You can want your e-scooter to be a jet ski, but you'll wind up having issues when you try to use it as one. LLMs are _really good_ at pretending to be something they aren't, but not always so good at being that thing, so you should be careful what you ask for.
The difference is what happens when you come to depend on it. Do you want your airplane to be flown by a pilot, or someone who's so good at talking like a pilot that they can fool almost everyone?
If they are only good at pretending to TALK like a pilot, then of course that is bad. However, if it is so good at pretending to BE a pilot that it can actually fly the plain really well, then that is good enough.
He may be able to talk the talk, and walk the walk, but today's guest pilot has a crippling heroin addiction. Let's hope that auto-landing system works.
