It’s not perfect, it has shortcomings, it sometimes produces bogus outputs. All of that is fine for a tool, it’s not fine when it pretends it’s a conscious being, because errors start to feel like lies and it becomes a bit too personal.
People want it to be Data from Star Trek, when it really should be the ship's computer. I want to tell it to run a simulation accurately, create some solved tool, etc.
I don't think there's any correction that can return LLMs to a purely tool-space. Too many AI boyfriend/girlfriends.
You can want your e-scooter to be a jet ski, but you'll wind up having issues when you try to use it as one. LLMs are _really good_ at pretending to be something they aren't, but not always so good at being that thing, so you should be careful what you ask for.
The difference is what happens when you come to depend on it. Do you want your airplane to be flown by a pilot, or someone who's so good at talking like a pilot that they can fool almost everyone?
If they are only good at pretending to TALK like a pilot, then of course that is bad. However, if it is so good at pretending to BE a pilot that it can actually fly the plain really well, then that is good enough.
He may be able to talk the talk, and walk the walk, but today's guest pilot has a crippling heroin addiction. Let's hope that auto-landing system works.
No because then the non-deterministic grifter Data tells you to kill yourself.
Data is an individual. LLMs are time-shared hallucinatory algorithms that will ultimately be used to sell ad space.
It is Stupid to create "the machine that melts people's brains", but these assholes are trying to sell you "the machine that melts people's brains" and love to market it as being "anything the user wants".
I think the problem is that we’ve anthropomorphised the LLMs in their training and through system prompts to the point that that seems reasonable.
We don’t thank our other tools like grep, or the compiler.
If you train your LLM to produce words that seem like human responses or conversations, they will. If you train them to seem like god-like superintelligences to people who think that’s what they’re creating, they will.
It’s not perfect, it has shortcomings, it sometimes produces bogus outputs. All of that is fine for a tool, it’s not fine when it pretends it’s a conscious being, because errors start to feel like lies and it becomes a bit too personal.