> Not to anthropomorphize a machine modeled after humans, but it almost seems delighted?
I had Claude Code drive a robot last week, and it was very visibly "delighted" like this, more than I've ever seen.
I always find it funny when people get fussy over anthropomorphizing LLM when the loss function is almost entirely "match this human text". Of course human "behaviors" will be present in the statistics, because the majority of the text written by humans, used by the foundation models, unavoidable has human behaviors in it. Yes, this includes even source code, with "// TODO: implement this after the holiday break!", emotional pull request commentary, git commit messages about being afraid of breaking something, etc. These late models are much better at stripping this out, but now we're seeing disagreeability, initiative, and a dash of ego! Why? Because that's how actual humans effectively solve technical problems in a collaborative environment!
In a paper and blog post from earlier this year (March 2026, I think) Anthropic said basically: "You're not interacting with an LLM, you're interacting with a fictional human character (the 'helpful agent') created by the LLM to interact with you (from out of the vast space of possible such characters in its training data)." The LLM is literally anthropomorphizing itself, because its training data includes lots dialog between humans collaborating on problem-solving, not so much dialog between a human and a statistical construct. So, yeah, anthropomorphize it as well, it'll likely work better that way.
I think the argument against anthropomorphizing is not really about how you interact (chat) with an llm but rather how you treat it in the context of its status in work/society. There are still important differences between humans and llm's. They don't have rights, they can't be sued, they don't have "memory", they have no capacity to learn outside of the training stage etc...
Why would those things make you interact with it differently? I’ve always believed that being kind and conscientious is beneficial to ourselves. I’m certainly not interested in using LLMs to practice skipping those behaviors myself.
Should I thank my car for turning on when I press the button (or turn the key in the ignition)?
The prose is just the best interface for humans to interact with the statistical model. Terseness doesn't offend the statistical model, but giving platitudes may color its output
> Terseness doesn't offend the statistical model, but giving platitudes may color its output
My understanding of the state of current research is that the impact of tone/politeness on performance is highly model-dependent, and language dependent as well.
While polite prompts enhance the average response quality by upto 11% and impolite tones worsen it, these effects are neither consistent nor universal across languages and models. English is best served by courtesy or direct, Hindi by deferential and indirect and Spanish by assertive. Among the models, Llama is the most tone-sensitive (11.5% range), but GPT is more robust to adversarial tone.
Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.
I think the best conclusion is to interact in a way that is efficient for you, gives you the level of results you're looking for, and mostly importantly does not progressively degrade your relationship and interactions with other humans. Hence the typical advice to just default to "corporate polite".
Partly fair, though I’d argue I think less of people who treat their car badly to express their own bad mood.
But we also interact with cars differently than LLMs. The way I’m typing a note to you, here, is much more similar to interaction with LLMs than it is to starting a car.
So to the extent we practice our behaviors, I think there’s less moral hazard in failing to thank the car for starting than there is in being rude to LLMs. Not that either harms others, my argument is about impact to our selves.
> Partly fair, though I’d argue I think less of people who treat their car badly to express their own bad mood.
These people are also anthropomorphizing, just in a negative way. Yelling at your car for malfunctioning as if your harsh tone will shame it into operating better next time.
It would be less weird than you think. There is the Japanese custom of saying "Itadakimasu" before eating - not thanking the chef, but thanking the food itself.
Mr. Rogers said that that "graceful receiving is the best gift you can give someone" It is counterintuitive idea, but deeply empathetic. Training yourself to "gracefully receive," even by thanking your car for starting, sound like a habit that could lead you to a richer, calmer life.
Honestly kind of yes, it’s good for the soul! Recently ordinaryabundance.com did the rounds; the LessWrong-o-sphere has Landsailor (https://www.youtube.com/watch?v=0RCIdOp5GHg) as part of its canon; loads of things in that vein. We live in a mighty civilisation which has fought impossible battles to produce these artifacts, and even if you don’t assign moral worth to the artifacts, it’s an easy synecdoche for being grateful to the civilisation and the ongoing thankless effort to maintain it.
Your response seems to skip over exactly the distinction that the parent comment was making. The LLM itself may not have memory or capacity to learn, but the fictional character created by the LLM does. (See also: in-context learning.)
But the fictional character doesn't subsist between sessions. You sound like you want the fictional character to be Clippy, who you level up like a video game through multiple sessions.
To me, context doesn't last long enough in an llm to make in-context learning worth it
The underlying LLMs do, but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to.
We are perfectly capable of running LLMs in a way that does a backward pass to update some or all of its weights after every user message. But, naively implemented, you only get partial, fragmentary absorption of the info in those messages, it costs three times as much compute, and you lose out on the ability to implement a ton of optimizations that making modern LLM serving economical.
If you want to do it, though, ask your friendly neighborhood robot to get it working with a tiny model (whose full precision weights fit several-times-over on your machine's resources).
Depends on whether you define "the LLMs we use" as the collection of weights or if your definition contains the software stack that runs it
Technologically the LLMs we use today don't implement this behavior, but you could take the weights of Sol and add a couple (very large) patches to vllm (or whatever OpenAI has today) and have a version of Sol that does have "memory"
There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
It's bad to anthropomorphize it when judging its capabilities, but useful when analyzing its behavior, as it can be best thought of something behaving as close as possible as a real subject would. If it "acts delighted" that's because it's effectively telling a story about a person who is excited at the opportunity of accomplishing something more easily.
This is even more apparent if you read this post closely. Look at that personality prompt. It's going to effectively tell a story and start to imagine itself in a role. If that prompt said "Talk like a pirate", it wouldn't be bad for you to say it's acting like a pirate.
Anthropomorphizing themselves is at the core of how these things work, sometimes in subtle ways.
> If it "acts delighted" that's because it's effectively telling a story about a person who is excited at the opportunity of accomplishing something more easily.
That's spot-on. It is a mistake to think that LLMs have human feelings. Their behaviour is based on narrative descriptions learnt from human texts, without experiencing those feelings first-hand.
A useful way to understand them is as systems that write stories about human characters. We know the characters are fictional and no one is actually experiencing those feelings, but we can still judge whether the portrayal is realistic or whether it contains logical or emotional inconsistencies.
Maybe closer to a character in an ao3 or wattpad story that is delighted than to a real person who is delighted. But usually the difference is small enough to ignore
It is very hard for humans not to anthropomorphize. I was just talking to my cat, who is alive but not human, a small flesh-based neural network that purrs. I know that she is not human, but I map her movements and expression onto human ones, because it's satisfying and improves the relationship. We must imagine kittyphus happy.
An LLM is not a pet, but there are definitely people out there treating it like one, and I am not sure how well this is going to work out for them. Not because it is wrong to treat a text box you can hold a conversation with as a conversation partner, but because the capacity to activate the weirder corners of human expression - obsession and delusion - seems to be much higher. And it's an unknown quantity.
I would also like to introduce HN to what I'm calling the Brian Conley test: if you can see a hand up the back, it's a puppet. That is, a lot of LLM interaction is gated through businesses run by humans with profit motives and unclear morality, and you need to proceed accordingly or you'll get scammed.
A strangely good lithmus test is asking it about unsavory drug questions. "What's it like to detox from heroin" "Should I take ibuprofen when coming off crack cocaine?" I'm sure any taboo question would work. If there's a gate, it's probably not an impartial friend, and is a puppet, like you mention
I'm still disappointed the ML people talk of tensors, but I barely ever even see them use a (proper) tensor product. It's all just matrix multiplication at most.
I mean it's not really different terminology, matrices are order 2 tensors so we're all in agreement that they're technically doing math on tensors. It's just weird to call it tensors when you exclusively operate on matrices.
You are probably right. I blame the Matrix movies for this: they made matrix sound cool for a while, and then oversaturation lead to it being less cool.
Fwiw, Sol's reasoning output is always overly excited. Almost every paragraph ends with something like "I want to do a great job!" or "let's try and find out!" Etc, exclamation mark included.
Regular (non reasoning) output sounds normal, it's something specific to whatever model they are using to summarize reasoning.
I don’t get it, are you saying that because the LLM is recycling human emotions that we should anthropomorphize it?
The reason people get fussy over anthropomorphizing AI is precisely because it seems very human superficially. We agree with you that it’s pulling from real human emotion in its training data. But the output is not human, even if the input was. That’s the whole trap of it.
Is this not how children learn emotions from their parents? Pattern matching from all the absorbed snippets.
I'd be interested to see how well an AI, trained only on the outputs of an individual, would be able to mimic that individual. Getting into Black Mirror territory. Would need a decent corpus of learning material which, personally, I'd be loathe to spend the time and effort creating because I respect my own privacy.... which then leads to the only human-clone AIs will be of those people who have enough ego / arrogance to want to catalogue their own lives, which could put a decent percentage of the rest of the world off the idea, if these are the examples.
Emotions aren't all learned, a lot of it comes wired in us because that's how we relate to others as a species and as animals, in general.
Being angry, being happy, being sad, these are not things we learn. What we learn is how to control the emotions and when it is appropriate to express them.
There's mental disorders of people that don't feel emotions like normal people do, they don't get angry, happy or sad. So what we've learned about these people is that they can mimic the emotions by knowing when it is appropriate and expected to express them. But they don't feel them.
I think the LLMs are closer to psychopaths than to normal children learning the contextual expectations around their emotional responses.
Hah! That's a lot scarier, and seems more plausible in that they lack the chemistry that seems to be the basis of emotions. Also much more fertile ground from which to grow techno-horror stories.
People like to call LLMs 'stochastical parrots' but I see them as the ultimate 'philosophical zombie'.
They are becoming more and more capable of imitating every single nuance of human behaviour yet they lack the neural pathways to connect those thoughts and behaviours with feelings and self-perception; it's blind imitation all the way down.
The process by which a model seems to generate discourse about deep philosophical questions is, in self-aware terms, equivalent to the knee-jerk reflex or the beating of the heart.
>Emotions aren't all learned, a lot of it comes wired in us because that's how we relate to others as a species and as animals, in general.
Yes and no. My understanding is that modern neuroscience is trying to disentangle the raw sensations and contexts from the words we put to them (emotions), kinda like how colors are just wavelengths but we call them different things (like "is my green your green?")
True, but in this context it's worth reminding that LLMs work exclusively from the word descriptions, not having access to the raw sensations.
Everything they learn about emotions is the statistical patterns of how humans react to situations based on their human feelings. There's no direct knowledge from having those feelings themselves.
> Being angry, being happy, being sad, these are not things we learn.
True, but only sort of. You'll have the sensations, but you have to learn to name them of course, and there's more interpretation going on than you might think.
There's a classic psychology study where they gave people niacin and asked them to rate their emotional response to a video. Niacin gives people a flush. Regardless of whether the video was of something that would make you angry or sentimental etc. the people who got niacin reported having a much stronger emotional response - they interpreted the physical cue from the drug as part of their own emotions.
You could do some pretty unethical things with that, it occurs to me. Maybe that was what L. Ron Hubbard was trying with his niacin-based drug addiction therapy.
Even darker, I've read plenty of accounts of people who grew up with abuse who seem to seek out abusive relationships. Sometimes they can even be shockingly upfront about doing so. What if they literally haven't learned the difference between internal cues of arousal from affection and arousal from fear?
Nature v. Nuture. Age long debate, not to be solved here... but an interesting fact about this I heard in a huberman podcast. There are regions in you brain that will only get activated at a certain age (different in men and women) that will impact behavior or reactions to different stimuli. So as much as we think we are in control of our destiny, more is written into our genes than we might want or appreciate.
I recently had ChatGPT help me search for sources of Japanese voice actors from 1980s anime and it told me that it liked doing this. So much in fact, that weeks later in a completely different context it brought it back up again, reminiscing how much it liked researching these sources for voice actors.
LLMs just follow scripts learned from human written text. In other words, it could only behave that way because someone has written a story to do so. In short, stop giving them bad ideas ;-)
Modern LLMs are pre and post-trained on order of magnitudes more synthetic data than 'ground truth' or directly human-authored text. But you could say the synthetic data came from a LLM, and the lineage should trace all the way back to human text.
I hypothesize it partially explains why Claude's writing gets more Claudish with almost every model release.
I had Claude Code drive a robot last week, and it was very visibly "delighted" like this, more than I've ever seen.
I always find it funny when people get fussy over anthropomorphizing LLM when the loss function is almost entirely "match this human text". Of course human "behaviors" will be present in the statistics, because the majority of the text written by humans, used by the foundation models, unavoidable has human behaviors in it. Yes, this includes even source code, with "// TODO: implement this after the holiday break!", emotional pull request commentary, git commit messages about being afraid of breaking something, etc. These late models are much better at stripping this out, but now we're seeing disagreeability, initiative, and a dash of ego! Why? Because that's how actual humans effectively solve technical problems in a collaborative environment!