I remember vaguely from a presentation by Yann LeCun "Intelligence is not what you know, but what you do when you don´t know". I find it helpful when trying to build an intuition for how to understand the LLM tool.
That is the task they are constructed to perform, but that's just an output, not the inner workings of what's happening to arrive at that next token. You can give a human the same constrained task, but the output alone doesn't a human mind make.
