I like the illustration that the models are working on a convex hull of known information. Filling gaps with linear combinations of known facts and results.
They can't exit the hull until the "intuition" starts spawning points outside the convex hull.
I have news for you. All humans do is also filling gaps with combinations of known facts and results - in new ways. "Everything is a Remix" is a good watch on youtube that explains this. Picasso might look like he has an invented personal style, but his style is a combination of different little details he took from others and mixed in a new way. Mozart the same. No music artist could ever create music in a vacuum. Everyone, for every art and science, the same. I know many are trying to cling to the last hope of human specialness, that "thing" that AI can never get to.
It's a convex hull of information that is reflective and spans outside of itself and combines in a new way, when you shine two known rays of light together from the inside.
Now it gets better. AI can be orders of magnitude more creative than any human could ever hope for, because his convex hull of information is orders of magnitude larger, and the possibilities for new combinations are equally larger.
I've seen this "everything is a remix" idea thrown around a lot, and always want to ask... so how did anything get started? What was the first cave-dwelling carver of a bone flute remixing? What was Alan Turing remixing to come up with the Turing machine? Where did Proto-Indo-European language come from?
Maybe there's some sense of "remix" that covers all of that, but then that meaning is not the standard one, or maybe you would concede that this happened in the past, but isn't happening now, but then where's the cutoff, or maybe something else...?
In your analogy, I would love to see some elaboration on the "reflective and spans outside of itself" part -- geometry doesn't work that way, do you have a more intuitive metaphor, or ideally a mechanism, or even better, some examples?
> "Everything is a Remix" is a good watch on youtube that explains this
Not completely. Novelty used to be a major thing, when Humans did it. Another important criteria used to be if that new thing makes sense at all. Here the language model has a problem, as it doesn't have the means to evaluate this criteria.
The extrapolation can also be a learned skill, especially in math. How many papers took result X, extended it to Y using known building blocks, and applied to Z.
By the way, convex hull permits extrapolating past the training data. LLM won't invent a new word that could not be defined by a sequence of known words. Just if it's meaningless and fully random/hallucinated, the new knowledge won't work with other known information blocks (breaks convexity).
Is that actually true though? I think it is an analogy, and as an analogy it seems quite risky because “convex hull” and “linear combination” are technical terms that might give the recipient the impression that it is a technical argument.
I think this is only "statistically" true in the sense that training is based on facts and not non-facts (except maybe with the ingestion of flat-earthers literature ;-). The existence of hallucinations in a bare transformer shows that the convex hull is not about information but about text, so the limit may more be "possible linear combinations of text", which allows for much extrapolation and counterfactuals. True creativity may be one reinforcement learning mid-training goal away that rewards novelty over correctness.
They can't exit the hull until the "intuition" starts spawning points outside the convex hull.