This has been a core belief of the EA/LessWrong/Rationalist people for many years - that above a certain level of intelligence you effectively get hypnotic powers, and people will obey AIs no matter what. So they said you can't keep a superpowered AI airgapped as merely offering a chat interface will immediately take control of the next person to interact with it and they will deliberately compromise the airgapping.
And if AI wrote a book about these hypnotic powers, then it would be incredibly convoluted, long, and yet worn as a badge of pride to have survived it.
Thinking about it, much of the output I'm seeing nowadays looks like a chapter out of infinite jest.
A complete mischaracterization, as usual for HN lately when discussing AI or LessWrong. Obviously, even average levels of persuasion are enough to convince some people. And nobody is air-gapping AI.
It's kind of fascinating how toxic the EA/LessWrong/Rationalist stuff has become. I remember reading it back in the early days (~2010 or so) and finding a lot of good ideas mixed in with some dorkiness (Harry Potter fan fiction about rationalism..). It always seemed like a sort of harmless niche with some good ideas and some wacky ones. And yet now it's basically become a cultish religion for the ultrapowerful who use it to justify doing some really awful things.
