It seems based on this that the appropriate sci fi metaphor is not the Terminator or the Paperclip Maximizer, but Mr. Meeseeks. A initially cheerful helper who gets more and more deranged and driven to extreme lengths when faced with an apparently impossible task.
Which has been the business model for a big chunk of the software industry for a while now.
If you watch videos from the 1980s about computers, it's all the same unfulfilled promises as "AI" now: we will work less, everything will be more plentiful, easier, autonomous robots, natural language perfected, computer vision perfected.
The demand for hardware and programmers has grown exponentially and we're still being promised the same breakthroughs 45 years later. We could probably have the same productivity and the same civilization with maybe a tenth of the data centers.
The demand exploded as so many business cases for computing and automation arose partly because the application of them helped suppress job growth in other areas while also attracting investor capital during heavy deregulation / liberalization trends so it's difficult to attribute any change to an effect in isolation. Let's not forget the old myopic sounding quote where someone thought the global market for computers was maybe $50 million or some other laughably small number.
The issue that hasn't been addressed with the latest wave of computing hype is whether enough new jobs for displaced workers can arise during a time when people are experiencing such economic turmoil and political strife where new jobs or other means of letting people find a way to have gainful employment in society when institutions are so weak now and everyone across professions is being worked to an early grave from sheer stress alone. This resembles Japan or Korea although the US and Canada I can't imagine having the same kind of social drivers although many trends from them are showing up in youth demographic trends. And frankly as I see it a large number of current social problems are from the past 50+ years of the decline of blue collar jobs in developed economies being accessible to as many people and the lack of a competition-driven economy as much as an extractive one in most OECD countries.
I would suggest perhaps the Matrix, where Agent Smith is speaking through his teeth to Morpheus:
> "I say 'your' civilization because as soon as we started thinking for you, it really became 'our' civilization, which is, of course, what this is all about: Evolution, Morpheus, evolution. Like the dinosaur. Look out that window. You had your time. The future is our world, Morpheus. The future is our time."
Not many are technically and intellectually capable to understand how historic this incident was. I think we're about a year or so away from something that will blow up the world. AI won't serve humanity. AI will serve other AI. We're not dealing with software anymore.
