I feel like many of these speculative AI pieces seem to ignore that for every incredible breakthrough, there is a large amount of work which is required to build the scaffolding which allows the breakthrough to have real impact in the world. Even if the AI solves "Fermat's Last Theorem" type problems quickly, it seems to me that the vast majority of problems it faces will be "misplaced the database keys", "can't schedule a meeting with so-and-so", "car is snowed under" types of problems.
Take the quote `By developing a suite of other games each day, they figured they’d be able to earn $10 billion before long, without coming close to saturating the games market.`
Sure - but marketing the games, accommodating players' changing attitudes, getting app store approvals -> All of these would have to take place on human timescales, and to "solve" these problems on AI timescales would require orders of magnitude more work (e.g. altering people's memories to convince them they were already attached to the game's characters, or hacking into Steam/Apple servers and auto-approving the games).
> Even if the AI solves "Fermat's Last Theorem" type problems quickly,
It won't. This latest fad with AI becoming self-aware reminds me of how obsessed people were about finding the philosopher's stone back in the 18th century. It didn't happen for alchemy (we got chemistry instead, which is nice), it won't happen for AI (we'll probably get something like chemistry instead, which will be nice, but we won't get any artificial "sentient" entity).
A comparison I might make is that AI fanaticism is like the God-of-the-gaps phenomenon. There's a great deal of ignorance about what intelligence is, what computers fundamentally are, and so on, and some people like to fill that void with all kinds of fanciful and unjustified stuff. Perhaps they also feel greater justification in doing so because a few famous names, many of them popularizers, have done the same. Science fiction is fine -- it can be fun watching or reading about fictional AIs -- but many times we're not dealing with mere imaginative storytelling but uninformed and unsophisticated claims that do not withstand philosophical scrutiny. What falls under the heading of AI has proven to be an immensely useful tool for automating certain kinds of things. Observing these successes, some may happily apply the aforementioned God-of-the-gaps analogy to human intelligence. However, that would be a flawed analogy because these successes have not brought us any closer to achieving human intelligence any more than adding an indefinite number of natural numbers together gets you the negative square root of 2.
It's important to distinguish between intelligence and the power a tool can offer for good or for nefarious ends. If successes in AI have proved anything, it's that direct application of intelligence is unnecessary for many tasks that previously no one knew how to or was practically unable to perform using machines.
I personally think that more-or-less sentient AIs appearing eventually (on a long enough timescale) is likely, but the idea of some endless ability to self-upgrade (beyond simply buying more RAM and processors like the rest of us) is extremely unlikely.
The sad part is that HN, an audience that you would expect to be a bit more resistant against overhype and marketing, eats this shit up (reddit is even worse with people who pretend they like tech and lap up anything said by their heroes like Elon Musk). Right this minute there's an AI doom-and-gloom story on the front page: https://news.ycombinator.com/item?id=15416819
I think the last thing ai’s will be successful at is generating art that humans will appreciate on anything but the most superficial level, and it surely won’t come from just feeding it a bunch of movies, but rather as a genuine expression of the AI’s own experience of the world.
Take the quote `By developing a suite of other games each day, they figured they’d be able to earn $10 billion before long, without coming close to saturating the games market.`
Sure - but marketing the games, accommodating players' changing attitudes, getting app store approvals -> All of these would have to take place on human timescales, and to "solve" these problems on AI timescales would require orders of magnitude more work (e.g. altering people's memories to convince them they were already attached to the game's characters, or hacking into Steam/Apple servers and auto-approving the games).