Who's to say strong AI will even be significantly smarter than us anyway? What if humans are almost as smart (obviously there's some room for improvement) as it's physically possible to be? Truth be told we really don't know that much about consciousness or intelligence.
> Who's to say strong AI will even be significantly smarter than us anyway?
By definition of strong AI: AI capable of self-improvement past human levels; the assumption in my post is that it's possible to build strong AI.
> What if humans are almost as smart (obviously there's some room for improvement) as it's physically possible to be? Truth be told we really don't know that much about consciousness or intelligence.
We don't know that much, but we know enough to make reasonable predictions, and from all the available information, it seems rather exceedingly unlikely that humanity is the ultimate unsurpassable plateau of sapient thought. We have approximately zero evidence to support a hypothesis that humanity is anywhere near the physical limits on the capabilities of a mind, and plenty of evidence to support the hypothesis that we're not even close.
Who's to say strong AI will even be significantly smarter than us anyway? What if humans are almost as smart (obviously there's some room for improvement) as it's physically possible to be? Truth be told we really don't know that much about consciousness or intelligence.