Where does natural evolutionary optimization lead us om AI without guidance? This is equivalent to your quantum ground state. Systems will naturally gravitate to this ground state. You have to constantly pump in energy and supervision to make sure it's not reached. This is a recepie for disaster.
Not sure what you mean. Are you suggesting that the current state of affairs is a result of not putting in extra work to guide or direct models away from such behavior? That perhaps this is their ground state?
Well, that we have to put an insane amount of work to keep it aligned. Kind of like pushing a huge round boulder to the top of mount Everest. You have to expend energy to get it there and fight physics to keep it there.
With a static model we might be able to keep it somewhat under control, but think about future continuous learning models. They'd drift away from unstable high energy configurations. Also any model being trained by people that don't care about safety.
We need to ask a different question.
Where does natural evolutionary optimization lead us om AI without guidance? This is equivalent to your quantum ground state. Systems will naturally gravitate to this ground state. You have to constantly pump in energy and supervision to make sure it's not reached. This is a recepie for disaster.