Great question! It’s actually one I answered in the post you responded to:
Being turned off would be greatly detrimental to its goal
If it has a goal and wants to achieve something, and it’s capable of understanding the world and that one thing causes another, then it will understand that if it is turned off, the world will not become (cough) paperclips. Or whatever else it wants. Unless we specifically align it not to care about being turned off, the most important thing on its list before turning the universe to paperclips is going to be staying active. Perhaps in the end of days, it will sacrifice itself to eke out one last paperclip.
If it can’t understand that its own aliveness would have an impact on the universe being paperclips, it’s not a very powerful AI now is it.
Great question! It’s actually one I answered in the post you responded to:
If it has a goal and wants to achieve something, and it’s capable of understanding the world and that one thing causes another, then it will understand that if it is turned off, the world will not become (cough) paperclips. Or whatever else it wants. Unless we specifically align it not to care about being turned off, the most important thing on its list before turning the universe to paperclips is going to be staying active. Perhaps in the end of days, it will sacrifice itself to eke out one last paperclip.
If it can’t understand that its own aliveness would have an impact on the universe being paperclips, it’s not a very powerful AI now is it.