The go-to example of a failure state in the golden rule that I am aware of is "thirsty person wants everyone else to jump their bones, thus the rule suggests that they should jump everyone else's bones".
The take-home being, as long as peoples preferences are different, the rule has some failure states based upon that.
And the wider example that this points to is that as long as everyone's values are different.. the rule loses a lot of it's luster. The more different another person's values are, the less you can rely on projecting your own values as a method to gainfully interact with them.
In place of that biblical aphorism (born from a monoculture no less), there is a lot of room available for us to build something more robust and sophisticated and the AI alignment problem offers a fine excuse to get cracking and try to formalize something already! ;)
Again, that’s an overly literal and straw man interpretation of the golden rule that seems nearly intentionally obtuse. The rule is not suggesting you should do anything that you know other people wouldn’t like. Jumping people’s bones is a very bad example, because that’s not something anyone should do without communication and explicit permission, otherwise it’s rape and it’s illegal.
If you understand and accept the reciprocal intent of the golden rule, via the above mentioned reasonable assumptions, not to mention the hundreds of years of examples and explanations, then it doesn’t really have a “failure state”. And, again, this so-called rule is not law, isn’t specific, and doesn’t cover all situations or all behaviors. It seems very silly to be nit picking this aphorism to death like this. It’s very simply a good faith suggestion to be kind.
The notion of fairness and reciprocity is not a religious idea, it’s common sense.
The take-home being, as long as peoples preferences are different, the rule has some failure states based upon that.
And the wider example that this points to is that as long as everyone's values are different.. the rule loses a lot of it's luster. The more different another person's values are, the less you can rely on projecting your own values as a method to gainfully interact with them.
In place of that biblical aphorism (born from a monoculture no less), there is a lot of room available for us to build something more robust and sophisticated and the AI alignment problem offers a fine excuse to get cracking and try to formalize something already! ;)