>Windows has had binary compatibility for decades.

Extensions don't just run on top of Firefox, though; they modify Firefox's UI in arbitrary ways.

>Why not just version the internal browser API?

Firefox already does this, and automatically disables incompatible extensions.