Well, that's just because Earth and moon are too far apart and don't both have an atmosphere. If they were closer, you could totally do that, just watch for the microgravity around the barycenter.

Better still, you could do that even with a hit air baloon and late middle-age technology! There is even a SF book series about that:

https://en.m.wikipedia.org/wiki/The_Ragged_Astronauts