YouTube will need to keep its h264 files for Flash (and mobile devices), so Ogg would require petabytes of new storage space. Real money. According to YouTube's engineers to get comparable quality video with Ogg the file size will have to increase. Real money.
If it would be cheaper there would be no reason not to do it. Google built support for Ogg into Chrome, so they aren't scared of the format. They aren't doing it so I'm going to go out on a limb here and say it's not cheaper.
I haven't seen a substantiated claim that shows that Ogg is bigger than H.264. I've see Chris DiBona claim it on WHATWG, but not back it up with numbers or experimentation. In fact, I posted a link above to someone who refutes that assertion:
It doesn't matter whether the same video produces larger or smaller files in Ogg than H.264. It matters that YouTube still has to keep an H.264 copy around regardless (in order to have a version everybody can watch), which means that supporting Ogg involves YouTube storing (size of YouTube as H.264) + (size of YouTube as Ogg), which can be expected to be prohibitively larger than (size of YouTube as H.264).
Ah, that comparison had the opposite of the intended effect on me. The Theora encoded video looks much worse in every instance. Not sure if I would notice the harsh contrast and encoding artifacts when viewing a YouTube video without comparison, though. But still, the comparison pretty much convinced me that Theora has a very long way to go to be able to compete on quality.
If it would be cheaper there would be no reason not to do it. Google built support for Ogg into Chrome, so they aren't scared of the format. They aren't doing it so I'm going to go out on a limb here and say it's not cheaper.