I guess I don't have a big problem with this, law enforcement would need a warrant to get the information, and in most cases it would only confirm or repudiate what they already know or suspect. It's just a more convenient way to surveil a suspect, and for the suspect it's easier to counter: turn off your phone, or better yet leave it somewhere where you're not.
The NSA isn't a law enforcement agency. Rightly or wrongly, US intelligence agencies seem to have a rather different set of rules these days than the rest of us are following, including most law enforcement. Bottom line is if they're wanting to use it to convict you of a crime they'll need a warrant for it (before most judges, at least).
If you're worried about NSA (llegal) spying in general, presumably the carriers retaining the data only makes it a bit easier for them. Since the info will need to be traveling around the network while you're active, they could easily just intercept that like they intercepted the voice calls.
