The even more historical problem is that surveillance powers were operated under the royal prerogative with no oversight of any kind. RIPA (that, correctly, much maligned predecessor to the Snoopers' Charter) was in one sense a step forward in that it at least put the surveillance that had always gone on onto a statutory basis.

Same with local councils. They'd always carried out some physical surveillance (benefit fraud and lying on school application forms being the main scope IIRC) but without oversight.

Unfortunately, the oversight regime sucks. But as you say, it's an improvement on a complete free-for-all.