Another kind of hackability: There is a whole little music genre called "tape loops" where people make music by cutting open cassettes and taping the ends of the tape to make infinite loops. You record onto a few of those of different lengths, play them back in parallel and get neat rhythmic effects as the loops go in and out of phase with each other.
One of the great pioneers of electronic music, Delia Derbyshire, part of the equally pioneering BBC Radiophonic Workshop, hacked reel audio tapes and machines in the way you describe and much more, to create her sounds. She was so OCD in doing so that it was a bit damaging, and may have viewed the later digital synths as a godsend or with disdain. She created the Dr. Who theme.
Speaking of creativity, the Beatles famously used a tape-based keyboard, the Mellotron [0]. For instance, in Strawberry Fields Forever, the spooky instrument heard in the introduction is a flute replayed in reverse on the Mellotron [1].
AMULETS is a good example of a musician making songs like this: https://www.youtube.com/watch?v=CVon_9VmMEc
The r/tapeloops subreddit is a nice little community for it.