Are there any standard alternatives today? Especially for bundles of data - music, images, json, etc. There's .tar.gz which has (I think) a variety of implementations, but it has issues with seeking and the compression isn't great, and brings with it a whole bagload of linux stuff. Maybe something using zstd?
I think ISO is seekable and from the sound of it, you want something to work on an OS where you have very little control over the utilities it offers (i.e. MS Windows), where, I think ISO is supported by the pre-installed system utilities. The downside is that these aren't compressed. I don't think the OS that you chose gives you more options... it has native support for more filesystems, but using those as archives seems very problematic (but maybe I don't know something).
Had you not been constrained by the choice of the operating system, a reasonable OS + FS combination will offer you some way to make snapshots. Filesystems are usually optimized for random access (with exception of things like TAR, which is designed for streaming). Most modern general purpose filesystems support compression, so that wouldn't be a problem. Often these filesystems offer a way to mount the file with the filesystem image w/o involving OS utilities like loop devices, so it would give you a way to access archive contents as if you were accessing any other filesystem.
The bioinformatics community uses block based gzip compression (bgzip) [0]. The gzip standard allows for blocks so, using an additional index file, you can use it to seek to arbitrary locations and uncompress the block.
gzip compression is maybe not optimal now and the block segmentation reduces the efficiency even further.
Though not very standard, there is also a tar indexer program [1] that allows you to create an index on tar files to do the same.
My information is at least a couple years old so things may have changed.
Do you consider rar a standard? It's a pretty good format, even though there aren't any good open-source implementations. But if you're willing to pay for your software, WinRAR command line versions are available for most platforms.
7zip is the most obvious free alternative. There is also a 7zip fork that offers zstd [1]. The command line experience for 7zip isn't very good however.
tar.gz/bz2/zstd might be more sane, as it separates the concept of concatenating multiple files together from the compression aspect. Perhaps the thing we can improve upon is the tar format, which IIRC was designed for tape devices.
Separating these concepts makes seekability much harder, and imho has dubious benefits at best.
I'm all for formats that separate the exact compression algorithm from the file storage format. Both rar and 7zip do this. For example 7zip offers you 4 compression algorithms (LZMA/LZMA2/PPMD/Bzip2) and forks extending the choice while keeping the same file format.
And if you don't keep the storage and compression layer in the dark about the existence of the other you can chunk compression more sensibly, keep your file directory uncompressed (or compressed separately), embed recovery information (like RAR's embedded par2), and choose different algorithms for different files (including not compressing certain file types)
True, you have convinced me. That said, I think the reality is that Zip and tar+compression are good enough and there is no real need to change the status quo. Both are extensible in their compression algo, which is the thing people care the most about their archives. I would go as far as to say people these days use 7z and rar out of inertia, rather than actual need.
Not exactly standardized, but .7z should be versatile enough to cover most use cases except for a basis of application data files. For that case, you can actually design your own file format, which is surprisingly easy if you know your own requirements.
tar is just the way you pack multiple files and directories into one file, you can pipe the tar file through any compression algorithm you like it doesn't matter.
The tar command has many compression options builtin https://man.archlinux.org/man/tar.1#Compression_options
And you can also specify any compression program you like through `-I CMD` e.g. `-I "zstd -10"` would compress the tar archive with zstd and compression level 10.
