Do you consider rar a standard? It's a pretty good format, even though there aren't any good open-source implementations. But if you're willing to pay for your software, WinRAR command line versions are available for most platforms.
7zip is the most obvious free alternative. There is also a 7zip fork that offers zstd [1]. The command line experience for 7zip isn't very good however.
tar.gz/bz2/zstd might be more sane, as it separates the concept of concatenating multiple files together from the compression aspect. Perhaps the thing we can improve upon is the tar format, which IIRC was designed for tape devices.
Separating these concepts makes seekability much harder, and imho has dubious benefits at best.
I'm all for formats that separate the exact compression algorithm from the file storage format. Both rar and 7zip do this. For example 7zip offers you 4 compression algorithms (LZMA/LZMA2/PPMD/Bzip2) and forks extending the choice while keeping the same file format.
And if you don't keep the storage and compression layer in the dark about the existence of the other you can chunk compression more sensibly, keep your file directory uncompressed (or compressed separately), embed recovery information (like RAR's embedded par2), and choose different algorithms for different files (including not compressing certain file types)
True, you have convinced me. That said, I think the reality is that Zip and tar+compression are good enough and there is no real need to change the status quo. Both are extensible in their compression algo, which is the thing people care the most about their archives. I would go as far as to say people these days use 7z and rar out of inertia, rather than actual need.
7zip is the most obvious free alternative. There is also a 7zip fork that offers zstd [1]. The command line experience for 7zip isn't very good however.
1: https://github.com/mcmilk/7-Zip-zstd