Re: Zstandard support for setup

[Achim Gratz <Stromeko at nexgo dot de>]

While it seems like I'm talking to myself, in case anybody is listening:

With the "normal" maximum compression level of 19 the package archive
for my installation (including sources and debuginfo) got about 10%
larger, taking about the same time as Xz for compression.  Using the
"ultra" compression level 22 gets that difference to about 3% at the
expense of twice the CPU time for the compression.  My repo is about
8GiB compressed and it took about 8 (resp. 16 hours) of single-core CPU
time to re-compress (uncompress from BZip2 or Xz and recompress
on-the-fly to ZStdandard).  The higher compression level needs just a
bit over 1.5GiB on average and around 3GiB peak.

I've had 16 cores to run that on and those stayed at 2.8GHz all the
time, so it didn't take too long.  Here at home I've tasked two slower
4-core machines with it, so they were running a few hours of wall-time.
I might try long-range matching to get even better compression at work
since I've got 128GB memory now to see if there's an improvement.  At
home I've run out of 8GiB memory more than once even with a single
compression for the largest debuginfo packages.  Incidentally, these
packages always take the longest time to compress, so if you want to do
a parallel mass conversion it's advisable to start these first (keeping
an eye on available memory) and have the faster compression of the
smaller package archives fill the remaining CPU/time.

I've installed and updated over a dozen of machines by now with this, so
I'm reasonably certain that the implementation in setup is OK.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada