mirror of
https://github.com/webinstall/webi-installers.git
synced 2026-05-16 21:56:33 +00:00
9f28505af7
Stacked on the modifications PR. Now that no live code path references
the per-package fetchers, the shared HTTP/parsing helpers, the
in-process normalizer, or the example template, delete them. Pure
deletion — no behavior change.
- ~93 per-package <pkg>/releases.js fetcher modules.
- _common/{brew,fetcher,git-tag,gitea,github,github-source,
githubish,githubish-source}.js shared HTTP/parsing helpers.
- _webi/normalize.js in-process normalization layer (cache files
arrive normalized from webicached).
- _example/releases.js fetcher template for new packages.
The Go cache daemon (webicached) is now the sole producer of release
metadata; the Node process never makes an upstream request.
title, homepage, tagline
| title | homepage | tagline |
|---|---|---|
| Crabz | https://github.com/sstadick/crabz | crabz: multi-threaded gzip (like pigz, but in Rust) |
To update or switch versions, run webi crabz@stable (or @0.8, @beta, etc).
Cheat Sheet
crabzbrings the power of multi-core compression to gzip and deflate.
(and a few other formats + other useful features)
gzip, faster.
crabz -I ./example.json
crabz -d -I ./example.json.gz
Compressing (gzip) with 8 threads at compression level 6.
Decompressing (gzip) with 8 threads available.
Table of Contents
- Files
- Tar
- Other Formats
Files
These are the files/directories that are created and/or modified with this install:
~/.config/envman/PATH.env
~/.local/bin/crabz
How to Optimize
| Flag | Value | Comments |
|---|---|---|
-l, --compression-level |
1-9 | higher is slower |
-p, --compression-threads |
8 | set to the number of available cores |
| (but no more than 4 for decompression) | ||
-P, --pin-at |
0 | pin to physical cores, starting at N |
| (so 4 threads starting at 0 is 0, 1, 2, 3) |
crabz -l 9 -p 8 -I ./example.tar
crabz -d -p 4 -I ./example.tar.gz
How to use with Tar
Tar and then compress:
tar cv ./example/ | crabz -o ./example.tar.gz
Or decompress and then untar:
crabz -d ./example.tar.gz | tar xv
How to use with other formats
crabz supports most of the LZ77 with Huffman coding compression formats:
| Format | Extension | Notes |
|---|---|---|
gzip |
.gz |
of GNU fame |
bgzf |
.gz |
supports random-access decompression |
mgzip |
.gz |
of python fame |
zlib |
.zz |
of PNG fame, also .z |
snap |
.sz |
of LevelDB and MongoDB fame |
deflate |
.gz |
the O.G. LZ77 |
crabz --format mgzip -I ./example.tar
# DO NOT decompress in-place
crabz --format mgzip -d ./example.tar.gz -o ./example.tar
# verify before removing the original
tar tf ./example.tar
⚠️ Warnings:
- DO NOT deflate in-place with non-standard formats:
Althoughgunzipwill work correctly on files compressed withmgziporbgzf, some combinations (ex: decompressing frommgzipwithbgzf) could result in corruption! tar xvfandgzip -lmay report incorrect information, even thoughgunzipwill work
See also:
(p.s. zip isn't in the list because it's a container format like tar, not a
zip format)