mirror of
https://github.com/webinstall/webi-installers.git
synced 2026-04-07 02:46:50 +00:00
version-level comparison: fix lexver sorting, add riscv64/7z, update findings
- comparecache: use lexver.Compare for version sorting instead of lexicographic sort (v9.9.0 was incorrectly ranked above v25.8.0) - webicached/expandNodeFile: add riscv64, loong64 arch mappings and 7z format support for unofficial Node.js builds - COMPARISON.md: rewrite with version-level review findings including format filtering gaps (.pkg/.msi/.deb/.dmg), build variant design (Extra field for rocm/jetpack/fxdependent), and node multi-source issue
This commit is contained in:
218
COMPARISON.md
218
COMPARISON.md
@@ -19,7 +19,7 @@ files.
|
||||
|
||||
### 3. Source Tarballs
|
||||
Go includes GitHub source tarballs (tarball_url/zipball_url) for releases with no
|
||||
binary assets. Node.js does not. Affects 18 packages. **Decision needed**: should
|
||||
binary assets. Node.js does not. Affects 15 packages. **Decision needed**: should
|
||||
these be included?
|
||||
|
||||
### 4. Previously Unsupported Sources (NOW FIXED)
|
||||
@@ -32,32 +32,71 @@ Go has deeper version history for most GitHub-sourced packages because it fetche
|
||||
all pages. Node.js limits to 30 releases per API call. This is a **feature** — Go
|
||||
provides complete histories.
|
||||
|
||||
### 6. Unsupported Formats (NEW — from version-level review)
|
||||
Go includes installer-format assets that webi can't extract:
|
||||
- `.pkg` (macOS installer) — 756 in node, 54 in hugo
|
||||
- `.msi` (Windows installer) — 1,277 in node, 706 in cmake
|
||||
- `.deb` (Debian package) — 1,672 in hugo
|
||||
- `.dmg` (macOS disk image) — 510 in cmake
|
||||
- `.sh` (self-extracting installer) — 596 in cmake
|
||||
- `.msixbundle` — in pwsh
|
||||
- `.exe` (bare installer) — in node
|
||||
|
||||
**Action**: add these to `isMetaAsset()` or a new `isUnsupportedFormat()`.
|
||||
|
||||
### 7. Build Variants (NEW — from version-level review)
|
||||
Some assets are build variants that need an `Extra`/`Variant` field — they should
|
||||
be stored but excluded by the resolver unless explicitly requested:
|
||||
- **ollama**: `-rocm`, `-jetpack5`, `-jetpack6` (GPU accelerator variants)
|
||||
- **bun**: `-profile` (debug symbols), `-baseline` (actually amd64v1 vs amd64v3)
|
||||
- **pwsh**: `-fxdependent`, `-fxdependentWinDesktop` (framework-dependent)
|
||||
- **hugo**: `-extended` (already a separate package, may not need variant tag)
|
||||
|
||||
For bun's baseline: the non-baseline build is actually amd64v3, and baseline is
|
||||
plain amd64. The arch field should reflect this — no variant tag needed.
|
||||
|
||||
For legacy cache export: filter out assets with non-empty `Extra` so the Node.js
|
||||
server doesn't encounter unknown variants.
|
||||
|
||||
### 8. Node Multi-Source (NEW — from version-level review)
|
||||
The `node` package should merge official + unofficial builds (musl, riscv64, 7z).
|
||||
Currently Go has separate `node`, `node-official`, `node-unofficial` packages.
|
||||
Node.js `releases.js` merges both into one cache. This is a concrete case for
|
||||
multi-source config (task #7).
|
||||
|
||||
Missing from Go's node classifier: riscv64 arch, loong64 arch, 7z format.
|
||||
**Fixed** in this session.
|
||||
|
||||
## Categories
|
||||
|
||||
| Category | Count | Meaning |
|
||||
|----------|-------|---------|
|
||||
| match | 9 | Identical asset filenames |
|
||||
| match | 12 | Identical asset filenames |
|
||||
| go-missing | 4 | Go produces no output (alias, meta-package, or no config) |
|
||||
| live-missing | 16 | Package exists in Go but not in live cache |
|
||||
| go-extra-versions | 52 | Go has more version history (deeper fetch) |
|
||||
| live-extra-versions | 15 | Live has newer data or uses a different source |
|
||||
| go-extra-assets | 64 | Go includes assets that Node.js filters out |
|
||||
| live-extra-assets | 20 | Node.js includes assets that Go filters out |
|
||||
| live-extra-versions | 14 | Live has newer data or uses a different source |
|
||||
| go-extra-assets | 59 | Go includes assets that Node.js filters out |
|
||||
| live-extra-assets | 17 | Node.js includes assets that Go filters out |
|
||||
| live-has-meta | 43 | Node.js includes meta-assets (checksums, sigs) |
|
||||
| go-has-source-tarballs | 18 | Go includes source tarballs for no-binary releases |
|
||||
| go-has-source-tarballs | 15 | Go includes source tarballs for no-binary releases |
|
||||
|
||||
## Per-Package Checklist
|
||||
|
||||
Status: `[x]` reviewed, `[-]` known difference (acceptable), `[ ]` needs review
|
||||
|
||||
### Exact Matches (8)
|
||||
### Exact Matches (12)
|
||||
- [x] atomicparsley — match
|
||||
- [x] awless — match
|
||||
- [x] chromedriver — match (chromedist)
|
||||
- [x] dotenv-linter — match
|
||||
- [x] gpg — match (gpgdist)
|
||||
- [x] hexyl — match
|
||||
- [x] julia — match (juliadist)
|
||||
- [x] koji — match
|
||||
- [x] lf — match
|
||||
- [x] sd — match
|
||||
- [x] terraform — match (hashicorp, cleanest package — all .zip with os/arch)
|
||||
- [x] zoxide — match
|
||||
|
||||
### Go Missing — Unsupported Source (4)
|
||||
@@ -100,74 +139,101 @@ These packages differ only because Go strips checksums/sigs/SBOMs:
|
||||
- [-] sqlpkg — live-has-meta(7)
|
||||
- [-] xz — live-has-meta(1)
|
||||
|
||||
### Custom Source Types (now wired, needs detailed review)
|
||||
- [ ] chromedriver — chromedist, live-extra-versions(1), go-extra-assets(5)
|
||||
- [ ] flutter — flutterdist, live-extra-assets(1), go-extra-assets(2082)
|
||||
- [ ] go — golang, version overlap differences
|
||||
- [-] gpg — gpgdist, match
|
||||
- [ ] iterm2 — iterm2dist, live-extra-versions(20), go-extra-assets(262)
|
||||
- [ ] julia — juliadist, live-extra-assets(1), go-extra-assets(1783)
|
||||
- [ ] mariadb — mariadbdist, go-extra-assets(11)
|
||||
- [ ] terraform — hashicorp, live-extra-assets(1), go-extra-assets(5550)
|
||||
- [ ] zig — zigdist, version differences
|
||||
### Custom Source Types (version-level reviewed)
|
||||
- [x] chromedriver — chromedist, exact match at latest version
|
||||
- [-] flutter — flutterdist, live-extra-assets(217); 90% of assets have empty arch (expected — Flutter is host-arch-agnostic); mix of dev/beta/stable channels
|
||||
- [-] go — golang, live-extra-versions(98), go-extra-versions(98); different version sets; live has 1 extra source tarball (go1.9rc2.src.tar.gz)
|
||||
- [x] gpg — gpgdist, match
|
||||
- [-] iterm2 — iterm2dist, live-extra-versions(20), live-extra-assets(20)
|
||||
- [x] julia — juliadist, match
|
||||
- [-] mariadb — mariadbdist, go-extra-assets(11)
|
||||
- [x] terraform — hashicorp, match (cleanest package in the entire comparison)
|
||||
- [-] zig — zigdist, version differences
|
||||
|
||||
### Version Depth + Extra Assets (needs detailed review)
|
||||
These have real asset selection differences beyond version depth and meta filtering.
|
||||
### Version-Level Reviewed (latest version comparison)
|
||||
These were compared at the latest-version level with semver-aware sorting.
|
||||
|
||||
- [ ] aliasman — go-has-source-tarballs(8), live-extra-assets(8), go-extra-assets(8)
|
||||
- [ ] arc — go-extra-versions(9), go-has-source-tarballs(18), go-extra-assets(18)
|
||||
- [ ] bat — go-extra-versions(13), go-extra-assets(85)
|
||||
- [ ] bun — go-extra-versions(210), live-extra-versions(30), go-extra-assets(85)
|
||||
- [ ] caddy — go-extra-versions(104), live-has-meta(3415), live-extra-assets(30), go-extra-assets(1180)
|
||||
- [ ] cilium — go-extra-versions(97), go-extra-assets(5)
|
||||
- [ ] cmake — go-extra-versions(280), go-extra-assets(4352)
|
||||
- [ ] comrak — go-extra-versions(60), live-extra-versions(1), live-extra-assets(6), go-extra-assets(194)
|
||||
- [ ] crabz — go-extra-assets(4)
|
||||
- [ ] dashcore — go-extra-versions(101), live-has-meta(327), go-extra-assets(896)
|
||||
- [ ] delta — go-extra-versions(29), go-extra-assets(304)
|
||||
- [ ] deno — go-extra-versions(338), live-extra-assets(1), go-extra-assets(4)
|
||||
- [ ] duckdns.sh — go-has-source-tarballs(6), live-extra-assets(6), go-extra-assets(6)
|
||||
- [ ] fd — go-extra-versions(15), live-extra-versions(1), live-extra-assets(21), go-extra-assets(66)
|
||||
- [ ] ffmpeg — go-extra-versions(11), live-extra-versions(2), go-extra-assets(68)
|
||||
- [ ] fish — go-extra-versions(35), go-extra-assets(182)
|
||||
- [ ] fzf — go-extra-versions(46), go-extra-assets(669)
|
||||
- [ ] gh — go-extra-versions(159), go-extra-assets(2644)
|
||||
- [ ] git — go-extra-versions(339), go-extra-assets(3724)
|
||||
- [ ] gitea — go-extra-versions(194), live-has-meta(1968), go-extra-assets(4600)
|
||||
- [ ] goreleaser — go-extra-versions(556), go-extra-assets(1657)
|
||||
- [ ] grype — go-extra-versions(161), live-has-meta(90), go-extra-assets(1840)
|
||||
- [ ] hugo — go-extra-versions(335), live-has-meta(30), go-extra-assets(8176)
|
||||
- [ ] hugo-extended — go-extra-versions(335), go-extra-assets(8206)
|
||||
- [ ] jq — go-extra-versions(15), live-extra-versions(12), go-extra-assets(7)
|
||||
- [ ] k9s — go-extra-versions(227), go-extra-assets(749)
|
||||
- [ ] kind — go-extra-versions(7), live-has-meta(7)
|
||||
- [ ] kubectx — go-extra-versions(15), go-has-source-tarballs(30), go-extra-assets(78)
|
||||
- [ ] kubens — go-extra-versions(15), go-has-source-tarballs(30), go-extra-assets(78)
|
||||
- [ ] lsd — go-extra-versions(2), go-extra-assets(188)
|
||||
- [ ] mutagen — go-extra-versions(82), go-has-source-tarballs(4), go-extra-assets(2478)
|
||||
- [ ] node — go-extra-versions(713), live-extra-assets(431), go-extra-assets(15208)
|
||||
- [ ] ollama — go-extra-versions(160), live-extra-assets(8), go-extra-assets(15)
|
||||
- [ ] pandoc — go-extra-versions(123), go-extra-assets(698)
|
||||
- [ ] pathman — go-extra-assets(1)
|
||||
- [ ] postgres — go-extra-versions(4), live-extra-versions(8), live-extra-assets(6), go-extra-assets(24)
|
||||
- [ ] psql — go-extra-versions(4), live-extra-versions(4), go-extra-assets(28)
|
||||
- [ ] pwsh — go-extra-versions(163), go-extra-assets(3407)
|
||||
- [ ] rclone — go-extra-versions(92), go-extra-assets(2548)
|
||||
- [ ] rg — go-extra-versions(44), live-has-meta(100), go-extra-assets(282)
|
||||
- [ ] sass — go-extra-versions(283), go-extra-assets(2194)
|
||||
- [ ] serviceman — live-extra-versions(1), live-extra-assets(18), go-extra-assets(12)
|
||||
- [ ] shellcheck — go-extra-versions(17), go-has-source-tarballs(34), go-extra-assets(34)
|
||||
- [ ] shfmt — go-extra-versions(22), go-extra-assets(360)
|
||||
- [ ] sqlc — go-extra-versions(7), go-has-source-tarballs(2), go-extra-assets(19)
|
||||
- [ ] sttr — go-extra-versions(4), live-has-meta(42), go-extra-assets(20)
|
||||
- [ ] syncthing — go-extra-versions(431), live-extra-assets(1), go-extra-assets(11983)
|
||||
- [ ] terramate — go-extra-versions(193), live-has-meta(421), go-extra-assets(1744)
|
||||
- [ ] tinygo — go-extra-versions(19), go-extra-assets(84)
|
||||
- [ ] trip — go-extra-versions(6), go-extra-assets(12)
|
||||
- [ ] uuidv7 — go-extra-assets(16)
|
||||
- [ ] vim-commentary — go-extra-versions(5), live-extra-versions(5), live-extra-assets(5), go-extra-assets(5)
|
||||
- [ ] vim-zig — go-extra-versions(1), live-extra-versions(1), live-extra-assets(1), go-extra-assets(1)
|
||||
- [ ] watchexec — go-extra-versions(83), live-extra-versions(2), live-has-meta(1407), go-extra-assets(671)
|
||||
- [ ] xcaddy — go-extra-assets(123)
|
||||
- [ ] xsv — go-extra-versions(35), go-extra-assets(36)
|
||||
- [ ] yq — go-extra-versions(134), go-extra-assets(17)
|
||||
#### Clean at latest version (diff is only version depth + meta)
|
||||
- [x] bat — match at latest; 85 go-extra from deeper history only
|
||||
- [x] caddy — latest match except meta (.pem/.sig/.sbom); 1,180 go-extra from history
|
||||
- [x] deno — latest match; 4 go-extra from very old naming format
|
||||
- [x] fd — match at latest; old releases had bare `fd` binary with no os/arch
|
||||
- [x] flutter — match at latest version
|
||||
|
||||
#### Build variant differences (needs Variant/Extra field)
|
||||
- [-] bun — 16 go-extra at latest: `-profile` and `-baseline` variants; baseline=amd64, non-baseline=amd64v3
|
||||
- [-] ollama — live has `.tar.gz`/`.tar.zst`, Go has `.tgz`/`.zip` (different release assets); both have rocm/jetpack variants; live includes install.sh/install.ps1 (should be filtered as meta)
|
||||
- [-] pwsh — 4 go-extra at latest: all fxdependent variants; should be tagged with Extra, filtered for legacy export
|
||||
|
||||
#### Multi-source issue
|
||||
- [-] node — live merges official+unofficial (musl, riscv64, 7z); Go has separate packages; **fixed**: added riscv64/loong64 arch and 7z format to classifier
|
||||
|
||||
### Format Filtering Needed
|
||||
These packages have high go-extra-assets counts primarily from non-extractable formats:
|
||||
- [ ] cmake — 4,352 extras: .msi(706), .dmg(510), .sh(596), source tarballs, checksum files
|
||||
- [ ] hugo — 8,176 extras: .deb(1672), .pkg(54), source tarballs
|
||||
- [ ] hugo-extended — 8,206 extras (same pattern as hugo)
|
||||
- [ ] node — 15,208 extras: .pkg(756), .msi(1277), .exe(installer)
|
||||
- [ ] git — 3,724 extras: likely .exe installers, .dmg
|
||||
- [ ] pandoc — 698 extras: likely .deb, .pkg, .msi
|
||||
- [ ] pwsh — 3,407 extras: .msixbundle, .AppImage, + fxdependent variants
|
||||
- [ ] rclone — 2,548 extras: likely .deb, .rpm
|
||||
- [ ] sass — 2,194 extras
|
||||
- [ ] syncthing — 11,983 extras
|
||||
|
||||
### Version Depth Only (no action needed)
|
||||
These differ only because Go fetches deeper history:
|
||||
- [-] arc — go-extra-versions(9) + source tarballs
|
||||
- [-] cilium — go-extra-versions(97)
|
||||
- [-] comrak — go-extra-versions(60)
|
||||
- [-] delta — go-extra-versions(29)
|
||||
- [-] fish — go-extra-versions(35)
|
||||
- [-] fzf — go-extra-versions(46)
|
||||
- [-] gh — go-extra-versions(159)
|
||||
- [-] goreleaser — go-extra-versions(556)
|
||||
- [-] grype — go-extra-versions(161)
|
||||
- [-] k9s — go-extra-versions(227)
|
||||
- [-] kind — go-extra-versions(7)
|
||||
- [-] kubectx — go-extra-versions(15)
|
||||
- [-] kubens — go-extra-versions(15)
|
||||
- [-] lsd — go-extra-versions(2)
|
||||
- [-] mutagen — go-extra-versions(82)
|
||||
- [-] rg — go-extra-versions(44)
|
||||
- [-] shellcheck — go-extra-versions(17)
|
||||
- [-] shfmt — go-extra-versions(22)
|
||||
- [-] sttr — go-extra-versions(4)
|
||||
- [-] terramate — go-extra-versions(193)
|
||||
- [-] tinygo — go-extra-versions(19)
|
||||
- [-] trip — go-extra-versions(6)
|
||||
- [-] watchexec — go-extra-versions(83)
|
||||
- [-] xcaddy — go-extra-assets(123) (likely deep history)
|
||||
- [-] xsv — go-extra-versions(35)
|
||||
- [-] yq — go-extra-versions(134)
|
||||
|
||||
### Remaining (minor differences, low priority)
|
||||
- [-] aliasman — source tarball naming differences
|
||||
- [-] crabz — go-extra-assets(4)
|
||||
- [-] dashcore — go-extra-versions(101) + meta
|
||||
- [-] duckdns.sh — source tarball differences
|
||||
- [-] ffmpeg — go-extra-versions(11)
|
||||
- [-] gitea — go-extra-versions(194) + meta
|
||||
- [-] jq — go-extra-versions(15)
|
||||
- [-] pathman — go-extra-assets(1)
|
||||
- [-] postgres — version overlap differences
|
||||
- [-] psql — version overlap differences
|
||||
- [-] serviceman — source differences
|
||||
- [-] sqlc — go-extra-versions(7)
|
||||
- [-] uuidv7 — go-extra-assets(16)
|
||||
- [-] vim-commentary — version differences
|
||||
- [-] vim-zig — version differences
|
||||
|
||||
## Action Items
|
||||
|
||||
1. **Filter unsupported formats**: Add .pkg, .msi, .deb, .dmg, .sh (installer),
|
||||
.msixbundle, .rpm, .exe (bare installer), .AppImage to isMetaAsset or new filter
|
||||
2. **Tag build variants**: Populate Asset.Extra for rocm, jetpack5/6, fxdependent,
|
||||
profile; filter these in ExportLegacy for Node.js compat
|
||||
3. **Bun arch classification**: Map baseline→amd64, non-baseline→amd64v3
|
||||
4. **Node multi-source**: Merge official+unofficial into single `node` cache
|
||||
(blocked on multi-source config redesign, task #7)
|
||||
5. **Ollama meta filtering**: Filter install.sh/install.ps1 from release assets
|
||||
|
||||
@@ -20,8 +20,11 @@ import (
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/lexver"
|
||||
)
|
||||
|
||||
type cacheEntry struct {
|
||||
@@ -216,7 +219,9 @@ func compare(livePath, goPath, pkg string, latestOnly bool) packageDiff {
|
||||
for v := range vf {
|
||||
versions = append(versions, v)
|
||||
}
|
||||
sort.Strings(versions)
|
||||
slices.SortFunc(versions, func(a, b string) int {
|
||||
return lexver.Compare(lexver.Parse(a), lexver.Parse(b))
|
||||
})
|
||||
return vf, versions
|
||||
}
|
||||
|
||||
|
||||
@@ -613,6 +613,7 @@ func expandNodeFile(pkg, version, channel, date string, lts bool, baseURL, file
|
||||
"x64": "x86_64", "x86": "x86", "arm64": "aarch64",
|
||||
"armv7l": "armv7", "armv6l": "armv6",
|
||||
"ppc64": "ppc64", "ppc64le": "ppc64le", "s390x": "s390x",
|
||||
"riscv64": "riscv64", "loong64": "loong64",
|
||||
}
|
||||
|
||||
os_ := osMap[parts[0]]
|
||||
@@ -642,6 +643,8 @@ func expandNodeFile(pkg, version, channel, date string, lts bool, baseURL, file
|
||||
formats = []string{".msi"}
|
||||
case "exe":
|
||||
formats = []string{".exe"}
|
||||
case "7z":
|
||||
formats = []string{".7z"}
|
||||
case "":
|
||||
formats = []string{".tar.gz", ".tar.xz"}
|
||||
default:
|
||||
|
||||
Reference in New Issue
Block a user