From 992d50eacac5eecf418c2493adea54be70aff76f Mon Sep 17 00:00:00 2001 From: AJ ONeal Date: Wed, 11 Mar 2026 12:48:34 -0600 Subject: [PATCH] docs: detailed Go cache classification issues from Node tests 9 categories: universal2, solaris/illumos, armhf, armel, windows arm, android, winx64, minor arch mismatches, sttr pkg misclassification. Plus broad sweep failures and live-compare known diffs. --- QUESTIONS.md | 157 ++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 123 insertions(+), 34 deletions(-) diff --git a/QUESTIONS.md b/QUESTIONS.md index 3a45f81..fe905fc 100644 --- a/QUESTIONS.md +++ b/QUESTIONS.md @@ -1,47 +1,136 @@ # Questions/Notes from ref-webi-go-2 agent (Node cache-only work) -## Issue 1: WATERFALL libc vs gnu (build-classifier submodule) +## Resolved -The Go cache correctly uses `libc='gnu'` for glibc-linked Linux binaries (bat, -rg, node — any Rust or C project with explicit gnu triplets in filenames). This -is more precise than the old `normalize.js` which mapped everything to -`libc='none'`. +- [x] Issues 1-3, 5: WATERFALL, .git shadowing, man pages, musl libc — all fixed +- [x] Node-side fixes: WATERFALL patch, ANYOS-last ordering, version-first iteration -However, the build-classifier `host-targets.js` WATERFALL maps: -``` -libc: ['none', 'libc'] -``` +## Open: Go cache classification issues -A host reporting `libc` (meaning "I have standard glibc") never tries `gnu` -triplets. So `curl webi.sh/bat` on Linux glibc fails to find a match. +These are all issues where the Go cache has `os`, `arch`, or `libc` values that +the Node build-classifier doesn't recognize, causing PACKAGE FORMAT CHANGE +warnings and, in some cases, resolution failures. -**Fix needed**: Update WATERFALL in `host-targets.js`: -``` -libc: ['none', 'gnu', 'libc'] -``` +### 1. `universal2` arch not mapped to aarch64/x86_64 -This should be safe — a `libc` host (standard glibc) can run `gnu`-linked -binaries. The ordering puts `none` first (static/no-libc-dep preferred), then -`gnu` (glibc-linked), then `libc` as final fallback. +The Go cache classifies darwin universal builds as `arch: "universal2"`. The +Node classifier expects `aarch64` or `x86_64`. This causes ~7k warnings across +hugo, hugo-extended, cmake, syncthing, gh. -Note: `musl` hosts already work correctly — the WATERFALL has -`musl: ['none', 'musl']` and musl-specific builds are properly classified. +**Affected packages**: hugo, hugo-extended, cmake, syncthing, gh -## Issue 2: Go cache includes `.git` source URLs as releases +**Fix options** (pick one): +- Go cache emits **two entries** per universal build (one aarch64, one x86_64) +- Go cache maps `universal`/`universal2` → `aarch64` (and Node WATERFALL adds + `aarch64` fallback for `x86_64` hosts) -**This is a regression.** Production jq and caddy work correctly today — the old -`releases.js` fetchers never included `.git` source URLs. The Go cache adds -them, which creates `ANYOS-ANYARCH-none` triplets. Since the triplet enumeration -tries ANYOS/ANYARCH *before* specific OS/arch, the `.git` entry matches first -and shadows all platform-specific binaries. +Hugo is the worst case — v0.100+ **only** has universal macOS builds (`.pkg` +format). No recent aarch64/x86_64 `.tar.gz` exist. Last non-universal: v0.99.1. -Verified against production: -- `curl 'https://webinstall.dev/api/releases/jq@stable.json?os=macos&arch=arm64&limit=1'` - → returns `jq-macos-arm64` (exe, correct) -- Our code → returns `jq.git` (ANYOS, wrong) +### 2. `solaris`/`illumos` not mapped to `sunos` -**Fix**: The Go cache should not include `.git` source repo URLs as release -assets. They aren't installable binaries. +The Go cache uses `solaris` and `illumos` as OS values. The Node classifier +expects `sunos` for both. -Affected packages (at minimum): jq, caddy — any package whose GitHub releases -page includes the auto-generated source links that GitHub adds to every release. +**Affected packages**: go, hugo, caddy, lf, syncthing, terraform, mutagen, +rclone, monorel, runzip, uuidv7 + +**Fix**: Map `solaris` → `sunos` and `illumos` → `sunos` in the Go cache. +(Or keep both and let Node handle the mapping — but fixing in Go is cleaner.) + +### 3. `armhf` not mapped to a recognized arch + +The Go cache classifies `arm-unknown-linux-gnueabihf` as `arch: "armhf"`. The +Node classifier expects this to be `armv6` (which is what production uses). + +`gnueabihf` = ARMv6 hard-float. The Rust triplet `arm-unknown-linux-gnueabihf` +targets ARMv6 (Raspberry Pi 1/Zero). + +**Affected packages**: bat, delta, fd, hexyl, lsd, rg, sd, jq, shellcheck, +dashcore, kubectx, kubens, uuidv7 + +**Fix**: Map `gnueabihf` → `armv6` in the Go classifier. + +### 4. `armel` not mapped to a recognized arch + +The Go cache classifies some ARM builds as `arch: "armel"`. The Node classifier +expects `armv6` or `armv7`. + +`armel` = soft-float ARM, typically ARMv5/v6. In webi's context these should +map to `armv6`. + +**Affected packages**: caddy, fzf, gitea, pathman, xcaddy + +**Fix**: Map `armel` → `armv6` in the Go classifier. + +### 5. `windows arm` classified as `aarch64` instead of `arm` + +Several packages have `windows-arm` builds that the Go cache classifies with +`arch: "aarch64"`. The Node classifier says `arm != aarch64`. + +These are genuinely `arm` (32-bit ARM) builds, not `aarch64`. The Go classifier +is wrong here — `windows-arm` ≠ `windows-arm64`. + +**Affected packages**: caddy, curlie, dashmsg, ffuf, fzf, goreleaser, gprox, +runzip, sclient, uuidv7, xcaddy + +**Fix**: Don't promote `arm` to `aarch64` for Windows. `arm` = 32-bit ARM, +`arm64` = aarch64. + +### 6. `android` classified as `linux` + +The Go cache maps Android builds to `os: "linux"`. Production treats Android +as its own OS. + +**Affected packages**: fzf, lf, runzip, sass, uuidv7 + +**Fix**: Keep `android` as a distinct OS in the Go cache (don't collapse to +`linux`). + +### 7. `winx64` not parsed — mariadb OS missing + +MariaDB's download URLs use `winx64` (e.g. `mariadb-11.4.0-winx64.zip`). The +Go cache doesn't detect `win` as `windows` in this context, leaving `os` empty. + +**Affected packages**: mariadb (61 versions) + +**Fix**: Teach the Go classifier that `winx64` → `os: "windows"`, `arch: "x86_64"`. + +### 8. Minor arch mismatches + +- `mips64r6`/`mips64r6el` → should be `mips64` (jq) +- `ppc64el` → should be `ppc64le` (jq) +- `arma` → should be `arm` (zig) + +### 9. `sttr` macOS `.pkg` misclassified as Linux + +sttr's `.pkg` files are classified with `os: "linux"` but they're macOS +packages: `sttr_linux_386.pkg`, `sttr_linux_amd64.pkg`, `sttr_linux_arm64.pkg`. +The filename says "linux" but `.pkg` is a macOS format. This is an upstream +naming bug in sttr — the Go cache is correctly parsing the filename, but the +result is wrong. + +**Fix**: Low priority — this is sttr's bug, not the classifier's. + +## Broad sweep failures (6/196) + +These packages fail with `v0.0.0 ext=err` on both production and local: +- **git** (macOS arm64, Linux amd64) +- **gpg** (Linux amd64) +- **iterm2** (macOS arm64, Linux amd64) +- **mariadb** (macOS arm64) + +git, gpg, and iterm2 are self-hosted installers that don't have downloadable +binaries. mariadb fails because `winx64` URLs aren't parsed (see issue 7 above) +and no macOS builds exist. + +## Live-compare known diffs (5/49) + +These differ from production because the Go cache correctly excludes +non-installable files that production accidentally includes: +- **bat linux/amd64**: live returns `.deb`, local returns `.tar.gz` (correct) +- **caddy macos/amd64, macos/arm64**: live returns `.pem`, local returns `.tar.gz` +- **caddy linux/amd64**: live returns `.deb`, local returns `.tar.gz` +- **caddy windows/amd64**: live returns `.pem`, local returns `.zip` + +These are improvements — `.deb` and `.pem` aren't installable by webi.