universal2 → emit x86_64 (cmake/hugo losing years of releases)
solaris/illumos → keep original values, don't translate to sunos
ARM → keep classifier-matching values (armhf, armel, armv7a)
android → drop from legacy export
The Node build-classifier re-parses filenames independently and drops any
cache entry where its extraction doesn't match the pre-classified field.
New drops in ExportLegacy (with LegacyDropStats tracking):
- universal2/universal1 arch: classifier maps 'universal' in filename to
x86_64 and rejects entries with arch='universal2'
- solaris/illumos OS: Node never served these; classifier mismatches
are unfixable without changing the filename
- android OS: classifier maps android filenames to linux
ARM arch translations in legacyFieldBackport (filename-based):
- gnueabihf / armhf filename → 'armhf' (Go normalizes to armv6/armv7;
Node classifier preserves the original Debian/Rust naming)
- armel filename → 'armel' (Go normalizes to armv6)
- armv5 filename → 'armel' (Node tiered map: armv5 falls back to armel)
- armv7a filename → 'armv7a' (Go normalizes to armv7)
- armv7l / armv6l: no translation needed (both Go and Node say armv7/armv6)
LIVE_cache confirms: go.json uses 'illumos' and 'solaris' as distinct
values. No package in the live cache uses 'sunos'. The build-classifier
(triplet.js) also keeps all three distinct (illumos, solaris, sunos).
The previous sunos translation matched the old normalize.js path, not
the build-classifier path that the Go rewrite targets.
Covers all cases where canonical data can't be expressed in the legacy
Node.js cache format, with explicit verification:
- Variant builds dropped and counted (Node.js has no variant logic)
- Unknown formats dropped and counted (.apk, .AppImage, .deb, .rpm)
- Empty format passes through (bare binaries, git sources)
- solaris/illumos translated to sunos (Node.js only knows sunos)
- ffmpeg windows .gz overridden to .exe (per-package compat rule)
- ffmpeg translation not applied to other packages
- universal2 passes through as-is (Node.js WATERFALL handles it)
- Mixed: correct counts when drops and translations occur together
universal2 entries, solaris/illumos, android, and ARM variant mismatches
all need to be filtered or translated in ExportLegacy before writing to
cache JSON. The Node classifier is not being modified.
Go armv6l dropped from 1,936 to 648 warnings. All tests passing.
Remaining 7,606 warnings are informational classifier limitations.
No further GOER action needed.
The Node classifier maps armv6l filenames → armv6, so the cache should
emit armv6 not arm. The previous backport predates the armv6l→armv6
normalizeGoArch fix and is no longer needed.
Expanding universal2 into aarch64+x86_64 entries creates filename/arch
mismatches: the filename still contains 'universal' so the Node classifier
re-parses it as universal2, then sees arch=aarch64 and flags a mismatch.
Keep universal2 as-is; the Node agent will add it to their WATERFALL.
Classifier fixes:
- Remove Windows arm→arm64 auto-promotion; packages like caddy/fzf/goreleaser
have genuine arm32 Windows builds (windows_armv6) that were wrongly promoted
- Add armel and gnueabihf as ARMv6 aliases (jq, caddy and others use these)
- Add winx64 to Windows OS pattern (MariaDB uses winx64 in filenames)
- Add ppc64el as ppc64le alias (Debian/Ubuntu naming, used by jq)
- Normalize armv6l → armv6 in normalizeGoArch (Go dist had armv6l filenames)
- Fix classifyGPGDist hardcoded "amd64" → buildmeta.ArchAMD64 ("x86_64")
Legacy export fixes:
- Map solaris/illumos → sunos globally (Node.js only knows "sunos")
- Expand universal2 → two entries (aarch64 + x86_64) so Hugo/cmake/gh/syncthing
work on both Apple Silicon and Intel Mac in the legacy resolver
- Remove double-application of legacyFieldBackport (toLegacy no longer calls it)
- Remove LegacyBackport from classifypkg and webicached; canonical values
now flow through storage untouched
- Add legacyFieldBackport() in storage/legacy.go, called only at export time
(go: armv6→arm, ffmpeg windows: .gz/.empty→.exe)
- ExportLegacy now takes pkg name and returns LegacyDropStats (variants + formats dropped)
- fsstore.Commit logs dropped assets so filtering is visible
- Add FormatAPK (.apk) and FormatAppImage (.AppImage) to buildmeta and classify
so those files are properly classified and then correctly dropped from legacy export
rather than passing through as empty-format
Implements storage.Store for PostgreSQL using pgx/v5.
Schema uses double-buffered generations per package — write into the
inactive gen, then atomically swap the active pointer on Commit. Readers
always see a complete consistent snapshot.
Write path: BeginRefresh → Put (staged in-memory) → Commit (CopyFrom + swap)
Read path: Load → reads active gen from webi_packages, fetches assets
Both webid and webicached now accept -pg=<dsn> to use pgstore instead
of fsstore. Schema is applied idempotently on startup.
Also:
- storage.Store interface gains ListPackages(ctx) — fsstore reads the
directory; pgstore queries webi_packages
- webid.loadAll() uses ListPackages instead of filepath.ReadDir
- Fixed .gitignore: /webid (root binary) was incorrectly matching cmd/webid/
Move legacy-specific field translations out of the core classifier into
LegacyBackport(), called by webicached before writing the JSON cache.
Core classifier now outputs canonical values:
- Go dist arm → armv6 (correct per GOARM default)
- ffmpeg Windows .gz → .gz (correct file extension)
LegacyBackport remaps for Node.js compat:
- Go dist armv6 → arm (production keeps raw API value)
- ffmpeg Windows .gz → exe (production releases.js override)
sass armv6→armv7 stays in classifier (Dart Sass genuinely targets ARMv7).
The comparecache equivalence matching hides the issue. The Node
build-classifier needs normalized values (armv6 not armhf, sunos not
solaris) in the actual cache JSON files.
- sass: bare arm → armv7 (Dart Sass targets ARMv7, not v6)
- ffmpeg: Windows .gz → ext exe (gzipped bare executables)
- go: keep bare arm as-is from Go dist API (matches production)
Reduces comparecache diffs from 6 packages to 3 (iterm2 channel edge
cases, postgres legacy ext, terraform alpha detection — all understood).
The golang dist API provides structured os/arch fields. Using these
instead of filename-based classification fixes:
- illumos/solaris kept distinct (not merged to sunos)
- arm arch correctly mapped per GOARCH convention
- buildmeta: add OSIllumos and OSSolaris constants
Replace direct string comparison with canonical equivalence checks so
naming convention differences (darwin/macos, x86_64/amd64, aarch64/arm64)
don't appear as false diffs. Now only real classification disagreements
surface:
- go: illumos/solaris→sunos mapping, arm ambiguity per OS
- sass: bare "arm" should be armv7, not armv6
- ffmpeg: Windows .gz ext classified as exe in prod
- terraform: alpha channel detected correctly by Go, missed by prod
- postgres: legacy EDB ext "tar" vs "tar.gz"
- pg/releases.conf: add asset_filter=postgres so pg only returns server
assets (which include the client), matching production releases.js
- classifypkg: add "pg" to postgres version normalizer switch case
- comparecache: compare os/arch/libc/ext/channel fields on shared assets,
distinguishing real disagreements (diff-*) from expected fill diffs
where Go classifies at write time but Node.js leaves fields empty
9 categories: universal2, solaris/illumos, armhf, armel, windows arm,
android, winx64, minor arch mismatches, sttr pkg misclassification.
Plus broad sweep failures and live-compare known diffs.
- Windows gnu (MinGW) builds are self-contained: classify as libc='none'
- Pad install.sh content to 8 spaces to match production template indent
- Use replaceMarkerLine for both bash and PS1 installer injection
Rust *-unknown-linux-musl builds are statically linked with zero
runtime libc dependency. Detect this pattern in classifyGitHub and
override libc from 'musl' to 'none'. Hard-musl packages (pwsh, bun,
node) use different filename patterns and keep libc='musl'.
Add PowerShell() function to render .ps1 installers by injecting
$Env: variables and splicing install.ps1 content. Wire it into
the webid server for .ps1 extension requests.
