The Node build-classifier re-parses filenames independently and drops any
cache entry where its extraction doesn't match the pre-classified field.
New drops in ExportLegacy (with LegacyDropStats tracking):
- universal2/universal1 arch: classifier maps 'universal' in filename to
x86_64 and rejects entries with arch='universal2'
- solaris/illumos OS: Node never served these; classifier mismatches
are unfixable without changing the filename
- android OS: classifier maps android filenames to linux
ARM arch translations in legacyFieldBackport (filename-based):
- gnueabihf / armhf filename → 'armhf' (Go normalizes to armv6/armv7;
Node classifier preserves the original Debian/Rust naming)
- armel filename → 'armel' (Go normalizes to armv6)
- armv5 filename → 'armel' (Node tiered map: armv5 falls back to armel)
- armv7a filename → 'armv7a' (Go normalizes to armv7)
- armv7l / armv6l: no translation needed (both Go and Node say armv7/armv6)
LIVE_cache confirms: go.json uses 'illumos' and 'solaris' as distinct
values. No package in the live cache uses 'sunos'. The build-classifier
(triplet.js) also keeps all three distinct (illumos, solaris, sunos).
The previous sunos translation matched the old normalize.js path, not
the build-classifier path that the Go rewrite targets.
Covers all cases where canonical data can't be expressed in the legacy
Node.js cache format, with explicit verification:
- Variant builds dropped and counted (Node.js has no variant logic)
- Unknown formats dropped and counted (.apk, .AppImage, .deb, .rpm)
- Empty format passes through (bare binaries, git sources)
- solaris/illumos translated to sunos (Node.js only knows sunos)
- ffmpeg windows .gz overridden to .exe (per-package compat rule)
- ffmpeg translation not applied to other packages
- universal2 passes through as-is (Node.js WATERFALL handles it)
- Mixed: correct counts when drops and translations occur together
The Node classifier maps armv6l filenames → armv6, so the cache should
emit armv6 not arm. The previous backport predates the armv6l→armv6
normalizeGoArch fix and is no longer needed.
Expanding universal2 into aarch64+x86_64 entries creates filename/arch
mismatches: the filename still contains 'universal' so the Node classifier
re-parses it as universal2, then sees arch=aarch64 and flags a mismatch.
Keep universal2 as-is; the Node agent will add it to their WATERFALL.
Classifier fixes:
- Remove Windows arm→arm64 auto-promotion; packages like caddy/fzf/goreleaser
have genuine arm32 Windows builds (windows_armv6) that were wrongly promoted
- Add armel and gnueabihf as ARMv6 aliases (jq, caddy and others use these)
- Add winx64 to Windows OS pattern (MariaDB uses winx64 in filenames)
- Add ppc64el as ppc64le alias (Debian/Ubuntu naming, used by jq)
- Normalize armv6l → armv6 in normalizeGoArch (Go dist had armv6l filenames)
- Fix classifyGPGDist hardcoded "amd64" → buildmeta.ArchAMD64 ("x86_64")
Legacy export fixes:
- Map solaris/illumos → sunos globally (Node.js only knows "sunos")
- Expand universal2 → two entries (aarch64 + x86_64) so Hugo/cmake/gh/syncthing
work on both Apple Silicon and Intel Mac in the legacy resolver
- Remove double-application of legacyFieldBackport (toLegacy no longer calls it)
- Remove LegacyBackport from classifypkg and webicached; canonical values
now flow through storage untouched
- Add legacyFieldBackport() in storage/legacy.go, called only at export time
(go: armv6→arm, ffmpeg windows: .gz/.empty→.exe)
- ExportLegacy now takes pkg name and returns LegacyDropStats (variants + formats dropped)
- fsstore.Commit logs dropped assets so filtering is visible
- Add FormatAPK (.apk) and FormatAppImage (.AppImage) to buildmeta and classify
so those files are properly classified and then correctly dropped from legacy export
rather than passing through as empty-format
Implements storage.Store for PostgreSQL using pgx/v5.
Schema uses double-buffered generations per package — write into the
inactive gen, then atomically swap the active pointer on Commit. Readers
always see a complete consistent snapshot.
Write path: BeginRefresh → Put (staged in-memory) → Commit (CopyFrom + swap)
Read path: Load → reads active gen from webi_packages, fetches assets
Both webid and webicached now accept -pg=<dsn> to use pgstore instead
of fsstore. Schema is applied idempotently on startup.
Also:
- storage.Store interface gains ListPackages(ctx) — fsstore reads the
directory; pgstore queries webi_packages
- webid.loadAll() uses ListPackages instead of filepath.ReadDir
- Fixed .gitignore: /webid (root binary) was incorrectly matching cmd/webid/
Serves /api/releases/{pkg}@{version}.json and .tab matching the
Node.js format. Supports query params for os, arch, libc, channel,
formats, lts, limit. Handles selfhosted packages (install.sh only).
Pre-loads all cached packages on startup. Includes /api/debug for
UA detection and /api/health endpoint.
Packages with alias_of in releases.conf (e.g. dashd → dashcore,
golang → go) now get symlinked cache files so they resolve to the
same JSON as their target. 13 aliases total.
Added AliasOf as a proper field in installerconf.Conf, LinkAlias
method to fsstore, and alias handling in webicached's Run loop.
- ffmpeg: add version_prefix = b to strip 'b' from tags (b6.0 → 6.0)
- legacy.go: add .gz to legacyFormats for bare gzipped binaries
- iterm2: broaden regex to handle preview/beta variants, skip empty
versions
Match count: 75/106
- Add .app.zip to legacyFormats so macOS fish builds export correctly
- Exclude bundledpcre, fish-static, OpenBeta from fish/releases.conf
- Add fish Linux binaries to comparecache noise (Go improvement)
Match count: 72/106
- Hugo: exclude Linux-64bit legacy filename alias
- Hugo-extended: exclude Linux-64bit legacy filename alias
- Gitea: exclude -src- and -docs- tarballs
- Pathman: exclude armv8 legacy alias
- UUID v7: exclude exotic architectures (thumb, armeb, loong, gnux32, risc)
- comparecache: filter bare executables and docs tarballs as noise,
apply noise filter to both live and Go sides
- legacy.go: add .tar.bz2 to legacyFormats
Match count: 69/106 (up from 58)
- gittag classifier: use "{repo}-{tag}" filenames (matching Node.js),
strip "v" prefix from version, synthesize date-based version for
tagless repos (HEAD of master/main)
- GitHub source-only: use "git" format (no dot) and "{repo}-{tag}"
filename for clone assets
- Legacy export: add "git" to recognized formats so gittag packages
appear in the legacy cache
- Derives repo name from the git URL in releases.conf
vim-commentary now matches. vim-zig matches on format but has newer
data (expected — Go fetched more recently than Node.js).
ExportLegacy now skips assets with non-empty Variants (installer,
rocm, fxdependent, etc.) and formats Node.js doesn't handle (.deb,
.rpm, .snap, .appx, .tar.zst, .tar.bz2, .7z). This ensures the
_cache/ JSON files are compatible with the legacy Node.js server.
Also fix test data to use dotted format strings (.tar.gz) matching
what the classifier actually produces.
Drop VariantTagger interface and map-based lookup. Each per-installer
package now exports a plain TagVariants function. webicached dispatches
via a switch on package name, consistent with fetchRaw and
classifyPackage.
Move variant detection logic from inline functions in webicached to
per-installer packages (internal/releases/{bun,fish,git,lsd,node,
ollama,pwsh,xcaddy}). Each exports a Tagger implementing the new
storage.VariantTagger interface. webicached uses an explicit map
of package name → tagger, no magic registration.
Extra is for version-related sort metadata (build numbers, etc.).
Variants captures build qualifiers like "rocm", "jetpack5",
"fxdependent", "installer" — things the resolver should skip by
default unless explicitly requested.
Also update format classification docs: most formats (.pkg, .deb,
.dmg, .msi) are extractable — only .exe is ambiguous and needs
the "installer" variant tag when it's not the actual binary.
storage.Store is the read/write interface for release asset storage.
storage.Asset uses correct terminology (Filename, Format) internally.
storage.LegacyAsset / LegacyCache preserve the Node.js wire format
("releases", "name", "ext") for backward compatibility.
fsstore writes to _cache/YYYY-MM/{pkg}.json with atomic rename,
matching the existing Node.js layout. The Node.js server can read
files written by Go and vice versa.
