Git for Windows uses tags like v2.53.0.windows.1. Node.js strips
".windows.1" and replaces ".windows.N" (N>1) with ".N".
Add NormalizeVersions to the git package and wire it into the classify
pipeline. Also add version normalization to comparecache so the
comparison uses canonical versions for both caches.
Remaining git diffs: data freshness (.windows.2 releases Go hasn't
fetched) and RC versions in Go that live doesn't have.
rocm and jetpack variants are tagged by Go's variant system but kept
by Node.js with special arch names. Filter them from comparison noise
to avoid false positives.
Match count: 70/106
Node.js releases.js only keeps MinGit assets and excludes busybox.
Add asset_filter and exclude to releases.conf to match.
Remaining diff: version normalization (.windows.N suffix stripping)
and data freshness (Go missing .windows.2 releases).
- Hugo: exclude Linux-64bit legacy filename alias
- Hugo-extended: exclude Linux-64bit legacy filename alias
- Gitea: exclude -src- and -docs- tarballs
- Pathman: exclude armv8 legacy alias
- UUID v7: exclude exotic architectures (thumb, armeb, loong, gnux32, risc)
- comparecache: filter bare executables and docs tarballs as noise,
apply noise filter to both live and Go sides
- legacy.go: add .tar.bz2 to legacyFormats
Match count: 69/106 (up from 58)
Move all source-specific classifiers, variant tagging, config filtering,
and readAllRaw out of cmd/webicached into internal/classifypkg. The new
Package() function runs the full classify pipeline: source dispatch →
tag variants → apply config.
webicached now only handles fetching raw data and writing to fsstore.
The classification logic is reusable by comparecache and future tools.
- gittag classifier: use "{repo}-{tag}" filenames (matching Node.js),
strip "v" prefix from version, synthesize date-based version for
tagless repos (HEAD of master/main)
- GitHub source-only: use "git" format (no dot) and "{repo}-{tag}"
filename for clone assets
- Legacy export: add "git" to recognized formats so gittag packages
appear in the legacy cache
- Derives repo name from the git URL in releases.conf
vim-commentary now matches. vim-zig matches on format but has newer
data (expected — Go fetched more recently than Node.js).
Moved isMetaAsset from cmd/webicached to classify.IsMetaAsset so
both webicached and comparecache use the same logic. Removed
duplicated isMetaFile from comparecache. The comparecache
isLiveNoise now delegates to classify.IsMetaAsset and adds
live-specific filters (.deb, .rpm, -src-).
Drop the Owner-Repo prefix from source tarball filenames — the
actual download name comes from Content-Disposition. Added TODO
to resolve the full filename via HEAD at fetch time.
Source-only releases (no uploaded assets) now also emit a .git
asset with the GitHub clone URL, matching how gittag-sourced
packages like vim-commentary and vim-zig work. This allows
install via git clone --branch <tag> as an alternative to
downloading the tarball.
Source-only GitHub releases now use the API-provided tarball_url
and zipball_url directly. Filename follows the legacy pattern
(Owner-Repo-Tag.ext) to approximate the Content-Disposition
filename that Node.js gets by following the redirect.
Source-only releases now use the API-provided tarball_url and
zipball_url directly instead of guessing the archive URL format.
The filename uses the git tag, and the download URL is what
GitHub's API actually returns.
The macOS .pkg is a pkgutil installer, not a plain archive.
Tagged as installer so it's excluded from legacy export but
available for Go's native installer support.
The node package already merges both sources via unofficial_url
in releases.conf. The split packages were a workaround that
produced cache files not present in the live Node.js cache.
Strips known noise from the live cache before comparison: .deb, .rpm,
.asc, .sig, .gpg, .sbom, .sha256, checksums, install.sh, install.ps1,
.txt, and other non-installable files. Matches went from 16 to 50.
Uses Node.js version range (2nd to 2nd-to-last) as the window.
All Node.js versions in the window are included so missing Go
versions/assets are visible. Go-only versions are hidden since
those are just deeper fetch history, not real gaps.
shellcheck has no Windows builds, xz has no arm64 builds — these are
real upstream gaps that the test suite now surfaces as failures rather
than silently excluding. 891 pass, 2 known upstream gaps.
The resolver now handles:
- ANYOS assets match any query OS
- posix_2017/posix_2024 assets match any non-Windows OS
- ANYARCH assets match any query architecture (ranked below specific)
14 tests covering: exact match, version constraints, arch fallback
(Rosetta 2, Windows ARM64, micro-arch), format preference, libc
filtering, base-over-variant preference, POSIX/ANYOS/ANYARCH fallback,
Survey catalog, and no-match.
- yq: move man_page_only from general isMetaAsset to yq-specific tagger
- node: restore .exe as stored asset with "bare-exe" variant (installable
by Go, excluded from legacy)
- ollama: rename Ollama-darwin.zip variant from "installer" to "app"
(.app bundle is installable by Go, just not by legacy Node.js)
The distinction: general classification/filter (isMetaAsset) handles
truly non-installable files. Installer-specific taggers handle assets
that are installable but need variant tagging. Legacy filter strips
variants and unsupported formats for Node.js compat.
Node.js normalizes .tgz extensions to .tar.gz in the cache name field
while keeping the real .tgz URL in download. Match this behavior so
legacy export filenames are consistent. Affects ollama-darwin.tgz and
any other packages using .tgz.
Node.js index lists "win-x64-exe" but there's no .exe file on the
download server. The MSI installer (separate "msi" entry) is the actual
Windows installer. The "exe" entry was generating a phantom filename.
asset_filter is a substring that asset filenames must contain. Used when
multiple packages share a GitHub release (kubectx/kubens both come from
ahmetb/kubectx). Added as a first-class Conf field and applied in
webicached's applyConfig.
MinGit-busybox is a stripped-down MinGit using busybox instead of MSYS2.
pdbs-for-git-* filenames weren't caught by the existing "-pdb" check.
Both are now tagged as variants and excluded from legacy export.
fish-{version}.tar.xz is an uploaded source tarball with no OS/arch in
the filename. GitHub API doesn't distinguish it from binaries. Tag assets
with no OS and no arch as "source" variant so they're filtered from
legacy export. The linux .tar.xz binaries classify correctly and are
kept — Node.js just doesn't have them yet.
Baseline builds (-baseline suffix) are plain x86_64 and match what Node.js
serves. Strip -baseline from Filename (keep in Download URL) so legacy
export sees a clean name. Non-baseline builds get Arch: x86_64_v3 and
Variants: ["v3"], excluding them from legacy output.
Add .tar.bz2 to classifier format detection (was slipping through
as empty format). Update COMPARISON.md with fresh results: 21 exact
matches, .deb/.rpm/.tar.zst/.tar.bz2 now correctly filtered from
legacy export. Document remaining items for review.
ExportLegacy now skips assets with non-empty Variants (installer,
rocm, fxdependent, etc.) and formats Node.js doesn't handle (.deb,
.rpm, .snap, .appx, .tar.zst, .tar.bz2, .7z). This ensures the
_cache/ JSON files are compatible with the legacy Node.js server.
Also fix test data to use dotted format strings (.tar.gz) matching
what the classifier actually produces.
Drop VariantTagger interface and map-based lookup. Each per-installer
package now exports a plain TagVariants function. webicached dispatches
via a switch on package name, consistent with fetchRaw and
classifyPackage.
Move variant detection logic from inline functions in webicached to
per-installer packages (internal/releases/{bun,fish,git,lsd,node,
ollama,pwsh,xcaddy}). Each exports a Tagger implementing the new
storage.VariantTagger interface. webicached uses an explicit map
of package name → tagger, no magic registration.
28 exact matches at latest version (up from 12). Reorganize by
difference category. Update action items to reflect current design
(Variants field, format handling, node multi-source fix).
Extract shared state (store, client, auth, rawDir, config flags) into
a WebiCache struct. Convert refreshPackage, fetchRaw, and paginated
fetchers (github, gitea, gittag, nodedist) to methods.
Add -shallow flag: fetches only the first page of releases from
paginated sources. Single-index sources (nodedist, chromedist, etc.)
are always complete in one request.
Extra is for version-related sort metadata (build numbers, etc.).
Variants captures build qualifiers like "rocm", "jetpack5",
"fxdependent", "installer" — things the resolver should skip by
default unless explicitly requested.
Also update format classification docs: most formats (.pkg, .deb,
.dmg, .msi) are extractable — only .exe is ambiguous and needs
the "installer" variant tag when it's not the actual binary.
Installer formats (.pkg, .msi, .deb, etc.) get Extra="installer"
rather than being filtered at classification time. The resolver
skips them by default but the full API can still serve them.
Add unofficial_url to node/releases.conf and update the nodedist
fetcher/classifier to fetch from both URLs. Raw entries are stored
with "official/" or "unofficial/" tag prefixes so they don't overwrite
each other. The classifier picks the correct base URL from the prefix.
This matches the Node.js releases.js behavior which merges both sources,
adding musl, riscv64, loong64, and 7z builds from unofficial.
