vim-ale/PROD_NOTES.md

Production Notes: Node.js Cache-Only Migration

Current State

The Node.js server no longer fetches from upstream APIs. It reads only from _cache/YYYY-MM/{pkg}.json files generated by the Go webicached daemon. All per-package releases.js files and _common/ fetcher libraries have been deleted — they are no longer needed.

Changes Made

• builds.js: Removed freshenRandomPackage() calls and background refresh
• builds-cacher.js: Removed getLatestBuilds(), stale re-fetch, and freshenRandomPackage(). Missing cache files return empty metadata.
• builds-cacher.js getProjectTypeByEntry(): Replaced require(releases.js) with cache file existence check. Packages are valid if they have a cache file, selfhosted if they don't.
• transform-releases.js: Legacy release API now reads from cache files instead of fetching upstream. Pre-classified fields are cleared before normalize() so the output matches the legacy format (e.g. darwin → macos, versions without v prefix).
• classify-one.js: Updated to read from cache instead of releases.js. Fixed hardcoded triplet key.
• Deleted 94 {pkg}/releases.js files and 8 _common/*.js files

Data Format Note

The Go cache uses different OS/arch names than the legacy normalize.js:

• Go: darwin → normalize.js: macos
• Go: amd64 → normalize.js: amd64 (same)
• Go: aarch64 → normalize.js: arm64

For the legacy /api/releases/ endpoint (transform-releases.js), we clear pre-classified fields and re-normalize from filenames to match production output.

For the installer endpoint (builds-cacher.js + serve-installer.js), the Go naming is used directly — the build-classifier handles its own mappings.

Known Pre-existing Issues

• transform-releases.js self-test: The if (require.main === module) block calls module.exports({...}) but the module exports an object.
• armhf classification warnings: arm-unknown-linux-gnueabihf triggers "wrong arch" warnings in the build-classifier (armhf vs armv6).
• Go illumos/solaris warnings: Go's illumos and solaris builds trigger "wrong os" warnings (expected sunos).

Testing

All paths verified locally:

# builds-cacher: load packages from cache
node -e "let B = require('./_webi/builds.js'); B.init().then(...);"

# transform-releases: legacy API from cache
node -e "let R = require('./_webi/transform-releases.js'); R.getReleases({pkg:'bat', ...});"

# serve-installer: full end-to-end resolution
node -e "let I = require('./_webi/serve-installer.js'); I.helper({unameAgent:'aarch64/unknown Darwin/24.2.0 libc', ...});"

# classify-one: dev tool reads cache
node _webi/classify-one.js bat

Public API Endpoints (live at webinstall.dev)

• GET /api/releases/{pkg}.json — Returns JSON with oses, arches, libcs, formats, and releases array. Uses macos not darwin.
• GET /api/releases/{pkg}.tab — Tab-separated release data
• GET /{pkg}@{tag} — Returns installer script (bash or ps1 based on UA)

The HTTP routing is NOT in this repo — an external server calls into the Node modules (Releases.getReleases() and InstallerServer.serveInstaller()).

Project Type Detection

builds-cacher.js:getProjectTypeByEntry() classifies packages:

Type	Meaning	Check
alias	Symlink or README has alias: x	symlink or README frontmatter
valid	Has cache file	_cache/YYYY-MM/{name}.json exists
selfhosted	No cache file	cache file missing
hidden	System dirs, _*, .* etc	naming convention
invalid	No README.md	file check

Package Counts

• 99 packages detected as valid (have cache files)
• 58 packages detected as selfhosted (no cache)
• 33 packages detected as alias