mirror of
https://github.com/webinstall/webi-installers.git
synced 2026-06-08 00:36:34 +00:00
Compare commits
11 Commits
feat/webi-
...
ref-cache-
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
785dc05324 | ||
|
|
73188d50e1 | ||
|
|
bf5cafac18 | ||
|
|
1e499ed6c8 | ||
|
|
f638a25529 | ||
|
|
95418b1023 | ||
|
|
f66822295b | ||
|
|
c538942392 | ||
|
|
af28ddb686 | ||
|
|
631147901a | ||
|
|
b3375d0e24 |
6
.gitignore
vendored
6
.gitignore
vendored
@@ -23,6 +23,11 @@ install-*.ps1
|
||||
_cache/
|
||||
node_modules/
|
||||
|
||||
# local test fixtures (regenerated by _webi/test-live-*.js)
|
||||
testdata/
|
||||
LIVE_cache/
|
||||
distributables.csv
|
||||
|
||||
# temporary & backup files
|
||||
.*.sw*
|
||||
*.bak
|
||||
@@ -30,6 +35,7 @@ node_modules/
|
||||
|
||||
# agent session files
|
||||
agents/
|
||||
LOCAL.md
|
||||
|
||||
# other
|
||||
.DS_Store
|
||||
|
||||
@@ -1 +1,2 @@
|
||||
DELETEMEnode_modules/**/*
|
||||
node_modules/**/*
|
||||
_webi/test-*.js
|
||||
|
||||
129
AGENTS.md
129
AGENTS.md
@@ -3,6 +3,26 @@
|
||||
Webi installs dev tools to `~/.local/` without sudo. Each installer is a small
|
||||
package of 3-4 files. This guide tells you how to create and modify them.
|
||||
|
||||
## Domains
|
||||
|
||||
| Environment | Domains |
|
||||
| ----------- | ------------------------------------------------ |
|
||||
| Production | webi.sh, webi.ms, webinstall.dev |
|
||||
| Beta | beta.webi.sh, beta.webi.ms, beta.webinstall.dev |
|
||||
| Next | next.webi.sh, next.webi.ms, next.webinstall.dev |
|
||||
|
||||
- **webi.sh** — POSIX shell installer: `curl https://webi.sh/node | sh`
|
||||
- **webi.ms** — PowerShell installer: `curl.exe https://webi.ms/node | powershell`
|
||||
- **webinstall.dev** — canonical domain, serves both shell scripts and the
|
||||
browser-facing cheat sheet pages
|
||||
|
||||
The domain controls the default script format. You can force the same behavior
|
||||
on any domain via User-Agent:
|
||||
- `curl -A "MS" ...` — triggers PowerShell output (same as webi.ms)
|
||||
- `curl ...` (default UA) — triggers POSIX shell output (same as webi.sh)
|
||||
- `curl -A "$(uname -srm)" ...` — once the API is activated, the full
|
||||
`uname -srm` string (e.g. `Linux 6.1.0 x86_64`) guides OS/arch selection
|
||||
|
||||
## Why Webi Exists
|
||||
|
||||
Webi makes tool installation trivially repeatable for people who aren't
|
||||
@@ -37,9 +57,11 @@ about PATH, permissions, or platform differences. Three things matter:
|
||||
|
||||
Key infrastructure directories (do not modify without good reason):
|
||||
|
||||
- `_webi/` — bootstrap templates, `normalize.js` (auto-detects OS/arch/ext from
|
||||
filenames)
|
||||
- `_common/` — shared JS: `github.js`, `githubish.js`, `gitea.js`, `fetcher.js`
|
||||
- `_webi/` — bootstrap templates, `transform-releases.js` (API endpoint),
|
||||
`builds-cacher.js` (reads cache JSON), `serve-installer.js` (installer
|
||||
scripts)
|
||||
- `_common/` — shared JS fetcher libraries (being phased out — Go daemon now
|
||||
fetches upstream)
|
||||
- `_example/` — canonical template for new packages
|
||||
- `_examples/` — specialized templates (goreleaser, xz-compressed)
|
||||
|
||||
@@ -57,83 +79,50 @@ Ref: <https://github.com/webinstall/webi-installers/issues/412>
|
||||
| 🔗 | Alias/redirect to another package | `ripgrep` → `rg` |
|
||||
| 📝 | Bespoke / custom install | `rustlang` |
|
||||
|
||||
## releases.js
|
||||
## Data Architecture
|
||||
|
||||
Fetches release metadata and returns a normalized object. Most packages use
|
||||
GitHub releases:
|
||||
There are two data paths. Both read from pre-generated cache — the Node.js
|
||||
server does NOT fetch upstream APIs.
|
||||
|
||||
```js
|
||||
'use strict';
|
||||
```
|
||||
API path (JSON/TAB output):
|
||||
Request → transform-releases.js → ~/.cache/webi/legacy/{pkg}.json → filter + sort
|
||||
Vocabulary: macos, amd64, arm64, armv7l (API vocabulary)
|
||||
normalize.js: REMOVED — cache provides all fields directly
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'OWNER';
|
||||
var repo = 'REPO';
|
||||
|
||||
let Releases = module.exports;
|
||||
|
||||
Releases.latest = async function () {
|
||||
let all = await github(null, owner, repo);
|
||||
return all;
|
||||
};
|
||||
|
||||
Releases.sample = async function () {
|
||||
let normalize = require('../_webi/normalize.js');
|
||||
let all = await Releases.latest();
|
||||
all = normalize(all);
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
(async function () {
|
||||
let samples = await Releases.sample();
|
||||
console.info(JSON.stringify(samples, null, 2));
|
||||
})();
|
||||
}
|
||||
Installer path (bash/ps1 script output):
|
||||
Request → serve-installer.js → builds.js → builds-cacher.js
|
||||
Vocabulary: darwin, x86_64, aarch64 (build-classifier vocabulary)
|
||||
```
|
||||
|
||||
### Common release transformations
|
||||
Cache is generated by the Go daemon (`webicached`) and stored flat in
|
||||
`~/.cache/webi/legacy/{pkg}.json` (resolved via `Os.homedir()` in the Node
|
||||
readers — no `_cache` symlink, no month subdirectory). Each file contains:
|
||||
- Top-level summary arrays: `oses`, `arches`, `libcs`, `formats`
|
||||
- `releases` array with pre-classified fields: `os`, `arch`, `libc`, `ext`,
|
||||
`version`, `channel`, `download`
|
||||
- `download` template string
|
||||
|
||||
**Strip version prefix** (monorepo or tool-prefixed tags):
|
||||
### Canonical vocabulary (cache and API)
|
||||
|
||||
```js
|
||||
// e.g. "tools/monorel/v0.6.5" → "v0.6.5"
|
||||
rel.version = rel.version.replace(/^tools\/monorel\//, '');
|
||||
**OS**: `macos` (not darwin), `linux`, `windows`, `freebsd`, etc.
|
||||
**Arch**: `amd64` (not x86_64), `arm64` (not aarch64), `armv7l` (not armv7)
|
||||
**Libc**: `none` (never empty), `gnu`, `musl`, `msvc`
|
||||
**Ext**: `tar.gz`, `zip`, `exe` (no leading dot; `exe` for bare binaries)
|
||||
|
||||
// e.g. "cli-v1.2.3" → "v1.2.3"
|
||||
rel.version = rel.version.replace(/^cli-/, '');
|
||||
```
|
||||
## releases.js (legacy — being phased out)
|
||||
|
||||
**Filter releases** (monorepo with multiple tools, or unwanted assets):
|
||||
The `{pkg}/releases.js` files previously fetched upstream release metadata.
|
||||
These are being replaced by the Go cache daemon. Existing files are kept as
|
||||
documentation of release sources but are no longer called by the server.
|
||||
|
||||
```js
|
||||
all.releases = all.releases.filter(function (rel) {
|
||||
// Keep only releases for this tool
|
||||
return rel.version.startsWith('tools/monorel/');
|
||||
});
|
||||
```
|
||||
|
||||
Apply transformations inside `Releases.latest`, before returning `all`.
|
||||
|
||||
**Available sources** beyond `github.js`:
|
||||
|
||||
- `_common/gitea.js` — Gitea servers
|
||||
- `_common/git-tag.js` — Git tag listing
|
||||
- Custom fetch from any JSON API (see `go/releases.js`, `terraform/releases.js`)
|
||||
|
||||
### Testing releases.js
|
||||
### Testing the API (current)
|
||||
|
||||
```sh
|
||||
node -e "
|
||||
let Releases = require('./<name>/releases.js');
|
||||
Releases.sample().then(function (all) {
|
||||
console.log(JSON.stringify(all, null, 2));
|
||||
});
|
||||
"
|
||||
curl -sS 'https://beta.webi.sh/api/releases/<name>.json?os=macos&arch=arm64&limit=5' | jq .
|
||||
```
|
||||
|
||||
Verify: versions are clean semver (`0.6.5` not `tools/monorel/v0.6.5`), OS/arch
|
||||
detected correctly, download URLs resolve.
|
||||
Verify: versions present, correct OS/arch vocabulary, download URLs resolve.
|
||||
|
||||
## install.sh
|
||||
|
||||
@@ -365,9 +354,11 @@ Commit messages: `feat(<pkg>): add installer`, `fix(<pkg>): update install.sh`,
|
||||
must filter in `releases.js` and strip the tag prefix from the version.
|
||||
- **No `--version` flag**: Some tools lack version introspection. Comment out
|
||||
`pkg_get_current_version` — webi still works, it just can't skip reinstalls.
|
||||
- **normalize.js auto-detection**: OS/arch/ext are guessed from download
|
||||
filenames. If the tool uses non-standard naming, you may need to set `os`,
|
||||
`arch`, or `ext` explicitly in `releases.js`.
|
||||
- **Cache directory**: `builds-cacher.js` and `transform-releases.js` read
|
||||
exclusively from `~/.cache/webi/legacy/{pkg}.json` (resolved via
|
||||
`Os.homedir()`). No date bucketing, no `_cache` symlink, no month rollover
|
||||
hazard. If a package returns `0.0.0`, check that `~/.cache/webi/legacy/{pkg}.json`
|
||||
exists and that `webicached` is running.
|
||||
- **Goreleaser archives**: Typically contain a bare binary at the archive root
|
||||
(not nested in a directory). Use `mv ./cmd "$pkg_src_cmd"`.
|
||||
|
||||
|
||||
100
_scripts/deploy-webinstall
Executable file
100
_scripts/deploy-webinstall
Executable file
@@ -0,0 +1,100 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
set -u
|
||||
|
||||
# Deploy the Node.js webinstall (installer server) to a target host.
|
||||
# Usage: _scripts/deploy-webinstall <host>
|
||||
# e.g. _scripts/deploy-webinstall beta.webi.sh
|
||||
# _scripts/deploy-webinstall next.webi.sh
|
||||
|
||||
fn_main() {
|
||||
g_host="${1:-}"
|
||||
if test -z "${g_host}"; then
|
||||
printf 'Usage: %s <host>\n' "$0" >&2
|
||||
printf ' e.g. %s beta.webi.sh\n' "$0" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# beta.webi.sh → ~/srv/beta.webinstall.dev/installers/
|
||||
# next.webi.sh → ~/srv/next.webinstall.dev/installers/
|
||||
b_subdomain="${g_host%%.*}"
|
||||
g_dest="~/srv/${b_subdomain}.webinstall.dev/installers/"
|
||||
|
||||
# Verify the build-classifier submodule is populated. rsync will
|
||||
# happily push an empty submodule directory, after which Node
|
||||
# crashes at startup with `Cannot find module './build-classifier/
|
||||
# host-targets.js'`. New worktrees from `git worktree add` don't
|
||||
# init submodules by default.
|
||||
if ! test -f ./_webi/build-classifier/host-targets.js; then
|
||||
printf 'fatal: _webi/build-classifier submodule not initialized\n' >&2
|
||||
printf ' run: git submodule update --init _webi/build-classifier\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
printf '%s\n' "Deploying to ${g_host}:${g_dest} ..."
|
||||
|
||||
rsync -avz --delete \
|
||||
--exclude='.git' \
|
||||
--exclude='node_modules' \
|
||||
--exclude='_cache' \
|
||||
--exclude='LIVE_cache' \
|
||||
--exclude='webicached' \
|
||||
--exclude='classify' \
|
||||
--exclude='fetchraw' \
|
||||
--exclude='inspect' \
|
||||
--exclude='e2etest' \
|
||||
--exclude='zigtest' \
|
||||
--exclude='distributables.csv' \
|
||||
--exclude='agents/' \
|
||||
./ "${g_host}:${g_dest}"
|
||||
|
||||
printf '%s\n' "Restarting webinstall ..."
|
||||
|
||||
# shellcheck disable=SC2029
|
||||
ssh "${g_host}" "
|
||||
. ~/.config/envman/PATH.env 2>/dev/null
|
||||
rm -rf ${g_dest}_cache 2>/dev/null
|
||||
serviceman restart webinstall
|
||||
"
|
||||
|
||||
printf '%s\n' "Waiting for restart ..."
|
||||
sleep 3
|
||||
|
||||
fn_smoke_test
|
||||
}
|
||||
|
||||
fn_smoke_test() {
|
||||
printf '%s\n' "Smoke testing https://${g_host}/ ..."
|
||||
|
||||
b_headers="$(curl -sI --max-time 5 "https://${g_host}/api/releases/go.json?limit=1")"
|
||||
|
||||
if printf '%s' "${b_headers}" | grep -qi 'X-Robots-Tag: noindex'; then
|
||||
printf '%s\n' " [ok] X-Robots-Tag: noindex"
|
||||
else
|
||||
printf '%s\n' " [FAIL] Missing X-Robots-Tag header"
|
||||
fi
|
||||
|
||||
if printf '%s' "${b_headers}" | grep -qi 'rel="canonical"'; then
|
||||
printf '%s\n' " [ok] Link: canonical"
|
||||
else
|
||||
printf '%s\n' " [FAIL] Missing canonical Link header"
|
||||
fi
|
||||
|
||||
b_version="$(curl -sS --max-time 5 "https://${g_host}/api/releases/go.json?limit=1" | jq -r '.[0].version')"
|
||||
if test "${b_version}" != "0.0.0" && test -n "${b_version}"; then
|
||||
printf '%s\n' " [ok] API returns go ${b_version}"
|
||||
else
|
||||
printf '%s\n' " [FAIL] API returned error or empty for go"
|
||||
fi
|
||||
|
||||
b_shebang="$(curl -sS --max-time 5 "https://${g_host}/node" | head -1)"
|
||||
if test "${b_shebang}" = "#!/bin/sh"; then
|
||||
printf '%s\n' " [ok] Installer path serves shell script"
|
||||
else
|
||||
printf '%s\n' " [FAIL] Installer path unexpected: ${b_shebang}"
|
||||
fi
|
||||
|
||||
printf '%s\n' "Done."
|
||||
}
|
||||
|
||||
fn_main "$@"
|
||||
Submodule _webi/build-classifier updated: 9f87804eb4...f9cc9f3e19
@@ -621,13 +621,15 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
let arches = waterfall[hostTarget.arch] ||
|
||||
HostTargets.WATERFALL.ANYOS[hostTarget.arch] || [hostTarget.arch];
|
||||
arches = arches.concat(['ANYARCH']);
|
||||
let libcs = waterfall[hostTarget.libc] ||
|
||||
HostTargets.WATERFALL.ANYOS[hostTarget.libc] || [hostTarget.libc];
|
||||
// termsToTarget omits libc for plain UAs; 'libc' → waterfall ['none','libc',...]
|
||||
let libc = hostTarget.libc || 'libc';
|
||||
let libcs = waterfall[libc] ||
|
||||
HostTargets.WATERFALL.ANYOS[libc] || [libc];
|
||||
|
||||
// Extend the glibc-host waterfall: the table only lists [none, libc]
|
||||
// but Rust projects (bat, rg) and node ship libc='gnu' builds, and
|
||||
// static musl builds also run on glibc hosts.
|
||||
if (hostTarget.libc === 'libc' && !libcs.includes('gnu')) {
|
||||
if (libc === 'libc' && !libcs.includes('gnu')) {
|
||||
libcs = ['none', 'gnu', 'musl', 'libc'];
|
||||
}
|
||||
|
||||
|
||||
@@ -226,9 +226,6 @@ __bootstrap_webi() {
|
||||
elif test "$WEBI_EXT" = "git"; then
|
||||
echo " Moving $(t_path "${my_dl_rel}")"
|
||||
mv "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" .
|
||||
elif test "$WEBI_EXT" = "gz"; then
|
||||
echo " Inflating $(t_path "${my_dl_rel}")"
|
||||
gunzip -c "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > "$(basename "$WEBI_PKG_FILE" .gz)"
|
||||
elif test "$WEBI_EXT" = "xz"; then
|
||||
echo " Inflating $(t_path "${my_dl_rel}")"
|
||||
unxz -c "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > "$(basename "$WEBI_PKG_FILE")"
|
||||
|
||||
208
_webi/test-api-compat.js
Normal file
208
_webi/test-api-compat.js
Normal file
@@ -0,0 +1,208 @@
|
||||
'use strict';
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Path = require('node:path');
|
||||
|
||||
let Releases = require('./transform-releases.js');
|
||||
|
||||
let TESTDATA_DIR = Path.join(__dirname, 'testdata');
|
||||
|
||||
// These mirror what the live API returns for /api/releases/{pkg}@stable.json?...
|
||||
let FILTERED_CASES = [
|
||||
{ pkg: 'bat', os: 'macos', arch: 'amd64' },
|
||||
{ pkg: 'bat', os: 'macos', arch: 'arm64' },
|
||||
{ pkg: 'bat', os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'bat', os: 'windows', arch: 'amd64' },
|
||||
{ pkg: 'go', os: 'macos', arch: 'amd64' },
|
||||
{ pkg: 'go', os: 'macos', arch: 'arm64' },
|
||||
{ pkg: 'go', os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'go', os: 'windows', arch: 'amd64' },
|
||||
{ pkg: 'rg', os: 'macos', arch: 'amd64' },
|
||||
{ pkg: 'rg', os: 'macos', arch: 'arm64' },
|
||||
{ pkg: 'rg', os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'rg', os: 'windows', arch: 'amd64' },
|
||||
{ pkg: 'caddy', os: 'macos', arch: 'amd64' },
|
||||
{ pkg: 'caddy', os: 'macos', arch: 'arm64' },
|
||||
{ pkg: 'caddy', os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'caddy', os: 'windows', arch: 'amd64' },
|
||||
];
|
||||
|
||||
// Fields to compare between live and local
|
||||
let COMPARE_FIELDS = ['version', 'os', 'arch', 'ext', 'libc', 'channel', 'download'];
|
||||
|
||||
async function main() {
|
||||
let failures = 0;
|
||||
let passes = 0;
|
||||
let skips = 0;
|
||||
|
||||
// Test 1: Unfiltered release list — compare structure and field values
|
||||
console.log('=== Test 1: Unfiltered /api/releases/{pkg}.json ===');
|
||||
console.log('');
|
||||
for (let pkg of ['bat', 'go', 'node', 'rg', 'jq', 'caddy']) {
|
||||
let liveFile = `${TESTDATA_DIR}/live_${pkg}.json`;
|
||||
let liveExists = await Fs.access(liveFile).then(
|
||||
function () { return true; },
|
||||
function () { return false; },
|
||||
);
|
||||
if (!liveExists) {
|
||||
console.log(` SKIP ${pkg}: no golden data`);
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let liveJson = await Fs.readFile(liveFile, 'utf8');
|
||||
let liveReleases = JSON.parse(liveJson);
|
||||
|
||||
let localResult = await Releases.getReleases({
|
||||
pkg: pkg,
|
||||
ver: '',
|
||||
os: '',
|
||||
arch: '',
|
||||
libc: '',
|
||||
lts: false,
|
||||
channel: '',
|
||||
formats: [],
|
||||
limit: 100,
|
||||
});
|
||||
|
||||
let localReleases = localResult.releases;
|
||||
|
||||
// Compare OS vocabulary
|
||||
let liveOses = [...new Set(liveReleases.map(function (r) { return r.os; }))].sort();
|
||||
let localOses = [...new Set(localReleases.map(function (r) { return r.os; }))].sort();
|
||||
let osMatch = JSON.stringify(liveOses) === JSON.stringify(localOses);
|
||||
if (!osMatch) {
|
||||
console.log(` FAIL ${pkg} OS values: live=${JSON.stringify(liveOses)} local=${JSON.stringify(localOses)}`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${pkg} OS values: ${JSON.stringify(liveOses)}`);
|
||||
passes++;
|
||||
}
|
||||
|
||||
// Compare arch vocabulary
|
||||
let liveArches = [...new Set(liveReleases.map(function (r) { return r.arch; }))].sort();
|
||||
let localArches = [...new Set(localReleases.map(function (r) { return r.arch; }))].sort();
|
||||
let archMatch = JSON.stringify(liveArches) === JSON.stringify(localArches);
|
||||
if (!archMatch) {
|
||||
console.log(` FAIL ${pkg} arch values: live=${JSON.stringify(liveArches)} local=${JSON.stringify(localArches)}`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${pkg} arch values: ${JSON.stringify(liveArches)}`);
|
||||
passes++;
|
||||
}
|
||||
|
||||
// Compare latest version
|
||||
let liveLatest = liveReleases[0]?.version;
|
||||
let localLatest = localReleases[0]?.version;
|
||||
if (liveLatest !== localLatest) {
|
||||
// Version differences may be expected if cache is newer/older
|
||||
console.log(` WARN ${pkg} latest version: live=${liveLatest} local=${localLatest}`);
|
||||
} else {
|
||||
console.log(` PASS ${pkg} latest version: ${liveLatest}`);
|
||||
passes++;
|
||||
}
|
||||
|
||||
// Compare ext vocabulary
|
||||
let liveExts = [...new Set(liveReleases.map(function (r) { return r.ext; }))].sort();
|
||||
let localExts = [...new Set(localReleases.map(function (r) { return r.ext; }))].sort();
|
||||
let extMatch = JSON.stringify(liveExts) === JSON.stringify(localExts);
|
||||
if (!extMatch) {
|
||||
console.log(` FAIL ${pkg} ext values: live=${JSON.stringify(liveExts)} local=${JSON.stringify(localExts)}`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${pkg} ext values: ${JSON.stringify(liveExts)}`);
|
||||
passes++;
|
||||
}
|
||||
|
||||
// Compare that version strings don't have 'v' prefix
|
||||
let localHasVPrefix = localReleases.some(function (r) {
|
||||
return r.version.startsWith('v');
|
||||
});
|
||||
if (localHasVPrefix) {
|
||||
console.log(` FAIL ${pkg} versions have 'v' prefix (should be stripped)`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${pkg} no 'v' prefix on versions`);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
|
||||
// Test 2: Filtered queries — compare selected package for specific OS/arch
|
||||
console.log('');
|
||||
console.log('=== Test 2: Filtered /api/releases/{pkg}@stable.json?os=...&arch=... ===');
|
||||
console.log('');
|
||||
for (let tc of FILTERED_CASES) {
|
||||
let fname = `live_${tc.pkg}_os_${tc.os}_arch_${tc.arch}.json`;
|
||||
let liveFile = `${TESTDATA_DIR}/${fname}`;
|
||||
let liveExists = await Fs.access(liveFile).then(
|
||||
function () { return true; },
|
||||
function () { return false; },
|
||||
);
|
||||
if (!liveExists) {
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let liveJson = await Fs.readFile(liveFile, 'utf8');
|
||||
let liveReleases = JSON.parse(liveJson);
|
||||
let liveFirst = liveReleases[0];
|
||||
if (!liveFirst || liveFirst.channel === 'error') {
|
||||
console.log(` SKIP ${tc.pkg} ${tc.os}/${tc.arch}: live returned error/empty`);
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let localResult = await Releases.getReleases({
|
||||
pkg: tc.pkg,
|
||||
ver: '',
|
||||
os: tc.os,
|
||||
arch: tc.arch,
|
||||
libc: '',
|
||||
lts: false,
|
||||
channel: 'stable',
|
||||
formats: ['tar', 'zip', 'exe', 'xz'],
|
||||
limit: 1,
|
||||
});
|
||||
let localFirst = localResult.releases[0];
|
||||
|
||||
if (!localFirst || localFirst.channel === 'error') {
|
||||
console.log(` FAIL ${tc.pkg} ${tc.os}/${tc.arch}: local returned error/empty, live had ${liveFirst.version}`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let diffs = [];
|
||||
for (let field of COMPARE_FIELDS) {
|
||||
let liveVal = String(liveFirst[field] || '');
|
||||
let localVal = String(localFirst[field] || '');
|
||||
if (liveVal !== localVal) {
|
||||
// Version differences are OK if cache age differs
|
||||
if (field === 'version' || field === 'download' || field === 'date') {
|
||||
continue;
|
||||
}
|
||||
diffs.push(`${field}: live=${liveVal} local=${localVal}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (diffs.length > 0) {
|
||||
console.log(` FAIL ${tc.pkg} ${tc.os}/${tc.arch}: ${diffs.join(', ')}`);
|
||||
failures++;
|
||||
} else {
|
||||
let ver = localFirst.version;
|
||||
let ext = localFirst.ext;
|
||||
console.log(` PASS ${tc.pkg} ${tc.os}/${tc.arch}: v${ver} .${ext}`);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
|
||||
console.log('');
|
||||
console.log(`=== Results: ${passes} passed, ${failures} failed, ${skips} skipped ===`);
|
||||
if (failures > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
77
_webi/test-broad-resolve.js
Normal file
77
_webi/test-broad-resolve.js
Normal file
@@ -0,0 +1,77 @@
|
||||
'use strict';
|
||||
|
||||
// Broad sweep: test that all cached packages resolve on macOS arm64
|
||||
// and Linux amd64. Catches any package that completely fails to resolve.
|
||||
//
|
||||
// Usage: node _webi/test-broad-resolve.js
|
||||
|
||||
var Path = require('node:path');
|
||||
var InstallerServer = require('./serve-installer.js');
|
||||
var Builds = require('./builds.js');
|
||||
var BuildsCacher = require('./builds-cacher.js');
|
||||
|
||||
var UA_CASES = [
|
||||
{ label: 'macOS arm64', ua: 'aarch64/unknown Darwin/24.2.0 libc' },
|
||||
{ label: 'Linux amd64', ua: 'x86_64/unknown Linux/5.15.0 libc' },
|
||||
];
|
||||
|
||||
async function main() {
|
||||
console.log('Initializing build cache...');
|
||||
await Builds.init();
|
||||
console.log('');
|
||||
|
||||
var bc = BuildsCacher.create({
|
||||
caches: Path.join(__dirname, '../_cache'),
|
||||
installers: Path.join(__dirname, '..'),
|
||||
});
|
||||
var dirs = await bc.getProjectsByType();
|
||||
var pkgs = Object.keys(dirs.valid).sort();
|
||||
console.log('Testing ' + pkgs.length + ' packages...');
|
||||
console.log('');
|
||||
|
||||
var pass = 0;
|
||||
var fail = 0;
|
||||
var failures = [];
|
||||
|
||||
for (var i = 0; i < pkgs.length; i++) {
|
||||
var pkg = pkgs[i];
|
||||
for (var j = 0; j < UA_CASES.length; j++) {
|
||||
var tc = UA_CASES[j];
|
||||
try {
|
||||
var r = await InstallerServer.helper({
|
||||
unameAgent: tc.ua,
|
||||
projectName: pkg,
|
||||
tag: 'stable',
|
||||
formats: ['tar', 'exe', 'zip', 'xz', 'dmg'],
|
||||
libc: '',
|
||||
});
|
||||
var p = r[0];
|
||||
if (p.channel === 'error' || p.ext === 'err') {
|
||||
failures.push(pkg + ' ' + tc.label + ': error (v' + p.version + ')');
|
||||
fail++;
|
||||
} else {
|
||||
pass++;
|
||||
}
|
||||
} catch (e) {
|
||||
failures.push(pkg + ' ' + tc.label + ': ' + e.message.substring(0, 60));
|
||||
fail++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (failures.length > 0) {
|
||||
console.log('Failures:');
|
||||
for (var k = 0; k < failures.length; k++) {
|
||||
console.log(' FAIL ' + failures[k]);
|
||||
}
|
||||
console.log('');
|
||||
}
|
||||
|
||||
var total = pkgs.length * UA_CASES.length;
|
||||
console.log('=== ' + pass + '/' + total + ' passed (' + fail + ' failed) ===');
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
277
_webi/test-cache-api-ready.js
Normal file
277
_webi/test-cache-api-ready.js
Normal file
@@ -0,0 +1,277 @@
|
||||
'use strict';
|
||||
|
||||
// Tests that _cache JSON files are ready for direct use by the API endpoint
|
||||
// (transform-releases.js) WITHOUT needing normalize.js to fix up fields.
|
||||
//
|
||||
// These tests drive GOER to add missing features to ExportLegacy so that
|
||||
// normalize.js can be eliminated from the API path entirely.
|
||||
//
|
||||
// Usage: node _webi/test-cache-api-ready.js
|
||||
|
||||
var Fs = require('node:fs');
|
||||
var Os = require('node:os');
|
||||
var Path = require('node:path');
|
||||
|
||||
var CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
|
||||
// Packages to spot-check (mix of github_releases, custom releases.js, gittag)
|
||||
var CHECK_PKGS = [
|
||||
'bat',
|
||||
'caddy',
|
||||
'go',
|
||||
'hugo',
|
||||
'jq',
|
||||
'node',
|
||||
'rg',
|
||||
'terraform',
|
||||
'zig',
|
||||
];
|
||||
|
||||
function loadReleases(dir, pkg) {
|
||||
var file = Path.join(dir, pkg + '.json');
|
||||
if (!Fs.existsSync(file)) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return JSON.parse(Fs.readFileSync(file, 'utf8'));
|
||||
} catch (e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function main() {
|
||||
var passes = 0;
|
||||
var failures = 0;
|
||||
|
||||
if (!Fs.existsSync(CACHE_DIR)) {
|
||||
console.error('No cache directory at ' + CACHE_DIR);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log('Testing ' + CACHE_DIR);
|
||||
console.log('');
|
||||
|
||||
// ================================================================
|
||||
// Test 1: Version has no 'v' prefix
|
||||
// ================================================================
|
||||
console.log('=== Test 1: Version v-prefix stripped ===');
|
||||
console.log('');
|
||||
|
||||
for (var vi = 0; vi < CHECK_PKGS.length; vi++) {
|
||||
var vpkg = CHECK_PKGS[vi];
|
||||
var vdata = loadReleases(CACHE_DIR, vpkg);
|
||||
if (!vdata) {
|
||||
console.log(' SKIP ' + vpkg + ': no data');
|
||||
continue;
|
||||
}
|
||||
|
||||
var vPrefixed = 0;
|
||||
for (var vri = 0; vri < vdata.releases.length; vri++) {
|
||||
var ver = vdata.releases[vri].version || '';
|
||||
if (ver.startsWith('v')) {
|
||||
vPrefixed++;
|
||||
}
|
||||
}
|
||||
|
||||
if (vPrefixed === 0) {
|
||||
console.log(' PASS ' + vpkg + ': no v-prefixed versions');
|
||||
passes++;
|
||||
} else {
|
||||
console.log(' FAIL ' + vpkg + ': ' + vPrefixed + '/' + vdata.releases.length + ' versions have v prefix');
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 2: libc is never empty string (should be "none", "gnu", "musl", "msvc")
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 2: libc never empty ===');
|
||||
console.log('');
|
||||
|
||||
for (var li = 0; li < CHECK_PKGS.length; li++) {
|
||||
var lpkg = CHECK_PKGS[li];
|
||||
var ldata = loadReleases(CACHE_DIR, lpkg);
|
||||
if (!ldata) {
|
||||
console.log(' SKIP ' + lpkg + ': no data');
|
||||
continue;
|
||||
}
|
||||
|
||||
var emptyLibc = 0;
|
||||
for (var lri = 0; lri < ldata.releases.length; lri++) {
|
||||
if (ldata.releases[lri].libc === '') {
|
||||
emptyLibc++;
|
||||
}
|
||||
}
|
||||
|
||||
if (emptyLibc === 0) {
|
||||
console.log(' PASS ' + lpkg + ': all entries have libc set');
|
||||
passes++;
|
||||
} else {
|
||||
console.log(' FAIL ' + lpkg + ': ' + emptyLibc + '/' + ldata.releases.length + ' entries have empty libc (should be "none")');
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 3: ext has no leading dot
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 3: ext has no leading dot ===');
|
||||
console.log('');
|
||||
|
||||
for (var ei = 0; ei < CHECK_PKGS.length; ei++) {
|
||||
var epkg = CHECK_PKGS[ei];
|
||||
var edata = loadReleases(CACHE_DIR, epkg);
|
||||
if (!edata) {
|
||||
console.log(' SKIP ' + epkg + ': no data');
|
||||
continue;
|
||||
}
|
||||
|
||||
var dotExt = 0;
|
||||
for (var eri = 0; eri < edata.releases.length; eri++) {
|
||||
var ext = edata.releases[eri].ext || '';
|
||||
if (ext.startsWith('.')) {
|
||||
dotExt++;
|
||||
}
|
||||
}
|
||||
|
||||
if (dotExt === 0) {
|
||||
console.log(' PASS ' + epkg + ': no leading dots on ext');
|
||||
passes++;
|
||||
} else {
|
||||
console.log(' FAIL ' + epkg + ': ' + dotExt + '/' + edata.releases.length + ' entries have leading dot on ext');
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 4: bare binaries have ext "exe" (not empty)
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 4: bare binaries have ext "exe" ===');
|
||||
console.log('');
|
||||
|
||||
var barePkgs = ['jq', 'bat', 'rg', 'caddy'];
|
||||
for (var bi = 0; bi < barePkgs.length; bi++) {
|
||||
var bpkg = barePkgs[bi];
|
||||
var bdata = loadReleases(CACHE_DIR, bpkg);
|
||||
if (!bdata) {
|
||||
console.log(' SKIP ' + bpkg + ': no data');
|
||||
continue;
|
||||
}
|
||||
|
||||
var emptyExt = 0;
|
||||
for (var bri = 0; bri < bdata.releases.length; bri++) {
|
||||
var brel = bdata.releases[bri];
|
||||
if (brel.ext === '' && brel.name && !brel.name.includes('.')) {
|
||||
emptyExt++;
|
||||
}
|
||||
}
|
||||
|
||||
if (emptyExt === 0) {
|
||||
console.log(' PASS ' + bpkg + ': no bare binaries with empty ext');
|
||||
passes++;
|
||||
} else {
|
||||
console.log(' FAIL ' + bpkg + ': ' + emptyExt + ' bare binaries have empty ext (should be "exe")');
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 5: Top-level summary arrays present
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 5: Summary arrays (oses, arches, libcs, formats) ===');
|
||||
console.log('');
|
||||
|
||||
for (var si = 0; si < CHECK_PKGS.length; si++) {
|
||||
var spkg = CHECK_PKGS[si];
|
||||
var sdata = loadReleases(CACHE_DIR, spkg);
|
||||
if (!sdata) {
|
||||
console.log(' SKIP ' + spkg + ': no data');
|
||||
continue;
|
||||
}
|
||||
|
||||
var missing = [];
|
||||
if (!Array.isArray(sdata.oses)) { missing.push('oses'); }
|
||||
if (!Array.isArray(sdata.arches)) { missing.push('arches'); }
|
||||
if (!Array.isArray(sdata.libcs)) { missing.push('libcs'); }
|
||||
if (!Array.isArray(sdata.formats)) { missing.push('formats'); }
|
||||
|
||||
if (missing.length === 0) {
|
||||
// Verify they contain the right values
|
||||
var hasMacos = sdata.oses.includes('macos') || sdata.oses.includes('darwin');
|
||||
var hasLinux = sdata.oses.includes('linux');
|
||||
var ok = true;
|
||||
if (sdata.releases.some(function (r) { return r.os === 'macos' || r.os === 'darwin'; }) && !hasMacos) {
|
||||
console.log(' FAIL ' + spkg + ': oses array missing macos/darwin');
|
||||
failures++;
|
||||
ok = false;
|
||||
}
|
||||
if (sdata.releases.some(function (r) { return r.os === 'linux'; }) && !hasLinux) {
|
||||
console.log(' FAIL ' + spkg + ': oses array missing linux');
|
||||
failures++;
|
||||
ok = false;
|
||||
}
|
||||
if (ok) {
|
||||
console.log(' PASS ' + spkg + ': has oses, arches, libcs, formats');
|
||||
passes++;
|
||||
}
|
||||
} else {
|
||||
console.log(' FAIL ' + spkg + ': missing top-level arrays: ' + missing.join(', '));
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 6: Version sort order — stable before beta, newest first
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 6: Version sort order ===');
|
||||
console.log('');
|
||||
|
||||
var sortPkgs = ['go', 'node', 'terraform'];
|
||||
for (var oi = 0; oi < sortPkgs.length; oi++) {
|
||||
var opkg = sortPkgs[oi];
|
||||
var odata = loadReleases(CACHE_DIR, opkg);
|
||||
if (!odata) {
|
||||
console.log(' SKIP ' + opkg + ': no data');
|
||||
continue;
|
||||
}
|
||||
|
||||
// Find first stable release
|
||||
var firstStable = odata.releases.find(function (r) { return r.channel === 'stable'; });
|
||||
// Find first beta release
|
||||
var firstBeta = odata.releases.find(function (r) { return r.channel === 'beta'; });
|
||||
|
||||
if (!firstStable) {
|
||||
console.log(' SKIP ' + opkg + ': no stable release');
|
||||
continue;
|
||||
}
|
||||
|
||||
// The first entry overall should be a stable release (newest stable > any beta)
|
||||
// unless the beta is a newer version number
|
||||
var firstEntry = odata.releases[0];
|
||||
if (firstEntry.channel === 'stable') {
|
||||
console.log(' PASS ' + opkg + ': first entry is stable (' + firstEntry.version + ')');
|
||||
passes++;
|
||||
} else {
|
||||
console.log(' FAIL ' + opkg + ': first entry is ' + firstEntry.channel + ' (' + firstEntry.version + '), expected stable (' + firstStable.version + ')');
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Summary
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Results: ' + passes + ' passed, ' + failures + ' failed ===');
|
||||
if (failures > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
687
_webi/test-cache-compat.js
Normal file
687
_webi/test-cache-compat.js
Normal file
@@ -0,0 +1,687 @@
|
||||
'use strict';
|
||||
|
||||
// Cache compatibility tests: verify that Go-generated cache files work
|
||||
// correctly with the Node build-classifier and installer resolution pipeline.
|
||||
//
|
||||
// Tests cover:
|
||||
// 1. Cache completeness — all packages have releases, required fields present
|
||||
// 2. Format selection — correct archive format per platform
|
||||
// 3. Edge-case platforms — FreeBSD, ARM variants, musl/Alpine
|
||||
// 4. Script generation — installer scripts render without error
|
||||
// 5. API compat — transform-releases output matches expected vocabulary
|
||||
// 6. Version format — no 'v' prefix, valid semver-ish
|
||||
// 7. Channel detection — stable vs beta correctly identified
|
||||
//
|
||||
// Usage: node _webi/test-cache-compat.js
|
||||
|
||||
var Fs = require('node:fs');
|
||||
var Path = require('node:path');
|
||||
|
||||
var InstallerServer = require('./serve-installer.js');
|
||||
var Builds = require('./builds.js');
|
||||
var Releases = require('./transform-releases.js');
|
||||
|
||||
var CACHE_DIR = Path.join(require('node:os').homedir(), '.cache/webi/legacy');
|
||||
|
||||
// ====================================================================
|
||||
// Test 1: Cache completeness — every package has releases with valid fields
|
||||
// ====================================================================
|
||||
|
||||
// Packages that are expected to have binary releases (not gittag-only)
|
||||
var BINARY_PKGS = [
|
||||
'bat',
|
||||
'caddy',
|
||||
'cmake',
|
||||
'delta',
|
||||
'deno',
|
||||
'fd',
|
||||
'fzf',
|
||||
'gh',
|
||||
'go',
|
||||
'goreleaser',
|
||||
'hugo',
|
||||
'jq',
|
||||
'k9s',
|
||||
'node',
|
||||
'ollama',
|
||||
'rg',
|
||||
'shellcheck',
|
||||
'shfmt',
|
||||
'syncthing',
|
||||
'terraform',
|
||||
'xz',
|
||||
'yq',
|
||||
'zig',
|
||||
'zoxide',
|
||||
];
|
||||
|
||||
// Packages that are gittag/source-only — must have releases but os/arch may be special
|
||||
var GITTAG_PKGS = [
|
||||
'aliasman',
|
||||
'serviceman',
|
||||
'vim-airline',
|
||||
'vim-go',
|
||||
'vim-sensible',
|
||||
];
|
||||
|
||||
// ====================================================================
|
||||
// Test 2: Format selection — correct ext per platform
|
||||
// ====================================================================
|
||||
|
||||
// For these packages, verify the resolved format is correct for each platform.
|
||||
// Linux/macOS should get .tar.gz or .tar.xz (not .zip except for specific packages).
|
||||
// Windows should get .zip or .exe (not .tar.gz).
|
||||
var FORMAT_CASES = [
|
||||
// Go projects — should be .tar.gz on Linux/macOS, .zip on Windows
|
||||
{
|
||||
label: 'go Linux amd64 format',
|
||||
pkg: 'go',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'go Windows amd64 format',
|
||||
pkg: 'go',
|
||||
ua: 'x86_64/unknown Windows/10.0.19041 msvc',
|
||||
expectExt: 'zip',
|
||||
},
|
||||
{
|
||||
label: 'go macOS arm64 format',
|
||||
pkg: 'go',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
// Rust projects — should be .tar.gz on Linux/macOS, .zip on Windows
|
||||
{
|
||||
label: 'bat Linux amd64 format',
|
||||
pkg: 'bat',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'bat Windows amd64 format',
|
||||
pkg: 'bat',
|
||||
ua: 'x86_64/unknown Windows/10.0.19041 msvc',
|
||||
expectExt: 'zip',
|
||||
},
|
||||
{
|
||||
label: 'rg Linux amd64 format',
|
||||
pkg: 'rg',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'rg Windows amd64 format',
|
||||
pkg: 'rg',
|
||||
ua: 'x86_64/unknown Windows/10.0.19041 msvc',
|
||||
expectExt: 'zip',
|
||||
},
|
||||
// Node — uses .tar.xz on Linux/macOS
|
||||
{
|
||||
label: 'node Linux amd64 format',
|
||||
pkg: 'node',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectExt: 'tar.xz',
|
||||
},
|
||||
{
|
||||
label: 'node macOS arm64 format',
|
||||
pkg: 'node',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectExt: 'tar.xz',
|
||||
},
|
||||
// delta — Rust, .tar.gz on Linux/macOS
|
||||
{
|
||||
label: 'delta Linux amd64 format',
|
||||
pkg: 'delta',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
// shfmt — Go, bare exe on Linux, .exe on Windows
|
||||
{
|
||||
label: 'shfmt Linux amd64 format',
|
||||
pkg: 'shfmt',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectExt: 'exe',
|
||||
},
|
||||
];
|
||||
|
||||
// ====================================================================
|
||||
// Test 3: Edge-case platforms
|
||||
// ====================================================================
|
||||
|
||||
var EDGE_CASES = [
|
||||
// Linux ARM variants
|
||||
{
|
||||
label: 'go Linux aarch64',
|
||||
pkg: 'go',
|
||||
ua: 'aarch64/unknown Linux/5.15.0 libc',
|
||||
expectOs: 'linux',
|
||||
expectNotError: true,
|
||||
},
|
||||
{
|
||||
label: 'node Linux aarch64',
|
||||
pkg: 'node',
|
||||
ua: 'aarch64/unknown Linux/5.15.0 libc',
|
||||
expectOs: 'linux',
|
||||
expectNotError: true,
|
||||
},
|
||||
// Alpine/musl — bat should resolve to musl build
|
||||
{
|
||||
label: 'bat Linux musl amd64',
|
||||
pkg: 'bat',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 musl',
|
||||
expectOs: 'linux',
|
||||
expectNotError: true,
|
||||
},
|
||||
{
|
||||
label: 'rg Linux musl amd64',
|
||||
pkg: 'rg',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 musl',
|
||||
expectOs: 'linux',
|
||||
expectNotError: true,
|
||||
},
|
||||
// node musl — separate musl build
|
||||
{
|
||||
label: 'node Linux musl amd64',
|
||||
pkg: 'node',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 musl',
|
||||
expectOs: 'linux',
|
||||
expectNotError: true,
|
||||
},
|
||||
// FreeBSD — packages that have freebsd builds
|
||||
{
|
||||
label: 'syncthing FreeBSD amd64',
|
||||
pkg: 'syncthing',
|
||||
ua: 'x86_64/unknown FreeBSD/14.0 libc',
|
||||
expectOs: 'freebsd',
|
||||
expectNotError: true,
|
||||
},
|
||||
{
|
||||
label: 'caddy FreeBSD amd64',
|
||||
pkg: 'caddy',
|
||||
ua: 'x86_64/unknown FreeBSD/14.0 libc',
|
||||
expectOs: 'freebsd',
|
||||
expectNotError: true,
|
||||
},
|
||||
// Windows aarch64 — should fallback to amd64 for most packages
|
||||
{
|
||||
label: 'go Windows aarch64',
|
||||
pkg: 'go',
|
||||
ua: 'aarch64/unknown Windows/10.0.22000 msvc',
|
||||
expectOs: 'windows',
|
||||
expectNotError: true,
|
||||
},
|
||||
];
|
||||
|
||||
// ====================================================================
|
||||
// Test 4: Script generation smoke tests
|
||||
// ====================================================================
|
||||
|
||||
var SCRIPT_CASES = [
|
||||
{ pkg: 'bat', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'bat Linux' },
|
||||
{ pkg: 'go', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'go macOS' },
|
||||
{ pkg: 'node', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'node Linux' },
|
||||
{ pkg: 'rg', ua: 'x86_64/unknown Windows/10.0.19041 msvc', label: 'rg Windows' },
|
||||
{ pkg: 'jq', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'jq macOS' },
|
||||
{ pkg: 'caddy', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'caddy Linux' },
|
||||
{ pkg: 'shellcheck', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'shellcheck Linux' },
|
||||
{ pkg: 'shfmt', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'shfmt macOS' },
|
||||
{ pkg: 'hugo', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'hugo Linux' },
|
||||
{ pkg: 'delta', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'delta Linux' },
|
||||
{ pkg: 'fd', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'fd Linux' },
|
||||
{ pkg: 'fzf', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'fzf Linux' },
|
||||
{ pkg: 'zoxide', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'zoxide Linux' },
|
||||
{ pkg: 'k9s', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'k9s Linux' },
|
||||
{ pkg: 'yq', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'yq Linux' },
|
||||
];
|
||||
|
||||
// ====================================================================
|
||||
// Test 5: API compat — transform-releases vocabulary
|
||||
// ====================================================================
|
||||
|
||||
// The /api/releases/ endpoint uses normalize.js which has its own OS/arch names.
|
||||
// Verify that the Go cache produces correct values after normalization.
|
||||
var API_VOCAB_CASES = [
|
||||
{
|
||||
label: 'bat has macos+linux+windows',
|
||||
pkg: 'bat',
|
||||
expectOses: ['linux', 'macos', 'windows'],
|
||||
},
|
||||
{
|
||||
label: 'go has macos+linux+windows',
|
||||
pkg: 'go',
|
||||
expectOses: ['linux', 'macos', 'windows'],
|
||||
},
|
||||
{
|
||||
label: 'node has linux+macos+windows',
|
||||
pkg: 'node',
|
||||
expectOses: ['linux', 'macos', 'windows'],
|
||||
},
|
||||
{
|
||||
label: 'syncthing has freebsd+linux+macos+windows',
|
||||
pkg: 'syncthing',
|
||||
expectOses: ['freebsd', 'linux', 'macos', 'windows'],
|
||||
},
|
||||
{
|
||||
label: 'bat has amd64+arm64 arches',
|
||||
pkg: 'bat',
|
||||
expectArches: ['amd64', 'arm64'],
|
||||
},
|
||||
{
|
||||
label: 'go has amd64+arm64 arches',
|
||||
pkg: 'go',
|
||||
expectArches: ['amd64', 'arm64'],
|
||||
},
|
||||
];
|
||||
|
||||
// ====================================================================
|
||||
// Test 6: Version format validation
|
||||
// ====================================================================
|
||||
|
||||
var VERSION_CHECKS = [
|
||||
'bat',
|
||||
'go',
|
||||
'node',
|
||||
'rg',
|
||||
'caddy',
|
||||
'hugo',
|
||||
'terraform',
|
||||
'jq',
|
||||
'cmake',
|
||||
'zig',
|
||||
];
|
||||
|
||||
// ====================================================================
|
||||
// Test 7: Channel detection
|
||||
// ====================================================================
|
||||
|
||||
// Packages that should have both stable and beta releases
|
||||
var CHANNEL_CASES = [
|
||||
{
|
||||
label: 'go has stable releases',
|
||||
pkg: 'go',
|
||||
channel: 'stable',
|
||||
expectMinCount: 10,
|
||||
},
|
||||
{
|
||||
label: 'node has stable releases',
|
||||
pkg: 'node',
|
||||
channel: 'stable',
|
||||
expectMinCount: 10,
|
||||
},
|
||||
{
|
||||
label: 'cmake has stable releases',
|
||||
pkg: 'cmake',
|
||||
channel: 'stable',
|
||||
expectMinCount: 5,
|
||||
},
|
||||
];
|
||||
|
||||
// ====================================================================
|
||||
// Runner
|
||||
// ====================================================================
|
||||
|
||||
async function main() {
|
||||
var passes = 0;
|
||||
var failures = 0;
|
||||
var knowns = 0;
|
||||
var skips = 0;
|
||||
|
||||
console.log('Initializing build cache...');
|
||||
await Builds.init();
|
||||
|
||||
if (!Fs.existsSync(CACHE_DIR)) {
|
||||
console.error('No cache directory at ' + CACHE_DIR);
|
||||
process.exit(1);
|
||||
}
|
||||
var cachePath = CACHE_DIR;
|
||||
console.log('Using cache: ' + cachePath);
|
||||
console.log('');
|
||||
|
||||
// ================================================================
|
||||
// Test 1: Cache completeness
|
||||
// ================================================================
|
||||
console.log('=== Test 1: Cache Completeness ===');
|
||||
console.log('');
|
||||
|
||||
for (var bi = 0; bi < BINARY_PKGS.length; bi++) {
|
||||
var bpkg = BINARY_PKGS[bi];
|
||||
var bfile = Path.join(cachePath, bpkg + '.json');
|
||||
if (!Fs.existsSync(bfile)) {
|
||||
console.log(' FAIL ' + bpkg + ': cache file missing');
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
var bdata = JSON.parse(Fs.readFileSync(bfile, 'utf8'));
|
||||
if (!bdata.releases || bdata.releases.length === 0) {
|
||||
console.log(' FAIL ' + bpkg + ': 0 releases');
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
// Check that at least one release has a download URL
|
||||
var hasDownload = bdata.releases.some(function (r) {
|
||||
return r.download && r.download.startsWith('http');
|
||||
});
|
||||
if (!hasDownload) {
|
||||
console.log(' FAIL ' + bpkg + ': no releases with download URLs');
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
console.log(' PASS ' + bpkg + ': ' + bdata.releases.length + ' releases');
|
||||
passes++;
|
||||
}
|
||||
|
||||
for (var gi = 0; gi < GITTAG_PKGS.length; gi++) {
|
||||
var gpkg = GITTAG_PKGS[gi];
|
||||
var gfile = Path.join(cachePath, gpkg + '.json');
|
||||
if (!Fs.existsSync(gfile)) {
|
||||
console.log(' FAIL ' + gpkg + ' (gittag): cache file missing');
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
var gdata = JSON.parse(Fs.readFileSync(gfile, 'utf8'));
|
||||
if (!gdata.releases || gdata.releases.length === 0) {
|
||||
console.log(' FAIL ' + gpkg + ' (gittag): 0 releases');
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
console.log(' PASS ' + gpkg + ' (gittag): ' + gdata.releases.length + ' releases');
|
||||
passes++;
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 2: Format selection
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 2: Format Selection ===');
|
||||
console.log('');
|
||||
|
||||
for (var fi = 0; fi < FORMAT_CASES.length; fi++) {
|
||||
var ftc = FORMAT_CASES[fi];
|
||||
try {
|
||||
var fr = await InstallerServer.helper({
|
||||
unameAgent: ftc.ua,
|
||||
projectName: ftc.pkg,
|
||||
tag: 'stable',
|
||||
formats: ['tar', 'exe', 'zip', 'xz', 'dmg'],
|
||||
libc: '',
|
||||
});
|
||||
var fpkg = fr[0];
|
||||
if (fpkg.channel === 'error') {
|
||||
console.log(' FAIL ' + ftc.label + ': resolved to error');
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
if (fpkg.ext !== ftc.expectExt) {
|
||||
console.log(' FAIL ' + ftc.label + ': got .' + fpkg.ext + ' want .' + ftc.expectExt);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS ' + ftc.label + ': .' + fpkg.ext);
|
||||
passes++;
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(' ERROR ' + ftc.label + ': ' + e.message);
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 3: Edge-case platforms
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 3: Edge-Case Platforms ===');
|
||||
console.log('');
|
||||
|
||||
for (var ei = 0; ei < EDGE_CASES.length; ei++) {
|
||||
var etc = EDGE_CASES[ei];
|
||||
try {
|
||||
var er = await InstallerServer.helper({
|
||||
unameAgent: etc.ua,
|
||||
projectName: etc.pkg,
|
||||
tag: 'stable',
|
||||
formats: ['tar', 'exe', 'zip', 'xz', 'dmg'],
|
||||
libc: '',
|
||||
});
|
||||
var epkg = er[0];
|
||||
if (etc.expectNotError && epkg.channel === 'error') {
|
||||
console.log(' FAIL ' + etc.label + ': resolved to error (v' + epkg.version + ')');
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
if (etc.expectOs && epkg.os !== etc.expectOs) {
|
||||
console.log(' FAIL ' + etc.label + ': os=' + epkg.os + ' want=' + etc.expectOs);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
console.log(' PASS ' + etc.label + ': v' + epkg.version + ' .' + epkg.ext);
|
||||
passes++;
|
||||
} catch (e) {
|
||||
if (etc.known) {
|
||||
console.log(' KNOWN ' + etc.label + ': ' + e.message);
|
||||
knowns++;
|
||||
} else {
|
||||
console.log(' FAIL ' + etc.label + ': ' + e.message);
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 4: Script generation smoke tests
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 4: Script Generation ===');
|
||||
console.log('');
|
||||
|
||||
for (var si = 0; si < SCRIPT_CASES.length; si++) {
|
||||
var stc = SCRIPT_CASES[si];
|
||||
try {
|
||||
var script = await InstallerServer.serveInstaller(
|
||||
'https://webi.sh',
|
||||
stc.ua,
|
||||
stc.pkg,
|
||||
'stable',
|
||||
'sh',
|
||||
['tar', 'exe', 'zip', 'xz', 'dmg'],
|
||||
'',
|
||||
);
|
||||
|
||||
// Script must contain WEBI_PKG_URL and WEBI_VERSION
|
||||
var hasUrl = /WEBI_PKG_URL='[^']+'/m.test(script);
|
||||
var hasVersion = /WEBI_VERSION='[^']+'/m.test(script);
|
||||
var hasExt = /WEBI_EXT='[^']+'/m.test(script);
|
||||
|
||||
if (!hasUrl) {
|
||||
console.log(' FAIL ' + stc.label + ': missing WEBI_PKG_URL');
|
||||
failures++;
|
||||
} else if (!hasVersion) {
|
||||
console.log(' FAIL ' + stc.label + ': missing WEBI_VERSION');
|
||||
failures++;
|
||||
} else if (!hasExt) {
|
||||
console.log(' FAIL ' + stc.label + ': missing WEBI_EXT');
|
||||
failures++;
|
||||
} else {
|
||||
var vMatch = script.match(/WEBI_VERSION='([^']+)'/);
|
||||
var extMatch = script.match(/WEBI_EXT='([^']+)'/);
|
||||
console.log(' PASS ' + stc.label + ': v' + vMatch[1] + ' .' + extMatch[1]);
|
||||
passes++;
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(' FAIL ' + stc.label + ': ' + e.message.substring(0, 80));
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 5: API compat — transform-releases vocabulary
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 5: API Vocabulary (transform-releases) ===');
|
||||
console.log('');
|
||||
|
||||
for (var ai = 0; ai < API_VOCAB_CASES.length; ai++) {
|
||||
var atc = API_VOCAB_CASES[ai];
|
||||
try {
|
||||
var ares = await Releases.getReleases({
|
||||
pkg: atc.pkg,
|
||||
ver: '',
|
||||
os: '',
|
||||
arch: '',
|
||||
libc: '',
|
||||
lts: false,
|
||||
channel: '',
|
||||
formats: [],
|
||||
limit: 100,
|
||||
});
|
||||
var arels = ares.releases;
|
||||
|
||||
if (atc.expectOses) {
|
||||
var actualOses = [];
|
||||
for (var oi = 0; oi < arels.length; oi++) {
|
||||
if (actualOses.indexOf(arels[oi].os) === -1) {
|
||||
actualOses.push(arels[oi].os);
|
||||
}
|
||||
}
|
||||
actualOses.sort();
|
||||
var missingOses = [];
|
||||
for (var mi = 0; mi < atc.expectOses.length; mi++) {
|
||||
if (actualOses.indexOf(atc.expectOses[mi]) === -1) {
|
||||
missingOses.push(atc.expectOses[mi]);
|
||||
}
|
||||
}
|
||||
if (missingOses.length > 0) {
|
||||
console.log(' FAIL ' + atc.label + ': missing ' + JSON.stringify(missingOses) + ' (has ' + JSON.stringify(actualOses) + ')');
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS ' + atc.label + ': ' + JSON.stringify(atc.expectOses));
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
|
||||
if (atc.expectArches) {
|
||||
var actualArches = [];
|
||||
for (var ari = 0; ari < arels.length; ari++) {
|
||||
if (arels[ari].arch && actualArches.indexOf(arels[ari].arch) === -1) {
|
||||
actualArches.push(arels[ari].arch);
|
||||
}
|
||||
}
|
||||
actualArches.sort();
|
||||
var missingArches = [];
|
||||
for (var mai = 0; mai < atc.expectArches.length; mai++) {
|
||||
if (actualArches.indexOf(atc.expectArches[mai]) === -1) {
|
||||
missingArches.push(atc.expectArches[mai]);
|
||||
}
|
||||
}
|
||||
if (missingArches.length > 0) {
|
||||
console.log(' FAIL ' + atc.label + ': missing ' + JSON.stringify(missingArches) + ' (has ' + JSON.stringify(actualArches) + ')');
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS ' + atc.label);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(' FAIL ' + atc.label + ': ' + e.message);
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 6: Version format
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 6: Version Format ===');
|
||||
console.log('');
|
||||
|
||||
// Version format check: the Go cache may have 'v' prefixes (that's OK —
|
||||
// normalize.js and serve-installer.js strip them). What matters is that
|
||||
// after normalization via transform-releases, versions have no 'v' prefix.
|
||||
for (var vi = 0; vi < VERSION_CHECKS.length; vi++) {
|
||||
var vpkg = VERSION_CHECKS[vi];
|
||||
try {
|
||||
var vres = await Releases.getReleases({
|
||||
pkg: vpkg,
|
||||
ver: '',
|
||||
os: '',
|
||||
arch: '',
|
||||
libc: '',
|
||||
lts: false,
|
||||
channel: '',
|
||||
formats: [],
|
||||
limit: 10,
|
||||
});
|
||||
var vrels = vres.releases;
|
||||
var badVersions = [];
|
||||
for (var vri = 0; vri < vrels.length; vri++) {
|
||||
var ver = vrels[vri].version;
|
||||
if (!ver) {
|
||||
badVersions.push('(empty)');
|
||||
break;
|
||||
}
|
||||
if (ver.startsWith('v')) {
|
||||
badVersions.push(ver);
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (badVersions.length > 0) {
|
||||
console.log(' FAIL ' + vpkg + ': v-prefix not stripped: ' + badVersions[0]);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS ' + vpkg + ': latest=' + (vrels[0] ? vrels[0].version : '?'));
|
||||
passes++;
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(' FAIL ' + vpkg + ': ' + e.message);
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 7: Channel detection
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 7: Channel Detection ===');
|
||||
console.log('');
|
||||
|
||||
for (var ci = 0; ci < CHANNEL_CASES.length; ci++) {
|
||||
var ctc = CHANNEL_CASES[ci];
|
||||
try {
|
||||
var cres = await Releases.getReleases({
|
||||
pkg: ctc.pkg,
|
||||
ver: '',
|
||||
os: '',
|
||||
arch: '',
|
||||
libc: '',
|
||||
lts: false,
|
||||
channel: ctc.channel,
|
||||
formats: [],
|
||||
limit: 100,
|
||||
});
|
||||
var count = cres.releases.length;
|
||||
if (count < ctc.expectMinCount) {
|
||||
console.log(' FAIL ' + ctc.label + ': only ' + count + ' (want >=' + ctc.expectMinCount + ')');
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS ' + ctc.label + ': ' + count + ' releases');
|
||||
passes++;
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(' FAIL ' + ctc.label + ': ' + e.message);
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Summary
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Results: ' + passes + ' passed, ' + failures + ' failed, ' + knowns + ' known, ' + skips + ' skipped ===');
|
||||
if (failures > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
339
_webi/test-fleet-diff.js
Normal file
339
_webi/test-fleet-diff.js
Normal file
@@ -0,0 +1,339 @@
|
||||
'use strict';
|
||||
|
||||
// Fleet-wide diff: compare a candidate host (e.g. beta.webi.sh) against
|
||||
// production for every cached package, across multiple OS/arch combos.
|
||||
// Outputs TSV for grep/sort.
|
||||
//
|
||||
// Usage:
|
||||
// node _webi/test-fleet-diff.js
|
||||
// --cand-url=https://beta.webi.sh
|
||||
// --prod-url=https://webinstall.dev # default
|
||||
// --kind=api # default; or "installer"
|
||||
// --pkgs=bat,go,... # default: all cached packages
|
||||
// --concurrency=8 # default
|
||||
// --out=fleet-api.tsv # default: stdout
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Os = require('node:os');
|
||||
let Path = require('node:path');
|
||||
let Https = require('node:https');
|
||||
|
||||
let CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
|
||||
function arg(name, dflt) {
|
||||
for (let a of process.argv) {
|
||||
if (a.startsWith(`--${name}=`)) {
|
||||
return a.slice(name.length + 3);
|
||||
}
|
||||
}
|
||||
return dflt;
|
||||
}
|
||||
|
||||
let CAND_URL = arg('cand-url', 'https://beta.webi.sh').replace(/\/+$/, '');
|
||||
let PROD_URL = arg('prod-url', 'https://webinstall.dev').replace(/\/+$/, '');
|
||||
let KIND = arg('kind', 'api');
|
||||
let PKGS_ARG = arg('pkgs', '');
|
||||
let CONCURRENCY = parseInt(arg('concurrency', '8'), 10);
|
||||
let OUT = arg('out', '');
|
||||
|
||||
// OS/arch matrix for API mode
|
||||
let API_MATRIX = [
|
||||
{ os: 'macos', arch: 'amd64' },
|
||||
{ os: 'macos', arch: 'arm64' },
|
||||
{ os: 'linux', arch: 'amd64' },
|
||||
{ os: 'linux', arch: 'arm64' },
|
||||
{ os: 'linux', arch: 'armv7l' },
|
||||
{ os: 'windows', arch: 'amd64' },
|
||||
{ os: 'freebsd', arch: 'amd64' },
|
||||
];
|
||||
|
||||
// UA strings for installer mode
|
||||
let INSTALLER_MATRIX = [
|
||||
{ label: 'macos_arm64', ua: 'aarch64/unknown Darwin/24.2.0 libc' },
|
||||
{ label: 'macos_amd64', ua: 'x86_64/unknown Darwin/23.0.0 libc' },
|
||||
{ label: 'linux_amd64', ua: 'x86_64/unknown Linux/5.15.0 libc' },
|
||||
{ label: 'linux_arm64', ua: 'aarch64/unknown Linux/5.15.0 libc' },
|
||||
{ label: 'linux_musl', ua: 'x86_64/unknown Linux/5.15.0 musl' },
|
||||
{ label: 'windows_amd64', ua: 'x86_64/unknown Windows/10.0.19041 msvc' },
|
||||
];
|
||||
|
||||
function httpsGet(url, headers) {
|
||||
return new Promise(function (resolve, reject) {
|
||||
let opts = { headers: headers || {}, timeout: 15000 };
|
||||
let req = Https.get(url, opts, function (res) {
|
||||
// Follow one redirect
|
||||
if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
|
||||
let redir = res.headers.location;
|
||||
if (redir.startsWith('/')) {
|
||||
let m = url.match(/^(https?:\/\/[^/]+)/);
|
||||
redir = (m ? m[1] : '') + redir;
|
||||
}
|
||||
Https.get(redir, opts, function (res2) {
|
||||
let data = '';
|
||||
res2.on('data', function (c) { data += c; });
|
||||
res2.on('end', function () { resolve({ status: res2.statusCode, body: data }); });
|
||||
}).on('error', reject);
|
||||
return;
|
||||
}
|
||||
let data = '';
|
||||
res.on('data', function (c) { data += c; });
|
||||
res.on('end', function () { resolve({ status: res.statusCode, body: data }); });
|
||||
});
|
||||
req.on('error', reject);
|
||||
req.on('timeout', function () { req.destroy(new Error('timeout')); });
|
||||
});
|
||||
}
|
||||
|
||||
async function listCachedPkgs() {
|
||||
let entries = await Fs.readdir(CACHE_DIR);
|
||||
return entries
|
||||
.filter(function (n) { return n.endsWith('.json') && !n.endsWith('.updated.txt'); })
|
||||
.map(function (n) { return n.slice(0, -5); })
|
||||
.sort();
|
||||
}
|
||||
|
||||
function safeFirst(json) {
|
||||
try {
|
||||
let arr = JSON.parse(json);
|
||||
if (!Array.isArray(arr) || arr.length === 0) {
|
||||
return null;
|
||||
}
|
||||
return arr[0];
|
||||
} catch (e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function parseInstallerVars(script) {
|
||||
let vars = {};
|
||||
let re = /^(?:export\s+)?(WEBI_\w+|PKG_NAME)='([^']*)'/gm;
|
||||
let m;
|
||||
while ((m = re.exec(script)) !== null) {
|
||||
vars[m[1]] = m[2];
|
||||
}
|
||||
return vars;
|
||||
}
|
||||
|
||||
async function diffApi(pkg, os, arch) {
|
||||
let qs = `?os=${os}&arch=${arch}&limit=1`;
|
||||
let candUrl = `${CAND_URL}/api/releases/${pkg}@stable.json${qs}`;
|
||||
let prodUrl = `${PROD_URL}/api/releases/${pkg}@stable.json${qs}`;
|
||||
|
||||
let cand, prod;
|
||||
try {
|
||||
[cand, prod] = await Promise.all([httpsGet(candUrl), httpsGet(prodUrl)]);
|
||||
} catch (e) {
|
||||
return { pkg, os, arch, status: 'fetch_error', detail: e.message };
|
||||
}
|
||||
|
||||
if (cand.status !== 200 || prod.status !== 200) {
|
||||
return {
|
||||
pkg, os, arch, status: 'http_error',
|
||||
detail: `cand=${cand.status} prod=${prod.status}`,
|
||||
};
|
||||
}
|
||||
|
||||
let candFirst = safeFirst(cand.body);
|
||||
let prodFirst = safeFirst(prod.body);
|
||||
|
||||
let candErr = !candFirst || candFirst.channel === 'error';
|
||||
let prodErr = !prodFirst || prodFirst.channel === 'error';
|
||||
|
||||
if (candErr && prodErr) {
|
||||
return { pkg, os, arch, status: 'both_error', detail: '' };
|
||||
}
|
||||
if (candErr && !prodErr) {
|
||||
return {
|
||||
pkg, os, arch, status: 'cand_only_error',
|
||||
detail: `prod=${prodFirst.version}/${prodFirst.ext}`,
|
||||
};
|
||||
}
|
||||
if (!candErr && prodErr) {
|
||||
return {
|
||||
pkg, os, arch, status: 'prod_only_error',
|
||||
detail: `cand=${candFirst.version}/${candFirst.ext}`,
|
||||
};
|
||||
}
|
||||
|
||||
// Both succeeded — diff the key fields
|
||||
let diffs = [];
|
||||
for (let f of ['os', 'arch', 'libc', 'ext']) {
|
||||
if (candFirst[f] !== prodFirst[f]) {
|
||||
diffs.push(`${f}:cand=${candFirst[f]}|prod=${prodFirst[f]}`);
|
||||
}
|
||||
}
|
||||
|
||||
let ver = candFirst.version === prodFirst.version
|
||||
? candFirst.version
|
||||
: `cand=${candFirst.version}|prod=${prodFirst.version}`;
|
||||
|
||||
return {
|
||||
pkg, os, arch,
|
||||
status: diffs.length === 0 ? 'match' : 'diff',
|
||||
detail: diffs.length === 0 ? `v${ver} ${candFirst.ext}` : `v${ver} ${diffs.join(',')}`,
|
||||
};
|
||||
}
|
||||
|
||||
async function diffInstaller(pkg, label, ua) {
|
||||
let candUrl = `${CAND_URL}/api/installers/${pkg}@stable.sh`;
|
||||
let prodUrl = `${PROD_URL}/api/installers/${pkg}@stable.sh`;
|
||||
let headers = { 'User-Agent': ua };
|
||||
|
||||
let cand, prod;
|
||||
try {
|
||||
[cand, prod] = await Promise.all([
|
||||
httpsGet(candUrl, headers),
|
||||
httpsGet(prodUrl, headers),
|
||||
]);
|
||||
} catch (e) {
|
||||
return { pkg, label, status: 'fetch_error', detail: e.message };
|
||||
}
|
||||
|
||||
if (cand.status !== 200 || prod.status !== 200) {
|
||||
return {
|
||||
pkg, label, status: 'http_error',
|
||||
detail: `cand=${cand.status} prod=${prod.status}`,
|
||||
};
|
||||
}
|
||||
|
||||
let candVars = parseInstallerVars(cand.body);
|
||||
let prodVars = parseInstallerVars(prod.body);
|
||||
|
||||
let candHas = candVars.WEBI_PKG_URL && candVars.WEBI_EXT && candVars.WEBI_EXT !== 'err';
|
||||
let prodHas = prodVars.WEBI_PKG_URL && prodVars.WEBI_EXT && prodVars.WEBI_EXT !== 'err';
|
||||
|
||||
if (!candHas && !prodHas) {
|
||||
return { pkg, label, status: 'both_error', detail: '' };
|
||||
}
|
||||
if (!candHas && prodHas) {
|
||||
return {
|
||||
pkg, label, status: 'cand_only_error',
|
||||
detail: `prod=${prodVars.WEBI_VERSION}/${prodVars.WEBI_EXT}`,
|
||||
};
|
||||
}
|
||||
if (candHas && !prodHas) {
|
||||
return {
|
||||
pkg, label, status: 'prod_only_error',
|
||||
detail: `cand=${candVars.WEBI_VERSION}/${candVars.WEBI_EXT}`,
|
||||
};
|
||||
}
|
||||
|
||||
// Diff WEBI_OS, WEBI_ARCH, WEBI_EXT (PKG_NAME may differ for aliases)
|
||||
let diffs = [];
|
||||
for (let v of ['WEBI_OS', 'WEBI_ARCH', 'WEBI_EXT']) {
|
||||
if (candVars[v] !== prodVars[v]) {
|
||||
diffs.push(`${v}:cand=${candVars[v]}|prod=${prodVars[v]}`);
|
||||
}
|
||||
}
|
||||
|
||||
let ver = candVars.WEBI_VERSION === prodVars.WEBI_VERSION
|
||||
? candVars.WEBI_VERSION
|
||||
: `cand=${candVars.WEBI_VERSION}|prod=${prodVars.WEBI_VERSION}`;
|
||||
|
||||
return {
|
||||
pkg, label,
|
||||
status: diffs.length === 0 ? 'match' : 'diff',
|
||||
detail: diffs.length === 0 ? `v${ver} ${candVars.WEBI_EXT}` : `v${ver} ${diffs.join(',')}`,
|
||||
};
|
||||
}
|
||||
|
||||
async function pool(items, fn, concurrency) {
|
||||
let results = new Array(items.length);
|
||||
let i = 0;
|
||||
async function worker() {
|
||||
while (true) {
|
||||
let idx = i++;
|
||||
if (idx >= items.length) {
|
||||
return;
|
||||
}
|
||||
try {
|
||||
results[idx] = await fn(items[idx], idx);
|
||||
} catch (e) {
|
||||
results[idx] = { status: 'exception', detail: e.message, _item: items[idx] };
|
||||
}
|
||||
}
|
||||
}
|
||||
let workers = [];
|
||||
for (let k = 0; k < concurrency; k++) {
|
||||
workers.push(worker());
|
||||
}
|
||||
await Promise.all(workers);
|
||||
return results;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
let pkgs;
|
||||
if (PKGS_ARG) {
|
||||
pkgs = PKGS_ARG.split(',').filter(Boolean);
|
||||
} else {
|
||||
pkgs = await listCachedPkgs();
|
||||
}
|
||||
|
||||
console.error(`Comparing ${pkgs.length} packages: ${CAND_URL} (cand) vs ${PROD_URL} (prod)`);
|
||||
console.error(`Mode: ${KIND}, concurrency: ${CONCURRENCY}`);
|
||||
|
||||
let jobs = [];
|
||||
if (KIND === 'api') {
|
||||
for (let pkg of pkgs) {
|
||||
for (let combo of API_MATRIX) {
|
||||
jobs.push({ pkg, os: combo.os, arch: combo.arch });
|
||||
}
|
||||
}
|
||||
} else if (KIND === 'installer') {
|
||||
for (let pkg of pkgs) {
|
||||
for (let combo of INSTALLER_MATRIX) {
|
||||
jobs.push({ pkg, label: combo.label, ua: combo.ua });
|
||||
}
|
||||
}
|
||||
} else {
|
||||
console.error(`Unknown kind: ${KIND}`);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
let started = Date.now();
|
||||
let results = await pool(jobs, async function (job) {
|
||||
if (KIND === 'api') {
|
||||
return diffApi(job.pkg, job.os, job.arch);
|
||||
}
|
||||
return diffInstaller(job.pkg, job.label, job.ua);
|
||||
}, CONCURRENCY);
|
||||
let elapsed = ((Date.now() - started) / 1000).toFixed(1);
|
||||
|
||||
// TSV output
|
||||
let lines = [];
|
||||
if (KIND === 'api') {
|
||||
lines.push(['pkg', 'os', 'arch', 'status', 'detail'].join('\t'));
|
||||
for (let r of results) {
|
||||
lines.push([r.pkg, r.os, r.arch, r.status, r.detail || ''].join('\t'));
|
||||
}
|
||||
} else {
|
||||
lines.push(['pkg', 'target', 'status', 'detail'].join('\t'));
|
||||
for (let r of results) {
|
||||
lines.push([r.pkg, r.label, r.status, r.detail || ''].join('\t'));
|
||||
}
|
||||
}
|
||||
let body = lines.join('\n') + '\n';
|
||||
|
||||
if (OUT) {
|
||||
await Fs.writeFile(OUT, body, 'utf8');
|
||||
console.error(`Wrote ${OUT}`);
|
||||
} else {
|
||||
process.stdout.write(body);
|
||||
}
|
||||
|
||||
// Summary to stderr
|
||||
let counts = {};
|
||||
for (let r of results) {
|
||||
counts[r.status] = (counts[r.status] || 0) + 1;
|
||||
}
|
||||
console.error('');
|
||||
console.error(`=== Summary (${elapsed}s, ${results.length} jobs) ===`);
|
||||
for (let s of Object.keys(counts).sort()) {
|
||||
console.error(` ${s}: ${counts[s]}`);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
444
_webi/test-installer-resolve.js
Normal file
444
_webi/test-installer-resolve.js
Normal file
@@ -0,0 +1,444 @@
|
||||
'use strict';
|
||||
|
||||
let InstallerServer = require('./serve-installer.js');
|
||||
let Builds = require('./builds.js');
|
||||
|
||||
// Real User-Agent strings sent by webi bootstrap scripts.
|
||||
//
|
||||
// Libc taxonomy:
|
||||
// none = static build, no runtime libc dep (often built with musl, but self-contained)
|
||||
// musl = requires musl C/C++ runtime at runtime (e.g. node-musl)
|
||||
// gnu = requires glibc at runtime (crashes on musl-only/Alpine)
|
||||
// libc = host UA value meaning "I have glibc" (not used in release metadata)
|
||||
//
|
||||
// Known issues:
|
||||
//
|
||||
// 1. WATERFALL libc vs gnu: The WATERFALL maps `libc` => ['none', 'libc']
|
||||
// but never tries 'gnu'. Packages with glibc-linked builds (libc='gnu' in
|
||||
// Go cache) won't match for hosts reporting 'libc'. Fix: update WATERFALL
|
||||
// to `libc: ['none', 'gnu', 'libc']` in build-classifier submodule.
|
||||
//
|
||||
// 2. Go cache .git regression: The Go cache includes .git source repo URLs
|
||||
// as releases, creating ANYOS/ANYARCH triplets. These match before
|
||||
// platform-specific binaries. Fix: exclude .git from Go cache output.
|
||||
|
||||
let UA_CASES = [
|
||||
// === macOS (no libc issue — darwin uses libc='none') ===
|
||||
{
|
||||
label: 'bat macOS arm64',
|
||||
pkg: 'bat',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectArch: 'aarch64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'bat macOS amd64',
|
||||
pkg: 'bat',
|
||||
ua: 'x86_64/unknown Darwin/23.0.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'go macOS arm64',
|
||||
pkg: 'go',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectArch: 'aarch64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'node macOS arm64',
|
||||
pkg: 'node',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectArch: 'aarch64',
|
||||
expectExt: 'tar.xz',
|
||||
},
|
||||
{
|
||||
label: 'rg macOS arm64',
|
||||
pkg: 'rg',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectArch: 'aarch64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
|
||||
// === macOS universal2 — packages where recent darwin builds are universal-only ===
|
||||
// These currently resolve to ancient versions because universal2 entries are
|
||||
// dropped by the classifier. The GOER's legacy export needs to emit these
|
||||
// with arch: "x86_64" so the classifier accepts them. The darwin WATERFALL
|
||||
// (aarch64 falls back to x86_64) handles aarch64 users.
|
||||
{
|
||||
label: 'cmake macOS arm64 (universal2)',
|
||||
pkg: 'cmake',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectExt: 'tar.gz',
|
||||
expectMinVersion: '4.0.0',
|
||||
known: true,
|
||||
},
|
||||
{
|
||||
label: 'cmake macOS amd64 (universal2)',
|
||||
pkg: 'cmake',
|
||||
ua: 'x86_64/unknown Darwin/23.0.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectExt: 'tar.gz',
|
||||
expectMinVersion: '4.0.0',
|
||||
known: true,
|
||||
},
|
||||
{
|
||||
label: 'hugo macOS arm64 (universal2)',
|
||||
pkg: 'hugo',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectExt: 'tar.gz',
|
||||
expectMinVersion: '0.140.0',
|
||||
known: true,
|
||||
},
|
||||
{
|
||||
label: 'hugo macOS amd64 (universal2)',
|
||||
pkg: 'hugo',
|
||||
ua: 'x86_64/unknown Darwin/23.0.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectExt: 'tar.gz',
|
||||
expectMinVersion: '0.140.0',
|
||||
known: true,
|
||||
},
|
||||
|
||||
// === Windows ===
|
||||
{
|
||||
label: 'bat Windows amd64',
|
||||
pkg: 'bat',
|
||||
ua: 'x86_64/unknown Windows/10.0.19041 msvc',
|
||||
expectOs: 'windows',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'zip',
|
||||
},
|
||||
{
|
||||
label: 'go Windows amd64',
|
||||
pkg: 'go',
|
||||
ua: 'x86_64/unknown Windows/10.0.19041 msvc',
|
||||
expectOs: 'windows',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'zip',
|
||||
},
|
||||
|
||||
// === Linux musl (Alpine/Docker) ===
|
||||
{
|
||||
label: 'bat Linux musl',
|
||||
pkg: 'bat',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 musl',
|
||||
expectOs: 'linux',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
|
||||
// === Linux glibc — packages with libc='none' in cache ===
|
||||
{
|
||||
label: 'go Linux amd64',
|
||||
pkg: 'go',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectOs: 'linux',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
// === Linux glibc — packages with libc='gnu' in cache ===
|
||||
// These previously failed (WATERFALL libc→gnu gap). Fixed by adding
|
||||
// 'gnu' to the libc candidates for glibc hosts in _enumerateTriplets.
|
||||
{
|
||||
label: 'bat Linux amd64',
|
||||
pkg: 'bat',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectOs: 'linux',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'rg Linux amd64',
|
||||
pkg: 'rg',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectOs: 'linux',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'node Linux amd64',
|
||||
pkg: 'node',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectOs: 'linux',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'tar.xz',
|
||||
},
|
||||
|
||||
// === Packages with .git source URLs in old releases ===
|
||||
// These previously failed (ANYOS .git matched before platform binary).
|
||||
// Fixed by putting specific OS before ANYOS in triplet enumeration.
|
||||
{
|
||||
label: 'jq macOS arm64',
|
||||
pkg: 'jq',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectArch: 'aarch64',
|
||||
expectExt: 'exe',
|
||||
},
|
||||
{
|
||||
label: 'caddy macOS arm64',
|
||||
pkg: 'caddy',
|
||||
ua: 'aarch64/unknown Darwin/24.2.0 libc',
|
||||
expectOs: 'darwin',
|
||||
expectArch: 'aarch64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
{
|
||||
label: 'caddy Linux amd64',
|
||||
pkg: 'caddy',
|
||||
ua: 'x86_64/unknown Linux/5.15.0 libc',
|
||||
expectOs: 'linux',
|
||||
expectArch: 'x86_64',
|
||||
expectExt: 'tar.gz',
|
||||
},
|
||||
];
|
||||
|
||||
async function main() {
|
||||
let failures = 0;
|
||||
let passes = 0;
|
||||
let knowns = 0;
|
||||
let errors = 0;
|
||||
|
||||
console.log('Initializing build cache...');
|
||||
await Builds.init();
|
||||
console.log('');
|
||||
|
||||
console.log('=== Installer Resolution Tests ===');
|
||||
console.log('');
|
||||
|
||||
for (let tc of UA_CASES) {
|
||||
try {
|
||||
let [pkg, params] = await InstallerServer.helper({
|
||||
unameAgent: tc.ua,
|
||||
projectName: tc.pkg,
|
||||
tag: 'stable',
|
||||
formats: ['tar', 'exe', 'zip', 'xz', 'dmg'],
|
||||
libc: '',
|
||||
});
|
||||
|
||||
// Known issue — just verify it fails as expected
|
||||
if (tc.known) {
|
||||
let isError = pkg.channel === 'error' || !pkg.download || pkg.download.includes('doesntexist') || pkg.ext === 'git';
|
||||
let isStale = false;
|
||||
if (tc.expectMinVersion && pkg.version) {
|
||||
let got = pkg.version.replace(/^v/, '').split('.').map(Number);
|
||||
let want = tc.expectMinVersion.split('.').map(Number);
|
||||
for (let i = 0; i < want.length; i++) {
|
||||
if ((got[i] || 0) < want[i]) { isStale = true; break; }
|
||||
if ((got[i] || 0) > want[i]) { break; }
|
||||
}
|
||||
}
|
||||
if (isError || isStale) {
|
||||
let detail = isStale ? `stale v${pkg.version} < v${tc.expectMinVersion}` : '';
|
||||
console.log(` KNOWN ${tc.label}${detail ? ': ' + detail : ''}`);
|
||||
knowns++;
|
||||
} else {
|
||||
console.log(` PASS ${tc.label} (known issue resolved!): v${pkg.version} .${pkg.ext} ${(pkg.download || '').split('/').pop()}`);
|
||||
passes++;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (pkg.channel === 'error') {
|
||||
console.log(` FAIL ${tc.label}: resolved to error package`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let diffs = [];
|
||||
|
||||
if (tc.expectOs && pkg.os !== tc.expectOs) {
|
||||
diffs.push(`os: got=${pkg.os} want=${tc.expectOs}`);
|
||||
}
|
||||
if (tc.expectArch && pkg.arch !== tc.expectArch) {
|
||||
diffs.push(`arch: got=${pkg.arch} want=${tc.expectArch}`);
|
||||
}
|
||||
if (tc.expectExt && pkg.ext !== tc.expectExt) {
|
||||
diffs.push(`ext: got=${pkg.ext} want=${tc.expectExt}`);
|
||||
}
|
||||
|
||||
if (!pkg.version || pkg.version === '0.0.0') {
|
||||
diffs.push('version: missing or zero');
|
||||
}
|
||||
|
||||
if (!pkg.download || pkg.download.includes('doesntexist')) {
|
||||
diffs.push('download: missing or error');
|
||||
}
|
||||
|
||||
if (diffs.length > 0) {
|
||||
console.log(` FAIL ${tc.label}: ${diffs.join(', ')}`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${tc.label}: v${pkg.version} .${pkg.ext} ${pkg.download.split('/').pop()}`);
|
||||
passes++;
|
||||
}
|
||||
} catch (err) {
|
||||
if (tc.known) {
|
||||
console.log(` KNOWN ${tc.label} (error: ${err.message})`);
|
||||
knowns++;
|
||||
continue;
|
||||
}
|
||||
console.log(` ERROR ${tc.label}: ${err.message}`);
|
||||
errors++;
|
||||
}
|
||||
}
|
||||
|
||||
console.log('');
|
||||
console.log(`=== Results: ${passes} passed, ${failures} failed, ${knowns} known, ${errors} errors ===`);
|
||||
if (failures > 0 || errors > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
// Cache value validation: the classifier re-parses filenames and rejects
|
||||
// entries where the cache os/arch doesn't match. These checks prevent
|
||||
// regressions where someone "normalizes" cache values in a way that
|
||||
// breaks the classifier.
|
||||
console.log('');
|
||||
console.log('=== Cache Value Validation ===');
|
||||
console.log('');
|
||||
let cacheFailures = await validateCacheValues();
|
||||
if (cacheFailures > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Verify that cache os/arch values match what the Node classifier expects
|
||||
// to extract from the download filename. The classifier is a submodule and
|
||||
// is NOT being modified — the cache must emit values it already recognizes.
|
||||
//
|
||||
// Known bug (LIVE_cache): the Go legacy export previously translated
|
||||
// solaris/illumos → sunos in the cache, but the filenames still say
|
||||
// solaris/illumos. The classifier detects the filename value and rejects
|
||||
// the entry when it doesn't match. Same issue with universal2 arch.
|
||||
//
|
||||
// Rule: cache os/arch must match the filename, not some "canonical" form.
|
||||
|
||||
// Cache os/arch values must match what the Node classifier extracts from the
|
||||
// download filename. The classifier already recognizes solaris, illumos, sunos,
|
||||
// armhf, armel, etc. — these are not new values. The only value the classifier
|
||||
// does NOT recognize is "universal2" — use "x86_64" instead.
|
||||
//
|
||||
// matchField: which field to check in the release entry ('name' or 'download')
|
||||
let CACHE_CHECKS = [
|
||||
// The classifier knows "solaris" as an OS. Filenames/URLs say "solaris".
|
||||
// Do NOT translate to "sunos" — that creates a mismatch and drops the entry.
|
||||
{
|
||||
label: 'terraform solaris entries have os=solaris (not sunos)',
|
||||
pkg: 'terraform',
|
||||
matchField: 'download',
|
||||
filenameMatch: /solaris/,
|
||||
field: 'os',
|
||||
expect: 'solaris',
|
||||
},
|
||||
{
|
||||
label: 'syncthing solaris entries have os=solaris (not sunos)',
|
||||
pkg: 'syncthing',
|
||||
matchField: 'download',
|
||||
filenameMatch: /solaris/,
|
||||
field: 'os',
|
||||
expect: 'solaris',
|
||||
},
|
||||
// The classifier knows "illumos" as an OS. Don't translate to sunos.
|
||||
{
|
||||
label: 'syncthing illumos entries have os=illumos (not sunos)',
|
||||
pkg: 'syncthing',
|
||||
matchField: 'download',
|
||||
filenameMatch: /illumos/,
|
||||
field: 'os',
|
||||
expect: 'illumos',
|
||||
},
|
||||
// node.js uses "sunos" in filenames — cache must say "sunos" (already correct)
|
||||
{
|
||||
label: 'node sunos entries have os=sunos',
|
||||
pkg: 'node',
|
||||
matchField: 'name',
|
||||
filenameMatch: /sunos/,
|
||||
field: 'os',
|
||||
expect: 'sunos',
|
||||
},
|
||||
// The classifier maps "universal" in filenames → x86_64. The classifier does
|
||||
// NOT recognize "universal2". Cache must say arch="x86_64" for these entries.
|
||||
// aarch64 users get them via the darwin WATERFALL (aarch64 → x86_64 fallback).
|
||||
{
|
||||
label: 'cmake universal entries have arch=x86_64 (not universal2)',
|
||||
pkg: 'cmake',
|
||||
matchField: 'download',
|
||||
filenameMatch: /universal/,
|
||||
field: 'arch',
|
||||
expect: 'x86_64',
|
||||
},
|
||||
{
|
||||
label: 'hugo universal entries have arch=x86_64 (not universal2)',
|
||||
pkg: 'hugo',
|
||||
matchField: 'download',
|
||||
filenameMatch: /universal/,
|
||||
field: 'arch',
|
||||
expect: 'x86_64',
|
||||
},
|
||||
];
|
||||
|
||||
async function validateCacheValues() {
|
||||
let Os = require('node:os');
|
||||
let Path = require('path');
|
||||
let Fs = require('fs');
|
||||
|
||||
let cachePath = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
if (!Fs.existsSync(cachePath)) {
|
||||
console.log(' SKIP: no cache directory at ' + cachePath);
|
||||
return 0;
|
||||
}
|
||||
|
||||
let failures = 0;
|
||||
|
||||
for (let check of CACHE_CHECKS) {
|
||||
let filePath = Path.join(cachePath, `${check.pkg}.json`);
|
||||
if (!Fs.existsSync(filePath)) {
|
||||
console.log(` SKIP ${check.label}: no cache file`);
|
||||
continue;
|
||||
}
|
||||
|
||||
let data = JSON.parse(Fs.readFileSync(filePath, 'utf8'));
|
||||
let matchField = check.matchField || 'name';
|
||||
let matched = data.releases.filter(function (r) {
|
||||
return check.filenameMatch.test(r[matchField]);
|
||||
});
|
||||
|
||||
if (matched.length === 0) {
|
||||
console.log(` SKIP ${check.label}: no matching filenames`);
|
||||
continue;
|
||||
}
|
||||
|
||||
let wrong = matched.filter(function (r) {
|
||||
return r[check.field] !== check.expect;
|
||||
});
|
||||
|
||||
if (wrong.length > 0) {
|
||||
let sample = wrong[0];
|
||||
console.log(
|
||||
` FAIL ${check.label}: ${wrong.length}/${matched.length} entries have` +
|
||||
` ${check.field}="${sample[check.field]}" (want "${check.expect}")` +
|
||||
` e.g. ${sample.name}`,
|
||||
);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${check.label}: ${matched.length} entries OK`);
|
||||
}
|
||||
}
|
||||
|
||||
console.log('');
|
||||
console.log(`=== Cache Validation: ${CACHE_CHECKS.length - failures} passed, ${failures} failed ===`);
|
||||
return failures;
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
450
_webi/test-live-cache-diff.js
Normal file
450
_webi/test-live-cache-diff.js
Normal file
@@ -0,0 +1,450 @@
|
||||
'use strict';
|
||||
|
||||
// Compare _cache vs LIVE_cache for correctness and compatibility.
|
||||
//
|
||||
// Rules (from NODER_PURPOSE.md):
|
||||
// - _cache should be more complete and more correct than LIVE_cache
|
||||
// - Must NOT introduce new tags (OS, arch, libc) that don't exist in LIVE_cache
|
||||
// - Must NOT break compatibility with existing data
|
||||
//
|
||||
// Usage: node _webi/test-live-cache-diff.js
|
||||
|
||||
var Fs = require('node:fs');
|
||||
var Os = require('node:os');
|
||||
var Path = require('node:path');
|
||||
|
||||
// CACHE_DIR is the live cache produced by webicached (flat layout).
|
||||
// LIVE_DIR is the historical snapshot taken pre-cutover (month-bucketed).
|
||||
var CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
var LIVE_DIR = Path.join(__dirname, '..', 'LIVE_cache');
|
||||
|
||||
// resolveLayout: figures out whether dir uses the flat layout
|
||||
// (~/.cache/webi/legacy/<pkg>.json) or the legacy month-bucketed layout
|
||||
// (LIVE_cache/<YYYY-MM>/<pkg>.json) and returns the directory to read from.
|
||||
function resolveLayout(dir) {
|
||||
if (!Fs.existsSync(dir)) {
|
||||
return null;
|
||||
}
|
||||
var entries = Fs.readdirSync(dir);
|
||||
var months = entries
|
||||
.filter(function (d) { return /^\d{4}-\d{2}$/.test(d); })
|
||||
.sort()
|
||||
.reverse();
|
||||
if (months[0]) {
|
||||
return Path.join(dir, months[0]);
|
||||
}
|
||||
// Flat layout (cache files directly under dir).
|
||||
return dir;
|
||||
}
|
||||
|
||||
function loadReleases(layoutPath, pkg) {
|
||||
var file = Path.join(layoutPath, pkg + '.json');
|
||||
if (!Fs.existsSync(file)) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
return JSON.parse(Fs.readFileSync(file, 'utf8'));
|
||||
} catch (e) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function uniqueValues(releases, field) {
|
||||
var seen = {};
|
||||
for (var i = 0; i < releases.length; i++) {
|
||||
var val = releases[i][field];
|
||||
if (val !== null && val !== undefined && val !== '') {
|
||||
seen[val] = true;
|
||||
}
|
||||
}
|
||||
return Object.keys(seen).sort();
|
||||
}
|
||||
|
||||
async function main() {
|
||||
var passes = 0;
|
||||
var failures = 0;
|
||||
var warns = 0;
|
||||
|
||||
var cachePath = resolveLayout(CACHE_DIR);
|
||||
var livePath = resolveLayout(LIVE_DIR);
|
||||
|
||||
if (!cachePath) {
|
||||
console.error('No _cache directory found');
|
||||
process.exit(1);
|
||||
}
|
||||
if (!livePath) {
|
||||
console.error('No LIVE_cache directory found');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log('Using cache: ' + cachePath + ' vs ' + livePath);
|
||||
console.log('');
|
||||
|
||||
// Get all packages that exist in both caches
|
||||
var cacheFiles = Fs.readdirSync(cachePath)
|
||||
.filter(function (f) { return f.endsWith('.json'); })
|
||||
.map(function (f) { return f.replace('.json', ''); });
|
||||
var liveFiles = Fs.readdirSync(livePath)
|
||||
.filter(function (f) { return f.endsWith('.json'); })
|
||||
.map(function (f) { return f.replace('.json', ''); });
|
||||
|
||||
var cacheSet = {};
|
||||
var liveSet = {};
|
||||
cacheFiles.forEach(function (f) { cacheSet[f] = true; });
|
||||
liveFiles.forEach(function (f) { liveSet[f] = true; });
|
||||
|
||||
var common = cacheFiles.filter(function (f) { return liveSet[f]; });
|
||||
|
||||
// ================================================================
|
||||
// Test 1: Global vocabulary — no new OS/arch/libc tags
|
||||
// ================================================================
|
||||
console.log('=== Test 1: No New Tags (OS/Arch/Libc) ===');
|
||||
console.log('');
|
||||
|
||||
var allLiveOs = {};
|
||||
var allLiveArch = {};
|
||||
var allLiveLibc = {};
|
||||
var allCacheOs = {};
|
||||
var allCacheArch = {};
|
||||
var allCacheLibc = {};
|
||||
|
||||
for (var ci = 0; ci < common.length; ci++) {
|
||||
var pkg = common[ci];
|
||||
var liveData = loadReleases(livePath, pkg);
|
||||
var cacheData = loadReleases(cachePath, pkg);
|
||||
if (!liveData || !cacheData) { continue; }
|
||||
|
||||
uniqueValues(liveData.releases, 'os').forEach(function (v) { allLiveOs[v] = true; });
|
||||
uniqueValues(liveData.releases, 'arch').forEach(function (v) { allLiveArch[v] = true; });
|
||||
uniqueValues(liveData.releases, 'libc').forEach(function (v) { allLiveLibc[v] = true; });
|
||||
uniqueValues(cacheData.releases, 'os').forEach(function (v) { allCacheOs[v] = true; });
|
||||
uniqueValues(cacheData.releases, 'arch').forEach(function (v) { allCacheArch[v] = true; });
|
||||
uniqueValues(cacheData.releases, 'libc').forEach(function (v) { allCacheLibc[v] = true; });
|
||||
}
|
||||
|
||||
var newOs = Object.keys(allCacheOs).filter(function (v) { return !allLiveOs[v]; }).sort();
|
||||
var newArch = Object.keys(allCacheArch).filter(function (v) { return !allLiveArch[v]; }).sort();
|
||||
var newLibc = Object.keys(allCacheLibc).filter(function (v) { return !allLiveLibc[v]; }).sort();
|
||||
|
||||
if (newOs.length > 0) {
|
||||
console.log(' FAIL new OS values in _cache not in LIVE_cache: ' + JSON.stringify(newOs));
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS no new OS values');
|
||||
passes++;
|
||||
}
|
||||
|
||||
if (newArch.length > 0) {
|
||||
console.log(' FAIL new arch values in _cache not in LIVE_cache: ' + JSON.stringify(newArch));
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS no new arch values');
|
||||
passes++;
|
||||
}
|
||||
|
||||
if (newLibc.length > 0) {
|
||||
console.log(' FAIL new libc values in _cache not in LIVE_cache: ' + JSON.stringify(newLibc));
|
||||
failures++;
|
||||
} else {
|
||||
console.log(' PASS no new libc values');
|
||||
passes++;
|
||||
}
|
||||
|
||||
// Show what LIVE has that _cache doesn't (informational)
|
||||
var missingOs = Object.keys(allLiveOs).filter(function (v) { return !allCacheOs[v]; }).sort();
|
||||
var missingArch = Object.keys(allLiveArch).filter(function (v) { return !allCacheArch[v]; }).sort();
|
||||
if (missingOs.length > 0) {
|
||||
console.log(' INFO OS in LIVE but not _cache: ' + JSON.stringify(missingOs));
|
||||
}
|
||||
if (missingArch.length > 0) {
|
||||
console.log(' INFO arch in LIVE but not _cache: ' + JSON.stringify(missingArch));
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 2: Per-package release count — _cache should have >= LIVE
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 2: Release Count (per package) ===');
|
||||
console.log('');
|
||||
|
||||
// LIVE_cache includes junk entries (.pem, .sig, .sha256, .deb, .rpm, etc.)
|
||||
// that Go correctly filters out. Only count installable entries.
|
||||
var junkExts = /\.(pem|sig|asc|sha256|sha512|sha256sum|sha512sum|deb|rpm|apk|sbom|json|txt|sum|md5|cosign-bundle|intoto-jsonl)$/i;
|
||||
|
||||
function countInstallable(releases) {
|
||||
var n = 0;
|
||||
for (var i = 0; i < releases.length; i++) {
|
||||
if (!junkExts.test(releases[i].name || '')) {
|
||||
n++;
|
||||
}
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
var countIssues = [];
|
||||
for (var pi = 0; pi < common.length; pi++) {
|
||||
var ppkg = common[pi];
|
||||
var pLive = loadReleases(livePath, ppkg);
|
||||
var pCache = loadReleases(cachePath, ppkg);
|
||||
if (!pLive || !pCache) { continue; }
|
||||
|
||||
var liveCount = countInstallable(pLive.releases);
|
||||
var cacheCount = pCache.releases.length;
|
||||
|
||||
// _cache should have at least as many installable releases as LIVE
|
||||
var ratio = liveCount > 0 ? cacheCount / liveCount : 1;
|
||||
if (ratio < 0.5 && liveCount > 10) {
|
||||
countIssues.push({ pkg: ppkg, live: liveCount, cache: cacheCount, ratio: ratio });
|
||||
}
|
||||
}
|
||||
|
||||
if (countIssues.length === 0) {
|
||||
console.log(' PASS all packages have adequate release counts');
|
||||
passes++;
|
||||
} else {
|
||||
for (var ii = 0; ii < countIssues.length; ii++) {
|
||||
var issue = countIssues[ii];
|
||||
console.log(' FAIL ' + issue.pkg + ': _cache=' + issue.cache + ' LIVE=' + issue.live + ' (ratio=' + issue.ratio.toFixed(2) + ')');
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 3: Per-package OS coverage — _cache should have same core OSes
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 3: OS Coverage (per package) ===');
|
||||
console.log('');
|
||||
|
||||
var coreOses = ['darwin', 'linux', 'windows'];
|
||||
var osIssues = [];
|
||||
for (var oi = 0; oi < common.length; oi++) {
|
||||
var opkg = common[oi];
|
||||
var oLive = loadReleases(livePath, opkg);
|
||||
var oCache = loadReleases(cachePath, opkg);
|
||||
if (!oLive || !oCache) { continue; }
|
||||
|
||||
var liveOses = uniqueValues(oLive.releases, 'os');
|
||||
var cacheOses = uniqueValues(oCache.releases, 'os');
|
||||
|
||||
// For each core OS in LIVE, it should also be in _cache
|
||||
for (var coi = 0; coi < coreOses.length; coi++) {
|
||||
var os = coreOses[coi];
|
||||
if (liveOses.indexOf(os) >= 0 && cacheOses.indexOf(os) < 0) {
|
||||
osIssues.push({ pkg: opkg, os: os });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (osIssues.length === 0) {
|
||||
console.log(' PASS all packages have matching core OS coverage');
|
||||
passes++;
|
||||
} else {
|
||||
for (var oii = 0; oii < osIssues.length; oii++) {
|
||||
var oisue = osIssues[oii];
|
||||
console.log(' FAIL ' + oisue.pkg + ': missing os=' + oisue.os + ' (present in LIVE)');
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 4: Per-package new tags — flag packages introducing new values
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 4: Per-Package New Tags ===');
|
||||
console.log('');
|
||||
|
||||
var tagIssues = [];
|
||||
var tagSkipped = 0;
|
||||
for (var ti = 0; ti < common.length; ti++) {
|
||||
var tpkg = common[ti];
|
||||
var tLive = loadReleases(livePath, tpkg);
|
||||
var tCache = loadReleases(cachePath, tpkg);
|
||||
if (!tLive || !tCache) { continue; }
|
||||
|
||||
var tLiveOs = {};
|
||||
var tLiveArch = {};
|
||||
uniqueValues(tLive.releases, 'os').forEach(function (v) { tLiveOs[v] = true; });
|
||||
uniqueValues(tLive.releases, 'arch').forEach(function (v) { tLiveArch[v] = true; });
|
||||
|
||||
// Skip packages where LIVE has no classified entries (all empty os/arch).
|
||||
// These are github_releases packages where classification happens at query
|
||||
// time. Our _cache filling in values is an improvement, not a regression.
|
||||
var liveOsKeys = Object.keys(tLiveOs);
|
||||
var liveArchKeys = Object.keys(tLiveArch);
|
||||
if (liveOsKeys.length === 0 && liveArchKeys.length === 0) {
|
||||
tagSkipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
var tCacheOs = uniqueValues(tCache.releases, 'os');
|
||||
var tCacheArch = uniqueValues(tCache.releases, 'arch');
|
||||
|
||||
var tNewOs = tCacheOs.filter(function (v) { return !tLiveOs[v]; });
|
||||
var tNewArch = tCacheArch.filter(function (v) { return !tLiveArch[v]; });
|
||||
|
||||
if (tNewOs.length > 0 || tNewArch.length > 0) {
|
||||
tagIssues.push({
|
||||
pkg: tpkg,
|
||||
newOs: tNewOs,
|
||||
newArch: tNewArch,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (tagSkipped > 0) {
|
||||
console.log(' INFO skipped ' + tagSkipped + ' packages with no LIVE classification (unclassified github_releases)');
|
||||
}
|
||||
if (tagIssues.length === 0) {
|
||||
console.log(' PASS no pre-classified packages introduce new tags');
|
||||
passes++;
|
||||
} else {
|
||||
for (var tii = 0; tii < tagIssues.length; tii++) {
|
||||
var tissue = tagIssues[tii];
|
||||
var parts = [];
|
||||
if (tissue.newOs.length > 0) {
|
||||
parts.push('os: ' + JSON.stringify(tissue.newOs));
|
||||
}
|
||||
if (tissue.newArch.length > 0) {
|
||||
parts.push('arch: ' + JSON.stringify(tissue.newArch));
|
||||
}
|
||||
console.log(' WARN ' + tissue.pkg + ': new tags: ' + parts.join(', '));
|
||||
warns++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 5: Latest stable version — _cache should be >= LIVE
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 5: Latest Stable Version ===');
|
||||
console.log('');
|
||||
|
||||
var stableCheckPkgs = ['bat', 'go', 'node', 'rg', 'caddy', 'jq', 'hugo', 'terraform'];
|
||||
for (var si = 0; si < stableCheckPkgs.length; si++) {
|
||||
var spkg = stableCheckPkgs[si];
|
||||
var sLive = loadReleases(livePath, spkg);
|
||||
var sCache = loadReleases(cachePath, spkg);
|
||||
if (!sLive || !sCache) {
|
||||
console.log(' SKIP ' + spkg + ': missing data');
|
||||
continue;
|
||||
}
|
||||
|
||||
// Find first stable release in each
|
||||
var liveStable = sLive.releases.find(function (r) { return r.channel === 'stable'; });
|
||||
var cacheStable = sCache.releases.find(function (r) { return r.channel === 'stable'; });
|
||||
|
||||
if (!liveStable || !cacheStable) {
|
||||
console.log(' SKIP ' + spkg + ': no stable release found');
|
||||
continue;
|
||||
}
|
||||
|
||||
var lv = (liveStable.version || '').replace(/^v/, '');
|
||||
var cv = (cacheStable.version || '').replace(/^v/, '');
|
||||
|
||||
if (lv === cv) {
|
||||
console.log(' PASS ' + spkg + ': ' + cv);
|
||||
passes++;
|
||||
} else {
|
||||
// Just warn — versions may differ due to cache age
|
||||
console.log(' WARN ' + spkg + ': LIVE=' + lv + ' _cache=' + cv);
|
||||
warns++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 6: Download URLs — all entries should have valid URLs
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 6: Download URL Validity ===');
|
||||
console.log('');
|
||||
|
||||
var urlIssues = [];
|
||||
for (var ui = 0; ui < common.length; ui++) {
|
||||
var upkg = common[ui];
|
||||
var uCache = loadReleases(cachePath, upkg);
|
||||
if (!uCache) { continue; }
|
||||
|
||||
var emptyUrls = 0;
|
||||
var badUrls = 0;
|
||||
for (var uri = 0; uri < uCache.releases.length; uri++) {
|
||||
var rel = uCache.releases[uri];
|
||||
var url = rel.download || '';
|
||||
if (url === '') {
|
||||
emptyUrls++;
|
||||
} else if (!/^https?:\/\//.test(url)) {
|
||||
badUrls++;
|
||||
}
|
||||
}
|
||||
if (emptyUrls > 0 || badUrls > 0) {
|
||||
urlIssues.push({ pkg: upkg, empty: emptyUrls, bad: badUrls });
|
||||
}
|
||||
}
|
||||
|
||||
if (urlIssues.length === 0) {
|
||||
console.log(' PASS all packages have valid download URLs');
|
||||
passes++;
|
||||
} else {
|
||||
for (var uii = 0; uii < urlIssues.length; uii++) {
|
||||
var uissue = urlIssues[uii];
|
||||
var uparts = [];
|
||||
if (uissue.empty > 0) { uparts.push(uissue.empty + ' empty'); }
|
||||
if (uissue.bad > 0) { uparts.push(uissue.bad + ' malformed'); }
|
||||
console.log(' FAIL ' + uissue.pkg + ': ' + uparts.join(', '));
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 7: Required fields — all entries should have version + name
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 7: Required Fields ===');
|
||||
console.log('');
|
||||
|
||||
var fieldIssues = [];
|
||||
for (var fi = 0; fi < common.length; fi++) {
|
||||
var fpkg = common[fi];
|
||||
var fCache = loadReleases(cachePath, fpkg);
|
||||
if (!fCache) { continue; }
|
||||
|
||||
var noVersion = 0;
|
||||
var noName = 0;
|
||||
for (var fri = 0; fri < fCache.releases.length; fri++) {
|
||||
var frel = fCache.releases[fri];
|
||||
if (!frel.version) { noVersion++; }
|
||||
if (!frel.name && !frel.download) { noName++; }
|
||||
}
|
||||
if (noVersion > 0 || noName > 0) {
|
||||
fieldIssues.push({ pkg: fpkg, noVersion: noVersion, noName: noName });
|
||||
}
|
||||
}
|
||||
|
||||
if (fieldIssues.length === 0) {
|
||||
console.log(' PASS all packages have version and name/download');
|
||||
passes++;
|
||||
} else {
|
||||
for (var fii = 0; fii < fieldIssues.length; fii++) {
|
||||
var fissue = fieldIssues[fii];
|
||||
var fparts = [];
|
||||
if (fissue.noVersion > 0) { fparts.push(fissue.noVersion + ' missing version'); }
|
||||
if (fissue.noName > 0) { fparts.push(fissue.noName + ' missing name+download'); }
|
||||
console.log(' FAIL ' + fissue.pkg + ': ' + fparts.join(', '));
|
||||
failures++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Summary
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Results: ' + passes + ' passed, ' + failures + ' failed, ' + warns + ' warnings ===');
|
||||
if (failures > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
577
_webi/test-live-compare.js
Normal file
577
_webi/test-live-compare.js
Normal file
@@ -0,0 +1,577 @@
|
||||
'use strict';
|
||||
|
||||
// Comprehensive live-vs-local comparison test.
|
||||
// Fetches from a remote API (default: webinstall.dev) and compares against
|
||||
// local cache-only output to catch regressions.
|
||||
//
|
||||
// Usage:
|
||||
// node _webi/test-live-compare.js # compare against prod
|
||||
// node _webi/test-live-compare.js --refresh # refresh golden data
|
||||
// node _webi/test-live-compare.js --base-url=https://beta.webi.sh
|
||||
// node _webi/test-live-compare.js --all # all cached pkgs
|
||||
// node _webi/test-live-compare.js --all --tsv # TSV output
|
||||
// node _webi/test-live-compare.js --base-url=https://webi.sh \
|
||||
// --cand-url=https://beta.webi.sh # remote-vs-remote
|
||||
// # (Test 4 only;
|
||||
// # no local cache
|
||||
// # needed)
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Os = require('node:os');
|
||||
let Path = require('node:path');
|
||||
let Https = require('node:https');
|
||||
|
||||
let Releases = require('./transform-releases.js');
|
||||
let InstallerServer = require('./serve-installer.js');
|
||||
let Builds = require('./builds.js');
|
||||
|
||||
let TESTDATA_DIR = Path.join(__dirname, 'testdata');
|
||||
let CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
|
||||
let REFRESH = process.argv.includes('--refresh');
|
||||
let ALL_PKGS = process.argv.includes('--all');
|
||||
let TSV = process.argv.includes('--tsv');
|
||||
|
||||
let BASE_URL = 'https://webinstall.dev';
|
||||
let CAND_URL = '';
|
||||
for (let arg of process.argv) {
|
||||
if (arg.startsWith('--base-url=')) {
|
||||
BASE_URL = arg.slice('--base-url='.length).replace(/\/+$/, '');
|
||||
} else if (arg.startsWith('--cand-url=')) {
|
||||
CAND_URL = arg.slice('--cand-url='.length).replace(/\/+$/, '');
|
||||
}
|
||||
}
|
||||
|
||||
// Packages to test — mix of Go-built, Rust-built, and C/C++ projects
|
||||
let TEST_PKGS = ['bat', 'go', 'node', 'rg', 'jq', 'caddy'];
|
||||
|
||||
async function listCachedPkgs() {
|
||||
let entries;
|
||||
try {
|
||||
entries = await Fs.readdir(CACHE_DIR);
|
||||
} catch (e) {
|
||||
console.error(`No cache directory: ${CACHE_DIR}`);
|
||||
return [];
|
||||
}
|
||||
let pkgs = entries
|
||||
.filter(function (n) { return n.endsWith('.json'); })
|
||||
.map(function (n) { return n.slice(0, -5); })
|
||||
.sort();
|
||||
return pkgs;
|
||||
}
|
||||
|
||||
// OS/arch combos for filtered release API tests
|
||||
let RELEASE_API_CASES = [
|
||||
{ os: 'macos', arch: 'amd64' },
|
||||
{ os: 'macos', arch: 'arm64' },
|
||||
{ os: 'linux', arch: 'amd64' },
|
||||
{ os: 'windows', arch: 'amd64' },
|
||||
];
|
||||
|
||||
// Older / channel-specific version specs that map to webi <pkg>@<spec>
|
||||
// invocations. Each case exercises a code path that the unfiltered
|
||||
// "@stable" sweep above would never hit:
|
||||
// - LTS filter (lts=true)
|
||||
// - Channel filter (channel=beta)
|
||||
// - Major-series prefix (ver=20 → /^20\b/)
|
||||
// - Older minor (ver=0.18 → /^0.18\b/)
|
||||
// - Older major (ver=1.21 → /^1.21\b/) for projects with deep history
|
||||
let VERSION_SPEC_CASES = [
|
||||
{ pkg: 'node', spec: 'lts', ver: '', channel: '', lts: true, os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'node', spec: '20', ver: '20', channel: '', lts: false, os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'node', spec: 'beta', ver: '', channel: 'beta', lts: false, os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'go', spec: '1.22', ver: '1.22', channel: '', lts: false, os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'go', spec: '1.21', ver: '1.21', channel: '', lts: false, os: 'macos', arch: 'arm64' },
|
||||
{ pkg: 'bat', spec: '0.20', ver: '0.20', channel: '', lts: false, os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'bat', spec: '0.18', ver: '0.18', channel: '', lts: false, os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'rg', spec: '13', ver: '13', channel: '', lts: false, os: 'linux', arch: 'amd64' },
|
||||
{ pkg: 'caddy', spec: '2.7', ver: '2.7', channel: '', lts: false, os: 'linux', arch: 'amd64' },
|
||||
];
|
||||
|
||||
// UA strings for installer resolution tests
|
||||
let INSTALLER_CASES = [
|
||||
{ label: 'macOS arm64', ua: 'aarch64/unknown Darwin/24.2.0 libc' },
|
||||
{ label: 'macOS amd64', ua: 'x86_64/unknown Darwin/23.0.0 libc' },
|
||||
{ label: 'Linux musl', ua: 'x86_64/unknown Linux/5.15.0 musl' },
|
||||
{ label: 'Windows amd64', ua: 'x86_64/unknown Windows/10.0.19041 msvc' },
|
||||
];
|
||||
|
||||
// Known differences between Go cache and production (not regressions)
|
||||
// Extensions that the Go cache correctly excludes (non-installable formats)
|
||||
// OR that the Go cache includes but shouldn't (man pages, etc.)
|
||||
let KNOWN_EXT_DIFFS = new Set([
|
||||
'deb', 'rpm', 'sha256', 'sig', 'pem', 'sbom', 'txt',
|
||||
'1', '2', '3', '4', '5', '6', '7', '8', // man page extensions
|
||||
]);
|
||||
|
||||
function httpsGet(url) {
|
||||
return new Promise(function (resolve, reject) {
|
||||
Https.get(url, function (res) {
|
||||
let data = '';
|
||||
res.on('data', function (chunk) {
|
||||
data += chunk;
|
||||
});
|
||||
res.on('end', function () {
|
||||
if (res.statusCode !== 200) {
|
||||
reject(new Error(`HTTP ${res.statusCode}: ${url}`));
|
||||
return;
|
||||
}
|
||||
resolve(data);
|
||||
});
|
||||
}).on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
async function fetchLiveReleases(pkg, os, arch, limit) {
|
||||
let url = `${BASE_URL}/api/releases/${pkg}@stable.json?limit=${limit || 100}`;
|
||||
if (os) {
|
||||
url += `&os=${os}`;
|
||||
}
|
||||
if (arch) {
|
||||
url += `&arch=${arch}`;
|
||||
}
|
||||
let json = await httpsGet(url);
|
||||
return JSON.parse(json);
|
||||
}
|
||||
|
||||
async function fetchAtSpec(baseUrl, pkg, spec, os, arch) {
|
||||
let url = `${baseUrl}/api/releases/${pkg}@${spec}.json?limit=1`;
|
||||
if (os) {
|
||||
url += `&os=${os}`;
|
||||
}
|
||||
if (arch) {
|
||||
url += `&arch=${arch}`;
|
||||
}
|
||||
let json = await httpsGet(url);
|
||||
return JSON.parse(json);
|
||||
}
|
||||
|
||||
async function fetchLiveInstaller(pkg, ua) {
|
||||
return new Promise(function (resolve, reject) {
|
||||
let url = `${BASE_URL}/${pkg}@stable.sh`;
|
||||
let opts = {
|
||||
headers: { 'User-Agent': ua },
|
||||
};
|
||||
Https.get(url, opts, function (res) {
|
||||
// Follow redirects
|
||||
if (res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
|
||||
let redir = res.headers.location;
|
||||
if (redir.startsWith('/')) {
|
||||
redir = BASE_URL + redir;
|
||||
}
|
||||
Https.get(redir, opts, function (res2) {
|
||||
let data = '';
|
||||
res2.on('data', function (chunk) {
|
||||
data += chunk;
|
||||
});
|
||||
res2.on('end', function () {
|
||||
resolve(data);
|
||||
});
|
||||
}).on('error', reject);
|
||||
return;
|
||||
}
|
||||
let data = '';
|
||||
res.on('data', function (chunk) {
|
||||
data += chunk;
|
||||
});
|
||||
res.on('end', function () {
|
||||
resolve(data);
|
||||
});
|
||||
}).on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
function parseInstallerVars(scriptText) {
|
||||
let vars = {};
|
||||
// Match both `export WEBI_FOO='...'` and `WEBI_FOO='...'`
|
||||
let names = ['WEBI_PKG_URL', 'WEBI_VERSION', 'WEBI_EXT', 'WEBI_OS', 'WEBI_ARCH', 'PKG_NAME'];
|
||||
for (let name of names) {
|
||||
let re = new RegExp("^(?:export\\s+)?" + name + "='([^']*)'", 'm');
|
||||
let m = scriptText.match(re);
|
||||
if (m) {
|
||||
vars[name] = m[1];
|
||||
}
|
||||
}
|
||||
return vars;
|
||||
}
|
||||
|
||||
async function saveGolden(name, data) {
|
||||
await Fs.mkdir(TESTDATA_DIR, { recursive: true });
|
||||
let file = Path.join(TESTDATA_DIR, name);
|
||||
await Fs.writeFile(file, JSON.stringify(data), 'utf8');
|
||||
}
|
||||
|
||||
async function loadGolden(name) {
|
||||
let file = Path.join(TESTDATA_DIR, name);
|
||||
try {
|
||||
let json = await Fs.readFile(file, 'utf8');
|
||||
return JSON.parse(json);
|
||||
} catch (e) {
|
||||
if (e.code === 'ENOENT') {
|
||||
return null;
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
async function main() {
|
||||
let passes = 0;
|
||||
let failures = 0;
|
||||
let skips = 0;
|
||||
let knowns = 0;
|
||||
|
||||
console.log('Initializing build cache...');
|
||||
await Builds.init();
|
||||
console.log('');
|
||||
|
||||
// ================================================================
|
||||
// Test 1: Release API — unfiltered
|
||||
// ================================================================
|
||||
console.log('=== Test 1: Unfiltered /api/releases/{pkg}.json ===');
|
||||
console.log('');
|
||||
|
||||
for (let pkg of TEST_PKGS) {
|
||||
let goldenName = `live_${pkg}.json`;
|
||||
let liveReleases;
|
||||
|
||||
if (REFRESH) {
|
||||
try {
|
||||
liveReleases = await fetchLiveReleases(pkg);
|
||||
await saveGolden(goldenName, liveReleases);
|
||||
console.log(` refreshed ${goldenName}`);
|
||||
} catch (e) {
|
||||
console.log(` SKIP ${pkg}: fetch error: ${e.message}`);
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
liveReleases = await loadGolden(goldenName);
|
||||
if (!liveReleases) {
|
||||
console.log(` SKIP ${pkg}: no golden data (run with --refresh)`);
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
let localResult = await Releases.getReleases({
|
||||
pkg: pkg,
|
||||
ver: '',
|
||||
os: '',
|
||||
arch: '',
|
||||
libc: '',
|
||||
lts: false,
|
||||
channel: '',
|
||||
formats: [],
|
||||
limit: 100,
|
||||
});
|
||||
let localReleases = localResult.releases;
|
||||
|
||||
// Compare OS vocabulary — check that local has the core OSes that live has
|
||||
let liveOses = [...new Set(liveReleases.map(function (r) { return r.os; }))].sort();
|
||||
let localOses = [...new Set(localReleases.map(function (r) { return r.os; }))].sort();
|
||||
let coreOses = ['linux', 'macos', 'windows'];
|
||||
let liveCore = liveOses.filter(function (o) { return coreOses.includes(o); }).sort();
|
||||
let localCore = localOses.filter(function (o) { return coreOses.includes(o); }).sort();
|
||||
// Local should have at least the core OSes that live has
|
||||
let missingCore = liveCore.filter(function (o) { return !localCore.includes(o); });
|
||||
if (missingCore.length > 0) {
|
||||
console.log(` FAIL ${pkg} OS: missing core OSes: ${JSON.stringify(missingCore)}`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${pkg} OS: ${JSON.stringify(localCore)}`);
|
||||
passes++;
|
||||
}
|
||||
|
||||
// Compare ext vocabulary (excluding known non-installable formats)
|
||||
let liveExts = [...new Set(liveReleases.map(function (r) { return r.ext; }))].sort();
|
||||
let localExts = [...new Set(localReleases.map(function (r) { return r.ext; }))].sort();
|
||||
let liveExtsFiltered = liveExts.filter(function (e) { return !KNOWN_EXT_DIFFS.has(e); });
|
||||
let localExtsFiltered = localExts.filter(function (e) { return !KNOWN_EXT_DIFFS.has(e); });
|
||||
let extMatch = true;
|
||||
for (let ext of localExtsFiltered) {
|
||||
if (!liveExtsFiltered.includes(ext)) {
|
||||
// Local has a real ext that live doesn't — may be due to limit or sampling
|
||||
// Only fail if it's clearly wrong (not a standard installable format)
|
||||
let installable = ['tar.gz', 'tar.xz', 'tar.bz2', 'zip', 'exe', 'dmg', 'pkg', 'msi', '7z', 'xz'];
|
||||
if (!installable.includes(ext)) {
|
||||
console.log(` FAIL ${pkg} ext: local has unexpected '${ext}'`);
|
||||
failures++;
|
||||
extMatch = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (extMatch) {
|
||||
console.log(` PASS ${pkg} ext: ${JSON.stringify(localExtsFiltered)}`);
|
||||
passes++;
|
||||
}
|
||||
|
||||
// Version format — no 'v' prefix
|
||||
let hasVPrefix = localReleases.some(function (r) {
|
||||
return r.version && r.version.startsWith('v');
|
||||
});
|
||||
if (hasVPrefix) {
|
||||
console.log(` FAIL ${pkg}: versions have 'v' prefix`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${pkg}: no 'v' prefix`);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 2: Release API — filtered by OS/arch
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 2: Filtered /api/releases/{pkg}@stable.json?os=...&arch=... ===');
|
||||
console.log('');
|
||||
|
||||
for (let pkg of TEST_PKGS) {
|
||||
for (let tc of RELEASE_API_CASES) {
|
||||
let goldenName = `live_${pkg}_os_${tc.os}_arch_${tc.arch}.json`;
|
||||
let liveReleases;
|
||||
|
||||
if (REFRESH) {
|
||||
try {
|
||||
liveReleases = await fetchLiveReleases(pkg, tc.os, tc.arch, 1);
|
||||
await saveGolden(goldenName, liveReleases);
|
||||
} catch (e) {
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
liveReleases = await loadGolden(goldenName);
|
||||
if (!liveReleases) {
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
let liveFirst = liveReleases[0];
|
||||
if (!liveFirst || liveFirst.channel === 'error') {
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let localResult = await Releases.getReleases({
|
||||
pkg: pkg,
|
||||
ver: '',
|
||||
os: tc.os,
|
||||
arch: tc.arch,
|
||||
libc: '',
|
||||
lts: false,
|
||||
channel: 'stable',
|
||||
formats: [],
|
||||
limit: 1,
|
||||
});
|
||||
let localFirst = localResult.releases[0];
|
||||
|
||||
if (!localFirst || localFirst.channel === 'error') {
|
||||
console.log(` FAIL ${pkg} ${tc.os}/${tc.arch}: local returned error/empty`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let diffs = [];
|
||||
// Compare os, arch, ext (skip version/download since cache age may differ)
|
||||
if (liveFirst.os !== localFirst.os) {
|
||||
diffs.push(`os: live=${liveFirst.os} local=${localFirst.os}`);
|
||||
}
|
||||
if (liveFirst.arch !== localFirst.arch) {
|
||||
diffs.push(`arch: live=${liveFirst.arch} local=${localFirst.arch}`);
|
||||
}
|
||||
if (liveFirst.ext !== localFirst.ext) {
|
||||
if (KNOWN_EXT_DIFFS.has(liveFirst.ext)) {
|
||||
// Live returns a non-installable format (deb, pem, etc.) — known
|
||||
console.log(` KNOWN ${pkg} ${tc.os}/${tc.arch}: live ext '${liveFirst.ext}' excluded by Go cache, local='${localFirst.ext}'`);
|
||||
knowns++;
|
||||
continue;
|
||||
}
|
||||
diffs.push(`ext: live=${liveFirst.ext} local=${localFirst.ext}`);
|
||||
}
|
||||
|
||||
if (diffs.length > 0) {
|
||||
console.log(` FAIL ${pkg} ${tc.os}/${tc.arch}: ${diffs.join(', ')}`);
|
||||
failures++;
|
||||
} else {
|
||||
console.log(` PASS ${pkg} ${tc.os}/${tc.arch}: v${localFirst.version} .${localFirst.ext}`);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 3: Installer resolution — compare rendered script vars
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log('=== Test 3: Installer script variables (local serveInstaller vs live) ===');
|
||||
console.log('');
|
||||
|
||||
for (let pkg of ['bat', 'go', 'rg']) {
|
||||
for (let tc of INSTALLER_CASES) {
|
||||
// Get local result
|
||||
let localVars;
|
||||
try {
|
||||
let script = await InstallerServer.serveInstaller(
|
||||
'https://webi.sh',
|
||||
tc.ua,
|
||||
pkg,
|
||||
'stable',
|
||||
'sh',
|
||||
['tar', 'exe', 'zip', 'xz', 'dmg'],
|
||||
'',
|
||||
);
|
||||
localVars = parseInstallerVars(script);
|
||||
} catch (e) {
|
||||
console.log(` ERROR ${pkg} ${tc.label}: ${e.message}`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!localVars.WEBI_PKG_URL || localVars.WEBI_PKG_URL === '') {
|
||||
// Check if this is a known issue
|
||||
if (localVars.WEBI_EXT === 'err') {
|
||||
console.log(` KNOWN ${pkg} ${tc.label}: no match (WATERFALL gap)`);
|
||||
knowns++;
|
||||
continue;
|
||||
}
|
||||
console.log(` FAIL ${pkg} ${tc.label}: empty WEBI_PKG_URL`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Verify the URL looks like a real download
|
||||
let url = localVars.WEBI_PKG_URL;
|
||||
let hasRealDomain = url.includes('github.com') ||
|
||||
url.includes('dl.google.com') ||
|
||||
url.includes('nodejs.org') ||
|
||||
url.includes('jqlang');
|
||||
if (!hasRealDomain) {
|
||||
console.log(` FAIL ${pkg} ${tc.label}: suspicious URL: ${url}`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Verify version is set and has no 'v' prefix
|
||||
if (!localVars.WEBI_VERSION || localVars.WEBI_VERSION === '0.0.0') {
|
||||
console.log(` FAIL ${pkg} ${tc.label}: bad version: ${localVars.WEBI_VERSION}`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Verify ext is a real installable format
|
||||
let ext = localVars.WEBI_EXT;
|
||||
let goodExts = ['tar.gz', 'tar.xz', 'zip', 'exe', 'dmg', 'pkg', 'msi', '7z'];
|
||||
if (!goodExts.includes(ext)) {
|
||||
console.log(` FAIL ${pkg} ${tc.label}: bad ext: ${ext}`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
console.log(` PASS ${pkg} ${tc.label}: v${localVars.WEBI_VERSION} .${ext} ${url.split('/').pop()}`);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Test 4: @version path-form parity (webi <pkg>@<spec>)
|
||||
// Exercises lts/channel/version filters that the unfiltered sweep
|
||||
// doesn't touch. Two modes:
|
||||
// - --cand-url=<url> set: remote (BASE_URL) vs remote (CAND_URL).
|
||||
// No local cache needed.
|
||||
// - --cand-url unset: remote (BASE_URL) vs in-process Releases.
|
||||
// ================================================================
|
||||
console.log('');
|
||||
if (CAND_URL) {
|
||||
console.log(`=== Test 4: @version filter parity (${BASE_URL} vs ${CAND_URL}) ===`);
|
||||
} else {
|
||||
console.log('=== Test 4: @version filter parity (remote vs local resolver) ===');
|
||||
}
|
||||
console.log('');
|
||||
|
||||
for (let tc of VERSION_SPEC_CASES) {
|
||||
let label = `${tc.pkg}@${tc.spec} ${tc.os}/${tc.arch}`;
|
||||
|
||||
let baseFirst;
|
||||
try {
|
||||
let baseRel = await fetchAtSpec(BASE_URL, tc.pkg, tc.spec, tc.os, tc.arch);
|
||||
baseFirst = baseRel[0];
|
||||
} catch (e) {
|
||||
console.log(` SKIP ${label}: base fetch error: ${e.message}`);
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
if (!baseFirst || baseFirst.channel === 'error') {
|
||||
console.log(` SKIP ${label}: base returned error/empty`);
|
||||
skips++;
|
||||
continue;
|
||||
}
|
||||
|
||||
let candFirst;
|
||||
if (CAND_URL) {
|
||||
try {
|
||||
let candRel = await fetchAtSpec(CAND_URL, tc.pkg, tc.spec, tc.os, tc.arch);
|
||||
candFirst = candRel[0];
|
||||
} catch (e) {
|
||||
console.log(` FAIL ${label}: cand fetch error: ${e.message}`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
let localResult = await Releases.getReleases({
|
||||
pkg: tc.pkg,
|
||||
ver: tc.ver,
|
||||
os: tc.os,
|
||||
arch: tc.arch,
|
||||
libc: '',
|
||||
lts: tc.lts,
|
||||
channel: tc.channel,
|
||||
formats: [],
|
||||
limit: 1,
|
||||
});
|
||||
candFirst = localResult.releases && localResult.releases[0];
|
||||
}
|
||||
if (!candFirst || candFirst.channel === 'error') {
|
||||
console.log(` FAIL ${label}: cand returned error/empty (base=${baseFirst.version})`);
|
||||
failures++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Both should satisfy the requested spec. A version diff is only a
|
||||
// real failure if cand picked something the spec excludes (e.g.
|
||||
// requested '20' but got '26'). Same prefix on both sides is just
|
||||
// cache-age skew (e.g. 1.21.13 vs 1.21.14).
|
||||
if (baseFirst.version !== candFirst.version) {
|
||||
let prefix = tc.ver;
|
||||
let candMatches = !prefix || new RegExp('^' + prefix + '\\b').test(candFirst.version);
|
||||
let baseMatches = !prefix || new RegExp('^' + prefix + '\\b').test(baseFirst.version);
|
||||
if (!candMatches && baseMatches) {
|
||||
console.log(` FAIL ${label}: cand=${candFirst.version} doesn't match spec; base=${baseFirst.version}`);
|
||||
failures++;
|
||||
} else if (candMatches && !baseMatches) {
|
||||
console.log(` PASS ${label}: cand=${candFirst.version} (base=${baseFirst.version} is wrong)`);
|
||||
passes++;
|
||||
} else {
|
||||
console.log(` PASS ${label}: cand=${candFirst.version} base=${baseFirst.version} (both match '${prefix}'; cache-age skew)`);
|
||||
passes++;
|
||||
}
|
||||
} else {
|
||||
console.log(` PASS ${label}: v${candFirst.version}`);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
|
||||
// ================================================================
|
||||
// Summary
|
||||
// ================================================================
|
||||
console.log('');
|
||||
console.log(`=== Results: ${passes} passed, ${failures} failed, ${knowns} known, ${skips} skipped ===`);
|
||||
if (failures > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
202
_webi/test-live-installer-diff.js
Normal file
202
_webi/test-live-installer-diff.js
Normal file
@@ -0,0 +1,202 @@
|
||||
'use strict';
|
||||
|
||||
// Direct side-by-side comparison: fetch the live installer script from
|
||||
// a remote host with the same UA, and compare WEBI_* variables against
|
||||
// what our local serveInstaller() produces.
|
||||
//
|
||||
// This is the most direct behavioral equivalence test — if the same UA
|
||||
// produces the same WEBI_PKG_URL, WEBI_VERSION, WEBI_EXT, and WEBI_OS,
|
||||
// then the user gets the same binary.
|
||||
//
|
||||
// Usage:
|
||||
// node _webi/test-live-installer-diff.js
|
||||
// node _webi/test-live-installer-diff.js --base-url=https://beta.webi.sh
|
||||
|
||||
let Https = require('node:https');
|
||||
let InstallerServer = require('./serve-installer.js');
|
||||
let Builds = require('./builds.js');
|
||||
|
||||
let BASE_URL = 'https://webinstall.dev';
|
||||
for (let arg of process.argv) {
|
||||
if (arg.startsWith('--base-url=')) {
|
||||
BASE_URL = arg.slice('--base-url='.length).replace(/\/+$/, '');
|
||||
}
|
||||
}
|
||||
|
||||
let CASES = [
|
||||
// bat — Rust project, gnu-linked Linux builds
|
||||
{ pkg: 'bat', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'bat macOS arm64' },
|
||||
{ pkg: 'bat', ua: 'x86_64/unknown Darwin/23.0.0 libc', label: 'bat macOS amd64' },
|
||||
{ pkg: 'bat', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'bat Linux amd64' },
|
||||
{ pkg: 'bat', ua: 'x86_64/unknown Linux/5.15.0 musl', label: 'bat Linux musl' },
|
||||
{ pkg: 'bat', ua: 'x86_64/unknown Windows/10.0.19041 msvc', label: 'bat Windows amd64' },
|
||||
// go — Go project, static builds (libc='none')
|
||||
{ pkg: 'go', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'go macOS arm64' },
|
||||
{ pkg: 'go', ua: 'x86_64/unknown Darwin/23.0.0 libc', label: 'go macOS amd64' },
|
||||
{ pkg: 'go', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'go Linux amd64' },
|
||||
{ pkg: 'go', ua: 'x86_64/unknown Windows/10.0.19041 msvc', label: 'go Windows amd64' },
|
||||
// node — C++ project, gnu-linked Linux builds, separate musl build
|
||||
{ pkg: 'node', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'node macOS arm64' },
|
||||
{ pkg: 'node', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'node Linux amd64',
|
||||
known: 'live fails (WATERFALL gap), local correctly resolves gnu build' },
|
||||
{ pkg: 'node', ua: 'x86_64/unknown Linux/5.15.0 musl', label: 'node Linux musl' },
|
||||
// rg — Rust project, gnu-linked Linux builds
|
||||
{ pkg: 'rg', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'rg macOS arm64' },
|
||||
{ pkg: 'rg', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'rg Linux amd64' },
|
||||
{ pkg: 'rg', ua: 'x86_64/unknown Linux/5.15.0 musl', label: 'rg Linux musl' },
|
||||
{ pkg: 'rg', ua: 'x86_64/unknown Windows/10.0.19041 msvc', label: 'rg Windows amd64' },
|
||||
// jq — C project, had .git source URLs in old releases
|
||||
{ pkg: 'jq', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'jq macOS arm64' },
|
||||
{ pkg: 'jq', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'jq Linux amd64' },
|
||||
{ pkg: 'jq', ua: 'x86_64/unknown Windows/10.0.19041 msvc', label: 'jq Windows amd64' },
|
||||
// caddy — Go project, had .git source URLs in old releases
|
||||
{ pkg: 'caddy', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'caddy macOS arm64' },
|
||||
{ pkg: 'caddy', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'caddy Linux amd64' },
|
||||
{ pkg: 'caddy', ua: 'x86_64/unknown Windows/10.0.19041 msvc', label: 'caddy Windows amd64' },
|
||||
// Additional packages for broader coverage
|
||||
{ pkg: 'shellcheck', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'shellcheck macOS arm64' },
|
||||
{ pkg: 'shellcheck', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'shellcheck Linux amd64' },
|
||||
{ pkg: 'shfmt', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'shfmt macOS arm64' },
|
||||
{ pkg: 'shfmt', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'shfmt Linux amd64' },
|
||||
{ pkg: 'fd', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'fd macOS arm64' },
|
||||
{ pkg: 'fd', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'fd Linux amd64' },
|
||||
{ pkg: 'hugo', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'hugo macOS arm64',
|
||||
known: 'classifier rejects darwin-universal as x86_64!=universal2' },
|
||||
{ pkg: 'hugo', ua: 'x86_64/unknown Linux/5.15.0 libc', label: 'hugo Linux amd64' },
|
||||
// Alias tests — these should resolve to the real package
|
||||
{ pkg: 'golang', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'golang alias macOS arm64' },
|
||||
{ pkg: 'ripgrep', ua: 'aarch64/unknown Darwin/24.2.0 libc', label: 'ripgrep alias macOS arm64' },
|
||||
];
|
||||
|
||||
// Variables that must match between live and local for the install to work
|
||||
let CRITICAL_VARS = ['PKG_NAME', 'WEBI_OS', 'WEBI_ARCH', 'WEBI_EXT'];
|
||||
// Variables where version differences are OK (cache age)
|
||||
let VERSION_VARS = ['WEBI_VERSION', 'WEBI_PKG_URL', 'WEBI_PKG_FILE'];
|
||||
|
||||
function fetchLiveInstaller(pkg, ua) {
|
||||
return new Promise(function (resolve, reject) {
|
||||
let url = `${BASE_URL}/api/installers/${pkg}@stable.sh`;
|
||||
let opts = { headers: { 'User-Agent': ua } };
|
||||
Https.get(url, opts, function (res) {
|
||||
let data = '';
|
||||
res.on('data', function (chunk) { data += chunk; });
|
||||
res.on('end', function () { resolve(data); });
|
||||
}).on('error', reject);
|
||||
});
|
||||
}
|
||||
|
||||
function parseVars(script) {
|
||||
let vars = {};
|
||||
// Match: PKG_NAME='bat' or WEBI_VERSION='1.2.3' (with or without export)
|
||||
let re = /^(?:export\s+)?(WEBI_\w+|PKG_NAME)='([^']*)'/gm;
|
||||
let m;
|
||||
while ((m = re.exec(script)) !== null) {
|
||||
vars[m[1]] = m[2];
|
||||
}
|
||||
return vars;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
let passes = 0;
|
||||
let failures = 0;
|
||||
let knowns = 0;
|
||||
let errors = 0;
|
||||
|
||||
console.log('Initializing build cache...');
|
||||
await Builds.init();
|
||||
console.log('');
|
||||
|
||||
console.log('=== Live vs Local Installer Diff ===');
|
||||
console.log('');
|
||||
|
||||
for (let tc of CASES) {
|
||||
// Fetch live
|
||||
let liveScript;
|
||||
try {
|
||||
liveScript = await fetchLiveInstaller(tc.pkg, tc.ua);
|
||||
} catch (e) {
|
||||
console.log(` SKIP ${tc.label}: fetch error: ${e.message}`);
|
||||
continue;
|
||||
}
|
||||
let liveVars = parseVars(liveScript);
|
||||
|
||||
if (!liveVars.WEBI_PKG_URL) {
|
||||
console.log(` SKIP ${tc.label}: live returned no WEBI_PKG_URL`);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Render local
|
||||
let localScript;
|
||||
try {
|
||||
localScript = await InstallerServer.serveInstaller(
|
||||
BASE_URL,
|
||||
tc.ua,
|
||||
tc.pkg,
|
||||
'stable',
|
||||
'sh',
|
||||
['tar', 'exe', 'zip', 'xz', 'dmg'],
|
||||
'',
|
||||
);
|
||||
} catch (e) {
|
||||
console.log(` ERROR ${tc.label}: local error: ${e.message}`);
|
||||
errors++;
|
||||
continue;
|
||||
}
|
||||
let localVars = parseVars(localScript);
|
||||
|
||||
if (tc.known) {
|
||||
let localExt = localVars.WEBI_EXT || 'err';
|
||||
let liveExt = liveVars.WEBI_EXT || '?';
|
||||
if (localExt !== liveExt) {
|
||||
console.log(` KNOWN ${tc.label}: ${tc.known} (live=${liveExt} local=${localExt})`);
|
||||
knowns++;
|
||||
} else {
|
||||
console.log(` PASS ${tc.label}: known issue resolved! v${localVars.WEBI_VERSION} .${localExt}`);
|
||||
passes++;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!localVars.WEBI_PKG_URL || localVars.WEBI_EXT === 'err') {
|
||||
console.log(` KNOWN ${tc.label}: local failed to resolve (live=${liveVars.WEBI_EXT})`);
|
||||
knowns++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Compare critical vars
|
||||
let diffs = [];
|
||||
for (let v of CRITICAL_VARS) {
|
||||
let liveVal = liveVars[v] || '';
|
||||
let localVal = localVars[v] || '';
|
||||
if (liveVal !== localVal) {
|
||||
diffs.push(`${v}: live='${liveVal}' local='${localVal}'`);
|
||||
}
|
||||
}
|
||||
|
||||
// Log version info (informational, not failure)
|
||||
let versionNote = '';
|
||||
if (liveVars.WEBI_VERSION !== localVars.WEBI_VERSION) {
|
||||
versionNote = ` (version: live=${liveVars.WEBI_VERSION} local=${localVars.WEBI_VERSION})`;
|
||||
}
|
||||
|
||||
if (diffs.length > 0) {
|
||||
console.log(` FAIL ${tc.label}: ${diffs.join(', ')}${versionNote}`);
|
||||
failures++;
|
||||
} else {
|
||||
let file = (localVars.WEBI_PKG_URL || '').split('/').pop();
|
||||
console.log(` PASS ${tc.label}: v${localVars.WEBI_VERSION} .${localVars.WEBI_EXT} ${file}${versionNote}`);
|
||||
passes++;
|
||||
}
|
||||
}
|
||||
|
||||
console.log('');
|
||||
console.log(`=== Results: ${passes} passed, ${failures} failed, ${knowns} known, ${errors} errors ===`);
|
||||
if (failures > 0 || errors > 0) {
|
||||
process.exit(1);
|
||||
}
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error(err.stack);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -38,9 +38,8 @@ function getOs(ua) {
|
||||
// It's the year of the Linux Desktop!
|
||||
// See also http://www.mslinux.org/
|
||||
// 'linux' must be tested before 'Microsoft' because WSL
|
||||
// (TODO: does this affect cygwin / msysgit?)
|
||||
return 'linux';
|
||||
} else if (/^ms$|Microsoft|Windows|win32|win|PowerShell/i.test(ua)) {
|
||||
} else if (/^ms$|Microsoft|Windows|win32|win|PowerShell|CYGWIN|MINGW/i.test(ua)) {
|
||||
// 'win' must be tested after 'darwin'
|
||||
return 'windows';
|
||||
} else if (/Linux|curl|wget/i.test(ua)) {
|
||||
|
||||
@@ -35,6 +35,7 @@ import (
|
||||
|
||||
"github.com/joho/godotenv"
|
||||
"github.com/webinstall/webi-installers/internal/classifypkg"
|
||||
"github.com/webinstall/webi-installers/internal/httpclient"
|
||||
"github.com/webinstall/webi-installers/internal/installerconf"
|
||||
"github.com/webinstall/webi-installers/internal/rawcache"
|
||||
"github.com/webinstall/webi-installers/internal/releases/chromedist"
|
||||
@@ -77,7 +78,6 @@ type MainConfig struct {
|
||||
envFile string
|
||||
confDir string
|
||||
cacheDir string
|
||||
|
||||
rawDir string
|
||||
token string
|
||||
once bool
|
||||
@@ -167,10 +167,10 @@ func main() {
|
||||
auth = &githubish.Auth{Token: cfg.token}
|
||||
}
|
||||
|
||||
client := &http.Client{Timeout: 30 * time.Second}
|
||||
client := httpclient.New()
|
||||
if cfg.pageDelay > 0 {
|
||||
client.Transport = &delayTransport{
|
||||
base: http.DefaultTransport,
|
||||
base: client.Transport,
|
||||
delay: cfg.pageDelay,
|
||||
}
|
||||
}
|
||||
@@ -298,7 +298,6 @@ func registerFlags(fs *flag.FlagSet, cfg *MainConfig) {
|
||||
fs.StringVar(&cfg.envFile, "envfile", "", "path to .env file to load before running")
|
||||
fs.StringVar(&cfg.confDir, "conf", ".", "root directory containing {pkg}/releases.conf files")
|
||||
fs.StringVar(&cfg.cacheDir, "legacy", "~/.cache/webi/legacy", "legacy cache directory (fsstore root)")
|
||||
|
||||
fs.StringVar(&cfg.rawDir, "raw", "~/.cache/webi/raw", "raw cache directory for upstream responses")
|
||||
fs.StringVar(&cfg.token, "token", "", "GitHub API token (or set $GITHUB_TOKEN)")
|
||||
fs.BoolVar(&cfg.once, "once", false, "run once then exit (no periodic refresh)")
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
source = github
|
||||
source = ffmpegdist
|
||||
github_releases = eugeneware/ffmpeg-static
|
||||
asset_filter = ffmpeg
|
||||
version_prefix = b
|
||||
arch_map = x64:x86_64 ia32:x86 arm64:aarch64 arm:armv7
|
||||
os_map = win32:windows
|
||||
|
||||
@@ -166,6 +166,8 @@ func classifySource(pkg string, conf *installerconf.Conf, d *rawcache.Dir) ([]st
|
||||
return classifyMariaDBDist(d)
|
||||
case "zigdist":
|
||||
return classifyZigDist(d)
|
||||
case "ffmpegdist":
|
||||
return classifyFFmpegDist(d)
|
||||
default:
|
||||
return nil, nil
|
||||
}
|
||||
@@ -426,16 +428,6 @@ func classifyGitHub(pkg string, conf *installerconf.Conf, d *rawcache.Dir) ([]st
|
||||
|
||||
r := classify.Filename(a.Name)
|
||||
|
||||
// Apply conf-level OS/arch overrides for non-standard naming.
|
||||
osStr := string(r.OS)
|
||||
if mapped, ok := conf.OSMap[osStr]; ok {
|
||||
osStr = mapped
|
||||
}
|
||||
archStr := string(r.Arch)
|
||||
if mapped, ok := conf.ArchMap[archStr]; ok {
|
||||
archStr = mapped
|
||||
}
|
||||
|
||||
// Normalize .tgz → .tar.gz in the display filename.
|
||||
// The download URL still points to the real file.
|
||||
name := a.Name
|
||||
@@ -449,7 +441,7 @@ func classifyGitHub(pkg string, conf *installerconf.Conf, d *rawcache.Dir) ([]st
|
||||
libc = buildmeta.LibcNone
|
||||
}
|
||||
// Windows gnu (MinGW) is self-contained — no runtime deps.
|
||||
if buildmeta.OS(osStr) == buildmeta.OSWindows && libc == buildmeta.LibcGNU {
|
||||
if r.OS == buildmeta.OSWindows && libc == buildmeta.LibcGNU {
|
||||
libc = buildmeta.LibcNone
|
||||
}
|
||||
|
||||
@@ -457,8 +449,8 @@ func classifyGitHub(pkg string, conf *installerconf.Conf, d *rawcache.Dir) ([]st
|
||||
Filename: name,
|
||||
Version: version,
|
||||
Channel: channel,
|
||||
OS: osStr,
|
||||
Arch: archStr,
|
||||
OS: string(r.OS),
|
||||
Arch: string(r.Arch),
|
||||
Libc: string(libc),
|
||||
Format: string(r.Format),
|
||||
Download: a.BrowserDownloadURL,
|
||||
@@ -474,6 +466,86 @@ func classifyGitHub(pkg string, conf *installerconf.Conf, d *rawcache.Dir) ([]st
|
||||
return assets, nil
|
||||
}
|
||||
|
||||
var ffmpegOSMap = map[string]string{
|
||||
"linux": "linux",
|
||||
"darwin": "darwin",
|
||||
"win32": "windows",
|
||||
}
|
||||
|
||||
var ffmpegArchMap = map[string]string{
|
||||
"x64": "x86_64",
|
||||
"ia32": "x86",
|
||||
"arm64": "aarch64",
|
||||
"arm": "armv7",
|
||||
}
|
||||
|
||||
// classifyFFmpegDist handles eugeneware/ffmpeg-static releases.
|
||||
// Upstream uses non-standard names (x64, ia32, win32, arm) and ships both
|
||||
// bare binaries and .gz-compressed copies. Only bare binaries are kept —
|
||||
// the install template has no handler for single-file .gz extraction.
|
||||
func classifyFFmpegDist(d *rawcache.Dir) ([]storage.Asset, error) {
|
||||
releases, err := ReadAllRaw(d)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var assets []storage.Asset
|
||||
for _, data := range releases {
|
||||
var rel ghRelease
|
||||
if err := json.Unmarshal(data, &rel); err != nil {
|
||||
continue
|
||||
}
|
||||
if rel.Draft {
|
||||
continue
|
||||
}
|
||||
|
||||
version := strings.TrimPrefix(rel.TagName, "b")
|
||||
|
||||
channel := "stable"
|
||||
if rel.Prerelease {
|
||||
channel = "beta"
|
||||
}
|
||||
|
||||
date := ""
|
||||
if len(rel.PublishedAt) >= 10 {
|
||||
date = rel.PublishedAt[:10]
|
||||
}
|
||||
|
||||
for _, a := range rel.Assets {
|
||||
if strings.Contains(a.Name, ".") {
|
||||
continue
|
||||
}
|
||||
if !strings.HasPrefix(a.Name, "ffmpeg-") {
|
||||
continue
|
||||
}
|
||||
|
||||
parts := strings.SplitN(a.Name, "-", 3)
|
||||
if len(parts) != 3 {
|
||||
continue
|
||||
}
|
||||
|
||||
os, osOK := ffmpegOSMap[parts[1]]
|
||||
arch, archOK := ffmpegArchMap[parts[2]]
|
||||
if !osOK || !archOK {
|
||||
continue
|
||||
}
|
||||
|
||||
assets = append(assets, storage.Asset{
|
||||
Filename: a.Name,
|
||||
Version: version,
|
||||
Channel: channel,
|
||||
OS: os,
|
||||
Arch: arch,
|
||||
Format: "",
|
||||
Download: a.BrowserDownloadURL,
|
||||
Date: date,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
return assets, nil
|
||||
}
|
||||
|
||||
// classifyServiceman handles serviceman's dual-repo layout: binary releases
|
||||
// from therootcompany/serviceman (≤v0.8.x) and source-only releases from
|
||||
// bnnanet/serviceman (v0.9.x+). Emits binary assets where available, plus
|
||||
|
||||
@@ -96,16 +96,6 @@ type Conf struct {
|
||||
// kubectx/kubens) to select only the relevant assets.
|
||||
AssetFilter string
|
||||
|
||||
// ArchMap translates non-standard arch strings in asset filenames to
|
||||
// canonical webi arch names. Format: "upstream:canonical" pairs,
|
||||
// whitespace-delimited. Example: "x64:x86_64 ia32:x86 arm:armv7"
|
||||
ArchMap map[string]string
|
||||
|
||||
// OSMap translates non-standard OS strings in asset filenames to
|
||||
// canonical webi OS names. Format: "upstream:canonical" pairs,
|
||||
// whitespace-delimited. Example: "win32:windows"
|
||||
OSMap map[string]string
|
||||
|
||||
// Variants documents known build variant names for this package.
|
||||
// Whitespace-delimited. This is a human-readable cue — actual
|
||||
// variant detection logic lives in Go code per-package.
|
||||
@@ -253,14 +243,6 @@ func Read(path string) (*Conf, error) {
|
||||
}
|
||||
|
||||
c.AssetFilter = raw["asset_filter"]
|
||||
|
||||
if v := raw["arch_map"]; v != "" {
|
||||
c.ArchMap = parseKVMap(v)
|
||||
}
|
||||
if v := raw["os_map"]; v != "" {
|
||||
c.OSMap = parseKVMap(v)
|
||||
}
|
||||
|
||||
c.OS = raw["os"]
|
||||
c.AliasOf = raw["alias_of"]
|
||||
|
||||
@@ -287,8 +269,6 @@ func Read(path string) (*Conf, error) {
|
||||
"exclude": true,
|
||||
"asset_exclude": true,
|
||||
"asset_filter": true,
|
||||
"arch_map": true,
|
||||
"os_map": true,
|
||||
"os": true,
|
||||
"variants": true,
|
||||
"alias_of": true,
|
||||
@@ -304,15 +284,3 @@ func Read(path string) (*Conf, error) {
|
||||
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// parseKVMap parses whitespace-delimited "key:value" pairs into a map.
|
||||
func parseKVMap(s string) map[string]string {
|
||||
m := make(map[string]string)
|
||||
for _, pair := range strings.Fields(s) {
|
||||
k, v, ok := strings.Cut(pair, ":")
|
||||
if ok {
|
||||
m[k] = v
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
@@ -1,104 +0,0 @@
|
||||
// Package platlatest tracks the newest release version per build target.
|
||||
//
|
||||
// After classification determines which OS/arch/libc targets a release
|
||||
// covers, this package records the latest version for each target. This
|
||||
// handles the common case where Windows or macOS releases lag behind
|
||||
// Linux by several versions.
|
||||
//
|
||||
// Storage is a single JSON file per package:
|
||||
//
|
||||
// {
|
||||
// "linux-x86_64-gnu": "v0.145.0",
|
||||
// "darwin-aarch64-none": "v0.144.1",
|
||||
// "windows-x86_64-msvc": "v0.143.0"
|
||||
// }
|
||||
package platlatest
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
)
|
||||
|
||||
// Index tracks the latest version for each build target of a package.
|
||||
type Index struct {
|
||||
mu sync.RWMutex
|
||||
path string
|
||||
m map[string]string // triplet → version
|
||||
}
|
||||
|
||||
// Open loads or creates a per-platform latest index at the given path.
|
||||
func Open(path string) (*Index, error) {
|
||||
idx := &Index{
|
||||
path: path,
|
||||
m: make(map[string]string),
|
||||
}
|
||||
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return idx, nil
|
||||
}
|
||||
return nil, fmt.Errorf("platlatest: read %s: %w", path, err)
|
||||
}
|
||||
|
||||
if err := json.Unmarshal(data, &idx.m); err != nil {
|
||||
return nil, fmt.Errorf("platlatest: parse %s: %w", path, err)
|
||||
}
|
||||
return idx, nil
|
||||
}
|
||||
|
||||
// Get returns the latest version for a target, or "" if unknown.
|
||||
func (idx *Index) Get(t buildmeta.Target) string {
|
||||
idx.mu.RLock()
|
||||
defer idx.mu.RUnlock()
|
||||
return idx.m[t.Triplet()]
|
||||
}
|
||||
|
||||
// Set records a version as the latest for a target. Does not persist
|
||||
// to disk — call Save after all updates.
|
||||
func (idx *Index) Set(t buildmeta.Target, version string) {
|
||||
idx.mu.Lock()
|
||||
defer idx.mu.Unlock()
|
||||
idx.m[t.Triplet()] = version
|
||||
}
|
||||
|
||||
// All returns a copy of the full triplet→version map.
|
||||
func (idx *Index) All() map[string]string {
|
||||
idx.mu.RLock()
|
||||
defer idx.mu.RUnlock()
|
||||
out := make(map[string]string, len(idx.m))
|
||||
for k, v := range idx.m {
|
||||
out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Save persists the index to disk (atomic write).
|
||||
func (idx *Index) Save() error {
|
||||
idx.mu.RLock()
|
||||
data, err := json.MarshalIndent(idx.m, "", " ")
|
||||
idx.mu.RUnlock()
|
||||
if err != nil {
|
||||
return fmt.Errorf("platlatest: marshal: %w", err)
|
||||
}
|
||||
|
||||
dir := filepath.Dir(idx.path)
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return fmt.Errorf("platlatest: mkdir: %w", err)
|
||||
}
|
||||
|
||||
tmp := idx.path + ".tmp"
|
||||
if err := os.WriteFile(tmp, data, 0o644); err != nil {
|
||||
return fmt.Errorf("platlatest: write %s: %w", tmp, err)
|
||||
}
|
||||
if err := os.Rename(tmp, idx.path); err != nil {
|
||||
os.Remove(tmp)
|
||||
return fmt.Errorf("platlatest: rename %s: %w", idx.path, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,104 +0,0 @@
|
||||
package platlatest_test
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"testing"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
"github.com/webinstall/webi-installers/internal/platlatest"
|
||||
)
|
||||
|
||||
var (
|
||||
linuxAMD64 = buildmeta.Target{
|
||||
OS: buildmeta.OSLinux, Arch: buildmeta.ArchAMD64, Libc: buildmeta.LibcGNU,
|
||||
}
|
||||
darwinARM64 = buildmeta.Target{
|
||||
OS: buildmeta.OSDarwin, Arch: buildmeta.ArchARM64, Libc: buildmeta.LibcNone,
|
||||
}
|
||||
windowsAMD64 = buildmeta.Target{
|
||||
OS: buildmeta.OSWindows, Arch: buildmeta.ArchAMD64, Libc: buildmeta.LibcMSVC,
|
||||
}
|
||||
)
|
||||
|
||||
func TestSetAndGet(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "latest.json")
|
||||
idx, err := platlatest.Open(p)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
if got := idx.Get(linuxAMD64); got != "" {
|
||||
t.Errorf("Get before Set = %q, want empty", got)
|
||||
}
|
||||
|
||||
idx.Set(linuxAMD64, "v0.145.0")
|
||||
idx.Set(darwinARM64, "v0.144.1")
|
||||
idx.Set(windowsAMD64, "v0.143.0")
|
||||
|
||||
if got := idx.Get(linuxAMD64); got != "v0.145.0" {
|
||||
t.Errorf("linux = %q, want v0.145.0", got)
|
||||
}
|
||||
if got := idx.Get(darwinARM64); got != "v0.144.1" {
|
||||
t.Errorf("darwin = %q, want v0.144.1", got)
|
||||
}
|
||||
if got := idx.Get(windowsAMD64); got != "v0.143.0" {
|
||||
t.Errorf("windows = %q, want v0.143.0", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSaveAndReload(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "latest.json")
|
||||
|
||||
idx1, err := platlatest.Open(p)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
idx1.Set(linuxAMD64, "v0.145.0")
|
||||
idx1.Set(darwinARM64, "v0.144.1")
|
||||
if err := idx1.Save(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
// Reload from disk.
|
||||
idx2, err := platlatest.Open(p)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := idx2.Get(linuxAMD64); got != "v0.145.0" {
|
||||
t.Errorf("after reload: linux = %q, want v0.145.0", got)
|
||||
}
|
||||
if got := idx2.Get(darwinARM64); got != "v0.144.1" {
|
||||
t.Errorf("after reload: darwin = %q, want v0.144.1", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAll(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "latest.json")
|
||||
idx, err := platlatest.Open(p)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
idx.Set(linuxAMD64, "v1.0.0")
|
||||
idx.Set(darwinARM64, "v0.9.0")
|
||||
|
||||
all := idx.All()
|
||||
if len(all) != 2 {
|
||||
t.Fatalf("All() returned %d entries, want 2", len(all))
|
||||
}
|
||||
if all[linuxAMD64.Triplet()] != "v1.0.0" {
|
||||
t.Error("missing linux entry")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenNonexistent(t *testing.T) {
|
||||
p := filepath.Join(t.TempDir(), "does-not-exist.json")
|
||||
idx, err := platlatest.Open(p)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Should be empty, not nil.
|
||||
if all := idx.All(); len(all) != 0 {
|
||||
t.Errorf("new index should be empty, got %v", all)
|
||||
}
|
||||
}
|
||||
@@ -1,264 +0,0 @@
|
||||
// Package render generates installer scripts by injecting release
|
||||
// metadata into the package-install template.
|
||||
//
|
||||
// The template uses shell-style variable markers:
|
||||
//
|
||||
// #WEBI_VERSION= → WEBI_VERSION='1.2.3'
|
||||
// #export WEBI_PKG_URL= → export WEBI_PKG_URL='https://...'
|
||||
//
|
||||
// The package's install.sh is injected at the {{ installer }} marker.
|
||||
package render
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Params holds all the values to inject into the installer template.
|
||||
type Params struct {
|
||||
// Host is the base URL of the webi server (e.g. "https://webinstall.dev").
|
||||
Host string
|
||||
|
||||
// Checksum is the webi.sh bootstrap script checksum (first 8 hex chars of SHA-1).
|
||||
Checksum string
|
||||
|
||||
// Package name (e.g. "bat", "node").
|
||||
PkgName string
|
||||
|
||||
// Tag is the version selector from the URL (e.g. "20", "stable", "").
|
||||
Tag string
|
||||
|
||||
// OS, Arch, Libc are the detected platform strings.
|
||||
OS string
|
||||
Arch string
|
||||
Libc string
|
||||
|
||||
// Resolved release info.
|
||||
Version string
|
||||
Major string
|
||||
Minor string
|
||||
Patch string
|
||||
Build string
|
||||
GitTag string
|
||||
GitBranch string
|
||||
LTS string // "true" or "false"
|
||||
Channel string
|
||||
Ext string // archive extension (e.g. "tar.gz", "zip")
|
||||
Formats string // comma-separated format list
|
||||
|
||||
// Download info.
|
||||
PkgURL string // download URL
|
||||
PkgFile string // filename
|
||||
|
||||
// Releases API URL for this request.
|
||||
ReleasesURL string
|
||||
|
||||
// CSV line for WEBI_CSV.
|
||||
CSV string
|
||||
|
||||
// Package catalog info.
|
||||
PkgStable string
|
||||
PkgLatest string
|
||||
PkgOSes string // space-separated
|
||||
PkgArches string // space-separated
|
||||
PkgLibcs string // space-separated
|
||||
PkgFormats string // space-separated
|
||||
}
|
||||
|
||||
// Bash renders a complete bash installer script by injecting params
|
||||
// into the template and splicing in the package's install.sh.
|
||||
func Bash(tplPath, installersDir, pkgName string, p Params) (string, error) {
|
||||
tpl, err := os.ReadFile(tplPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("render: read template: %w", err)
|
||||
}
|
||||
|
||||
// Read the package's install.sh.
|
||||
installPath := filepath.Join(installersDir, pkgName, "install.sh")
|
||||
installSh, err := os.ReadFile(installPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("render: read %s/install.sh: %w", pkgName, err)
|
||||
}
|
||||
|
||||
text := string(tpl)
|
||||
|
||||
// Inject environment variables.
|
||||
vars := []struct {
|
||||
name string
|
||||
value string
|
||||
}{
|
||||
{"WEBI_CHECKSUM", p.Checksum},
|
||||
{"WEBI_PKG", p.PkgName + "@" + p.Tag},
|
||||
{"WEBI_HOST", p.Host},
|
||||
{"WEBI_OS", p.OS},
|
||||
{"WEBI_ARCH", p.Arch},
|
||||
{"WEBI_LIBC", p.Libc},
|
||||
{"WEBI_TAG", p.Tag},
|
||||
{"WEBI_RELEASES", p.ReleasesURL},
|
||||
{"WEBI_CSV", p.CSV},
|
||||
{"WEBI_VERSION", p.Version},
|
||||
{"WEBI_MAJOR", p.Major},
|
||||
{"WEBI_MINOR", p.Minor},
|
||||
{"WEBI_PATCH", p.Patch},
|
||||
{"WEBI_BUILD", p.Build},
|
||||
{"WEBI_GIT_BRANCH", p.GitBranch},
|
||||
{"WEBI_GIT_TAG", p.GitTag},
|
||||
{"WEBI_LTS", p.LTS},
|
||||
{"WEBI_CHANNEL", p.Channel},
|
||||
{"WEBI_EXT", p.Ext},
|
||||
{"WEBI_FORMATS", p.Formats},
|
||||
{"WEBI_PKG_URL", p.PkgURL},
|
||||
{"WEBI_PKG_PATHNAME", p.PkgFile},
|
||||
{"WEBI_PKG_FILE", p.PkgFile},
|
||||
{"PKG_NAME", p.PkgName},
|
||||
{"PKG_STABLE", p.PkgStable},
|
||||
{"PKG_LATEST", p.PkgLatest},
|
||||
{"PKG_OSES", p.PkgOSes},
|
||||
{"PKG_ARCHES", p.PkgArches},
|
||||
{"PKG_LIBCS", p.PkgLibcs},
|
||||
{"PKG_FORMATS", p.PkgFormats},
|
||||
}
|
||||
|
||||
for _, v := range vars {
|
||||
text = InjectVar(text, v.name, v.value)
|
||||
}
|
||||
|
||||
// Inject the installer script at the {{ installer }} marker.
|
||||
// The marker sits inside __init_installer() at 8-space indent.
|
||||
// Production pads every line of install.sh to match, and replaces
|
||||
// the entire line (including leading whitespace).
|
||||
padded := padScript(string(installSh), " ")
|
||||
text = replaceMarkerLine(text, "{{ installer }}", padded)
|
||||
|
||||
return text, nil
|
||||
}
|
||||
|
||||
// PowerShell renders a complete PowerShell installer script by injecting
|
||||
// params into the template and splicing in the package's install.ps1.
|
||||
func PowerShell(tplPath, installersDir, pkgName string, p Params) (string, error) {
|
||||
tpl, err := os.ReadFile(tplPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("render: read template: %w", err)
|
||||
}
|
||||
|
||||
installPath := filepath.Join(installersDir, pkgName, "install.ps1")
|
||||
installPs1, err := os.ReadFile(installPath)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("render: read %s/install.ps1: %w", pkgName, err)
|
||||
}
|
||||
|
||||
text := string(tpl)
|
||||
|
||||
vars := []struct {
|
||||
name string
|
||||
value string
|
||||
}{
|
||||
{"WEBI_PKG", p.PkgName + "@" + p.Tag},
|
||||
{"WEBI_HOST", p.Host},
|
||||
{"WEBI_VERSION", p.Version},
|
||||
{"WEBI_GIT_TAG", p.GitTag},
|
||||
{"WEBI_PKG_URL", p.PkgURL},
|
||||
{"WEBI_PKG_FILE", p.PkgFile},
|
||||
{"WEBI_PKG_PATHNAME", p.PkgFile},
|
||||
{"PKG_NAME", p.PkgName},
|
||||
}
|
||||
|
||||
for _, v := range vars {
|
||||
text = InjectPSVar(text, v.name, v.value)
|
||||
}
|
||||
|
||||
// PS1 marker is at column 0, no padding needed.
|
||||
text = replaceMarkerLine(text, "{{ installer }}", string(installPs1))
|
||||
|
||||
return text, nil
|
||||
}
|
||||
|
||||
// InjectPSVar replaces a PowerShell template variable line with its value.
|
||||
// Matches lines like:
|
||||
//
|
||||
// #$Env:WEBI_VERSION = v12.16.2
|
||||
// $Env:WEBI_HOST = 'https://webinstall.dev'
|
||||
func InjectPSVar(text, name, value string) string {
|
||||
p := getPSVarPattern(name)
|
||||
return p.ReplaceAllString(text, "${1}$$Env:"+name+" = '"+sanitizePSValue(value)+"'")
|
||||
}
|
||||
|
||||
var psVarPatterns = map[string]*regexp.Regexp{}
|
||||
|
||||
func getPSVarPattern(name string) *regexp.Regexp {
|
||||
if p, ok := psVarPatterns[name]; ok {
|
||||
return p
|
||||
}
|
||||
// Match: optional leading whitespace, optional #, $Env:NAME, =, rest of line
|
||||
p := regexp.MustCompile(`(?m)^([ \t]*)#?\$Env:` + regexp.QuoteMeta(name) + `\s*=.*$`)
|
||||
psVarPatterns[name] = p
|
||||
return p
|
||||
}
|
||||
|
||||
// sanitizePSValue escapes single quotes for PowerShell single-quoted strings.
|
||||
// In PowerShell, single quotes inside single-quoted strings are doubled: ''
|
||||
func sanitizePSValue(s string) string {
|
||||
return strings.ReplaceAll(s, "'", "''")
|
||||
}
|
||||
|
||||
// varPattern matches shell variable declarations in the template.
|
||||
// Matches lines like:
|
||||
//
|
||||
// #WEBI_VERSION=
|
||||
// #export WEBI_PKG_URL=
|
||||
// #WEBI_OS=
|
||||
var varPatterns = map[string]*regexp.Regexp{}
|
||||
|
||||
func getVarPattern(name string) *regexp.Regexp {
|
||||
if p, ok := varPatterns[name]; ok {
|
||||
return p
|
||||
}
|
||||
// Match: optional leading whitespace, optional #, optional export, the var name, =, rest of line
|
||||
p := regexp.MustCompile(`(?m)^([ \t]*)#?([ \t]*)(export[ \t]+)?[ \t]*(` + regexp.QuoteMeta(name) + `)=.*$`)
|
||||
varPatterns[name] = p
|
||||
return p
|
||||
}
|
||||
|
||||
// InjectVar replaces a template variable line with its value.
|
||||
// It matches lines like:
|
||||
//
|
||||
// #WEBI_VERSION=
|
||||
// #export WEBI_PKG_URL=
|
||||
// export WEBI_HOST=
|
||||
//
|
||||
// and replaces them with the value in single quotes.
|
||||
func InjectVar(text, name, value string) string {
|
||||
p := getVarPattern(name)
|
||||
return p.ReplaceAllString(text, "${1}${3}"+name+"='"+sanitizeShellValue(value)+"'")
|
||||
}
|
||||
|
||||
// sanitizeShellValue ensures a value is safe to embed in single quotes.
|
||||
// Single quotes in shell can't be escaped inside single quotes, so we
|
||||
// close-quote, add escaped quote, re-open quote: 'foo'\''bar'
|
||||
func sanitizeShellValue(s string) string {
|
||||
return strings.ReplaceAll(s, "'", `'\''`)
|
||||
}
|
||||
|
||||
// padScript prepends each line of a script with the given indent string.
|
||||
// This matches production behavior where install.sh content is indented
|
||||
// to align with the surrounding template code.
|
||||
func padScript(script, indent string) string {
|
||||
lines := strings.Split(script, "\n")
|
||||
for i, line := range lines {
|
||||
if line != "" {
|
||||
lines[i] = indent + line
|
||||
}
|
||||
}
|
||||
return strings.Join(lines, "\n")
|
||||
}
|
||||
|
||||
// replaceMarkerLine replaces an entire line containing the marker
|
||||
// (including any leading whitespace) with the replacement text.
|
||||
// This matches production's regex: /\s*#?\s*{{ installer }}/
|
||||
func replaceMarkerLine(text, marker, replacement string) string {
|
||||
re := regexp.MustCompile(`(?m)^[ \t]*#?[ \t]*` + regexp.QuoteMeta(marker) + `[^\n]*`)
|
||||
return re.ReplaceAllLiteralString(text, replacement)
|
||||
}
|
||||
@@ -1,90 +0,0 @@
|
||||
package render
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestInjectVar(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
input string
|
||||
key string
|
||||
value string
|
||||
want string
|
||||
}{
|
||||
{
|
||||
name: "commented var",
|
||||
input: " #WEBI_VERSION=",
|
||||
key: "WEBI_VERSION",
|
||||
value: "1.2.3",
|
||||
want: " WEBI_VERSION='1.2.3'",
|
||||
},
|
||||
{
|
||||
name: "commented export var",
|
||||
input: " #export WEBI_PKG_URL=",
|
||||
key: "WEBI_PKG_URL",
|
||||
value: "https://example.com/foo.tar.gz",
|
||||
want: " export WEBI_PKG_URL='https://example.com/foo.tar.gz'",
|
||||
},
|
||||
{
|
||||
name: "existing value replaced",
|
||||
input: " export WEBI_HOST=",
|
||||
key: "WEBI_HOST",
|
||||
value: "https://webinstall.dev",
|
||||
want: " export WEBI_HOST='https://webinstall.dev'",
|
||||
},
|
||||
{
|
||||
name: "value with single quotes",
|
||||
input: " #PKG_NAME=",
|
||||
key: "PKG_NAME",
|
||||
value: "it's-a-test",
|
||||
want: " PKG_NAME='it'\\''s-a-test'",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := InjectVar(tt.input, tt.key, tt.value)
|
||||
if strings.TrimSpace(got) != strings.TrimSpace(tt.want) {
|
||||
t.Errorf("got %q\nwant %q", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestInjectVarInTemplate(t *testing.T) {
|
||||
tpl := `#!/bin/sh
|
||||
__bootstrap_webi() {
|
||||
#PKG_NAME=
|
||||
#WEBI_OS=
|
||||
#WEBI_ARCH=
|
||||
#WEBI_VERSION=
|
||||
export WEBI_HOST=
|
||||
WEBI_PKG_DOWNLOAD=""
|
||||
`
|
||||
|
||||
result := tpl
|
||||
result = InjectVar(result, "PKG_NAME", "bat")
|
||||
result = InjectVar(result, "WEBI_OS", "linux")
|
||||
result = InjectVar(result, "WEBI_ARCH", "x86_64")
|
||||
result = InjectVar(result, "WEBI_VERSION", "0.26.1")
|
||||
result = InjectVar(result, "WEBI_HOST", "https://webinstall.dev")
|
||||
|
||||
if !strings.Contains(result, "PKG_NAME='bat'") {
|
||||
t.Error("PKG_NAME not injected")
|
||||
}
|
||||
if !strings.Contains(result, "WEBI_OS='linux'") {
|
||||
t.Error("WEBI_OS not injected")
|
||||
}
|
||||
if !strings.Contains(result, "WEBI_VERSION='0.26.1'") {
|
||||
t.Error("WEBI_VERSION not injected")
|
||||
}
|
||||
if !strings.Contains(result, "export WEBI_HOST='https://webinstall.dev'") {
|
||||
t.Error("WEBI_HOST not injected")
|
||||
}
|
||||
// Should not have #PKG_NAME= anymore.
|
||||
if strings.Contains(result, "#PKG_NAME=") {
|
||||
t.Error("#PKG_NAME= should have been replaced")
|
||||
}
|
||||
}
|
||||
@@ -1,303 +0,0 @@
|
||||
// Package resolve picks the best release for a given platform query.
|
||||
//
|
||||
// Given a set of classified distributables and a target query (OS, arch,
|
||||
// libc, format preferences, version constraint), it returns the single
|
||||
// best matching release — or nil if nothing matches.
|
||||
package resolve
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
"github.com/webinstall/webi-installers/internal/lexver"
|
||||
)
|
||||
|
||||
// Dist is one downloadable distributable — matches the CSV row from classify.
|
||||
type Dist struct {
|
||||
Package string
|
||||
Version string
|
||||
Channel string
|
||||
OS string
|
||||
Arch string
|
||||
Libc string
|
||||
Format string
|
||||
Download string
|
||||
Filename string
|
||||
SHA256 string
|
||||
Size int64
|
||||
LTS bool
|
||||
Date string
|
||||
Extra string // extra version info for sorting
|
||||
GitTag string // original git tag or branch — only for format="git"
|
||||
GitCommitHash string // short commit hash — only for format="git"
|
||||
Variants []string // build qualifiers: "installer", "rocm", "fxdependent", etc.
|
||||
}
|
||||
|
||||
// Query describes what the caller wants.
|
||||
type Query struct {
|
||||
OS buildmeta.OS
|
||||
Arch buildmeta.Arch
|
||||
Libc buildmeta.Libc
|
||||
Formats []string // acceptable formats (e.g. ".tar.gz", ".zip"), in preference order
|
||||
Channel string // "stable" (default), "beta", etc.
|
||||
Version string // version prefix constraint ("24", "24.14", ""), empty = latest
|
||||
Variants []string // if non-empty, only match assets with these variants
|
||||
}
|
||||
|
||||
// Match is the resolved release.
|
||||
type Match struct {
|
||||
Version string
|
||||
OS string
|
||||
Arch string
|
||||
Libc string
|
||||
Format string
|
||||
Download string
|
||||
Filename string
|
||||
LTS bool
|
||||
Date string
|
||||
Channel string
|
||||
}
|
||||
|
||||
// Best finds the single best release matching the query.
|
||||
// Returns nil if nothing matches.
|
||||
func Best(dists []Dist, q Query) *Match {
|
||||
channel := q.Channel
|
||||
if channel == "" {
|
||||
channel = "stable"
|
||||
}
|
||||
|
||||
// Build format set for fast lookup + rank map for preference.
|
||||
formatRank := make(map[string]int, len(q.Formats))
|
||||
for i, f := range q.Formats {
|
||||
formatRank[f] = i
|
||||
}
|
||||
|
||||
// Build the set of acceptable architectures (native + compat).
|
||||
compatArches := buildmeta.CompatArches(q.OS, q.Arch)
|
||||
archRank := make(map[string]int, len(compatArches))
|
||||
for i, a := range compatArches {
|
||||
archRank[string(a)] = i
|
||||
}
|
||||
|
||||
// Parse version prefix for constraint matching.
|
||||
var versionPrefix lexver.Version
|
||||
hasVersionConstraint := q.Version != ""
|
||||
if hasVersionConstraint {
|
||||
versionPrefix = lexver.Parse(q.Version)
|
||||
}
|
||||
|
||||
var best *candidate
|
||||
for i := range dists {
|
||||
d := &dists[i]
|
||||
|
||||
// Channel filter.
|
||||
if channel == "stable" && d.Channel != "stable" && d.Channel != "" {
|
||||
continue
|
||||
}
|
||||
|
||||
// OS filter: exact match, POSIX fallback, or ANYOS.
|
||||
if !osMatches(q.OS, d.OS) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Arch filter (including compat arches).
|
||||
// Empty arch, ANYARCH, or "*" means "universal/platform-agnostic" —
|
||||
// accept it but rank it lower than an exact match.
|
||||
aRank, archOK := archRank[d.Arch]
|
||||
if !archOK && (d.Arch == "" || d.Arch == "*" || d.Arch == string(buildmeta.ArchAny)) {
|
||||
// Universal binary — rank after all specific arches.
|
||||
aRank = len(compatArches)
|
||||
archOK = true
|
||||
}
|
||||
if !archOK {
|
||||
continue
|
||||
}
|
||||
|
||||
// Libc filter.
|
||||
if !libcMatches(q.OS, q.Libc, d.Libc) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Format filter.
|
||||
// Empty format means bare binary — accept as last resort.
|
||||
fRank, formatOK := formatRank[d.Format]
|
||||
if !formatOK && d.Format == "" {
|
||||
// Bare binary — rank after all explicit formats.
|
||||
fRank = len(q.Formats)
|
||||
formatOK = true
|
||||
}
|
||||
if !formatOK && len(q.Formats) > 0 {
|
||||
continue
|
||||
}
|
||||
if !formatOK {
|
||||
fRank = 999
|
||||
}
|
||||
|
||||
// Version constraint.
|
||||
ver := lexver.Parse(d.Version)
|
||||
if hasVersionConstraint && !ver.HasPrefix(versionPrefix) {
|
||||
continue
|
||||
}
|
||||
|
||||
c := &candidate{
|
||||
dist: d,
|
||||
ver: ver,
|
||||
archRank: aRank,
|
||||
formatRank: fRank,
|
||||
hasVariants: len(d.Variants) > 0,
|
||||
}
|
||||
|
||||
if best == nil || c.betterThan(best) {
|
||||
best = c
|
||||
}
|
||||
}
|
||||
|
||||
if best == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
d := best.dist
|
||||
return &Match{
|
||||
Version: d.Version,
|
||||
OS: d.OS,
|
||||
Arch: d.Arch,
|
||||
Libc: d.Libc,
|
||||
Format: d.Format,
|
||||
Download: d.Download,
|
||||
Filename: d.Filename,
|
||||
LTS: d.LTS,
|
||||
Date: d.Date,
|
||||
Channel: d.Channel,
|
||||
}
|
||||
}
|
||||
|
||||
// Catalog computes aggregate metadata across all stable dists for a package.
|
||||
type Catalog struct {
|
||||
OSes []string
|
||||
Arches []string
|
||||
Libcs []string
|
||||
Formats []string
|
||||
Latest string // highest version of any channel
|
||||
Stable string // highest stable version
|
||||
}
|
||||
|
||||
// Survey scans all dists and returns the catalog.
|
||||
func Survey(dists []Dist) Catalog {
|
||||
oses := make(map[string]bool)
|
||||
arches := make(map[string]bool)
|
||||
libcs := make(map[string]bool)
|
||||
formats := make(map[string]bool)
|
||||
|
||||
var latest, stable string
|
||||
for _, d := range dists {
|
||||
if d.OS != "" {
|
||||
oses[d.OS] = true
|
||||
}
|
||||
if d.Arch != "" {
|
||||
arches[d.Arch] = true
|
||||
}
|
||||
if d.Libc != "" {
|
||||
libcs[d.Libc] = true
|
||||
}
|
||||
if d.Format != "" {
|
||||
formats[d.Format] = true
|
||||
}
|
||||
|
||||
v := lexver.Parse(d.Version)
|
||||
if latest == "" || lexver.Compare(v, lexver.Parse(latest)) > 0 {
|
||||
latest = d.Version
|
||||
}
|
||||
if d.Channel == "stable" || d.Channel == "" {
|
||||
if stable == "" || lexver.Compare(v, lexver.Parse(stable)) > 0 {
|
||||
stable = d.Version
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return Catalog{
|
||||
OSes: sortedKeys(oses),
|
||||
Arches: sortedKeys(arches),
|
||||
Libcs: sortedKeys(libcs),
|
||||
Formats: sortedKeys(formats),
|
||||
Latest: latest,
|
||||
Stable: stable,
|
||||
}
|
||||
}
|
||||
|
||||
type candidate struct {
|
||||
dist *Dist
|
||||
ver lexver.Version
|
||||
archRank int
|
||||
formatRank int
|
||||
hasVariants bool // true if dist has variant qualifiers (GPU, installer, etc.)
|
||||
}
|
||||
|
||||
// betterThan returns true if c is a better match than other.
|
||||
// Priority: version (higher) > base over variant > arch rank (lower=native) > format rank (lower=preferred).
|
||||
func (c *candidate) betterThan(other *candidate) bool {
|
||||
cmp := lexver.Compare(c.ver, other.ver)
|
||||
if cmp != 0 {
|
||||
return cmp > 0
|
||||
}
|
||||
// Prefer base build over variant builds (rocm, installer, etc.)
|
||||
if c.hasVariants != other.hasVariants {
|
||||
return !c.hasVariants
|
||||
}
|
||||
if c.archRank != other.archRank {
|
||||
return c.archRank < other.archRank
|
||||
}
|
||||
return c.formatRank < other.formatRank
|
||||
}
|
||||
|
||||
// osMatches checks whether a dist's OS is acceptable for the query.
|
||||
// Matches exact OS, ANYOS (universal), and POSIX compatibility levels
|
||||
// (posix_2017 matches any non-Windows OS).
|
||||
func osMatches(want buildmeta.OS, have string) bool {
|
||||
if have == string(want) {
|
||||
return true
|
||||
}
|
||||
if have == string(buildmeta.OSAny) {
|
||||
return true
|
||||
}
|
||||
// POSIX assets run on any non-Windows system.
|
||||
if want != buildmeta.OSWindows {
|
||||
if have == string(buildmeta.OSPosix2017) || have == string(buildmeta.OSPosix2024) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// libcMatches checks whether a dist's libc is acceptable for the query.
|
||||
func libcMatches(os buildmeta.OS, want buildmeta.Libc, have string) bool {
|
||||
// Darwin and Windows don't use libc tagging — accept anything.
|
||||
if os == buildmeta.OSDarwin || os == buildmeta.OSWindows {
|
||||
return true
|
||||
}
|
||||
|
||||
// If the dist has no libc tag, accept it (likely statically linked).
|
||||
if have == "" || have == "none" || have == string(buildmeta.LibcNone) {
|
||||
return true
|
||||
}
|
||||
|
||||
// If the query has no libc preference, accept any.
|
||||
if want == "" || want == buildmeta.LibcNone {
|
||||
return true
|
||||
}
|
||||
|
||||
return have == string(want)
|
||||
}
|
||||
|
||||
func sortedKeys(m map[string]bool) []string {
|
||||
keys := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
// Simple insertion sort — these are tiny sets.
|
||||
for i := 1; i < len(keys); i++ {
|
||||
for j := i; j > 0 && strings.Compare(keys[j-1], keys[j]) > 0; j-- {
|
||||
keys[j-1], keys[j] = keys[j], keys[j-1]
|
||||
}
|
||||
}
|
||||
return keys
|
||||
}
|
||||
@@ -1,422 +0,0 @@
|
||||
package resolve_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
"github.com/webinstall/webi-installers/internal/resolve"
|
||||
)
|
||||
|
||||
// legacyAsset matches the _cache/ JSON format.
|
||||
type legacyAsset struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
LTS bool `json:"lts"`
|
||||
Channel string `json:"channel"`
|
||||
Date string `json:"date"`
|
||||
OS string `json:"os"`
|
||||
Arch string `json:"arch"`
|
||||
Libc string `json:"libc"`
|
||||
Ext string `json:"ext"`
|
||||
Download string `json:"download"`
|
||||
}
|
||||
|
||||
type legacyCache struct {
|
||||
Releases []legacyAsset `json:"releases"`
|
||||
}
|
||||
|
||||
func loadCacheDists(t *testing.T, pkg string) []resolve.Dist {
|
||||
t.Helper()
|
||||
cacheDir := filepath.Join("..", "..", "_cache", "2026-03")
|
||||
path := filepath.Join(cacheDir, pkg+".json")
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Skipf("no cache file for %s: %v", pkg, err)
|
||||
}
|
||||
var lc legacyCache
|
||||
if err := json.Unmarshal(data, &lc); err != nil {
|
||||
t.Fatalf("parse %s: %v", pkg, err)
|
||||
}
|
||||
dists := make([]resolve.Dist, len(lc.Releases))
|
||||
for i, la := range lc.Releases {
|
||||
// Reverse-translate legacy Node.js vocabulary to Go canonical names.
|
||||
// The cache file uses macos/amd64/arm64; the resolver uses darwin/x86_64/aarch64.
|
||||
osStr := la.OS
|
||||
if osStr == "macos" {
|
||||
osStr = "darwin"
|
||||
}
|
||||
archStr := la.Arch
|
||||
switch archStr {
|
||||
case "amd64":
|
||||
archStr = "x86_64"
|
||||
case "arm64":
|
||||
archStr = "aarch64"
|
||||
}
|
||||
// Restore dot-prefix convention: cache stores "tar.gz", resolver needs ".tar.gz".
|
||||
// "exe" with no dot in filename = bare binary (Format ""), otherwise ".exe".
|
||||
format := la.Ext
|
||||
switch {
|
||||
case format == "exe" && !strings.Contains(la.Name, "."):
|
||||
format = ""
|
||||
case format != "":
|
||||
format = "." + format
|
||||
}
|
||||
dists[i] = resolve.Dist{
|
||||
Filename: la.Name,
|
||||
Version: la.Version,
|
||||
LTS: la.LTS,
|
||||
Channel: la.Channel,
|
||||
Date: la.Date,
|
||||
OS: osStr,
|
||||
Arch: archStr,
|
||||
Libc: la.Libc, // "none" = buildmeta.LibcNone (statically linked)
|
||||
Format: format,
|
||||
Download: la.Download,
|
||||
}
|
||||
}
|
||||
return dists
|
||||
}
|
||||
|
||||
// platforms is the standard webi target matrix.
|
||||
var platforms = []struct {
|
||||
name string
|
||||
os buildmeta.OS
|
||||
arch buildmeta.Arch
|
||||
formats []string
|
||||
}{
|
||||
{"darwin-arm64", buildmeta.OSDarwin, buildmeta.ArchARM64, []string{".tar.xz", ".tar.gz", ".zip"}},
|
||||
{"darwin-amd64", buildmeta.OSDarwin, buildmeta.ArchAMD64, []string{".tar.xz", ".tar.gz", ".zip"}},
|
||||
{"linux-amd64", buildmeta.OSLinux, buildmeta.ArchAMD64, []string{".tar.xz", ".exe.xz", ".tar.gz", ".xz", ".gz", ".zip"}},
|
||||
{"linux-arm64", buildmeta.OSLinux, buildmeta.ArchARM64, []string{".tar.xz", ".exe.xz", ".tar.gz", ".xz", ".gz", ".zip"}},
|
||||
{"linux-armv7", buildmeta.OSLinux, buildmeta.ArchARMv7, []string{".tar.xz", ".tar.gz", ".xz", ".gz", ".zip"}},
|
||||
{"linux-armv6", buildmeta.OSLinux, buildmeta.ArchARMv6, []string{".tar.xz", ".tar.gz", ".xz", ".gz", ".zip"}},
|
||||
{"windows-amd64", buildmeta.OSWindows, buildmeta.ArchAMD64, []string{".zip", ".tar.gz", ".exe", ".7z"}},
|
||||
{"windows-arm64", buildmeta.OSWindows, buildmeta.ArchARM64, []string{".zip", ".tar.gz", ".exe", ".7z"}},
|
||||
}
|
||||
|
||||
// TestResolveAllPackages loads every package from the cache and verifies
|
||||
// the resolver finds a match for each platform the package supports.
|
||||
func TestResolveAllPackages(t *testing.T) {
|
||||
cacheDir := filepath.Join("..", "..", "_cache", "2026-03")
|
||||
entries, err := os.ReadDir(cacheDir)
|
||||
if err != nil {
|
||||
t.Skipf("no cache dir: %v", err)
|
||||
}
|
||||
|
||||
var pkgs []string
|
||||
for _, e := range entries {
|
||||
if strings.HasSuffix(e.Name(), ".json") {
|
||||
pkgs = append(pkgs, strings.TrimSuffix(e.Name(), ".json"))
|
||||
}
|
||||
}
|
||||
|
||||
if len(pkgs) < 50 {
|
||||
t.Fatalf("expected at least 50 packages, got %d", len(pkgs))
|
||||
}
|
||||
|
||||
for _, pkg := range pkgs {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
dists := loadCacheDists(t, pkg)
|
||||
if len(dists) == 0 {
|
||||
t.Skip("no releases")
|
||||
}
|
||||
|
||||
// Determine which platforms this package supports.
|
||||
cat := resolve.Survey(dists)
|
||||
osSet := make(map[string]bool, len(cat.OSes))
|
||||
for _, o := range cat.OSes {
|
||||
osSet[o] = true
|
||||
}
|
||||
|
||||
for _, plat := range platforms {
|
||||
platOS := string(plat.os)
|
||||
// Check if this package has any assets for this OS
|
||||
// (including POSIX/ANYOS which are compatible).
|
||||
supported := osSet[platOS] ||
|
||||
osSet[string(buildmeta.OSAny)] ||
|
||||
(platOS != "windows" && (osSet[string(buildmeta.OSPosix2017)] || osSet[string(buildmeta.OSPosix2024)]))
|
||||
|
||||
if !supported {
|
||||
continue
|
||||
}
|
||||
|
||||
t.Run(plat.name, func(t *testing.T) {
|
||||
m := resolve.Best(dists, resolve.Query{
|
||||
OS: plat.os,
|
||||
Arch: plat.arch,
|
||||
Formats: plat.formats,
|
||||
})
|
||||
if m == nil {
|
||||
// This is a warning, not a failure — some packages
|
||||
// legitimately don't have builds for all arches.
|
||||
// But log it so we can spot unexpected gaps.
|
||||
t.Logf("WARN: no match for %s on %s (has OSes: %v, Arches: %v)",
|
||||
pkg, plat.name, cat.OSes, cat.Arches)
|
||||
return
|
||||
}
|
||||
if m.Version == "" {
|
||||
t.Error("matched but Version is empty")
|
||||
}
|
||||
if m.Download == "" {
|
||||
t.Error("matched but Download is empty")
|
||||
}
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Packages with known platform expectations. Each entry specifies
|
||||
// platforms that MUST resolve and the expected latest version.
|
||||
var knownPackages = []struct {
|
||||
pkg string
|
||||
version string // expected latest stable version (prefix match)
|
||||
platforms []string // platform names from the platforms table
|
||||
}{
|
||||
{"bat", "0.26", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"caddy", "2.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv7", "linux-armv6", "windows-amd64"}},
|
||||
{"delta", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"fd", "10.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv7", "windows-amd64"}},
|
||||
{"fzf", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv7", "windows-amd64"}},
|
||||
{"gh", "2.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv6", "windows-amd64"}},
|
||||
{"rg", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"shellcheck", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv6", "windows-amd64"}},
|
||||
{"shfmt", "3.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv6", "windows-amd64"}},
|
||||
{"xz", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"yq", "4.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv6", "windows-amd64"}},
|
||||
{"zoxide", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "linux-armv7", "windows-amd64"}},
|
||||
{"aliasman", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64"}},
|
||||
{"comrak", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "windows-amd64"}},
|
||||
{"hugo", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"node", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"terraform", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"zig", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
}
|
||||
|
||||
// TestKnownPackages verifies specific packages resolve correctly
|
||||
// with expected versions and platform coverage.
|
||||
func TestKnownPackages(t *testing.T) {
|
||||
platMap := make(map[string]struct {
|
||||
os buildmeta.OS
|
||||
arch buildmeta.Arch
|
||||
formats []string
|
||||
})
|
||||
for _, p := range platforms {
|
||||
platMap[p.name] = struct {
|
||||
os buildmeta.OS
|
||||
arch buildmeta.Arch
|
||||
formats []string
|
||||
}{p.os, p.arch, p.formats}
|
||||
}
|
||||
|
||||
for _, kp := range knownPackages {
|
||||
t.Run(kp.pkg, func(t *testing.T) {
|
||||
dists := loadCacheDists(t, kp.pkg)
|
||||
|
||||
for _, platName := range kp.platforms {
|
||||
plat := platMap[platName]
|
||||
t.Run(platName, func(t *testing.T) {
|
||||
m := resolve.Best(dists, resolve.Query{
|
||||
OS: plat.os,
|
||||
Arch: plat.arch,
|
||||
Formats: plat.formats,
|
||||
})
|
||||
if m == nil {
|
||||
t.Skipf("no build available for %s on %s — upstream gap", kp.pkg, platName)
|
||||
return
|
||||
}
|
||||
if kp.version != "" {
|
||||
// Strip leading "v" for prefix comparison.
|
||||
v := strings.TrimPrefix(m.Version, "v")
|
||||
if !strings.HasPrefix(v, kp.version) {
|
||||
t.Errorf("Version = %q, want prefix %q", m.Version, kp.version)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestResolveVersionConstraints tests version pinning across real packages.
|
||||
func TestResolveVersionConstraints(t *testing.T) {
|
||||
tests := []struct {
|
||||
pkg string
|
||||
version string // constraint
|
||||
wantPfx string // expected version prefix in result
|
||||
}{
|
||||
{"bat", "0.25", "0.25"},
|
||||
{"bat", "0.26", "0.26"},
|
||||
{"gh", "2.40", "2.40"},
|
||||
{"node", "20", "20."},
|
||||
{"node", "22", "22."},
|
||||
{"hugo", "0.121", "0.121"},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
name := fmt.Sprintf("%s@%s", tt.pkg, tt.version)
|
||||
t.Run(name, func(t *testing.T) {
|
||||
dists := loadCacheDists(t, tt.pkg)
|
||||
m := resolve.Best(dists, resolve.Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.xz", ".tar.gz", ".zip"},
|
||||
Version: tt.version,
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatalf("no match for %s@%s", tt.pkg, tt.version)
|
||||
}
|
||||
v := strings.TrimPrefix(m.Version, "v")
|
||||
if !strings.HasPrefix(v, tt.wantPfx) {
|
||||
t.Errorf("Version = %q, want prefix %q", m.Version, tt.wantPfx)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestResolveArchFallbackReal tests arch fallback with real package data.
|
||||
func TestResolveArchFallbackReal(t *testing.T) {
|
||||
// awless only has amd64 builds — macOS ARM64 should fall back.
|
||||
dists := loadCacheDists(t, "awless")
|
||||
m := resolve.Best(dists, resolve.Query{
|
||||
OS: buildmeta.OSDarwin,
|
||||
Arch: buildmeta.ArchARM64,
|
||||
Formats: []string{".tar.gz", ".zip"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected Rosetta 2 fallback for awless")
|
||||
}
|
||||
if m.Arch != "x86_64" {
|
||||
t.Errorf("Arch = %q, want x86_64", m.Arch)
|
||||
}
|
||||
}
|
||||
|
||||
// TestResolvePosixPackages tests packages that use posix_2017/ANYOS.
|
||||
func TestResolvePosixPackages(t *testing.T) {
|
||||
posixPkgs := []string{"aliasman", "pathman", "serviceman"}
|
||||
for _, pkg := range posixPkgs {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
dists := loadCacheDists(t, pkg)
|
||||
if len(dists) == 0 {
|
||||
t.Skip("no releases")
|
||||
}
|
||||
|
||||
// Should resolve on Linux.
|
||||
m := resolve.Best(dists, resolve.Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.xz", ".tar.gz", ".zip", ".xz", ".gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Error("expected match on Linux for POSIX package")
|
||||
}
|
||||
|
||||
// Should resolve on macOS.
|
||||
m = resolve.Best(dists, resolve.Query{
|
||||
OS: buildmeta.OSDarwin,
|
||||
Arch: buildmeta.ArchARM64,
|
||||
Formats: []string{".tar.xz", ".tar.gz", ".zip"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Error("expected match on macOS for POSIX package")
|
||||
}
|
||||
|
||||
// Should NOT resolve on Windows (POSIX packages aren't Windows-compatible).
|
||||
m = resolve.Best(dists, resolve.Query{
|
||||
OS: buildmeta.OSWindows,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".zip", ".tar.gz"},
|
||||
})
|
||||
// This may or may not resolve depending on whether the package
|
||||
// also has Windows builds. Don't assert nil — just check it
|
||||
// doesn't return a posix_2017 match for Windows.
|
||||
if m != nil && (m.OS == "posix_2017" || m.OS == "posix_2024") {
|
||||
t.Errorf("POSIX package should not match Windows, got OS=%q", m.OS)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestResolveLibcPreference tests libc selection.
|
||||
// bat is a Rust project — its musl builds are static (libc='none').
|
||||
// pwsh has hard musl dependencies (libc='musl').
|
||||
func TestResolveLibcPreference(t *testing.T) {
|
||||
batDists := loadCacheDists(t, "bat")
|
||||
|
||||
// Musl host requesting bat: gets the static musl build (tagged 'none').
|
||||
m := resolve.Best(batDists, resolve.Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Libc: buildmeta.LibcMusl,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match for musl host")
|
||||
}
|
||||
// Rust musl builds are static — tagged as 'none', not 'musl'.
|
||||
if m.Libc != "none" {
|
||||
t.Errorf("bat musl request: Libc = %q, want none (static musl)", m.Libc)
|
||||
}
|
||||
|
||||
// Explicit gnu request.
|
||||
m = resolve.Best(batDists, resolve.Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Libc: buildmeta.LibcGNU,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected gnu match")
|
||||
}
|
||||
if m.Libc != "gnu" {
|
||||
t.Errorf("Libc = %q, want gnu", m.Libc)
|
||||
}
|
||||
|
||||
// No preference — should still match (accepts any).
|
||||
m = resolve.Best(batDists, resolve.Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match with no libc preference")
|
||||
}
|
||||
|
||||
// pwsh has hard musl builds (dynamically linked, requires musl runtime).
|
||||
pwshDists := loadCacheDists(t, "pwsh")
|
||||
m = resolve.Best(pwshDists, resolve.Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Libc: buildmeta.LibcMusl,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected pwsh musl match")
|
||||
}
|
||||
if m.Libc != "musl" {
|
||||
t.Errorf("pwsh musl request: Libc = %q, want musl", m.Libc)
|
||||
}
|
||||
}
|
||||
|
||||
// TestResolveFormatFallback tests format preference cascading.
|
||||
func TestResolveFormatFallback(t *testing.T) {
|
||||
// Request .tar.xz first, fall back to .tar.gz.
|
||||
dists := loadCacheDists(t, "bat")
|
||||
m := resolve.Best(dists, resolve.Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.xz", ".tar.gz", ".zip"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match")
|
||||
}
|
||||
// bat only has .tar.gz — should fall back from .tar.xz.
|
||||
if m.Format != ".tar.gz" {
|
||||
t.Errorf("Format = %q, want .tar.gz (fallback from .tar.xz)", m.Format)
|
||||
}
|
||||
}
|
||||
@@ -1,250 +0,0 @@
|
||||
package resolve
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
)
|
||||
|
||||
// bat-style dists: standard goreleaser output.
|
||||
var batDists = []Dist{
|
||||
{Version: "0.26.1", Channel: "stable", OS: "darwin", Arch: "aarch64", Format: ".tar.gz", Filename: "bat-v0.26.1-aarch64-apple-darwin.tar.gz"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "darwin", Arch: "x86_64", Format: ".tar.gz", Filename: "bat-v0.26.1-x86_64-apple-darwin.tar.gz"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "linux", Arch: "aarch64", Libc: "gnu", Format: ".tar.gz", Filename: "bat-v0.26.1-aarch64-unknown-linux-gnu.tar.gz"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "linux", Arch: "aarch64", Libc: "musl", Format: ".tar.gz", Filename: "bat-v0.26.1-aarch64-unknown-linux-musl.tar.gz"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "linux", Arch: "x86_64", Libc: "gnu", Format: ".tar.gz", Filename: "bat-v0.26.1-x86_64-unknown-linux-gnu.tar.gz"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "linux", Arch: "x86_64", Libc: "musl", Format: ".tar.gz", Filename: "bat-v0.26.1-x86_64-unknown-linux-musl.tar.gz"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "windows", Arch: "aarch64", Format: ".zip", Filename: "bat-v0.26.1-aarch64-pc-windows-msvc.zip"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "windows", Arch: "x86_64", Libc: "gnu", Format: ".zip", Filename: "bat-v0.26.1-x86_64-pc-windows-gnu.zip"},
|
||||
{Version: "0.26.1", Channel: "stable", OS: "windows", Arch: "x86_64", Libc: "msvc", Format: ".zip", Filename: "bat-v0.26.1-x86_64-pc-windows-msvc.zip"},
|
||||
// Older version.
|
||||
{Version: "0.25.0", Channel: "stable", OS: "darwin", Arch: "aarch64", Format: ".tar.gz", Filename: "bat-v0.25.0-aarch64-apple-darwin.tar.gz"},
|
||||
{Version: "0.25.0", Channel: "stable", OS: "linux", Arch: "x86_64", Libc: "gnu", Format: ".tar.gz", Filename: "bat-v0.25.0-x86_64-unknown-linux-gnu.tar.gz"},
|
||||
}
|
||||
|
||||
func TestBestExactMatch(t *testing.T) {
|
||||
m := Best(batDists, Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match")
|
||||
}
|
||||
if m.Version != "0.26.1" {
|
||||
t.Errorf("Version = %q, want 0.26.1", m.Version)
|
||||
}
|
||||
if m.Filename != "bat-v0.26.1-x86_64-unknown-linux-gnu.tar.gz" {
|
||||
t.Errorf("Filename = %q", m.Filename)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestVersionConstraint(t *testing.T) {
|
||||
m := Best(batDists, Query{
|
||||
OS: buildmeta.OSDarwin,
|
||||
Arch: buildmeta.ArchARM64,
|
||||
Formats: []string{".tar.gz"},
|
||||
Version: "0.25",
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match")
|
||||
}
|
||||
if m.Version != "0.25.0" {
|
||||
t.Errorf("Version = %q, want 0.25.0", m.Version)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestArchFallback(t *testing.T) {
|
||||
// macOS ARM64 should fall back to x86_64 via Rosetta 2
|
||||
// when no ARM64 build exists.
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "darwin", Arch: "x86_64", Format: ".tar.gz", Filename: "tool-darwin-amd64.tar.gz"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSDarwin,
|
||||
Arch: buildmeta.ArchARM64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match via Rosetta 2 fallback")
|
||||
}
|
||||
if m.Arch != "x86_64" {
|
||||
t.Errorf("Arch = %q, want x86_64", m.Arch)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestPrefersNativeOverCompat(t *testing.T) {
|
||||
// When both native and compat builds exist, prefer native.
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "darwin", Arch: "x86_64", Format: ".tar.gz", Filename: "tool-darwin-amd64.tar.gz"},
|
||||
{Version: "1.0.0", Channel: "stable", OS: "darwin", Arch: "aarch64", Format: ".tar.gz", Filename: "tool-darwin-arm64.tar.gz"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSDarwin,
|
||||
Arch: buildmeta.ArchARM64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match")
|
||||
}
|
||||
if m.Arch != "aarch64" {
|
||||
t.Errorf("Arch = %q, want aarch64 (native)", m.Arch)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestFormatPreference(t *testing.T) {
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "linux", Arch: "x86_64", Format: ".zip", Filename: "tool.zip"},
|
||||
{Version: "1.0.0", Channel: "stable", OS: "linux", Arch: "x86_64", Format: ".tar.gz", Filename: "tool.tar.gz"},
|
||||
{Version: "1.0.0", Channel: "stable", OS: "linux", Arch: "x86_64", Format: ".tar.xz", Filename: "tool.tar.xz"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.xz", ".tar.gz", ".zip"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match")
|
||||
}
|
||||
if m.Format != ".tar.xz" {
|
||||
t.Errorf("Format = %q, want .tar.xz", m.Format)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestNoMatch(t *testing.T) {
|
||||
m := Best(batDists, Query{
|
||||
OS: buildmeta.OSFreeBSD,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m != nil {
|
||||
t.Errorf("expected nil, got %+v", m)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestLibcMusl(t *testing.T) {
|
||||
m := Best(batDists, Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Libc: buildmeta.LibcMusl,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match")
|
||||
}
|
||||
if m.Libc != "musl" {
|
||||
t.Errorf("Libc = %q, want musl", m.Libc)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestPrefersBaseOverVariant(t *testing.T) {
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "linux", Arch: "x86_64", Format: ".tar.gz", Filename: "tool.tar.gz"},
|
||||
{Version: "1.0.0", Channel: "stable", OS: "linux", Arch: "x86_64", Format: ".tar.gz", Filename: "tool-rocm.tar.gz", Variants: []string{"rocm"}},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match")
|
||||
}
|
||||
if m.Filename != "tool.tar.gz" {
|
||||
t.Errorf("got variant build %q, want base", m.Filename)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestPosixFallback(t *testing.T) {
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "posix_2017", Format: ".tar.gz", Filename: "script.tar.gz"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match via POSIX fallback")
|
||||
}
|
||||
if m.OS != "posix_2017" {
|
||||
t.Errorf("OS = %q, want posix_2017", m.OS)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestAnyOS(t *testing.T) {
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "ANYOS", Format: ".tar.gz", Filename: "tool.tar.gz"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSWindows,
|
||||
Arch: buildmeta.ArchAMD64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match via ANYOS")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestAnyArch(t *testing.T) {
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "linux", Arch: "ANYARCH", Format: ".tar.gz", Filename: "tool.tar.gz"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchARM64,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match via ANYARCH")
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestWindowsArchFallback(t *testing.T) {
|
||||
// Windows ARM64 should fall back to x86_64 via emulation.
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "windows", Arch: "x86_64", Format: ".zip", Filename: "tool-win64.zip"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSWindows,
|
||||
Arch: buildmeta.ArchARM64,
|
||||
Formats: []string{".zip"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match via Windows ARM64 emulation")
|
||||
}
|
||||
if m.Arch != "x86_64" {
|
||||
t.Errorf("Arch = %q, want x86_64", m.Arch)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBestMicroArchFallback(t *testing.T) {
|
||||
// amd64v3 query should fall back to amd64 baseline.
|
||||
dists := []Dist{
|
||||
{Version: "1.0.0", Channel: "stable", OS: "linux", Arch: "x86_64", Format: ".tar.gz", Filename: "tool-amd64.tar.gz"},
|
||||
}
|
||||
m := Best(dists, Query{
|
||||
OS: buildmeta.OSLinux,
|
||||
Arch: buildmeta.ArchAMD64v3,
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if m == nil {
|
||||
t.Fatal("expected match via micro-arch fallback")
|
||||
}
|
||||
if m.Arch != "x86_64" {
|
||||
t.Errorf("Arch = %q, want x86_64 (baseline)", m.Arch)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSurvey(t *testing.T) {
|
||||
cat := Survey(batDists)
|
||||
if cat.Stable != "0.26.1" {
|
||||
t.Errorf("Stable = %q, want 0.26.1", cat.Stable)
|
||||
}
|
||||
if cat.Latest != "0.26.1" {
|
||||
t.Errorf("Latest = %q, want 0.26.1", cat.Latest)
|
||||
}
|
||||
if len(cat.OSes) != 3 {
|
||||
t.Errorf("OSes = %v, want 3", cat.OSes)
|
||||
}
|
||||
}
|
||||
@@ -1,416 +0,0 @@
|
||||
// Package resolver selects the best release asset for a given platform
|
||||
// and version constraint.
|
||||
//
|
||||
// The resolver takes a package's full asset list and a request describing
|
||||
// what the client needs (OS, arch, libc, version prefix, channel, format
|
||||
// preferences). It returns the single best matching asset or an error.
|
||||
//
|
||||
// Resolution order:
|
||||
// 1. Filter assets by channel (inclusive: @stable includes stable+lts)
|
||||
// 2. Sort versions descending, filter by version prefix if given
|
||||
// 3. For each candidate version, try compatible platform triplets
|
||||
// (OS × CompatArches fallback × libc) in preference order
|
||||
// 4. Among platform matches, pick the best format
|
||||
// 5. Among format matches, prefer assets without build variants
|
||||
package resolver
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
"github.com/webinstall/webi-installers/internal/lexver"
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
)
|
||||
|
||||
// ErrNoMatch is returned when no asset matches the request.
|
||||
var ErrNoMatch = errors.New("resolver: no matching asset")
|
||||
|
||||
// Request describes what the client is looking for.
|
||||
type Request struct {
|
||||
// OS is the target operating system (e.g. "linux", "darwin", "windows").
|
||||
OS string
|
||||
|
||||
// Arch is the target architecture (e.g. "aarch64", "x86_64").
|
||||
Arch string
|
||||
|
||||
// Libc is the preferred C library (e.g. "gnu", "musl", "msvc").
|
||||
// Empty means no preference — the resolver tries all libc values.
|
||||
Libc string
|
||||
|
||||
// Version is a version prefix constraint (e.g. "1.20", "1", "").
|
||||
// Empty means latest. Exact versions like "1.20.3" also work.
|
||||
Version string
|
||||
|
||||
// Channel selects the release stability level. Values:
|
||||
// ""/"stable" — stable and LTS only (default)
|
||||
// "lts" — LTS releases only
|
||||
// "rc" — rc + stable + LTS
|
||||
// "beta" — beta + rc + stable + LTS
|
||||
// "alpha" — everything (alpha + beta + rc + stable + LTS)
|
||||
// "pre" — alias for beta (package-specific meaning)
|
||||
Channel string
|
||||
|
||||
// LTS when true selects only LTS-flagged releases.
|
||||
LTS bool
|
||||
|
||||
// Formats lists acceptable archive formats in preference order.
|
||||
// If empty, a default preference order is used.
|
||||
Formats []string
|
||||
|
||||
// Variant selects a specific build variant (e.g. "rocm", "jetpack6").
|
||||
// If empty, assets with variants are deprioritized.
|
||||
Variant string
|
||||
}
|
||||
|
||||
// Result holds the resolved asset and metadata about the match.
|
||||
type Result struct {
|
||||
// Asset is the selected download.
|
||||
Asset storage.Asset
|
||||
|
||||
// Version is the matched version string.
|
||||
Version string
|
||||
|
||||
// Triplet is the matched platform triplet (os-arch-libc).
|
||||
Triplet string
|
||||
}
|
||||
|
||||
// Resolve finds the best matching asset for the given request.
|
||||
func Resolve(assets []storage.Asset, req Request) (Result, error) {
|
||||
if len(assets) == 0 {
|
||||
return Result{}, ErrNoMatch
|
||||
}
|
||||
|
||||
// Parse the version prefix for filtering.
|
||||
var versionPrefix lexver.Version
|
||||
hasPrefix := req.Version != ""
|
||||
if hasPrefix {
|
||||
versionPrefix = lexver.Parse(req.Version)
|
||||
}
|
||||
|
||||
// Build the channel filter.
|
||||
channelOK := channelFilter(req.Channel, req.LTS)
|
||||
|
||||
// Parse and sort all unique versions descending.
|
||||
type versionEntry struct {
|
||||
parsed lexver.Version
|
||||
raw string
|
||||
}
|
||||
seen := make(map[string]bool)
|
||||
var versions []versionEntry
|
||||
for _, a := range assets {
|
||||
if seen[a.Version] {
|
||||
continue
|
||||
}
|
||||
seen[a.Version] = true
|
||||
v := lexver.Parse(a.Version)
|
||||
v.Raw = a.Version
|
||||
versions = append(versions, versionEntry{parsed: v, raw: a.Version})
|
||||
}
|
||||
slices.SortFunc(versions, func(a, b versionEntry) int {
|
||||
return lexver.Compare(b.parsed, a.parsed) // descending
|
||||
})
|
||||
|
||||
// Build platform fallback list: ordered (os, arch, libc) combinations.
|
||||
triplets := enumerateTriplets(req.OS, req.Arch, req.Libc)
|
||||
|
||||
// Build format preference list.
|
||||
formats := req.Formats
|
||||
if len(formats) == 0 {
|
||||
formats = defaultFormats(req.OS)
|
||||
}
|
||||
|
||||
// Index assets by version+triplet for fast lookup.
|
||||
// Assets with empty OS/Arch (like git repos) use "" keys.
|
||||
type tripletKey struct {
|
||||
version string
|
||||
os string
|
||||
arch string
|
||||
libc string
|
||||
}
|
||||
index := make(map[tripletKey][]storage.Asset)
|
||||
for _, a := range assets {
|
||||
key := tripletKey{
|
||||
version: a.Version,
|
||||
os: a.OS,
|
||||
arch: a.Arch,
|
||||
libc: a.Libc,
|
||||
}
|
||||
index[key] = append(index[key], a)
|
||||
}
|
||||
|
||||
// Walk versions in descending order.
|
||||
for _, ve := range versions {
|
||||
// Check version prefix.
|
||||
if hasPrefix && !ve.parsed.HasPrefix(versionPrefix) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Check channel.
|
||||
if !channelOK(ve.parsed.Channel, ve.raw) {
|
||||
continue
|
||||
}
|
||||
|
||||
// Try each compatible triplet.
|
||||
for _, tri := range triplets {
|
||||
key := tripletKey{
|
||||
version: ve.raw,
|
||||
os: tri.os,
|
||||
arch: tri.arch,
|
||||
libc: tri.libc,
|
||||
}
|
||||
candidates := index[key]
|
||||
if len(candidates) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
// Pick the best asset from candidates.
|
||||
best, ok := pickBest(candidates, formats, req.Variant, req.LTS)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
|
||||
triplet := tri.os + "-" + tri.arch + "-" + tri.libc
|
||||
return Result{
|
||||
Asset: best,
|
||||
Version: ve.raw,
|
||||
Triplet: triplet,
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
|
||||
return Result{}, ErrNoMatch
|
||||
}
|
||||
|
||||
// channelFilter returns a function that checks whether a given channel
|
||||
// is acceptable for the requested channel level.
|
||||
func channelFilter(requested string, ltsOnly bool) func(channel string, version string) bool {
|
||||
if ltsOnly {
|
||||
return func(_ string, _ string) bool {
|
||||
// LTS filtering happens at the asset level, not version level.
|
||||
// We let all versions through and filter by LTS flag later.
|
||||
// Actually, LTS is per-asset, so we handle it in pickBest.
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
requested = strings.ToLower(requested)
|
||||
if requested == "" {
|
||||
requested = "stable"
|
||||
}
|
||||
if requested == "pre" {
|
||||
requested = "beta"
|
||||
}
|
||||
if requested == "latest" {
|
||||
requested = "stable"
|
||||
}
|
||||
|
||||
// channelRank maps channel names to a numeric rank.
|
||||
// Higher rank = less stable. A request for rank N accepts
|
||||
// everything at rank N or below.
|
||||
rank := func(ch string) int {
|
||||
ch = strings.ToLower(ch)
|
||||
switch ch {
|
||||
case "", "stable":
|
||||
return 0
|
||||
case "rc":
|
||||
return 1
|
||||
case "beta", "preview":
|
||||
return 2
|
||||
case "alpha", "dev":
|
||||
return 3
|
||||
default:
|
||||
return 2 // unknown pre-release channels default to beta-level
|
||||
}
|
||||
}
|
||||
|
||||
maxRank := rank(requested)
|
||||
return func(channel string, _ string) bool {
|
||||
return rank(channel) <= maxRank
|
||||
}
|
||||
}
|
||||
|
||||
type platformTriple struct {
|
||||
os string
|
||||
arch string
|
||||
libc string
|
||||
}
|
||||
|
||||
// enumerateTriplets builds the ordered list of platform combinations to try.
|
||||
// It uses CompatArches for arch fallback and tries multiple libc values.
|
||||
func enumerateTriplets(osStr, archStr, libcStr string) []platformTriple {
|
||||
// OS candidates: specific OS first, then POSIX compat, then any.
|
||||
var oses []string
|
||||
switch osStr {
|
||||
case "windows":
|
||||
oses = []string{"windows", "ANYOS", ""}
|
||||
case "android":
|
||||
oses = []string{"android", "linux", "posix_2024", "posix_2017", "ANYOS", ""}
|
||||
case "":
|
||||
oses = []string{"ANYOS", ""}
|
||||
default:
|
||||
oses = []string{osStr, "posix_2024", "posix_2017", "ANYOS", ""}
|
||||
}
|
||||
|
||||
// Arch candidates: use CompatArches for fallback chain.
|
||||
arches := buildmeta.CompatArches(buildmeta.OS(osStr), buildmeta.Arch(archStr))
|
||||
var archStrs []string
|
||||
for _, a := range arches {
|
||||
archStrs = append(archStrs, string(a))
|
||||
}
|
||||
// Also try ANYARCH and empty (for platform-agnostic assets like git repos).
|
||||
archStrs = append(archStrs, "ANYARCH", "")
|
||||
|
||||
// Libc candidates.
|
||||
var libcs []string
|
||||
if libcStr != "" {
|
||||
libcs = []string{libcStr, "none", ""}
|
||||
} else {
|
||||
// No preference: try all common options.
|
||||
switch osStr {
|
||||
case "linux":
|
||||
// none first (static, no deps), then gnu, musl, empty.
|
||||
libcs = []string{"none", "gnu", "musl", ""}
|
||||
case "windows":
|
||||
// none first (no deps), msvc last (needs vcredist).
|
||||
libcs = []string{"none", "msvc", ""}
|
||||
default:
|
||||
libcs = []string{"none", ""}
|
||||
}
|
||||
}
|
||||
|
||||
var triplets []platformTriple
|
||||
for _, os := range oses {
|
||||
for _, arch := range archStrs {
|
||||
for _, libc := range libcs {
|
||||
triplets = append(triplets, platformTriple{
|
||||
os: os,
|
||||
arch: arch,
|
||||
libc: libc,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
return triplets
|
||||
}
|
||||
|
||||
// pickBest selects the best asset from a set of candidates for the same
|
||||
// version and platform. Prefers the requested variant (or no-variant if
|
||||
// none requested), then picks by format preference.
|
||||
func pickBest(candidates []storage.Asset, formats []string, wantVariant string, ltsOnly bool) (storage.Asset, bool) {
|
||||
// Filter by LTS if requested.
|
||||
if ltsOnly {
|
||||
var lts []storage.Asset
|
||||
for _, a := range candidates {
|
||||
if a.LTS {
|
||||
lts = append(lts, a)
|
||||
}
|
||||
}
|
||||
if len(lts) == 0 {
|
||||
return storage.Asset{}, false
|
||||
}
|
||||
candidates = lts
|
||||
}
|
||||
|
||||
// Separate into variant-matched and non-variant pools.
|
||||
var preferred []storage.Asset
|
||||
var fallback []storage.Asset
|
||||
|
||||
for _, a := range candidates {
|
||||
if wantVariant != "" {
|
||||
// User requested a specific variant.
|
||||
if hasVariant(a.Variants, wantVariant) {
|
||||
preferred = append(preferred, a)
|
||||
} else if len(a.Variants) == 0 {
|
||||
fallback = append(fallback, a)
|
||||
}
|
||||
} else {
|
||||
// No variant requested: prefer no-variant assets.
|
||||
if len(a.Variants) == 0 {
|
||||
preferred = append(preferred, a)
|
||||
} else {
|
||||
fallback = append(fallback, a)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Try preferred pool first, then fallback.
|
||||
for _, pool := range [][]storage.Asset{preferred, fallback} {
|
||||
if len(pool) == 0 {
|
||||
continue
|
||||
}
|
||||
if best, ok := pickByFormat(pool, formats); ok {
|
||||
return best, true
|
||||
}
|
||||
}
|
||||
|
||||
return storage.Asset{}, false
|
||||
}
|
||||
|
||||
// pickByFormat selects the asset with the most preferred format.
|
||||
func pickByFormat(assets []storage.Asset, formats []string) (storage.Asset, bool) {
|
||||
for _, fmt := range formats {
|
||||
for _, a := range assets {
|
||||
if a.Format == fmt {
|
||||
return a, true
|
||||
}
|
||||
}
|
||||
}
|
||||
// No format match — return the first asset as last resort.
|
||||
if len(assets) > 0 {
|
||||
return assets[0], true
|
||||
}
|
||||
return storage.Asset{}, false
|
||||
}
|
||||
|
||||
func hasVariant(variants []string, want string) bool {
|
||||
for _, v := range variants {
|
||||
if v == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// defaultFormats returns the format preference order for an OS.
|
||||
// zst is preferred as the modern standard, but availability varies.
|
||||
func defaultFormats(os string) []string {
|
||||
switch os {
|
||||
case "windows":
|
||||
return []string{
|
||||
".tar.zst",
|
||||
".tar.xz",
|
||||
".zip",
|
||||
".tar.gz",
|
||||
".exe.xz",
|
||||
".7z",
|
||||
".exe",
|
||||
".msi",
|
||||
"git",
|
||||
}
|
||||
case "darwin":
|
||||
return []string{
|
||||
".tar.zst",
|
||||
".tar.xz",
|
||||
".zip",
|
||||
".tar.gz",
|
||||
".gz",
|
||||
".app.zip",
|
||||
".dmg",
|
||||
".pkg",
|
||||
"git",
|
||||
}
|
||||
default:
|
||||
// Linux and other POSIX.
|
||||
return []string{
|
||||
".tar.zst",
|
||||
".tar.xz",
|
||||
".tar.gz",
|
||||
".gz",
|
||||
".zip",
|
||||
".xz",
|
||||
"git",
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,290 +0,0 @@
|
||||
package resolver_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/resolver"
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
)
|
||||
|
||||
func loadAssets(t *testing.T, pkg string) []storage.Asset {
|
||||
t.Helper()
|
||||
cacheDir := filepath.Join("..", "..", "_cache", "2026-03")
|
||||
path := filepath.Join(cacheDir, pkg+".json")
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Skipf("no cache file for %s: %v", pkg, err)
|
||||
}
|
||||
var lc storage.LegacyCache
|
||||
if err := json.Unmarshal(data, &lc); err != nil {
|
||||
t.Fatalf("parse %s: %v", pkg, err)
|
||||
}
|
||||
pd := storage.ImportLegacy(lc)
|
||||
return pd.Assets
|
||||
}
|
||||
|
||||
// TestCacheResolveAllPackages loads every package from the cache and verifies
|
||||
// the resolver finds a match for each standard platform.
|
||||
func TestCacheResolveAllPackages(t *testing.T) {
|
||||
cacheDir := filepath.Join("..", "..", "_cache", "2026-03")
|
||||
entries, err := os.ReadDir(cacheDir)
|
||||
if err != nil {
|
||||
t.Skipf("no cache dir: %v", err)
|
||||
}
|
||||
|
||||
var pkgs []string
|
||||
for _, e := range entries {
|
||||
if strings.HasSuffix(e.Name(), ".json") {
|
||||
pkgs = append(pkgs, strings.TrimSuffix(e.Name(), ".json"))
|
||||
}
|
||||
}
|
||||
|
||||
if len(pkgs) < 50 {
|
||||
t.Fatalf("expected at least 50 packages, got %d", len(pkgs))
|
||||
}
|
||||
|
||||
platforms := []struct {
|
||||
name string
|
||||
os string
|
||||
arch string
|
||||
}{
|
||||
{"darwin-arm64", "darwin", "aarch64"},
|
||||
{"darwin-amd64", "darwin", "x86_64"},
|
||||
{"linux-amd64", "linux", "x86_64"},
|
||||
{"linux-arm64", "linux", "aarch64"},
|
||||
{"windows-amd64", "windows", "x86_64"},
|
||||
}
|
||||
|
||||
for _, pkg := range pkgs {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
assets := loadAssets(t, pkg)
|
||||
if len(assets) == 0 {
|
||||
t.Skip("no releases")
|
||||
}
|
||||
|
||||
// Determine which OSes this package has.
|
||||
osSet := make(map[string]bool)
|
||||
for _, a := range assets {
|
||||
if a.OS != "" {
|
||||
osSet[a.OS] = true
|
||||
}
|
||||
}
|
||||
// Also check for platform-agnostic assets.
|
||||
hasAgnostic := false
|
||||
for _, a := range assets {
|
||||
if a.OS == "" {
|
||||
hasAgnostic = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
for _, plat := range platforms {
|
||||
supported := osSet[plat.os] ||
|
||||
osSet["ANYOS"] ||
|
||||
hasAgnostic ||
|
||||
(plat.os != "windows" && (osSet["posix_2017"] || osSet["posix_2024"]))
|
||||
|
||||
if !supported {
|
||||
continue
|
||||
}
|
||||
|
||||
t.Run(plat.name, func(t *testing.T) {
|
||||
res, err := resolver.Resolve(assets, resolver.Request{
|
||||
OS: plat.os,
|
||||
Arch: plat.arch,
|
||||
})
|
||||
if err != nil {
|
||||
// Not a test failure — some packages don't have
|
||||
// all arch builds. Log for visibility.
|
||||
t.Logf("WARN: no match for %s on %s (has OSes: %v)",
|
||||
pkg, plat.name, sortedOSes(osSet))
|
||||
return
|
||||
}
|
||||
if res.Version == "" {
|
||||
t.Error("matched but Version is empty")
|
||||
}
|
||||
if res.Asset.Download == "" {
|
||||
t.Error("matched but Download is empty")
|
||||
}
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCacheKnownPackages verifies specific packages resolve correctly.
|
||||
var knownPackages = []struct {
|
||||
pkg string
|
||||
version string // expected latest stable version prefix
|
||||
platforms []string
|
||||
}{
|
||||
{"bat", "0.26", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"caddy", "2.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"delta", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"fd", "10.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"fzf", "0.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"gh", "2.", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"rg", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"node", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"terraform", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
{"zig", "", []string{"darwin-arm64", "darwin-amd64", "linux-amd64", "linux-arm64", "windows-amd64"}},
|
||||
}
|
||||
|
||||
func TestCacheKnownPackages(t *testing.T) {
|
||||
platMap := map[string]resolver.Request{
|
||||
"darwin-arm64": {OS: "darwin", Arch: "aarch64"},
|
||||
"darwin-amd64": {OS: "darwin", Arch: "x86_64"},
|
||||
"linux-amd64": {OS: "linux", Arch: "x86_64"},
|
||||
"linux-arm64": {OS: "linux", Arch: "aarch64"},
|
||||
"windows-amd64": {OS: "windows", Arch: "x86_64"},
|
||||
}
|
||||
|
||||
for _, kp := range knownPackages {
|
||||
t.Run(kp.pkg, func(t *testing.T) {
|
||||
assets := loadAssets(t, kp.pkg)
|
||||
|
||||
for _, platName := range kp.platforms {
|
||||
req := platMap[platName]
|
||||
t.Run(platName, func(t *testing.T) {
|
||||
res, err := resolver.Resolve(assets, req)
|
||||
if err != nil {
|
||||
t.Fatalf("no match for %s on %s", kp.pkg, platName)
|
||||
}
|
||||
if kp.version != "" {
|
||||
v := strings.TrimPrefix(res.Version, "v")
|
||||
if !strings.HasPrefix(v, kp.version) {
|
||||
t.Errorf("Version = %q, want prefix %q", res.Version, kp.version)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCacheVersionConstraints tests version pinning with real data.
|
||||
func TestCacheVersionConstraints(t *testing.T) {
|
||||
tests := []struct {
|
||||
pkg string
|
||||
version string
|
||||
wantPfx string
|
||||
}{
|
||||
{"bat", "0.25", "0.25"},
|
||||
{"bat", "0.26", "0.26"},
|
||||
{"gh", "2.40", "2.40"},
|
||||
{"node", "20", "20."},
|
||||
{"node", "22", "22."},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.pkg+"@"+tt.version, func(t *testing.T) {
|
||||
assets := loadAssets(t, tt.pkg)
|
||||
res, err := resolver.Resolve(assets, resolver.Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Version: tt.version,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("no match for %s@%s", tt.pkg, tt.version)
|
||||
}
|
||||
v := strings.TrimPrefix(res.Version, "v")
|
||||
if !strings.HasPrefix(v, tt.wantPfx) {
|
||||
t.Errorf("Version = %q, want prefix %q", res.Version, tt.wantPfx)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCacheArchFallback verifies Rosetta-style fallback with real data.
|
||||
func TestCacheArchFallback(t *testing.T) {
|
||||
// awless only has amd64 builds — macOS ARM64 should fall back.
|
||||
assets := loadAssets(t, "awless")
|
||||
res, err := resolver.Resolve(assets, resolver.Request{
|
||||
OS: "darwin",
|
||||
Arch: "aarch64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal("expected Rosetta 2 fallback for awless")
|
||||
}
|
||||
if res.Asset.Arch != "x86_64" {
|
||||
t.Errorf("Arch = %q, want x86_64", res.Asset.Arch)
|
||||
}
|
||||
}
|
||||
|
||||
// TestCacheGitPackages verifies git-only packages resolve on any platform.
|
||||
func TestCacheGitPackages(t *testing.T) {
|
||||
gitPkgs := []string{"vim-essentials", "vim-spell"}
|
||||
for _, pkg := range gitPkgs {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
assets := loadAssets(t, pkg)
|
||||
if len(assets) == 0 {
|
||||
t.Skip("no releases")
|
||||
}
|
||||
|
||||
// Should work on any platform.
|
||||
for _, plat := range []struct {
|
||||
os, arch string
|
||||
}{
|
||||
{"linux", "x86_64"},
|
||||
{"darwin", "aarch64"},
|
||||
{"windows", "x86_64"},
|
||||
} {
|
||||
res, err := resolver.Resolve(assets, resolver.Request{
|
||||
OS: plat.os,
|
||||
Arch: plat.arch,
|
||||
})
|
||||
if err != nil {
|
||||
t.Errorf("expected match on %s-%s", plat.os, plat.arch)
|
||||
continue
|
||||
}
|
||||
if res.Asset.Format != "git" {
|
||||
t.Errorf("format = %q, want git", res.Asset.Format)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestCacheLibcPreference tests explicit libc selection.
|
||||
// bat is Rust — its musl builds are static (tagged 'none').
|
||||
func TestCacheLibcPreference(t *testing.T) {
|
||||
assets := loadAssets(t, "bat")
|
||||
|
||||
// Musl host requesting bat: gets static musl build (tagged 'none').
|
||||
res, err := resolver.Resolve(assets, resolver.Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Libc: "musl",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal("expected match for musl host")
|
||||
}
|
||||
if res.Asset.Libc != "none" {
|
||||
t.Errorf("Libc = %q, want none (static musl)", res.Asset.Libc)
|
||||
}
|
||||
|
||||
// Explicit gnu.
|
||||
res, err = resolver.Resolve(assets, resolver.Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Libc: "gnu",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal("expected gnu match")
|
||||
}
|
||||
if res.Asset.Libc != "gnu" {
|
||||
t.Errorf("Libc = %q, want gnu", res.Asset.Libc)
|
||||
}
|
||||
}
|
||||
|
||||
func sortedOSes(m map[string]bool) []string {
|
||||
var keys []string
|
||||
for k := range m {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
return keys
|
||||
}
|
||||
@@ -1,397 +0,0 @@
|
||||
package resolver
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
)
|
||||
|
||||
func TestResolveSimple(t *testing.T) {
|
||||
assets := []storage.Asset{
|
||||
{
|
||||
Filename: "bat-v0.25.0-x86_64-unknown-linux-musl.tar.gz",
|
||||
Version: "0.25.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Libc: "musl",
|
||||
Format: ".tar.gz",
|
||||
Download: "https://example.com/bat-0.25.0-linux-x86_64.tar.gz",
|
||||
},
|
||||
{
|
||||
Filename: "bat-v0.26.0-x86_64-unknown-linux-musl.tar.gz",
|
||||
Version: "0.26.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Libc: "musl",
|
||||
Format: ".tar.gz",
|
||||
Download: "https://example.com/bat-0.26.0-linux-x86_64.tar.gz",
|
||||
},
|
||||
{
|
||||
Filename: "bat-v0.26.0-aarch64-unknown-linux-musl.tar.gz",
|
||||
Version: "0.26.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "aarch64",
|
||||
Libc: "musl",
|
||||
Format: ".tar.gz",
|
||||
Download: "https://example.com/bat-0.26.0-linux-aarch64.tar.gz",
|
||||
},
|
||||
{
|
||||
Filename: "bat-v0.26.0-x86_64-pc-windows-msvc.zip",
|
||||
Version: "0.26.0",
|
||||
Channel: "stable",
|
||||
OS: "windows",
|
||||
Arch: "x86_64",
|
||||
Libc: "msvc",
|
||||
Format: ".zip",
|
||||
Download: "https://example.com/bat-0.26.0-windows-x86_64.zip",
|
||||
},
|
||||
{
|
||||
Filename: "bat-v0.26.0-x86_64-apple-darwin.tar.gz",
|
||||
Version: "0.26.0",
|
||||
Channel: "stable",
|
||||
OS: "darwin",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
Download: "https://example.com/bat-0.26.0-darwin-x86_64.tar.gz",
|
||||
},
|
||||
}
|
||||
|
||||
t.Run("latest linux x86_64", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Version != "0.26.0" {
|
||||
t.Errorf("version = %q, want 0.26.0", res.Version)
|
||||
}
|
||||
if res.Asset.OS != "linux" {
|
||||
t.Errorf("os = %q, want linux", res.Asset.OS)
|
||||
}
|
||||
if res.Asset.Arch != "x86_64" {
|
||||
t.Errorf("arch = %q, want x86_64", res.Asset.Arch)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("latest linux aarch64", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "aarch64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Version != "0.26.0" {
|
||||
t.Errorf("version = %q, want 0.26.0", res.Version)
|
||||
}
|
||||
if res.Asset.Arch != "aarch64" {
|
||||
t.Errorf("arch = %q, want aarch64", res.Asset.Arch)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("version prefix 0.25", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Version: "0.25",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Version != "0.25.0" {
|
||||
t.Errorf("version = %q, want 0.25.0", res.Version)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("darwin arm64 falls back to x86_64", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "darwin",
|
||||
Arch: "aarch64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Asset.Arch != "x86_64" {
|
||||
t.Errorf("arch = %q, want x86_64 (Rosetta fallback)", res.Asset.Arch)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("no match returns error", func(t *testing.T) {
|
||||
_, err := Resolve(assets, Request{
|
||||
OS: "freebsd",
|
||||
Arch: "x86_64",
|
||||
})
|
||||
if err != ErrNoMatch {
|
||||
t.Errorf("err = %v, want ErrNoMatch", err)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("windows gets zip", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "windows",
|
||||
Arch: "x86_64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Asset.Format != ".zip" {
|
||||
t.Errorf("format = %q, want .zip", res.Asset.Format)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestResolveChannels(t *testing.T) {
|
||||
assets := []storage.Asset{
|
||||
{
|
||||
Filename: "tool-v2.0.0-rc1-linux-x86_64.tar.gz",
|
||||
Version: "2.0.0-rc1",
|
||||
Channel: "rc",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
},
|
||||
{
|
||||
Filename: "tool-v1.5.0-linux-x86_64.tar.gz",
|
||||
Version: "1.5.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
},
|
||||
{
|
||||
Filename: "tool-v2.0.0-beta2-linux-x86_64.tar.gz",
|
||||
Version: "2.0.0-beta2",
|
||||
Channel: "beta",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
},
|
||||
}
|
||||
|
||||
t.Run("stable skips rc and beta", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Version != "1.5.0" {
|
||||
t.Errorf("version = %q, want 1.5.0", res.Version)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("rc includes rc and stable", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Channel: "rc",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Version != "2.0.0-rc1" {
|
||||
t.Errorf("version = %q, want 2.0.0-rc1", res.Version)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("beta includes beta, rc, and stable", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Channel: "beta",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// beta2 sorts after rc1 for the same numeric version (2.0.0),
|
||||
// but rc1 is more stable. However, the user asked for beta channel
|
||||
// which includes everything — and beta sorts before rc alphabetically.
|
||||
// With lexver: 2.0.0-rc1 > 2.0.0-beta2 (rc > beta alphabetically).
|
||||
if res.Version != "2.0.0-rc1" {
|
||||
t.Errorf("version = %q, want 2.0.0-rc1", res.Version)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestResolveVariants(t *testing.T) {
|
||||
assets := []storage.Asset{
|
||||
{
|
||||
Filename: "ollama-linux-amd64.tgz",
|
||||
Version: "0.6.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
},
|
||||
{
|
||||
Filename: "ollama-linux-amd64-rocm.tgz",
|
||||
Version: "0.6.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
Variants: []string{"rocm"},
|
||||
},
|
||||
}
|
||||
|
||||
t.Run("no variant prefers plain", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(res.Asset.Variants) != 0 {
|
||||
t.Errorf("variants = %v, want empty", res.Asset.Variants)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("explicit variant selects it", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Variant: "rocm",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !hasVariant(res.Asset.Variants, "rocm") {
|
||||
t.Errorf("variants = %v, want [rocm]", res.Asset.Variants)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestResolveFormatPreference(t *testing.T) {
|
||||
assets := []storage.Asset{
|
||||
{
|
||||
Filename: "tool-v1.0.0-linux-x86_64.tar.gz",
|
||||
Version: "1.0.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
},
|
||||
{
|
||||
Filename: "tool-v1.0.0-linux-x86_64.tar.xz",
|
||||
Version: "1.0.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.xz",
|
||||
},
|
||||
{
|
||||
Filename: "tool-v1.0.0-linux-x86_64.tar.zst",
|
||||
Version: "1.0.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.zst",
|
||||
},
|
||||
}
|
||||
|
||||
t.Run("default prefers zst", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Asset.Format != ".tar.zst" {
|
||||
t.Errorf("format = %q, want .tar.zst", res.Asset.Format)
|
||||
}
|
||||
})
|
||||
|
||||
t.Run("explicit format preference", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Formats: []string{".tar.gz"},
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Asset.Format != ".tar.gz" {
|
||||
t.Errorf("format = %q, want .tar.gz", res.Asset.Format)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestResolveGitAssets(t *testing.T) {
|
||||
assets := []storage.Asset{
|
||||
{
|
||||
Filename: "vim-commentary-v1.2",
|
||||
Version: "1.2",
|
||||
Channel: "stable",
|
||||
Format: "git",
|
||||
Download: "https://github.com/tpope/vim-commentary.git",
|
||||
},
|
||||
{
|
||||
Filename: "vim-commentary-v1.1",
|
||||
Version: "1.1",
|
||||
Channel: "stable",
|
||||
Format: "git",
|
||||
Download: "https://github.com/tpope/vim-commentary.git",
|
||||
},
|
||||
}
|
||||
|
||||
t.Run("git assets match any platform", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Version != "1.2" {
|
||||
t.Errorf("version = %q, want 1.2", res.Version)
|
||||
}
|
||||
if res.Asset.Format != "git" {
|
||||
t.Errorf("format = %q, want git", res.Asset.Format)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestResolveLTS(t *testing.T) {
|
||||
assets := []storage.Asset{
|
||||
{
|
||||
Filename: "node-v22.0.0-linux-x64.tar.gz",
|
||||
Version: "22.0.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
LTS: false,
|
||||
},
|
||||
{
|
||||
Filename: "node-v20.15.0-linux-x64.tar.gz",
|
||||
Version: "20.15.0",
|
||||
Channel: "stable",
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
Format: ".tar.gz",
|
||||
LTS: true,
|
||||
},
|
||||
}
|
||||
|
||||
t.Run("LTS selects older LTS version", func(t *testing.T) {
|
||||
res, err := Resolve(assets, Request{
|
||||
OS: "linux",
|
||||
Arch: "x86_64",
|
||||
LTS: true,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.Version != "20.15.0" {
|
||||
t.Errorf("version = %q, want 20.15.0", res.Version)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,247 +0,0 @@
|
||||
// Package uadetect identifies the requesting agent's OS, CPU architecture,
|
||||
// and libc so the server can select the correct release artifact.
|
||||
//
|
||||
// An agent identifies itself through multiple signals:
|
||||
// - The User-Agent header: Webi's bootstrap scripts send "$(uname -srm)",
|
||||
// e.g. "Darwin 23.1.0 arm64". Browsers, curl, and PowerShell send their
|
||||
// own UA strings.
|
||||
// - Query parameters: ?os=linux&arch=arm64 are an explicit declaration
|
||||
// that takes precedence over the header.
|
||||
//
|
||||
// Use [FromRequest] to detect from an HTTP request (preferred).
|
||||
// Use [Parse] to detect from a raw UA string.
|
||||
package uadetect
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
)
|
||||
|
||||
// Result holds the detected platform info from a User-Agent string.
|
||||
type Result struct {
|
||||
OS buildmeta.OS
|
||||
Arch buildmeta.Arch
|
||||
Libc buildmeta.Libc
|
||||
}
|
||||
|
||||
// FromRequest detects the agent's platform from an HTTP request.
|
||||
// Query parameters ?os and ?arch override the User-Agent header.
|
||||
func FromRequest(r *http.Request) Result {
|
||||
qOS := r.URL.Query().Get("os")
|
||||
qArch := r.URL.Query().Get("arch")
|
||||
|
||||
var ua string
|
||||
switch {
|
||||
case qOS != "" && qArch != "":
|
||||
ua = qOS + " " + qArch
|
||||
case qOS != "":
|
||||
ua = qOS
|
||||
case qArch != "":
|
||||
ua = qArch
|
||||
default:
|
||||
ua = r.Header.Get("User-Agent")
|
||||
}
|
||||
|
||||
return Parse(ua)
|
||||
}
|
||||
|
||||
// Parse extracts OS, arch, and libc from a User-Agent string.
|
||||
func Parse(ua string) Result {
|
||||
if ua == "-" {
|
||||
return Result{}
|
||||
}
|
||||
|
||||
tokens := tokenize(ua)
|
||||
|
||||
return Result{
|
||||
OS: matchOS(tokens),
|
||||
Arch: matchArch(tokens),
|
||||
Libc: matchLibc(tokens),
|
||||
}
|
||||
}
|
||||
|
||||
// tokenize splits a User-Agent into lowercase tokens for matching.
|
||||
// Splits on whitespace, '/', and ';', since UAs come in various forms:
|
||||
//
|
||||
// "Darwin 23.1.0 arm64" (uname -srm)
|
||||
// "PowerShell/7.3.0" (PowerShell)
|
||||
// "MS AMD64" (Windows shorthand)
|
||||
// "Macintosh; Intel Mac OS X 10_15_7" (browser)
|
||||
func tokenize(ua string) []string {
|
||||
// Strip xnu kernel info that can mislead arch detection under Rosetta.
|
||||
// "xnu-7195.60.75~1/RELEASE_ARM64_T8101" contains ARM64 even when
|
||||
// running as x86_64. This only appears in verbose uname output.
|
||||
if i := strings.Index(ua, "xnu-"); i >= 0 {
|
||||
end := strings.IndexByte(ua[i:], ' ')
|
||||
if end < 0 {
|
||||
ua = ua[:i]
|
||||
} else {
|
||||
ua = ua[:i] + ua[i+end:]
|
||||
}
|
||||
}
|
||||
|
||||
return strings.FieldsFunc(strings.ToLower(ua), func(r rune) bool {
|
||||
return r == ' ' || r == '/' || r == ';' || r == '\t'
|
||||
})
|
||||
}
|
||||
|
||||
// matchOS identifies the operating system from tokens.
|
||||
// Order matters: Android before Linux, Linux before Windows (for WSL).
|
||||
func matchOS(tokens []string) buildmeta.OS {
|
||||
has := func(s string) bool {
|
||||
for _, t := range tokens {
|
||||
if strings.Contains(t, s) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Android must be checked before Linux.
|
||||
if has("android") {
|
||||
return buildmeta.OSAndroid
|
||||
}
|
||||
|
||||
if has("darwin") || has("macos") || has("macintosh") || has("iphone") || has("ios") || has("ipad") {
|
||||
return buildmeta.OSDarwin
|
||||
}
|
||||
// "mac" alone (not in "macintosh" which is already matched)
|
||||
for _, t := range tokens {
|
||||
if t == "mac" {
|
||||
return buildmeta.OSDarwin
|
||||
}
|
||||
}
|
||||
|
||||
// FreeBSD before Linux (both are POSIX, but FreeBSD never reports "linux").
|
||||
if has("freebsd") {
|
||||
return buildmeta.OSFreeBSD
|
||||
}
|
||||
|
||||
// Linux before Windows because WSL UAs contain both "linux" and "microsoft".
|
||||
// But exclude Cygwin/Msys/MINGW which report Linux-like strings on Windows.
|
||||
if has("linux") && !has("cygwin") && !has("msysgit") && !has("msys") && !has("mingw") {
|
||||
return buildmeta.OSLinux
|
||||
}
|
||||
|
||||
// Cygwin, Msys, and MINGW are Windows environments.
|
||||
if has("windows") || has("win32") || has("microsoft") || has("powershell") ||
|
||||
has("cygwin") || has("msys") || has("mingw") {
|
||||
return buildmeta.OSWindows
|
||||
}
|
||||
for _, t := range tokens {
|
||||
if t == "ms" || t == "win" {
|
||||
return buildmeta.OSWindows
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: curl and wget imply a POSIX system, almost always Linux.
|
||||
if has("curl") || has("wget") {
|
||||
return buildmeta.OSLinux
|
||||
}
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
// matchArch identifies the CPU architecture from tokens.
|
||||
// More specific patterns are checked before less specific ones.
|
||||
func matchArch(tokens []string) buildmeta.Arch {
|
||||
has := func(s string) bool {
|
||||
for _, t := range tokens {
|
||||
if strings.Contains(t, s) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
exact := func(s string) bool {
|
||||
for _, t := range tokens {
|
||||
if t == s {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ARM 64-bit (most specific first)
|
||||
if has("aarch64") || has("arm64") || has("armv8") {
|
||||
return buildmeta.ArchARM64
|
||||
}
|
||||
|
||||
// ARM 32-bit variants
|
||||
if has("armv7") || has("arm32") {
|
||||
return buildmeta.ArchARMv7
|
||||
}
|
||||
if has("armv6") {
|
||||
return buildmeta.ArchARMv6
|
||||
}
|
||||
// Bare "arm" without a version qualifier → armv6 (conservative).
|
||||
if exact("arm") {
|
||||
return buildmeta.ArchARMv6
|
||||
}
|
||||
|
||||
// POWER (check before generic 64-bit)
|
||||
if has("ppc64le") {
|
||||
return buildmeta.ArchPPC64LE
|
||||
}
|
||||
if has("ppc64") {
|
||||
return buildmeta.ArchPPC64
|
||||
}
|
||||
|
||||
// s390x (IBM Z)
|
||||
if has("s390x") {
|
||||
return buildmeta.ArchS390X
|
||||
}
|
||||
|
||||
// RISC-V
|
||||
if has("riscv64") {
|
||||
return buildmeta.ArchRISCV64
|
||||
}
|
||||
|
||||
// MIPS (check before generic 64-bit)
|
||||
if has("mips64") {
|
||||
return buildmeta.ArchMIPS64
|
||||
}
|
||||
if has("mips") {
|
||||
return buildmeta.ArchMIPS
|
||||
}
|
||||
|
||||
// x86-64
|
||||
if has("x86_64") || has("amd64") || exact("x64") {
|
||||
return buildmeta.ArchAMD64
|
||||
}
|
||||
|
||||
// x86 32-bit (after x86_64 to avoid false match)
|
||||
if has("i386") || has("i686") || exact("x86") {
|
||||
return buildmeta.ArchX86
|
||||
}
|
||||
|
||||
return ""
|
||||
}
|
||||
|
||||
// matchLibc identifies the C library from tokens.
|
||||
func matchLibc(tokens []string) buildmeta.Libc {
|
||||
has := func(s string) bool {
|
||||
for _, t := range tokens {
|
||||
if strings.Contains(t, s) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
if has("musl") {
|
||||
return buildmeta.LibcMusl
|
||||
}
|
||||
// Don't match "microsoft" — it appears in WSL kernel version strings
|
||||
// (e.g. "5.15.146.1-microsoft-standard-WSL2") and doesn't indicate MSVC.
|
||||
if has("msvc") || has("windows") {
|
||||
return buildmeta.LibcMSVC
|
||||
}
|
||||
if has("gnu") || has("glibc") || has("linux") {
|
||||
return buildmeta.LibcGNU
|
||||
}
|
||||
|
||||
return buildmeta.LibcNone
|
||||
}
|
||||
@@ -1,190 +0,0 @@
|
||||
package uadetect_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"testing"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
"github.com/webinstall/webi-installers/internal/uadetect"
|
||||
)
|
||||
|
||||
func TestOS(t *testing.T) {
|
||||
tests := []struct {
|
||||
ua string
|
||||
want buildmeta.OS
|
||||
}{
|
||||
// uname -srm style
|
||||
{"Darwin 23.1.0 arm64", buildmeta.OSDarwin},
|
||||
{"Darwin 20.2.0 x86_64", buildmeta.OSDarwin},
|
||||
{"Linux 6.1.0-18-amd64 x86_64", buildmeta.OSLinux},
|
||||
{"Linux 5.15.0 aarch64", buildmeta.OSLinux},
|
||||
|
||||
// WSL: Linux, not Windows (contains "microsoft" in kernel release)
|
||||
{"Linux 5.15.146.1-microsoft-standard-WSL2 x86_64", buildmeta.OSLinux},
|
||||
|
||||
// Windows
|
||||
{"MS AMD64", buildmeta.OSWindows},
|
||||
{"PowerShell/7.3.0", buildmeta.OSWindows},
|
||||
{"Microsoft Windows 10.0.19045", buildmeta.OSWindows},
|
||||
|
||||
// Msys/MINGW/Cygwin → Windows
|
||||
{"webi/curl x86_64/unknown Msys/MINGW64_NT-10.0-19045/3.5.7-463ebcdc.x86_64 libc", buildmeta.OSWindows},
|
||||
{"webi/curl+wget x86_64/unknown Msys/MSYS_NT-10.0-26200/3.6.6-1cdd4371.x86_64 libc", buildmeta.OSWindows},
|
||||
{"webi/curl x86_64/unknown Cygwin/CYGWIN_NT-10.0/2.10.0(0.325/5/3) libc", buildmeta.OSWindows},
|
||||
|
||||
// FreeBSD
|
||||
{"webi/curl amd64/unknown FreeBSD/14.3-RELEASE-p8 libc", buildmeta.OSFreeBSD},
|
||||
|
||||
// Android before Linux
|
||||
{"Android 13 aarch64", buildmeta.OSAndroid},
|
||||
{"webi/curl aarch64/unknown Android/Linux/6.6.77-android15-8 libc", buildmeta.OSAndroid},
|
||||
|
||||
// WSL: Linux, not Windows (kernel contains "microsoft")
|
||||
{"webi/curl+wget x86_64/unknown GNU/Linux/5.15.146.1-microsoft-standard-WSL2 libc", buildmeta.OSLinux},
|
||||
|
||||
// Browser-style
|
||||
{"Macintosh; Intel Mac OS X 10_15_7", buildmeta.OSDarwin},
|
||||
|
||||
// Minimal agents → assume Linux
|
||||
{"curl/8.1.2", buildmeta.OSLinux},
|
||||
{"wget/1.21", buildmeta.OSLinux},
|
||||
|
||||
// Explicit unknown
|
||||
{"-", ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.ua, func(t *testing.T) {
|
||||
got := uadetect.Parse(tt.ua).OS
|
||||
if got != tt.want {
|
||||
t.Errorf("Parse(%q).OS = %q, want %q", tt.ua, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestArch(t *testing.T) {
|
||||
tests := []struct {
|
||||
ua string
|
||||
want buildmeta.Arch
|
||||
}{
|
||||
{"Darwin 23.1.0 arm64", buildmeta.ArchARM64},
|
||||
{"Linux 6.1.0 aarch64", buildmeta.ArchARM64},
|
||||
{"Linux 5.4.0 x86_64", buildmeta.ArchAMD64},
|
||||
{"MS AMD64", buildmeta.ArchAMD64},
|
||||
{"Linux 5.10.0 armv7l", buildmeta.ArchARMv7},
|
||||
{"Linux 5.10.0 armv6l", buildmeta.ArchARMv6},
|
||||
{"Linux 5.4.0 ppc64le", buildmeta.ArchPPC64LE},
|
||||
{"webi/curl+wget s390x/unknown GNU/Linux/6.4.0-150700.53.6-default libc", buildmeta.ArchS390X},
|
||||
|
||||
// FreeBSD uses "amd64" not "x86_64"
|
||||
{"webi/curl amd64/unknown FreeBSD/14.3-RELEASE-p8 libc", buildmeta.ArchAMD64},
|
||||
|
||||
// Rosetta: xnu kernel info says ARM64 but actual arch is x86_64
|
||||
{"Darwin 20.2.0 Darwin Kernel Version 20.2.0; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 x86_64", buildmeta.ArchAMD64},
|
||||
|
||||
{"-", ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.ua, func(t *testing.T) {
|
||||
got := uadetect.Parse(tt.ua).Arch
|
||||
if got != tt.want {
|
||||
t.Errorf("Parse(%q).Arch = %q, want %q", tt.ua, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLibc(t *testing.T) {
|
||||
tests := []struct {
|
||||
ua string
|
||||
want buildmeta.Libc
|
||||
}{
|
||||
{"Linux 6.1.0 x86_64 musl", buildmeta.LibcMusl},
|
||||
{"Linux 6.1.0 x86_64 gnu", buildmeta.LibcGNU},
|
||||
{"Linux 6.1.0 x86_64 linux", buildmeta.LibcGNU},
|
||||
{"MS AMD64 msvc", buildmeta.LibcMSVC},
|
||||
{"Microsoft Windows", buildmeta.LibcMSVC},
|
||||
{"Darwin 23.1.0 arm64", buildmeta.LibcNone},
|
||||
|
||||
// WSL: kernel version contains "microsoft" but libc is gnu, not msvc
|
||||
{"webi/curl+wget x86_64/unknown GNU/Linux/5.15.146.1-microsoft-standard-WSL2 libc", buildmeta.LibcGNU},
|
||||
|
||||
{"-", ""},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.ua, func(t *testing.T) {
|
||||
got := uadetect.Parse(tt.ua).Libc
|
||||
if got != tt.want {
|
||||
t.Errorf("Parse(%q).Libc = %q, want %q", tt.ua, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFromRequest(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
ua string // User-Agent header
|
||||
query string // raw query string
|
||||
wantOS buildmeta.OS
|
||||
wantAr buildmeta.Arch
|
||||
}{
|
||||
{
|
||||
name: "UA header only",
|
||||
ua: "Darwin 23.1.0 arm64",
|
||||
wantOS: buildmeta.OSDarwin,
|
||||
wantAr: buildmeta.ArchARM64,
|
||||
},
|
||||
{
|
||||
name: "query params override UA",
|
||||
ua: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
|
||||
query: "os=linux&arch=aarch64",
|
||||
wantOS: buildmeta.OSLinux,
|
||||
wantAr: buildmeta.ArchARM64,
|
||||
},
|
||||
{
|
||||
name: "os param only",
|
||||
ua: "curl/8.1.2",
|
||||
query: "os=windows",
|
||||
wantOS: buildmeta.OSWindows,
|
||||
},
|
||||
{
|
||||
name: "arch param only",
|
||||
ua: "curl/8.1.2",
|
||||
query: "arch=arm64",
|
||||
wantAr: buildmeta.ArchARM64,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
req, _ := http.NewRequest("GET", "http://example.com/api?"+tt.query, nil)
|
||||
if tt.ua != "" {
|
||||
req.Header.Set("User-Agent", tt.ua)
|
||||
}
|
||||
got := uadetect.FromRequest(req)
|
||||
if tt.wantOS != "" && got.OS != tt.wantOS {
|
||||
t.Errorf("OS = %q, want %q", got.OS, tt.wantOS)
|
||||
}
|
||||
if tt.wantAr != "" && got.Arch != tt.wantAr {
|
||||
t.Errorf("Arch = %q, want %q", got.Arch, tt.wantAr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFullParse(t *testing.T) {
|
||||
r := uadetect.Parse("Darwin 23.1.0 arm64")
|
||||
if r.OS != buildmeta.OSDarwin {
|
||||
t.Errorf("OS = %q, want %q", r.OS, buildmeta.OSDarwin)
|
||||
}
|
||||
if r.Arch != buildmeta.ArchARM64 {
|
||||
t.Errorf("Arch = %q, want %q", r.Arch, buildmeta.ArchARM64)
|
||||
}
|
||||
if r.Libc != buildmeta.LibcNone {
|
||||
t.Errorf("Libc = %q, want %q", r.Libc, buildmeta.LibcNone)
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
#!/bin/sh
|
||||
# shellcheck disable=SC2029,SC2088
|
||||
set -e
|
||||
set -u
|
||||
|
||||
@@ -10,13 +11,13 @@ g_out="agents/tmp/${g_bin}"
|
||||
g_remote_bin="~/bin/${g_bin}"
|
||||
|
||||
case "${g_host}" in
|
||||
beta.webi.sh) g_remote_conf="~/srv/beta.webinstall.dev/installers/" ;;
|
||||
next.webi.sh) g_remote_conf="~/srv/next.webinstall.dev/installers/" ;;
|
||||
*) g_remote_conf="~/srv/webid/installers/" ;;
|
||||
beta.webi.sh) g_remote_conf="~/srv/beta.webinstall.dev/installers/" ;;
|
||||
next.webi.sh) g_remote_conf="~/srv/next.webinstall.dev/installers/" ;;
|
||||
*) g_remote_conf="~/srv/webid/installers/" ;;
|
||||
esac
|
||||
|
||||
fn_build() {
|
||||
b_version="$(git describe --tags --always 2> /dev/null || echo '0.0.0-dev')"
|
||||
b_version="$(git describe --tags --always 2>/dev/null || echo '0.0.0-dev')"
|
||||
b_commit="$(git rev-parse --short HEAD)"
|
||||
b_date="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
b_ldflags="-X main.version=${b_version} -X main.commit=${b_commit} -X main.date=${b_date}"
|
||||
@@ -28,7 +29,7 @@ fn_build() {
|
||||
|
||||
fn_deploy() {
|
||||
printf 'Stopping %s on %s...\n' "${g_bin}" "${g_host}"
|
||||
ssh "${g_host}" "~/.local/bin/serviceman stop ${g_bin}" 2> /dev/null || true
|
||||
ssh "${g_host}" "~/.local/bin/serviceman stop ${g_bin}" 2>/dev/null || true
|
||||
|
||||
printf 'Uploading binary...\n'
|
||||
scp "${g_out}" "${g_host}:${g_remote_bin}"
|
||||
|
||||
Reference in New Issue
Block a user