Commit Graph

1281 Commits

Author SHA1 Message Date
AJ ONeal
efda7c60aa add gittag conf for vim plugins, alias confs, fix psql as own package
Vim plugins with gittag source:
- vim-airline, vim-airline-themes, vim-ale, vim-devicons, vim-go
- vim-nerdtree, vim-prettier, vim-rust, vim-sensible, vim-shfmt
- vim-syntastic

rust.vim is a directory symlink to vim-rust, so it shares the same
releases.conf automatically.

Alias confs (alias_of):
- postgresql → postgres
- postgresql-client, postgres-client → psql
- mariadb-server, mariadbd → mariadb
- gnupg → gpg, iterm → iterm2, ziglang → zig
- trippy → trip, powershell → pwsh

Fix: psql is its own package (postgres client), not an alias of
postgres (server). Both use the same GitHub repo
(bnnanet/postgresql-releases) but install different binaries.
2026-03-09 23:24:35 -06:00
AJ ONeal
7f0c92e262 add releases.conf for all remaining packages and wire new fetchers
New fetcher packages:
- chromedist: Chrome for Testing API (googlechromelabs.github.io)
- gpgdist: SourceForge RSS for GPG macOS
- mariadbdist: MariaDB downloads REST API

New releases.conf files for:
- GitHub: aliasman, awless, duckdns.sh, hugo-extended, kubens, rg, postgres
- gittag: vim-commentary, vim-zig
- gitea: pathman
- chromedist: chromedriver
- gpgdist: gpg
- mariadbdist: mariadb
- nodedist: node

Alias support (alias_of key):
- golang → go, dashd → dashcore, psql → postgres, zig.vim → vim-zig
- Aliases skip fetching and share cache with their target

Every package with a releases.js now has a releases.conf (except the
dead macos package). fetchraw dispatches to all 13 source types.
2026-03-09 22:48:11 -06:00
AJ ONeal
990221454e add fetchers for non-GitHub release sources
New fetcher packages:
- golang: golang.org/dl/?mode=json&include=all
- zigdist: ziglang.org/download/index.json
- flutterdist: Google Storage per-OS release indexes
- iterm2dist: scrapes iterm2.com/downloads.html
- hashicorp: releases.hashicorp.com/{product}/index.json
- juliadist: julialang-s3.julialang.org/bin/versions.json

Each follows the same iter.Seq2 pattern as the existing nodedist/github
fetchers. Added releases.conf files for all six packages and wired them
into cmd/fetchraw.

Fixed latest-version detection for sources that return unordered data
(hashicorp, zigdist, juliadist) by comparing all versions with lexver
instead of taking the first stable one found.
2026-03-09 22:39:16 -06:00
AJ ONeal
c4a358e5a5 add example releases.conf and skip _-prefixed dirs in fetchraw
The discover() function now skips directories starting with _ (like
_example, _webi, _common) so infrastructure dirs aren't treated as
packages to fetch.
2026-03-09 22:33:15 -06:00
AJ ONeal
caae40df27 ref(fetchraw): read from releases.conf instead of hardcoded list
Discovers packages by globbing {confDir}/*/releases.conf. Adding a
new package is now just creating a conf file — no Go code changes.
Dispatches to the right fetcher based on source= (github, nodedist).
2026-03-09 22:29:03 -06:00
AJ ONeal
b98cbc975c feat: add releases.conf files and installerconf parser
Simple key=value config per package declaring the fetch source and
its parameters. Greppable, no dependencies needed to parse.

  grep 'source = github' */releases.conf
  grep 'owner = therootcompany' */releases.conf

70 packages configured. installerconf package provides the reader.
fetchraw will be updated to read these instead of a hardcoded list.
2026-03-09 22:27:26 -06:00
AJ ONeal
2c3b21a5ae add releases.conf for all GitHub and nodedist packages
Declarative key=value config files that specify the release source
(github or nodedist), owner/repo, and optional tag_prefix for
monorepo packages. These replace the per-package releases.js logic
for the Go rewrite.
2026-03-09 22:27:02 -06:00
AJ ONeal
69a23f3592 feat: add audit log, merge strategy, and all GitHub packages
- rawcache: add Merge() that skips unchanged releases, logs added/
  changed events to an append-only JSONL audit log with SHA-256
- rawcache: drop .json extension from filenames — raw cache stores
  opaque bytes (upstream may be JSON, CSV, XML, or bespoke)
- fetchraw: add all 68 GitHub packages, use Merge instead of Put
- fetchraw: log format shows +added ~changed =skipped
2026-03-09 22:19:11 -06:00
AJ ONeal
215c996eb7 fix(fetchraw): use merge strategy instead of full replace
Put directly into the active slot instead of BeginRefresh. Existing
releases are skipped (Has check), new ones are added, _latest is
only updated if the candidate is newer. Safe to run repeatedly —
backports and delayed releases accumulate without losing history.
2026-03-09 22:12:31 -06:00
AJ ONeal
c8e5a007f5 feat: add fetchraw tool for populating raw release cache
Fetches complete release histories from upstream APIs and stores
them in rawcache. Supports GitHub (with pagination, auth, monorepo
tag prefix filtering) and Node.js dist API (official + unofficial
as separate caches to avoid version collisions).

Tested with: node-official (834), node-unofficial (387),
hugo (365), caddy (134), monorel (3).
2026-03-09 22:11:05 -06:00
AJ ONeal
bdf7ad4a56 docs: update GO_WEBI.md with current progress and design decisions
Reflect completed work (all fetchers, rawcache, classify, platlatest,
CompatArches), update repo layout to match actual packages, document
the fallback/compatibility design (classifier is 80/20 default,
per-installer config is the authority), add open questions for CPU
micro-arch detection and installer config format.
2026-03-09 22:07:26 -06:00
AJ ONeal
5dba2de20b feat(buildmeta): add CompatArches and universal binary arch types
CompatArches returns what a given OS+arch can execute — OS-level
facts like Rosetta 2 (darwin arm64 runs x86_64), Windows ARM
emulation, and x86-64 micro-arch backward compat. Also adds
ArchUniversal1 (PPC+x86) and ArchUniversal2 (x86_64+ARM64).

Per-package/per-version overrides (libc compat, nonstandard naming)
remain the installer config's responsibility.
2026-03-09 21:57:43 -06:00
AJ ONeal
1253fcd671 ref: remove universal fallback chains from buildmeta and platlatest
Arch and libc fallbacks are not universal — they depend on the OS,
the package, and even the version. ARM64 on macOS/Windows can run
x64 (Rosetta/emulation) but not on Linux. Musl can be static or
dynamically linked depending on the package version. Windows GNU
may or may not need mingw.

These rules belong in per-installer config, not in shared types.
platlatest stays as a simple fact store (triplet → version).
Resolution with fallbacks will be the caller's job.
2026-03-09 21:50:10 -06:00
AJ ONeal
34cfe32492 feat: add arch/libc fallback chains and version waterfall resolution
Prefer latest version over best CPU match. An amd64v4 machine gets
v2.0.0 (baseline only) instead of v1.0.0 (which had a v4 build)
because recency beats specificity.

- buildmeta: add amd64v2/v3/v4 micro-levels, ArchFallbacks, LibcFallbacks
- classify: detect micro-arch levels, treat Windows "arm" as ARM64
- platlatest: add Resolve() that walks fallback chains picking newest
2026-03-09 21:44:06 -06:00
AJ ONeal
1e26a3e5ec feat: add classify and platlatest packages
classify extracts OS, arch, libc, and format from release asset
filenames using regex pattern matching with priority ordering
(x86_64 before x86, arm64 before armv7, etc.).

platlatest tracks the newest release version per build target
(OS+arch+libc triplet) to handle the common case where Windows
or macOS releases lag behind Linux by several versions.
2026-03-09 21:33:59 -06:00
AJ ONeal
ae39837145 feat(rawcache): add double-buffered raw release cache
Stores one JSON file per release, named by tag. Supports:
- Incremental updates: atomic writes to the active slot
- Full refresh: write to standby slot, atomic symlink swap
- O(1) existence check and latest-tag lookup
2026-03-09 21:28:03 -06:00
AJ ONeal
574e5be929 feat(releases): add source archive fetchers for GitHub, Gitea, GitLab
For packages installed from auto-generated source tarballs rather
than uploaded binary assets (shell scripts, vim plugins, etc.).
Each delegates to its respective forge fetcher — the distinction
is organizational, signaling which fields the consumer should use.
2026-03-09 21:10:18 -06:00
AJ ONeal
e1bd6bb82f ref(gitea): rewrite as standalone fetcher, not a githubish wrapper
Gitea's API is similar to GitHub's but not identical (different URL
prefix, limit vs per_page, token auth header). Give it its own types
and pagination logic rather than coupling through githubish.
2026-03-09 21:06:58 -06:00
AJ ONeal
fd9d5ca080 feat(releases): add GitLab release fetcher
GitLab's API differs from GitHub: different URL pattern
(/api/v4/projects/:id/releases), nested asset structure
(sources + links), page/per_page pagination with X-Total-Pages
header, and PRIVATE-TOKEN auth.
2026-03-09 21:05:51 -06:00
AJ ONeal
6576ca65b6 feat(githubish): add TarballURL and ZipballURL to Release
Some packages (shell scripts, vim plugins) use the auto-generated
source archives rather than uploaded binary assets. These URLs are
already in the API response — just needed to be deserialized.
2026-03-09 20:57:01 -06:00
AJ ONeal
1116dd3935 feat(releases): add Gitea and git-tag fetchers
gitea: thin wrapper over githubish that appends /api/v1 to the base URL.

gittag: clones/fetches a bare repo, lists version-like tags with
commit metadata, includes HEAD. For packages installed by cloning
(vim plugins, shell scripts) rather than downloading binaries.
2026-03-09 20:55:32 -06:00
AJ ONeal
befb1fb425 feat(releases): add GitHub-compatible release fetcher with pagination
githubish: generic fetcher for any GitHub-compatible API (GitHub,
Gitea, Forgejo). Paginates via Link headers, supports Bearer auth.
Returns raw API data with no transformation.

github: thin wrapper that sets the base URL to api.github.com.
2026-03-08 23:20:39 -06:00
AJ ONeal
b7e3fe69ad feat(releases): add Node.js distribution fetchers
nodedist: generic fetcher for any Node.js-style dist index.json API.
Returns raw API entries with no transformation or normalization.
Uses iter.Seq2 for a paginated interface consistent across sources.

node: calls nodedist twice — official builds and unofficial builds
(musl, loong64, etc.) — yielding one batch per source.
2026-03-08 23:12:54 -06:00
AJ ONeal
4f3bdd7d58 feat(uadetect): add FromRequest for full agent detection
The user agent identifies itself through multiple signals — the
User-Agent header and query parameters (?os, ?arch). FromRequest
unifies both, with explicit query params taking precedence.
2026-03-08 22:58:59 -06:00
AJ ONeal
43ab591061 ref(internal): rewrite buildmeta, uadetect, httpclient from scratch
buildmeta: remove premature Release/PackageMeta structs and
ChannelNames slice — keep only the shared vocabulary types.

uadetect: replace regex-based matching with token-based matching.
Split UA on whitespace/slash/semicolon, match lowercase tokens.
Strip xnu kernel info for Rosetta. Single Parse() entry point.

httpclient: return plain *http.Client from New(). Make Do() and Get()
free functions. Only retry idempotent methods (GET/HEAD).
2026-03-08 22:58:58 -06:00
AJ ONeal
1374bca46b feat(lexver): add Original field and ExtraSort tiebreaker
Original preserves the upstream tag as the releaser published it (e.g.
"REL_17_0"), while Raw holds Webi's normalized form ("17.0").

ExtraSort is an opaque string for package-specific ordering where Nums
alone can't capture sort order (e.g. flutter "2.3.0-16.0.pre"). Set by
release-fetcher code using zero-padded strings or whatever works for
that package.
2026-03-08 22:58:58 -06:00
AJ ONeal
c1a40cebf3 ref(lexver): support arbitrary version depth, use time.Time for dates
Versions aren't always semver — chromedriver uses 4 parts, gpg uses 4
parts, atomicparsley uses dates. Replaced fixed Major/Minor/Patch/Build
fields with a Nums slice. Date is now time.Time for minute-level
precision.

Also adds ODDITIES.md cataloging non-standard version formats across
Webi packages for future reference.
2026-03-08 22:58:58 -06:00
AJ ONeal
66f9f5f5fe ref(lexver): replace Build field with Date for tiebreaking
Build is often a hash with no ordering meaning. When release dates are
known, they're a better tiebreaker for versions with the same
major.minor.patch. Date is only used when both sides have one.
2026-03-08 22:58:58 -06:00
AJ ONeal
c2c54e54ea ref(lexver): rewrite as structured type with Compare function
Instead of encoding versions as padded strings (a JS-ism), parse into a
Version struct and compare fields directly. Pre-releases sort before
their corresponding stable release via channel comparison.
2026-03-08 22:58:58 -06:00
AJ ONeal
1fc595f265 chore: set go version to 1.26.1 2026-03-08 22:58:58 -06:00
AJ ONeal
d1b2b3d1e5 docs(GO_WEBI): mark Phase 0 complete, fix Go version to 1.24+ 2026-03-08 22:58:58 -06:00
AJ ONeal
ed377c93c6 docs: rewrite package comments to focus on what/why, not how
Each package doc now explains what problem it solves and why it exists,
with the public interface as the only "how" detail. Implementation
notes removed from doc comments.
2026-03-08 22:58:58 -06:00
AJ ONeal
cf9dd4d2e2 feat: add Phase 0 foundation packages for Go rewrite
- internal/buildmeta: canonical constants for OS, arch, libc, format, channel
- internal/lexver: version string → lexicographically sortable string
- internal/uadetect: User-Agent → OS/arch/libc detection
- internal/httpclient: resilient net/http client with retry and backoff
- go.mod: initialize module (stdlib only, no dependencies)
2026-03-08 22:58:58 -06:00
AJ ONeal
0acc6b06aa docs(GO_WEBI): webid checks storage per-request, no package registry
Looks up releases by name in filesystem and/or Postgres on each request.
If releases exist for that name, it's valid. No fsnotify, no scanning.
2026-03-08 22:58:58 -06:00
AJ ONeal
e83a9abaeb docs(GO_WEBI): webicached is rebuilt for new packages, not just restarted 2026-03-08 22:58:58 -06:00
AJ ONeal
60961494d5 docs(GO_WEBI): webicached requires restart for new packages
webid still discovers new packages from storage without restart.
webicached knows its package set at startup and periodically refreshes
releases to both Postgres and filesystem.
2026-03-08 22:58:58 -06:00
AJ ONeal
52d9ccc744 docs(GO_WEBI): capture hot-add constraint for new installers
Adding a new package must not require restarting either server.
webicached watches for new packages, webid discovers them from storage.
2026-03-08 22:58:58 -06:00
AJ ONeal
0d4ca55e6f docs: add GO_WEBI.md planning doc for Go rewrite
Captures the full migration plan: architecture, API surface inventory,
storage double-buffer design, incremental migration phases, and key
design decisions. Not a straight port — redesigning internals while
preserving the public API contract.
2026-03-08 22:58:58 -06:00
AJ ONeal
3e2e7f2f65 feat(monorel): add installer for monorepo release tool
Adds releases.js, install.sh, install.ps1, and README.md for monorel,
a Go monorepo release tool from therootcompany/golib. Filters monorepo
releases by tools/monorel/ prefix and auto-installs prerequisites
(git, gh, goreleaser).
2026-03-08 22:50:34 -06:00
AJ ONeal
ca81127b93 fix(docs): fix typos in goreleaser, ssh-authorize, and node READMEs
- goreleaser: "you should the git tag" → "you should see the git tag"
- ssh-authorize: "will to do" → "will be able to do"
- node: "jhint" → "jshint"
2026-03-08 19:53:26 -06:00
AJ ONeal
3c8b66be55 docs: add AGENTS.md with conventions and design principles 2026-03-08 19:53:26 -06:00
AJ ONeal
8f9b9da4a3 chore: npm run fmt 2026-03-08 19:38:49 -06:00
bry-val
81ffcf3182 doc(deno): update Hello World example to use deno.com URL
Signed-off-by: bry-val <94031627+bry-val@users.noreply.github.com>
2025-03-06 20:48:54 +00:00
AJ ONeal
3d1a75102f ref(koji): keep backwards-compat 2025-02-20 10:56:41 +00:00
Finley Thomalla
e6b3aec8c0 docs(koji): fix mistakes, improve 2025-02-20 10:48:52 +00:00
Finley Thomalla
ed8058deb8 fix(koji): update binary path
The release process of koji has been updated a while ago, resulting in the binary not being nested in the archive anymore.

Fixes cococonscious/koji#137
2025-02-20 10:48:51 +00:00
AJ ONeal
480169beac fix(terramate): link both terramate and terramate-ls 2025-02-03 21:18:43 +00:00
AJ ONeal
625168156f fix(terramate): don't exclude package files 2025-02-03 21:18:43 +00:00
Michael Dubner
75e39c54a2 fix: add '386' and 'i386' to tab regexp (fixes GH-941) 2025-01-29 22:59:56 +00:00
AJ ONeal
cac2e62da8 feat(mariadb): add mysql and mariadb-server aliases 2025-01-26 00:35:51 +00:00