vim-ale/AGENTS.md

# Webi Installers — Agent Guide

Webi installs dev tools to `~/.local/` without sudo. Each installer is a small
package of 3-4 files. This guide tells you how to create and modify them.

## Domains

| Environment | Domains                                          |
| ----------- | ------------------------------------------------ |
| Production  | webi.sh, webi.ms, webinstall.dev                 |
| Beta        | beta.webi.sh, beta.webi.ms, beta.webinstall.dev  |
| Next        | next.webi.sh, next.webi.ms, next.webinstall.dev  |

- **webi.sh** — POSIX shell installer: `curl https://webi.sh/node | sh`
- **webi.ms** — PowerShell installer: `curl.exe https://webi.ms/node | powershell`
- **webinstall.dev** — canonical domain, serves both shell scripts and the
  browser-facing cheat sheet pages

The domain controls the default script format. You can force the same behavior
on any domain via User-Agent:
- `curl -A "MS" ...` — triggers PowerShell output (same as webi.ms)
- `curl ...` (default UA) — triggers POSIX shell output (same as webi.sh)
- `curl -A "$(uname -srm)" ...` — once the API is activated, the full
  `uname -srm` string (e.g. `Linux 6.1.0 x86_64`) guides OS/arch selection

## Why Webi Exists

Webi makes tool installation trivially repeatable for people who aren't
sysadmins — freelance clients, junior devs, anyone who shouldn't have to care
about PATH, permissions, or platform differences. Three things matter:

1. **Install without friction.** No sudo, no manual PATH edits, no "necessary
   but unimportant" steps leaking into the experience.
2. **Know where things are.** The Files section tells you exactly what got
   created or modified. Nothing should be mysterious.
3. **Copy-paste recipes.** The cheat sheet is what you'd send someone less
   experienced than yourself instead of a project's full README — scannable,
   concrete, easy to reference by name.

## Quick Start: Adding a New Installer

1. Identify the **package type** (see [Categories](#categories) below)
2. Find an existing installer of the same type to use as a template
3. Create `<name>/releases.js`, `install.sh`, `install.ps1`, `README.md`
4. Test with the command in [Testing releases.js](#testing-releasesjs)
5. Run formatters before committing (see [Code Style](#code-style))

## Directory Layout

```
<package-name>/
  README.md          # YAML frontmatter + docs
  releases.js        # Fetches release metadata (Node.js)
  install.sh         # POSIX shell installer (macOS/Linux)
  install.ps1        # PowerShell installer (Windows) — optional
```

Key infrastructure directories (do not modify without good reason):

- `_webi/` — bootstrap templates, `transform-releases.js` (API endpoint),
  `builds-cacher.js` (reads cache JSON), `serve-installer.js` (installer
  scripts)
- `_common/` — shared JS fetcher libraries (being phased out — Go daemon now
  fetches upstream)
- `_example/` — canonical template for new packages
- `_examples/` — specialized templates (goreleaser, xz-compressed)

## Categories

Ref: <https://github.com/webinstall/webi-installers/issues/412>

| Type  | Description                            | Template to copy |
| ----- | -------------------------------------- | ---------------- |
| `bin` | Single binary in tar/zip               | `koji`, `delta`  |
| `bin` | Single bare binary (no archive)        | `arc`, `shfmt`   |
| `bin` | Goreleaser-style archives              | `keypairs`       |
| 📦    | Self-contained package (bin/man/share) | `node`, `go`     |
| 📂    | Multiple binaries/scripts              | `pg`             |
| 🔗    | Alias/redirect to another package      | `ripgrep` → `rg` |
| 📝    | Bespoke / custom install               | `rustlang`       |

## Data Architecture

There are two data paths. Both read from pre-generated cache — the Node.js
server does NOT fetch upstream APIs.

```
API path (JSON/TAB output):
  Request → transform-releases.js → ~/.cache/webi/legacy/{pkg}.json → filter + sort
  Vocabulary: macos, amd64, arm64, armv7l (API vocabulary)
  normalize.js: REMOVED — cache provides all fields directly

Installer path (bash/ps1 script output):
  Request → serve-installer.js → builds.js → builds-cacher.js
  Vocabulary: darwin, x86_64, aarch64 (build-classifier vocabulary)
```

Cache is generated by the Go daemon (`webicached`) and stored flat in
`~/.cache/webi/legacy/{pkg}.json` (resolved via `Os.homedir()` in the Node
readers — no `_cache` symlink, no month subdirectory). Each file contains:
- Top-level summary arrays: `oses`, `arches`, `libcs`, `formats`
- `releases` array with pre-classified fields: `os`, `arch`, `libc`, `ext`,
  `version`, `channel`, `download`
- `download` template string

### Canonical vocabulary (cache and API)

**OS**: `macos` (not darwin), `linux`, `windows`, `freebsd`, etc.
**Arch**: `amd64` (not x86_64), `arm64` (not aarch64), `armv7l` (not armv7)
**Libc**: `none` (never empty), `gnu`, `musl`, `msvc`
**Ext**: `tar.gz`, `zip`, `exe` (no leading dot; `exe` for bare binaries)

## releases.js (legacy — being phased out)

The `{pkg}/releases.js` files previously fetched upstream release metadata.
These are being replaced by the Go cache daemon. Existing files are kept as
documentation of release sources but are no longer called by the server.

### Testing the API (current)

```sh
curl -sS 'https://beta.webi.sh/api/releases/<name>.json?os=macos&arch=arm64&limit=5' | jq .
```

Verify: versions present, correct OS/arch vocabulary, download URLs resolve.

## install.sh

POSIX shell (`sh`, not bash). Always wrapped in a function:

```sh
#!/bin/sh
# shellcheck disable=SC2034

set -e
set -u

__init_pkgname() {
    # These 6 variables are required
    pkg_cmd_name="cmd"

    pkg_dst_cmd="$HOME/.local/bin/cmd"
    pkg_dst="$pkg_dst_cmd"

    pkg_src_cmd="$HOME/.local/opt/cmd-v$WEBI_VERSION/bin/cmd"
    pkg_src_dir="$HOME/.local/opt/cmd-v$WEBI_VERSION"
    pkg_src="$pkg_src_cmd"

    pkg_install() {
        mkdir -p "$(dirname "$pkg_src_cmd")"
        mv ./cmd "$pkg_src_cmd"
    }

    pkg_get_current_version() {
        cmd --version 2> /dev/null | head -n 1 | cut -d' ' -f2
    }
}

__init_pkgname
```

### Framework variables available in install.sh

Set by the webi bootstrap (`_webi/package-install.tpl.sh`):

| Variable        | Example             | Description           |
| --------------- | ------------------- | --------------------- |
| `WEBI_VERSION`  | `1.2.3`             | Selected version      |
| `WEBI_PKG_URL`  | `https://...`       | Download URL          |
| `WEBI_PKG_FILE` | `foo-v1.2.3.tar.gz` | Download filename     |
| `WEBI_OS`       | `linux`             | Detected OS           |
| `WEBI_ARCH`     | `amd64`             | Detected architecture |
| `WEBI_EXT`      | `tar.gz`            | Archive extension     |
| `WEBI_CHANNEL`  | `stable`            | Release channel       |
| `PKG_NAME`      | `foo`               | Package name          |

### Override functions

| Function                    | Purpose                                       |
| --------------------------- | --------------------------------------------- |
| `pkg_install()`             | **Required.** Move files to `$pkg_src`        |
| `pkg_get_current_version()` | Parse installed version from command output   |
| `pkg_post_install()`        | Post-install setup (git config, shell config) |
| `pkg_done_message()`        | Custom completion message                     |
| `pkg_link()`                | Override default symlink behavior             |
| `pkg_pre_install()`         | Custom pre-install logic                      |

### Framework helper functions

| Function                 | Purpose                            |
| ------------------------ | ---------------------------------- |
| `webi_download()`        | Download package if not cached     |
| `webi_extract()`         | Extract archive by extension       |
| `webi_path_add <dir>`    | Add to PATH via envman             |
| `webi_link()`            | Create versioned symlinks          |
| `webi_check_installed()` | Check if version already installed |

### pkg_install patterns

**Bare binary in archive root:**

```sh
mv ./cmd "$pkg_src_cmd"
```

**Binary in a subdirectory (goreleaser-style `cmd-OS-arch/cmd`):**

```sh
mv ./cmd-*/cmd "$pkg_src_cmd"
```

**Flexible detection (handles multiple archive layouts):**

```sh
if test -f ./cmd; then
    mv ./cmd "$pkg_src_cmd"
elif test -e ./cmd-*/cmd; then
    mv ./cmd-*/cmd "$pkg_src_cmd"
elif test -e ./cmd-*; then
    mv ./cmd-* "$pkg_src_cmd"
fi
```

## install.ps1

PowerShell for Windows. Uses `$Env:` variables. See `_example/install.ps1` for
the full template. Key differences from install.sh:

- Paths use backslashes, commands end in `.exe`
- `$Env:USERPROFILE` instead of `$HOME`
- `Test-Path`, `Move-Item`, `Copy-Item` instead of shell equivalents
- Downloads go to `$Env:USERPROFILE\Downloads\webi\`
- Temp work in `.local\tmp`, use `Push-Location`/`Pop-Location`
- Symlinks done via `Copy-Item` (not actual symlinks)

## README.md

````markdown
---
title: toolname
homepage: https://github.com/owner/repo
tagline: |
  toolname: A short one-line description.
---

To update or switch versions, run `webi toolname@stable` (or `@v2`, `@beta`,
etc).

### Files

These are the files that are created and/or modified with this installer:

```text
~/.config/envman/PATH.env
~/.local/bin/toolname
~/.local/opt/toolname-VERSION/bin/toolname
```

## Cheat Sheet

> `toolname` does X. Brief description.

### How to use toolname

```sh
toolname --example
```
````

Note: **Files goes above Cheat Sheet**, not inside it.

### Cheat Sheet tone and style

Webi cheat sheets are **opinionated quick-reference guides**, not comprehensive
documentation. Think "colleague's sticky note" — not the project's official
README.

The tool is the topic, but **the problem is the reason**. Cheat sheets are
organized around tasks the reader already wants to do — the tool is how they get
there. Headings reference the tool (the reader came to this page on purpose),
but the content solves the underlying problem completely:

- "How to reverse proxy to Node" (caddy knowledge, not just node)
- "How to run a Node app as a System Service" (serviceman knowledge)
- "How to Enable Secure Remote Postgres Access" (openssl, pg_hba.conf, systemd)
- "How to manually configure git to use delta" (gitconfig, not delta flags)
- "How to make fish the default shell in iTerm2" (iTerm2 knowledge, not fish)

The reader's question is "how do I do X?" and the cheat sheet answers it
completely — including configs, adjacent tools, and platform-specific
variations. A goreleaser cheat sheet teaches you goreleaser YAML. A postgres
cheat sheet teaches you pg_hba.conf, openssl certs, and systemd units.

Cheat sheets cross tool boundaries freely. Node's references caddy, serviceman,
setcap-netbind, GitHub Actions. Postgres references serviceman, openrc, launchd.
They link to each other's webi pages. The scope is "everything you need to
accomplish this task," not "everything this one binary does."

They show the actual files and configs that matter — not documentation _about_
configs, but the configs themselves, copy-pasteable, with inline comments
explaining the non-obvious parts.

**Guidelines:**

- **Show the 3-5 things someone will actually do**, with copy-pasteable
  commands. Skip exhaustive flag lists and API docs.
- **Lead with practical integration.** Show the exact `git config` lines, the
  exact hook script, the exact shell alias — don't just explain the feature and
  leave wiring up to the reader.
- **Skip what they already know.** No need to re-explain what the tool is at
  length — the tagline and one-liner blockquote handle that. Get to the
  commands.
- **Prefer concrete over abstract.** Instead of "you can configure X via a
  config file", show the config file contents.

## Shell Naming Conventions

**Variables:**

- `ALL_CAPS` — environment variables only (`PATH`, `HOME`, `WEBI_VERSION`)
- `b_varname` — block-scoped (inside a function, loop, or conditional)
- `g_varname` — global to the script (and sourced scripts)
- `a_varname` — function arguments

**Functions and commands:**

- `fn_name` — helper functions (anything other than the script's main/entry
  function)
- `cmd_name` — command aliases, e.g. `cmd_curl='curl --fail-with-body -sSL'`

## Code Style

Requires `node`, `shfmt`, `pwsh`, and `pwsh-essentials` (install all via webi).
Run before committing:

```sh
npm run fmt         # prettier (JS/MD) + shfmt (sh) + pwsh-fmt (ps1)
npm run lint        # jshint + shellcheck + pwsh-lint
```

Commit messages: `feat(<pkg>): add installer`, `fix(<pkg>): update install.sh`,
`docs(<pkg>): add cheat sheet`.

## Naming Conventions

- The canonical package name is the **command name** you type: `go`, `node`,
  `rg`
- The alternate/alias name is the project name: `golang`, `nodejs`, `ripgrep`
- Package directories are lowercase with hyphens

## Common Pitfalls

- **Monorepo releases**: The GitHub API returns ALL releases for the repo. You
  must filter in `releases.js` and strip the tag prefix from the version.
- **No `--version` flag**: Some tools lack version introspection. Comment out
  `pkg_get_current_version` — webi still works, it just can't skip reinstalls.
- **Cache directory**: `builds-cacher.js` and `transform-releases.js` read
  exclusively from `~/.cache/webi/legacy/{pkg}.json` (resolved via
  `Os.homedir()`). No date bucketing, no `_cache` symlink, no month rollover
  hazard. If a package returns `0.0.0`, check that `~/.cache/webi/legacy/{pkg}.json`
  exists and that `webicached` is running.
- **Goreleaser archives**: Typically contain a bare binary at the archive root
  (not nested in a directory). Use `mv ./cmd "$pkg_src_cmd"`.

---

## Go Cache Daemon

The Go pipeline (`cmd/webicached`) replaces the Node.js release-fetching code.
It reads `releases.conf` files, fetches upstream release metadata, classifies
build assets, and writes to `~/.cache/webi/legacy/` in the format the Node.js server expects.

### Canonical Vocabulary

The classifier MUST use exactly these strings. They match the production API.

**OS**: `macos` (NOT `darwin`), `linux`, `windows`, `freebsd`, `openbsd`,
`netbsd`, `dragonfly`, `aix`, `illumos`, `plan9`, `solaris`, `posix_2017`

**Arch** — exact equivalences:
- `amd64` (NOT `x86_64`), `x86` (NOT `i386`/`i686`/`386`)
- `arm64` (NOT `aarch64`)
- `armv7l` (NOT `armv7`), `armv6l` (NOT `armv6`)
- `mipsle` (NOT `mipsel`), `mips64le` (NOT `mips64el`)

**Arch** — compatibility downcasts:
- `armhf` → `armv7l`, `armv7a` → `armv7l`, `armel` → `arm`

**Arch** — other: `arm`, `ppc64le`, `ppc64`, `loong64`, `riscv64`, `s390x`,
`mips`, `mips64`

**Libc**: `none` (never empty), `gnu`, `musl`, `msvc`

**Ext**: `tar.gz`, `tar.xz`, `zip`, `exe`, `7z`, `pkg`, `msi`
(no leading dot; `exe` for bare binaries)

### releases.conf

Each package directory contains a `releases.conf` that tells the daemon where
to fetch releases. Format is `key = value`, one per line. `#` comments and
blank lines are ignored.

#### Source types (mutually exclusive — pick one)

```ini
# GitHub binary releases (most common)
github_releases = sharkdp/bat

# GitHub source tarballs (with optional git fallback)
github_sources = bnnanet/serviceman
git_url = https://github.com/bnnanet/serviceman.git

# Git tag enumeration (vim plugins, shell scripts — git_url alone)
git_url = https://github.com/tpope/vim-commentary.git

# Gitea (full URL required, or short form + base_url)
gitea_releases = https://git.rootprojects.org/root/pathman

# GitLab (defaults to gitlab.com)
gitlab_releases = owner/repo

# HashiCorp releases API
hashicorp_product = terraform

# Custom source (servicemandist, nodedist, zigdist, etc.)
source = nodedist
url = https://nodejs.org/download/release
```

#### Filtering, versioning, and platform

```ini
tag_prefix = bun-                    # monorepo: strip prefix from version
version_prefixes = jq-               # strip from version string (space-separated)
asset_filter = MinGit                # filename must contain this substring
exclude = busybox -src- -docs-       # skip assets containing these (space-separated)
os = posix_2017                      # restrict ALL versions to this OS (blanket)
alias_of = rg                        # mirrors another package's releases
```

#### Design rules

- `os` is a blanket tag on ALL versions. Only use for packages that are always
  POSIX-only. For version-dependent OS tagging, use a custom `TagVariants` in
  `internal/releases/{pkg}/variants.go`.
- `git_url` can be primary (gittag source when it's the only key) or secondary
  fallback alongside `github_sources`/`gitea_sources`.
- Full URL forms accepted for github/gitea/gitlab (e.g.
  `github_releases = https://github.com/sharkdp/bat`).

### Testing

Test tools: `cmd/e2etest` (pipeline comparison), `cmd/comparecache` (cache diff),
`cmd/inspect` (single-package debug). Run each with `--help` for usage.

### Classifier vs Per-Package Tagger

The general classifier (`internal/classify/`) handles patterns common across
many projects. It MUST NOT contain one-off logic for a single package.

Per-package taggers (`internal/releases/{pkg}/variants.go`) handle
project-specific knowledge. Read the existing taggers for conventions.

MUST: Derive arch/OS from concrete evidence — not blanket defaults.
MUST: New general classifier patterns must apply to 2-3+ packages.

### Deploying

```sh
./scripts/deploy-webicached.sh beta.webi.sh
./scripts/deploy-webicached.sh next.webi.sh
```

First-time setup on a new host uses `serviceman`:

```sh
serviceman add --name webicached \
  --workdir ~/srv/webid/installers/ -- \
  ~/bin/webicached \
    --envfile ~/srv/webid/.env.secret \
    --conf ~/srv/webid/installers/ \
    --raw ~/.cache/webi/raw
```