chore: add types via jsdoc

Signed-off-by: bry-val <94031627+bry-val@users.noreply.github.com>

.github/workflows/node.js.yml

@@ -0,0 +1,37 @@
+name: Node.js CI
+on:
+  push:
+    branches: ['main']
+  pull_request:
+jobs:
+  build:
+    name: "Fmt, Lint, & Test"
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        node-version:
+          #- 20.x
+          - latest
+    steps:
+      - uses: actions/checkout@v3
+      - name: Use Node.js ${{ matrix.node-version }}
+        uses: actions/setup-node@v3
+        with:
+          node-version: ${{ matrix.node-version }}
+      - name: "Webi: Install 'shfmt' and 'shellcheck', and update PATH"
+        run: |
+          sh ./_scripts/install-ci-deps
+          echo "${HOME}/.local/bin" >> $GITHUB_PATH
+          echo "${HOME}/.local/opt/pwsh" >> $GITHUB_PATH
+      - run: |
+          git submodule init
+          git submodule update
+      - run: shfmt --version
+      - run: shellcheck -V
+      - run: node --version
+      - run: npm run fmt
+      - run: npm ci
+      - run: npm run lint
+      - env:
+          GITHUB_TOKEN: ${{ github.token }}
+        run: npm run test

.gitignore

@@ -1,6 +1,24 @@
-.env
-node_modules
+# generated artifacts
 install-*.sh
 install-*.bat
 install-*.ps1
+
+# local config
+.env.*
+*.env
+.env
+!example.env
+
+# caches
+_cache/
+node_modules/
+
+# temporary & backup files
 .*.sw*
+*.bak
+*.bak.*
+
+# other
+.DS_Store
+desktop.ini
+.directory

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "_webi/build-classifier"]
+	path = _webi/build-classifier
+	url = https://github.com/webinstall/webi-build-classifier.git

.jshintignore

@@ -0,0 +1 @@
+DELETEMEnode_modules/**/*

.jshintrc

@@ -0,0 +1,26 @@
+{
+  "globals": {
+    "AbortController": false
+  },
+  "browser": true,
+  "node": true,
+  "esversion": 11,
+  "curly": true,
+  "sub": true,
+
+  "bitwise": true,
+  "eqeqeq": true,
+  "forin": true,
+  "freeze": true,
+  "immed": true,
+  "latedef": "nofunc",
+  "nonbsp": true,
+  "nonew": true,
+  "plusplus": true,
+  "undef": true,
+  "unused": "vars",
+  "strict": true,
+  "maxdepth": 4,
+  "maxstatements": 100,
+  "maxcomplexity": 20
+}

.prettierignore

@@ -0,0 +1,4 @@
+node_modules
+jsconfig.json
+package.json
+package-lock.json

.prettierrc

@@ -1,6 +0,0 @@
-{
-  "trailingComma": "none",
-  "tabWidth": 2,
-  "singleQuote": true,
-  "proseWrap": "always"
-}

.prettierrc.json

@@ -0,0 +1,9 @@
+{
+  "bracketSpacing": true,
+  "printWidth": 80,
+  "proseWrap": "always",
+  "singleQuote": true,
+  "tabWidth": 2,
+  "trailingComma": "all",
+  "semi": true
+}

.vimrc

@@ -0,0 +1,6 @@
+" Note: You must enable per-project .vimrc in your ~/.vimrc
+" set secure
+" set exrc
+
+" PowerShell settings
+let g:ale_powershell_psscriptanalyzer_exclusions = "PSAvoidUsingWriteHost,PSUseDeclaredVarsMoreThanAssignments"

CONTRIBUTING.md

@@ -0,0 +1,123 @@
+# Guidelines for Contributing to Webi
+
+**Before you start**:
+
+- It's a good idea to try [Webi](https://webinstall.dev/) out for installing
+  developer tools for yourself before creating an installer.
+- It's also best if you take on issues for tools that you're familiar with.
+
+**Before you PR**:
+
+- You'll be asked to make changes if you don't run the code formatters and
+  linters:
+
+  - Node / JavaScript:
+    - [prettier](https://webinstall.dev/prettier)
+      ```sh
+      npm run prettier
+      ```
+    - [jshint](https://webinstall.dev/jshint)
+      ```sh
+      npm run lint
+      ```
+  - Bash
+
+    - [shfmt](https://webinstall.dev/shfmt)
+      ```sh
+      npm run shfmt
+      ```
+      Or
+      ```bash
+      shfmt -w -i 4 -sr -ci -s ./
+      ```
+    - [shellcheck](https://webinstall.dev/shellcheck) \
+      To check for all warnings and errors (except the ones we ignore):
+      ```bash
+      # to check all errors
+      shellcheck -s sh --exclude=SC2154,SC2034 */*.sh */*/*.sh
+      ```
+      To check for only specific warnings and errors:
+      ```bash
+      # to check specific errors
+      shellcheck -s sh --include=SC2005 */*.sh */*/*.sh
+      ```
+      Enumerated shellcheck codes:
+      <https://gist.github.com/nicerobot/53cee11ee0abbdc997661e65b348f375>
+    - Common exceptions:
+
+      ```text
+      # We make use of `.` (source) to import without exports
+      SC2034: foo appears unused. Verify it or export it.
+      SC2154: var is referenced but not assigned.
+      ```
+
+- If you use vim, [vim-essentials](https://webinstall.dev/vim-essentials)
+  includes everything you need to automatically format and lint on save.
+- If you use VS Code, the same plugins are also available in the VS Code store.
+
+**Not strictly mandatory, but we appreciate**:
+
+- [x] [Signed Commits](/git-config-gpg)
+- [x] Semantic Commit Messages
+- [x] Update `test` psuedo-package
+
+## Signed Commits
+
+Please **enable gpg-signing**.
+
+You can do this **in about 30 seconds**:
+
+1. Run [`git-config-gpg`](https://webinstall.dev/git-config-gpg) from Webi:
+   ```sh
+   # On Mac & Linux
+   curl https://webi.sh/git-config-gpg | sh
+   ```
+2. Copy the GPG public key (it will be printed to your screen)
+3. Add it to your GitHub profile: <https://github.com/settings/gpg/new>
+
+## Semantic Commit Messages
+
+We try to follow "semantic commits" to some degree. Especially since this is a
+project with many sub-projects.
+
+The general format is `<type>(<package>): <description>`, using these _types_:
+
+| _type_   | _usage_                                                            |
+| :------- | :----------------------------------------------------------------- |
+| feat     | new feature for the user, not a new feature for build script       |
+| fix      | bug fix for the user, not a fix to a build script                  |
+| docs     | changes to the documentation                                       |
+| style    | formatting, missing semi colons, etc; no production code change    |
+| refactor | refactoring production code, eg. renaming a variable               |
+| test     | adding missing tests, refactoring tests; no production code change |
+| chore    | updating grunt tasks etc; no production code change                |
+
+Try to write your commit messages (in the present tense) like this:
+
+```text
+fix(node): update install.sh (fix #200)
+```
+
+```text
+feat(delta): add cheat sheet and install.sh
+```
+
+```text
+docs(ssh-adduser): document that foo does bar
+```
+
+See <https://gist.github.com/joshbuchea/6f47e86d2510bce28f8e7f42ae84c716> for
+some more examples.
+
+## Also update the `test` installer (a 3-line change)
+
+Whenever adding a new installer, please also update `test/install.sh`
+
+- `rm -rf ~/.local/opt/YOUR_PACKAGE`
+- `rm -f ~/.local/bin/YOUR_PACKAGE`
+- `webi YOUR_PACKAGE`
+- (and please keep it in alphabetical order)
+
+See
+<https://github.com/webinstall/webi-installers/pull/346/files#diff-db3af85ef45ed7ac0d1d9c473cf4d858657c127dc24d931fe18a9961f17e05b1>
+for an example.

README.md

@@ -2,13 +2,15 @@
 
 > [webi](https://webinstall.dev) is how developers install their tools
 
+[![webinstall-dev-ogimage-github](https://user-images.githubusercontent.com/122831/129465590-136b5a8a-f8f5-4e8d-a010-784eaa9f21bb.png)](https://webinstall.dev)
+
 - no `sudo`
 - no package manager
 - no messing with system permissions
 - in short: no nonsense
 
-```bash
-curl https://webinstall.dev/webi | bash
+```sh
+curl https://webi.sh/webi | sh
 ```
 
 This repository contains the primary and community-submitted packages for
@@ -30,7 +32,7 @@ More technically:
    - common release APIs are in `_common/` (i.e. `_common/github.js`)
 2. `_webi/bootstrap.sh` is a template that exchanges system information for a
    correct installer
-   - contructs a user agent with os, cpu, and utility info (i.e. `macos`,
+   - constructs a user agent with os, cpu, and utility info (i.e. `macos`,
      `amd64`, can unpack `tar,zip,xz`)
 3. `_webi/template.sh` is the base installer template with common functions for
    - checking versions
@@ -39,10 +41,10 @@ More technically:
    - (re-)linking directories
 4. `<package>/install.sh` may provide functions to override `_webi/template.sh`
 5. Recap:
-   - `curl https://webinstall.dev/<pkg>` => `bootstrap-<pkg>.sh`
-   - `bash bootstrap-<pkg>.sh` =>
+   - `curl https://webi.sh/<pkg>` => `bootstrap-<pkg>.sh`
+   - `sh bootstrap-<pkg>.sh` =>
      `https://webinstall.dev/api/installers/<pkg>@<ver>.sh?formats=zip,tar`
-   - `bash install-<pkg>.sh` => download, unpack, move, link, update PATH
+   - `sh install-<pkg>.sh` => download, unpack, move, link, update PATH
 
 # Philosophy (for package authors / maintainers publishing with webi)
 
@@ -71,7 +73,7 @@ An install consists of 5 parts in 4 files:
 my-new-package/
   - README.md (package info in frontmatter)
   - releases.js
-  - install.sh (bash)
+  - install.sh (POSIX Shell)
   - install.ps1 (PowerShell)
 ```
 
@@ -86,7 +88,7 @@ See these **examples**:
 - https://github.com/webinstall/packages/blob/master/golang/
 
 The `webinstall.dev` server uses the list of releases returned by
-`<your-package>/releases.js` to generate a bash script with most necessary
+`<your-package>/releases.js` to generate a shell script with most necessary
 variables and functions pre-defined.
 
 You just fill in the blanks.
@@ -95,13 +97,14 @@ You just fill in the blanks.
 
 Just create an empty directory and run the tests until you get a good result.
 
-```bash
-git clone git@github.com:webinstall/packages.git
-pushd packages
-npm install
+```sh
+git clone git@github.com:webinstall/webi-installers.git
+pushd ./webi-installers/
+git submodule update --init
+npm clean-install
 ```
 
-```bash
+```sh
 mkdir -p ./new-package/
 node _webi/test.js ./new-package/
 ```
@@ -116,12 +119,13 @@ Just copy the format from any of the existing packages. It's like this:
 ---
 title: Node.js
 homepage: https://nodejs.org
-tagline: JavaScript V8 runtime
+tagline: |
+  JavaScript V8 runtime
 description: |
   Node.js® is a JavaScript runtime built on Chrome's V8 JavaScript engine
 ---
 
-```bash
+```sh
 node -e 'console.log("Hello, World!")'
 > Hello, World!
 ```
@@ -141,8 +145,8 @@ It looks like this:
 `releases.js`:
 
 ```js
-module.exports = function (request) {
-  return github(request, owner, repo).then(function (all) {
+module.exports = function () {
+  return github(null, owner, repo).then(function (all) {
     // if you need to do something special, you can do it here
     // ...
     return all;
@@ -158,7 +162,7 @@ module.exports = function (request) {
 
 (optional, if needed) Bash variables that you _may_ define:
 
-```bash
+```sh
 # Define this if the package name is different from the command name (i.e. golang => go)
 pkg_cmd_name="foobar"
 
@@ -175,16 +179,16 @@ pkg_src_cmd="$HOME/.local/opt/foobar-v$WEBI_VERSION/bin/foobar"
 
 (required) A version check function that strips all non-version junk
 
-```bash
+```sh
 pkg_get_current_version() {
     # foobar-v1.1.7 => 1.1.7
     echo "$(foobar --version | head -n 1 | sed 's:foobar-v::')"
 }
 ```
 
-For the rest of the functions you can like copy/paste from the examples:
+For the rest of the functions you can copy/paste from the examples:
 
-```bash
+```sh
 pkg_format_cmd_version() {}         # Override, pretty prints version
 
 pkg_link                            # Override, replaces webi_link()
@@ -211,7 +215,7 @@ See `webi/template.sh`
 
 These variables will be set by the server:
 
-```bash
+```sh
 WEBI_PKG=example@v1
 WEBI_TAG=v1
 WEBI_HOST=https://webinstall.dev
@@ -228,21 +232,21 @@ WEBI_PKG_URL=https://cdn.example.com/example-macos-amd64.tar.gz
 WEBI_PKG_FILE=example-macos-amd64.tar.gz
 ```
 
-```bash
+```sh
 PKG_NAME=example
 PKG_OSES=macos,linux,windows
 PKG_ARCHES=amd64,arm64,x86
 PKG_FORMATS=zip,xz
 ```
 
-```bash
+```sh
 WEBI_TMP=${WEBI_TMP:-"$(mktemp -d -t webinstall-foobar.XXXXXXXX)"}
 WEBI_SINGLE=""
 ```
 
-```bash
+```sh
 webi_check              # Checks to see if the selected version is already installed (and re-links if so)
-webi_download           # Downloads the selected release to $HOME/Downloads/<package-name>.tar.gz
+webi_download           # Downloads the selected release to $HOME/Downloads/webi/<package-name>.tar.gz
 webi_extract            # Extracts the download to /tmp/<package-name>-<random>/
 webi_path_add /new/path # Adds /new/path to PATH for bash, zsh, and fish
 webi_pre_install        # Runs webi_check, webi_download, and webi_extract
@@ -267,10 +271,10 @@ webi_post_install       # Runs `webi_path_add $pkg_dst_bin`
 set WEBI_HOST=https://webinstall.dev
 ```
 
-Windows 10 has curl too!?
+Windows has curl too!?
 
 ```bat
-curl.exe -sL -A "MS" https://webinstall.dev/node | powershell
+curl.exe -sL https://webi.ms/node | powershell
 ```
 
 And it's easy enough to ignore the execution policy
@@ -283,11 +287,11 @@ And if we want something that looks as complicated as we expect Windows to be,
 historically, we have options:
 
 ```bat
-powershell "Invoke-Expression ( Invoke-WebRequest -UseBasicParsing https://webinstall.dev/node ).Contents"
+powershell "Invoke-Expression ( Invoke-WebRequest -UseBasicParsing https://webi.ms/node ).Contents"
 ```
 
 ```bat
-powershell ( Invoke-WebRequest -UseBasicParsing https://webinstall.dev/node ).Contents | powershell
+powershell ( Invoke-WebRequest -UseBasicParsing https://webi.ms/node ).Contents | powershell
 ```
 
 -->

_common/brew.js

@@ -1,64 +1,62 @@
 'use strict';
 
+let Fetcher = require('../_common/fetcher.js');
+
 /**
- * Gets a releases from 'brew'.
+ * Gets releases from 'brew'.
  *
- * @param request
+ * @param {null} _
  * @param {string} formula
- * @returns {PromiseLike<any> | Promise<any>}
+ * @returns {Promise<any>}
  */
-function getAllReleases(request, formula) {
+async function getDistributables(_, formula) {
   if (!formula) {
     return Promise.reject('missing formula for brew');
   }
-  return request({
-    url: 'https://formulae.brew.sh/api/formula/' + formula + '.json',
-    fail: true, // https://git.coolaj86.com/coolaj86/request.js/issues/2
-    json: true
-  })
-    .then(failOnBadStatus)
-    .then(function (resp) {
-      var ver = resp.body.versions.stable;
-      var dl = (
-        resp.body.bottle.stable.files.high_sierra ||
-        resp.body.bottle.stable.files.catalina
-      ).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
-      return [
-        {
-          version: ver,
-          download: dl.replace(/{{ v }}/g, ver)
-        }
-      ].concat(
-        resp.body.versioned_formulae.map(function (f) {
-          var ver = f.replace(/.*@/, '');
-          return {
-            version: ver,
-            download: dl
-          };
-        })
-      );
-    })
-    .catch(function (err) {
-      console.error('Error fetching MariaDB versions (brew)');
-      console.error(err);
-      return [];
+
+  let resp;
+  try {
+    let url = `https://formulae.brew.sh/api/formula/${formula}.json`;
+    resp = await Fetcher.fetch(url, {
+      headers: { Accept: 'application/json' },
     });
-}
-
-function failOnBadStatus(resp) {
-  if (resp.statusCode >= 400) {
-    var err = new Error('Non-successful status code: ' + resp.statusCode);
-    err.code = 'ESTATUS';
-    err.response = resp;
-    throw err;
+  } catch (e) {
+    /** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
+    let err = e;
+    if (err.code === 'E_FETCH_RELEASES') {
+      err.message = `failed to fetch '${formula}' release data from 'brew': ${err.response.status} ${err.response.body}`;
+    }
+    throw e;
   }
-  return resp;
+  let body = JSON.parse(resp.body);
+
+  var ver = body.versions.stable;
+  var dl = (
+    body.bottle.stable.files.high_sierra || body.bottle.stable.files.catalina
+  ).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
+  return [
+    {
+      version: ver,
+      download: dl.replace(/{{ v }}/g, ver),
+    },
+  ].concat(
+    body.versioned_formulae.map(
+      /** @param {String} f */
+      function (f) {
+        var ver = f.replace(/.*@/, '');
+        return {
+          version: ver,
+          download: dl,
+        };
+      },
+    ),
+  );
 }
 
-module.exports = getAllReleases;
+module.exports = getDistributables;
 
 if (module === require.main) {
-  getAllReleases(require('@root/request'), 'mariadb').then(function (all) {
+  getDistributables(null, 'mariadb').then(function (all) {
     console.info(JSON.stringify(all, null, 2));
   });
 }

_common/fetcher.js

@@ -0,0 +1,56 @@
+'use strict';
+
+let Fetcher = module.exports;
+
+/**
+ * @typedef ResponseSummary
+ * @prop {Boolean} ok
+ * @prop {Headers} headers
+ * @prop {Number} status
+ * @prop {String} body
+ */
+
+/**
+ * @param {String} url
+ * @param {RequestInit} opts
+ * @returns {Promise<ResponseSummary>}
+ */
+Fetcher.fetch = async function (url, opts) {
+  let resp = await fetch(url, opts);
+  let summary = Fetcher.throwIfNotOk(resp);
+
+  return summary;
+};
+
+/**
+ * @param {Response} resp
+ * @returns {Promise<ResponseSummary>}
+ */
+Fetcher.throwIfNotOk = async function (resp) {
+  let text = await resp.text();
+
+  if (!resp.ok) {
+    let headers = Array.from(resp.headers);
+    console.error('[Fetcher] error: Response Headers:', headers);
+    console.error('[Fetcher] error: Response Text:', text);
+    let err = new Error(`fetch was not ok`);
+    Object.assign({
+      status: 503,
+      code: 'E_FETCH_RELEASES',
+      response: {
+        status: resp.status,
+        headers: headers,
+        body: text,
+      },
+    });
+    throw err;
+  }
+
+  let summary = {
+    ok: resp.ok,
+    headers: resp.headers,
+    status: resp.status,
+    body: text,
+  };
+  return summary;
+};

_common/git-tag.js

@@ -0,0 +1,218 @@
+'use strict';
+
+require('dotenv').config({ path: '.env' });
+
+var Crypto = require('crypto');
+var util = require('util');
+var exec = util.promisify(require('child_process').exec);
+var Fs = require('node:fs/promises');
+var FsSync = require('node:fs');
+var Path = require('node:path');
+
+var repoBaseDir = process.env.REPO_BASE_DIR || '';
+if (!repoBaseDir) {
+  repoBaseDir = Path.resolve('./_repos');
+  // for stderr
+  console.error(`[Warn] REPO_BASE_DIR= not set, ${repoBaseDir}`);
+}
+
+var Repos = {};
+
+Repos.clone = async function (repoPath, gitUrl) {
+  let uuid = Crypto.randomUUID();
+  let tmpPath = `${repoPath}.${uuid}.tmp`;
+  let bakPath = `${repoPath}.${uuid}.dup`;
+  await exec(`git clone --bare --filter=tree:0 ${gitUrl} ${tmpPath}`);
+
+  try {
+    FsSync.accessSync(repoPath);
+    return;
+  } catch (e) {
+    if (e.code !== 'ENOENT') {
+      throw e;
+    }
+  }
+
+  // sync to avoid race conditions
+  try {
+    FsSync.renameSync(repoPath, bakPath);
+  } catch (e) {
+    if (e.code !== 'ENOENT') {
+      throw e;
+    }
+  }
+  FsSync.renameSync(tmpPath, repoPath);
+
+  await Fs.rm(bakPath, { force: true, recursive: true });
+};
+
+Repos.checkExists = async function (repoPath) {
+  let err = await Fs.access(repoPath).catch(Object);
+  if (!err) {
+    return true;
+  }
+
+  if (err.code !== 'ENOENT') {
+    throw err;
+  }
+  return false;
+};
+
+Repos.fetch = async function (repoPath) {
+  await exec(`git --git-dir=${repoPath} fetch`);
+};
+
+Repos.getTags = async function (repoPath) {
+  var { stdout } = await exec(`git --git-dir=${repoPath} tag`);
+  var rawTags = stdout.trim().split('\n');
+
+  let tags = [];
+  for (let tag of rawTags) {
+    // ex: v1, v2, v1.1, 1.1.0-rc
+    let maybeVersionRe = /^(v\d+|v?\d+\.\d+)/;
+    let maybeVersion = maybeVersionRe.test(tag);
+    if (maybeVersion) {
+      tags.push(tag);
+    }
+  }
+
+  tags = tags.reverse();
+  return tags;
+};
+
+Repos.getTipInfo = async function (repoPath) {
+  var { stdout } = await exec(
+    `git --git-dir=${repoPath} rev-parse --abbrev-ref HEAD`,
+  );
+  var branch = stdout.trim();
+
+  var info = await Repos.getCommitInfo(repoPath, 'HEAD');
+  info.commitish = branch;
+
+  return info;
+};
+
+Repos.getCommitInfo = async function (repoPath, commitish) {
+  var { stdout } = await exec(
+    `git --git-dir=${repoPath} log -1 --format="%h %H %ad %cd" --date=iso-strict ${commitish}`,
+  );
+  stdout = stdout.trim();
+  var commitParts = stdout.split(/\s+/g);
+  return {
+    commitish: commitish,
+    commit_id: `${commitParts[0]}`,
+    commit: `${commitParts[1]}`,
+    date: commitParts[2],
+    date_authored: commitParts[3],
+  };
+};
+
+/**
+ * Lists GitHub Releases (w/ uploaded assets)
+ *
+ * @param request
+ * @param {string} owner
+ * @param {string} gitUrl
+ * @returns {PromiseLike<any> | Promise<any>}
+ */
+async function getDistributables(gitUrl) {
+  let all = {
+    releases: [],
+    download: '',
+  };
+
+  let repoName = gitUrl.split('/').pop();
+  repoName = repoName.replace(/\.git$/, '');
+
+  let repoPath = `${repoBaseDir}/${repoName}.git`;
+
+  let isCloned = await Repos.checkExists(repoPath);
+  if (!isCloned) {
+    await Repos.clone(repoPath, gitUrl);
+  } else {
+    await Repos.fetch(repoPath);
+  }
+
+  let commitInfos = [];
+  let tags = await Repos.getTags(repoPath);
+  for (let tag of tags) {
+    let commitInfo = await Repos.getCommitInfo(repoPath, tag);
+    Object.assign(commitInfo, { version: tag, channel: '' });
+    commitInfos.push(commitInfo);
+  }
+
+  {
+    let tipInfo = await Repos.getTipInfo(repoPath);
+    // "2024-01-01T00:00:00-05:00" => "2024-01-01T05:00:00"
+    let date = new Date(tipInfo.date);
+    // "2024-01-01T05:00:00" => "v2024.01.01-05.00.00"
+    let version = date.toISOString();
+    // strip '.000Z'
+    version = version.replace(/\.\d+Z/, '');
+    version = version.replace(/[:\-]/g, '.');
+    version = version.replace(/T/, '-');
+    Object.assign(tipInfo, { version: `v${version}`, channel: '' });
+    if (commitInfos.length > 1) {
+      tipInfo.channel = 'beta';
+    }
+    commitInfos.push(tipInfo);
+  }
+
+  let releases = [];
+  for (let commitInfo of commitInfos) {
+    let version = commitInfo.version.replace(/^v/, '');
+
+    let date = new Date(commitInfo.date);
+    let isoDate = date.toISOString();
+    isoDate = isoDate.replace(/\.\d+Z/, '');
+
+    // tags and HEAD qualify for '--branch <branchish>'
+    let branch = commitInfo.commitish;
+
+    let rel = {
+      name: `${repoName}-v${version}`,
+      version: version,
+      git_tag: branch,
+      git_commit_hash: commitInfo.commit_id,
+      lts: false,
+      channel: commitInfo.channel,
+      date: isoDate,
+      os: '*',
+      arch: '*',
+      ext: 'git',
+      download: gitUrl,
+    };
+
+    releases.push(rel);
+  }
+  all.releases = releases;
+
+  return all;
+}
+
+module.exports = getDistributables;
+
+if (module === require.main) {
+  (async function main() {
+    let testRepos = [
+      // just a few tags, and a different HEAD
+      'https://github.com/tpope/vim-commentary.git',
+      // no tags, just HEAD
+      'https://github.com/ziglang/zig.vim.git',
+      // many, many tags
+      //'https://github.com/dense-analysis/ale.git',
+    ];
+    for (let url of testRepos) {
+      let all = await getDistributables(url);
+
+      all = require('../_webi/normalize.js')(all);
+      console.info(JSON.stringify(all, null, 2));
+    }
+  })()
+    .then(function () {
+      process.exit(0);
+    })
+    .catch(function (err) {
+      console.error(err);
+    });
+}

_common/gitea.js

@@ -1,39 +1,47 @@
 'use strict';
 
-var ghRelease = require('./github.js');
+var GitHubish = require('./githubish.js');
 
 /**
- * Gets the releases for 'ripgrep'. This function could be trimmed down and made
- * for use with any github release.
+ * Lists Gitea Releases (w/ uploaded assets)
  *
- * @param request
- * @param {string} owner
- * @param {string} repo
- * @returns {PromiseLike<any> | Promise<any>}
+ * @param {null} _ - deprecated
+ * @param {String} owner
+ * @param {String} repo
+ * @param {String} baseurl
+ * @param {String} [username]
+ * @param {String} [token]
  */
-function getAllReleases(request, owner, repo, baseurl) {
-  if (!baseurl) {
-    return Promise.reject('missing baseurl');
-  }
-  return ghRelease(request, owner, repo, baseurl + '/api/v1').then(function (
-    all
-  ) {
-    return all;
+async function getDistributables(
+  _,
+  owner,
+  repo,
+  baseurl,
+  username = '',
+  token = '',
+) {
+  baseurl = `${baseurl}/api/v1`;
+  let all = await GitHubish.getDistributables({
+    owner,
+    repo,
+    baseurl,
+    username,
+    token,
   });
+  return all;
 }
 
-module.exports = getAllReleases;
+module.exports = getDistributables;
 
 if (module === require.main) {
-  getAllReleases(
-    require('@root/request'),
-    'coolaj86',
-    'go-pathman',
-    'https://git.coolaj86.com'
-  ).then(
-    //getAllReleases(require('@root/request'), 'root', 'serviceman', 'https://git.rootprojects.org').then(
-    function (all) {
-      console.info(JSON.stringify(all, null, 2));
-    }
-  );
+  getDistributables(
+    null,
+    'root',
+    'pathman',
+    'https://git.rootprojects.org',
+    '',
+    '',
+  ).then(function (all) {
+    console.info(JSON.stringify(all, null, 2));
+  });
 }

_common/github-source.js

@@ -0,0 +1,40 @@
+'use strict';
+
+require('dotenv').config({ path: '.env' });
+
+let GitHubSource = module.exports;
+
+let GitHubishSource = require('./githubish-source.js');
+
+/**
+ * @param {Object} opts
+ * @param {String} opts.owner
+ * @param {String} opts.repo
+ * @param {String} [opts.baseurl]
+ * @param {String} [opts.username]
+ * @param {String} [opts.token]
+ */
+GitHubSource.getDistributables = async function ({
+  owner,
+  repo,
+  baseurl = 'https://api.github.com',
+  username = process.env.GITHUB_USERNAME || '',
+  token = process.env.GITHUB_TOKEN || '',
+}) {
+  let all = await GitHubishSource.getDistributables({
+    owner,
+    repo,
+    baseurl,
+    username,
+    token,
+  });
+  return all;
+};
+
+if (module === require.main) {
+  GitHubSource.getDistributables(null, 'BeyondCodeBootcamp', 'DuckDNS.sh').then(
+    function (all) {
+      console.info(JSON.stringify(all, null, 2));
+    },
+  );
+}

_common/github.js

@@ -1,76 +1,42 @@
 'use strict';
 
-require('dotenv').config();
+require('dotenv').config({ path: '.env' });
+
+let GitHubish = require('./githubish.js');
 
 /**
- * Gets the releases for 'ripgrep'. This function could be trimmed down and made
- * for use with any github release.
+ * Lists GitHub Releases (w/ uploaded assets)
  *
- * @param request
- * @param {string} owner
- * @param {string} repo
- * @returns {PromiseLike<any> | Promise<any>}
+ * @param {null} _ - deprecated
+ * @param {String} owner
+ * @param {String} repo
+ * @param {String} [baseurl]
+ * @param {String} [username]
+ * @param {String} [token]
  */
-function getAllReleases(
-  request,
+module.exports = async function (
+  _,
   owner,
   repo,
-  baseurl = 'https://api.github.com'
+  baseurl = 'https://api.github.com',
+  username = process.env.GITHUB_USERNAME || '',
+  token = process.env.GITHUB_TOKEN || '',
 ) {
-  if (!owner) {
-    return Promise.reject('missing owner for repo');
-  }
-  if (!repo) {
-    return Promise.reject('missing repo name');
-  }
-
-  var req = {
-    url: `${baseurl}/repos/${owner}/${repo}/releases`,
-    json: true
-  };
-  // TODO I really don't like global config, find a way to do better
-  if (process.env.GITHUB_USERNAME) {
-    req.auth = {
-      user: process.env.GITHUB_USERNAME,
-      pass: process.env.GITHUB_TOKEN
-    };
-  }
-
-  return request(req).then((resp) => {
-    const gHubResp = resp.body;
-    const all = {
-      releases: [],
-      // todo make this ':baseurl' + ':releasename'
-      download: ''
-    };
-
-    gHubResp.forEach((release) => {
-      release['assets'].forEach((asset) => {
-        const name = asset['name'];
-        all.releases.push({
-          name: name,
-          version: release['tag_name'], // TODO tags aren't always semver / sensical
-          lts: /(\b|_)(lts)(\b|_)/.test(release['tag_name']),
-          channel: !release['prerelease'] ? 'stable' : 'beta',
-          date: (release['published_at'] || '').replace(/T.*/, ''),
-          os: '', // will be guessed by download filename
-          arch: '', // will be guessed by download filename
-          ext: '', // will be normalized
-          download: asset['browser_download_url']
-        });
-      });
-    });
-
-    return all;
+  let all = await GitHubish.getDistributables({
+    owner,
+    repo,
+    baseurl,
+    username,
+    token,
   });
-}
+  return all;
+};
 
-module.exports = getAllReleases;
+let GitHub = module.exports;
+GitHub.getDistributables = module.exports;
 
 if (module === require.main) {
-  getAllReleases(require('@root/request'), 'BurntSushi', 'ripgrep').then(
-    function (all) {
-      console.info(JSON.stringify(all, null, 2));
-    }
-  );
+  GitHub.getDistributables(null, 'BurntSushi', 'ripgrep').then(function (all) {
+    console.info(JSON.stringify(all, null, 2));
+  });
 }

_common/githubish-source.js

@@ -0,0 +1,174 @@
+'use strict';
+
+let Fetcher = require('../_common/fetcher.js');
+
+let GitHubishSource = module.exports;
+
+/**
+ * Lists GitHub-Like Releases (source tarball & zip)
+ *
+ * @param {Object} opts
+ * @param {String} opts.owner
+ * @param {String} opts.repo
+ * @param {String} opts.baseurl
+ * @param {String} [opts.username]
+ * @param {String} [opts.token]
+ */
+GitHubishSource.getDistributables = async function ({
+  owner,
+  repo,
+  baseurl,
+  username = '',
+  token = '',
+}) {
+  if (!owner) {
+    throw new Error('missing owner for repo');
+  }
+  if (!repo) {
+    throw new Error('missing repo name');
+  }
+  if (!baseurl) {
+    throw new Error('missing baseurl');
+  }
+
+  let url = `${baseurl}/repos/${owner}/${repo}/releases`;
+  let opts = {
+    headers: {
+      'Content-Type': 'appplication/json',
+    },
+  };
+
+  if (token) {
+    let userpass = `${username}:${token}`;
+    let basicAuth = btoa(userpass);
+    Object.assign(opts.headers, {
+      Authorization: `Basic ${basicAuth}`,
+    });
+  }
+
+  let resp;
+  try {
+    resp = await Fetcher.fetch(url, opts);
+  } catch (e) {
+    /** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
+    let err = e;
+    if (err.code === 'E_FETCH_RELEASES') {
+      err.message = `failed to fetch '${baseurl}' (githubish-source, user '${username}) release data: ${err.response.status} ${err.response.body}`;
+    }
+    throw e;
+  }
+  let gHubResp = JSON.parse(resp.body);
+
+  let all = {
+    /** @type {Array<BuildInfo>} */
+    releases: [],
+    download: '',
+  };
+
+  for (let release of gHubResp) {
+    let dists = GitHubishSource.releaseToDistributables(release);
+    for (let dist of dists) {
+      let updates =
+        await GitHubishSource.followDistributableDownloadAttachment(dist);
+      Object.assign(dist, updates);
+      all.releases.push(dist);
+    }
+  }
+
+  return all;
+};
+
+/**
+ * @typedef BuildInfo
+ * @prop {String} [name] - name to use instead of filename for hash urls
+ * @prop {String} version
+ * @prop {String} [_version]
+ * @prop {String} [arch]
+ * @prop {String} channel
+ * @prop {String} date
+ * @prop {String} download
+ * @prop {String} [ext]
+ * @prop {String} [_filename]
+ * @prop {String} [hash]
+ * @prop {String} [libc]
+ * @prop {Boolean} [_musl]
+ * @prop {Boolean} [lts]
+ * @prop {String} [size]
+ * @prop {String} os
+ */
+
+/**
+ * @param {any} ghRelease - TODO
+ * @returns {Array<BuildInfo>}
+ */
+GitHubishSource.releaseToDistributables = function (ghRelease) {
+  let ghTag = ghRelease['tag_name']; // TODO tags aren't always semver / sensical
+  let lts = /(\b|_)(lts)(\b|_)/.test(ghRelease['tag_name']);
+  let channel = 'stable';
+  if (ghRelease['prerelease']) {
+    channel = 'beta';
+  }
+  let date = ghRelease['published_at'] || '';
+  date = date.replace(/T.*/, '');
+
+  let urls = [ghRelease.tarball_url, ghRelease.zipball_url];
+  /** @type {Array<BuildInfo>} */
+  let dists = [];
+  for (let url of urls) {
+    dists.push({
+      name: '',
+      version: ghTag,
+      lts: lts,
+      channel: channel,
+      date: date,
+      os: '*',
+      arch: '*',
+      libc: '',
+      ext: '',
+      download: url,
+    });
+  }
+
+  return dists;
+};
+
+/**
+ * @param {BuildInfo} dist
+ */
+GitHubishSource.followDistributableDownloadAttachment = async function (dist) {
+  let abortCtrl = new AbortController();
+  let resp = await fetch(dist.download, {
+    method: 'HEAD',
+    redirect: 'follow',
+    signal: abortCtrl.signal,
+  });
+  let headers = Object.fromEntries(resp.headers);
+
+  // Workaround for bug where METHOD changes to GET
+  abortCtrl.abort();
+  await resp.text().catch(function (err) {
+    if (err.name !== 'AbortError') {
+      throw err;
+    }
+  });
+
+  // ex: content-disposition: attachment; filename=BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
+  //  => BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
+  let name = headers['content-disposition'].replace(
+    /.*filename=([^;]+)(;|$)/,
+    '$1',
+  );
+  let download = resp.url;
+
+  return { name, download };
+};
+
+if (module === require.main) {
+  GitHubishSource.getDistributables({
+    owner: 'BeyondCodeBootcamp',
+    repo: 'DuckDNS.sh',
+    baseurl: 'https://api.github.com',
+  }).then(function (all) {
+    console.info(JSON.stringify(all, null, 2));
+  });
+}

_common/githubish.js

@@ -0,0 +1,137 @@
+'use strict';
+
+let Fetcher = require('../_common/fetcher.js');
+
+/**
+ * @typedef DistributableRaw
+ * @prop {String} name
+ * @prop {String} version
+ * @prop {Boolean} lts
+ * @prop {String} [channel]
+ * @prop {String} date
+ * @prop {String} os
+ * @prop {String} arch
+ * @prop {String} ext
+ * @prop {String} download
+ */
+
+let GitHubish = module.exports;
+
+/**
+ * Lists GitHub-Like Releases (w/ uploaded assets)
+ *
+ * @param {Object} opts
+ * @param {String} opts.owner
+ * @param {String} opts.repo
+ * @param {String} opts.baseurl
+ * @param {String} [opts.username]
+ * @param {String} [opts.token]
+ */
+GitHubish.getDistributables = async function ({
+  owner,
+  repo,
+  baseurl,
+  username = '',
+  token = '',
+}) {
+  if (!owner) {
+    throw new Error('missing owner for repo');
+  }
+  if (!repo) {
+    throw new Error('missing repo name');
+  }
+  if (!baseurl) {
+    throw new Error('missing baseurl');
+  }
+
+  let url = `${baseurl}/repos/${owner}/${repo}/releases`;
+  let opts = {
+    headers: {
+      'Content-Type': 'appplication/json',
+    },
+  };
+
+  if (token) {
+    let userpass = `${username}:${token}`;
+    let basicAuth = btoa(userpass);
+    Object.assign(opts.headers, {
+      Authorization: `Basic ${basicAuth}`,
+    });
+  }
+
+  let resp;
+  try {
+    resp = await Fetcher.fetch(url, opts);
+  } catch (e) {
+    /** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
+    let err = e;
+    if (err.code === 'E_FETCH_RELEASES') {
+      err.message = `failed to fetch '${baseurl}' (githubish, user '${username}) release data: ${err.response.status} ${err.response.body}`;
+    }
+    throw e;
+  }
+  let gHubResp = JSON.parse(resp.body);
+
+  let all = {
+    /** @type {Array<DistributableRaw>} */
+    releases: [],
+    // todo make this ':baseurl' + ':releasename'
+    download: '',
+  };
+
+  try {
+    gHubResp.forEach(transformReleases);
+  } catch (e) {
+    /** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
+    let err = e;
+    console.error(err.message);
+    console.error('Error Headers:', resp.headers);
+    console.error('Error Body:', resp.body);
+    let msg = `failed to transform releases from '${baseurl}' with user '${username}'`;
+    throw new Error(msg);
+  }
+
+  /**
+   * @param {any} release - TODO
+   */
+  function transformReleases(release) {
+    for (let asset of release['assets']) {
+      let name = asset['name'];
+      let date = release['published_at']?.replace(/T.*/, '');
+      let download = asset['browser_download_url'];
+
+      // TODO tags aren't always semver / sensical
+      let version = release['tag_name'];
+      let channel;
+      if (release['prerelease']) {
+        // -rcX, -preview, -beta, etc will be checked in _webi/normalize.js
+        channel = 'beta';
+      }
+      let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
+
+      all.releases.push({
+        name: name,
+        version: version,
+        lts: lts,
+        channel: channel,
+        date: date,
+        os: '', // will be guessed by download filename
+        arch: '', // will be guessed by download filename
+        ext: '', // will be normalized
+        download: download,
+      });
+    }
+  }
+
+  return all;
+};
+
+if (module === require.main) {
+  GitHubish.getDistributables({
+    owner: 'BurntSushi',
+    repo: 'ripgrep',
+    baseurl: 'https://api.github.com',
+  }).then(function (all) {
+    console.info(JSON.stringify(all, null, 2));
+  });
+}

_example/README.md

@@ -15,6 +15,17 @@ tagline: |
 To update or switch versions, run `webi example@stable` (or `@v2`, `@beta`,
 etc).
 
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.local/bin/foo
+~/.local/opt/foo/
+```
+
 ## Cheat Sheet
 
 > `foo` doesn't exist and this text should have been replaced. It doesn't do
@@ -22,7 +33,7 @@ etc).
 
 To run foo:
 
-```bash
+```sh
 foo
 ```
 
@@ -30,6 +41,6 @@ foo
 
 To run foo with both bar and baz highlighting turned on:
 
-```bash
+```sh
 foo --bar=baz
 ```

_example/install.ps1

@@ -15,47 +15,45 @@ $pkg_src_bin = "$Env:USERPROFILE\.local\opt\foobar-v$Env:WEBI_VERSION\bin"
 $pkg_src_dir = "$Env:USERPROFILE\.local\opt\foobar-v$Env:WEBI_VERSION"
 $pkg_src = "$pkg_src_cmd"
 
-$pkg_download = "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"
+New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
+$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
 
 # Fetch archive
-IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"))
-{
-    # TODO: arch detection
-    echo "Downloading foobar from $Env:WEBI_PKG_URL to $pkg_download"
+IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
+    Write-Output "Downloading foobar from $Env:WEBI_PKG_URL to $pkg_download"
     & curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
-    & move "$pkg_download.part" "$pkg_download"
+    & Move-Item "$pkg_download.part" "$pkg_download"
 }
 
-IF (!(Test-Path -Path "$pkg_src_cmd"))
-{
-    echo "Installing foobar"
+IF (!(Test-Path -Path "$pkg_src_cmd")) {
+    Write-Output "Installing foobar"
 
     # TODO: create package-specific temp directory
     # Enter tmp
-    pushd .local\tmp
+    Push-Location .local\tmp
 
-        # Remove any leftover tmp cruft
-        Remove-Item -Path ".\foobar-v*" -Recurse -ErrorAction Ignore
-        Remove-Item -Path ".\foo.exe" -Recurse -ErrorAction Ignore
+    # Remove any leftover tmp cruft
+    Remove-Item -Path ".\foobar-v*" -Recurse -ErrorAction Ignore
+    Remove-Item -Path ".\foo.exe" -Recurse -ErrorAction Ignore
 
-        # NOTE: DELETE THIS COMMENT IF NOT USED
-        # Move single binary into root of temporary folder
-        #& move "$pkg_download" "foo.exe"
+    # NOTE: DELETE THIS COMMENT IF NOT USED
+    # Move single binary into root of temporary folder
+    #& move "$pkg_download" "foo.exe"
 
-        # Unpack archive file into this temporary directory
-        # Windows BSD-tar handles zip. Imagine that.
-        echo "Unpacking $pkg_download"
-        & tar xf "$pkg_download"
+    # Unpack archive file into this temporary directory
+    # Windows BSD-tar handles zip. Imagine that.
+    Write-Output "Unpacking $pkg_download"
+    & tar xf "$pkg_download"
 
-        # Settle unpacked archive into place
-        echo "Install Location: $pkg_src_cmd"
-        New-Item "$pkg_src_bin" -ItemType Directory -Force | out-null
-        Move-Item -Path ".\foobar-*\foo.exe" -Destination "$pkg_src_bin"
+    # Settle unpacked archive into place
+    Write-Output "Install Location: $pkg_src_cmd"
+    New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
+    Move-Item -Path ".\foobar-*\foo.exe" -Destination "$pkg_src_bin"
 
     # Exit tmp
-    popd
+    Pop-Location
 }
 
-echo "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
-Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | out-null
+Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
+Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
 Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

_example/install.sh

@@ -1,6 +1,9 @@
-#!/bin/bash
+#!/bin/sh
 
-function __init_foobar() {
+# shellcheck disable=SC2034
+# "'pkg_cmd_name' appears unused. Verify it or export it."
+
+__init_foobar() {
     set -e
     set -u
 
@@ -21,19 +24,21 @@ function __init_foobar() {
     # pkg_install must be defined by every package
     pkg_install() {
         # ~/.local/opt/foobar-v0.99.9/bin
-        mkdir -p "$(dirname $pkg_src_cmd)"
+        mkdir -p "$(dirname "${pkg_src_cmd}")"
 
         # mv ./foobar-*/foo ~/.local/opt/foobar-v0.99.9/bin/foo
-        mv ./foobar-*/foo "$pkg_src_cmd"
+        mv ./foobar-*/foo "${pkg_src_cmd}"
     }
 
-    # pkg_get_current_version is recommended, but (soon) not required
+    # pkg_get_current_version is recommended, but not required
     pkg_get_current_version() {
         # 'foo --version' has output in this format:
         #       foobar 0.99.9 (rev abcdef0123)
         # This trims it down to just the version number:
         #       0.99.9
-        echo $(foo --version 2> /dev/null | head -n 1 | cut -d ' ' -f 2)
+        foo --version 2> /dev/null |
+            head -n 1 |
+            cut -d ' ' -f 2
     }
 
 }

_example/releases.js

@@ -12,17 +12,26 @@ var repo = 'ripgrep';
 /**                                                                          **/
 /******************************************************************************/
 
-module.exports = function (request) {
-  return github(request, owner, repo).then(function (all) {
-    return all;
-  });
+let Releases = module.exports;
+
+Releases.latest = async function () {
+  let all = await github(null, owner, repo);
+  return all;
+};
+
+Releases.sample = async function () {
+  let normalize = require('../_webi/normalize.js');
+  let all = await Releases.latest();
+  all = normalize(all);
+  // just select the first 5 for demonstration
+  all.releases = all.releases.slice(0, 5);
+  return all;
 };
 
 if (module === require.main) {
-  module.exports(require('@root/request')).then(function (all) {
-    all = require('../_webi/normalize.js')(all);
-    // just select the first 5 for demonstration
-    all.releases = all.releases.slice(0, 5);
-    console.info(JSON.stringify(all, null, 2));
-  });
+  (async function () {
+    let samples = await Releases.sample();
+
+    console.info(JSON.stringify(samples, null, 2));
+  })();
 }

_examples/built-with-goreleaser/README.md

@@ -16,7 +16,7 @@ Each example has four files that need to be modified, just slightly:
 
 See [./keypairs/](/keypairs/) as an example.
 
-```bash
+```sh
 rsync -av ./keypairs/ ./my-project/
 ```
 
@@ -26,7 +26,7 @@ Keypairs is packaged for Mac and Linux as `.tar.gz`, and as `.zip` for Windows.
 
 See [./arc/](/arc/) as an example.
 
-```bash
+```sh
 rsync -av ./arc/ ./my-project/
 ```
 

_examples/compressed-with-xz/README.md

@@ -16,7 +16,7 @@ Each example has four files that need to be modified, just slightly:
 
 See [./gitea/](/gitea/) as an example.
 
-```bash
+```sh
 rsync -av ./keypairs/ ./my-project/
 ```
 

_npm/README.md

@@ -11,7 +11,7 @@ Effortlessly install developer tools with easy-to-remember URLs.
 
 ## Install webi via npm:
 
-```bash
+```sh
 npm install -g webi
 ```
 
@@ -19,14 +19,14 @@ npm install -g webi
 
 Mac & Linux:
 
-```bash
-curl -fsS https://webinstall.dev/node | bash
+```sh
+curl -fsS https://webi.sh/node | sh
 ```
 
-Windows 10 (includes `curl.exe` and PowerShell by default):
+Windows (includes `curl.exe` and PowerShell by default):
 
-```bash
-curl.exe -fsSA "MS" https://webinstall.dev/node | powershell
+```sh
+curl.exe -fsSA "MS" https://webi.ms/node | powershell
 ```
 
 ## Example: Switching node versions
@@ -34,7 +34,7 @@ curl.exe -fsSA "MS" https://webinstall.dev/node | powershell
 Once `webi` is installed, you can then install commands or switch versions with
 webi itself:
 
-```bash
+```sh
 webi node@stable
 webi node@lts
 webi node@v10

_npm/lib/exec.js

@@ -1,15 +1,12 @@
 'use strict';
 
-var pkg = require('../package.json');
 var spawn = require('child_process').spawn;
-var os = require('os');
-var path = require('path');
 
 function spawner(args) {
   return new Promise(function (resolve, reject) {
     var bin = args.shift();
     var runner = spawn(bin, args, {
-      windowsHide: true
+      windowsHide: true,
     });
     runner.stdout.on('data', function (chunk) {
       console.info(chunk.toString('utf8'));

_npm/scripts/install-webi.js

@@ -4,7 +4,6 @@
 
 //var pkg = require('../package.json');
 var os = require('os');
-//var request = require('@root/request');
 //var promisify = require('util').promisify;
 //var exec = promisify(require('child_process').exec);
 var exec = require('child_process').exec;
@@ -14,18 +13,12 @@ if (/^win/i.test(os.platform())) {
   console.warn("This npm installer doesn't work on windows yet.");
   console.warn('Copy and paste this into cmd.exe or PowerShell instead:');
   console.warn('');
-  console.warn(
-    "    curl.exe -fsSA 'MS' https://webinstall.dev/webi | powershell"
-  );
+  console.warn('    curl.exe -fsS https://webi.ms/webi | powershell');
   console.warn('');
   return;
 }
 
-exec('curl -fsS https://webinstall.dev/webi | bash', function (
-  err,
-  stdout,
-  stderr
-) {
+exec('curl -fsS https://webi.sh/webi | sh', function (err, stdout, stderr) {
   if (err) {
     console.error(err);
   }

_scripts/git-hooks-init.js

@@ -0,0 +1,57 @@
+#!/usr/bin/env node
+'use strict';
+
+var Fs = require('node:fs/promises');
+var Path = require('node:path');
+
+async function copyScripts() {
+  var pkgDir = Path.join(__dirname, '..');
+  var scriptsDir = Path.join(pkgDir, '_scripts');
+  var gitFile = Path.join(pkgDir, '.git');
+
+  // if this is a submodule, '.git' will be a file with a
+  // path to the actual git module directory
+  var gitDir = await Fs.readFile(gitFile, 'utf8')
+    .catch(function (e) {
+      // console.error(e);
+      //return 'gitdir: ../.git/modules/installers';
+      return 'gitdir: ./.git';
+    })
+    .then(function (str) {
+      var parts = str.split(': ');
+      str = parts[1];
+      str = str.trim();
+
+      return Path.resolve(pkgDir, str);
+    });
+
+  var gitHooksDir = Path.join(gitDir, 'hooks');
+
+  var src = Path.join(scriptsDir, 'git-hooks-pre-commit.js');
+  var dst = Path.join(gitHooksDir, 'pre-commit');
+
+  console.info(`[git-hooks] Checking for pre-commit hooks...`);
+  var relSrc = Path.relative(pkgDir, src);
+  var relDst = Path.relative(pkgDir, dst);
+  await Fs.access(dst)
+    .then(function () {
+      console.info(`[git-hooks] Found ${relDst}`);
+    })
+    .catch(async function (e) {
+      // ignore e
+      await Fs.mkdir(gitHooksDir, { recursive: true });
+      await Fs.copyFile(src, dst);
+      await Fs.chmod(dst, 0o755);
+      console.info(`[git-hooks] Found template ${relSrc}`);
+      console.info(`[git-hooks] Initialized ${relDst}`);
+    });
+}
+
+copyScripts()
+  .then(function () {
+    process.exit(0);
+  })
+  .catch(function (e) {
+    console.error(e);
+    process.exit(1);
+  });

_scripts/git-hooks-pre-commit

@@ -0,0 +1,7 @@
+#!/bin/sh
+set -e
+set -u
+
+npm run fmt
+npm run lint
+npm run test

_scripts/git-hooks-pre-commit.js

@@ -0,0 +1,12 @@
+#!/usr/bin/env node
+'use strict';
+
+var Process = require('child_process');
+
+var procOpts = { stdio: 'inherit' };
+var commands = ['npm run fmt', 'npm run lint', 'npm run test'];
+
+for (let cmd of commands) {
+  console.info(`[pre-commit] exec: ${cmd}`);
+  Process.execSync(cmd, procOpts);
+}

_scripts/install-ci-deps

@@ -0,0 +1,15 @@
+#!/bin/sh
+set -e
+set -u
+
+# Install 'shfmt 3.7.x'
+curl -fsS https://webi.sh/shfmt@3.7 | sh
+
+# Install 'shellcheck v0.9.x'
+curl -fsS https://webi.sh/shellcheck@0.9 | sh
+
+# Install 'pwsh (PowerShell Core) v7.x'
+curl -fsS https://webi.sh/pwsh@7 | sh
+# shellcheck disable=SC1090
+. ~/.config/envman/PATH.env
+pwsh -Command "Install-Module -Name PSScriptAnalyzer -Scope CurrentUser -AllowClobber"

_scripts/pwsh-fmt

@@ -0,0 +1,27 @@
+#!/bin/sh
+set -e
+set -u
+
+echo "Formatting */*.ps1 ... "
+for my_ps1 in */*.ps1; do
+    my_dir="$(
+        dirname "${my_ps1}"
+    )"
+    if test -L "${my_ps1}" ||
+        test -L "${my_dir}" ||
+        ! test -f "${my_ps1}" ||
+        ! test -d "${my_dir}"; then
+        printf '    SKIP %s (non-regular file or parent directory)\n' "${my_ps1}"
+        continue
+    fi
+
+    printf "    %s" "${my_ps1}"
+
+    # -Settings ./Settings/CodeFormatting.psd1
+    my_new_file="$(
+        pwsh -Command "Invoke-Formatter -ScriptDefinition (Get-Content -Path '${my_ps1}' -Raw)"
+    )"
+    printf '%s\n' "${my_new_file}" > "${my_ps1}"
+
+    printf '\n'
+done

_scripts/pwsh-fmt.ps1

@@ -0,0 +1,40 @@
+#!/usr/bin/env pwsh
+
+$ErrorActionPreference = "Stop"
+
+Write-Host "Formatting */*.ps1 ... "
+
+$my_cwd = Get-Location
+$my_dirs = Get-ChildItem -Directory
+$my_status = 0
+
+foreach ($my_dir in $my_dirs) {
+
+    $my_files = Get-ChildItem -Path $my_dir.FullName -File -Filter *.ps1
+    foreach ($my_file in $my_files) {
+        $my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
+        $my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
+
+        if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
+            -Not (Test-Path -PathType Container -Path $my_dir)) {
+            Write-Host ("    SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
+            continue
+        }
+
+        Write-Host ("    {0}" -f $my_ps1)
+
+        $text = Get-Content -Path $my_ps1 -Raw
+        $my_new_file = Invoke-Formatter -ScriptDefinition $text
+        $my_new_file = $my_new_file.Trim()
+
+        # note: trailing newline is added back on write
+        $my_new_file | Set-Content -Path $my_ps1
+
+        $my_new_file = $my_new_file + "`n"
+        IF ($text -ne $my_new_file) {
+            $my_status = 1
+        }
+    }
+}
+
+exit $my_status

_scripts/pwsh-lint

@@ -0,0 +1,36 @@
+#!/bin/sh
+set -e
+set -u
+
+echo "Formatting */*.ps1 ..."
+
+for my_ps1 in */*.ps1; do
+    my_dir="$(
+        dirname "${my_ps1}"
+    )"
+    if test -L "${my_ps1}" ||
+        test -L "${my_dir}" ||
+        ! test -f "${my_ps1}" ||
+        ! test -d "${my_dir}"; then
+        printf '    SKIP %s (non-regular file or parent directory)\n' "${my_ps1}"
+        continue
+    fi
+
+    printf "    %s" "${my_ps1}"
+
+    # -Settings ./PSScriptAnalyzerSettings.psd1
+    pwsh -Command "Invoke-ScriptAnalyzer -Fix -ExcludeRule PSAvoidUsingWriteHost,PSUseDeclaredVarsMoreThanAssignment -Path '$my_ps1'"
+
+    #
+    # fmt MUST happen after lint due to Byte-Order Marker (BOM) issues
+    # See https://github.com/PowerShell/PSScriptAnalyzer/issues/1743
+    #
+
+    # -Settings ./Settings/CodeFormatting.psd1
+    my_new_file="$(
+        pwsh -Command "Invoke-Formatter -ScriptDefinition (Get-Content -Path '${my_ps1}' -Raw)"
+    )"
+    printf '%s\n' "${my_new_file}" > "${my_ps1}"
+
+    printf '\n'
+done

_scripts/pwsh-lint.ps1

@@ -0,0 +1,48 @@
+#!/usr/bin/env pwsh
+
+$ErrorActionPreference = "Stop"
+
+Write-Host "Linting */*.ps1 ... "
+
+$my_cwd = Get-Location
+$my_dirs = Get-ChildItem -Directory
+$my_status = 0
+
+foreach ($my_dir in $my_dirs) {
+
+    $my_files = Get-ChildItem -Path $my_dir.FullName -File -Filter *.ps1
+    foreach ($my_file in $my_files) {
+        $my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
+        $my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
+
+        if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
+            -Not (Test-Path -PathType Container -Path $my_dir)) {
+            Write-Host ("    SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
+            continue
+        }
+
+        Write-Host ("    {0}" -f $my_ps1)
+
+        $my_old_file = (Get-Content -Path $my_ps1 -Raw)
+        Invoke-ScriptAnalyzer -Fix -ExcludeRule PSAvoidUsingWriteHost, PSUseDeclaredVarsMoreThanAssignment -Path $my_ps1
+
+        #
+        # fmt MUST happen after lint due to Byte-Order Marker (BOM) issues
+        # See https://github.com/PowerShell/PSScriptAnalyzer/issues/1743
+        #
+
+        $my_fixed_file = (Get-Content -Path $my_ps1 -Raw)
+        $my_new_file = Invoke-Formatter -ScriptDefinition $my_fixed_file
+        $my_new_file = $my_new_file.Trim()
+
+        # note: trailing newline is added back on write
+        $my_new_file | Set-Content -Path $my_ps1
+
+        $my_new_file = $my_new_file + "`n"
+        IF ($my_old_file -ne $my_new_file) {
+            $my_status = 1
+        }
+    }
+}
+
+exit $my_status

_scripts/tooling-init.js

@@ -0,0 +1,32 @@
+#!/usr/bin/env node
+'use strict';
+
+var Cmd = require('node:child_process');
+var Os = require('node:os');
+
+var procOpts = { stdio: 'inherit' };
+var commands = ['shfmt --version', 'shellcheck --version'];
+
+for (let cmd of commands) {
+  console.info(`[tooling-init] checking for '${cmd}':`);
+  try {
+    Cmd.execSync(cmd, procOpts);
+  } catch (e) {
+    // ignore e
+    printInstallHelp(cmd);
+    process.exit(1);
+  }
+}
+
+function printInstallHelp(cmd) {
+  console.error('');
+  console.error('ERROR');
+  console.error(`        could not run '${cmd}'`);
+  console.error('POSSIBLE FIX');
+  if (/^win/i.test(Os.platform())) {
+    console.error(`        curl.exe https://webi.ms/${cmd} | powershell`);
+  } else {
+    console.error(`        curl -fsS https://webi.sh/${cmd} | sh`);
+  }
+  console.error('');
+}

_vim-example/README.md

@@ -26,7 +26,7 @@ If there are some important key bindings, use a table like this:
 
 1. Place EXAMPLE into your `~/.vim/pack/plugins/start`:
 
-   ```bash
+   ```sh
    mkdir -p ~/.vim/pack/plugins/start/
    git clone --depth=1 https://github.com/CHANGEME/EXAMPLE.git ~/.vim/pack/plugins/start/example
    ```

_vim-example/install.ps1

@@ -1,7 +1,7 @@
 #!/usr/bin/env pwsh
 
 IF (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
-    New-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start" -ItemType Directory -Force | out-null
+    New-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start" -ItemType Directory -Force | Out-Null
 }
 Remove-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start\example" -Recurse -ErrorAction Ignore
 & git clone --depth=1 https://github.com/CHANGEME/example.git "$Env:USERPROFILE\.vim\pack\plugins\start\example"

_vim-example/install.sh

@@ -1,6 +1,6 @@
-#!/bin/bash
+#!/bin/sh
 
-function __init_vim_example() {
+__init_vim_example() {
     set -e
     set -u
 

_webi/bootstrap.bat

@@ -1,28 +0,0 @@
-setlocal
-@echo off
-pushd "%userprofile%" || goto :error
-  IF NOT EXIST .local (
-    mkdir .local || goto :error
-  )
-  IF NOT EXIST .local\bin (
-    mkdir .local\bin || goto :error
-  )
-
-  rem {{ baseurl }}
-  rem {{ version }}
-  pushd .local\bin || goto :error
-    if NOT EXIST webi.bat (
-      rem without SilentlyContinue this is SLOOOOOOOOOOOOOOOW!
-      powershell $ProgressPreference = 'SilentlyContinue'; Invoke-WebRequest https://webinstall.dev/packages/_webi/webi.bat -OutFile webi.bat || goto :error
-    )
-    call .\webi {{ exename }} || goto :error
-    rem pathman add "%userprofile%\.local\bin" >nul 2>&1 || goto :error
-    pathman add "%userprofile%\.local\bin" || goto :error
-  popd || goto :error
-popd
-
-goto :EOF
-
-:error
-echo Failed with error #%errorlevel%.
-exit /b %errorlevel%

_webi/bootstrap.ps1

@@ -1,6 +0,0 @@
-# Download the latest webi, then install {{ exename }}
-New-Item -Path .local\bin -ItemType Directory -Force | out-null
-IF ($Env:WEBI_HOST -eq $null -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
-curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/_webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
-Set-ExecutionPolicy -Scope Process Bypass
-& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" "{{ exename }}"

_webi/bootstrap.sh

@@ -1,172 +0,0 @@
-#!/bin/bash
-
-#set -x
-
-function __install_webi() {
-
-    #WEBI_PKG=
-    #WEBI_HOST=https://webinstall.dev
-    export WEBI_HOST
-
-    echo ""
-    echo "Thanks for using webi to install '${WEBI_PKG:-}' on '$(uname -s)/$(uname -m)'."
-    echo "Have a problem? Experience a bug? Please let us know:"
-    echo "        https://github.com/webinstall/packages/issues"
-    echo ""
-    WEBI_WELCOME=true
-    export WEBI_WELCOME
-
-    set -e
-    set -u
-
-    mkdir -p "$HOME/.local/bin"
-
-    cat << EOF > "$HOME/.local/bin/webi"
-#!/bin/bash
-
-set -e
-set -u
-#set -x
-
-function __webi_main () {
-
-    export WEBI_TIMESTAMP=\$(date +%F_%H-%M-%S)
-    export _webi_tmp="\${_webi_tmp:-\$(mktemp -d -t webi-\$WEBI_TIMESTAMP.XXXXXXXX)}"
-
-    if [ -n "\${_WEBI_PARENT:-}" ]; then
-        export _WEBI_CHILD=true
-    else
-        export _WEBI_CHILD=
-    fi
-    export _WEBI_PARENT=true
-
-    ##
-    ## Detect acceptable package formats
-    ##
-
-    my_ext=""
-    set +e
-    # NOTE: the order here is least favorable to most favorable
-    if [ -n "\$(command -v pkgutil)" ]; then
-        my_ext="pkg,\$my_ext"
-    fi
-    # disable this check for the sake of building the macOS installer on Linux
-    #if [ -n "\$(command -v diskutil)" ]; then
-        # note: could also detect via hdiutil
-        my_ext="dmg,\$my_ext"
-    #fi
-    if [ -n "\$(command -v git)" ]; then
-        my_ext="git,\$my_ext"
-    fi
-    if [ -n "\$(command -v unxz)" ]; then
-        my_ext="xz,\$my_ext"
-    fi
-    if [ -n "\$(command -v unzip)" ]; then
-        my_ext="zip,\$my_ext"
-    fi
-    # for mac/linux 'exe' refers to the uncompressed binary without extension
-    my_ext="exe,\$my_ext"
-    if [ -n "\$(command -v tar)" ]; then
-        my_ext="tar,\$my_ext"
-    fi
-    my_ext="\$(echo "\$my_ext" | sed 's/,$//')" # nix trailing comma
-    set -e
-
-
-    ##
-    ## Detect http client
-    ##
-
-    set +e
-    export WEBI_CURL="\$(command -v curl)"
-    export WEBI_WGET="\$(command -v wget)"
-    set -e
-
-    export WEBI_HOST="\${WEBI_HOST:-https://webinstall.dev}"
-    export WEBI_UA="\$(uname -a)"
-
-
-    function webinstall() {
-
-        my_package="\${1:-}"
-        if [ -z "\$my_package" ]; then
-            >&2 echo "Usage: webi <package>@<version> ..."
-            >&2 echo "Example: webi node@lts rg"
-            exit 1
-        fi
-
-        export WEBI_BOOT="\$(mktemp -d -t "\$my_package-bootstrap.\$WEBI_TIMESTAMP.XXXXXXXX")"
-
-        my_installer_url="\$WEBI_HOST/api/installers/\$my_package.sh?formats=\$my_ext"
-        set +e
-        if [ -n "\$WEBI_CURL" ]; then
-            curl -fsSL "\$my_installer_url" -H "User-Agent: curl \$WEBI_UA" \\
-                -o "\$WEBI_BOOT/\$my_package-bootstrap.sh"
-        else
-            wget -q "\$my_installer_url" --user-agent="wget \$WEBI_UA" \\
-                -O "\$WEBI_BOOT/\$my_package-bootstrap.sh"
-        fi
-        if ! [ \$? -eq 0 ]; then
-          >&2 echo "error fetching '\$my_installer_url'"
-          exit 1
-        fi
-        set -e
-
-        pushd "\$WEBI_BOOT" 2>&1 > /dev/null
-            bash "\$my_package-bootstrap.sh"
-        popd 2>&1 > /dev/null
-
-        rm -rf "\$WEBI_BOOT"
-
-    }
-
-    show_path_updates() {
-
-        if ! [ -n "\${_WEBI_CHILD}" ]; then
-            if [ -f "\$_webi_tmp/.PATH.env" ]; then
-                my_paths=\$(cat "\$_webi_tmp/.PATH.env" | sort -u)
-                if [ -n "\$my_paths" ]; then
-                    echo "IMPORTANT: You must update you PATH to use the installed program(s)"
-                    echo ""
-                    echo "You can either"
-                    echo "A) can CLOSE and REOPEN Terminal or"
-                    echo "B) RUN these exports:"
-                    echo ""
-                    echo "\$my_paths"
-                    echo ""
-                fi
-                rm -f "\$_webi_tmp/.PATH.env"
-            fi
-        fi
-
-    }
-
-    for pkgname in "\$@"
-    do
-        webinstall "\$pkgname"
-    done
-
-    show_path_updates
-
-}
-
-__webi_main "\$@"
-
-EOF
-
-    chmod a+x "$HOME/.local/bin/webi"
-
-    if [ -n "${WEBI_PKG:-}" ]; then
-        "$HOME/.local/bin/webi" "${WEBI_PKG}"
-    else
-        echo ""
-        echo "Hmm... no WEBI_PKG was specified. This is probably an error in the script."
-        echo ""
-        echo "Please open an issue with this information: Package '${WEBI_PKG:-}' on '$(uname -s)/$(uname -m)'"
-        echo "    https://github.com/webinstall/packages/issues"
-        echo ""
-    fi
-
-}
-
-__install_webi

_webi/builds-cacher-test.js

@@ -0,0 +1,73 @@
+'use strict';
+
+let Path = require('node:path');
+
+let BuildsCacher = require('./builds-cacher.js');
+// let Parallel = require('./parallel.js');
+
+var INSTALLERS_DIR = Path.join(__dirname, '..');
+var CACHE_DIR = Path.join(__dirname, '../_cache');
+
+async function main() {
+  let bc = BuildsCacher.create({
+    caches: CACHE_DIR,
+    installers: INSTALLERS_DIR,
+  });
+  bc.freshenRandomPackage(600 * 1000);
+
+  // let dirs = await bc.getProjectsByType();
+  // let projNames = Object.keys(dirs.valid);
+
+  let lastUpdate;
+
+  let projName = 'k9s';
+  {
+    let packages = await bc.getPackages({
+      //Releases: Releases,
+      name: projName,
+      date: new Date(),
+    });
+    lastUpdate = packages.updated;
+    console.info(
+      `Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
+    );
+  }
+
+  console.info('Waiting 5s');
+  {
+    setTimeout(async function () {
+      let packages = await bc.getPackages({
+        //Releases: Releases,
+        name: projName,
+        date: new Date(),
+      });
+      console.info(
+        `Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
+      );
+      if (lastUpdate < packages.updated) {
+        console.info(`PASS`);
+      } else {
+        console.info(`MAYBE fail`);
+      }
+    }, 5 * 1000);
+  }
+
+  //let parallel = 25;
+  //await Parallel.run(parallel, projNames, getAll);
+  //async function getAll(name) {
+  //  void (await bc.getPackages({
+  //    //Releases: Releases,
+  //    name: name,
+  //    date: new Date(),
+  //  }));
+  //}
+}
+
+main()
+  .then(function () {
+    console.log('Done');
+  })
+  .catch(function (e) {
+    console.error(e.stack || e);
+    process.exit(1);
+  });

_webi/builds.js

@@ -0,0 +1,41 @@
+'use strict';
+
+let Builds = module.exports;
+
+let Path = require('node:path');
+
+let BuildsCacher = require('./builds-cacher.js');
+// let HostTargets = require('./build-classifier/host-targets.js');
+let Parallel = require('./parallel.js');
+
+var INSTALLERS_DIR = Path.join(__dirname, '..');
+var CACHE_DIR = Path.join(__dirname, '../_cache');
+
+let bc = BuildsCacher.create({
+  caches: CACHE_DIR,
+  installers: INSTALLERS_DIR,
+});
+bc.freshenRandomPackage(600 * 1000);
+
+Builds.init = async function () {
+  bc.freshenRandomPackage(600 * 1000);
+
+  let dirs = await bc.getProjectsByType();
+  let projNames = Object.keys(dirs.valid);
+
+  let parallel = 25;
+  await Parallel.run(parallel, projNames, getAll);
+  async function getAll(name) {
+    void (await bc.getPackages({
+      //Releases: Releases,
+      name: name,
+      date: new Date(),
+    }));
+  }
+};
+
+Builds.enumerateLatestVersions = bc.enumerateLatestVersions;
+Builds.findMatchingPackages = bc.findMatchingPackages;
+Builds.getPackage = bc.getPackages;
+Builds.getProjectType = bc.getProjectType;
+Builds.selectPackage = bc.selectPackage;

_webi/classify-one.js

@@ -0,0 +1,90 @@
+'use strict';
+
+let Path = require('node:path');
+
+// let Builds = require('./builds.js');
+let BuildsCacher = require('./builds-cacher.js');
+let Triplet = require('./build-classifier/triplet.js');
+
+async function main() {
+  let projName = process.argv[2];
+  if (!projName) {
+    console.error(``);
+    console.error(`USAGE`);
+    console.error(``);
+    console.error(`    classify-one <project-name>`);
+    console.error(``);
+    console.error(`EXAMPLE`);
+    console.error(``);
+    console.error(`    classify-one caddy`);
+    console.error(``);
+    return;
+  }
+
+  let tsDate = new Date(0);
+  let meta = {
+    // version info
+    versions: [],
+    lexvers: [],
+    lexversMap: {},
+    // culled release assets
+    packages: [],
+    releasesByTriplet: {},
+    // target info
+    triplets: [],
+    oses: [],
+    arches: [],
+    libcs: [],
+    formats: [],
+    // TODO channels: [],
+  };
+
+  let installersDir = Path.join(__dirname, '..');
+  let Releases = require(`${installersDir}/${projName}/releases.js`);
+  if (!Releases.latest) {
+    Releases.latest = Releases;
+  }
+
+  let projInfo = await Releases.latest();
+
+  // let packages = await Builds.getPackage({ name: projName });
+  // console.log(packages);
+
+  let bc = {};
+  bc.ALL_TERMS = Triplet.TERMS_PRIMARY_MAP;
+  bc.orphanTerms = Object.assign({}, bc.ALL_TERMS);
+  bc.unknownTerms = {};
+  bc.usedTerms = {};
+  bc.formats = [];
+  bc._targetsByBuildIdCache = {};
+  bc._triplets = {};
+
+  let transformed = BuildsCacher.transformAndUpdate(
+    projName,
+    projInfo,
+    meta,
+    tsDate,
+    bc,
+  );
+
+  console.log(`[DEBUG] transformed`);
+  let sample = transformed.packages.slice(0, 20);
+  console.log('packages:', sample, ':packages');
+  console.log(
+    'releasesByTriplet:',
+    transformed.releasesByTriplet['linux-x86_64-none'][transformed.versions[0]],
+    ':releasesByTriplet',
+  );
+  console.log('versions:', transformed.versions, ':versions');
+  console.log('triplets:', transformed.triplets, ':triplets');
+  console.log('oses:', transformed.oses, ':oses');
+  console.log('arches:', transformed.arches, ':arches');
+  console.log('libcs:', transformed.libcs, ':libcs');
+  console.log('formats:', transformed.formats, ':formats');
+  console.log(Object.keys(transformed));
+}
+
+main().catch(function (err) {
+  console.error('Error:');
+  console.error(err);
+});

_webi/curl-pipe-bootstrap.tpl.ps1

@@ -0,0 +1,12 @@
+# Download the latest webi, then install {{ exename }}
+# <pre>
+############################################################
+# <h1>Cheat Sheet at CHEATSHEET_URL</h1>
+# <meta http-equiv="refresh" content="3; URL='CHEATSHEET_URL'" />
+############################################################
+New-Item -Path "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
+New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
+IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
+curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
+Set-ExecutionPolicy -Scope Process Bypass
+& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" "{{ exename }}"

_webi/curl-pipe-bootstrap.tpl.sh

@@ -0,0 +1,349 @@
+#!/bin/sh
+#<pre>
+
+############################################################
+# <h1>Cheat Sheet at CHEATSHEET_URL</h1>
+# <meta http-equiv="refresh" content="3; URL='CHEATSHEET_URL'" />
+############################################################
+
+export WEBI_PKG=webi
+export WEBI_HOST=https://webinstall.dev
+export WEBI_CHECKSUM=06a7fb9f
+
+#########################################
+#                                       #
+# Display Debug Info in Case of Failure #
+#                                       #
+#########################################
+
+fn_show_welcome() { (
+    echo ""
+    echo ""
+    # invert t_task and t_pkg for top-level welcome message
+    printf -- ">>> %s %s  <<<\n" \
+        "$(t_pkg 'Welcome to') $(t_task 'Webi')$(t_pkg '!')" \
+        "$(t_dim "- modern tools, instant installs.")"
+    echo "    We expect your experience to be $(t_em 'absolutely perfect')!"
+    echo ""
+    echo "    $(t_attn 'Success')? Star it!   $(t_url 'https://github.com/webinstall/webi-installers')"
+    echo "    $(t_attn 'Problem')? Report it: $(t_url 'https://github.com/webinstall/webi-installers/issues')"
+    echo "                        $(t_dim "(your system is") $(t_host "$(fn_get_os)")/$(t_host "$(uname -m)") $(t_dim "with") $(t_host "$(fn_get_libc)") $(t_dim "&") $(t_host "$(fn_get_http_client_name)")$(t_dim ")")"
+
+    sleep 0.2
+); }
+
+fn_get_os() { (
+    # Ex:
+    #     GNU/Linux
+    #     Android
+    #     Linux (often Alpine, musl)
+    #     Darwin
+    b_os="$(uname -o 2> /dev/null || echo '')"
+    b_sys="$(uname -s)"
+    if test -z "${b_os}" || test "${b_os}" = "${b_sys}"; then
+        # ex: 'Darwin' (and plain, non-GNU 'Linux')
+        echo "${b_sys}"
+        return 0
+    fi
+
+    if echo "${b_os}" | grep -q "${b_sys}"; then
+        # ex: 'GNU/Linux'
+        echo "${b_os}"
+        return 0
+    fi
+
+    # ex: 'Android/Linux'
+    echo "${b_os}/${b_sys}"
+); }
+
+fn_get_libc() { (
+    # Ex:
+    #     musl
+    #     libc
+    if ldd /bin/ls 2> /dev/null | grep -q 'musl' 2> /dev/null; then
+        echo 'musl'
+    elif fn_get_os | grep -q 'GNU|Linux'; then
+        echo 'gnu'
+    else
+        echo 'libc'
+    fi
+); }
+
+fn_get_http_client_name() { (
+    # Ex:
+    #     curl
+    #     curl+wget
+    b_client=""
+    if command -v curl > /dev/null; then
+        b_client="curl"
+    fi
+    if command -v wget > /dev/null; then
+        if test -z "${b_client}"; then
+            b_client="wget"
+        else
+            b_client="curl+wget"
+        fi
+    fi
+
+    echo "${b_client}"
+); }
+
+#########################################
+#                                       #
+#      For Making the Display Nice      #
+#                                       #
+#########################################
+
+# Term Types
+t_cmd() { (fn_printf '\e[2m\e[35m%s\e[39m\e[22m' "${1}"); }
+t_host() { (fn_printf '\e[2m\e[33m%s\e[39m\e[22m' "${1}"); }
+t_link() { (fn_printf '\e[1m\e[36m%s\e[39m\e[22m' "${1}"); }
+t_path() { (fn_printf '\e[2m\e[32m%s\e[39m\e[22m' "${1}"); }
+t_pkg() { (fn_printf '\e[1m\e[32m%s\e[39m\e[22m' "${1}"); }
+t_task() { (fn_printf '\e[36m%s\e[39m' "${1}"); }
+t_url() { (fn_printf '\e[2m%s\e[22m' "${1}"); }
+
+# Levels
+t_info() { (fn_printf '\e[1m\e[36m%s\e[39m\e[22m' "${1}"); }
+t_attn() { (fn_printf '\e[1m\e[33m%s\e[39m\e[22m' "${1}"); }
+t_warn() { (fn_printf '\e[1m\e[33m%s\e[39m\e[22m' "${1}"); }
+t_err() { (fn_printf '\e[31m%s\e[39m' "${1}"); }
+
+# Styles
+t_bold() { (fn_printf '\e[1m%s\e[22m' "${1}"); }
+t_dim() { (fn_printf '\e[2m%s\e[22m' "${1}"); }
+t_em() { (fn_printf '\e[3m%s\e[23m' "${1}"); }
+t_under() { (fn_printf '\e[4m%s\e[24m' "${1}"); }
+
+# FG Colors
+t_cyan() { (fn_printf '\e[36m%s\e[39m' "${1}"); }
+t_green() { (fn_printf '\e[32m%s\e[39m' "${1}"); }
+t_magenta() { (fn_printf '\e[35m%s\e[39m' "${1}"); }
+t_yellow() { (fn_printf '\e[33m%s\e[39m' "${1}"); }
+
+fn_printf() { (
+    a_style="${1}"
+    a_text="${2}"
+    if fn_is_tty; then
+        #shellcheck disable=SC2059
+        printf -- "${a_style}" "${a_text}"
+    else
+        printf -- '%s' "${a_text}"
+    fi
+); }
+
+fn_sub_home() { (
+    my_rel=${HOME}
+    my_abs=${1}
+    echo "${my_abs}" | sed "s:^${my_rel}:~:"
+); }
+
+###################################
+#                                 #
+#       Detect HTTP Client        #
+#                                 #
+###################################
+
+fn_wget() { (
+    # Doc:
+    #     Downloads the file at the given url to the given path
+    a_url="${1}"
+    a_path="${2}"
+
+    cmd_wget="wget -c -q --user-agent"
+    if fn_is_tty; then
+        cmd_wget="wget -c -q --show-progress --user-agent"
+    fi
+    # busybox wget doesn't support --show-progress
+    # See <https://github.com/webinstall/webi-installers/pull/772>
+    if readlink "$(command -v wget)" | grep -q busybox; then
+        cmd_wget="wget --user-agent"
+    fi
+
+    b_triple_ua="$(fn_get_target_triple_user_agent)"
+    b_agent="webi/wget ${b_triple_ua}"
+    if command -v curl > /dev/null; then
+        b_agent="webi/wget+curl ${b_triple_ua}"
+    fi
+
+    if ! $cmd_wget "${b_agent}" "${a_url}" -O "${a_path}"; then
+        echo >&2 "    $(t_err "failed to download (wget)") '$(t_url "${a_url}")'"
+        echo >&2 "    $cmd_wget '${b_agent}' '${a_url}' -O '${a_path}'"
+        echo >&2 "    $(wget -V)"
+        return 1
+    fi
+); }
+
+fn_curl() { (
+    # Doc:
+    #     Downloads the file at the given url to the given path
+    a_url="${1}"
+    a_path="${2}"
+
+    cmd_curl="curl -f -sSL -#"
+    if fn_is_tty; then
+        cmd_curl="curl -f -sSL"
+    fi
+
+    b_triple_ua="$(fn_get_target_triple_user_agent)"
+    b_agent="webi/curl ${b_triple_ua}"
+    if command -v wget > /dev/null; then
+        b_agent="webi/curl+wget ${b_triple_ua}"
+    fi
+
+    if ! $cmd_curl -A "${b_agent}" "${a_url}" -o "${a_path}"; then
+        echo >&2 "    $(t_err "failed to download (curl)") '$(t_url "${a_url}")'"
+        echo >&2 "    $cmd_curl -A '${b_agent}' '${a_url}' -o '${a_path}'"
+        echo >&2 "    $(curl -V)"
+        return 1
+    fi
+); }
+
+fn_get_target_triple_user_agent() { (
+    # Ex:
+    #     x86_64/unknown GNU/Linux/5.15.107-2-pve gnu
+    #     arm64/unknown Darwin/22.6.0 libc
+    echo "$(uname -m)/unknown $(fn_get_os)/$(uname -r) $(fn_get_libc)"
+); }
+
+fn_download_to_path() { (
+    a_url="${1}"
+    a_path="${2}"
+
+    mkdir -p "$(dirname "${a_path}")"
+    if command -v curl > /dev/null; then
+        fn_curl "${a_url}" "${a_path}.part"
+    elif command -v wget > /dev/null; then
+        fn_wget "${a_url}" "${a_path}.part"
+    else
+        echo >&2 "    $(t_err "failed to detect HTTP client (curl, wget)")"
+        return 1
+    fi
+    mv "${a_path}.part" "${a_path}"
+); }
+
+##############################################
+#                                            #
+# Install or Update Webi and Install Package #
+#                                            #
+##############################################
+
+webi_bootstrap() { (
+    a_path="${1}"
+
+    echo ""
+    echo "$(t_task 'Bootstrapping') $(t_pkg 'Webi')"
+
+    b_path_rel="$(fn_sub_home "${a_path}")"
+    b_checksum=""
+    if test -r "${a_path}"; then
+        b_checksum="$(fn_checksum "${a_path}")"
+    fi
+    if test "$b_checksum" = "${WEBI_CHECKSUM}"; then
+        echo "    $(t_dim 'Found') $(t_path "${b_path_rel}")"
+        sleep 0.1
+        return 0
+    fi
+
+    b_webi_file_url="${WEBI_HOST}/packages/webi/webi.sh"
+    b_tmp=''
+    if test -r "${a_path}"; then
+        b_ts="$(date -u '+%s')"
+        b_tmp="${a_path}.${b_ts}.bak"
+        mv "${a_path}" "${b_tmp}"
+        echo "    Updating $(t_path "${b_path_rel}")"
+    fi
+
+    echo "    Downloading $(t_url "${b_webi_file_url}")"
+    echo "        to $(t_path "${b_path_rel}")"
+    fn_download_to_path "${b_webi_file_url}" "${a_path}"
+    chmod u+x "${a_path}"
+
+    if test -r "${b_tmp}"; then
+        rm -f "${b_tmp}"
+    fi
+); }
+
+fn_checksum() {
+    a_filepath="${1}"
+
+    if command -v sha1sum > /dev/null; then
+        sha1sum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
+        return 0
+    fi
+
+    if command -v shasum > /dev/null; then
+        shasum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
+        return 0
+    fi
+
+    if command -v sha1 > /dev/null; then
+        sha1 "${a_filepath}" | cut -d'=' -f2 | cut -c 2-9
+        return 0
+    fi
+
+    echo >&2 "    warn: no sha1 sum program"
+    date '+%F %H:%M'
+}
+
+##############################################
+#                                            #
+#          Detect TTY and run main           #
+#                                            #
+##############################################
+
+fn_is_tty() {
+    if test "${WEBI_TTY}" = 'tty'; then
+        return 0
+    fi
+    return 1
+}
+
+fn_detect_tty() { (
+    # stdin will NOT be a tty if it's being piped
+    # stdout & stderr WILL be a tty even when piped
+    # they are not a tty if being captured or redirected
+    # 'set -i' is NOT available in sh
+    if test -t 1 && test -t 2; then
+        return 0
+    fi
+
+    return 1
+); }
+
+main() { (
+    set -e
+    set -u
+
+    WEBI_TTY="${WEBI_TTY:-}"
+    if test -z "${WEBI_TTY}"; then
+        if fn_detect_tty; then
+            WEBI_TTY="tty"
+        fi
+        export WEBI_TTY
+    fi
+
+    if test -z "${WEBI_WELCOME:-}"; then
+        fn_show_welcome
+    fi
+    export WEBI_WELCOME='shown'
+
+    # note: we may support custom locations in the future
+    export WEBI_HOME="${HOME}/.local"
+    b_home="$(fn_sub_home "${WEBI_HOME}")"
+    b_webi_path="${WEBI_HOME}/bin/webi"
+    b_webi_path_rel="${b_home}/bin/webi"
+
+    WEBI_CURRENT="${WEBI_CURRENT:-}"
+    if test "${WEBI_CURRENT}" != "${WEBI_CHECKSUM}"; then
+        webi_bootstrap "${b_webi_path}"
+        export WEBI_CURRENT="${WEBI_CHECKSUM}"
+    fi
+
+    echo "    Running $(t_cmd "${b_webi_path_rel} ${WEBI_PKG}")"
+    echo ""
+
+    "${b_webi_path}" "${WEBI_PKG}"
+); }
+
+main

_webi/example_install_safe_copy.sh

@@ -1,14 +0,0 @@
-# For installing from the extracted package tmp directory
-pkg_install() {
-    pushd "$WEBI_TMP" 2>&1 > /dev/null
-
-    if [ -n "$(command -v rsync 2> /dev/null | grep rsync)" ]; then
-        rsync -Krl ./xmpl*/ "$pkg_src/" 2> /dev/null
-    else
-        cp -Hr ./xmpl*/* "$pkg_src/" 2> /dev/null
-        cp -Hr ./xmpl*/.* "$pkg_src/" 2> /dev/null
-    fi
-    rm -rf ./xmpl*
-
-    popd 2>&1 > /dev/null
-}

_webi/frontmarker.js

@@ -1,7 +1,7 @@
 'use strict';
 
 var fs = require('fs');
-var marked = require('marked');
+var marked = require('marked').marked;
 
 var frontmatter = '---';
 var keyValRe = /(\w+): (.*)/;
@@ -12,7 +12,6 @@ function parseYamlish(txt) {
   var block = false;
 
   var lines = txt.trim().split('\n');
-  var moreRe = /\s+/;
   var last;
 
   if (frontmatter !== lines.shift()) {
@@ -20,7 +19,7 @@ function parseYamlish(txt) {
   }
 
   function unblock() {
-    cfg[block] = marked(cfg[block]);
+    cfg[block] = marked.parse(cfg[block]);
     block = false;
   }
 
@@ -58,7 +57,7 @@ function parseYamlish(txt) {
     var m = line.match(keyValRe);
     if (!m) {
       throw new Error(
-        'invalid key format for: ' + JSON.stringify(line) + ' ' + i
+        'invalid key format for: ' + JSON.stringify(line) + ' ' + i,
       );
     }
     if ('|' === m[2]) {
@@ -69,9 +68,9 @@ function parseYamlish(txt) {
   });
 
   if (block) {
-    cfg[block] = marked(cfg[block]);
+    cfg[block] = marked.parse(cfg[block]);
   }
-  cfg.examples = marked(lines.slice(last).join('\n'));
+  cfg.examples = marked.parse(lines.slice(last).join('\n'));
 
   return cfg;
 }
@@ -80,8 +79,6 @@ module.exports.parse = parseYamlish;
 
 if (require.main === module) {
   console.info(
-    parseYamlish(
-      fs.readFileSync(__dirname + '/../node/README.md', 'utf8')
-    )
+    parseYamlish(fs.readFileSync(__dirname + '/../node/README.md', 'utf8')),
   );
 }

_webi/installers.js

@@ -0,0 +1,233 @@
+'use strict';
+
+var Installers = module.exports;
+
+var Crypto = require('crypto');
+var Fs = require('node:fs/promises');
+var path = require('node:path');
+
+var reInstallTpl = /\s*#?\s*{{ installer }}/;
+
+function padScript(txt) {
+  return txt.replace(/^/g, '        ');
+}
+
+var BAD_SH_RE = /[<>'"`$\\]/;
+Installers.renderBash = async function (
+  pkgdir,
+  rel,
+  { baseurl, pkg, tag, ver, os = '', arch = '', libc = '', formats },
+) {
+  if (!Array.isArray(formats)) {
+    formats = [];
+  }
+  if (!tag) {
+    tag = '';
+  }
+  let installTxt = await Fs.readFile(path.join(pkgdir, 'install.sh'), 'utf8');
+  installTxt = padScript(installTxt);
+  var vers = rel.version.split('.');
+  var v = {
+    major: vers.shift() || '',
+    minor: vers.shift() || '',
+    patch: vers.join('.').replace(/[+\-].*/, ''),
+    build: vers
+      .join('.')
+      .replace(/[^+\-]*/, '')
+      .replace(/^-/, ''),
+  };
+  var pkgFile = rel.filename || rel.name;
+  let tplTxt = await Fs.readFile(
+    path.join(__dirname, 'package-install.tpl.sh'),
+    'utf8',
+  );
+  // ex: 'node@lts' or 'node'
+  var webiPkg = pkg;
+  if (ver) {
+    webiPkg += `@${ver}`;
+  }
+
+  let releaseParams = new URLSearchParams({
+    os: rel.os,
+    arch: rel.arch,
+    libc: rel.libc,
+    formats: formats.join(','),
+    pretty: true,
+  });
+  let releaseSearch = releaseParams.toString();
+  releaseSearch = releaseSearch.replace(/%2C/g, ',');
+  let releaseUrl = `/api/releases/${pkg}@${tag}.tab?${releaseSearch}`;
+  let releaseCsv = [
+    rel.version,
+    rel.lts,
+    rel.channel,
+    rel.date,
+    rel.os,
+    rel.arch,
+    rel.ext,
+    '-',
+    rel.download,
+    rel.name,
+    rel.comment || '',
+  ]
+    .join(',')
+    .replace(/'/g, '');
+
+  let webiChecksum = await Installers.getWebiShChecksum();
+  let envReplacements = [
+    ['WEBI_CHECKSUM', webiChecksum],
+    ['WEBI_PKG', webiPkg],
+    ['WEBI_HOST', baseurl],
+    ['WEBI_OS', os],
+    ['WEBI_ARCH', arch],
+    ['WEBI_LIBC', libc],
+    ['WEBI_TAG', tag],
+    ['WEBI_RELEASES', `${baseurl}${releaseUrl}`],
+    ['WEBI_CSV', releaseCsv],
+    ['WEBI_VERSION', rel.version],
+    ['WEBI_MAJOR', v.major],
+    ['WEBI_MINOR', v.minor],
+    ['WEBI_PATCH', v.patch],
+    ['WEBI_BUILD', v.build],
+    ['WEBI_GIT_BRANCH', rel.git_branch || rel.git_tag],
+    ['WEBI_GIT_TAG', rel.git_tag], // TODO replace with branch
+    ['WEBI_LTS', rel.lts],
+    ['WEBI_CHANNEL', rel.channel],
+    ['WEBI_EXT', rel.ext],
+    ['WEBI_FORMATS', formats.join(',')],
+    ['WEBI_PKG_URL', rel.download],
+    ['WEBI_PKG_PATHNAME', pkgFile],
+    ['WEBI_PKG_FILE', pkgFile], // TODO replace with pathname
+    ['PKG_NAME', pkg],
+    ['PKG_STABLE', rel.stable],
+    ['PKG_LATEST', rel.latest],
+    ['PKG_OSES', (rel.oses || []).join(' ')],
+    ['PKG_ARCHES', (rel.arches || []).join(' ')],
+    ['PKG_LIBCS', (rel.libcs || []).join(' ')],
+    ['PKG_FORMATS', (rel.formats || []).join(' ')],
+  ];
+
+  for (let env of envReplacements) {
+    let name = env[0];
+    let value = env[1];
+    // Ex:
+    // #export WEBI_FOO=xyz => export WEBI_FOO='123'
+    // export WEBI_FOO=     => export WEBI_FOO='123'
+    let envRe = new RegExp(
+      `^[ \\t]*#?[ \\t]*(export[ \\t])?[ \\t]*(${name})=.*`,
+      'm',
+    );
+    if (BAD_SH_RE.test(value)) {
+      throw new Error(`key '${name}' has invalid value '${value}'`);
+    }
+    tplTxt = tplTxt.replace(envRe, `$1$2='${value}'`);
+  }
+
+  tplTxt = tplTxt
+    .replace(/CHEATSHEET_URL/g, `${baseurl}/${pkg}`)
+    // $', $0, ... $9, $`, $&, and $_ all have special meaning
+    // (see https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/RegExp)
+    // However, it can be escaped with $$ (which must be escaped with $$)
+    .replace(reInstallTpl, '\n' + installTxt.replace(/\$/g, '$$$$'));
+  return tplTxt;
+};
+
+Installers.renderPowerShell = async function (
+  pkgdir,
+  rel,
+  { baseurl, pkg, tag, ver, os, arch, libc = '', formats },
+) {
+  if (!Array.isArray(formats)) {
+    formats = [];
+  }
+  if (!tag) {
+    tag = '';
+  }
+  let installTxt = await Fs.readFile(path.join(pkgdir, 'install.ps1'), 'utf8');
+  installTxt = padScript(installTxt);
+  /*
+    var vers = rel.version.split('.');
+    var v = {
+      major: vers.shift() || '',
+      minor: vers.shift() || '',
+      patch: vers.join('.').replace(/[+\-].*$/, ''),
+      build: vers
+        .join('.')
+        .replace(/[^+\-]*()/, '')
+        .replace(/^-/, '')
+    };
+  */
+  let tplTxt = await Fs.readFile(
+    path.join(__dirname, 'package-install.tpl.ps1'),
+    'utf8',
+  );
+  var pkgver = pkg + '@' + ver;
+  return (
+    tplTxt
+      .replace(
+        /^(#)?\$Env:WEBI_LIBC\s*=.*/im,
+        "$Env:WEBI_LIBC = '" + libc + "'",
+      )
+      .replace(
+        /^(#)?\$Env:WEBI_HOST\s*=.*/im,
+        "$Env:WEBI_HOST = '" + baseurl + "'",
+      )
+      .replace(
+        /^(#)?\$Env:WEBI_PKG\s*=.*/im,
+        "$Env:WEBI_PKG = '" + pkgver + "'",
+      )
+      .replace(/^(#)?\$Env:PKG_NAME\s*=.*/im, "$Env:PKG_NAME = '" + pkg + "'")
+      .replace(
+        /^(#)?\$Env:WEBI_VERSION\s*=.*/im,
+        "$Env:WEBI_VERSION = '" + rel.version + "'",
+      )
+      .replace(
+        /^(#)?\$Env:WEBI_GIT_TAG\s*=.*/im,
+        "$Env:WEBI_GIT_TAG = '" + rel.git_tag + "'",
+      )
+      .replace(
+        /^(#)?\$Env:WEBI_PKG_URL\s*=.*/im,
+        "$Env:WEBI_PKG_URL = '" + rel.download + "'",
+      )
+      // TODO replace WEBI_PKG_FILE (which is sometimes a dir)
+      .replace(
+        /^(#)?\$Env:WEBI_PKG_PATHNAME\s*=.*/im,
+        "$Env:WEBI_PKG_PATHNAME = '" + rel.name + "'",
+      )
+      // TODO deprecate
+      .replace(
+        /^(#)?\$Env:WEBI_PKG_FILE\s*=.*/im,
+        "$Env:WEBI_PKG_FILE = '" + rel.name + "'",
+      )
+      .replace(reInstallTpl, '\n' + installTxt)
+  );
+};
+
+var _webiShMeta = {
+  stale: 10 * 1000,
+  updated_at: 0,
+  checksum: '',
+  mtime: 0,
+};
+Installers.getWebiShChecksum = async function () {
+  let now = Date.now();
+  let ago = now - _webiShMeta.updated_at;
+  if (ago <= _webiShMeta.stale) {
+    return _webiShMeta.checksum;
+  }
+
+  let webiPath = path.join(__dirname, '../webi/webi.sh');
+  let stat = await Fs.stat(webiPath);
+  if (stat.mtimeMs === _webiShMeta.mtime) {
+    return _webiShMeta.checksum;
+  }
+
+  let webiBuf = await Fs.readFile(webiPath, null);
+  let webiHash = Crypto.createHash('sha1').update(webiBuf).digest('hex');
+  let webiChecksum = webiHash.slice(0, 8);
+
+  _webiShMeta.mtime = stat.mtimeMs;
+  _webiShMeta.updated_at = now;
+  _webiShMeta.checksum = webiChecksum;
+  return _webiShMeta.checksum;
+};

_webi/lint-builds.js

@@ -0,0 +1,338 @@
+#!/usr/bin/env node
+'use strict';
+
+let Fs = require('node:fs/promises');
+let Path = require('node:path');
+
+let BuildsCacher = require('./builds-cacher.js');
+let HostTargets = require('./build-classifier/host-targets.js');
+let Parallel = require('./parallel.js');
+
+var INSTALLERS_DIR = Path.join(__dirname, '..');
+var CACHE_DIR = Path.join(__dirname, '../_cache');
+
+let UserAgentsMap = require('./build-classifier/uas.json');
+let uas = Object.keys(UserAgentsMap);
+let uaTargetsMap = {};
+for (let ua of uas) {
+  let terms = ua.split(/[\s\/]+/g);
+  let target = {};
+  void HostTargets.termsToTarget(target, terms);
+  if (!target) {
+    continue;
+  }
+  if (target.errors.length) {
+    throw target.errors[0];
+  }
+  if (!target.os) {
+    // TODO make target null, or create error for this
+    console.warn(`no os for terms: ${terms}`);
+    //throw new Error(`terms: ${terms}`);
+    continue;
+  }
+  if (!target.arch) {
+    // TODO make target null, or create error for this
+    console.warn(`no arch for terms: ${terms}`);
+    //throw new Error(`terms: ${terms}`);
+    continue;
+  }
+  if (!target.libc) {
+    // TODO make target null, or create error for this
+    console.warn(`no libc for terms: ${terms}`);
+    //throw new Error(`terms: ${terms}`);
+    continue;
+  }
+  let triplet = `${target.os}-${target.arch}-${target.libc}`;
+  uaTargetsMap[triplet] = target;
+}
+let uaTargets = [];
+let triplets = Object.keys(uaTargetsMap);
+for (let triplet of triplets) {
+  let target = uaTargetsMap[triplet];
+  uaTargets.push(target);
+}
+
+function showDirs(dirs) {
+  {
+    let errors = Object.keys(dirs.errors);
+    console.error('');
+    console.error(`Errors: ${errors.length}`);
+    for (let name of errors) {
+      let err = dirs.errors[name];
+      console.error(`${name}/: ${err.message}`);
+    }
+  }
+
+  {
+    let hidden = Object.keys(dirs.hidden);
+    console.debug('');
+    console.debug(`Hidden: ${hidden.length}`);
+    for (let name of hidden) {
+      let kind = dirs.hidden[name];
+      if (kind === '!directory') {
+        console.debug(`    ${name}`);
+      } else {
+        console.debug(`    ${name}/`);
+      }
+    }
+  }
+
+  {
+    let alias = Object.keys(dirs.alias);
+    console.debug('');
+    console.debug(`Alias: ${alias.length}`);
+    for (let name of alias) {
+      let kind = dirs.alias[name];
+      if (kind === 'symlink') {
+        console.debug(`    ${name} => ...`);
+      } else {
+        console.debug(`    ${name}/`);
+      }
+    }
+  }
+
+  {
+    let invalids = Object.keys(dirs.invalid);
+    console.warn('');
+    console.warn(`Invalid: ${invalids.length}`);
+    for (let name of invalids) {
+      console.warn(`    ${name}/`);
+    }
+  }
+
+  {
+    let selfhosted = Object.keys(dirs.selfhosted);
+    console.info('');
+    console.info(`Self-Hosted: ${selfhosted.length}`);
+    for (let name of selfhosted) {
+      console.info(`    ${name}/`);
+    }
+  }
+
+  {
+    let valids = Object.keys(dirs.valid);
+    console.info('');
+    console.info(`Found: ${valids.length}`);
+    for (let name of valids) {
+      console.info(`    ${name}/`);
+    }
+  }
+}
+
+let bc = BuildsCacher.create({
+  caches: CACHE_DIR,
+  installers: INSTALLERS_DIR,
+});
+
+async function main() {
+  /* jshint maxcomplexity: 25 */
+  // TODO
+  //     node ./_webi/lint-builds.js caddy@beta 'x86_64/unknown Darwin libc'
+  //
+  //let [projName, userAgent] = process.argv.slice(2);
+  let projName = process.argv[2];
+  // create test case for zoxide, goreleaser, go, yq, caddy, rg
+
+  let dirs = await bc.getProjectsByType();
+  if (!projName) {
+    showDirs(dirs);
+    console.info('');
+  }
+
+  bc.freshenRandomPackage(600 * 1000);
+
+  let rows = [];
+  let triples = [];
+  let valids = Object.keys(dirs.valid);
+
+  if (projName) {
+    if (!valids.includes(projName)) {
+      throw new Error(`'${projName}' is not a valid installable project`);
+    }
+    valids = [projName];
+  }
+  //valids = ['atomicparsley', 'caddy', 'macos'];
+  //valids = ['atomicparsley'];
+
+  console.info('');
+  console.info(`Fetching project release assets`);
+  let parallel = 25;
+  let projects = [];
+  await Parallel.run(parallel, valids, getAll);
+  async function getAll(name, i) {
+    console.info(`    ${name}`);
+    let projInfo = await bc.getPackages({
+      //Releases: Releases,
+      name: name,
+      date: new Date(),
+    });
+    projects[i] = projInfo;
+  }
+
+  console.info(`Classifying build assets for...`);
+  for (let projInfo of projects) {
+    console.info(`    ${projInfo.name}`);
+
+    let nStr = projInfo.releases.length.toString();
+    let n = nStr.padStart(5, ' ');
+    let row = `##### ${n}\t${projInfo.name}\tv`;
+    rows.push(row);
+
+    // ignore known, non-package extensions
+    for (let build of projInfo.releases) {
+      let target = bc.classify(projInfo, build);
+      if (!target) {
+        // non-build file
+        continue;
+      }
+
+      if (target.error) {
+        let e = target.error;
+        if (e.code === 'E_BUILD_NO_PATTERN') {
+          console.warn(`>>> ${e.message} <<<`);
+          console.warn(projInfo);
+          console.warn(build);
+          console.warn(`^^^ ${e.message} ^^^`);
+        }
+        throw e;
+      }
+
+      if (target.unknownTerms?.length) {
+        let msg = `${projInfo.name}: unrecognized term(s) '${target.unknownTerms}' in '${build.download}'`;
+        let err = new Error(msg);
+        throw err;
+      }
+
+      triples.push(target.triplet);
+      // if (!build.version) {
+      //   throw new Error(`no version for ${pkg.name} ${build.name}`);
+      // }
+      // // For debug printing versions
+      // console.error(build.version);
+      rows.push(`${target.triplet}\t${projInfo.name}\t${build.version}`);
+    }
+  }
+
+  console.info(`Fetching builds for`);
+  for (let projInfo of projects) {
+    console.info('');
+    console.info('');
+    console.info(`    ${projInfo.name}`);
+
+    for (let target of uaTargets) {
+      let libc = target.libc || 'libc';
+      let hostTriplet = `${target.os}-${target.arch}-${libc}`;
+      console.info('');
+      console.info(`    target: ${hostTriplet}`);
+      let match = bc.findMatchingPackages(projInfo, target, {
+        ver: '',
+      });
+      if (!match) {
+        console.info(
+          `    project: ${projInfo.name}: missing build for os '${target.os}'`,
+        );
+        continue;
+      }
+
+      if (!match.releases) {
+        console.info(
+          `    project: ${projInfo.name}: missing build for os '${target.os}-${target.arch}-${libc}'`,
+        );
+      } else if (match.triplet === hostTriplet) {
+        let releaseNames = Object.keys(match.releases);
+        console.info(`    selected ${releaseNames.length}`);
+      } else {
+        let releaseNames = Object.keys(match.releases);
+        console.info(
+          `    selected ${releaseNames.length} (${match.triplet} fallback)`,
+        );
+      }
+    }
+  }
+
+  let tsv = rows.join('\n');
+  console.info('');
+  console.info('#rows', rows.length);
+  await Fs.writeFile('builds.tsv', tsv, 'utf8');
+
+  console.info('');
+  console.info('Triplets Detected:');
+  let triplets = Object.keys(bc._triplets);
+  if (triplets.length) {
+    triplets.sort();
+    console.info('   ', triplets.join('\n    '));
+  } else {
+    console.info('    (none)');
+  }
+
+  console.info('');
+  console.info('New / Unknown Terms:');
+  let unknowns = Object.keys(bc.unknownTerms);
+  if (unknowns.length) {
+    unknowns.sort();
+    console.warn('   ', unknowns.join('\n    '));
+  } else {
+    console.info('    (none)');
+  }
+
+  console.info('');
+  console.info('Unused Terms:');
+  let unuseds = Object.keys(bc.orphanTerms);
+  if (unuseds.length) {
+    unuseds.sort();
+    console.warn('   ', unuseds.join('\n    '));
+  } else {
+    console.info('    (none)');
+  }
+
+  console.info('');
+  console.info('Formats:');
+  if (bc.formats.length) {
+    let formats = bc.formats.slice();
+    formats.sort();
+    if (!formats[0]) {
+      formats[0] = '(bin)';
+    }
+    console.warn('   ', formats.join('\n    '));
+  } else {
+    console.info('    (none)');
+  }
+
+  // sort -u -k1 builds.tsv | rg -v '^#|^https?:' | rg -i arm
+  // cut -f1 builds.tsv | sort -u -k1 | rg -v '^#|^https?:' | rg -i arm
+}
+
+if (module === require.main) {
+  let times = [];
+  let now = Date.now();
+  main()
+    .then(async function () {
+      let then = Date.now();
+      let delta = then - now;
+      times.push(delta);
+      now = then;
+      await main();
+      then = Date.now();
+      delta = then - now;
+      times.push(delta);
+    })
+    .then(function () {
+      console.info('');
+      console.info('Run times');
+      for (let delta of times) {
+        let s = delta / 1000;
+        console.info(`    ${s}`);
+      }
+
+      function forceExit() {
+        console.warn(`warn: dangling event loop reference`);
+        process.exit(0);
+      }
+      let exitTimeout = setTimeout(forceExit, 250);
+      exitTimeout.unref();
+    })
+    .catch(function (err) {
+      console.error(err.stack || err);
+      process.exit(1);
+    });
+}

_webi/normalize.js

@@ -7,20 +7,21 @@ var osMap = {
   freebsd: /(\b|_)(freebsd)/i,
   windows: /(\b|_)(win|microsoft|msft)/i,
   sunos: /(\b|_)(sun)/i,
-  aix: /(\b|_)(aix)/i
+  aix: /(\b|_)(aix)/i,
 };
 
 var maps = {
   oses: {},
   arches: {},
-  formats: {}
+  libcs: {},
+  formats: {},
 };
 
 Object.keys(osMap).forEach(function (name) {
   maps.oses[name] = true;
 });
 
-var formats = ['zip', 'xz', 'tar', 'pkg', 'msi', 'git', 'exe', 'dmg'];
+var formats = ['zip', 'xz', 'tar', 'pkg', 'msi', 'git', 'exe', 'dmg', 'git'];
 formats.forEach(function (name) {
   maps.formats[name] = true;
 });
@@ -28,59 +29,90 @@ formats.forEach(function (name) {
 // evaluation order matters
 // (i.e. otherwise x86 and x64 can cross match)
 var arches = [
-  'amd64', // first and most likely match
+  // arm64/aarch64 has very high specificity, so it comes first
   'arm64',
+  // arm 7 is also generic aarch/arm/arm32
+  'armv7l',
+  // arm6 can run on armv7
+  'armv6l',
+  // amd64 is more likely and less often specified than arm64
+  'amd64',
   'x86',
   'ppc64le',
   'ppc64',
-  'armv7l',
-  'armv6l',
-  's390x'
+  's390x',
 ];
+// Used for detecting system arch from package download url, for example:
+//
+// https://git.com/org/foo/releases/v0.7.9/foo-aarch64-linux-musl.tar.gz
+// https://git.com/org/foo/releases/v0.7.9/foo-arm-linux-musleabihf.tar.gz
+// https://git.com/org/foo/releases/v0.7.9/foo-armv7-linux-musleabihf.tar.gz
+// https://git.com/org/foo/releases/v0.7.9/foo-x86_64-linux-musl.tar.gz
+//
 var archMap = {
+  arm64: /(\b|_)(aarch64|arm64)/i,
+  armv7l: /(\b|_)(arm32|arm[_\-]?v?7l?)/i,
+  armv6l: /(\b|_)(arm|aarch32|arm[_\-]?v?6l?)(\b|_)/i,
   //amd64: /(amd.?64|x64|[_\-]64)/i,
-  amd64: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)64([_\-]?bit)?(\b|_)/i,
+  amd64:
+    /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)64([_\-]?bit)?(\b|_)/i,
   //x86: /(86)(\b|_)/i,
-  x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32)([_\-]?bit)(\b|_)/i,
+  x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32|i?386)([_\-]?bit)?(\b|_)/i,
   ppc64le: /(\b|_)(ppc64le)/i,
   ppc64: /(\b|_)(ppc64)(\b|_)/i,
-  arm64: /(\b|_)((aarch|arm)64|arm)/i,
-  armv7l: /(\b|_)(armv?7l)/i,
-  armv6l: /(\b|_)(aarch32|armv?6l)/i,
-  s390x: /(\b|_)(s390x)/i
+  s390x: /(\b|_)(s390x)/i,
 };
 arches.forEach(function (name) {
   maps.arches[name] = true;
 });
 
+var libcs = ['none', 'musl', 'gnu', 'msvc', 'libc'];
+libcs.forEach(function (name) {
+  maps.libcs[name] = true;
+});
+
 function normalize(all) {
+  /* jshint maxcomplexity:50 */
+  /* jshint maxdepth:10 */
   var supported = {
     oses: {},
     arches: {},
-    formats: {}
+    libcs: {},
+    formats: {},
   };
 
-  all.releases.forEach(function (rel) {
+  for (let rel of all.releases) {
     rel.version = rel.version.replace(/^v/i, '');
+
     if (!rel.name) {
       rel.name = rel.download.replace(/.*\//, '');
     }
+
     if (!rel.os) {
-      rel.os =
-        Object.keys(osMap).find(function (regKey) {
-          return osMap[regKey].test(rel.name || rel.download);
-        }) || 'unknown';
+      rel.os = 'unknown';
+
+      let osNames = Object.keys(osMap);
+      for (let osName of osNames) {
+        let relName = rel.name || rel.download;
+        let osRegExp = osMap[osName];
+        let matches = osRegExp.test(relName);
+        if (matches) {
+          rel.os = osName;
+          break;
+        }
+      }
     }
     supported.oses[rel.os] = true;
 
     if (!rel.arch) {
-      arches.some(function (regKey) {
-        var arch = (rel.name || rel.download).match(archMap[regKey]) && regKey;
-        if (arch) {
+      for (let arch of arches) {
+        let name = rel.name || rel.download;
+        let isArch = name.match(archMap[arch]);
+        if (isArch) {
           rel.arch = arch;
-          return true;
+          break;
         }
-      });
+      }
     }
     if (!rel.arch) {
       if ('macos' === rel.os) {
@@ -89,6 +121,60 @@ function normalize(all) {
     }
     supported.arches[rel.arch] = true;
 
+    // note: depends on rel.os
+    if (!rel.libc) {
+      let isMusl;
+      let isMsvc;
+      let isStatic;
+      let isGnu;
+
+      // extra blocks to prevent copy pasta errors
+
+      {
+        let muslRe = /(\b|_)(musl)(\b|_)/i;
+        isMusl = muslRe.test(rel.download) || muslRe.test(rel.name);
+      }
+
+      {
+        let msvcRe = /(\b|_)(msvc)(\b|_)/i;
+        isMsvc = msvcRe.test(rel.download) || msvcRe.test(rel.name);
+      }
+
+      {
+        let staticRe = /(\b|_)(static)(\b|_)/i;
+        isStatic = staticRe.test(rel.download) || staticRe.test(rel.name);
+      }
+
+      {
+        let gnuRe = /(\b|_)(gnu|glibc|libc)(\b|_)/i;
+        isGnu = gnuRe.test(rel.download) || gnuRe.test(rel.name);
+      }
+
+      if (isMusl) {
+        // we specifically tag things that need musl++ in their own releases
+        rel.libc = 'none';
+      } else if (isStatic) {
+        rel.libc = 'none';
+      } else if (isGnu) {
+        rel.libc = 'gnu';
+        if (rel.os === 'windows') {
+          // windows gnu is static
+          rel.libc = 'none';
+        } else if (rel.os === 'darwin') {
+          // if glibc is required on macos, it'll be static
+          rel.libc = 'none';
+        }
+      } else if (isMsvc) {
+        rel.libc = 'msvc';
+      } else {
+        // The default is no requirement for any particular libc
+        // (Go, Zig, POSIX Shell, JS, etc)
+        // and hopefully we never have to worry about mingw and friends
+        rel.libc = 'none';
+      }
+    }
+    supported.libcs[rel.libc] = true;
+
     var tarExt;
     if (!rel.ext) {
       // pkg-v1.0.tar.gz => ['gz', 'tar', '0', 'pkg-v1']
@@ -103,7 +189,7 @@ function normalize(all) {
       if ('tar' === exts[1]) {
         rel.ext = exts.reverse().join('.');
         tarExt = 'tar';
-      } else if ('tgz' == exts[0]) {
+      } else if ('tgz' === exts[0]) {
         rel.ext = 'tar.gz';
         tarExt = 'tar';
       } else {
@@ -116,10 +202,29 @@ function normalize(all) {
     }
     supported.formats[tarExt || rel.ext] = true;
 
+    if (!rel.channel) {
+      // basically like this: (+.-_)(beta|rc)(0-9)(+.-_)
+      // matches:
+      //   - v1.0-beta
+      //   - v1.0-beta1.1
+      //   - v1.0-beta-11
+      // won't match:
+      //   - v1.0beta
+      //   - v1.0-beta1b
+      let isBetaRe =
+        /(\b|_)(alpha|beta|dev|developer|prev|preview|rc)(\d+)(\b|_)/;
+      let isBeta = isBetaRe.test(rel.name);
+      if (isBeta) {
+        rel.channel = 'beta';
+      } else {
+        rel.channel = 'stable';
+      }
+    }
+
     if (all.download) {
       rel.download = all.download.replace(/{{ download }}/, rel.download);
     }
-  });
+  }
 
   all.oses = Object.keys(supported.oses).filter(function (name) {
     return maps.oses[name];
@@ -127,6 +232,9 @@ function normalize(all) {
   all.arches = Object.keys(supported.arches).filter(function (name) {
     return maps.arches[name];
   });
+  all.libcs = Object.keys(supported.libcs).filter(function (name) {
+    return maps.libcs[name];
+  });
   all.formats = Object.keys(supported.formats).filter(function (name) {
     return maps.formats[name];
   });
@@ -147,4 +255,5 @@ module.exports._debug = function (all) {
 // NOT in order of priority (which would be tar, xz, zip, ...)
 module.exports.formats = formats;
 module.exports.arches = arches;
+module.exports.libcs = libcs;
 module.exports.formatsMap = maps.formats;

_webi/package-install.tpl.ps1

@@ -0,0 +1,155 @@
+#!/usr/bin/env pwsh
+#350 check if windows user run as admin
+
+# this allows us to call ps1 files, which allows us to have spaces in filenames
+# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
+# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
+Set-ExecutionPolicy -Scope Process Bypass
+
+# If a command returns an error, halt the script.
+$ErrorActionPreference = 'Stop'
+
+# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
+$ProgressPreference = 'SilentlyContinue'
+
+$Env:WEBI_HOST = 'https://webinstall.dev'
+#$Env:WEBI_PKG = 'node@lts'
+#$Env:PKG_NAME = node
+#$Env:WEBI_VERSION = v12.16.2
+#$Env:WEBI_GIT_TAG = 12.16.2
+#$Env:WEBI_PKG_URL = "https://.../node-....zip"
+#$Env:WEBI_PKG_FILE = "node-v12.16.2-win-x64.zip"
+#$Env:WEBI_PKG_PATHNAME = "node-v12.16.2-win-x64.zip"
+
+# Switch to userprofile
+Push-Location $Env:USERPROFILE
+
+# Make paths
+New-Item -Path "$Env:USERPROFILE\Downloads" -ItemType Directory -Force | Out-Null
+New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
+New-Item -Path "$Env:USERPROFILE\.local\opt" -ItemType Directory -Force | Out-Null
+
+# {{ baseurl }}
+# {{ version }}
+
+# See
+#   - <https://superuser.com/q/1264444>
+#   - <https://stackoverflow.com/a/60572643/151312>
+$Esc = [char]27
+$TTask = "${Esc}[36m"
+$TName = "${Esc}[1m${Esc}[32m"
+$TUrl = "${Esc}[2m"
+$TPath = "${Esc}[2m${Esc}[32m"
+$TCmd = "${Esc}[2m${Esc}[35m"
+$TDim = "${Esc}[2m"
+$TReset = "${Esc}[0m"
+
+function Invoke-DownloadUrl {
+    Param (
+        [string]$URL,
+        [string]$Params,
+        [string]$Path,
+        [switch]$Force
+    )
+
+    IF (Test-Path -Path "$Path") {
+        IF (-Not $Force.IsPresent) {
+            Write-Host "    ${TDim}Found${TReset} $Path"
+            return
+        }
+        Write-Host "    Updating ${TDim}${Path}${TDim}"
+    }
+
+    $TmpPath = "${Path}.part"
+    Remove-Item -Path $TmpPath -Force -ErrorAction Ignore
+
+    Write-Host "    Downloading ${TDim}from${TReset}"
+    Write-Host "      ${TDim}${URL}${TReset}"
+    IF ($Params.Length -ne 0) {
+        Write-Host "        ?$Params"
+        $URL = "${URL}?${Params}"
+    }
+    curl.exe '-#' -f -sS -A $Env:WEBI_UA -L $URL | Out-File $TmpPath
+
+    Remove-Item -Path $Path -Force -ErrorAction Ignore
+    Move-Item $TmpPath $Path
+    Write-Host "      Saved ${TPath}${Path}${TReset}"
+}
+
+function Get-UserAgent {
+    # This is the canonical CPU arch when the process is emulated
+    $my_arch = "$Env:PROCESSOR_ARCHITEW6432"
+
+    IF ($my_arch -eq $null -or $my_arch -eq "") {
+        # This is the canonical CPU arch when the process is native
+        $my_arch = "$Env:PROCESSOR_ARCHITECTURE"
+    }
+
+    IF ($my_arch -eq "AMD64") {
+        # Because PowerShell is sometimes AMD64 on Windows 10 ARM
+        # See https://oofhours.com/2020/02/04/powershell-on-windows-10-arm64/
+        $my_os_arch = (Get-CimInstance -ClassName Win32_OperatingSystem).OSArchitecture
+
+        # Using -clike because of the trailing newline
+        IF ($my_os_arch -clike "ARM 64*") {
+            $my_arch = "ARM64"
+        }
+    }
+
+    "PowerShell+curl Windows/10+ $my_arch msvc"
+}
+
+function webi_path_add($pathname) {
+    # C:\Users\me => C:/Users/me
+    $my_home = $Env:UserProfile
+    $my_home = $my_home.replace('\\', '/')
+    $my_home_re = [regex]::escape($my_home)
+
+    # ~/bin => %USERPROFILE%/bin
+    $pathname = $pathname.replace('~/', "$Env:UserProfile/")
+
+    # C:\Users\me\bin => %USERPROFILE%/bin
+    $my_pathname = $pathname.replace('\\', '/')
+    $my_pathname = $my_pathname -ireplace $my_home_re, "%USERPROFILE%"
+
+    $all_user_paths = [Environment]::GetEnvironmentVariable("Path", "User")
+    $user_paths = "${all_user_paths}".Trim(';').Split(';')
+    $exists_in_path = $false
+    foreach ($user_path in $user_paths) {
+        # C:\Users\me\bin => %USERPROFILE%/bin
+        $my_user_path = $user_path.replace('\\', '/')
+        $my_user_path = $my_user_path -ireplace $my_home_re, "%USERPROFILE%"
+
+        if ($my_user_path -ieq $my_pathname) {
+            $exists_in_path = $true
+        }
+    }
+    if (-Not $exists_in_path) {
+        $all_user_paths = "${pathname};${all_user_paths}".Trim(';')
+        [Environment]::SetEnvironmentVariable("Path", $all_user_paths, "User")
+        $null = Sync-EnvPath
+    }
+}
+
+function Sync-EnvPath {
+    $UserPath = [Environment]::GetEnvironmentVariable("Path", "User").Trim(';')
+    $MachinePath = [Environment]::GetEnvironmentVariable("Path", "Machine").Trim(';')
+    $Env:Path = "${UserPath};${MachinePath}"
+    [Environment]::SetEnvironmentVariable("Path", $Env:Path)
+    $Env:Path
+}
+
+$Env:WEBI_UA = Get-UserAgent
+
+#$has_local_bin = echo "$Env:PATH" | Select-String -Pattern '\.local.bin'
+#if (!$has_local_bin)
+#{
+webi_path_add ~/.local/bin
+#}
+
+# {{ installer }}
+
+webi_path_add ~/.local/bin
+
+# Done
+Pop-Location

_webi/packages.js

@@ -1,114 +0,0 @@
-'use strict';
-
-var frontmarker = require('./frontmarker.js');
-var shmatter = require('shmatter');
-var fs = require('fs');
-var path = require('path');
-
-var pkgs = module.exports;
-pkgs.create = function (Pkgs, basepath) {
-  if (!Pkgs) {
-    Pkgs = {};
-  }
-  if (!basepath) {
-    basepath = path.join(__dirname, '../');
-  }
-
-  Pkgs.all = function () {
-    return fs.promises.readdir(basepath).then(function (nodes) {
-      var items = [];
-      return nodes
-        .reduce(function (p, node) {
-          return p.then(function () {
-            return pkgs.get(node).then(function (meta) {
-              if (meta && '_' !== node[0]) {
-                meta.name = node;
-                items.push(meta);
-              }
-            });
-          });
-        }, Promise.resolve())
-        .then(function () {
-          return items;
-        });
-    });
-  };
-
-  Pkgs.get = function (node) {
-    return fs.promises.access(path.join(basepath, node)).then(function () {
-      return Pkgs._get(node);
-    });
-  };
-  Pkgs._get = function (node) {
-    var yash = path.join(basepath, node, 'package.yash');
-    var curlbash = path.join(basepath, node, 'install.sh');
-    var readme = path.join(basepath, node, 'README.md');
-    var winstall = path.join(basepath, node, 'install.ps1');
-    return Promise.all([
-      fs.promises
-        .readFile(readme, 'utf-8')
-        .then(function (txt) {
-          // TODO
-          return frontmarker.parse(txt);
-        })
-        .catch(function (e) {
-          if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
-            console.error("failed to read '" + node + "/README.md'");
-            console.error(e);
-          }
-        }),
-      fs.promises
-        .readFile(yash, 'utf-8')
-        .then(function (txt) {
-          return shmatter.parse(txt);
-        })
-        .catch(function (e) {
-          // no yash package description
-          yash = '';
-          if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
-            console.error("failed to parse '" + node + "/package.yash'");
-            console.error(e);
-          }
-          return fs.promises.readFile(curlbash, 'utf-8').then(function (txt) {
-            return shmatter.parse(txt);
-          });
-        })
-        .catch(function (e) {
-          // no *nix installer
-          curlbash = '';
-          if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
-            console.error("failed to parse '" + node + "/install.sh'");
-            console.error(e);
-          }
-        }),
-      fs.promises.access(winstall).catch(function (e) {
-        // no winstaller
-        winstall = '';
-        if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
-          console.error("failed to read '" + node + "/install.ps1'");
-          console.error(e);
-        }
-      })
-    ]).then(function (items) {
-      var meta = items[0] || items[1];
-      if (!meta) {
-        // doesn't exist
-        return;
-      }
-      meta.windows = !!winstall;
-      meta.bash = !!curlbash;
-
-      return meta;
-    });
-  };
-
-  return Pkgs;
-};
-pkgs.create(pkgs);
-
-if (module === require.main) {
-  pkgs.all().then(function (data) {
-    console.info('package info:');
-    console.info(data);
-  });
-}

_webi/packages.js

@@ -0,0 +1 @@
+projects.js

_webi/parallel.js

@@ -0,0 +1,44 @@
+'use strict';
+
+var Parallel = module.exports;
+Parallel.run = async function (limit, arr, fn) {
+  let index = 0;
+  let actives = [];
+  let results = [];
+  limit = Math.min(limit, arr.length);
+
+  function launch() {
+    let _index = index;
+    let p = fn(arr[_index], _index, arr);
+
+    // some tasks may be synchronous
+    // so we must push before removing
+    actives.push(p);
+
+    p.then(function _resolve(result) {
+      let i = actives.indexOf(p);
+      actives.splice(i, 1);
+      results[_index] = result;
+    });
+
+    index += 1;
+  }
+
+  // start tasks in parallel, up to limit
+  for (; actives.length < limit; ) {
+    launch();
+  }
+
+  // keep the task queue full
+  for (; index < arr.length; ) {
+    // wait for one task to complete
+    await Promise.race(actives);
+    // add one task again
+    launch();
+  }
+
+  // wait for all remaining tasks
+  await Promise.all(actives);
+
+  return results;
+};

_webi/projects.js

@@ -0,0 +1,96 @@
+'use strict';
+
+var frontmarker = require('./frontmarker.js');
+var fs = require('fs');
+var path = require('path');
+
+var pkgs = module.exports;
+pkgs.create = function (Pkgs, basepath) {
+  if (!Pkgs) {
+    Pkgs = {};
+  }
+  if (!basepath) {
+    basepath = path.join(__dirname, '../');
+  }
+
+  Pkgs.all = function () {
+    return fs.promises.readdir(basepath).then(function (nodes) {
+      var items = [];
+      return nodes
+        .reduce(function (p, node) {
+          return p.then(function () {
+            return pkgs.get(node).then(function (meta) {
+              if (meta && '_' !== node[0]) {
+                meta.name = node;
+                items.push(meta);
+              }
+            });
+          });
+        }, Promise.resolve())
+        .then(function () {
+          return items;
+        });
+    });
+  };
+
+  Pkgs.get = function (node) {
+    return fs.promises.access(path.join(basepath, node)).then(function () {
+      return Pkgs._get(node);
+    });
+  };
+  Pkgs._get = function (node) {
+    var curlbash = path.join(basepath, node, 'install.sh');
+    var readme = path.join(basepath, node, 'README.md');
+    var winstall = path.join(basepath, node, 'install.ps1');
+    return Promise.all([
+      fs.promises
+        .readFile(readme, 'utf-8')
+        .then(function (txt) {
+          // TODO
+          return frontmarker.parse(txt);
+        })
+        .catch(function (e) {
+          if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
+            console.error("failed to read '" + node + "/README.md'");
+            console.error(e);
+          }
+        }),
+      fs.promises.access(curlbash).catch(function (e) {
+        // no *nix installer
+        curlbash = '';
+        if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
+          console.error("failed to parse '" + node + "/install.sh'");
+          console.error(e);
+        }
+      }),
+      fs.promises.access(winstall).catch(function (e) {
+        // no winstaller
+        winstall = '';
+        if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
+          console.error("failed to read '" + node + "/install.ps1'");
+          console.error(e);
+        }
+      }),
+    ]).then(function (items) {
+      var meta = items[0] || items[1];
+      if (!meta) {
+        // doesn't exist
+        return;
+      }
+      meta.windows = !!winstall;
+      meta.bash = !!curlbash;
+
+      return meta;
+    });
+  };
+
+  return Pkgs;
+};
+pkgs.create(pkgs);
+
+if (module === require.main) {
+  pkgs.all().then(function (data) {
+    console.info('package info:');
+    console.info(data);
+  });
+}

_webi/releases.js

@@ -1,251 +0,0 @@
-'use strict';
-
-var fs = require('fs');
-var path = require('path');
-var request = require('@root/request');
-var _normalize = require('../_webi/normalize.js');
-
-var reInstallTpl = /\s*#?\s*{{ installer }}/;
-
-var Releases = module.exports;
-Releases.get = async function (pkgdir) {
-  var get;
-  try {
-    get = require(path.join(pkgdir, 'releases.js'));
-  } catch (e) {
-    throw new Error('no releases.js for', pkgdir.split(/[\/\\]+/).pop());
-  }
-  return get(request).then(function (all) {
-    return _normalize(all);
-  });
-};
-
-function padScript(txt) {
-  return txt.replace(/^/g, '        ');
-}
-
-Releases.renderBash = function (
-  pkgdir,
-  rel,
-  { baseurl, pkg, tag, ver, os, arch, formats }
-) {
-  if (!Array.isArray(formats)) {
-    formats = [];
-  }
-  if (!tag) {
-    tag = '';
-  }
-  return fs.promises
-    .readFile(path.join(pkgdir, 'install.sh'), 'utf8')
-    .then(function (installTxt) {
-      installTxt = padScript(installTxt);
-      var vers = rel.version.split('.');
-      var v = {
-        major: vers.shift() || '',
-        minor: vers.shift() || '',
-        patch: vers.join('.').replace(/[+\-].*/, ''),
-        build: vers
-          .join('.')
-          .replace(/[^+\-]*/, '')
-          .replace(/^-/, '')
-      };
-      var pkgFile = rel.filename || rel.name;
-      return fs.promises
-        .readFile(path.join(__dirname, 'template.sh'), 'utf8')
-        .then(function (tplTxt) {
-          return (
-            tplTxt
-              .replace(
-                /^\s*#?WEBI_PKG=.*/m,
-                "WEBI_PKG='" + pkg + '@' + ver + "'"
-              )
-              .replace(/^\s*#?WEBI_HOST=.*/m, "WEBI_HOST='" + baseurl + "'")
-              .replace(/^\s*#?WEBI_OS=.*/m, "WEBI_OS='" + (os || '') + "'")
-              .replace(
-                /^\s*#?WEBI_ARCH=.*/m,
-                "WEBI_ARCH='" + (arch || '') + "'"
-              )
-              .replace(/^\s*#?WEBI_TAG=.*/m, "WEBI_TAG='" + tag + "'")
-              .replace(
-                /^\s*#?WEBI_RELEASES=.*/m,
-                "WEBI_RELEASES='" +
-                  baseurl +
-                  '/api/releases/' +
-                  pkg +
-                  '@' +
-                  tag +
-                  '.tab?os=' +
-                  rel.os +
-                  '&arch=' +
-                  rel.arch +
-                  '&formats=' +
-                  formats.join(',') +
-                  '&pretty=true' +
-                  "'"
-              )
-              .replace(
-                /^\s*#?WEBI_CSV=.*/m,
-                "WEBI_CSV='" +
-                  [
-                    rel.version,
-                    rel.lts,
-                    rel.channel,
-                    rel.date,
-                    rel.os,
-                    rel.arch,
-                    rel.ext,
-                    '-',
-                    rel.download,
-                    rel.name,
-                    rel.comment || ''
-                  ]
-                    .join(',')
-                    .replace(/'/g, '') +
-                  "'"
-              )
-              .replace(
-                /^\s*#?WEBI_VERSION=.*/m,
-                'WEBI_VERSION=' + JSON.stringify(rel.version)
-              )
-              .replace(/^\s*#?WEBI_MAJOR=.*/m, 'WEBI_MAJOR=' + v.major)
-              .replace(/^\s*#?WEBI_MINOR=.*/m, 'WEBI_MINOR=' + v.minor)
-              .replace(/^\s*#?WEBI_PATCH=.*/m, 'WEBI_PATCH=' + v.patch)
-              .replace(/^\s*#?WEBI_BUILD=.*/m, 'WEBI_BUILD=' + v.build)
-              .replace(/^\s*#?WEBI_LTS=.*/m, 'WEBI_LTS=' + rel.lts)
-              .replace(/^\s*#?WEBI_CHANNEL=.*/m, 'WEBI_CHANNEL=' + rel.channel)
-              .replace(
-                /^\s*#?WEBI_EXT=.*/m,
-                'WEBI_EXT=' + rel.ext.replace(/tar.*/, 'tar')
-              )
-              .replace(
-                /^\s*#?WEBI_FORMATS=.*/m,
-                "WEBI_FORMATS='" + formats.join(',') + "'"
-              )
-              .replace(
-                /^\s*#?WEBI_PKG_URL=.*/m,
-                "WEBI_PKG_URL='" + rel.download + "'"
-              )
-              .replace(
-                /^\s*#?WEBI_PKG_FILE=.*/m,
-                "WEBI_PKG_FILE='" + pkgFile + "'"
-              )
-              // PKG details
-              .replace(/^\s*#?PKG_NAME=.*/m, "PKG_NAME='" + pkg + "'")
-              .replace(
-                /^\s*#?PKG_OSES=.*/m,
-                "PKG_OSES='" + ((rel && rel.oses) || []).join(',') + "'"
-              )
-              .replace(
-                /^\s*#?PKG_ARCHES=.*/m,
-                "PKG_ARCHES='" + ((rel && rel.arches) || []).join(',') + "'"
-              )
-              .replace(
-                /^\s*#?PKG_FORMATS=.*/m,
-                "PKG_FORMATS='" + ((rel && rel.formats) || []).join(',') + "'"
-              )
-              .replace(reInstallTpl, '\n' + installTxt)
-          );
-        });
-    });
-};
-
-Releases.renderBatch = function (
-  pkgdir,
-  rel,
-  { baseurl, pkg, tag, ver, os, arch, formats }
-) {
-  if (!Array.isArray(formats)) {
-    formats = [];
-  }
-  if (!tag) {
-    tag = '';
-  }
-  return fs.promises
-    .readFile(path.join(pkgdir, 'install.bat'), 'utf8')
-    .then(function (installTxt) {
-      installTxt = padScript(installTxt);
-      /*
-      var vers = rel.version.split('.');
-      var v = {
-        major: vers.shift() || '',
-        minor: vers.shift() || '',
-        patch: vers.join('.').replace(/[+\-].*$/, ''),
-        build: vers
-          .join('.')
-          .replace(/[^+\-]*()/, '')
-          .replace(/^-/, '')
-      };
-      */
-      return fs.promises
-        .readFile(path.join(__dirname, 'template.bat'), 'utf8')
-        .then(function (tplTxt) {
-          return tplTxt
-            .replace(
-              /^(REM )?WEBI_PKG=.*/im,
-              "WEBI_PKG='" + pkg + '@' + ver + "'"
-            )
-            .replace(reInstallTpl, '\n' + installTxt);
-        });
-    });
-};
-
-Releases.renderPowerShell = function (
-  pkgdir,
-  rel,
-  { baseurl, pkg, tag, ver, os, arch, formats }
-) {
-  if (!Array.isArray(formats)) {
-    formats = [];
-  }
-  if (!tag) {
-    tag = '';
-  }
-  return fs.promises
-    .readFile(path.join(pkgdir, 'install.ps1'), 'utf8')
-    .then(function (installTxt) {
-      installTxt = padScript(installTxt);
-      /*
-      var vers = rel.version.split('.');
-      var v = {
-        major: vers.shift() || '',
-        minor: vers.shift() || '',
-        patch: vers.join('.').replace(/[+\-].*$/, ''),
-        build: vers
-          .join('.')
-          .replace(/[^+\-]*()/, '')
-          .replace(/^-/, '')
-      };
-      */
-      return fs.promises
-        .readFile(path.join(__dirname, 'template.ps1'), 'utf8')
-        .then(function (tplTxt) {
-          var pkgver = pkg + '@' + ver;
-          return tplTxt
-            .replace(
-              /^(#)?\$Env:WEBI_HOST\s*=.*/im,
-              "$Env:WEBI_HOST = '" + baseurl + "'"
-            )
-            .replace(
-              /^(#)?\$Env:WEBI_PKG\s*=.*/im,
-              "$Env:WEBI_PKG = '" + pkgver + "'"
-            )
-            .replace(
-              /^(#)?\$Env:PKG_NAME\s*=.*/im,
-              "$Env:PKG_NAME = '" + pkg + "'"
-            )
-            .replace(
-              /^(#)?\$Env:WEBI_VERSION\s*=.*/im,
-              "$Env:WEBI_VERSION = '" + rel.version + "'"
-            )
-            .replace(
-              /^(#)?\$Env:WEBI_PKG_URL\s*=.*/im,
-              "$Env:WEBI_PKG_URL = '" + rel.download + "'"
-            )
-            .replace(
-              /^(#)?\$Env:WEBI_PKG_FILE\s*=.*/im,
-              "$Env:WEBI_PKG_FILE = '" + rel.name + "'"
-            )
-            .replace(reInstallTpl, '\n' + installTxt);
-        });
-    });
-};

_webi/serve-installer.js

@@ -0,0 +1,304 @@
+'use strict';
+
+var InstallerServer = module.exports;
+
+let Fs = require('fs/promises');
+let Path = require('path');
+
+let HostTargets = require('./build-classifier/host-targets.js');
+let Builds = require('./builds.js');
+let Installers = require('./installers.js');
+
+InstallerServer.INSTALLERS_DIR = Path.join(__dirname, '..');
+InstallerServer.serveInstaller = async function (
+  baseurl,
+  ua,
+  pkg,
+  tag,
+  ext,
+  formats,
+  libc,
+) {
+  let unameAgent = ua;
+  let projectName = pkg;
+  let [rel, tmplParams] = await InstallerServer.helper({
+    unameAgent,
+    projectName,
+    tag,
+    formats,
+    libc,
+  });
+  Object.assign(tmplParams, {
+    baseurl,
+  });
+
+  var pkgdir = Path.join(InstallerServer.INSTALLERS_DIR, projectName);
+  if ('ps1' === ext) {
+    return Installers.renderPowerShell(pkgdir, rel, tmplParams);
+  }
+  return Installers.renderBash(pkgdir, rel, tmplParams);
+};
+
+// TODO put some of this in a middleware? or common function?
+// TODO maybe move package/version/lts/channel detection into getReleases
+InstallerServer.helper = async function ({
+  unameAgent,
+  projectName,
+  tag,
+  formats,
+  libc,
+}) {
+  console.log(`dbg: Installer User-Agent: ${unameAgent}`);
+
+  let releaseTarget = toReleaseTarget(tag);
+  let hostFormats = formats;
+  let terms = unameAgent.split(/[\s\/]+/g);
+  let hostTarget = {};
+  try {
+    void HostTargets.termsToTarget(hostTarget, terms);
+  } catch (e) {
+    // if we can't guarantee the results...
+    // "in the face of ambiguity, refuse the temptation to guess"
+    throw e;
+  }
+  console.log(`dbg: Installer Host Target:`);
+  console.log(hostTarget);
+
+  if (!hostTarget.os) {
+    throw new Error(`OS could not be identified by User-Agent '${unameAgent}'`);
+  }
+
+  console.log(`dbg: Get Project Installer Type for '${projectName}':`);
+  let proj = await Builds.getProjectType(projectName);
+  if (proj.type === 'alias') {
+    console.log(`dbg: alias`, proj);
+    projectName = proj.detail;
+    proj = await Builds.getProjectType(projectName); // an alias should never resolve to an alias
+  }
+  console.log(`dbg: proj`, proj);
+
+  let validTypes = ['selfhosted', 'valid'];
+  if (!validTypes.includes(proj.type)) {
+    let msg = `'${projectName}' doesn't have an installer: '${proj.type}': '${proj.detail}'`;
+    let err = new Error(msg);
+    err.code = 'ENOENT';
+    throw err;
+  }
+
+  let tmplParams = {
+    pkg: projectName,
+    tag: tag,
+    os: hostTarget.os,
+    arch: hostTarget.arch,
+    libc: hostTarget.libc,
+    formats: hostFormats,
+    limit: 1,
+  };
+  Object.assign(tmplParams, releaseTarget);
+  console.log('tmplParams', tmplParams);
+
+  let errPackage = {
+    name: 'doesntexist.ext',
+    version: '0.0.0',
+    lts: '-',
+    channel: 'error',
+    date: '1970-01-01',
+    os: hostTarget.os || '-',
+    arch: hostTarget.arch || '-',
+    libc: hostTarget.libc || '-',
+    ext: 'err',
+    download: 'https://example.com/doesntexist.ext',
+    comment:
+      'No matches found. Could be bad or missing version info' +
+      ',' +
+      "Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
+  };
+
+  if (proj.type === 'selfhosted') {
+    return [errPackage, tmplParams];
+  }
+
+  let projInfo = await Builds.getPackage({
+    name: projectName,
+    date: new Date(),
+  });
+  let latestVersions = Builds.enumerateLatestVersions(projInfo);
+  //console.log('projInfo', projInfo);
+
+  let buildTargetInfo = {
+    triplets: projInfo.triplets,
+    oses: projInfo.oses,
+    arches: projInfo.arches,
+    libcs: projInfo.libcs,
+    formats: projInfo.formats,
+    latest: latestVersions.latest,
+    stable: latestVersions.stable,
+  };
+
+  // TODO .findMatchingPackages() should probably account for this
+  let hasOs = projInfo.oses.includes(hostTarget.os);
+  let maybePosix = !hasOs && hostTarget.os !== 'windows';
+  if (maybePosix) {
+    let posixes = ['posix_2017', 'posix_2024'];
+    for (let posixYear of posixes) {
+      let hasPosix = projInfo.oses.includes(posixYear);
+      if (hasPosix) {
+        hasOs = true;
+        break;
+      }
+    }
+  }
+  if (!hasOs) {
+    hasOs = projInfo.oses.includes('ANYOS');
+  }
+
+  if (!hasOs) {
+    let pkg1 = Object.assign(buildTargetInfo, errPackage);
+    return [pkg1, tmplParams];
+  }
+
+  let targetRelease = Builds.findMatchingPackages(
+    projInfo,
+    hostTarget,
+    releaseTarget,
+  );
+  // { triplet: `${os}-${arch}-${libc}`, packages: targetPackages
+  // , latest: projInfo.versions[0], versions: matchInfo
+  // }
+
+  if (!targetRelease?.packages) {
+    let pkg1 = Object.assign(buildTargetInfo, errPackage);
+    return [pkg1, tmplParams];
+  }
+
+  let buildPkg = Builds.selectPackage(targetRelease.packages, hostFormats);
+  let ext = buildPkg.ext || '.exe';
+  if (ext.startsWith('.')) {
+    ext = ext.slice(1);
+  }
+
+  let version = targetRelease.version;
+  if (version.startsWith('v')) {
+    version = version.slice(1);
+  }
+
+  buildPkg = Object.assign(buildTargetInfo, buildPkg, { ext, version });
+  console.log('dbg: buildPkg', buildPkg);
+  console.log('dbg: tmplParams', tmplParams);
+  return [buildPkg, tmplParams];
+};
+
+let channelNames = [
+  'stable',
+  // 'hotfix',
+  'latest',
+  'rc',
+  'preview',
+  'pre',
+  'dev',
+  'beta',
+  'alpha',
+];
+
+function toReleaseTarget(tag) {
+  tag = tag.replace(/^v/, '');
+
+  let releaseTarget = {
+    channel: '',
+    lts: false,
+    version: '',
+  };
+
+  if (tag === 'lts') {
+    releaseTarget.lts = true;
+    releaseTarget.channel = 'stable';
+  } else if (channelNames.includes(tag)) {
+    releaseTarget.channel = tag;
+  } else {
+    releaseTarget.version = tag;
+  }
+
+  return releaseTarget;
+}
+
+var CURL_PIPE_PS1_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.ps1');
+var CURL_PIPE_SH_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.sh');
+var BAD_SH_RE = /[<>'"`$\\]/;
+
+InstallerServer.getPosixCurlPipeBootstrap = async function ({
+  baseurl,
+  pkg,
+  ver,
+}) {
+  let bootTxt = await Fs.readFile(CURL_PIPE_SH_BOOT, 'utf8');
+
+  var webiPkg = [pkg, ver].filter(Boolean).join('@');
+  var webiChecksum = await Installers.getWebiShChecksum();
+  var envReplacements = [
+    ['WEBI_PKG', webiPkg],
+    ['WEBI_HOST', baseurl],
+    ['WEBI_CHECKSUM', webiChecksum],
+  ];
+
+  for (let env of envReplacements) {
+    let name = env[0];
+    let value = env[1];
+
+    let envRe = new RegExp(
+      `^[ \\t]*#?[ \\t]*(export[ \\t])?[ \\t]*(${name})=.*`,
+      'm',
+    );
+
+    if (BAD_SH_RE.test(value)) {
+      throw new Error(`key '${name}' has invalid value '${value}'`);
+    }
+
+    bootTxt = bootTxt.replace(envRe, `$1$2='${value}'`);
+  }
+  // TODO init config here
+  //bootTxt.replace(/CHEATSHEET_URL/g, `${Config.cheatUrl}/${pkg}`);
+
+  return bootTxt;
+};
+
+InstallerServer.getPwshCurlPipeBootstrap = async function ({
+  baseurl,
+  pkg,
+  ver,
+  exename,
+}) {
+  let bootTxt = await Fs.readFile(CURL_PIPE_PS1_BOOT, 'utf8');
+
+  var webiPkg = [pkg, ver].filter(Boolean).join('@');
+  //var webiChecksum = await InstallerServer.getWebiPs1Checksum();
+  var envReplacements = [
+    ['Env:WEBI_PKG', webiPkg],
+    ['Env:WEBI_HOST', baseurl],
+    //['Env:WEBI_CHECKSUM', webiChecksum],
+    ['baseurl', baseurl],
+    ['exename', exename],
+    ['version', ver],
+  ];
+
+  for (let env of envReplacements) {
+    let name = env[0];
+    let value = env[1];
+
+    if (BAD_SH_RE.test(value)) {
+      throw new Error(`key '${name}' has invalid value '${value}'`);
+    }
+
+    let tplRe = new RegExp(`{{ (${name}) }}`, 'g');
+    bootTxt = bootTxt.replace(tplRe, `${value}`);
+
+    let setRe = new RegExp(
+      `(#[ \\t]*)?(\\$${name})[ \\t]*=[ \\t]['"].*['"][ \\t]`,
+      'im',
+    );
+    bootTxt = bootTxt.replace(setRe, `$$${name} = '${value}'`);
+  }
+  // TODO init config here
+  //bootTxt.replace(/CHEATSHEET_URL/g, `${Config.cheatUrl}/${pkg}`);
+
+  return bootTxt;
+};

_webi/template.bat

@@ -1,5 +0,0 @@
-REM REM debug
-
-REM WEBI_PKG=
-
-{{ installer }}

_webi/template.ps1

@@ -1,47 +0,0 @@
-#!/usr/bin/env pwsh
-
-# this allows us to call ps1 files, which allows us to have spaces in filenames
-# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
-# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
-Set-ExecutionPolicy -Scope Process Bypass
-
-# If a command returns an error, halt the script.
-$ErrorActionPreference = 'Stop'
-
-# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
-$ProgressPreference = 'SilentlyContinue'
-
-$Env:WEBI_HOST = 'https://webinstall.dev'
-#$Env:WEBI_PKG = 'node@lts'
-#$Env:PKG_NAME = node
-#$Env:WEBI_VERSION = v12.16.2
-#$Env:WEBI_PKG_URL = "https://.../node-....zip"
-#$Env:WEBI_PKG_FILE = "node-v12.16.2-win-x64.zip"
-
-# Switch to userprofile
-pushd $Env:USERPROFILE
-
-# Make paths
-New-Item -Path Downloads -ItemType Directory -Force | out-null
-New-Item -Path .local\bin -ItemType Directory -Force | out-null
-New-Item -Path .local\opt -ItemType Directory -Force | out-null
-
-# {{ baseurl }}
-# {{ version }}
-
-function webi_add_path
-{
-    & "$Env:USERPROFILE\.local\bin\pathman.exe" add "$args[0]"
-    # Note: not all of these work as expected, so we use the unix-style, which is most consistent
-    #& "$Env:USERPROFILE\.local\bin\pathman.exe" add ~/.local/bin
-    #& "$Env:USERPROFILE\.local\bin\pathman.exe" add "$Env:USERPROFILE\.local\bin"
-    #& "$Env:USERPROFILE\.local\bin\pathman.exe" add %USERPROFILE%\.local\bin
-}
-
-# Run pathman to set up the folder
-& "$Env:USERPROFILE\.local\bin\pathman.exe" add ~/.local/bin
-
-{{ installer }}
-
-# Done
-popd

_webi/template.sh

@@ -1,391 +0,0 @@
-#!/bin/bash
-
-# shellcheck disable=2001
-# because I prefer to use sed rather than bash replace
-# (there's too little space in my head to learn both syntaxes)
-
-function __bootstrap_webi() {
-
-    set -e
-    set -u
-    #set -x
-
-    #WEBI_PKG=
-    #PKG_NAME=
-    # TODO should this be BASEURL instead?
-    #WEBI_OS=
-    #WEBI_ARCH=
-    #WEBI_HOST=
-    #WEBI_RELEASES=
-    #WEBI_CSV=
-    #WEBI_TAG=
-    #WEBI_VERSION=
-    #WEBI_MAJOR=
-    #WEBI_MINOR=
-    #WEBI_PATCH=
-    # TODO not sure if BUILD is the best name for this
-    #WEBI_BUILD=
-    #WEBI_LTS=
-    #WEBI_CHANNEL=
-    #WEBI_EXT=
-    #WEBI_FORMATS=
-    #WEBI_PKG_URL=
-    #WEBI_PKG_FILE=
-    #PKG_OSES=
-    #PKG_ARCHES=
-    #PKG_FORMATS=
-    WEBI_UA="$(uname -a)"
-    export WEBI_HOST
-
-    ##
-    ## Set up tmp, download, and install directories
-    ##
-
-    WEBI_TMP=${WEBI_TMP:-"$(mktemp -d -t webinstall-"${WEBI_PKG:-}".XXXXXXXX)"}
-    export _webi_tmp="${_webi_tmp:-"$HOME/.local/opt/webi-tmp.d"}"
-
-    mkdir -p "$HOME/Downloads"
-    mkdir -p "$HOME/.local/bin"
-    mkdir -p "$HOME/.local/opt"
-
-    ##
-    ## Detect http client
-    ##
-    set +e
-    WEBI_CURL="$(command -v curl)"
-    export WEBI_CURL
-    WEBI_WGET="$(command -v wget)"
-    export WEBI_WGET
-    set -e
-
-    # get the special formatted version (i.e. "go is go1.14" while node is "node v12.10.8")
-    my_versioned_name=""
-    _webi_canonical_name() {
-        if [ -n "$my_versioned_name" ]; then
-            echo "$my_versioned_name"
-            return 0
-        fi
-
-        if [ -n "$(command -v pkg_format_cmd_version)" ]; then
-            my_versioned_name="$(pkg_format_cmd_version "$WEBI_VERSION")"
-        else
-            my_versioned_name="'$pkg_cmd_name' v$WEBI_VERSION"
-        fi
-
-        echo "$my_versioned_name"
-    }
-
-    # update symlinks according to $HOME/.local/opt and $HOME/.local/bin install paths.
-    # shellcheck disable=2120
-    # webi_link may be used in the templated install script
-    webi_link() {
-        if [ -n "$(command -v pkg_link)" ]; then
-            pkg_link
-            return 0
-        fi
-
-        if [ -n "$WEBI_SINGLE" ] || [ "single" == "${1:-}" ]; then
-            rm -rf "$pkg_dst_cmd"
-            ln -s "$pkg_src_cmd" "$pkg_dst_cmd"
-        else
-            # 'pkg_dst' will default to $HOME/.local/opt/<pkg>
-            # 'pkg_src' will be the installed version, such as to $HOME/.local/opt/<pkg>-<version>
-            rm -rf "$pkg_dst"
-            ln -s "$pkg_src" "$pkg_dst"
-        fi
-    }
-
-    # detect if this program is already installed or if an installed version may cause conflict
-    webi_check() {
-        # Test for existing version
-        set +e
-        my_path="$PATH"
-        PATH="$(dirname "$pkg_dst_cmd"):$PATH"
-        export PATH
-        my_current_cmd="$(command -v "$pkg_cmd_name")"
-        set -e
-        if [ -n "$my_current_cmd" ]; then
-            pkg_current_version="$(pkg_get_current_version 2> /dev/null | head -n 1)"
-            # remove trailing '.0's for golang's sake
-            my_current_version="$(echo "$pkg_current_version" | sed 's:\.0::g')"
-            my_src_version="$(echo "$WEBI_VERSION" | sed 's:\.0::g')"
-            my_canonical_name="$(_webi_canonical_name)"
-            if [ "$my_src_version" == "$my_current_version" ]; then
-                echo "$my_canonical_name already installed at $my_current_cmd"
-                exit 0
-            else
-                if [ "$my_current_cmd" != "$pkg_dst_cmd" ]; then
-                    echo >&2 "WARN: possible conflict between $my_canonical_name and $pkg_current_version at $my_current_cmd"
-                fi
-                if [ -x "$pkg_src_cmd" ]; then
-                    # shellcheck disable=2119
-                    # this function takes no args
-                    webi_link
-                    echo "switched to $my_canonical_name at $pkg_src"
-                    exit 0
-                fi
-            fi
-        fi
-        export PATH="$my_path"
-    }
-
-    # detect if file is downloaded, and how to download it
-    webi_download() {
-        if [ -n "${1:-}" ]; then
-            my_url="$1"
-        else
-            if [ "error" == "$WEBI_CHANNEL" ]; then
-                # TODO pass back requested OS / Arch / Version
-                echo >&2 "Error: no '$PKG_NAME' release for '${WEBI_OS:-}' on '$WEBI_ARCH' as one of '$WEBI_FORMATS' by the tag '${WEBI_TAG:-}'"
-                echo >&2 "       '$PKG_NAME' is available for '$PKG_OSES' on '$PKG_ARCHES' as one of '$PKG_FORMATS'"
-                echo >&2 "       (check that the package name and version are correct)"
-                echo >&2 ""
-                echo >&2 "       Double check at $(echo "$WEBI_RELEASES" | sed 's:\?.*::')"
-                echo >&2 ""
-                exit 1
-            fi
-            my_url="$WEBI_PKG_URL"
-        fi
-        if [ -n "${2:-}" ]; then
-            my_dl="$2"
-        else
-            my_dl="$HOME/Downloads/$WEBI_PKG_FILE"
-        fi
-
-        if [ -e "$my_dl" ]; then
-            echo "Found $my_dl"
-            return 0
-        fi
-
-        echo "Downloading $PKG_NAME from"
-        echo "$my_url"
-
-        # It's only 2020, we can't expect to have reliable CLI tools
-        # to tell us the size of a file as part of a base system...
-        if [ -n "$WEBI_WGET" ]; then
-            # wget has resumable downloads
-            # TODO wget -c --content-disposition "$my_url"
-            set +e
-            my_show_progress=""
-            if [[ $- == *i* ]]; then
-                my_show_progress="--show-progress"
-            fi
-            if ! wget -q $my_show_progress --user-agent="wget $WEBI_UA" -c "$my_url" -O "$my_dl.part"; then
-                echo >&2 "failed to download from $WEBI_PKG_URL"
-                exit 1
-            fi
-            set -e
-        else
-            # Neither GNU nor BSD curl have sane resume download options, hence we don't bother
-            # TODO curl -fsSL --remote-name --remote-header-name --write-out "$my_url"
-            my_show_progress="-#"
-            if [[ $- == *i* ]]; then
-                my_show_progress=""
-            fi
-            # shellcheck disable=SC2086
-            # we want the flags to be split
-            curl -fSL $my_show_progress -H "User-Agent: curl $WEBI_UA" "$my_url" -o "$my_dl.part"
-        fi
-        mv "$my_dl.part" "$my_dl"
-
-        echo ""
-        echo "Saved as $my_dl"
-    }
-
-    # detect which archives can be used
-    webi_extract() {
-        pushd "$WEBI_TMP" > /dev/null 2>&1
-        if [ "tar" == "$WEBI_EXT" ]; then
-            echo "Extracting $HOME/Downloads/$WEBI_PKG_FILE"
-            tar xf "$HOME/Downloads/$WEBI_PKG_FILE"
-        elif [ "zip" == "$WEBI_EXT" ]; then
-            echo "Extracting $HOME/Downloads/$WEBI_PKG_FILE"
-            unzip "$HOME/Downloads/$WEBI_PKG_FILE" > __unzip__.log
-        elif [ "exe" == "$WEBI_EXT" ]; then
-            echo "Moving $HOME/Downloads/$WEBI_PKG_FILE"
-            mv "$HOME/Downloads/$WEBI_PKG_FILE" .
-        elif [ "xz" == "$WEBI_EXT" ]; then
-            echo "Inflating $HOME/Downloads/$WEBI_PKG_FILE"
-            unxz -c "$HOME/Downloads/$WEBI_PKG_FILE" > "$(basename "$WEBI_PKG_FILE")"
-        else
-            # do nothing
-            echo "Failed to extract $HOME/Downloads/$WEBI_PKG_FILE"
-            exit 1
-        fi
-        popd > /dev/null 2>&1
-    }
-
-    # use 'pathman' to update $HOME/.config/envman/PATH.env
-    webi_path_add() {
-        # make sure that we don't recursively install pathman with webi
-        my_path="$PATH"
-        export PATH="$HOME/.local/bin:$PATH"
-
-        # install pathman if not already installed
-        if [ -z "$(command -v pathman)" ]; then
-            "$HOME/.local/bin/webi" pathman > /dev/null
-        fi
-
-        export PATH="$my_path"
-
-        # in case pathman was recently installed and the PATH not updated
-        mkdir -p "$_webi_tmp"
-        # prevent "too few arguments" output on bash when there are 0 lines of stdout
-        "$HOME/.local/bin/pathman" add "$1" | grep "export" 2> /dev/null >> "$_webi_tmp/.PATH.env" || true
-    }
-
-    # group common pre-install tasks as default
-    webi_pre_install() {
-        webi_check
-        webi_download
-        webi_extract
-    }
-
-    # move commands from the extracted archive directory to $HOME/.local/opt or $HOME/.local/bin
-    # shellcheck disable=2120
-    # webi_install may be sourced and used elsewhere
-    webi_install() {
-        if [ -n "$WEBI_SINGLE" ] || [ "single" == "${1:-}" ]; then
-            mkdir -p "$(dirname "$pkg_src_cmd")"
-            mv ./"$pkg_cmd_name"* "$pkg_src_cmd"
-        else
-            rm -rf "$pkg_src"
-            mv ./"$pkg_cmd_name"* "$pkg_src"
-        fi
-    }
-
-    # run post-install functions - just updating PATH by default
-    webi_post_install() {
-        webi_path_add "$(dirname "$pkg_dst_cmd")"
-    }
-
-    _webi_enable_exec() {
-        if [ -n "$(command -v spctl)" ] && [ -n "$(command -v xattr)" ]; then
-            # note: some packages contain files that cannot be affected by xattr
-            xattr -r -d com.apple.quarantine "$pkg_src" || true
-            return 0
-        fi
-        # TODO need to test that the above actually worked
-        # (and proceed to this below if it did not)
-        if [ -n "$(command -v spctl)" ]; then
-            echo "Checking permission to execute '$pkg_cmd_name' on macOS 11+"
-            set +e
-            is_allowed="$(spctl -a "$pkg_src_cmd" 2>&1 | grep valid)"
-            set -e
-            if [ -z "$is_allowed" ]; then
-                echo ""
-                echo "##########################################"
-                echo "#  IMPORTANT: Permission Grant Required  #"
-                echo "##########################################"
-                echo ""
-                echo "Requesting permission to execute '$pkg_cmd_name' on macOS 10.14+"
-                echo ""
-                sleep 3
-                spctl --add "$pkg_src_cmd"
-            fi
-        fi
-    }
-
-    # a friendly message when all is well, showing the final install path in $HOME/.local
-    _webi_done_message() {
-        echo "Installed $(_webi_canonical_name) as $pkg_dst_cmd"
-    }
-
-    ##
-    ##
-    ## BEGIN custom override functions from <package>/install.sh
-    ##
-    ##
-
-    WEBI_SINGLE=
-
-    if [[ -z ${WEBI_WELCOME:-} ]]; then
-        echo ""
-        echo "Thanks for using webi to install '$PKG_NAME' on '$WEBI_OS/$WEBI_ARCH'."
-        echo "Have a problem? Experience a bug? Please let us know:"
-        echo "        https://github.com/webinstall/packages/issues"
-        echo ""
-    fi
-
-    function __init_installer() {
-
-        # do nothing - to satisfy parser prior to templating
-        echo -n ""
-
-        # {{ installer }}
-
-    }
-
-    __init_installer
-
-    ##
-    ##
-    ## END custom override functions
-    ##
-    ##
-
-    # run everything with defaults or overrides as needed
-    if [ -n "$(command -v pkg_get_current_version)" ]; then
-        pkg_cmd_name="${pkg_cmd_name:-$PKG_NAME}"
-
-        if [ -n "$WEBI_SINGLE" ]; then
-            pkg_dst_cmd="${pkg_dst_cmd:-$HOME/.local/bin/$pkg_cmd_name}"
-            pkg_dst="$pkg_dst_cmd" # "$(dirname "$(dirname $pkg_dst_cmd)")"
-
-            #pkg_src_cmd="${pkg_src_cmd:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION/bin/$pkg_cmd_name-v$WEBI_VERSION}"
-            pkg_src_cmd="${pkg_src_cmd:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION/bin/$pkg_cmd_name}"
-            pkg_src="$pkg_src_cmd" # "$(dirname "$(dirname $pkg_src_cmd)")"
-        else
-            pkg_dst="${pkg_dst:-$HOME/.local/opt/$pkg_cmd_name}"
-            pkg_dst_cmd="${pkg_dst_cmd:-$pkg_dst/bin/$pkg_cmd_name}"
-
-            pkg_src="${pkg_src:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION}"
-            pkg_src_cmd="${pkg_src_cmd:-$pkg_src/bin/$pkg_cmd_name}"
-        fi
-        # this script is templated and these are used elsewhere
-        # shellcheck disable=SC2034
-        pkg_src_bin="$(dirname "$pkg_src_cmd")"
-        # shellcheck disable=SC2034
-        pkg_dst_bin="$(dirname "$pkg_dst_cmd")"
-
-        if [[ -n "$(command -v pkg_pre_install)" ]]; then pkg_pre_install; else webi_pre_install; fi
-
-        pushd "$WEBI_TMP" > /dev/null 2>&1
-        echo "Installing to $pkg_src_cmd"
-        if [[ -n "$(command -v pkg_install)" ]]; then pkg_install; else webi_install; fi
-        chmod a+x "$pkg_src"
-        chmod a+x "$pkg_src_cmd"
-        popd > /dev/null 2>&1
-
-        webi_link
-
-        _webi_enable_exec
-        pushd "$WEBI_TMP" > /dev/null 2>&1
-        if [[ -n "$(command -v pkg_post_install)" ]]; then pkg_post_install; else webi_post_install; fi
-        popd > /dev/null 2>&1
-
-        pushd "$WEBI_TMP" > /dev/null 2>&1
-        if [[ -n "$(command -v pkg_done_message)" ]]; then pkg_done_message; else _webi_done_message; fi
-        popd > /dev/null 2>&1
-
-        echo ""
-    fi
-
-    webi_path_add "$HOME/.local/bin"
-    if [[ -z ${_WEBI_CHILD:-} ]] && [[ -f "$_webi_tmp/.PATH.env" ]]; then
-        if [[ -n $(cat "$_webi_tmp/.PATH.env") ]]; then
-            echo "You need to update your PATH to use $PKG_NAME:"
-            echo ""
-            sort -u "$_webi_tmp/.PATH.env"
-            rm -f "$_webi_tmp/.PATH.env"
-        fi
-    fi
-
-    # cleanup the temp directory
-    rm -rf "$WEBI_TMP"
-
-    # See? No magic. Just downloading and moving files.
-
-}
-
-__bootstrap_webi

_webi/test.js

@@ -5,7 +5,7 @@
 //
 var usage = [
   'Usage: node _webi/test.js --debug <path-to-package>',
-  'Example: node _webi/test.js --debug ./node/'
+  'Example: node _webi/test.js --debug ./node/',
 ].join('\n');
 
 var count = 3;
@@ -32,8 +32,10 @@ if (/\b-?-h(elp)?\b/.test(process.argv.join(' '))) {
 var os = require('os');
 var fs = require('fs');
 var path = require('path');
-var Releases = require('./releases.js');
-var uaDetect = require('./ua-detect.js');
+var Builds = require('./builds.js');
+var Installers = require('./installers.js');
+var ServeInstaller = require('./serve-installer.js');
+
 var pkg = process.argv[2].split('@');
 var pkgdir = pkg[0];
 var pkgtag = pkg[1] || '';
@@ -42,6 +44,7 @@ var nodes = fs.readdirSync(pkgdir);
 nodes.forEach(function (node) {
   nodesMap[node] = true;
 });
+var baseurl = 'https://webinstall.dev';
 
 var maxLen = 0;
 console.info('');
@@ -62,31 +65,49 @@ console.info('Has the necessary files?');
   });
 
 console.info('');
-Releases.get(path.join(process.cwd(), pkgdir)).then(function (all) {
+let projName = pkgdir.split('/').filter(Boolean).pop();
+Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
   var pkgname = path.basename(pkgdir.replace(/\/$/, ''));
-  var osrel = os.platform() + '-' + os.release();
-  var arch = os.arch();
-  var formats = ['exe', 'xz', 'tar', 'zip'];
+  var nodeOs = os.platform();
+  var nodeOsRelease = os.release();
+  var nodeArch = os.arch();
+  var nodeLibc = 'libc';
+  if (process.platform === 'linux') {
+    nodeLibc = 'gnu';
+    let isUnofficial =
+      process.config.variables.node_release_urlbase.includes('unofficial');
+    if (isUnofficial) {
+      nodeLibc = 'musl';
+    }
+  }
+  var formats = ['exe', 'xz', 'tar', 'zip', 'git'];
 
-  var rel = all.releases.filter(function (rel) {
-    return (
-      formats.filter(function (ext) {
-        return rel.ext.match(ext);
-      })[0] &&
-      'stable' === rel.channel &&
-      rel.os === uaDetect.os(osrel) &&
-      rel.arch === uaDetect.arch(arch) &&
-      (!pkgtag ||
-        rel.tag === pkgtag ||
-        new RegExp('^' + pkgtag).test(rel.version))
-    );
-  })[0];
-  rel.oses = all.oses;
-  rel.arches = all.arches;
-  rel.formats = all.formats;
+  let [rel, opts] = await ServeInstaller.helper({
+    unameAgent: `${nodeOs}/${nodeOsRelease} ${nodeArch}/unknown ${nodeLibc}`,
+    projectName: pkgname,
+    tag: pkgtag || '',
+    formats: formats,
+    libc: nodeLibc,
+  });
+  Object.assign(
+    {
+      ver: '',
+      lts: null,
+      channel: '',
+      os: '',
+      arch: '',
+      limit: 0,
+    },
+    opts,
+    {
+      baseurl,
+    },
+  );
 
   if (!rel) {
-    console.error('Error: ❌ no release found for current os, arch, and tag');
+    console.error(
+      `Error: ❌ no release found for @${pkgtag}?os=${nodeOs}&arch=${nodeArch}&libc=${nodeLibc}&formats=${formats}`,
+    );
     process.exit(1);
     return;
   }
@@ -97,24 +118,8 @@ Releases.get(path.join(process.cwd(), pkgdir)).then(function (all) {
   console.info('');
 
   return Promise.all([
-    Releases.renderBash(pkgdir, rel, {
-      baseurl: 'https://webinstall.dev',
-      pkg: pkgname,
-      tag: pkgtag || '',
-      ver: '',
-      os: osrel,
-      arch,
-      formats: formats
-    }).catch(function () {}),
-    Releases.renderPowerShell(pkgdir, rel, {
-      baseurl: 'https://webinstall.dev',
-      pkg: pkgname,
-      tag: pkgtag || '',
-      ver: '',
-      os: osrel,
-      arch,
-      formats: formats
-    }).catch(function () {})
+    Installers.renderBash(pkgdir, rel, opts).catch(function () {}),
+    Installers.renderPowerShell(pkgdir, rel, opts).catch(function () {}),
   ]).then(function (scripts) {
     var bashTxt = scripts[0];
     var ps1Txt = scripts[1];
@@ -125,18 +130,22 @@ Releases.get(path.join(process.cwd(), pkgdir)).then(function (all) {
       bashTxt = (bashTxt || 'echo ERROR').replace(/#set -x/g, 'set -x');
       ps1Txt = (ps1Txt || 'echo ERROR').replace(
         /REM REM todo debug/g,
-        'REM todo debug'
+        'REM todo debug',
       );
     }
     console.info('Do the scripts actually work?');
     if (bashFile && bashTxt) {
       fs.writeFileSync(bashFile, bashTxt, 'utf-8');
-      console.info('\tNEEDS MANUAL TEST: bash %s', bashFile);
+      console.info('\tNEEDS MANUAL TEST: sh %s', bashFile);
     }
     if (ps1File && ps1Txt) {
       fs.writeFileSync(ps1File, ps1Txt, 'utf-8');
       console.info('\tNEEDS MANUAL TEST: powershell.exe %s', ps1File);
     }
     console.info('');
+    setTimeout(function () {
+      console.warn(`[warn] dangling event loop handle`);
+      process.exit(0);
+    }, 300).unref();
   });
 });

_webi/transform-releases.js

@@ -0,0 +1,434 @@
+'use strict';
+
+var Releases = module.exports;
+
+var path = require('path');
+var _normalize = require('./normalize.js');
+
+var cache = {};
+//var staleAge = 5 * 1000;
+//var expiredAge = 15 * 1000;
+var staleAge = 5 * 60 * 1000;
+var expiredAge = 15 * 60 * 1000;
+
+let installerDir = path.join(__dirname, '..');
+
+Releases.get = async function (pkgdir) {
+  let get;
+  try {
+    get = require(`${pkgdir}/releases.js`);
+    // TODO update all releases files with module.exports.xxxx = 'foo';
+    if (!get.latest) {
+      get.latest = get;
+    }
+  } catch (e) {
+    let err = new Error('no releases.js for', pkgdir.split(/[\/\\]+/).pop());
+    err.code = 'E_NO_RELEASE';
+    throw err;
+  }
+
+  let all = await get.latest();
+
+  return _normalize(all);
+};
+
+// TODO needs a proper test, and more accurate (though perhaps far less simple) code
+function createFormatsSorter(formats) {
+  return function sortByVerExt(a, b) {
+    function lexver(semver) {
+      // v1.20.156 => 00001.00020.00156.zzzzz
+      // TODO BUG: v1.20.156-rc2 => 00001.00020.00156.rc2zz
+      var parts = semver.split(/[+\.\-]/g);
+      while (parts.length < 4) {
+        parts.push('');
+      }
+      return parts
+        .map(function (num, i) {
+          if (3 === i) {
+            return num.toString().padEnd(10, 'z');
+          }
+          return num.toString().padStart(10, '0');
+        })
+        .join('.');
+    }
+
+    var aver = lexver(a.version);
+    var bver = lexver(b.version);
+    if (aver > bver) {
+      //console.log(aver, '>', bver);
+      return -1;
+    }
+    if (aver < bver) {
+      //console.log(aver, '<', bver);
+      return 1;
+    }
+
+    var aExtPri = formats.indexOf(a.ext.replace(/tar\..*/, 'tar'));
+    var bExtPri = formats.indexOf(b.ext.replace(/tar\..*/, 'tar'));
+    if (aExtPri > bExtPri) {
+      //console.log(a.ext, aExtPri, '>', b.ext, bExtPri);
+      return -1;
+    }
+    if (aExtPri < bExtPri) {
+      //console.log(a.ext, aExtPri, '<', b.ext, bExtPri);
+      return 1;
+    }
+
+    // rank builds that don't depend on any form of libc first
+    if (a.libc === 'none' && b.libc !== 'none') {
+      return -1;
+    }
+    if (a.libc !== 'none' && b.libc === 'none') {
+      return 1;
+    }
+
+    return 0;
+  };
+}
+
+async function getCachedReleases(pkg) {
+  // returns { download: '<template string>', releases: [{ version, date, os, arch, lts, channel, download}] }
+
+  async function chainCachePromise(fn) {
+    cache[pkg].promise = cache[pkg].promise.then(fn);
+    return cache[pkg].promise;
+  }
+
+  async function sleep(ms) {
+    return await new Promise(function (resolve, reject) {
+      setTimeout(resolve, ms);
+    });
+  }
+
+  async function putCache() {
+    var age = Date.now() - cache[pkg].updatedAt;
+    if (age < staleAge) {
+      //console.debug('NOT STALE ANYMORE - updated in previous promise');
+      return cache[pkg].all;
+    }
+
+    //console.debug('DOWNLOADING NEW "%s" releases', pkg);
+    var pkgdir = path.join(installerDir, pkg);
+
+    // workaround for request timeout seeming to not work
+    let complete = false;
+    await Promise.race([
+      Releases.get(pkgdir)
+        .catch(function (err) {
+          if ('E_NO_RELEASE' === err.code) {
+            let all = { _error: 'E_NO_RELEASE', download: '', releases: [] };
+            return all;
+          }
+
+          throw err;
+        })
+        .catch(function (err) {
+          let hasReleases = cache[pkg].all?.releases?.length > 1;
+          if (!hasReleases) {
+            throw err;
+          }
+
+          console.error(`Error: the BOOGEYMAN got us!`);
+          console.error(err.stack);
+
+          return cache[pkg].all;
+        })
+        .then(function (all) {
+          // Note: it is possible for slightly older data
+          // to replace slightly newer data, but this is better
+          // than being in a cycle where release updates _always_
+          // take longer than expected.
+          //console.debug('DOWNLOADED NEW "%s" releases', pkg);
+          cache[pkg].updatedAt = Date.now();
+          cache[pkg].all = all;
+          complete = true;
+        }),
+      sleep(15000).then(function () {
+        if (complete) {
+          return;
+        }
+        console.error(`request timeout waiting for '${pkg}' release info`);
+      }),
+    ]);
+
+    return cache[pkg].all;
+  }
+
+  if (!cache[pkg]) {
+    cache[pkg] = {
+      updatedAt: 0,
+      all: { download: '', releases: [] },
+      promise: Promise.resolve(),
+    };
+  }
+
+  var bgRenewal;
+  var age = Date.now() - cache[pkg].updatedAt;
+  var fresh = age < staleAge;
+  if (!fresh) {
+    bgRenewal = chainCachePromise(putCache);
+  }
+
+  var tooStale = age > expiredAge;
+  if (!tooStale) {
+    return await cache[pkg].all;
+  }
+
+  return await Promise.race([
+    bgRenewal,
+    sleep(5000).then(function () {
+      return cache[pkg].all;
+    }),
+  ]);
+}
+
+async function filterReleases(
+  all,
+  { ver, os, arch, libc, lts, channel, formats, limit },
+) {
+  // When multiple formats are downloadable (i.e. .zip and .pkg)
+  // sort the most compatible format first
+  // (i.e. so that we don't do .pkg on linux except on purpose)
+  var rformats = formats.slice(0).reverse();
+  var sortByVerExt = createFormatsSorter(rformats);
+  var reVer = new RegExp('^' + ver + '\\b');
+
+  function selectMatches(rel) {
+    if (os) {
+      if (rel.os !== '*') {
+        if (rel.os !== os) {
+          return false;
+        }
+      }
+    }
+
+    if (arch) {
+      if (rel.arch !== '*') {
+        if (rel.arch !== arch) {
+          return false;
+        }
+      }
+    }
+
+    if (rel.libc !== 'none') {
+      let releaseRequiresMusl = rel.libc === 'musl';
+      // goal: handle non-glibc (Alpine / Docker / musl)
+      let osHasMusl = libc === 'musl';
+      if (osHasMusl) {
+        // goal: fail if dependent on libc
+        let releaseRequiresLibc = rel.libc === 'gnu';
+        if (releaseRequiresLibc) {
+          return false;
+        }
+      } else if (releaseRequiresMusl) {
+        // goal: don't use musl++ on glibc (Ubuntu, GNU, etc)
+        return false;
+      }
+    }
+
+    if (lts) {
+      if (!rel.lts) {
+        return false;
+      }
+    }
+
+    if (channel) {
+      if (rel.channel !== channel) {
+        return false;
+      }
+    }
+
+    // to match 'tar.gz' and 'tar.xz' with just 'tar'
+    function hasExt(ext) {
+      return rel.ext.match(ext);
+    }
+    if (formats.length) {
+      if (!formats.some(hasExt)) {
+        return false;
+      }
+    }
+
+    if (ver) {
+      if (!rel.version.match(reVer)) {
+        return false;
+      }
+    }
+
+    return true;
+  }
+
+  var sortedRels = all.releases.filter(selectMatches).sort(sortByVerExt);
+  //console.log(sortedRels.slice(0, 4));
+
+  return sortedRels.slice(0, limit || 1000);
+}
+
+Releases.getReleases = function ({
+  _count,
+  pkg,
+  ver,
+  os,
+  arch,
+  libc,
+  lts,
+  channel,
+  formats,
+  limit,
+}) {
+  if (!_count) {
+    _count = 0;
+  }
+  return getCachedReleases(pkg).then(function (all) {
+    return filterReleases(all, {
+      ver,
+      os,
+      arch,
+      libc,
+      lts,
+      channel,
+      formats,
+      limit,
+    })
+      .catch(function (err) {
+        if ('MODULE_NOT_FOUND' === err.code) {
+          return null;
+        }
+        console.error(
+          'TODO: lib/release.js: check type of error, such as MODULE_NOT_FOUND',
+        );
+        console.error(err);
+      })
+      .then(function (releases) {
+        if (releases.length) {
+          return {
+            oses: all.oses,
+            arches: all.arches,
+            libcs: all.libcs,
+            formats: all.formats,
+            releases: releases,
+          };
+        }
+        if (_count < 1) {
+          // Apple Silicon M1 hacky-do workaround fix
+          if ('macos' === os && 'arm64' === arch) {
+            return Releases.getReleases({
+              pkg,
+              ver,
+              os,
+              arch: 'amd64',
+              libc,
+              lts,
+              channel,
+              formats,
+              limit,
+            });
+          }
+          // Windows ARM hacky-do workaround fix
+          if ('windows' === os && 'arm64' === arch) {
+            return Releases.getReleases({
+              pkg,
+              ver,
+              os,
+              arch: 'amd64',
+              libc,
+              lts,
+              channel,
+              formats,
+              limit,
+            });
+          }
+          // Raspberry Pi 3+ on Ubuntu arm64 (via Bionic?)
+          if ('linux' === os && 'arm64' === arch) {
+            return Releases.getReleases({
+              _count: _count + 1,
+              pkg,
+              ver,
+              os,
+              arch: 'armv7l',
+              libc,
+              lts,
+              channel,
+              formats,
+              limit,
+            });
+          }
+          // armv7 can run armv6
+          if ('linux' === os && 'armv7l' === arch) {
+            return Releases.getReleases({
+              _count: _count + 1,
+              pkg,
+              ver,
+              os,
+              arch: 'armv6l',
+              libc,
+              lts,
+              channel,
+              formats,
+              limit,
+            });
+          }
+        }
+        if (_count < 2) {
+          // Raspberry Pi 3+ on Raspbian arm7 (not Ubuntu arm64)
+          // hail mary
+          if ('linux' === os && 'armv7l' === arch) {
+            return Releases.getReleases({
+              _count: _count + 1,
+              pkg,
+              ver,
+              os,
+              arch: 'arm64',
+              libc,
+              lts,
+              channel,
+              formats,
+              limit,
+            });
+          }
+        }
+        releases = [
+          {
+            name: 'doesntexist.ext',
+            version: '0.0.0',
+            lts: '-',
+            channel: 'error',
+            date: '1970-01-01',
+            os: os || '-',
+            arch: arch || '-',
+            libc: libc || '-',
+            ext: 'err',
+            download: 'https://example.com/doesntexist.ext',
+            comment:
+              'No matches found. Could be bad or missing version info' +
+              ',' +
+              "Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
+          },
+        ];
+        return {
+          oses: all.oses,
+          arches: all.arches,
+          libcs: all.libcs,
+          formats: all.formats,
+          releases: releases,
+        };
+      });
+  });
+};
+
+if (require.main === module) {
+  return module
+    .exports({
+      pkg: 'node',
+      ver: '',
+      os: 'macos',
+      arch: 'amd64',
+      lts: true,
+      libc: 'libc',
+      channel: 'stable',
+      formats: ['tar', 'exe', 'zip', 'xz', 'dmg', 'pkg'],
+      limit: 10,
+    })
+    .then(function (all) {
+      console.info(JSON.stringify(all));
+    });
+}

_webi/ua-detect.js

@@ -1,5 +1,7 @@
 'use strict';
 
+var uaDetect = module.exports;
+
 function getRequest(req) {
   var ua = req.headers['user-agent'] || '';
   var os = req.query.os;
@@ -14,11 +16,11 @@ function getRequest(req) {
   }
 
   return {
-    unix: 'curl -fsSA "$(uname -a)" ' + url,
+    unix: 'curl -fsSA "$(uname -srm)" ' + url,
     windows: 'curl.exe -fsSA "MS $Env:PROCESSOR_ARCHITECTURE" ' + url,
     ua: ua,
     os: uaDetect.os(ua),
-    arch: uaDetect.arch(ua)
+    arch: uaDetect.arch(ua),
   };
 }
 
@@ -54,16 +56,23 @@ function getArch(ua) {
     return '-';
   }
 
-  // quick hack for Apple Silicon M1
+  // Quick hack for Apple Silicon M1
+  //
+  // Note: we now use `uname -srm` which does not have the native arch
+  // info included with `uname -v` and `uname -a`.
+  //
   // Native:  Darwin boomer.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec  2 20:40:21 PST 2020; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 arm64
   // Resetta: Darwin boomer.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec  2 20:40:21 PST 2020; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 x86_64
-  ua = ua.replace(/xnu-.*RELEASE_[^\s]*/, '')
+  ua = ua.replace(/xnu-.*RELEASE_[^\s]*/, '');
+
   if (/aarch64|arm64|arm8|armv8/i.test(ua)) {
     return 'arm64';
-  } else if (/aarch|arm7|armv7/i.test(ua)) {
+  } else if (/aarch|arm7|armv7|arm32/i.test(ua)) {
     return 'armv7l';
-  } else if (/arm6|armv6/i.test(ua)) {
+  } else if (/arm6|armv6|arm(\b|_)/i.test(ua)) {
     return 'armv6l';
+  } else if (/ppc64le/i.test(ua)) {
+    return 'ppc64le';
   } else if (/ppc64/i.test(ua)) {
     return 'ppc64';
   } else if (/mips64/i.test(ua)) {
@@ -82,7 +91,63 @@ function getArch(ua) {
   }
 }
 
-var uaDetect = module.exports;
+function getLibc(ua) {
+  if ('-' === ua) {
+    return '-';
+  }
+
+  // How to see a bunch of target host quadruples:
+  //     go tool dist list
+  //     rustup target list
+  //     zig targets | jq -r '.libc[]' | sort -
+
+  // Generally, we prefer 'musl' builds because they DO work on glibc systems (Ubuntu),
+  // but 'glibc' builds will NOT work on musl systems (Alpine / Docker).
+  //
+  // However, there are a few instances (ex: Node.js), where the 'musl' builds
+  // DO NOT work on glibc systems.
+
+  {
+    let muslRe = /(\b|_)(musl)(\b|_)/i;
+    if (muslRe.test(ua)) {
+      return 'musl';
+    }
+  }
+
+  {
+    let msvcRe = /(\b|_)(msvc|windows|microsoft)(\b|_)/i;
+    if (msvcRe.test(ua)) {
+      return 'msvc';
+    }
+  }
+
+  {
+    let gnuRe = /(\b|_)(gnu|glibc|linux)(\b|_)/i;
+    if (gnuRe.test(ua)) {
+      return 'gnu';
+    }
+  }
+
+  {
+    let libcRe = /(\b|_)(libc)(\b|_)/i;
+    if (libcRe.test(ua)) {
+      return 'libc';
+    }
+  }
+
+  {
+    let darwinRe = /(\b|_)(darwin)(\b|_)/i;
+    if (darwinRe.test(ua)) {
+      // not sure whether this should be "darwin" or "libc" or "none"
+      // https://opensource.apple.com/source/Libc/
+      return 'libc';
+    }
+  }
+
+  return 'libc';
+}
+
 uaDetect.os = getOs;
 uaDetect.arch = getArch;
+uaDetect.libc = getLibc;
 uaDetect.request = getRequest;

_webi/webi-pwsh.ps1

@@ -1,78 +0,0 @@
-#!/usr/bin/env pwsh
-
-# this allows us to call ps1 files, which allows us to have spaces in filenames
-# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
-# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
-Set-ExecutionPolicy -Scope Process Bypass
-
-# If a command returns an error, halt the script.
-$ErrorActionPreference = 'Stop'
-
-# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
-$ProgressPreference = 'SilentlyContinue'
-
-# This is the canonical CPU arch when the process is emulated
-$my_arch = "$Env:PROCESSOR_ARCHITEW6432"
-IF ($my_arch -eq $null -or $my_arch -eq "") {
-  # This is the canonical CPU arch when the process is native
-  $my_arch = "$Env:PROCESSOR_ARCHITECTURE"
-}
-# TODO API should know to prefer x86 for windows when arm binary is not available
-$Env:WEBI_UA = "Windows/10 $my_arch"
-$exename = $args[0]
-
-# Switch to userprofile
-pushd $Env:USERPROFILE
-
-# Make paths if needed
-New-Item -Path .local\bin -ItemType Directory -Force | out-null
-# TODO replace all xbin with opt\bin\
-New-Item -Path .local\xbin -ItemType Directory -Force | out-null
-
-# See note on Set-ExecutionPolicy above
-Set-Content -Path .local\bin\webi.bat -Value "@echo off`r`npushd %USERPROFILE%`r`npowershell -ExecutionPolicy Bypass .local\bin\webi-pwsh.ps1 %1`r`npopd"
-# Backwards-compat bugfix: remove old webi-pwsh.ps1 location
-Remove-Item -Path .local\bin\webi.ps1 -Recurse -ErrorAction Ignore
-if (!(Test-Path -Path .local\opt))
-{
-    New-Item -Path .local\opt -ItemType Directory -Force | out-null
-}
-# TODO windows version of mktemp -d
-if (!(Test-Path -Path .local\tmp))
-{
-    New-Item -Path .local\tmp -ItemType Directory -Force | out-null
-}
-
-# TODO SetStrictMode
-# TODO Test-Path variable:global:Env:WEBI_HOST ???
-IF($Env:WEBI_HOST -eq $null -or $Env:WEBI_HOST -eq "")
-{
-    $Env:WEBI_HOST = "https://webinstall.dev"
-}
-
-if (!(Test-Path -Path .local\bin\pathman.exe))
-{
-    & curl.exe -fsSL -A "$Env:WEBI_UA" "$Env:WEBI_HOST/packages/pathman/install.ps1" -o .\.local\tmp\pathman-setup.ps1
-    powershell .\.local\tmp\pathman-setup.ps1
-    # TODO del .\.local\tmp\pathman-setup.bat
-}
-
-# Run pathman to set up the folder
-# (using unix style path because... cmd vs powershell vs whatever)
-& "$Env:USERPROFILE\.local\bin\pathman.exe" add ~/.local/bin
-
-# {{ baseurl }}
-# {{ version }}
-
-# Fetch <whatever>.ps1
-# TODO detect formats
-$PKG_URL = "$Env:WEBI_HOST/api/installers/$exename.ps1?formats=zip,exe,tar"
-echo "Downloading $PKG_URL"
-# Invoke-WebRequest -UserAgent "Windows amd64" "$PKG_URL" -OutFile ".\.local\tmp\$exename.install.ps1"
-& curl.exe -fsSL -A "$Env:WEBI_UA" "$PKG_URL" -o .\.local\tmp\$exename.install.ps1
-
-# Run <whatever>.ps1
-powershell .\.local\tmp\$exename.install.ps1
-
-# Done
-popd

_webi/webi.bat

@@ -1,38 +0,0 @@
-@echo off
-pushd "%userprofile%" || goto :error
-  IF NOT EXIST .local (
-    mkdir .local || goto :error
-  )
-  IF NOT EXIST .local\bin (
-    mkdir .local\bin || goto :error
-  )
-  IF NOT EXIST .local\opt (
-    mkdir .local\opt || goto :error
-  )
-
-  pushd .local\bin || goto :error
-    if NOT EXIST pathman.exe (
-      echo updating PATH management
-      powershell $ProgressPreference = 'SilentlyContinue'; Invoke-WebRequest https://webinstall.dev/packages/pathman/install.bat -OutFile pathman-setup.bat || goto :error
-      call .\pathman-setup.bat || goto :error
-      del pathman-setup.bat  || goto :error
-      rem TODO there's rumor of a windows tool called 'pathman' that does the same thing?
-    )
-  popd || goto :error
-  .\.local\bin\pathman add ".local\bin" || goto :error
-
-  echo downloading and installing %1
-  powershell $ProgressPreference = 'SilentlyContinue'; Invoke-WebRequest https://webinstall.dev/packages/%1/install.ps1 -OutFile %1-webinstall.bat || goto :error
-
-  rem TODO only add if it's not in there already
-  PATH .local\bin;%PATH%
-
-  call %1-webinstall.bat || goto :error
-  del %1-webinstall.bat || goto :error
-popd
-
-goto :EOF
-
-:error
-echo Failed with error #%errorlevel%.
-exit /b %errorlevel%

_webi/webi.ps1.bat

@@ -1 +0,0 @@
-powershell -ExecutionPolicy Bypass .\.local\bin\webi-pwsh.ps1 %1

aliasman/README.md

@@ -0,0 +1,125 @@
+---
+title: aliasman
+homepage: https://github.com/BeyondCodeBootcamp/aliasman
+tagline: |
+  aliasman: A cross-shell (POSIX-compliant) alias manager for bash, zsh, and fish
+---
+
+To update or switch versions, run `webi aliasman@stable` (or `@v1.0.0`, `@beta`,
+etc).
+
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.config/envman/alias.env
+~/.local/bin/aliasman
+```
+
+## Cheat Sheet
+
+> `aliasman` helps you love your *alias*es again! 🥸 \
+> Set 'em once, use 'em everywhere! \
+> (and all in just one dotfile, for an on-the-go friendly config)
+
+```sh
+aliasman <alias-name> <command-name> [args, pipes, redirs, etc]
+```
+
+### What are Aliases?
+
+An _alias_ is just shorthand for a shell function.
+
+Take a long command like this:
+
+```sh
+git commit -m "feat: new feature"
+```
+
+And turn it into a short command, like this:
+
+```sh
+gc "feat: new feature"
+```
+
+(that would be `aliasman gc 'git commit -m'`)
+
+### Imagine the possibilities!
+
+1. What if you could quickly create a _command_, `ll`, \
+   that does the work of `ls -lAhF`!?
+2. Set an _alias_ to do just that!
+   ```sh
+   aliasman ll 'ls -lAhF'
+   ```
+3. Reload your alias config (or open a _new terminal_)
+   ```sh
+   source ~/.config/envman/alias.env
+   ```
+4. Use it!
+   ```sh
+   ll
+   ```
+   ```text
+   drwxr-xr-x aj wheel 416 B  Thu Feb  9 02:08:39 2023 📂 .git/
+   .rwxr-xr-x aj staff 6.2 KB Thu Feb  9 01:36:30 2023 💻 aliasman*
+   .rw-r--r-- aj wheel  16 KB Wed Feb  8 21:51:06 2023 🔑 LICENSE
+   .rw-r--r-- aj wheel 1.4 KB Thu Feb  9 01:47:13 2023 📄 README.md
+   ```
+
+### Common aliases
+
+Use *alias*es to make other tools you find around webi even _more_ convenient
+⚡️ (and powerful 💪).
+
+```sh
+aliasman curl 'curlie'
+
+aliasman diffy 'diff -y --suppress-common-lines'
+
+aliasman gc 'git commit -m'
+aliasman gri 'git rebase -i'
+
+aliasman la 'lsd -AF'
+aliasman ll 'lsd -lAhF'
+aliasman ls 'lsd -F'
+
+aliasman rgi 'rg -i'
+
+aliasman tree 'lsd -F --tree --group-dirs=last'
+
+# random password generator
+aliasman rnd 'xxd -l24 -ps /dev/urandom'
+```
+
+### How to replace an alias
+
+Just run the command again!
+
+```sh
+aliasman ll 'lsd -l'
+aliasman ll 'lsd -lAhF'
+```
+
+### How to delete an alias
+
+With `--delete`!
+
+```sh
+aliasman --delete ll
+```
+
+### How to see an alias
+
+Just supply the name!
+
+```sh
+aliasman rnd
+```
+
+```text
+alias rnd='xxd -l24 -ps /dev/urandom'
+```

aliasman/install.sh

@@ -0,0 +1,41 @@
+#!/bin/sh
+set -e
+set -u
+
+__init_aliasman() {
+
+    ######################
+    # Install aliasman #
+    ######################
+
+    # Every package should define these 6 variables
+    pkg_cmd_name="aliasman"
+
+    pkg_dst_cmd="$HOME/.local/bin/aliasman"
+    pkg_dst="$pkg_dst_cmd"
+
+    pkg_src_cmd="$HOME/.local/opt/aliasman-v$WEBI_VERSION/bin/aliasman"
+    pkg_src_dir="$HOME/.local/opt/aliasman-v$WEBI_VERSION"
+    pkg_src="$pkg_src_cmd"
+
+    # pkg_install must be defined by every package
+    pkg_install() {
+        # ~/.local/opt/aliasman-v1.0.0/bin
+        mkdir -p "$(dirname "$pkg_src_cmd")"
+
+        # mv ./*aliasman*/aliasman ~/.local/opt/aliasman-v1.0.0/bin/aliasman
+        mv ./*aliasman*/aliasman "$pkg_src_cmd"
+    }
+
+    # pkg_get_current_version is recommended, but (soon) not required
+    pkg_get_current_version() {
+        # 'aliasman version' has output in this format:
+        #       aliasman v1.0.0 (2023-01-15)
+        #       Copyright 2023 AJ ONeal
+        # This trims it down to just the version number:
+        #       1.0.0
+        aliasman version | head -n 1 | cut -d ' ' -f 2 | sed 's:^v::'
+    }
+}
+
+__init_aliasman

aliasman/releases.js

@@ -0,0 +1,22 @@
+'use strict';
+
+let Releases = module.exports;
+
+let GitHubSource = require('../_common/github-source.js');
+let owner = 'BeyondCodeBootcamp';
+let repo = 'aliasman';
+
+Releases.latest = async function () {
+  let all = await GitHubSource.getDistributables({ owner, repo });
+  for (let pkg of all.releases) {
+    pkg.os = 'posix_2017';
+  }
+  return all;
+};
+
+if (module === require.main) {
+  Releases.latest().then(function (all) {
+    all = require('../_webi/normalize.js')(all);
+    console.info(JSON.stringify(all, null, 2));
+  });
+}

arc/README.md

@@ -5,11 +5,17 @@ tagline: |
   Arc is a cross-platform, multi-format archive utility.
 ---
 
-## Updating `arc`
+To update or switch versions, run `webi arc@stable` (or `@v3.5`, `@beta`, etc).
 
-`webi arc@stable`
+### Files
 
-Use the `@beta` tag for pre-releases.
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.local/bin/arc
+```
 
 ## Cheat Sheet
 
@@ -22,42 +28,42 @@ create a top-level directory if one does not exist.
 
 ### List
 
-```txt
+```text
 # arc ls <archive file>
 arc ls   example.zip
 ```
 
 ### Unarchive (whole)
 
-```txt
+```text
 # arc unarchive <archive file>
 arc unarchive   example.zip
 ```
 
 ### Extract (partial)
 
-```txt
+```text
 # arc extract <archive file> <archived path> <extracted path>
 arc extract   example.zip    example/foo     ~/Downloads/foo
 ```
 
 ### Archive (recursive)
 
-```txt
+```text
 # arc archive <archive file> <files or folders ...>
 arc archive   example.zip    ./README.md ./bin ./src
 ```
 
 ### Compress (single file)
 
-```txt
+```text
 # arc compress <single file> <format>
 arc compress   ./example.tar xz
 ```
 
 ### Decompress (single file)
 
-```txt
+```text
 # arc decompress <archive file>
 arc decompress   ./example.tar.xz
 ```

arc/install.ps1

@@ -15,41 +15,39 @@ $pkg_src_bin = "$Env:USERPROFILE\.local\opt\archiver-v$Env:WEBI_VERSION\bin"
 $pkg_src_dir = "$Env:USERPROFILE\.local\opt\archiver-v$Env:WEBI_VERSION"
 $pkg_src = "$pkg_src_cmd"
 
-$pkg_download = "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"
+New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
+$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
 
 # Fetch archive
-IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"))
-{
-    # TODO: arch detection
-    echo "Downloading archiver from $Env:WEBI_PKG_URL to $pkg_download"
+IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
+    Write-Output "Downloading archiver from $Env:WEBI_PKG_URL to $pkg_download"
     & curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
-    & move "$pkg_download.part" "$pkg_download"
+    & Move-Item "$pkg_download.part" "$pkg_download"
 }
 
-IF (!(Test-Path -Path "$pkg_src_cmd"))
-{
-    echo "Installing archiver"
+IF (!(Test-Path -Path "$pkg_src_cmd")) {
+    Write-Output "Installing archiver"
 
     # TODO: create package-specific temp directory
     # Enter tmp
-    pushd .local\tmp
+    Push-Location .local\tmp
 
-    	# Remove any leftover tmp cruft
-        Remove-Item -Path ".\arc-*" -Recurse -ErrorAction Ignore
-        Remove-Item -Path ".\arc.exe" -Recurse -ErrorAction Ignore
+    # Remove any leftover tmp cruft
+    Remove-Item -Path ".\arc-*" -Recurse -ErrorAction Ignore
+    Remove-Item -Path ".\arc.exe" -Recurse -ErrorAction Ignore
 
-        # Move single binary into root of temporary folder
-        & move "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE" "arc.exe"
+    # Move single binary into root of temporary folder
+    & Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE" "arc.exe"
 
-        # Settle unpacked archive into place
-        echo "Install Location: $pkg_src_cmd"
-        New-Item "$pkg_src_bin" -ItemType Directory -Force | out-null
-        Move-Item -Path "arc.exe" -Destination "$pkg_src_bin"
+    # Settle unpacked archive into place
+    Write-Output "Install Location: $pkg_src_cmd"
+    New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
+    Move-Item -Path "arc.exe" -Destination "$pkg_src_bin"
 
     # Exit tmp
-    popd
+    Pop-Location
 }
 
-echo "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
-Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | out-null
+Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
+Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
 Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

arc/install.sh

@@ -1,6 +1,8 @@
-{
-    set -e
-    set -u
+#!/bin/sh
+set -e
+set -u
+
+__init_arc() {
 
     ####################
     # Install archiver #
@@ -19,7 +21,7 @@
     # pkg_install must be defined by every package
     pkg_install() {
         # ~/.local/opt/arc-v3.2.0/bin
-        mkdir -p "$(dirname $pkg_src_cmd)"
+        mkdir -p "$(dirname "$pkg_src_cmd")"
 
         # mv ./arc_* ~/.local/opt/arc-v3.2.0/bin/arc
         mv ./arc_* "$pkg_src_cmd"
@@ -31,6 +33,8 @@
         #       arc v3.5.0 (25e050d) 2020-10-30T03:27:58Z
         # This trims it down to just the version number:
         #       3.5.0
-        echo "$(arc version 2> /dev/null | head -n 1 | cut -d' ' -f2 | sed 's:^v::')"
+        arc version 2> /dev/null | head -n 1 | cut -d' ' -f2 | sed 's:^v::'
     }
 }
+
+__init_arc

arc/releases.js

@@ -4,14 +4,15 @@ var github = require('../_common/github.js');
 var owner = 'mholt';
 var repo = 'archiver';
 
-module.exports = function (request) {
-  return github(request, owner, repo).then(function (all) {
+module.exports = function () {
+  return github(null, owner, repo).then(function (all) {
+    all._names = ['archiver', 'arc'];
     return all;
   });
 };
 
 if (module === require.main) {
-  module.exports(require('@root/request')).then(function (all) {
+  module.exports().then(function (all) {
     all = require('../_webi/normalize.js')(all);
     // just select the first 5 for demonstration
     all.releases = all.releases.slice(0, 5);

archiver/README.md

@@ -0,0 +1,11 @@
+---
+title: Archiver (arc alias)
+homepage: https://webinstall.dev/arc
+tagline: |
+  Alias for https://webinstall.dev/arc
+alias: arc
+description: |
+  See https://webinstall.dev/arc
+---
+
+Alias for https://webinstall.dev/arc

archiver/install.ps1

@@ -1,5 +1,5 @@
 #!/bin/pwsh
 
-echo "'archiver@$Env:WEBI_TAG' is an alias for 'arc@$Env:WEBI_VERSION'"
-IF ($Env:WEBI_HOST -eq $null -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
-curl.exe -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell
+Write-Output "'archiver@$Env:WEBI_TAG' is an alias for 'arc@$Env:WEBI_VERSION'"
+IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
+curl.exe -A MS -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell

archiver/install.sh

@@ -1,10 +1,11 @@
-# title: Archiver (arc alias)
-# homepage: https://webinstall.dev/arc
-# tagline: Alias for https://webinstall.dev/arc
-# alias: arc
-# description: |
-#   See https://webinstall.dev/arc
+#!/bin/sh
+set -e
+set -u
 
-echo "'archiver@${WEBI_TAG:-stable}' is an alias for 'arc@${WEBI_VERSION:-}'"
-WEBI_HOST=${WEBI_HOST:-"https://webinstall.dev"}
-curl -fsSL "$WEBI_HOST/arc@${WEBI_VERSION:-}" | bash
+__redirect_alias_arc() {
+    echo "'archiver@${WEBI_TAG:-stable}' is an alias for 'arc@${WEBI_VERSION-}'"
+    WEBI_HOST=${WEBI_HOST:-"https://webinstall.dev"}
+    curl -fsSL "$WEBI_HOST/arc@${WEBI_VERSION-}" | sh
+}
+
+__redirect_alias_arc

atomicparsley/README.md

@@ -0,0 +1,538 @@
+---
+title: atomicparsley
+homepage: https://github.com/wez/atomicparsley
+tagline: |
+  AtomicParsley is a lightweight tool for reading, parsing and setting iTunes-style metadata.
+---
+
+To update or switch versions, run `webi atomicparsley@stable` (or `@v20221229`,
+`@beta`, etc).
+
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.local/bin/AtomicParsley
+```
+
+**Windows Users**
+
+```text
+\Windows\System32\vcruntime140.dll
+```
+
+This will also attempt to install the
+[Microsoft Visual C++ Redistributable](/vcruntime) via `webi vcruntime`. If it
+fails and you get the error _`vcruntime140.dll` was not found_, you'll need to
+[install it manually](https://learn.microsoft.com/en-US/cpp/windows/latest-supported-vc-redist?view=msvc-170).
+
+## Cheat Sheet
+
+> `AtomicParsley` is an alternative to `ffmpeg` and `ffprobe` for viewing and
+> changing metadata in MPEG-4 files with .3gp, .3g2, .mp4, .m4a, .m4b, .m4p,
+> .m4r, and .m4v extensions. Because it's purpose-built for MP4 containers, it
+> can do some things `ffmpeg` can't.
+
+### How to Change Album Art
+
+1. Save any existing cover art
+   ```sh
+   AtomicParsley ./my-song.m4a --extractPixToPath ./ 'my-song'
+   ```
+2. Remove the artwork
+   ```sh
+   AtomicParsley ./my-song.m4a --artwork REMOVE_ALL
+   ```
+3. **(macOS ONLY)** Add new artwork at the specified DPI (and other constraints)
+
+   ```sh
+   export PIC_OPTIONS="DPI=72"
+
+   AtomicParsley ./my-podcast-audiobook.m4b --artwork ./my-season-1-cover.jpg
+   ```
+
+Only JPEG and PNG are supported. See `PIC_OPTIONS` down below for more options.
+If you get an error, try exporting your file with a DPI to 72 (or up to 300 with
+`PIC_OPTIONS` set) and a smaller resolution - perhaps 600x600 or 1500x1500 (what
+old versions iTunes versions used).
+
+### How to Remove Apple Account Info
+
+```sh
+AtomicParsley ./my-file.m4a \
+    --DeepScan \
+    --manualAtomRemove "moov.trak.mdia.minf.stbl.mp4a.pinf" \
+    --manualAtomRemove "moov.udta.meta.ilst.apID" \
+    --manualAtomRemove "moov.udta.meta.ilst.cnID" \
+    --manualAtomRemove "moov.udta.meta.ilst.purd" \
+    --manualAtomRemove "moov.udta.meta.ilst.sfID" \
+    --manualAtomRemove "moov.udta.meta.ilst.soal" \
+    --manualAtomRemove "moov.udta.meta.ilst.xid"
+```
+
+If you wanted to also remove information that indicates which Country or
+Language the song was purchased in, or which album it was purchased from, there
+are some additional IDs to consider:
+
+| Metadata Tag                             | Description                                  |
+| ---------------------------------------- | -------------------------------------------- |
+| moov.udta.meta.ilst.apID                 | Apple account email address                  |
+| moov.udta.meta.ilst.ownr                 | Apple account username                       |
+| moov.udta.meta.ilst.atID                 | Artist-track ID                              |
+| moov.udta.meta.ilst.cnID                 | iTunes Catalog ID                            |
+| moov.udta.meta.ilst.geID                 | Genre ID                                     |
+| moov.udta.meta.ilst.plID                 | Playlist ID (identifies album)               |
+| moov.udta.meta.ilst.sfID                 | iTunes store identifier (location/number)    |
+| moov.udta.meta.ilst.cprt                 | Copyright information                        |
+| moov.udta.meta.ilst.flvr                 | Bitrate/video size related                   |
+| moov.udta.meta.ilst.purd                 | Date purchased                               |
+| moov.udta.meta.ilst.rtng                 | Explicit/Clean information                   |
+| moov.udta.meta.ilst.soal                 | Album sort name                              |
+| moov.udta.meta.ilst.stik                 | Media type information                       |
+| moov.udta.meta.ilst.xid                  | Vendor xID                                   |
+| moov.udta.meta.ilst.----.name:[iTunMOVI] | Embedded plist contains filesize and flavor. |
+| moov.trak.mdia.minf.stbl.stsd.mp4a.pinf  | Purchase information related                 |
+
+See <https://gist.github.com/riophae/f5694fd2952cb64982689b971ca6ec79>.
+
+### Genre Lists
+
+All values are **case sensitive**.
+
+#### The "stik" List
+
+These values are **case sensitive**:
+
+0. `Home Video`
+1. `Normal` (meaning music)
+2. `Audiobook` (changes extension to .m4b)
+3. `Whacked Bookmark`
+4. `Music Video`
+5. `Movie`
+6. `Short Film`
+7. `TV Show`
+8. `Booklet`
+
+#### Standard Music Genres
+
+```txt
+(1.)  Blues
+(2.)  Classic Rock
+(3.)  Country
+(4.)  Dance
+(5.)  Disco
+(6.)  Funk
+(7.)  Grunge
+(8.)  Hip-Hop
+(9.)  Jazz
+(10.)  Metal
+(11.)  New Age
+(12.)  Oldies
+(13.)  Other
+(14.)  Pop
+(15.)  R&B
+(16.)  Rap
+(17.)  Reggae
+(18.)  Rock
+(19.)  Techno
+(20.)  Industrial
+(21.)  Alternative
+(22.)  Ska
+(23.)  Death Metal
+(24.)  Pranks
+(25.)  Soundtrack
+(26.)  Euro-Techno
+(27.)  Ambient
+(28.)  Trip-Hop
+(29.)  Vocal
+(30.)  Jazz+Funk
+(31.)  Fusion
+(32.)  Trance
+(33.)  Classical
+(34.)  Instrumental
+(35.)  Acid
+(36.)  House
+(37.)  Game
+(38.)  Sound Clip
+(39.)  Gospel
+(40.)  Noise
+(41.)  AlternRock
+(42.)  Bass
+(43.)  Soul
+(44.)  Punk
+(45.)  Space
+(46.)  Meditative
+(47.)  Instrumental Pop
+(48.)  Instrumental Rock
+(49.)  Ethnic
+(50.)  Gothic
+(51.)  Darkwave
+(52.)  Techno-Industrial
+(53.)  Electronic
+(54.)  Pop-Folk
+(55.)  Eurodance
+(56.)  Dream
+(57.)  Southern Rock
+(58.)  Comedy
+(59.)  Cult
+(60.)  Gangsta
+(61.)  Top 40
+(62.)  Christian Rap
+(63.)  Pop/Funk
+(64.)  Jungle
+(65.)  Native American
+(66.)  Cabaret
+(67.)  New Wave
+(68.)  Psychadelic
+(69.)  Rave
+(70.)  Showtunes
+(71.)  Trailer
+(72.)  Lo-Fi
+(73.)  Tribal
+(74.)  Acid Punk
+(75.)  Acid Jazz
+(76.)  Polka
+(77.)  Retro
+(78.)  Musical
+(79.)  Rock & Roll
+(80.)  Hard Rock
+(81.)  Folk
+(82.)  Folk/Rock
+(83.)  National Folk
+(84.)  Swing
+(85.)  Fast Fusion
+(86.)  Bebob
+(87.)  Latin
+(88.)  Revival
+(89.)  Celtic
+(90.)  Bluegrass
+(91.)  Avantgarde
+(92.)  Gothic Rock
+(93.)  Progressive Rock
+(94.)  Psychedelic Rock
+(95.)  Symphonic Rock
+(96.)  Slow Rock
+(97.)  Big Band
+(98.)  Chorus
+(99.)  Easy Listening
+(100.)  Acoustic
+(101.)  Humour
+(102.)  Speech
+(103.)  Chanson
+(104.)  Opera
+(105.)  Chamber Music
+(106.)  Sonata
+(107.)  Symphony
+(108.)  Booty Bass
+(109.)  Primus
+(110.)  Porn Groove
+(111.)  Satire
+(112.)  Slow Jam
+(113.)  Club
+(114.)  Tango
+(115.)  Samba
+(116.)  Folklore
+(117.)  Ballad
+(118.)  Power Ballad
+(119.)  Rhythmic Soul
+(120.)  Freestyle
+(121.)  Duet
+(122.)  Punk Rock
+(123.)  Drum Solo
+(124.)  A Capella
+(125.)  Euro-House
+(126.)  Dance Hall
+```
+
+#### iTunes Movie Genre IDs
+
+```sh
+AtomicParsley --genre-movie-id-list
+```
+
+```text
+(4401) Action & Adventure
+(4402) Anime
+(4403) Classics
+(4404) Comedy
+(4405) Documentary
+(4406) Drama
+(4407) Foreign
+(4408) Horror
+(4409) Independent
+(4410) Kids & Family
+(4411) Musicals
+(4412) Romance
+(4413) Sci-Fi & Fantasy
+(4414) Short Films
+(4415) Special Interest
+(4416) Thriller
+(4417) Sports
+(4418) Western
+(4419) Urban
+(4420) Holiday
+(4421) Made for TV
+(4422) Concert Films
+(4423) Music Documentaries
+(4424) Music Feature Films
+(4425) Japanese Cinema
+(4426) Jidaigeki
+(4427) Tokusatsu
+(4428) Korean Cinema
+```
+
+#### iTunes TV Genre IDs
+
+```sh
+AtomicParsley --genre-tv-id-list
+```
+
+```text
+(4000) Comedy
+(4001) Drama
+(4002) Animation
+(4003) Action & Adventure
+(4004) Classic
+(4005) Kids
+(4005) Nonfiction
+(4007) Reality TV
+(4008) Sci-Fi & Fantasy
+(4009) Sports
+(4010) Teens
+(4011) Latino TV
+```
+
+### Help
+
+Note: Normally we don't include full help text, but since AtomicParsley has a
+long history across various maintainers and repos, we feel it's appropriate to
+do so in this case.
+
+```text
+AtomicParsley sets metadata into MPEG-4 files & derivatives supporting 3 tag
+ schemes: iTunes-style, 3GPP assets & ISO defined copyright notifications.
+
+AtomicParsley quick help for setting iTunes-style metadata into MPEG-4 files.
+
+General usage examples:
+  AtomicParsley /path/to.mp4 -T 1
+  AtomicParsley /path/to.mp4 -t +
+  AtomicParsley /path/to.mp4 --artist "Me" --artwork /path/to/art.jpg
+  Atomicparsley /path/to.mp4 --albumArtist "You" --podcastFlag true
+  Atomicparsley /path/to.mp4 --stik "TV Show" --advisory explicit
+
+Getting information about the file & tags:
+  -T  --test        Test file for mpeg4-ishness & print atom tree
+  -t  --textdata    Prints tags embedded within the file
+  -E  --extractPix  Extracts pix to the same folder as the mpeg-4 file
+
+Setting iTunes-style metadata tags
+  --artist       (string)     Set the artist tag
+  --title        (string)     Set the title tag
+  --album        (string)     Set the album tag
+  --genre        (string)     Genre tag (see --longhelp for more info)
+  --tracknum     (num)[/tot]  Track number (or track number/total tracks)
+  --disk         (num)[/tot]  Disk number (or disk number/total disks)
+  --comment      (string)     Set the comment tag
+  --year         (num|UTC)    Year tag (see --longhelp for "Release Date")
+  --lyrics       (string)     Set lyrics (not subject to 256 byte limit)
+  --lyricsFile   (/path)      Set lyrics to the content of a file
+  --composer     (string)     Set the composer tag
+  --copyright    (string)     Set the copyright tag
+  --grouping     (string)     Set the grouping tag
+  --artwork      (/path)      Set a piece of artwork (jpeg or png only)
+  --bpm          (number)     Set the tempo/bpm
+  --albumArtist  (string)     Set the album artist tag
+  --compilation  (boolean)    Set the compilation flag (true or false)
+  --hdvideo      (number)     Set the hdvideo flag to one of:
+                              false or 0 for standard definition
+                              true or 1 for 720p
+                              2 for 1080p
+  --advisory     (string*)    Content advisory (*values: 'clean', 'explicit')
+  --stik         (string*)    Sets the iTunes "stik" atom (see --longhelp)
+  --description  (string)     Set the description tag
+  --longdesc     (string)     Set the long description tag
+  --storedesc    (string)     Set the store description tag
+  --TVNetwork    (string)     Set the TV Network name
+  --TVShowName   (string)     Set the TV Show name
+  --TVEpisode    (string)     Set the TV episode/production code
+  --TVSeasonNum  (number)     Set the TV Season number
+  --TVEpisodeNum (number)     Set the TV Episode number
+  --podcastFlag  (boolean)    Set the podcast flag (true or false)
+  --category     (string)     Sets the podcast category
+  --keyword      (string)     Sets the podcast keyword
+  --podcastURL   (URL)        Set the podcast feed URL
+  --podcastGUID  (URL)        Set the episode's URL tag
+  --purchaseDate (UTC)        Set time of purchase
+  --encodingTool (string)     Set the name of the encoder
+  --encodedBy    (string)     Set the name of the Person/company who encoded the file
+  --apID         (string)     Set the Account Name
+  --cnID         (number)     Set the iTunes Catalog ID (see --longhelp)
+  --geID         (number)     Set the iTunes Genre ID (see --longhelp)
+  --xID          (string)     Set the vendor-supplied iTunes xID (see --longhelp)
+  --gapless      (boolean)    Set the gapless playback flag
+  --contentRating (string*)   Set tv/mpaa rating (see -rDNS-help)
+
+Deleting tags
+  Set the value to "":        --artist "" --stik "" --bpm ""
+  To delete (all) artwork:    --artwork REMOVE_ALL
+  manually removal:           --manualAtomRemove "moov.udta.meta.ilst.ATOM"
+
+More detailed iTunes help is available with AtomicParsley --longhelp
+Setting reverse DNS forms for iTunes files: see --reverseDNS-help
+Setting 3gp assets into 3GPP & derivative files: see --3gp-help
+Setting copyright notices for all files: see --ISO-help
+For file-level options & padding info: see --file-help
+Setting custom private tag extensions: see --uuid-help
+Setting ID3 tags onto mpeg-4 files: see --ID3-help
+
+----------------------------------------------------------------------
+AtomicParsley version: 20221229.172126.0 d813aa6e0304ed3ab6d92f1ae96cd52b586181ec (utf8)
+
+Submit bug fixes to https://github.com/wez/atomicparsley
+```
+
+### `--longhelp`
+
+```text
+AtomicParsley help page for setting iTunes-style metadata into MPEG-4 files.
+              (3gp help available with AtomicParsley --3gp-help)
+          (ISO copyright help available with AtomicParsley --ISO-help)
+      (reverse DNS form help available with AtomicParsley --reverseDNS-help)
+Usage: AtomicParsley [mp4FILE]... [OPTION]... [ARGUMENT]... [ [OPTION2]...[ARGUMENT2]...]
+
+example: AtomicParsley /path/to.mp4 -e ~/Desktop/pix
+example: AtomicParsley /path/to.mp4 --podcastURL "http://www.url.net" --tracknum 45/356
+example: AtomicParsley /path/to.mp4 --copyright "℗ © 2006"
+example: AtomicParsley /path/to.mp4 --year "2006-07-27T14:00:43Z" --purchaseDate timestamp
+example: AtomicParsley /path/to.mp4 --sortOrder artist "Mighty Dub Cats, The
+------------------------------------------------------------------------------------------------
+  Extract any pictures in user data "covr" atoms to separate files.
+  --extractPix       ,  -E                     Extract to same folder (basename derived from file).
+  --extractPixToPath ,  -e  (/path/basename)   Extract to specific path (numbers added to basename).
+                                                 example: --e ~/Desktop/SomeText
+                                                 gives: SomeText_artwork_1.jpg  SomeText_artwork_2.png
+                                               Note: extension comes from embedded image file format
+------------------------------------------------------------------------------------------------
+ Tag setting options:
+
+  --artist           ,  -a   (str)    Set the artist tag: "moov.udta.meta.ilst.©ART.data"
+  --title            ,  -s   (str)    Set the title tag: "moov.udta.meta.ilst.©nam.data"
+  --album            ,  -b   (str)    Set the album tag: "moov.udta.meta.ilst.©alb.data"
+  --genre            ,  -g   (str)    Set the genre tag: "©gen" (custom) or "gnre" (standard).
+                                          see the standard list with "AtomicParsley --genre-list"
+  --tracknum         ,  -k   (num)[/tot]  Set the track number (or track number & total tracks).
+  --disk             ,  -d   (num)[/tot]  Set the disk number (or disk number & total disks).
+  --comment          ,  -c   (str)    Set the comment tag: "moov.udta.meta.ilst.©cmt.data"
+  --year             ,  -y   (num|UTC)    Set the year tag: "moov.udta.meta.ilst.©day.data"
+                                          set with UTC "2006-09-11T09:00:00Z" for Release Date
+  --lyrics           ,  -l   (str)    Set the lyrics tag: "moov.udta.meta.ilst.©lyr.data"
+  --lyricsFile       ,       (/path)  Set the lyrics tag to the content of a file
+  --composer         ,  -w   (str)    Set the composer tag: "moov.udta.meta.ilst.©wrt.data"
+  --copyright        ,  -x   (str)    Set the copyright tag: "moov.udta.meta.ilst.cprt.data"
+  --grouping         ,  -G   (str)    Set the grouping tag: "moov.udta.meta.ilst.©grp.data"
+  --artwork          ,  -A   (/path)  Set a piece of artwork (jpeg or png) on "covr.data"
+                                          Note: multiple pieces are allowed with more --artwork args
+  --bpm              ,  -B   (num)    Set the tempo/bpm tag: "moov.udta.meta.ilst.tmpo.data"
+  --albumArtist      ,  -A   (str)    Set the album artist tag: "moov.udta.meta.ilst.aART.data"
+  --compilation      ,  -C   (bool)   Sets the "cpil" atom (true or false to delete the atom)
+  --hdvideo          ,  -V   (bool)   Sets the "hdvd" atom (true or false to delete the atom)
+  --advisory         ,  -y   (1of3)   Sets the iTunes lyrics advisory ('remove', 'clean', 'explicit')
+  --stik             ,  -S   (1of7)   Sets the iTunes "stik" atom (--stik "remove" to delete)
+                                           "Movie", "Normal", "TV Show" .... others:
+                                           see the full list with "AtomicParsley --stik-list"
+                                           or set in an integer value with --stik value=(num)
+                                      Note: --stik Audiobook will change file extension to '.m4b'
+  --description      ,  -p   (str)    Sets the description on the "desc" atom
+  --Rating           ,       (str)    Sets the Rating on the "rate" atom
+  --longdesc         ,  -j   (str)    Sets the long description on the "ldes" atom
+  --storedesc        ,       (str)    Sets the iTunes store description on the "sdes" atom
+  --TVNetwork        ,  -n   (str)    Sets the TV Network name on the "tvnn" atom
+  --TVShowName       ,  -H   (str)    Sets the TV Show name on the "tvsh" atom
+  --TVEpisode        ,  -I   (str)    Sets the TV Episode on "tven":"209", but it is a string: "209 Part 1"
+  --TVSeasonNum      ,  -U   (num)    Sets the TV Season number on the "tvsn" atom
+  --TVEpisodeNum     ,  -N   (num)    Sets the TV Episode number on the "tves" atom
+  --podcastFlag      ,  -f   (bool)   Sets the podcast flag (values are "true" or "false")
+  --category         ,  -q   (str)    Sets the podcast category; typically a duplicate of its genre
+  --keyword          ,  -K   (str)    Sets the podcast keyword; invisible to MacOSX Spotlight
+  --podcastURL       ,  -L   (URL)    Set the podcast feed URL on the "purl" atom
+  --podcastGUID      ,  -J   (URL)    Set the episode's URL tag on the "egid" atom
+  --purchaseDate     ,  -D   (UTC)    Set Universal Coordinated Time of purchase on a "purd" atom
+                                       (use "timestamp" to set UTC to now; can be akin to id3v2 TDTG tag)
+  --encodingTool     ,       (str)    Set the name of the encoder on the "©too" atom
+  --encodedBy        ,       (str)    Set the name of the Person/company who encoded the file on the "©enc" atom
+  --apID             ,  -Y   (str)    Set the name of the Account Name on the "apID" atom
+  --cnID             ,       (num)    Set iTunes Catalog ID, used for combining SD and HD encodes in iTunes on the "cnID" atom
+
+                                      To combine you must set "hdvd" atom on one file and must have same "stik" on both file
+                                      Must not use "stik" of value Home Video(0), use Movie(9)
+
+                                      iTunes Catalog numbers can be obtained by finding the item in the iTunes Store.  Once item
+                                      is found in the iTunes Store right click on picture of item and select copy link.  Paste this link
+                                      into a document or web browser to display the catalog number ID.
+
+                                      An example link for the video Street Kings is:
+                                      http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewMovie?id=278743714&s=143441
+                                      Here you can see the cnID is 278743714
+
+                                      Alternatively you can use iMDB numbers, however these will not match the iTunes catalog.
+
+  --geID             ,       (num)    Set iTunes Genre ID.  This does not necessarily have to match genre.
+                                      See --genre-movie-id-list and --genre-tv-id-list
+
+  --xID              ,       (str)    Set iTunes vendor-supplied xID, used to allow iTunes LPs and iTunes Extras to interact
+                                            with other content in your iTunes Library
+  --gapless          ,       (bool)   Sets the gapless playback flag for a track in a gapless album
+  --sortOrder    (type)      (str)    Sets the sort order string for that type of tag.
+                                       (available types are: "name", "artist", "albumartist",
+                                        "album", "composer", "show")
+
+NOTE: Except for artwork, only 1 of each tag is allowed; artwork allows multiple pieces.
+NOTE: Tags that carry text(str) have a limit of 255 utf8 characters;
+however lyrics and long descriptions have no limit.
+------------------------------------------------------------------------------------------------
+ To delete a single atom, set the tag to null (except artwork):
+  --artist "" --lyrics ""
+  --artwork REMOVE_ALL
+  --metaEnema        ,  -P            Douches away every atom under "moov.udta.meta.ilst"
+  --foobar2000Enema  ,  -2            Eliminates foobar2000's non-compliant so-out-o-spec tagging scheme
+  --manualAtomRemove "some.atom.path" where some.atom.path can be:
+      keys to using manualAtomRemove:
+         ilst.ATOM.data or ilst.ATOM target an iTunes-style metadata tag
+         ATOM:lang=foo               target an atom with this language setting; like 3gp assets
+         ATOM.----.name:[foo]        target a reverseDNS metadata tag; like iTunNORM
+                                     Note: these atoms show up with 'AP -t' as: Atom "----" [foo]
+                                         'foo' is actually carried on the 'name' atom
+         ATOM[x]                     target an atom with an index other than 1; like trak[2]
+         ATOM.uuid=hex-hex-hex-hex   targt a uuid atom with the uuid of hex string representation
+    examples:
+        moov.udta.meta.ilst.----.name:[iTunNORM]      moov.trak[3].cprt:lang=urd
+        moov.trak[2].uuid=55534d54-21d2-4fce-bb88-695cfac9c740
+------------------------------------------------------------------------------------------------
+                   Environmental Variables (affecting picture placement) (macOS ONLY)
+
+  set PIC_OPTIONS in your shell to set these flags; preferences are separated by colons (:)
+
+ MaxDimensions=num (default: 0; unlimited); sets maximum pixel dimensions
+ DPI=num           (default: 72); sets dpi
+ MaxKBytes=num     (default: 0; unlimited);  maximum kilobytes for file (jpeg only)
+ AddBothPix=bool   (default: false); add original & converted pic (for archival purposes)
+ AllPixJPEG | AllPixPNG =bool (default: false); force conversion to a specific picture format
+ SquareUp          (include to square images to largest dimension, allows an [ugly] 160x1200->1200x1200)
+ removeTempPix     (include to delete temp pic files created when resizing images after tagging)
+ ForceHeight=num   (must also specify width, below) force image pixel height
+ ForceWidth=num    (must also specify height, above) force image pixel width
+
+ Examples: (bash-style)
+ export PIC_OPTIONS="MaxDimensions=400:DPI=72:MaxKBytes=100:AddBothPix=true:AllPixJPEG=true"
+ export PIC_OPTIONS="SquareUp:removeTempPix"
+ export PIC_OPTIONS="ForceHeight=999:ForceWidth=333:removeTempPix"
+------------------------------------------------------------------------------------------------
+```
+
+```
+
+```

atomicparsley/install.ps1

@@ -0,0 +1,60 @@
+#!/usr/bin/env pwsh
+
+#########################
+# Install AtomicParsley #
+#########################
+
+# Every package should define these variables
+$pkg_cmd_name = "AtomicParsley"
+
+$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\AtomicParsley.exe"
+$pkg_dst_bin = "$Env:USERPROFILE\.local\bin"
+$pkg_dst = "$pkg_dst_cmd"
+
+$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\AtomicParsley-v$Env:WEBI_VERSION\bin\AtomicParsley.exe"
+$pkg_src_bin = "$Env:USERPROFILE\.local\opt\AtomicParsley-v$Env:WEBI_VERSION\bin"
+$pkg_src_dir = "$Env:USERPROFILE\.local\opt\AtomicParsley-v$Env:WEBI_VERSION"
+$pkg_src = "$pkg_src_cmd"
+
+New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
+$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
+
+# Fetch archive
+IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
+    Write-Output "Checking for (or Installing) MSVC Runtime..."
+    & "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
+
+    Write-Output "Downloading AtomicParsley from $Env:WEBI_PKG_URL to $pkg_download"
+    & curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
+    & Move-Item "$pkg_download.part" "$pkg_download"
+}
+
+IF (!(Test-Path -Path "$pkg_src_cmd")) {
+    Write-Output "Installing AtomicParsley"
+
+    # TODO: create package-specific temp directory
+    # Enter tmp
+    Push-Location .local\tmp
+
+    # Remove any leftover tmp cruft
+    Remove-Item -Path ".\AtomicParsley-v*" -Recurse -ErrorAction Ignore
+    Remove-Item -Path ".\AtomicParsley.exe" -Recurse -ErrorAction Ignore
+
+    # Unpack archive file into this temporary directory
+    # Windows BSD-tar handles zip. Imagine that.
+    Write-Output "Unpacking $pkg_download"
+    & tar xf "$pkg_download"
+
+    # Settle unpacked archive into place
+    Write-Output "Install Location: $pkg_src_cmd"
+    New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
+    Move-Item -Path ".\AtomicParsley.exe" -Destination "$pkg_src_bin"
+
+    # Exit tmp
+    Pop-Location
+}
+
+Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
+Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
+New-Item "$pkg_dst_bin" -ItemType Directory -Force | Out-Null
+Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

atomicparsley/install.sh

@@ -0,0 +1,41 @@
+#!/bin/sh
+set -e
+set -u
+
+__init_atomicparsley() {
+
+    #########################
+    # Install AtomicParsley #
+    #########################
+
+    WEBI_SINGLE=true
+
+    # Every package should define these 6 variables
+    pkg_cmd_name="AtomicParsley"
+
+    pkg_dst_cmd="$HOME/.local/bin/AtomicParsley"
+    pkg_dst="$pkg_dst_cmd"
+
+    pkg_src_cmd="$HOME/.local/opt/AtomicParsley-v$WEBI_VERSION/bin/AtomicParsley"
+    pkg_src_dir="$HOME/.local/opt/AtomicParsley-v$WEBI_VERSION"
+    pkg_src="$pkg_src_cmd"
+
+    # pkg_install must be defined by every package
+    pkg_install() {
+        # ~/.local/opt/AtomicParsley-v20221229.172126.0/bin
+        mkdir -p "$(dirname "$pkg_src_cmd")"
+
+        # mv ./AtomicParsley ~/.local/opt/AtomicParsley-v20221229.172126.0/bin/AtomicParsley
+        mv ./"$pkg_cmd_name" "$pkg_src_cmd"
+    }
+
+    pkg_get_current_version() {
+        # 'AtomicParsley --version' has output in this format:
+        #       AtomicParsley version: 20221229.172126.0 d813aa6e0304ed3ab6d92f1ae96cd52b586181ec (utf8)
+        # This trims it down to just the version number:
+        #       20221229.172126.0
+        AtomicParsley --version 2> /dev/null | head -n 1 | cut -d' ' -f3
+    }
+}
+
+__init_atomicparsley

atomicparsley/releases.js

@@ -0,0 +1,79 @@
+'use strict';
+
+var github = require('../_common/github.js');
+var owner = 'wez';
+var repo = 'atomicparsley';
+
+let targets = {
+  x86win: {
+    os: 'windows',
+    arch: 'x86',
+    libc: 'msvc',
+  },
+  x64win: {
+    os: 'windows',
+    arch: 'amd64',
+    // https://github.com/wez/atomicparsley/issues/6#issuecomment-1364523028
+    libc: 'msvc',
+  },
+  x64mac: {
+    os: 'macos',
+    arch: 'amd64',
+  },
+  x64lin: {
+    os: 'linux',
+    arch: 'amd64',
+    libc: 'gnu',
+  },
+  x64musl: {
+    os: 'linux',
+    arch: 'amd64',
+    libc: 'musl',
+  },
+};
+
+module.exports = function () {
+  return github(null, owner, repo).then(function (all) {
+    for (let rel of all.releases) {
+      let windows32 = rel.name.includes('WindowsX86.');
+      if (windows32) {
+        Object.assign(rel, targets.x86win);
+        continue;
+      }
+
+      let windows64 = rel.name.includes('Windows.');
+      if (windows64) {
+        Object.assign(rel, targets.x64win);
+        continue;
+      }
+
+      let macos64 = rel.name.includes('MacOS');
+      if (macos64) {
+        Object.assign(rel, targets.x64mac);
+        continue;
+      }
+
+      let musl64 = rel.name.includes('Alpine');
+      if (musl64) {
+        Object.assign(rel, targets.x64musl);
+        continue;
+      }
+
+      let lin64 = rel.name.includes('Linux.');
+      if (lin64) {
+        Object.assign(rel, targets.x64lin);
+        continue;
+      }
+    }
+    all._names = ['AtomicParsley', 'atomicparsley'];
+    return all;
+  });
+};
+
+if (module === require.main) {
+  module.exports().then(function (all) {
+    all = require('../_webi/normalize.js')(all);
+    console.info(JSON.stringify(all));
+    //console.info(JSON.stringify(all, null, 2));
+  });
+}

awless/README.md

@@ -7,12 +7,23 @@ tagline: |
 
 To update or switch versions, run `webi awless@stable` (or `@v2`, `@beta`, etc).
 
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.local/bin/awless
+~/.aws/credentials
+```
+
 ## Cheat Sheet
 
 awless is modeled after popular command-line tools such as Git. Most commands
 are in the form of:
 
-```bash
+```sh
 awless verb [entity] [parameter=value ...]
 ```
 
@@ -26,13 +37,13 @@ Unlike the standard awscli tools, `awless` aims to be more human readable.
 
 For instance, let's list some resources:
 
-```bash
+```sh
 awless list vpcs
 ```
 
 Which outputs a friendly human readable table!
 
-```bash
+```sh
 |         ID ▲          | NAME | DEFAULT |   STATE   |     CIDR      |
 |-----------------------|------|---------|-----------|---------------|
 | vpc-00fd208a070000000 |      | false   | available | 172.16.0.0/16 |
@@ -42,13 +53,13 @@ Which outputs a friendly human readable table!
 There's also filter capabilities, in case the list is long. For example, let's
 list all EC2 instances with "api" in the name:
 
-```bash
+```sh
 awless list instances --filter name=api
 ```
 
 In addition to the default table output, there's also csv, tsv, json.
 
-```bash
+```sh
 awless list loadbalancers --format csv
 ```
 
@@ -56,7 +67,7 @@ awless list loadbalancers --format csv
 
 awless allows specifying things by name rather than ID by using the `@` prefix.
 
-```bash
+```sh
 awless create subnet cidr=10.0.0.0/24 vpc=@wordpress-vpc name=wordpress-public-subnet
 ```
 
@@ -65,13 +76,13 @@ awless create subnet cidr=10.0.0.0/24 vpc=@wordpress-vpc name=wordpress-public-s
 If you leave out a parameter, awless will prompt you for the missing
 information.
 
-```bash
+```sh
 awless delete i-123456789000abcd
 ```
 
 It will correctly detect what you were probably trying to do:
 
-```bash
+```sh
 Did you mean `awless delete instance ids=i-051fcef0537a53eb0` ?  [Y/n]
 ```
 

awless/install.ps1

@@ -16,22 +16,22 @@ $pkg_src_bin = "$Env:USERPROFILE\.local\opt\awless-v$Env:WEBI_VERSION\bin"
 $pkg_src_dir = "$Env:USERPROFILE\.local\opt\awless-v$Env:WEBI_VERSION"
 $pkg_src = "$pkg_src_cmd"
 
-$pkg_download = "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"
+New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
+$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
 
 # Fetch archive
-IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE")) {
-    # TODO: arch detection
-    echo "Downloading awless from $Env:WEBI_PKG_URL to $pkg_download"
+IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
+    Write-Output "Downloading awless from $Env:WEBI_PKG_URL to $pkg_download"
     & curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
-    & move "$pkg_download.part" "$pkg_download"
+    & Move-Item "$pkg_download.part" "$pkg_download"
 }
 
 IF (!(Test-Path -Path "$pkg_src_cmd")) {
-    echo "Installing awless"
+    Write-Output "Installing awless"
 
     # TODO: create package-specific temp directory
     # Enter tmp
-    pushd .local\tmp
+    Push-Location .local\tmp
 
     # Remove any leftover tmp cruft
     Remove-Item -Path ".\awless-v*" -Recurse -ErrorAction Ignore
@@ -39,19 +39,19 @@ IF (!(Test-Path -Path "$pkg_src_cmd")) {
 
     # Unpack archive file into this temporary directory
     # Windows BSD-tar handles zip. Imagine that.
-    echo "Unpacking $pkg_download"
+    Write-Output "Unpacking $pkg_download"
     & tar xf "$pkg_download"
 
     # Settle unpacked archive into place
-    echo "Install Location: $pkg_src_cmd"
-    New-Item "$pkg_src_bin" -ItemType Directory -Force | out-null
+    Write-Output "Install Location: $pkg_src_cmd"
+    New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
     Move-Item -Path ".\awless.exe" -Destination "$pkg_src_bin"
 
     # Exit tmp
-    popd
+    Pop-Location
 }
 
-echo "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
-Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | out-null
-New-Item "$pkg_dst_bin" -ItemType Directory -Force | out-null
+Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
+Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
+New-Item "$pkg_dst_bin" -ItemType Directory -Force | Out-Null
 Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

awless/install.sh

@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
 
 # "This is too simple" you say! "Where is the magic!?" you ask.
 # There is no magic!
@@ -19,7 +19,7 @@ pkg_get_current_version() {
     #       v2.1.0 h1:pQSaIJGFluFvu8KDGDODV8u4/QRED/OPyIR+MWYYse8=
     # This trims it down to just the version number:
     #       2.1.0
-    echo "$(awless version 2> /dev/null | head -n 1 | cut -d' ' -f1 | sed 's:^v::')"
+    awless version 2> /dev/null | head -n 1 | cut -d' ' -f1 | sed 's:^v::'
 }
 
 pkg_install() {

awless/releases.js

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
 var owner = 'wallix';
 var repo = 'awless';
 
-module.exports = function (request) {
-  return github(request, owner, repo).then(function (all) {
+module.exports = function () {
+  return github(null, owner, repo).then(function (all) {
     // remove checksums and .deb
     all.releases = all.releases.filter(function (rel) {
       return !/(\.txt)|(\.deb)$/i.test(rel.name);
@@ -15,7 +15,7 @@ module.exports = function (request) {
 };
 
 if (module === require.main) {
-  module.exports(require('@root/request')).then(function (all) {
+  module.exports().then(function (all) {
     all = require('../_webi/normalize.js')(all);
     console.info(JSON.stringify(all));
   });

bat/README.md

@@ -5,39 +5,56 @@ tagline: |
   bat: A cat(1) clone with syntax highlighting and Git integration.
 ---
 
-## Updating `bat`
+To update or switch versions, run `webi bat@stable` (or `@v0.18`, `@beta`, etc).
 
-```bash
-webi bat@stable
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.config/bat/config
+~/.local/opt/bat/
 ```
 
-Use the `@beta` tag for pre-releases.
+**Windows Users**
+
+```text
+\Windows\System32\vcruntime140.dll
+```
+
+This will also attempt to install the
+[Microsoft Visual C++ Redistributable](/vcruntime) via `webi vcruntime`. If it
+fails and you get the error _`vcruntime140.dll` was not found_, you'll need to
+[install it manually](https://learn.microsoft.com/en-US/cpp/windows/latest-supported-vc-redist?view=msvc-170).
 
 ## Cheat Sheet
 
 > `bat` is pretty much what `cat` would be if it were developed today's in the
 > world of Markdown, git, etc.
 
-### How to run on Windows 10
-
-On Windows 10 you'll get an error like this:
-
-> execution cannot proceed run because vcruntime140.dll was not found
-
-You need to download and install the
-[Microsoft Visual C++ Redistributable](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads)
-
 ### How to alias as `cat`
 
-Update your `.bashrc`, `.zshrc`, or `.profile`
+Use [aliasman](/aliasman):
 
-```bash
+```sh
+aliasman cat 'bat --style=plain'
+alias cat='bat --style=plain'
+```
+
+Or place this in `~/.config/envman/alias.env` and manually update your
+`.bashrc`, `.zshrc`, `.profile`, and/or `~/.config/fish/config.fish` to source
+it.
+
+```sh
 alias cat="bat --style=plain"
 ```
 
-For situations in which you must use `cat` exactly, remember that you can escape the alias:
+For situations in which you must use `cat` exactly, remember that you can escape
+the alias:
 
-```bash
+```sh
 \cat foo
 ```
 
@@ -45,13 +62,13 @@ For situations in which you must use `cat` exactly, remember that you can escape
 
 Take a look at the config options:
 
-```bash
+```sh
 bat --help
 ```
 
 Check to see where your config file is:
 
-```bash
+```sh
 echo 'N' | bat --generate-config-file
 ```
 
@@ -59,7 +76,7 @@ Edit the config file:
 
 `~/.config/bat/config`:
 
-```txt
+```text
 # no numbers or headers, just highlighting and such
 --style="plain"
 ```

bat/install.ps1

@@ -3,42 +3,45 @@
 $VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
 $EXENAME = "$Env:PKG_NAME.exe"
 # Fetch archive
-IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"))
-{
-    # TODO: arch detection
-    echo "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"
-    & curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE.part"
-    & move "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"
+IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
+    Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
+    & curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
+    & Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
 }
 
-IF (!(Test-Path -Path "$Env:USERPROFILE\.local\xbin\$VERNAME"))
-{
-    echo "Installing $Env:PKG_NAME"
+# Fetch MSVC Runtime
+Write-Output "Checking for MSVC Runtime..."
+IF (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
+    & "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
+}
+
+IF (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
+    Write-Output "Installing $Env:PKG_NAME"
     # TODO: temp directory
 
     # Enter tmp
-    pushd .local\tmp
+    Push-Location .local\tmp
 
-        # Remove any leftover tmp cruft
-        Remove-Item -Path "$Env:PKG_NAME-v*" -Recurse -ErrorAction Ignore
+    # Remove any leftover tmp cruft
+    Remove-Item -Path "$Env:PKG_NAME-v*" -Recurse -ErrorAction Ignore
 
-        # Unpack archive
-        # Windows BSD-tar handles zip. Imagine that.
-        echo "Unpacking $Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"
-        & tar xf "$Env:USERPROFILE\Downloads\$Env:WEBI_PKG_FILE"
-        # Move single binary into root of temporary folder
-        & move "bat-*\$EXENAME" "$VERNAME"
+    # Unpack archive
+    # Windows BSD-tar handles zip. Imagine that.
+    Write-Output "Unpacking $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
+    & tar xf "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
+    # Move single binary into root of temporary folder
+    & Move-Item "bat-*\$EXENAME" "$VERNAME"
 
 
-        # Settle unpacked archive into place
-        echo "New Name: $VERNAME"
-        echo "New Location: $Env:USERPROFILE\.local\xbin\$VERNAME"
-        Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\xbin"
+    # Settle unpacked archive into place
+    Write-Output "New Name: $VERNAME"
+    Write-Output "New Location: $Env:USERPROFILE\.local\bin\$VERNAME"
+    Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\bin"
 
     # Exit tmp
-    popd
+    Pop-Location
 }
 
-echo "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\xbin\$VERNAME'"
+Write-Output "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\bin\$VERNAME'"
 Remove-Item -Path "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse -ErrorAction Ignore
-Copy-Item -Path "$Env:USERPROFILE\.local\xbin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse
+Copy-Item -Path "$Env:USERPROFILE\.local\bin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse

bat/install.sh

@@ -1,8 +1,8 @@
-#!/bin/bash
+#!/bin/sh
+set -e
+set -u
 
-{
-    set -e
-    set -u
+__init_bat() {
 
     ###############
     # Install bat #
@@ -15,11 +15,11 @@
         #       bat 0.15.4
         # This trims it down to just the version number:
         #       0.15.4
-        echo $(bat --version 2> /dev/null | head -n 1 | cut -d' ' -f 2)
+        bat --version 2> /dev/null | head -n 1 | cut -d' ' -f 2
     }
 
     pkg_install() {
-        # ~/.local/xbin
+        # ~/.local/bin
         mkdir -p "$pkg_src_bin"
 
         # mv ./bat-*/bat ~/.local/opt/bat-v0.15.4/bin/bat
@@ -27,5 +27,12 @@
 
         # chmod a+x ~/.local/opt/bat-v0.15.4/bin/bat
         chmod a+x "$pkg_src_cmd"
+
+        if ! [ -e ~/.config/bat/config ]; then
+            mkdir -p ~/.config/bat/
+            touch ~/.config/bat/config
+        fi
     }
 }
+
+__init_bat

bat/releases.js

@@ -4,14 +4,14 @@ var github = require('../_common/github.js');
 var owner = 'sharkdp';
 var repo = 'bat';
 
-module.exports = function (request) {
-  return github(request, owner, repo).then(function (all) {
+module.exports = function () {
+  return github(null, owner, repo).then(function (all) {
     return all;
   });
 };
 
 if (module === require.main) {
-  module.exports(require('@root/request')).then(function (all) {
+  module.exports().then(function (all) {
     all = require('../_webi/normalize.js')(all);
     all.releases = all.releases.slice(0, 10);
     //console.info(JSON.stringify(all));

beyond-shell/README.md

@@ -0,0 +1,83 @@
+---
+title: beyond-shell
+homepage: https://webinstall.dev/beyond-shell
+tagline: |
+  meta package for Beyond Code workshops
+---
+
+To update, run the relevant installers individually. For example:
+`webi node@lts`.
+
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/bin
+~/.config/envman/alias.env
+~/.config/envman/PATH.env
+~/.iterm2/
+~/.local/bin/
+~/.local/opt/
+~/.local/share/font/ (or ~/Library/Fonts)
+~/.vim/pack/plugins/start/
+~/.vim/plugins/
+~/.vimrc
+```
+
+## Cheat Sheet
+
+> Installs the format and linter tools and code you'll need for the Beyond Code
+> Shell Scripting Workshop
+
+This meta package will install the full set of plugins and settings we
+recommended.
+
+## Prerequisites for Windows
+
+- https://webinstall.dev/wsl
+- https://webinstall.dev/nerdfont
+
+## Post Install for macOS
+
+- https://webinstall.dev/iterm2-themes
+- https://webinstall.dev/nerdfont
+
+## What's installed?
+
+- Formatters & Linters
+  - [`prettier`](/prettier)
+  - [`shellcheck`](/shellcheck)
+  - [`shfmt`](/shfmt)
+- Vim Plugins & Config
+  - [vim-leader](/vim-leader)
+  - [vim-shell](/vim-shell)
+  - [vim-sensible](/vim-sensible)
+  - [vim-viminfo](/vim-viminfo)
+  - [vim-lastplace](/vim-lastplace)
+  - [vim-smartcase](/vim-smartcase)
+  - [vim-spell](/vim-spell)
+  - [vim-ale](/vim-ale)
+  - [vim-prettier](/vim-prettier)
+  - [vim-shfmt](/vim-shfmt)
+  - [vim-whitespace](/vim-whitespace)
+- Commandline Tools
+  - [`aliasman`](/aliasman)
+  - [`bat`](/bat)
+  - [`curlie`](/curlie)
+  - [`jq`](/jq)
+  - [`pathman`](/pathman)
+  - [`ssh-pubkey`](/ssh-pubkey)
+  - [`webi`](/webi)
+- Aliases
+  - `cat='bat --style=plain --pager=none'`
+  - `curl=curlie`
+  - `setalias='aliasman'`
+- System
+  - [iterm2](/iterm2)
+  - [fish](/fish)
+  - [NerdFont](/nerdfont)
+  - git
+  - vim
+  - zip

beyond-shell/install.sh

@@ -0,0 +1,415 @@
+#!/bin/sh
+set -e
+set -u
+
+webi_cmd="$HOME/.local/bin/webi"
+aliasman_cmd="$HOME/.local/bin/aliasman"
+pathman_cmd="$HOME/.local/bin/pathman"
+
+my_date="$(
+    date -u '+%F_%H.%M.%S'
+)"
+my_log="${HOME}/.local/share/beyond-code/var/${my_date}.log"
+
+fn_node_version() { (
+    my_node="${1:-}"
+
+    if test -n "${my_node}"; then
+        "${my_node}" --version | sed 's/v//'
+    fi
+); }
+
+fn_node_path() { (
+    if command -v node; then
+        return 0
+    fi
+
+    if test ! -e ~/.local/opt/node/bin/node; then
+        return 0
+    fi
+
+    echo ~/.local/opt/node/bin/node
+); }
+
+fn_node_install() { (
+    node_minimum="18"
+
+    my_path_actual="$(
+        fn_node_path
+    )"
+    my_node="$(
+        fn_node_path
+    )"
+    my_node_version="$(
+        fn_node_version "${my_node}"
+    )"
+
+    if test -n "${my_path_actual}"; then
+        my_bin_dir="$(
+            dirname "${my_node}"
+        )"
+        my_node_dir="$(
+            dirname "${my_bin_dir}"
+        )"
+        my_lib_dir="${my_node_dir}/lib/node_modules"
+        if test ! -w "${my_bin_dir}" ||
+            test ! -w "${my_node_dir}" ||
+            test ! -w "${my_lib_dir}"; then
+            printf "\e[31m[Warning]\e[0m \e[32mNode.js\e[0m at %s is not user-writable\n" "${my_path_actual}"
+            sleep 1
+
+            my_path_actual=""
+        fi
+    fi
+
+    if test -n "${my_path_actual}"; then
+        my_node_major="$(
+            echo "${my_node_version}" | cut -d'.' -f1
+        )"
+        if test -z "${my_node_major}"; then
+            my_path_actual=""
+        elif test "${my_node_major}" -lt "${node_minimum}"; then
+
+            printf "\e[31m[Warning]\e[0m \e[32mNode.js v%s\e[0m is too old\n" "${my_node_version}"
+            sleep 1
+
+            my_path_actual=""
+        fi
+    fi
+
+    if test -z "${my_path_actual}"; then
+        printf "\e[32m[Fixup]\e[0m Installing \e[32mNode.js\e[0m to \e[34m%s\e[0m\n" '~'/.local/opt/node/
+        sleep 1
+
+        "${webi_cmd}" node@lts >> "${my_log}"
+        if test -e ~/.local/opt/node/bin/node; then
+            "${pathman_cmd}" add ~/.local/opt/node/bin/ > /dev/null 2> /dev/null
+        fi
+    else
+        echo "[Info] Found ${my_path_actual}/node (v${my_node_version})"
+    fi
+); }
+
+fn_git_install() { (
+    if [ "Darwin" = "$(uname -s)" ]; then
+        needs_xcode="$(
+            /usr/bin/xcode-select -p > /dev/null 2> /dev/null || echo "true"
+        )"
+        if [ -n "${needs_xcode}" ]; then
+            echo ""
+            echo ""
+            printf "\e[31m[Error]\e[0m: Run this command to install XCode Command Line Tools first:\n"
+            echo ""
+            echo "    xcode-select --install"
+            echo ""
+            echo "After the install, close this terminal, open a new one, and try again."
+            echo ""
+            exit 1
+        else
+            my_git="$(command -v git)"
+            echo "[Info] Found ${my_git}"
+        fi
+        return 0
+    fi
+
+    if ! command -v git > /dev/null; then
+        fn_apt_get git git
+        return 0
+    fi
+
+    my_git="$(command -v git)"
+    echo "[Info] Found ${my_git}"
+); }
+
+fn_apt_get() { (
+    my_cmd="${1:-}"
+    my_pkg="${2:-}"
+
+    printf "\e[31m[Warning]\e[0m \e[32m%s\e[0m not found\n" "${my_cmd}"
+
+    printf "\e[32m[Fixup]\e[0m install \e[34m%s\e[0m (may require password)\n" "${my_pkg}"
+    sleep 1
+
+    echo "sudo apt-get install ${my_pkg}"
+
+    # shellcheck disable=SC2030
+    export DEBIAN_FRONTEND=noninteractive
+    if ! sudo apt-get install -qq -y -o=Dpkg::Use-Pty=0 "${my_pkg}"; then
+        echo "failed to install ${my_cmd} on"
+        cat /etc/issue
+    fi
+); }
+
+fn_vim_install() { (
+    if command -v vim > /dev/null; then
+        my_vim="$(command -v vim)"
+        echo "[Info] Found ${my_vim}"
+        return 0
+    fi
+    fn_apt_get vim vim
+); }
+
+fn_zip_install() { (
+    if command -v zip > /dev/null; then
+        my_zip="$(command -v zip)"
+        echo "[Info] Found ${my_zip}"
+        return 0
+    fi
+    fn_apt_get zip zip
+); }
+
+fn_fish_install() { (
+    if ! command -v fish > /dev/null; then
+        if [ "Darwin" = "$(uname -s)" ]; then
+            printf "\e[31m[Warning]\e[0m \e[32mfish\e[0m not found\n"
+
+            printf "\e[32m[Fixup]\e[0m install \e[34mfish\e[0m\n"
+            sleep 1
+
+            "${webi_cmd}" fish > /dev/null
+        else
+            fn_apt_get fish fish
+        fi
+    fi
+
+    if ! test -f ~/.config/fish/config.fish; then
+        printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" '~'/.config/fish/config.fish
+        mkdir -p ~/.config/fish/
+        touch ~/.config/fish/config.fish
+        chmod 0600 ~/.config/fish/config.fish
+    fi
+); }
+
+fn_touch() { (
+    my_file_path="${1:-}"
+    my_file_name="${2:-}"
+
+    if test -f "${my_file_path}"; then
+        echo "[Info] Found ${my_file_name}"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" "${my_file_name}"
+    my_base_path="$(
+        basename "${my_file_path}"
+    )"
+
+    if test ! -e "${my_base_path}"; then
+        mkdir -p "${my_base_path}"
+        chmod 0700 "${my_base_path}"
+    fi
+
+    touch "${my_file_path}"
+    chmod 0600 "${my_file_path}"
+); }
+
+fn_mkdir() { (
+    my_dir_name="${1:-}"
+    my_dir_path="${2:-}"
+    if test -d "${my_dir_path}"; then
+        echo "[Info] Found ${my_dir_name}"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" "${my_dir_name}"
+    mkdir -p "${my_dir_path}"
+    chmod 0700 "${my_dir_path}"
+); }
+
+fn_path_bin() { (
+    if test -d ~/bin/; then
+        echo "[Info] Found ~/bin/"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" '~'/bin/
+    mkdir -p ~/bin/
+    "${pathman_cmd}" add ~/bin/ > /dev/null 2> /dev/null
+); }
+
+fn_webi_bin() { (
+    my_cmd="$(
+        echo "${1:-}" | sed 's/@.*//'
+    )"
+
+    if test -e ~/.local/bin/"${my_cmd}"; then
+        echo "[Info] Found ~/.local/bin/${my_cmd}"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "${my_cmd}" '~'"/.local/bin/${my_cmd}"
+
+    "${webi_cmd}" "${my_cmd}" > /dev/null 2> /dev/null
+); }
+
+fn_webi_opt() { (
+    my_cmd="${1:-}"
+
+    # shellcheck disable=SC2010
+    # we do want to use ls with grep
+    if ls ~/.local/opt/ | grep -q -E "^${my_cmd}(-|\$)"; then
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s/\e[0m\n" "${my_cmd}" '~'"/.local/opt/${my_cmd}"
+
+    "${webi_cmd}" "${my_cmd}" > /dev/null
+); }
+
+fn_webi_vim_config() { (
+    my_cmd="${1:-}"
+
+    if test -e ~/.vim/plugins/"${my_cmd}.vim"; then
+        echo "[Info] Found ~/.vim/plugins/${my_cmd}.vim"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s.vim\e[0m\n" "vim-${my_cmd}" '~'"/.vim/plugins/${my_cmd}.vim"
+
+    "${webi_cmd}" "vim-${my_cmd}" > /dev/null
+); }
+
+fn_webi_vim_plugin_ale() { (
+    my_cmd="ale"
+
+    if test -e ~/.vim/pack/plugins/start/"${my_cmd}"; then
+        echo "[Info] Found ~/.vim/pack/plugins/start/${my_cmd}"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "vim-${my_cmd}" '~'"/.vim/pack/plugins/start/${my_cmd}"
+    printf "\e[32m[Fixup]\e[0m     and config at \e[34m%s.vim\e[0m\n" '~'"/.vim/plugins/${my_cmd}"
+
+    "${webi_cmd}" "vim-${my_cmd}" > /dev/null 2> /dev/null
+); }
+
+fn_webi_vim_plugin() { (
+    my_cmd="${1:-}"
+
+    if test -e ~/.vim/pack/plugins/start/"vim-${my_cmd}"; then
+        echo "[Info] Found ~/.vim/pack/plugins/start/vim-${my_cmd}"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "vim-${my_cmd}" '~'"/.vim/pack/plugins/start/vim-${my_cmd}"
+    printf "\e[32m[Fixup]\e[0m     and config at \e[34m%s.vim\e[0m\n" '~'"/.vim/plugins/${my_cmd}"
+
+    "${webi_cmd}" "vim-${my_cmd}" > /dev/null 2> /dev/null
+); }
+
+fn_webi_vimrc() { (
+    my_cmd="${1:-}"
+
+    if grep -q "${my_cmd}" ~/.vimrc; then
+        echo "[Info] Found vim-${my_cmd} config in ~/.vimrc"
+        return 0
+    fi
+
+    printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "vim-${my_cmd}" '~'/.vimrc
+
+    "${webi_cmd}" "vim-${my_cmd}" > /dev/null
+); }
+
+fn_setalias() { (
+    my_name="${1:-}"
+    my_cmd="${2:-}"
+
+    if grep -q "${my_name}" ~/.config/envman/alias.env; then
+        echo "[Info] Found '${my_name}' alias"
+        return 0
+    fi
+
+    printf "\e[32m%s\e[0m aliased to '\e[34m%s\e[0m'\n" "${my_name}" "${my_cmd}"
+
+    "${aliasman_cmd}" "${my_name}" "${my_cmd}" > /dev/null
+); }
+
+__install() {
+    #printf "\e[31mRED\e[0m\n"
+    #printf "\e[32mYELLOW\e[0m\n"
+    #printf "\e[34mBLUE\e[0m\n"
+
+    echo ""
+    printf "\e[34mLog\e[0m will be written to \e[32m%s\e[0m\n" '~'/.local/share/beyond-code/var/
+    sleep 2
+
+    echo ""
+    mkdir -p ~/.local/share/beyond-code/var/
+    mkdir -p ~/.local/opt/
+    {
+        printf "\e[32mwebi\e[0m at \e[32m%s\e[0m\n" '~'/.local/bin/webi
+        echo ""
+
+        if [ "Darwin" = "$(uname -s)" ]; then
+            if test -e /Applications/iTerm.app; then
+                echo "[Info] Found iTerm.app in /Applications/"
+            else
+                printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m into \e[34m%s\e[0m\n" "iTerm.app" "/Applications/"
+                "${webi_cmd}" iterm2 > /dev/null
+            fi
+        fi
+
+        fn_git_install
+        fn_fish_install
+        fn_zip_install
+
+        if test -e ~/.local/share/fonts/'Droid Sans Mono for Powerline Nerd Font Complete.otf'; then
+            echo "[Info] Found NerdFont in ~/.local/share/fonts"
+        elif test -e ~/Library/Fonts/'Droid Sans Mono for Powerline Nerd Font Complete.otf'; then
+            echo "[Info] Found NerdFont in ~/Library/Fonts"
+        else
+            "${webi_cmd}" nerdfont | tail -n 1
+        fi
+
+        if test -e ~/.iterm2/; then
+            echo "[Info] Found iTerm2 utils ~/.iterm2/"
+        else
+            printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m into \e[34m%s\e[0m\n" "iterm2-utils" '~'"/.iterm2/"
+            "${webi_cmd}" iterm2-utils > /dev/null
+        fi
+
+        echo ""
+        echo "[Info] PATH updates in ~/.config/envman/PATH.env"
+        fn_path_bin
+        fn_touch ~/.config/envman/alias.env '~'/.config/envman/alias.env
+        fn_touch ~/.config/envman/PATH.env '~'/.config/envman/PATH.env
+
+        echo ""
+        fn_node_install
+        for my_cmd in aliasman bat curlie jq pathman ssh-pubkey; do
+            fn_webi_bin "${my_cmd}"
+        done
+
+        echo ""
+        echo "[Info] Aliases in ~/.config/envman/alias.env"
+        fn_setalias setalias 'aliasman'
+        fn_setalias cat 'bat --style=plain --pager=none'
+        fn_setalias curl 'curlie'
+
+        # vim config and plugins
+        echo ""
+        fn_vim_install
+        fn_touch ~/.vimrc '~'/.vimrc
+        for my_plugin in leader shell; do
+            fn_webi_vimrc "${my_plugin}"
+        done
+        fn_webi_vim_plugin sensible
+        for my_plugin in viminfo smartcase spell whitespace; do
+            fn_webi_vim_config "${my_plugin}"
+        done
+        fn_webi_vim_plugin_ale
+        for my_plugin in commentary lastplace shfmt prettier; do
+            fn_webi_vim_plugin "${my_plugin}"
+        done
+        for my_cmd in shfmt@3.5 shellcheck; do
+            fn_webi_bin "${my_cmd}"
+        done
+
+    } | tee -a "${my_log}"
+
+    echo ""
+    printf "\e[34mLog\e[0m written to \e[32m%s\e[0m\n" "${my_log}"
+    echo ""
+}
+
+__install

brew/README.md

@@ -5,13 +5,24 @@ tagline: |
   Brew: The Missing Package Manager for macOS (and Linux).
 ---
 
-## Updating `brew`
+To update or switch versions, use brew's built-in upgrade management, or our
+handy update service script:
 
-`brew` has its own built-in upgrade management.
-
-```bash
+```sh
 brew update
 brew upgrade
+brew-update-service-install # Webi helper script
+```
+
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.local/opt/brew/
+~/.local/bin/brew-update-service-install
 ```
 
 ## Cheat Sheet
@@ -26,27 +37,27 @@ way, report it to brew.
 
 ### How to install CLI packages
 
-```bash
+```sh
 brew update
 brew install node
 ```
 
 ### How to install GUI packages
 
-```bash
+```sh
 brew update
 brew cask install docker
 ```
 
 ### Where are things installed?
 
-```bash
+```sh
 ~/.local/opt/brew/
 ```
 
 For reference, traditional `brew` installs here:
 
-```bash
+```sh
 /usr/local/Cellar/
 /opt/homebrew-cask/Caskroom/
 ```
@@ -58,7 +69,7 @@ like `brew upgrade` or `brew update python`, and your whole system gets borked.
 
 If you need to _uninstall_ and _reinstall_ local brew:
 
-```bash
+```sh
 rm -rf ~/.local/opt/brew
 webi brew
 ```
@@ -67,12 +78,12 @@ If you need to _uninstall_ global brew:
 
 For _Mojave_, _Catalina_, _Big Sur_, and above:
 
-```bash
+```sh
 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall.sh)"
 ```
 
 For _High Sierra_ and below:
 
-```bash
+```sh
 /bin/bash -c ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/uninstall)"
 ```

brew/brew-update-service-install

@@ -0,0 +1,56 @@
+#!/bin/sh
+set -e
+set -u
+
+main() { (
+    sed '1,/^#~\/.local\/bin\/brew-updater/d' "${0}" > ~/.local/bin/brew-update-hourly
+    chmod a+x ~/.local/bin/brew-update-hourly
+
+    echo "Checking for serviceman..."
+    ~/.local/bin/webi serviceman
+    if ! command -v serviceman > /dev/null; then
+        export PATH="$HOME/.local/bin:$PATH"
+    fi
+    serviceman --version
+
+    serviceman add --agent \
+        --workdir ~/.local/opt/brew/ \
+        --name sh.brew.updater -- \
+        ~/.local/bin/brew-update-hourly
+); }
+
+if ! main; then
+    exit 1
+fi
+exit 0
+
+#~/.local/bin/brew-updater
+#!/bin/sh
+#set -e
+set -u
+
+if test -e ~/.config/envman/PATH.env; then
+    # shellcheck disable=SC1090
+    . ~/.config/envman/PATH.env
+fi
+
+while true; do
+    my_start="$(date '+%s')"
+
+    my_date="$(date '+%F %T')"
+    echo "[$my_date] Updating brew..."
+    brew update
+    echo ''
+
+    my_end="$(date '+%s')"
+    my_elapsed="$((my_end - my_start))"
+    my_date="$(date '+%F %T')"
+    echo "[$my_date] Updated in ${my_elapsed}s."
+
+    echo "[$my_date] Cleaning up..."
+    brew cleanup
+
+    echo "[$my_date] Waiting 24 hours..."
+    my_wait="$((24 * 60 * 60))"
+    sleep "$my_wait"
+done

brew/install.sh

@@ -1,15 +1,15 @@
-#!/bin/bash
+#!/bin/sh
 
 set -e
 set -u
 
-function _install_brew() {
+_install_brew() {
     # Straight from https://brew.sh
     #/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
 
-    if [[ -n "$(uname -a | grep -i darwin)" ]]; then
+    if test "Darwin" = "$(uname -s)"; then
         needs_xcode="$(/usr/bin/xcode-select -p > /dev/null 2> /dev/null || echo "true")"
-        if [[ -n ${needs_xcode} ]]; then
+        if test -n "${needs_xcode}"; then
             echo ""
             echo ""
             echo "ERROR: Run this command to install XCode Command Line Tools first:"
@@ -20,11 +20,11 @@ function _install_brew() {
             echo ""
         fi
     else
-        if [ -z "$(command -v gcc)" ]; then
+        if ! command -v gcc > /dev/null; then
             echo >&2 "Warning: to install 'gcc' et al on Linux use the built-in package manager."
             echo >&2 "       For example, try: sudo apt install -y build-essential"
         fi
-        if [ -z "$(command -v git)" ]; then
+        if ! command -v git > /dev/null; then
             echo >&2 "Error: to install 'git' on Linux use the built-in package manager."
             echo >&2 "       For example, try: sudo apt install -y git"
             exit 1
@@ -32,17 +32,25 @@ function _install_brew() {
     fi
 
     # From Straight from https://brew.sh
-    if ! [ -d "$HOME/.local/opt/brew" ]; then
+    if ! test -d "$HOME/.local/opt/brew"; then
         echo "Installing to '$HOME/.local/opt/brew'"
         echo ""
         echo "If you prefer to have brew installed to '/usr/local' cancel now and do the following:"
         echo "        rm -rf '$HOME/.local/opt/brew'"
+        # shellcheck disable=2016
         echo '        /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"'
         echo ""
         sleep 3
         git clone --depth=1 https://github.com/Homebrew/brew "$HOME/.local/opt/brew"
     fi
 
+    rm -rf "$HOME/.local/bin/brew-update-service-install"
+    webi_download \
+        "$WEBI_HOST/packages/brew/brew-update-service-install" \
+        "$HOME/.local/bin/brew-update-service-install" \
+        brew-update-service-install
+    chmod a+x "$HOME/.local/bin/brew-update-service-install"
+
     webi_path_add "$HOME/.local/opt/brew/bin"
     export PATH="$HOME/.local/opt/brew/bin:$PATH"
 
@@ -56,8 +64,13 @@ function _install_brew() {
     echo ""
     echo "If you prefer to have brew installed to '/usr/local' do the following:"
     echo "        mv '$HOME/.local/opt/brew' '$HOME/.local/opt/brew.$(date '+%F_%H-%M-%S').bak'"
+    # shellcheck disable=2016
     echo '        /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"'
     echo ""
+
+    echo "To register 'brew update' as a hourly system service:"
+    echo "        brew-update-service-install"
+    echo ""
 }
 
 _install_brew

bun/README.md

@@ -0,0 +1,158 @@
+---
+title: Bun
+homepage: https://bun.sh
+tagline: |
+  Bun is a fast all-in-one JavaScript runtime
+---
+
+To update or switch versions, run `webi bun@<tag>`. \
+(you can use `@beta` for pre-releases, or `@x.y.z` for a specific version)
+
+### Files
+
+These are the files / directories that are created and/or modified with this
+install:
+
+```text
+~/.config/envman/PATH.env
+~/.local/opt/bun/
+~/.local/opt/bun-<VERSION>/
+```
+
+## Cheat Sheet
+
+> Bun is a wicked-fast JavaScript runtime for developer tooling, API servers,
+> and edge computing.
+>
+> It's built in Zig and provides a more curated, "batteries-included" approach
+> to developing with JavaScript and JavaScript-ish languages - such as
+> TypeScript, JSX, and TSX.
+
+Run some <strong><em>x</em></strong>Script:
+
+```sh
+bun run ./hello.js
+bun run ./hello.jsx
+bun run ./hello.ts
+bun run ./hello.tsx
+```
+
+Run a package from `npm`:
+
+```sh
+bun x jswt
+```
+
+More goodies:
+
+```
+bun help
+```
+
+(there's also a built-in **development server** and lots of stuff)
+
+### bun<span>.</span>sh/install vs webi
+
+Bun has an official installer:
+
+```sh
+export BUN_INSTALL="$HOME/.bun"
+curl -fsSL https://bun.sh/install | bash
+```
+
+You might want to still use webi if you want to be able to switch between
+versions.
+
+### How to install command line completions
+
+```sh
+bun completions
+```
+
+### How to create a bun executable
+
+1. Create your script
+   ```sh
+   vim ./hello
+   ```
+   ```js
+   #!/usr/bin/env bun
+   console.log('hello');
+   ```
+2. Make it executable
+   ```sh
+   chmod a+x ./hello
+   ```
+3. Run it
+   ```sh
+   ./hello
+   ```
+
+### How to publish bun packages
+
+At the time of this writing (bun v0.5.1), you'll still need to publish with
+`npm`.
+
+`npm` is installed with [node](/node).
+
+See
+[Getting Started with NPM (as a developer)](https://gist.github.com/coolaj86/1318304).
+
+### How to install bun packages
+
+You can run it with `bun x`:
+
+```sh
+bun x <whatever>
+```
+
+Or you can put the `#!/usr/bin/env bun` shebang before publishing, and install
+from npm:
+
+```sh
+bun install -g <whatever>
+<whatever>
+```
+
+### How to run a Bun program as a service
+
+As a system service on Linux:
+
+(**note**: swap 'my-project' and './my-project' for the name of your project and
+file)
+
+1. Install serviceman (compatible with systemd)
+   ```sh
+   webi serviceman
+   source ~/.config/envman/PATH.env
+   ```
+2. Go into your program's directory
+   ```sh
+   pushd ./my-project/
+   ```
+3. Add your project to the system launcher, running as the current user
+   ```sh
+   serviceman add --name 'my-project' --daemon -- \
+       bun run ./my-project.js
+   ```
+4. Restart the logging service
+   ```sh
+   sudo systemctl restart systemd-journald
+   ```
+
+For **macOS**:
+
+1. Install serviceman (compatible with `launchctl`)
+   ```sh
+   webi serviceman
+   source ~/.config/envman/PATH.env
+   ```
+2. Go into your program's directory
+   ```sh
+   pushd ./my-project/
+   ```
+3. Add your project to the system launcher, running as the current user
+   ```sh
+   serviceman add --agent --name 'my-project' -- \
+       bun run ./my-project.js
+   ```