Compare commits

..

1 Commits

Author SHA1 Message Date
AJ ONeal
5bfc166f1b docs(caddy): a few more updates 2022-10-21 22:34:40 +00:00
607 changed files with 5445 additions and 29638 deletions

View File

@@ -35,18 +35,15 @@ info, and doing a find and replace on a few file system path names.
```
2. Copy the example template and update with info from Official Releases:
<https://github.com/___CHANGE/ME___/releases>
```bash
rsync -av _example/ CHANGE-ME/
```
- [ ] update `CHANGE-ME/release.js` to use the official repo
- [ ] Learn how `CHANGE-ME` unpacks (i.e. as a single file? as a .tar.gz? as
a .tar.gz with a folder named CHANGE-ME?)
- [ ] find and replace to change the name
- [ ] update `CHANGE-ME/install.sh` (see `bat` and `jq` as examples)
- [ ] update `CHANGE-ME/install.ps1` (see `bat` and `jq` as examples)
3. Needs an updated tagline and cheat sheet
- [ ] update `CHANGE-ME/README.md`
- [ ] official URL

View File

@@ -1,37 +0,0 @@
name: Node.js CI
on:
push:
branches: ['main']
pull_request:
jobs:
build:
name: "Fmt, Lint, & Test"
runs-on: ubuntu-latest
strategy:
matrix:
node-version:
#- 20.x
- latest
steps:
- uses: actions/checkout@v3
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
- name: "Webi: Install 'shfmt' and 'shellcheck', and update PATH"
run: |
sh ./_scripts/install-ci-deps
echo "${HOME}/.local/bin" >> $GITHUB_PATH
echo "${HOME}/.local/opt/pwsh" >> $GITHUB_PATH
- run: |
git submodule init
git submodule update
- run: shfmt --version
- run: shellcheck -V
- run: node --version
- run: npm run fmt
- run: npm ci
- run: npm run lint
- env:
GITHUB_TOKEN: ${{ github.token }}
run: npm run test

21
.gitignore vendored
View File

@@ -1,24 +1,7 @@
# generated artifacts
.env
node_modules
install-*.sh
install-*.bat
install-*.ps1
# local config
.env.*
*.env
.env
!example.env
# caches
_cache/
node_modules/
# temporary & backup files
.*.sw*
*.bak
*.bak.*
# other
.DS_Store
desktop.ini
.directory

3
.gitmodules vendored
View File

@@ -1,3 +0,0 @@
[submodule "_webi/build-classifier"]
path = _webi/build-classifier
url = https://github.com/webinstall/webi-build-classifier.git

View File

@@ -1 +0,0 @@
DELETEMEnode_modules/**/*

View File

@@ -1,7 +1,4 @@
{
"globals": {
"AbortController": false
},
"browser": true,
"node": true,
"esversion": 11,

View File

@@ -1,4 +0,0 @@
node_modules
jsconfig.json
package.json
package-lock.json

6
.prettierrc Normal file
View File

@@ -0,0 +1,6 @@
{
"trailingComma": "none",
"tabWidth": 2,
"singleQuote": true,
"proseWrap": "always"
}

View File

@@ -1,9 +0,0 @@
{
"bracketSpacing": true,
"printWidth": 80,
"proseWrap": "always",
"singleQuote": true,
"tabWidth": 2,
"trailingComma": "all",
"semi": true
}

6
.vimrc
View File

@@ -1,6 +0,0 @@
" Note: You must enable per-project .vimrc in your ~/.vimrc
" set secure
" set exrc
" PowerShell settings
let g:ale_powershell_psscriptanalyzer_exclusions = "PSAvoidUsingWriteHost,PSUseDeclaredVarsMoreThanAssignments"

372
AGENTS.md
View File

@@ -1,372 +0,0 @@
# Webi Installers — Agent Guide
Webi installs dev tools to `~/.local/` without sudo. Each installer is a small
package of 3-4 files. This guide tells you how to create and modify them.
## Why Webi Exists
Webi makes tool installation trivially repeatable for people who aren't
sysadmins — freelance clients, junior devs, anyone who shouldn't have to care
about PATH, permissions, or platform differences. Three things matter:
1. **Install without friction.** No sudo, no manual PATH edits, no "necessary
but unimportant" steps leaking into the experience.
2. **Know where things are.** The Files section tells you exactly what got
created or modified. Nothing should be mysterious.
3. **Copy-paste recipes.** The cheat sheet is what you'd send someone less
experienced than yourself instead of a project's full README — scannable,
concrete, easy to reference by name.
## Quick Start: Adding a New Installer
1. Identify the **package type** (see [Categories](#categories) below)
2. Find an existing installer of the same type to use as a template
3. Create `<name>/releases.js`, `install.sh`, `install.ps1`, `README.md`
4. Test with the command in [Testing releases.js](#testing-releasesjs)
5. Run formatters before committing (see [Code Style](#code-style))
## Directory Layout
```
<package-name>/
README.md # YAML frontmatter + docs
releases.js # Fetches release metadata (Node.js)
install.sh # POSIX shell installer (macOS/Linux)
install.ps1 # PowerShell installer (Windows) — optional
```
Key infrastructure directories (do not modify without good reason):
- `_webi/` — bootstrap templates, `normalize.js` (auto-detects OS/arch/ext from
filenames)
- `_common/` — shared JS: `github.js`, `githubish.js`, `gitea.js`, `fetcher.js`
- `_example/` — canonical template for new packages
- `_examples/` — specialized templates (goreleaser, xz-compressed)
## Categories
Ref: <https://github.com/webinstall/webi-installers/issues/412>
| Type | Description | Template to copy |
| ----- | -------------------------------------- | ---------------- |
| `bin` | Single binary in tar/zip | `koji`, `delta` |
| `bin` | Single bare binary (no archive) | `arc`, `shfmt` |
| `bin` | Goreleaser-style archives | `keypairs` |
| 📦 | Self-contained package (bin/man/share) | `node`, `go` |
| 📂 | Multiple binaries/scripts | `pg` |
| 🔗 | Alias/redirect to another package | `ripgrep``rg` |
| 📝 | Bespoke / custom install | `rustlang` |
## releases.js
Fetches release metadata and returns a normalized object. Most packages use
GitHub releases:
```js
'use strict';
var github = require('../_common/github.js');
var owner = 'OWNER';
var repo = 'REPO';
let Releases = module.exports;
Releases.latest = async function () {
let all = await github(null, owner, repo);
return all;
};
Releases.sample = async function () {
let normalize = require('../_webi/normalize.js');
let all = await Releases.latest();
all = normalize(all);
all.releases = all.releases.slice(0, 5);
return all;
};
if (module === require.main) {
(async function () {
let samples = await Releases.sample();
console.info(JSON.stringify(samples, null, 2));
})();
}
```
### Common release transformations
**Strip version prefix** (monorepo or tool-prefixed tags):
```js
// e.g. "tools/monorel/v0.6.5" → "v0.6.5"
rel.version = rel.version.replace(/^tools\/monorel\//, '');
// e.g. "cli-v1.2.3" → "v1.2.3"
rel.version = rel.version.replace(/^cli-/, '');
```
**Filter releases** (monorepo with multiple tools, or unwanted assets):
```js
all.releases = all.releases.filter(function (rel) {
// Keep only releases for this tool
return rel.version.startsWith('tools/monorel/');
});
```
Apply transformations inside `Releases.latest`, before returning `all`.
**Available sources** beyond `github.js`:
- `_common/gitea.js` — Gitea servers
- `_common/git-tag.js` — Git tag listing
- Custom fetch from any JSON API (see `go/releases.js`, `terraform/releases.js`)
### Testing releases.js
```sh
node -e "
let Releases = require('./<name>/releases.js');
Releases.sample().then(function (all) {
console.log(JSON.stringify(all, null, 2));
});
"
```
Verify: versions are clean semver (`0.6.5` not `tools/monorel/v0.6.5`), OS/arch
detected correctly, download URLs resolve.
## install.sh
POSIX shell (`sh`, not bash). Always wrapped in a function:
```sh
#!/bin/sh
# shellcheck disable=SC2034
set -e
set -u
__init_pkgname() {
# These 6 variables are required
pkg_cmd_name="cmd"
pkg_dst_cmd="$HOME/.local/bin/cmd"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/cmd-v$WEBI_VERSION/bin/cmd"
pkg_src_dir="$HOME/.local/opt/cmd-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_cmd")"
mv ./cmd "$pkg_src_cmd"
}
pkg_get_current_version() {
cmd --version 2> /dev/null | head -n 1 | cut -d' ' -f2
}
}
__init_pkgname
```
### Framework variables available in install.sh
Set by the webi bootstrap (`_webi/package-install.tpl.sh`):
| Variable | Example | Description |
| --------------- | ------------------- | --------------------- |
| `WEBI_VERSION` | `1.2.3` | Selected version |
| `WEBI_PKG_URL` | `https://...` | Download URL |
| `WEBI_PKG_FILE` | `foo-v1.2.3.tar.gz` | Download filename |
| `WEBI_OS` | `linux` | Detected OS |
| `WEBI_ARCH` | `amd64` | Detected architecture |
| `WEBI_EXT` | `tar.gz` | Archive extension |
| `WEBI_CHANNEL` | `stable` | Release channel |
| `PKG_NAME` | `foo` | Package name |
### Override functions
| Function | Purpose |
| --------------------------- | --------------------------------------------- |
| `pkg_install()` | **Required.** Move files to `$pkg_src` |
| `pkg_get_current_version()` | Parse installed version from command output |
| `pkg_post_install()` | Post-install setup (git config, shell config) |
| `pkg_done_message()` | Custom completion message |
| `pkg_link()` | Override default symlink behavior |
| `pkg_pre_install()` | Custom pre-install logic |
### Framework helper functions
| Function | Purpose |
| ------------------------ | ---------------------------------- |
| `webi_download()` | Download package if not cached |
| `webi_extract()` | Extract archive by extension |
| `webi_path_add <dir>` | Add to PATH via envman |
| `webi_link()` | Create versioned symlinks |
| `webi_check_installed()` | Check if version already installed |
### pkg_install patterns
**Bare binary in archive root:**
```sh
mv ./cmd "$pkg_src_cmd"
```
**Binary in a subdirectory (goreleaser-style `cmd-OS-arch/cmd`):**
```sh
mv ./cmd-*/cmd "$pkg_src_cmd"
```
**Flexible detection (handles multiple archive layouts):**
```sh
if test -f ./cmd; then
mv ./cmd "$pkg_src_cmd"
elif test -e ./cmd-*/cmd; then
mv ./cmd-*/cmd "$pkg_src_cmd"
elif test -e ./cmd-*; then
mv ./cmd-* "$pkg_src_cmd"
fi
```
## install.ps1
PowerShell for Windows. Uses `$Env:` variables. See `_example/install.ps1` for
the full template. Key differences from install.sh:
- Paths use backslashes, commands end in `.exe`
- `$Env:USERPROFILE` instead of `$HOME`
- `Test-Path`, `Move-Item`, `Copy-Item` instead of shell equivalents
- Downloads go to `$Env:USERPROFILE\Downloads\webi\`
- Temp work in `.local\tmp`, use `Push-Location`/`Pop-Location`
- Symlinks done via `Copy-Item` (not actual symlinks)
## README.md
````markdown
---
title: toolname
homepage: https://github.com/owner/repo
tagline: |
toolname: A short one-line description.
---
To update or switch versions, run `webi toolname@stable` (or `@v2`, `@beta`,
etc).
### Files
These are the files that are created and/or modified with this installer:
```text
~/.config/envman/PATH.env
~/.local/bin/toolname
~/.local/opt/toolname-VERSION/bin/toolname
```
## Cheat Sheet
> `toolname` does X. Brief description.
### How to use toolname
```sh
toolname --example
```
````
Note: **Files goes above Cheat Sheet**, not inside it.
### Cheat Sheet tone and style
Webi cheat sheets are **opinionated quick-reference guides**, not comprehensive
documentation. Think "colleague's sticky note" — not the project's official
README.
The tool is the topic, but **the problem is the reason**. Cheat sheets are
organized around tasks the reader already wants to do — the tool is how they get
there. Headings reference the tool (the reader came to this page on purpose),
but the content solves the underlying problem completely:
- "How to reverse proxy to Node" (caddy knowledge, not just node)
- "How to run a Node app as a System Service" (serviceman knowledge)
- "How to Enable Secure Remote Postgres Access" (openssl, pg_hba.conf, systemd)
- "How to manually configure git to use delta" (gitconfig, not delta flags)
- "How to make fish the default shell in iTerm2" (iTerm2 knowledge, not fish)
The reader's question is "how do I do X?" and the cheat sheet answers it
completely — including configs, adjacent tools, and platform-specific
variations. A goreleaser cheat sheet teaches you goreleaser YAML. A postgres
cheat sheet teaches you pg_hba.conf, openssl certs, and systemd units.
Cheat sheets cross tool boundaries freely. Node's references caddy, serviceman,
setcap-netbind, GitHub Actions. Postgres references serviceman, openrc, launchd.
They link to each other's webi pages. The scope is "everything you need to
accomplish this task," not "everything this one binary does."
They show the actual files and configs that matter — not documentation _about_
configs, but the configs themselves, copy-pasteable, with inline comments
explaining the non-obvious parts.
**Guidelines:**
- **Show the 3-5 things someone will actually do**, with copy-pasteable
commands. Skip exhaustive flag lists and API docs.
- **Lead with practical integration.** Show the exact `git config` lines, the
exact hook script, the exact shell alias — don't just explain the feature and
leave wiring up to the reader.
- **Skip what they already know.** No need to re-explain what the tool is at
length — the tagline and one-liner blockquote handle that. Get to the
commands.
- **Prefer concrete over abstract.** Instead of "you can configure X via a
config file", show the config file contents.
## Shell Naming Conventions
**Variables:**
- `ALL_CAPS` — environment variables only (`PATH`, `HOME`, `WEBI_VERSION`)
- `b_varname` — block-scoped (inside a function, loop, or conditional)
- `g_varname` — global to the script (and sourced scripts)
- `a_varname` — function arguments
**Functions and commands:**
- `fn_name` — helper functions (anything other than the script's main/entry
function)
- `cmd_name` — command aliases, e.g. `cmd_curl='curl --fail-with-body -sSL'`
## Code Style
Requires `node`, `shfmt`, `pwsh`, and `pwsh-essentials` (install all via webi).
Run before committing:
```sh
npm run fmt # prettier (JS/MD) + shfmt (sh) + pwsh-fmt (ps1)
npm run lint # jshint + shellcheck + pwsh-lint
```
Commit messages: `feat(<pkg>): add installer`, `fix(<pkg>): update install.sh`,
`docs(<pkg>): add cheat sheet`.
## Naming Conventions
- The canonical package name is the **command name** you type: `go`, `node`,
`rg`
- The alternate/alias name is the project name: `golang`, `nodejs`, `ripgrep`
- Package directories are lowercase with hyphens
## Common Pitfalls
- **Monorepo releases**: The GitHub API returns ALL releases for the repo. You
must filter in `releases.js` and strip the tag prefix from the version.
- **No `--version` flag**: Some tools lack version introspection. Comment out
`pkg_get_current_version` — webi still works, it just can't skip reinstalls.
- **normalize.js auto-detection**: OS/arch/ext are guessed from download
filenames. If the tool uses non-standard naming, you may need to set `os`,
`arch`, or `ext` explicitly in `releases.js`.
- **Goreleaser archives**: Typically contain a bare binary at the archive root
(not nested in a directory). Use `mv ./cmd "$pkg_src_cmd"`.

View File

@@ -10,6 +10,7 @@
- You'll be asked to make changes if you don't run the code formatters and
linters:
- Node / JavaScript:
- [prettier](https://webinstall.dev/prettier)
```sh
@@ -20,6 +21,7 @@
npm run lint
```
- Bash
- [shfmt](https://webinstall.dev/shfmt)
```sh
npm run shfmt
@@ -43,7 +45,7 @@
<https://gist.github.com/nicerobot/53cee11ee0abbdc997661e65b348f375>
- Common exceptions:
```text
```txt
# We make use of `.` (source) to import without exports
SC2034: foo appears unused. Verify it or export it.
SC2154: var is referenced but not assigned.
@@ -92,15 +94,15 @@ The general format is `<type>(<package>): <description>`, using these _types_:
Try to write your commit messages (in the present tense) like this:
```text
```txt
fix(node): update install.sh (fix #200)
```
```text
```txt
feat(delta): add cheat sheet and install.sh
```
```text
```txt
docs(ssh-adduser): document that foo does bar
```

View File

@@ -98,10 +98,9 @@ You just fill in the blanks.
Just create an empty directory and run the tests until you get a good result.
```sh
git clone git@github.com:webinstall/webi-installers.git
pushd ./webi-installers/
git submodule update --init
npm clean-install
git clone git@github.com:webinstall/packages.git
pushd packages
npm install
```
```sh
@@ -145,8 +144,8 @@ It looks like this:
`releases.js`:
```js
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
// if you need to do something special, you can do it here
// ...
return all;

View File

@@ -1,62 +1,64 @@
'use strict';
let Fetcher = require('../_common/fetcher.js');
/**
* Gets releases from 'brew'.
* Gets a releases from 'brew'.
*
* @param {null} _
* @param request
* @param {string} formula
* @returns {Promise<any>}
* @returns {PromiseLike<any> | Promise<any>}
*/
async function getDistributables(_, formula) {
function getAllReleases(request, formula) {
if (!formula) {
return Promise.reject('missing formula for brew');
}
let resp;
try {
let url = `https://formulae.brew.sh/api/formula/${formula}.json`;
resp = await Fetcher.fetch(url, {
headers: { Accept: 'application/json' },
});
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
if (err.code === 'E_FETCH_RELEASES') {
err.message = `failed to fetch '${formula}' release data from 'brew': ${err.response.status} ${err.response.body}`;
}
throw e;
}
let body = JSON.parse(resp.body);
var ver = body.versions.stable;
var dl = (
body.bottle.stable.files.high_sierra || body.bottle.stable.files.catalina
).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
return [
{
version: ver,
download: dl.replace(/{{ v }}/g, ver),
},
].concat(
body.versioned_formulae.map(
/** @param {String} f */
function (f) {
var ver = f.replace(/.*@/, '');
return {
return request({
url: 'https://formulae.brew.sh/api/formula/' + formula + '.json',
fail: true, // https://git.coolaj86.com/coolaj86/request.js/issues/2
json: true
})
.then(failOnBadStatus)
.then(function (resp) {
var ver = resp.body.versions.stable;
var dl = (
resp.body.bottle.stable.files.high_sierra ||
resp.body.bottle.stable.files.catalina
).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
return [
{
version: ver,
download: dl,
};
},
),
);
download: dl.replace(/{{ v }}/g, ver)
}
].concat(
resp.body.versioned_formulae.map(function (f) {
var ver = f.replace(/.*@/, '');
return {
version: ver,
download: dl
};
})
);
})
.catch(function (err) {
console.error('Error fetching MariaDB versions (brew)');
console.error(err);
return [];
});
}
module.exports = getDistributables;
function failOnBadStatus(resp) {
if (resp.statusCode >= 400) {
var err = new Error('Non-successful status code: ' + resp.statusCode);
err.code = 'ESTATUS';
err.response = resp;
throw err;
}
return resp;
}
module.exports = getAllReleases;
if (module === require.main) {
getDistributables(null, 'mariadb').then(function (all) {
getAllReleases(require('@root/request'), 'mariadb').then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -1,56 +0,0 @@
'use strict';
let Fetcher = module.exports;
/**
* @typedef ResponseSummary
* @prop {Boolean} ok
* @prop {Headers} headers
* @prop {Number} status
* @prop {String} body
*/
/**
* @param {String} url
* @param {RequestInit} opts
* @returns {Promise<ResponseSummary>}
*/
Fetcher.fetch = async function (url, opts) {
let resp = await fetch(url, opts);
let summary = Fetcher.throwIfNotOk(resp);
return summary;
};
/**
* @param {Response} resp
* @returns {Promise<ResponseSummary>}
*/
Fetcher.throwIfNotOk = async function (resp) {
let text = await resp.text();
if (!resp.ok) {
let headers = Array.from(resp.headers);
console.error('[Fetcher] error: Response Headers:', headers);
console.error('[Fetcher] error: Response Text:', text);
let err = new Error(`fetch was not ok`);
Object.assign({
status: 503,
code: 'E_FETCH_RELEASES',
response: {
status: resp.status,
headers: headers,
body: text,
},
});
throw err;
}
let summary = {
ok: resp.ok,
headers: resp.headers,
status: resp.status,
body: text,
};
return summary;
};

View File

@@ -1,218 +0,0 @@
'use strict';
require('dotenv').config({ path: '.env' });
var Crypto = require('crypto');
var util = require('util');
var exec = util.promisify(require('child_process').exec);
var Fs = require('node:fs/promises');
var FsSync = require('node:fs');
var Path = require('node:path');
var repoBaseDir = process.env.REPO_BASE_DIR || '';
if (!repoBaseDir) {
repoBaseDir = Path.resolve('./_repos');
// for stderr
console.error(`[Warn] REPO_BASE_DIR= not set, ${repoBaseDir}`);
}
var Repos = {};
Repos.clone = async function (repoPath, gitUrl) {
let uuid = Crypto.randomUUID();
let tmpPath = `${repoPath}.${uuid}.tmp`;
let bakPath = `${repoPath}.${uuid}.dup`;
await exec(`git clone --bare --filter=tree:0 ${gitUrl} ${tmpPath}`);
try {
FsSync.accessSync(repoPath);
return;
} catch (e) {
if (e.code !== 'ENOENT') {
throw e;
}
}
// sync to avoid race conditions
try {
FsSync.renameSync(repoPath, bakPath);
} catch (e) {
if (e.code !== 'ENOENT') {
throw e;
}
}
FsSync.renameSync(tmpPath, repoPath);
await Fs.rm(bakPath, { force: true, recursive: true });
};
Repos.checkExists = async function (repoPath) {
let err = await Fs.access(repoPath).catch(Object);
if (!err) {
return true;
}
if (err.code !== 'ENOENT') {
throw err;
}
return false;
};
Repos.fetch = async function (repoPath) {
await exec(`git --git-dir=${repoPath} fetch`);
};
Repos.getTags = async function (repoPath) {
var { stdout } = await exec(`git --git-dir=${repoPath} tag`);
var rawTags = stdout.trim().split('\n');
let tags = [];
for (let tag of rawTags) {
// ex: v1, v2, v1.1, 1.1.0-rc
let maybeVersionRe = /^(v\d+|v?\d+\.\d+)/;
let maybeVersion = maybeVersionRe.test(tag);
if (maybeVersion) {
tags.push(tag);
}
}
tags = tags.reverse();
return tags;
};
Repos.getTipInfo = async function (repoPath) {
var { stdout } = await exec(
`git --git-dir=${repoPath} rev-parse --abbrev-ref HEAD`,
);
var branch = stdout.trim();
var info = await Repos.getCommitInfo(repoPath, 'HEAD');
info.commitish = branch;
return info;
};
Repos.getCommitInfo = async function (repoPath, commitish) {
var { stdout } = await exec(
`git --git-dir=${repoPath} log -1 --format="%h %H %ad %cd" --date=iso-strict ${commitish}`,
);
stdout = stdout.trim();
var commitParts = stdout.split(/\s+/g);
return {
commitish: commitish,
commit_id: `${commitParts[0]}`,
commit: `${commitParts[1]}`,
date: commitParts[2],
date_authored: commitParts[3],
};
};
/**
* Lists GitHub Releases (w/ uploaded assets)
*
* @param request
* @param {string} owner
* @param {string} gitUrl
* @returns {PromiseLike<any> | Promise<any>}
*/
async function getDistributables(gitUrl) {
let all = {
releases: [],
download: '',
};
let repoName = gitUrl.split('/').pop();
repoName = repoName.replace(/\.git$/, '');
let repoPath = `${repoBaseDir}/${repoName}.git`;
let isCloned = await Repos.checkExists(repoPath);
if (!isCloned) {
await Repos.clone(repoPath, gitUrl);
} else {
await Repos.fetch(repoPath);
}
let commitInfos = [];
let tags = await Repos.getTags(repoPath);
for (let tag of tags) {
let commitInfo = await Repos.getCommitInfo(repoPath, tag);
Object.assign(commitInfo, { version: tag, channel: '' });
commitInfos.push(commitInfo);
}
{
let tipInfo = await Repos.getTipInfo(repoPath);
// "2024-01-01T00:00:00-05:00" => "2024-01-01T05:00:00"
let date = new Date(tipInfo.date);
// "2024-01-01T05:00:00" => "v2024.01.01-05.00.00"
let version = date.toISOString();
// strip '.000Z'
version = version.replace(/\.\d+Z/, '');
version = version.replace(/[:\-]/g, '.');
version = version.replace(/T/, '-');
Object.assign(tipInfo, { version: `v${version}`, channel: '' });
if (commitInfos.length > 1) {
tipInfo.channel = 'beta';
}
commitInfos.push(tipInfo);
}
let releases = [];
for (let commitInfo of commitInfos) {
let version = commitInfo.version.replace(/^v/, '');
let date = new Date(commitInfo.date);
let isoDate = date.toISOString();
isoDate = isoDate.replace(/\.\d+Z/, '');
// tags and HEAD qualify for '--branch <branchish>'
let branch = commitInfo.commitish;
let rel = {
name: `${repoName}-v${version}`,
version: version,
git_tag: branch,
git_commit_hash: commitInfo.commit_id,
lts: false,
channel: commitInfo.channel,
date: isoDate,
os: '*',
arch: '*',
ext: 'git',
download: gitUrl,
};
releases.push(rel);
}
all.releases = releases;
return all;
}
module.exports = getDistributables;
if (module === require.main) {
(async function main() {
let testRepos = [
// just a few tags, and a different HEAD
'https://github.com/tpope/vim-commentary.git',
// no tags, just HEAD
'https://github.com/ziglang/zig.vim.git',
// many, many tags
//'https://github.com/dense-analysis/ale.git',
];
for (let url of testRepos) {
let all = await getDistributables(url);
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all, null, 2));
}
})()
.then(function () {
process.exit(0);
})
.catch(function (err) {
console.error(err);
});
}

View File

@@ -1,47 +1,39 @@
'use strict';
var GitHubish = require('./githubish.js');
var ghRelease = require('./github.js');
/**
* Lists Gitea Releases (w/ uploaded assets)
* Gets the releases for 'ripgrep'. This function could be trimmed down and made
* for use with any github release.
*
* @param {null} _ - deprecated
* @param {String} owner
* @param {String} repo
* @param {String} baseurl
* @param {String} [username]
* @param {String} [token]
* @param request
* @param {string} owner
* @param {string} repo
* @returns {PromiseLike<any> | Promise<any>}
*/
async function getDistributables(
_,
owner,
repo,
baseurl,
username = '',
token = '',
) {
baseurl = `${baseurl}/api/v1`;
let all = await GitHubish.getDistributables({
owner,
repo,
baseurl,
username,
token,
function getAllReleases(request, owner, repo, baseurl) {
if (!baseurl) {
return Promise.reject('missing baseurl');
}
return ghRelease(request, owner, repo, baseurl + '/api/v1').then(function (
all
) {
return all;
});
return all;
}
module.exports = getDistributables;
module.exports = getAllReleases;
if (module === require.main) {
getDistributables(
null,
'root',
'pathman',
'https://git.rootprojects.org',
'',
'',
).then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
getAllReleases(
require('@root/request'),
'coolaj86',
'go-pathman',
'https://git.coolaj86.com'
).then(
//getAllReleases(require('@root/request'), 'root', 'serviceman', 'https://git.rootprojects.org').then(
function (all) {
console.info(JSON.stringify(all, null, 2));
}
);
}

View File

@@ -1,40 +0,0 @@
'use strict';
require('dotenv').config({ path: '.env' });
let GitHubSource = module.exports;
let GitHubishSource = require('./githubish-source.js');
/**
* @param {Object} opts
* @param {String} opts.owner
* @param {String} opts.repo
* @param {String} [opts.baseurl]
* @param {String} [opts.username]
* @param {String} [opts.token]
*/
GitHubSource.getDistributables = async function ({
owner,
repo,
baseurl = 'https://api.github.com',
username = process.env.GITHUB_USERNAME || '',
token = process.env.GITHUB_TOKEN || '',
}) {
let all = await GitHubishSource.getDistributables({
owner,
repo,
baseurl,
username,
token,
});
return all;
};
if (module === require.main) {
GitHubSource.getDistributables(null, 'BeyondCodeBootcamp', 'DuckDNS.sh').then(
function (all) {
console.info(JSON.stringify(all, null, 2));
},
);
}

View File

@@ -1,42 +1,76 @@
'use strict';
require('dotenv').config({ path: '.env' });
let GitHubish = require('./githubish.js');
require('dotenv').config();
/**
* Lists GitHub Releases (w/ uploaded assets)
* Gets the releases for 'ripgrep'. This function could be trimmed down and made
* for use with any github release.
*
* @param {null} _ - deprecated
* @param {String} owner
* @param {String} repo
* @param {String} [baseurl]
* @param {String} [username]
* @param {String} [token]
* @param request
* @param {string} owner
* @param {string} repo
* @returns {PromiseLike<any> | Promise<any>}
*/
module.exports = async function (
_,
function getAllReleases(
request,
owner,
repo,
baseurl = 'https://api.github.com',
username = process.env.GITHUB_USERNAME || '',
token = process.env.GITHUB_TOKEN || '',
baseurl = 'https://api.github.com'
) {
let all = await GitHubish.getDistributables({
owner,
repo,
baseurl,
username,
token,
});
return all;
};
if (!owner) {
return Promise.reject('missing owner for repo');
}
if (!repo) {
return Promise.reject('missing repo name');
}
let GitHub = module.exports;
GitHub.getDistributables = module.exports;
var req = {
url: `${baseurl}/repos/${owner}/${repo}/releases`,
json: true
};
// TODO I really don't like global config, find a way to do better
if (process.env.GITHUB_USERNAME) {
req.auth = {
user: process.env.GITHUB_USERNAME,
pass: process.env.GITHUB_TOKEN
};
}
if (module === require.main) {
GitHub.getDistributables(null, 'BurntSushi', 'ripgrep').then(function (all) {
console.info(JSON.stringify(all, null, 2));
return request(req).then((resp) => {
const gHubResp = resp.body;
const all = {
releases: [],
// todo make this ':baseurl' + ':releasename'
download: ''
};
gHubResp.forEach((release) => {
release['assets'].forEach((asset) => {
const name = asset['name'];
all.releases.push({
name: name,
version: release['tag_name'], // TODO tags aren't always semver / sensical
lts: /(\b|_)(lts)(\b|_)/.test(release['tag_name']),
channel: !release['prerelease'] ? 'stable' : 'beta',
date: (release['published_at'] || '').replace(/T.*/, ''),
os: '', // will be guessed by download filename
arch: '', // will be guessed by download filename
ext: '', // will be normalized
download: asset['browser_download_url']
});
});
});
return all;
});
}
module.exports = getAllReleases;
if (module === require.main) {
getAllReleases(require('@root/request'), 'BurntSushi', 'ripgrep').then(
function (all) {
console.info(JSON.stringify(all, null, 2));
}
);
}

View File

@@ -1,174 +0,0 @@
'use strict';
let Fetcher = require('../_common/fetcher.js');
let GitHubishSource = module.exports;
/**
* Lists GitHub-Like Releases (source tarball & zip)
*
* @param {Object} opts
* @param {String} opts.owner
* @param {String} opts.repo
* @param {String} opts.baseurl
* @param {String} [opts.username]
* @param {String} [opts.token]
*/
GitHubishSource.getDistributables = async function ({
owner,
repo,
baseurl,
username = '',
token = '',
}) {
if (!owner) {
throw new Error('missing owner for repo');
}
if (!repo) {
throw new Error('missing repo name');
}
if (!baseurl) {
throw new Error('missing baseurl');
}
let url = `${baseurl}/repos/${owner}/${repo}/releases`;
let opts = {
headers: {
'Content-Type': 'appplication/json',
},
};
if (token) {
let userpass = `${username}:${token}`;
let basicAuth = btoa(userpass);
Object.assign(opts.headers, {
Authorization: `Basic ${basicAuth}`,
});
}
let resp;
try {
resp = await Fetcher.fetch(url, opts);
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
if (err.code === 'E_FETCH_RELEASES') {
err.message = `failed to fetch '${baseurl}' (githubish-source, user '${username}) release data: ${err.response.status} ${err.response.body}`;
}
throw e;
}
let gHubResp = JSON.parse(resp.body);
let all = {
/** @type {Array<BuildInfo>} */
releases: [],
download: '',
};
for (let release of gHubResp) {
let dists = GitHubishSource.releaseToDistributables(release);
for (let dist of dists) {
let updates =
await GitHubishSource.followDistributableDownloadAttachment(dist);
Object.assign(dist, updates);
all.releases.push(dist);
}
}
return all;
};
/**
* @typedef BuildInfo
* @prop {String} [name] - name to use instead of filename for hash urls
* @prop {String} version
* @prop {String} [_version]
* @prop {String} [arch]
* @prop {String} channel
* @prop {String} date
* @prop {String} download
* @prop {String} [ext]
* @prop {String} [_filename]
* @prop {String} [hash]
* @prop {String} [libc]
* @prop {Boolean} [_musl]
* @prop {Boolean} [lts]
* @prop {String} [size]
* @prop {String} os
*/
/**
* @param {any} ghRelease - TODO
* @returns {Array<BuildInfo>}
*/
GitHubishSource.releaseToDistributables = function (ghRelease) {
let ghTag = ghRelease['tag_name']; // TODO tags aren't always semver / sensical
let lts = /(\b|_)(lts)(\b|_)/.test(ghRelease['tag_name']);
let channel = 'stable';
if (ghRelease['prerelease']) {
channel = 'beta';
}
let date = ghRelease['published_at'] || '';
date = date.replace(/T.*/, '');
let urls = [ghRelease.tarball_url, ghRelease.zipball_url];
/** @type {Array<BuildInfo>} */
let dists = [];
for (let url of urls) {
dists.push({
name: '',
version: ghTag,
lts: lts,
channel: channel,
date: date,
os: '*',
arch: '*',
libc: '',
ext: '',
download: url,
});
}
return dists;
};
/**
* @param {BuildInfo} dist
*/
GitHubishSource.followDistributableDownloadAttachment = async function (dist) {
let abortCtrl = new AbortController();
let resp = await fetch(dist.download, {
method: 'HEAD',
redirect: 'follow',
signal: abortCtrl.signal,
});
let headers = Object.fromEntries(resp.headers);
// Workaround for bug where METHOD changes to GET
abortCtrl.abort();
await resp.text().catch(function (err) {
if (err.name !== 'AbortError') {
throw err;
}
});
// ex: content-disposition: attachment; filename=BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
// => BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
let name = headers['content-disposition'].replace(
/.*filename=([^;]+)(;|$)/,
'$1',
);
let download = resp.url;
return { name, download };
};
if (module === require.main) {
GitHubishSource.getDistributables({
owner: 'BeyondCodeBootcamp',
repo: 'DuckDNS.sh',
baseurl: 'https://api.github.com',
}).then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -1,137 +0,0 @@
'use strict';
let Fetcher = require('../_common/fetcher.js');
/**
* @typedef DistributableRaw
* @prop {String} name
* @prop {String} version
* @prop {Boolean} lts
* @prop {String} [channel]
* @prop {String} date
* @prop {String} os
* @prop {String} arch
* @prop {String} ext
* @prop {String} download
*/
let GitHubish = module.exports;
/**
* Lists GitHub-Like Releases (w/ uploaded assets)
*
* @param {Object} opts
* @param {String} opts.owner
* @param {String} opts.repo
* @param {String} opts.baseurl
* @param {String} [opts.username]
* @param {String} [opts.token]
*/
GitHubish.getDistributables = async function ({
owner,
repo,
baseurl,
username = '',
token = '',
}) {
if (!owner) {
throw new Error('missing owner for repo');
}
if (!repo) {
throw new Error('missing repo name');
}
if (!baseurl) {
throw new Error('missing baseurl');
}
let url = `${baseurl}/repos/${owner}/${repo}/releases`;
let opts = {
headers: {
'Content-Type': 'appplication/json',
},
};
if (token) {
let userpass = `${username}:${token}`;
let basicAuth = btoa(userpass);
Object.assign(opts.headers, {
Authorization: `Basic ${basicAuth}`,
});
}
let resp;
try {
resp = await Fetcher.fetch(url, opts);
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
if (err.code === 'E_FETCH_RELEASES') {
err.message = `failed to fetch '${baseurl}' (githubish, user '${username}) release data: ${err.response.status} ${err.response.body}`;
}
throw e;
}
let gHubResp = JSON.parse(resp.body);
let all = {
/** @type {Array<DistributableRaw>} */
releases: [],
// todo make this ':baseurl' + ':releasename'
download: '',
};
try {
gHubResp.forEach(transformReleases);
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
console.error(err.message);
console.error('Error Headers:', resp.headers);
console.error('Error Body:', resp.body);
let msg = `failed to transform releases from '${baseurl}' with user '${username}'`;
throw new Error(msg);
}
/**
* @param {any} release - TODO
*/
function transformReleases(release) {
for (let asset of release['assets']) {
let name = asset['name'];
let date = release['published_at']?.replace(/T.*/, '');
let download = asset['browser_download_url'];
// TODO tags aren't always semver / sensical
let version = release['tag_name'];
let channel;
if (release['prerelease']) {
// -rcX, -preview, -beta, etc will be checked in _webi/normalize.js
channel = 'beta';
}
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
all.releases.push({
name: name,
version: version,
lts: lts,
channel: channel,
date: date,
os: '', // will be guessed by download filename
arch: '', // will be guessed by download filename
ext: '', // will be normalized
download: download,
});
}
}
return all;
};
if (module === require.main) {
GitHubish.getDistributables({
owner: 'BurntSushi',
repo: 'ripgrep',
baseurl: 'https://api.github.com',
}).then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -20,10 +20,10 @@ etc).
These are the files / directories that are created and/or modified with this
install:
```text
```txt
~/.config/envman/PATH.env
~/.local/bin/foo
~/.local/opt/foo/
~/.local/opt/foo
```
## Cheat Sheet

View File

@@ -15,45 +15,47 @@ $pkg_src_bin = "$Env:USERPROFILE\.local\opt\foobar-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\foobar-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | out-null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading foobar from $Env:WEBI_PKG_URL to $pkg_download"
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"))
{
echo "Downloading foobar from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
& move "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing foobar"
IF (!(Test-Path -Path "$pkg_src_cmd"))
{
echo "Installing foobar"
# TODO: create package-specific temp directory
# Enter tmp
Push-Location .local\tmp
pushd .local\tmp
# Remove any leftover tmp cruft
Remove-Item -Path ".\foobar-v*" -Recurse -ErrorAction Ignore
Remove-Item -Path ".\foo.exe" -Recurse -ErrorAction Ignore
# Remove any leftover tmp cruft
Remove-Item -Path ".\foobar-v*" -Recurse -ErrorAction Ignore
Remove-Item -Path ".\foo.exe" -Recurse -ErrorAction Ignore
# NOTE: DELETE THIS COMMENT IF NOT USED
# Move single binary into root of temporary folder
#& move "$pkg_download" "foo.exe"
# NOTE: DELETE THIS COMMENT IF NOT USED
# Move single binary into root of temporary folder
#& move "$pkg_download" "foo.exe"
# Unpack archive file into this temporary directory
# Windows BSD-tar handles zip. Imagine that.
Write-Output "Unpacking $pkg_download"
& tar xf "$pkg_download"
# Unpack archive file into this temporary directory
# Windows BSD-tar handles zip. Imagine that.
echo "Unpacking $pkg_download"
& tar xf "$pkg_download"
# Settle unpacked archive into place
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
Move-Item -Path ".\foobar-*\foo.exe" -Destination "$pkg_src_bin"
# Settle unpacked archive into place
echo "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | out-null
Move-Item -Path ".\foobar-*\foo.exe" -Destination "$pkg_src_bin"
# Exit tmp
Pop-Location
popd
}
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
echo "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | out-null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

View File

@@ -12,26 +12,17 @@ var repo = 'ripgrep';
/** **/
/******************************************************************************/
let Releases = module.exports;
Releases.latest = async function () {
let all = await github(null, owner, repo);
return all;
};
Releases.sample = async function () {
let normalize = require('../_webi/normalize.js');
let all = await Releases.latest();
all = normalize(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);
return all;
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
(async function () {
let samples = await Releases.sample();
console.info(JSON.stringify(samples, null, 2));
})();
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -1,12 +1,15 @@
'use strict';
var pkg = require('../package.json');
var spawn = require('child_process').spawn;
var os = require('os');
var path = require('path');
function spawner(args) {
return new Promise(function (resolve, reject) {
var bin = args.shift();
var runner = spawn(bin, args, {
windowsHide: true,
windowsHide: true
});
runner.stdout.on('data', function (chunk) {
console.info(chunk.toString('utf8'));

View File

@@ -4,6 +4,7 @@
//var pkg = require('../package.json');
var os = require('os');
//var request = require('@root/request');
//var promisify = require('util').promisify;
//var exec = promisify(require('child_process').exec);
var exec = require('child_process').exec;
@@ -18,7 +19,7 @@ if (/^win/i.test(os.platform())) {
return;
}
exec('curl -fsS https://webi.sh/webi | sh', function (err, stdout, stderr) {
exec('curl -fsS https://webi.sh/webi | sh', function(err, stdout, stderr) {
if (err) {
console.error(err);
}

View File

@@ -1,57 +0,0 @@
#!/usr/bin/env node
'use strict';
var Fs = require('node:fs/promises');
var Path = require('node:path');
async function copyScripts() {
var pkgDir = Path.join(__dirname, '..');
var scriptsDir = Path.join(pkgDir, '_scripts');
var gitFile = Path.join(pkgDir, '.git');
// if this is a submodule, '.git' will be a file with a
// path to the actual git module directory
var gitDir = await Fs.readFile(gitFile, 'utf8')
.catch(function (e) {
// console.error(e);
//return 'gitdir: ../.git/modules/installers';
return 'gitdir: ./.git';
})
.then(function (str) {
var parts = str.split(': ');
str = parts[1];
str = str.trim();
return Path.resolve(pkgDir, str);
});
var gitHooksDir = Path.join(gitDir, 'hooks');
var src = Path.join(scriptsDir, 'git-hooks-pre-commit.js');
var dst = Path.join(gitHooksDir, 'pre-commit');
console.info(`[git-hooks] Checking for pre-commit hooks...`);
var relSrc = Path.relative(pkgDir, src);
var relDst = Path.relative(pkgDir, dst);
await Fs.access(dst)
.then(function () {
console.info(`[git-hooks] Found ${relDst}`);
})
.catch(async function (e) {
// ignore e
await Fs.mkdir(gitHooksDir, { recursive: true });
await Fs.copyFile(src, dst);
await Fs.chmod(dst, 0o755);
console.info(`[git-hooks] Found template ${relSrc}`);
console.info(`[git-hooks] Initialized ${relDst}`);
});
}
copyScripts()
.then(function () {
process.exit(0);
})
.catch(function (e) {
console.error(e);
process.exit(1);
});

View File

@@ -1,7 +0,0 @@
#!/bin/sh
set -e
set -u
npm run fmt
npm run lint
npm run test

View File

@@ -1,12 +0,0 @@
#!/usr/bin/env node
'use strict';
var Process = require('child_process');
var procOpts = { stdio: 'inherit' };
var commands = ['npm run fmt', 'npm run lint', 'npm run test'];
for (let cmd of commands) {
console.info(`[pre-commit] exec: ${cmd}`);
Process.execSync(cmd, procOpts);
}

View File

@@ -1,15 +0,0 @@
#!/bin/sh
set -e
set -u
# Install 'shfmt 3.7.x'
curl -fsS https://webi.sh/shfmt@3.7 | sh
# Install 'shellcheck v0.9.x'
curl -fsS https://webi.sh/shellcheck@0.9 | sh
# Install 'pwsh (PowerShell Core) v7.x'
curl -fsS https://webi.sh/pwsh@7 | sh
# shellcheck disable=SC1090
. ~/.config/envman/PATH.env
pwsh -Command "Install-Module -Name PSScriptAnalyzer -Scope CurrentUser -AllowClobber"

View File

@@ -1,27 +0,0 @@
#!/bin/sh
set -e
set -u
echo "Formatting */*.ps1 ... "
for my_ps1 in */*.ps1; do
my_dir="$(
dirname "${my_ps1}"
)"
if test -L "${my_ps1}" ||
test -L "${my_dir}" ||
! test -f "${my_ps1}" ||
! test -d "${my_dir}"; then
printf ' SKIP %s (non-regular file or parent directory)\n' "${my_ps1}"
continue
fi
printf " %s" "${my_ps1}"
# -Settings ./Settings/CodeFormatting.psd1
my_new_file="$(
pwsh -Command "Invoke-Formatter -ScriptDefinition (Get-Content -Path '${my_ps1}' -Raw)"
)"
printf '%s\n' "${my_new_file}" > "${my_ps1}"
printf '\n'
done

View File

@@ -1,40 +0,0 @@
#!/usr/bin/env pwsh
$ErrorActionPreference = "Stop"
Write-Host "Formatting */*.ps1 ... "
$my_cwd = Get-Location
$my_dirs = Get-ChildItem -Directory
$my_status = 0
foreach ($my_dir in $my_dirs) {
$my_files = Get-ChildItem -Path $my_dir.FullName -File -Filter *.ps1
foreach ($my_file in $my_files) {
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
-not (Test-Path -PathType Container -Path $my_dir)) {
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
continue
}
Write-Host (" {0}" -f $my_ps1)
$text = Get-Content -Path $my_ps1 -Raw
$my_new_file = Invoke-Formatter -ScriptDefinition $text
$my_new_file = $my_new_file.Trim()
# note: trailing newline is added back on write
$my_new_file | Set-Content -Path $my_ps1
$my_new_file = $my_new_file + "`n"
if ($text -ne $my_new_file) {
$my_status = 1
}
}
}
exit $my_status

View File

@@ -1,36 +0,0 @@
#!/bin/sh
set -e
set -u
echo "Formatting */*.ps1 ..."
for my_ps1 in */*.ps1; do
my_dir="$(
dirname "${my_ps1}"
)"
if test -L "${my_ps1}" ||
test -L "${my_dir}" ||
! test -f "${my_ps1}" ||
! test -d "${my_dir}"; then
printf ' SKIP %s (non-regular file or parent directory)\n' "${my_ps1}"
continue
fi
printf " %s" "${my_ps1}"
# -Settings ./PSScriptAnalyzerSettings.psd1
pwsh -Command "Invoke-ScriptAnalyzer -Fix -ExcludeRule PSAvoidUsingWriteHost,PSUseDeclaredVarsMoreThanAssignment -Path '$my_ps1'"
#
# fmt MUST happen after lint due to Byte-Order Marker (BOM) issues
# See https://github.com/PowerShell/PSScriptAnalyzer/issues/1743
#
# -Settings ./Settings/CodeFormatting.psd1
my_new_file="$(
pwsh -Command "Invoke-Formatter -ScriptDefinition (Get-Content -Path '${my_ps1}' -Raw)"
)"
printf '%s\n' "${my_new_file}" > "${my_ps1}"
printf '\n'
done

View File

@@ -1,48 +0,0 @@
#!/usr/bin/env pwsh
$ErrorActionPreference = "Stop"
Write-Host "Linting */*.ps1 ... "
$my_cwd = Get-Location
$my_dirs = Get-ChildItem -Directory
$my_status = 0
foreach ($my_dir in $my_dirs) {
$my_files = Get-ChildItem -Path $my_dir.FullName -File -Filter *.ps1
foreach ($my_file in $my_files) {
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
-not (Test-Path -PathType Container -Path $my_dir)) {
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
continue
}
Write-Host (" {0}" -f $my_ps1)
$my_old_file = (Get-Content -Path $my_ps1 -Raw)
Invoke-ScriptAnalyzer -Fix -ExcludeRule PSAvoidUsingWriteHost, PSUseDeclaredVarsMoreThanAssignment -Path $my_ps1
#
# fmt MUST happen after lint due to Byte-Order Marker (BOM) issues
# See https://github.com/PowerShell/PSScriptAnalyzer/issues/1743
#
$my_fixed_file = (Get-Content -Path $my_ps1 -Raw)
$my_new_file = Invoke-Formatter -ScriptDefinition $my_fixed_file
$my_new_file = $my_new_file.Trim()
# note: trailing newline is added back on write
$my_new_file | Set-Content -Path $my_ps1
$my_new_file = $my_new_file + "`n"
if ($my_old_file -ne $my_new_file) {
$my_status = 1
}
}
}
exit $my_status

View File

@@ -1,32 +0,0 @@
#!/usr/bin/env node
'use strict';
var Cmd = require('node:child_process');
var Os = require('node:os');
var procOpts = { stdio: 'inherit' };
var commands = ['shfmt --version', 'shellcheck --version'];
for (let cmd of commands) {
console.info(`[tooling-init] checking for '${cmd}':`);
try {
Cmd.execSync(cmd, procOpts);
} catch (e) {
// ignore e
printInstallHelp(cmd);
process.exit(1);
}
}
function printInstallHelp(cmd) {
console.error('');
console.error('ERROR');
console.error(` could not run '${cmd}'`);
console.error('POSSIBLE FIX');
if (/^win/i.test(Os.platform())) {
console.error(` curl.exe https://webi.ms/${cmd} | powershell`);
} else {
console.error(` curl -fsS https://webi.sh/${cmd} | sh`);
}
console.error('');
}

View File

@@ -1,442 +0,0 @@
---
name: installer
description: >
Create or update install.sh and install.ps1 scripts for a webi package.
Use when adding a new package to webi-installers, or when an existing
install script needs to be updated to match a changed archive structure.
Covers discovering archive layout from GitHub releases, identifying the
right install pattern (AI), and writing both the POSIX shell and
PowerShell scripts that the webi framework calls.
Note: this skill covers install scripts only — writing releases.js /
releases.conf (the release-fetcher config) is a separate concern.
license: MIT
compatibility: Requires git, curl, tar. GitHub API access needed for
discovery phase. Designed for Claude Code in the webi-installers repo.
metadata:
author: AJ ONeal
version: "1.1"
---
# Webi Installer Skill
Write `install.sh` and `install.ps1` for a webi package. These scripts are
called by the webi framework **after** it has already downloaded and verified
the archive — your job is only to unpack and place the files.
> **Scope:** This skill covers `install.sh` and `install.ps1` only. A
> separate `releases.js` / `releases.conf` file is needed to tell webi where
> to fetch releases from. That config must already exist (or be written
> separately) before these install scripts are useful.
## Quick overview
1. [Discover the archive layout](#1-discover-the-archive-layout) — inspect
GitHub releases with `curl` + `tar -t` to understand what's inside.
2. [Choose the install pattern](#2-choose-the-install-pattern) — nine
patterns (AI) cover almost every real-world case.
3. [Write `install.sh`](#3-write-installsh) — POSIX shell, ~2040 lines.
4. [Write `install.ps1`](#4-write-installps1) — PowerShell, ~4060 lines.
5. [Check for classification issues](#5-check-for-classification-issues) —
look for variant assets, non-standard OS/arch naming, or installer .exe
files that need special handling.
Full reference: [`references/PATTERNS.md`](references/PATTERNS.md)
Archive layout details: [`references/ARCHIVE-LAYOUTS.md`](references/ARCHIVE-LAYOUTS.md)
Classification guide: [`references/CLASSIFICATION.md`](references/CLASSIFICATION.md)
---
## 1. Discover the archive layout
### Use the webi releases API (fastest, if the package already exists)
```sh
# JSON with all releases for a package
curl -s https://webinstall.dev/api/releases/bat.json | jq '.releases[:3]'
```
Each entry has `name` (filename), `version`, `os`, `arch`, `ext`, `download`.
### Or inspect GitHub releases directly
```sh
# List asset filenames for the latest release
curl -s "https://api.github.com/repos/sharkdp/bat/releases?per_page=3" \
| jq '.[0].assets[] | .name'
```
### Inspect what's inside an archive
Download one representative asset and list its contents **without extracting**:
```sh
# tar.gz / tar.xz
curl -fsSL "$DOWNLOAD_URL" | tar -tz
# tar.zst (modern systems — GNU tar / bsdtar both support this)
curl -fsSL "$DOWNLOAD_URL" | tar --zstd -tz
# zip
curl -fsSL "$DOWNLOAD_URL" -o /tmp/pkg.zip && unzip -l /tmp/pkg.zip
# bare binary (no archive extension, e.g. jq-linux-amd64)
# The file IS the binary — no unpacking needed. Set WEBI_SINGLE=true.
```
Look for:
- Is the binary at the top level or inside a subdirectory?
- Does the subdirectory name include the version and/or triplet?
- Are there completions (`completions/`, `autocomplete/`, `complete/`)?
- Are there man pages (`*.1`, `doc/*.1`, `man/man1/`)?
- Are there shared libraries (`.so`, `.dylib`, `.dll`) alongside the binary?
- Is the binary name different from the package command name?
See [`references/ARCHIVE-LAYOUTS.md`](references/ARCHIVE-LAYOUTS.md) for
what each pattern looks like, with real examples.
---
## 2. Choose the install pattern
| Pattern | Description | Examples |
|---------|-------------|---------|
| **A** | Bare binary (or binary+docs) at archive root | caddy, fzf, k9s, terraform |
| **B** | Binary inside a version/triplet-named subdirectory | delta, shellcheck, trip, xsv |
| **C** | Like B, plus shell completions and/or man pages | bat, fd, rg, sd, watchexec, zoxide |
| **D** | Binary + shared libraries (bundled) | ollama (Linux), psql, sass, syncthing |
| **E** | FHS-like layout (`bin/`, `share/man/`) | gh, pandoc |
| **F** | Renamed binary needing install-time rename | pathman, yq |
| **G** | Full SDK/toolchain (many files) | go, node, zig, flutter, julia |
| **H** | .NET runtime bundle | pwsh |
| **I** | Multi-binary distribution | dashcore, mutagen |
**Pattern A** is by far the most common (~28 packages). When in doubt,
download the archive and `tar -tz` it before writing a single line of code.
---
## 3. Write `install.sh`
The framework (`_webi/package-install.tpl.sh`) handles: user-agent detection,
version resolution, download, checksum verification, and PATH management.
Your script is **injected into** the framework and provides the
package-specific part: where to find the binary and how to move it.
### Script structure
Every `install.sh` wraps its definitions in an `__init_pkgname()` function
and immediately calls it. This prevents variable leakage when the script is
sourced by the framework:
```sh
#!/bin/sh
__init_toolname() {
set -e
set -u
####################
# Install toolname #
####################
pkg_cmd_name="toolname"
WEBI_SINGLE=true # if applicable — see below
pkg_dst_cmd="$HOME/.local/bin/toolname"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/toolname-v$WEBI_VERSION/bin/toolname"
pkg_src_dir="$HOME/.local/opt/toolname-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
# ...
}
pkg_get_current_version() {
# ...
}
}
__init_toolname
```
### Variables
| Variable | Description |
|----------|-------------|
| `pkg_cmd_name` | The command name that ends up on `$PATH` |
| `pkg_dst_cmd` | Final destination: `~/.local/bin/<cmd>` (the symlink) |
| `pkg_dst` | Same as `pkg_dst_cmd` for single-binary packages; `~/.local/opt/<cmd>` for SDKs |
| `pkg_src_cmd` | Versioned binary: `~/.local/opt/<pkg>-v<ver>/bin/<cmd>` |
| `pkg_src_dir` | Versioned install dir: `~/.local/opt/<pkg>-v<ver>` |
| `pkg_src` | Same as `pkg_src_cmd` for single-binary packages; same as `pkg_src_dir` for SDKs |
**Framework-derived (set by the framework before calling `pkg_install` — do not set manually):**
- `pkg_src_bin``$(dirname "$pkg_src_cmd")` — the versioned `bin/` dir
- `pkg_dst_bin``$(dirname "$pkg_dst_cmd")``~/.local/bin`
### `WEBI_SINGLE`
`WEBI_SINGLE=true` affects the default values the framework uses for
`pkg_src` and `pkg_dst`, and how `webi_link()` creates the symlink:
- **With `WEBI_SINGLE=true`**: links the binary file directly:
`~/.local/bin/cmd → ~/.local/opt/cmd-vX.Y.Z/bin/cmd`
- **Without it (default)**: links the directory:
`~/.local/opt/cmd → ~/.local/opt/cmd-vX.Y.Z`
Set `WEBI_SINGLE=true` when using the conventional Pattern A skeleton
(where `pkg_src` and `pkg_dst` are not set to custom values). When you
explicitly assign all six variables yourself (as in Patterns BF),
`WEBI_SINGLE` is not strictly required but can still be set for clarity.
Pattern G (SDKs) and Pattern H (.NET bundles) do NOT use `WEBI_SINGLE`
they define `pkg_link()` manually because the whole directory tree must
be linked, not just a single binary.
### Required function: `pkg_install`
Moves files from the extracted archive into the versioned opt directory.
The framework has already extracted the archive into a temp directory and
`cd`'d into it before calling `pkg_install`.
```sh
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./tool-*/tool "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
### Recommended function: `pkg_get_current_version`
Used to detect whether the package is already installed at the right version:
```sh
pkg_get_current_version() {
# 'tool --version' output: "tool 1.2.3 (rev abc)"
# trim to just the version number
tool --version 2>/dev/null | head -n 1 | cut -d' ' -f2
}
```
### Skeletons by pattern
**Pattern A** — binary at archive root (`WEBI_SINGLE=true`):
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./"$pkg_cmd_name"* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
Use `$pkg_cmd_name*` as the glob — it matches the binary and avoids
accidentally moving LICENSE or README into the binary path.
**Pattern B** — binary inside a `tool-{ver}-{triplet}/` subdirectory:
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./tool-*/tool "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
**Pattern C** — like B, plus completions and man pages.
The completion directory and filename vary per package — always check
`tar -tz` output first. Common variants: `completions/`, `autocomplete/`,
`complete/`. See [`references/PATTERNS.md`](references/PATTERNS.md) for
a full example with guards:
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./tool-*/tool "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
# bash completion (directory name varies — check tar -tz)
if test -e ./tool-*/completions/tool.bash; then
mkdir -p "$pkg_src_dir/share/bash-completion/completions"
mv ./tool-*/completions/tool.bash \
"$pkg_src_dir/share/bash-completion/completions/tool"
fi
if test -e ./tool-*/completions/tool.fish; then
mkdir -p "$pkg_src_dir/share/fish/vendor_completions.d"
mv ./tool-*/completions/tool.fish \
"$pkg_src_dir/share/fish/vendor_completions.d/tool.fish"
fi
if test -e ./tool-*/completions/_tool; then
mkdir -p "$pkg_src_dir/share/zsh/site-functions"
mv ./tool-*/completions/_tool \
"$pkg_src_dir/share/zsh/site-functions/_tool"
fi
if test -e ./tool-*/tool.1; then
mkdir -p "$pkg_src_dir/share/man/man1"
mv ./tool-*/tool.1 "$pkg_src_dir/share/man/man1/tool.1"
fi
}
```
**Pattern D** — binary + shared libraries. The entire directory structure
must be preserved. See [`references/PATTERNS.md`](references/PATTERNS.md)
for the ollama and psql examples.
**Pattern E** — FHS layout (archive already has `bin/` and `share/`):
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./tool-*/ "$pkg_src_dir"
}
```
**Pattern F** — binary needs rename (archive name ≠ command name).
Use when the binary in the archive cannot be matched by `$pkg_cmd_name*`
— e.g., `yq_linux_amd64` for a command named `yq`:
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./yq_* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
**Pattern G** — full SDK (do NOT set `WEBI_SINGLE`):
```sh
# pkg_src = directory, not a binary
pkg_src="$pkg_src_dir"
pkg_dst="$HOME/.local/opt/tool"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./tool-*/ "$pkg_src_dir"
}
pkg_link() {
rm -f "$pkg_dst"
ln -s "$pkg_src" "$pkg_dst"
}
```
---
## 4. Write `install.ps1`
A PowerShell framework template exists (`_webi/package-install.tpl.ps1`)
and injects the `install.ps1` script at the `# {{ installer }}` placeholder.
The template provides: error handling, directory setup, `Invoke-DownloadUrl`
helper, and PATH management via `webi_path_add`. However, unlike the shell
side, the PS1 framework does **not** download or extract the archive — the
package script must handle that itself. The same path conventions apply
(opt/bin layout), but Windows uses `Copy-Item` instead of symlinks for
the final `bin/` step.
### Variable block (always at top)
```powershell
$pkg_cmd_name = "tool"
$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\tool.exe"
$pkg_dst_bin = "$Env:USERPROFILE\.local\bin"
$pkg_dst = "$pkg_dst_cmd"
$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\tool-v$Env:WEBI_VERSION\bin\tool.exe"
$pkg_src_bin = "$Env:USERPROFILE\.local\opt\tool-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\tool-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
```
### Standard body
```powershell
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$pkg_download")) {
Write-Output "Downloading tool from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing tool"
Push-Location .local\tmp
Remove-Item -Path ".\tool-v*" -Recurse -ErrorAction Ignore
# Unpack — Windows BSD-tar handles zip too
Write-Output "Unpacking $pkg_download"
& tar xf "$pkg_download"
# Move binary into place — adjust glob for your archive structure
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
Move-Item -Path ".\tool-*\tool.exe" -Destination "$pkg_src_bin"
Pop-Location
}
# Windows has no symlinks in the webi sense — copy to bin/
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
New-Item "$pkg_dst_bin" -ItemType Directory -Force | Out-Null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse
```
For Pattern A (binary at archive root), change the `Move-Item` line to:
```powershell
Move-Item -Path ".\tool.exe" -Destination "$pkg_src_bin"
```
---
## 5. Check for classification issues
Before writing any scripts, scan the asset list for red flags:
### Non-standard OS/arch names in filenames
The webi classifier recognises most patterns automatically. Watch for:
- `darwin` vs `macos` — both recognised; output normalised to `macos`
- `x86_64` vs `amd64` — both recognised; output normalised to `amd64`
- `aarch64` vs `arm64` — both recognised; output normalised to `arm64`
- `armv7` (missing trailing `l`) — normalised to `armv7l`
These are handled automatically. Only flag them if the asset list contains
something genuinely unusual that the classifier would not recognise.
### Variant assets needing tags
Flag if you see multiple assets for the same OS/arch that serve different
hardware or runtime requirements:
- **GPU variants**: `*-rocm*`, `*-cuda*`, `*-vulkan*` alongside a baseline build
- **Windows installer**: `*Setup.exe` or `*Install.exe` alongside a bare `*.exe`
- **Framework-dependent .NET**: `*-fxdependent*` vs self-contained
- **AppImage**: `*.AppImage` — not supported by the webi installer
- **Electron/GUI app**: `*.dmg` or `*.AppImage` that is a full GUI app, not a CLI
If you find variants, see [`references/CLASSIFICATION.md`](references/CLASSIFICATION.md)
for how to write a variant tagger.
### Formats to drop
These are automatically filtered by the framework — no action needed:
- `.deb`, `.rpm`, `.snap`, `.AppImage`
- Checksums (`*.sha256`, `*.sha512`, `*.asc`, `*.sig`)
- Source archives (`*-src.tar.gz`, `*.tar.gz` with no OS in name)
---
## Reference files
- [`references/PATTERNS.md`](references/PATTERNS.md) — detailed pattern
descriptions with real package examples and complete install script snippets
- [`references/ARCHIVE-LAYOUTS.md`](references/ARCHIVE-LAYOUTS.md) — actual
`tar -t` output for representative packages in each pattern
- [`references/CLASSIFICATION.md`](references/CLASSIFICATION.md) — when and
how to write variant taggers; non-standard filename conventions

View File

@@ -1,289 +0,0 @@
# Archive Layouts — Real Package Examples
Actual `tar -t` / `unzip -l` output for representative packages.
Use these to calibrate your eye for what each pattern looks like.
---
## Pattern A — Flat archive (no subdirectory)
### caddy 2.9.1 — linux/amd64 tar.gz
```
caddy
LICENSE
README.md
```
Binary `caddy` is at the top level. Set `WEBI_SINGLE=true`.
### fzf 0.70.0 — linux/amd64 tar.gz
```
fzf
```
Minimal — just the binary.
### terraform 1.9.8 — linux/amd64 zip
```
terraform
LICENSE.txt
```
Zip archive but same flat layout.
### k9s — linux/amd64 tar.gz
```
k9s
LICENSE
README.md
```
---
## Pattern B — Named subdirectory, binary only
### delta 0.18.2 — linux/amd64 tar.gz
```
delta-0.18.2-x86_64-unknown-linux-musl/
delta-0.18.2-x86_64-unknown-linux-musl/delta
delta-0.18.2-x86_64-unknown-linux-musl/LICENSE
delta-0.18.2-x86_64-unknown-linux-musl/README.md
```
Glob to move: `./delta-*/delta`
### shellcheck 0.10.0 — linux/x86_64 tar.xz
```
shellcheck-v0.10.0/
shellcheck-v0.10.0/shellcheck
shellcheck-v0.10.0/LICENSE.txt
shellcheck-v0.10.0/README.txt
```
Glob to move: `./shellcheck-*/shellcheck`
### xsv 0.13.0 — linux/x86_64 tar.gz
```
xsv-0.13.0-x86_64-unknown-linux-musl/
xsv-0.13.0-x86_64-unknown-linux-musl/xsv
xsv-0.13.0-x86_64-unknown-linux-musl/UNLICENSE
```
---
## Pattern C — Subdirectory + completions + man pages
### rg/ripgrep 14.1.1 — linux/amd64 tar.gz
```
ripgrep-14.1.1-x86_64-unknown-linux-musl/
ripgrep-14.1.1-x86_64-unknown-linux-musl/rg
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/_rg
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/_rg.ps1
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/rg.bash
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/rg.fish
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/rg.1
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/FAQ.md
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/GUIDE.md
ripgrep-14.1.1-x86_64-unknown-linux-musl/CHANGELOG.md
ripgrep-14.1.1-x86_64-unknown-linux-musl/LICENSE-MIT
ripgrep-14.1.1-x86_64-unknown-linux-musl/README.md
```
Note: completions are in `complete/` (not `completions/`). Man page is `doc/rg.1`.
### sd 1.1.0 — linux/x86_64 tar.gz
```
sd-v1.1.0-x86_64-unknown-linux-musl/
sd-v1.1.0-x86_64-unknown-linux-musl/sd
sd-v1.1.0-x86_64-unknown-linux-musl/sd.1
sd-v1.1.0-x86_64-unknown-linux-musl/completions/
sd-v1.1.0-x86_64-unknown-linux-musl/completions/sd.bash
sd-v1.1.0-x86_64-unknown-linux-musl/completions/sd.elv
sd-v1.1.0-x86_64-unknown-linux-musl/completions/sd.fish
sd-v1.1.0-x86_64-unknown-linux-musl/completions/_sd
sd-v1.1.0-x86_64-unknown-linux-musl/completions/_sd.ps1
sd-v1.1.0-x86_64-unknown-linux-musl/CHANGELOG.md
sd-v1.1.0-x86_64-unknown-linux-musl/LICENSE
sd-v1.1.0-x86_64-unknown-linux-musl/README.md
```
Note: man page `sd.1` is at subdirectory root. Completions in `completions/`.
### bat 0.26.1 — linux/amd64 tar.gz
```
bat-v0.26.1-x86_64-unknown-linux-musl/
bat-v0.26.1-x86_64-unknown-linux-musl/bat
bat-v0.26.1-x86_64-unknown-linux-musl/bat.1
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/bat.bash
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/bat.fish
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/bat.zsh
bat-v0.26.1-x86_64-unknown-linux-musl/LICENSE-APACHE
bat-v0.26.1-x86_64-unknown-linux-musl/LICENSE-MIT
bat-v0.26.1-x86_64-unknown-linux-musl/README.md
```
Note: completions in `autocomplete/` (not `completions/`). Zsh file is `bat.zsh` not `_bat`.
### goreleaser — linux/amd64 tar.gz
```
goreleaser
completions/
completions/goreleaser.bash
completions/goreleaser.fish
completions/goreleaser.zsh
manpages/
manpages/goreleaser.1.gz
LICENSE.md
README.md
```
Note: goreleaser uses Pattern A layout (binary at root, no subdirectory)
but includes completions and a gzipped man page. Set `WEBI_SINGLE=true`;
move completions and man page after the binary.
---
## Pattern D — Binary + shared libraries
### ollama 0.17.7 — linux/amd64 tar.zst
```
bin/
bin/ollama
lib/
lib/ollama/
lib/ollama/libggml-base.so
lib/ollama/libggml-cpu-alderlake.so
lib/ollama/libggml-cpu-haswell.so
lib/ollama/libggml-cpu-icelake.so
lib/ollama/libggml-cpu-sandybridge.so
lib/ollama/libggml-cpu-skylakex.so
lib/ollama/libggml-cpu-sse42.so
lib/ollama/libggml-cpu-x64.so
lib/ollama/cuda_v12/
lib/ollama/cuda_v12/libcublas.so.12
lib/ollama/cuda_v12/libcublasLt.so.12
lib/ollama/cuda_v12/libcudart.so.12
lib/ollama/cuda_v12/libggml-cuda.so
... (66 files total)
```
Extract bin/ and lib/ directories separately or together.
### psql (postgres client) — linux/amd64 tar.gz
```
psql-17.2-linux-x86_64/
psql-17.2-linux-x86_64/bin/
psql-17.2-linux-x86_64/bin/psql
psql-17.2-linux-x86_64/lib/
psql-17.2-linux-x86_64/lib/libpq.so.5
psql-17.2-linux-x86_64/lib/libz.so.1
psql-17.2-linux-x86_64/lib/libzstd.so.1
psql-17.2-linux-x86_64/lib/libssl.so.3
psql-17.2-linux-x86_64/lib/libcrypto.so.3
psql-17.2-linux-x86_64/include/
... (75 files total)
```
Move the entire `psql-{ver}-{triplet}/` directory: `mv ./psql-*/ "$pkg_src_dir"`
---
## Pattern E — FHS layout
### gh 2.67.0 — linux/amd64 tar.gz
```
gh_2.67.0_linux_amd64/
gh_2.67.0_linux_amd64/bin/
gh_2.67.0_linux_amd64/bin/gh
gh_2.67.0_linux_amd64/share/
gh_2.67.0_linux_amd64/share/man/
gh_2.67.0_linux_amd64/share/man/man1/
gh_2.67.0_linux_amd64/share/man/man1/gh-actions-cache-delete.1
gh_2.67.0_linux_amd64/share/man/man1/gh-actions-cache-list.1
... (129 man pages)
gh_2.67.0_linux_amd64/LICENSE
```
Move the entire `gh_*/` directory: `mv ./gh_*/ "$pkg_src_dir"`
---
## Pattern F — Binary needs rename
### yq — linux/amd64 tar.gz (WEBI_SINGLE=true)
```
yq_linux_amd64
yq.1
```
Binary is `yq_linux_amd64` — must rename to `yq` during install.
### pathman 0.6.0 — linux/amd64 tar.gz (WEBI_SINGLE=true)
```
pathman-v0.6.0-linux-amd64_v1
```
Binary name includes the full release tag. Rename to `pathman`.
---
## Pattern G — Full SDK
### node 24.14.0 — linux/amd64 tar.xz
```
node-v24.14.0-linux-x64/
node-v24.14.0-linux-x64/bin/
node-v24.14.0-linux-x64/bin/node
node-v24.14.0-linux-x64/bin/npm -> ../lib/node_modules/npm/bin/npm-cli.js
node-v24.14.0-linux-x64/bin/npx -> ../lib/node_modules/npm/bin/npx-cli.js
node-v24.14.0-linux-x64/include/
node-v24.14.0-linux-x64/lib/
node-v24.14.0-linux-x64/lib/node_modules/
node-v24.14.0-linux-x64/share/
... (thousands of files)
```
Move entire directory: `mv ./node-*/ "$pkg_src_dir"`
### go 1.24.1 — linux/amd64 tar.gz
```
go/
go/bin/
go/bin/go
go/bin/gofmt
go/src/
go/pkg/
... (thousands of files)
```
Note: go's archive root directory is literally `go/` with no version in the name.
---
## Pattern H — .NET runtime bundle
### pwsh 7.4.6 — linux/amd64 tar.gz
```
pwsh
Accessibility.dll
clrcompression.dll
clrjit.dll
createdump
cs/
cs/System.Private.CoreLib.resources.dll
de/
de/System.Private.CoreLib.resources.dll
... (727 files, all in same flat directory)
```
No subdirectory. Move all files into `$pkg_src_bin/`.
---
## Inspecting archives yourself
```sh
# tar.gz / tar.xz / tar.zst — list contents only (no extraction)
curl -fsSL "$URL" | tar -tz | head -20
# zip
curl -fsSL "$URL" -o /tmp/pkg.zip
unzip -l /tmp/pkg.zip | head -20
# For a .zst file when tar doesn't support zstd natively:
curl -fsSL "$URL" -o /tmp/pkg.tar.zst && zstd -dc /tmp/pkg.tar.zst | tar -tz | head -20
```
**What to look for**:
1. Is there a top-level directory? (Pattern B/C/D/E/G) or no directory? (Pattern A/F/H)
2. What is the directory named? Does it contain version? triplet?
3. Are there `completions/`, `autocomplete/`, `complete/` subdirs? (Pattern C)
4. Are there `.so`/`.dylib`/`.dll` files? (Pattern D or H)
5. Does the binary name match the command you want on PATH? (Pattern F if not)
6. Is there a `bin/` directory at the top level? (Pattern E or G)

View File

@@ -1,183 +0,0 @@
# Classification Reference
When to flag classification issues, what the webi classifier does automatically,
and what needs manual annotation.
---
## What the classifier handles automatically
The webi classifier (`internal/classify/classify.go`) parses asset filenames
using regex patterns and produces canonical `os`, `arch`, `libc`, and `ext`
values. It handles the vast majority of packages with no configuration needed.
### OS recognition
Filenames containing these terms are classified automatically:
- `darwin`, `macos`, `osx`, `apple``macos` in legacy cache
- `linux``linux`
- `windows`, `win`, `win32`, `win64``windows`
- `freebsd`, `openbsd`, `netbsd`, `dragonfly` → respective values
- `.deb`, `.rpm`, `.snap``linux` (but dropped from legacy cache)
- `.dmg`, `.app.zip``macos`
### Arch recognition
Filenames containing these terms are classified automatically:
- `x86_64`, `amd64`, `64bit`, `x64``amd64`
- `aarch64`, `arm64``arm64`
- `armv7`, `armv7l`, `armhf`, `gnueabihf``armv7l`
- `armv6`, `armv6l``armv6l`
- `i386`, `i686`, `386`, `x86``x86`
- `universal`, `universal2``amd64` (fat binary; arm64 falls back to this)
### Format recognition
- `.tar.gz`, `.tar.xz`, `.tar.zst`, `.tar.bz2`, `.zip`, `.7z` → compressed archive
- `.pkg`, `.msi`, `.dmg` → platform installer
- `.exe` → either bare binary or GUI installer (see below)
- No extension in filename → bare binary (ext = `exe` in cache)
### Automatically dropped
These asset types are recognised and excluded without any configuration:
- Checksums: `*.sha256`, `*.sha512`, `*.md5`, `*.sha256sum`
- Signatures: `*.asc`, `*.sig`, `*.cosign`, `*.sbom`
- Source archives: files with `source`, `src` in the name but no OS
- Package formats not supported by the Node installer: `.deb`, `.rpm`, `.snap`,
`.AppImage`, `.apk`
---
## When you need to add configuration
### Variant assets
A **variant** is a secondary build that serves the same OS/arch as a baseline
build but requires different hardware or runtime support. The Node.js installer
can't choose between variants — it only knows OS, arch, and libc. Variants
must be tagged and then excluded at export time.
**Common variants and how to identify them**:
| Variant | Filename pattern | Notes |
|---------|-----------------|-------|
| CUDA (GPU) | `*-cuda*`, `*cuda12*` | NVIDIA GPU support |
| ROCm (GPU) | `*-rocm*` | AMD GPU support |
| Vulkan | `*-vulkan*` | Cross-vendor GPU |
| AppImage | `*.AppImage` | Linux sandboxed app |
| .NET fxdependent | `*-fxdependent*` | Requires .NET runtime |
| Windows installer | `*Setup.exe`, `*Install.exe` | GUI installer, not the binary |
**Rule**: if there are multiple assets for the same OS/arch combination and
they serve the same users differently, they need variant tags. The baseline
(most widely compatible) build should be kept; variants should be tagged and
excluded.
**Example**: ollama publishes for linux/amd64:
- `ollama-linux-amd64.tar.zst` — baseline (CPU + any GPU auto-detected)
- `ollama-linux-amd64-rocm.tar.zst` — ROCm variant
- `ollama-linux-amd64-jetpack6.tar.zst` — NVIDIA Jetson variant
Only the baseline is useful via webi. The ROCm and Jetpack builds should be
tagged as variants and excluded.
---
### Windows .exe: bare binary vs GUI installer
`.exe` assets are ambiguous — they could be:
1. A bare binary (the tool itself, run from command line)
2. A GUI installer (runs a setup wizard, not useful for webi)
**How to tell**:
- GUI installer: filename contains `Setup`, `Install`, `Installer`, `inno`, `nsis`
- GUI installer: the tool also has a `.zip` or `.tar.gz` for Windows
- Bare binary: filename matches the tool name with minimal decoration
**When you see both**, the `.zip`/archive build is what webi uses. The `.exe`
installer should be tagged as a variant (`installer`) so it's excluded.
**When there's only a `.exe`** (no archive), it's probably the bare binary.
Test by downloading and running it — a bare binary runs immediately.
---
### Packages with no OS/arch in filenames
Some packages (rare) release with minimal filename decoration. Examples:
- `tool-v1.2.3.tar.gz` — no OS, no arch
- `tool.tar.gz` — version not even in filename
These are usually source archives (not compiled binaries) and should be
dropped entirely from the release list. If they are compiled binaries for a
specific OS, the releases.js config needs an `asset_filter` key to match the
right file, plus OS/arch metadata added.
---
### Non-standard OS naming in filenames
A few upstreams use unusual OS names:
- `sunos` — should map to `solaris` (the webi classifier does this)
- `osx` or `macosx` — recognised as `macos`
- `apple-darwin` (Rust triplet) — recognised as `macos`
If a package uses a genuinely unknown OS string, the classifier will produce
`os = ""` for that asset. Those entries are dropped from the legacy cache.
---
### Asset filter configuration
If GitHub releases for a package include multiple builds that would otherwise
collide (e.g. `extended` vs non-extended for hugo, or specific project builds
in a monorepo), add to the package's `releases.conf`:
```ini
# Only include assets containing "extended" in the name
asset_filter = extended
# Exclude assets containing "legacy" in the name
asset_exclude = legacy
```
These filters run before classification.
---
## Quick checklist when inspecting a new package
1. **Look at the latest 23 releases** on GitHub. Note all asset filenames.
2. **Find the "standard" builds** — the ones a normal user would download for
their OS. Usually there are ≤4 per OS (amd64, arm64, x86, armv7l).
3. **Check for extras**:
- Are there GPU-specific builds for the same OS/arch? → variant
- Are there `.exe` installer files alongside a `.zip`? → variant
- Are there `.deb`/`.rpm`/`.AppImage`? → auto-dropped, no action needed
- Does the Windows build have no archive and only a bare `.exe`? → fine
4. **Check OS/arch naming** — does the filename use standard terms, or
something unusual that might confuse the classifier?
5. **Check format changes** — do old releases use a different archive type
or directory layout than recent ones? The install script may need to
handle both.
---
## Canonical vocabulary reference
All cache output must use exactly these values.
**OS**: `macos`, `linux`, `windows`, `freebsd`, `openbsd`, `netbsd`,
`dragonfly`, `aix`, `illumos`, `plan9`, `solaris`
**Arch**:
- `amd64` (not `x86_64`)
- `arm64` (not `aarch64`)
- `armv7l` (not `armv7` — the `l` stands for little-endian; `uname -m` reports `armv7l`)
- `armv6l` (not `armv6`)
- `x86` (not `i386`, `i686`, `386`)
- `mipsle` (not `mipsel`)
- `mips64le` (not `mips64el`)
- Other: `arm`, `ppc64le`, `ppc64`, `loong64`, `riscv64`, `s390x`, `mips`, `mips64`
**Libc**: `none` (static/Go/Zig — never empty), `gnu`, `musl`, `msvc`
**Ext**: `tar.gz`, `tar.xz`, `zip`, `exe`, `7z`, `pkg`, `msi`
(no leading dot; `exe` for bare binaries with no file extension)

View File

@@ -1,388 +0,0 @@
# Install Patterns Reference
Nine patterns cover the full range of webi packages. Pattern A is by far
the most common. Check `tar -tz $ARCHIVE` before writing any code.
---
## Pattern A — Bare binary at archive root
The archive extracts directly to the current directory with no wrapper
subdirectory. Binary (and optional LICENSE/README) is at the top level.
**Set `WEBI_SINGLE=true`** — tells the framework to link the binary file
directly (`~/.local/bin/cmd → ~/.local/opt/cmd-vX/bin/cmd`) rather than
linking the versioned directory.
Representative packages: caddy, fzf, k9s, terraform, sttr, lf, monorel,
awless, bun, cilium, curlie, dashmsg, dotenv, dotenv-linter, ffuf,
gitdeploy, gprox, grype, hugo, keypairs, koji, ots, runzip, sclient,
sqlc, sqlpkg, uuidv7, xcaddy, deno
**install.sh**:
```sh
pkg_cmd_name="caddy"
WEBI_SINGLE=true
pkg_dst_cmd="$HOME/.local/bin/caddy"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/caddy-v$WEBI_VERSION/bin/caddy"
pkg_src_dir="$HOME/.local/opt/caddy-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./"$pkg_cmd_name"* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
pkg_get_current_version() {
caddy version 2>/dev/null | head -n 1 | cut -d' ' -f1 | sed 's:^v::'
}
```
**install.ps1** key lines:
```powershell
# No subdirectory — binary is at the top level of the archive
Move-Item -Path ".\caddy.exe" -Destination "$pkg_src_bin"
```
---
## Pattern B — Binary inside a version/triplet subdirectory
Archive extracts to a single directory named with the version and/or
platform triplet. Binary (and docs) live inside that directory.
Representative packages: delta, hexyl, shellcheck, trip, xsv, kubectx, kubens
**Subdirectory naming conventions seen in the wild**:
- `tool-{ver}-{triplet}/` — most Rust tools (delta, shellcheck, xsv)
- `tool-{ver}/` — simpler version-only dirs
- flat (no dir) — kubectx/kubens use flat archives despite being "B-ish"
**install.sh**:
```sh
pkg_cmd_name="delta"
# WEBI_SINGLE not set (or false)
pkg_dst_cmd="$HOME/.local/bin/delta"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/delta-v$WEBI_VERSION/bin/delta"
pkg_src_dir="$HOME/.local/opt/delta-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./delta-*/delta "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
pkg_get_current_version() {
delta --version 2>/dev/null | head -n 1 | cut -d' ' -f2
}
```
**install.ps1** key lines:
```powershell
Move-Item -Path ".\delta-*\delta.exe" -Destination "$pkg_src_bin"
```
---
## Pattern C — Subdirectory with binary + completions and/or man pages
Same as B but the archive also contains shell completions and/or man pages
worth installing.
Representative packages: bat, fd, lsd, rg/ripgrep, sd, watchexec, zoxide
Note: goreleaser has a flat archive (Pattern A layout) but with completions at
the archive root. See the goreleaser entry in ARCHIVE-LAYOUTS.md.
**Completion directory name varies by package**:
- `completions/` — sd, watchexec, zoxide
- `autocomplete/` — bat, fd, lsd
- `complete/` — rg/ripgrep
**Completion filename conventions**:
- Bash: `tool.bash`, `tool.bash-completion`, `_tool.bash`
- Fish: `tool.fish`
- Zsh: `_tool`
- PowerShell: `_tool.ps1`, `tool.ps1`
**Man page location varies**:
- `tool.1` at subdirectory root — sd, bat, fd, lsd
- `doc/tool.1` — rg/ripgrep
- `man/man1/tool.1` — zoxide (deepest path)
**install.sh** (rg as example):
```sh
pkg_cmd_name="rg"
pkg_dst_cmd="$HOME/.local/bin/rg"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/rg-v$WEBI_VERSION/bin/rg"
pkg_src_dir="$HOME/.local/opt/rg-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./ripgrep-*/rg "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
# bash completion
if test -e ./ripgrep-*/complete/rg.bash; then
mkdir -p "$pkg_src_dir/share/bash-completion/completions"
mv ./ripgrep-*/complete/rg.bash \
"$pkg_src_dir/share/bash-completion/completions/rg"
fi
# fish completion
if test -e ./ripgrep-*/complete/rg.fish; then
mkdir -p "$pkg_src_dir/share/fish/vendor_completions.d"
mv ./ripgrep-*/complete/rg.fish \
"$pkg_src_dir/share/fish/vendor_completions.d/rg.fish"
fi
# zsh completion
if test -e ./ripgrep-*/complete/_rg; then
mkdir -p "$pkg_src_dir/share/zsh/site-functions"
mv ./ripgrep-*/complete/_rg \
"$pkg_src_dir/share/zsh/site-functions/_rg"
fi
# man page
if test -e ./ripgrep-*/doc/rg.1; then
mkdir -p "$pkg_src_dir/share/man/man1"
mv ./ripgrep-*/doc/rg.1 "$pkg_src_dir/share/man/man1/rg.1"
fi
}
pkg_get_current_version() {
rg --version 2>/dev/null | head -n 1 | cut -d' ' -f2
}
```
**Note**: Completion paths in completions/man install are best-effort
— use `if test -e ...` guards so the script still works on older releases
that didn't include them.
---
## Pattern D — Binary + shared libraries
The package bundles shared libraries alongside the binary. The entire
directory tree must be preserved.
Representative packages: ollama (Linux), psql/postgres, sass (Dart VM),
syncthing, xz
**install.sh**:
```sh
pkg_cmd_name="ollama"
pkg_dst_cmd="$HOME/.local/bin/ollama"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/ollama-v$WEBI_VERSION/bin/ollama"
pkg_src_dir="$HOME/.local/opt/ollama-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
# Archive already has bin/ and lib/ layout
mv ./bin "$pkg_src_dir/bin"
mv ./lib "$pkg_src_dir/lib"
}
```
For psql (archive has a `psql-{ver}-{triplet}/` wrapper dir):
```sh
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./psql-*/ "$pkg_src_dir"
}
```
---
## Pattern E — FHS-like layout
Archive already follows `bin/`, `share/man/`, `share/doc/` hierarchy.
Extract the whole thing directly into the versioned opt directory.
Representative packages: gh (GitHub CLI), pandoc
**install.sh**:
```sh
pkg_cmd_name="gh"
pkg_dst_cmd="$HOME/.local/bin/gh"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/gh-v$WEBI_VERSION/bin/gh"
pkg_src_dir="$HOME/.local/opt/gh-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./gh_*/ "$pkg_src_dir"
}
pkg_get_current_version() {
gh --version 2>/dev/null | head -n 1 | cut -d' ' -f3
}
```
No `chmod` needed — binary is already executable inside the archive.
---
## Pattern F — Binary needs rename
Binary in the archive doesn't match the expected command name.
Representative packages: pathman (`pathman-v0.6.0-linux-amd64_v1``pathman`),
yq (`yq_linux_amd64``yq`)
**install.sh**:
```sh
pkg_cmd_name="yq"
WEBI_SINGLE=true
pkg_dst_cmd="$HOME/.local/bin/yq"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/yq-v$WEBI_VERSION/bin/yq"
pkg_src_dir="$HOME/.local/opt/yq-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
# Binary is named yq_linux_amd64 (or yq_darwin_amd64 etc)
mv ./yq_* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
---
## Pattern G — Full SDK / toolchain
Archive contains a complete runtime or SDK (hundreds to thousands of files).
The entire tree goes into opt; multiple binaries are linked from `bin/`.
Representative packages: go, node, zig, flutter, julia, cmake, tinygo
**install.sh** (node as example):
```sh
pkg_cmd_name="node"
# NOTE: pkg_src points to the directory, not a binary
pkg_dst_cmd="$HOME/.local/bin/node"
pkg_dst="$HOME/.local/opt/node" # versioned-dir symlink target
pkg_src_cmd="$HOME/.local/opt/node-v$WEBI_VERSION/bin/node"
pkg_src_dir="$HOME/.local/opt/node-v$WEBI_VERSION"
pkg_src="$pkg_src_dir" # pkg_src = the directory
pkg_install() {
mkdir -p "$(dirname "$pkg_src")"
mv ./node-*/ "$pkg_src"
}
pkg_link() {
rm -f "$pkg_dst"
ln -s "$pkg_src" "$pkg_dst"
}
pkg_get_current_version() {
node --version 2>/dev/null | head -n 1 | sed 's:^v::'
}
```
---
## Pattern H — .NET runtime bundle
Flat directory with one binary and hundreds of `.dll` files. The entire
directory must be preserved. Like Pattern G (SDK) in structure — the
versioned directory is the package root, with the binary directly inside
(no `bin/` subdirectory). A `pkg_link()` creates the unversioned symlink.
Representative packages: pwsh (PowerShell Core)
**install.sh**:
```sh
pkg_cmd_name="pwsh"
# note: binary is at pkg_src_dir root, no bin/ subdirectory
pkg_src_cmd="$HOME/.local/opt/pwsh-v$WEBI_VERSION/pwsh"
pkg_src_dir="$HOME/.local/opt/pwsh-v$WEBI_VERSION"
pkg_src="$pkg_src_dir"
pkg_dst_cmd="$HOME/.local/opt/pwsh/pwsh"
pkg_dst="$HOME/.local/opt/pwsh"
pkg_install() {
# Archive extracts flat — move all contents into the versioned dir
mkdir -p "$pkg_src_dir"
mv ./* "$pkg_src_dir"
chmod a+x "$pkg_src_cmd"
}
pkg_link() {
rm -rf "$pkg_dst"
ln -s "$pkg_src" "$pkg_dst"
}
```
---
## Pattern I — Multi-binary distribution
Archive contains multiple related binaries. Install the primary one and
link only that.
Representative packages: dashcore (dashd + dash-cli + dash-qt + ...),
mutagen (mutagen + mutagen-agents.tar.gz)
**install.sh** (dashcore-style):
```sh
pkg_cmd_name="dashd"
pkg_dst_cmd="$HOME/.local/bin/dashd"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/dashcore-v$WEBI_VERSION/bin/dashd"
pkg_src_dir="$HOME/.local/opt/dashcore-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./dashcore-*/ "$pkg_src_dir"
}
```
---
## Choosing between patterns
```
Archive root contains a single binary (or binary + docs)?
→ Pattern A (set WEBI_SINGLE=true)
Archive has a named subdirectory wrapping the binary?
├─ Binary only inside subdir? → Pattern B
├─ Binary + completions/man pages? → Pattern C
└─ Binary + shared libraries (.so)? → Pattern D
Archive already has bin/ and share/ layout?
→ Pattern E
Binary name doesn't match the command name?
→ Pattern F (rename during install)
Archive is a full SDK (compiler, runtime, stdlib)?
→ Pattern G (pkg_src = pkg_src_dir)
Flat directory with many DLLs (.NET)?
→ Pattern H
Multiple binaries for a single distributed system?
→ Pattern I
```

View File

@@ -1,7 +1,7 @@
#!/usr/bin/env pwsh
if (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
New-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start" -ItemType Directory -Force | Out-Null
IF (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
New-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start" -ItemType Directory -Force | out-null
}
Remove-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start\example" -Recurse -ErrorAction Ignore
& git clone --depth=1 https://github.com/CHANGEME/example.git "$Env:USERPROFILE\.vim\pack\plugins\start\example"

28
_webi/bootstrap.bat Normal file
View File

@@ -0,0 +1,28 @@
setlocal
@echo off
pushd "%userprofile%" || goto :error
IF NOT EXIST .local (
mkdir .local || goto :error
)
IF NOT EXIST .local\bin (
mkdir .local\bin || goto :error
)
rem {{ baseurl }}
rem {{ version }}
pushd .local\bin || goto :error
if NOT EXIST webi.bat (
rem without SilentlyContinue this is SLOOOOOOOOOOOOOOOW!
powershell $ProgressPreference = 'SilentlyContinue'; Invoke-WebRequest https://webinstall.dev/packages/_webi/webi.bat -OutFile webi.bat || goto :error
)
call .\webi {{ exename }} || goto :error
rem pathman add "%userprofile%\.local\bin" >nul 2>&1 || goto :error
pathman add "%userprofile%\.local\bin" || goto :error
popd || goto :error
popd
goto :EOF
:error
echo Failed with error #%errorlevel%.
exit /b %errorlevel%

View File

@@ -4,9 +4,9 @@
# <h1>Cheat Sheet at CHEATSHEET_URL</h1>
# <meta http-equiv="refresh" content="3; URL='CHEATSHEET_URL'" />
############################################################
New-Item -Path "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
New-Item -Path "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | out-null
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | out-null
IF ($Env:WEBI_HOST -eq $null -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/_webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
Set-ExecutionPolicy -Scope Process Bypass
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" "{{ exename }}"

230
_webi/bootstrap.sh Normal file
View File

@@ -0,0 +1,230 @@
#!/bin/sh
#<pre>
############################################################
# <h1>Cheat Sheet at CHEATSHEET_URL</h1>
# <meta http-equiv="refresh" content="3; URL='CHEATSHEET_URL'" />
############################################################
#set -x
__install_webi() {
#WEBI_PKG=
#WEBI_HOST=https://webinstall.dev
export WEBI_HOST
echo ""
printf "Thanks for using webi to install '\e[32m%s\e[0m' on '\e[31m%s/%s\e[0m'.\n" "${WEBI_PKG:-}" "$(uname -s)" "$(uname -m)"
echo "Have a problem? Experience a bug? Please let us know:"
echo " https://github.com/webinstall/webi-installers/issues"
echo ""
printf "\e[31mLovin'\e[0m it? Say thanks with a \e[34mStar on GitHub\e[0m:\n"
printf " \e[32mhttps://github.com/webinstall/webi-installers\e[0m\n"
echo ""
WEBI_WELCOME=true
export WEBI_WELCOME
set -e
set -u
mkdir -p "$HOME/.local/bin"
cat << EOF > "$HOME/.local/bin/webi"
#!/bin/sh
set -e
set -u
#set -x
__webi_main() {
export WEBI_TIMESTAMP="\$(date +%F_%H-%M-%S)"
export _webi_tmp="\${_webi_tmp:-\$(mktemp -d -t webi-\$WEBI_TIMESTAMP.XXXXXXXX)}"
if [ -n "\${_WEBI_PARENT:-}" ]; then
export _WEBI_CHILD=true
else
export _WEBI_CHILD=
fi
export _WEBI_PARENT=true
##
## Detect acceptable package formats
##
my_ext=""
set +e
# NOTE: the order here is least favorable to most favorable
if [ -n "\$(command -v pkgutil)" ]; then
my_ext="pkg,\$my_ext"
fi
# disable this check for the sake of building the macOS installer on Linux
#if [ -n "\$(command -v diskutil)" ]; then
# note: could also detect via hdiutil
my_ext="dmg,\$my_ext"
#fi
if [ -n "\$(command -v git)" ]; then
my_ext="git,\$my_ext"
fi
if [ -n "\$(command -v unxz)" ]; then
my_ext="xz,\$my_ext"
fi
if [ -n "\$(command -v unzip)" ]; then
my_ext="zip,\$my_ext"
fi
# for mac/linux 'exe' refers to the uncompressed binary without extension
my_ext="exe,\$my_ext"
if [ -n "\$(command -v tar)" ]; then
my_ext="tar,\$my_ext"
fi
my_ext="\$(echo "\$my_ext" | sed 's/,$//')" # nix trailing comma
set -e
##
## Detect http client
##
set +e
export WEBI_CURL="\$(command -v curl)"
export WEBI_WGET="\$(command -v wget)"
set -e
export WEBI_HOST="\${WEBI_HOST:-https://webinstall.dev}"
export WEBI_UA="\$(uname -a)"
webinstall() {
my_package="\${1:-}"
if [ -z "\$my_package" ]; then
>&2 echo "Usage: webi <package>@<version> ..."
>&2 echo "Example: webi node@lts rg"
exit 1
fi
export WEBI_BOOT="\$(mktemp -d -t "\$my_package-bootstrap.\$WEBI_TIMESTAMP.XXXXXXXX")"
my_installer_url="\$WEBI_HOST/api/installers/\$my_package.sh?formats=\$my_ext"
set +e
if [ -n "\$WEBI_CURL" ]; then
curl -fsSL "\$my_installer_url" -H "User-Agent: curl \$WEBI_UA" \\
-o "\$WEBI_BOOT/\$my_package-bootstrap.sh"
else
wget -q "\$my_installer_url" --user-agent="wget \$WEBI_UA" \\
-O "\$WEBI_BOOT/\$my_package-bootstrap.sh"
fi
if ! [ \$? -eq 0 ]; then
>&2 echo "error fetching '\$my_installer_url'"
exit 1
fi
set -e
(
cd "\$WEBI_BOOT"
sh "\$my_package-bootstrap.sh"
)
rm -rf "\$WEBI_BOOT"
}
show_path_updates() {
if ! [ -n "\${_WEBI_CHILD}" ]; then
if [ -f "\$_webi_tmp/.PATH.env" ]; then
my_paths=\$(cat "\$_webi_tmp/.PATH.env" | sort -u)
if [ -n "\$my_paths" ]; then
printf 'PATH.env updated with:\\n'
printf "%s\\n" "\$my_paths"
printf '\\n'
printf "\\e[31mTO FINISH\\e[0m: copy, paste & run the following command:\\n"
printf "\\n"
printf " \\e[34msource ~/.config/envman/PATH.env\\e[0m\\n"
printf " (newly opened terminal windows will update automatically)\\n"
fi
rm -f "\$_webi_tmp/.PATH.env"
fi
fi
}
version() {
my_version=v1.1.15
printf "\\e[31mwebi\\e[32m %s\\e[0m Copyright 2020+ AJ ONeal\\n" "\${my_version}"
printf " \\e[34mhttps://webinstall.dev/webi\\e[0m\\n"
}
# show help if no params given or help flags are used
usage() {
echo ""
version
echo ""
printf "\\e[1mSUMMARY\\e[0m\\n"
echo " Webi is the best way to install the modern developer tools you love."
echo " It's fast, easy-to-remember, and conflict free."
echo ""
printf "\\e[1mUSAGE\\e[0m\\n"
echo " webi <thing1>[@version] [thing2] ..."
echo ""
printf "\\e[1mUNINSTALL\\e[0m\\n"
echo " Almost everything that is installed with webi is scoped to"
echo " ~/.local/opt/<thing1>, so you can remove it like so:"
echo ""
echo " rm -rf ~/.local/opt/<thing1>"
echo " rm -f ~/.local/bin/<thing1>"
echo ""
echo " Some packages have special uninstall instructions, check"
echo " https://webinstall.dev/<thing1> to be sure."
echo ""
printf "\\e[1mFAQ\\e[0m\\n"
printf " See \\e[34mhttps://webinstall.dev/faq\\e[0m\\n"
echo ""
printf "\\e[1mALWAYS REMEMBER\\e[0m\\n"
echo " Friends don't let friends use brew for simple, modern tools that don't need it."
echo " (and certainly not apt either **shudder**)"
echo ""
}
if [ \$# -eq 0 ] || echo "\$1" | grep -q -E '^(-V|--version|version)$'; then
version
exit 0
fi
if echo "\$1" | grep -q -E '^(-h|--help|help)$'; then
usage "\$@"
exit 0
fi
for pkgname in "\$@"
do
webinstall "\$pkgname"
done
show_path_updates
}
__webi_main "\$@"
EOF
chmod a+x "$HOME/.local/bin/webi"
if [ -n "${WEBI_PKG:-}" ]; then
"$HOME/.local/bin/webi" "${WEBI_PKG}"
else
echo ""
echo "Hmm... no WEBI_PKG was specified. This is probably an error in the script."
echo ""
echo "Please open an issue with this information: Package '${WEBI_PKG:-}' on '$(uname -s)/$(uname -m)'"
echo " https://github.com/webinstall/packages/issues"
echo ""
fi
}
__install_webi

View File

@@ -1,73 +0,0 @@
'use strict';
let Path = require('node:path');
let BuildsCacher = require('./builds-cacher.js');
// let Parallel = require('./parallel.js');
var INSTALLERS_DIR = Path.join(__dirname, '..');
var CACHE_DIR = Path.join(__dirname, '../_cache');
async function main() {
let bc = BuildsCacher.create({
caches: CACHE_DIR,
installers: INSTALLERS_DIR,
});
bc.freshenRandomPackage(600 * 1000);
// let dirs = await bc.getProjectsByType();
// let projNames = Object.keys(dirs.valid);
let lastUpdate;
let projName = 'k9s';
{
let packages = await bc.getPackages({
//Releases: Releases,
name: projName,
date: new Date(),
});
lastUpdate = packages.updated;
console.info(
`Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
);
}
console.info('Waiting 5s');
{
setTimeout(async function () {
let packages = await bc.getPackages({
//Releases: Releases,
name: projName,
date: new Date(),
});
console.info(
`Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
);
if (lastUpdate < packages.updated) {
console.info(`PASS`);
} else {
console.info(`MAYBE fail`);
}
}, 5 * 1000);
}
//let parallel = 25;
//await Parallel.run(parallel, projNames, getAll);
//async function getAll(name) {
// void (await bc.getPackages({
// //Releases: Releases,
// name: name,
// date: new Date(),
// }));
//}
}
main()
.then(function () {
console.log('Done');
})
.catch(function (e) {
console.error(e.stack || e);
process.exit(1);
});

View File

@@ -1,992 +0,0 @@
'use strict';
var BuildsCacher = module.exports;
let Fs = require('node:fs/promises');
let Path = require('node:path');
let HostTargets = require('./build-classifier/host-targets.js');
let Lexver = require('./build-classifier/lexver.js');
let Triplet = require('./build-classifier/triplet.js');
var ALIAS_RE = /^alias: (\w+)$/m;
var LEGACY_ARCH_MAP = {
'*': 'ANYARCH',
arm64: 'aarch64',
armv6l: 'armv6',
armv7l: 'armv7',
amd64: 'x86_64',
mipsle: 'mipsel',
mips64le: 'mips64el',
mipsr6le: 'mipsr6el',
mips64r6le: 'mips64r6el',
// yes... el for arm and mips, but le for ppc
// (perhaps the joke got old?)
ppc64el: 'ppc64le',
386: 'x86',
};
var LEGACY_OS_MAP = {
'*': 'ANYOS',
macos: 'darwin',
posix: 'posix_2017',
};
var TERMS_META = [
// pattern
'{ARCH}',
'{EXT}',
'{LIBC}',
'{NAME}',
'{OS}',
'{VENDOR}',
// // os-/arch-indepedent
// 'ANYARCH',
// 'ANYOS',
// // libc
// 'none',
// channel
'beta',
'dev',
'preview',
'stable',
];
/** @typedef {String} TripletString - {arch}-{vendor}-{os}-{libc} */
/** @typedef {String} VersionString */
/** @typedef {Object.<VersionString, Array<BuildAsset>>} PackagesByRelease */
/**
* @typedef ProjectInfo
* @prop {Array<BuildAsset>} releases
* @prop {Array<BuildAsset>} packages
* @prop {Object.<TripletString, PackagesByRelease>} releasesByTriplet
* @prop {Array<import('./build-classifier/types.js').ArchString>} arches
* @prop {Array<import('./build-classifier/types.js').OsString>} oses
* @prop {Array<import('./build-classifier/types.js').LibcString>} libcs
* @prop {Array<String>} channels
* @prop {Array<String>} formats
* @prop {Array<String>} triplets
* @prop {Array<String>} versions
* @prop {Array<String>} lexvers
* @prop {Object.<String, String>} lexversMap
*/
/**
* @typedef BuildAsset
* @prop {String} name
* @prop {String} version
* @prop {Boolean} lts
* @prop {String} date
* @prop {String} arch
* @prop {String} os
* @prop {String} libc
* @prop {String} ext
* @prop {String} download
*/
/**
* @typedef VersionTarget
* @prop {String} version
* @prop {Boolean} lts
* @prop {String} channel
*/
/** @typedef {TargetTriplet & HostTargetPartial} HostTarget */
/** @typedef {import('./build-classifier/types.js').TargetTriplet} TargetTriplet */
/**
* @typedef HostTargetPartial
* @prop {String} target.triplet - os-vendor-arch-libc
* @prop {Error} target.error
*/
async function getPartialHeader(path) {
let readme = `${path}/README.md`;
let head = await readFirstBytes(readme).catch(function (err) {
if (err.code !== 'ENOENT') {
console.warn(`warn: ${path}: ${err.message}`);
}
return null;
});
return head;
}
// let fsOpen = util.promisify(Fs.open);
// let fsRead = util.promisify(Fs.read);
async function readFirstBytes(path) {
let start = 0;
let n = 1024;
let fh = await Fs.open(path, 'r');
let buf = Buffer.alloc(n);
let result = await fh.read(buf, start, n);
let str = result.buffer.toString('utf8');
await fh.close();
return str;
}
let promises = {};
async function getLatestBuilds(Releases, installersDir, cacheDir, name, date) {
console.info(`[INFO] getLatestBuilds: ${name}`);
if (!Releases) {
Releases = require(`${installersDir}/${name}/releases.js`);
}
// TODO update all releases files with module.exports.xxxx = 'foo';
if (!Releases.latest) {
Releases.latest = Releases;
}
let id = `${cacheDir}/${name}`;
if (!promises[id]) {
promises[id] = Promise.resolve();
}
promises[id] = promises[id].then(async function () {
return await getLatestBuildsInner(Releases, cacheDir, name, date);
});
return await promises[id];
}
async function getLatestBuildsInner(Releases, cacheDir, name, date) {
let data = await Releases.latest();
if (!date) {
date = new Date();
}
let isoDate = date.toISOString();
let yearMonth = isoDate.slice(0, 7);
// TODO hash file
let dataFile = `${cacheDir}/${yearMonth}/${name}.json`;
// TODO fsstat releases.js vs require-ing time as well
let tsFile = `${cacheDir}/${yearMonth}/${name}.updated.txt`;
let dirPath = Path.dirname(dataFile);
await Fs.mkdir(dirPath, { recursive: true });
let json = JSON.stringify(data, null, 2);
await Fs.writeFile(dataFile, json, 'utf8');
let seconds = date.valueOf();
let ms = seconds / 1000;
let msStr = ms.toFixed(3);
await Fs.writeFile(tsFile, msStr, 'utf8');
return data;
}
BuildsCacher.create = function ({ ALL_TERMS, installers, caches }) {
let installersDir = installers;
let cacheDir = caches;
if (!ALL_TERMS) {
ALL_TERMS = Triplet.TERMS_PRIMARY_MAP;
}
let bc = {};
bc.ALL_TERMS = ALL_TERMS;
bc.orphanTerms = Object.assign({}, bc.ALL_TERMS);
bc.unknownTerms = {};
bc.usedTerms = {};
bc.formats = [];
bc._triplets = {};
bc._targetsByBuildIdCache = {};
bc._caches = {};
bc._staleAge = 15 * 60 * 1000;
bc._allFormats = {};
bc._allTriplets = {};
for (let term of TERMS_META) {
delete bc.orphanTerms[term];
}
bc.getProjectsByType = async function () {
let dirs = {
hidden: {},
errors: {},
alias: {},
invalid: {},
selfhosted: {},
valid: {},
};
let entries = await Fs.readdir(installersDir, { withFileTypes: true });
for (let entry of entries) {
let meta = await bc.getProjectTypeByEntry(entry);
if (meta.type === 'not_found') {
let err = meta.detail;
console.error('');
console.error('PROBLEM');
console.error(` ${err.message}`);
console.error('');
console.error('SOLUTION');
console.error(' npm clean-install');
console.error('');
throw new Error(
'[SANITY FAIL] should never have missing modules in prod',
);
}
dirs[meta.type][entry.name] = meta.detail;
}
return dirs;
};
/**
* Get project type and detail - alias, selfhosted, valid (and the invalids)
* @param {String} name - filename
*/
bc.getProjectType = async function (name) {
let filepath = Path.join(installersDir, name);
let entry;
try {
entry = await Fs.lstat(filepath);
Object.assign(entry, { name: name });
} catch (e) {
return { type: 'errors', detail: 'not found' };
}
let info = await bc.getProjectTypeByEntry(entry);
return info;
};
/**
* Get project type and detail - alias, selfhosted, valid (and the invalids)
* @param {fs.Stats|fs.Dirent} entry
*/
bc.getProjectTypeByEntry = async function (entry) {
let path = Path.join(installersDir, entry.name);
// skip non-installer dirs
if (entry.isSymbolicLink()) {
let link = await Fs.readlink(path);
return { type: 'alias', detail: link };
}
if (!entry.isDirectory()) {
return { type: 'hidden', detail: '!directory' };
}
if (entry.name === 'node_modules') {
return { type: 'hidden', detail: 'node_modules' };
}
if (entry.name.startsWith('_')) {
return { type: 'hidden', detail: '_*' };
}
if (entry.name.startsWith('.')) {
return { type: 'hidden', detail: '.*' };
}
if (entry.name.startsWith('~')) {
return { type: 'hidden', detail: '~*' };
}
if (entry.name.endsWith('~')) {
return { type: 'hidden', detail: '*~' };
}
// skip invalid installers
let head = await getPartialHeader(path);
if (!head) {
return { type: 'invalid', detail: '!README.md' };
}
let alias = head.match(ALIAS_RE);
if (alias) {
let link = alias[1];
return { type: 'alias', detail: link };
}
let releasesPath = Path.join(path, 'releases.js');
try {
void require(releasesPath);
} catch (err) {
if (err.code !== 'MODULE_NOT_FOUND') {
return { type: 'errors', detail: err };
}
if (err.message.includes(`Cannot find module '${releasesPath}'`)) {
return { type: 'selfhosted', detail: true };
}
return { type: 'not_found', detail: err };
}
return { type: 'valid', detail: true };
};
// Typically a package is organized by release (ex: go has 1.20, 1.21, etc),
// but we will organize by the build (ex: go1.20-darwin-arm64.tar.gz, etc).
bc.getPackages = async function ({ Releases, name, date }) {
if (!date) {
date = new Date();
}
let isoDate = date.toISOString();
let yearMonth = isoDate.slice(0, 7);
let dataFile = `${cacheDir}/${yearMonth}/${name}.json`;
let tsFile = `${cacheDir}/${yearMonth}/${name}.updated.txt`;
let tsDate;
{
let secondsStr = await Fs.readFile(tsFile, 'ascii').catch(function (err) {
if (err.code !== 'ENOENT') {
throw err;
}
return '0';
});
secondsStr = secondsStr.trim();
let seconds = parseFloat(secondsStr) || 0;
let ms = seconds * 1000;
tsDate = new Date(ms);
}
let projInfo = bc._caches[name];
let meta = {
// version info
versions: projInfo?.versions || [],
lexvers: projInfo?.lexvers || [],
lexversMap: projInfo?.lexversMap || {},
// culled release assets
packages: projInfo?.packages || [],
releasesByTriplet: projInfo?.releasesByTriplet || {},
// target info
triplets: projInfo?.triplets || [],
oses: projInfo?.oses || [],
arches: projInfo?.arches || [],
libcs: projInfo?.libcs || [],
formats: projInfo?.formats || [],
// TODO channels: projInfo?.channels || [],
};
if (!projInfo) {
let json = await Fs.readFile(dataFile, 'ascii').catch(
async function (err) {
if (err.code !== 'ENOENT') {
throw err;
}
return null;
},
);
try {
projInfo = JSON.parse(json);
} catch (e) {
console.error(`error: ${dataFile}:\n\t${e.message}`);
projInfo = null;
}
}
if (!projInfo) {
projInfo = await getLatestBuilds(Releases, installersDir, cacheDir, name);
}
let latestProjInfo = await BuildsCacher.transformAndUpdate(
name,
projInfo,
meta,
tsDate,
bc,
);
bc._caches[name] = latestProjInfo;
process.nextTick(async function () {
let now = date.valueOf();
let age = now - projInfo.updated;
let fresh = age < bc._staleAge;
if (fresh) {
return;
}
projInfo = await getLatestBuilds(Releases, installersDir, cacheDir, name);
let latestProjInfo = BuildsCacher.transformAndUpdate(
name,
projInfo,
meta,
date,
bc,
);
bc._caches[name] = latestProjInfo;
});
return projInfo;
};
// Makes sure that packages are updated once an hour, on average
bc._staleNames = [];
bc._freshenTimeout = null;
bc.freshenRandomPackage = async function (minDelay) {
if (!minDelay) {
minDelay = 15 * 1000;
}
if (bc._staleNames.length === 0) {
let dirs = await bc.getProjectsByType();
bc._staleNames = Object.keys(dirs.valid);
bc._staleNames.sort(function () {
return 0.5 - Math.random();
});
}
let name = bc._staleNames.pop();
void (await bc.getPackages({
//Releases: Releases,
name: name,
date: new Date(),
}));
console.info(`[INFO] freshenRandomPackage: ${name}`);
let hour = 60 * 60 * 1000;
let delay = minDelay;
let spread = hour / bc._staleNames.length;
let seed = Math.random();
delay += seed * spread;
clearTimeout(bc._freshenTimeout);
bc._freshenTimeout = setTimeout(bc.freshenRandomPackage, delay);
bc._freshenTimeout.unref();
};
/**
* Given a list of acceptable formats, get the sorted list of of formats.
* Actually used (as per node _webi/lint-builds.js):
* .7z
* .app.zip
* .dmg
* .exe
* .exe.xz
* .git
* .gz
* .msi
* .pkg
* .pkg.tar.zst
* .sh
* .tar.gz
* .tar.xz
* .xz
* .zip
*/
bc.getSortedFormats = function (formats) {
/* jshint maxcomplexity: 25 */
formats.sort();
let id = formats.join(',');
if (bc._allFormats[id]) {
return bc._allFormats[id];
}
// we don't know how to handle any of these yet
// let exclude = [];
// let isAndroid = false;
// if (!isAndroid) {
// exclude.push('.apk');
// }
// let isDebian = false;
// if (!isDebian) {
// exclude.push('.deb');
// }
// let isEnterpriseLinux = false;
// if (!isEnterpriseLinux) {
// exclude.push('.rpm');
// }
// let isArch = false;
// if (!isArch) {
// exclude.push('.pkg.tar.zst');
// }
let hasExe = formats.includes('exe') || formats.includes('.exe');
/** @type {Array<String>} */
let exts = [];
let hasXz = formats.includes('xz') || formats.includes('.xz');
if (hasXz) {
exts.push('.tar.xz');
if (hasExe) {
exts.push('.exe.xz');
}
exts.push('.xz');
}
let hasZst = formats.includes('zst') || formats.includes('.zst');
if (hasZst) {
exts.push('.tar.zst');
exts.push('.zst');
}
let hasZip = formats.includes('zip') || formats.includes('.zip');
if (hasZip) {
exts.push('.zip');
}
let has7z = false;
if (has7z) {
exts.push('.7z');
}
// let hasBz2 = formats.includes('bz2') || formats.includes('.bz2');
// if (hasBz2) {
// exts.push('.bz2');
// }
if (hasExe) {
if (!hasZip) {
exts.push('.zip');
}
exts.push('.tar.gz');
exts.push('.gz');
exts.push('.exe');
exts.push('.msi');
//exts.push('.msixbundle');
} else {
exts.push('.tar.gz');
exts.push('.gz');
exts.push('.sh');
}
exts.push('.git');
// Fallbacks
// (we include everything to bubble an extract error over not found)
exts.push('.app.zip');
exts.push('.dmg');
exts.push('.pkg');
if (!hasXz) {
exts.push('.tar.xz');
if (hasExe) {
exts.push('.exe.xz');
}
exts.push('.xz');
}
if (!hasZip) {
if (!hasExe) {
exts.push('.zip');
}
}
if (!hasZst) {
exts.push('.tar.zst');
exts.push('.zst');
}
if (!has7z) {
exts.push('.7z');
}
// if (!hasRar) {
// exts.push('.rar');
// }
// exts.push('.tar.bz2');
// exts.push('.bz2');
bc._allFormats[id] = exts;
return exts;
};
bc.selectPackage = function (packages, formats) {
if (packages.length === 1) {
return packages[0];
}
let exts = bc.getSortedFormats(formats);
for (let ext of exts) {
for (let build of packages) {
if (build.ext === ext) {
return build;
}
}
}
return packages[0];
};
/**
* @param {ProjectInfo} projInfo
*/
bc.enumerateLatestVersions = function (projInfo) {
let lexPrefix = '';
let matchInfo = Lexver.matchSorted(projInfo.lexvers, lexPrefix);
let verInfo = {
default: projInfo.lexversMap[matchInfo.default],
previous: projInfo.lexversMap[matchInfo.previous],
stable: projInfo.lexversMap[matchInfo.stable],
latest: projInfo.lexversMap[matchInfo.latest],
};
return verInfo;
};
/**
* @param {ProjectInfo} projInfo
* @param {HostTarget} hostTarget
* @param {VersionTarget} verTarget
*/
bc.findMatchingPackages = function (projInfo, hostTarget, verTarget) {
let matchInfo = bc._enumerateVersions(projInfo, verTarget.version);
let triplets = bc._enumerateTriplets(hostTarget);
//console.log('dbg: matchInfo', matchInfo);
if (matchInfo) {
for (let _triplet of triplets) {
let targetReleases = projInfo.releasesByTriplet[_triplet];
if (!targetReleases) {
continue;
}
// Make sure that these releases are the expected version
// (ex: jq1.7 => darwin-arm64-libc, jq1.6 => darwin-x86_64-libc)
for (let matchver of matchInfo.matches) {
let ver = projInfo.lexversMap[matchver] || matchver;
let packages = targetReleases[ver];
if (!packages) {
continue;
}
let match = {
triplet: _triplet,
packages: packages,
latest: projInfo.versions[0],
version: ver,
versions: matchInfo,
};
return match;
}
}
return null;
}
for (let _triplet of triplets) {
let targetReleases = projInfo.releasesByTriplet[_triplet];
if (!targetReleases) {
continue;
}
let versions = Object.keys(targetReleases);
//console.log('dbg: targetRelease versions', versions);
let lexvers = [];
for (let version of versions) {
let lexPrefix = Lexver.parseVersion(version);
lexvers.push(lexPrefix);
}
lexvers.sort();
lexvers.reverse();
// TODO get the other matchInfo props
// Make sure that these releases are the expected version
// (ex: jq1.7 => darwin-arm64-libc, jq1.6 => darwin-x86_64-libc)
for (let matchver of lexvers) {
let ver = projInfo.lexversMap[matchver] || matchver;
let packages = targetReleases[ver];
//console.log('dbg: packages', packages);
if (!packages) {
continue;
}
let pkg = packages[0];
if (verTarget.lts) {
if (!pkg.lts) {
continue;
}
let match = {
triplet: _triplet,
packages: packages,
latest: projInfo.versions[0],
version: ver,
versions: matchInfo,
};
return match;
}
let wantChannel = verTarget.channel || 'stable';
let isChannel = pkg.channel || 'stable';
if (wantChannel === 'stable') {
if (isChannel !== 'stable') {
continue;
}
}
// latest, beta, alpha, rc, preview
let match = {
triplet: _triplet,
packages: packages,
latest: projInfo.versions[0],
version: ver,
versions: matchInfo,
};
return match;
}
}
return null;
};
bc._enumerateTriplets = function (hostTarget) {
let id = [hostTarget.os, hostTarget.arch, hostTarget.libc].join(',');
let triplets = bc._allTriplets[id] || [];
if (triplets.length > 0) {
return triplets;
}
let oses = [];
if (hostTarget.os === 'windows') {
oses = ['ANYOS', 'windows'];
} else if (hostTarget.os === 'android') {
oses = ['ANYOS', 'posix_2017', 'posix_2024', 'android', 'linux'];
} else {
oses = ['ANYOS', 'posix_2017', 'posix_2024', hostTarget.os];
}
let waterfall = HostTargets.WATERFALL[hostTarget.os] || {};
let arches = waterfall[hostTarget.arch] ||
HostTargets.WATERFALL.ANYOS[hostTarget.arch] || [hostTarget.arch];
arches = ['ANYARCH'].concat(arches);
let libcs = waterfall[hostTarget.libc] ||
HostTargets.WATERFALL.ANYOS[hostTarget.libc] || [hostTarget.libc];
for (let os of oses) {
for (let arch of arches) {
for (let libc of libcs) {
let triplet = `${os}-${arch}-${libc}`;
triplets.push(triplet);
}
}
}
bc._allTriplets[id] = triplets;
return triplets;
};
bc._enumerateVersions = function (projInfo, ver) {
if (!ver) {
return null;
}
let lexPrefix = Lexver.parsePrefix(ver);
let matchInfo = Lexver.matchSorted(projInfo.lexvers, lexPrefix);
return matchInfo;
};
return bc;
};
BuildsCacher._classify = function (bc, projInfo, build) {
/* jshint maxcomplexity: 25 */
let maybeInstallable = Triplet.maybeInstallable(projInfo, build);
if (!maybeInstallable) {
return null;
}
if (LEGACY_OS_MAP[build.os]) {
build.os = LEGACY_OS_MAP[build.os];
}
if (LEGACY_ARCH_MAP[build.arch]) {
build.arch = LEGACY_ARCH_MAP[build.arch];
}
// because some packages are shimmed to match a single download against
let preTarget = Object.assign({ os: '', arch: '', libc: '' }, build);
let targetId = `${preTarget.os}:${preTarget.arch}:${preTarget.libc}`;
let buildId = `${projInfo.name}:${targetId}@${build.download}`;
//console.log(`dbg: buildId`, buildId);
let target = bc._targetsByBuildIdCache[buildId];
if (target) {
Object.assign(build, { target: target, triplet: target.triplet });
return target;
}
let pattern = Triplet.toPattern(projInfo, build);
//console.log(`dbg: pattern`, pattern);
if (!pattern) {
let err = new Error(`no pattern generated for ${projInfo.name}`);
err.code = 'E_BUILD_NO_PATTERN';
target = { error: err };
bc._targetsByBuildIdCache[buildId] = target;
return target;
}
let rawTerms = pattern.split(/[_\{\}\/\.\-]+/g);
//console.log(`dbg: rawTerms`, rawTerms);
for (let term of rawTerms) {
delete bc.orphanTerms[term];
bc.usedTerms[term] = true;
}
// {NAME}/{NAME}-{VER}-Windows-x86_64_v2-musl.exe =>
// {NAME}.windows.x86_64v2.musl.exe
let terms = Triplet.patternToTerms(pattern);
//console.log(`dbg: terms`, terms);
if (!terms.length) {
let err = new Error(`'${terms}' was trimmed to ''`);
target = { error: err };
bc._targetsByBuildIdCache[buildId] = target;
return target;
}
for (let term of terms) {
if (!term) {
continue;
}
if (bc.ALL_TERMS[term]) {
delete bc.orphanTerms[term];
bc.usedTerms[term] = true;
continue;
}
bc.unknownTerms[term] = true;
}
// {NAME}.windows.x86_64v2.musl.exe
// windows-x86_64_v2-musl
target = { triplet: '' };
try {
void Triplet.termsToTarget(target, projInfo, build, terms);
} catch (e) {
console.error(`PACKAGE FORMAT CHANGE for '${projInfo.name}':`);
console.error(e.message);
console.error(build);
return null;
}
target.triplet = `${target.arch}-${target.vendor}-${target.os}-${target.libc}`;
{
// TODO I don't love this hidden behavior
// perhaps classify should just happen when the package is loaded
// (and the sanity error should be removed, or thrown after the loop is complete)
let hasTriplet = projInfo.triplets.includes(target.triplet);
if (!hasTriplet) {
projInfo.triplets.push(target.triplet);
}
let hasOs = projInfo.oses.includes(target.os);
if (!hasOs) {
projInfo.oses.push(target.os);
}
let hasArch = projInfo.arches.includes(target.arch);
if (!hasArch) {
projInfo.arches.push(target.arch);
}
let hasLibc = projInfo.libcs.includes(target.libc);
if (!hasLibc) {
projInfo.libcs.push(target.libc);
}
if (!build.ext) {
build.ext = Triplet.buildToPackageType(build);
}
if (build.ext) {
if (!build.ext.startsWith('.')) {
build.ext = `.${build.ext}`;
}
}
let hasExt = projInfo.formats.includes(build.ext);
if (!hasExt) {
projInfo.formats.push(build.ext);
}
let hasGlobalExt = bc.formats.includes(build.ext);
if (!hasGlobalExt) {
bc.formats.push(build.ext);
}
}
bc._triplets[target.triplet] = true;
bc._targetsByBuildIdCache[buildId] = target;
let triple = [target.arch, target.vendor, target.os, target.libc];
for (let term of triple) {
if (!bc.ALL_TERMS[term]) {
throw new Error(
`[SANITY FAIL] '${projInfo.name}' '${target.triplet}' generated unknown term '${term}'`,
);
}
delete bc.orphanTerms[term];
bc.usedTerms[term] = true;
}
return target;
};
BuildsCacher.transformAndUpdate = function (name, projInfo, meta, date, bc) {
meta.packages = [];
let updated = date.valueOf();
Object.assign(projInfo, { name, updated }, meta);
for (let build of projInfo.releases) {
let buildTarget = BuildsCacher._classify(bc, projInfo, build);
if (!buildTarget) {
// ignore known, non-package extensions
continue;
}
if (buildTarget.error) {
let err = buildTarget.error;
let code = err.code || '';
console.error(`[ERROR]: ${code} ${projInfo.name}: ${build.name}`);
console.error(`>>> ${err.message} <<<`);
console.error(projInfo);
console.error(build);
console.error(`^^^ ${err.message} ^^^`);
console.error(err.stack);
continue;
}
if (!build.name) {
build.name = build.download.replace(/.*\//, '');
}
build.target = buildTarget;
meta.packages.push(build);
}
BuildsCacher.updateReleasesByTriplet(meta);
BuildsCacher.updateAndSortVersions(projInfo, meta);
Object.assign(projInfo, { name, updated }, meta);
return projInfo;
};
// TODO
// - tag channels
BuildsCacher.updateAndSortVersions = function (projInfo, meta) {
for (let build of projInfo.packages) {
let hasVersion = meta.versions.includes(build.version);
if (!hasVersion) {
build.lexver = Lexver.parseVersion(build.version);
meta.lexversMap[build.lexver] = build.version;
}
}
meta.lexvers = Object.keys(meta.lexversMap);
meta.lexvers.sort();
meta.lexvers.reverse();
meta.versions = [];
for (let lexver of meta.lexvers) {
let version = meta.lexversMap[lexver];
meta.versions.push(version);
}
projInfo.packages.sort(function (a, b) {
if (a.lexver > b.lexver) {
return -1;
}
if (a.lexver < b.lexver) {
return 1;
}
return 0;
});
};
BuildsCacher.updateReleasesByTriplet = function (meta) {
for (let build of meta.packages) {
let target = build.target;
let triplet = `${target.os}-${target.arch}-${target.libc}`;
if (!meta.releasesByTriplet[triplet]) {
meta.releasesByTriplet[triplet] = {};
}
let buildsByRelease = meta.releasesByTriplet[triplet];
if (!buildsByRelease[build.version]) {
buildsByRelease[build.version] = [];
}
let packages = buildsByRelease[build.version];
packages.push(build);
}
};

View File

@@ -1,41 +0,0 @@
'use strict';
let Builds = module.exports;
let Path = require('node:path');
let BuildsCacher = require('./builds-cacher.js');
// let HostTargets = require('./build-classifier/host-targets.js');
let Parallel = require('./parallel.js');
var INSTALLERS_DIR = Path.join(__dirname, '..');
var CACHE_DIR = Path.join(__dirname, '../_cache');
let bc = BuildsCacher.create({
caches: CACHE_DIR,
installers: INSTALLERS_DIR,
});
bc.freshenRandomPackage(600 * 1000);
Builds.init = async function () {
bc.freshenRandomPackage(600 * 1000);
let dirs = await bc.getProjectsByType();
let projNames = Object.keys(dirs.valid);
let parallel = 25;
await Parallel.run(parallel, projNames, getAll);
async function getAll(name) {
void (await bc.getPackages({
//Releases: Releases,
name: name,
date: new Date(),
}));
}
};
Builds.enumerateLatestVersions = bc.enumerateLatestVersions;
Builds.findMatchingPackages = bc.findMatchingPackages;
Builds.getPackage = bc.getPackages;
Builds.getProjectType = bc.getProjectType;
Builds.selectPackage = bc.selectPackage;

View File

@@ -1,90 +0,0 @@
'use strict';
let Path = require('node:path');
// let Builds = require('./builds.js');
let BuildsCacher = require('./builds-cacher.js');
let Triplet = require('./build-classifier/triplet.js');
async function main() {
let projName = process.argv[2];
if (!projName) {
console.error(``);
console.error(`USAGE`);
console.error(``);
console.error(` classify-one <project-name>`);
console.error(``);
console.error(`EXAMPLE`);
console.error(``);
console.error(` classify-one caddy`);
console.error(``);
return;
}
let tsDate = new Date(0);
let meta = {
// version info
versions: [],
lexvers: [],
lexversMap: {},
// culled release assets
packages: [],
releasesByTriplet: {},
// target info
triplets: [],
oses: [],
arches: [],
libcs: [],
formats: [],
// TODO channels: [],
};
let installersDir = Path.join(__dirname, '..');
let Releases = require(`${installersDir}/${projName}/releases.js`);
if (!Releases.latest) {
Releases.latest = Releases;
}
let projInfo = await Releases.latest();
// let packages = await Builds.getPackage({ name: projName });
// console.log(packages);
let bc = {};
bc.ALL_TERMS = Triplet.TERMS_PRIMARY_MAP;
bc.orphanTerms = Object.assign({}, bc.ALL_TERMS);
bc.unknownTerms = {};
bc.usedTerms = {};
bc.formats = [];
bc._targetsByBuildIdCache = {};
bc._triplets = {};
let transformed = BuildsCacher.transformAndUpdate(
projName,
projInfo,
meta,
tsDate,
bc,
);
console.log(`[DEBUG] transformed`);
let sample = transformed.packages.slice(0, 20);
console.log('packages:', sample, ':packages');
console.log(
'releasesByTriplet:',
transformed.releasesByTriplet['linux-x86_64-none'][transformed.versions[0]],
':releasesByTriplet',
);
console.log('versions:', transformed.versions, ':versions');
console.log('triplets:', transformed.triplets, ':triplets');
console.log('oses:', transformed.oses, ':oses');
console.log('arches:', transformed.arches, ':arches');
console.log('libcs:', transformed.libcs, ':libcs');
console.log('formats:', transformed.formats, ':formats');
console.log(Object.keys(transformed));
}
main().catch(function (err) {
console.error('Error:');
console.error(err);
});

View File

@@ -1,349 +0,0 @@
#!/bin/sh
#<pre>
############################################################
# <h1>Cheat Sheet at CHEATSHEET_URL</h1>
# <meta http-equiv="refresh" content="3; URL='CHEATSHEET_URL'" />
############################################################
export WEBI_PKG=webi
export WEBI_HOST=https://webinstall.dev
export WEBI_CHECKSUM=06a7fb9f
#########################################
# #
# Display Debug Info in Case of Failure #
# #
#########################################
fn_show_welcome() { (
echo ""
echo ""
# invert t_task and t_pkg for top-level welcome message
printf -- ">>> %s %s <<<\n" \
"$(t_pkg 'Welcome to') $(t_task 'Webi')$(t_pkg '!')" \
"$(t_dim "- modern tools, instant installs.")"
echo " We expect your experience to be $(t_em 'absolutely perfect')!"
echo ""
echo " $(t_attn 'Success')? Star it! $(t_url 'https://github.com/webinstall/webi-installers')"
echo " $(t_attn 'Problem')? Report it: $(t_url 'https://github.com/webinstall/webi-installers/issues')"
echo " $(t_dim "(your system is") $(t_host "$(fn_get_os)")/$(t_host "$(uname -m)") $(t_dim "with") $(t_host "$(fn_get_libc)") $(t_dim "&") $(t_host "$(fn_get_http_client_name)")$(t_dim ")")"
sleep 0.2
); }
fn_get_os() { (
# Ex:
# GNU/Linux
# Android
# Linux (often Alpine, musl)
# Darwin
b_os="$(uname -o 2> /dev/null || echo '')"
b_sys="$(uname -s)"
if test -z "${b_os}" || test "${b_os}" = "${b_sys}"; then
# ex: 'Darwin' (and plain, non-GNU 'Linux')
echo "${b_sys}"
return 0
fi
if echo "${b_os}" | grep -q "${b_sys}"; then
# ex: 'GNU/Linux'
echo "${b_os}"
return 0
fi
# ex: 'Android/Linux'
echo "${b_os}/${b_sys}"
); }
fn_get_libc() { (
# Ex:
# musl
# libc
if ldd /bin/ls 2> /dev/null | grep -q 'musl' 2> /dev/null; then
echo 'musl'
elif fn_get_os | grep -q 'GNU|Linux'; then
echo 'gnu'
else
echo 'libc'
fi
); }
fn_get_http_client_name() { (
# Ex:
# curl
# curl+wget
b_client=""
if command -v curl > /dev/null; then
b_client="curl"
fi
if command -v wget > /dev/null; then
if test -z "${b_client}"; then
b_client="wget"
else
b_client="curl+wget"
fi
fi
echo "${b_client}"
); }
#########################################
# #
# For Making the Display Nice #
# #
#########################################
# Term Types
t_cmd() { (fn_printf '\e[2m\e[35m%s\e[39m\e[22m' "${1}"); }
t_host() { (fn_printf '\e[2m\e[33m%s\e[39m\e[22m' "${1}"); }
t_link() { (fn_printf '\e[1m\e[36m%s\e[39m\e[22m' "${1}"); }
t_path() { (fn_printf '\e[2m\e[32m%s\e[39m\e[22m' "${1}"); }
t_pkg() { (fn_printf '\e[1m\e[32m%s\e[39m\e[22m' "${1}"); }
t_task() { (fn_printf '\e[36m%s\e[39m' "${1}"); }
t_url() { (fn_printf '\e[2m%s\e[22m' "${1}"); }
# Levels
t_info() { (fn_printf '\e[1m\e[36m%s\e[39m\e[22m' "${1}"); }
t_attn() { (fn_printf '\e[1m\e[33m%s\e[39m\e[22m' "${1}"); }
t_warn() { (fn_printf '\e[1m\e[33m%s\e[39m\e[22m' "${1}"); }
t_err() { (fn_printf '\e[31m%s\e[39m' "${1}"); }
# Styles
t_bold() { (fn_printf '\e[1m%s\e[22m' "${1}"); }
t_dim() { (fn_printf '\e[2m%s\e[22m' "${1}"); }
t_em() { (fn_printf '\e[3m%s\e[23m' "${1}"); }
t_under() { (fn_printf '\e[4m%s\e[24m' "${1}"); }
# FG Colors
t_cyan() { (fn_printf '\e[36m%s\e[39m' "${1}"); }
t_green() { (fn_printf '\e[32m%s\e[39m' "${1}"); }
t_magenta() { (fn_printf '\e[35m%s\e[39m' "${1}"); }
t_yellow() { (fn_printf '\e[33m%s\e[39m' "${1}"); }
fn_printf() { (
a_style="${1}"
a_text="${2}"
if fn_is_tty; then
#shellcheck disable=SC2059
printf -- "${a_style}" "${a_text}"
else
printf -- '%s' "${a_text}"
fi
); }
fn_sub_home() { (
my_rel=${HOME}
my_abs=${1}
echo "${my_abs}" | sed "s:^${my_rel}:~:"
); }
###################################
# #
# Detect HTTP Client #
# #
###################################
fn_wget() { (
# Doc:
# Downloads the file at the given url to the given path
a_url="${1}"
a_path="${2}"
cmd_wget="wget -c -q --user-agent"
if fn_is_tty; then
cmd_wget="wget -c -q --show-progress --user-agent"
fi
# busybox wget doesn't support --show-progress
# See <https://github.com/webinstall/webi-installers/pull/772>
if readlink "$(command -v wget)" | grep -q busybox; then
cmd_wget="wget --user-agent"
fi
b_triple_ua="$(fn_get_target_triple_user_agent)"
b_agent="webi/wget ${b_triple_ua}"
if command -v curl > /dev/null; then
b_agent="webi/wget+curl ${b_triple_ua}"
fi
if ! $cmd_wget "${b_agent}" "${a_url}" -O "${a_path}"; then
echo >&2 " $(t_err "failed to download (wget)") '$(t_url "${a_url}")'"
echo >&2 " $cmd_wget '${b_agent}' '${a_url}' -O '${a_path}'"
echo >&2 " $(wget -V)"
return 1
fi
); }
fn_curl() { (
# Doc:
# Downloads the file at the given url to the given path
a_url="${1}"
a_path="${2}"
cmd_curl="curl -f -sSL -#"
if fn_is_tty; then
cmd_curl="curl -f -sSL"
fi
b_triple_ua="$(fn_get_target_triple_user_agent)"
b_agent="webi/curl ${b_triple_ua}"
if command -v wget > /dev/null; then
b_agent="webi/curl+wget ${b_triple_ua}"
fi
if ! $cmd_curl -A "${b_agent}" "${a_url}" -o "${a_path}"; then
echo >&2 " $(t_err "failed to download (curl)") '$(t_url "${a_url}")'"
echo >&2 " $cmd_curl -A '${b_agent}' '${a_url}' -o '${a_path}'"
echo >&2 " $(curl -V)"
return 1
fi
); }
fn_get_target_triple_user_agent() { (
# Ex:
# x86_64/unknown GNU/Linux/5.15.107-2-pve gnu
# arm64/unknown Darwin/22.6.0 libc
echo "$(uname -m)/unknown $(fn_get_os)/$(uname -r) $(fn_get_libc)"
); }
fn_download_to_path() { (
a_url="${1}"
a_path="${2}"
mkdir -p "$(dirname "${a_path}")"
if command -v curl > /dev/null; then
fn_curl "${a_url}" "${a_path}.part"
elif command -v wget > /dev/null; then
fn_wget "${a_url}" "${a_path}.part"
else
echo >&2 " $(t_err "failed to detect HTTP client (curl, wget)")"
return 1
fi
mv "${a_path}.part" "${a_path}"
); }
##############################################
# #
# Install or Update Webi and Install Package #
# #
##############################################
webi_bootstrap() { (
a_path="${1}"
echo ""
echo "$(t_task 'Bootstrapping') $(t_pkg 'Webi')"
b_path_rel="$(fn_sub_home "${a_path}")"
b_checksum=""
if test -r "${a_path}"; then
b_checksum="$(fn_checksum "${a_path}")"
fi
if test "$b_checksum" = "${WEBI_CHECKSUM}"; then
echo " $(t_dim 'Found') $(t_path "${b_path_rel}")"
sleep 0.1
return 0
fi
b_webi_file_url="${WEBI_HOST}/packages/webi/webi.sh"
b_tmp=''
if test -r "${a_path}"; then
b_ts="$(date -u '+%s')"
b_tmp="${a_path}.${b_ts}.bak"
mv "${a_path}" "${b_tmp}"
echo " Updating $(t_path "${b_path_rel}")"
fi
echo " Downloading $(t_url "${b_webi_file_url}")"
echo " to $(t_path "${b_path_rel}")"
fn_download_to_path "${b_webi_file_url}" "${a_path}"
chmod u+x "${a_path}"
if test -r "${b_tmp}"; then
rm -f "${b_tmp}"
fi
); }
fn_checksum() {
a_filepath="${1}"
if command -v sha1sum > /dev/null; then
sha1sum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
return 0
fi
if command -v shasum > /dev/null; then
shasum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
return 0
fi
if command -v sha1 > /dev/null; then
sha1 "${a_filepath}" | cut -d'=' -f2 | cut -c 2-9
return 0
fi
echo >&2 " warn: no sha1 sum program"
date '+%F %H:%M'
}
##############################################
# #
# Detect TTY and run main #
# #
##############################################
fn_is_tty() {
if test "${WEBI_TTY}" = 'tty'; then
return 0
fi
return 1
}
fn_detect_tty() { (
# stdin will NOT be a tty if it's being piped
# stdout & stderr WILL be a tty even when piped
# they are not a tty if being captured or redirected
# 'set -i' is NOT available in sh
if test -t 1 && test -t 2; then
return 0
fi
return 1
); }
main() { (
set -e
set -u
WEBI_TTY="${WEBI_TTY:-}"
if test -z "${WEBI_TTY}"; then
if fn_detect_tty; then
WEBI_TTY="tty"
fi
export WEBI_TTY
fi
if test -z "${WEBI_WELCOME:-}"; then
fn_show_welcome
fi
export WEBI_WELCOME='shown'
# note: we may support custom locations in the future
export WEBI_HOME="${HOME}/.local"
b_home="$(fn_sub_home "${WEBI_HOME}")"
b_webi_path="${WEBI_HOME}/bin/webi"
b_webi_path_rel="${b_home}/bin/webi"
WEBI_CURRENT="${WEBI_CURRENT:-}"
if test "${WEBI_CURRENT}" != "${WEBI_CHECKSUM}"; then
webi_bootstrap "${b_webi_path}"
export WEBI_CURRENT="${WEBI_CHECKSUM}"
fi
echo " Running $(t_cmd "${b_webi_path_rel} ${WEBI_PKG}")"
echo ""
"${b_webi_path}" "${WEBI_PKG}"
); }
main

View File

@@ -0,0 +1,19 @@
#!/bin/sh
set -e
set -u
# For installing from the extracted package tmp directory
pkg_install() {
(
cd "$WEBI_TMP"
if command -v rsync 2> /dev/null | grep -q rsync; then
rsync -Krl ./xmpl*/ "$pkg_src/" 2> /dev/null
else
cp -Hr ./xmpl*/* "$pkg_src/" 2> /dev/null
cp -Hr ./xmpl*/.* "$pkg_src/" 2> /dev/null
fi
rm -rf ./xmpl*
)
}

View File

@@ -1,7 +1,7 @@
'use strict';
var fs = require('fs');
var marked = require('marked').marked;
var marked = require('marked');
var frontmatter = '---';
var keyValRe = /(\w+): (.*)/;
@@ -19,7 +19,7 @@ function parseYamlish(txt) {
}
function unblock() {
cfg[block] = marked.parse(cfg[block]);
cfg[block] = marked(cfg[block]);
block = false;
}
@@ -57,7 +57,7 @@ function parseYamlish(txt) {
var m = line.match(keyValRe);
if (!m) {
throw new Error(
'invalid key format for: ' + JSON.stringify(line) + ' ' + i,
'invalid key format for: ' + JSON.stringify(line) + ' ' + i
);
}
if ('|' === m[2]) {
@@ -68,9 +68,9 @@ function parseYamlish(txt) {
});
if (block) {
cfg[block] = marked.parse(cfg[block]);
cfg[block] = marked(cfg[block]);
}
cfg.examples = marked.parse(lines.slice(last).join('\n'));
cfg.examples = marked(lines.slice(last).join('\n'));
return cfg;
}
@@ -79,6 +79,6 @@ module.exports.parse = parseYamlish;
if (require.main === module) {
console.info(
parseYamlish(fs.readFileSync(__dirname + '/../node/README.md', 'utf8')),
parseYamlish(fs.readFileSync(__dirname + '/../node/README.md', 'utf8'))
);
}

View File

@@ -1,233 +0,0 @@
'use strict';
var Installers = module.exports;
var Crypto = require('crypto');
var Fs = require('node:fs/promises');
var path = require('node:path');
var reInstallTpl = /\s*#?\s*{{ installer }}/;
function padScript(txt) {
return txt.replace(/^/g, ' ');
}
var BAD_SH_RE = /[<>'"`$\\]/;
Installers.renderBash = async function (
pkgdir,
rel,
{ baseurl, pkg, tag, ver, os = '', arch = '', libc = '', formats },
) {
if (!Array.isArray(formats)) {
formats = [];
}
if (!tag) {
tag = '';
}
let installTxt = await Fs.readFile(path.join(pkgdir, 'install.sh'), 'utf8');
installTxt = padScript(installTxt);
var vers = rel.version.split('.');
var v = {
major: vers.shift() || '',
minor: vers.shift() || '',
patch: vers.join('.').replace(/[+\-].*/, ''),
build: vers
.join('.')
.replace(/[^+\-]*/, '')
.replace(/^-/, ''),
};
var pkgFile = rel.filename || rel.name;
let tplTxt = await Fs.readFile(
path.join(__dirname, 'package-install.tpl.sh'),
'utf8',
);
// ex: 'node@lts' or 'node'
var webiPkg = pkg;
if (ver) {
webiPkg += `@${ver}`;
}
let releaseParams = new URLSearchParams({
os: rel.os,
arch: rel.arch,
libc: rel.libc,
formats: formats.join(','),
pretty: true,
});
let releaseSearch = releaseParams.toString();
releaseSearch = releaseSearch.replace(/%2C/g, ',');
let releaseUrl = `/api/releases/${pkg}@${tag}.tab?${releaseSearch}`;
let releaseCsv = [
rel.version,
rel.lts,
rel.channel,
rel.date,
rel.os,
rel.arch,
rel.ext,
'-',
rel.download,
rel.name,
rel.comment || '',
]
.join(',')
.replace(/'/g, '');
let webiChecksum = await Installers.getWebiShChecksum();
let envReplacements = [
['WEBI_CHECKSUM', webiChecksum],
['WEBI_PKG', webiPkg],
['WEBI_HOST', baseurl],
['WEBI_OS', os],
['WEBI_ARCH', arch],
['WEBI_LIBC', libc],
['WEBI_TAG', tag],
['WEBI_RELEASES', `${baseurl}${releaseUrl}`],
['WEBI_CSV', releaseCsv],
['WEBI_VERSION', rel.version],
['WEBI_MAJOR', v.major],
['WEBI_MINOR', v.minor],
['WEBI_PATCH', v.patch],
['WEBI_BUILD', v.build],
['WEBI_GIT_BRANCH', rel.git_branch || rel.git_tag],
['WEBI_GIT_TAG', rel.git_tag], // TODO replace with branch
['WEBI_LTS', rel.lts],
['WEBI_CHANNEL', rel.channel],
['WEBI_EXT', rel.ext],
['WEBI_FORMATS', formats.join(',')],
['WEBI_PKG_URL', rel.download],
['WEBI_PKG_PATHNAME', pkgFile],
['WEBI_PKG_FILE', pkgFile], // TODO replace with pathname
['PKG_NAME', pkg],
['PKG_STABLE', rel.stable],
['PKG_LATEST', rel.latest],
['PKG_OSES', (rel.oses || []).join(' ')],
['PKG_ARCHES', (rel.arches || []).join(' ')],
['PKG_LIBCS', (rel.libcs || []).join(' ')],
['PKG_FORMATS', (rel.formats || []).join(' ')],
];
for (let env of envReplacements) {
let name = env[0];
let value = env[1];
// Ex:
// #export WEBI_FOO=xyz => export WEBI_FOO='123'
// export WEBI_FOO= => export WEBI_FOO='123'
let envRe = new RegExp(
`^[ \\t]*#?[ \\t]*(export[ \\t])?[ \\t]*(${name})=.*`,
'm',
);
if (BAD_SH_RE.test(value)) {
throw new Error(`key '${name}' has invalid value '${value}'`);
}
tplTxt = tplTxt.replace(envRe, `$1$2='${value}'`);
}
tplTxt = tplTxt
.replace(/CHEATSHEET_URL/g, `${baseurl}/${pkg}`)
// $', $0, ... $9, $`, $&, and $_ all have special meaning
// (see https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/RegExp)
// However, it can be escaped with $$ (which must be escaped with $$)
.replace(reInstallTpl, '\n' + installTxt.replace(/\$/g, '$$$$'));
return tplTxt;
};
Installers.renderPowerShell = async function (
pkgdir,
rel,
{ baseurl, pkg, tag, ver, os, arch, libc = '', formats },
) {
if (!Array.isArray(formats)) {
formats = [];
}
if (!tag) {
tag = '';
}
let installTxt = await Fs.readFile(path.join(pkgdir, 'install.ps1'), 'utf8');
installTxt = padScript(installTxt);
/*
var vers = rel.version.split('.');
var v = {
major: vers.shift() || '',
minor: vers.shift() || '',
patch: vers.join('.').replace(/[+\-].*$/, ''),
build: vers
.join('.')
.replace(/[^+\-]*()/, '')
.replace(/^-/, '')
};
*/
let tplTxt = await Fs.readFile(
path.join(__dirname, 'package-install.tpl.ps1'),
'utf8',
);
var pkgver = pkg + '@' + ver;
return (
tplTxt
.replace(
/^(#)?\$Env:WEBI_LIBC\s*=.*/im,
"$Env:WEBI_LIBC = '" + libc + "'",
)
.replace(
/^(#)?\$Env:WEBI_HOST\s*=.*/im,
"$Env:WEBI_HOST = '" + baseurl + "'",
)
.replace(
/^(#)?\$Env:WEBI_PKG\s*=.*/im,
"$Env:WEBI_PKG = '" + pkgver + "'",
)
.replace(/^(#)?\$Env:PKG_NAME\s*=.*/im, "$Env:PKG_NAME = '" + pkg + "'")
.replace(
/^(#)?\$Env:WEBI_VERSION\s*=.*/im,
"$Env:WEBI_VERSION = '" + rel.version + "'",
)
.replace(
/^(#)?\$Env:WEBI_GIT_TAG\s*=.*/im,
"$Env:WEBI_GIT_TAG = '" + rel.git_tag + "'",
)
.replace(
/^(#)?\$Env:WEBI_PKG_URL\s*=.*/im,
"$Env:WEBI_PKG_URL = '" + rel.download + "'",
)
// TODO replace WEBI_PKG_FILE (which is sometimes a dir)
.replace(
/^(#)?\$Env:WEBI_PKG_PATHNAME\s*=.*/im,
"$Env:WEBI_PKG_PATHNAME = '" + rel.name + "'",
)
// TODO deprecate
.replace(
/^(#)?\$Env:WEBI_PKG_FILE\s*=.*/im,
"$Env:WEBI_PKG_FILE = '" + rel.name + "'",
)
.replace(reInstallTpl, '\n' + installTxt)
);
};
var _webiShMeta = {
stale: 10 * 1000,
updated_at: 0,
checksum: '',
mtime: 0,
};
Installers.getWebiShChecksum = async function () {
let now = Date.now();
let ago = now - _webiShMeta.updated_at;
if (ago <= _webiShMeta.stale) {
return _webiShMeta.checksum;
}
let webiPath = path.join(__dirname, '../webi/webi.sh');
let stat = await Fs.stat(webiPath);
if (stat.mtimeMs === _webiShMeta.mtime) {
return _webiShMeta.checksum;
}
let webiBuf = await Fs.readFile(webiPath, null);
let webiHash = Crypto.createHash('sha1').update(webiBuf).digest('hex');
let webiChecksum = webiHash.slice(0, 8);
_webiShMeta.mtime = stat.mtimeMs;
_webiShMeta.updated_at = now;
_webiShMeta.checksum = webiChecksum;
return _webiShMeta.checksum;
};

View File

@@ -1,338 +0,0 @@
#!/usr/bin/env node
'use strict';
let Fs = require('node:fs/promises');
let Path = require('node:path');
let BuildsCacher = require('./builds-cacher.js');
let HostTargets = require('./build-classifier/host-targets.js');
let Parallel = require('./parallel.js');
var INSTALLERS_DIR = Path.join(__dirname, '..');
var CACHE_DIR = Path.join(__dirname, '../_cache');
let UserAgentsMap = require('./build-classifier/uas.json');
let uas = Object.keys(UserAgentsMap);
let uaTargetsMap = {};
for (let ua of uas) {
let terms = ua.split(/[\s\/]+/g);
let target = {};
void HostTargets.termsToTarget(target, terms);
if (!target) {
continue;
}
if (target.errors.length) {
throw target.errors[0];
}
if (!target.os) {
// TODO make target null, or create error for this
console.warn(`no os for terms: ${terms}`);
//throw new Error(`terms: ${terms}`);
continue;
}
if (!target.arch) {
// TODO make target null, or create error for this
console.warn(`no arch for terms: ${terms}`);
//throw new Error(`terms: ${terms}`);
continue;
}
if (!target.libc) {
// TODO make target null, or create error for this
console.warn(`no libc for terms: ${terms}`);
//throw new Error(`terms: ${terms}`);
continue;
}
let triplet = `${target.os}-${target.arch}-${target.libc}`;
uaTargetsMap[triplet] = target;
}
let uaTargets = [];
let triplets = Object.keys(uaTargetsMap);
for (let triplet of triplets) {
let target = uaTargetsMap[triplet];
uaTargets.push(target);
}
function showDirs(dirs) {
{
let errors = Object.keys(dirs.errors);
console.error('');
console.error(`Errors: ${errors.length}`);
for (let name of errors) {
let err = dirs.errors[name];
console.error(`${name}/: ${err.message}`);
}
}
{
let hidden = Object.keys(dirs.hidden);
console.debug('');
console.debug(`Hidden: ${hidden.length}`);
for (let name of hidden) {
let kind = dirs.hidden[name];
if (kind === '!directory') {
console.debug(` ${name}`);
} else {
console.debug(` ${name}/`);
}
}
}
{
let alias = Object.keys(dirs.alias);
console.debug('');
console.debug(`Alias: ${alias.length}`);
for (let name of alias) {
let kind = dirs.alias[name];
if (kind === 'symlink') {
console.debug(` ${name} => ...`);
} else {
console.debug(` ${name}/`);
}
}
}
{
let invalids = Object.keys(dirs.invalid);
console.warn('');
console.warn(`Invalid: ${invalids.length}`);
for (let name of invalids) {
console.warn(` ${name}/`);
}
}
{
let selfhosted = Object.keys(dirs.selfhosted);
console.info('');
console.info(`Self-Hosted: ${selfhosted.length}`);
for (let name of selfhosted) {
console.info(` ${name}/`);
}
}
{
let valids = Object.keys(dirs.valid);
console.info('');
console.info(`Found: ${valids.length}`);
for (let name of valids) {
console.info(` ${name}/`);
}
}
}
let bc = BuildsCacher.create({
caches: CACHE_DIR,
installers: INSTALLERS_DIR,
});
async function main() {
/* jshint maxcomplexity: 25 */
// TODO
// node ./_webi/lint-builds.js caddy@beta 'x86_64/unknown Darwin libc'
//
//let [projName, userAgent] = process.argv.slice(2);
let projName = process.argv[2];
// create test case for zoxide, goreleaser, go, yq, caddy, rg
let dirs = await bc.getProjectsByType();
if (!projName) {
showDirs(dirs);
console.info('');
}
bc.freshenRandomPackage(600 * 1000);
let rows = [];
let triples = [];
let valids = Object.keys(dirs.valid);
if (projName) {
if (!valids.includes(projName)) {
throw new Error(`'${projName}' is not a valid installable project`);
}
valids = [projName];
}
//valids = ['atomicparsley', 'caddy', 'macos'];
//valids = ['atomicparsley'];
console.info('');
console.info(`Fetching project release assets`);
let parallel = 25;
let projects = [];
await Parallel.run(parallel, valids, getAll);
async function getAll(name, i) {
console.info(` ${name}`);
let projInfo = await bc.getPackages({
//Releases: Releases,
name: name,
date: new Date(),
});
projects[i] = projInfo;
}
console.info(`Classifying build assets for...`);
for (let projInfo of projects) {
console.info(` ${projInfo.name}`);
let nStr = projInfo.releases.length.toString();
let n = nStr.padStart(5, ' ');
let row = `##### ${n}\t${projInfo.name}\tv`;
rows.push(row);
// ignore known, non-package extensions
for (let build of projInfo.releases) {
let target = bc.classify(projInfo, build);
if (!target) {
// non-build file
continue;
}
if (target.error) {
let e = target.error;
if (e.code === 'E_BUILD_NO_PATTERN') {
console.warn(`>>> ${e.message} <<<`);
console.warn(projInfo);
console.warn(build);
console.warn(`^^^ ${e.message} ^^^`);
}
throw e;
}
if (target.unknownTerms?.length) {
let msg = `${projInfo.name}: unrecognized term(s) '${target.unknownTerms}' in '${build.download}'`;
let err = new Error(msg);
throw err;
}
triples.push(target.triplet);
// if (!build.version) {
// throw new Error(`no version for ${pkg.name} ${build.name}`);
// }
// // For debug printing versions
// console.error(build.version);
rows.push(`${target.triplet}\t${projInfo.name}\t${build.version}`);
}
}
console.info(`Fetching builds for`);
for (let projInfo of projects) {
console.info('');
console.info('');
console.info(` ${projInfo.name}`);
for (let target of uaTargets) {
let libc = target.libc || 'libc';
let hostTriplet = `${target.os}-${target.arch}-${libc}`;
console.info('');
console.info(` target: ${hostTriplet}`);
let match = bc.findMatchingPackages(projInfo, target, {
ver: '',
});
if (!match) {
console.info(
` project: ${projInfo.name}: missing build for os '${target.os}'`,
);
continue;
}
if (!match.releases) {
console.info(
` project: ${projInfo.name}: missing build for os '${target.os}-${target.arch}-${libc}'`,
);
} else if (match.triplet === hostTriplet) {
let releaseNames = Object.keys(match.releases);
console.info(` selected ${releaseNames.length}`);
} else {
let releaseNames = Object.keys(match.releases);
console.info(
` selected ${releaseNames.length} (${match.triplet} fallback)`,
);
}
}
}
let tsv = rows.join('\n');
console.info('');
console.info('#rows', rows.length);
await Fs.writeFile('builds.tsv', tsv, 'utf8');
console.info('');
console.info('Triplets Detected:');
let triplets = Object.keys(bc._triplets);
if (triplets.length) {
triplets.sort();
console.info(' ', triplets.join('\n '));
} else {
console.info(' (none)');
}
console.info('');
console.info('New / Unknown Terms:');
let unknowns = Object.keys(bc.unknownTerms);
if (unknowns.length) {
unknowns.sort();
console.warn(' ', unknowns.join('\n '));
} else {
console.info(' (none)');
}
console.info('');
console.info('Unused Terms:');
let unuseds = Object.keys(bc.orphanTerms);
if (unuseds.length) {
unuseds.sort();
console.warn(' ', unuseds.join('\n '));
} else {
console.info(' (none)');
}
console.info('');
console.info('Formats:');
if (bc.formats.length) {
let formats = bc.formats.slice();
formats.sort();
if (!formats[0]) {
formats[0] = '(bin)';
}
console.warn(' ', formats.join('\n '));
} else {
console.info(' (none)');
}
// sort -u -k1 builds.tsv | rg -v '^#|^https?:' | rg -i arm
// cut -f1 builds.tsv | sort -u -k1 | rg -v '^#|^https?:' | rg -i arm
}
if (module === require.main) {
let times = [];
let now = Date.now();
main()
.then(async function () {
let then = Date.now();
let delta = then - now;
times.push(delta);
now = then;
await main();
then = Date.now();
delta = then - now;
times.push(delta);
})
.then(function () {
console.info('');
console.info('Run times');
for (let delta of times) {
let s = delta / 1000;
console.info(` ${s}`);
}
function forceExit() {
console.warn(`warn: dangling event loop reference`);
process.exit(0);
}
let exitTimeout = setTimeout(forceExit, 250);
exitTimeout.unref();
})
.catch(function (err) {
console.error(err.stack || err);
process.exit(1);
});
}

View File

@@ -7,21 +7,20 @@ var osMap = {
freebsd: /(\b|_)(freebsd)/i,
windows: /(\b|_)(win|microsoft|msft)/i,
sunos: /(\b|_)(sun)/i,
aix: /(\b|_)(aix)/i,
aix: /(\b|_)(aix)/i
};
var maps = {
oses: {},
arches: {},
libcs: {},
formats: {},
formats: {}
};
Object.keys(osMap).forEach(function (name) {
maps.oses[name] = true;
});
var formats = ['zip', 'xz', 'tar', 'pkg', 'msi', 'git', 'exe', 'dmg', 'git'];
var formats = ['zip', 'xz', 'tar', 'pkg', 'msi', 'git', 'exe', 'dmg'];
formats.forEach(function (name) {
maps.formats[name] = true;
});
@@ -29,18 +28,18 @@ formats.forEach(function (name) {
// evaluation order matters
// (i.e. otherwise x86 and x64 can cross match)
var arches = [
// arm64/aarch64 has very high specificity, so it comes first
'arm64',
// arm 7 is also generic aarch/arm/arm32
// arm 7 cannot be confused with arm64
'armv7l',
// arm6 can run on armv7
'armv6l',
// amd64 is more likely and less often specified than arm64
// amd64 is more likely than arm64
'amd64',
// arm6 has the same prefix as arm64
'armv6l',
// arm64 is more likely than arm6, and should be the default
'arm64',
'x86',
'ppc64le',
'ppc64',
's390x',
's390x'
];
// Used for detecting system arch from package download url, for example:
//
@@ -50,69 +49,57 @@ var arches = [
// https://git.com/org/foo/releases/v0.7.9/foo-x86_64-linux-musl.tar.gz
//
var archMap = {
arm64: /(\b|_)(aarch64|arm64)/i,
armv7l: /(\b|_)(arm32|arm[_\-]?v?7l?)/i,
armv6l: /(\b|_)(arm|aarch32|arm[_\-]?v?6l?)(\b|_)/i,
armv7l: /(\b|_)(armv?7l?)/i,
//amd64: /(amd.?64|x64|[_\-]64)/i,
amd64:
/(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)64([_\-]?bit)?(\b|_)/i,
//x86: /(86)(\b|_)/i,
x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32|i?386)([_\-]?bit)?(\b|_)/i,
armv6l: /(\b|_)(aarch32|armv?6l?)(\b|_)/i,
arm64: /(\b|_)((aarch|arm)64|arm)/i,
x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32)([_\-]?bit)(\b|_)/i,
ppc64le: /(\b|_)(ppc64le)/i,
ppc64: /(\b|_)(ppc64)(\b|_)/i,
s390x: /(\b|_)(s390x)/i,
s390x: /(\b|_)(s390x)/i
};
arches.forEach(function (name) {
maps.arches[name] = true;
});
var libcs = ['none', 'musl', 'gnu', 'msvc', 'libc'];
libcs.forEach(function (name) {
maps.libcs[name] = true;
});
function normalize(all) {
/* jshint maxcomplexity:50 */
/* jshint maxdepth:10 */
var supported = {
oses: {},
arches: {},
libcs: {},
formats: {},
formats: {}
};
for (let rel of all.releases) {
all.releases.forEach(function (rel) {
rel.version = rel.version.replace(/^v/i, '');
if (!rel.name) {
rel.name = rel.download.replace(/.*\//, '');
}
if (!rel.os) {
rel.os = 'unknown';
let osNames = Object.keys(osMap);
for (let osName of osNames) {
let relName = rel.name || rel.download;
let osRegExp = osMap[osName];
let matches = osRegExp.test(relName);
if (matches) {
rel.os = osName;
break;
}
rel.os =
Object.keys(osMap).find(function (regKey) {
return osMap[regKey].test(rel.name || rel.download);
}) || 'unknown';
}
// Hacky-doo for musl
// TODO some sort of glibc vs musl tag?
if (!rel._musl) {
if (/(\b|\.|_|-)(musl)(\b|\.|_|-)/.test(rel.download)) {
rel._musl = true;
}
}
supported.oses[rel.os] = true;
if (!rel.arch) {
for (let arch of arches) {
let name = rel.name || rel.download;
let isArch = name.match(archMap[arch]);
if (isArch) {
arches.some(function (regKey) {
var arch = (rel.name || rel.download).match(archMap[regKey]) && regKey;
if (arch) {
rel.arch = arch;
break;
return true;
}
}
});
}
if (!rel.arch) {
if ('macos' === rel.os) {
@@ -121,60 +108,6 @@ function normalize(all) {
}
supported.arches[rel.arch] = true;
// note: depends on rel.os
if (!rel.libc) {
let isMusl;
let isMsvc;
let isStatic;
let isGnu;
// extra blocks to prevent copy pasta errors
{
let muslRe = /(\b|_)(musl)(\b|_)/i;
isMusl = muslRe.test(rel.download) || muslRe.test(rel.name);
}
{
let msvcRe = /(\b|_)(msvc)(\b|_)/i;
isMsvc = msvcRe.test(rel.download) || msvcRe.test(rel.name);
}
{
let staticRe = /(\b|_)(static)(\b|_)/i;
isStatic = staticRe.test(rel.download) || staticRe.test(rel.name);
}
{
let gnuRe = /(\b|_)(gnu|glibc|libc)(\b|_)/i;
isGnu = gnuRe.test(rel.download) || gnuRe.test(rel.name);
}
if (isMusl) {
// we specifically tag things that need musl++ in their own releases
rel.libc = 'none';
} else if (isStatic) {
rel.libc = 'none';
} else if (isGnu) {
rel.libc = 'gnu';
if (rel.os === 'windows') {
// windows gnu is static
rel.libc = 'none';
} else if (rel.os === 'darwin') {
// if glibc is required on macos, it'll be static
rel.libc = 'none';
}
} else if (isMsvc) {
rel.libc = 'msvc';
} else {
// The default is no requirement for any particular libc
// (Go, Zig, POSIX Shell, JS, etc)
// and hopefully we never have to worry about mingw and friends
rel.libc = 'none';
}
}
supported.libcs[rel.libc] = true;
var tarExt;
if (!rel.ext) {
// pkg-v1.0.tar.gz => ['gz', 'tar', '0', 'pkg-v1']
@@ -202,29 +135,10 @@ function normalize(all) {
}
supported.formats[tarExt || rel.ext] = true;
if (!rel.channel) {
// basically like this: (+.-_)(beta|rc)(0-9)(+.-_)
// matches:
// - v1.0-beta
// - v1.0-beta1.1
// - v1.0-beta-11
// won't match:
// - v1.0beta
// - v1.0-beta1b
let isBetaRe =
/(\b|_)(alpha|beta|dev|developer|prev|preview|rc)(\d+)(\b|_)/;
let isBeta = isBetaRe.test(rel.name);
if (isBeta) {
rel.channel = 'beta';
} else {
rel.channel = 'stable';
}
}
if (all.download) {
rel.download = all.download.replace(/{{ download }}/, rel.download);
}
}
});
all.oses = Object.keys(supported.oses).filter(function (name) {
return maps.oses[name];
@@ -232,9 +146,6 @@ function normalize(all) {
all.arches = Object.keys(supported.arches).filter(function (name) {
return maps.arches[name];
});
all.libcs = Object.keys(supported.libcs).filter(function (name) {
return maps.libcs[name];
});
all.formats = Object.keys(supported.formats).filter(function (name) {
return maps.formats[name];
});
@@ -255,5 +166,4 @@ module.exports._debug = function (all) {
// NOT in order of priority (which would be tar, xz, zip, ...)
module.exports.formats = formats;
module.exports.arches = arches;
module.exports.libcs = libcs;
module.exports.formatsMap = maps.formats;

View File

@@ -1,155 +0,0 @@
#!/usr/bin/env pwsh
#350 check if windows user run as admin
# this allows us to call ps1 files, which allows us to have spaces in filenames
# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
Set-ExecutionPolicy -Scope Process Bypass
# If a command returns an error, halt the script.
$ErrorActionPreference = 'Stop'
# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
$ProgressPreference = 'SilentlyContinue'
$Env:WEBI_HOST = 'https://webinstall.dev'
#$Env:WEBI_PKG = 'node@lts'
#$Env:PKG_NAME = node
#$Env:WEBI_VERSION = v12.16.2
#$Env:WEBI_GIT_TAG = 12.16.2
#$Env:WEBI_PKG_URL = "https://.../node-....zip"
#$Env:WEBI_PKG_FILE = "node-v12.16.2-win-x64.zip"
#$Env:WEBI_PKG_PATHNAME = "node-v12.16.2-win-x64.zip"
# Switch to userprofile
Push-Location $Env:USERPROFILE
# Make paths
New-Item -Path "$Env:USERPROFILE\Downloads" -ItemType Directory -Force | Out-Null
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
New-Item -Path "$Env:USERPROFILE\.local\opt" -ItemType Directory -Force | Out-Null
# {{ baseurl }}
# {{ version }}
# See
# - <https://superuser.com/q/1264444>
# - <https://stackoverflow.com/a/60572643/151312>
$Esc = [char]27
$TTask = "${Esc}[36m"
$TName = "${Esc}[1m${Esc}[32m"
$TUrl = "${Esc}[2m"
$TPath = "${Esc}[2m${Esc}[32m"
$TCmd = "${Esc}[2m${Esc}[35m"
$TDim = "${Esc}[2m"
$TReset = "${Esc}[0m"
function Invoke-DownloadUrl {
param (
[string]$URL,
[string]$Params,
[string]$Path,
[switch]$Force
)
if (Test-Path -Path "$Path") {
if (-not $Force.IsPresent) {
Write-Host " ${TDim}Found${TReset} $Path"
return
}
Write-Host " Updating ${TDim}${Path}${TDim}"
}
$TmpPath = "${Path}.part"
Remove-Item -Path $TmpPath -Force -ErrorAction Ignore
Write-Host " Downloading ${TDim}from${TReset}"
Write-Host " ${TDim}${URL}${TReset}"
if ($Params.Length -ne 0) {
Write-Host " ?$Params"
$URL = "${URL}?${Params}"
}
curl.exe '-#' -f -sS -A $Env:WEBI_UA -L $URL | Out-File $TmpPath
Remove-Item -Path $Path -Force -ErrorAction Ignore
Move-Item $TmpPath $Path
Write-Host " Saved ${TPath}${Path}${TReset}"
}
function Get-UserAgent {
# This is the canonical CPU arch when the process is emulated
$my_arch = "$Env:PROCESSOR_ARCHITEW6432"
if ($my_arch -eq $null -or $my_arch -eq "") {
# This is the canonical CPU arch when the process is native
$my_arch = "$Env:PROCESSOR_ARCHITECTURE"
}
if ($my_arch -eq "AMD64") {
# Because PowerShell is sometimes AMD64 on Windows 10 ARM
# See https://oofhours.com/2020/02/04/powershell-on-windows-10-arm64/
$my_os_arch = (Get-CimInstance -ClassName Win32_OperatingSystem).OSArchitecture
# Using -clike because of the trailing newline
if ($my_os_arch -clike "ARM 64*") {
$my_arch = "ARM64"
}
}
"PowerShell+curl Windows/10+ $my_arch msvc"
}
function webi_path_add($pathname) {
# C:\Users\me => C:/Users/me
$my_home = $Env:UserProfile
$my_home = $my_home.replace('\\', '/')
$my_home_re = [regex]::escape($my_home)
# ~/bin => %USERPROFILE%/bin
$pathname = $pathname.replace('~/', "$Env:UserProfile/")
# C:\Users\me\bin => %USERPROFILE%/bin
$my_pathname = $pathname.replace('\\', '/')
$my_pathname = $my_pathname -ireplace $my_home_re, "%USERPROFILE%"
$all_user_paths = [Environment]::GetEnvironmentVariable("Path", "User")
$user_paths = "${all_user_paths}".Trim(';').Split(';')
$exists_in_path = $false
foreach ($user_path in $user_paths) {
# C:\Users\me\bin => %USERPROFILE%/bin
$my_user_path = $user_path.replace('\\', '/')
$my_user_path = $my_user_path -ireplace $my_home_re, "%USERPROFILE%"
if ($my_user_path -ieq $my_pathname) {
$exists_in_path = $true
}
}
if (-not $exists_in_path) {
$all_user_paths = "${pathname};${all_user_paths}".Trim(';')
[Environment]::SetEnvironmentVariable("Path", $all_user_paths, "User")
$null = Sync-EnvPath
}
}
function Sync-EnvPath {
$UserPath = [Environment]::GetEnvironmentVariable("Path", "User").Trim(';')
$MachinePath = [Environment]::GetEnvironmentVariable("Path", "Machine").Trim(';')
$Env:Path = "${UserPath};${MachinePath}"
[Environment]::SetEnvironmentVariable("Path", $Env:Path)
$Env:Path
}
$Env:WEBI_UA = Get-UserAgent
#$has_local_bin = echo "$Env:PATH" | Select-String -Pattern '\.local.bin'
#if (!$has_local_bin)
#{
webi_path_add ~/.local/bin
#}
# {{ installer }}
webi_path_add ~/.local/bin
# Done
Pop-Location

File diff suppressed because it is too large Load Diff

View File

@@ -1 +0,0 @@
projects.js

114
_webi/packages.js Normal file
View File

@@ -0,0 +1,114 @@
'use strict';
var frontmarker = require('./frontmarker.js');
var shmatter = require('shmatter');
var fs = require('fs');
var path = require('path');
var pkgs = module.exports;
pkgs.create = function (Pkgs, basepath) {
if (!Pkgs) {
Pkgs = {};
}
if (!basepath) {
basepath = path.join(__dirname, '../');
}
Pkgs.all = function () {
return fs.promises.readdir(basepath).then(function (nodes) {
var items = [];
return nodes
.reduce(function (p, node) {
return p.then(function () {
return pkgs.get(node).then(function (meta) {
if (meta && '_' !== node[0]) {
meta.name = node;
items.push(meta);
}
});
});
}, Promise.resolve())
.then(function () {
return items;
});
});
};
Pkgs.get = function (node) {
return fs.promises.access(path.join(basepath, node)).then(function () {
return Pkgs._get(node);
});
};
Pkgs._get = function (node) {
var yash = path.join(basepath, node, 'package.yash');
var curlbash = path.join(basepath, node, 'install.sh');
var readme = path.join(basepath, node, 'README.md');
var winstall = path.join(basepath, node, 'install.ps1');
return Promise.all([
fs.promises
.readFile(readme, 'utf-8')
.then(function (txt) {
// TODO
return frontmarker.parse(txt);
})
.catch(function (e) {
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
console.error("failed to read '" + node + "/README.md'");
console.error(e);
}
}),
fs.promises
.readFile(yash, 'utf-8')
.then(function (txt) {
return shmatter.parse(txt);
})
.catch(function (e) {
// no yash package description
yash = '';
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
console.error("failed to parse '" + node + "/package.yash'");
console.error(e);
}
return fs.promises.readFile(curlbash, 'utf-8').then(function (txt) {
return shmatter.parse(txt);
});
})
.catch(function (e) {
// no *nix installer
curlbash = '';
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
console.error("failed to parse '" + node + "/install.sh'");
console.error(e);
}
}),
fs.promises.access(winstall).catch(function (e) {
// no winstaller
winstall = '';
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
console.error("failed to read '" + node + "/install.ps1'");
console.error(e);
}
})
]).then(function (items) {
var meta = items[0] || items[1];
if (!meta) {
// doesn't exist
return;
}
meta.windows = !!winstall;
meta.bash = !!curlbash;
return meta;
});
};
return Pkgs;
};
pkgs.create(pkgs);
if (module === require.main) {
pkgs.all().then(function (data) {
console.info('package info:');
console.info(data);
});
}

View File

@@ -1,44 +0,0 @@
'use strict';
var Parallel = module.exports;
Parallel.run = async function (limit, arr, fn) {
let index = 0;
let actives = [];
let results = [];
limit = Math.min(limit, arr.length);
function launch() {
let _index = index;
let p = fn(arr[_index], _index, arr);
// some tasks may be synchronous
// so we must push before removing
actives.push(p);
p.then(function _resolve(result) {
let i = actives.indexOf(p);
actives.splice(i, 1);
results[_index] = result;
});
index += 1;
}
// start tasks in parallel, up to limit
for (; actives.length < limit; ) {
launch();
}
// keep the task queue full
for (; index < arr.length; ) {
// wait for one task to complete
await Promise.race(actives);
// add one task again
launch();
}
// wait for all remaining tasks
await Promise.all(actives);
return results;
};

View File

@@ -1,96 +0,0 @@
'use strict';
var frontmarker = require('./frontmarker.js');
var fs = require('fs');
var path = require('path');
var pkgs = module.exports;
pkgs.create = function (Pkgs, basepath) {
if (!Pkgs) {
Pkgs = {};
}
if (!basepath) {
basepath = path.join(__dirname, '../');
}
Pkgs.all = function () {
return fs.promises.readdir(basepath).then(function (nodes) {
var items = [];
return nodes
.reduce(function (p, node) {
return p.then(function () {
return pkgs.get(node).then(function (meta) {
if (meta && '_' !== node[0]) {
meta.name = node;
items.push(meta);
}
});
});
}, Promise.resolve())
.then(function () {
return items;
});
});
};
Pkgs.get = function (node) {
return fs.promises.access(path.join(basepath, node)).then(function () {
return Pkgs._get(node);
});
};
Pkgs._get = function (node) {
var curlbash = path.join(basepath, node, 'install.sh');
var readme = path.join(basepath, node, 'README.md');
var winstall = path.join(basepath, node, 'install.ps1');
return Promise.all([
fs.promises
.readFile(readme, 'utf-8')
.then(function (txt) {
// TODO
return frontmarker.parse(txt);
})
.catch(function (e) {
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
console.error("failed to read '" + node + "/README.md'");
console.error(e);
}
}),
fs.promises.access(curlbash).catch(function (e) {
// no *nix installer
curlbash = '';
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
console.error("failed to parse '" + node + "/install.sh'");
console.error(e);
}
}),
fs.promises.access(winstall).catch(function (e) {
// no winstaller
winstall = '';
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
console.error("failed to read '" + node + "/install.ps1'");
console.error(e);
}
}),
]).then(function (items) {
var meta = items[0] || items[1];
if (!meta) {
// doesn't exist
return;
}
meta.windows = !!winstall;
meta.bash = !!curlbash;
return meta;
});
};
return Pkgs;
};
pkgs.create(pkgs);
if (module === require.main) {
pkgs.all().then(function (data) {
console.info('package info:');
console.info(data);
});
}

255
_webi/releases.js Normal file
View File

@@ -0,0 +1,255 @@
'use strict';
var fs = require('fs');
var path = require('path');
var request = require('@root/request');
var _normalize = require('../_webi/normalize.js');
var reInstallTpl = /\s*#?\s*{{ installer }}/;
var Releases = module.exports;
Releases.get = async function (pkgdir) {
var get;
try {
get = require(path.join(pkgdir, 'releases.js'));
} catch (e) {
throw new Error('no releases.js for', pkgdir.split(/[\/\\]+/).pop());
}
return get(request).then(function (all) {
return _normalize(all);
});
};
function padScript(txt) {
return txt.replace(/^/g, ' ');
}
Releases.renderBash = function (
pkgdir,
rel,
{ baseurl, pkg, tag, ver, os = '', arch = '', formats }
) {
if (!Array.isArray(formats)) {
formats = [];
}
if (!tag) {
tag = '';
}
return fs.promises
.readFile(path.join(pkgdir, 'install.sh'), 'utf8')
.then(function (installTxt) {
installTxt = padScript(installTxt);
var vers = rel.version.split('.');
var v = {
major: vers.shift() || '',
minor: vers.shift() || '',
patch: vers.join('.').replace(/[+\-].*/, ''),
build: vers
.join('.')
.replace(/[^+\-]*/, '')
.replace(/^-/, '')
};
var pkgFile = rel.filename || rel.name;
return fs.promises
.readFile(path.join(__dirname, 'template.sh'), 'utf8')
.then(function (tplTxt) {
// ex: 'node@lts' or 'node'
var webiPkg = pkg;
if (ver) {
webiPkg += `@${ver}`;
}
return (
tplTxt
.replace(/CHEATSHEET_URL/g, `${baseurl}/${pkg}`)
.replace(/^\s*#?WEBI_PKG=.*/m, `WEBI_PKG='${webiPkg}'`)
.replace(/^\s*#?WEBI_HOST=.*/m, `WEBI_HOST='${baseurl}'`)
.replace(/^\s*#?WEBI_OS=.*/m, `WEBI_OS='${os}'`)
.replace(/^\s*#?WEBI_ARCH=.*/m, `WEBI_ARCH='${arch}'`)
.replace(/^\s*#?WEBI_TAG=.*/m, `WEBI_TAG='${tag}'`)
.replace(
/^\s*#?WEBI_RELEASES=.*/m,
"WEBI_RELEASES='" +
baseurl +
'/api/releases/' +
pkg +
'@' +
tag +
'.tab?os=' +
rel.os +
'&arch=' +
rel.arch +
'&formats=' +
formats.join(',') +
'&pretty=true' +
"'"
)
.replace(
/^\s*#?WEBI_CSV=.*/m,
"WEBI_CSV='" +
[
rel.version,
rel.lts,
rel.channel,
rel.date,
rel.os,
rel.arch,
rel.ext,
'-',
rel.download,
rel.name,
rel.comment || ''
]
.join(',')
.replace(/'/g, '') +
"'"
)
.replace(
/^\s*#?WEBI_VERSION=.*/m,
'WEBI_VERSION=' + JSON.stringify(rel.version)
)
.replace(/^\s*#?WEBI_MAJOR=.*/m, 'WEBI_MAJOR=' + v.major)
.replace(/^\s*#?WEBI_MINOR=.*/m, 'WEBI_MINOR=' + v.minor)
.replace(/^\s*#?WEBI_PATCH=.*/m, 'WEBI_PATCH=' + v.patch)
.replace(/^\s*#?WEBI_BUILD=.*/m, 'WEBI_BUILD=' + v.build)
.replace(/^\s*#?WEBI_LTS=.*/m, 'WEBI_LTS=' + rel.lts)
.replace(/^\s*#?WEBI_CHANNEL=.*/m, 'WEBI_CHANNEL=' + rel.channel)
.replace(
/^\s*#?WEBI_EXT=.*/m,
'WEBI_EXT=' + rel.ext.replace(/tar.*/, 'tar')
)
.replace(
/^\s*#?WEBI_FORMATS=.*/m,
"WEBI_FORMATS='" + formats.join(',') + "'"
)
.replace(
/^\s*#?WEBI_PKG_URL=.*/m,
"WEBI_PKG_URL='" + rel.download + "'"
)
.replace(
/^\s*#?WEBI_PKG_FILE=.*/m,
"WEBI_PKG_FILE='" + pkgFile + "'"
)
// PKG details
.replace(/^\s*#?PKG_NAME=.*/m, "PKG_NAME='" + pkg + "'")
.replace(
/^\s*#?PKG_OSES=.*/m,
"PKG_OSES='" + ((rel && rel.oses) || []).join(',') + "'"
)
.replace(
/^\s*#?PKG_ARCHES=.*/m,
"PKG_ARCHES='" + ((rel && rel.arches) || []).join(',') + "'"
)
.replace(
/^\s*#?PKG_FORMATS=.*/m,
"PKG_FORMATS='" + ((rel && rel.formats) || []).join(',') + "'"
)
// $', $0, ... $9, $`, $&, and $_ all have special meaning
// (see https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/RegExp)
// However, it can be escaped with $$ (which must be escaped with $$)
.replace(reInstallTpl, '\n' + installTxt.replace(/\$/g, '$$$$'))
);
});
});
};
Releases.renderBatch = function (
pkgdir,
rel,
{ baseurl, pkg, tag, ver, os, arch, formats }
) {
if (!Array.isArray(formats)) {
formats = [];
}
if (!tag) {
tag = '';
}
return fs.promises
.readFile(path.join(pkgdir, 'install.bat'), 'utf8')
.then(function (installTxt) {
installTxt = padScript(installTxt);
/*
var vers = rel.version.split('.');
var v = {
major: vers.shift() || '',
minor: vers.shift() || '',
patch: vers.join('.').replace(/[+\-].*$/, ''),
build: vers
.join('.')
.replace(/[^+\-]*()/, '')
.replace(/^-/, '')
};
*/
return fs.promises
.readFile(path.join(__dirname, 'template.bat'), 'utf8')
.then(function (tplTxt) {
return tplTxt
.replace(
/^(REM )?WEBI_PKG=.*/im,
"WEBI_PKG='" + pkg + '@' + ver + "'"
)
.replace(reInstallTpl, '\n' + installTxt);
});
});
};
Releases.renderPowerShell = function (
pkgdir,
rel,
{ baseurl, pkg, tag, ver, os, arch, formats }
) {
if (!Array.isArray(formats)) {
formats = [];
}
if (!tag) {
tag = '';
}
return fs.promises
.readFile(path.join(pkgdir, 'install.ps1'), 'utf8')
.then(function (installTxt) {
installTxt = padScript(installTxt);
/*
var vers = rel.version.split('.');
var v = {
major: vers.shift() || '',
minor: vers.shift() || '',
patch: vers.join('.').replace(/[+\-].*$/, ''),
build: vers
.join('.')
.replace(/[^+\-]*()/, '')
.replace(/^-/, '')
};
*/
return fs.promises
.readFile(path.join(__dirname, 'template.ps1'), 'utf8')
.then(function (tplTxt) {
var pkgver = pkg + '@' + ver;
return tplTxt
.replace(
/^(#)?\$Env:WEBI_HOST\s*=.*/im,
"$Env:WEBI_HOST = '" + baseurl + "'"
)
.replace(
/^(#)?\$Env:WEBI_PKG\s*=.*/im,
"$Env:WEBI_PKG = '" + pkgver + "'"
)
.replace(
/^(#)?\$Env:PKG_NAME\s*=.*/im,
"$Env:PKG_NAME = '" + pkg + "'"
)
.replace(
/^(#)?\$Env:WEBI_VERSION\s*=.*/im,
"$Env:WEBI_VERSION = '" + rel.version + "'"
)
.replace(
/^(#)?\$Env:WEBI_PKG_URL\s*=.*/im,
"$Env:WEBI_PKG_URL = '" + rel.download + "'"
)
.replace(
/^(#)?\$Env:WEBI_PKG_FILE\s*=.*/im,
"$Env:WEBI_PKG_FILE = '" + rel.name + "'"
)
.replace(reInstallTpl, '\n' + installTxt);
});
});
};

View File

@@ -1,304 +1,92 @@
'use strict';
var InstallerServer = module.exports;
var path = require('path');
let Fs = require('fs/promises');
let Path = require('path');
var uaDetect = require('./ua-detect.js');
var packages = require('./packages.js');
var Releases = require('./releases.js');
let HostTargets = require('./build-classifier/host-targets.js');
let Builds = require('./builds.js');
let Installers = require('./installers.js');
// handlers caching and transformation, probably should be broken down
var getReleases = require('./transform-releases.js');
InstallerServer.INSTALLERS_DIR = Path.join(__dirname, '..');
InstallerServer.serveInstaller = async function (
var installersDir = path.join(__dirname, '..');
module.exports = async function serveInstaller(
baseurl,
ua,
pkg,
tag,
ext,
formats,
libc,
formats
) {
let unameAgent = ua;
let projectName = pkg;
let [rel, tmplParams] = await InstallerServer.helper({
unameAgent,
projectName,
tag,
formats,
libc,
});
Object.assign(tmplParams, {
baseurl,
});
// TODO put some of this in a middleware? or common function?
var pkgdir = Path.join(InstallerServer.INSTALLERS_DIR, projectName);
if ('ps1' === ext) {
return Installers.renderPowerShell(pkgdir, rel, tmplParams);
}
return Installers.renderBash(pkgdir, rel, tmplParams);
};
var ver = tag.replace(/^v/, '');
var lts;
var channel;
// TODO put some of this in a middleware? or common function?
// TODO maybe move package/version/lts/channel detection into getReleases
InstallerServer.helper = async function ({
unameAgent,
projectName,
tag,
formats,
libc,
}) {
console.log(`dbg: Installer User-Agent: ${unameAgent}`);
let releaseTarget = toReleaseTarget(tag);
let hostFormats = formats;
let terms = unameAgent.split(/[\s\/]+/g);
let hostTarget = {};
try {
void HostTargets.termsToTarget(hostTarget, terms);
} catch (e) {
// if we can't guarantee the results...
// "in the face of ambiguity, refuse the temptation to guess"
throw e;
}
console.log(`dbg: Installer Host Target:`);
console.log(hostTarget);
if (!hostTarget.os) {
throw new Error(`OS could not be identified by User-Agent '${unameAgent}'`);
switch (ver) {
case 'latest':
ver = '';
channel = 'stable';
break;
case 'lts':
lts = true;
channel = 'stable';
ver = '';
break;
case 'stable':
channel = 'stable';
ver = '';
break;
case 'beta':
channel = 'beta';
ver = '';
break;
case 'dev':
channel = 'dev';
ver = '';
break;
}
console.log(`dbg: Get Project Installer Type for '${projectName}':`);
let proj = await Builds.getProjectType(projectName);
if (proj.type === 'alias') {
console.log(`dbg: alias`, proj);
projectName = proj.detail;
proj = await Builds.getProjectType(projectName); // an alias should never resolve to an alias
}
console.log(`dbg: proj`, proj);
let validTypes = ['selfhosted', 'valid'];
if (!validTypes.includes(proj.type)) {
let msg = `'${projectName}' doesn't have an installer: '${proj.type}': '${proj.detail}'`;
let err = new Error(msg);
err.code = 'ENOENT';
throw err;
}
let tmplParams = {
pkg: projectName,
tag: tag,
os: hostTarget.os,
arch: hostTarget.arch,
libc: hostTarget.libc,
formats: hostFormats,
limit: 1,
};
Object.assign(tmplParams, releaseTarget);
console.log('tmplParams', tmplParams);
let errPackage = {
name: 'doesntexist.ext',
version: '0.0.0',
lts: '-',
channel: 'error',
date: '1970-01-01',
os: hostTarget.os || '-',
arch: hostTarget.arch || '-',
libc: hostTarget.libc || '-',
ext: 'err',
download: 'https://example.com/doesntexist.ext',
comment:
'No matches found. Could be bad or missing version info' +
',' +
"Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
};
if (proj.type === 'selfhosted') {
return [errPackage, tmplParams];
}
let projInfo = await Builds.getPackage({
name: projectName,
date: new Date(),
});
let latestVersions = Builds.enumerateLatestVersions(projInfo);
//console.log('projInfo', projInfo);
let buildTargetInfo = {
triplets: projInfo.triplets,
oses: projInfo.oses,
arches: projInfo.arches,
libcs: projInfo.libcs,
formats: projInfo.formats,
latest: latestVersions.latest,
stable: latestVersions.stable,
};
// TODO .findMatchingPackages() should probably account for this
let hasOs = projInfo.oses.includes(hostTarget.os);
let maybePosix = !hasOs && hostTarget.os !== 'windows';
if (maybePosix) {
let posixes = ['posix_2017', 'posix_2024'];
for (let posixYear of posixes) {
let hasPosix = projInfo.oses.includes(posixYear);
if (hasPosix) {
hasOs = true;
break;
// TODO maybe move package/version/lts/channel detection into getReleases
var myOs = uaDetect.os(ua);
var myArch = uaDetect.arch(ua);
return packages.get(pkg).then(function (cfg) {
return getReleases({
pkg: cfg.alias || pkg,
ver,
os: myOs,
arch: myArch,
lts,
channel,
formats,
limit: 1
}).then(function (rels) {
var rel = rels.releases[0];
var pkgdir = path.join(installersDir, pkg);
var opts = {
baseurl,
pkg: cfg.alias || pkg,
ver,
tag,
os: myOs,
arch: myArch,
lts,
channel,
formats,
limit: 1
};
rel.oses = rels.oses;
rel.arches = rels.arches;
rel.formats = rels.formats;
if ('bat' === ext) {
return Releases.renderBatch(pkgdir, rel, opts);
} else if ('ps1' === ext) {
return Releases.renderPowerShell(pkgdir, rel, opts);
} else {
return Releases.renderBash(pkgdir, rel, opts);
}
}
}
if (!hasOs) {
hasOs = projInfo.oses.includes('ANYOS');
}
if (!hasOs) {
let pkg1 = Object.assign(buildTargetInfo, errPackage);
return [pkg1, tmplParams];
}
let targetRelease = Builds.findMatchingPackages(
projInfo,
hostTarget,
releaseTarget,
);
// { triplet: `${os}-${arch}-${libc}`, packages: targetPackages
// , latest: projInfo.versions[0], versions: matchInfo
// }
if (!targetRelease?.packages) {
let pkg1 = Object.assign(buildTargetInfo, errPackage);
return [pkg1, tmplParams];
}
let buildPkg = Builds.selectPackage(targetRelease.packages, hostFormats);
let ext = buildPkg.ext || '.exe';
if (ext.startsWith('.')) {
ext = ext.slice(1);
}
let version = targetRelease.version;
if (version.startsWith('v')) {
version = version.slice(1);
}
buildPkg = Object.assign(buildTargetInfo, buildPkg, { ext, version });
console.log('dbg: buildPkg', buildPkg);
console.log('dbg: tmplParams', tmplParams);
return [buildPkg, tmplParams];
};
let channelNames = [
'stable',
// 'hotfix',
'latest',
'rc',
'preview',
'pre',
'dev',
'beta',
'alpha',
];
function toReleaseTarget(tag) {
tag = tag.replace(/^v/, '');
let releaseTarget = {
channel: '',
lts: false,
version: '',
};
if (tag === 'lts') {
releaseTarget.lts = true;
releaseTarget.channel = 'stable';
} else if (channelNames.includes(tag)) {
releaseTarget.channel = tag;
} else {
releaseTarget.version = tag;
}
return releaseTarget;
}
var CURL_PIPE_PS1_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.ps1');
var CURL_PIPE_SH_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.sh');
var BAD_SH_RE = /[<>'"`$\\]/;
InstallerServer.getPosixCurlPipeBootstrap = async function ({
baseurl,
pkg,
ver,
}) {
let bootTxt = await Fs.readFile(CURL_PIPE_SH_BOOT, 'utf8');
var webiPkg = [pkg, ver].filter(Boolean).join('@');
var webiChecksum = await Installers.getWebiShChecksum();
var envReplacements = [
['WEBI_PKG', webiPkg],
['WEBI_HOST', baseurl],
['WEBI_CHECKSUM', webiChecksum],
];
for (let env of envReplacements) {
let name = env[0];
let value = env[1];
let envRe = new RegExp(
`^[ \\t]*#?[ \\t]*(export[ \\t])?[ \\t]*(${name})=.*`,
'm',
);
if (BAD_SH_RE.test(value)) {
throw new Error(`key '${name}' has invalid value '${value}'`);
}
bootTxt = bootTxt.replace(envRe, `$1$2='${value}'`);
}
// TODO init config here
//bootTxt.replace(/CHEATSHEET_URL/g, `${Config.cheatUrl}/${pkg}`);
return bootTxt;
};
InstallerServer.getPwshCurlPipeBootstrap = async function ({
baseurl,
pkg,
ver,
exename,
}) {
let bootTxt = await Fs.readFile(CURL_PIPE_PS1_BOOT, 'utf8');
var webiPkg = [pkg, ver].filter(Boolean).join('@');
//var webiChecksum = await InstallerServer.getWebiPs1Checksum();
var envReplacements = [
['Env:WEBI_PKG', webiPkg],
['Env:WEBI_HOST', baseurl],
//['Env:WEBI_CHECKSUM', webiChecksum],
['baseurl', baseurl],
['exename', exename],
['version', ver],
];
for (let env of envReplacements) {
let name = env[0];
let value = env[1];
if (BAD_SH_RE.test(value)) {
throw new Error(`key '${name}' has invalid value '${value}'`);
}
let tplRe = new RegExp(`{{ (${name}) }}`, 'g');
bootTxt = bootTxt.replace(tplRe, `${value}`);
let setRe = new RegExp(
`(#[ \\t]*)?(\\$${name})[ \\t]*=[ \\t]['"].*['"][ \\t]`,
'im',
);
bootTxt = bootTxt.replace(setRe, `$$${name} = '${value}'`);
}
// TODO init config here
//bootTxt.replace(/CHEATSHEET_URL/g, `${Config.cheatUrl}/${pkg}`);
return bootTxt;
});
});
};

5
_webi/template.bat Normal file
View File

@@ -0,0 +1,5 @@
REM REM debug
REM WEBI_PKG=
{{ installer }}

65
_webi/template.ps1 Normal file
View File

@@ -0,0 +1,65 @@
#!/usr/bin/env pwsh
#350 check if windows user run as admin
function Confirm-IsElevated {
$id = [System.Security.Principal.WindowsIdentity]::GetCurrent()
$p = New-Object System.Security.Principal.WindowsPrincipal($id)
if ($p.IsInRole([System.Security.Principal.WindowsBuiltInRole]::Administrator))
{ Write-Output $true }
else
{ Write-Output $false }
}
if (Confirm-IsElevated)
{ throw "Webi MUST NOT be run with elevated privileges. Please run again as a normal user, NOT as administrator." }
# this allows us to call ps1 files, which allows us to have spaces in filenames
# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
Set-ExecutionPolicy -Scope Process Bypass
# If a command returns an error, halt the script.
$ErrorActionPreference = 'Stop'
# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
$ProgressPreference = 'SilentlyContinue'
$Env:WEBI_HOST = 'https://webinstall.dev'
#$Env:WEBI_PKG = 'node@lts'
#$Env:PKG_NAME = node
#$Env:WEBI_VERSION = v12.16.2
#$Env:WEBI_PKG_URL = "https://.../node-....zip"
#$Env:WEBI_PKG_FILE = "node-v12.16.2-win-x64.zip"
# Switch to userprofile
pushd $Env:USERPROFILE
# Make paths
New-Item -Path Downloads -ItemType Directory -Force | out-null
New-Item -Path .local\bin -ItemType Directory -Force | out-null
New-Item -Path .local\opt -ItemType Directory -Force | out-null
# {{ baseurl }}
# {{ version }}
function webi_add_path
{
Write-Host ''
Write-Host '*****************************' -ForegroundColor red -BackgroundColor white
Write-Host '* IMPORTANT - READ ME *' -ForegroundColor red -BackgroundColor white
Write-Host '*****************************' -ForegroundColor red -BackgroundColor white
Write-Host ''
& "$Env:USERPROFILE\.local\bin\pathman.exe" add "$args[0]"
# Note: not all of these work as expected, so we use the unix-style, which is most consistent
#& "$Env:USERPROFILE\.local\bin\pathman.exe" add ~/.local/bin
#& "$Env:USERPROFILE\.local\bin\pathman.exe" add "$Env:USERPROFILE\.local\bin"
#& "$Env:USERPROFILE\.local\bin\pathman.exe" add %USERPROFILE%\.local\bin
}
# Run pathman to set up the folder
& "$Env:USERPROFILE\.local\bin\pathman.exe" add ~/.local/bin
{{ installer }}
# Done
popd

445
_webi/template.sh Normal file
View File

@@ -0,0 +1,445 @@
#!/bin/sh
__bootstrap_webi() {
set -e
set -u
#set -x
#WEBI_PKG=
#PKG_NAME=
# TODO should this be BASEURL instead?
#WEBI_OS=
#WEBI_ARCH=
#WEBI_HOST=
#WEBI_RELEASES=
#WEBI_CSV=
#WEBI_TAG=
#WEBI_VERSION=
#WEBI_MAJOR=
#WEBI_MINOR=
#WEBI_PATCH=
# TODO not sure if BUILD is the best name for this
#WEBI_BUILD=
#WEBI_LTS=
#WEBI_CHANNEL=
#WEBI_EXT=
#WEBI_FORMATS=
#WEBI_PKG_URL=
#WEBI_PKG_FILE=
#PKG_OSES=
#PKG_ARCHES=
#PKG_FORMATS=
WEBI_UA="$(uname -a)"
WEBI_PKG_DOWNLOAD=""
WEBI_PKG_PATH="${HOME}/Downloads/webi/${PKG_NAME:-error}/${WEBI_VERSION:-latest}"
export WEBI_HOST
##
## Set up tmp, download, and install directories
##
WEBI_TMP=${WEBI_TMP:-"$(mktemp -d -t webinstall-"${WEBI_PKG:-}".XXXXXXXX)"}
export _webi_tmp="${_webi_tmp:-"$HOME/.local/opt/webi-tmp.d"}"
mkdir -p "${WEBI_PKG_PATH}"
mkdir -p "$HOME/.local/bin"
mkdir -p "$HOME/.local/opt"
##
## Detect http client
##
set +e
WEBI_CURL="$(command -v curl)"
export WEBI_CURL
WEBI_WGET="$(command -v wget)"
export WEBI_WGET
set -e
# get the special formatted version (i.e. "go is go1.14" while node is "node v12.10.8")
my_versioned_name=""
_webi_canonical_name() {
if [ -n "$my_versioned_name" ]; then
echo "$my_versioned_name"
return 0
fi
if [ -n "$(command -v pkg_format_cmd_version)" ]; then
my_versioned_name="'$(pkg_format_cmd_version "$WEBI_VERSION")'"
else
my_versioned_name="'$pkg_cmd_name v$WEBI_VERSION'"
fi
echo "$my_versioned_name"
}
# update symlinks according to $HOME/.local/opt and $HOME/.local/bin install paths.
# shellcheck disable=2120
# webi_link may be used in the templated install script
webi_link() {
if [ -n "$(command -v pkg_link)" ]; then
pkg_link
return 0
fi
if [ -n "$WEBI_SINGLE" ] || [ "single" = "${1:-}" ]; then
rm -rf "$pkg_dst_cmd"
ln -s "$pkg_src_cmd" "$pkg_dst_cmd"
else
# 'pkg_dst' will default to $HOME/.local/opt/<pkg>
# 'pkg_src' will be the installed version, such as to $HOME/.local/opt/<pkg>-<version>
rm -rf "$pkg_dst"
ln -s "$pkg_src" "$pkg_dst"
fi
}
# detect if this program is already installed or if an installed version may cause conflict
webi_check() {
# Test for existing version
set +e
my_path="$PATH"
PATH="$(dirname "$pkg_dst_cmd"):$PATH"
export PATH
my_current_cmd="$(command -v "$pkg_cmd_name")"
set -e
if [ -n "$my_current_cmd" ]; then
my_canonical_name="$(_webi_canonical_name)"
if [ "$my_current_cmd" != "$pkg_dst_cmd" ]; then
echo >&2 "WARN: possible PATH conflict between $my_canonical_name and currently installed version"
echo >&2 " ${pkg_dst_cmd} (new)"
echo >&2 " ${my_current_cmd} (existing)"
#my_current_version=false
fi
# 'readlink' can't read links in paths on macOS 🤦
# but that's okay, 'cmp -s' is good enough for us
if cmp -s "${pkg_src_cmd}" "${my_current_cmd}"; then
echo "${my_canonical_name} already installed:"
printf " %s" "${pkg_dst}"
if [ "${pkg_src_cmd}" != "${my_current_cmd}" ]; then
printf " => %s" "${pkg_src}"
fi
echo ""
exit 0
fi
if [ -x "$pkg_src_cmd" ]; then
# shellcheck disable=2119
# this function takes no args
webi_link
echo "switched to $my_canonical_name:"
echo " ${pkg_dst} => ${pkg_src}"
exit 0
fi
fi
export PATH="$my_path"
}
is_interactive_shell() {
# $- shows shell flags (error,unset,interactive,etc)
case $- in
*i*)
# true
return 0
;;
*)
# false
return 1
;;
esac
}
# detect if file is downloaded, and how to download it
webi_download() {
# determine the url to download
if [ -n "${1:-}" ]; then
my_url="$1"
else
if [ "error" = "$WEBI_CHANNEL" ]; then
# TODO pass back requested OS / Arch / Version
echo >&2 "Error: no '$PKG_NAME' release for '${WEBI_OS:-}' on '$WEBI_ARCH' as one of '$WEBI_FORMATS' by the tag '${WEBI_TAG:-}'"
echo >&2 " '$PKG_NAME' is available for '$PKG_OSES' on '$PKG_ARCHES' as one of '$PKG_FORMATS'"
echo >&2 " (check that the package name and version are correct)"
echo >&2 ""
echo >&2 " Double check at $(echo "$WEBI_RELEASES" | sed 's:\?.*::')"
echo >&2 ""
exit 1
fi
my_url="$WEBI_PKG_URL"
fi
# determine the location to download to
if [ -n "${2:-}" ]; then
my_dl="$2"
else
my_dl="${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
fi
WEBI_PKG_DOWNLOAD="${my_dl}"
export WEBI_PKG_DOWNLOAD
if [ -e "$my_dl" ]; then
echo "Found $my_dl"
return 0
fi
echo "Downloading $PKG_NAME from"
echo "$my_url"
# It's only 2020, we can't expect to have reliable CLI tools
# to tell us the size of a file as part of a base system...
if [ -n "$WEBI_WGET" ]; then
# wget has resumable downloads
# TODO wget -c --content-disposition "$my_url"
set +e
my_show_progress=""
if is_interactive_shell; then
my_show_progress="--show-progress"
fi
if ! wget -q $my_show_progress --user-agent="wget $WEBI_UA" -c "$my_url" -O "$my_dl.part"; then
echo >&2 "failed to download from $WEBI_PKG_URL"
exit 1
fi
set -e
else
# Neither GNU nor BSD curl have sane resume download options, hence we don't bother
# TODO curl -fsSL --remote-name --remote-header-name --write-out "$my_url"
my_show_progress="-#"
if is_interactive_shell; then
my_show_progress=""
fi
# shellcheck disable=SC2086
# we want the flags to be split
curl -fSL $my_show_progress -H "User-Agent: curl $WEBI_UA" "$my_url" -o "$my_dl.part"
fi
mv "$my_dl.part" "$my_dl"
echo ""
echo "Saved as $my_dl"
}
# detect which archives can be used
webi_extract() {
(
cd "$WEBI_TMP"
if [ "tar" = "$WEBI_EXT" ]; then
echo "Extracting ${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
tar xf "${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
elif [ "zip" = "$WEBI_EXT" ]; then
echo "Extracting ${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
unzip "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > __unzip__.log
elif [ "exe" = "$WEBI_EXT" ]; then
echo "Moving ${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
mv "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" .
elif [ "xz" = "$WEBI_EXT" ]; then
echo "Inflating ${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
unxz -c "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > "$(basename "$WEBI_PKG_FILE")"
else
# do nothing
echo "Failed to extract ${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
exit 1
fi
)
}
# use 'pathman' to update $HOME/.config/envman/PATH.env
webi_path_add() {
# make sure that we don't recursively install pathman with webi
my_path="$PATH"
export PATH="$HOME/.local/bin:$PATH"
# install pathman if not already installed
if [ -z "$(command -v pathman)" ]; then
"$HOME/.local/bin/webi" pathman > /dev/null
fi
export PATH="$my_path"
# in case pathman was recently installed and the PATH not updated
mkdir -p "$_webi_tmp"
# 'true' to prevent "too few arguments" output
# when there are 0 lines of stdout
"$HOME/.local/bin/pathman" add "$1" |
grep "export" 2> /dev/null \
>> "$_webi_tmp/.PATH.env" ||
true
}
# group common pre-install tasks as default
webi_pre_install() {
webi_check
webi_download
webi_extract
}
# move commands from the extracted archive directory to $HOME/.local/opt or $HOME/.local/bin
# shellcheck disable=2120
# webi_install may be sourced and used elsewhere
webi_install() {
if [ -n "$WEBI_SINGLE" ] || [ "single" = "${1:-}" ]; then
mkdir -p "$(dirname "$pkg_src_cmd")"
mv ./"$pkg_cmd_name"* "$pkg_src_cmd"
else
rm -rf "$pkg_src"
mv ./"$pkg_cmd_name"* "$pkg_src"
fi
}
# run post-install functions - just updating PATH by default
webi_post_install() {
webi_path_add "$(dirname "$pkg_dst_cmd")"
}
_webi_enable_exec() {
if [ -n "$(command -v spctl)" ] && [ -n "$(command -v xattr)" ]; then
# note: some packages contain files that cannot be affected by xattr
xattr -r -d com.apple.quarantine "$pkg_src" || true
return 0
fi
# TODO need to test that the above actually worked
# (and proceed to this below if it did not)
if [ -n "$(command -v spctl)" ]; then
echo "Checking permission to execute '$pkg_cmd_name' on macOS 11+"
set +e
is_allowed="$(spctl -a "$pkg_src_cmd" 2>&1 | grep valid)"
set -e
if [ -z "$is_allowed" ]; then
echo ""
echo "##########################################"
echo "# IMPORTANT: Permission Grant Required #"
echo "##########################################"
echo ""
echo "Requesting permission to execute '$pkg_cmd_name' on macOS 10.14+"
echo ""
sleep 3
spctl --add "$pkg_src_cmd"
fi
fi
}
# a friendly message when all is well, showing the final install path in $HOME/.local
_webi_done_message() {
echo "Installed $(_webi_canonical_name) as $pkg_dst_cmd"
}
##
##
## BEGIN custom override functions from <package>/install.sh
##
##
WEBI_SINGLE=
if [ -z "${WEBI_WELCOME:-}" ]; then
echo ""
printf "Thanks for using webi to install '\e[32m%s\e[0m' on '\e[31m%s/%s\e[0m'.\n" "${WEBI_PKG:-}" "$(uname -s)" "$(uname -m)"
echo "Have a problem? Experience a bug? Please let us know:"
echo " https://github.com/webinstall/webi-installers/issues"
echo ""
printf "\e[31mLovin'\e[0m it? Say thanks with a \e[34mStar on GitHub\e[0m:\n"
printf " \e[32mhttps://github.com/webinstall/webi-installers\e[0m\n"
echo ""
fi
__init_installer() {
# do nothing - to satisfy parser prior to templating
printf ""
# {{ installer }}
}
__init_installer
##
##
## END custom override functions
##
##
# run everything with defaults or overrides as needed
if command -v pkg_install > /dev/null ||
command -v pkg_link > /dev/null ||
command -v pkg_post_install > /dev/null ||
command -v pkg_done_message > /dev/null ||
command -v pkg_format_cmd_version > /dev/null ||
[ -n "${WEBI_SINGLE:-}" ] ||
[ -n "${pkg_cmd_name:-}" ] ||
[ -n "${pkg_dst_cmd:-}" ] ||
[ -n "${pkg_dst_dir:-}" ] ||
[ -n "${pkg_dst:-}" ] ||
[ -n "${pkg_src_cmd:-}" ] ||
[ -n "${pkg_src_dir:-}" ] ||
[ -n "${pkg_src:-}" ]; then
pkg_cmd_name="${pkg_cmd_name:-$PKG_NAME}"
if [ -n "$WEBI_SINGLE" ]; then
pkg_dst_cmd="${pkg_dst_cmd:-$HOME/.local/bin/$pkg_cmd_name}"
pkg_dst="$pkg_dst_cmd" # "$(dirname "$(dirname $pkg_dst_cmd)")"
#pkg_src_cmd="${pkg_src_cmd:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION/bin/$pkg_cmd_name-v$WEBI_VERSION}"
pkg_src_cmd="${pkg_src_cmd:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION/bin/$pkg_cmd_name}"
pkg_src="$pkg_src_cmd" # "$(dirname "$(dirname $pkg_src_cmd)")"
else
pkg_dst="${pkg_dst:-$HOME/.local/opt/$pkg_cmd_name}"
pkg_dst_cmd="${pkg_dst_cmd:-$pkg_dst/bin/$pkg_cmd_name}"
pkg_src="${pkg_src:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION}"
pkg_src_cmd="${pkg_src_cmd:-$pkg_src/bin/$pkg_cmd_name}"
fi
# this script is templated and these are used elsewhere
# shellcheck disable=SC2034
pkg_src_bin="$(dirname "$pkg_src_cmd")"
# shellcheck disable=SC2034
pkg_dst_bin="$(dirname "$pkg_dst_cmd")"
if [ -n "$(command -v pkg_pre_install)" ]; then pkg_pre_install; else webi_pre_install; fi
(
cd "$WEBI_TMP"
echo "Installing to $pkg_src_cmd"
if [ -n "$(command -v pkg_install)" ]; then pkg_install; else webi_install; fi
chmod a+x "$pkg_src"
chmod a+x "$pkg_src_cmd"
)
webi_link
_webi_enable_exec
(
cd "$WEBI_TMP"
if [ -n "$(command -v pkg_post_install)" ]; then pkg_post_install; else webi_post_install; fi
)
(
cd "$WEBI_TMP"
if [ -n "$(command -v pkg_done_message)" ]; then pkg_done_message; else _webi_done_message; fi
)
echo ""
fi
webi_path_add "$HOME/.local/bin"
if [ -z "${_WEBI_CHILD:-}" ] && [ -f "$_webi_tmp/.PATH.env" ]; then
if [ -n "$(cat "$_webi_tmp/.PATH.env")" ]; then
printf 'PATH.env updated with:\n'
sort -u "$_webi_tmp/.PATH.env"
printf "\n"
rm -f "$_webi_tmp/.PATH.env"
printf "\e[31mTO FINISH\e[0m: copy, paste & run the following command:\n"
printf "\n"
printf " \e[34msource ~/.config/envman/PATH.env\e[0m\n"
printf " (newly opened terminal windows will update automatically)\n"
fi
fi
# cleanup the temp directory
rm -rf "$WEBI_TMP"
# See? No magic. Just downloading and moving files.
}
__bootstrap_webi

View File

@@ -5,7 +5,7 @@
//
var usage = [
'Usage: node _webi/test.js --debug <path-to-package>',
'Example: node _webi/test.js --debug ./node/',
'Example: node _webi/test.js --debug ./node/'
].join('\n');
var count = 3;
@@ -32,10 +32,8 @@ if (/\b-?-h(elp)?\b/.test(process.argv.join(' '))) {
var os = require('os');
var fs = require('fs');
var path = require('path');
var Builds = require('./builds.js');
var Installers = require('./installers.js');
var ServeInstaller = require('./serve-installer.js');
var Releases = require('./releases.js');
var uaDetect = require('./ua-detect.js');
var pkg = process.argv[2].split('@');
var pkgdir = pkg[0];
var pkgtag = pkg[1] || '';
@@ -44,7 +42,6 @@ var nodes = fs.readdirSync(pkgdir);
nodes.forEach(function (node) {
nodesMap[node] = true;
});
var baseurl = 'https://webinstall.dev';
var maxLen = 0;
console.info('');
@@ -65,49 +62,31 @@ console.info('Has the necessary files?');
});
console.info('');
let projName = pkgdir.split('/').filter(Boolean).pop();
Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
Releases.get(path.join(process.cwd(), pkgdir)).then(function (all) {
var pkgname = path.basename(pkgdir.replace(/\/$/, ''));
var nodeOs = os.platform();
var nodeOsRelease = os.release();
var nodeArch = os.arch();
var nodeLibc = 'libc';
if (process.platform === 'linux') {
nodeLibc = 'gnu';
let isUnofficial =
process.config.variables.node_release_urlbase.includes('unofficial');
if (isUnofficial) {
nodeLibc = 'musl';
}
}
var formats = ['exe', 'xz', 'tar', 'zip', 'git'];
var osrel = os.platform() + '-' + os.release();
var arch = os.arch();
var formats = ['exe', 'xz', 'tar', 'zip'];
let [rel, opts] = await ServeInstaller.helper({
unameAgent: `${nodeOs}/${nodeOsRelease} ${nodeArch}/unknown ${nodeLibc}`,
projectName: pkgname,
tag: pkgtag || '',
formats: formats,
libc: nodeLibc,
});
Object.assign(
{
ver: '',
lts: null,
channel: '',
os: '',
arch: '',
limit: 0,
},
opts,
{
baseurl,
},
);
var rel = all.releases.filter(function (rel) {
return (
formats.filter(function (ext) {
return rel.ext.match(ext);
})[0] &&
'stable' === rel.channel &&
rel.os === uaDetect.os(osrel) &&
rel.arch === uaDetect.arch(arch) &&
(!pkgtag ||
rel.tag === pkgtag ||
new RegExp('^' + pkgtag).test(rel.version))
);
})[0];
rel.oses = all.oses;
rel.arches = all.arches;
rel.formats = all.formats;
if (!rel) {
console.error(
`Error: ❌ no release found for @${pkgtag}?os=${nodeOs}&arch=${nodeArch}&libc=${nodeLibc}&formats=${formats}`,
);
console.error('Error: ❌ no release found for current os, arch, and tag');
process.exit(1);
return;
}
@@ -118,8 +97,24 @@ Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
console.info('');
return Promise.all([
Installers.renderBash(pkgdir, rel, opts).catch(function () {}),
Installers.renderPowerShell(pkgdir, rel, opts).catch(function () {}),
Releases.renderBash(pkgdir, rel, {
baseurl: 'https://webinstall.dev',
pkg: pkgname,
tag: pkgtag || '',
ver: '',
os: osrel,
arch,
formats: formats
}).catch(function () {}),
Releases.renderPowerShell(pkgdir, rel, {
baseurl: 'https://webinstall.dev',
pkg: pkgname,
tag: pkgtag || '',
ver: '',
os: osrel,
arch,
formats: formats
}).catch(function () {})
]).then(function (scripts) {
var bashTxt = scripts[0];
var ps1Txt = scripts[1];
@@ -130,7 +125,7 @@ Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
bashTxt = (bashTxt || 'echo ERROR').replace(/#set -x/g, 'set -x');
ps1Txt = (ps1Txt || 'echo ERROR').replace(
/REM REM todo debug/g,
'REM todo debug',
'REM todo debug'
);
}
console.info('Do the scripts actually work?');
@@ -143,9 +138,5 @@ Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
console.info('\tNEEDS MANUAL TEST: powershell.exe %s', ps1File);
}
console.info('');
setTimeout(function () {
console.warn(`[warn] dangling event loop handle`);
process.exit(0);
}, 300).unref();
});
});

View File

@@ -1,37 +1,13 @@
'use strict';
var Releases = module.exports;
var path = require('path');
var _normalize = require('./normalize.js');
var Releases = require('./releases.js');
var cache = {};
//var staleAge = 5 * 1000;
//var expiredAge = 15 * 1000;
var staleAge = 5 * 60 * 1000;
var expiredAge = 15 * 60 * 1000;
var staleAge = 5 * 1000;
var expiredAge = 15 * 1000;
let installerDir = path.join(__dirname, '..');
Releases.get = async function (pkgdir) {
let get;
try {
get = require(`${pkgdir}/releases.js`);
// TODO update all releases files with module.exports.xxxx = 'foo';
if (!get.latest) {
get.latest = get;
}
} catch (e) {
let err = new Error('no releases.js for', pkgdir.split(/[\/\\]+/).pop());
err.code = 'E_NO_RELEASE';
throw err;
}
let all = await get.latest();
return _normalize(all);
};
// TODO needs a proper test, and more accurate (though perhaps far less simple) code
function createFormatsSorter(formats) {
return function sortByVerExt(a, b) {
@@ -74,14 +50,13 @@ function createFormatsSorter(formats) {
return 1;
}
// rank builds that don't depend on any form of libc first
if (a.libc === 'none' && b.libc !== 'none') {
// Hacky-doo for linux: prefer musl
if (a._musl && !b._musl) {
return -1;
}
if (a.libc !== 'none' && b.libc === 'none') {
if (!a._musl && b._musl) {
return 1;
}
return 0;
};
}
@@ -89,102 +64,64 @@ function createFormatsSorter(formats) {
async function getCachedReleases(pkg) {
// returns { download: '<template string>', releases: [{ version, date, os, arch, lts, channel, download}] }
async function chainCachePromise(fn) {
cache[pkg].promise = cache[pkg].promise.then(fn);
return cache[pkg].promise;
}
async function sleep(ms) {
return await new Promise(function (resolve, reject) {
setTimeout(resolve, ms);
});
}
async function putCache() {
var age = Date.now() - cache[pkg].updatedAt;
if (age < staleAge) {
//console.debug('NOT STALE ANYMORE - updated in previous promise');
return cache[pkg].all;
}
//console.debug('DOWNLOADING NEW "%s" releases', pkg);
var pkgdir = path.join(installerDir, pkg);
// workaround for request timeout seeming to not work
let complete = false;
await Promise.race([
Releases.get(pkgdir)
.catch(function (err) {
if ('E_NO_RELEASE' === err.code) {
let all = { _error: 'E_NO_RELEASE', download: '', releases: [] };
return all;
}
throw err;
})
.catch(function (err) {
let hasReleases = cache[pkg].all?.releases?.length > 1;
if (!hasReleases) {
throw err;
}
console.error(`Error: the BOOGEYMAN got us!`);
console.error(err.stack);
return cache[pkg].all;
})
function putCache() {
cache[pkg].promise = cache[pkg].promise.then(function () {
var age = Date.now() - cache[pkg].updatedAt;
if (age < staleAge) {
//console.debug('NOT STALE ANYMORE - updated in previous promise');
return cache[pkg].all;
}
//console.debug('DOWNLOADING NEW "%s" releases', pkg);
var pkgdir = path.join(installerDir, pkg);
return Releases.get(pkgdir)
.then(function (all) {
// Note: it is possible for slightly older data
// to replace slightly newer data, but this is better
// than being in a cycle where release updates _always_
// take longer than expected.
//console.debug('DOWNLOADED NEW "%s" releases', pkg);
cache[pkg].updatedAt = Date.now();
cache[pkg].all = all;
complete = true;
}),
sleep(15000).then(function () {
if (complete) {
return;
}
console.error(`request timeout waiting for '${pkg}' release info`);
}),
]);
return cache[pkg].all;
})
.catch(function (e) {
console.error(
'Error fetching releases for "%s": %s',
pkg,
e.toString()
);
cache[pkg].all = { download: '', releases: [] };
})
.then(function () {
return cache[pkg].all;
});
});
return cache[pkg].promise;
}
var p;
if (!cache[pkg]) {
cache[pkg] = {
updatedAt: 0,
all: { download: '', releases: [] },
promise: Promise.resolve(),
all: null,
promise: Promise.resolve()
};
}
var bgRenewal;
var age = Date.now() - cache[pkg].updatedAt;
var fresh = age < staleAge;
if (!fresh) {
bgRenewal = chainCachePromise(putCache);
if (age >= expiredAge) {
//console.debug("EXPIRED - waiting");
p = putCache();
} else if (age >= staleAge) {
//console.debug("STALE - background update");
putCache();
p = Promise.resolve(cache[pkg].all);
} else {
//console.debug("FRESH");
p = Promise.resolve(cache[pkg].all);
}
var tooStale = age > expiredAge;
if (!tooStale) {
return await cache[pkg].all;
}
return await Promise.race([
bgRenewal,
sleep(5000).then(function () {
return cache[pkg].all;
}),
]);
return p;
}
async function filterReleases(
all,
{ ver, os, arch, libc, lts, channel, formats, limit },
{ ver, os, arch, lts, channel, formats, limit }
) {
// When multiple formats are downloadable (i.e. .zip and .pkg)
// sort the most compatible format first
@@ -193,87 +130,40 @@ async function filterReleases(
var sortByVerExt = createFormatsSorter(rformats);
var reVer = new RegExp('^' + ver + '\\b');
function selectMatches(rel) {
if (os) {
if (rel.os !== '*') {
if (rel.os !== os) {
return false;
}
}
}
if (arch) {
if (rel.arch !== '*') {
if (rel.arch !== arch) {
return false;
}
}
}
if (rel.libc !== 'none') {
let releaseRequiresMusl = rel.libc === 'musl';
// goal: handle non-glibc (Alpine / Docker / musl)
let osHasMusl = libc === 'musl';
if (osHasMusl) {
// goal: fail if dependent on libc
let releaseRequiresLibc = rel.libc === 'gnu';
if (releaseRequiresLibc) {
return false;
}
} else if (releaseRequiresMusl) {
// goal: don't use musl++ on glibc (Ubuntu, GNU, etc)
var sortedRels = all.releases
.filter(function (rel) {
if (
(os && rel.os !== os) ||
// Hacky-doo for linux musl
(arch && rel.arch !== arch) ||
(lts && !rel.lts) ||
(channel && rel.channel !== channel) ||
// to match 'tar.gz' and 'tar.xz' with just 'tar'
(formats.length &&
!formats.some(function (ext) {
return rel.ext.match(ext);
})) ||
(ver && !rel.version.match(reVer))
) {
return false;
}
}
if (lts) {
if (!rel.lts) {
return false;
}
}
if (channel) {
if (rel.channel !== channel) {
return false;
}
}
// to match 'tar.gz' and 'tar.xz' with just 'tar'
function hasExt(ext) {
return rel.ext.match(ext);
}
if (formats.length) {
if (!formats.some(hasExt)) {
return false;
}
}
if (ver) {
if (!rel.version.match(reVer)) {
return false;
}
}
return true;
}
var sortedRels = all.releases.filter(selectMatches).sort(sortByVerExt);
return true;
})
.sort(sortByVerExt);
//console.log(sortedRels.slice(0, 4));
return sortedRels.slice(0, limit || 1000);
}
Releases.getReleases = function ({
module.exports = function getReleases({
_count,
pkg,
ver,
os,
arch,
libc,
lts,
channel,
formats,
limit,
limit
}) {
if (!_count) {
_count = 0;
@@ -283,133 +173,102 @@ Releases.getReleases = function ({
ver,
os,
arch,
libc,
lts,
channel,
formats,
limit,
limit
})
.catch(function (err) {
if ('MODULE_NOT_FOUND' === err.code) {
return null;
}
console.error(
'TODO: lib/release.js: check type of error, such as MODULE_NOT_FOUND',
'TODO: lib/release.js: check type of error, such as MODULE_NOT_FOUND'
);
console.error(err);
})
.then(function (releases) {
if (releases.length) {
return {
oses: all.oses,
arches: all.arches,
libcs: all.libcs,
formats: all.formats,
releases: releases,
};
}
if (_count < 1) {
// Apple Silicon M1 hacky-do workaround fix
if (!releases.length) {
// Apple Silicon M1 hack-y do workaround fix
if ('macos' === os && 'arm64' === arch) {
return Releases.getReleases({
return getReleases({
pkg,
ver,
os,
arch: 'amd64',
libc,
lts,
channel,
formats,
limit,
limit
});
}
// Windows ARM hacky-do workaround fix
if ('windows' === os && 'arm64' === arch) {
return Releases.getReleases({
pkg,
ver,
os,
arch: 'amd64',
libc,
lts,
channel,
formats,
limit,
});
}
// Raspberry Pi 3+ on Ubuntu arm64 (via Bionic?)
if ('linux' === os && 'arm64' === arch) {
return Releases.getReleases({
_count: _count + 1,
pkg,
ver,
os,
arch: 'armv7l',
libc,
lts,
channel,
formats,
limit,
});
}
// armv7 can run armv6
if ('linux' === os && 'armv7l' === arch) {
return Releases.getReleases({
_count: _count + 1,
pkg,
ver,
os,
arch: 'armv6l',
libc,
lts,
channel,
formats,
limit,
});
}
}
if (_count < 2) {
// Raspberry Pi 3+ on Raspbian arm7 (not Ubuntu arm64)
// hail mary
if ('linux' === os && 'armv7l' === arch) {
return Releases.getReleases({
// Raspberry Pi 3+ on Raspbian x86 (not Ubuntu arm64)
if (!_count && 'linux' === os && 'armv7l' === arch) {
return getReleases({
_count: _count + 1,
pkg,
ver,
os,
arch: 'arm64',
libc,
lts,
channel,
formats,
limit,
limit
});
}
// Raspberry Pi 3+ on Ubuntu arm64 (via Bionic?)
// (this may be the same as the prior search, that's okay)
if ('linux' === os && 'arm64' === arch) {
return getReleases({
_count: _count + 1,
pkg,
ver,
os,
arch: 'armv7l',
lts,
channel,
formats,
limit
});
}
// Raspberry Pi 3+ on Ubuntu arm64 (via Bionic?)
if ('linux' === os && 'armv7l' === arch) {
return getReleases({
_count: _count + 1,
pkg,
ver,
os,
arch: 'armv6l',
lts,
channel,
formats,
limit
});
}
releases = [
{
name: 'doesntexist.ext',
version: '0.0.0',
lts: '-',
channel: 'error',
date: '1970-01-01',
os: os || '-',
arch: arch || '-',
_musl: undefined,
ext: 'err',
download: 'https://example.com/doesntexist.ext',
comment:
'No matches found. Could be bad or missing version info' +
',' +
"Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&limit=100'"
}
];
}
releases = [
{
name: 'doesntexist.ext',
version: '0.0.0',
lts: '-',
channel: 'error',
date: '1970-01-01',
os: os || '-',
arch: arch || '-',
libc: libc || '-',
ext: 'err',
download: 'https://example.com/doesntexist.ext',
comment:
'No matches found. Could be bad or missing version info' +
',' +
"Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
},
];
return {
oses: all.oses,
arches: all.arches,
libcs: all.libcs,
formats: all.formats,
releases: releases,
releases: releases
};
});
});
@@ -423,10 +282,9 @@ if (require.main === module) {
os: 'macos',
arch: 'amd64',
lts: true,
libc: 'libc',
channel: 'stable',
formats: ['tar', 'exe', 'zip', 'xz', 'dmg', 'pkg'],
limit: 10,
limit: 10
})
.then(function (all) {
console.info(JSON.stringify(all));

View File

@@ -16,11 +16,11 @@ function getRequest(req) {
}
return {
unix: 'curl -fsSA "$(uname -srm)" ' + url,
unix: 'curl -fsSA "$(uname -a)" ' + url,
windows: 'curl.exe -fsSA "MS $Env:PROCESSOR_ARCHITECTURE" ' + url,
ua: ua,
os: uaDetect.os(ua),
arch: uaDetect.arch(ua),
arch: uaDetect.arch(ua)
};
}
@@ -56,23 +56,16 @@ function getArch(ua) {
return '-';
}
// Quick hack for Apple Silicon M1
//
// Note: we now use `uname -srm` which does not have the native arch
// info included with `uname -v` and `uname -a`.
//
// quick hack for Apple Silicon M1
// Native: Darwin boomer.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec 2 20:40:21 PST 2020; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 arm64
// Resetta: Darwin boomer.local 20.2.0 Darwin Kernel Version 20.2.0: Wed Dec 2 20:40:21 PST 2020; root:xnu-7195.60.75~1/RELEASE_ARM64_T8101 x86_64
ua = ua.replace(/xnu-.*RELEASE_[^\s]*/, '');
if (/aarch64|arm64|arm8|armv8/i.test(ua)) {
return 'arm64';
} else if (/aarch|arm7|armv7|arm32/i.test(ua)) {
} else if (/aarch|arm7|armv7/i.test(ua)) {
return 'armv7l';
} else if (/arm6|armv6|arm(\b|_)/i.test(ua)) {
} else if (/arm6|armv6/i.test(ua)) {
return 'armv6l';
} else if (/ppc64le/i.test(ua)) {
return 'ppc64le';
} else if (/ppc64/i.test(ua)) {
return 'ppc64';
} else if (/mips64/i.test(ua)) {
@@ -91,63 +84,6 @@ function getArch(ua) {
}
}
function getLibc(ua) {
if ('-' === ua) {
return '-';
}
// How to see a bunch of target host quadruples:
// go tool dist list
// rustup target list
// zig targets | jq -r '.libc[]' | sort -
// Generally, we prefer 'musl' builds because they DO work on glibc systems (Ubuntu),
// but 'glibc' builds will NOT work on musl systems (Alpine / Docker).
//
// However, there are a few instances (ex: Node.js), where the 'musl' builds
// DO NOT work on glibc systems.
{
let muslRe = /(\b|_)(musl)(\b|_)/i;
if (muslRe.test(ua)) {
return 'musl';
}
}
{
let msvcRe = /(\b|_)(msvc|windows|microsoft)(\b|_)/i;
if (msvcRe.test(ua)) {
return 'msvc';
}
}
{
let gnuRe = /(\b|_)(gnu|glibc|linux)(\b|_)/i;
if (gnuRe.test(ua)) {
return 'gnu';
}
}
{
let libcRe = /(\b|_)(libc)(\b|_)/i;
if (libcRe.test(ua)) {
return 'libc';
}
}
{
let darwinRe = /(\b|_)(darwin)(\b|_)/i;
if (darwinRe.test(ua)) {
// not sure whether this should be "darwin" or "libc" or "none"
// https://opensource.apple.com/source/Libc/
return 'libc';
}
}
return 'libc';
}
uaDetect.os = getOs;
uaDetect.arch = getArch;
uaDetect.libc = getLibc;
uaDetect.request = getRequest;

126
_webi/webi-pwsh.ps1 Normal file
View File

@@ -0,0 +1,126 @@
#!/usr/bin/env pwsh
# this allows us to call ps1 files, which allows us to have spaces in filenames
# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
Set-ExecutionPolicy -Scope Process Bypass
# If a command returns an error, halt the script.
$ErrorActionPreference = 'Stop'
# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
$ProgressPreference = 'SilentlyContinue'
# This is the canonical CPU arch when the process is emulated
$my_arch = "$Env:PROCESSOR_ARCHITEW6432"
IF ($my_arch -eq $null -or $my_arch -eq "") {
# This is the canonical CPU arch when the process is native
$my_arch = "$Env:PROCESSOR_ARCHITECTURE"
}
# TODO API should know to prefer x86 for windows when arm binary is not available
$Env:WEBI_UA = "Windows/10 $my_arch"
$exename = $args[0]
# Switch to userprofile
pushd $Env:USERPROFILE
# Make paths if needed
New-Item -Path .local\bin -ItemType Directory -Force | out-null
# TODO replace all xbin with opt\bin\
New-Item -Path .local\xbin -ItemType Directory -Force | out-null
# See note on Set-ExecutionPolicy above
Set-Content -Path .local\bin\webi.bat -Value "@echo off`r`npushd %USERPROFILE%`r`npowershell -ExecutionPolicy Bypass .local\bin\webi-pwsh.ps1 %1`r`npopd"
# Backwards-compat bugfix: remove old webi-pwsh.ps1 location
Remove-Item -Path .local\bin\webi.ps1 -Recurse -ErrorAction Ignore
if (!(Test-Path -Path .local\opt))
{
New-Item -Path .local\opt -ItemType Directory -Force | out-null
}
# TODO windows version of mktemp -d
if (!(Test-Path -Path .local\tmp))
{
New-Item -Path .local\tmp -ItemType Directory -Force | out-null
}
# TODO SetStrictMode
# TODO Test-Path variable:global:Env:WEBI_HOST ???
IF($Env:WEBI_HOST -eq $null -or $Env:WEBI_HOST -eq "")
{
$Env:WEBI_HOST = "https://webinstall.dev"
}
if (!(Test-Path -Path .local\bin\pathman.exe))
{
& curl.exe -fsSL -A "$Env:WEBI_UA" "$Env:WEBI_HOST/packages/pathman/install.ps1" -o .\.local\tmp\pathman-setup.ps1
powershell .\.local\tmp\pathman-setup.ps1
# TODO del .\.local\tmp\pathman-setup.bat
}
# Run pathman to set up the folder
# (using unix style path because... cmd vs powershell vs whatever)
$has_local_bin = echo "$Env:PATH" | Select-String -Pattern '\.local.bin'
if (!$has_local_bin)
{
Write-Host ''
Write-Host '**********************************' -ForegroundColor red -BackgroundColor white
Write-Host '* IMPORTANT -- READ ME *' -ForegroundColor red -BackgroundColor white
Write-Host '* (run the PATH command below) *' -ForegroundColor red -BackgroundColor white
Write-Host '**********************************' -ForegroundColor red -BackgroundColor white
Write-Host ''
& "$Env:USERPROFILE\.local\bin\pathman.exe" add ~/.local/bin
}
# {{ baseurl }}
# {{ version }}
$my_version = 'v1.1.15'
## show help if no params given or help flags are used
if ($exename -eq $null -or $exename -eq "-h" -or $exename -eq "--help" -or $exename -eq "help" -or $exename -eq "/?") {
Write-Host "webi " -ForegroundColor Green -NoNewline; Write-Host "$my_version " -ForegroundColor Red -NoNewline; Write-Host "Copyright 2020+ AJ ONeal"
Write-Host " https://webinstall.dev/webi" -ForegroundColor blue
echo ""
echo "SUMMARY"
echo " Webi is the best way to install the modern developer tools you love."
echo " It's fast, easy-to-remember, and conflict free."
echo ""
echo "USAGE"
echo " webi <thing1>[@version] [thing2] ..."
echo ""
echo "UNINSTALL"
echo " Almost everything that is installed with webi is scoped to"
echo " ~/.local/opt/<thing1>, so you can remove it like so:"
echo ""
echo " rmdir /s %USERPROFILE%\.local\opt\<thing1>"
echo " del %USERPROFILE%\.local\bin\<thing1>"
echo ""
echo " Some packages have special uninstall instructions, check"
echo " https://webinstall.dev/<thing1> to be sure."
echo ""
echo "FAQ"
Write-Host " See " -NoNewline; Write-Host "https://webinstall.dev/faq" -ForegroundColor blue
echo ""
echo "ALWAYS REMEMBER"
echo " Friends don't let friends use brew for simple, modern tools that don't need it."
exit 0
}
if ($exename -eq "-V" -or $exename -eq "--version" -or $exename -eq "version" -or $exename -eq "/v") {
Write-Host "webi " -ForegroundColor Green -NoNewline; Write-Host "$my_version " -ForegroundColor Red -NoNewline; Write-Host "Copyright 2020+ AJ ONeal"
Write-Host " https://webinstall.dev/webi" -ForegroundColor blue
exit 0
}
# Fetch <whatever>.ps1
# TODO detect formats
$PKG_URL = "$Env:WEBI_HOST/api/installers/$exename.ps1?formats=zip,exe,tar"
echo "Downloading $PKG_URL"
# Invoke-WebRequest -UserAgent "Windows amd64" "$PKG_URL" -OutFile ".\.local\tmp\$exename.install.ps1"
& curl.exe -fsSL -A "$Env:WEBI_UA" "$PKG_URL" -o .\.local\tmp\$exename.install.ps1
# Run <whatever>.ps1
powershell .\.local\tmp\$exename.install.ps1
# Done
popd

38
_webi/webi.bat Normal file
View File

@@ -0,0 +1,38 @@
@echo off
pushd "%userprofile%" || goto :error
IF NOT EXIST .local (
mkdir .local || goto :error
)
IF NOT EXIST .local\bin (
mkdir .local\bin || goto :error
)
IF NOT EXIST .local\opt (
mkdir .local\opt || goto :error
)
pushd .local\bin || goto :error
if NOT EXIST pathman.exe (
echo updating PATH management
powershell $ProgressPreference = 'SilentlyContinue'; Invoke-WebRequest https://webinstall.dev/packages/pathman/install.bat -OutFile pathman-setup.bat || goto :error
call .\pathman-setup.bat || goto :error
del pathman-setup.bat || goto :error
rem TODO there's rumor of a windows tool called 'pathman' that does the same thing?
)
popd || goto :error
.\.local\bin\pathman add ".local\bin" || goto :error
echo downloading and installing %1
powershell $ProgressPreference = 'SilentlyContinue'; Invoke-WebRequest https://webinstall.dev/packages/%1/install.ps1 -OutFile %1-webinstall.bat || goto :error
rem TODO only add if it's not in there already
PATH .local\bin;%PATH%
call %1-webinstall.bat || goto :error
del %1-webinstall.bat || goto :error
popd
goto :EOF
:error
echo Failed with error #%errorlevel%.
exit /b %errorlevel%

1
_webi/webi.ps1.bat Normal file
View File

@@ -0,0 +1 @@
powershell -ExecutionPolicy Bypass .\.local\bin\webi-pwsh.ps1 %1

View File

@@ -1,125 +0,0 @@
---
title: aliasman
homepage: https://github.com/BeyondCodeBootcamp/aliasman
tagline: |
aliasman: A cross-shell (POSIX-compliant) alias manager for bash, zsh, and fish
---
To update or switch versions, run `webi aliasman@stable` (or `@v1.0.0`, `@beta`,
etc).
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.config/envman/alias.env
~/.local/bin/aliasman
```
## Cheat Sheet
> `aliasman` helps you love your *alias*es again! 🥸 \
> Set 'em once, use 'em everywhere! \
> (and all in just one dotfile, for an on-the-go friendly config)
```sh
aliasman <alias-name> <command-name> [args, pipes, redirs, etc]
```
### What are Aliases?
An _alias_ is just shorthand for a shell function.
Take a long command like this:
```sh
git commit -m "feat: new feature"
```
And turn it into a short command, like this:
```sh
gc "feat: new feature"
```
(that would be `aliasman gc 'git commit -m'`)
### Imagine the possibilities!
1. What if you could quickly create a _command_, `ll`, \
that does the work of `ls -lAhF`!?
2. Set an _alias_ to do just that!
```sh
aliasman ll 'ls -lAhF'
```
3. Reload your alias config (or open a _new terminal_)
```sh
source ~/.config/envman/alias.env
```
4. Use it!
```sh
ll
```
```text
drwxr-xr-x aj wheel 416 B Thu Feb 9 02:08:39 2023 📂 .git/
.rwxr-xr-x aj staff 6.2 KB Thu Feb 9 01:36:30 2023 💻 aliasman*
.rw-r--r-- aj wheel 16 KB Wed Feb 8 21:51:06 2023 🔑 LICENSE
.rw-r--r-- aj wheel 1.4 KB Thu Feb 9 01:47:13 2023 📄 README.md
```
### Common aliases
Use *alias*es to make other tools you find around webi even _more_ convenient ⚡️
(and powerful 💪).
```sh
aliasman curl 'curlie'
aliasman diffy 'diff -y --suppress-common-lines'
aliasman gc 'git commit -m'
aliasman gri 'git rebase -i'
aliasman la 'lsd -AF'
aliasman ll 'lsd -lAhF'
aliasman ls 'lsd -F'
aliasman rgi 'rg -i'
aliasman tree 'lsd -F --tree --group-dirs=last'
# random password generator
aliasman rnd 'xxd -l24 -ps /dev/urandom'
```
### How to replace an alias
Just run the command again!
```sh
aliasman ll 'lsd -l'
aliasman ll 'lsd -lAhF'
```
### How to delete an alias
With `--delete`!
```sh
aliasman --delete ll
```
### How to see an alias
Just supply the name!
```sh
aliasman rnd
```
```text
alias rnd='xxd -l24 -ps /dev/urandom'
```

View File

@@ -1,41 +0,0 @@
#!/bin/sh
set -e
set -u
__init_aliasman() {
######################
# Install aliasman #
######################
# Every package should define these 6 variables
pkg_cmd_name="aliasman"
pkg_dst_cmd="$HOME/.local/bin/aliasman"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/aliasman-v$WEBI_VERSION/bin/aliasman"
pkg_src_dir="$HOME/.local/opt/aliasman-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
# pkg_install must be defined by every package
pkg_install() {
# ~/.local/opt/aliasman-v1.0.0/bin
mkdir -p "$(dirname "$pkg_src_cmd")"
# mv ./*aliasman*/aliasman ~/.local/opt/aliasman-v1.0.0/bin/aliasman
mv ./*aliasman*/aliasman "$pkg_src_cmd"
}
# pkg_get_current_version is recommended, but (soon) not required
pkg_get_current_version() {
# 'aliasman version' has output in this format:
# aliasman v1.0.0 (2023-01-15)
# Copyright 2023 AJ ONeal
# This trims it down to just the version number:
# 1.0.0
aliasman version | head -n 1 | cut -d ' ' -f 2 | sed 's:^v::'
}
}
__init_aliasman

View File

@@ -1,22 +0,0 @@
'use strict';
let Releases = module.exports;
let GitHubSource = require('../_common/github-source.js');
let owner = 'BeyondCodeBootcamp';
let repo = 'aliasman';
Releases.latest = async function () {
let all = await GitHubSource.getDistributables({ owner, repo });
for (let pkg of all.releases) {
pkg.os = 'posix_2017';
}
return all;
};
if (module === require.main) {
Releases.latest().then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -7,16 +7,6 @@ tagline: |
To update or switch versions, run `webi arc@stable` (or `@v3.5`, `@beta`, etc).
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.local/bin/arc
```
## Cheat Sheet
> Archiver (`arc`) is a powerful and flexible library meets an elegant CLI in
@@ -28,42 +18,42 @@ create a top-level directory if one does not exist.
### List
```text
```txt
# arc ls <archive file>
arc ls example.zip
```
### Unarchive (whole)
```text
```txt
# arc unarchive <archive file>
arc unarchive example.zip
```
### Extract (partial)
```text
```txt
# arc extract <archive file> <archived path> <extracted path>
arc extract example.zip example/foo ~/Downloads/foo
```
### Archive (recursive)
```text
```txt
# arc archive <archive file> <files or folders ...>
arc archive example.zip ./README.md ./bin ./src
```
### Compress (single file)
```text
```txt
# arc compress <single file> <format>
arc compress ./example.tar xz
```
### Decompress (single file)
```text
```txt
# arc decompress <archive file>
arc decompress ./example.tar.xz
```

View File

@@ -15,39 +15,41 @@ $pkg_src_bin = "$Env:USERPROFILE\.local\opt\archiver-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\archiver-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | out-null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading archiver from $Env:WEBI_PKG_URL to $pkg_download"
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"))
{
echo "Downloading archiver from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
& move "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing archiver"
IF (!(Test-Path -Path "$pkg_src_cmd"))
{
echo "Installing archiver"
# TODO: create package-specific temp directory
# Enter tmp
Push-Location .local\tmp
pushd .local\tmp
# Remove any leftover tmp cruft
Remove-Item -Path ".\arc-*" -Recurse -ErrorAction Ignore
Remove-Item -Path ".\arc.exe" -Recurse -ErrorAction Ignore
# Remove any leftover tmp cruft
Remove-Item -Path ".\arc-*" -Recurse -ErrorAction Ignore
Remove-Item -Path ".\arc.exe" -Recurse -ErrorAction Ignore
# Move single binary into root of temporary folder
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE" "arc.exe"
# Move single binary into root of temporary folder
& move "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE" "arc.exe"
# Settle unpacked archive into place
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
Move-Item -Path "arc.exe" -Destination "$pkg_src_bin"
# Settle unpacked archive into place
echo "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | out-null
Move-Item -Path "arc.exe" -Destination "$pkg_src_bin"
# Exit tmp
Pop-Location
popd
}
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
echo "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | out-null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

View File

@@ -4,15 +4,14 @@ var github = require('../_common/github.js');
var owner = 'mholt';
var repo = 'archiver';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
all._names = ['archiver', 'arc'];
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);

View File

@@ -1,5 +1,5 @@
#!/bin/pwsh
Write-Output "'archiver@$Env:WEBI_TAG' is an alias for 'arc@$Env:WEBI_VERSION'"
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
curl.exe -A MS -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell
echo "'archiver@$Env:WEBI_TAG' is an alias for 'arc@$Env:WEBI_VERSION'"
IF ($Env:WEBI_HOST -eq $null -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
curl.exe -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell

View File

@@ -3,9 +3,9 @@ set -e
set -u
__redirect_alias_arc() {
echo "'archiver@${WEBI_TAG:-stable}' is an alias for 'arc@${WEBI_VERSION-}'"
echo "'archiver@${WEBI_TAG:-stable}' is an alias for 'arc@${WEBI_VERSION:-}'"
WEBI_HOST=${WEBI_HOST:-"https://webinstall.dev"}
curl -fsSL "$WEBI_HOST/arc@${WEBI_VERSION-}" | sh
curl -fsSL "$WEBI_HOST/arc@${WEBI_VERSION:-}" | sh
}
__redirect_alias_arc

View File

@@ -1,538 +0,0 @@
---
title: atomicparsley
homepage: https://github.com/wez/atomicparsley
tagline: |
AtomicParsley is a lightweight tool for reading, parsing and setting iTunes-style metadata.
---
To update or switch versions, run `webi atomicparsley@stable` (or `@v20221229`,
`@beta`, etc).
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.local/bin/AtomicParsley
```
**Windows Users**
```text
\Windows\System32\vcruntime140.dll
```
This will also attempt to install the
[Microsoft Visual C++ Redistributable](/vcruntime) via `webi vcruntime`. If it
fails and you get the error _`vcruntime140.dll` was not found_, you'll need to
[install it manually](https://learn.microsoft.com/en-US/cpp/windows/latest-supported-vc-redist?view=msvc-170).
## Cheat Sheet
> `AtomicParsley` is an alternative to `ffmpeg` and `ffprobe` for viewing and
> changing metadata in MPEG-4 files with .3gp, .3g2, .mp4, .m4a, .m4b, .m4p,
> .m4r, and .m4v extensions. Because it's purpose-built for MP4 containers, it
> can do some things `ffmpeg` can't.
### How to Change Album Art
1. Save any existing cover art
```sh
AtomicParsley ./my-song.m4a --extractPixToPath ./ 'my-song'
```
2. Remove the artwork
```sh
AtomicParsley ./my-song.m4a --artwork REMOVE_ALL
```
3. **(macOS ONLY)** Add new artwork at the specified DPI (and other constraints)
```sh
export PIC_OPTIONS="DPI=72"
AtomicParsley ./my-podcast-audiobook.m4b --artwork ./my-season-1-cover.jpg
```
Only JPEG and PNG are supported. See `PIC_OPTIONS` down below for more options.
If you get an error, try exporting your file with a DPI to 72 (or up to 300 with
`PIC_OPTIONS` set) and a smaller resolution - perhaps 600x600 or 1500x1500 (what
old versions iTunes versions used).
### How to Remove Apple Account Info
```sh
AtomicParsley ./my-file.m4a \
--DeepScan \
--manualAtomRemove "moov.trak.mdia.minf.stbl.mp4a.pinf" \
--manualAtomRemove "moov.udta.meta.ilst.apID" \
--manualAtomRemove "moov.udta.meta.ilst.cnID" \
--manualAtomRemove "moov.udta.meta.ilst.purd" \
--manualAtomRemove "moov.udta.meta.ilst.sfID" \
--manualAtomRemove "moov.udta.meta.ilst.soal" \
--manualAtomRemove "moov.udta.meta.ilst.xid"
```
If you wanted to also remove information that indicates which Country or
Language the song was purchased in, or which album it was purchased from, there
are some additional IDs to consider:
| Metadata Tag | Description |
| ---------------------------------------- | -------------------------------------------- |
| moov.udta.meta.ilst.apID | Apple account email address |
| moov.udta.meta.ilst.ownr | Apple account username |
| moov.udta.meta.ilst.atID | Artist-track ID |
| moov.udta.meta.ilst.cnID | iTunes Catalog ID |
| moov.udta.meta.ilst.geID | Genre ID |
| moov.udta.meta.ilst.plID | Playlist ID (identifies album) |
| moov.udta.meta.ilst.sfID | iTunes store identifier (location/number) |
| moov.udta.meta.ilst.cprt | Copyright information |
| moov.udta.meta.ilst.flvr | Bitrate/video size related |
| moov.udta.meta.ilst.purd | Date purchased |
| moov.udta.meta.ilst.rtng | Explicit/Clean information |
| moov.udta.meta.ilst.soal | Album sort name |
| moov.udta.meta.ilst.stik | Media type information |
| moov.udta.meta.ilst.xid | Vendor xID |
| moov.udta.meta.ilst.----.name:[iTunMOVI] | Embedded plist contains filesize and flavor. |
| moov.trak.mdia.minf.stbl.stsd.mp4a.pinf | Purchase information related |
See <https://gist.github.com/riophae/f5694fd2952cb64982689b971ca6ec79>.
### Genre Lists
All values are **case sensitive**.
#### The "stik" List
These values are **case sensitive**:
0. `Home Video`
1. `Normal` (meaning music)
2. `Audiobook` (changes extension to .m4b)
3. `Whacked Bookmark`
4. `Music Video`
5. `Movie`
6. `Short Film`
7. `TV Show`
8. `Booklet`
#### Standard Music Genres
```txt
(1.) Blues
(2.) Classic Rock
(3.) Country
(4.) Dance
(5.) Disco
(6.) Funk
(7.) Grunge
(8.) Hip-Hop
(9.) Jazz
(10.) Metal
(11.) New Age
(12.) Oldies
(13.) Other
(14.) Pop
(15.) R&B
(16.) Rap
(17.) Reggae
(18.) Rock
(19.) Techno
(20.) Industrial
(21.) Alternative
(22.) Ska
(23.) Death Metal
(24.) Pranks
(25.) Soundtrack
(26.) Euro-Techno
(27.) Ambient
(28.) Trip-Hop
(29.) Vocal
(30.) Jazz+Funk
(31.) Fusion
(32.) Trance
(33.) Classical
(34.) Instrumental
(35.) Acid
(36.) House
(37.) Game
(38.) Sound Clip
(39.) Gospel
(40.) Noise
(41.) AlternRock
(42.) Bass
(43.) Soul
(44.) Punk
(45.) Space
(46.) Meditative
(47.) Instrumental Pop
(48.) Instrumental Rock
(49.) Ethnic
(50.) Gothic
(51.) Darkwave
(52.) Techno-Industrial
(53.) Electronic
(54.) Pop-Folk
(55.) Eurodance
(56.) Dream
(57.) Southern Rock
(58.) Comedy
(59.) Cult
(60.) Gangsta
(61.) Top 40
(62.) Christian Rap
(63.) Pop/Funk
(64.) Jungle
(65.) Native American
(66.) Cabaret
(67.) New Wave
(68.) Psychadelic
(69.) Rave
(70.) Showtunes
(71.) Trailer
(72.) Lo-Fi
(73.) Tribal
(74.) Acid Punk
(75.) Acid Jazz
(76.) Polka
(77.) Retro
(78.) Musical
(79.) Rock & Roll
(80.) Hard Rock
(81.) Folk
(82.) Folk/Rock
(83.) National Folk
(84.) Swing
(85.) Fast Fusion
(86.) Bebob
(87.) Latin
(88.) Revival
(89.) Celtic
(90.) Bluegrass
(91.) Avantgarde
(92.) Gothic Rock
(93.) Progressive Rock
(94.) Psychedelic Rock
(95.) Symphonic Rock
(96.) Slow Rock
(97.) Big Band
(98.) Chorus
(99.) Easy Listening
(100.) Acoustic
(101.) Humour
(102.) Speech
(103.) Chanson
(104.) Opera
(105.) Chamber Music
(106.) Sonata
(107.) Symphony
(108.) Booty Bass
(109.) Primus
(110.) Porn Groove
(111.) Satire
(112.) Slow Jam
(113.) Club
(114.) Tango
(115.) Samba
(116.) Folklore
(117.) Ballad
(118.) Power Ballad
(119.) Rhythmic Soul
(120.) Freestyle
(121.) Duet
(122.) Punk Rock
(123.) Drum Solo
(124.) A Capella
(125.) Euro-House
(126.) Dance Hall
```
#### iTunes Movie Genre IDs
```sh
AtomicParsley --genre-movie-id-list
```
```text
(4401) Action & Adventure
(4402) Anime
(4403) Classics
(4404) Comedy
(4405) Documentary
(4406) Drama
(4407) Foreign
(4408) Horror
(4409) Independent
(4410) Kids & Family
(4411) Musicals
(4412) Romance
(4413) Sci-Fi & Fantasy
(4414) Short Films
(4415) Special Interest
(4416) Thriller
(4417) Sports
(4418) Western
(4419) Urban
(4420) Holiday
(4421) Made for TV
(4422) Concert Films
(4423) Music Documentaries
(4424) Music Feature Films
(4425) Japanese Cinema
(4426) Jidaigeki
(4427) Tokusatsu
(4428) Korean Cinema
```
#### iTunes TV Genre IDs
```sh
AtomicParsley --genre-tv-id-list
```
```text
(4000) Comedy
(4001) Drama
(4002) Animation
(4003) Action & Adventure
(4004) Classic
(4005) Kids
(4005) Nonfiction
(4007) Reality TV
(4008) Sci-Fi & Fantasy
(4009) Sports
(4010) Teens
(4011) Latino TV
```
### Help
Note: Normally we don't include full help text, but since AtomicParsley has a
long history across various maintainers and repos, we feel it's appropriate to
do so in this case.
```text
AtomicParsley sets metadata into MPEG-4 files & derivatives supporting 3 tag
schemes: iTunes-style, 3GPP assets & ISO defined copyright notifications.
AtomicParsley quick help for setting iTunes-style metadata into MPEG-4 files.
General usage examples:
AtomicParsley /path/to.mp4 -T 1
AtomicParsley /path/to.mp4 -t +
AtomicParsley /path/to.mp4 --artist "Me" --artwork /path/to/art.jpg
Atomicparsley /path/to.mp4 --albumArtist "You" --podcastFlag true
Atomicparsley /path/to.mp4 --stik "TV Show" --advisory explicit
Getting information about the file & tags:
-T --test Test file for mpeg4-ishness & print atom tree
-t --textdata Prints tags embedded within the file
-E --extractPix Extracts pix to the same folder as the mpeg-4 file
Setting iTunes-style metadata tags
--artist (string) Set the artist tag
--title (string) Set the title tag
--album (string) Set the album tag
--genre (string) Genre tag (see --longhelp for more info)
--tracknum (num)[/tot] Track number (or track number/total tracks)
--disk (num)[/tot] Disk number (or disk number/total disks)
--comment (string) Set the comment tag
--year (num|UTC) Year tag (see --longhelp for "Release Date")
--lyrics (string) Set lyrics (not subject to 256 byte limit)
--lyricsFile (/path) Set lyrics to the content of a file
--composer (string) Set the composer tag
--copyright (string) Set the copyright tag
--grouping (string) Set the grouping tag
--artwork (/path) Set a piece of artwork (jpeg or png only)
--bpm (number) Set the tempo/bpm
--albumArtist (string) Set the album artist tag
--compilation (boolean) Set the compilation flag (true or false)
--hdvideo (number) Set the hdvideo flag to one of:
false or 0 for standard definition
true or 1 for 720p
2 for 1080p
--advisory (string*) Content advisory (*values: 'clean', 'explicit')
--stik (string*) Sets the iTunes "stik" atom (see --longhelp)
--description (string) Set the description tag
--longdesc (string) Set the long description tag
--storedesc (string) Set the store description tag
--TVNetwork (string) Set the TV Network name
--TVShowName (string) Set the TV Show name
--TVEpisode (string) Set the TV episode/production code
--TVSeasonNum (number) Set the TV Season number
--TVEpisodeNum (number) Set the TV Episode number
--podcastFlag (boolean) Set the podcast flag (true or false)
--category (string) Sets the podcast category
--keyword (string) Sets the podcast keyword
--podcastURL (URL) Set the podcast feed URL
--podcastGUID (URL) Set the episode's URL tag
--purchaseDate (UTC) Set time of purchase
--encodingTool (string) Set the name of the encoder
--encodedBy (string) Set the name of the Person/company who encoded the file
--apID (string) Set the Account Name
--cnID (number) Set the iTunes Catalog ID (see --longhelp)
--geID (number) Set the iTunes Genre ID (see --longhelp)
--xID (string) Set the vendor-supplied iTunes xID (see --longhelp)
--gapless (boolean) Set the gapless playback flag
--contentRating (string*) Set tv/mpaa rating (see -rDNS-help)
Deleting tags
Set the value to "": --artist "" --stik "" --bpm ""
To delete (all) artwork: --artwork REMOVE_ALL
manually removal: --manualAtomRemove "moov.udta.meta.ilst.ATOM"
More detailed iTunes help is available with AtomicParsley --longhelp
Setting reverse DNS forms for iTunes files: see --reverseDNS-help
Setting 3gp assets into 3GPP & derivative files: see --3gp-help
Setting copyright notices for all files: see --ISO-help
For file-level options & padding info: see --file-help
Setting custom private tag extensions: see --uuid-help
Setting ID3 tags onto mpeg-4 files: see --ID3-help
----------------------------------------------------------------------
AtomicParsley version: 20221229.172126.0 d813aa6e0304ed3ab6d92f1ae96cd52b586181ec (utf8)
Submit bug fixes to https://github.com/wez/atomicparsley
```
### `--longhelp`
```text
AtomicParsley help page for setting iTunes-style metadata into MPEG-4 files.
(3gp help available with AtomicParsley --3gp-help)
(ISO copyright help available with AtomicParsley --ISO-help)
(reverse DNS form help available with AtomicParsley --reverseDNS-help)
Usage: AtomicParsley [mp4FILE]... [OPTION]... [ARGUMENT]... [ [OPTION2]...[ARGUMENT2]...]
example: AtomicParsley /path/to.mp4 -e ~/Desktop/pix
example: AtomicParsley /path/to.mp4 --podcastURL "http://www.url.net" --tracknum 45/356
example: AtomicParsley /path/to.mp4 --copyright "℗ © 2006"
example: AtomicParsley /path/to.mp4 --year "2006-07-27T14:00:43Z" --purchaseDate timestamp
example: AtomicParsley /path/to.mp4 --sortOrder artist "Mighty Dub Cats, The
------------------------------------------------------------------------------------------------
Extract any pictures in user data "covr" atoms to separate files.
--extractPix , -E Extract to same folder (basename derived from file).
--extractPixToPath , -e (/path/basename) Extract to specific path (numbers added to basename).
example: --e ~/Desktop/SomeText
gives: SomeText_artwork_1.jpg SomeText_artwork_2.png
Note: extension comes from embedded image file format
------------------------------------------------------------------------------------------------
Tag setting options:
--artist , -a (str) Set the artist tag: "moov.udta.meta.ilst.©ART.data"
--title , -s (str) Set the title tag: "moov.udta.meta.ilst.©nam.data"
--album , -b (str) Set the album tag: "moov.udta.meta.ilst.©alb.data"
--genre , -g (str) Set the genre tag: "©gen" (custom) or "gnre" (standard).
see the standard list with "AtomicParsley --genre-list"
--tracknum , -k (num)[/tot] Set the track number (or track number & total tracks).
--disk , -d (num)[/tot] Set the disk number (or disk number & total disks).
--comment , -c (str) Set the comment tag: "moov.udta.meta.ilst.©cmt.data"
--year , -y (num|UTC) Set the year tag: "moov.udta.meta.ilst.©day.data"
set with UTC "2006-09-11T09:00:00Z" for Release Date
--lyrics , -l (str) Set the lyrics tag: "moov.udta.meta.ilst.©lyr.data"
--lyricsFile , (/path) Set the lyrics tag to the content of a file
--composer , -w (str) Set the composer tag: "moov.udta.meta.ilst.©wrt.data"
--copyright , -x (str) Set the copyright tag: "moov.udta.meta.ilst.cprt.data"
--grouping , -G (str) Set the grouping tag: "moov.udta.meta.ilst.©grp.data"
--artwork , -A (/path) Set a piece of artwork (jpeg or png) on "covr.data"
Note: multiple pieces are allowed with more --artwork args
--bpm , -B (num) Set the tempo/bpm tag: "moov.udta.meta.ilst.tmpo.data"
--albumArtist , -A (str) Set the album artist tag: "moov.udta.meta.ilst.aART.data"
--compilation , -C (bool) Sets the "cpil" atom (true or false to delete the atom)
--hdvideo , -V (bool) Sets the "hdvd" atom (true or false to delete the atom)
--advisory , -y (1of3) Sets the iTunes lyrics advisory ('remove', 'clean', 'explicit')
--stik , -S (1of7) Sets the iTunes "stik" atom (--stik "remove" to delete)
"Movie", "Normal", "TV Show" .... others:
see the full list with "AtomicParsley --stik-list"
or set in an integer value with --stik value=(num)
Note: --stik Audiobook will change file extension to '.m4b'
--description , -p (str) Sets the description on the "desc" atom
--Rating , (str) Sets the Rating on the "rate" atom
--longdesc , -j (str) Sets the long description on the "ldes" atom
--storedesc , (str) Sets the iTunes store description on the "sdes" atom
--TVNetwork , -n (str) Sets the TV Network name on the "tvnn" atom
--TVShowName , -H (str) Sets the TV Show name on the "tvsh" atom
--TVEpisode , -I (str) Sets the TV Episode on "tven":"209", but it is a string: "209 Part 1"
--TVSeasonNum , -U (num) Sets the TV Season number on the "tvsn" atom
--TVEpisodeNum , -N (num) Sets the TV Episode number on the "tves" atom
--podcastFlag , -f (bool) Sets the podcast flag (values are "true" or "false")
--category , -q (str) Sets the podcast category; typically a duplicate of its genre
--keyword , -K (str) Sets the podcast keyword; invisible to MacOSX Spotlight
--podcastURL , -L (URL) Set the podcast feed URL on the "purl" atom
--podcastGUID , -J (URL) Set the episode's URL tag on the "egid" atom
--purchaseDate , -D (UTC) Set Universal Coordinated Time of purchase on a "purd" atom
(use "timestamp" to set UTC to now; can be akin to id3v2 TDTG tag)
--encodingTool , (str) Set the name of the encoder on the "©too" atom
--encodedBy , (str) Set the name of the Person/company who encoded the file on the "©enc" atom
--apID , -Y (str) Set the name of the Account Name on the "apID" atom
--cnID , (num) Set iTunes Catalog ID, used for combining SD and HD encodes in iTunes on the "cnID" atom
To combine you must set "hdvd" atom on one file and must have same "stik" on both file
Must not use "stik" of value Home Video(0), use Movie(9)
iTunes Catalog numbers can be obtained by finding the item in the iTunes Store. Once item
is found in the iTunes Store right click on picture of item and select copy link. Paste this link
into a document or web browser to display the catalog number ID.
An example link for the video Street Kings is:
http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewMovie?id=278743714&s=143441
Here you can see the cnID is 278743714
Alternatively you can use iMDB numbers, however these will not match the iTunes catalog.
--geID , (num) Set iTunes Genre ID. This does not necessarily have to match genre.
See --genre-movie-id-list and --genre-tv-id-list
--xID , (str) Set iTunes vendor-supplied xID, used to allow iTunes LPs and iTunes Extras to interact
with other content in your iTunes Library
--gapless , (bool) Sets the gapless playback flag for a track in a gapless album
--sortOrder (type) (str) Sets the sort order string for that type of tag.
(available types are: "name", "artist", "albumartist",
"album", "composer", "show")
NOTE: Except for artwork, only 1 of each tag is allowed; artwork allows multiple pieces.
NOTE: Tags that carry text(str) have a limit of 255 utf8 characters;
however lyrics and long descriptions have no limit.
------------------------------------------------------------------------------------------------
To delete a single atom, set the tag to null (except artwork):
--artist "" --lyrics ""
--artwork REMOVE_ALL
--metaEnema , -P Douches away every atom under "moov.udta.meta.ilst"
--foobar2000Enema , -2 Eliminates foobar2000's non-compliant so-out-o-spec tagging scheme
--manualAtomRemove "some.atom.path" where some.atom.path can be:
keys to using manualAtomRemove:
ilst.ATOM.data or ilst.ATOM target an iTunes-style metadata tag
ATOM:lang=foo target an atom with this language setting; like 3gp assets
ATOM.----.name:[foo] target a reverseDNS metadata tag; like iTunNORM
Note: these atoms show up with 'AP -t' as: Atom "----" [foo]
'foo' is actually carried on the 'name' atom
ATOM[x] target an atom with an index other than 1; like trak[2]
ATOM.uuid=hex-hex-hex-hex targt a uuid atom with the uuid of hex string representation
examples:
moov.udta.meta.ilst.----.name:[iTunNORM] moov.trak[3].cprt:lang=urd
moov.trak[2].uuid=55534d54-21d2-4fce-bb88-695cfac9c740
------------------------------------------------------------------------------------------------
Environmental Variables (affecting picture placement) (macOS ONLY)
set PIC_OPTIONS in your shell to set these flags; preferences are separated by colons (:)
MaxDimensions=num (default: 0; unlimited); sets maximum pixel dimensions
DPI=num (default: 72); sets dpi
MaxKBytes=num (default: 0; unlimited); maximum kilobytes for file (jpeg only)
AddBothPix=bool (default: false); add original & converted pic (for archival purposes)
AllPixJPEG | AllPixPNG =bool (default: false); force conversion to a specific picture format
SquareUp (include to square images to largest dimension, allows an [ugly] 160x1200->1200x1200)
removeTempPix (include to delete temp pic files created when resizing images after tagging)
ForceHeight=num (must also specify width, below) force image pixel height
ForceWidth=num (must also specify height, above) force image pixel width
Examples: (bash-style)
export PIC_OPTIONS="MaxDimensions=400:DPI=72:MaxKBytes=100:AddBothPix=true:AllPixJPEG=true"
export PIC_OPTIONS="SquareUp:removeTempPix"
export PIC_OPTIONS="ForceHeight=999:ForceWidth=333:removeTempPix"
------------------------------------------------------------------------------------------------
```
```
```

View File

@@ -1,60 +0,0 @@
#!/usr/bin/env pwsh
#########################
# Install AtomicParsley #
#########################
# Every package should define these variables
$pkg_cmd_name = "AtomicParsley"
$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\AtomicParsley.exe"
$pkg_dst_bin = "$Env:USERPROFILE\.local\bin"
$pkg_dst = "$pkg_dst_cmd"
$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\AtomicParsley-v$Env:WEBI_VERSION\bin\AtomicParsley.exe"
$pkg_src_bin = "$Env:USERPROFILE\.local\opt\AtomicParsley-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\AtomicParsley-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Checking for (or Installing) MSVC Runtime..."
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
Write-Output "Downloading AtomicParsley from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing AtomicParsley"
# TODO: create package-specific temp directory
# Enter tmp
Push-Location .local\tmp
# Remove any leftover tmp cruft
Remove-Item -Path ".\AtomicParsley-v*" -Recurse -ErrorAction Ignore
Remove-Item -Path ".\AtomicParsley.exe" -Recurse -ErrorAction Ignore
# Unpack archive file into this temporary directory
# Windows BSD-tar handles zip. Imagine that.
Write-Output "Unpacking $pkg_download"
& tar xf "$pkg_download"
# Settle unpacked archive into place
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
Move-Item -Path ".\AtomicParsley.exe" -Destination "$pkg_src_bin"
# Exit tmp
Pop-Location
}
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
New-Item "$pkg_dst_bin" -ItemType Directory -Force | Out-Null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

View File

@@ -1,41 +0,0 @@
#!/bin/sh
set -e
set -u
__init_atomicparsley() {
#########################
# Install AtomicParsley #
#########################
WEBI_SINGLE=true
# Every package should define these 6 variables
pkg_cmd_name="AtomicParsley"
pkg_dst_cmd="$HOME/.local/bin/AtomicParsley"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/AtomicParsley-v$WEBI_VERSION/bin/AtomicParsley"
pkg_src_dir="$HOME/.local/opt/AtomicParsley-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
# pkg_install must be defined by every package
pkg_install() {
# ~/.local/opt/AtomicParsley-v20221229.172126.0/bin
mkdir -p "$(dirname "$pkg_src_cmd")"
# mv ./AtomicParsley ~/.local/opt/AtomicParsley-v20221229.172126.0/bin/AtomicParsley
mv ./"$pkg_cmd_name" "$pkg_src_cmd"
}
pkg_get_current_version() {
# 'AtomicParsley --version' has output in this format:
# AtomicParsley version: 20221229.172126.0 d813aa6e0304ed3ab6d92f1ae96cd52b586181ec (utf8)
# This trims it down to just the version number:
# 20221229.172126.0
AtomicParsley --version 2> /dev/null | head -n 1 | cut -d' ' -f3
}
}
__init_atomicparsley

View File

@@ -1,79 +0,0 @@
'use strict';
var github = require('../_common/github.js');
var owner = 'wez';
var repo = 'atomicparsley';
let targets = {
x86win: {
os: 'windows',
arch: 'x86',
libc: 'msvc',
},
x64win: {
os: 'windows',
arch: 'amd64',
// https://github.com/wez/atomicparsley/issues/6#issuecomment-1364523028
libc: 'msvc',
},
x64mac: {
os: 'macos',
arch: 'amd64',
},
x64lin: {
os: 'linux',
arch: 'amd64',
libc: 'gnu',
},
x64musl: {
os: 'linux',
arch: 'amd64',
libc: 'musl',
},
};
module.exports = function () {
return github(null, owner, repo).then(function (all) {
for (let rel of all.releases) {
let windows32 = rel.name.includes('WindowsX86.');
if (windows32) {
Object.assign(rel, targets.x86win);
continue;
}
let windows64 = rel.name.includes('Windows.');
if (windows64) {
Object.assign(rel, targets.x64win);
continue;
}
let macos64 = rel.name.includes('MacOS');
if (macos64) {
Object.assign(rel, targets.x64mac);
continue;
}
let musl64 = rel.name.includes('Alpine');
if (musl64) {
Object.assign(rel, targets.x64musl);
continue;
}
let lin64 = rel.name.includes('Linux.');
if (lin64) {
Object.assign(rel, targets.x64lin);
continue;
}
}
all._names = ['AtomicParsley', 'atomicparsley'];
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all));
//console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -7,17 +7,6 @@ tagline: |
To update or switch versions, run `webi awless@stable` (or `@v2`, `@beta`, etc).
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.local/bin/awless
~/.aws/credentials
```
## Cheat Sheet
awless is modeled after popular command-line tools such as Git. Most commands

View File

@@ -16,22 +16,22 @@ $pkg_src_bin = "$Env:USERPROFILE\.local\opt\awless-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\awless-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | out-null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading awless from $Env:WEBI_PKG_URL to $pkg_download"
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
echo "Downloading awless from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
& move "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing awless"
IF (!(Test-Path -Path "$pkg_src_cmd")) {
echo "Installing awless"
# TODO: create package-specific temp directory
# Enter tmp
Push-Location .local\tmp
pushd .local\tmp
# Remove any leftover tmp cruft
Remove-Item -Path ".\awless-v*" -Recurse -ErrorAction Ignore
@@ -39,19 +39,19 @@ if (!(Test-Path -Path "$pkg_src_cmd")) {
# Unpack archive file into this temporary directory
# Windows BSD-tar handles zip. Imagine that.
Write-Output "Unpacking $pkg_download"
echo "Unpacking $pkg_download"
& tar xf "$pkg_download"
# Settle unpacked archive into place
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
echo "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | out-null
Move-Item -Path ".\awless.exe" -Destination "$pkg_src_bin"
# Exit tmp
Pop-Location
popd
}
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
New-Item "$pkg_dst_bin" -ItemType Directory -Force | Out-Null
echo "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | out-null
New-Item "$pkg_dst_bin" -ItemType Directory -Force | out-null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

View File

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
var owner = 'wallix';
var repo = 'awless';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
// remove checksums and .deb
all.releases = all.releases.filter(function (rel) {
return !/(\.txt)|(\.deb)$/i.test(rel.name);
@@ -15,7 +15,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all));
});

View File

@@ -7,45 +7,23 @@ tagline: |
To update or switch versions, run `webi bat@stable` (or `@v0.18`, `@beta`, etc).
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.config/bat/config
~/.local/opt/bat/
```
**Windows Users**
```text
\Windows\System32\vcruntime140.dll
```
This will also attempt to install the
[Microsoft Visual C++ Redistributable](/vcruntime) via `webi vcruntime`. If it
fails and you get the error _`vcruntime140.dll` was not found_, you'll need to
[install it manually](https://learn.microsoft.com/en-US/cpp/windows/latest-supported-vc-redist?view=msvc-170).
## Cheat Sheet
> `bat` is pretty much what `cat` would be if it were developed today's in the
> world of Markdown, git, etc.
### How to run on Windows
On Windows you'll get an error like this:
> execution cannot proceed run because vcruntime140.dll was not found
You need to download and install the
[Microsoft Visual C++ Redistributable](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads)
### How to alias as `cat`
Use [aliasman](/aliasman):
```sh
aliasman cat 'bat --style=plain'
alias cat='bat --style=plain'
```
Or place this in `~/.config/envman/alias.env` and manually update your
`.bashrc`, `.zshrc`, `.profile`, and/or `~/.config/fish/config.fish` to source
it.
Update your `.bashrc`, `.zshrc`, or `.profile`
```sh
alias cat="bat --style=plain"
@@ -76,7 +54,7 @@ Edit the config file:
`~/.config/bat/config`:
```text
```txt
# no numbers or headers, just highlighting and such
--style="plain"
```

View File

@@ -3,45 +3,41 @@
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
$EXENAME = "$Env:PKG_NAME.exe"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"))
{
echo "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& move "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
}
# Fetch MSVC Runtime
Write-Output "Checking for MSVC Runtime..."
if (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
}
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
Write-Output "Installing $Env:PKG_NAME"
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\xbin\$VERNAME"))
{
echo "Installing $Env:PKG_NAME"
# TODO: temp directory
# Enter tmp
Push-Location .local\tmp
pushd .local\tmp
# Remove any leftover tmp cruft
Remove-Item -Path "$Env:PKG_NAME-v*" -Recurse -ErrorAction Ignore
# Remove any leftover tmp cruft
Remove-Item -Path "$Env:PKG_NAME-v*" -Recurse -ErrorAction Ignore
# Unpack archive
# Windows BSD-tar handles zip. Imagine that.
Write-Output "Unpacking $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& tar xf "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Move single binary into root of temporary folder
& Move-Item "bat-*\$EXENAME" "$VERNAME"
# Unpack archive
# Windows BSD-tar handles zip. Imagine that.
echo "Unpacking $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& tar xf "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Move single binary into root of temporary folder
& move "bat-*\$EXENAME" "$VERNAME"
# Settle unpacked archive into place
Write-Output "New Name: $VERNAME"
Write-Output "New Location: $Env:USERPROFILE\.local\bin\$VERNAME"
Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\bin"
# Settle unpacked archive into place
echo "New Name: $VERNAME"
echo "New Location: $Env:USERPROFILE\.local\xbin\$VERNAME"
Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\xbin"
# Exit tmp
Pop-Location
popd
}
Write-Output "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\bin\$VERNAME'"
echo "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\xbin\$VERNAME'"
Remove-Item -Path "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse -ErrorAction Ignore
Copy-Item -Path "$Env:USERPROFILE\.local\bin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse
Copy-Item -Path "$Env:USERPROFILE\.local\xbin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse

View File

@@ -19,7 +19,7 @@ __init_bat() {
}
pkg_install() {
# ~/.local/bin
# ~/.local/xbin
mkdir -p "$pkg_src_bin"
# mv ./bat-*/bat ~/.local/opt/bat-v0.15.4/bin/bat
@@ -27,11 +27,6 @@ __init_bat() {
# chmod a+x ~/.local/opt/bat-v0.15.4/bin/bat
chmod a+x "$pkg_src_cmd"
if ! [ -e ~/.config/bat/config ]; then
mkdir -p ~/.config/bat/
touch ~/.config/bat/config
fi
}
}

View File

@@ -4,14 +4,14 @@ var github = require('../_common/github.js');
var owner = 'sharkdp';
var repo = 'bat';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
all.releases = all.releases.slice(0, 10);
//console.info(JSON.stringify(all));

View File

@@ -1,83 +0,0 @@
---
title: beyond-shell
homepage: https://webinstall.dev/beyond-shell
tagline: |
meta package for Beyond Code workshops
---
To update, run the relevant installers individually. For example:
`webi node@lts`.
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/bin
~/.config/envman/alias.env
~/.config/envman/PATH.env
~/.iterm2/
~/.local/bin/
~/.local/opt/
~/.local/share/font/ (or ~/Library/Fonts)
~/.vim/pack/plugins/start/
~/.vim/plugins/
~/.vimrc
```
## Cheat Sheet
> Installs the format and linter tools and code you'll need for the Beyond Code
> Shell Scripting Workshop
This meta package will install the full set of plugins and settings we
recommended.
## Prerequisites for Windows
- https://webinstall.dev/wsl
- https://webinstall.dev/nerdfont
## Post Install for macOS
- https://webinstall.dev/iterm2-themes
- https://webinstall.dev/nerdfont
## What's installed?
- Formatters & Linters
- [`prettier`](/prettier)
- [`shellcheck`](/shellcheck)
- [`shfmt`](/shfmt)
- Vim Plugins & Config
- [vim-leader](/vim-leader)
- [vim-shell](/vim-shell)
- [vim-sensible](/vim-sensible)
- [vim-viminfo](/vim-viminfo)
- [vim-lastplace](/vim-lastplace)
- [vim-smartcase](/vim-smartcase)
- [vim-spell](/vim-spell)
- [vim-ale](/vim-ale)
- [vim-prettier](/vim-prettier)
- [vim-shfmt](/vim-shfmt)
- [vim-whitespace](/vim-whitespace)
- Commandline Tools
- [`aliasman`](/aliasman)
- [`bat`](/bat)
- [`curlie`](/curlie)
- [`jq`](/jq)
- [`pathman`](/pathman)
- [`ssh-pubkey`](/ssh-pubkey)
- [`webi`](/webi)
- Aliases
- `cat='bat --style=plain --pager=none'`
- `curl=curlie`
- `setalias='aliasman'`
- System
- [iterm2](/iterm2)
- [fish](/fish)
- [NerdFont](/nerdfont)
- git
- vim
- zip

View File

@@ -1,415 +0,0 @@
#!/bin/sh
set -e
set -u
webi_cmd="$HOME/.local/bin/webi"
aliasman_cmd="$HOME/.local/bin/aliasman"
pathman_cmd="$HOME/.local/bin/pathman"
my_date="$(
date -u '+%F_%H.%M.%S'
)"
my_log="${HOME}/.local/share/beyond-code/var/${my_date}.log"
fn_node_version() { (
my_node="${1:-}"
if test -n "${my_node}"; then
"${my_node}" --version | sed 's/v//'
fi
); }
fn_node_path() { (
if command -v node; then
return 0
fi
if test ! -e ~/.local/opt/node/bin/node; then
return 0
fi
echo ~/.local/opt/node/bin/node
); }
fn_node_install() { (
node_minimum="18"
my_path_actual="$(
fn_node_path
)"
my_node="$(
fn_node_path
)"
my_node_version="$(
fn_node_version "${my_node}"
)"
if test -n "${my_path_actual}"; then
my_bin_dir="$(
dirname "${my_node}"
)"
my_node_dir="$(
dirname "${my_bin_dir}"
)"
my_lib_dir="${my_node_dir}/lib/node_modules"
if test ! -w "${my_bin_dir}" ||
test ! -w "${my_node_dir}" ||
test ! -w "${my_lib_dir}"; then
printf "\e[31m[Warning]\e[0m \e[32mNode.js\e[0m at %s is not user-writable\n" "${my_path_actual}"
sleep 1
my_path_actual=""
fi
fi
if test -n "${my_path_actual}"; then
my_node_major="$(
echo "${my_node_version}" | cut -d'.' -f1
)"
if test -z "${my_node_major}"; then
my_path_actual=""
elif test "${my_node_major}" -lt "${node_minimum}"; then
printf "\e[31m[Warning]\e[0m \e[32mNode.js v%s\e[0m is too old\n" "${my_node_version}"
sleep 1
my_path_actual=""
fi
fi
if test -z "${my_path_actual}"; then
printf "\e[32m[Fixup]\e[0m Installing \e[32mNode.js\e[0m to \e[34m%s\e[0m\n" '~'/.local/opt/node/
sleep 1
"${webi_cmd}" node@lts >> "${my_log}"
if test -e ~/.local/opt/node/bin/node; then
"${pathman_cmd}" add ~/.local/opt/node/bin/ > /dev/null 2> /dev/null
fi
else
echo "[Info] Found ${my_path_actual}/node (v${my_node_version})"
fi
); }
fn_git_install() { (
if [ "Darwin" = "$(uname -s)" ]; then
needs_xcode="$(
/usr/bin/xcode-select -p > /dev/null 2> /dev/null || echo "true"
)"
if [ -n "${needs_xcode}" ]; then
echo ""
echo ""
printf "\e[31m[Error]\e[0m: Run this command to install XCode Command Line Tools first:\n"
echo ""
echo " xcode-select --install"
echo ""
echo "After the install, close this terminal, open a new one, and try again."
echo ""
exit 1
else
my_git="$(command -v git)"
echo "[Info] Found ${my_git}"
fi
return 0
fi
if ! command -v git > /dev/null; then
fn_apt_get git git
return 0
fi
my_git="$(command -v git)"
echo "[Info] Found ${my_git}"
); }
fn_apt_get() { (
my_cmd="${1:-}"
my_pkg="${2:-}"
printf "\e[31m[Warning]\e[0m \e[32m%s\e[0m not found\n" "${my_cmd}"
printf "\e[32m[Fixup]\e[0m install \e[34m%s\e[0m (may require password)\n" "${my_pkg}"
sleep 1
echo "sudo apt-get install ${my_pkg}"
# shellcheck disable=SC2030
export DEBIAN_FRONTEND=noninteractive
if ! sudo apt-get install -qq -y -o=Dpkg::Use-Pty=0 "${my_pkg}"; then
echo "failed to install ${my_cmd} on"
cat /etc/issue
fi
); }
fn_vim_install() { (
if command -v vim > /dev/null; then
my_vim="$(command -v vim)"
echo "[Info] Found ${my_vim}"
return 0
fi
fn_apt_get vim vim
); }
fn_zip_install() { (
if command -v zip > /dev/null; then
my_zip="$(command -v zip)"
echo "[Info] Found ${my_zip}"
return 0
fi
fn_apt_get zip zip
); }
fn_fish_install() { (
if ! command -v fish > /dev/null; then
if [ "Darwin" = "$(uname -s)" ]; then
printf "\e[31m[Warning]\e[0m \e[32mfish\e[0m not found\n"
printf "\e[32m[Fixup]\e[0m install \e[34mfish\e[0m\n"
sleep 1
"${webi_cmd}" fish > /dev/null
else
fn_apt_get fish fish
fi
fi
if ! test -f ~/.config/fish/config.fish; then
printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" '~'/.config/fish/config.fish
mkdir -p ~/.config/fish/
touch ~/.config/fish/config.fish
chmod 0600 ~/.config/fish/config.fish
fi
); }
fn_touch() { (
my_file_path="${1:-}"
my_file_name="${2:-}"
if test -f "${my_file_path}"; then
echo "[Info] Found ${my_file_name}"
return 0
fi
printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" "${my_file_name}"
my_base_path="$(
basename "${my_file_path}"
)"
if test ! -e "${my_base_path}"; then
mkdir -p "${my_base_path}"
chmod 0700 "${my_base_path}"
fi
touch "${my_file_path}"
chmod 0600 "${my_file_path}"
); }
fn_mkdir() { (
my_dir_name="${1:-}"
my_dir_path="${2:-}"
if test -d "${my_dir_path}"; then
echo "[Info] Found ${my_dir_name}"
return 0
fi
printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" "${my_dir_name}"
mkdir -p "${my_dir_path}"
chmod 0700 "${my_dir_path}"
); }
fn_path_bin() { (
if test -d ~/bin/; then
echo "[Info] Found ~/bin/"
return 0
fi
printf "\e[32m[Fixup]\e[0m create \e[34m%s\e[0m\n" '~'/bin/
mkdir -p ~/bin/
"${pathman_cmd}" add ~/bin/ > /dev/null 2> /dev/null
); }
fn_webi_bin() { (
my_cmd="$(
echo "${1:-}" | sed 's/@.*//'
)"
if test -e ~/.local/bin/"${my_cmd}"; then
echo "[Info] Found ~/.local/bin/${my_cmd}"
return 0
fi
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "${my_cmd}" '~'"/.local/bin/${my_cmd}"
"${webi_cmd}" "${my_cmd}" > /dev/null 2> /dev/null
); }
fn_webi_opt() { (
my_cmd="${1:-}"
# shellcheck disable=SC2010
# we do want to use ls with grep
if ls ~/.local/opt/ | grep -q -E "^${my_cmd}(-|\$)"; then
return 0
fi
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s/\e[0m\n" "${my_cmd}" '~'"/.local/opt/${my_cmd}"
"${webi_cmd}" "${my_cmd}" > /dev/null
); }
fn_webi_vim_config() { (
my_cmd="${1:-}"
if test -e ~/.vim/plugins/"${my_cmd}.vim"; then
echo "[Info] Found ~/.vim/plugins/${my_cmd}.vim"
return 0
fi
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s.vim\e[0m\n" "vim-${my_cmd}" '~'"/.vim/plugins/${my_cmd}.vim"
"${webi_cmd}" "vim-${my_cmd}" > /dev/null
); }
fn_webi_vim_plugin_ale() { (
my_cmd="ale"
if test -e ~/.vim/pack/plugins/start/"${my_cmd}"; then
echo "[Info] Found ~/.vim/pack/plugins/start/${my_cmd}"
return 0
fi
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "vim-${my_cmd}" '~'"/.vim/pack/plugins/start/${my_cmd}"
printf "\e[32m[Fixup]\e[0m and config at \e[34m%s.vim\e[0m\n" '~'"/.vim/plugins/${my_cmd}"
"${webi_cmd}" "vim-${my_cmd}" > /dev/null 2> /dev/null
); }
fn_webi_vim_plugin() { (
my_cmd="${1:-}"
if test -e ~/.vim/pack/plugins/start/"vim-${my_cmd}"; then
echo "[Info] Found ~/.vim/pack/plugins/start/vim-${my_cmd}"
return 0
fi
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "vim-${my_cmd}" '~'"/.vim/pack/plugins/start/vim-${my_cmd}"
printf "\e[32m[Fixup]\e[0m and config at \e[34m%s.vim\e[0m\n" '~'"/.vim/plugins/${my_cmd}"
"${webi_cmd}" "vim-${my_cmd}" > /dev/null 2> /dev/null
); }
fn_webi_vimrc() { (
my_cmd="${1:-}"
if grep -q "${my_cmd}" ~/.vimrc; then
echo "[Info] Found vim-${my_cmd} config in ~/.vimrc"
return 0
fi
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m to \e[34m%s\e[0m\n" "vim-${my_cmd}" '~'/.vimrc
"${webi_cmd}" "vim-${my_cmd}" > /dev/null
); }
fn_setalias() { (
my_name="${1:-}"
my_cmd="${2:-}"
if grep -q "${my_name}" ~/.config/envman/alias.env; then
echo "[Info] Found '${my_name}' alias"
return 0
fi
printf "\e[32m%s\e[0m aliased to '\e[34m%s\e[0m'\n" "${my_name}" "${my_cmd}"
"${aliasman_cmd}" "${my_name}" "${my_cmd}" > /dev/null
); }
__install() {
#printf "\e[31mRED\e[0m\n"
#printf "\e[32mYELLOW\e[0m\n"
#printf "\e[34mBLUE\e[0m\n"
echo ""
printf "\e[34mLog\e[0m will be written to \e[32m%s\e[0m\n" '~'/.local/share/beyond-code/var/
sleep 2
echo ""
mkdir -p ~/.local/share/beyond-code/var/
mkdir -p ~/.local/opt/
{
printf "\e[32mwebi\e[0m at \e[32m%s\e[0m\n" '~'/.local/bin/webi
echo ""
if [ "Darwin" = "$(uname -s)" ]; then
if test -e /Applications/iTerm.app; then
echo "[Info] Found iTerm.app in /Applications/"
else
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m into \e[34m%s\e[0m\n" "iTerm.app" "/Applications/"
"${webi_cmd}" iterm2 > /dev/null
fi
fi
fn_git_install
fn_fish_install
fn_zip_install
if test -e ~/.local/share/fonts/'Droid Sans Mono for Powerline Nerd Font Complete.otf'; then
echo "[Info] Found NerdFont in ~/.local/share/fonts"
elif test -e ~/Library/Fonts/'Droid Sans Mono for Powerline Nerd Font Complete.otf'; then
echo "[Info] Found NerdFont in ~/Library/Fonts"
else
"${webi_cmd}" nerdfont | tail -n 1
fi
if test -e ~/.iterm2/; then
echo "[Info] Found iTerm2 utils ~/.iterm2/"
else
printf "\e[32m[Fixup]\e[0m Installing \e[32m%s\e[0m into \e[34m%s\e[0m\n" "iterm2-utils" '~'"/.iterm2/"
"${webi_cmd}" iterm2-utils > /dev/null
fi
echo ""
echo "[Info] PATH updates in ~/.config/envman/PATH.env"
fn_path_bin
fn_touch ~/.config/envman/alias.env '~'/.config/envman/alias.env
fn_touch ~/.config/envman/PATH.env '~'/.config/envman/PATH.env
echo ""
fn_node_install
for my_cmd in aliasman bat curlie jq pathman ssh-pubkey; do
fn_webi_bin "${my_cmd}"
done
echo ""
echo "[Info] Aliases in ~/.config/envman/alias.env"
fn_setalias setalias 'aliasman'
fn_setalias cat 'bat --style=plain --pager=none'
fn_setalias curl 'curlie'
# vim config and plugins
echo ""
fn_vim_install
fn_touch ~/.vimrc '~'/.vimrc
for my_plugin in leader shell; do
fn_webi_vimrc "${my_plugin}"
done
fn_webi_vim_plugin sensible
for my_plugin in viminfo smartcase spell whitespace; do
fn_webi_vim_config "${my_plugin}"
done
fn_webi_vim_plugin_ale
for my_plugin in commentary lastplace shfmt prettier; do
fn_webi_vim_plugin "${my_plugin}"
done
for my_cmd in shfmt@3.5 shellcheck; do
fn_webi_bin "${my_cmd}"
done
} | tee -a "${my_log}"
echo ""
printf "\e[34mLog\e[0m written to \e[32m%s\e[0m\n" "${my_log}"
echo ""
}
__install

View File

@@ -5,24 +5,13 @@ tagline: |
Brew: The Missing Package Manager for macOS (and Linux).
---
To update or switch versions, use brew's built-in upgrade management, or our
handy update service script:
## Updating `brew`
`brew` has its own built-in upgrade management.
```sh
brew update
brew upgrade
brew-update-service-install # Webi helper script
```
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.local/opt/brew/
~/.local/bin/brew-update-service-install
```
## Cheat Sheet

View File

@@ -1,56 +0,0 @@
#!/bin/sh
set -e
set -u
main() { (
sed '1,/^#~\/.local\/bin\/brew-updater/d' "${0}" > ~/.local/bin/brew-update-hourly
chmod a+x ~/.local/bin/brew-update-hourly
echo "Checking for serviceman..."
~/.local/bin/webi serviceman
if ! command -v serviceman > /dev/null; then
export PATH="$HOME/.local/bin:$PATH"
fi
serviceman --version
serviceman add --agent \
--workdir ~/.local/opt/brew/ \
--name sh.brew.updater -- \
~/.local/bin/brew-update-hourly
); }
if ! main; then
exit 1
fi
exit 0
#~/.local/bin/brew-updater
#!/bin/sh
#set -e
set -u
if test -e ~/.config/envman/PATH.env; then
# shellcheck disable=SC1090
. ~/.config/envman/PATH.env
fi
while true; do
my_start="$(date '+%s')"
my_date="$(date '+%F %T')"
echo "[$my_date] Updating brew..."
brew update
echo ''
my_end="$(date '+%s')"
my_elapsed="$((my_end - my_start))"
my_date="$(date '+%F %T')"
echo "[$my_date] Updated in ${my_elapsed}s."
echo "[$my_date] Cleaning up..."
brew cleanup
echo "[$my_date] Waiting 24 hours..."
my_wait="$((24 * 60 * 60))"
sleep "$my_wait"
done

View File

@@ -7,9 +7,9 @@ _install_brew() {
# Straight from https://brew.sh
#/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
if test "Darwin" = "$(uname -s)"; then
if uname -a | grep -q -i darwin; then
needs_xcode="$(/usr/bin/xcode-select -p > /dev/null 2> /dev/null || echo "true")"
if test -n "${needs_xcode}"; then
if [ -n "${needs_xcode}" ]; then
echo ""
echo ""
echo "ERROR: Run this command to install XCode Command Line Tools first:"
@@ -20,11 +20,11 @@ _install_brew() {
echo ""
fi
else
if ! command -v gcc > /dev/null; then
if [ -z "$(command -v gcc)" ]; then
echo >&2 "Warning: to install 'gcc' et al on Linux use the built-in package manager."
echo >&2 " For example, try: sudo apt install -y build-essential"
fi
if ! command -v git > /dev/null; then
if [ -z "$(command -v git)" ]; then
echo >&2 "Error: to install 'git' on Linux use the built-in package manager."
echo >&2 " For example, try: sudo apt install -y git"
exit 1
@@ -32,7 +32,7 @@ _install_brew() {
fi
# From Straight from https://brew.sh
if ! test -d "$HOME/.local/opt/brew"; then
if ! [ -d "$HOME/.local/opt/brew" ]; then
echo "Installing to '$HOME/.local/opt/brew'"
echo ""
echo "If you prefer to have brew installed to '/usr/local' cancel now and do the following:"
@@ -44,13 +44,6 @@ _install_brew() {
git clone --depth=1 https://github.com/Homebrew/brew "$HOME/.local/opt/brew"
fi
rm -rf "$HOME/.local/bin/brew-update-service-install"
webi_download \
"$WEBI_HOST/packages/brew/brew-update-service-install" \
"$HOME/.local/bin/brew-update-service-install" \
brew-update-service-install
chmod a+x "$HOME/.local/bin/brew-update-service-install"
webi_path_add "$HOME/.local/opt/brew/bin"
export PATH="$HOME/.local/opt/brew/bin:$PATH"
@@ -67,10 +60,6 @@ _install_brew() {
# shellcheck disable=2016
echo ' /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"'
echo ""
echo "To register 'brew update' as a hourly system service:"
echo " brew-update-service-install"
echo ""
}
_install_brew

View File

@@ -1,158 +0,0 @@
---
title: Bun
homepage: https://bun.sh
tagline: |
Bun is a fast all-in-one JavaScript runtime
---
To update or switch versions, run `webi bun@<tag>`. \
(you can use `@beta` for pre-releases, or `@x.y.z` for a specific version)
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.local/opt/bun/
~/.local/opt/bun-<VERSION>/
```
## Cheat Sheet
> Bun is a wicked-fast JavaScript runtime for developer tooling, API servers,
> and edge computing.
>
> It's built in Zig and provides a more curated, "batteries-included" approach
> to developing with JavaScript and JavaScript-ish languages - such as
> TypeScript, JSX, and TSX.
Run some <strong><em>x</em></strong>Script:
```sh
bun run ./hello.js
bun run ./hello.jsx
bun run ./hello.ts
bun run ./hello.tsx
```
Run a package from `npm`:
```sh
bun x jswt
```
More goodies:
```
bun help
```
(there's also a built-in **development server** and lots of stuff)
### bun<span>.</span>sh/install vs webi
Bun has an official installer:
```sh
export BUN_INSTALL="$HOME/.bun"
curl -fsSL https://bun.sh/install | bash
```
You might want to still use webi if you want to be able to switch between
versions.
### How to install command line completions
```sh
bun completions
```
### How to create a bun executable
1. Create your script
```sh
vim ./hello
```
```js
#!/usr/bin/env bun
console.log('hello');
```
2. Make it executable
```sh
chmod a+x ./hello
```
3. Run it
```sh
./hello
```
### How to publish bun packages
At the time of this writing (bun v0.5.1), you'll still need to publish with
`npm`.
`npm` is installed with [node](/node).
See
[Getting Started with NPM (as a developer)](https://gist.github.com/coolaj86/1318304).
### How to install bun packages
You can run it with `bun x`:
```sh
bun x <whatever>
```
Or you can put the `#!/usr/bin/env bun` shebang before publishing, and install
from npm:
```sh
bun install -g <whatever>
<whatever>
```
### How to run a Bun program as a service
As a system service on Linux:
(**note**: swap 'my-project' and './my-project' for the name of your project and
file)
1. Install serviceman (compatible with systemd)
```sh
webi serviceman
source ~/.config/envman/PATH.env
```
2. Go into your program's directory
```sh
pushd ./my-project/
```
3. Add your project to the system launcher, running as the current user
```sh
serviceman add --name 'my-project' --daemon -- \
bun run ./my-project.js
```
4. Restart the logging service
```sh
sudo systemctl restart systemd-journald
```
For **macOS**:
1. Install serviceman (compatible with `launchctl`)
```sh
webi serviceman
source ~/.config/envman/PATH.env
```
2. Go into your program's directory
```sh
pushd ./my-project/
```
3. Add your project to the system launcher, running as the current user
```sh
serviceman add --agent --name 'my-project' -- \
bun run ./my-project.js
```

View File

@@ -1,39 +0,0 @@
#!/bin/sh
set -e
set -u
# NOTE: pkg_* variables can be defined here
# pkg_cmd_name
# pkg_src, pkg_src_bin, pkg_src_cmd
# pkg_dst, pkg_dst_bin, pkg_dst_cmd
#
# Their defaults are defined in _webi/template.sh at https://github.com/webinstall/packages
# Every package should define these 6 variables
pkg_cmd_name="bun"
pkg_dst_cmd="$HOME/.local/opt/bun/bin/bun"
pkg_dst_dir="$HOME/.local/opt/bun"
pkg_dst="$pkg_dst_dir"
pkg_src_cmd="$HOME/.local/opt/bun-v$WEBI_VERSION/bin/bun"
pkg_src_dir="$HOME/.local/opt/bun-v$WEBI_VERSION"
pkg_src="$pkg_src_dir"
pkg_get_current_version() {
# 'bun --version' only outputs the version:
# 0.5.1
# But we future-proof it a little anyway
# 0.5.1
bun --version 2> /dev/null | head -n 1 | cut -d' ' -f1
}
# pkg_install must be defined by every package
pkg_install() {
# ~/.local/opt/bun-v0.5.1/bin
mkdir -p "$(dirname "$pkg_src_cmd")"
# mv ./bun-*/bun ~/.local/opt/bun-v0.5.1/bin/bun
mv ./bun-*/bun* "$pkg_src_cmd"
}

View File

@@ -1,61 +0,0 @@
'use strict';
var github = require('../_common/github.js');
var owner = 'oven-sh';
var repo = 'bun';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
// collect baseline asset names so we can prefer them over non-baseline
// (baseline builds avoid SIGILL on older/container CPUs)
let baselineNames = new Set();
all.releases.forEach(function (r) {
if (r.name.includes('-baseline')) {
baselineNames.add(r.name.replace('-baseline', ''));
}
});
all.releases = all.releases
.filter(function (r) {
if (r.name.includes('-profile')) {
return false;
}
if (r.name.endsWith('.txt') || r.name.endsWith('.asc')) {
return false;
}
// drop the non-baseline asset when a baseline twin exists
if (!r.name.includes('-baseline') && baselineNames.has(r.name)) {
return false;
}
let isMusl = r.name.includes('-musl');
if (isMusl) {
r._musl = true;
r.libc = 'musl';
} else if (r.name.includes('-linux-')) {
r.libc = 'gnu';
}
return true;
})
.map(function (r) {
// bun-linux-x64-baseline.zip => bun-linux-x64
r.name = r.name.replace('-baseline', '').replace(/\.zip$/, '');
// bun-v0.5.1 => v0.5.1
r.version = r.version.replace(/bun-/g, '');
return r;
});
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);
console.info(JSON.stringify(all, null, 2));
});
}

File diff suppressed because it is too large Load Diff

View File

@@ -16,22 +16,22 @@ $pkg_src_bin = "$Env:USERPROFILE\.local\opt\caddy-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\caddy-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | out-null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading caddy from $Env:WEBI_PKG_URL to $pkg_download"
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
echo "Downloading caddy from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
& move "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing caddy"
IF (!(Test-Path -Path "$pkg_src_cmd")) {
echo "Installing caddy"
# TODO: create package-specific temp directory
# Enter tmp
Push-Location .local\tmp
pushd .local\tmp
# Remove any leftover tmp cruft
Remove-Item -Path ".\caddy-v*" -Recurse -ErrorAction Ignore
@@ -39,19 +39,19 @@ if (!(Test-Path -Path "$pkg_src_cmd")) {
# Unpack archive file into this temporary directory
# Windows BSD-tar handles zip. Imagine that.
Write-Output "Unpacking $pkg_download"
echo "Unpacking $pkg_download"
& tar xf "$pkg_download"
# Settle unpacked archive into place
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
echo "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | out-null
Move-Item -Path ".\caddy.exe" -Destination "$pkg_src_bin"
# Exit tmp
Pop-Location
popd
}
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
New-Item "$pkg_dst_bin" -ItemType Directory -Force | Out-Null
echo "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | out-null
New-Item "$pkg_dst_bin" -ItemType Directory -Force | out-null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

Some files were not shown because too many files have changed in this diff Show More