Compare commits

..

41 Commits

Author SHA1 Message Date
AJ ONeal
f0471206ec chore: update deps 2023-12-28 02:05:21 -07:00
AJ ONeal
d0c0805a1f f: update build classifier 2023-12-28 02:05:21 -07:00
AJ ONeal
edfb064075 WIPPY WIP: add todo for next test 2023-12-28 02:05:21 -07:00
AJ ONeal
22d48c12cb f: match hosts 2023-12-28 02:05:20 -07:00
AJ ONeal
3b3df248d1 f: match hosts 2023-12-28 02:05:20 -07:00
AJ ONeal
00e828645b feat: match hosts to builds 2023-12-28 02:05:20 -07:00
AJ ONeal
9b9dc6d4c9 feat: add pkg.triplets when classifying 2023-12-28 02:05:20 -07:00
AJ ONeal
5509686fba f: versions: assign 2023-12-28 02:05:20 -07:00
AJ ONeal
76e21b6e0a f: versions: unshift 2023-12-28 02:05:20 -07:00
AJ ONeal
060f9b7026 feat(builds): add pkg.versions 2023-12-28 02:05:20 -07:00
AJ ONeal
f28bd86ce8 wip: get builds for a given target 2023-12-28 02:05:19 -07:00
AJ ONeal
d4458c1076 wip: get builds for a given target 2023-12-28 02:05:19 -07:00
AJ ONeal
619d77c6fa chore(builds): remove dead comment cruft 2023-12-28 02:05:19 -07:00
AJ ONeal
0111df7bb7 fix(builds): include explicit target as part of build id 2023-12-28 02:05:19 -07:00
AJ ONeal
ce9c111fa9 WIPPY WIP: feat: create builds tree 2023-12-28 02:05:19 -07:00
AJ ONeal
48ca227feb f: update build-classifier 2023-12-28 02:05:19 -07:00
AJ ONeal
8d8f809563 wip: create build groups by os, arch+libc 2023-12-28 02:05:19 -07:00
AJ ONeal
70121fc60c WIP: todos for the next thing 2023-12-28 02:05:19 -07:00
AJ ONeal
d0d2dce17c ref: termsToTriplet => termsToTarget 2023-12-28 02:05:18 -07:00
AJ ONeal
1eb28c7d22 f: rename buildId with @ 2023-12-28 02:05:18 -07:00
AJ ONeal
c6fe0f7fb6 fix: mipsXle => mipsXel, ppcXel => ppcXle 2023-12-28 02:05:18 -07:00
AJ ONeal
fb74b98960 fix: ppc64el => ppc64le 2023-12-28 02:05:18 -07:00
AJ ONeal
867fe16f27 fix: mipsle => mipsel 2023-12-28 02:05:18 -07:00
AJ ONeal
40766bea4c feat: cold-boot caching w/ lazy warm stale refreshes 2023-12-28 02:05:18 -07:00
AJ ONeal
65c27afe49 f: make parallel requests 2023-12-28 02:05:18 -07:00
AJ ONeal
36c3de2ffc f: move cacher functions 2023-12-28 02:05:17 -07:00
AJ ONeal
1b10b18454 f: move cacher functions 2023-12-28 02:05:17 -07:00
AJ ONeal
8abff3f0b9 f: separate build linter from classifier lib 2023-12-28 02:05:17 -07:00
AJ ONeal
14bb497303 f: move state 2023-12-28 02:05:17 -07:00
AJ ONeal
2a785a86dc f: rewrite 2023-12-28 02:05:17 -07:00
AJ ONeal
1d3f733cc4 f: refactor names 2023-12-28 02:05:17 -07:00
AJ ONeal
1f1ea2b8ba f: refactor names 2023-12-28 02:05:17 -07:00
AJ ONeal
8f92da28bb f: show unused terms, make importable 2023-12-28 02:05:17 -07:00
AJ ONeal
d102f18480 f: move triplet classification to own repo 2023-12-28 02:05:16 -07:00
AJ ONeal
e85db4ecc4 f: name refactor 2023-12-28 02:05:16 -07:00
AJ ONeal
f5b1de7751 f: simplify fetch, show triplets 2023-12-28 02:05:16 -07:00
AJ ONeal
f93ca34e68 f: JSON.stringify(data, null, 2) 2023-12-28 02:05:16 -07:00
AJ ONeal
de92e02bb1 wip: update classifier submodule to latest 2023-12-28 02:05:16 -07:00
AJ ONeal
c11bc55d5b WIP: new caching releases fetcher 2023-12-28 02:05:16 -07:00
AJ ONeal
f24df1a731 feat: add target triplet classifier 2023-12-28 02:05:16 -07:00
AJ ONeal
fca66feb3e feat: list all releases across all of Webi 2023-12-28 02:05:15 -07:00
330 changed files with 2633 additions and 9743 deletions

View File

@@ -35,18 +35,15 @@ info, and doing a find and replace on a few file system path names.
```
2. Copy the example template and update with info from Official Releases:
<https://github.com/___CHANGE/ME___/releases>
```bash
rsync -av _example/ CHANGE-ME/
```
- [ ] update `CHANGE-ME/release.js` to use the official repo
- [ ] Learn how `CHANGE-ME` unpacks (i.e. as a single file? as a .tar.gz? as
a .tar.gz with a folder named CHANGE-ME?)
- [ ] find and replace to change the name
- [ ] update `CHANGE-ME/install.sh` (see `bat` and `jq` as examples)
- [ ] update `CHANGE-ME/install.ps1` (see `bat` and `jq` as examples)
3. Needs an updated tagline and cheat sheet
- [ ] update `CHANGE-ME/README.md`
- [ ] official URL

View File

@@ -23,15 +23,10 @@ jobs:
sh ./_scripts/install-ci-deps
echo "${HOME}/.local/bin" >> $GITHUB_PATH
echo "${HOME}/.local/opt/pwsh" >> $GITHUB_PATH
- run: |
git submodule init
git submodule update
- run: shfmt --version
- run: shellcheck -V
- run: node --version
- run: npm run fmt
- run: npm ci
- run: npm run lint
- env:
GITHUB_TOKEN: ${{ github.token }}
run: npm run test
- run: npm run test

21
.gitignore vendored
View File

@@ -1,24 +1,7 @@
# generated artifacts
.env
node_modules
install-*.sh
install-*.bat
install-*.ps1
# local config
.env.*
*.env
.env
!example.env
# caches
_cache/
node_modules/
# temporary & backup files
.*.sw*
*.bak
*.bak.*
# other
.DS_Store
desktop.ini
.directory

View File

@@ -1,7 +1,4 @@
{
"globals": {
"AbortController": false
},
"browser": true,
"node": true,
"esversion": 11,

372
AGENTS.md
View File

@@ -1,372 +0,0 @@
# Webi Installers — Agent Guide
Webi installs dev tools to `~/.local/` without sudo. Each installer is a small
package of 3-4 files. This guide tells you how to create and modify them.
## Why Webi Exists
Webi makes tool installation trivially repeatable for people who aren't
sysadmins — freelance clients, junior devs, anyone who shouldn't have to care
about PATH, permissions, or platform differences. Three things matter:
1. **Install without friction.** No sudo, no manual PATH edits, no "necessary
but unimportant" steps leaking into the experience.
2. **Know where things are.** The Files section tells you exactly what got
created or modified. Nothing should be mysterious.
3. **Copy-paste recipes.** The cheat sheet is what you'd send someone less
experienced than yourself instead of a project's full README — scannable,
concrete, easy to reference by name.
## Quick Start: Adding a New Installer
1. Identify the **package type** (see [Categories](#categories) below)
2. Find an existing installer of the same type to use as a template
3. Create `<name>/releases.js`, `install.sh`, `install.ps1`, `README.md`
4. Test with the command in [Testing releases.js](#testing-releasesjs)
5. Run formatters before committing (see [Code Style](#code-style))
## Directory Layout
```
<package-name>/
README.md # YAML frontmatter + docs
releases.js # Fetches release metadata (Node.js)
install.sh # POSIX shell installer (macOS/Linux)
install.ps1 # PowerShell installer (Windows) — optional
```
Key infrastructure directories (do not modify without good reason):
- `_webi/` — bootstrap templates, `normalize.js` (auto-detects OS/arch/ext from
filenames)
- `_common/` — shared JS: `github.js`, `githubish.js`, `gitea.js`, `fetcher.js`
- `_example/` — canonical template for new packages
- `_examples/` — specialized templates (goreleaser, xz-compressed)
## Categories
Ref: <https://github.com/webinstall/webi-installers/issues/412>
| Type | Description | Template to copy |
| ----- | -------------------------------------- | ---------------- |
| `bin` | Single binary in tar/zip | `koji`, `delta` |
| `bin` | Single bare binary (no archive) | `arc`, `shfmt` |
| `bin` | Goreleaser-style archives | `keypairs` |
| 📦 | Self-contained package (bin/man/share) | `node`, `go` |
| 📂 | Multiple binaries/scripts | `pg` |
| 🔗 | Alias/redirect to another package | `ripgrep``rg` |
| 📝 | Bespoke / custom install | `rustlang` |
## releases.js
Fetches release metadata and returns a normalized object. Most packages use
GitHub releases:
```js
'use strict';
var github = require('../_common/github.js');
var owner = 'OWNER';
var repo = 'REPO';
let Releases = module.exports;
Releases.latest = async function () {
let all = await github(null, owner, repo);
return all;
};
Releases.sample = async function () {
let normalize = require('../_webi/normalize.js');
let all = await Releases.latest();
all = normalize(all);
all.releases = all.releases.slice(0, 5);
return all;
};
if (module === require.main) {
(async function () {
let samples = await Releases.sample();
console.info(JSON.stringify(samples, null, 2));
})();
}
```
### Common release transformations
**Strip version prefix** (monorepo or tool-prefixed tags):
```js
// e.g. "tools/monorel/v0.6.5" → "v0.6.5"
rel.version = rel.version.replace(/^tools\/monorel\//, '');
// e.g. "cli-v1.2.3" → "v1.2.3"
rel.version = rel.version.replace(/^cli-/, '');
```
**Filter releases** (monorepo with multiple tools, or unwanted assets):
```js
all.releases = all.releases.filter(function (rel) {
// Keep only releases for this tool
return rel.version.startsWith('tools/monorel/');
});
```
Apply transformations inside `Releases.latest`, before returning `all`.
**Available sources** beyond `github.js`:
- `_common/gitea.js` — Gitea servers
- `_common/git-tag.js` — Git tag listing
- Custom fetch from any JSON API (see `go/releases.js`, `terraform/releases.js`)
### Testing releases.js
```sh
node -e "
let Releases = require('./<name>/releases.js');
Releases.sample().then(function (all) {
console.log(JSON.stringify(all, null, 2));
});
"
```
Verify: versions are clean semver (`0.6.5` not `tools/monorel/v0.6.5`), OS/arch
detected correctly, download URLs resolve.
## install.sh
POSIX shell (`sh`, not bash). Always wrapped in a function:
```sh
#!/bin/sh
# shellcheck disable=SC2034
set -e
set -u
__init_pkgname() {
# These 6 variables are required
pkg_cmd_name="cmd"
pkg_dst_cmd="$HOME/.local/bin/cmd"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/cmd-v$WEBI_VERSION/bin/cmd"
pkg_src_dir="$HOME/.local/opt/cmd-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_cmd")"
mv ./cmd "$pkg_src_cmd"
}
pkg_get_current_version() {
cmd --version 2> /dev/null | head -n 1 | cut -d' ' -f2
}
}
__init_pkgname
```
### Framework variables available in install.sh
Set by the webi bootstrap (`_webi/package-install.tpl.sh`):
| Variable | Example | Description |
| --------------- | ------------------- | --------------------- |
| `WEBI_VERSION` | `1.2.3` | Selected version |
| `WEBI_PKG_URL` | `https://...` | Download URL |
| `WEBI_PKG_FILE` | `foo-v1.2.3.tar.gz` | Download filename |
| `WEBI_OS` | `linux` | Detected OS |
| `WEBI_ARCH` | `amd64` | Detected architecture |
| `WEBI_EXT` | `tar.gz` | Archive extension |
| `WEBI_CHANNEL` | `stable` | Release channel |
| `PKG_NAME` | `foo` | Package name |
### Override functions
| Function | Purpose |
| --------------------------- | --------------------------------------------- |
| `pkg_install()` | **Required.** Move files to `$pkg_src` |
| `pkg_get_current_version()` | Parse installed version from command output |
| `pkg_post_install()` | Post-install setup (git config, shell config) |
| `pkg_done_message()` | Custom completion message |
| `pkg_link()` | Override default symlink behavior |
| `pkg_pre_install()` | Custom pre-install logic |
### Framework helper functions
| Function | Purpose |
| ------------------------ | ---------------------------------- |
| `webi_download()` | Download package if not cached |
| `webi_extract()` | Extract archive by extension |
| `webi_path_add <dir>` | Add to PATH via envman |
| `webi_link()` | Create versioned symlinks |
| `webi_check_installed()` | Check if version already installed |
### pkg_install patterns
**Bare binary in archive root:**
```sh
mv ./cmd "$pkg_src_cmd"
```
**Binary in a subdirectory (goreleaser-style `cmd-OS-arch/cmd`):**
```sh
mv ./cmd-*/cmd "$pkg_src_cmd"
```
**Flexible detection (handles multiple archive layouts):**
```sh
if test -f ./cmd; then
mv ./cmd "$pkg_src_cmd"
elif test -e ./cmd-*/cmd; then
mv ./cmd-*/cmd "$pkg_src_cmd"
elif test -e ./cmd-*; then
mv ./cmd-* "$pkg_src_cmd"
fi
```
## install.ps1
PowerShell for Windows. Uses `$Env:` variables. See `_example/install.ps1` for
the full template. Key differences from install.sh:
- Paths use backslashes, commands end in `.exe`
- `$Env:USERPROFILE` instead of `$HOME`
- `Test-Path`, `Move-Item`, `Copy-Item` instead of shell equivalents
- Downloads go to `$Env:USERPROFILE\Downloads\webi\`
- Temp work in `.local\tmp`, use `Push-Location`/`Pop-Location`
- Symlinks done via `Copy-Item` (not actual symlinks)
## README.md
````markdown
---
title: toolname
homepage: https://github.com/owner/repo
tagline: |
toolname: A short one-line description.
---
To update or switch versions, run `webi toolname@stable` (or `@v2`, `@beta`,
etc).
### Files
These are the files that are created and/or modified with this installer:
```text
~/.config/envman/PATH.env
~/.local/bin/toolname
~/.local/opt/toolname-VERSION/bin/toolname
```
## Cheat Sheet
> `toolname` does X. Brief description.
### How to use toolname
```sh
toolname --example
```
````
Note: **Files goes above Cheat Sheet**, not inside it.
### Cheat Sheet tone and style
Webi cheat sheets are **opinionated quick-reference guides**, not comprehensive
documentation. Think "colleague's sticky note" — not the project's official
README.
The tool is the topic, but **the problem is the reason**. Cheat sheets are
organized around tasks the reader already wants to do — the tool is how they get
there. Headings reference the tool (the reader came to this page on purpose),
but the content solves the underlying problem completely:
- "How to reverse proxy to Node" (caddy knowledge, not just node)
- "How to run a Node app as a System Service" (serviceman knowledge)
- "How to Enable Secure Remote Postgres Access" (openssl, pg_hba.conf, systemd)
- "How to manually configure git to use delta" (gitconfig, not delta flags)
- "How to make fish the default shell in iTerm2" (iTerm2 knowledge, not fish)
The reader's question is "how do I do X?" and the cheat sheet answers it
completely — including configs, adjacent tools, and platform-specific
variations. A goreleaser cheat sheet teaches you goreleaser YAML. A postgres
cheat sheet teaches you pg_hba.conf, openssl certs, and systemd units.
Cheat sheets cross tool boundaries freely. Node's references caddy, serviceman,
setcap-netbind, GitHub Actions. Postgres references serviceman, openrc, launchd.
They link to each other's webi pages. The scope is "everything you need to
accomplish this task," not "everything this one binary does."
They show the actual files and configs that matter — not documentation _about_
configs, but the configs themselves, copy-pasteable, with inline comments
explaining the non-obvious parts.
**Guidelines:**
- **Show the 3-5 things someone will actually do**, with copy-pasteable
commands. Skip exhaustive flag lists and API docs.
- **Lead with practical integration.** Show the exact `git config` lines, the
exact hook script, the exact shell alias — don't just explain the feature and
leave wiring up to the reader.
- **Skip what they already know.** No need to re-explain what the tool is at
length — the tagline and one-liner blockquote handle that. Get to the
commands.
- **Prefer concrete over abstract.** Instead of "you can configure X via a
config file", show the config file contents.
## Shell Naming Conventions
**Variables:**
- `ALL_CAPS` — environment variables only (`PATH`, `HOME`, `WEBI_VERSION`)
- `b_varname` — block-scoped (inside a function, loop, or conditional)
- `g_varname` — global to the script (and sourced scripts)
- `a_varname` — function arguments
**Functions and commands:**
- `fn_name` — helper functions (anything other than the script's main/entry
function)
- `cmd_name` — command aliases, e.g. `cmd_curl='curl --fail-with-body -sSL'`
## Code Style
Requires `node`, `shfmt`, `pwsh`, and `pwsh-essentials` (install all via webi).
Run before committing:
```sh
npm run fmt # prettier (JS/MD) + shfmt (sh) + pwsh-fmt (ps1)
npm run lint # jshint + shellcheck + pwsh-lint
```
Commit messages: `feat(<pkg>): add installer`, `fix(<pkg>): update install.sh`,
`docs(<pkg>): add cheat sheet`.
## Naming Conventions
- The canonical package name is the **command name** you type: `go`, `node`,
`rg`
- The alternate/alias name is the project name: `golang`, `nodejs`, `ripgrep`
- Package directories are lowercase with hyphens
## Common Pitfalls
- **Monorepo releases**: The GitHub API returns ALL releases for the repo. You
must filter in `releases.js` and strip the tag prefix from the version.
- **No `--version` flag**: Some tools lack version introspection. Comment out
`pkg_get_current_version` — webi still works, it just can't skip reinstalls.
- **normalize.js auto-detection**: OS/arch/ext are guessed from download
filenames. If the tool uses non-standard naming, you may need to set `os`,
`arch`, or `ext` explicitly in `releases.js`.
- **Goreleaser archives**: Typically contain a bare binary at the archive root
(not nested in a directory). Use `mv ./cmd "$pkg_src_cmd"`.

View File

@@ -10,6 +10,7 @@
- You'll be asked to make changes if you don't run the code formatters and
linters:
- Node / JavaScript:
- [prettier](https://webinstall.dev/prettier)
```sh
@@ -20,6 +21,7 @@
npm run lint
```
- Bash
- [shfmt](https://webinstall.dev/shfmt)
```sh
npm run shfmt

View File

@@ -98,10 +98,9 @@ You just fill in the blanks.
Just create an empty directory and run the tests until you get a good result.
```sh
git clone git@github.com:webinstall/webi-installers.git
pushd ./webi-installers/
git submodule update --init
npm clean-install
git clone git@github.com:webinstall/packages.git
pushd packages
npm install
```
```sh
@@ -145,8 +144,8 @@ It looks like this:
`releases.js`:
```js
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
// if you need to do something special, you can do it here
// ...
return all;

View File

@@ -1,62 +1,64 @@
'use strict';
let Fetcher = require('../_common/fetcher.js');
/**
* Gets releases from 'brew'.
* Gets a releases from 'brew'.
*
* @param {null} _
* @param request
* @param {string} formula
* @returns {Promise<any>}
* @returns {PromiseLike<any> | Promise<any>}
*/
async function getDistributables(_, formula) {
function getAllReleases(request, formula) {
if (!formula) {
return Promise.reject('missing formula for brew');
}
let resp;
try {
let url = `https://formulae.brew.sh/api/formula/${formula}.json`;
resp = await Fetcher.fetch(url, {
headers: { Accept: 'application/json' },
});
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
if (err.code === 'E_FETCH_RELEASES') {
err.message = `failed to fetch '${formula}' release data from 'brew': ${err.response.status} ${err.response.body}`;
}
throw e;
}
let body = JSON.parse(resp.body);
var ver = body.versions.stable;
var dl = (
body.bottle.stable.files.high_sierra || body.bottle.stable.files.catalina
).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
return [
{
version: ver,
download: dl.replace(/{{ v }}/g, ver),
},
].concat(
body.versioned_formulae.map(
/** @param {String} f */
function (f) {
var ver = f.replace(/.*@/, '');
return {
return request({
url: 'https://formulae.brew.sh/api/formula/' + formula + '.json',
fail: true, // https://git.coolaj86.com/coolaj86/request.js/issues/2
json: true,
})
.then(failOnBadStatus)
.then(function (resp) {
var ver = resp.body.versions.stable;
var dl = (
resp.body.bottle.stable.files.high_sierra ||
resp.body.bottle.stable.files.catalina
).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
return [
{
version: ver,
download: dl,
};
},
),
);
download: dl.replace(/{{ v }}/g, ver),
},
].concat(
resp.body.versioned_formulae.map(function (f) {
var ver = f.replace(/.*@/, '');
return {
version: ver,
download: dl,
};
}),
);
})
.catch(function (err) {
console.error('Error fetching MariaDB versions (brew)');
console.error(err);
return [];
});
}
module.exports = getDistributables;
function failOnBadStatus(resp) {
if (resp.statusCode >= 400) {
var err = new Error('Non-successful status code: ' + resp.statusCode);
err.code = 'ESTATUS';
err.response = resp;
throw err;
}
return resp;
}
module.exports = getAllReleases;
if (module === require.main) {
getDistributables(null, 'mariadb').then(function (all) {
getAllReleases(require('@root/request'), 'mariadb').then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -1,56 +0,0 @@
'use strict';
let Fetcher = module.exports;
/**
* @typedef ResponseSummary
* @prop {Boolean} ok
* @prop {Headers} headers
* @prop {Number} status
* @prop {String} body
*/
/**
* @param {String} url
* @param {RequestInit} opts
* @returns {Promise<ResponseSummary>}
*/
Fetcher.fetch = async function (url, opts) {
let resp = await fetch(url, opts);
let summary = Fetcher.throwIfNotOk(resp);
return summary;
};
/**
* @param {Response} resp
* @returns {Promise<ResponseSummary>}
*/
Fetcher.throwIfNotOk = async function (resp) {
let text = await resp.text();
if (!resp.ok) {
let headers = Array.from(resp.headers);
console.error('[Fetcher] error: Response Headers:', headers);
console.error('[Fetcher] error: Response Text:', text);
let err = new Error(`fetch was not ok`);
Object.assign({
status: 503,
code: 'E_FETCH_RELEASES',
response: {
status: resp.status,
headers: headers,
body: text,
},
});
throw err;
}
let summary = {
ok: resp.ok,
headers: resp.headers,
status: resp.status,
body: text,
};
return summary;
};

View File

@@ -115,7 +115,7 @@ Repos.getCommitInfo = async function (repoPath, commitish) {
* @param {string} gitUrl
* @returns {PromiseLike<any> | Promise<any>}
*/
async function getDistributables(gitUrl) {
async function getAllReleases(gitUrl) {
let all = {
releases: [],
download: '',
@@ -190,7 +190,7 @@ async function getDistributables(gitUrl) {
return all;
}
module.exports = getDistributables;
module.exports = getAllReleases;
if (module === require.main) {
(async function main() {
@@ -203,7 +203,7 @@ if (module === require.main) {
//'https://github.com/dense-analysis/ale.git',
];
for (let url of testRepos) {
let all = await getDistributables(url);
let all = await getAllReleases(url);
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all, null, 2));

View File

@@ -1,47 +1,39 @@
'use strict';
var GitHubish = require('./githubish.js');
var ghRelease = require('./github.js');
/**
* Lists Gitea Releases (w/ uploaded assets)
* Gets the releases for 'ripgrep'. This function could be trimmed down and made
* for use with any github release.
*
* @param {null} _ - deprecated
* @param {String} owner
* @param {String} repo
* @param {String} baseurl
* @param {String} [username]
* @param {String} [token]
* @param request
* @param {string} owner
* @param {string} repo
* @returns {PromiseLike<any> | Promise<any>}
*/
async function getDistributables(
_,
owner,
repo,
baseurl,
username = '',
token = '',
) {
baseurl = `${baseurl}/api/v1`;
let all = await GitHubish.getDistributables({
owner,
repo,
baseurl,
username,
token,
});
return all;
function getAllReleases(request, owner, repo, baseurl) {
if (!baseurl) {
return Promise.reject('missing baseurl');
}
return ghRelease(request, owner, repo, baseurl + '/api/v1').then(
function (all) {
return all;
},
);
}
module.exports = getDistributables;
module.exports = getAllReleases;
if (module === require.main) {
getDistributables(
null,
'root',
'pathman',
'https://git.rootprojects.org',
'',
'',
).then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
getAllReleases(
require('@root/request'),
'coolaj86',
'go-pathman',
'https://git.coolaj86.com',
).then(
//getAllReleases(require('@root/request'), 'root', 'serviceman', 'https://git.rootprojects.org').then(
function (all) {
console.info(JSON.stringify(all, null, 2));
},
);
}

View File

@@ -1,40 +1,133 @@
'use strict';
require('dotenv').config({ path: '.env' });
let GitHubSource = module.exports;
let GitHubishSource = require('./githubish-source.js');
require('dotenv').config();
/**
* @param {Object} opts
* @param {String} opts.owner
* @param {String} opts.repo
* @param {String} [opts.baseurl]
* @param {String} [opts.username]
* @param {String} [opts.token]
* Gets the releases for 'ripgrep'. This function could be trimmed down and made
* for use with any github release.
*
* @param request
* @param {string} owner
* @param {string} repo
* @returns {PromiseLike<any> | Promise<any>}
*/
GitHubSource.getDistributables = async function ({
async function getAllReleases(
request,
owner,
repo,
oses,
arches,
baseurl = 'https://api.github.com',
username = process.env.GITHUB_USERNAME || '',
token = process.env.GITHUB_TOKEN || '',
}) {
let all = await GitHubishSource.getDistributables({
owner,
repo,
baseurl,
username,
token,
});
) {
if (!owner) {
return Promise.reject('missing owner for repo');
}
if (!repo) {
return Promise.reject('missing repo name');
}
let req = {
url: `${baseurl}/repos/${owner}/${repo}/releases`,
json: true,
};
// TODO I really don't like global config, find a way to do better
if (process.env.GITHUB_USERNAME) {
req.auth = {
user: process.env.GITHUB_USERNAME,
pass: process.env.GITHUB_TOKEN,
};
}
let resp = await request(req);
let gHubResp = resp.body;
let all = {
releases: [],
// TODO make this ':baseurl' + ':releasename'
download: '',
};
for (let release of gHubResp) {
// TODO tags aren't always semver / sensical
let tag = release['tag_name'];
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
let channel = 'stable';
if (release['prerelease']) {
channel = 'beta';
}
let date = release['published_at'] || '';
date = date.replace(/T.*/, '');
let urls = [release.tarball_url, release.zipball_url];
for (let url of urls) {
let resp = await request({
method: 'HEAD',
followRedirect: true,
followAllRedirects: true,
followOriginalHttpMethod: true,
url: url,
stream: true,
});
// Workaround for bug where method changes to GET
resp.destroy();
// content-disposition: attachment; filename=BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
let name = resp.headers['content-disposition'].replace(
/.*filename=([^;]+)(;|$)/,
'$1',
);
all.releases.push({
name: name,
version: tag,
lts: lts,
channel: channel,
date: date,
os: '', // will be guessed by download filename
arch: '', // will be guessed by download filename
ext: '', // will be normalized
download: resp.request.uri.href,
});
}
}
if (oses) {
return combinate(all, oses, arches);
}
return all;
};
}
function combinate(all, oses, arches) {
let releases = all.releases;
// ex: arches = ['amd64', 'arm64', 'armv7l', 'armv6l', 'x86'];
// ex: oses = ['macos', 'linux', 'bsd', 'posix'];
let combos = [];
for (let release of releases) {
for (let arch of arches) {
for (let os of oses) {
let combo = {
arch: arch,
os: os,
};
let rel = Object.assign({}, release, combo);
combos.push(rel);
}
}
}
all.releases = combos;
return all;
}
module.exports = getAllReleases;
if (module === require.main) {
GitHubSource.getDistributables(null, 'BeyondCodeBootcamp', 'DuckDNS.sh').then(
function (all) {
console.info(JSON.stringify(all, null, 2));
},
);
getAllReleases(
require('@root/request'),
'BeyondCodeBootcamp',
'DuckDNS.sh',
).then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -1,42 +1,102 @@
'use strict';
require('dotenv').config({ path: '.env' });
let GitHubish = require('./githubish.js');
require('dotenv').config();
/**
* Lists GitHub Releases (w/ uploaded assets)
*
* @param {null} _ - deprecated
* @param {String} owner
* @param {String} repo
* @param {String} [baseurl]
* @param {String} [username]
* @param {String} [token]
* @param request
* @param {string} owner
* @param {string} repo
* @returns {PromiseLike<any> | Promise<any>}
*/
module.exports = async function (
_,
async function getAllReleases(
request,
owner,
repo,
baseurl = 'https://api.github.com',
username = process.env.GITHUB_USERNAME || '',
token = process.env.GITHUB_TOKEN || '',
) {
let all = await GitHubish.getDistributables({
owner,
repo,
baseurl,
username,
token,
});
return all;
};
if (!owner) {
throw new Error('missing owner for repo');
}
if (!repo) {
throw new Error('missing repo name');
}
let GitHub = module.exports;
GitHub.getDistributables = module.exports;
var req = {
url: `${baseurl}/repos/${owner}/${repo}/releases`,
json: true,
};
// TODO I really don't like global config, find a way to do better
if (process.env.GITHUB_USERNAME) {
req.auth = {
user: process.env.GITHUB_USERNAME,
pass: process.env.GITHUB_TOKEN,
};
}
let resp = await request(req);
if (!resp.ok) {
console.error('Bad Resp Headers:', resp.headers);
console.error('Bad Resp Body:', resp.body);
throw new Error('the elusive releases BOOGEYMAN strikes again');
}
let gHubResp = resp.body;
let all = {
releases: [],
// todo make this ':baseurl' + ':releasename'
download: '',
};
try {
gHubResp.forEach(transformReleases);
} catch (e) {
console.error(e.message);
console.error('Error Headers:', resp.headers);
console.error('Error Body:', resp.body);
throw e;
}
function transformReleases(release) {
for (let asset of release['assets']) {
let name = asset['name'];
let date = release['published_at']?.replace(/T.*/, '');
let download = asset['browser_download_url'];
// TODO tags aren't always semver / sensical
let version = release['tag_name'];
let channel;
if (release['prerelease']) {
// -rcX, -preview, -beta, etc will be checked in _webi/normalize.js
channel = 'beta';
}
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
all.releases.push({
name: name,
version: version,
lts: lts,
channel: channel,
date: date,
os: '', // will be guessed by download filename
arch: '', // will be guessed by download filename
ext: '', // will be normalized
download: download,
});
}
}
return all;
}
module.exports = getAllReleases;
if (module === require.main) {
GitHub.getDistributables(null, 'BurntSushi', 'ripgrep').then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
getAllReleases(require('@root/request'), 'BurntSushi', 'ripgrep').then(
function (all) {
console.info(JSON.stringify(all, null, 2));
},
);
}

View File

@@ -1,174 +0,0 @@
'use strict';
let Fetcher = require('../_common/fetcher.js');
let GitHubishSource = module.exports;
/**
* Lists GitHub-Like Releases (source tarball & zip)
*
* @param {Object} opts
* @param {String} opts.owner
* @param {String} opts.repo
* @param {String} opts.baseurl
* @param {String} [opts.username]
* @param {String} [opts.token]
*/
GitHubishSource.getDistributables = async function ({
owner,
repo,
baseurl,
username = '',
token = '',
}) {
if (!owner) {
throw new Error('missing owner for repo');
}
if (!repo) {
throw new Error('missing repo name');
}
if (!baseurl) {
throw new Error('missing baseurl');
}
let url = `${baseurl}/repos/${owner}/${repo}/releases`;
let opts = {
headers: {
'Content-Type': 'appplication/json',
},
};
if (token) {
let userpass = `${username}:${token}`;
let basicAuth = btoa(userpass);
Object.assign(opts.headers, {
Authorization: `Basic ${basicAuth}`,
});
}
let resp;
try {
resp = await Fetcher.fetch(url, opts);
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
if (err.code === 'E_FETCH_RELEASES') {
err.message = `failed to fetch '${baseurl}' (githubish-source, user '${username}) release data: ${err.response.status} ${err.response.body}`;
}
throw e;
}
let gHubResp = JSON.parse(resp.body);
let all = {
/** @type {Array<BuildInfo>} */
releases: [],
download: '',
};
for (let release of gHubResp) {
let dists = GitHubishSource.releaseToDistributables(release);
for (let dist of dists) {
let updates =
await GitHubishSource.followDistributableDownloadAttachment(dist);
Object.assign(dist, updates);
all.releases.push(dist);
}
}
return all;
};
/**
* @typedef BuildInfo
* @prop {String} [name] - name to use instead of filename for hash urls
* @prop {String} version
* @prop {String} [_version]
* @prop {String} [arch]
* @prop {String} channel
* @prop {String} date
* @prop {String} download
* @prop {String} [ext]
* @prop {String} [_filename]
* @prop {String} [hash]
* @prop {String} [libc]
* @prop {Boolean} [_musl]
* @prop {Boolean} [lts]
* @prop {String} [size]
* @prop {String} os
*/
/**
* @param {any} ghRelease - TODO
* @returns {Array<BuildInfo>}
*/
GitHubishSource.releaseToDistributables = function (ghRelease) {
let ghTag = ghRelease['tag_name']; // TODO tags aren't always semver / sensical
let lts = /(\b|_)(lts)(\b|_)/.test(ghRelease['tag_name']);
let channel = 'stable';
if (ghRelease['prerelease']) {
channel = 'beta';
}
let date = ghRelease['published_at'] || '';
date = date.replace(/T.*/, '');
let urls = [ghRelease.tarball_url, ghRelease.zipball_url];
/** @type {Array<BuildInfo>} */
let dists = [];
for (let url of urls) {
dists.push({
name: '',
version: ghTag,
lts: lts,
channel: channel,
date: date,
os: '*',
arch: '*',
libc: '',
ext: '',
download: url,
});
}
return dists;
};
/**
* @param {BuildInfo} dist
*/
GitHubishSource.followDistributableDownloadAttachment = async function (dist) {
let abortCtrl = new AbortController();
let resp = await fetch(dist.download, {
method: 'HEAD',
redirect: 'follow',
signal: abortCtrl.signal,
});
let headers = Object.fromEntries(resp.headers);
// Workaround for bug where METHOD changes to GET
abortCtrl.abort();
await resp.text().catch(function (err) {
if (err.name !== 'AbortError') {
throw err;
}
});
// ex: content-disposition: attachment; filename=BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
// => BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
let name = headers['content-disposition'].replace(
/.*filename=([^;]+)(;|$)/,
'$1',
);
let download = resp.url;
return { name, download };
};
if (module === require.main) {
GitHubishSource.getDistributables({
owner: 'BeyondCodeBootcamp',
repo: 'DuckDNS.sh',
baseurl: 'https://api.github.com',
}).then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -1,137 +0,0 @@
'use strict';
let Fetcher = require('../_common/fetcher.js');
/**
* @typedef DistributableRaw
* @prop {String} name
* @prop {String} version
* @prop {Boolean} lts
* @prop {String} [channel]
* @prop {String} date
* @prop {String} os
* @prop {String} arch
* @prop {String} ext
* @prop {String} download
*/
let GitHubish = module.exports;
/**
* Lists GitHub-Like Releases (w/ uploaded assets)
*
* @param {Object} opts
* @param {String} opts.owner
* @param {String} opts.repo
* @param {String} opts.baseurl
* @param {String} [opts.username]
* @param {String} [opts.token]
*/
GitHubish.getDistributables = async function ({
owner,
repo,
baseurl,
username = '',
token = '',
}) {
if (!owner) {
throw new Error('missing owner for repo');
}
if (!repo) {
throw new Error('missing repo name');
}
if (!baseurl) {
throw new Error('missing baseurl');
}
let url = `${baseurl}/repos/${owner}/${repo}/releases`;
let opts = {
headers: {
'Content-Type': 'appplication/json',
},
};
if (token) {
let userpass = `${username}:${token}`;
let basicAuth = btoa(userpass);
Object.assign(opts.headers, {
Authorization: `Basic ${basicAuth}`,
});
}
let resp;
try {
resp = await Fetcher.fetch(url, opts);
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
if (err.code === 'E_FETCH_RELEASES') {
err.message = `failed to fetch '${baseurl}' (githubish, user '${username}) release data: ${err.response.status} ${err.response.body}`;
}
throw e;
}
let gHubResp = JSON.parse(resp.body);
let all = {
/** @type {Array<DistributableRaw>} */
releases: [],
// todo make this ':baseurl' + ':releasename'
download: '',
};
try {
gHubResp.forEach(transformReleases);
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
console.error(err.message);
console.error('Error Headers:', resp.headers);
console.error('Error Body:', resp.body);
let msg = `failed to transform releases from '${baseurl}' with user '${username}'`;
throw new Error(msg);
}
/**
* @param {any} release - TODO
*/
function transformReleases(release) {
for (let asset of release['assets']) {
let name = asset['name'];
let date = release['published_at']?.replace(/T.*/, '');
let download = asset['browser_download_url'];
// TODO tags aren't always semver / sensical
let version = release['tag_name'];
let channel;
if (release['prerelease']) {
// -rcX, -preview, -beta, etc will be checked in _webi/normalize.js
channel = 'beta';
}
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
all.releases.push({
name: name,
version: version,
lts: lts,
channel: channel,
date: date,
os: '', // will be guessed by download filename
arch: '', // will be guessed by download filename
ext: '', // will be normalized
download: download,
});
}
}
return all;
};
if (module === require.main) {
GitHubish.getDistributables({
owner: 'BurntSushi',
repo: 'ripgrep',
baseurl: 'https://api.github.com',
}).then(function (all) {
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -23,7 +23,7 @@ install:
```text
~/.config/envman/PATH.env
~/.local/bin/foo
~/.local/opt/foo/
~/.local/opt/foo
```
## Cheat Sheet

View File

@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading foobar from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing foobar"
# TODO: create package-specific temp directory

View File

@@ -12,26 +12,17 @@ var repo = 'ripgrep';
/** **/
/******************************************************************************/
let Releases = module.exports;
Releases.latest = async function () {
let all = await github(null, owner, repo);
return all;
};
Releases.sample = async function () {
let normalize = require('../_webi/normalize.js');
let all = await Releases.latest();
all = normalize(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);
return all;
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
(async function () {
let samples = await Releases.sample();
console.info(JSON.stringify(samples, null, 2));
})();
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -4,6 +4,7 @@
//var pkg = require('../package.json');
var os = require('os');
//var request = require('@root/request');
//var promisify = require('util').promisify;
//var exec = promisify(require('child_process').exec);
var exec = require('child_process').exec;

View File

@@ -15,8 +15,8 @@ foreach ($my_dir in $my_dirs) {
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
-not (Test-Path -PathType Container -Path $my_dir)) {
if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
-Not (Test-Path -PathType Container -Path $my_dir)) {
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
continue
}
@@ -31,7 +31,7 @@ foreach ($my_dir in $my_dirs) {
$my_new_file | Set-Content -Path $my_ps1
$my_new_file = $my_new_file + "`n"
if ($text -ne $my_new_file) {
IF ($text -ne $my_new_file) {
$my_status = 1
}
}

View File

@@ -15,8 +15,8 @@ foreach ($my_dir in $my_dirs) {
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
-not (Test-Path -PathType Container -Path $my_dir)) {
if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
-Not (Test-Path -PathType Container -Path $my_dir)) {
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
continue
}
@@ -39,7 +39,7 @@ foreach ($my_dir in $my_dirs) {
$my_new_file | Set-Content -Path $my_ps1
$my_new_file = $my_new_file + "`n"
if ($my_old_file -ne $my_new_file) {
IF ($my_old_file -ne $my_new_file) {
$my_status = 1
}
}

View File

@@ -1,442 +0,0 @@
---
name: installer
description: >
Create or update install.sh and install.ps1 scripts for a webi package.
Use when adding a new package to webi-installers, or when an existing
install script needs to be updated to match a changed archive structure.
Covers discovering archive layout from GitHub releases, identifying the
right install pattern (AI), and writing both the POSIX shell and
PowerShell scripts that the webi framework calls.
Note: this skill covers install scripts only — writing releases.js /
releases.conf (the release-fetcher config) is a separate concern.
license: MIT
compatibility: Requires git, curl, tar. GitHub API access needed for
discovery phase. Designed for Claude Code in the webi-installers repo.
metadata:
author: AJ ONeal
version: "1.1"
---
# Webi Installer Skill
Write `install.sh` and `install.ps1` for a webi package. These scripts are
called by the webi framework **after** it has already downloaded and verified
the archive — your job is only to unpack and place the files.
> **Scope:** This skill covers `install.sh` and `install.ps1` only. A
> separate `releases.js` / `releases.conf` file is needed to tell webi where
> to fetch releases from. That config must already exist (or be written
> separately) before these install scripts are useful.
## Quick overview
1. [Discover the archive layout](#1-discover-the-archive-layout) — inspect
GitHub releases with `curl` + `tar -t` to understand what's inside.
2. [Choose the install pattern](#2-choose-the-install-pattern) — nine
patterns (AI) cover almost every real-world case.
3. [Write `install.sh`](#3-write-installsh) — POSIX shell, ~2040 lines.
4. [Write `install.ps1`](#4-write-installps1) — PowerShell, ~4060 lines.
5. [Check for classification issues](#5-check-for-classification-issues) —
look for variant assets, non-standard OS/arch naming, or installer .exe
files that need special handling.
Full reference: [`references/PATTERNS.md`](references/PATTERNS.md)
Archive layout details: [`references/ARCHIVE-LAYOUTS.md`](references/ARCHIVE-LAYOUTS.md)
Classification guide: [`references/CLASSIFICATION.md`](references/CLASSIFICATION.md)
---
## 1. Discover the archive layout
### Use the webi releases API (fastest, if the package already exists)
```sh
# JSON with all releases for a package
curl -s https://webinstall.dev/api/releases/bat.json | jq '.releases[:3]'
```
Each entry has `name` (filename), `version`, `os`, `arch`, `ext`, `download`.
### Or inspect GitHub releases directly
```sh
# List asset filenames for the latest release
curl -s "https://api.github.com/repos/sharkdp/bat/releases?per_page=3" \
| jq '.[0].assets[] | .name'
```
### Inspect what's inside an archive
Download one representative asset and list its contents **without extracting**:
```sh
# tar.gz / tar.xz
curl -fsSL "$DOWNLOAD_URL" | tar -tz
# tar.zst (modern systems — GNU tar / bsdtar both support this)
curl -fsSL "$DOWNLOAD_URL" | tar --zstd -tz
# zip
curl -fsSL "$DOWNLOAD_URL" -o /tmp/pkg.zip && unzip -l /tmp/pkg.zip
# bare binary (no archive extension, e.g. jq-linux-amd64)
# The file IS the binary — no unpacking needed. Set WEBI_SINGLE=true.
```
Look for:
- Is the binary at the top level or inside a subdirectory?
- Does the subdirectory name include the version and/or triplet?
- Are there completions (`completions/`, `autocomplete/`, `complete/`)?
- Are there man pages (`*.1`, `doc/*.1`, `man/man1/`)?
- Are there shared libraries (`.so`, `.dylib`, `.dll`) alongside the binary?
- Is the binary name different from the package command name?
See [`references/ARCHIVE-LAYOUTS.md`](references/ARCHIVE-LAYOUTS.md) for
what each pattern looks like, with real examples.
---
## 2. Choose the install pattern
| Pattern | Description | Examples |
|---------|-------------|---------|
| **A** | Bare binary (or binary+docs) at archive root | caddy, fzf, k9s, terraform |
| **B** | Binary inside a version/triplet-named subdirectory | delta, shellcheck, trip, xsv |
| **C** | Like B, plus shell completions and/or man pages | bat, fd, rg, sd, watchexec, zoxide |
| **D** | Binary + shared libraries (bundled) | ollama (Linux), psql, sass, syncthing |
| **E** | FHS-like layout (`bin/`, `share/man/`) | gh, pandoc |
| **F** | Renamed binary needing install-time rename | pathman, yq |
| **G** | Full SDK/toolchain (many files) | go, node, zig, flutter, julia |
| **H** | .NET runtime bundle | pwsh |
| **I** | Multi-binary distribution | dashcore, mutagen |
**Pattern A** is by far the most common (~28 packages). When in doubt,
download the archive and `tar -tz` it before writing a single line of code.
---
## 3. Write `install.sh`
The framework (`_webi/package-install.tpl.sh`) handles: user-agent detection,
version resolution, download, checksum verification, and PATH management.
Your script is **injected into** the framework and provides the
package-specific part: where to find the binary and how to move it.
### Script structure
Every `install.sh` wraps its definitions in an `__init_pkgname()` function
and immediately calls it. This prevents variable leakage when the script is
sourced by the framework:
```sh
#!/bin/sh
__init_toolname() {
set -e
set -u
####################
# Install toolname #
####################
pkg_cmd_name="toolname"
WEBI_SINGLE=true # if applicable — see below
pkg_dst_cmd="$HOME/.local/bin/toolname"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/toolname-v$WEBI_VERSION/bin/toolname"
pkg_src_dir="$HOME/.local/opt/toolname-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
# ...
}
pkg_get_current_version() {
# ...
}
}
__init_toolname
```
### Variables
| Variable | Description |
|----------|-------------|
| `pkg_cmd_name` | The command name that ends up on `$PATH` |
| `pkg_dst_cmd` | Final destination: `~/.local/bin/<cmd>` (the symlink) |
| `pkg_dst` | Same as `pkg_dst_cmd` for single-binary packages; `~/.local/opt/<cmd>` for SDKs |
| `pkg_src_cmd` | Versioned binary: `~/.local/opt/<pkg>-v<ver>/bin/<cmd>` |
| `pkg_src_dir` | Versioned install dir: `~/.local/opt/<pkg>-v<ver>` |
| `pkg_src` | Same as `pkg_src_cmd` for single-binary packages; same as `pkg_src_dir` for SDKs |
**Framework-derived (set by the framework before calling `pkg_install` — do not set manually):**
- `pkg_src_bin``$(dirname "$pkg_src_cmd")` — the versioned `bin/` dir
- `pkg_dst_bin``$(dirname "$pkg_dst_cmd")``~/.local/bin`
### `WEBI_SINGLE`
`WEBI_SINGLE=true` affects the default values the framework uses for
`pkg_src` and `pkg_dst`, and how `webi_link()` creates the symlink:
- **With `WEBI_SINGLE=true`**: links the binary file directly:
`~/.local/bin/cmd → ~/.local/opt/cmd-vX.Y.Z/bin/cmd`
- **Without it (default)**: links the directory:
`~/.local/opt/cmd → ~/.local/opt/cmd-vX.Y.Z`
Set `WEBI_SINGLE=true` when using the conventional Pattern A skeleton
(where `pkg_src` and `pkg_dst` are not set to custom values). When you
explicitly assign all six variables yourself (as in Patterns BF),
`WEBI_SINGLE` is not strictly required but can still be set for clarity.
Pattern G (SDKs) and Pattern H (.NET bundles) do NOT use `WEBI_SINGLE`
they define `pkg_link()` manually because the whole directory tree must
be linked, not just a single binary.
### Required function: `pkg_install`
Moves files from the extracted archive into the versioned opt directory.
The framework has already extracted the archive into a temp directory and
`cd`'d into it before calling `pkg_install`.
```sh
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./tool-*/tool "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
### Recommended function: `pkg_get_current_version`
Used to detect whether the package is already installed at the right version:
```sh
pkg_get_current_version() {
# 'tool --version' output: "tool 1.2.3 (rev abc)"
# trim to just the version number
tool --version 2>/dev/null | head -n 1 | cut -d' ' -f2
}
```
### Skeletons by pattern
**Pattern A** — binary at archive root (`WEBI_SINGLE=true`):
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./"$pkg_cmd_name"* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
Use `$pkg_cmd_name*` as the glob — it matches the binary and avoids
accidentally moving LICENSE or README into the binary path.
**Pattern B** — binary inside a `tool-{ver}-{triplet}/` subdirectory:
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./tool-*/tool "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
**Pattern C** — like B, plus completions and man pages.
The completion directory and filename vary per package — always check
`tar -tz` output first. Common variants: `completions/`, `autocomplete/`,
`complete/`. See [`references/PATTERNS.md`](references/PATTERNS.md) for
a full example with guards:
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./tool-*/tool "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
# bash completion (directory name varies — check tar -tz)
if test -e ./tool-*/completions/tool.bash; then
mkdir -p "$pkg_src_dir/share/bash-completion/completions"
mv ./tool-*/completions/tool.bash \
"$pkg_src_dir/share/bash-completion/completions/tool"
fi
if test -e ./tool-*/completions/tool.fish; then
mkdir -p "$pkg_src_dir/share/fish/vendor_completions.d"
mv ./tool-*/completions/tool.fish \
"$pkg_src_dir/share/fish/vendor_completions.d/tool.fish"
fi
if test -e ./tool-*/completions/_tool; then
mkdir -p "$pkg_src_dir/share/zsh/site-functions"
mv ./tool-*/completions/_tool \
"$pkg_src_dir/share/zsh/site-functions/_tool"
fi
if test -e ./tool-*/tool.1; then
mkdir -p "$pkg_src_dir/share/man/man1"
mv ./tool-*/tool.1 "$pkg_src_dir/share/man/man1/tool.1"
fi
}
```
**Pattern D** — binary + shared libraries. The entire directory structure
must be preserved. See [`references/PATTERNS.md`](references/PATTERNS.md)
for the ollama and psql examples.
**Pattern E** — FHS layout (archive already has `bin/` and `share/`):
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./tool-*/ "$pkg_src_dir"
}
```
**Pattern F** — binary needs rename (archive name ≠ command name).
Use when the binary in the archive cannot be matched by `$pkg_cmd_name*`
— e.g., `yq_linux_amd64` for a command named `yq`:
```sh
WEBI_SINGLE=true
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./yq_* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
**Pattern G** — full SDK (do NOT set `WEBI_SINGLE`):
```sh
# pkg_src = directory, not a binary
pkg_src="$pkg_src_dir"
pkg_dst="$HOME/.local/opt/tool"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./tool-*/ "$pkg_src_dir"
}
pkg_link() {
rm -f "$pkg_dst"
ln -s "$pkg_src" "$pkg_dst"
}
```
---
## 4. Write `install.ps1`
A PowerShell framework template exists (`_webi/package-install.tpl.ps1`)
and injects the `install.ps1` script at the `# {{ installer }}` placeholder.
The template provides: error handling, directory setup, `Invoke-DownloadUrl`
helper, and PATH management via `webi_path_add`. However, unlike the shell
side, the PS1 framework does **not** download or extract the archive — the
package script must handle that itself. The same path conventions apply
(opt/bin layout), but Windows uses `Copy-Item` instead of symlinks for
the final `bin/` step.
### Variable block (always at top)
```powershell
$pkg_cmd_name = "tool"
$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\tool.exe"
$pkg_dst_bin = "$Env:USERPROFILE\.local\bin"
$pkg_dst = "$pkg_dst_cmd"
$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\tool-v$Env:WEBI_VERSION\bin\tool.exe"
$pkg_src_bin = "$Env:USERPROFILE\.local\opt\tool-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\tool-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
```
### Standard body
```powershell
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$pkg_download")) {
Write-Output "Downloading tool from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing tool"
Push-Location .local\tmp
Remove-Item -Path ".\tool-v*" -Recurse -ErrorAction Ignore
# Unpack — Windows BSD-tar handles zip too
Write-Output "Unpacking $pkg_download"
& tar xf "$pkg_download"
# Move binary into place — adjust glob for your archive structure
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
Move-Item -Path ".\tool-*\tool.exe" -Destination "$pkg_src_bin"
Pop-Location
}
# Windows has no symlinks in the webi sense — copy to bin/
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
New-Item "$pkg_dst_bin" -ItemType Directory -Force | Out-Null
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse
```
For Pattern A (binary at archive root), change the `Move-Item` line to:
```powershell
Move-Item -Path ".\tool.exe" -Destination "$pkg_src_bin"
```
---
## 5. Check for classification issues
Before writing any scripts, scan the asset list for red flags:
### Non-standard OS/arch names in filenames
The webi classifier recognises most patterns automatically. Watch for:
- `darwin` vs `macos` — both recognised; output normalised to `macos`
- `x86_64` vs `amd64` — both recognised; output normalised to `amd64`
- `aarch64` vs `arm64` — both recognised; output normalised to `arm64`
- `armv7` (missing trailing `l`) — normalised to `armv7l`
These are handled automatically. Only flag them if the asset list contains
something genuinely unusual that the classifier would not recognise.
### Variant assets needing tags
Flag if you see multiple assets for the same OS/arch that serve different
hardware or runtime requirements:
- **GPU variants**: `*-rocm*`, `*-cuda*`, `*-vulkan*` alongside a baseline build
- **Windows installer**: `*Setup.exe` or `*Install.exe` alongside a bare `*.exe`
- **Framework-dependent .NET**: `*-fxdependent*` vs self-contained
- **AppImage**: `*.AppImage` — not supported by the webi installer
- **Electron/GUI app**: `*.dmg` or `*.AppImage` that is a full GUI app, not a CLI
If you find variants, see [`references/CLASSIFICATION.md`](references/CLASSIFICATION.md)
for how to write a variant tagger.
### Formats to drop
These are automatically filtered by the framework — no action needed:
- `.deb`, `.rpm`, `.snap`, `.AppImage`
- Checksums (`*.sha256`, `*.sha512`, `*.asc`, `*.sig`)
- Source archives (`*-src.tar.gz`, `*.tar.gz` with no OS in name)
---
## Reference files
- [`references/PATTERNS.md`](references/PATTERNS.md) — detailed pattern
descriptions with real package examples and complete install script snippets
- [`references/ARCHIVE-LAYOUTS.md`](references/ARCHIVE-LAYOUTS.md) — actual
`tar -t` output for representative packages in each pattern
- [`references/CLASSIFICATION.md`](references/CLASSIFICATION.md) — when and
how to write variant taggers; non-standard filename conventions

View File

@@ -1,289 +0,0 @@
# Archive Layouts — Real Package Examples
Actual `tar -t` / `unzip -l` output for representative packages.
Use these to calibrate your eye for what each pattern looks like.
---
## Pattern A — Flat archive (no subdirectory)
### caddy 2.9.1 — linux/amd64 tar.gz
```
caddy
LICENSE
README.md
```
Binary `caddy` is at the top level. Set `WEBI_SINGLE=true`.
### fzf 0.70.0 — linux/amd64 tar.gz
```
fzf
```
Minimal — just the binary.
### terraform 1.9.8 — linux/amd64 zip
```
terraform
LICENSE.txt
```
Zip archive but same flat layout.
### k9s — linux/amd64 tar.gz
```
k9s
LICENSE
README.md
```
---
## Pattern B — Named subdirectory, binary only
### delta 0.18.2 — linux/amd64 tar.gz
```
delta-0.18.2-x86_64-unknown-linux-musl/
delta-0.18.2-x86_64-unknown-linux-musl/delta
delta-0.18.2-x86_64-unknown-linux-musl/LICENSE
delta-0.18.2-x86_64-unknown-linux-musl/README.md
```
Glob to move: `./delta-*/delta`
### shellcheck 0.10.0 — linux/x86_64 tar.xz
```
shellcheck-v0.10.0/
shellcheck-v0.10.0/shellcheck
shellcheck-v0.10.0/LICENSE.txt
shellcheck-v0.10.0/README.txt
```
Glob to move: `./shellcheck-*/shellcheck`
### xsv 0.13.0 — linux/x86_64 tar.gz
```
xsv-0.13.0-x86_64-unknown-linux-musl/
xsv-0.13.0-x86_64-unknown-linux-musl/xsv
xsv-0.13.0-x86_64-unknown-linux-musl/UNLICENSE
```
---
## Pattern C — Subdirectory + completions + man pages
### rg/ripgrep 14.1.1 — linux/amd64 tar.gz
```
ripgrep-14.1.1-x86_64-unknown-linux-musl/
ripgrep-14.1.1-x86_64-unknown-linux-musl/rg
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/_rg
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/_rg.ps1
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/rg.bash
ripgrep-14.1.1-x86_64-unknown-linux-musl/complete/rg.fish
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/rg.1
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/FAQ.md
ripgrep-14.1.1-x86_64-unknown-linux-musl/doc/GUIDE.md
ripgrep-14.1.1-x86_64-unknown-linux-musl/CHANGELOG.md
ripgrep-14.1.1-x86_64-unknown-linux-musl/LICENSE-MIT
ripgrep-14.1.1-x86_64-unknown-linux-musl/README.md
```
Note: completions are in `complete/` (not `completions/`). Man page is `doc/rg.1`.
### sd 1.1.0 — linux/x86_64 tar.gz
```
sd-v1.1.0-x86_64-unknown-linux-musl/
sd-v1.1.0-x86_64-unknown-linux-musl/sd
sd-v1.1.0-x86_64-unknown-linux-musl/sd.1
sd-v1.1.0-x86_64-unknown-linux-musl/completions/
sd-v1.1.0-x86_64-unknown-linux-musl/completions/sd.bash
sd-v1.1.0-x86_64-unknown-linux-musl/completions/sd.elv
sd-v1.1.0-x86_64-unknown-linux-musl/completions/sd.fish
sd-v1.1.0-x86_64-unknown-linux-musl/completions/_sd
sd-v1.1.0-x86_64-unknown-linux-musl/completions/_sd.ps1
sd-v1.1.0-x86_64-unknown-linux-musl/CHANGELOG.md
sd-v1.1.0-x86_64-unknown-linux-musl/LICENSE
sd-v1.1.0-x86_64-unknown-linux-musl/README.md
```
Note: man page `sd.1` is at subdirectory root. Completions in `completions/`.
### bat 0.26.1 — linux/amd64 tar.gz
```
bat-v0.26.1-x86_64-unknown-linux-musl/
bat-v0.26.1-x86_64-unknown-linux-musl/bat
bat-v0.26.1-x86_64-unknown-linux-musl/bat.1
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/bat.bash
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/bat.fish
bat-v0.26.1-x86_64-unknown-linux-musl/autocomplete/bat.zsh
bat-v0.26.1-x86_64-unknown-linux-musl/LICENSE-APACHE
bat-v0.26.1-x86_64-unknown-linux-musl/LICENSE-MIT
bat-v0.26.1-x86_64-unknown-linux-musl/README.md
```
Note: completions in `autocomplete/` (not `completions/`). Zsh file is `bat.zsh` not `_bat`.
### goreleaser — linux/amd64 tar.gz
```
goreleaser
completions/
completions/goreleaser.bash
completions/goreleaser.fish
completions/goreleaser.zsh
manpages/
manpages/goreleaser.1.gz
LICENSE.md
README.md
```
Note: goreleaser uses Pattern A layout (binary at root, no subdirectory)
but includes completions and a gzipped man page. Set `WEBI_SINGLE=true`;
move completions and man page after the binary.
---
## Pattern D — Binary + shared libraries
### ollama 0.17.7 — linux/amd64 tar.zst
```
bin/
bin/ollama
lib/
lib/ollama/
lib/ollama/libggml-base.so
lib/ollama/libggml-cpu-alderlake.so
lib/ollama/libggml-cpu-haswell.so
lib/ollama/libggml-cpu-icelake.so
lib/ollama/libggml-cpu-sandybridge.so
lib/ollama/libggml-cpu-skylakex.so
lib/ollama/libggml-cpu-sse42.so
lib/ollama/libggml-cpu-x64.so
lib/ollama/cuda_v12/
lib/ollama/cuda_v12/libcublas.so.12
lib/ollama/cuda_v12/libcublasLt.so.12
lib/ollama/cuda_v12/libcudart.so.12
lib/ollama/cuda_v12/libggml-cuda.so
... (66 files total)
```
Extract bin/ and lib/ directories separately or together.
### psql (postgres client) — linux/amd64 tar.gz
```
psql-17.2-linux-x86_64/
psql-17.2-linux-x86_64/bin/
psql-17.2-linux-x86_64/bin/psql
psql-17.2-linux-x86_64/lib/
psql-17.2-linux-x86_64/lib/libpq.so.5
psql-17.2-linux-x86_64/lib/libz.so.1
psql-17.2-linux-x86_64/lib/libzstd.so.1
psql-17.2-linux-x86_64/lib/libssl.so.3
psql-17.2-linux-x86_64/lib/libcrypto.so.3
psql-17.2-linux-x86_64/include/
... (75 files total)
```
Move the entire `psql-{ver}-{triplet}/` directory: `mv ./psql-*/ "$pkg_src_dir"`
---
## Pattern E — FHS layout
### gh 2.67.0 — linux/amd64 tar.gz
```
gh_2.67.0_linux_amd64/
gh_2.67.0_linux_amd64/bin/
gh_2.67.0_linux_amd64/bin/gh
gh_2.67.0_linux_amd64/share/
gh_2.67.0_linux_amd64/share/man/
gh_2.67.0_linux_amd64/share/man/man1/
gh_2.67.0_linux_amd64/share/man/man1/gh-actions-cache-delete.1
gh_2.67.0_linux_amd64/share/man/man1/gh-actions-cache-list.1
... (129 man pages)
gh_2.67.0_linux_amd64/LICENSE
```
Move the entire `gh_*/` directory: `mv ./gh_*/ "$pkg_src_dir"`
---
## Pattern F — Binary needs rename
### yq — linux/amd64 tar.gz (WEBI_SINGLE=true)
```
yq_linux_amd64
yq.1
```
Binary is `yq_linux_amd64` — must rename to `yq` during install.
### pathman 0.6.0 — linux/amd64 tar.gz (WEBI_SINGLE=true)
```
pathman-v0.6.0-linux-amd64_v1
```
Binary name includes the full release tag. Rename to `pathman`.
---
## Pattern G — Full SDK
### node 24.14.0 — linux/amd64 tar.xz
```
node-v24.14.0-linux-x64/
node-v24.14.0-linux-x64/bin/
node-v24.14.0-linux-x64/bin/node
node-v24.14.0-linux-x64/bin/npm -> ../lib/node_modules/npm/bin/npm-cli.js
node-v24.14.0-linux-x64/bin/npx -> ../lib/node_modules/npm/bin/npx-cli.js
node-v24.14.0-linux-x64/include/
node-v24.14.0-linux-x64/lib/
node-v24.14.0-linux-x64/lib/node_modules/
node-v24.14.0-linux-x64/share/
... (thousands of files)
```
Move entire directory: `mv ./node-*/ "$pkg_src_dir"`
### go 1.24.1 — linux/amd64 tar.gz
```
go/
go/bin/
go/bin/go
go/bin/gofmt
go/src/
go/pkg/
... (thousands of files)
```
Note: go's archive root directory is literally `go/` with no version in the name.
---
## Pattern H — .NET runtime bundle
### pwsh 7.4.6 — linux/amd64 tar.gz
```
pwsh
Accessibility.dll
clrcompression.dll
clrjit.dll
createdump
cs/
cs/System.Private.CoreLib.resources.dll
de/
de/System.Private.CoreLib.resources.dll
... (727 files, all in same flat directory)
```
No subdirectory. Move all files into `$pkg_src_bin/`.
---
## Inspecting archives yourself
```sh
# tar.gz / tar.xz / tar.zst — list contents only (no extraction)
curl -fsSL "$URL" | tar -tz | head -20
# zip
curl -fsSL "$URL" -o /tmp/pkg.zip
unzip -l /tmp/pkg.zip | head -20
# For a .zst file when tar doesn't support zstd natively:
curl -fsSL "$URL" -o /tmp/pkg.tar.zst && zstd -dc /tmp/pkg.tar.zst | tar -tz | head -20
```
**What to look for**:
1. Is there a top-level directory? (Pattern B/C/D/E/G) or no directory? (Pattern A/F/H)
2. What is the directory named? Does it contain version? triplet?
3. Are there `completions/`, `autocomplete/`, `complete/` subdirs? (Pattern C)
4. Are there `.so`/`.dylib`/`.dll` files? (Pattern D or H)
5. Does the binary name match the command you want on PATH? (Pattern F if not)
6. Is there a `bin/` directory at the top level? (Pattern E or G)

View File

@@ -1,183 +0,0 @@
# Classification Reference
When to flag classification issues, what the webi classifier does automatically,
and what needs manual annotation.
---
## What the classifier handles automatically
The webi classifier (`internal/classify/classify.go`) parses asset filenames
using regex patterns and produces canonical `os`, `arch`, `libc`, and `ext`
values. It handles the vast majority of packages with no configuration needed.
### OS recognition
Filenames containing these terms are classified automatically:
- `darwin`, `macos`, `osx`, `apple``macos` in legacy cache
- `linux``linux`
- `windows`, `win`, `win32`, `win64``windows`
- `freebsd`, `openbsd`, `netbsd`, `dragonfly` → respective values
- `.deb`, `.rpm`, `.snap``linux` (but dropped from legacy cache)
- `.dmg`, `.app.zip``macos`
### Arch recognition
Filenames containing these terms are classified automatically:
- `x86_64`, `amd64`, `64bit`, `x64``amd64`
- `aarch64`, `arm64``arm64`
- `armv7`, `armv7l`, `armhf`, `gnueabihf``armv7l`
- `armv6`, `armv6l``armv6l`
- `i386`, `i686`, `386`, `x86``x86`
- `universal`, `universal2``amd64` (fat binary; arm64 falls back to this)
### Format recognition
- `.tar.gz`, `.tar.xz`, `.tar.zst`, `.tar.bz2`, `.zip`, `.7z` → compressed archive
- `.pkg`, `.msi`, `.dmg` → platform installer
- `.exe` → either bare binary or GUI installer (see below)
- No extension in filename → bare binary (ext = `exe` in cache)
### Automatically dropped
These asset types are recognised and excluded without any configuration:
- Checksums: `*.sha256`, `*.sha512`, `*.md5`, `*.sha256sum`
- Signatures: `*.asc`, `*.sig`, `*.cosign`, `*.sbom`
- Source archives: files with `source`, `src` in the name but no OS
- Package formats not supported by the Node installer: `.deb`, `.rpm`, `.snap`,
`.AppImage`, `.apk`
---
## When you need to add configuration
### Variant assets
A **variant** is a secondary build that serves the same OS/arch as a baseline
build but requires different hardware or runtime support. The Node.js installer
can't choose between variants — it only knows OS, arch, and libc. Variants
must be tagged and then excluded at export time.
**Common variants and how to identify them**:
| Variant | Filename pattern | Notes |
|---------|-----------------|-------|
| CUDA (GPU) | `*-cuda*`, `*cuda12*` | NVIDIA GPU support |
| ROCm (GPU) | `*-rocm*` | AMD GPU support |
| Vulkan | `*-vulkan*` | Cross-vendor GPU |
| AppImage | `*.AppImage` | Linux sandboxed app |
| .NET fxdependent | `*-fxdependent*` | Requires .NET runtime |
| Windows installer | `*Setup.exe`, `*Install.exe` | GUI installer, not the binary |
**Rule**: if there are multiple assets for the same OS/arch combination and
they serve the same users differently, they need variant tags. The baseline
(most widely compatible) build should be kept; variants should be tagged and
excluded.
**Example**: ollama publishes for linux/amd64:
- `ollama-linux-amd64.tar.zst` — baseline (CPU + any GPU auto-detected)
- `ollama-linux-amd64-rocm.tar.zst` — ROCm variant
- `ollama-linux-amd64-jetpack6.tar.zst` — NVIDIA Jetson variant
Only the baseline is useful via webi. The ROCm and Jetpack builds should be
tagged as variants and excluded.
---
### Windows .exe: bare binary vs GUI installer
`.exe` assets are ambiguous — they could be:
1. A bare binary (the tool itself, run from command line)
2. A GUI installer (runs a setup wizard, not useful for webi)
**How to tell**:
- GUI installer: filename contains `Setup`, `Install`, `Installer`, `inno`, `nsis`
- GUI installer: the tool also has a `.zip` or `.tar.gz` for Windows
- Bare binary: filename matches the tool name with minimal decoration
**When you see both**, the `.zip`/archive build is what webi uses. The `.exe`
installer should be tagged as a variant (`installer`) so it's excluded.
**When there's only a `.exe`** (no archive), it's probably the bare binary.
Test by downloading and running it — a bare binary runs immediately.
---
### Packages with no OS/arch in filenames
Some packages (rare) release with minimal filename decoration. Examples:
- `tool-v1.2.3.tar.gz` — no OS, no arch
- `tool.tar.gz` — version not even in filename
These are usually source archives (not compiled binaries) and should be
dropped entirely from the release list. If they are compiled binaries for a
specific OS, the releases.js config needs an `asset_filter` key to match the
right file, plus OS/arch metadata added.
---
### Non-standard OS naming in filenames
A few upstreams use unusual OS names:
- `sunos` — should map to `solaris` (the webi classifier does this)
- `osx` or `macosx` — recognised as `macos`
- `apple-darwin` (Rust triplet) — recognised as `macos`
If a package uses a genuinely unknown OS string, the classifier will produce
`os = ""` for that asset. Those entries are dropped from the legacy cache.
---
### Asset filter configuration
If GitHub releases for a package include multiple builds that would otherwise
collide (e.g. `extended` vs non-extended for hugo, or specific project builds
in a monorepo), add to the package's `releases.conf`:
```ini
# Only include assets containing "extended" in the name
asset_filter = extended
# Exclude assets containing "legacy" in the name
asset_exclude = legacy
```
These filters run before classification.
---
## Quick checklist when inspecting a new package
1. **Look at the latest 23 releases** on GitHub. Note all asset filenames.
2. **Find the "standard" builds** — the ones a normal user would download for
their OS. Usually there are ≤4 per OS (amd64, arm64, x86, armv7l).
3. **Check for extras**:
- Are there GPU-specific builds for the same OS/arch? → variant
- Are there `.exe` installer files alongside a `.zip`? → variant
- Are there `.deb`/`.rpm`/`.AppImage`? → auto-dropped, no action needed
- Does the Windows build have no archive and only a bare `.exe`? → fine
4. **Check OS/arch naming** — does the filename use standard terms, or
something unusual that might confuse the classifier?
5. **Check format changes** — do old releases use a different archive type
or directory layout than recent ones? The install script may need to
handle both.
---
## Canonical vocabulary reference
All cache output must use exactly these values.
**OS**: `macos`, `linux`, `windows`, `freebsd`, `openbsd`, `netbsd`,
`dragonfly`, `aix`, `illumos`, `plan9`, `solaris`
**Arch**:
- `amd64` (not `x86_64`)
- `arm64` (not `aarch64`)
- `armv7l` (not `armv7` — the `l` stands for little-endian; `uname -m` reports `armv7l`)
- `armv6l` (not `armv6`)
- `x86` (not `i386`, `i686`, `386`)
- `mipsle` (not `mipsel`)
- `mips64le` (not `mips64el`)
- Other: `arm`, `ppc64le`, `ppc64`, `loong64`, `riscv64`, `s390x`, `mips`, `mips64`
**Libc**: `none` (static/Go/Zig — never empty), `gnu`, `musl`, `msvc`
**Ext**: `tar.gz`, `tar.xz`, `zip`, `exe`, `7z`, `pkg`, `msi`
(no leading dot; `exe` for bare binaries with no file extension)

View File

@@ -1,388 +0,0 @@
# Install Patterns Reference
Nine patterns cover the full range of webi packages. Pattern A is by far
the most common. Check `tar -tz $ARCHIVE` before writing any code.
---
## Pattern A — Bare binary at archive root
The archive extracts directly to the current directory with no wrapper
subdirectory. Binary (and optional LICENSE/README) is at the top level.
**Set `WEBI_SINGLE=true`** — tells the framework to link the binary file
directly (`~/.local/bin/cmd → ~/.local/opt/cmd-vX/bin/cmd`) rather than
linking the versioned directory.
Representative packages: caddy, fzf, k9s, terraform, sttr, lf, monorel,
awless, bun, cilium, curlie, dashmsg, dotenv, dotenv-linter, ffuf,
gitdeploy, gprox, grype, hugo, keypairs, koji, ots, runzip, sclient,
sqlc, sqlpkg, uuidv7, xcaddy, deno
**install.sh**:
```sh
pkg_cmd_name="caddy"
WEBI_SINGLE=true
pkg_dst_cmd="$HOME/.local/bin/caddy"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/caddy-v$WEBI_VERSION/bin/caddy"
pkg_src_dir="$HOME/.local/opt/caddy-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./"$pkg_cmd_name"* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
pkg_get_current_version() {
caddy version 2>/dev/null | head -n 1 | cut -d' ' -f1 | sed 's:^v::'
}
```
**install.ps1** key lines:
```powershell
# No subdirectory — binary is at the top level of the archive
Move-Item -Path ".\caddy.exe" -Destination "$pkg_src_bin"
```
---
## Pattern B — Binary inside a version/triplet subdirectory
Archive extracts to a single directory named with the version and/or
platform triplet. Binary (and docs) live inside that directory.
Representative packages: delta, hexyl, shellcheck, trip, xsv, kubectx, kubens
**Subdirectory naming conventions seen in the wild**:
- `tool-{ver}-{triplet}/` — most Rust tools (delta, shellcheck, xsv)
- `tool-{ver}/` — simpler version-only dirs
- flat (no dir) — kubectx/kubens use flat archives despite being "B-ish"
**install.sh**:
```sh
pkg_cmd_name="delta"
# WEBI_SINGLE not set (or false)
pkg_dst_cmd="$HOME/.local/bin/delta"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/delta-v$WEBI_VERSION/bin/delta"
pkg_src_dir="$HOME/.local/opt/delta-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./delta-*/delta "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
pkg_get_current_version() {
delta --version 2>/dev/null | head -n 1 | cut -d' ' -f2
}
```
**install.ps1** key lines:
```powershell
Move-Item -Path ".\delta-*\delta.exe" -Destination "$pkg_src_bin"
```
---
## Pattern C — Subdirectory with binary + completions and/or man pages
Same as B but the archive also contains shell completions and/or man pages
worth installing.
Representative packages: bat, fd, lsd, rg/ripgrep, sd, watchexec, zoxide
Note: goreleaser has a flat archive (Pattern A layout) but with completions at
the archive root. See the goreleaser entry in ARCHIVE-LAYOUTS.md.
**Completion directory name varies by package**:
- `completions/` — sd, watchexec, zoxide
- `autocomplete/` — bat, fd, lsd
- `complete/` — rg/ripgrep
**Completion filename conventions**:
- Bash: `tool.bash`, `tool.bash-completion`, `_tool.bash`
- Fish: `tool.fish`
- Zsh: `_tool`
- PowerShell: `_tool.ps1`, `tool.ps1`
**Man page location varies**:
- `tool.1` at subdirectory root — sd, bat, fd, lsd
- `doc/tool.1` — rg/ripgrep
- `man/man1/tool.1` — zoxide (deepest path)
**install.sh** (rg as example):
```sh
pkg_cmd_name="rg"
pkg_dst_cmd="$HOME/.local/bin/rg"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/rg-v$WEBI_VERSION/bin/rg"
pkg_src_dir="$HOME/.local/opt/rg-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
mv ./ripgrep-*/rg "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
# bash completion
if test -e ./ripgrep-*/complete/rg.bash; then
mkdir -p "$pkg_src_dir/share/bash-completion/completions"
mv ./ripgrep-*/complete/rg.bash \
"$pkg_src_dir/share/bash-completion/completions/rg"
fi
# fish completion
if test -e ./ripgrep-*/complete/rg.fish; then
mkdir -p "$pkg_src_dir/share/fish/vendor_completions.d"
mv ./ripgrep-*/complete/rg.fish \
"$pkg_src_dir/share/fish/vendor_completions.d/rg.fish"
fi
# zsh completion
if test -e ./ripgrep-*/complete/_rg; then
mkdir -p "$pkg_src_dir/share/zsh/site-functions"
mv ./ripgrep-*/complete/_rg \
"$pkg_src_dir/share/zsh/site-functions/_rg"
fi
# man page
if test -e ./ripgrep-*/doc/rg.1; then
mkdir -p "$pkg_src_dir/share/man/man1"
mv ./ripgrep-*/doc/rg.1 "$pkg_src_dir/share/man/man1/rg.1"
fi
}
pkg_get_current_version() {
rg --version 2>/dev/null | head -n 1 | cut -d' ' -f2
}
```
**Note**: Completion paths in completions/man install are best-effort
— use `if test -e ...` guards so the script still works on older releases
that didn't include them.
---
## Pattern D — Binary + shared libraries
The package bundles shared libraries alongside the binary. The entire
directory tree must be preserved.
Representative packages: ollama (Linux), psql/postgres, sass (Dart VM),
syncthing, xz
**install.sh**:
```sh
pkg_cmd_name="ollama"
pkg_dst_cmd="$HOME/.local/bin/ollama"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/ollama-v$WEBI_VERSION/bin/ollama"
pkg_src_dir="$HOME/.local/opt/ollama-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
# Archive already has bin/ and lib/ layout
mv ./bin "$pkg_src_dir/bin"
mv ./lib "$pkg_src_dir/lib"
}
```
For psql (archive has a `psql-{ver}-{triplet}/` wrapper dir):
```sh
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./psql-*/ "$pkg_src_dir"
}
```
---
## Pattern E — FHS-like layout
Archive already follows `bin/`, `share/man/`, `share/doc/` hierarchy.
Extract the whole thing directly into the versioned opt directory.
Representative packages: gh (GitHub CLI), pandoc
**install.sh**:
```sh
pkg_cmd_name="gh"
pkg_dst_cmd="$HOME/.local/bin/gh"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/gh-v$WEBI_VERSION/bin/gh"
pkg_src_dir="$HOME/.local/opt/gh-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./gh_*/ "$pkg_src_dir"
}
pkg_get_current_version() {
gh --version 2>/dev/null | head -n 1 | cut -d' ' -f3
}
```
No `chmod` needed — binary is already executable inside the archive.
---
## Pattern F — Binary needs rename
Binary in the archive doesn't match the expected command name.
Representative packages: pathman (`pathman-v0.6.0-linux-amd64_v1``pathman`),
yq (`yq_linux_amd64``yq`)
**install.sh**:
```sh
pkg_cmd_name="yq"
WEBI_SINGLE=true
pkg_dst_cmd="$HOME/.local/bin/yq"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/yq-v$WEBI_VERSION/bin/yq"
pkg_src_dir="$HOME/.local/opt/yq-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$pkg_src_bin"
# Binary is named yq_linux_amd64 (or yq_darwin_amd64 etc)
mv ./yq_* "$pkg_src_cmd"
chmod a+x "$pkg_src_cmd"
}
```
---
## Pattern G — Full SDK / toolchain
Archive contains a complete runtime or SDK (hundreds to thousands of files).
The entire tree goes into opt; multiple binaries are linked from `bin/`.
Representative packages: go, node, zig, flutter, julia, cmake, tinygo
**install.sh** (node as example):
```sh
pkg_cmd_name="node"
# NOTE: pkg_src points to the directory, not a binary
pkg_dst_cmd="$HOME/.local/bin/node"
pkg_dst="$HOME/.local/opt/node" # versioned-dir symlink target
pkg_src_cmd="$HOME/.local/opt/node-v$WEBI_VERSION/bin/node"
pkg_src_dir="$HOME/.local/opt/node-v$WEBI_VERSION"
pkg_src="$pkg_src_dir" # pkg_src = the directory
pkg_install() {
mkdir -p "$(dirname "$pkg_src")"
mv ./node-*/ "$pkg_src"
}
pkg_link() {
rm -f "$pkg_dst"
ln -s "$pkg_src" "$pkg_dst"
}
pkg_get_current_version() {
node --version 2>/dev/null | head -n 1 | sed 's:^v::'
}
```
---
## Pattern H — .NET runtime bundle
Flat directory with one binary and hundreds of `.dll` files. The entire
directory must be preserved. Like Pattern G (SDK) in structure — the
versioned directory is the package root, with the binary directly inside
(no `bin/` subdirectory). A `pkg_link()` creates the unversioned symlink.
Representative packages: pwsh (PowerShell Core)
**install.sh**:
```sh
pkg_cmd_name="pwsh"
# note: binary is at pkg_src_dir root, no bin/ subdirectory
pkg_src_cmd="$HOME/.local/opt/pwsh-v$WEBI_VERSION/pwsh"
pkg_src_dir="$HOME/.local/opt/pwsh-v$WEBI_VERSION"
pkg_src="$pkg_src_dir"
pkg_dst_cmd="$HOME/.local/opt/pwsh/pwsh"
pkg_dst="$HOME/.local/opt/pwsh"
pkg_install() {
# Archive extracts flat — move all contents into the versioned dir
mkdir -p "$pkg_src_dir"
mv ./* "$pkg_src_dir"
chmod a+x "$pkg_src_cmd"
}
pkg_link() {
rm -rf "$pkg_dst"
ln -s "$pkg_src" "$pkg_dst"
}
```
---
## Pattern I — Multi-binary distribution
Archive contains multiple related binaries. Install the primary one and
link only that.
Representative packages: dashcore (dashd + dash-cli + dash-qt + ...),
mutagen (mutagen + mutagen-agents.tar.gz)
**install.sh** (dashcore-style):
```sh
pkg_cmd_name="dashd"
pkg_dst_cmd="$HOME/.local/bin/dashd"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/dashcore-v$WEBI_VERSION/bin/dashd"
pkg_src_dir="$HOME/.local/opt/dashcore-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
pkg_install() {
mkdir -p "$(dirname "$pkg_src_dir")"
mv ./dashcore-*/ "$pkg_src_dir"
}
```
---
## Choosing between patterns
```
Archive root contains a single binary (or binary + docs)?
→ Pattern A (set WEBI_SINGLE=true)
Archive has a named subdirectory wrapping the binary?
├─ Binary only inside subdir? → Pattern B
├─ Binary + completions/man pages? → Pattern C
└─ Binary + shared libraries (.so)? → Pattern D
Archive already has bin/ and share/ layout?
→ Pattern E
Binary name doesn't match the command name?
→ Pattern F (rename during install)
Archive is a full SDK (compiler, runtime, stdlib)?
→ Pattern G (pkg_src = pkg_src_dir)
Flat directory with many DLLs (.NET)?
→ Pattern H
Multiple binaries for a single distributed system?
→ Pattern I
```

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env pwsh
if (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
New-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start" -ItemType Directory -Force | Out-Null
}
Remove-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start\example" -Recurse -ErrorAction Ignore

View File

@@ -1,73 +0,0 @@
'use strict';
let Path = require('node:path');
let BuildsCacher = require('./builds-cacher.js');
// let Parallel = require('./parallel.js');
var INSTALLERS_DIR = Path.join(__dirname, '..');
var CACHE_DIR = Path.join(__dirname, '../_cache');
async function main() {
let bc = BuildsCacher.create({
caches: CACHE_DIR,
installers: INSTALLERS_DIR,
});
bc.freshenRandomPackage(600 * 1000);
// let dirs = await bc.getProjectsByType();
// let projNames = Object.keys(dirs.valid);
let lastUpdate;
let projName = 'k9s';
{
let packages = await bc.getPackages({
//Releases: Releases,
name: projName,
date: new Date(),
});
lastUpdate = packages.updated;
console.info(
`Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
);
}
console.info('Waiting 5s');
{
setTimeout(async function () {
let packages = await bc.getPackages({
//Releases: Releases,
name: projName,
date: new Date(),
});
console.info(
`Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
);
if (lastUpdate < packages.updated) {
console.info(`PASS`);
} else {
console.info(`MAYBE fail`);
}
}, 5 * 1000);
}
//let parallel = 25;
//await Parallel.run(parallel, projNames, getAll);
//async function getAll(name) {
// void (await bc.getPackages({
// //Releases: Releases,
// name: name,
// date: new Date(),
// }));
//}
}
main()
.then(function () {
console.log('Done');
})
.catch(function (e) {
console.error(e.stack || e);
process.exit(1);
});

File diff suppressed because it is too large Load Diff

View File

@@ -1,41 +0,0 @@
'use strict';
let Builds = module.exports;
let Path = require('node:path');
let BuildsCacher = require('./builds-cacher.js');
// let HostTargets = require('./build-classifier/host-targets.js');
let Parallel = require('./parallel.js');
var INSTALLERS_DIR = Path.join(__dirname, '..');
var CACHE_DIR = Path.join(__dirname, '../_cache');
let bc = BuildsCacher.create({
caches: CACHE_DIR,
installers: INSTALLERS_DIR,
});
bc.freshenRandomPackage(600 * 1000);
Builds.init = async function () {
bc.freshenRandomPackage(600 * 1000);
let dirs = await bc.getProjectsByType();
let projNames = Object.keys(dirs.valid);
let parallel = 25;
await Parallel.run(parallel, projNames, getAll);
async function getAll(name) {
void (await bc.getPackages({
//Releases: Releases,
name: name,
date: new Date(),
}));
}
};
Builds.enumerateLatestVersions = bc.enumerateLatestVersions;
Builds.findMatchingPackages = bc.findMatchingPackages;
Builds.getPackage = bc.getPackages;
Builds.getProjectType = bc.getProjectType;
Builds.selectPackage = bc.selectPackage;

View File

@@ -1,90 +0,0 @@
'use strict';
let Path = require('node:path');
// let Builds = require('./builds.js');
let BuildsCacher = require('./builds-cacher.js');
let Triplet = require('./build-classifier/triplet.js');
async function main() {
let projName = process.argv[2];
if (!projName) {
console.error(``);
console.error(`USAGE`);
console.error(``);
console.error(` classify-one <project-name>`);
console.error(``);
console.error(`EXAMPLE`);
console.error(``);
console.error(` classify-one caddy`);
console.error(``);
return;
}
let tsDate = new Date(0);
let meta = {
// version info
versions: [],
lexvers: [],
lexversMap: {},
// culled release assets
packages: [],
releasesByTriplet: {},
// target info
triplets: [],
oses: [],
arches: [],
libcs: [],
formats: [],
// TODO channels: [],
};
let installersDir = Path.join(__dirname, '..');
let Releases = require(`${installersDir}/${projName}/releases.js`);
if (!Releases.latest) {
Releases.latest = Releases;
}
let projInfo = await Releases.latest();
// let packages = await Builds.getPackage({ name: projName });
// console.log(packages);
let bc = {};
bc.ALL_TERMS = Triplet.TERMS_PRIMARY_MAP;
bc.orphanTerms = Object.assign({}, bc.ALL_TERMS);
bc.unknownTerms = {};
bc.usedTerms = {};
bc.formats = [];
bc._targetsByBuildIdCache = {};
bc._triplets = {};
let transformed = BuildsCacher.transformAndUpdate(
projName,
projInfo,
meta,
tsDate,
bc,
);
console.log(`[DEBUG] transformed`);
let sample = transformed.packages.slice(0, 20);
console.log('packages:', sample, ':packages');
console.log(
'releasesByTriplet:',
transformed.releasesByTriplet['linux-x86_64-none'][transformed.versions[0]],
':releasesByTriplet',
);
console.log('versions:', transformed.versions, ':versions');
console.log('triplets:', transformed.triplets, ':triplets');
console.log('oses:', transformed.oses, ':oses');
console.log('arches:', transformed.arches, ':arches');
console.log('libcs:', transformed.libcs, ':libcs');
console.log('formats:', transformed.formats, ':formats');
console.log(Object.keys(transformed));
}
main().catch(function (err) {
console.error('Error:');
console.error(err);
});

View File

@@ -6,7 +6,7 @@
############################################################
New-Item -Path "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
Set-ExecutionPolicy -Scope Process Bypass
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" "{{ exename }}"

View File

@@ -267,23 +267,12 @@ webi_bootstrap() { (
fn_checksum() {
a_filepath="${1}"
if command -v sha1sum > /dev/null; then
sha1sum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
return 0
fi
cmd_shasum='sha1sum'
if command -v shasum > /dev/null; then
shasum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
return 0
cmd_shasum='shasum'
fi
if command -v sha1 > /dev/null; then
sha1 "${a_filepath}" | cut -d'=' -f2 | cut -c 2-9
return 0
fi
echo >&2 " warn: no sha1 sum program"
date '+%F %H:%M'
$cmd_shasum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
}
##############################################

View File

@@ -93,18 +93,16 @@ Installers.renderBash = async function (
['WEBI_GIT_TAG', rel.git_tag], // TODO replace with branch
['WEBI_LTS', rel.lts],
['WEBI_CHANNEL', rel.channel],
['WEBI_EXT', rel.ext],
['WEBI_EXT', rel.ext.replace(/tar.*/, 'tar')],
['WEBI_FORMATS', formats.join(',')],
['WEBI_PKG_URL', rel.download],
['WEBI_PKG_PATHNAME', pkgFile],
['WEBI_PKG_FILE', pkgFile], // TODO replace with pathname
['PKG_NAME', pkg],
['PKG_STABLE', rel.stable],
['PKG_LATEST', rel.latest],
['PKG_OSES', (rel.oses || []).join(' ')],
['PKG_ARCHES', (rel.arches || []).join(' ')],
['PKG_LIBCS', (rel.libcs || []).join(' ')],
['PKG_FORMATS', (rel.formats || []).join(' ')],
['PKG_OSES', rel.oses],
['PKG_ARCHES', rel.arches],
['PKG_LIBCS', rel.libcs],
['PKG_FORMATS', (rel.formats || []).join(',')],
];
for (let env of envReplacements) {

View File

@@ -124,20 +124,184 @@ let bc = BuildsCacher.create({
installers: INSTALLERS_DIR,
});
async function getPackagesWithBuilds(installersDir, pkgNames, parallel = 25) {
let packages = [];
await Parallel.run(parallel, pkgNames, getAll);
async function getAll(name, i) {
let Releases = require(`${installersDir}/${name}/releases.js`);
let pkg = await bc.getBuilds({
Releases: Releases,
name: name,
date: new Date(),
});
packages[i] = pkg;
}
return packages;
}
function getBuildsByTarget(packages) {
let packagesByName = {};
for (let pkg of packages) {
let buildsByOs = getBuildsByOs(pkg);
packagesByName[pkg.name] = buildsByOs;
}
return packagesByName;
}
function getBuildsByOs(pkg) {
let buildsByOs = {};
for (let build of pkg.releases) {
// TODO check targets cache
let target = bc.classify(pkg, build);
if (!target) {
// ignore known, non-package extensions
continue;
}
if (target.error) {
let err = target.error;
let code = err.code || '';
console.error(`[ERROR]: ${code} ${pkg.name}: ${build.name}`);
console.error(`>>> ${err.message} <<<`);
console.error(pkg);
console.error(build);
console.error(`^^^ ${err.message} ^^^`);
console.error(err.stack);
continue;
}
let buildsByRelease = getBuildsByRelease(build, buildsByOs, target);
buildsByRelease.push(build);
}
return buildsByOs;
}
function getBuildsByRelease(build, buildsByOs, target) {
let archLibc = `${target.arch}-${target.libc}`;
if (!buildsByOs[target.os]) {
buildsByOs[target.os] = {};
}
let buildsByVersion = buildsByOs[target.os];
if (!buildsByVersion[build.version]) {
buildsByVersion[build.version] = {};
}
let buildsByArchLibc = buildsByVersion[build.version];
if (!buildsByArchLibc[archLibc]) {
buildsByArchLibc[archLibc] = [];
}
let buildsByRelease = buildsByArchLibc[archLibc];
return buildsByRelease;
}
function matchBuildsByTarget(pkg, buildsTree, target) {
let oses = [];
let targetOs = target.os;
if (target.os === 'windows') {
oses = ['ANYOS', 'windows'];
//buildsByOs = buildsTree.ANYOS || buildsTree[target.os];
} else if (target.os === 'android') {
oses = ['ANYOS', 'posix_2017', 'android', 'linux'];
// buildsByOs =
// buildsTree.ANYOS || buildsTree.posix_2017 || buildsTree[target.os];
// if (!buildsByOs) {
// targetOs = 'linux';
// buildsByOs = buildsTree.linux;
// }
} else {
oses = ['ANYOS', 'posix_2017', target.os];
// buildsByOs =
// buildsTree.ANYOS || buildsTree.posix_2017 || buildsTree[target.os];
}
// TODO can we move sortByOsAndArchLibc(builds, anything) down to the lib?
// and then the matcher
// and make the waterfall more optional?
let waterfall = HostTargets.WATERFALL[target.os] || {};
let arches = waterfall[target.arch] ||
HostTargets.WATERFALL.ANYOS[target.arch] || [target.arch];
arches = ['ANYARCH'].concat(arches);
let libcs = waterfall[target.libc] ||
HostTargets.WATERFALL.ANYOS[target.libc] || [target.libc];
//console.log('waterfalls', arches, libcs);
// TODO flatten earlier and precache?
let duplets = [];
for (let arch of arches) {
for (let libc of libcs) {
let duplet = `${arch}-${libc}`;
duplets.push(duplet);
}
}
let duplet;
let targetBuilds;
for (let os of oses) {
let buildsByOs = buildsTree[os];
if (!buildsByOs) {
continue;
}
// TODO
// - latest supported triplets
// - historical supported triplets
// TODO sort versions first, get channel (or 'stable' or 'latest') from user
for (let version of pkg.versions) {
let versionBuilds = buildsByOs[version];
if (!versionBuilds) {
continue;
}
for (let _duplet of duplets) {
targetBuilds = versionBuilds[_duplet];
//console.log(` duplet: ${_duplet}`, versionBuilds, targetBuilds);
if (targetBuilds?.length > 0) {
targetOs = os;
duplet = _duplet;
break;
}
}
if (targetBuilds?.length > 0) {
break;
}
}
if (targetBuilds?.length > 0) {
break;
}
}
if (!targetBuilds?.length) {
// console.log(' no builds:', buildsByOs);
targetBuilds = [];
}
let match = { triplet: `${targetOs}-${duplet}`, builds: targetBuilds };
return match;
}
async function main() {
/* jshint maxcomplexity: 25 */
// TODO
// node ./_webi/lint-builds.js caddy@beta 'x86_64/unknown Darwin libc'
//
//let [projName, userAgent] = process.argv.slice(2);
let projName = process.argv[2];
// create test case for zoxide, goreleaser, go, yq, caddy, rg
// let [pkgName, userAgent] = process.argv[2].slice(0);
// create test case for zoxide, caddy, rg
let dirs = await bc.getProjectsByType();
if (!projName) {
showDirs(dirs);
console.info('');
}
let dirs = await bc.getPackages();
showDirs(dirs);
console.info('');
bc.freshenRandomPackage(600 * 1000);
@@ -145,106 +309,86 @@ async function main() {
let triples = [];
let valids = Object.keys(dirs.valid);
if (projName) {
if (!valids.includes(projName)) {
throw new Error(`'${projName}' is not a valid installable project`);
}
valids = [projName];
let index = valids.indexOf('webi');
if (index >= 0) {
// TODO fix the webi faux package
// (not sure why I even created it)
void valids.splice(index, 1);
}
let parallel = 25;
//valids = ['atomicparsley', 'caddy', 'macos'];
//valids = ['atomicparsley'];
let packages = await getPackagesWithBuilds(INSTALLERS_DIR, valids, parallel);
console.info('');
console.info(`Fetching project release assets`);
let parallel = 25;
let projects = [];
await Parallel.run(parallel, valids, getAll);
async function getAll(name, i) {
console.info(` ${name}`);
let projInfo = await bc.getPackages({
//Releases: Releases,
name: name,
date: new Date(),
});
projects[i] = projInfo;
}
console.info(`Fetching builds for`);
for (let pkg of packages) {
console.info(` ${pkg.name}`);
console.info(`Classifying build assets for...`);
for (let projInfo of projects) {
console.info(` ${projInfo.name}`);
let nStr = projInfo.releases.length.toString();
let nStr = pkg.releases.length.toString();
let n = nStr.padStart(5, ' ');
let row = `##### ${n}\t${projInfo.name}\tv`;
let row = `##### ${n}\t${pkg.name}\tv`;
rows.push(row);
// ignore known, non-package extensions
for (let build of projInfo.releases) {
let target = bc.classify(projInfo, build);
for (let build of pkg.releases) {
let target = bc.classify(pkg, build);
if (!target) {
// non-build file
continue;
}
if (target.error) {
let e = target.error;
if (e.code === 'E_BUILD_NO_PATTERN') {
console.warn(`>>> ${e.message} <<<`);
console.warn(projInfo);
console.warn(pkg);
console.warn(build);
console.warn(`^^^ ${e.message} ^^^`);
}
throw e;
}
if (target.unknownTerms?.length) {
let msg = `${projInfo.name}: unrecognized term(s) '${target.unknownTerms}' in '${build.download}'`;
let err = new Error(msg);
throw err;
}
triples.push(target.triplet);
// if (!build.version) {
// throw new Error(`no version for ${pkg.name} ${build.name}`);
// }
// // For debug printing versions
// console.error(build.version);
rows.push(`${target.triplet}\t${projInfo.name}\t${build.version}`);
rows.push(`${target.triplet}\t${pkg.name}\t${build.version}`);
}
}
console.info(`Fetching builds for`);
for (let projInfo of projects) {
console.info('');
console.info('');
console.info(` ${projInfo.name}`);
let packagesTree = getBuildsByTarget(packages);
//console.log(`packagesTree`, packagesTree);
for (let pkg of packages) {
console.log('');
console.log('');
console.log('pkg', pkg.name);
let buildsTree = packagesTree[pkg.name];
console.log(buildsTree);
for (let target of uaTargets) {
let libc = target.libc || 'libc';
let hostTriplet = `${target.os}-${target.arch}-${libc}`;
console.info('');
console.info(` target: ${hostTriplet}`);
let match = bc.findMatchingPackages(projInfo, target, {
ver: '',
});
console.log('');
console.log(`target: ${hostTriplet}`);
let match = matchBuildsByTarget(pkg, buildsTree, target);
if (!match) {
console.info(
` project: ${projInfo.name}: missing build for os '${target.os}'`,
console.log(
` pkg: ${pkg.name}: missing build for os '${target.os}'`,
);
continue;
}
if (!match.releases) {
console.info(
` project: ${projInfo.name}: missing build for os '${target.os}-${target.arch}-${libc}'`,
if (match.builds.length === 0) {
console.log(
` pkg: ${pkg.name}: missing build for os '${target.os}-${target.arch}-${libc}'`,
);
} else if (match.triplet === hostTriplet) {
let releaseNames = Object.keys(match.releases);
console.info(` selected ${releaseNames.length}`);
console.log(` selected ${match.builds.length}`);
} else {
let releaseNames = Object.keys(match.releases);
console.info(
` selected ${releaseNames.length} (${match.triplet} fallback)`,
console.log(
` selected ${match.builds.length} (${match.triplet} fallback)`,
);
}
}
@@ -286,18 +430,6 @@ async function main() {
}
console.info('');
console.info('Formats:');
if (bc.formats.length) {
let formats = bc.formats.slice();
formats.sort();
if (!formats[0]) {
formats[0] = '(bin)';
}
console.warn(' ', formats.join('\n '));
} else {
console.info(' (none)');
}
// sort -u -k1 builds.tsv | rg -v '^#|^https?:' | rg -i arm
// cut -f1 builds.tsv | sort -u -k1 | rg -v '^#|^https?:' | rg -i arm
}

View File

@@ -57,7 +57,7 @@ var archMap = {
amd64:
/(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)64([_\-]?bit)?(\b|_)/i,
//x86: /(86)(\b|_)/i,
x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32|i?386)([_\-]?bit)?(\b|_)/i,
x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32)([_\-]?bit)(\b|_)/i,
ppc64le: /(\b|_)(ppc64le)/i,
ppc64: /(\b|_)(ppc64)(\b|_)/i,
s390x: /(\b|_)(s390x)/i,
@@ -211,8 +211,7 @@ function normalize(all) {
// won't match:
// - v1.0beta
// - v1.0-beta1b
let isBetaRe =
/(\b|_)(alpha|beta|dev|developer|prev|preview|rc)(\d+)(\b|_)/;
let isBetaRe = /(\b|_)(preview|rc|beta|alpha)(\d+)(\b|_)/;
let isBeta = isBetaRe.test(rel.name);
if (isBeta) {
rel.channel = 'beta';

View File

@@ -45,15 +45,15 @@ $TDim = "${Esc}[2m"
$TReset = "${Esc}[0m"
function Invoke-DownloadUrl {
param (
Param (
[string]$URL,
[string]$Params,
[string]$Path,
[switch]$Force
)
if (Test-Path -Path "$Path") {
if (-not $Force.IsPresent) {
IF (Test-Path -Path "$Path") {
IF (-Not $Force.IsPresent) {
Write-Host " ${TDim}Found${TReset} $Path"
return
}
@@ -65,7 +65,7 @@ function Invoke-DownloadUrl {
Write-Host " Downloading ${TDim}from${TReset}"
Write-Host " ${TDim}${URL}${TReset}"
if ($Params.Length -ne 0) {
IF ($Params.Length -ne 0) {
Write-Host " ?$Params"
$URL = "${URL}?${Params}"
}
@@ -80,18 +80,18 @@ function Get-UserAgent {
# This is the canonical CPU arch when the process is emulated
$my_arch = "$Env:PROCESSOR_ARCHITEW6432"
if ($my_arch -eq $null -or $my_arch -eq "") {
IF ($my_arch -eq $null -or $my_arch -eq "") {
# This is the canonical CPU arch when the process is native
$my_arch = "$Env:PROCESSOR_ARCHITECTURE"
}
if ($my_arch -eq "AMD64") {
IF ($my_arch -eq "AMD64") {
# Because PowerShell is sometimes AMD64 on Windows 10 ARM
# See https://oofhours.com/2020/02/04/powershell-on-windows-10-arm64/
$my_os_arch = (Get-CimInstance -ClassName Win32_OperatingSystem).OSArchitecture
$my_os_arch = wmic os get osarchitecture
# Using -clike because of the trailing newline
if ($my_os_arch -clike "ARM 64*") {
IF ($my_os_arch -clike "ARM 64*") {
$my_arch = "ARM64"
}
}
@@ -124,7 +124,7 @@ function webi_path_add($pathname) {
$exists_in_path = $true
}
}
if (-not $exists_in_path) {
if (-Not $exists_in_path) {
$all_user_paths = "${pathname};${all_user_paths}".Trim(';')
[Environment]::SetEnvironmentVariable("Path", $all_user_paths, "User")
$null = Sync-EnvPath

View File

@@ -27,7 +27,6 @@ __bootstrap_webi() {
#PKG_LIBCS=
#PKG_FORMATS=
#PKG_LATEST=
#PKG_STABLE=
WEBI_PKG_DOWNLOAD=""
WEBI_DOWNLOAD_DIR="${HOME}/Downloads"
if command -v xdg-user-dir > /dev/null; then
@@ -37,7 +36,7 @@ __bootstrap_webi() {
fi
fi
WEBI_PKG_PATH="${WEBI_DOWNLOAD_DIR}/webi/${PKG_NAME:-error}/${WEBI_VERSION:-stable}"
WEBI_PKG_PATH="${WEBI_DOWNLOAD_DIR}/webi/${PKG_NAME:-error}/${WEBI_VERSION:-latest}"
# get the special formatted version
# (i.e. "go is go1.14" while node is "node v12.10.8")
@@ -128,10 +127,6 @@ __bootstrap_webi() {
echo ""
echo " $(t_err "Error: no '${PKG_NAME:-"Unknown Package"}@${WEBI_TAG:-"Unknown Tag"}' release for '${WEBI_OS:-"Unknown OS"}' (${WEBI_LIBC:-"Unknown Libc"}) on '${WEBI_ARCH:-"Unknown CPU"}' as one of '${WEBI_FORMATS:-"Unknown File Type"}'")"
echo ""
echo " Latest Stable: ${PKG_STABLE}"
if test "${PKG_LATEST}" != "${PKG_STABLE}"; then
echo " Next Version: ${PKG_LATEST}"
fi
echo " CPUs: $PKG_ARCHES"
echo " OSes: $PKG_OSES"
echo " libcs: $PKG_LIBCS"
@@ -201,32 +196,20 @@ __bootstrap_webi() {
my_dl_rel="$(
fn_sub_home "${WEBI_PKG_PATH}/${WEBI_PKG_FILE}"
)"
if test "$WEBI_EXT" = "tar.zst"; then
echo " Extracting $(t_path "${my_dl_rel}")"
unzstd -c --keep "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" | tar xf -
elif test "$WEBI_EXT" = "tar.xz"; then
echo " Extracting $(t_path "${my_dl_rel}")"
unxz -c -k "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" | tar xf -
elif test "$WEBI_EXT" = "tar.gz"; then
echo " Extracting $(t_path "${my_dl_rel}")"
tar xzf "${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
elif test "$WEBI_EXT" = "tar.bz2"; then
echo " Extracting $(t_path "${my_dl_rel}")"
tar xjf "${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
elif test "$WEBI_EXT" = "tar"; then
if [ "tar" = "$WEBI_EXT" ]; then
echo " Extracting $(t_path "${my_dl_rel}")"
tar xf "${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
elif test "$WEBI_EXT" = "zip" || test "$WEBI_EXT" = "app.zip"; then
elif [ "zip" = "$WEBI_EXT" ]; then
echo " Extracting $(t_path "${my_dl_rel}")"
unzip "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > __unzip__.log
elif test "$WEBI_EXT" = "exe"; then
elif [ "exe" = "$WEBI_EXT" ]; then
echo " Moving $(t_path "${my_dl_rel}")"
echo " to $(t_path "$(fn_sub_home "$(pwd)")")"
mv "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" .
elif test "$WEBI_EXT" = "git"; then
elif [ "git" = "$WEBI_EXT" ]; then
echo " Moving $(t_path "${my_dl_rel}")"
mv "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" .
elif test "$WEBI_EXT" = "xz"; then
elif [ "xz" = "$WEBI_EXT" ]; then
echo " Inflating $(t_path "${my_dl_rel}")"
unxz -c "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > "$(basename "$WEBI_PKG_FILE")"
else
@@ -239,6 +222,8 @@ __bootstrap_webi() {
webi_path_add() {
my_path="${1}"
fn_envman_init
# \v was chosen as it is extremely unlikely for a filename
# \1 could be an even better choice, but needs more testing.
# (currently tested working on: linux & mac)
@@ -401,6 +386,7 @@ __bootstrap_webi() {
export _webi_tmp="${_webi_tmp:-"$HOME/.local/opt/webi-tmp.d"}"
mkdir -p "${WEBI_PKG_PATH}"
mkdir -p "$HOME/.local/bin"
mkdir -p "$HOME/.local/opt"
if test -e ~/.local/bin; then
@@ -409,7 +395,6 @@ __bootstrap_webi() {
echo " Creating$(t_path ' ~/.local/bin')"
mkdir -p "$HOME/.local/bin"
fi
fn_envman_init
##
##
@@ -790,23 +775,12 @@ webi_upgrade() { (
fn_checksum() {
a_filepath="${1}"
if command -v sha1sum > /dev/null; then
sha1sum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
return 0
fi
cmd_shasum='sha1sum'
if command -v shasum > /dev/null; then
shasum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
return 0
cmd_shasum='shasum'
fi
if command -v sha1 > /dev/null; then
sha1 "${a_filepath}" | cut -d'=' -f2 | cut -c 2-9
return 0
fi
echo >&2 " warn: no sha1 sum program"
date '+%F %H:%M'
$cmd_shasum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
}
##############################################

View File

@@ -2,14 +2,17 @@
var InstallerServer = module.exports;
let Fs = require('fs/promises');
let Path = require('path');
var Fs = require('fs/promises');
var path = require('path');
let HostTargets = require('./build-classifier/host-targets.js');
let Builds = require('./builds.js');
let Installers = require('./installers.js');
var uaDetect = require('./ua-detect.js');
var Projects = require('./projects.js');
var Installers = require('./installers.js');
InstallerServer.INSTALLERS_DIR = Path.join(__dirname, '..');
// handlers caching and transformation, probably should be broken down
var Releases = require('./transform-releases.js');
InstallerServer.INSTALLERS_DIR = path.join(__dirname, '..');
InstallerServer.serveInstaller = async function (
baseurl,
ua,
@@ -19,210 +22,113 @@ InstallerServer.serveInstaller = async function (
formats,
libc,
) {
let unameAgent = ua;
let projectName = pkg;
let [rel, tmplParams] = await InstallerServer.helper({
unameAgent,
projectName,
let [rel, opts] = await InstallerServer.helper({
ua,
pkg,
tag,
formats,
libc,
});
Object.assign(tmplParams, {
Object.assign(opts, {
baseurl,
});
var pkgdir = Path.join(InstallerServer.INSTALLERS_DIR, projectName);
var pkgdir = path.join(InstallerServer.INSTALLERS_DIR, pkg);
if ('ps1' === ext) {
return Installers.renderPowerShell(pkgdir, rel, tmplParams);
return Installers.renderPowerShell(pkgdir, rel, opts);
}
return Installers.renderBash(pkgdir, rel, tmplParams);
return Installers.renderBash(pkgdir, rel, opts);
};
InstallerServer.helper = async function ({ ua, pkg, tag, formats, libc }) {
// TODO put some of this in a middleware? or common function?
// TODO put some of this in a middleware? or common function?
// TODO maybe move package/version/lts/channel detection into getReleases
InstallerServer.helper = async function ({
unameAgent,
projectName,
tag,
formats,
libc,
}) {
console.log(`dbg: Installer User-Agent: ${unameAgent}`);
// TODO maybe move package/version/lts/channel detection into getReleases
var ver = tag.replace(/^v/, '');
var lts;
var channel;
let releaseTarget = toReleaseTarget(tag);
let hostFormats = formats;
let terms = unameAgent.split(/[\s\/]+/g);
let hostTarget = {};
try {
void HostTargets.termsToTarget(hostTarget, terms);
} catch (e) {
// if we can't guarantee the results...
// "in the face of ambiguity, refuse the temptation to guess"
throw e;
}
console.log(`dbg: Installer Host Target:`);
console.log(hostTarget);
if (!hostTarget.os) {
throw new Error(`OS could not be identified by User-Agent '${unameAgent}'`);
switch (ver) {
case 'latest':
ver = '';
channel = 'stable';
break;
case 'lts':
lts = true;
channel = 'stable';
ver = '';
break;
case 'stable':
channel = 'stable';
ver = '';
break;
case 'beta':
channel = 'beta';
ver = '';
break;
case 'dev':
channel = 'dev';
ver = '';
break;
}
console.log(`dbg: Get Project Installer Type for '${projectName}':`);
let proj = await Builds.getProjectType(projectName);
if (proj.type === 'alias') {
console.log(`dbg: alias`, proj);
projectName = proj.detail;
proj = await Builds.getProjectType(projectName); // an alias should never resolve to an alias
var myOs = uaDetect.os(ua);
var myArch = uaDetect.arch(ua);
var myLibc;
if (libc) {
myLibc = uaDetect.libc(libc);
}
console.log(`dbg: proj`, proj);
let validTypes = ['selfhosted', 'valid'];
if (!validTypes.includes(proj.type)) {
let msg = `'${projectName}' doesn't have an installer: '${proj.type}': '${proj.detail}'`;
let err = new Error(msg);
err.code = 'ENOENT';
throw err;
if (!myLibc) {
myLibc = uaDetect.libc(ua);
}
if (!myLibc) {
myLibc = 'libc';
}
let tmplParams = {
pkg: projectName,
tag: tag,
os: hostTarget.os,
arch: hostTarget.arch,
libc: hostTarget.libc,
formats: hostFormats,
let cfg = await Projects.get(pkg);
let releaseQuery = {
pkg: cfg.alias || pkg,
ver,
os: myOs,
arch: myArch,
libc: myLibc,
lts,
channel,
// TODO use formats for sorting, not exclusion
// (it's better to install xz or report an error to install zip)
formats,
limit: 1,
};
Object.assign(tmplParams, releaseTarget);
console.log('tmplParams', tmplParams);
let errPackage = {
name: 'doesntexist.ext',
version: '0.0.0',
lts: '-',
channel: 'error',
date: '1970-01-01',
os: hostTarget.os || '-',
arch: hostTarget.arch || '-',
libc: hostTarget.libc || '-',
ext: 'err',
download: 'https://example.com/doesntexist.ext',
comment:
'No matches found. Could be bad or missing version info' +
',' +
"Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
let rels = await Releases.getReleases(releaseQuery);
var rel = rels.releases[0];
var opts = {
pkg: cfg.alias || pkg,
ver,
tag,
os: myOs,
arch: myArch,
libc: myLibc,
lts,
channel,
formats,
limit: 1,
};
if (proj.type === 'selfhosted') {
return [errPackage, tmplParams];
}
let projInfo = await Builds.getPackage({
name: projectName,
date: new Date(),
});
let latestVersions = Builds.enumerateLatestVersions(projInfo);
//console.log('projInfo', projInfo);
let buildTargetInfo = {
triplets: projInfo.triplets,
oses: projInfo.oses,
arches: projInfo.arches,
libcs: projInfo.libcs,
formats: projInfo.formats,
latest: latestVersions.latest,
stable: latestVersions.stable,
};
// TODO .findMatchingPackages() should probably account for this
let hasOs = projInfo.oses.includes(hostTarget.os);
let maybePosix = !hasOs && hostTarget.os !== 'windows';
if (maybePosix) {
let posixes = ['posix_2017', 'posix_2024'];
for (let posixYear of posixes) {
let hasPosix = projInfo.oses.includes(posixYear);
if (hasPosix) {
hasOs = true;
break;
}
}
}
if (!hasOs) {
hasOs = projInfo.oses.includes('ANYOS');
}
if (!hasOs) {
let pkg1 = Object.assign(buildTargetInfo, errPackage);
return [pkg1, tmplParams];
}
let targetRelease = Builds.findMatchingPackages(
projInfo,
hostTarget,
releaseTarget,
rel = Object.assign(
{
oses: rels.oses,
arches: rels.arches,
libcs: rels.libcs,
formats: rels.formats,
},
rel,
);
// { triplet: `${os}-${arch}-${libc}`, packages: targetPackages
// , latest: projInfo.versions[0], versions: matchInfo
// }
if (!targetRelease?.packages) {
let pkg1 = Object.assign(buildTargetInfo, errPackage);
return [pkg1, tmplParams];
}
let buildPkg = Builds.selectPackage(targetRelease.packages, hostFormats);
let ext = buildPkg.ext || '.exe';
if (ext.startsWith('.')) {
ext = ext.slice(1);
}
let version = targetRelease.version;
if (version.startsWith('v')) {
version = version.slice(1);
}
buildPkg = Object.assign(buildTargetInfo, buildPkg, { ext, version });
console.log('dbg: buildPkg', buildPkg);
console.log('dbg: tmplParams', tmplParams);
return [buildPkg, tmplParams];
return [rel, opts];
};
let channelNames = [
'stable',
// 'hotfix',
'latest',
'rc',
'preview',
'pre',
'dev',
'beta',
'alpha',
];
function toReleaseTarget(tag) {
tag = tag.replace(/^v/, '');
let releaseTarget = {
channel: '',
lts: false,
version: '',
};
if (tag === 'lts') {
releaseTarget.lts = true;
releaseTarget.channel = 'stable';
} else if (channelNames.includes(tag)) {
releaseTarget.channel = tag;
} else {
releaseTarget.version = tag;
}
return releaseTarget;
}
var CURL_PIPE_PS1_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.ps1');
var CURL_PIPE_SH_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.sh');
var CURL_PIPE_PS1_BOOT = path.join(__dirname, 'curl-pipe-bootstrap.tpl.ps1');
var CURL_PIPE_SH_BOOT = path.join(__dirname, 'curl-pipe-bootstrap.tpl.sh');
var BAD_SH_RE = /[<>'"`$\\]/;
InstallerServer.getPosixCurlPipeBootstrap = async function ({
@@ -244,6 +150,7 @@ InstallerServer.getPosixCurlPipeBootstrap = async function ({
let name = env[0];
let value = env[1];
// TODO create REs once, in higher scope
let envRe = new RegExp(
`^[ \\t]*#?[ \\t]*(export[ \\t])?[ \\t]*(${name})=.*`,
'm',
@@ -291,6 +198,9 @@ InstallerServer.getPwshCurlPipeBootstrap = async function ({
let tplRe = new RegExp(`{{ (${name}) }}`, 'g');
bootTxt = bootTxt.replace(tplRe, `${value}`);
// let envRe = new RegExp(`^[ \\t]*#?[ \\t]*($$${name})[ \\t]*=.*`, 'im');
// bootTxt = bootTxt.replace(envRe, `$$${name} = '${value}'`);
let setRe = new RegExp(
`(#[ \\t]*)?(\\$${name})[ \\t]*=[ \\t]['"].*['"][ \\t]`,
'im',

View File

@@ -32,7 +32,7 @@ if (/\b-?-h(elp)?\b/.test(process.argv.join(' '))) {
var os = require('os');
var fs = require('fs');
var path = require('path');
var Builds = require('./builds.js');
var Releases = require('./transform-releases.js');
var Installers = require('./installers.js');
var ServeInstaller = require('./serve-installer.js');
@@ -65,8 +65,7 @@ console.info('Has the necessary files?');
});
console.info('');
let projName = pkgdir.split('/').filter(Boolean).pop();
Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
Releases.get(path.join(process.cwd(), pkgdir)).then(async function (all) {
var pkgname = path.basename(pkgdir.replace(/\/$/, ''));
var nodeOs = os.platform();
var nodeOsRelease = os.release();
@@ -83,8 +82,8 @@ Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
var formats = ['exe', 'xz', 'tar', 'zip', 'git'];
let [rel, opts] = await ServeInstaller.helper({
unameAgent: `${nodeOs}/${nodeOsRelease} ${nodeArch}/unknown ${nodeLibc}`,
projectName: pkgname,
ua: `${nodeOs}/${nodeOsRelease} ${nodeArch}/unknown ${nodeLibc}`,
pkg: pkgname,
tag: pkgtag || '',
formats: formats,
libc: nodeLibc,

View File

@@ -3,6 +3,7 @@
var Releases = module.exports;
var path = require('path');
var request = require('@root/request');
var _normalize = require('./normalize.js');
var cache = {};
@@ -16,18 +17,14 @@ let installerDir = path.join(__dirname, '..');
Releases.get = async function (pkgdir) {
let get;
try {
get = require(`${pkgdir}/releases.js`);
// TODO update all releases files with module.exports.xxxx = 'foo';
if (!get.latest) {
get.latest = get;
}
get = require(path.join(pkgdir, 'releases.js'));
} catch (e) {
let err = new Error('no releases.js for', pkgdir.split(/[\/\\]+/).pop());
err.code = 'E_NO_RELEASE';
throw err;
}
let all = await get.latest();
let all = await get(request);
return _normalize(all);
};

View File

@@ -72,8 +72,8 @@ gc "feat: new feature"
### Common aliases
Use *alias*es to make other tools you find around webi even _more_ convenient ⚡️
(and powerful 💪).
Use *alias*es to make other tools you find around webi even _more_ convenient
⚡️ (and powerful 💪).
```sh
aliasman curl 'curlie'

View File

@@ -1,21 +1,29 @@
'use strict';
let Releases = module.exports;
var githubSource = require('../_common/github-source.js');
var owner = 'BeyondCodeBootcamp';
var repo = 'aliasman';
let GitHubSource = require('../_common/github-source.js');
let owner = 'BeyondCodeBootcamp';
let repo = 'aliasman';
Releases.latest = async function () {
let all = await GitHubSource.getDistributables({ owner, repo });
for (let pkg of all.releases) {
pkg.os = 'posix_2017';
}
return all;
module.exports = function (request) {
let arches = [
'amd64',
'arm64',
'armv6l',
'armv7l',
'ppc64le',
'ppc64',
's390x',
'x86',
];
let oses = ['freebsd', 'linux', 'macos', 'posix'];
return githubSource(request, owner, repo, oses, arches).then(function (all) {
all._names = ['aliasman', 'legacy'];
return all;
});
};
if (module === require.main) {
Releases.latest().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all, null, 2));
});

View File

@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading archiver from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing archiver"
# TODO: create package-specific temp directory

View File

@@ -4,15 +4,15 @@ var github = require('../_common/github.js');
var owner = 'mholt';
var repo = 'archiver';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
all._names = ['archiver', 'arc'];
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);

View File

@@ -1,5 +1,5 @@
#!/bin/pwsh
Write-Output "'archiver@$Env:WEBI_TAG' is an alias for 'arc@$Env:WEBI_VERSION'"
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
curl.exe -A MS -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell

View File

@@ -20,7 +20,7 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Checking for (or Installing) MSVC Runtime..."
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
@@ -29,7 +29,7 @@ if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing AtomicParsley"
# TODO: create package-specific temp directory

View File

@@ -32,8 +32,8 @@ let targets = {
},
};
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
for (let rel of all.releases) {
let windows32 = rel.name.includes('WindowsX86.');
if (windows32) {
@@ -71,7 +71,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all));
//console.info(JSON.stringify(all, null, 2));

View File

@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading awless from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing awless"
# TODO: create package-specific temp directory

View File

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
var owner = 'wallix';
var repo = 'awless';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
// remove checksums and .deb
all.releases = all.releases.filter(function (rel) {
return !/(\.txt)|(\.deb)$/i.test(rel.name);
@@ -15,7 +15,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all));
});

View File

@@ -3,7 +3,7 @@
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
$EXENAME = "$Env:PKG_NAME.exe"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
@@ -11,11 +11,11 @@ if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
# Fetch MSVC Runtime
Write-Output "Checking for MSVC Runtime..."
if (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
IF (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
}
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
Write-Output "Installing $Env:PKG_NAME"
# TODO: temp directory

View File

@@ -4,14 +4,14 @@ var github = require('../_common/github.js');
var owner = 'sharkdp';
var repo = 'bat';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
all.releases = all.releases.slice(0, 10);
//console.info(JSON.stringify(all));

View File

@@ -6,23 +6,15 @@ main() { (
sed '1,/^#~\/.local\/bin\/brew-updater/d' "${0}" > ~/.local/bin/brew-update-hourly
chmod a+x ~/.local/bin/brew-update-hourly
echo "Checking for serviceman..."
~/.local/bin/webi serviceman
if ! command -v serviceman > /dev/null; then
export PATH="$HOME/.local/bin:$PATH"
fi
serviceman --version
serviceman add --agent \
env PATH="$PATH" serviceman add --user \
--workdir ~/.local/opt/brew/ \
--name sh.brew.updater -- \
~/.local/bin/brew-update-hourly
); }
if ! main; then
exit 1
if main; then
exit 0
fi
exit 0
#~/.local/bin/brew-updater
#!/bin/sh

View File

@@ -132,7 +132,9 @@ file)
```
3. Add your project to the system launcher, running as the current user
```sh
serviceman add --name 'my-project' --daemon -- \
sudo env PATH="$PATH" \
serviceman add --path="$PATH" --system \
--username "$(whoami)" --name my-project -- \
bun run ./my-project.js
```
4. Restart the logging service
@@ -153,6 +155,6 @@ For **macOS**:
```
3. Add your project to the system launcher, running as the current user
```sh
serviceman add --agent --name 'my-project' -- \
serviceman add --path="$PATH" --user --name my-project -- \
bun run ./my-project.js
```

View File

@@ -4,45 +4,23 @@ var github = require('../_common/github.js');
var owner = 'oven-sh';
var repo = 'bun';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
// collect baseline asset names so we can prefer them over non-baseline
// (baseline builds avoid SIGILL on older/container CPUs)
let baselineNames = new Set();
all.releases.forEach(function (r) {
if (r.name.includes('-baseline')) {
baselineNames.add(r.name.replace('-baseline', ''));
}
});
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
all.releases = all.releases
.filter(function (r) {
if (r.name.includes('-profile')) {
let isDebug = r.name.includes('-profile');
if (isDebug) {
return false;
}
if (r.name.endsWith('.txt') || r.name.endsWith('.asc')) {
let isAncient = r.name.includes('-baseline');
if (isAncient) {
return false;
}
// drop the non-baseline asset when a baseline twin exists
if (!r.name.includes('-baseline') && baselineNames.has(r.name)) {
return false;
}
let isMusl = r.name.includes('-musl');
if (isMusl) {
r._musl = true;
r.libc = 'musl';
} else if (r.name.includes('-linux-')) {
r.libc = 'gnu';
}
return true;
})
.map(function (r) {
// bun-linux-x64-baseline.zip => bun-linux-x64
r.name = r.name.replace('-baseline', '').replace(/\.zip$/, '');
// bun-v0.5.1 => v0.5.1
r.version = r.version.replace(/bun-/g, '');
return r;
@@ -52,7 +30,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);

View File

@@ -819,10 +819,10 @@ To avoid the nitty-gritty details of `launchd` plist files, you can use
2. Use Serviceman to create a _launchd_ plist file
```sh
my_username="$(id -u -n)"
my_username="$( id -u -n )"
serviceman add --agent --name 'caddy' --workdir ./ -- \
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
serviceman add --user --name caddy -- \
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
```
(this will create `~/Library/LaunchAgents/caddy.plist`)
@@ -837,8 +837,8 @@ This process creates a _User-Level_ service in `~/Library/LaunchAgents`. To
create a _System-Level_ service in `/Library/LaunchDaemons/` instead:
```sh
serviceman add --name 'caddy' --workdir ./ --daemon -- \
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
sudo serviceman add --system --name caddy -- \
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
```
### How to run Caddy as a Windows Service
@@ -856,7 +856,7 @@ serviceman add --name 'caddy' --workdir ./ --daemon -- \
3. Create a **Startup Registry Entry** with Serviceman.
```sh
serviceman.exe add --name caddy -- \
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
```
4. You can manage the service directly with Serviceman. For example:
```sh
@@ -901,8 +901,10 @@ See the notes below to run as a **User Service** or use the JSON Config.
```
4. Use Serviceman to create a _systemd_ config file.
```sh
serviceman add --name 'caddy' --daemon -- \
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
my_username="$( id -u -n )"
sudo env PATH="$PATH" \
serviceman add --system --username "${my_username}" --name caddy -- \
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
```
(this will create `/etc/systemd/system/caddy.service`)
5. Manage the service with `systemctl` and `journalctl`:
@@ -913,10 +915,10 @@ See the notes below to run as a **User Service** or use the JSON Config.
To create a **User Service** instead:
- use `--agent` when running `serviceman`:
- don't use `sudo`, but do use `--user` when running `serviceman`:
```sh
serviceman add --agent --name caddy -- \
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
serviceman add --user --name caddy -- \
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
```
(this will create `~/.config/systemd/user/`)
- user the `--user` flag to manage services and logs:
@@ -1181,8 +1183,7 @@ To prevent search engine and browser confusion
- _DO NOT_ prevent crawling via `robots.txt` \
(counter-intuitive, but pages _must_ be crawled for links to _NOT_ be indexed)
- _all_ domains using public TLS certs _will_ be indexed by default \
(they are all linked to and crawled from various Certificate Transparency
reports)
(they are all linked to and crawled from various Certificate Transparency reports)
- follow these guidelines even if the dev sites use HTTP Basic Auth
```Caddyfile
@@ -1362,13 +1363,19 @@ See also: <https://caddyserver.com/docs/running>
2. Generate the `service` file: \
- JSON Config
```sh
serviceman add --name 'caddy' --daemon -- \
caddy run --resume --envfile ./caddy.env
my_app_user="$( id -u -n )"
sudo env PATH="${PATH}" \
serviceman add --system --cap-net-bind \
--username "${my_app_user}" --name caddy -- \
caddy run --resume --envfile ./caddy.env
```
- Caddyfile
```sh
serviceman add --name 'caddy' --daemon -- \
caddy run --config ./Caddyfile --envfile ./caddy.env
my_app_user="$( id -u -n )"
sudo env PATH="${PATH}" \
serviceman add --system --cap-net-bind \
--username "${my_app_user}" --name caddy -- \
caddy run --config ./Caddyfile --envfile ./caddy.env
```
3. Reload `systemd` config files, the logging service (it may not be started on
a new VPS), and caddy

View File

@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading caddy from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing caddy"
# TODO: create package-specific temp directory

View File

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
var owner = 'caddyserver';
var repo = 'caddy';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
// remove checksums and .deb
all.releases = all.releases.filter(function (rel) {
let isOneOffAsset = rel.download.includes('buildable-artifact');
@@ -20,7 +20,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all));
});

View File

@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading chromedriver from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing chromedriver"
# TODO: create package-specific temp directory

View File

@@ -1,9 +1,7 @@
'use strict';
let Fetcher = require('../_common/fetcher.js');
// See <https://googlechromelabs.github.io/chrome-for-testing/>
const releaseApiUrl =
var releaseApiUrl =
'https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json';
// {
@@ -42,24 +40,14 @@ const releaseApiUrl =
// ]
// }
module.exports = async function () {
let resp;
try {
resp = await Fetcher.fetch(releaseApiUrl, {
headers: { Accept: 'application/json' },
});
} catch (e) {
/** @type {Error & { code: string, response: { status: number, body: string } }} */ //@ts-expect-error
let err = e;
if (err.code === 'E_FETCH_RELEASES') {
err.message = `failed to fetch 'chromedriver' release data: ${err.response.status} ${err.response.body}`;
}
throw e;
}
let data = JSON.parse(resp.body);
module.exports = async function (request) {
let resp = await request({
url: releaseApiUrl,
json: true,
});
let builds = [];
for (let release of data.versions) {
for (let release of resp.body.versions) {
if (!release.downloads.chromedriver) {
continue;
}
@@ -70,7 +58,7 @@ module.exports = async function () {
version: version,
download: asset.url,
// I' not sure that this is actually statically built but it
// seems to be and at worst we'll just get bug reports for Alpine
// seems to be and at worst we'll just get bug reports for Apline
libc: 'none',
};
@@ -87,15 +75,10 @@ module.exports = async function () {
};
if (module === require.main) {
module
.exports()
.then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the latest 20 for demonstration
all.releases = all.releases.slice(-20);
console.info(JSON.stringify(all, null, 2));
})
.catch(function (err) {
console.error('Error:', err);
});
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the latest 5 for demonstration
all.releases = all.releases.slice(-20);
console.info(JSON.stringify(all, null, 2));
});
}

View File

@@ -1,47 +0,0 @@
---
title: cilium
homepage: https://github.com/cilium/cilium-cli
tagline: |
cilium: manage & troubleshoot Kubernetes clusters running Cilium
---
To update or switch versions, run `webi cilium@stable` (or `@v2`, `@beta`,etc).
### Files
These are the files / directories that are created and/or modified with this
install:
```text
~/.config/envman/PATH.env
~/.local/bin/cilium
~/.local/opt/cilium/
```
## Cheat Sheet
> Cilium is an open source, cloud native solution for providing, securing, and
> observing network connectivity between workloads, fueled by the revolutionary
> Kernel technology eBPF.
Quick Start User Guide:
<https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/#k8s-install-quick>
To install the default version of the Cilium image:
```sh
cilium install
```
To upgrade to a specific version of the Cilium image:
```sh
cilium upgrade --version v1.15.3
```
To check the status of the current Cilium deployment:
```sh
cilium status
```

View File

@@ -1,45 +0,0 @@
#!/usr/bin/env pwsh
##################
# Install cilium #
##################
$pkg_cmd_name = "cilium"
$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\cilium.exe"
$pkg_dst = "$pkg_dst_cmd"
$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\cilium-v$Env:WEBI_VERSION\bin\cilium.exe"
$pkg_src_bin = "$Env:USERPROFILE\.local\opt\cilium-v$Env:WEBI_VERSION\bin"
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\cilium-v$Env:WEBI_VERSION"
$pkg_src = "$pkg_src_cmd"
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading cilium from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing cilium"
Push-Location .local\tmp
Remove-Item -Path ".\cilium-v*" -Recurse -ErrorAction Ignore
Remove-Item -Path ".\cilium.exe" -Recurse -ErrorAction Ignore
Write-Output "Unpacking $pkg_download"
& tar xf "$pkg_download"
Write-Output "Install Location: $pkg_src_cmd"
New-Item "$pkg_src_bin" -ItemType Directory -Force
Move-Item -Path ".\cilium-*\cilium.exe" -Destination "$pkg_src_bin"
Pop-Location
}
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse

View File

@@ -1,39 +0,0 @@
#!/bin/sh
__init_cilium() {
set -e
set -u
##################
# Install cilium #
##################
pkg_cmd_name="cilium"
pkg_dst_cmd="$HOME/.local/bin/cilium"
pkg_dst="$pkg_dst_cmd"
pkg_src_cmd="$HOME/.local/opt/cilium-v$WEBI_VERSION/bin/cilium"
pkg_src_dir="$HOME/.local/opt/cilium-v$WEBI_VERSION"
pkg_src="$pkg_src_cmd"
WEBI_SINGLE=true
# pkg_install must be defined by every package
pkg_install() {
# ~/.local/opt/cilium-v0.16.16/bin
mkdir -p "$(dirname "${pkg_src_cmd}")"
# mv ./hugo ~/.local/opt/cilium-v0.16.16/bin/
mv ./cilium "${pkg_src_cmd}"
}
pkg_get_current_version() {
cilium version 2> /dev/null |
head -n 1 |
cut -d ' ' -f 2
}
}
__init_cilium

View File

@@ -1,21 +0,0 @@
'use strict';
var github = require('../_common/github.js');
var owner = 'cilium';
var repo = 'cilium-cli';
module.exports = async function () {
let all = await github(null, owner, repo);
return all;
};
if (module === require.main) {
(async function () {
let normalize = require('../_webi/normalize.js');
let all = await module.exports();
all = normalize(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);
console.info(JSON.stringify(all, null, 2));
})();
}

View File

@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$pkg_download")) {
IF (!(Test-Path -Path "$pkg_download")) {
Write-Output "Downloading cmake from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_dir")) {
IF (!(Test-Path -Path "$pkg_src_dir")) {
Write-Output "Installing cmake"
# TODO: create package-specific temp directory

View File

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
var owner = 'Kitware';
var repo = 'CMake';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
for (let rel of all.releases) {
if (rel.version.startsWith('v')) {
rel._version = rel.version.slice(1);
@@ -44,7 +44,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);

View File

@@ -1,55 +0,0 @@
---
title: XCode Command Line Tools
homepage: https://webinstall.dev/commandlinetools
tagline: |
The XCode Command Line Tools include git, swift, make, clang, and other developer tools
---
## Cheat Sheet
> The developer tools provided by Apple for macOS.
- git
- swift
- make
- clang
- etc
This is also part of [webi-essentials](../webi-essentials/).
## Table of Contents
- Files
- Manual Install
- macOS
- Linux
- Alpine
- Windows
### Files
These are the files / directories that are created and/or modified with this
install:
```sh
/Library/Developer/CommandLineTools/
```
### How to Install Manually
It's very easy to start the installer:
```sh
xcode-select --install
```
The trick is to also have a mechanism to know when it has finished:
```sh
while ! test -x /Library/Developer/CommandLineTools/usr/bin/git ||
! test -x /Library/Developer/CommandLineTools/usr/bin/make; do
sleep 0.25
done
echo "Command Line Tools Installed"
```

View File

@@ -1,62 +0,0 @@
#!/bin/sh
set -e
set -u
fn_install_xcode_commandlinetools() { (
b_os="$(uname -s)"
if test "${b_os}" != 'Darwin'; then
echo >&2 'XCode Command Line Tools are for macOS only'
return 1
fi
# streamline the output to be pretty
fn_check_pkg '/Library/Developer/CommandLineTools/usr/bin/clang' 'clang'
fn_check_pkg '/Library/Developer/CommandLineTools/usr/bin/git' 'git'
fn_check_pkg '/Library/Developer/CommandLineTools/usr/bin/make' 'make'
echo >&2 ""
# git
if xcode-select -p > /dev/null 2> /dev/null; then
echo ""
return 0
fi
cmd_xcode_cli_tools_install="xcode-select --install"
echo " Running $(t_cmd "${cmd_xcode_cli_tools_install}")"
$cmd_xcode_cli_tools_install 2> /dev/null
echo ""
echo ">>> $(t_attn 'ACTION REQUIRED') <<<"
echo ""
echo " $(t_attn "Click") '$(t_bold 'Install')' $(t_attn "in the pop-up")"
echo " (it may appear $(t_em 'under') this window)"
echo ""
echo "^^^ $(t_attn 'ACTION REQUIRED') ^^^"
echo ""
printf " waiting %s to finish installing Command Line Developer Tools ..." "$(t_em 'for you')"
while ! test -x /Library/Developer/CommandLineTools/usr/bin/git ||
! test -x /Library/Developer/CommandLineTools/usr/bin/make; do
sleep 0.25
done
echo " $(t_info 'OK')"
echo " Installed to $(t_path '/Library/Developer/CommandLineTools/')"
sleep 1
); }
fn_check_pkg() { (
a_pkg="${1}"
a_pkgname="${2:-$a_pkg}"
printf >&2 ' %s %s %s' \
"$(t_dim "Checking for")" \
"$(t_pkg "${a_pkgname}")" \
"$(t_dim "...")"
if command -v "${a_pkg}" > /dev/null; then
echo >&2 " $(t_dim 'OK')"
return 0
fi
echo >&2 ' missing'
); }
fn_install_xcode_commandlinetools

View File

@@ -3,13 +3,13 @@
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
$EXENAME = "$Env:PKG_NAME.exe"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
}
if (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION\bin\$VERNAME")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION\bin\$VERNAME")) {
Write-Output "Installing $Env:PKG_NAME"
# TODO: temp directory

View File

@@ -6,8 +6,8 @@ var repo = 'comrak';
var ODDITIES = ['-musleabihf.1-'];
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
let builds = [];
loopBuilds: for (let build of all.releases) {
@@ -31,7 +31,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
all.releases = all.releases.slice(0, 10);
//console.info(JSON.stringify(all));

View File

@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$pkg_download")) {
IF (!(Test-Path -Path "$pkg_download")) {
Write-Output "Downloading crabz from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing crabz"
# TODO: create package-specific temp directory

View File

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
var owner = 'sstadick';
var repo = 'crabz';
module.exports = async function () {
let all = await github(null, owner, repo);
module.exports = async function (request) {
let all = await github(request, owner, repo);
let releases = [];
for (let rel of all.releases) {
@@ -22,7 +22,7 @@ module.exports = async function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);

View File

@@ -3,13 +3,13 @@
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
$EXENAME = "$Env:PKG_NAME.exe"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
}
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
Write-Output "Installing $Env:PKG_NAME"
# TODO: temp directory

View File

@@ -4,15 +4,15 @@ var github = require('../_common/github.js');
var owner = 'rs';
var repo = 'curlie';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
all._names = ['curlie', 'curl-httpie'];
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
all.releases = all.releases.slice(0, 10);
//console.info(JSON.stringify(all));

View File

@@ -100,7 +100,8 @@ mkdir -p ~/.dashcore/wallets/
mkdir -p /mnt/slc1_vol_100g/dashcore/_data
mkdir -p /mnt/slc1_vol_100g/dashcore/_caches
serviceman add --name 'dashd' --daemon -- \
sudo env PATH="$PATH" serviceman add \
--system --user "$my_user" --path "$PATH" --name dashd --force -- \
dashd \
-usehd \
-conf="$HOME/.dashcore/dash.conf" \

View File

@@ -1,14 +1,7 @@
# for exposing RPCs, building APIs
txindex=1
addressindex=1
timestampindex=1
spentindex=1
# listen as server (explicit default)
listen=1
# because its already run as a service (systemd, openrc)
daemon=0
# for evonodes
#server=1
[main]
rpcuser=RPCUSER_MAIN

View File

@@ -17,7 +17,7 @@ fn_usage() { (
); }
fn_datadir_help() { (
my_vol="${1}"
my_vol="${1:-}"
my_user="$(
id -n -u
)"
@@ -25,7 +25,6 @@ fn_datadir_help() { (
id -n -g
)"
my_mount="$(dirname "${my_vol}")"
echo >&2 ""
echo >&2 "ERROR"
echo >&2 " '${my_vol}' is not writable"
@@ -33,8 +32,8 @@ fn_datadir_help() { (
echo >&2 "SOLUTION"
echo >&2 " 1. Mount a large (50gb+) volume"
echo >&2 ""
echo >&2 " sudo mkdir -p ${my_mount}"
echo >&2 " sudo mount /dev/sdx1 ${my_mount}"
echo >&2 " sudo mkdir -p /mnt/EXAMPLE"
echo >&2 " sudo mount /dev/sdx1 /mnt/EXAMPLE"
echo >&2 ""
echo >&2 " 2. Create a 'dashcore' inside of it"
echo >&2 ""
@@ -84,8 +83,20 @@ fn_srv_install() { (
my_name="dashd-${my_netname}"
fi
my_system_args=""
my_kernel="$(
uname -s
)"
if test "Darwin" != "${my_kernel}"; then
my_user="$(
id -u -n
)"
my_system_args="--system --username ${my_user}"
fi
# shellcheck disable=SC2016,SC1090
echo "serviceman add --name \"${my_name}\" --" \
echo 'sudo env PATH="$PATH"' \
"serviceman add ${my_system_args} --path \"\$PATH\" --name \"${my_name}\" --force --" \
"dashd " \
"${my_net_flag}" \
-usehd \
@@ -95,16 +106,16 @@ fn_srv_install() { (
"-datadir=\"${my_datadir}\"" \
"-blocksdir=\"${my_blocksdir}\""
echo ""
echo "Installing latest 'serviceman'..."
echo ""
"$HOME/.local/bin/webi" serviceman > /dev/null
if ! command -v serviceman > /dev/null; then
export PATH="$HOME/.local/bin:$PATH"
fi
serviceman --version
if ! command -v dashd > /dev/null; then
export PATH="$HOME/.local/opt/dashcore/bin:$PATH"
echo ""
echo "Installing 'serviceman'..."
echo ""
{
curl -fsSL "${WEBI_HOST}/serviceman" | sh
} > /dev/null
# shellcheck disable=SC1090
. ~/.config/envman/PATH.env || true
fi
mkdir -p "$HOME/.dashcore/wallets/"
@@ -119,7 +130,8 @@ fn_srv_install() { (
cd "${my_vol}" || return 1
# leave options unquoted so they're interpreted separately
# shellcheck disable=SC2086
serviceman add --name "${my_name}" -- \
sudo env PATH="${PATH}" \
serviceman add ${my_system_args} --path "${PATH}" --name "${my_name}" --force -- \
dashd \
${my_net_flag} \
-usehd \

View File

@@ -5,38 +5,32 @@ set -u
__install_dashcore_utils() {
webi_download \
"$WEBI_HOST/packages/dashcore-utils/dash-qt-hd" \
"$HOME/.local/bin/dash-qt-hd" \
"dash-qt-hd"
"$HOME/.local/bin/dash-qt-hd"
chmod a+x "$HOME/.local/bin/dash-qt-hd"
webi_download \
"$WEBI_HOST/packages/dashcore-utils/dash-qt-testnet" \
"$HOME/.local/bin/dash-qt-testnet" \
"dash-qt-testnet"
"$HOME/.local/bin/dash-qt-testnet"
chmod a+x "$HOME/.local/bin/dash-qt-testnet"
webi_download \
"$WEBI_HOST/packages/dashcore-utils/dashd-hd" \
"$HOME/.local/bin/dashd-hd" \
"dashd-hd"
"$HOME/.local/bin/dashd-hd"
chmod a+x "$HOME/.local/bin/dashd-hd"
webi_download \
"$WEBI_HOST/packages/dashcore-utils/dashd-testnet" \
"$HOME/.local/bin/dashd-testnet" \
"dashd-testnet"
"$HOME/.local/bin/dashd-testnet"
chmod a+x "$HOME/.local/bin/dashd-testnet"
webi_download \
"$WEBI_HOST/packages/dashcore-utils/dashd-hd-service-install" \
"$HOME/.local/bin/dashd-hd-service-install" \
"dashd-hd-service-install"
"$HOME/.local/bin/dashd-hd-service-install"
chmod a+x "$HOME/.local/bin/dashd-hd-service-install"
webi_download \
"$WEBI_HOST/packages/dashcore-utils/dashd-testnet-service-install" \
"$HOME/.local/bin/dashd-testnet-service-install" \
"dashd-testnet-service-install"
"$HOME/.local/bin/dashd-testnet-service-install"
chmod a+x "$HOME/.local/bin/dashd-testnet-service-install"
if ! test -e "${HOME}/.dashcore"; then
@@ -50,8 +44,7 @@ __install_dashcore_utils() {
webi_download \
"$WEBI_HOST/packages/dashcore-utils/dash.example.conf" \
"$HOME/.dashcore/dash.example.conf" \
"dash.example.conf"
"$HOME/.dashcore/dash.example.conf"
if ! grep -q rpcuser ~/.dashcore/dash.conf; then
cat ~/.dashcore/dash.example.conf >> ~/.dashcore/dash.conf

View File

@@ -147,8 +147,8 @@ dash-qt \
CoinJoin aids in preventing some bad actors and malicious observers being able
to easily reconstruct details about your transactions from the publicly
available data by creating many excess transactions. \
(be aware, however, that dedicated bad actors can use sophisticated software
that will reveal much of the same information over time)
(be aware, however, that dedicated bad actors can use sophisticated software that
will reveal much of the same information over time)
`dash-qt` does not enable CoinJoin mixing by default.

View File

@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading dashcore from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_dir")) {
IF (!(Test-Path -Path "$pkg_src_dir")) {
Write-Output "Installing dashcore"
# TODO: create package-specific temp directory

View File

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
var owner = 'dashpay';
var repo = 'dash';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
all.releases.forEach(function (rel) {
if (rel.name.includes('osx64')) {
rel.os = 'macos';
@@ -22,7 +22,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);

View File

@@ -24,7 +24,7 @@ install:
~/.config/envman/PATH.env
~/.dashcore/dash.conf
~/.dashcore/wallets/
~/.local/bin/dashd-hd-service-install
~/.local/bin/bin/dashd-hd-service-install
~/.local/opt/dashcore/
/mnt/<BLK_VOL>/dashcore/
```
@@ -219,7 +219,14 @@ You can use [`serviceman`](../serviceman/):
**Linux**
```sh
serviceman add --name 'dashd' -- \
sudo env PATH="$PATH" \
serviceman add \
--system \
--username "$(id -n -u)" \
--path "$PATH" \
--name dashd \
--force \
-- \
dashd \
-usehd \
-conf="$HOME/.dashcore/dash.conf" \
@@ -232,7 +239,11 @@ serviceman add --name 'dashd' -- \
**Mac**
```sh
serviceman add --name 'dashd' -- \
serviceman add \
--path "$PATH" \
--name dashd \
--force \
-- \
dashd \
-usehd \
-conf="$HOME/.dashcore/dash.conf" \

View File

@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading dashd from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_dir")) {
IF (!(Test-Path -Path "$pkg_src_dir")) {
Write-Output "Installing dashd"
# TODO: create package-specific temp directory

View File

@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading dashmsg from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing dashmsg"
# TODO: create package-specific temp directory

View File

@@ -4,14 +4,14 @@ var github = require('../_common/github.js');
var owner = 'dashhive';
var repo = 'dashmsg';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all, null, 2));
});

View File

@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading delta from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing delta"
# TODO: create package-specific temp directory

View File

@@ -4,14 +4,14 @@ var github = require('../_common/github.js');
var owner = 'dandavison';
var repo = 'delta';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 15 for demonstration
all.releases = all.releases.slice(0, 15);

View File

@@ -18,7 +18,7 @@ etc).
The obligatory Hello World
```sh
deno run https://docs.deno.com/examples/scripts/hello_world.ts
deno run https://deno.land/std/examples/welcome.ts
```
Run a local file

View File

@@ -1,14 +1,14 @@
#!/usr/bin/env pwsh
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
#Invoke-WebRequest https://nodejs.org/dist/v12.16.2/node-v12.16.2-win-x64.zip -OutFile node-v12.16.2-win-x64.zip
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
}
if (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION")) {
Write-Output "Installing $Env:PKG_NAME"
# TODO: temp directory

View File

@@ -6,8 +6,8 @@ var github = require('../_common/github.js');
var owner = 'denoland';
var repo = 'deno';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
// remove checksums and .deb
all.releases = all.releases
.filter(function (rel) {
@@ -35,7 +35,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all, null, 2));
});

View File

@@ -20,18 +20,18 @@ $pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch MSVC Runtime
Write-Output "Checking for MSVC Runtime..."
if (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
IF (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
}
# Fetch archive
if (!(Test-Path -Path "$pkg_download")) {
IF (!(Test-Path -Path "$pkg_download")) {
Write-Output "Downloading dotenv-linter from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing dotenv-linter"
# TODO: create package-specific temp directory

View File

@@ -4,14 +4,14 @@ var github = require('../_common/github.js');
var owner = 'dotenv-linter';
var repo = 'dotenv-linter';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
// just select the first 5 for demonstration
all.releases = all.releases.slice(0, 5);

View File

@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading dotenv from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
& Move-Item "$pkg_download.part" "$pkg_download"
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing dotenv"
# TODO: create package-specific temp directory

View File

@@ -4,14 +4,14 @@ var github = require('../_common/github.js');
var owner = 'therootcompany';
var repo = 'dotenv';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
return all;
});
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all));
});

View File

@@ -1,21 +1,29 @@
'use strict';
let Releases = module.exports;
var githubSource = require('../_common/github-source.js');
var owner = 'BeyondCodeBootcamp';
var repo = 'DuckDNS.sh';
let GitHubSource = require('../_common/github-source.js');
let owner = 'BeyondCodeBootcamp';
let repo = 'DuckDNS.sh';
Releases.latest = async function () {
let all = await GitHubSource.getDistributables({ owner, repo });
for (let pkg of all.releases) {
pkg.os = 'posix_2017';
}
return all;
module.exports = function (request) {
let arches = [
'amd64',
'arm64',
'armv6l',
'armv7l',
'ppc64le',
'ppc64',
's390x',
'x86',
];
let oses = ['freebsd', 'linux', 'macos', 'posix'];
return githubSource(request, owner, repo, oses, arches).then(function (all) {
all._names = ['DuckDNS.sh', 'duckdns.sh', 'legacy'];
return all;
});
};
if (module === require.main) {
Releases.latest().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all, null, 2));
});

View File

@@ -2,16 +2,10 @@
set -e
set -u
echo "'duckdns@${WEBI_TAG:-stable}' is reserved for future use."
echo
echo "Did you mean 'duckdns.sh@${WEBI_VERSION-}'?"
WEBI_HOST=${WEBI_HOST:-"https://webi.sh"}
echo ""
echo "ERROR"
echo " installer name 'duckdns' is reserved for future use"
echo ""
echo "SOLUTION"
echo " Did you mean 'duckdns.sh'?"
echo ""
echo " curl -fsSL '$WEBI_HOST/duckdns.sh' | sh"
echo ""
echo " curl -fsSL '$WEBI_HOST/duckdns.sh@${WEBI_VERSION-}' | sh"
exit 1

View File

@@ -3,13 +3,13 @@
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
$EXENAME = "$Env:PKG_NAME.exe"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
}
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
Write-Output "Installing $Env:PKG_NAME"
# TODO: temp directory

View File

@@ -4,8 +4,8 @@ var github = require('../_common/github.js');
var owner = 'sharkdp';
var repo = 'fd';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
let builds = [];
for (let build of all.releases) {
@@ -22,7 +22,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
all.releases = all.releases.slice(0, 10);
//console.info(JSON.stringify(all));

View File

@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
# Fetch archive
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
Write-Output "Downloading ffmpeg from $Env:WEBI_PKG_URL to $pkg_download"
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
Move-Item -Path "$pkg_download.part" -Destination "$pkg_download" -Force
}
if (!(Test-Path -Path "$pkg_src_cmd")) {
IF (!(Test-Path -Path "$pkg_src_cmd")) {
Write-Output "Installing ffmpeg"
# TODO: create package-specific temp directory

View File

@@ -6,8 +6,8 @@ var github = require('../_common/github.js');
var owner = 'eugeneware';
var repo = 'ffmpeg-static';
module.exports = function () {
return github(null, owner, repo).then(function (all) {
module.exports = function (request) {
return github(request, owner, repo).then(function (all) {
all.releases = all.releases
.filter(function (rel) {
let isFfmpeg = rel.name.includes('ffmpeg');
@@ -38,7 +38,7 @@ module.exports = function () {
};
if (module === require.main) {
module.exports().then(function (all) {
module.exports(require('@root/request')).then(function (all) {
all = require('../_webi/normalize.js')(all);
console.info(JSON.stringify(all));
});

Some files were not shown because too many files have changed in this diff Show More