mirror of
https://github.com/webinstall/webi-installers.git
synced 2026-05-30 04:22:47 +00:00
Compare commits
4 Commits
feat/webi-
...
add-feroxb
| Author | SHA1 | Date | |
|---|---|---|---|
|
1f6e574030 | ||
|
|
6626a0df86 | ||
|
|
b61a678371 | ||
|
eba10b52e6 |
3
.github/ISSUE_TEMPLATE/new_installer.md
vendored
3
.github/ISSUE_TEMPLATE/new_installer.md
vendored
@@ -35,18 +35,15 @@ info, and doing a find and replace on a few file system path names.
|
||||
```
|
||||
2. Copy the example template and update with info from Official Releases:
|
||||
<https://github.com/___CHANGE/ME___/releases>
|
||||
|
||||
```bash
|
||||
rsync -av _example/ CHANGE-ME/
|
||||
```
|
||||
|
||||
- [ ] update `CHANGE-ME/release.js` to use the official repo
|
||||
- [ ] Learn how `CHANGE-ME` unpacks (i.e. as a single file? as a .tar.gz? as
|
||||
a .tar.gz with a folder named CHANGE-ME?)
|
||||
- [ ] find and replace to change the name
|
||||
- [ ] update `CHANGE-ME/install.sh` (see `bat` and `jq` as examples)
|
||||
- [ ] update `CHANGE-ME/install.ps1` (see `bat` and `jq` as examples)
|
||||
|
||||
3. Needs an updated tagline and cheat sheet
|
||||
- [ ] update `CHANGE-ME/README.md`
|
||||
- [ ] official URL
|
||||
|
||||
7
.github/workflows/node.js.yml
vendored
7
.github/workflows/node.js.yml
vendored
@@ -23,15 +23,10 @@ jobs:
|
||||
sh ./_scripts/install-ci-deps
|
||||
echo "${HOME}/.local/bin" >> $GITHUB_PATH
|
||||
echo "${HOME}/.local/opt/pwsh" >> $GITHUB_PATH
|
||||
- run: |
|
||||
git submodule init
|
||||
git submodule update
|
||||
- run: shfmt --version
|
||||
- run: shellcheck -V
|
||||
- run: node --version
|
||||
- run: npm run fmt
|
||||
- run: npm ci
|
||||
- run: npm run lint
|
||||
- env:
|
||||
GITHUB_TOKEN: ${{ github.token }}
|
||||
run: npm run test
|
||||
- run: npm run test
|
||||
|
||||
37
.gitignore
vendored
37
.gitignore
vendored
@@ -1,40 +1,7 @@
|
||||
# generated artifacts
|
||||
.env
|
||||
node_modules
|
||||
install-*.sh
|
||||
install-*.bat
|
||||
install-*.ps1
|
||||
|
||||
# Go build outputs (from go run/build in repo root)
|
||||
/classify
|
||||
/e2etest
|
||||
/fetchraw
|
||||
/inspect
|
||||
/uaparse
|
||||
/webicached
|
||||
/zigtest
|
||||
/distributables.csv
|
||||
|
||||
# local config
|
||||
.env.*
|
||||
*.env
|
||||
.env
|
||||
!example.env
|
||||
|
||||
# caches
|
||||
_cache/
|
||||
node_modules/
|
||||
|
||||
# temporary & backup files
|
||||
.*.sw*
|
||||
*.bak
|
||||
*.bak.*
|
||||
|
||||
# agent session files
|
||||
agents/
|
||||
|
||||
# other
|
||||
.DS_Store
|
||||
desktop.ini
|
||||
.directory
|
||||
LIVE_cache
|
||||
/webid
|
||||
bin/
|
||||
|
||||
3
.gitmodules
vendored
3
.gitmodules
vendored
@@ -1,3 +0,0 @@
|
||||
[submodule "_webi/build-classifier"]
|
||||
path = _webi/build-classifier
|
||||
url = https://github.com/webinstall/webi-build-classifier.git
|
||||
@@ -1,7 +1,4 @@
|
||||
{
|
||||
"globals": {
|
||||
"AbortController": false
|
||||
},
|
||||
"browser": true,
|
||||
"node": true,
|
||||
"esversion": 11,
|
||||
|
||||
492
AGENTS.md
492
AGENTS.md
@@ -1,492 +0,0 @@
|
||||
# Webi Installers — Agent Guide
|
||||
|
||||
Webi installs dev tools to `~/.local/` without sudo. Each installer is a small
|
||||
package of 3-4 files. This guide tells you how to create and modify them.
|
||||
|
||||
## Why Webi Exists
|
||||
|
||||
Webi makes tool installation trivially repeatable for people who aren't
|
||||
sysadmins — freelance clients, junior devs, anyone who shouldn't have to care
|
||||
about PATH, permissions, or platform differences. Three things matter:
|
||||
|
||||
1. **Install without friction.** No sudo, no manual PATH edits, no "necessary
|
||||
but unimportant" steps leaking into the experience.
|
||||
2. **Know where things are.** The Files section tells you exactly what got
|
||||
created or modified. Nothing should be mysterious.
|
||||
3. **Copy-paste recipes.** The cheat sheet is what you'd send someone less
|
||||
experienced than yourself instead of a project's full README — scannable,
|
||||
concrete, easy to reference by name.
|
||||
|
||||
## Quick Start: Adding a New Installer
|
||||
|
||||
1. Identify the **package type** (see [Categories](#categories) below)
|
||||
2. Find an existing installer of the same type to use as a template
|
||||
3. Create `<name>/releases.js`, `install.sh`, `install.ps1`, `README.md`
|
||||
4. Test with the command in [Testing releases.js](#testing-releasesjs)
|
||||
5. Run formatters before committing (see [Code Style](#code-style))
|
||||
|
||||
## Directory Layout
|
||||
|
||||
```
|
||||
<package-name>/
|
||||
README.md # YAML frontmatter + docs
|
||||
releases.js # Fetches release metadata (Node.js)
|
||||
install.sh # POSIX shell installer (macOS/Linux)
|
||||
install.ps1 # PowerShell installer (Windows) — optional
|
||||
```
|
||||
|
||||
Key infrastructure directories (do not modify without good reason):
|
||||
|
||||
- `_webi/` — bootstrap templates, `normalize.js` (auto-detects OS/arch/ext from
|
||||
filenames)
|
||||
- `_common/` — shared JS: `github.js`, `githubish.js`, `gitea.js`, `fetcher.js`
|
||||
- `_example/` — canonical template for new packages
|
||||
- `_examples/` — specialized templates (goreleaser, xz-compressed)
|
||||
|
||||
## Categories
|
||||
|
||||
Ref: <https://github.com/webinstall/webi-installers/issues/412>
|
||||
|
||||
| Type | Description | Template to copy |
|
||||
| ----- | -------------------------------------- | ---------------- |
|
||||
| `bin` | Single binary in tar/zip | `koji`, `delta` |
|
||||
| `bin` | Single bare binary (no archive) | `arc`, `shfmt` |
|
||||
| `bin` | Goreleaser-style archives | `keypairs` |
|
||||
| 📦 | Self-contained package (bin/man/share) | `node`, `go` |
|
||||
| 📂 | Multiple binaries/scripts | `pg` |
|
||||
| 🔗 | Alias/redirect to another package | `ripgrep` → `rg` |
|
||||
| 📝 | Bespoke / custom install | `rustlang` |
|
||||
|
||||
## releases.js
|
||||
|
||||
Fetches release metadata and returns a normalized object. Most packages use
|
||||
GitHub releases:
|
||||
|
||||
```js
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'OWNER';
|
||||
var repo = 'REPO';
|
||||
|
||||
let Releases = module.exports;
|
||||
|
||||
Releases.latest = async function () {
|
||||
let all = await github(null, owner, repo);
|
||||
return all;
|
||||
};
|
||||
|
||||
Releases.sample = async function () {
|
||||
let normalize = require('../_webi/normalize.js');
|
||||
let all = await Releases.latest();
|
||||
all = normalize(all);
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
(async function () {
|
||||
let samples = await Releases.sample();
|
||||
console.info(JSON.stringify(samples, null, 2));
|
||||
})();
|
||||
}
|
||||
```
|
||||
|
||||
### Common release transformations
|
||||
|
||||
**Strip version prefix** (monorepo or tool-prefixed tags):
|
||||
|
||||
```js
|
||||
// e.g. "tools/monorel/v0.6.5" → "v0.6.5"
|
||||
rel.version = rel.version.replace(/^tools\/monorel\//, '');
|
||||
|
||||
// e.g. "cli-v1.2.3" → "v1.2.3"
|
||||
rel.version = rel.version.replace(/^cli-/, '');
|
||||
```
|
||||
|
||||
**Filter releases** (monorepo with multiple tools, or unwanted assets):
|
||||
|
||||
```js
|
||||
all.releases = all.releases.filter(function (rel) {
|
||||
// Keep only releases for this tool
|
||||
return rel.version.startsWith('tools/monorel/');
|
||||
});
|
||||
```
|
||||
|
||||
Apply transformations inside `Releases.latest`, before returning `all`.
|
||||
|
||||
**Available sources** beyond `github.js`:
|
||||
|
||||
- `_common/gitea.js` — Gitea servers
|
||||
- `_common/git-tag.js` — Git tag listing
|
||||
- Custom fetch from any JSON API (see `go/releases.js`, `terraform/releases.js`)
|
||||
|
||||
### Testing releases.js
|
||||
|
||||
```sh
|
||||
node -e "
|
||||
let Releases = require('./<name>/releases.js');
|
||||
Releases.sample().then(function (all) {
|
||||
console.log(JSON.stringify(all, null, 2));
|
||||
});
|
||||
"
|
||||
```
|
||||
|
||||
Verify: versions are clean semver (`0.6.5` not `tools/monorel/v0.6.5`), OS/arch
|
||||
detected correctly, download URLs resolve.
|
||||
|
||||
## install.sh
|
||||
|
||||
POSIX shell (`sh`, not bash). Always wrapped in a function:
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
set -e
|
||||
set -u
|
||||
|
||||
__init_pkgname() {
|
||||
# These 6 variables are required
|
||||
pkg_cmd_name="cmd"
|
||||
|
||||
pkg_dst_cmd="$HOME/.local/bin/cmd"
|
||||
pkg_dst="$pkg_dst_cmd"
|
||||
|
||||
pkg_src_cmd="$HOME/.local/opt/cmd-v$WEBI_VERSION/bin/cmd"
|
||||
pkg_src_dir="$HOME/.local/opt/cmd-v$WEBI_VERSION"
|
||||
pkg_src="$pkg_src_cmd"
|
||||
|
||||
pkg_install() {
|
||||
mkdir -p "$(dirname "$pkg_src_cmd")"
|
||||
mv ./cmd "$pkg_src_cmd"
|
||||
}
|
||||
|
||||
pkg_get_current_version() {
|
||||
cmd --version 2> /dev/null | head -n 1 | cut -d' ' -f2
|
||||
}
|
||||
}
|
||||
|
||||
__init_pkgname
|
||||
```
|
||||
|
||||
### Framework variables available in install.sh
|
||||
|
||||
Set by the webi bootstrap (`_webi/package-install.tpl.sh`):
|
||||
|
||||
| Variable | Example | Description |
|
||||
| --------------- | ------------------- | --------------------- |
|
||||
| `WEBI_VERSION` | `1.2.3` | Selected version |
|
||||
| `WEBI_PKG_URL` | `https://...` | Download URL |
|
||||
| `WEBI_PKG_FILE` | `foo-v1.2.3.tar.gz` | Download filename |
|
||||
| `WEBI_OS` | `linux` | Detected OS |
|
||||
| `WEBI_ARCH` | `amd64` | Detected architecture |
|
||||
| `WEBI_EXT` | `tar.gz` | Archive extension |
|
||||
| `WEBI_CHANNEL` | `stable` | Release channel |
|
||||
| `PKG_NAME` | `foo` | Package name |
|
||||
|
||||
### Override functions
|
||||
|
||||
| Function | Purpose |
|
||||
| --------------------------- | --------------------------------------------- |
|
||||
| `pkg_install()` | **Required.** Move files to `$pkg_src` |
|
||||
| `pkg_get_current_version()` | Parse installed version from command output |
|
||||
| `pkg_post_install()` | Post-install setup (git config, shell config) |
|
||||
| `pkg_done_message()` | Custom completion message |
|
||||
| `pkg_link()` | Override default symlink behavior |
|
||||
| `pkg_pre_install()` | Custom pre-install logic |
|
||||
|
||||
### Framework helper functions
|
||||
|
||||
| Function | Purpose |
|
||||
| ------------------------ | ---------------------------------- |
|
||||
| `webi_download()` | Download package if not cached |
|
||||
| `webi_extract()` | Extract archive by extension |
|
||||
| `webi_path_add <dir>` | Add to PATH via envman |
|
||||
| `webi_link()` | Create versioned symlinks |
|
||||
| `webi_check_installed()` | Check if version already installed |
|
||||
|
||||
### pkg_install patterns
|
||||
|
||||
**Bare binary in archive root:**
|
||||
|
||||
```sh
|
||||
mv ./cmd "$pkg_src_cmd"
|
||||
```
|
||||
|
||||
**Binary in a subdirectory (goreleaser-style `cmd-OS-arch/cmd`):**
|
||||
|
||||
```sh
|
||||
mv ./cmd-*/cmd "$pkg_src_cmd"
|
||||
```
|
||||
|
||||
**Flexible detection (handles multiple archive layouts):**
|
||||
|
||||
```sh
|
||||
if test -f ./cmd; then
|
||||
mv ./cmd "$pkg_src_cmd"
|
||||
elif test -e ./cmd-*/cmd; then
|
||||
mv ./cmd-*/cmd "$pkg_src_cmd"
|
||||
elif test -e ./cmd-*; then
|
||||
mv ./cmd-* "$pkg_src_cmd"
|
||||
fi
|
||||
```
|
||||
|
||||
## install.ps1
|
||||
|
||||
PowerShell for Windows. Uses `$Env:` variables. See `_example/install.ps1` for
|
||||
the full template. Key differences from install.sh:
|
||||
|
||||
- Paths use backslashes, commands end in `.exe`
|
||||
- `$Env:USERPROFILE` instead of `$HOME`
|
||||
- `Test-Path`, `Move-Item`, `Copy-Item` instead of shell equivalents
|
||||
- Downloads go to `$Env:USERPROFILE\Downloads\webi\`
|
||||
- Temp work in `.local\tmp`, use `Push-Location`/`Pop-Location`
|
||||
- Symlinks done via `Copy-Item` (not actual symlinks)
|
||||
|
||||
## README.md
|
||||
|
||||
````markdown
|
||||
---
|
||||
title: toolname
|
||||
homepage: https://github.com/owner/repo
|
||||
tagline: |
|
||||
toolname: A short one-line description.
|
||||
---
|
||||
|
||||
To update or switch versions, run `webi toolname@stable` (or `@v2`, `@beta`,
|
||||
etc).
|
||||
|
||||
### Files
|
||||
|
||||
These are the files that are created and/or modified with this installer:
|
||||
|
||||
```text
|
||||
~/.config/envman/PATH.env
|
||||
~/.local/bin/toolname
|
||||
~/.local/opt/toolname-VERSION/bin/toolname
|
||||
```
|
||||
|
||||
## Cheat Sheet
|
||||
|
||||
> `toolname` does X. Brief description.
|
||||
|
||||
### How to use toolname
|
||||
|
||||
```sh
|
||||
toolname --example
|
||||
```
|
||||
````
|
||||
|
||||
Note: **Files goes above Cheat Sheet**, not inside it.
|
||||
|
||||
### Cheat Sheet tone and style
|
||||
|
||||
Webi cheat sheets are **opinionated quick-reference guides**, not comprehensive
|
||||
documentation. Think "colleague's sticky note" — not the project's official
|
||||
README.
|
||||
|
||||
The tool is the topic, but **the problem is the reason**. Cheat sheets are
|
||||
organized around tasks the reader already wants to do — the tool is how they get
|
||||
there. Headings reference the tool (the reader came to this page on purpose),
|
||||
but the content solves the underlying problem completely:
|
||||
|
||||
- "How to reverse proxy to Node" (caddy knowledge, not just node)
|
||||
- "How to run a Node app as a System Service" (serviceman knowledge)
|
||||
- "How to Enable Secure Remote Postgres Access" (openssl, pg_hba.conf, systemd)
|
||||
- "How to manually configure git to use delta" (gitconfig, not delta flags)
|
||||
- "How to make fish the default shell in iTerm2" (iTerm2 knowledge, not fish)
|
||||
|
||||
The reader's question is "how do I do X?" and the cheat sheet answers it
|
||||
completely — including configs, adjacent tools, and platform-specific
|
||||
variations. A goreleaser cheat sheet teaches you goreleaser YAML. A postgres
|
||||
cheat sheet teaches you pg_hba.conf, openssl certs, and systemd units.
|
||||
|
||||
Cheat sheets cross tool boundaries freely. Node's references caddy, serviceman,
|
||||
setcap-netbind, GitHub Actions. Postgres references serviceman, openrc, launchd.
|
||||
They link to each other's webi pages. The scope is "everything you need to
|
||||
accomplish this task," not "everything this one binary does."
|
||||
|
||||
They show the actual files and configs that matter — not documentation _about_
|
||||
configs, but the configs themselves, copy-pasteable, with inline comments
|
||||
explaining the non-obvious parts.
|
||||
|
||||
**Guidelines:**
|
||||
|
||||
- **Show the 3-5 things someone will actually do**, with copy-pasteable
|
||||
commands. Skip exhaustive flag lists and API docs.
|
||||
- **Lead with practical integration.** Show the exact `git config` lines, the
|
||||
exact hook script, the exact shell alias — don't just explain the feature and
|
||||
leave wiring up to the reader.
|
||||
- **Skip what they already know.** No need to re-explain what the tool is at
|
||||
length — the tagline and one-liner blockquote handle that. Get to the
|
||||
commands.
|
||||
- **Prefer concrete over abstract.** Instead of "you can configure X via a
|
||||
config file", show the config file contents.
|
||||
|
||||
## Shell Naming Conventions
|
||||
|
||||
**Variables:**
|
||||
|
||||
- `ALL_CAPS` — environment variables only (`PATH`, `HOME`, `WEBI_VERSION`)
|
||||
- `b_varname` — block-scoped (inside a function, loop, or conditional)
|
||||
- `g_varname` — global to the script (and sourced scripts)
|
||||
- `a_varname` — function arguments
|
||||
|
||||
**Functions and commands:**
|
||||
|
||||
- `fn_name` — helper functions (anything other than the script's main/entry
|
||||
function)
|
||||
- `cmd_name` — command aliases, e.g. `cmd_curl='curl --fail-with-body -sSL'`
|
||||
|
||||
## Code Style
|
||||
|
||||
Requires `node`, `shfmt`, `pwsh`, and `pwsh-essentials` (install all via webi).
|
||||
Run before committing:
|
||||
|
||||
```sh
|
||||
npm run fmt # prettier (JS/MD) + shfmt (sh) + pwsh-fmt (ps1)
|
||||
npm run lint # jshint + shellcheck + pwsh-lint
|
||||
```
|
||||
|
||||
Commit messages: `feat(<pkg>): add installer`, `fix(<pkg>): update install.sh`,
|
||||
`docs(<pkg>): add cheat sheet`.
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
- The canonical package name is the **command name** you type: `go`, `node`,
|
||||
`rg`
|
||||
- The alternate/alias name is the project name: `golang`, `nodejs`, `ripgrep`
|
||||
- Package directories are lowercase with hyphens
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- **Monorepo releases**: The GitHub API returns ALL releases for the repo. You
|
||||
must filter in `releases.js` and strip the tag prefix from the version.
|
||||
- **No `--version` flag**: Some tools lack version introspection. Comment out
|
||||
`pkg_get_current_version` — webi still works, it just can't skip reinstalls.
|
||||
- **normalize.js auto-detection**: OS/arch/ext are guessed from download
|
||||
filenames. If the tool uses non-standard naming, you may need to set `os`,
|
||||
`arch`, or `ext` explicitly in `releases.js`.
|
||||
- **Goreleaser archives**: Typically contain a bare binary at the archive root
|
||||
(not nested in a directory). Use `mv ./cmd "$pkg_src_cmd"`.
|
||||
|
||||
---
|
||||
|
||||
## Go Cache Daemon
|
||||
|
||||
The Go pipeline (`cmd/webicached`) replaces the Node.js release-fetching code.
|
||||
It reads `releases.conf` files, fetches upstream release metadata, classifies
|
||||
build assets, and writes to `~/.cache/webi/legacy/` in the format the Node.js server expects.
|
||||
|
||||
### Canonical Vocabulary
|
||||
|
||||
The classifier MUST use exactly these strings. They match the production API.
|
||||
|
||||
**OS**: `macos` (NOT `darwin`), `linux`, `windows`, `freebsd`, `openbsd`,
|
||||
`netbsd`, `dragonfly`, `aix`, `illumos`, `plan9`, `solaris`, `posix_2017`
|
||||
|
||||
**Arch** — exact equivalences:
|
||||
- `amd64` (NOT `x86_64`), `x86` (NOT `i386`/`i686`/`386`)
|
||||
- `arm64` (NOT `aarch64`)
|
||||
- `armv7l` (NOT `armv7`), `armv6l` (NOT `armv6`)
|
||||
- `mipsle` (NOT `mipsel`), `mips64le` (NOT `mips64el`)
|
||||
|
||||
**Arch** — compatibility downcasts:
|
||||
- `armhf` → `armv7l`, `armv7a` → `armv7l`, `armel` → `arm`
|
||||
|
||||
**Arch** — other: `arm`, `ppc64le`, `ppc64`, `loong64`, `riscv64`, `s390x`,
|
||||
`mips`, `mips64`
|
||||
|
||||
**Libc**: `none` (never empty), `gnu`, `musl`, `msvc`
|
||||
|
||||
**Ext**: `tar.gz`, `tar.xz`, `zip`, `exe`, `7z`, `pkg`, `msi`
|
||||
(no leading dot; `exe` for bare binaries)
|
||||
|
||||
### releases.conf
|
||||
|
||||
Each package directory contains a `releases.conf` that tells the daemon where
|
||||
to fetch releases. Format is `key = value`, one per line. `#` comments and
|
||||
blank lines are ignored.
|
||||
|
||||
#### Source types (mutually exclusive — pick one)
|
||||
|
||||
```ini
|
||||
# GitHub binary releases (most common)
|
||||
github_releases = sharkdp/bat
|
||||
|
||||
# GitHub source tarballs (with optional git fallback)
|
||||
github_sources = bnnanet/serviceman
|
||||
git_url = https://github.com/bnnanet/serviceman.git
|
||||
|
||||
# Git tag enumeration (vim plugins, shell scripts — git_url alone)
|
||||
git_url = https://github.com/tpope/vim-commentary.git
|
||||
|
||||
# Gitea (full URL required, or short form + base_url)
|
||||
gitea_releases = https://git.rootprojects.org/root/pathman
|
||||
|
||||
# GitLab (defaults to gitlab.com)
|
||||
gitlab_releases = owner/repo
|
||||
|
||||
# HashiCorp releases API
|
||||
hashicorp_product = terraform
|
||||
|
||||
# Custom source (servicemandist, nodedist, zigdist, etc.)
|
||||
source = nodedist
|
||||
url = https://nodejs.org/download/release
|
||||
```
|
||||
|
||||
#### Filtering, versioning, and platform
|
||||
|
||||
```ini
|
||||
tag_prefix = bun- # monorepo: strip prefix from version
|
||||
version_prefixes = jq- # strip from version string (space-separated)
|
||||
asset_filter = MinGit # filename must contain this substring
|
||||
exclude = busybox -src- -docs- # skip assets containing these (space-separated)
|
||||
os = posix_2017 # restrict ALL versions to this OS (blanket)
|
||||
alias_of = rg # mirrors another package's releases
|
||||
```
|
||||
|
||||
#### Design rules
|
||||
|
||||
- `os` is a blanket tag on ALL versions. Only use for packages that are always
|
||||
POSIX-only. For version-dependent OS tagging, use a custom `TagVariants` in
|
||||
`internal/releases/{pkg}/variants.go`.
|
||||
- `git_url` can be primary (gittag source when it's the only key) or secondary
|
||||
fallback alongside `github_sources`/`gitea_sources`.
|
||||
- Full URL forms accepted for github/gitea/gitlab (e.g.
|
||||
`github_releases = https://github.com/sharkdp/bat`).
|
||||
|
||||
### Testing
|
||||
|
||||
Test tools: `cmd/e2etest` (pipeline comparison), `cmd/comparecache` (cache diff),
|
||||
`cmd/inspect` (single-package debug). Run each with `--help` for usage.
|
||||
|
||||
### Classifier vs Per-Package Tagger
|
||||
|
||||
The general classifier (`internal/classify/`) handles patterns common across
|
||||
many projects. It MUST NOT contain one-off logic for a single package.
|
||||
|
||||
Per-package taggers (`internal/releases/{pkg}/variants.go`) handle
|
||||
project-specific knowledge. Read the existing taggers for conventions.
|
||||
|
||||
MUST: Derive arch/OS from concrete evidence — not blanket defaults.
|
||||
MUST: New general classifier patterns must apply to 2-3+ packages.
|
||||
|
||||
### Deploying
|
||||
|
||||
```sh
|
||||
./scripts/deploy-webicached.sh beta.webi.sh
|
||||
./scripts/deploy-webicached.sh next.webi.sh
|
||||
```
|
||||
|
||||
First-time setup on a new host uses `serviceman`:
|
||||
|
||||
```sh
|
||||
serviceman add --name webicached \
|
||||
--workdir ~/srv/webid/installers/ -- \
|
||||
~/bin/webicached \
|
||||
--envfile ~/srv/webid/.env.secret \
|
||||
--conf ~/srv/webid/installers/ \
|
||||
--raw ~/.cache/webi/raw
|
||||
```
|
||||
@@ -10,6 +10,7 @@
|
||||
|
||||
- You'll be asked to make changes if you don't run the code formatters and
|
||||
linters:
|
||||
|
||||
- Node / JavaScript:
|
||||
- [prettier](https://webinstall.dev/prettier)
|
||||
```sh
|
||||
@@ -20,6 +21,7 @@
|
||||
npm run lint
|
||||
```
|
||||
- Bash
|
||||
|
||||
- [shfmt](https://webinstall.dev/shfmt)
|
||||
```sh
|
||||
npm run shfmt
|
||||
|
||||
11
README.md
11
README.md
@@ -98,10 +98,9 @@ You just fill in the blanks.
|
||||
Just create an empty directory and run the tests until you get a good result.
|
||||
|
||||
```sh
|
||||
git clone git@github.com:webinstall/webi-installers.git
|
||||
pushd ./webi-installers/
|
||||
git submodule update --init
|
||||
npm clean-install
|
||||
git clone git@github.com:webinstall/packages.git
|
||||
pushd packages
|
||||
npm install
|
||||
```
|
||||
|
||||
```sh
|
||||
@@ -145,8 +144,8 @@ It looks like this:
|
||||
`releases.js`:
|
||||
|
||||
```js
|
||||
module.exports = function () {
|
||||
return github(null, owner, repo).then(function (all) {
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
// if you need to do something special, you can do it here
|
||||
// ...
|
||||
return all;
|
||||
|
||||
64
_common/brew.js
Normal file
64
_common/brew.js
Normal file
@@ -0,0 +1,64 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Gets a releases from 'brew'.
|
||||
*
|
||||
* @param request
|
||||
* @param {string} formula
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
function getAllReleases(request, formula) {
|
||||
if (!formula) {
|
||||
return Promise.reject('missing formula for brew');
|
||||
}
|
||||
return request({
|
||||
url: 'https://formulae.brew.sh/api/formula/' + formula + '.json',
|
||||
fail: true, // https://git.coolaj86.com/coolaj86/request.js/issues/2
|
||||
json: true,
|
||||
})
|
||||
.then(failOnBadStatus)
|
||||
.then(function (resp) {
|
||||
var ver = resp.body.versions.stable;
|
||||
var dl = (
|
||||
resp.body.bottle.stable.files.high_sierra ||
|
||||
resp.body.bottle.stable.files.catalina
|
||||
).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
|
||||
return [
|
||||
{
|
||||
version: ver,
|
||||
download: dl.replace(/{{ v }}/g, ver),
|
||||
},
|
||||
].concat(
|
||||
resp.body.versioned_formulae.map(function (f) {
|
||||
var ver = f.replace(/.*@/, '');
|
||||
return {
|
||||
version: ver,
|
||||
download: dl,
|
||||
};
|
||||
}),
|
||||
);
|
||||
})
|
||||
.catch(function (err) {
|
||||
console.error('Error fetching MariaDB versions (brew)');
|
||||
console.error(err);
|
||||
return [];
|
||||
});
|
||||
}
|
||||
|
||||
function failOnBadStatus(resp) {
|
||||
if (resp.statusCode >= 400) {
|
||||
var err = new Error('Non-successful status code: ' + resp.statusCode);
|
||||
err.code = 'ESTATUS';
|
||||
err.response = resp;
|
||||
throw err;
|
||||
}
|
||||
return resp;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(require('@root/request'), 'mariadb').then(function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
196
_common/git-tag.js
Normal file
196
_common/git-tag.js
Normal file
@@ -0,0 +1,196 @@
|
||||
'use strict';
|
||||
|
||||
require('dotenv').config({ path: '.env' });
|
||||
|
||||
var Crypto = require('crypto');
|
||||
var util = require('util');
|
||||
var exec = util.promisify(require('child_process').exec);
|
||||
var Fs = require('node:fs/promises');
|
||||
var Path = require('node:path');
|
||||
|
||||
var repoBaseDir = process.env.REPO_BASE_DIR || '';
|
||||
if (!repoBaseDir) {
|
||||
repoBaseDir = Path.resolve('./repos');
|
||||
// for stderr
|
||||
console.error(`[Warn] REPO_BASE_DIR= not set, ${repoBaseDir}`);
|
||||
}
|
||||
|
||||
var Repos = {};
|
||||
|
||||
Repos.clone = async function (repoPath, gitUrl) {
|
||||
let uuid = Crypto.randomUUID();
|
||||
let tmpPath = `${repoPath}.${uuid}.tmp`;
|
||||
await exec(`git clone --bare --filter=tree:0 ${gitUrl} ${tmpPath}`);
|
||||
await Fs.rename(tmpPath, repoPath);
|
||||
};
|
||||
|
||||
Repos.checkExists = async function (repoPath) {
|
||||
let err = await Fs.access(repoPath).catch(Object);
|
||||
if (!err) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if (err.code !== 'ENOENT') {
|
||||
throw err;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
Repos.fetch = async function (repoPath) {
|
||||
await exec(`git --git-dir=${repoPath} fetch`);
|
||||
};
|
||||
|
||||
Repos.getTags = async function (repoPath) {
|
||||
var { stdout } = await exec(`git --git-dir=${repoPath} tag`);
|
||||
var rawTags = stdout.trim().split('\n');
|
||||
|
||||
let tags = [];
|
||||
for (let tag of rawTags) {
|
||||
// ex: v1, v2, v1.1, 1.1.0-rc
|
||||
let maybeVersionRe = /^(v\d+|v?\d+\.\d+)/;
|
||||
let maybeVersion = maybeVersionRe.test(tag);
|
||||
if (maybeVersion) {
|
||||
tags.push(tag);
|
||||
}
|
||||
}
|
||||
|
||||
tags = tags.reverse();
|
||||
return tags;
|
||||
};
|
||||
|
||||
Repos.getTipInfo = async function (repoPath) {
|
||||
var { stdout } = await exec(
|
||||
`git --git-dir=${repoPath} rev-parse --abbrev-ref HEAD`,
|
||||
);
|
||||
var branch = stdout.trim();
|
||||
|
||||
var info = await Repos.getCommitInfo(repoPath, 'HEAD');
|
||||
info.commitish = branch;
|
||||
|
||||
return info;
|
||||
};
|
||||
|
||||
Repos.getCommitInfo = async function (repoPath, commitish) {
|
||||
var { stdout } = await exec(
|
||||
`git --git-dir=${repoPath} log -1 --format="%h %H %ad %cd" --date=iso-strict ${commitish}`,
|
||||
);
|
||||
stdout = stdout.trim();
|
||||
var commitParts = stdout.split(/\s+/g);
|
||||
return {
|
||||
commitish: commitish,
|
||||
commit_id: `${commitParts[0]}`,
|
||||
commit: `${commitParts[1]}`,
|
||||
date: commitParts[2],
|
||||
date_authored: commitParts[3],
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Lists GitHub Releases (w/ uploaded assets)
|
||||
*
|
||||
* @param request
|
||||
* @param {string} owner
|
||||
* @param {string} gitUrl
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
async function getAllReleases(gitUrl) {
|
||||
let all = {
|
||||
releases: [],
|
||||
download: '',
|
||||
};
|
||||
|
||||
let repoName = gitUrl.split('/').pop();
|
||||
repoName = repoName.replace(/\.git$/, '');
|
||||
|
||||
let repoPath = `${repoBaseDir}/${repoName}.git`;
|
||||
|
||||
let isCloned = await Repos.checkExists(repoPath);
|
||||
if (!isCloned) {
|
||||
await Repos.clone(repoPath, gitUrl);
|
||||
} else {
|
||||
await Repos.fetch(repoPath);
|
||||
}
|
||||
|
||||
let commitInfos = [];
|
||||
let tags = await Repos.getTags(repoPath);
|
||||
for (let tag of tags) {
|
||||
let commitInfo = await Repos.getCommitInfo(repoPath, tag);
|
||||
Object.assign(commitInfo, { version: tag, channel: '' });
|
||||
commitInfos.push(commitInfo);
|
||||
}
|
||||
|
||||
{
|
||||
let tipInfo = await Repos.getTipInfo(repoPath);
|
||||
// "2024-01-01T00:00:00-05:00" => "2024-01-01T05:00:00"
|
||||
let date = new Date(tipInfo.date);
|
||||
// "2024-01-01T05:00:00" => "v2024.01.01-05.00.00"
|
||||
let version = date.toISOString();
|
||||
// strip '.000Z'
|
||||
version = version.replace(/\.\d+Z/, '');
|
||||
version = version.replace(/[:\-]/g, '.');
|
||||
version = version.replace(/T/, '-');
|
||||
Object.assign(tipInfo, { version: `v${version}`, channel: '' });
|
||||
if (commitInfos.length > 1) {
|
||||
tipInfo.channel = 'beta';
|
||||
}
|
||||
commitInfos.push(tipInfo);
|
||||
}
|
||||
|
||||
let releases = [];
|
||||
for (let commitInfo of commitInfos) {
|
||||
let version = commitInfo.version.replace(/^v/, '');
|
||||
|
||||
let date = new Date(commitInfo.date);
|
||||
let isoDate = date.toISOString();
|
||||
isoDate = isoDate.replace(/\.\d+Z/, '');
|
||||
|
||||
// tags and HEAD qualify for '--branch <branchish>'
|
||||
let branch = commitInfo.commitish;
|
||||
|
||||
let rel = {
|
||||
name: `${repoName}-v${version}`,
|
||||
version: version,
|
||||
git_tag: branch,
|
||||
git_commit_hash: commitInfo.commit_id,
|
||||
lts: false,
|
||||
channel: commitInfo.channel,
|
||||
date: isoDate,
|
||||
os: '*',
|
||||
arch: '*',
|
||||
ext: 'git',
|
||||
download: gitUrl,
|
||||
};
|
||||
|
||||
releases.push(rel);
|
||||
}
|
||||
all.releases = releases;
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
(async function main() {
|
||||
let testRepos = [
|
||||
// just a few tags, and a different HEAD
|
||||
'https://github.com/tpope/vim-commentary.git',
|
||||
// no tags, just HEAD
|
||||
'https://github.com/ziglang/zig.vim.git',
|
||||
// many, many tags
|
||||
//'https://github.com/dense-analysis/ale.git',
|
||||
];
|
||||
for (let url of testRepos) {
|
||||
let all = await getAllReleases(url);
|
||||
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
}
|
||||
})()
|
||||
.then(function () {
|
||||
process.exit(0);
|
||||
})
|
||||
.catch(function (err) {
|
||||
console.error(err);
|
||||
});
|
||||
}
|
||||
39
_common/gitea.js
Normal file
39
_common/gitea.js
Normal file
@@ -0,0 +1,39 @@
|
||||
'use strict';
|
||||
|
||||
var ghRelease = require('./github.js');
|
||||
|
||||
/**
|
||||
* Gets the releases for 'ripgrep'. This function could be trimmed down and made
|
||||
* for use with any github release.
|
||||
*
|
||||
* @param request
|
||||
* @param {string} owner
|
||||
* @param {string} repo
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
function getAllReleases(request, owner, repo, baseurl) {
|
||||
if (!baseurl) {
|
||||
return Promise.reject('missing baseurl');
|
||||
}
|
||||
return ghRelease(request, owner, repo, baseurl + '/api/v1').then(
|
||||
function (all) {
|
||||
return all;
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(
|
||||
require('@root/request'),
|
||||
'coolaj86',
|
||||
'go-pathman',
|
||||
'https://git.coolaj86.com',
|
||||
).then(
|
||||
//getAllReleases(require('@root/request'), 'root', 'serviceman', 'https://git.rootprojects.org').then(
|
||||
function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
},
|
||||
);
|
||||
}
|
||||
133
_common/github-source.js
Normal file
133
_common/github-source.js
Normal file
@@ -0,0 +1,133 @@
|
||||
'use strict';
|
||||
|
||||
require('dotenv').config();
|
||||
|
||||
/**
|
||||
* Gets the releases for 'ripgrep'. This function could be trimmed down and made
|
||||
* for use with any github release.
|
||||
*
|
||||
* @param request
|
||||
* @param {string} owner
|
||||
* @param {string} repo
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
async function getAllReleases(
|
||||
request,
|
||||
owner,
|
||||
repo,
|
||||
oses,
|
||||
arches,
|
||||
baseurl = 'https://api.github.com',
|
||||
) {
|
||||
if (!owner) {
|
||||
return Promise.reject('missing owner for repo');
|
||||
}
|
||||
if (!repo) {
|
||||
return Promise.reject('missing repo name');
|
||||
}
|
||||
|
||||
let req = {
|
||||
url: `${baseurl}/repos/${owner}/${repo}/releases`,
|
||||
json: true,
|
||||
};
|
||||
|
||||
// TODO I really don't like global config, find a way to do better
|
||||
if (process.env.GITHUB_USERNAME) {
|
||||
req.auth = {
|
||||
user: process.env.GITHUB_USERNAME,
|
||||
pass: process.env.GITHUB_TOKEN,
|
||||
};
|
||||
}
|
||||
|
||||
let resp = await request(req);
|
||||
let gHubResp = resp.body;
|
||||
let all = {
|
||||
releases: [],
|
||||
// TODO make this ':baseurl' + ':releasename'
|
||||
download: '',
|
||||
};
|
||||
|
||||
for (let release of gHubResp) {
|
||||
// TODO tags aren't always semver / sensical
|
||||
let tag = release['tag_name'];
|
||||
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
|
||||
let channel = 'stable';
|
||||
if (release['prerelease']) {
|
||||
channel = 'beta';
|
||||
}
|
||||
let date = release['published_at'] || '';
|
||||
date = date.replace(/T.*/, '');
|
||||
|
||||
let urls = [release.tarball_url, release.zipball_url];
|
||||
for (let url of urls) {
|
||||
let resp = await request({
|
||||
method: 'HEAD',
|
||||
followRedirect: true,
|
||||
followAllRedirects: true,
|
||||
followOriginalHttpMethod: true,
|
||||
url: url,
|
||||
stream: true,
|
||||
});
|
||||
// Workaround for bug where method changes to GET
|
||||
resp.destroy();
|
||||
|
||||
// content-disposition: attachment; filename=BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
|
||||
let name = resp.headers['content-disposition'].replace(
|
||||
/.*filename=([^;]+)(;|$)/,
|
||||
'$1',
|
||||
);
|
||||
all.releases.push({
|
||||
name: name,
|
||||
version: tag,
|
||||
lts: lts,
|
||||
channel: channel,
|
||||
date: date,
|
||||
os: '', // will be guessed by download filename
|
||||
arch: '', // will be guessed by download filename
|
||||
ext: '', // will be normalized
|
||||
download: resp.request.uri.href,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (oses) {
|
||||
return combinate(all, oses, arches);
|
||||
}
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
function combinate(all, oses, arches) {
|
||||
let releases = all.releases;
|
||||
// ex: arches = ['amd64', 'arm64', 'armv7l', 'armv6l', 'x86'];
|
||||
// ex: oses = ['macos', 'linux', 'bsd', 'posix'];
|
||||
|
||||
let combos = [];
|
||||
for (let release of releases) {
|
||||
for (let arch of arches) {
|
||||
for (let os of oses) {
|
||||
let combo = {
|
||||
arch: arch,
|
||||
os: os,
|
||||
};
|
||||
let rel = Object.assign({}, release, combo);
|
||||
combos.push(rel);
|
||||
}
|
||||
}
|
||||
}
|
||||
all.releases = combos;
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(
|
||||
require('@root/request'),
|
||||
'BeyondCodeBootcamp',
|
||||
'DuckDNS.sh',
|
||||
).then(function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
102
_common/github.js
Normal file
102
_common/github.js
Normal file
@@ -0,0 +1,102 @@
|
||||
'use strict';
|
||||
|
||||
require('dotenv').config();
|
||||
|
||||
/**
|
||||
* Lists GitHub Releases (w/ uploaded assets)
|
||||
*
|
||||
* @param request
|
||||
* @param {string} owner
|
||||
* @param {string} repo
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
async function getAllReleases(
|
||||
request,
|
||||
owner,
|
||||
repo,
|
||||
baseurl = 'https://api.github.com',
|
||||
) {
|
||||
if (!owner) {
|
||||
throw new Error('missing owner for repo');
|
||||
}
|
||||
if (!repo) {
|
||||
throw new Error('missing repo name');
|
||||
}
|
||||
|
||||
var req = {
|
||||
url: `${baseurl}/repos/${owner}/${repo}/releases`,
|
||||
json: true,
|
||||
};
|
||||
|
||||
// TODO I really don't like global config, find a way to do better
|
||||
if (process.env.GITHUB_USERNAME) {
|
||||
req.auth = {
|
||||
user: process.env.GITHUB_USERNAME,
|
||||
pass: process.env.GITHUB_TOKEN,
|
||||
};
|
||||
}
|
||||
|
||||
let resp = await request(req);
|
||||
if (!resp.ok) {
|
||||
console.error('Bad Resp Headers:', resp.headers);
|
||||
console.error('Bad Resp Body:', resp.body);
|
||||
throw new Error('the elusive releases BOOGEYMAN strikes again');
|
||||
}
|
||||
|
||||
let gHubResp = resp.body;
|
||||
let all = {
|
||||
releases: [],
|
||||
// todo make this ':baseurl' + ':releasename'
|
||||
download: '',
|
||||
};
|
||||
|
||||
try {
|
||||
gHubResp.forEach(transformReleases);
|
||||
} catch (e) {
|
||||
console.error(e.message);
|
||||
console.error('Error Headers:', resp.headers);
|
||||
console.error('Error Body:', resp.body);
|
||||
throw e;
|
||||
}
|
||||
|
||||
function transformReleases(release) {
|
||||
for (let asset of release['assets']) {
|
||||
let name = asset['name'];
|
||||
let date = release['published_at']?.replace(/T.*/, '');
|
||||
let download = asset['browser_download_url'];
|
||||
|
||||
// TODO tags aren't always semver / sensical
|
||||
let version = release['tag_name'];
|
||||
let channel;
|
||||
if (release['prerelease']) {
|
||||
// -rcX, -preview, -beta, etc will be checked in _webi/normalize.js
|
||||
channel = 'beta';
|
||||
}
|
||||
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
|
||||
|
||||
all.releases.push({
|
||||
name: name,
|
||||
version: version,
|
||||
lts: lts,
|
||||
channel: channel,
|
||||
date: date,
|
||||
os: '', // will be guessed by download filename
|
||||
arch: '', // will be guessed by download filename
|
||||
ext: '', // will be normalized
|
||||
download: download,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(require('@root/request'), 'BurntSushi', 'ripgrep').then(
|
||||
function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -23,7 +23,7 @@ install:
|
||||
```text
|
||||
~/.config/envman/PATH.env
|
||||
~/.local/bin/foo
|
||||
~/.local/opt/foo/
|
||||
~/.local/opt/foo
|
||||
```
|
||||
|
||||
## Cheat Sheet
|
||||
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading foobar from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing foobar"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
# Example releases.conf — uses ripgrep as a sample project.
|
||||
# Copy this file into your package directory and adjust.
|
||||
github_releases = BurntSushi/ripgrep
|
||||
28
_example/releases.js
Normal file
28
_example/releases.js
Normal file
@@ -0,0 +1,28 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'BurntSushi';
|
||||
var repo = 'ripgrep';
|
||||
|
||||
/******************************************************************************/
|
||||
/** Note: Delete this Comment! **/
|
||||
/** **/
|
||||
/** Need a an example that filters out miscellaneous release files? **/
|
||||
/** See `deno`, `gitea`, or `caddy` **/
|
||||
/** **/
|
||||
/******************************************************************************/
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -4,6 +4,7 @@
|
||||
|
||||
//var pkg = require('../package.json');
|
||||
var os = require('os');
|
||||
//var request = require('@root/request');
|
||||
//var promisify = require('util').promisify;
|
||||
//var exec = promisify(require('child_process').exec);
|
||||
var exec = require('child_process').exec;
|
||||
|
||||
@@ -15,8 +15,8 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
|
||||
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
|
||||
|
||||
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-Not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
|
||||
continue
|
||||
}
|
||||
@@ -31,7 +31,7 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_new_file | Set-Content -Path $my_ps1
|
||||
|
||||
$my_new_file = $my_new_file + "`n"
|
||||
if ($text -ne $my_new_file) {
|
||||
IF ($text -ne $my_new_file) {
|
||||
$my_status = 1
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,8 +15,8 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
|
||||
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
|
||||
|
||||
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-Not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
|
||||
continue
|
||||
}
|
||||
@@ -39,7 +39,7 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_new_file | Set-Content -Path $my_ps1
|
||||
|
||||
$my_new_file = $my_new_file + "`n"
|
||||
if ($my_old_file -ne $my_new_file) {
|
||||
IF ($my_old_file -ne $my_new_file) {
|
||||
$my_status = 1
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env pwsh
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
|
||||
New-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start" -ItemType Directory -Force | Out-Null
|
||||
}
|
||||
Remove-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start\example" -Recurse -ErrorAction Ignore
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
############################################################
|
||||
New-Item -Path "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
|
||||
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
|
||||
IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/_webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
|
||||
Set-ExecutionPolicy -Scope Process Bypass
|
||||
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" "{{ exename }}"
|
||||
359
_webi/bootstrap.sh
Normal file
359
_webi/bootstrap.sh
Normal file
@@ -0,0 +1,359 @@
|
||||
#!/bin/sh
|
||||
#<pre>
|
||||
|
||||
############################################################
|
||||
# <h1>Cheat Sheet at CHEATSHEET_URL</h1>
|
||||
# <meta http-equiv="refresh" content="3; URL='CHEATSHEET_URL'" />
|
||||
############################################################
|
||||
|
||||
#set -x
|
||||
|
||||
__install_webi() {
|
||||
|
||||
#WEBI_PKG=
|
||||
#WEBI_HOST=https://webinstall.dev
|
||||
export WEBI_HOST
|
||||
|
||||
my_libc=''
|
||||
if ldd /bin/ls 2> /dev/null | grep -q 'musl' 2> /dev/null; then
|
||||
my_libc='musl'
|
||||
elif uname -o | grep -q 'GNU' || uname -s | grep -q 'Linux'; then
|
||||
my_libc='gnu'
|
||||
else
|
||||
my_libc='libc'
|
||||
fi
|
||||
|
||||
if test -z "$WEBI_WELCOME"; then
|
||||
echo ""
|
||||
printf "Thanks for using webi to install '\e[32m%s\e[0m' on '\e[33m%s (%s) %s\e[0m'.\n" "${WEBI_PKG:-"Unknown Package"}" "$(uname -s)" "${my_libc:-"Unknown Libc"}" "$(uname -m)"
|
||||
echo "Have a problem? Experience a bug? Please let us know:"
|
||||
printf " \e[2m\e[36mhttps://github.com/webinstall/webi-installers/issues\e[0m\n"
|
||||
echo ""
|
||||
printf "\e[35mLovin'\e[0m it? Say thanks with a \e[1m\e[33mStar on GitHub\e[0m:\n"
|
||||
printf " \e[36mhttps://github.com/webinstall/webi-installers\e[0m\n"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
WEBI_WELCOME=true
|
||||
export WEBI_WELCOME
|
||||
|
||||
set -e
|
||||
set -u
|
||||
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
|
||||
cat << EOF > "$HOME/.local/bin/webi"
|
||||
#!/bin/sh
|
||||
|
||||
set -e
|
||||
set -u
|
||||
#set -x
|
||||
|
||||
__webi_main() {
|
||||
|
||||
my_date="\$(date +%F_%H-%M-%S)"
|
||||
export WEBI_TIMESTAMP="\${my_date}"
|
||||
export _webi_tmp="\${_webi_tmp:-\$(
|
||||
mktemp -d -t "webi-\$WEBI_TIMESTAMP.XXXXXXXX"
|
||||
)}"
|
||||
|
||||
if [ -n "\${_WEBI_PARENT:-}" ]; then
|
||||
export _WEBI_CHILD=true
|
||||
else
|
||||
export _WEBI_CHILD=
|
||||
fi
|
||||
export _WEBI_PARENT=true
|
||||
|
||||
my_os="\$(uname -s)"
|
||||
my_arch="\$(uname -m)"
|
||||
|
||||
##
|
||||
## Detect acceptable package formats
|
||||
##
|
||||
|
||||
my_ext=""
|
||||
set +e
|
||||
# NOTE: the order here is least favorable to most favorable
|
||||
if [ -n "\$(command -v pkgutil)" ]; then
|
||||
my_ext="pkg,\$my_ext"
|
||||
fi
|
||||
# disable this check for the sake of building the macOS installer on Linux
|
||||
#if [ -n "\$(command -v diskutil)" ]; then
|
||||
# note: could also detect via hdiutil
|
||||
my_ext="dmg,\$my_ext"
|
||||
#fi
|
||||
if [ -n "\$(command -v git)" ]; then
|
||||
my_ext="git,\$my_ext"
|
||||
fi
|
||||
if [ -n "\$(command -v unxz)" ]; then
|
||||
my_ext="xz,\$my_ext"
|
||||
fi
|
||||
if [ -n "\$(command -v unzip)" ]; then
|
||||
my_ext="zip,\$my_ext"
|
||||
fi
|
||||
# for mac/linux 'exe' refers to the uncompressed binary without extension
|
||||
my_ext="exe,\$my_ext"
|
||||
if [ -n "\$(command -v tar)" ]; then
|
||||
my_ext="tar,\$my_ext"
|
||||
fi
|
||||
my_ext="\$(echo "\$my_ext" | sed 's/,$//')" # nix trailing comma
|
||||
set -e
|
||||
|
||||
|
||||
##
|
||||
## Detect http client
|
||||
##
|
||||
|
||||
set +e
|
||||
WEBI_CURL="\$(command -v curl)"
|
||||
export WEBI_URL
|
||||
WEBI_WGET="\$(command -v wget)"
|
||||
export WEBI_WGET
|
||||
set -e
|
||||
|
||||
my_libc=''
|
||||
if ldd /bin/ls 2> /dev/null | grep -q 'musl' 2> /dev/null; then
|
||||
my_libc='musl'
|
||||
elif uname -o | grep -q 'GNU' || uname -s | grep -q 'Linux'; then
|
||||
my_libc='gnu'
|
||||
else
|
||||
my_libc='libc'
|
||||
fi
|
||||
|
||||
export WEBI_HOST="\${WEBI_HOST:-https://webinstall.dev}"
|
||||
|
||||
# ex: Darwin or Linux
|
||||
my_sys="\$(uname -s)"
|
||||
# ex: 22.6.0
|
||||
my_rev="\$(uname -r)"
|
||||
# ex: arm64
|
||||
my_machine="\$(uname -m)"
|
||||
|
||||
export WEBI_UA="\${my_sys}/\${my_rev} \${my_machine}/unknown \${my_libc}"
|
||||
|
||||
webinstall() {
|
||||
|
||||
my_package="\${1:-}"
|
||||
if [ -z "\$my_package" ]; then
|
||||
echo >&2 "Usage: webi <package>@<version> ..."
|
||||
echo >&2 "Example: webi node@lts rg"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
WEBI_BOOT="\$(
|
||||
mktemp -d -t "\$my_package-bootstrap.\$WEBI_TIMESTAMP.XXXXXXXX"
|
||||
)"
|
||||
export WEBI_BOOT
|
||||
|
||||
my_installer_url="\$WEBI_HOST/api/installers/\$my_package.sh?formats=\$my_ext"
|
||||
if [ -n "\$WEBI_CURL" ]; then
|
||||
if ! curl -fsSL "\$my_installer_url" -H "User-Agent: curl \$WEBI_UA" \\
|
||||
-o "\$WEBI_BOOT/\$my_package-bootstrap.sh"; then
|
||||
echo >&2 "error fetching '\$my_installer_url'"
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
if ! wget -q "\$my_installer_url" --user-agent="wget \$WEBI_UA" \\
|
||||
-O "\$WEBI_BOOT/\$my_package-bootstrap.sh"; then
|
||||
echo >&2 "error fetching '\$my_installer_url'"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
(
|
||||
cd "\$WEBI_BOOT"
|
||||
sh "\$my_package-bootstrap.sh"
|
||||
)
|
||||
|
||||
rm -rf "\$WEBI_BOOT"
|
||||
|
||||
}
|
||||
|
||||
show_path_updates() {
|
||||
|
||||
if test -z "\${_WEBI_CHILD}"; then
|
||||
if test -f "\$_webi_tmp/.PATH.env"; then
|
||||
my_paths=\$(sort -u < "\$_webi_tmp/.PATH.env")
|
||||
if test -n "\$my_paths"; then
|
||||
printf 'PATH.env updated with:\\n'
|
||||
printf "%s\\n" "\$my_paths"
|
||||
printf '\\n'
|
||||
printf "\\e[1m\\e[35mTO FINISH\\e[0m: copy, paste & run the following command:\\n"
|
||||
printf "\\n"
|
||||
printf " \\e[1m\\e[32msource ~/.config/envman/PATH.env\\e[0m\\n"
|
||||
printf " (newly opened terminal windows will update automatically)\\n"
|
||||
fi
|
||||
rm -f "\$_webi_tmp/.PATH.env"
|
||||
fi
|
||||
fi
|
||||
|
||||
}
|
||||
|
||||
fn_checksum() {
|
||||
cmd_shasum='sha1sum'
|
||||
if command -v shasum > /dev/null; then
|
||||
cmd_shasum='shasum'
|
||||
fi
|
||||
\$cmd_shasum "\${0}" | cut -d' ' -f1 | cut -c 1-8
|
||||
}
|
||||
|
||||
version() {
|
||||
my_checksum="\$(
|
||||
fn_checksum
|
||||
)"
|
||||
my_version=v1.2.0
|
||||
printf "\\e[35mwebi\\e[32m %s\\e[0m Copyright 2020+ AJ ONeal\\n" "\${my_version} (\${my_checksum})"
|
||||
printf " \\e[36mhttps://webinstall.dev/webi\\e[0m\\n"
|
||||
}
|
||||
|
||||
# show help if no params given or help flags are used
|
||||
usage() {
|
||||
echo ""
|
||||
version
|
||||
echo ""
|
||||
|
||||
printf "\\e[1mSUMMARY\\e[0m\\n"
|
||||
echo " Webi is the best way to install the modern developer tools you love."
|
||||
echo " It's fast, easy-to-remember, and conflict free."
|
||||
echo ""
|
||||
printf "\\e[1mUSAGE\\e[0m\\n"
|
||||
echo " webi <thing1>[@version] [thing2] ..."
|
||||
echo ""
|
||||
printf "\\e[1mUNINSTALL\\e[0m\\n"
|
||||
echo " Almost everything that is installed with webi is scoped to"
|
||||
echo " ~/.local/opt/<thing1>, so you can remove it like so:"
|
||||
echo ""
|
||||
echo " rm -rf ~/.local/opt/<thing1>"
|
||||
echo " rm -f ~/.local/bin/<thing1>"
|
||||
echo ""
|
||||
echo " Some packages have special uninstall instructions, check"
|
||||
echo " https://webinstall.dev/<thing1> to be sure."
|
||||
echo ""
|
||||
printf "\\e[1mOPTIONS\\e[0m\\n"
|
||||
echo " Generic Program Information"
|
||||
echo " --help Output a usage message and exit."
|
||||
echo ""
|
||||
echo " -V, --version"
|
||||
echo " Output the version number of webi and exit."
|
||||
echo ""
|
||||
echo " Helper Utilities"
|
||||
echo " --list Show everything webi has to offer."
|
||||
echo ""
|
||||
echo " --info <package>"
|
||||
echo " Show various links and example release."
|
||||
echo ""
|
||||
printf "\\e[1mFAQ\\e[0m\\n"
|
||||
printf " See \\e[34mhttps://webinstall.dev/faq\\e[0m\\n"
|
||||
echo ""
|
||||
printf "\\e[1mALWAYS REMEMBER\\e[0m\\n"
|
||||
echo " Friends don't let friends use brew for simple, modern tools that don't need it."
|
||||
echo " (and certainly not apt either **shudder**)"
|
||||
echo ""
|
||||
}
|
||||
|
||||
if [ \$# -eq 0 ] || echo "\$1" | grep -q -E '^(-V|--version|version)$'; then
|
||||
version
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if echo "\$1" | grep -q -E '^(-h|--help|help)$'; then
|
||||
usage "\$@"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if echo "\$1" | grep -q -E '^(-l|--list|list)$'; then
|
||||
echo >&2 "[warn] the format of --list output may change"
|
||||
|
||||
# because we don't have sitemap.xml for dev sites yet
|
||||
my_host="https://webinstall.dev"
|
||||
my_len="\${#my_host}"
|
||||
|
||||
# 6 because the field will looks like "loc>WEBI_HOST/PKG_NAME"
|
||||
# and the count is 1-indexed
|
||||
my_count="\$((my_len + 6))"
|
||||
|
||||
curl -fsS "\${my_host}/sitemap.xml" |
|
||||
grep -F "\${my_host}" |
|
||||
cut -d'<' -f2 |
|
||||
cut -c "\${my_count}"-
|
||||
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if echo "\${1}" | grep -q -E '^(--info|info)$'; then
|
||||
if test -z "\${2}"; then
|
||||
echo >&2 "Usage: webi --info <package>"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo >&2 "[warn] the output of --info is completely half-baked and will change"
|
||||
my_pkg="\${2}"
|
||||
# TODO need a way to check that it exists at all (readme, win, lin)
|
||||
echo ""
|
||||
echo " Cheat Sheet: \${WEBI_HOST}/\${my_pkg}"
|
||||
echo " POSIX: curl -sS \${WEBI_HOST}/\${my_pkg} | sh"
|
||||
echo " Windows: curl.exe -A MS \${WEBI_HOST}/\${my_pkg} | powershell"
|
||||
echo "Releases (JSON): \${WEBI_HOST}/api/releases/\${my_pkg}.json"
|
||||
echo " Releases (tsv): \${WEBI_HOST}/api/releases/\${my_pkg}.tab"
|
||||
echo " (query params): ?channel=stable&limit=10"
|
||||
echo " &os=\${my_os}&arch=\${my_arch}"
|
||||
echo " Install Script: \${WEBI_HOST}/api/installers/\${my_pkg}.sh?formats=tar,zip,xz,git,dmg,pkg"
|
||||
echo " Static Assets: \${WEBI_HOST}/packages/\${my_pkg}/README.md"
|
||||
echo ""
|
||||
|
||||
# TODO os=linux,macos,windows (limit to tagged releases)
|
||||
my_releases="\$(
|
||||
curl -fsS "\${WEBI_HOST}/api/releases/\${my_pkg}.json?channel=stable&limit=1&pretty=true"
|
||||
)"
|
||||
|
||||
if printf '%s\\n' "\${my_releases}" | grep -q "error"; then
|
||||
my_releases_beta="\$(
|
||||
curl -fsS "\${WEBI_HOST}/api/releases/\${my_pkg}.json?&limit=1&pretty=true"
|
||||
)"
|
||||
if printf '%s\\n' "\${my_releases_beta}" | grep -q "error"; then
|
||||
echo >&2 "'\${my_pkg}' is a special case that does not have releases"
|
||||
else
|
||||
echo >&2 "ERROR no stable releases for '\${my_pkg}'!"
|
||||
fi
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo >&2 "Stable '\${my_pkg}' releases:"
|
||||
if command -v jq > /dev/null; then
|
||||
printf '%s\\n' "\${my_releases}" |
|
||||
jq
|
||||
else
|
||||
printf '%s\\n' "\${my_releases}"
|
||||
fi
|
||||
|
||||
exit 0
|
||||
fi
|
||||
|
||||
for pkgname in "\$@"; do
|
||||
webinstall "\$pkgname"
|
||||
done
|
||||
|
||||
show_path_updates
|
||||
|
||||
}
|
||||
|
||||
__webi_main "\$@"
|
||||
|
||||
EOF
|
||||
|
||||
chmod a+x "$HOME/.local/bin/webi"
|
||||
|
||||
if [ -n "${WEBI_PKG-}" ]; then
|
||||
"$HOME/.local/bin/webi" "${WEBI_PKG}"
|
||||
else
|
||||
echo ""
|
||||
echo "Hmm... no WEBI_PKG was specified. This is probably an error in the script."
|
||||
echo ""
|
||||
echo "Please open an issue with this information: Package '${WEBI_PKG-}' on '$(uname -s)/$(uname -r) $(uname -m)'"
|
||||
echo " https://github.com/webinstall/packages/issues"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
}
|
||||
|
||||
__install_webi
|
||||
Submodule _webi/build-classifier deleted from f9cc9f3e19
@@ -1,72 +0,0 @@
|
||||
'use strict';
|
||||
|
||||
let Path = require('node:path');
|
||||
|
||||
let BuildsCacher = require('./builds-cacher.js');
|
||||
// let Parallel = require('./parallel.js');
|
||||
|
||||
var INSTALLERS_DIR = Path.join(__dirname, '..');
|
||||
var CACHE_DIR = Path.join(__dirname, '../_cache');
|
||||
|
||||
async function main() {
|
||||
let bc = BuildsCacher.create({
|
||||
caches: CACHE_DIR,
|
||||
installers: INSTALLERS_DIR,
|
||||
});
|
||||
|
||||
// let dirs = await bc.getProjectsByType();
|
||||
// let projNames = Object.keys(dirs.valid);
|
||||
|
||||
let lastUpdate;
|
||||
|
||||
let projName = 'k9s';
|
||||
{
|
||||
let packages = await bc.getPackages({
|
||||
//Releases: Releases,
|
||||
name: projName,
|
||||
date: new Date(),
|
||||
});
|
||||
lastUpdate = packages.updated;
|
||||
console.info(
|
||||
`Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
|
||||
);
|
||||
}
|
||||
|
||||
console.info('Waiting 5s');
|
||||
{
|
||||
setTimeout(async function () {
|
||||
let packages = await bc.getPackages({
|
||||
//Releases: Releases,
|
||||
name: projName,
|
||||
date: new Date(),
|
||||
});
|
||||
console.info(
|
||||
`Last update for '${projName}': ${packages.updated} (${packages.releases.length} assets)`,
|
||||
);
|
||||
if (lastUpdate < packages.updated) {
|
||||
console.info(`PASS`);
|
||||
} else {
|
||||
console.info(`MAYBE fail`);
|
||||
}
|
||||
}, 5 * 1000);
|
||||
}
|
||||
|
||||
//let parallel = 25;
|
||||
//await Parallel.run(parallel, projNames, getAll);
|
||||
//async function getAll(name) {
|
||||
// void (await bc.getPackages({
|
||||
// //Releases: Releases,
|
||||
// name: name,
|
||||
// date: new Date(),
|
||||
// }));
|
||||
//}
|
||||
}
|
||||
|
||||
main()
|
||||
.then(function () {
|
||||
console.log('Done');
|
||||
})
|
||||
.catch(function (e) {
|
||||
console.error(e.stack || e);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,907 +0,0 @@
|
||||
'use strict';
|
||||
|
||||
var BuildsCacher = module.exports;
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Os = require('node:os');
|
||||
let Path = require('node:path');
|
||||
|
||||
let LEGACY_CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
|
||||
let HostTargets = require('./build-classifier/host-targets.js');
|
||||
let Lexver = require('./build-classifier/lexver.js');
|
||||
let Triplet = require('./build-classifier/triplet.js');
|
||||
|
||||
var ALIAS_RE = /^alias: ([\w.-]+)$/m;
|
||||
|
||||
var LEGACY_ARCH_MAP = {
|
||||
'*': 'ANYARCH',
|
||||
arm64: 'aarch64',
|
||||
armv6l: 'armv6',
|
||||
armv7l: 'armv7',
|
||||
amd64: 'x86_64',
|
||||
mipsle: 'mipsel',
|
||||
mips64le: 'mips64el',
|
||||
mipsr6le: 'mipsr6el',
|
||||
mips64r6le: 'mips64r6el',
|
||||
// yes... el for arm and mips, but le for ppc
|
||||
// (perhaps the joke got old?)
|
||||
ppc64el: 'ppc64le',
|
||||
386: 'x86',
|
||||
};
|
||||
var LEGACY_OS_MAP = {
|
||||
'*': 'ANYOS',
|
||||
macos: 'darwin',
|
||||
posix: 'posix_2017',
|
||||
};
|
||||
|
||||
var TERMS_META = [
|
||||
// pattern
|
||||
'{ARCH}',
|
||||
'{EXT}',
|
||||
'{LIBC}',
|
||||
'{NAME}',
|
||||
'{OS}',
|
||||
'{VENDOR}',
|
||||
// // os-/arch-indepedent
|
||||
// 'ANYARCH',
|
||||
// 'ANYOS',
|
||||
// // libc
|
||||
// 'none',
|
||||
// channel
|
||||
'beta',
|
||||
'dev',
|
||||
'preview',
|
||||
'stable',
|
||||
];
|
||||
|
||||
/** @typedef {String} TripletString - {arch}-{vendor}-{os}-{libc} */
|
||||
/** @typedef {String} VersionString */
|
||||
/** @typedef {Object.<VersionString, Array<BuildAsset>>} PackagesByRelease */
|
||||
|
||||
/**
|
||||
* @typedef ProjectInfo
|
||||
* @prop {Array<BuildAsset>} releases
|
||||
* @prop {Array<BuildAsset>} packages
|
||||
* @prop {Object.<TripletString, PackagesByRelease>} releasesByTriplet
|
||||
* @prop {Array<import('./build-classifier/types.js').ArchString>} arches
|
||||
* @prop {Array<import('./build-classifier/types.js').OsString>} oses
|
||||
* @prop {Array<import('./build-classifier/types.js').LibcString>} libcs
|
||||
* @prop {Array<String>} channels
|
||||
* @prop {Array<String>} formats
|
||||
* @prop {Array<String>} triplets
|
||||
* @prop {Array<String>} versions
|
||||
* @prop {Array<String>} lexvers
|
||||
* @prop {Object.<String, String>} lexversMap
|
||||
*/
|
||||
|
||||
/**
|
||||
* @typedef BuildAsset
|
||||
* @prop {String} name
|
||||
* @prop {String} version
|
||||
* @prop {Boolean} lts
|
||||
* @prop {String} date
|
||||
* @prop {String} arch
|
||||
* @prop {String} os
|
||||
* @prop {String} libc
|
||||
* @prop {String} ext
|
||||
* @prop {String} download
|
||||
*/
|
||||
|
||||
/**
|
||||
* @typedef VersionTarget
|
||||
* @prop {String} version
|
||||
* @prop {Boolean} lts
|
||||
* @prop {String} channel
|
||||
*/
|
||||
|
||||
/** @typedef {TargetTriplet & HostTargetPartial} HostTarget */
|
||||
/** @typedef {import('./build-classifier/types.js').TargetTriplet} TargetTriplet */
|
||||
/**
|
||||
* @typedef HostTargetPartial
|
||||
* @prop {String} target.triplet - os-vendor-arch-libc
|
||||
* @prop {Error} target.error
|
||||
*/
|
||||
|
||||
async function getPartialHeader(path) {
|
||||
let readme = `${path}/README.md`;
|
||||
let head = await readFirstBytes(readme).catch(function (err) {
|
||||
if (err.code !== 'ENOENT') {
|
||||
console.warn(`warn: ${path}: ${err.message}`);
|
||||
}
|
||||
return null;
|
||||
});
|
||||
|
||||
return head;
|
||||
}
|
||||
|
||||
// let fsOpen = util.promisify(Fs.open);
|
||||
// let fsRead = util.promisify(Fs.read);
|
||||
async function readFirstBytes(path) {
|
||||
let start = 0;
|
||||
let n = 1024;
|
||||
let fh = await Fs.open(path, 'r');
|
||||
let buf = Buffer.alloc(n);
|
||||
let result = await fh.read(buf, start, n);
|
||||
let str = result.buffer.toString('utf8');
|
||||
await fh.close();
|
||||
|
||||
return str;
|
||||
}
|
||||
|
||||
BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
let installersDir = installers;
|
||||
|
||||
if (!ALL_TERMS) {
|
||||
ALL_TERMS = Triplet.TERMS_PRIMARY_MAP;
|
||||
}
|
||||
|
||||
let bc = {};
|
||||
bc.ALL_TERMS = ALL_TERMS;
|
||||
bc.orphanTerms = Object.assign({}, bc.ALL_TERMS);
|
||||
bc.unknownTerms = {};
|
||||
bc.usedTerms = {};
|
||||
bc.formats = [];
|
||||
bc._triplets = {};
|
||||
bc._targetsByBuildIdCache = {};
|
||||
bc._caches = {};
|
||||
bc._allFormats = {};
|
||||
bc._allTriplets = {};
|
||||
// Per-name lock: serializes cold-cache getPackages so concurrent
|
||||
// callers can't corrupt bc._caches[name] via a transformAndUpdate race.
|
||||
bc._inflight = {};
|
||||
|
||||
for (let term of TERMS_META) {
|
||||
delete bc.orphanTerms[term];
|
||||
}
|
||||
|
||||
bc.getProjectsByType = async function () {
|
||||
let dirs = {
|
||||
hidden: {},
|
||||
errors: {},
|
||||
alias: {},
|
||||
invalid: {},
|
||||
selfhosted: {},
|
||||
valid: {},
|
||||
};
|
||||
|
||||
let entries = await Fs.readdir(installersDir, { withFileTypes: true });
|
||||
for (let entry of entries) {
|
||||
let meta = await bc.getProjectTypeByEntry(entry);
|
||||
dirs[meta.type][entry.name] = meta.detail;
|
||||
}
|
||||
|
||||
return dirs;
|
||||
};
|
||||
|
||||
/**
|
||||
* Get project type and detail - alias, selfhosted, valid (and the invalids)
|
||||
* @param {String} name - filename
|
||||
*/
|
||||
bc.getProjectType = async function (name) {
|
||||
let filepath = Path.join(installersDir, name);
|
||||
let entry;
|
||||
try {
|
||||
entry = await Fs.lstat(filepath);
|
||||
Object.assign(entry, { name: name });
|
||||
} catch (e) {
|
||||
return { type: 'errors', detail: 'not found' };
|
||||
}
|
||||
let info = await bc.getProjectTypeByEntry(entry);
|
||||
|
||||
return info;
|
||||
};
|
||||
|
||||
/**
|
||||
* Get project type and detail - alias, selfhosted, valid (and the invalids)
|
||||
* @param {fs.Stats|fs.Dirent} entry
|
||||
*/
|
||||
bc.getProjectTypeByEntry = async function (entry) {
|
||||
let path = Path.join(installersDir, entry.name);
|
||||
|
||||
// skip non-installer dirs
|
||||
if (entry.isSymbolicLink()) {
|
||||
let link = await Fs.readlink(path);
|
||||
return { type: 'alias', detail: link };
|
||||
}
|
||||
|
||||
if (!entry.isDirectory()) {
|
||||
return { type: 'hidden', detail: '!directory' };
|
||||
}
|
||||
if (entry.name === 'node_modules') {
|
||||
return { type: 'hidden', detail: 'node_modules' };
|
||||
}
|
||||
if (entry.name.startsWith('_')) {
|
||||
return { type: 'hidden', detail: '_*' };
|
||||
}
|
||||
if (entry.name.startsWith('.')) {
|
||||
return { type: 'hidden', detail: '.*' };
|
||||
}
|
||||
if (entry.name.startsWith('~')) {
|
||||
return { type: 'hidden', detail: '~*' };
|
||||
}
|
||||
if (entry.name.endsWith('~')) {
|
||||
return { type: 'hidden', detail: '*~' };
|
||||
}
|
||||
|
||||
// skip invalid installers
|
||||
let head = await getPartialHeader(path);
|
||||
if (!head) {
|
||||
return { type: 'invalid', detail: '!README.md' };
|
||||
}
|
||||
|
||||
let alias = head.match(ALIAS_RE);
|
||||
if (alias) {
|
||||
let link = alias[1];
|
||||
return { type: 'alias', detail: link };
|
||||
}
|
||||
|
||||
let cacheFile = `${LEGACY_CACHE_DIR}/${entry.name}.json`;
|
||||
let hasCacheFile = await Fs.access(cacheFile)
|
||||
.then(function () {
|
||||
return true;
|
||||
})
|
||||
.catch(function () {
|
||||
return false;
|
||||
});
|
||||
if (!hasCacheFile) {
|
||||
return { type: 'selfhosted', detail: true };
|
||||
}
|
||||
|
||||
return { type: 'valid', detail: true };
|
||||
};
|
||||
|
||||
// Typically a package is organized by release (ex: go has 1.20, 1.21, etc),
|
||||
// but we will organize by the build (ex: go1.20-darwin-arm64.tar.gz, etc).
|
||||
bc.getPackages = async function (args) {
|
||||
let name = args.name;
|
||||
let warm = bc._caches[name];
|
||||
if (warm) {
|
||||
return _doGetPackages(args);
|
||||
}
|
||||
let inflight = bc._inflight[name];
|
||||
if (inflight) {
|
||||
return inflight;
|
||||
}
|
||||
let p = _doGetPackages(args).finally(function () {
|
||||
delete bc._inflight[name];
|
||||
});
|
||||
bc._inflight[name] = p;
|
||||
return p;
|
||||
};
|
||||
|
||||
async function _doGetPackages({ name }) {
|
||||
let dataFile = `${LEGACY_CACHE_DIR}/${name}.json`;
|
||||
let tsFile = `${LEGACY_CACHE_DIR}/${name}.updated.txt`;
|
||||
|
||||
let tsDate;
|
||||
{
|
||||
let secondsStr = await Fs.readFile(tsFile, 'ascii').catch(function (err) {
|
||||
if (err.code !== 'ENOENT') {
|
||||
throw err;
|
||||
}
|
||||
return '0';
|
||||
});
|
||||
secondsStr = secondsStr.trim();
|
||||
let seconds = parseFloat(secondsStr) || 0;
|
||||
let ms = seconds * 1000;
|
||||
tsDate = new Date(ms);
|
||||
}
|
||||
|
||||
let projInfo = bc._caches[name];
|
||||
|
||||
let meta = {
|
||||
// version info
|
||||
versions: projInfo?.versions || [],
|
||||
lexvers: projInfo?.lexvers || [],
|
||||
lexversMap: projInfo?.lexversMap || {},
|
||||
// culled release assets
|
||||
packages: projInfo?.packages || [],
|
||||
releasesByTriplet: projInfo?.releasesByTriplet || {},
|
||||
// target info
|
||||
triplets: projInfo?.triplets || [],
|
||||
oses: projInfo?.oses || [],
|
||||
arches: projInfo?.arches || [],
|
||||
libcs: projInfo?.libcs || [],
|
||||
formats: projInfo?.formats || [],
|
||||
// TODO channels: projInfo?.channels || [],
|
||||
};
|
||||
|
||||
if (!projInfo) {
|
||||
let json = await Fs.readFile(dataFile, 'ascii').catch(
|
||||
async function (err) {
|
||||
if (err.code !== 'ENOENT') {
|
||||
throw err;
|
||||
}
|
||||
|
||||
return null;
|
||||
},
|
||||
);
|
||||
|
||||
try {
|
||||
projInfo = JSON.parse(json);
|
||||
} catch (e) {
|
||||
console.error(`error: ${dataFile}:\n\t${e.message}`);
|
||||
projInfo = null;
|
||||
}
|
||||
}
|
||||
if (!projInfo) {
|
||||
return meta;
|
||||
}
|
||||
let latestProjInfo = await BuildsCacher.transformAndUpdate(
|
||||
name,
|
||||
projInfo,
|
||||
meta,
|
||||
tsDate,
|
||||
bc,
|
||||
);
|
||||
bc._caches[name] = latestProjInfo;
|
||||
|
||||
return latestProjInfo;
|
||||
}
|
||||
|
||||
/**
|
||||
* Given a list of acceptable formats, get the sorted list of of formats.
|
||||
* Actually used (as per node _webi/lint-builds.js):
|
||||
* .7z
|
||||
* .app.zip
|
||||
* .dmg
|
||||
* .exe
|
||||
* .exe.xz
|
||||
* .git
|
||||
* .gz
|
||||
* .msi
|
||||
* .pkg
|
||||
* .pkg.tar.zst
|
||||
* .sh
|
||||
* .tar.gz
|
||||
* .tar.xz
|
||||
* .xz
|
||||
* .zip
|
||||
*/
|
||||
bc.getSortedFormats = function (formats) {
|
||||
/* jshint maxcomplexity: 25 */
|
||||
formats.sort();
|
||||
let id = formats.join(',');
|
||||
if (bc._allFormats[id]) {
|
||||
return bc._allFormats[id];
|
||||
}
|
||||
|
||||
// we don't know how to handle any of these yet
|
||||
// let exclude = [];
|
||||
// let isAndroid = false;
|
||||
// if (!isAndroid) {
|
||||
// exclude.push('.apk');
|
||||
// }
|
||||
// let isDebian = false;
|
||||
// if (!isDebian) {
|
||||
// exclude.push('.deb');
|
||||
// }
|
||||
// let isEnterpriseLinux = false;
|
||||
// if (!isEnterpriseLinux) {
|
||||
// exclude.push('.rpm');
|
||||
// }
|
||||
// let isArch = false;
|
||||
// if (!isArch) {
|
||||
// exclude.push('.pkg.tar.zst');
|
||||
// }
|
||||
|
||||
let hasExe = formats.includes('exe') || formats.includes('.exe');
|
||||
|
||||
/** @type {Array<String>} */
|
||||
let exts = [];
|
||||
|
||||
let hasXz = formats.includes('xz') || formats.includes('.xz');
|
||||
if (hasXz) {
|
||||
exts.push('.tar.xz');
|
||||
if (hasExe) {
|
||||
exts.push('.exe.xz');
|
||||
}
|
||||
exts.push('.xz');
|
||||
}
|
||||
let hasZst = formats.includes('zst') || formats.includes('.zst');
|
||||
if (hasZst) {
|
||||
exts.push('.tar.zst');
|
||||
exts.push('.zst');
|
||||
}
|
||||
let hasZip = formats.includes('zip') || formats.includes('.zip');
|
||||
if (hasZip) {
|
||||
exts.push('.zip');
|
||||
}
|
||||
let has7z = false;
|
||||
if (has7z) {
|
||||
exts.push('.7z');
|
||||
}
|
||||
// let hasBz2 = formats.includes('bz2') || formats.includes('.bz2');
|
||||
// if (hasBz2) {
|
||||
// exts.push('.bz2');
|
||||
// }
|
||||
if (hasExe) {
|
||||
if (!hasZip) {
|
||||
exts.push('.zip');
|
||||
}
|
||||
exts.push('.tar.gz');
|
||||
exts.push('.gz');
|
||||
exts.push('.exe');
|
||||
exts.push('.msi');
|
||||
//exts.push('.msixbundle');
|
||||
} else {
|
||||
exts.push('.tar.gz');
|
||||
exts.push('.gz');
|
||||
exts.push('.sh');
|
||||
}
|
||||
let hasGit = formats.includes('git') || formats.includes('.git');
|
||||
if (hasGit) {
|
||||
exts.push('.git');
|
||||
}
|
||||
|
||||
// Fallbacks
|
||||
// (we include everything to bubble an extract error over not found)
|
||||
exts.push('.app.zip');
|
||||
exts.push('.dmg');
|
||||
exts.push('.pkg');
|
||||
if (!hasXz) {
|
||||
exts.push('.tar.xz');
|
||||
if (hasExe) {
|
||||
exts.push('.exe.xz');
|
||||
}
|
||||
exts.push('.xz');
|
||||
}
|
||||
if (!hasZip) {
|
||||
if (!hasExe) {
|
||||
exts.push('.zip');
|
||||
}
|
||||
}
|
||||
if (!hasZst) {
|
||||
exts.push('.tar.zst');
|
||||
exts.push('.zst');
|
||||
}
|
||||
if (!has7z) {
|
||||
exts.push('.7z');
|
||||
}
|
||||
// if (!hasRar) {
|
||||
// exts.push('.rar');
|
||||
// }
|
||||
// exts.push('.tar.bz2');
|
||||
// exts.push('.bz2');
|
||||
|
||||
bc._allFormats[id] = exts;
|
||||
return exts;
|
||||
};
|
||||
|
||||
bc.selectPackage = function (packages, formats) {
|
||||
if (packages.length === 1) {
|
||||
return packages[0];
|
||||
}
|
||||
|
||||
let exts = bc.getSortedFormats(formats);
|
||||
for (let ext of exts) {
|
||||
for (let build of packages) {
|
||||
if (build.ext === ext) {
|
||||
return build;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return packages[0];
|
||||
};
|
||||
|
||||
/**
|
||||
* @param {ProjectInfo} projInfo
|
||||
*/
|
||||
bc.enumerateLatestVersions = function (projInfo) {
|
||||
let lexPrefix = '';
|
||||
let matchInfo = Lexver.matchSorted(projInfo.lexvers, lexPrefix);
|
||||
let verInfo = {
|
||||
default: projInfo.lexversMap[matchInfo.default],
|
||||
previous: projInfo.lexversMap[matchInfo.previous],
|
||||
stable: projInfo.lexversMap[matchInfo.stable],
|
||||
latest: projInfo.lexversMap[matchInfo.latest],
|
||||
};
|
||||
|
||||
return verInfo;
|
||||
};
|
||||
|
||||
/**
|
||||
* @param {ProjectInfo} projInfo
|
||||
* @param {HostTarget} hostTarget
|
||||
* @param {VersionTarget} verTarget
|
||||
*/
|
||||
bc.findMatchingPackages = function (projInfo, hostTarget, verTarget) {
|
||||
let matchInfo = bc._enumerateVersions(projInfo, verTarget.version);
|
||||
let triplets = bc._enumerateTriplets(hostTarget);
|
||||
//console.log('dbg: matchInfo', matchInfo);
|
||||
|
||||
if (matchInfo) {
|
||||
for (let _triplet of triplets) {
|
||||
let targetReleases = projInfo.releasesByTriplet[_triplet];
|
||||
if (!targetReleases) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Make sure that these releases are the expected version
|
||||
// (ex: jq1.7 => darwin-arm64-libc, jq1.6 => darwin-x86_64-libc)
|
||||
for (let matchver of matchInfo.matches) {
|
||||
let ver = projInfo.lexversMap[matchver] || matchver;
|
||||
let packages = targetReleases[ver];
|
||||
if (!packages) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let match = {
|
||||
triplet: _triplet,
|
||||
packages: packages,
|
||||
latest: projInfo.versions[0],
|
||||
version: ver,
|
||||
versions: matchInfo,
|
||||
};
|
||||
return match;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
// Version-first iteration, not triplet-first: take the newest
|
||||
// version even when its only build lives in a fallback triplet
|
||||
// (e.g. serviceman v1.0.1 only exists at posix_2017-ANYARCH-none).
|
||||
for (let lexver of projInfo.lexvers) {
|
||||
let ver = projInfo.lexversMap[lexver] || lexver;
|
||||
|
||||
for (let _triplet of triplets) {
|
||||
let targetReleases = projInfo.releasesByTriplet[_triplet];
|
||||
if (!targetReleases) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let packages = targetReleases[ver];
|
||||
if (!packages) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let pkg = packages[0];
|
||||
if (verTarget.lts) {
|
||||
if (!pkg.lts) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let match = {
|
||||
triplet: _triplet,
|
||||
packages: packages,
|
||||
latest: projInfo.versions[0],
|
||||
version: ver,
|
||||
versions: matchInfo,
|
||||
};
|
||||
return match;
|
||||
}
|
||||
|
||||
let wantChannel = verTarget.channel || 'stable';
|
||||
let isChannel = pkg.channel || 'stable';
|
||||
if (wantChannel === 'stable') {
|
||||
if (isChannel !== 'stable') {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
// latest, beta, alpha, rc, preview
|
||||
|
||||
let match = {
|
||||
triplet: _triplet,
|
||||
packages: packages,
|
||||
latest: projInfo.versions[0],
|
||||
version: ver,
|
||||
versions: matchInfo,
|
||||
};
|
||||
return match;
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
};
|
||||
|
||||
bc._enumerateTriplets = function (hostTarget) {
|
||||
let id = [hostTarget.os, hostTarget.arch, hostTarget.libc].join(',');
|
||||
let triplets = bc._allTriplets[id] || [];
|
||||
if (triplets.length > 0) {
|
||||
return triplets;
|
||||
}
|
||||
|
||||
// Prefer platform-specific matches over ANYOS/ANYARCH fallbacks.
|
||||
// This ensures e.g. darwin-aarch64-none matches before
|
||||
// ANYOS-ANYARCH-none (.git source URLs from old releases).
|
||||
let oses = [];
|
||||
if (hostTarget.os === 'windows') {
|
||||
oses = ['windows', 'ANYOS'];
|
||||
} else if (hostTarget.os === 'android') {
|
||||
oses = ['android', 'linux', 'posix_2017', 'posix_2024', 'ANYOS'];
|
||||
} else {
|
||||
oses = [hostTarget.os, 'posix_2017', 'posix_2024', 'ANYOS'];
|
||||
}
|
||||
|
||||
let waterfall = HostTargets.WATERFALL[hostTarget.os] || {};
|
||||
let arches = waterfall[hostTarget.arch] ||
|
||||
HostTargets.WATERFALL.ANYOS[hostTarget.arch] || [hostTarget.arch];
|
||||
arches = arches.concat(['ANYARCH']);
|
||||
// termsToTarget omits libc for plain UAs; 'libc' → waterfall ['none','libc',...]
|
||||
let libc = hostTarget.libc || 'libc';
|
||||
let libcs = waterfall[libc] ||
|
||||
HostTargets.WATERFALL.ANYOS[libc] || [libc];
|
||||
|
||||
// Extend the glibc-host waterfall: the table only lists [none, libc]
|
||||
// but Rust projects (bat, rg) and node ship libc='gnu' builds, and
|
||||
// static musl builds also run on glibc hosts.
|
||||
if (libc === 'libc' && !libcs.includes('gnu')) {
|
||||
libcs = ['none', 'gnu', 'musl', 'libc'];
|
||||
}
|
||||
|
||||
for (let os of oses) {
|
||||
for (let arch of arches) {
|
||||
for (let libc of libcs) {
|
||||
let triplet = `${os}-${arch}-${libc}`;
|
||||
triplets.push(triplet);
|
||||
}
|
||||
}
|
||||
}
|
||||
bc._allTriplets[id] = triplets;
|
||||
|
||||
return triplets;
|
||||
};
|
||||
|
||||
bc._enumerateVersions = function (projInfo, ver) {
|
||||
if (!ver) {
|
||||
return null;
|
||||
}
|
||||
let lexPrefix = Lexver.parsePrefix(ver);
|
||||
let matchInfo = Lexver.matchSorted(projInfo.lexvers, lexPrefix);
|
||||
|
||||
return matchInfo;
|
||||
};
|
||||
|
||||
return bc;
|
||||
};
|
||||
|
||||
BuildsCacher._classify = function (bc, projInfo, build) {
|
||||
/* jshint maxcomplexity: 30 */
|
||||
// Cache entries arrive pre-classified (os/arch/libc/ext set). Skip
|
||||
// maybeInstallable for those — it false-rejects names ending in a
|
||||
// version tag (`serviceman-v1.0.1`, `v1.0.1.zip`).
|
||||
let cacheClassified =
|
||||
build.os && build.arch && build.libc && build.ext;
|
||||
if (!cacheClassified) {
|
||||
let maybeInstallable = Triplet.maybeInstallable(projInfo, build);
|
||||
if (!maybeInstallable) {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
if (LEGACY_OS_MAP[build.os]) {
|
||||
build.os = LEGACY_OS_MAP[build.os];
|
||||
}
|
||||
if (LEGACY_ARCH_MAP[build.arch]) {
|
||||
build.arch = LEGACY_ARCH_MAP[build.arch];
|
||||
}
|
||||
|
||||
// because some packages are shimmed to match a single download against
|
||||
let preTarget = Object.assign({ os: '', arch: '', libc: '' }, build);
|
||||
|
||||
let targetId = `${preTarget.os}:${preTarget.arch}:${preTarget.libc}`;
|
||||
let buildId = `${projInfo.name}:${targetId}@${build.download}`;
|
||||
//console.log(`dbg: buildId`, buildId);
|
||||
let target = bc._targetsByBuildIdCache[buildId];
|
||||
if (target) {
|
||||
Object.assign(build, { target: target, triplet: target.triplet });
|
||||
return target;
|
||||
}
|
||||
|
||||
let pattern = Triplet.toPattern(projInfo, build);
|
||||
//console.log(`dbg: pattern`, pattern);
|
||||
if (!pattern) {
|
||||
let err = new Error(`no pattern generated for ${projInfo.name}`);
|
||||
err.code = 'E_BUILD_NO_PATTERN';
|
||||
target = { error: err };
|
||||
bc._targetsByBuildIdCache[buildId] = target;
|
||||
return target;
|
||||
}
|
||||
|
||||
let rawTerms = pattern.split(/[_\{\}\/\.\-]+/g);
|
||||
//console.log(`dbg: rawTerms`, rawTerms);
|
||||
for (let term of rawTerms) {
|
||||
delete bc.orphanTerms[term];
|
||||
bc.usedTerms[term] = true;
|
||||
}
|
||||
|
||||
// {NAME}/{NAME}-{VER}-Windows-x86_64_v2-musl.exe =>
|
||||
// {NAME}.windows.x86_64v2.musl.exe
|
||||
let terms = Triplet.patternToTerms(pattern);
|
||||
//console.log(`dbg: terms`, terms);
|
||||
if (!terms.length) {
|
||||
let err = new Error(`'${terms}' was trimmed to ''`);
|
||||
target = { error: err };
|
||||
bc._targetsByBuildIdCache[buildId] = target;
|
||||
return target;
|
||||
}
|
||||
|
||||
for (let term of terms) {
|
||||
if (!term) {
|
||||
continue;
|
||||
}
|
||||
|
||||
if (bc.ALL_TERMS[term]) {
|
||||
delete bc.orphanTerms[term];
|
||||
bc.usedTerms[term] = true;
|
||||
continue;
|
||||
}
|
||||
|
||||
bc.unknownTerms[term] = true;
|
||||
}
|
||||
|
||||
// Skip termsToTarget for cache-classified entries: it false-flags
|
||||
// e.g. .git URLs as os=ANYOS while the cache says os=posix_2017,
|
||||
// and the mismatch check throws.
|
||||
target = { triplet: '' };
|
||||
if (cacheClassified) {
|
||||
target.os = build.os;
|
||||
target.arch = build.arch;
|
||||
target.libc = build.libc;
|
||||
target.vendor = build.vendor || 'unknown';
|
||||
target.android = false;
|
||||
target.unknownTerms = [];
|
||||
} else {
|
||||
try {
|
||||
void Triplet.termsToTarget(target, projInfo, build, terms);
|
||||
} catch (e) {
|
||||
console.error(`PACKAGE FORMAT CHANGE for '${projInfo.name}':`);
|
||||
console.error(e.message);
|
||||
console.error(build);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
target.triplet = `${target.arch}-${target.vendor}-${target.os}-${target.libc}`;
|
||||
|
||||
{
|
||||
// TODO I don't love this hidden behavior
|
||||
// perhaps classify should just happen when the package is loaded
|
||||
// (and the sanity error should be removed, or thrown after the loop is complete)
|
||||
let hasTriplet = projInfo.triplets.includes(target.triplet);
|
||||
if (!hasTriplet) {
|
||||
projInfo.triplets.push(target.triplet);
|
||||
}
|
||||
let hasOs = projInfo.oses.includes(target.os);
|
||||
if (!hasOs) {
|
||||
projInfo.oses.push(target.os);
|
||||
}
|
||||
let hasArch = projInfo.arches.includes(target.arch);
|
||||
if (!hasArch) {
|
||||
projInfo.arches.push(target.arch);
|
||||
}
|
||||
let hasLibc = projInfo.libcs.includes(target.libc);
|
||||
if (!hasLibc) {
|
||||
projInfo.libcs.push(target.libc);
|
||||
}
|
||||
|
||||
if (!build.ext) {
|
||||
build.ext = Triplet.buildToPackageType(build);
|
||||
}
|
||||
if (build.ext) {
|
||||
if (!build.ext.startsWith('.')) {
|
||||
build.ext = `.${build.ext}`;
|
||||
}
|
||||
}
|
||||
let hasExt = projInfo.formats.includes(build.ext);
|
||||
if (!hasExt) {
|
||||
projInfo.formats.push(build.ext);
|
||||
}
|
||||
let hasGlobalExt = bc.formats.includes(build.ext);
|
||||
if (!hasGlobalExt) {
|
||||
bc.formats.push(build.ext);
|
||||
}
|
||||
}
|
||||
|
||||
bc._triplets[target.triplet] = true;
|
||||
bc._targetsByBuildIdCache[buildId] = target;
|
||||
|
||||
let triple = [target.arch, target.vendor, target.os, target.libc];
|
||||
for (let term of triple) {
|
||||
if (!bc.ALL_TERMS[term]) {
|
||||
throw new Error(
|
||||
`[SANITY FAIL] '${projInfo.name}' '${target.triplet}' generated unknown term '${term}'`,
|
||||
);
|
||||
}
|
||||
|
||||
delete bc.orphanTerms[term];
|
||||
bc.usedTerms[term] = true;
|
||||
}
|
||||
|
||||
return target;
|
||||
};
|
||||
|
||||
BuildsCacher.transformAndUpdate = function (name, projInfo, meta, date, bc) {
|
||||
meta.packages = [];
|
||||
|
||||
let updated = date.valueOf();
|
||||
|
||||
Object.assign(projInfo, { name, updated }, meta);
|
||||
for (let build of projInfo.releases) {
|
||||
let buildTarget = BuildsCacher._classify(bc, projInfo, build);
|
||||
if (!buildTarget) {
|
||||
// ignore known, non-package extensions
|
||||
continue;
|
||||
}
|
||||
|
||||
if (buildTarget.error) {
|
||||
let err = buildTarget.error;
|
||||
let code = err.code || '';
|
||||
console.error(`[ERROR]: ${code} ${projInfo.name}: ${build.name}`);
|
||||
console.error(`>>> ${err.message} <<<`);
|
||||
console.error(projInfo);
|
||||
console.error(build);
|
||||
console.error(`^^^ ${err.message} ^^^`);
|
||||
console.error(err.stack);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!build.name) {
|
||||
build.name = build.download.replace(/.*\//, '');
|
||||
}
|
||||
|
||||
build.target = buildTarget;
|
||||
meta.packages.push(build);
|
||||
}
|
||||
|
||||
BuildsCacher.updateReleasesByTriplet(meta);
|
||||
BuildsCacher.updateAndSortVersions(projInfo, meta);
|
||||
|
||||
Object.assign(projInfo, { name, updated }, meta);
|
||||
return projInfo;
|
||||
};
|
||||
|
||||
// TODO
|
||||
// - tag channels
|
||||
BuildsCacher.updateAndSortVersions = function (projInfo, meta) {
|
||||
for (let build of projInfo.packages) {
|
||||
let hasVersion = meta.versions.includes(build.version);
|
||||
if (!hasVersion) {
|
||||
build.lexver = Lexver.parseVersion(build.version);
|
||||
meta.lexversMap[build.lexver] = build.version;
|
||||
}
|
||||
}
|
||||
|
||||
meta.lexvers = Object.keys(meta.lexversMap);
|
||||
meta.lexvers.sort();
|
||||
meta.lexvers.reverse();
|
||||
|
||||
meta.versions = [];
|
||||
for (let lexver of meta.lexvers) {
|
||||
let version = meta.lexversMap[lexver];
|
||||
meta.versions.push(version);
|
||||
}
|
||||
|
||||
projInfo.packages.sort(function (a, b) {
|
||||
if (a.lexver > b.lexver) {
|
||||
return -1;
|
||||
}
|
||||
if (a.lexver < b.lexver) {
|
||||
return 1;
|
||||
}
|
||||
return 0;
|
||||
});
|
||||
};
|
||||
|
||||
BuildsCacher.updateReleasesByTriplet = function (meta) {
|
||||
for (let build of meta.packages) {
|
||||
let target = build.target;
|
||||
|
||||
let triplet = `${target.os}-${target.arch}-${target.libc}`;
|
||||
if (!meta.releasesByTriplet[triplet]) {
|
||||
meta.releasesByTriplet[triplet] = {};
|
||||
}
|
||||
|
||||
let buildsByRelease = meta.releasesByTriplet[triplet];
|
||||
if (!buildsByRelease[build.version]) {
|
||||
buildsByRelease[build.version] = [];
|
||||
}
|
||||
|
||||
let packages = buildsByRelease[build.version];
|
||||
packages.push(build);
|
||||
}
|
||||
};
|
||||
@@ -1,37 +0,0 @@
|
||||
'use strict';
|
||||
|
||||
let Builds = module.exports;
|
||||
|
||||
let Path = require('node:path');
|
||||
|
||||
let BuildsCacher = require('./builds-cacher.js');
|
||||
// let HostTargets = require('./build-classifier/host-targets.js');
|
||||
let Parallel = require('./parallel.js');
|
||||
|
||||
var INSTALLERS_DIR = Path.join(__dirname, '..');
|
||||
var CACHE_DIR = Path.join(__dirname, '../_cache');
|
||||
|
||||
let bc = BuildsCacher.create({
|
||||
caches: CACHE_DIR,
|
||||
installers: INSTALLERS_DIR,
|
||||
});
|
||||
|
||||
Builds.init = async function () {
|
||||
let dirs = await bc.getProjectsByType();
|
||||
let projNames = Object.keys(dirs.valid);
|
||||
|
||||
let parallel = 25;
|
||||
await Parallel.run(parallel, projNames, getAll);
|
||||
async function getAll(name) {
|
||||
void (await bc.getPackages({
|
||||
name: name,
|
||||
date: new Date(),
|
||||
}));
|
||||
}
|
||||
};
|
||||
|
||||
Builds.enumerateLatestVersions = bc.enumerateLatestVersions;
|
||||
Builds.findMatchingPackages = bc.findMatchingPackages;
|
||||
Builds.getPackage = bc.getPackages;
|
||||
Builds.getProjectType = bc.getProjectType;
|
||||
Builds.selectPackage = bc.selectPackage;
|
||||
@@ -1,90 +0,0 @@
|
||||
'use strict';
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Os = require('node:os');
|
||||
let Path = require('node:path');
|
||||
|
||||
let BuildsCacher = require('./builds-cacher.js');
|
||||
let Triplet = require('./build-classifier/triplet.js');
|
||||
|
||||
let LEGACY_CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
|
||||
async function main() {
|
||||
let projName = process.argv[2];
|
||||
if (!projName) {
|
||||
console.error(``);
|
||||
console.error(`USAGE`);
|
||||
console.error(``);
|
||||
console.error(` classify-one <project-name>`);
|
||||
console.error(``);
|
||||
console.error(`EXAMPLE`);
|
||||
console.error(``);
|
||||
console.error(` classify-one caddy`);
|
||||
console.error(``);
|
||||
return;
|
||||
}
|
||||
|
||||
let tsDate = new Date(0);
|
||||
let meta = {
|
||||
// version info
|
||||
versions: [],
|
||||
lexvers: [],
|
||||
lexversMap: {},
|
||||
// culled release assets
|
||||
packages: [],
|
||||
releasesByTriplet: {},
|
||||
// target info
|
||||
triplets: [],
|
||||
oses: [],
|
||||
arches: [],
|
||||
libcs: [],
|
||||
formats: [],
|
||||
};
|
||||
|
||||
let dataFile = Path.join(LEGACY_CACHE_DIR, `${projName}.json`);
|
||||
let json = await Fs.readFile(dataFile, 'utf8');
|
||||
let projInfo = JSON.parse(json);
|
||||
|
||||
// let packages = await Builds.getPackage({ name: projName });
|
||||
// console.log(packages);
|
||||
|
||||
let bc = {};
|
||||
bc.ALL_TERMS = Triplet.TERMS_PRIMARY_MAP;
|
||||
bc.orphanTerms = Object.assign({}, bc.ALL_TERMS);
|
||||
bc.unknownTerms = {};
|
||||
bc.usedTerms = {};
|
||||
bc.formats = [];
|
||||
bc._targetsByBuildIdCache = {};
|
||||
bc._triplets = {};
|
||||
|
||||
let transformed = BuildsCacher.transformAndUpdate(
|
||||
projName,
|
||||
projInfo,
|
||||
meta,
|
||||
tsDate,
|
||||
bc,
|
||||
);
|
||||
|
||||
console.log(`[DEBUG] transformed`);
|
||||
let sample = transformed.packages.slice(0, 20);
|
||||
console.log('packages:', sample, ':packages');
|
||||
let firstTriplet = Object.keys(transformed.releasesByTriplet)[0];
|
||||
let firstVersion = transformed.versions[0];
|
||||
console.log(
|
||||
`releasesByTriplet[${firstTriplet}][${firstVersion}]:`,
|
||||
transformed.releasesByTriplet[firstTriplet]?.[firstVersion],
|
||||
':releasesByTriplet',
|
||||
);
|
||||
console.log('versions:', transformed.versions, ':versions');
|
||||
console.log('triplets:', transformed.triplets, ':triplets');
|
||||
console.log('oses:', transformed.oses, ':oses');
|
||||
console.log('arches:', transformed.arches, ':arches');
|
||||
console.log('libcs:', transformed.libcs, ':libcs');
|
||||
console.log('formats:', transformed.formats, ':formats');
|
||||
console.log(Object.keys(transformed));
|
||||
}
|
||||
|
||||
main().catch(function (err) {
|
||||
console.error('Error:');
|
||||
console.error(err);
|
||||
});
|
||||
@@ -1,349 +0,0 @@
|
||||
#!/bin/sh
|
||||
#<pre>
|
||||
|
||||
############################################################
|
||||
# <h1>Cheat Sheet at CHEATSHEET_URL</h1>
|
||||
# <meta http-equiv="refresh" content="3; URL='CHEATSHEET_URL'" />
|
||||
############################################################
|
||||
|
||||
export WEBI_PKG=webi
|
||||
export WEBI_HOST=https://webinstall.dev
|
||||
export WEBI_CHECKSUM=06a7fb9f
|
||||
|
||||
#########################################
|
||||
# #
|
||||
# Display Debug Info in Case of Failure #
|
||||
# #
|
||||
#########################################
|
||||
|
||||
fn_show_welcome() { (
|
||||
echo ""
|
||||
echo ""
|
||||
# invert t_task and t_pkg for top-level welcome message
|
||||
printf -- ">>> %s %s <<<\n" \
|
||||
"$(t_pkg 'Welcome to') $(t_task 'Webi')$(t_pkg '!')" \
|
||||
"$(t_dim "- modern tools, instant installs.")"
|
||||
echo " We expect your experience to be $(t_em 'absolutely perfect')!"
|
||||
echo ""
|
||||
echo " $(t_attn 'Success')? Star it! $(t_url 'https://github.com/webinstall/webi-installers')"
|
||||
echo " $(t_attn 'Problem')? Report it: $(t_url 'https://github.com/webinstall/webi-installers/issues')"
|
||||
echo " $(t_dim "(your system is") $(t_host "$(fn_get_os)")/$(t_host "$(uname -m)") $(t_dim "with") $(t_host "$(fn_get_libc)") $(t_dim "&") $(t_host "$(fn_get_http_client_name)")$(t_dim ")")"
|
||||
|
||||
sleep 0.2
|
||||
); }
|
||||
|
||||
fn_get_os() { (
|
||||
# Ex:
|
||||
# GNU/Linux
|
||||
# Android
|
||||
# Linux (often Alpine, musl)
|
||||
# Darwin
|
||||
b_os="$(uname -o 2> /dev/null || echo '')"
|
||||
b_sys="$(uname -s)"
|
||||
if test -z "${b_os}" || test "${b_os}" = "${b_sys}"; then
|
||||
# ex: 'Darwin' (and plain, non-GNU 'Linux')
|
||||
echo "${b_sys}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if echo "${b_os}" | grep -q "${b_sys}"; then
|
||||
# ex: 'GNU/Linux'
|
||||
echo "${b_os}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# ex: 'Android/Linux'
|
||||
echo "${b_os}/${b_sys}"
|
||||
); }
|
||||
|
||||
fn_get_libc() { (
|
||||
# Ex:
|
||||
# musl
|
||||
# libc
|
||||
if ldd /bin/ls 2> /dev/null | grep -q 'musl' 2> /dev/null; then
|
||||
echo 'musl'
|
||||
elif fn_get_os | grep -q 'GNU|Linux'; then
|
||||
echo 'gnu'
|
||||
else
|
||||
echo 'libc'
|
||||
fi
|
||||
); }
|
||||
|
||||
fn_get_http_client_name() { (
|
||||
# Ex:
|
||||
# curl
|
||||
# curl+wget
|
||||
b_client=""
|
||||
if command -v curl > /dev/null; then
|
||||
b_client="curl"
|
||||
fi
|
||||
if command -v wget > /dev/null; then
|
||||
if test -z "${b_client}"; then
|
||||
b_client="wget"
|
||||
else
|
||||
b_client="curl+wget"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "${b_client}"
|
||||
); }
|
||||
|
||||
#########################################
|
||||
# #
|
||||
# For Making the Display Nice #
|
||||
# #
|
||||
#########################################
|
||||
|
||||
# Term Types
|
||||
t_cmd() { (fn_printf '\e[2m\e[35m%s\e[39m\e[22m' "${1}"); }
|
||||
t_host() { (fn_printf '\e[2m\e[33m%s\e[39m\e[22m' "${1}"); }
|
||||
t_link() { (fn_printf '\e[1m\e[36m%s\e[39m\e[22m' "${1}"); }
|
||||
t_path() { (fn_printf '\e[2m\e[32m%s\e[39m\e[22m' "${1}"); }
|
||||
t_pkg() { (fn_printf '\e[1m\e[32m%s\e[39m\e[22m' "${1}"); }
|
||||
t_task() { (fn_printf '\e[36m%s\e[39m' "${1}"); }
|
||||
t_url() { (fn_printf '\e[2m%s\e[22m' "${1}"); }
|
||||
|
||||
# Levels
|
||||
t_info() { (fn_printf '\e[1m\e[36m%s\e[39m\e[22m' "${1}"); }
|
||||
t_attn() { (fn_printf '\e[1m\e[33m%s\e[39m\e[22m' "${1}"); }
|
||||
t_warn() { (fn_printf '\e[1m\e[33m%s\e[39m\e[22m' "${1}"); }
|
||||
t_err() { (fn_printf '\e[31m%s\e[39m' "${1}"); }
|
||||
|
||||
# Styles
|
||||
t_bold() { (fn_printf '\e[1m%s\e[22m' "${1}"); }
|
||||
t_dim() { (fn_printf '\e[2m%s\e[22m' "${1}"); }
|
||||
t_em() { (fn_printf '\e[3m%s\e[23m' "${1}"); }
|
||||
t_under() { (fn_printf '\e[4m%s\e[24m' "${1}"); }
|
||||
|
||||
# FG Colors
|
||||
t_cyan() { (fn_printf '\e[36m%s\e[39m' "${1}"); }
|
||||
t_green() { (fn_printf '\e[32m%s\e[39m' "${1}"); }
|
||||
t_magenta() { (fn_printf '\e[35m%s\e[39m' "${1}"); }
|
||||
t_yellow() { (fn_printf '\e[33m%s\e[39m' "${1}"); }
|
||||
|
||||
fn_printf() { (
|
||||
a_style="${1}"
|
||||
a_text="${2}"
|
||||
if fn_is_tty; then
|
||||
#shellcheck disable=SC2059
|
||||
printf -- "${a_style}" "${a_text}"
|
||||
else
|
||||
printf -- '%s' "${a_text}"
|
||||
fi
|
||||
); }
|
||||
|
||||
fn_sub_home() { (
|
||||
my_rel=${HOME}
|
||||
my_abs=${1}
|
||||
echo "${my_abs}" | sed "s:^${my_rel}:~:"
|
||||
); }
|
||||
|
||||
###################################
|
||||
# #
|
||||
# Detect HTTP Client #
|
||||
# #
|
||||
###################################
|
||||
|
||||
fn_wget() { (
|
||||
# Doc:
|
||||
# Downloads the file at the given url to the given path
|
||||
a_url="${1}"
|
||||
a_path="${2}"
|
||||
|
||||
cmd_wget="wget -c -q --user-agent"
|
||||
if fn_is_tty; then
|
||||
cmd_wget="wget -c -q --show-progress --user-agent"
|
||||
fi
|
||||
# busybox wget doesn't support --show-progress
|
||||
# See <https://github.com/webinstall/webi-installers/pull/772>
|
||||
if readlink "$(command -v wget)" | grep -q busybox; then
|
||||
cmd_wget="wget --user-agent"
|
||||
fi
|
||||
|
||||
b_triple_ua="$(fn_get_target_triple_user_agent)"
|
||||
b_agent="webi/wget ${b_triple_ua}"
|
||||
if command -v curl > /dev/null; then
|
||||
b_agent="webi/wget+curl ${b_triple_ua}"
|
||||
fi
|
||||
|
||||
if ! $cmd_wget "${b_agent}" "${a_url}" -O "${a_path}"; then
|
||||
echo >&2 " $(t_err "failed to download (wget)") '$(t_url "${a_url}")'"
|
||||
echo >&2 " $cmd_wget '${b_agent}' '${a_url}' -O '${a_path}'"
|
||||
echo >&2 " $(wget -V)"
|
||||
return 1
|
||||
fi
|
||||
); }
|
||||
|
||||
fn_curl() { (
|
||||
# Doc:
|
||||
# Downloads the file at the given url to the given path
|
||||
a_url="${1}"
|
||||
a_path="${2}"
|
||||
|
||||
cmd_curl="curl -f -sSL -#"
|
||||
if fn_is_tty; then
|
||||
cmd_curl="curl -f -sSL"
|
||||
fi
|
||||
|
||||
b_triple_ua="$(fn_get_target_triple_user_agent)"
|
||||
b_agent="webi/curl ${b_triple_ua}"
|
||||
if command -v wget > /dev/null; then
|
||||
b_agent="webi/curl+wget ${b_triple_ua}"
|
||||
fi
|
||||
|
||||
if ! $cmd_curl -A "${b_agent}" "${a_url}" -o "${a_path}"; then
|
||||
echo >&2 " $(t_err "failed to download (curl)") '$(t_url "${a_url}")'"
|
||||
echo >&2 " $cmd_curl -A '${b_agent}' '${a_url}' -o '${a_path}'"
|
||||
echo >&2 " $(curl -V)"
|
||||
return 1
|
||||
fi
|
||||
); }
|
||||
|
||||
fn_get_target_triple_user_agent() { (
|
||||
# Ex:
|
||||
# x86_64/unknown GNU/Linux/5.15.107-2-pve gnu
|
||||
# arm64/unknown Darwin/22.6.0 libc
|
||||
echo "$(uname -m)/unknown $(fn_get_os)/$(uname -r) $(fn_get_libc)"
|
||||
); }
|
||||
|
||||
fn_download_to_path() { (
|
||||
a_url="${1}"
|
||||
a_path="${2}"
|
||||
|
||||
mkdir -p "$(dirname "${a_path}")"
|
||||
if command -v curl > /dev/null; then
|
||||
fn_curl "${a_url}" "${a_path}.part"
|
||||
elif command -v wget > /dev/null; then
|
||||
fn_wget "${a_url}" "${a_path}.part"
|
||||
else
|
||||
echo >&2 " $(t_err "failed to detect HTTP client (curl, wget)")"
|
||||
return 1
|
||||
fi
|
||||
mv "${a_path}.part" "${a_path}"
|
||||
); }
|
||||
|
||||
##############################################
|
||||
# #
|
||||
# Install or Update Webi and Install Package #
|
||||
# #
|
||||
##############################################
|
||||
|
||||
webi_bootstrap() { (
|
||||
a_path="${1}"
|
||||
|
||||
echo ""
|
||||
echo "$(t_task 'Bootstrapping') $(t_pkg 'Webi')"
|
||||
|
||||
b_path_rel="$(fn_sub_home "${a_path}")"
|
||||
b_checksum=""
|
||||
if test -r "${a_path}"; then
|
||||
b_checksum="$(fn_checksum "${a_path}")"
|
||||
fi
|
||||
if test "$b_checksum" = "${WEBI_CHECKSUM}"; then
|
||||
echo " $(t_dim 'Found') $(t_path "${b_path_rel}")"
|
||||
sleep 0.1
|
||||
return 0
|
||||
fi
|
||||
|
||||
b_webi_file_url="${WEBI_HOST}/packages/webi/webi.sh"
|
||||
b_tmp=''
|
||||
if test -r "${a_path}"; then
|
||||
b_ts="$(date -u '+%s')"
|
||||
b_tmp="${a_path}.${b_ts}.bak"
|
||||
mv "${a_path}" "${b_tmp}"
|
||||
echo " Updating $(t_path "${b_path_rel}")"
|
||||
fi
|
||||
|
||||
echo " Downloading $(t_url "${b_webi_file_url}")"
|
||||
echo " to $(t_path "${b_path_rel}")"
|
||||
fn_download_to_path "${b_webi_file_url}" "${a_path}"
|
||||
chmod u+x "${a_path}"
|
||||
|
||||
if test -r "${b_tmp}"; then
|
||||
rm -f "${b_tmp}"
|
||||
fi
|
||||
); }
|
||||
|
||||
fn_checksum() {
|
||||
a_filepath="${1}"
|
||||
|
||||
if command -v sha1sum > /dev/null; then
|
||||
sha1sum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
|
||||
return 0
|
||||
fi
|
||||
|
||||
if command -v shasum > /dev/null; then
|
||||
shasum "${a_filepath}" | cut -d' ' -f1 | cut -c 1-8
|
||||
return 0
|
||||
fi
|
||||
|
||||
if command -v sha1 > /dev/null; then
|
||||
sha1 "${a_filepath}" | cut -d'=' -f2 | cut -c 2-9
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo >&2 " warn: no sha1 sum program"
|
||||
date '+%F %H:%M'
|
||||
}
|
||||
|
||||
##############################################
|
||||
# #
|
||||
# Detect TTY and run main #
|
||||
# #
|
||||
##############################################
|
||||
|
||||
fn_is_tty() {
|
||||
if test "${WEBI_TTY}" = 'tty'; then
|
||||
return 0
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
fn_detect_tty() { (
|
||||
# stdin will NOT be a tty if it's being piped
|
||||
# stdout & stderr WILL be a tty even when piped
|
||||
# they are not a tty if being captured or redirected
|
||||
# 'set -i' is NOT available in sh
|
||||
if test -t 1 && test -t 2; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1
|
||||
); }
|
||||
|
||||
main() { (
|
||||
set -e
|
||||
set -u
|
||||
|
||||
WEBI_TTY="${WEBI_TTY:-}"
|
||||
if test -z "${WEBI_TTY}"; then
|
||||
if fn_detect_tty; then
|
||||
WEBI_TTY="tty"
|
||||
fi
|
||||
export WEBI_TTY
|
||||
fi
|
||||
|
||||
if test -z "${WEBI_WELCOME:-}"; then
|
||||
fn_show_welcome
|
||||
fi
|
||||
export WEBI_WELCOME='shown'
|
||||
|
||||
# note: we may support custom locations in the future
|
||||
export WEBI_HOME="${HOME}/.local"
|
||||
b_home="$(fn_sub_home "${WEBI_HOME}")"
|
||||
b_webi_path="${WEBI_HOME}/bin/webi"
|
||||
b_webi_path_rel="${b_home}/bin/webi"
|
||||
|
||||
WEBI_CURRENT="${WEBI_CURRENT:-}"
|
||||
if test "${WEBI_CURRENT}" != "${WEBI_CHECKSUM}"; then
|
||||
webi_bootstrap "${b_webi_path}"
|
||||
export WEBI_CURRENT="${WEBI_CHECKSUM}"
|
||||
fi
|
||||
|
||||
echo " Running $(t_cmd "${b_webi_path_rel} ${WEBI_PKG}")"
|
||||
echo ""
|
||||
|
||||
"${b_webi_path}" "${WEBI_PKG}"
|
||||
); }
|
||||
|
||||
main
|
||||
@@ -1,336 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Path = require('node:path');
|
||||
|
||||
let BuildsCacher = require('./builds-cacher.js');
|
||||
let HostTargets = require('./build-classifier/host-targets.js');
|
||||
let Parallel = require('./parallel.js');
|
||||
|
||||
var INSTALLERS_DIR = Path.join(__dirname, '..');
|
||||
var CACHE_DIR = Path.join(__dirname, '../_cache');
|
||||
|
||||
let UserAgentsMap = require('./build-classifier/uas.json');
|
||||
let uas = Object.keys(UserAgentsMap);
|
||||
let uaTargetsMap = {};
|
||||
for (let ua of uas) {
|
||||
let terms = ua.split(/[\s\/]+/g);
|
||||
let target = {};
|
||||
void HostTargets.termsToTarget(target, terms);
|
||||
if (!target) {
|
||||
continue;
|
||||
}
|
||||
if (target.errors.length) {
|
||||
throw target.errors[0];
|
||||
}
|
||||
if (!target.os) {
|
||||
// TODO make target null, or create error for this
|
||||
console.warn(`no os for terms: ${terms}`);
|
||||
//throw new Error(`terms: ${terms}`);
|
||||
continue;
|
||||
}
|
||||
if (!target.arch) {
|
||||
// TODO make target null, or create error for this
|
||||
console.warn(`no arch for terms: ${terms}`);
|
||||
//throw new Error(`terms: ${terms}`);
|
||||
continue;
|
||||
}
|
||||
if (!target.libc) {
|
||||
// TODO make target null, or create error for this
|
||||
console.warn(`no libc for terms: ${terms}`);
|
||||
//throw new Error(`terms: ${terms}`);
|
||||
continue;
|
||||
}
|
||||
let triplet = `${target.os}-${target.arch}-${target.libc}`;
|
||||
uaTargetsMap[triplet] = target;
|
||||
}
|
||||
let uaTargets = [];
|
||||
let triplets = Object.keys(uaTargetsMap);
|
||||
for (let triplet of triplets) {
|
||||
let target = uaTargetsMap[triplet];
|
||||
uaTargets.push(target);
|
||||
}
|
||||
|
||||
function showDirs(dirs) {
|
||||
{
|
||||
let errors = Object.keys(dirs.errors);
|
||||
console.error('');
|
||||
console.error(`Errors: ${errors.length}`);
|
||||
for (let name of errors) {
|
||||
let err = dirs.errors[name];
|
||||
console.error(`${name}/: ${err.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let hidden = Object.keys(dirs.hidden);
|
||||
console.debug('');
|
||||
console.debug(`Hidden: ${hidden.length}`);
|
||||
for (let name of hidden) {
|
||||
let kind = dirs.hidden[name];
|
||||
if (kind === '!directory') {
|
||||
console.debug(` ${name}`);
|
||||
} else {
|
||||
console.debug(` ${name}/`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let alias = Object.keys(dirs.alias);
|
||||
console.debug('');
|
||||
console.debug(`Alias: ${alias.length}`);
|
||||
for (let name of alias) {
|
||||
let kind = dirs.alias[name];
|
||||
if (kind === 'symlink') {
|
||||
console.debug(` ${name} => ...`);
|
||||
} else {
|
||||
console.debug(` ${name}/`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let invalids = Object.keys(dirs.invalid);
|
||||
console.warn('');
|
||||
console.warn(`Invalid: ${invalids.length}`);
|
||||
for (let name of invalids) {
|
||||
console.warn(` ${name}/`);
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let selfhosted = Object.keys(dirs.selfhosted);
|
||||
console.info('');
|
||||
console.info(`Self-Hosted: ${selfhosted.length}`);
|
||||
for (let name of selfhosted) {
|
||||
console.info(` ${name}/`);
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let valids = Object.keys(dirs.valid);
|
||||
console.info('');
|
||||
console.info(`Found: ${valids.length}`);
|
||||
for (let name of valids) {
|
||||
console.info(` ${name}/`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let bc = BuildsCacher.create({
|
||||
caches: CACHE_DIR,
|
||||
installers: INSTALLERS_DIR,
|
||||
});
|
||||
|
||||
async function main() {
|
||||
/* jshint maxcomplexity: 25 */
|
||||
// TODO
|
||||
// node ./_webi/lint-builds.js caddy@beta 'x86_64/unknown Darwin libc'
|
||||
//
|
||||
//let [projName, userAgent] = process.argv.slice(2);
|
||||
let projName = process.argv[2];
|
||||
// create test case for zoxide, goreleaser, go, yq, caddy, rg
|
||||
|
||||
let dirs = await bc.getProjectsByType();
|
||||
if (!projName) {
|
||||
showDirs(dirs);
|
||||
console.info('');
|
||||
}
|
||||
|
||||
let rows = [];
|
||||
let triples = [];
|
||||
let valids = Object.keys(dirs.valid);
|
||||
|
||||
if (projName) {
|
||||
if (!valids.includes(projName)) {
|
||||
throw new Error(`'${projName}' is not a valid installable project`);
|
||||
}
|
||||
valids = [projName];
|
||||
}
|
||||
//valids = ['atomicparsley', 'caddy', 'macos'];
|
||||
//valids = ['atomicparsley'];
|
||||
|
||||
console.info('');
|
||||
console.info(`Fetching project release assets`);
|
||||
let parallel = 25;
|
||||
let projects = [];
|
||||
await Parallel.run(parallel, valids, getAll);
|
||||
async function getAll(name, i) {
|
||||
console.info(` ${name}`);
|
||||
let projInfo = await bc.getPackages({
|
||||
//Releases: Releases,
|
||||
name: name,
|
||||
date: new Date(),
|
||||
});
|
||||
projects[i] = projInfo;
|
||||
}
|
||||
|
||||
console.info(`Classifying build assets for...`);
|
||||
for (let projInfo of projects) {
|
||||
console.info(` ${projInfo.name}`);
|
||||
|
||||
let nStr = projInfo.releases.length.toString();
|
||||
let n = nStr.padStart(5, ' ');
|
||||
let row = `##### ${n}\t${projInfo.name}\tv`;
|
||||
rows.push(row);
|
||||
|
||||
// ignore known, non-package extensions
|
||||
for (let build of projInfo.releases) {
|
||||
let target = bc.classify(projInfo, build);
|
||||
if (!target) {
|
||||
// non-build file
|
||||
continue;
|
||||
}
|
||||
|
||||
if (target.error) {
|
||||
let e = target.error;
|
||||
if (e.code === 'E_BUILD_NO_PATTERN') {
|
||||
console.warn(`>>> ${e.message} <<<`);
|
||||
console.warn(projInfo);
|
||||
console.warn(build);
|
||||
console.warn(`^^^ ${e.message} ^^^`);
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
|
||||
if (target.unknownTerms?.length) {
|
||||
let msg = `${projInfo.name}: unrecognized term(s) '${target.unknownTerms}' in '${build.download}'`;
|
||||
let err = new Error(msg);
|
||||
throw err;
|
||||
}
|
||||
|
||||
triples.push(target.triplet);
|
||||
// if (!build.version) {
|
||||
// throw new Error(`no version for ${pkg.name} ${build.name}`);
|
||||
// }
|
||||
// // For debug printing versions
|
||||
// console.error(build.version);
|
||||
rows.push(`${target.triplet}\t${projInfo.name}\t${build.version}`);
|
||||
}
|
||||
}
|
||||
|
||||
console.info(`Fetching builds for`);
|
||||
for (let projInfo of projects) {
|
||||
console.info('');
|
||||
console.info('');
|
||||
console.info(` ${projInfo.name}`);
|
||||
|
||||
for (let target of uaTargets) {
|
||||
let libc = target.libc || 'libc';
|
||||
let hostTriplet = `${target.os}-${target.arch}-${libc}`;
|
||||
console.info('');
|
||||
console.info(` target: ${hostTriplet}`);
|
||||
let match = bc.findMatchingPackages(projInfo, target, {
|
||||
ver: '',
|
||||
});
|
||||
if (!match) {
|
||||
console.info(
|
||||
` project: ${projInfo.name}: missing build for os '${target.os}'`,
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (!match.releases) {
|
||||
console.info(
|
||||
` project: ${projInfo.name}: missing build for os '${target.os}-${target.arch}-${libc}'`,
|
||||
);
|
||||
} else if (match.triplet === hostTriplet) {
|
||||
let releaseNames = Object.keys(match.releases);
|
||||
console.info(` selected ${releaseNames.length}`);
|
||||
} else {
|
||||
let releaseNames = Object.keys(match.releases);
|
||||
console.info(
|
||||
` selected ${releaseNames.length} (${match.triplet} fallback)`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let tsv = rows.join('\n');
|
||||
console.info('');
|
||||
console.info('#rows', rows.length);
|
||||
await Fs.writeFile('builds.tsv', tsv, 'utf8');
|
||||
|
||||
console.info('');
|
||||
console.info('Triplets Detected:');
|
||||
let triplets = Object.keys(bc._triplets);
|
||||
if (triplets.length) {
|
||||
triplets.sort();
|
||||
console.info(' ', triplets.join('\n '));
|
||||
} else {
|
||||
console.info(' (none)');
|
||||
}
|
||||
|
||||
console.info('');
|
||||
console.info('New / Unknown Terms:');
|
||||
let unknowns = Object.keys(bc.unknownTerms);
|
||||
if (unknowns.length) {
|
||||
unknowns.sort();
|
||||
console.warn(' ', unknowns.join('\n '));
|
||||
} else {
|
||||
console.info(' (none)');
|
||||
}
|
||||
|
||||
console.info('');
|
||||
console.info('Unused Terms:');
|
||||
let unuseds = Object.keys(bc.orphanTerms);
|
||||
if (unuseds.length) {
|
||||
unuseds.sort();
|
||||
console.warn(' ', unuseds.join('\n '));
|
||||
} else {
|
||||
console.info(' (none)');
|
||||
}
|
||||
|
||||
console.info('');
|
||||
console.info('Formats:');
|
||||
if (bc.formats.length) {
|
||||
let formats = bc.formats.slice();
|
||||
formats.sort();
|
||||
if (!formats[0]) {
|
||||
formats[0] = '(bin)';
|
||||
}
|
||||
console.warn(' ', formats.join('\n '));
|
||||
} else {
|
||||
console.info(' (none)');
|
||||
}
|
||||
|
||||
// sort -u -k1 builds.tsv | rg -v '^#|^https?:' | rg -i arm
|
||||
// cut -f1 builds.tsv | sort -u -k1 | rg -v '^#|^https?:' | rg -i arm
|
||||
}
|
||||
|
||||
if (module === require.main) {
|
||||
let times = [];
|
||||
let now = Date.now();
|
||||
main()
|
||||
.then(async function () {
|
||||
let then = Date.now();
|
||||
let delta = then - now;
|
||||
times.push(delta);
|
||||
now = then;
|
||||
await main();
|
||||
then = Date.now();
|
||||
delta = then - now;
|
||||
times.push(delta);
|
||||
})
|
||||
.then(function () {
|
||||
console.info('');
|
||||
console.info('Run times');
|
||||
for (let delta of times) {
|
||||
let s = delta / 1000;
|
||||
console.info(` ${s}`);
|
||||
}
|
||||
|
||||
function forceExit() {
|
||||
console.warn(`warn: dangling event loop reference`);
|
||||
process.exit(0);
|
||||
}
|
||||
let exitTimeout = setTimeout(forceExit, 250);
|
||||
exitTimeout.unref();
|
||||
})
|
||||
.catch(function (err) {
|
||||
console.error(err.stack || err);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
258
_webi/normalize.js
Normal file
258
_webi/normalize.js
Normal file
@@ -0,0 +1,258 @@
|
||||
'use strict';
|
||||
|
||||
// this may need customizations between packages
|
||||
var osMap = {
|
||||
macos: /(\b|_)(apple|os(\s_-)?x\b|mac|darwin|iPhone|iOS|iPad)/i,
|
||||
linux: /(\b|_)(linux)/i,
|
||||
freebsd: /(\b|_)(freebsd)/i,
|
||||
windows: /(\b|_)(win|microsoft|msft)/i,
|
||||
sunos: /(\b|_)(sun)/i,
|
||||
aix: /(\b|_)(aix)/i,
|
||||
};
|
||||
|
||||
var maps = {
|
||||
oses: {},
|
||||
arches: {},
|
||||
libcs: {},
|
||||
formats: {},
|
||||
};
|
||||
|
||||
Object.keys(osMap).forEach(function (name) {
|
||||
maps.oses[name] = true;
|
||||
});
|
||||
|
||||
var formats = ['zip', 'xz', 'tar', 'pkg', 'msi', 'git', 'exe', 'dmg', 'git'];
|
||||
formats.forEach(function (name) {
|
||||
maps.formats[name] = true;
|
||||
});
|
||||
|
||||
// evaluation order matters
|
||||
// (i.e. otherwise x86 and x64 can cross match)
|
||||
var arches = [
|
||||
// arm 7 cannot be confused with arm64
|
||||
'armv7l',
|
||||
// amd64 is more likely than arm64
|
||||
'amd64',
|
||||
// arm6 has the same prefix as arm64
|
||||
'armv6l',
|
||||
// arm64 is more likely than arm6, and should be the default
|
||||
'arm64',
|
||||
'x86',
|
||||
'ppc64le',
|
||||
'ppc64',
|
||||
's390x',
|
||||
];
|
||||
// Used for detecting system arch from package download url, for example:
|
||||
//
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-aarch64-linux-musl.tar.gz
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-arm-linux-musleabihf.tar.gz
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-armv7-linux-musleabihf.tar.gz
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-x86_64-linux-musl.tar.gz
|
||||
//
|
||||
var archMap = {
|
||||
armv7l: /(\b|_)(armv?7l?)/i,
|
||||
//amd64: /(amd.?64|x64|[_\-]64)/i,
|
||||
amd64:
|
||||
/(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)64([_\-]?bit)?(\b|_)/i,
|
||||
//x86: /(86)(\b|_)/i,
|
||||
armv6l: /(\b|_)(aarch32|armv?6l?)(\b|_)/i,
|
||||
arm64: /(\b|_)((aarch|arm)64|arm)/i,
|
||||
x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32)([_\-]?bit)(\b|_)/i,
|
||||
ppc64le: /(\b|_)(ppc64le)/i,
|
||||
ppc64: /(\b|_)(ppc64)(\b|_)/i,
|
||||
s390x: /(\b|_)(s390x)/i,
|
||||
};
|
||||
arches.forEach(function (name) {
|
||||
maps.arches[name] = true;
|
||||
});
|
||||
|
||||
var libcs = ['none', 'musl', 'gnu', 'msvc', 'libc'];
|
||||
libcs.forEach(function (name) {
|
||||
maps.libcs[name] = true;
|
||||
});
|
||||
|
||||
function normalize(all) {
|
||||
/* jshint maxcomplexity:50 */
|
||||
/* jshint maxdepth:10 */
|
||||
var supported = {
|
||||
oses: {},
|
||||
arches: {},
|
||||
libcs: {},
|
||||
formats: {},
|
||||
};
|
||||
|
||||
for (let rel of all.releases) {
|
||||
rel.version = rel.version.replace(/^v/i, '');
|
||||
|
||||
if (!rel.name) {
|
||||
rel.name = rel.download.replace(/.*\//, '');
|
||||
}
|
||||
|
||||
if (!rel.os) {
|
||||
rel.os = 'unknown';
|
||||
|
||||
let osNames = Object.keys(osMap);
|
||||
for (let osName of osNames) {
|
||||
let relName = rel.name || rel.download;
|
||||
let osRegExp = osMap[osName];
|
||||
let matches = osRegExp.test(relName);
|
||||
if (matches) {
|
||||
rel.os = osName;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
supported.oses[rel.os] = true;
|
||||
|
||||
if (!rel.arch) {
|
||||
for (let arch of arches) {
|
||||
let name = rel.name || rel.download;
|
||||
let isArch = name.match(archMap[arch]);
|
||||
if (isArch) {
|
||||
rel.arch = arch;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!rel.arch) {
|
||||
if ('macos' === rel.os) {
|
||||
rel.arch = 'amd64';
|
||||
}
|
||||
}
|
||||
supported.arches[rel.arch] = true;
|
||||
|
||||
// note: depends on rel.os
|
||||
if (!rel.libc) {
|
||||
let isMusl;
|
||||
let isMsvc;
|
||||
let isStatic;
|
||||
let isGnu;
|
||||
|
||||
// extra blocks to prevent copy pasta errors
|
||||
|
||||
{
|
||||
let muslRe = /(\b|_)(musl)(\b|_)/i;
|
||||
isMusl = muslRe.test(rel.download) || muslRe.test(rel.name);
|
||||
}
|
||||
|
||||
{
|
||||
let msvcRe = /(\b|_)(msvc)(\b|_)/i;
|
||||
isMsvc = msvcRe.test(rel.download) || msvcRe.test(rel.name);
|
||||
}
|
||||
|
||||
{
|
||||
let staticRe = /(\b|_)(static)(\b|_)/i;
|
||||
isStatic = staticRe.test(rel.download) || staticRe.test(rel.name);
|
||||
}
|
||||
|
||||
{
|
||||
let gnuRe = /(\b|_)(gnu|glibc|libc)(\b|_)/i;
|
||||
isGnu = gnuRe.test(rel.download) || gnuRe.test(rel.name);
|
||||
}
|
||||
|
||||
if (isMusl) {
|
||||
// we specifically tag things that need musl++ in their own releases
|
||||
rel.libc = 'none';
|
||||
} else if (isStatic) {
|
||||
rel.libc = 'none';
|
||||
} else if (isGnu) {
|
||||
rel.libc = 'gnu';
|
||||
if (rel.os === 'windows') {
|
||||
// windows gnu is static
|
||||
rel.libc = 'none';
|
||||
} else if (rel.os === 'darwin') {
|
||||
// if glibc is required on macos, it'll be static
|
||||
rel.libc = 'none';
|
||||
}
|
||||
} else if (isMsvc) {
|
||||
rel.libc = 'msvc';
|
||||
} else {
|
||||
// The default is no requirement for any particular libc
|
||||
// (Go, Zig, POSIX Shell, JS, etc)
|
||||
// and hopefully we never have to worry about mingw and friends
|
||||
rel.libc = 'none';
|
||||
}
|
||||
}
|
||||
supported.libcs[rel.libc] = true;
|
||||
|
||||
var tarExt;
|
||||
if (!rel.ext) {
|
||||
// pkg-v1.0.tar.gz => ['gz', 'tar', '0', 'pkg-v1']
|
||||
// pkg-v1.0.tar => ['tar', '0' ,'pkg-v1']
|
||||
// pkg-v1.0.zip => ['zip', '0', 'pkg-v1']
|
||||
var exts = (rel.name || rel.download).split('.');
|
||||
if (1 === exts.length) {
|
||||
// for bare releases in the format of foo-linux-amd64
|
||||
rel.ext = 'exe';
|
||||
}
|
||||
exts = exts.reverse().slice(0, 2);
|
||||
if ('tar' === exts[1]) {
|
||||
rel.ext = exts.reverse().join('.');
|
||||
tarExt = 'tar';
|
||||
} else if ('tgz' === exts[0]) {
|
||||
rel.ext = 'tar.gz';
|
||||
tarExt = 'tar';
|
||||
} else {
|
||||
rel.ext = exts[0];
|
||||
}
|
||||
if (/\-|linux|mac|os[_\-]?x|arm|amd|86|64|mip/i.test(rel.ext)) {
|
||||
// for bare releases in the format of foo.linux-amd64
|
||||
rel.ext = 'exe';
|
||||
}
|
||||
}
|
||||
supported.formats[tarExt || rel.ext] = true;
|
||||
|
||||
if (!rel.channel) {
|
||||
// basically like this: (+.-_)(beta|rc)(0-9)(+.-_)
|
||||
// matches:
|
||||
// - v1.0-beta
|
||||
// - v1.0-beta1.1
|
||||
// - v1.0-beta-11
|
||||
// won't match:
|
||||
// - v1.0beta
|
||||
// - v1.0-beta1b
|
||||
let isBetaRe = /(\b|_)(preview|rc|beta|alpha)(\d+)(\b|_)/;
|
||||
let isBeta = isBetaRe.test(rel.name);
|
||||
if (isBeta) {
|
||||
rel.channel = 'beta';
|
||||
} else {
|
||||
rel.channel = 'stable';
|
||||
}
|
||||
}
|
||||
|
||||
if (all.download) {
|
||||
rel.download = all.download.replace(/{{ download }}/, rel.download);
|
||||
}
|
||||
}
|
||||
|
||||
all.oses = Object.keys(supported.oses).filter(function (name) {
|
||||
return maps.oses[name];
|
||||
});
|
||||
all.arches = Object.keys(supported.arches).filter(function (name) {
|
||||
return maps.arches[name];
|
||||
});
|
||||
all.libcs = Object.keys(supported.libcs).filter(function (name) {
|
||||
return maps.libcs[name];
|
||||
});
|
||||
all.formats = Object.keys(supported.formats).filter(function (name) {
|
||||
return maps.formats[name];
|
||||
});
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = normalize;
|
||||
module.exports._debug = function (all) {
|
||||
all = normalize(all);
|
||||
all.releases = all.releases
|
||||
.filter(function (r) {
|
||||
return ['windows', 'macos', 'linux'].includes(r.os) && 'amd64' === r.arch;
|
||||
})
|
||||
.slice(0, 10);
|
||||
return all;
|
||||
};
|
||||
// NOT in order of priority (which would be tar, xz, zip, ...)
|
||||
module.exports.formats = formats;
|
||||
module.exports.arches = arches;
|
||||
module.exports.libcs = libcs;
|
||||
module.exports.formatsMap = maps.formats;
|
||||
@@ -1,156 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
#350 check if windows user run as admin
|
||||
|
||||
# this allows us to call ps1 files, which allows us to have spaces in filenames
|
||||
# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
|
||||
# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
|
||||
Set-ExecutionPolicy -Scope Process Bypass
|
||||
|
||||
# If a command returns an error, halt the script.
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
|
||||
$ProgressPreference = 'SilentlyContinue'
|
||||
|
||||
$Env:WEBI_HOST = 'https://webinstall.dev'
|
||||
#$Env:WEBI_PKG = 'node@lts'
|
||||
#$Env:PKG_NAME = node
|
||||
#$Env:WEBI_VERSION = v12.16.2
|
||||
#$Env:WEBI_GIT_TAG = 12.16.2
|
||||
#$Env:WEBI_GIT_COMMIT_HASH =
|
||||
#$Env:WEBI_PKG_URL = "https://.../node-....zip"
|
||||
#$Env:WEBI_PKG_FILE = "node-v12.16.2-win-x64.zip"
|
||||
#$Env:WEBI_PKG_PATHNAME = "node-v12.16.2-win-x64.zip"
|
||||
|
||||
# Switch to userprofile
|
||||
Push-Location $Env:USERPROFILE
|
||||
|
||||
# Make paths
|
||||
New-Item -Path "$Env:USERPROFILE\Downloads" -ItemType Directory -Force | Out-Null
|
||||
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
|
||||
New-Item -Path "$Env:USERPROFILE\.local\opt" -ItemType Directory -Force | Out-Null
|
||||
|
||||
# {{ baseurl }}
|
||||
# {{ version }}
|
||||
|
||||
# See
|
||||
# - <https://superuser.com/q/1264444>
|
||||
# - <https://stackoverflow.com/a/60572643/151312>
|
||||
$Esc = [char]27
|
||||
$TTask = "${Esc}[36m"
|
||||
$TName = "${Esc}[1m${Esc}[32m"
|
||||
$TUrl = "${Esc}[2m"
|
||||
$TPath = "${Esc}[2m${Esc}[32m"
|
||||
$TCmd = "${Esc}[2m${Esc}[35m"
|
||||
$TDim = "${Esc}[2m"
|
||||
$TReset = "${Esc}[0m"
|
||||
|
||||
function Invoke-DownloadUrl {
|
||||
param (
|
||||
[string]$URL,
|
||||
[string]$Params,
|
||||
[string]$Path,
|
||||
[switch]$Force
|
||||
)
|
||||
|
||||
if (Test-Path -Path "$Path") {
|
||||
if (-not $Force.IsPresent) {
|
||||
Write-Host " ${TDim}Found${TReset} $Path"
|
||||
return
|
||||
}
|
||||
Write-Host " Updating ${TDim}${Path}${TDim}"
|
||||
}
|
||||
|
||||
$TmpPath = "${Path}.part"
|
||||
Remove-Item -Path $TmpPath -Force -ErrorAction Ignore
|
||||
|
||||
Write-Host " Downloading ${TDim}from${TReset}"
|
||||
Write-Host " ${TDim}${URL}${TReset}"
|
||||
if ($Params.Length -ne 0) {
|
||||
Write-Host " ?$Params"
|
||||
$URL = "${URL}?${Params}"
|
||||
}
|
||||
curl.exe '-#' -f -sS -A $Env:WEBI_UA -L $URL | Out-File $TmpPath
|
||||
|
||||
Remove-Item -Path $Path -Force -ErrorAction Ignore
|
||||
Move-Item $TmpPath $Path
|
||||
Write-Host " Saved ${TPath}${Path}${TReset}"
|
||||
}
|
||||
|
||||
function Get-UserAgent {
|
||||
# This is the canonical CPU arch when the process is emulated
|
||||
$my_arch = "$Env:PROCESSOR_ARCHITEW6432"
|
||||
|
||||
if ($my_arch -eq $null -or $my_arch -eq "") {
|
||||
# This is the canonical CPU arch when the process is native
|
||||
$my_arch = "$Env:PROCESSOR_ARCHITECTURE"
|
||||
}
|
||||
|
||||
if ($my_arch -eq "AMD64") {
|
||||
# Because PowerShell is sometimes AMD64 on Windows 10 ARM
|
||||
# See https://oofhours.com/2020/02/04/powershell-on-windows-10-arm64/
|
||||
$my_os_arch = (Get-CimInstance -ClassName Win32_OperatingSystem).OSArchitecture
|
||||
|
||||
# Using -clike because of the trailing newline
|
||||
if ($my_os_arch -clike "ARM 64*") {
|
||||
$my_arch = "ARM64"
|
||||
}
|
||||
}
|
||||
|
||||
"PowerShell+curl Windows/10+ $my_arch msvc"
|
||||
}
|
||||
|
||||
function webi_path_add($pathname) {
|
||||
# C:\Users\me => C:/Users/me
|
||||
$my_home = $Env:UserProfile
|
||||
$my_home = $my_home.replace('\\', '/')
|
||||
$my_home_re = [regex]::escape($my_home)
|
||||
|
||||
# ~/bin => %USERPROFILE%/bin
|
||||
$pathname = $pathname.replace('~/', "$Env:UserProfile/")
|
||||
|
||||
# C:\Users\me\bin => %USERPROFILE%/bin
|
||||
$my_pathname = $pathname.replace('\\', '/')
|
||||
$my_pathname = $my_pathname -ireplace $my_home_re, "%USERPROFILE%"
|
||||
|
||||
$all_user_paths = [Environment]::GetEnvironmentVariable("Path", "User")
|
||||
$user_paths = "${all_user_paths}".Trim(';').Split(';')
|
||||
$exists_in_path = $false
|
||||
foreach ($user_path in $user_paths) {
|
||||
# C:\Users\me\bin => %USERPROFILE%/bin
|
||||
$my_user_path = $user_path.replace('\\', '/')
|
||||
$my_user_path = $my_user_path -ireplace $my_home_re, "%USERPROFILE%"
|
||||
|
||||
if ($my_user_path -ieq $my_pathname) {
|
||||
$exists_in_path = $true
|
||||
}
|
||||
}
|
||||
if (-not $exists_in_path) {
|
||||
$all_user_paths = "${pathname};${all_user_paths}".Trim(';')
|
||||
[Environment]::SetEnvironmentVariable("Path", $all_user_paths, "User")
|
||||
$null = Sync-EnvPath
|
||||
}
|
||||
}
|
||||
|
||||
function Sync-EnvPath {
|
||||
$UserPath = [Environment]::GetEnvironmentVariable("Path", "User").Trim(';')
|
||||
$MachinePath = [Environment]::GetEnvironmentVariable("Path", "Machine").Trim(';')
|
||||
$Env:Path = "${UserPath};${MachinePath}"
|
||||
[Environment]::SetEnvironmentVariable("Path", $Env:Path)
|
||||
$Env:Path
|
||||
}
|
||||
|
||||
$Env:WEBI_UA = Get-UserAgent
|
||||
|
||||
#$has_local_bin = echo "$Env:PATH" | Select-String -Pattern '\.local.bin'
|
||||
#if (!$has_local_bin)
|
||||
#{
|
||||
webi_path_add ~/.local/bin
|
||||
#}
|
||||
|
||||
# {{ installer }}
|
||||
|
||||
webi_path_add ~/.local/bin
|
||||
|
||||
# Done
|
||||
Pop-Location
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1 +0,0 @@
|
||||
projects.js
|
||||
96
_webi/packages.js
Normal file
96
_webi/packages.js
Normal file
@@ -0,0 +1,96 @@
|
||||
'use strict';
|
||||
|
||||
var frontmarker = require('./frontmarker.js');
|
||||
var fs = require('fs');
|
||||
var path = require('path');
|
||||
|
||||
var pkgs = module.exports;
|
||||
pkgs.create = function (Pkgs, basepath) {
|
||||
if (!Pkgs) {
|
||||
Pkgs = {};
|
||||
}
|
||||
if (!basepath) {
|
||||
basepath = path.join(__dirname, '../');
|
||||
}
|
||||
|
||||
Pkgs.all = function () {
|
||||
return fs.promises.readdir(basepath).then(function (nodes) {
|
||||
var items = [];
|
||||
return nodes
|
||||
.reduce(function (p, node) {
|
||||
return p.then(function () {
|
||||
return pkgs.get(node).then(function (meta) {
|
||||
if (meta && '_' !== node[0]) {
|
||||
meta.name = node;
|
||||
items.push(meta);
|
||||
}
|
||||
});
|
||||
});
|
||||
}, Promise.resolve())
|
||||
.then(function () {
|
||||
return items;
|
||||
});
|
||||
});
|
||||
};
|
||||
|
||||
Pkgs.get = function (node) {
|
||||
return fs.promises.access(path.join(basepath, node)).then(function () {
|
||||
return Pkgs._get(node);
|
||||
});
|
||||
};
|
||||
Pkgs._get = function (node) {
|
||||
var curlbash = path.join(basepath, node, 'install.sh');
|
||||
var readme = path.join(basepath, node, 'README.md');
|
||||
var winstall = path.join(basepath, node, 'install.ps1');
|
||||
return Promise.all([
|
||||
fs.promises
|
||||
.readFile(readme, 'utf-8')
|
||||
.then(function (txt) {
|
||||
// TODO
|
||||
return frontmarker.parse(txt);
|
||||
})
|
||||
.catch(function (e) {
|
||||
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
|
||||
console.error("failed to read '" + node + "/README.md'");
|
||||
console.error(e);
|
||||
}
|
||||
}),
|
||||
fs.promises.access(curlbash).catch(function (e) {
|
||||
// no *nix installer
|
||||
curlbash = '';
|
||||
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
|
||||
console.error("failed to parse '" + node + "/install.sh'");
|
||||
console.error(e);
|
||||
}
|
||||
}),
|
||||
fs.promises.access(winstall).catch(function (e) {
|
||||
// no winstaller
|
||||
winstall = '';
|
||||
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
|
||||
console.error("failed to read '" + node + "/install.ps1'");
|
||||
console.error(e);
|
||||
}
|
||||
}),
|
||||
]).then(function (items) {
|
||||
var meta = items[0] || items[1];
|
||||
if (!meta) {
|
||||
// doesn't exist
|
||||
return;
|
||||
}
|
||||
meta.windows = !!winstall;
|
||||
meta.bash = !!curlbash;
|
||||
|
||||
return meta;
|
||||
});
|
||||
};
|
||||
|
||||
return Pkgs;
|
||||
};
|
||||
pkgs.create(pkgs);
|
||||
|
||||
if (module === require.main) {
|
||||
pkgs.all().then(function (data) {
|
||||
console.info('package info:');
|
||||
console.info(data);
|
||||
});
|
||||
}
|
||||
@@ -1,44 +0,0 @@
|
||||
'use strict';
|
||||
|
||||
var Parallel = module.exports;
|
||||
Parallel.run = async function (limit, arr, fn) {
|
||||
let index = 0;
|
||||
let actives = [];
|
||||
let results = [];
|
||||
limit = Math.min(limit, arr.length);
|
||||
|
||||
function launch() {
|
||||
let _index = index;
|
||||
let p = fn(arr[_index], _index, arr);
|
||||
|
||||
// some tasks may be synchronous
|
||||
// so we must push before removing
|
||||
actives.push(p);
|
||||
|
||||
p.then(function _resolve(result) {
|
||||
let i = actives.indexOf(p);
|
||||
actives.splice(i, 1);
|
||||
results[_index] = result;
|
||||
});
|
||||
|
||||
index += 1;
|
||||
}
|
||||
|
||||
// start tasks in parallel, up to limit
|
||||
for (; actives.length < limit; ) {
|
||||
launch();
|
||||
}
|
||||
|
||||
// keep the task queue full
|
||||
for (; index < arr.length; ) {
|
||||
// wait for one task to complete
|
||||
await Promise.race(actives);
|
||||
// add one task again
|
||||
launch();
|
||||
}
|
||||
|
||||
// wait for all remaining tasks
|
||||
await Promise.all(actives);
|
||||
|
||||
return results;
|
||||
};
|
||||
@@ -1,96 +0,0 @@
|
||||
'use strict';
|
||||
|
||||
var frontmarker = require('./frontmarker.js');
|
||||
var fs = require('fs');
|
||||
var path = require('path');
|
||||
|
||||
var pkgs = module.exports;
|
||||
pkgs.create = function (Pkgs, basepath) {
|
||||
if (!Pkgs) {
|
||||
Pkgs = {};
|
||||
}
|
||||
if (!basepath) {
|
||||
basepath = path.join(__dirname, '../');
|
||||
}
|
||||
|
||||
Pkgs.all = function () {
|
||||
return fs.promises.readdir(basepath).then(function (nodes) {
|
||||
var items = [];
|
||||
return nodes
|
||||
.reduce(function (p, node) {
|
||||
return p.then(function () {
|
||||
return pkgs.get(node).then(function (meta) {
|
||||
if (meta && '_' !== node[0]) {
|
||||
meta.name = node;
|
||||
items.push(meta);
|
||||
}
|
||||
});
|
||||
});
|
||||
}, Promise.resolve())
|
||||
.then(function () {
|
||||
return items;
|
||||
});
|
||||
});
|
||||
};
|
||||
|
||||
Pkgs.get = function (node) {
|
||||
return fs.promises.access(path.join(basepath, node)).then(function () {
|
||||
return Pkgs._get(node);
|
||||
});
|
||||
};
|
||||
Pkgs._get = function (node) {
|
||||
var curlbash = path.join(basepath, node, 'install.sh');
|
||||
var readme = path.join(basepath, node, 'README.md');
|
||||
var winstall = path.join(basepath, node, 'install.ps1');
|
||||
return Promise.all([
|
||||
fs.promises
|
||||
.readFile(readme, 'utf-8')
|
||||
.then(function (txt) {
|
||||
// TODO
|
||||
return frontmarker.parse(txt);
|
||||
})
|
||||
.catch(function (e) {
|
||||
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
|
||||
console.error("failed to read '" + node + "/README.md'");
|
||||
console.error(e);
|
||||
}
|
||||
}),
|
||||
fs.promises.access(curlbash).catch(function (e) {
|
||||
// no *nix installer
|
||||
curlbash = '';
|
||||
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
|
||||
console.error("failed to parse '" + node + "/install.sh'");
|
||||
console.error(e);
|
||||
}
|
||||
}),
|
||||
fs.promises.access(winstall).catch(function (e) {
|
||||
// no winstaller
|
||||
winstall = '';
|
||||
if ('ENOENT' !== e.code && 'ENOTDIR' !== e.code) {
|
||||
console.error("failed to read '" + node + "/install.ps1'");
|
||||
console.error(e);
|
||||
}
|
||||
}),
|
||||
]).then(function (items) {
|
||||
var meta = items[0] || items[1];
|
||||
if (!meta) {
|
||||
// doesn't exist
|
||||
return;
|
||||
}
|
||||
meta.windows = !!winstall;
|
||||
meta.bash = !!curlbash;
|
||||
|
||||
return meta;
|
||||
});
|
||||
};
|
||||
|
||||
return Pkgs;
|
||||
};
|
||||
pkgs.create(pkgs);
|
||||
|
||||
if (module === require.main) {
|
||||
pkgs.all().then(function (data) {
|
||||
console.info('package info:');
|
||||
console.info(data);
|
||||
});
|
||||
}
|
||||
@@ -1,19 +1,34 @@
|
||||
'use strict';
|
||||
|
||||
var Installers = module.exports;
|
||||
|
||||
var Crypto = require('crypto');
|
||||
var Fs = require('node:fs/promises');
|
||||
var fs = require('node:fs');
|
||||
var path = require('node:path');
|
||||
var request = require('@root/request');
|
||||
var _normalize = require('../_webi/normalize.js');
|
||||
|
||||
var reInstallTpl = /\s*#?\s*{{ installer }}/;
|
||||
|
||||
var Releases = module.exports;
|
||||
Releases.get = async function (pkgdir) {
|
||||
let get;
|
||||
try {
|
||||
get = require(path.join(pkgdir, 'releases.js'));
|
||||
} catch (e) {
|
||||
let err = new Error('no releases.js for', pkgdir.split(/[\/\\]+/).pop());
|
||||
err.code = 'E_NO_RELEASE';
|
||||
throw err;
|
||||
}
|
||||
|
||||
let all = await get(request);
|
||||
|
||||
return _normalize(all);
|
||||
};
|
||||
|
||||
function padScript(txt) {
|
||||
return txt.replace(/^/g, ' ');
|
||||
}
|
||||
|
||||
var BAD_SH_RE = /[<>'"`$\\]/;
|
||||
Installers.renderBash = async function (
|
||||
Releases.renderBash = async function (
|
||||
pkgdir,
|
||||
rel,
|
||||
{ baseurl, pkg, tag, ver, os = '', arch = '', libc = '', formats },
|
||||
@@ -24,7 +39,10 @@ Installers.renderBash = async function (
|
||||
if (!tag) {
|
||||
tag = '';
|
||||
}
|
||||
let installTxt = await Fs.readFile(path.join(pkgdir, 'install.sh'), 'utf8');
|
||||
let installTxt = await fs.promises.readFile(
|
||||
path.join(pkgdir, 'install.sh'),
|
||||
'utf8',
|
||||
);
|
||||
installTxt = padScript(installTxt);
|
||||
var vers = rel.version.split('.');
|
||||
var v = {
|
||||
@@ -37,8 +55,8 @@ Installers.renderBash = async function (
|
||||
.replace(/^-/, ''),
|
||||
};
|
||||
var pkgFile = rel.filename || rel.name;
|
||||
let tplTxt = await Fs.readFile(
|
||||
path.join(__dirname, 'package-install.tpl.sh'),
|
||||
let tplTxt = await fs.promises.readFile(
|
||||
path.join(__dirname, 'install-package.tpl.sh'),
|
||||
'utf8',
|
||||
);
|
||||
// ex: 'node@lts' or 'node'
|
||||
@@ -72,17 +90,14 @@ Installers.renderBash = async function (
|
||||
]
|
||||
.join(',')
|
||||
.replace(/'/g, '');
|
||||
|
||||
let webiChecksum = await Installers.getWebiShChecksum();
|
||||
let envReplacements = [
|
||||
['WEBI_CHECKSUM', webiChecksum],
|
||||
['WEBI_PKG', webiPkg],
|
||||
['WEBI_HOST', baseurl],
|
||||
['WEBI_OS', os],
|
||||
['WEBI_ARCH', arch],
|
||||
['WEBI_LIBC', libc],
|
||||
['WEBI_TAG', tag],
|
||||
['WEBI_RELEASES', `${baseurl}${releaseUrl}`],
|
||||
['WEBI_RELEASES', `${baseurl}/${releaseUrl}`],
|
||||
['WEBI_CSV', releaseCsv],
|
||||
['WEBI_VERSION', rel.version],
|
||||
['WEBI_MAJOR', v.major],
|
||||
@@ -93,18 +108,16 @@ Installers.renderBash = async function (
|
||||
['WEBI_GIT_TAG', rel.git_tag], // TODO replace with branch
|
||||
['WEBI_LTS', rel.lts],
|
||||
['WEBI_CHANNEL', rel.channel],
|
||||
['WEBI_EXT', rel.ext],
|
||||
['WEBI_EXT', rel.ext.replace(/tar.*/, 'tar')],
|
||||
['WEBI_FORMATS', formats.join(',')],
|
||||
['WEBI_PKG_URL', rel.download],
|
||||
['WEBI_PKG_PATHNAME', pkgFile],
|
||||
['WEBI_PKG_FILE', pkgFile], // TODO replace with pathname
|
||||
['PKG_NAME', pkg],
|
||||
['PKG_STABLE', rel.stable],
|
||||
['PKG_LATEST', rel.latest],
|
||||
['PKG_OSES', (rel.oses || []).join(' ')],
|
||||
['PKG_ARCHES', (rel.arches || []).join(' ')],
|
||||
['PKG_LIBCS', (rel.libcs || []).join(' ')],
|
||||
['PKG_FORMATS', (rel.formats || []).join(' ')],
|
||||
['PKG_OSES', rel.oses],
|
||||
['PKG_ARCHES', rel.arches],
|
||||
['PKG_LIBCS', rel.libcs],
|
||||
['PKG_FORMATS', (rel.formats || []).join(',')],
|
||||
];
|
||||
|
||||
for (let env of envReplacements) {
|
||||
@@ -114,7 +127,7 @@ Installers.renderBash = async function (
|
||||
// #export WEBI_FOO=xyz => export WEBI_FOO='123'
|
||||
// export WEBI_FOO= => export WEBI_FOO='123'
|
||||
let envRe = new RegExp(
|
||||
`^[ \\t]*#?[ \\t]*(export[ \\t])?[ \\t]*(${name})=.*`,
|
||||
`^[ \\t]*#?[ \\t]*(export\\s)?[ \\t]*(${name})=.*`,
|
||||
'm',
|
||||
);
|
||||
if (BAD_SH_RE.test(value)) {
|
||||
@@ -132,7 +145,7 @@ Installers.renderBash = async function (
|
||||
return tplTxt;
|
||||
};
|
||||
|
||||
Installers.renderPowerShell = async function (
|
||||
Releases.renderPowerShell = async function (
|
||||
pkgdir,
|
||||
rel,
|
||||
{ baseurl, pkg, tag, ver, os, arch, libc = '', formats },
|
||||
@@ -143,7 +156,10 @@ Installers.renderPowerShell = async function (
|
||||
if (!tag) {
|
||||
tag = '';
|
||||
}
|
||||
let installTxt = await Fs.readFile(path.join(pkgdir, 'install.ps1'), 'utf8');
|
||||
let installTxt = await fs.promises.readFile(
|
||||
path.join(pkgdir, 'install.ps1'),
|
||||
'utf8',
|
||||
);
|
||||
installTxt = padScript(installTxt);
|
||||
/*
|
||||
var vers = rel.version.split('.');
|
||||
@@ -157,8 +173,8 @@ Installers.renderPowerShell = async function (
|
||||
.replace(/^-/, '')
|
||||
};
|
||||
*/
|
||||
let tplTxt = await Fs.readFile(
|
||||
path.join(__dirname, 'package-install.tpl.ps1'),
|
||||
let tplTxt = await fs.promises.readFile(
|
||||
path.join(__dirname, 'install-package.tpl.ps1'),
|
||||
'utf8',
|
||||
);
|
||||
var pkgver = pkg + '@' + ver;
|
||||
@@ -202,32 +218,3 @@ Installers.renderPowerShell = async function (
|
||||
.replace(reInstallTpl, '\n' + installTxt)
|
||||
);
|
||||
};
|
||||
|
||||
var _webiShMeta = {
|
||||
stale: 10 * 1000,
|
||||
updated_at: 0,
|
||||
checksum: '',
|
||||
mtime: 0,
|
||||
};
|
||||
Installers.getWebiShChecksum = async function () {
|
||||
let now = Date.now();
|
||||
let ago = now - _webiShMeta.updated_at;
|
||||
if (ago <= _webiShMeta.stale) {
|
||||
return _webiShMeta.checksum;
|
||||
}
|
||||
|
||||
let webiPath = path.join(__dirname, '../webi/webi.sh');
|
||||
let stat = await Fs.stat(webiPath);
|
||||
if (stat.mtimeMs === _webiShMeta.mtime) {
|
||||
return _webiShMeta.checksum;
|
||||
}
|
||||
|
||||
let webiBuf = await Fs.readFile(webiPath, null);
|
||||
let webiHash = Crypto.createHash('sha1').update(webiBuf).digest('hex');
|
||||
let webiChecksum = webiHash.slice(0, 8);
|
||||
|
||||
_webiShMeta.mtime = stat.mtimeMs;
|
||||
_webiShMeta.updated_at = now;
|
||||
_webiShMeta.checksum = webiChecksum;
|
||||
return _webiShMeta.checksum;
|
||||
};
|
||||
@@ -1,16 +1,20 @@
|
||||
'use strict';
|
||||
|
||||
var InstallerServer = module.exports;
|
||||
var Installers = module.exports;
|
||||
|
||||
let Fs = require('fs/promises');
|
||||
let Path = require('path');
|
||||
var Crypto = require('crypto');
|
||||
var Fs = require('fs/promises');
|
||||
var path = require('path');
|
||||
|
||||
let HostTargets = require('./build-classifier/host-targets.js');
|
||||
let Builds = require('./builds.js');
|
||||
let Installers = require('./installers.js');
|
||||
var uaDetect = require('./ua-detect.js');
|
||||
var packages = require('./packages.js');
|
||||
var Releases = require('./releases.js');
|
||||
|
||||
InstallerServer.INSTALLERS_DIR = Path.join(__dirname, '..');
|
||||
InstallerServer.serveInstaller = async function (
|
||||
// handlers caching and transformation, probably should be broken down
|
||||
var getReleases = require('./transform-releases.js');
|
||||
|
||||
Installers.INSTALLERS_DIR = path.join(__dirname, '..');
|
||||
Installers.serveInstaller = async function (
|
||||
baseurl,
|
||||
ua,
|
||||
pkg,
|
||||
@@ -19,217 +23,116 @@ InstallerServer.serveInstaller = async function (
|
||||
formats,
|
||||
libc,
|
||||
) {
|
||||
let unameAgent = ua;
|
||||
let projectName = pkg;
|
||||
let [rel, tmplParams] = await InstallerServer.helper({
|
||||
unameAgent,
|
||||
projectName,
|
||||
let [rel, opts] = await Installers.helper({
|
||||
ua,
|
||||
pkg,
|
||||
tag,
|
||||
formats,
|
||||
libc,
|
||||
});
|
||||
Object.assign(tmplParams, {
|
||||
Object.assign(opts, {
|
||||
baseurl,
|
||||
});
|
||||
|
||||
var pkgdir = Path.join(InstallerServer.INSTALLERS_DIR, projectName);
|
||||
var pkgdir = path.join(Installers.INSTALLERS_DIR, pkg);
|
||||
if ('ps1' === ext) {
|
||||
return Installers.renderPowerShell(pkgdir, rel, tmplParams);
|
||||
return Releases.renderPowerShell(pkgdir, rel, opts);
|
||||
}
|
||||
return Installers.renderBash(pkgdir, rel, tmplParams);
|
||||
return Releases.renderBash(pkgdir, rel, opts);
|
||||
};
|
||||
Installers.helper = async function ({ ua, pkg, tag, formats, libc }) {
|
||||
// TODO put some of this in a middleware? or common function?
|
||||
|
||||
// TODO put some of this in a middleware? or common function?
|
||||
// TODO maybe move package/version/lts/channel detection into getReleases
|
||||
InstallerServer.helper = async function ({
|
||||
unameAgent,
|
||||
projectName,
|
||||
tag,
|
||||
formats,
|
||||
libc,
|
||||
}) {
|
||||
console.log(`dbg: Installer User-Agent: ${unameAgent}`);
|
||||
// TODO maybe move package/version/lts/channel detection into getReleases
|
||||
var ver = tag.replace(/^v/, '');
|
||||
var lts;
|
||||
var channel;
|
||||
|
||||
let releaseTarget = toReleaseTarget(tag);
|
||||
let hostFormats = formats;
|
||||
let terms = unameAgent.split(/[\s\/]+/g);
|
||||
let hostTarget = {};
|
||||
try {
|
||||
void HostTargets.termsToTarget(hostTarget, terms);
|
||||
} catch (e) {
|
||||
// if we can't guarantee the results...
|
||||
// "in the face of ambiguity, refuse the temptation to guess"
|
||||
throw e;
|
||||
}
|
||||
console.log(`dbg: Installer Host Target:`);
|
||||
console.log(hostTarget);
|
||||
|
||||
if (!hostTarget.os) {
|
||||
throw new Error(`OS could not be identified by User-Agent '${unameAgent}'`);
|
||||
switch (ver) {
|
||||
case 'latest':
|
||||
ver = '';
|
||||
channel = 'stable';
|
||||
break;
|
||||
case 'lts':
|
||||
lts = true;
|
||||
channel = 'stable';
|
||||
ver = '';
|
||||
break;
|
||||
case 'stable':
|
||||
channel = 'stable';
|
||||
ver = '';
|
||||
break;
|
||||
case 'beta':
|
||||
channel = 'beta';
|
||||
ver = '';
|
||||
break;
|
||||
case 'dev':
|
||||
channel = 'dev';
|
||||
ver = '';
|
||||
break;
|
||||
}
|
||||
|
||||
console.log(`dbg: Get Project Installer Type for '${projectName}':`);
|
||||
let proj = await Builds.getProjectType(projectName);
|
||||
if (proj.type === 'alias') {
|
||||
console.log(`dbg: alias`, proj);
|
||||
projectName = proj.detail;
|
||||
proj = await Builds.getProjectType(projectName); // an alias should never resolve to an alias
|
||||
var myOs = uaDetect.os(ua);
|
||||
var myArch = uaDetect.arch(ua);
|
||||
var myLibc;
|
||||
if (libc) {
|
||||
myLibc = uaDetect.libc(libc);
|
||||
}
|
||||
console.log(`dbg: proj`, proj);
|
||||
|
||||
let validTypes = ['selfhosted', 'valid'];
|
||||
if (!validTypes.includes(proj.type)) {
|
||||
let msg = `'${projectName}' doesn't have an installer: '${proj.type}': '${proj.detail}'`;
|
||||
let err = new Error(msg);
|
||||
err.code = 'ENOENT';
|
||||
throw err;
|
||||
if (!myLibc) {
|
||||
myLibc = uaDetect.libc(ua);
|
||||
}
|
||||
if (!myLibc) {
|
||||
myLibc = 'libc';
|
||||
}
|
||||
|
||||
let tmplParams = {
|
||||
pkg: projectName,
|
||||
tag: tag,
|
||||
os: hostTarget.os,
|
||||
arch: hostTarget.arch,
|
||||
libc: hostTarget.libc,
|
||||
formats: hostFormats,
|
||||
let cfg = await packages.get(pkg);
|
||||
let releaseQuery = {
|
||||
pkg: cfg.alias || pkg,
|
||||
ver,
|
||||
os: myOs,
|
||||
arch: myArch,
|
||||
libc: myLibc,
|
||||
lts,
|
||||
channel,
|
||||
// TODO use formats for sorting, not exclusion
|
||||
// (it's better to install xz or report an error to install zip)
|
||||
formats,
|
||||
limit: 1,
|
||||
};
|
||||
Object.assign(tmplParams, releaseTarget);
|
||||
console.log('tmplParams', tmplParams);
|
||||
|
||||
let errPackage = {
|
||||
name: 'doesntexist.ext',
|
||||
version: '0.0.0',
|
||||
lts: '-',
|
||||
channel: 'error',
|
||||
date: '1970-01-01',
|
||||
os: hostTarget.os || '-',
|
||||
arch: hostTarget.arch || '-',
|
||||
libc: hostTarget.libc || '-',
|
||||
ext: 'err',
|
||||
download: 'https://example.com/doesntexist.ext',
|
||||
comment:
|
||||
'No matches found. Could be bad or missing version info' +
|
||||
',' +
|
||||
"Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
|
||||
let rels = await getReleases(releaseQuery);
|
||||
|
||||
var rel = rels.releases[0];
|
||||
var opts = {
|
||||
pkg: cfg.alias || pkg,
|
||||
ver,
|
||||
tag,
|
||||
os: myOs,
|
||||
arch: myArch,
|
||||
libc: myLibc,
|
||||
lts,
|
||||
channel,
|
||||
formats,
|
||||
limit: 1,
|
||||
};
|
||||
|
||||
if (proj.type === 'selfhosted') {
|
||||
return [errPackage, tmplParams];
|
||||
}
|
||||
|
||||
let projInfo = await Builds.getPackage({
|
||||
name: projectName,
|
||||
date: new Date(),
|
||||
});
|
||||
let latestVersions = Builds.enumerateLatestVersions(projInfo);
|
||||
//console.log('projInfo', projInfo);
|
||||
|
||||
let buildTargetInfo = {
|
||||
triplets: projInfo.triplets,
|
||||
oses: projInfo.oses,
|
||||
arches: projInfo.arches,
|
||||
libcs: projInfo.libcs,
|
||||
formats: projInfo.formats,
|
||||
latest: latestVersions.latest,
|
||||
stable: latestVersions.stable,
|
||||
};
|
||||
|
||||
// TODO .findMatchingPackages() should probably account for this
|
||||
let hasOs = projInfo.oses.includes(hostTarget.os);
|
||||
let maybePosix = !hasOs && hostTarget.os !== 'windows';
|
||||
if (maybePosix) {
|
||||
let posixes = ['posix_2017', 'posix_2024'];
|
||||
for (let posixYear of posixes) {
|
||||
let hasPosix = projInfo.oses.includes(posixYear);
|
||||
if (hasPosix) {
|
||||
hasOs = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!hasOs) {
|
||||
hasOs = projInfo.oses.includes('ANYOS');
|
||||
}
|
||||
|
||||
if (!hasOs) {
|
||||
let pkg1 = Object.assign(buildTargetInfo, errPackage);
|
||||
return [pkg1, tmplParams];
|
||||
}
|
||||
|
||||
let targetRelease = Builds.findMatchingPackages(
|
||||
projInfo,
|
||||
hostTarget,
|
||||
releaseTarget,
|
||||
rel = Object.assign(
|
||||
{
|
||||
oses: rels.oses,
|
||||
arches: rels.arches,
|
||||
libcs: rels.libcs,
|
||||
formats: rels.formats,
|
||||
},
|
||||
rel,
|
||||
);
|
||||
// { triplet: `${os}-${arch}-${libc}`, packages: targetPackages
|
||||
// , latest: projInfo.versions[0], versions: matchInfo
|
||||
// }
|
||||
|
||||
if (!targetRelease?.packages) {
|
||||
let pkg1 = Object.assign(buildTargetInfo, errPackage);
|
||||
return [pkg1, tmplParams];
|
||||
}
|
||||
|
||||
let buildPkg = Builds.selectPackage(targetRelease.packages, hostFormats);
|
||||
let ext = buildPkg.ext || '.exe';
|
||||
if (ext.startsWith('.')) {
|
||||
ext = ext.slice(1);
|
||||
}
|
||||
|
||||
let version = targetRelease.version;
|
||||
if (version.startsWith('v')) {
|
||||
version = version.slice(1);
|
||||
}
|
||||
|
||||
buildPkg = Object.assign(buildTargetInfo, buildPkg, { ext, version });
|
||||
console.log('dbg: buildPkg', buildPkg);
|
||||
console.log('dbg: tmplParams', tmplParams);
|
||||
return [buildPkg, tmplParams];
|
||||
return [rel, opts];
|
||||
};
|
||||
|
||||
let channelNames = [
|
||||
'stable',
|
||||
// 'hotfix',
|
||||
'latest',
|
||||
'rc',
|
||||
'preview',
|
||||
'pre',
|
||||
'dev',
|
||||
'beta',
|
||||
'alpha',
|
||||
];
|
||||
|
||||
function toReleaseTarget(tag) {
|
||||
tag = tag.replace(/^v/, '');
|
||||
|
||||
let releaseTarget = {
|
||||
channel: '',
|
||||
lts: false,
|
||||
version: '',
|
||||
};
|
||||
|
||||
if (tag === 'lts') {
|
||||
releaseTarget.lts = true;
|
||||
releaseTarget.channel = 'stable';
|
||||
} else if (channelNames.includes(tag)) {
|
||||
releaseTarget.channel = tag;
|
||||
} else {
|
||||
releaseTarget.version = tag;
|
||||
}
|
||||
|
||||
return releaseTarget;
|
||||
}
|
||||
|
||||
var CURL_PIPE_PS1_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.ps1');
|
||||
var CURL_PIPE_SH_BOOT = Path.join(__dirname, 'curl-pipe-bootstrap.tpl.sh');
|
||||
var CURL_PIPE_PS1_BOOT = path.join(__dirname, 'bootstrap.ps1');
|
||||
var CURL_PIPE_SH_BOOT = path.join(__dirname, 'bootstrap.sh');
|
||||
var BAD_SH_RE = /[<>'"`$\\]/;
|
||||
|
||||
InstallerServer.getPosixCurlPipeBootstrap = async function ({
|
||||
baseurl,
|
||||
pkg,
|
||||
ver,
|
||||
}) {
|
||||
Installers.getPosixCurlPipeBootstrap = async function ({ baseurl, pkg, ver }) {
|
||||
let bootTxt = await Fs.readFile(CURL_PIPE_SH_BOOT, 'utf8');
|
||||
|
||||
var webiPkg = [pkg, ver].filter(Boolean).join('@');
|
||||
@@ -244,8 +147,9 @@ InstallerServer.getPosixCurlPipeBootstrap = async function ({
|
||||
let name = env[0];
|
||||
let value = env[1];
|
||||
|
||||
// TODO create REs once, in higher scope
|
||||
let envRe = new RegExp(
|
||||
`^[ \\t]*#?[ \\t]*(export[ \\t])?[ \\t]*(${name})=.*`,
|
||||
`^[ \\t]*#?[ \\t]*(export\\s)?[ \\t]*(${name})=.*`,
|
||||
'm',
|
||||
);
|
||||
|
||||
@@ -261,7 +165,7 @@ InstallerServer.getPosixCurlPipeBootstrap = async function ({
|
||||
return bootTxt;
|
||||
};
|
||||
|
||||
InstallerServer.getPwshCurlPipeBootstrap = async function ({
|
||||
Installers.getPwshCurlPipeBootstrap = async function ({
|
||||
baseurl,
|
||||
pkg,
|
||||
ver,
|
||||
@@ -270,7 +174,7 @@ InstallerServer.getPwshCurlPipeBootstrap = async function ({
|
||||
let bootTxt = await Fs.readFile(CURL_PIPE_PS1_BOOT, 'utf8');
|
||||
|
||||
var webiPkg = [pkg, ver].filter(Boolean).join('@');
|
||||
//var webiChecksum = await InstallerServer.getWebiPs1Checksum();
|
||||
//var webiChecksum = await Installers.getWebiPs1Checksum();
|
||||
var envReplacements = [
|
||||
['Env:WEBI_PKG', webiPkg],
|
||||
['Env:WEBI_HOST', baseurl],
|
||||
@@ -291,6 +195,9 @@ InstallerServer.getPwshCurlPipeBootstrap = async function ({
|
||||
let tplRe = new RegExp(`{{ (${name}) }}`, 'g');
|
||||
bootTxt = bootTxt.replace(tplRe, `${value}`);
|
||||
|
||||
// let envRe = new RegExp(`^[ \\t]*#?[ \\t]*($$${name})[ \\t]*=.*`, 'im');
|
||||
// bootTxt = bootTxt.replace(envRe, `$$${name} = '${value}'`);
|
||||
|
||||
let setRe = new RegExp(
|
||||
`(#[ \\t]*)?(\\$${name})[ \\t]*=[ \\t]['"].*['"][ \\t]`,
|
||||
'im',
|
||||
@@ -302,3 +209,32 @@ InstallerServer.getPwshCurlPipeBootstrap = async function ({
|
||||
|
||||
return bootTxt;
|
||||
};
|
||||
|
||||
var _webiShMeta = {
|
||||
stale: 10 * 1000,
|
||||
updated_at: 0,
|
||||
checksum: '',
|
||||
mtime: 0,
|
||||
};
|
||||
Installers.getWebiShChecksum = async function () {
|
||||
let now = Date.now();
|
||||
let ago = now - _webiShMeta.updated_at;
|
||||
if (ago <= _webiShMeta.stale) {
|
||||
return _webiShMeta.checksum;
|
||||
}
|
||||
|
||||
let webiPath = path.join(__dirname, '../webi/webi.sh');
|
||||
let stat = await Fs.stat(webiPath);
|
||||
if (stat.mtimeMs === _webiShMeta.mtime) {
|
||||
return _webiShMeta.checksum;
|
||||
}
|
||||
|
||||
let webiBuf = await Fs.readFile(webiPath, null);
|
||||
let webiHash = Crypto.createHash('sha1').update(webiBuf).digest('hex');
|
||||
let webiChecksum = webiHash.slice(0, 8);
|
||||
|
||||
_webiShMeta.mtime = stat.mtimeMs;
|
||||
_webiShMeta.updated_at = now;
|
||||
_webiShMeta.checksum = webiChecksum;
|
||||
return _webiShMeta.checksum;
|
||||
};
|
||||
|
||||
127
_webi/template.ps1
Normal file
127
_webi/template.ps1
Normal file
@@ -0,0 +1,127 @@
|
||||
#!/usr/bin/env pwsh
|
||||
#350 check if windows user run as admin
|
||||
|
||||
function Confirm-IsElevated {
|
||||
$id = [System.Security.Principal.WindowsIdentity]::GetCurrent()
|
||||
$p = New-Object System.Security.Principal.WindowsPrincipal($id)
|
||||
if ($p.IsInRole([System.Security.Principal.WindowsBuiltInRole]::Administrator))
|
||||
{ Write-Output $true }
|
||||
else
|
||||
{ Write-Output $false }
|
||||
}
|
||||
|
||||
if (Confirm-IsElevated)
|
||||
{ throw "Webi MUST NOT be run with elevated privileges. Please run again as a normal user, NOT as administrator." }
|
||||
|
||||
# this allows us to call ps1 files, which allows us to have spaces in filenames
|
||||
# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
|
||||
# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
|
||||
Set-ExecutionPolicy -Scope Process Bypass
|
||||
|
||||
# If a command returns an error, halt the script.
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
|
||||
$ProgressPreference = 'SilentlyContinue'
|
||||
|
||||
$Env:WEBI_HOST = 'https://webinstall.dev'
|
||||
#$Env:WEBI_PKG = 'node@lts'
|
||||
#$Env:PKG_NAME = node
|
||||
#$Env:WEBI_VERSION = v12.16.2
|
||||
#$Env:WEBI_GIT_TAG = 12.16.2
|
||||
#$Env:WEBI_PKG_URL = "https://.../node-....zip"
|
||||
#$Env:WEBI_PKG_FILE = "node-v12.16.2-win-x64.zip"
|
||||
#$Env:WEBI_PKG_PATHNAME = "node-v12.16.2-win-x64.zip"
|
||||
|
||||
# Switch to userprofile
|
||||
Push-Location $Env:USERPROFILE
|
||||
|
||||
# Make paths
|
||||
New-Item -Path "$Env:USERPROFILE\Downloads" -ItemType Directory -Force | Out-Null
|
||||
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
|
||||
New-Item -Path "$Env:USERPROFILE\.local\opt" -ItemType Directory -Force | Out-Null
|
||||
|
||||
# {{ baseurl }}
|
||||
# {{ version }}
|
||||
|
||||
function webi_path_add($pathname) {
|
||||
# C:\Users\me => C:/Users/me
|
||||
$my_home = $Env:UserProfile
|
||||
$my_home = $my_home.replace('\\', '/')
|
||||
$my_home_re = [regex]::escape($my_home)
|
||||
|
||||
# ~/bin => %USERPROFILE%/bin
|
||||
$pathname = $pathname.replace('~/', "$Env:UserProfile/")
|
||||
|
||||
# C:\Users\me\bin => %USERPROFILE%/bin
|
||||
$my_pathname = $pathname.replace('\\', '/')
|
||||
$my_pathname = $my_pathname -ireplace $my_home_re, "%USERPROFILE%"
|
||||
|
||||
$all_user_paths = [Environment]::GetEnvironmentVariable("Path", "User")
|
||||
$user_paths = $all_user_paths -Split (';')
|
||||
$exists_in_path = $false
|
||||
foreach ($user_path in $user_paths) {
|
||||
# C:\Users\me\bin => %USERPROFILE%/bin
|
||||
$my_user_path = $user_path.replace('\\', '/')
|
||||
$my_user_path = $my_user_path -ireplace $my_home_re, "%USERPROFILE%"
|
||||
|
||||
if ($my_user_path -ieq $my_pathname) {
|
||||
$exists_in_path = $true
|
||||
}
|
||||
}
|
||||
if (-Not $exists_in_path) {
|
||||
$all_user_paths = $pathname + ";" + $all_user_paths
|
||||
[Environment]::SetEnvironmentVariable("Path", $all_user_paths, "User")
|
||||
}
|
||||
|
||||
$session_paths = $Env:Path -Split (';')
|
||||
$in_session_path = $false
|
||||
foreach ($session_path in $session_paths) {
|
||||
# C:\Users\me\bin => %USERPROFILE%/bin
|
||||
$my_session_path = $session_path.replace('\\', '/')
|
||||
$my_session_path = $my_session_path -ireplace $my_home_re, "%USERPROFILE%"
|
||||
|
||||
if ($my_session_path -ieq $my_pathname) {
|
||||
$in_session_path = $true
|
||||
}
|
||||
}
|
||||
|
||||
if (-Not ($in_session_path)) {
|
||||
webi_path_add_followup $pathname
|
||||
}
|
||||
}
|
||||
|
||||
function webi_path_add_followup($pathname) {
|
||||
$my_cmd = 'PATH ' + "$pathname" + ';%PATH%'
|
||||
$my_pwsh = '$Env:Path = "' + "$pathname" + ';$Env:Path"'
|
||||
|
||||
Write-Host ''
|
||||
Write-Host '**********************************' -ForegroundColor yellow -BackgroundColor black
|
||||
Write-Host '* IMPORTANT -- READ ME *' -ForegroundColor yellow -BackgroundColor black
|
||||
Write-Host '* (run the PATH command below) *' -ForegroundColor yellow -BackgroundColor black
|
||||
Write-Host '**********************************' -ForegroundColor yellow -BackgroundColor black
|
||||
Write-Host ''
|
||||
Write-Output ""
|
||||
Write-Output "Copy, paste, and run the appropriate command to update your PATH:"
|
||||
Write-Output "(or close and reopen the terminal, or reboot)"
|
||||
Write-Output ""
|
||||
Write-Output "cmd.exe:"
|
||||
Write-Host " $my_cmd" -ForegroundColor yellow -BackgroundColor black
|
||||
Write-Output ""
|
||||
Write-Output "PowerShell:"
|
||||
Write-Host " $my_pwsh" -ForegroundColor yellow -BackgroundColor black
|
||||
Write-Output ""
|
||||
}
|
||||
|
||||
#$has_local_bin = echo "$Env:PATH" | Select-String -Pattern '\.local.bin'
|
||||
#if (!$has_local_bin)
|
||||
#{
|
||||
webi_path_add ~/.local/bin
|
||||
#}
|
||||
|
||||
# {{ installer }}
|
||||
|
||||
webi_path_add ~/.local/bin
|
||||
|
||||
# Done
|
||||
Pop-Location
|
||||
686
_webi/template.sh
Normal file
686
_webi/template.sh
Normal file
@@ -0,0 +1,686 @@
|
||||
#!/bin/sh
|
||||
|
||||
__bootstrap_webi() {
|
||||
|
||||
set -e
|
||||
set -u
|
||||
#set -x
|
||||
|
||||
my_libc=''
|
||||
if ldd /bin/ls 2> /dev/null | grep -q 'musl' 2> /dev/null; then
|
||||
my_libc='musl'
|
||||
elif uname -o | grep -q 'GNU' || uname -s | grep -q 'Linux'; then
|
||||
my_libc='gnu'
|
||||
else
|
||||
my_libc='libc'
|
||||
fi
|
||||
|
||||
#WEBI_PKG=
|
||||
#PKG_NAME=
|
||||
#WEBI_OS=
|
||||
#WEBI_ARCH=
|
||||
#WEBI_LIBC=
|
||||
#WEBI_HOST=
|
||||
#WEBI_RELEASES=
|
||||
#WEBI_CSV=
|
||||
#WEBI_TAG=
|
||||
#WEBI_VERSION=
|
||||
#WEBI_MAJOR=
|
||||
#WEBI_MINOR=
|
||||
#WEBI_PATCH=
|
||||
# TODO not sure if BUILD is the best name for this
|
||||
#WEBI_BUILD=
|
||||
#WEBI_GIT_TAG=
|
||||
#WEBI_LTS=
|
||||
#WEBI_CHANNEL=
|
||||
#WEBI_EXT=
|
||||
#WEBI_FORMATS=
|
||||
#WEBI_PKG_URL=
|
||||
#WEBI_PKG_FILE=
|
||||
#WEBI_PKG_PATHNAME=
|
||||
#PKG_OSES=
|
||||
#PKG_ARCHES=
|
||||
#PKG_LIBCS=
|
||||
#PKG_FORMATS=
|
||||
WEBI_UA="$(uname -s)/$(uname -r) $(uname -m)/unknown ${my_libc}"
|
||||
WEBI_PKG_DOWNLOAD=""
|
||||
WEBI_DOWNLOAD_DIR="${HOME}/Downloads"
|
||||
if command -v xdg-user-dir > /dev/null; then
|
||||
WEBI_DOWNLOAD_DIR="$(xdg-user-dir DOWNLOAD)"
|
||||
if [ "${WEBI_DOWNLOAD_DIR}" = "${HOME}" ]; then
|
||||
WEBI_DOWNLOAD_DIR="${HOME}/Downloads"
|
||||
fi
|
||||
fi
|
||||
|
||||
WEBI_PKG_PATH="${WEBI_DOWNLOAD_DIR}/webi/${PKG_NAME:-error}/${WEBI_VERSION:-latest}"
|
||||
|
||||
export WEBI_HOST
|
||||
|
||||
##
|
||||
## Set up tmp, download, and install directories
|
||||
##
|
||||
|
||||
WEBI_TMP=${WEBI_TMP:-"$(mktemp -d -t webinstall-"${WEBI_PKG-}".XXXXXXXX)"}
|
||||
export _webi_tmp="${_webi_tmp:-"$HOME/.local/opt/webi-tmp.d"}"
|
||||
|
||||
mkdir -p "${WEBI_PKG_PATH}"
|
||||
mkdir -p "$HOME/.local/bin"
|
||||
mkdir -p "$HOME/.local/opt"
|
||||
|
||||
##
|
||||
## Detect http client
|
||||
##
|
||||
set +e
|
||||
WEBI_CURL="$(command -v curl)"
|
||||
export WEBI_CURL
|
||||
WEBI_WGET="$(command -v wget)"
|
||||
export WEBI_WGET
|
||||
set -e
|
||||
|
||||
# get the special formatted version
|
||||
# (i.e. "go is go1.14" while node is "node v12.10.8")
|
||||
my_versioned_name=""
|
||||
_webi_canonical_name() {
|
||||
if [ -n "$my_versioned_name" ]; then
|
||||
echo "$my_versioned_name"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if command -v pkg_format_cmd_version > /dev/null; then
|
||||
my_versioned_name="'$(pkg_format_cmd_version "$WEBI_VERSION")'"
|
||||
else
|
||||
my_versioned_name="'$pkg_cmd_name v$WEBI_VERSION'"
|
||||
fi
|
||||
|
||||
echo "$my_versioned_name"
|
||||
}
|
||||
|
||||
# Update symlinks as per $HOME/.local/opt and $HOME/.local/bin install paths.
|
||||
webi_link() {
|
||||
if command -v pkg_link > /dev/null; then
|
||||
pkg_link
|
||||
return 0
|
||||
fi
|
||||
|
||||
if test -n "${WEBI_SINGLE}"; then
|
||||
rm -rf "$pkg_dst_cmd"
|
||||
ln -s "$pkg_src_cmd" "$pkg_dst_cmd"
|
||||
else
|
||||
# 'pkg_dst' will default to $HOME/.local/opt/<pkg>
|
||||
# 'pkg_src' will be the installed version,
|
||||
# such as to $HOME/.local/opt/<pkg>-<version>
|
||||
rm -rf "$pkg_dst"
|
||||
ln -s "$pkg_src" "$pkg_dst"
|
||||
fi
|
||||
}
|
||||
|
||||
# detect if this program is already installed
|
||||
# or if an installed version may cause conflict
|
||||
webi_check_installed() {
|
||||
# Test for existing version
|
||||
set +e
|
||||
my_path="$PATH"
|
||||
PATH="$(dirname "$pkg_dst_cmd"):$PATH"
|
||||
export PATH
|
||||
my_current_cmd="$(command -v "$pkg_cmd_name")"
|
||||
set -e
|
||||
if [ -n "$my_current_cmd" ]; then
|
||||
my_canonical_name="$(_webi_canonical_name)"
|
||||
if [ "$my_current_cmd" != "$pkg_dst_cmd" ]; then
|
||||
echo >&2 "WARN: possible PATH conflict between $my_canonical_name and currently installed version"
|
||||
echo >&2 " ${pkg_dst_cmd} (new)"
|
||||
echo >&2 " ${my_current_cmd} (existing)"
|
||||
#my_current_version=false
|
||||
fi
|
||||
# 'readlink' can't read links in paths on macOS 🤦
|
||||
# but that's okay, 'cmp -s' is good enough for us
|
||||
if cmp -s "${pkg_src_cmd}" "${my_current_cmd}"; then
|
||||
echo "${my_canonical_name} already installed:"
|
||||
my_dst_rel="$(
|
||||
webi_sub_home "${pkg_dst}"
|
||||
)"
|
||||
printf " %s" "${my_dst_rel}"
|
||||
if [ "${pkg_src_cmd}" != "${my_current_cmd}" ]; then
|
||||
my_src_rel="$(
|
||||
webi_sub_home "${pkg_src}"
|
||||
)"
|
||||
printf " => %s" "${my_src_rel}"
|
||||
fi
|
||||
echo ""
|
||||
exit 0
|
||||
fi
|
||||
if [ -x "$pkg_src_cmd" ]; then
|
||||
webi_link
|
||||
echo "switched to $my_canonical_name:"
|
||||
my_src_rel="$(
|
||||
webi_sub_home "${pkg_src}"
|
||||
)"
|
||||
my_dst_rel="$(
|
||||
webi_sub_home "${pkg_dst}"
|
||||
)"
|
||||
echo " ${my_dst_rel} => ${my_src_rel}"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
export PATH="$my_path"
|
||||
}
|
||||
|
||||
webi_check_available() {
|
||||
if test "$WEBI_CHANNEL" != "error"; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo >&2 "Error: no '${PKG_NAME:-"Unknown Package"}@${WEBI_TAG:-"Unknown Tag"}' release for '${WEBI_OS:-"Unknown OS"}' (${WEBI_LIBC:-"Unknown Libc"}) on '${WEBI_ARCH:-"Unknown CPU"}' as one of '${WEBI_FORMATS:-"Unknown File Type"}'"
|
||||
echo >&2 " '$PKG_NAME' is available for '$PKG_OSES' ($PKG_LIBCS) on '$PKG_ARCHES' as one of '$PKG_FORMATS'"
|
||||
echo >&2 " (check that the package name and version are correct)"
|
||||
echo >&2 ""
|
||||
my_release_url="$(
|
||||
echo "$WEBI_RELEASES" |
|
||||
sed 's:?.*::'
|
||||
)"
|
||||
my_release_params="$(
|
||||
echo "$WEBI_RELEASES" |
|
||||
sed 's:.*?:?:'
|
||||
)"
|
||||
echo >&2 " Double check at ${my_release_url}"
|
||||
echo >&2 " ${my_release_params}"
|
||||
echo >&2 ""
|
||||
|
||||
exit 1
|
||||
}
|
||||
|
||||
is_interactive_shell() {
|
||||
# $- shows shell flags (error,unset,interactive,etc)
|
||||
case $- in
|
||||
*i*)
|
||||
# true
|
||||
return 0
|
||||
;;
|
||||
*)
|
||||
# false
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
webi_sub_home() { (
|
||||
my_rel=${HOME}
|
||||
my_abs=${1}
|
||||
echo "${my_abs}" | sed "s:^${my_rel}:~:"
|
||||
); }
|
||||
|
||||
# detect if file is downloaded, and how to download it
|
||||
webi_download() {
|
||||
my_url="${1}"
|
||||
my_dl="${2}"
|
||||
my_dl_name="${3:-${PKG_NAME}}"
|
||||
|
||||
my_dl_rel="$(
|
||||
webi_sub_home "${my_dl}"
|
||||
)"
|
||||
|
||||
WEBI_PKG_DOWNLOAD="${my_dl}"
|
||||
export WEBI_PKG_DOWNLOAD
|
||||
|
||||
if [ -e "${my_dl}" ]; then
|
||||
echo "Found ${my_dl_rel}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Downloading ${my_dl_name} from"
|
||||
echo "$my_url"
|
||||
|
||||
# It's only 2020, we can't expect to have reliable CLI tools
|
||||
# to tell us the size of a file as part of a base system...
|
||||
if [ -n "$WEBI_WGET" ]; then
|
||||
# wget has resumable downloads
|
||||
# TODO wget -c --content-disposition "$my_url"
|
||||
set +e
|
||||
my_show_progress=""
|
||||
if is_interactive_shell; then
|
||||
my_show_progress="--show-progress"
|
||||
fi
|
||||
if ! wget -q $my_show_progress --user-agent="wget $WEBI_UA" -c "$my_url" -O "$my_dl.part"; then
|
||||
echo >&2 "failed to download from $WEBI_PKG_URL"
|
||||
exit 1
|
||||
fi
|
||||
set -e
|
||||
else
|
||||
# Neither GNU nor BSD curl have sane resume download options, hence we don't bother
|
||||
my_show_progress="-#"
|
||||
if is_interactive_shell; then
|
||||
my_show_progress=""
|
||||
fi
|
||||
# shellcheck disable=SC2086
|
||||
# we want the flags to be split
|
||||
curl -fSL $my_show_progress -H "User-Agent: curl $WEBI_UA" "$my_url" -o "$my_dl.part"
|
||||
fi
|
||||
mv "$my_dl.part" "$my_dl"
|
||||
|
||||
echo ""
|
||||
echo "Saved as ${my_dl_rel}"
|
||||
}
|
||||
|
||||
webi_git_clone() { (
|
||||
my_url="${1}"
|
||||
my_dl="${2}"
|
||||
|
||||
my_dl_rel="$(
|
||||
webi_sub_home "${my_dl}"
|
||||
)"
|
||||
if [ -e "${my_dl}" ]; then
|
||||
echo "Found ${my_dl_rel}"
|
||||
|
||||
cp -RPp "${my_dl}" "${WEBI_TMP}/${WEBI_PKG_FILE}/"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "Cloning ${my_url}"
|
||||
cmd_git="git clone --config advice.detachedHead=false --quiet --depth=1 --single-branch"
|
||||
rm -rf "${my_dl}.part"
|
||||
if ! $cmd_git "${my_url}" --branch "${WEBI_GIT_TAG}" "${my_dl}.part"; then
|
||||
echo >&2 "failed to git clone ${WEBI_PKG_URL}"
|
||||
exit 1
|
||||
fi
|
||||
mv "${my_dl}.part" "${my_dl}"
|
||||
|
||||
cp -RPp "${my_dl}" "${WEBI_TMP}/${WEBI_PKG_FILE}/"
|
||||
); }
|
||||
|
||||
# detect which archives can be used
|
||||
webi_extract() {
|
||||
(
|
||||
cd "$WEBI_TMP"
|
||||
|
||||
my_dl_rel="$(
|
||||
webi_sub_home "${WEBI_PKG_PATH}/${WEBI_PKG_FILE}"
|
||||
)"
|
||||
if [ "tar" = "$WEBI_EXT" ]; then
|
||||
echo "Extracting ${my_dl_rel}"
|
||||
tar xf "${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
|
||||
elif [ "zip" = "$WEBI_EXT" ]; then
|
||||
echo "Extracting ${my_dl_rel}"
|
||||
unzip "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > __unzip__.log
|
||||
elif [ "exe" = "$WEBI_EXT" ]; then
|
||||
echo "Moving ${my_dl_rel}"
|
||||
mv "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" .
|
||||
elif [ "git" = "$WEBI_EXT" ]; then
|
||||
echo "Moving ${my_dl_rel}"
|
||||
mv "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" .
|
||||
elif [ "xz" = "$WEBI_EXT" ]; then
|
||||
echo "Inflating ${my_dl_rel}"
|
||||
unxz -c "${WEBI_PKG_PATH}/$WEBI_PKG_FILE" > "$(basename "$WEBI_PKG_FILE")"
|
||||
else
|
||||
echo "Failed to extract ${WEBI_PKG_PATH}/$WEBI_PKG_FILE"
|
||||
exit 1
|
||||
fi
|
||||
)
|
||||
}
|
||||
|
||||
# use 'pathman' to update $HOME/.config/envman/PATH.env
|
||||
webi_path_add() {
|
||||
my_path="${1}"
|
||||
|
||||
fn_envman_init
|
||||
|
||||
# \v was chosen as it is extremely unlikely for a filename
|
||||
# \1 could be an even better choice, but needs more testing.
|
||||
# (currently tested working on: linux & mac)
|
||||
# "\0001" should also work
|
||||
my_delim="$(
|
||||
printf '\v'
|
||||
)"
|
||||
|
||||
my_path_expanded="$(
|
||||
echo "${my_path}" |
|
||||
sed -e "s${my_delim}\$HOME${my_delim}$HOME${my_delim}g" \
|
||||
-e "s${my_delim}\${HOME}${my_delim}$HOME${my_delim}g" \
|
||||
-e "s${my_delim}^~/${my_delim}$HOME/${my_delim}g"
|
||||
)"
|
||||
|
||||
# A gift for @adamcstephens.
|
||||
# See https://github.com/webinstall/webi-installers/issues/322
|
||||
case "${PATH}" in
|
||||
# matches whether the first, a middle, the last, or the only PATH entry
|
||||
"${my_path_expanded}":* | \
|
||||
*:"${my_path_expanded}":* | \
|
||||
*:"${my_path_expanded}" | \
|
||||
"${my_path_expanded}")
|
||||
|
||||
if fn_is_defined_in_all_shells "${my_path}"; then
|
||||
return 0
|
||||
fi
|
||||
;;
|
||||
*) ;;
|
||||
esac
|
||||
|
||||
my_path_export="$(
|
||||
echo "${my_path}" |
|
||||
sed -e "s${my_delim}${HOME}${my_delim}\$HOME${my_delim}g" \
|
||||
-e "s${my_delim}\${HOME}${my_delim}\$HOME${my_delim}g" \
|
||||
-e "s${my_delim}^~/${my_delim}\$HOME/${my_delim}g"
|
||||
)"
|
||||
|
||||
my_export="export PATH=\"$my_path_export:\$PATH\""
|
||||
if grep -q -F "${my_export}" ~/.config/envman/PATH.env; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo "${my_export}" >> ~/.config/envman/PATH.env
|
||||
|
||||
mkdir -p "$_webi_tmp"
|
||||
my_path_tilde="$(
|
||||
echo "${my_path}" |
|
||||
sed -e "s${my_delim}${HOME}${my_delim}~${my_delim}g"
|
||||
)"
|
||||
|
||||
if ! test -f "$_webi_tmp/.PATH.env" ||
|
||||
! grep -q -F "${my_path_tilde}" "$_webi_tmp/.PATH.env"; then
|
||||
echo "${my_path_tilde}" >> "$_webi_tmp/.PATH.env"
|
||||
fi
|
||||
}
|
||||
|
||||
fn_envman_init() {
|
||||
mkdir -p ~/.config/envman/
|
||||
if ! test -e ~/.config/envman/PATH.env; then
|
||||
touch ~/.config/envman/PATH.env
|
||||
fi
|
||||
|
||||
if ! test -e ~/.config/envman/load.sh; then
|
||||
# shellcheck disable=SC2016
|
||||
{
|
||||
echo '# Generated for envman. Do not edit.'
|
||||
echo 'for x in ~/.config/envman/*.env; do'
|
||||
echo ' my_basename="$(basename "${x}")"'
|
||||
echo ' if [ "*.env" = "${my_basename}" ]; then'
|
||||
echo ' continue'
|
||||
echo ' fi'
|
||||
echo ''
|
||||
echo ' # shellcheck source=/dev/null'
|
||||
echo ' . "${x}"'
|
||||
echo 'done'
|
||||
} > ~/.config/envman/load.sh
|
||||
fi
|
||||
|
||||
if command -v sh > /dev/null; then
|
||||
if test -e ~/.profile; then
|
||||
if ! grep -q -F '/.config/envman/load.sh' ~/.profile; then
|
||||
fn_echo_load_sh >> ~/.profile
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
if command -v bash > /dev/null; then
|
||||
if test -e ~/.bashrc; then
|
||||
if ! grep -q -F '/.config/envman/load.sh' ~/.bashrc; then
|
||||
fn_echo_load_sh >> ~/.bashrc
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
if command -v zsh > /dev/null; then
|
||||
if test -e ~/.zshrc; then
|
||||
if ! grep -q -F '/.config/envman/load.sh' ~/.zshrc; then
|
||||
fn_echo_load_sh >> ~/.zshrc
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if command -v fish > /dev/null; then
|
||||
if test ! -e ~/.config/envman/load.fish; then
|
||||
# shellcheck disable=SC2016
|
||||
{
|
||||
echo '# Generated for envman. Do not edit.'
|
||||
echo 'for x in ~/.config/envman/*.env'
|
||||
echo ' source "$x"'
|
||||
echo 'end'
|
||||
} > ~/.config/envman/load.fish
|
||||
fi
|
||||
|
||||
mkdir -p ~/.config/fish
|
||||
if test -e ~/.config/fish/config.fish; then
|
||||
touch ~/.config/fish/config.fish
|
||||
fi
|
||||
if ! grep -q -F '/.config/envman/load.fish' ~/.config/fish/config.fish; then
|
||||
fn_echo_load_fish >> ~/.config/fish/config.fish
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
fn_echo_load_fish() {
|
||||
echo ''
|
||||
echo '# Generated for envman. Do not edit.'
|
||||
# shellcheck disable=SC2016
|
||||
echo 'test -s "$HOME/.config/envman/load.fish"; and source "$HOME/.config/envman/load.fish"'
|
||||
}
|
||||
|
||||
fn_echo_load_sh() {
|
||||
echo ''
|
||||
echo '# Generated for envman. Do not edit.'
|
||||
# shellcheck disable=SC2016
|
||||
echo '[ -s "$HOME/.config/envman/load.sh" ] && source "$HOME/.config/envman/load.sh"'
|
||||
}
|
||||
|
||||
fn_is_defined_in_all_shells() {
|
||||
my_path="${1}"
|
||||
|
||||
my_path_expanded="$(
|
||||
echo "${my_path}" |
|
||||
sed -e "s${my_delim}\$HOME|${my_delim}$HOME${my_delim}g" \
|
||||
-e "s${my_delim}\${HOME}${my_delim}$HOME${my_delim}g" \
|
||||
-e "s${my_delim}^~/${my_delim}$HOME/${my_delim}g"
|
||||
)"
|
||||
my_paths="$(
|
||||
echo "${my_path_expanded}"
|
||||
# $HOME/foo
|
||||
echo "${my_path_expanded}" |
|
||||
sed "s${my_delim}${HOME}${my_delim}\$HOME${my_delim}g"
|
||||
# ${HOME}/foo
|
||||
echo "${my_path_expanded}" |
|
||||
sed "s${my_delim}${HOME}${my_delim}\${HOME}${my_delim}g"
|
||||
echo "${my_path}"
|
||||
)"
|
||||
|
||||
my_confs="$(
|
||||
echo "${HOME}/.profile"
|
||||
echo "${HOME}/.bashrc"
|
||||
echo "${HOME}/.zshrc"
|
||||
echo "${HOME}/.config/fish/config.fish"
|
||||
)"
|
||||
for my_conf in $my_confs; do
|
||||
if test -e "${my_conf}"; then
|
||||
if ! grep -q -F "${my_paths}" "${my_conf}"; then
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
# group common pre-install tasks as default
|
||||
webi_pre_install() {
|
||||
webi_check_installed
|
||||
webi_check_available
|
||||
if test "git" = "${WEBI_EXT}"; then
|
||||
webi_git_clone \
|
||||
"${WEBI_PKG_URL}" \
|
||||
"${WEBI_PKG_PATH}/${WEBI_PKG_FILE}"
|
||||
return 0
|
||||
fi
|
||||
webi_download \
|
||||
"${WEBI_PKG_URL}" \
|
||||
"${WEBI_PKG_PATH}/${WEBI_PKG_FILE}"
|
||||
webi_extract
|
||||
}
|
||||
|
||||
# move commands from the extracted archive directory
|
||||
# to $HOME/.local/opt or $HOME/.local/bin
|
||||
webi_install() {
|
||||
if test -n "${WEBI_SINGLE}"; then
|
||||
mkdir -p "$(dirname "$pkg_src_cmd")"
|
||||
mv ./"$pkg_cmd_name"* "$pkg_src_cmd"
|
||||
else
|
||||
rm -rf "$pkg_src"
|
||||
mv ./"$pkg_cmd_name"* "$pkg_src"
|
||||
fi
|
||||
}
|
||||
|
||||
# run post-install functions - just updating PATH by default
|
||||
webi_post_install() {
|
||||
if test -n "${pkg_no_exec}"; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
webi_path_add "$(dirname "$pkg_dst_cmd")"
|
||||
}
|
||||
|
||||
_webi_enable_exec() {
|
||||
if command -v spctl > /dev/null && command -v xattr > /dev/null; then
|
||||
# note: some packages contain files that cannot be affected by xattr
|
||||
xattr -r -d com.apple.quarantine "$pkg_src" || true
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
_webi_done_message() {
|
||||
my_dst_rel="$(
|
||||
webi_sub_home "${pkg_dst_cmd}"
|
||||
)"
|
||||
my_canonical_name="$(
|
||||
_webi_canonical_name
|
||||
)"
|
||||
echo "Installed ${my_canonical_name} as ${my_dst_rel}"
|
||||
}
|
||||
|
||||
##
|
||||
##
|
||||
## BEGIN custom override functions from <package>/install.sh
|
||||
##
|
||||
##
|
||||
|
||||
WEBI_SINGLE=
|
||||
|
||||
if [ -z "${WEBI_WELCOME-}" ]; then
|
||||
echo ""
|
||||
printf "Thanks for using webi to install '\e[32m%s\e[0m' on '\e[33m%s (%s) %s\e[0m'.\n" "${WEBI_PKG:-"Unknown Package"}" "$(uname -s)" "${WEBI_LIBC:-"Unknown Libc"}" "$(uname -m)"
|
||||
echo "Have a problem? Experience a bug? Please let us know:"
|
||||
printf " \e[2m\e[36mhttps://github.com/webinstall/webi-installers/issues\e[0m\n"
|
||||
echo ""
|
||||
printf "\e[35mLovin'\e[0m it? Say thanks with a \e[1m\e[33mStar on GitHub\e[0m:\n"
|
||||
printf " \e[36mhttps://github.com/webinstall/webi-installers\e[0m\n"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
WEBI_WELCOME=true
|
||||
export WEBI_WELCOME
|
||||
|
||||
__init_installer() {
|
||||
# the installer will be injected here
|
||||
# {{ installer }}
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
__init_installer
|
||||
|
||||
##
|
||||
##
|
||||
## END custom override functions
|
||||
##
|
||||
##
|
||||
|
||||
# run everything with defaults or overrides as needed
|
||||
if command -v pkg_install > /dev/null ||
|
||||
command -v pkg_link > /dev/null ||
|
||||
command -v pkg_post_install > /dev/null ||
|
||||
command -v pkg_done_message > /dev/null ||
|
||||
command -v pkg_format_cmd_version > /dev/null ||
|
||||
[ -n "${WEBI_SINGLE-}" ] ||
|
||||
[ -n "${pkg_cmd_name-}" ] ||
|
||||
[ -n "${pkg_dst_cmd-}" ] ||
|
||||
[ -n "${pkg_dst_dir-}" ] ||
|
||||
[ -n "${pkg_dst-}" ] ||
|
||||
[ -n "${pkg_src_cmd-}" ] ||
|
||||
[ -n "${pkg_src_dir-}" ] ||
|
||||
[ -n "${pkg_src-}" ]; then
|
||||
|
||||
pkg_cmd_name="${pkg_cmd_name:-$PKG_NAME}"
|
||||
|
||||
pkg_no_exec="${pkg_no_exec:-}"
|
||||
|
||||
if [ -n "${pkg_no_exec}" ]; then
|
||||
pkg_dst_cmd="${pkg_dst}"
|
||||
pkg_src="${pkg_dst}"
|
||||
pkg_src_cmd="${pkg_dst}"
|
||||
elif [ -n "${WEBI_SINGLE}" ]; then
|
||||
pkg_dst_cmd="${pkg_dst_cmd:-$HOME/.local/bin/$pkg_cmd_name}"
|
||||
pkg_dst="$pkg_dst_cmd"
|
||||
|
||||
#pkg_src_cmd="${pkg_src_cmd:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION/bin/$pkg_cmd_name-v$WEBI_VERSION}"
|
||||
pkg_src_cmd="${pkg_src_cmd:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION/bin/$pkg_cmd_name}"
|
||||
pkg_src="$pkg_src_cmd"
|
||||
else
|
||||
pkg_dst="${pkg_dst:-$HOME/.local/opt/$pkg_cmd_name}"
|
||||
pkg_dst_cmd="${pkg_dst_cmd:-$pkg_dst/bin/$pkg_cmd_name}"
|
||||
|
||||
pkg_src="${pkg_src:-$HOME/.local/opt/$pkg_cmd_name-v$WEBI_VERSION}"
|
||||
pkg_src_cmd="${pkg_src_cmd:-$pkg_src/bin/$pkg_cmd_name}"
|
||||
fi
|
||||
# shellcheck disable=SC2034 # used in ${WEBI_PKG}/install.sh
|
||||
pkg_src_bin="$(dirname "$pkg_src_cmd")"
|
||||
# shellcheck disable=SC2034 # used in ${WEBI_PKG}/install.sh
|
||||
pkg_dst_bin="$(dirname "$pkg_dst_cmd")"
|
||||
|
||||
if command -v pkg_pre_install > /dev/null; then pkg_pre_install; else webi_pre_install; fi
|
||||
|
||||
(
|
||||
cd "$WEBI_TMP"
|
||||
my_src_rel="$(
|
||||
webi_sub_home "${pkg_src_cmd}"
|
||||
)"
|
||||
if test -e "${pkg_src_cmd}"; then
|
||||
echo "Found ${my_src_rel} (remove to force reinstall)"
|
||||
else
|
||||
echo "Installing to ${my_src_rel}"
|
||||
if command -v pkg_install > /dev/null; then pkg_install; else webi_install; fi
|
||||
chmod a+x "$pkg_src"
|
||||
chmod a+x "$pkg_src_cmd"
|
||||
fi
|
||||
)
|
||||
|
||||
if test -z "${pkg_no_exec}"; then
|
||||
webi_link
|
||||
_webi_enable_exec
|
||||
fi
|
||||
(
|
||||
cd "$WEBI_TMP"
|
||||
if command -v pkg_post_install > /dev/null; then pkg_post_install; else webi_post_install; fi
|
||||
)
|
||||
|
||||
(
|
||||
cd "$WEBI_TMP"
|
||||
if command -v pkg_done_message > /dev/null; then pkg_done_message; else _webi_done_message; fi
|
||||
)
|
||||
|
||||
echo ""
|
||||
fi
|
||||
|
||||
webi_path_add "$HOME/.local/bin"
|
||||
if [ -z "${_WEBI_CHILD-}" ] && [ -f "$_webi_tmp/.PATH.env" ]; then
|
||||
if test -s "$_webi_tmp/.PATH.env"; then
|
||||
printf 'PATH.env updated with:\n'
|
||||
sort -u "$_webi_tmp/.PATH.env" | while read -r my_new_path; do
|
||||
echo " ${my_new_path}"
|
||||
done
|
||||
printf "\n"
|
||||
|
||||
rm -f "$_webi_tmp/.PATH.env"
|
||||
|
||||
printf "\e[1m\e[35mTO FINISH\e[0m: copy, paste & run the following command:\n"
|
||||
printf "\n"
|
||||
printf " \e[1m\e[32msource ~/.config/envman/PATH.env\e[0m\n"
|
||||
printf " (newly opened terminal windows will update automatically)\n"
|
||||
fi
|
||||
fi
|
||||
|
||||
rm -rf "$WEBI_TMP"
|
||||
}
|
||||
|
||||
__bootstrap_webi
|
||||
@@ -32,8 +32,7 @@ if (/\b-?-h(elp)?\b/.test(process.argv.join(' '))) {
|
||||
var os = require('os');
|
||||
var fs = require('fs');
|
||||
var path = require('path');
|
||||
var Builds = require('./builds.js');
|
||||
var Installers = require('./installers.js');
|
||||
var Releases = require('./releases.js');
|
||||
var ServeInstaller = require('./serve-installer.js');
|
||||
|
||||
var pkg = process.argv[2].split('@');
|
||||
@@ -49,7 +48,7 @@ var baseurl = 'https://webinstall.dev';
|
||||
var maxLen = 0;
|
||||
console.info('');
|
||||
console.info('Has the necessary files?');
|
||||
['README.md', 'install.sh', 'install.ps1']
|
||||
['README.md', 'releases.js', 'install.sh', 'install.ps1']
|
||||
.map(function (node) {
|
||||
maxLen = Math.max(maxLen, node.length);
|
||||
return node;
|
||||
@@ -65,8 +64,7 @@ console.info('Has the necessary files?');
|
||||
});
|
||||
|
||||
console.info('');
|
||||
let projName = pkgdir.split('/').filter(Boolean).pop();
|
||||
Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
|
||||
Releases.get(path.join(process.cwd(), pkgdir)).then(async function (all) {
|
||||
var pkgname = path.basename(pkgdir.replace(/\/$/, ''));
|
||||
var nodeOs = os.platform();
|
||||
var nodeOsRelease = os.release();
|
||||
@@ -83,8 +81,8 @@ Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
|
||||
var formats = ['exe', 'xz', 'tar', 'zip', 'git'];
|
||||
|
||||
let [rel, opts] = await ServeInstaller.helper({
|
||||
unameAgent: `${nodeOs}/${nodeOsRelease} ${nodeArch}/unknown ${nodeLibc}`,
|
||||
projectName: pkgname,
|
||||
ua: `${nodeOs}/${nodeOsRelease} ${nodeArch}/unknown ${nodeLibc}`,
|
||||
pkg: pkgname,
|
||||
tag: pkgtag || '',
|
||||
formats: formats,
|
||||
libc: nodeLibc,
|
||||
@@ -118,8 +116,8 @@ Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
|
||||
console.info('');
|
||||
|
||||
return Promise.all([
|
||||
Installers.renderBash(pkgdir, rel, opts).catch(function () {}),
|
||||
Installers.renderPowerShell(pkgdir, rel, opts).catch(function () {}),
|
||||
Releases.renderBash(pkgdir, rel, opts).catch(function () {}),
|
||||
Releases.renderPowerShell(pkgdir, rel, opts).catch(function () {}),
|
||||
]).then(function (scripts) {
|
||||
var bashTxt = scripts[0];
|
||||
var ps1Txt = scripts[1];
|
||||
|
||||
@@ -1,31 +1,55 @@
|
||||
'use strict';
|
||||
|
||||
var Releases = module.exports;
|
||||
|
||||
var Fs = require('node:fs/promises');
|
||||
var Os = require('node:os');
|
||||
var path = require('path');
|
||||
var Releases = require('./releases.js');
|
||||
|
||||
var cache = {};
|
||||
//var staleAge = 5 * 1000;
|
||||
//var expiredAge = 15 * 1000;
|
||||
var staleAge = 5 * 60 * 1000;
|
||||
var expiredAge = 15 * 60 * 1000;
|
||||
|
||||
var LEGACY_CACHE_DIR = path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
let installerDir = path.join(__dirname, '..');
|
||||
|
||||
// Sort releases by ext preference and libc within the same version.
|
||||
// The cache is already sorted by version (stable before beta, newest first),
|
||||
// so we only re-order within the same version string.
|
||||
// TODO needs a proper test, and more accurate (though perhaps far less simple) code
|
||||
function createFormatsSorter(formats) {
|
||||
return function sortByExtLibc(a, b) {
|
||||
if (a.version !== b.version) {
|
||||
// Array.sort is stable (V8, ES2019), so returning 0 across
|
||||
// versions preserves the cache's pre-sorted version-desc order.
|
||||
return 0;
|
||||
return function sortByVerExt(a, b) {
|
||||
function lexver(semver) {
|
||||
// v1.20.156 => 00001.00020.00156.zzzzz
|
||||
// TODO BUG: v1.20.156-rc2 => 00001.00020.00156.rc2zz
|
||||
var parts = semver.split(/[+\.\-]/g);
|
||||
while (parts.length < 4) {
|
||||
parts.push('');
|
||||
}
|
||||
return parts
|
||||
.map(function (num, i) {
|
||||
if (3 === i) {
|
||||
return num.toString().padEnd(10, 'z');
|
||||
}
|
||||
return num.toString().padStart(10, '0');
|
||||
})
|
||||
.join('.');
|
||||
}
|
||||
|
||||
var aver = lexver(a.version);
|
||||
var bver = lexver(b.version);
|
||||
if (aver > bver) {
|
||||
//console.log(aver, '>', bver);
|
||||
return -1;
|
||||
}
|
||||
if (aver < bver) {
|
||||
//console.log(aver, '<', bver);
|
||||
return 1;
|
||||
}
|
||||
|
||||
var aExtPri = formats.indexOf(a.ext.replace(/tar\..*/, 'tar'));
|
||||
var bExtPri = formats.indexOf(b.ext.replace(/tar\..*/, 'tar'));
|
||||
if (aExtPri > bExtPri) {
|
||||
//console.log(a.ext, aExtPri, '>', b.ext, bExtPri);
|
||||
return -1;
|
||||
}
|
||||
if (aExtPri < bExtPri) {
|
||||
//console.log(a.ext, aExtPri, '<', b.ext, bExtPri);
|
||||
return 1;
|
||||
}
|
||||
|
||||
@@ -42,39 +66,99 @@ function createFormatsSorter(formats) {
|
||||
}
|
||||
|
||||
async function getCachedReleases(pkg) {
|
||||
// returns { download: '', releases: [{ version, date, os, arch, lts, channel, download}] }
|
||||
// returns { download: '<template string>', releases: [{ version, date, os, arch, lts, channel, download}] }
|
||||
|
||||
if (cache[pkg]) {
|
||||
return cache[pkg];
|
||||
async function chainCachePromise(fn) {
|
||||
cache[pkg].promise = cache[pkg].promise.then(fn);
|
||||
return cache[pkg].promise;
|
||||
}
|
||||
|
||||
let dataFile = `${LEGACY_CACHE_DIR}/${pkg}.json`;
|
||||
async function sleep(ms) {
|
||||
return await new Promise(function (resolve, reject) {
|
||||
setTimeout(resolve, ms);
|
||||
});
|
||||
}
|
||||
|
||||
let json = await Fs.readFile(dataFile, 'utf8').catch(function (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return null;
|
||||
async function putCache() {
|
||||
var age = Date.now() - cache[pkg].updatedAt;
|
||||
if (age < staleAge) {
|
||||
//console.debug('NOT STALE ANYMORE - updated in previous promise');
|
||||
return cache[pkg].all;
|
||||
}
|
||||
throw err;
|
||||
});
|
||||
|
||||
if (!json) {
|
||||
let empty = { download: '', releases: [] };
|
||||
cache[pkg] = empty;
|
||||
return empty;
|
||||
//console.debug('DOWNLOADING NEW "%s" releases', pkg);
|
||||
var pkgdir = path.join(installerDir, pkg);
|
||||
|
||||
// workaround for request timeout seeming to not work
|
||||
let complete = false;
|
||||
await Promise.race([
|
||||
Releases.get(pkgdir)
|
||||
.catch(function (err) {
|
||||
if ('E_NO_RELEASE' === err.code) {
|
||||
let all = { _error: 'E_NO_RELEASE', download: '', releases: [] };
|
||||
return all;
|
||||
}
|
||||
|
||||
throw err;
|
||||
})
|
||||
.catch(function (err) {
|
||||
let hasReleases = cache[pkg].all?.releases?.length > 1;
|
||||
if (!hasReleases) {
|
||||
throw err;
|
||||
}
|
||||
|
||||
console.error(`Error: the BOOGEYMAN got us!`);
|
||||
console.error(err.stack);
|
||||
|
||||
return cache[pkg].all;
|
||||
})
|
||||
.then(function (all) {
|
||||
// Note: it is possible for slightly older data
|
||||
// to replace slightly newer data, but this is better
|
||||
// than being in a cycle where release updates _always_
|
||||
// take longer than expected.
|
||||
//console.debug('DOWNLOADED NEW "%s" releases', pkg);
|
||||
cache[pkg].updatedAt = Date.now();
|
||||
cache[pkg].all = all;
|
||||
complete = true;
|
||||
}),
|
||||
sleep(5000).then(function () {
|
||||
if (complete) {
|
||||
return;
|
||||
}
|
||||
console.error(`request timeout waiting for '${pkg}' release info`);
|
||||
}),
|
||||
]);
|
||||
|
||||
return cache[pkg].all;
|
||||
}
|
||||
|
||||
let all;
|
||||
try {
|
||||
all = JSON.parse(json);
|
||||
} catch (e) {
|
||||
console.error(`error: ${dataFile}:\n\t${e.message}`);
|
||||
let empty = { download: '', releases: [] };
|
||||
cache[pkg] = empty;
|
||||
return empty;
|
||||
if (!cache[pkg]) {
|
||||
cache[pkg] = {
|
||||
updatedAt: 0,
|
||||
all: { download: '', releases: [] },
|
||||
promise: Promise.resolve(),
|
||||
};
|
||||
}
|
||||
|
||||
cache[pkg] = all;
|
||||
return all;
|
||||
var bgRenewal;
|
||||
var age = Date.now() - cache[pkg].updatedAt;
|
||||
var fresh = age < staleAge;
|
||||
if (!fresh) {
|
||||
bgRenewal = chainCachePromise(putCache);
|
||||
}
|
||||
|
||||
var tooStale = age > expiredAge;
|
||||
if (!tooStale) {
|
||||
return await cache[pkg].all;
|
||||
}
|
||||
|
||||
return await Promise.race([
|
||||
bgRenewal,
|
||||
sleep(5000).then(function () {
|
||||
return cache[pkg].all;
|
||||
}),
|
||||
]);
|
||||
}
|
||||
|
||||
async function filterReleases(
|
||||
@@ -85,22 +169,15 @@ async function filterReleases(
|
||||
// sort the most compatible format first
|
||||
// (i.e. so that we don't do .pkg on linux except on purpose)
|
||||
var rformats = formats.slice(0).reverse();
|
||||
var sortByExtLibc = createFormatsSorter(rformats);
|
||||
var sortByVerExt = createFormatsSorter(rformats);
|
||||
var reVer = new RegExp('^' + ver + '\\b');
|
||||
|
||||
function selectMatches(rel) {
|
||||
/* jshint maxcomplexity: 25 */
|
||||
if (os) {
|
||||
// '*' = any OS (matches anything, including windows).
|
||||
// 'posix' / 'posix_20xx' = any POSIX OS (matches linux, macos,
|
||||
// freebsd, etc., but NOT windows).
|
||||
let isPosix = rel.os === 'posix' || rel.os.startsWith('posix_20');
|
||||
let osMatches =
|
||||
rel.os === '*' ||
|
||||
rel.os === os ||
|
||||
(isPosix && os !== 'windows');
|
||||
if (!osMatches) {
|
||||
return false;
|
||||
if (rel.os !== '*') {
|
||||
if (rel.os !== os) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -112,10 +189,7 @@ async function filterReleases(
|
||||
}
|
||||
}
|
||||
|
||||
// libc='libc' is serve-releases.js's default when the caller
|
||||
// didn't pin one — treat it as 'no preference', not a filter.
|
||||
let isMeaningfulLibc = libc && libc !== 'libc' && rel.libc !== 'none';
|
||||
if (isMeaningfulLibc) {
|
||||
if (rel.libc !== 'none') {
|
||||
let releaseRequiresMusl = rel.libc === 'musl';
|
||||
// goal: handle non-glibc (Alpine / Docker / musl)
|
||||
let osHasMusl = libc === 'musl';
|
||||
@@ -162,13 +236,13 @@ async function filterReleases(
|
||||
return true;
|
||||
}
|
||||
|
||||
var sortedRels = all.releases.filter(selectMatches).sort(sortByExtLibc);
|
||||
var sortedRels = all.releases.filter(selectMatches).sort(sortByVerExt);
|
||||
//console.log(sortedRels.slice(0, 4));
|
||||
|
||||
return sortedRels.slice(0, limit || 1000);
|
||||
}
|
||||
|
||||
Releases.getReleases = function ({
|
||||
module.exports = function getReleases({
|
||||
_count,
|
||||
pkg,
|
||||
ver,
|
||||
@@ -204,19 +278,10 @@ Releases.getReleases = function ({
|
||||
console.error(err);
|
||||
})
|
||||
.then(function (releases) {
|
||||
if (releases.length) {
|
||||
return {
|
||||
oses: all.oses,
|
||||
arches: all.arches,
|
||||
libcs: all.libcs,
|
||||
formats: all.formats,
|
||||
releases: releases,
|
||||
};
|
||||
}
|
||||
if (_count < 1) {
|
||||
if (!releases.length) {
|
||||
// Apple Silicon M1 hacky-do workaround fix
|
||||
if ('macos' === os && 'arm64' === arch) {
|
||||
return Releases.getReleases({
|
||||
return getReleases({
|
||||
pkg,
|
||||
ver,
|
||||
os,
|
||||
@@ -230,7 +295,7 @@ Releases.getReleases = function ({
|
||||
}
|
||||
// Windows ARM hacky-do workaround fix
|
||||
if ('windows' === os && 'arm64' === arch) {
|
||||
return Releases.getReleases({
|
||||
return getReleases({
|
||||
pkg,
|
||||
ver,
|
||||
os,
|
||||
@@ -242,42 +307,9 @@ Releases.getReleases = function ({
|
||||
limit,
|
||||
});
|
||||
}
|
||||
// Raspberry Pi 3+ on Ubuntu arm64 (via Bionic?)
|
||||
if ('linux' === os && 'arm64' === arch) {
|
||||
return Releases.getReleases({
|
||||
_count: _count + 1,
|
||||
pkg,
|
||||
ver,
|
||||
os,
|
||||
arch: 'armv7l',
|
||||
libc,
|
||||
lts,
|
||||
channel,
|
||||
formats,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
// armv7 can run armv6
|
||||
if ('linux' === os && 'armv7l' === arch) {
|
||||
return Releases.getReleases({
|
||||
_count: _count + 1,
|
||||
pkg,
|
||||
ver,
|
||||
os,
|
||||
arch: 'armv6l',
|
||||
libc,
|
||||
lts,
|
||||
channel,
|
||||
formats,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
}
|
||||
if (_count < 2) {
|
||||
// Raspberry Pi 3+ on Raspbian arm7 (not Ubuntu arm64)
|
||||
// hail mary
|
||||
if ('linux' === os && 'armv7l' === arch) {
|
||||
return Releases.getReleases({
|
||||
// Raspberry Pi 3+ on Raspbian x86 (not Ubuntu arm64)
|
||||
if (!_count && 'linux' === os && 'armv7l' === arch) {
|
||||
return getReleases({
|
||||
_count: _count + 1,
|
||||
pkg,
|
||||
ver,
|
||||
@@ -290,25 +322,56 @@ Releases.getReleases = function ({
|
||||
limit,
|
||||
});
|
||||
}
|
||||
// Raspberry Pi 3+ on Ubuntu arm64 (via Bionic?)
|
||||
// (this may be the same as the prior search, that's okay)
|
||||
if ('linux' === os && 'arm64' === arch) {
|
||||
return getReleases({
|
||||
_count: _count + 1,
|
||||
pkg,
|
||||
ver,
|
||||
os,
|
||||
arch: 'armv7l',
|
||||
libc,
|
||||
lts,
|
||||
channel,
|
||||
formats,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
// Raspberry Pi 3+ on Ubuntu arm64 (via Bionic?)
|
||||
if ('linux' === os && 'armv7l' === arch) {
|
||||
return getReleases({
|
||||
_count: _count + 1,
|
||||
pkg,
|
||||
ver,
|
||||
os,
|
||||
arch: 'armv6l',
|
||||
libc,
|
||||
lts,
|
||||
channel,
|
||||
formats,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
releases = [
|
||||
{
|
||||
name: 'doesntexist.ext',
|
||||
version: '0.0.0',
|
||||
lts: '-',
|
||||
channel: 'error',
|
||||
date: '1970-01-01',
|
||||
os: os || '-',
|
||||
arch: arch || '-',
|
||||
libc: libc || '-',
|
||||
ext: 'err',
|
||||
download: 'https://example.com/doesntexist.ext',
|
||||
comment:
|
||||
'No matches found. Could be bad or missing version info' +
|
||||
',' +
|
||||
"Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
|
||||
},
|
||||
];
|
||||
}
|
||||
releases = [
|
||||
{
|
||||
name: 'doesntexist.ext',
|
||||
version: '0.0.0',
|
||||
lts: '-',
|
||||
channel: 'error',
|
||||
date: '1970-01-01',
|
||||
os: os || '-',
|
||||
arch: arch || '-',
|
||||
libc: libc || '-',
|
||||
ext: 'err',
|
||||
download: 'https://example.com/doesntexist.ext',
|
||||
comment:
|
||||
'No matches found. Could be bad or missing version info' +
|
||||
',' +
|
||||
"Check query parameters. Should be something like '/api/releases/{package}@{version}.tab?os={macos|linux|windows|-}&arch={amd64|x86|aarch64|arm64|armv7l|-}&libc={musl|gnu|msvc|libc|static}&limit=10'",
|
||||
},
|
||||
];
|
||||
return {
|
||||
oses: all.oses,
|
||||
arches: all.arches,
|
||||
@@ -321,8 +384,8 @@ Releases.getReleases = function ({
|
||||
};
|
||||
|
||||
if (require.main === module) {
|
||||
return Releases
|
||||
.getReleases({
|
||||
return module
|
||||
.exports({
|
||||
pkg: 'node',
|
||||
ver: '',
|
||||
os: 'macos',
|
||||
|
||||
@@ -38,8 +38,9 @@ function getOs(ua) {
|
||||
// It's the year of the Linux Desktop!
|
||||
// See also http://www.mslinux.org/
|
||||
// 'linux' must be tested before 'Microsoft' because WSL
|
||||
// (TODO: does this affect cygwin / msysgit?)
|
||||
return 'linux';
|
||||
} else if (/^ms$|Microsoft|Windows|win32|win|PowerShell|CYGWIN|MINGW/i.test(ua)) {
|
||||
} else if (/^ms$|Microsoft|Windows|win32|win|PowerShell/i.test(ua)) {
|
||||
// 'win' must be tested after 'darwin'
|
||||
return 'windows';
|
||||
} else if (/Linux|curl|wget/i.test(ua)) {
|
||||
@@ -66,9 +67,9 @@ function getArch(ua) {
|
||||
|
||||
if (/aarch64|arm64|arm8|armv8/i.test(ua)) {
|
||||
return 'arm64';
|
||||
} else if (/aarch|arm7|armv7|arm32/i.test(ua)) {
|
||||
} else if (/aarch|arm7|armv7/i.test(ua)) {
|
||||
return 'armv7l';
|
||||
} else if (/arm6|armv6|arm(\b|_)/i.test(ua)) {
|
||||
} else if (/arm6|armv6/i.test(ua)) {
|
||||
return 'armv6l';
|
||||
} else if (/ppc64le/i.test(ua)) {
|
||||
return 'ppc64le';
|
||||
|
||||
112
_webi/webi-pwsh.ps1
Normal file
112
_webi/webi-pwsh.ps1
Normal file
@@ -0,0 +1,112 @@
|
||||
#!/usr/bin/env pwsh
|
||||
|
||||
# this allows us to call ps1 files, which allows us to have spaces in filenames
|
||||
# ('powershell "$Env:USERPROFILE\test.ps1" foo' will fail if it has a space in
|
||||
# the path but '& "$Env:USERPROFILE\test.ps1" foo' will work even with a space)
|
||||
Set-ExecutionPolicy -Scope Process Bypass
|
||||
|
||||
# If a command returns an error, halt the script.
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Ignore progress events from cmdlets so Invoke-WebRequest is not painfully slow
|
||||
$ProgressPreference = 'SilentlyContinue'
|
||||
|
||||
# This is the canonical CPU arch when the process is emulated
|
||||
$my_arch = "$Env:PROCESSOR_ARCHITEW6432"
|
||||
IF ($my_arch -eq $null -or $my_arch -eq "") {
|
||||
# This is the canonical CPU arch when the process is native
|
||||
$my_arch = "$Env:PROCESSOR_ARCHITECTURE"
|
||||
}
|
||||
IF ($my_arch -eq "AMD64") {
|
||||
# Because PowerShell isn't ARM yet.
|
||||
# See https://oofhours.com/2020/02/04/powershell-on-windows-10-arm64/
|
||||
$my_os_arch = wmic os get osarchitecture
|
||||
|
||||
# Using -clike because of the trailing newline
|
||||
IF ($my_os_arch -clike "ARM 64*") {
|
||||
$my_arch = "ARM64"
|
||||
}
|
||||
}
|
||||
|
||||
$Env:WEBI_UA = "Windows/10+ $my_arch msvc"
|
||||
$exename = $args[0]
|
||||
|
||||
# Switch to userprofile
|
||||
Push-Location $Env:USERPROFILE
|
||||
|
||||
# Make paths if needed
|
||||
New-Item -Path .local\bin -ItemType Directory -Force | Out-Null
|
||||
# TODO replace all xbin with opt\bin\
|
||||
New-Item -Path .local\xbin -ItemType Directory -Force | Out-Null
|
||||
|
||||
# See note on Set-ExecutionPolicy above
|
||||
Set-Content -Path .local\bin\webi.bat -Value "@echo off`r`npushd %USERPROFILE%`r`npowershell -ExecutionPolicy Bypass .local\bin\webi-pwsh.ps1 %1`r`npopd"
|
||||
# Backwards-compat bugfix: remove old webi-pwsh.ps1 location
|
||||
Remove-Item -Path .local\bin\webi.ps1 -Recurse -ErrorAction Ignore
|
||||
if (!(Test-Path -Path .local\opt)) {
|
||||
New-Item -Path .local\opt -ItemType Directory -Force | Out-Null
|
||||
}
|
||||
# TODO windows version of mktemp -d
|
||||
if (!(Test-Path -Path .local\tmp)) {
|
||||
New-Item -Path .local\tmp -ItemType Directory -Force | Out-Null
|
||||
}
|
||||
|
||||
# TODO SetStrictMode
|
||||
# TODO Test-Path variable:global:Env:WEBI_HOST ???
|
||||
IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") {
|
||||
$Env:WEBI_HOST = "https://webinstall.dev"
|
||||
}
|
||||
|
||||
# {{ baseurl }}
|
||||
# {{ version }}
|
||||
|
||||
$my_version = 'v1.1.15'
|
||||
|
||||
## show help if no params given or help flags are used
|
||||
if ($null -eq $exename -or $exename -eq "-h" -or $exename -eq "--help" -or $exename -eq "help" -or $exename -eq "/?") {
|
||||
Write-Host "webi " -ForegroundColor Green -NoNewline; Write-Host "$my_version " -ForegroundColor Red -NoNewline; Write-Host "Copyright 2020+ AJ ONeal"
|
||||
Write-Host " https://webinstall.dev/webi" -ForegroundColor blue
|
||||
Write-Output ""
|
||||
Write-Output "SUMMARY"
|
||||
Write-Output " Webi is the best way to install the modern developer tools you love."
|
||||
Write-Output " It's fast, easy-to-remember, and conflict free."
|
||||
Write-Output ""
|
||||
Write-Output "USAGE"
|
||||
Write-Output " webi <thing1>[@version] [thing2] ..."
|
||||
Write-Output ""
|
||||
Write-Output "UNINSTALL"
|
||||
Write-Output " Almost everything that is installed with webi is scoped to"
|
||||
Write-Output " ~/.local/opt/<thing1>, so you can remove it like so:"
|
||||
Write-Output ""
|
||||
Write-Output " rmdir /s %USERPROFILE%\.local\opt\<thing1>"
|
||||
Write-Output " del %USERPROFILE%\.local\bin\<thing1>"
|
||||
Write-Output ""
|
||||
Write-Output " Some packages have special uninstall instructions, check"
|
||||
Write-Output " https://webinstall.dev/<thing1> to be sure."
|
||||
Write-Output ""
|
||||
Write-Output "FAQ"
|
||||
Write-Host " See " -NoNewline; Write-Host "https://webinstall.dev/faq" -ForegroundColor blue
|
||||
Write-Output ""
|
||||
Write-Output "ALWAYS REMEMBER"
|
||||
Write-Output " Friends don't let friends use brew for simple, modern tools that don't need it."
|
||||
exit 0
|
||||
}
|
||||
|
||||
if ($exename -eq "-V" -or $exename -eq "--version" -or $exename -eq "version" -or $exename -eq "/v") {
|
||||
Write-Host "webi " -ForegroundColor Green -NoNewline; Write-Host "$my_version " -ForegroundColor Red -NoNewline; Write-Host "Copyright 2020+ AJ ONeal"
|
||||
Write-Host " https://webinstall.dev/webi" -ForegroundColor blue
|
||||
exit 0
|
||||
}
|
||||
|
||||
# Fetch <whatever>.ps1
|
||||
# TODO detect formats
|
||||
$PKG_URL = "$Env:WEBI_HOST/api/installers/$exename.ps1?formats=zip,exe,tar,git&libc=msvc"
|
||||
Write-Output "Downloading $PKG_URL"
|
||||
# Invoke-WebRequest -UserAgent "Windows amd64" "$PKG_URL" -OutFile ".\.local\tmp\$exename.install.ps1"
|
||||
& curl.exe -fsSL -A "$Env:WEBI_UA" "$PKG_URL" -o .\.local\tmp\$exename.install.ps1
|
||||
|
||||
# Run <whatever>.ps1
|
||||
powershell .\.local\tmp\$exename.install.ps1
|
||||
|
||||
# Done
|
||||
Pop-Location
|
||||
@@ -72,8 +72,8 @@ gc "feat: new feature"
|
||||
|
||||
### Common aliases
|
||||
|
||||
Use *alias*es to make other tools you find around webi even _more_ convenient ⚡️
|
||||
(and powerful 💪).
|
||||
Use *alias*es to make other tools you find around webi even _more_ convenient
|
||||
⚡️ (and powerful 💪).
|
||||
|
||||
```sh
|
||||
aliasman curl 'curlie'
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
github_sources = BeyondCodeBootcamp/aliasman
|
||||
git_url = https://github.com/BeyondCodeBootcamp/aliasman.git
|
||||
29
aliasman/releases.js
Normal file
29
aliasman/releases.js
Normal file
@@ -0,0 +1,29 @@
|
||||
'use strict';
|
||||
|
||||
var githubSource = require('../_common/github-source.js');
|
||||
var owner = 'BeyondCodeBootcamp';
|
||||
var repo = 'aliasman';
|
||||
|
||||
module.exports = function (request) {
|
||||
let arches = [
|
||||
'amd64',
|
||||
'arm64',
|
||||
'armv6l',
|
||||
'armv7l',
|
||||
'ppc64le',
|
||||
'ppc64',
|
||||
's390x',
|
||||
'x86',
|
||||
];
|
||||
let oses = ['freebsd', 'linux', 'macos', 'posix'];
|
||||
return githubSource(request, owner, repo, oses, arches).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading archiver from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing archiver"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = mholt/archiver
|
||||
20
arc/releases.js
Normal file
20
arc/releases.js
Normal file
@@ -0,0 +1,20 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'mholt';
|
||||
var repo = 'archiver';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/bin/pwsh
|
||||
|
||||
Write-Output "'archiver@$Env:WEBI_TAG' is an alias for 'arc@$Env:WEBI_VERSION'"
|
||||
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
curl.exe -A MS -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell
|
||||
IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
curl.exe -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell
|
||||
|
||||
@@ -20,7 +20,7 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Checking for (or Installing) MSVC Runtime..."
|
||||
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
|
||||
|
||||
@@ -29,7 +29,7 @@ if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing AtomicParsley"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = wez/atomicparsley
|
||||
78
atomicparsley/releases.js
Normal file
78
atomicparsley/releases.js
Normal file
@@ -0,0 +1,78 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'wez';
|
||||
var repo = 'atomicparsley';
|
||||
|
||||
let targets = {
|
||||
x86win: {
|
||||
os: 'windows',
|
||||
arch: 'x86',
|
||||
libc: 'msvc',
|
||||
},
|
||||
x64win: {
|
||||
os: 'windows',
|
||||
arch: 'amd64',
|
||||
// https://github.com/wez/atomicparsley/issues/6#issuecomment-1364523028
|
||||
libc: 'msvc',
|
||||
},
|
||||
x64mac: {
|
||||
os: 'macos',
|
||||
arch: 'amd64',
|
||||
},
|
||||
x64lin: {
|
||||
os: 'linux',
|
||||
arch: 'amd64',
|
||||
libc: 'gnu',
|
||||
},
|
||||
x64musl: {
|
||||
os: 'linux',
|
||||
arch: 'amd64',
|
||||
libc: 'musl',
|
||||
},
|
||||
};
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
for (let rel of all.releases) {
|
||||
let windows32 = rel.name.includes('WindowsX86.');
|
||||
if (windows32) {
|
||||
Object.assign(rel, targets.x86win);
|
||||
continue;
|
||||
}
|
||||
|
||||
let windows64 = rel.name.includes('Windows.');
|
||||
if (windows64) {
|
||||
Object.assign(rel, targets.x64win);
|
||||
continue;
|
||||
}
|
||||
|
||||
let macos64 = rel.name.includes('MacOS');
|
||||
if (macos64) {
|
||||
Object.assign(rel, targets.x64mac);
|
||||
continue;
|
||||
}
|
||||
|
||||
let musl64 = rel.name.includes('Alpine');
|
||||
if (musl64) {
|
||||
Object.assign(rel, targets.x64musl);
|
||||
continue;
|
||||
}
|
||||
|
||||
let lin64 = rel.name.includes('Linux.');
|
||||
if (lin64) {
|
||||
Object.assign(rel, targets.x64lin);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all));
|
||||
//console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading awless from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing awless"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = wallix/awless
|
||||
22
awless/releases.js
Normal file
22
awless/releases.js
Normal file
@@ -0,0 +1,22 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'wallix';
|
||||
var repo = 'awless';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
// remove checksums and .deb
|
||||
all.releases = all.releases.filter(function (rel) {
|
||||
return !/(\.txt)|(\.deb)$/i.test(rel.name);
|
||||
});
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all));
|
||||
});
|
||||
}
|
||||
@@ -3,7 +3,7 @@
|
||||
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
|
||||
$EXENAME = "$Env:PKG_NAME.exe"
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
|
||||
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
@@ -11,11 +11,11 @@ if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
|
||||
# Fetch MSVC Runtime
|
||||
Write-Output "Checking for MSVC Runtime..."
|
||||
if (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
|
||||
IF (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
|
||||
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\xbin\$VERNAME")) {
|
||||
Write-Output "Installing $Env:PKG_NAME"
|
||||
# TODO: temp directory
|
||||
|
||||
@@ -35,13 +35,13 @@ if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
|
||||
# Settle unpacked archive into place
|
||||
Write-Output "New Name: $VERNAME"
|
||||
Write-Output "New Location: $Env:USERPROFILE\.local\bin\$VERNAME"
|
||||
Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\bin"
|
||||
Write-Output "New Location: $Env:USERPROFILE\.local\xbin\$VERNAME"
|
||||
Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\xbin"
|
||||
|
||||
# Exit tmp
|
||||
Pop-Location
|
||||
}
|
||||
|
||||
Write-Output "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\bin\$VERNAME'"
|
||||
Write-Output "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\xbin\$VERNAME'"
|
||||
Remove-Item -Path "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse -ErrorAction Ignore
|
||||
Copy-Item -Path "$Env:USERPROFILE\.local\bin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse
|
||||
Copy-Item -Path "$Env:USERPROFILE\.local\xbin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse
|
||||
|
||||
@@ -19,7 +19,7 @@ __init_bat() {
|
||||
}
|
||||
|
||||
pkg_install() {
|
||||
# ~/.local/bin
|
||||
# ~/.local/xbin
|
||||
mkdir -p "$pkg_src_bin"
|
||||
|
||||
# mv ./bat-*/bat ~/.local/opt/bat-v0.15.4/bin/bat
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = sharkdp/bat
|
||||
20
bat/releases.js
Normal file
20
bat/releases.js
Normal file
@@ -0,0 +1,20 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'sharkdp';
|
||||
var repo = 'bat';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
all.releases = all.releases.slice(0, 10);
|
||||
//console.info(JSON.stringify(all));
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -6,23 +6,15 @@ main() { (
|
||||
sed '1,/^#~\/.local\/bin\/brew-updater/d' "${0}" > ~/.local/bin/brew-update-hourly
|
||||
chmod a+x ~/.local/bin/brew-update-hourly
|
||||
|
||||
echo "Checking for serviceman..."
|
||||
~/.local/bin/webi serviceman
|
||||
if ! command -v serviceman > /dev/null; then
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
fi
|
||||
serviceman --version
|
||||
|
||||
serviceman add --agent \
|
||||
env PATH="$PATH" serviceman add --user \
|
||||
--workdir ~/.local/opt/brew/ \
|
||||
--name sh.brew.updater -- \
|
||||
~/.local/bin/brew-update-hourly
|
||||
); }
|
||||
|
||||
if ! main; then
|
||||
exit 1
|
||||
if main; then
|
||||
exit 0
|
||||
fi
|
||||
exit 0
|
||||
|
||||
#~/.local/bin/brew-updater
|
||||
#!/bin/sh
|
||||
|
||||
@@ -132,7 +132,9 @@ file)
|
||||
```
|
||||
3. Add your project to the system launcher, running as the current user
|
||||
```sh
|
||||
serviceman add --name 'my-project' --daemon -- \
|
||||
sudo env PATH="$PATH" \
|
||||
serviceman add --path="$PATH" --system \
|
||||
--username "$(whoami)" --name my-project -- \
|
||||
bun run ./my-project.js
|
||||
```
|
||||
4. Restart the logging service
|
||||
@@ -153,6 +155,6 @@ For **macOS**:
|
||||
```
|
||||
3. Add your project to the system launcher, running as the current user
|
||||
```sh
|
||||
serviceman add --agent --name 'my-project' -- \
|
||||
serviceman add --path="$PATH" --user --name my-project -- \
|
||||
bun run ./my-project.js
|
||||
```
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
github_releases = oven-sh/bun
|
||||
tag_prefix = bun-
|
||||
default_x86_64 = x86_64_v3
|
||||
x86_64_v2 = baseline
|
||||
variants = profile
|
||||
32
bun/releases.js
Normal file
32
bun/releases.js
Normal file
@@ -0,0 +1,32 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'oven-sh';
|
||||
var repo = 'bun';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
all.releases = all.releases
|
||||
.filter(function (r) {
|
||||
let isDebug = /-profile/.test(r.name);
|
||||
if (!isDebug) {
|
||||
return true;
|
||||
}
|
||||
})
|
||||
.map(function (r) {
|
||||
// bun-v0.5.1 => v0.5.1
|
||||
r.version = r.version.replace(/bun-/g, '');
|
||||
return r;
|
||||
});
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -819,10 +819,10 @@ To avoid the nitty-gritty details of `launchd` plist files, you can use
|
||||
2. Use Serviceman to create a _launchd_ plist file
|
||||
|
||||
```sh
|
||||
my_username="$(id -u -n)"
|
||||
my_username="$( id -u -n )"
|
||||
|
||||
serviceman add --agent --name 'caddy' --workdir ./ -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
serviceman add --user --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
|
||||
(this will create `~/Library/LaunchAgents/caddy.plist`)
|
||||
@@ -837,8 +837,8 @@ This process creates a _User-Level_ service in `~/Library/LaunchAgents`. To
|
||||
create a _System-Level_ service in `/Library/LaunchDaemons/` instead:
|
||||
|
||||
```sh
|
||||
serviceman add --name 'caddy' --workdir ./ --daemon -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
sudo serviceman add --system --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
|
||||
### How to run Caddy as a Windows Service
|
||||
@@ -847,7 +847,7 @@ serviceman add --name 'caddy' --workdir ./ --daemon -- \
|
||||
You can do this with _PowerShell_ by changing `YOUR_USER` in the script below
|
||||
and running it in `cmd.exe` as Administrator:
|
||||
```pwsh
|
||||
powershell.exe -WindowStyle Hidden -Command $r = Get-NetFirewallRule -DisplayName 'Caddy Web Server' 2> $null; if ($r) {write-host 'found rule';} else {New-NetFirewallRule -DisplayName 'Caddy Web Server' -Direction Inbound $HOME\\.local\\bin\\caddy.exe -Action Allow}
|
||||
powershell.exe -WindowStyle Hidden -Command $r = Get-NetFirewallRule -DisplayName 'Caddy Web Server' 2> $null; if ($r) {write-host 'found rule';} else {New-NetFirewallRule -DisplayName 'Caddy Web Server' -Direction Inbound C:\\Users\\YOUR_USER\\.local\\bin\\caddy.exe -Action Allow}
|
||||
```
|
||||
2. Install [`serviceman`](../serviceman/)
|
||||
```sh
|
||||
@@ -856,7 +856,7 @@ serviceman add --name 'caddy' --workdir ./ --daemon -- \
|
||||
3. Create a **Startup Registry Entry** with Serviceman.
|
||||
```sh
|
||||
serviceman.exe add --name caddy -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
4. You can manage the service directly with Serviceman. For example:
|
||||
```sh
|
||||
@@ -901,8 +901,10 @@ See the notes below to run as a **User Service** or use the JSON Config.
|
||||
```
|
||||
4. Use Serviceman to create a _systemd_ config file.
|
||||
```sh
|
||||
serviceman add --name 'caddy' --daemon -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
my_username="$( id -u -n )"
|
||||
sudo env PATH="$PATH" \
|
||||
serviceman add --system --username "${my_username}" --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
(this will create `/etc/systemd/system/caddy.service`)
|
||||
5. Manage the service with `systemctl` and `journalctl`:
|
||||
@@ -913,10 +915,10 @@ See the notes below to run as a **User Service** or use the JSON Config.
|
||||
|
||||
To create a **User Service** instead:
|
||||
|
||||
- use `--agent` when running `serviceman`:
|
||||
- don't use `sudo`, but do use `--user` when running `serviceman`:
|
||||
```sh
|
||||
serviceman add --agent --name caddy -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
serviceman add --user --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
(this will create `~/.config/systemd/user/`)
|
||||
- user the `--user` flag to manage services and logs:
|
||||
@@ -1181,8 +1183,7 @@ To prevent search engine and browser confusion
|
||||
- _DO NOT_ prevent crawling via `robots.txt` \
|
||||
(counter-intuitive, but pages _must_ be crawled for links to _NOT_ be indexed)
|
||||
- _all_ domains using public TLS certs _will_ be indexed by default \
|
||||
(they are all linked to and crawled from various Certificate Transparency
|
||||
reports)
|
||||
(they are all linked to and crawled from various Certificate Transparency reports)
|
||||
- follow these guidelines even if the dev sites use HTTP Basic Auth
|
||||
|
||||
```Caddyfile
|
||||
@@ -1362,13 +1363,19 @@ See also: <https://caddyserver.com/docs/running>
|
||||
2. Generate the `service` file: \
|
||||
- JSON Config
|
||||
```sh
|
||||
serviceman add --name 'caddy' --daemon -- \
|
||||
caddy run --resume --envfile ./caddy.env
|
||||
my_app_user="$( id -u -n )"
|
||||
sudo env PATH="${PATH}" \
|
||||
serviceman add --system --cap-net-bind \
|
||||
--username "${my_app_user}" --name caddy -- \
|
||||
caddy run --resume --envfile ./caddy.env
|
||||
```
|
||||
- Caddyfile
|
||||
```sh
|
||||
serviceman add --name 'caddy' --daemon -- \
|
||||
caddy run --config ./Caddyfile --envfile ./caddy.env
|
||||
my_app_user="$( id -u -n )"
|
||||
sudo env PATH="${PATH}" \
|
||||
serviceman add --system --cap-net-bind \
|
||||
--username "${my_app_user}" --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ./caddy.env
|
||||
```
|
||||
3. Reload `systemd` config files, the logging service (it may not be started on
|
||||
a new VPS), and caddy
|
||||
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading caddy from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing caddy"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = caddyserver/caddy
|
||||
22
caddy/releases.js
Normal file
22
caddy/releases.js
Normal file
@@ -0,0 +1,22 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'caddyserver';
|
||||
var repo = 'caddy';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
// remove checksums and .deb
|
||||
all.releases = all.releases.filter(function (rel) {
|
||||
return !/(\.txt)|(\.deb)$/i.test(rel.name);
|
||||
});
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all));
|
||||
});
|
||||
}
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading chromedriver from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing chromedriver"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -20,28 +20,20 @@ __init_chromedriver() {
|
||||
|
||||
# pkg_install must be defined by every package
|
||||
pkg_install() {
|
||||
# ~/.local/opt/chromedriver-v121.0.6130.0/bin
|
||||
# ~/.local/opt/chromedriver-v88.0.4324.96/bin
|
||||
mkdir -p "$(dirname "$pkg_src_cmd")"
|
||||
|
||||
# mv ./chromedriver-macos-arm64/chromedriver \
|
||||
# ~/.local/opt/chromedriver-v121.0.6130.0/bin/chromedriver
|
||||
mv ./chromedriver-*/chromedriver "$pkg_src_cmd"
|
||||
|
||||
echo ""
|
||||
echo " $(t_warn 'MANUAL STEPS TO FINISH:') you may need to install libnss3:"
|
||||
echo " $(t_cmd 'sudo apt install -y') $(t_warn 'libnss3')"
|
||||
# mv ./chromedriver-*/chromedriver ~/.local/opt/chromedriver-v88.0.4324.96/bin/chromedriver
|
||||
mv ./chromedriver* "$pkg_src_cmd"
|
||||
}
|
||||
|
||||
# pkg_get_current_version is recommended, but (soon) not required
|
||||
pkg_get_current_version() {
|
||||
# 'chromedriver --version' has output in this format:
|
||||
# ChromeDriver 121.0.6130.0 (5fc19c7ab5e88bb674c2efc65db4b7890a52a4ec-refs/branch-heads/6130@{#1})
|
||||
# ChromeDriver 88.0.4324.96 (68dba2d8a0b149a1d3afac56fa74648032bcf46b-refs/branch-heads/4324@{#1784})
|
||||
# This trims it down to just the version number:
|
||||
# 121.0.6130.0
|
||||
chromedriver --version 2> /dev/null |
|
||||
head -n 1 |
|
||||
tr -s ' ' |
|
||||
cut -d' ' -f2
|
||||
# 88.0.4324.96
|
||||
chromedriver --version 2> /dev/null | head -n 1 | cut -d ' ' -f 2
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
source = chromedist
|
||||
88
chromedriver/releases.js
Normal file
88
chromedriver/releases.js
Normal file
@@ -0,0 +1,88 @@
|
||||
'use strict';
|
||||
|
||||
var matchers = {
|
||||
key: /.*Key>(.*)<\/Key.*/,
|
||||
generation: /.*Generation>(.*)<\/Generation.*/,
|
||||
metaGeneration: /.*MetaGeneration>(.*)<\/MetaGeneration.*/,
|
||||
lastModified: /.*LastModified>(.*)<\/LastModified.*/,
|
||||
etag: /.*ETag>(.*)<\/ETag.*/,
|
||||
size: /.*Size>(.*)<\/Size.*/,
|
||||
};
|
||||
var baseUrl = 'https://chromedriver.storage.googleapis.com';
|
||||
|
||||
module.exports = function (request) {
|
||||
var all = {
|
||||
download: '',
|
||||
releases: [],
|
||||
};
|
||||
|
||||
// XML
|
||||
return request({
|
||||
url: 'https://chromedriver.storage.googleapis.com/',
|
||||
json: false,
|
||||
})
|
||||
.then(function (resp) {
|
||||
var body = resp.body;
|
||||
var groups = body.split(/<\/?Contents>/g);
|
||||
// get rid of leading and trailing junk
|
||||
groups.shift();
|
||||
groups.pop();
|
||||
var metas = groups.map(function (group) {
|
||||
return {
|
||||
key: group.replace(matchers.key, '$1'),
|
||||
//generation: group.replace(matchers.generation, '$1'),
|
||||
//metaGeneration: group.replace(matchers.metaGeneration, '$1'),
|
||||
lastModified: group.replace(matchers.lastModified, '$1'),
|
||||
//etag: group.replace(matchers.etag, '$1'),
|
||||
//size: group.replace(matchers.size, '$1')
|
||||
};
|
||||
});
|
||||
all.download = baseUrl + '/{{ download }}';
|
||||
metas.forEach(function (asset) {
|
||||
if (!asset.key.includes('chromedriver')) {
|
||||
// skip the indexes, images, etc
|
||||
return null;
|
||||
}
|
||||
|
||||
var osname = asset.key.replace(/.*(win|mac|linux)/, '$1');
|
||||
var arch;
|
||||
if (asset.key.includes('linux')) {
|
||||
osname = 'linux';
|
||||
} else if (asset.key.includes('mac64')) {
|
||||
osname = 'macos';
|
||||
if (asset.key.includes('_m1.')) {
|
||||
arch = 'arm64';
|
||||
}
|
||||
} else if (asset.key.includes('win')) {
|
||||
osname = 'windows';
|
||||
arch = 'amd64';
|
||||
}
|
||||
all.releases.push({
|
||||
// 87.0.4280.88/chromedriver_win32.zip => 87.0.4280.88
|
||||
version: asset.key.replace(/(.*)\/.*/, '$1'),
|
||||
lts: false,
|
||||
channel: 'stable',
|
||||
date: asset.lastModified.replace(/T.*/, '$1'),
|
||||
os: osname,
|
||||
arch: arch,
|
||||
hash: '-', // not sure about including etag as hash yet
|
||||
download: asset.key,
|
||||
});
|
||||
});
|
||||
})
|
||||
.then(function () {
|
||||
all.releases.sort(function (a, b) {
|
||||
return new Date(b.date).valueOf() - new Date(a.date).valueOf();
|
||||
});
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the latest 5 for demonstration
|
||||
all.releases = all.releases.slice(-20);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -1,47 +0,0 @@
|
||||
---
|
||||
title: cilium
|
||||
homepage: https://github.com/cilium/cilium-cli
|
||||
tagline: |
|
||||
cilium: manage & troubleshoot Kubernetes clusters running Cilium
|
||||
---
|
||||
|
||||
To update or switch versions, run `webi cilium@stable` (or `@v2`, `@beta`,etc).
|
||||
|
||||
### Files
|
||||
|
||||
These are the files / directories that are created and/or modified with this
|
||||
install:
|
||||
|
||||
```text
|
||||
~/.config/envman/PATH.env
|
||||
~/.local/bin/cilium
|
||||
~/.local/opt/cilium/
|
||||
```
|
||||
|
||||
## Cheat Sheet
|
||||
|
||||
> Cilium is an open source, cloud native solution for providing, securing, and
|
||||
> observing network connectivity between workloads, fueled by the revolutionary
|
||||
> Kernel technology eBPF.
|
||||
|
||||
Quick Start User Guide:
|
||||
|
||||
<https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/#k8s-install-quick>
|
||||
|
||||
To install the default version of the Cilium image:
|
||||
|
||||
```sh
|
||||
cilium install
|
||||
```
|
||||
|
||||
To upgrade to a specific version of the Cilium image:
|
||||
|
||||
```sh
|
||||
cilium upgrade --version v1.15.3
|
||||
```
|
||||
|
||||
To check the status of the current Cilium deployment:
|
||||
|
||||
```sh
|
||||
cilium status
|
||||
```
|
||||
@@ -1,45 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
|
||||
##################
|
||||
# Install cilium #
|
||||
##################
|
||||
|
||||
$pkg_cmd_name = "cilium"
|
||||
|
||||
$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\cilium.exe"
|
||||
$pkg_dst = "$pkg_dst_cmd"
|
||||
|
||||
$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\cilium-v$Env:WEBI_VERSION\bin\cilium.exe"
|
||||
$pkg_src_bin = "$Env:USERPROFILE\.local\opt\cilium-v$Env:WEBI_VERSION\bin"
|
||||
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\cilium-v$Env:WEBI_VERSION"
|
||||
$pkg_src = "$pkg_src_cmd"
|
||||
|
||||
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading cilium from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing cilium"
|
||||
Push-Location .local\tmp
|
||||
|
||||
Remove-Item -Path ".\cilium-v*" -Recurse -ErrorAction Ignore
|
||||
Remove-Item -Path ".\cilium.exe" -Recurse -ErrorAction Ignore
|
||||
|
||||
Write-Output "Unpacking $pkg_download"
|
||||
& tar xf "$pkg_download"
|
||||
|
||||
Write-Output "Install Location: $pkg_src_cmd"
|
||||
New-Item "$pkg_src_bin" -ItemType Directory -Force
|
||||
Move-Item -Path ".\cilium-*\cilium.exe" -Destination "$pkg_src_bin"
|
||||
|
||||
Pop-Location
|
||||
}
|
||||
|
||||
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
|
||||
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore
|
||||
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse
|
||||
@@ -1,39 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
__init_cilium() {
|
||||
set -e
|
||||
set -u
|
||||
|
||||
##################
|
||||
# Install cilium #
|
||||
##################
|
||||
|
||||
pkg_cmd_name="cilium"
|
||||
|
||||
pkg_dst_cmd="$HOME/.local/bin/cilium"
|
||||
pkg_dst="$pkg_dst_cmd"
|
||||
|
||||
pkg_src_cmd="$HOME/.local/opt/cilium-v$WEBI_VERSION/bin/cilium"
|
||||
pkg_src_dir="$HOME/.local/opt/cilium-v$WEBI_VERSION"
|
||||
pkg_src="$pkg_src_cmd"
|
||||
|
||||
WEBI_SINGLE=true
|
||||
|
||||
# pkg_install must be defined by every package
|
||||
pkg_install() {
|
||||
# ~/.local/opt/cilium-v0.16.16/bin
|
||||
mkdir -p "$(dirname "${pkg_src_cmd}")"
|
||||
|
||||
# mv ./hugo ~/.local/opt/cilium-v0.16.16/bin/
|
||||
mv ./cilium "${pkg_src_cmd}"
|
||||
}
|
||||
|
||||
pkg_get_current_version() {
|
||||
cilium version 2> /dev/null |
|
||||
head -n 1 |
|
||||
cut -d ' ' -f 2
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
__init_cilium
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = cilium/cilium-cli
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$pkg_download")) {
|
||||
IF (!(Test-Path -Path "$pkg_download")) {
|
||||
Write-Output "Downloading cmake from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
Write-Output "Installing cmake"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = Kitware/CMake
|
||||
49
cmake/releases.js
Normal file
49
cmake/releases.js
Normal file
@@ -0,0 +1,49 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'Kitware';
|
||||
var repo = 'CMake';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
for (let rel of all.releases) {
|
||||
{
|
||||
let linuxRe = /(\b|_)(linux|gnu)(\b|_)/i;
|
||||
let isLinux = linuxRe.test(rel.download) || linuxRe.test(rel.name);
|
||||
|
||||
if (isLinux) {
|
||||
let muslRe = /(\b|_)(musl|alpine)(\b|_)/i;
|
||||
let isMusl = muslRe.test(rel.download) || muslRe.test(rel.name);
|
||||
if (isMusl) {
|
||||
rel.libc = 'musl';
|
||||
} else {
|
||||
rel.libc = 'gnu';
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let windowsRe = /(\b|_)(win\d*|windows\d*)(\b|_)/i;
|
||||
let isWindows =
|
||||
windowsRe.test(rel.download) || windowsRe.test(rel.name);
|
||||
|
||||
if (isWindows) {
|
||||
rel.libc = 'msvc';
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -1,924 +0,0 @@
|
||||
// Command webicached is the release cache daemon. It fetches releases
|
||||
// from upstream sources, classifies build assets, and writes them to
|
||||
// the _cache/ directory in the format the Node.js server expects.
|
||||
//
|
||||
// This is the Go replacement for the Node.js release-fetching pipeline.
|
||||
// It reads releases.conf files to discover packages, fetches from the
|
||||
// configured source, classifies assets, and writes to fsstore.
|
||||
//
|
||||
// Default mode: classify all from existing rawcache on startup, then
|
||||
// fetch+refresh one package per tick (round-robin, 15m default).
|
||||
//
|
||||
// Usage:
|
||||
//
|
||||
// go run ./cmd/webicached # default: round-robin, one per tick
|
||||
// go run ./cmd/webicached -eager # fetch all packages on startup
|
||||
// go run ./cmd/webicached -once -no-fetch # classify from rawcache and exit
|
||||
// go run ./cmd/webicached bat goreleaser # only these packages
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"math/rand/v2"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/joho/godotenv"
|
||||
"github.com/webinstall/webi-installers/internal/classifypkg"
|
||||
"github.com/webinstall/webi-installers/internal/httpclient"
|
||||
"github.com/webinstall/webi-installers/internal/installerconf"
|
||||
"github.com/webinstall/webi-installers/internal/rawcache"
|
||||
"github.com/webinstall/webi-installers/internal/releases/chromedist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/flutterdist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/gitea"
|
||||
"github.com/webinstall/webi-installers/internal/releases/github"
|
||||
"github.com/webinstall/webi-installers/internal/releases/githubish"
|
||||
"github.com/webinstall/webi-installers/internal/releases/gittag"
|
||||
"github.com/webinstall/webi-installers/internal/releases/golang"
|
||||
"github.com/webinstall/webi-installers/internal/releases/gpgdist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/hashicorp"
|
||||
"github.com/webinstall/webi-installers/internal/releases/iterm2dist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/juliadist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/mariadbdist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/nodedist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/servicemandist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/zigdist"
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
"github.com/webinstall/webi-installers/internal/storage/fsstore"
|
||||
)
|
||||
|
||||
var (
|
||||
name = "webicached"
|
||||
version = "0.0.0-dev"
|
||||
commit = "0000000"
|
||||
date = "0001-01-01"
|
||||
licenseYear = "2024"
|
||||
licenseOwner = "AJ ONeal"
|
||||
licenseType = "MPL-2.0"
|
||||
)
|
||||
|
||||
func printVersion(w io.Writer) {
|
||||
b_ver := strings.TrimPrefix(version, "v")
|
||||
_, _ = fmt.Fprintf(w, "%s v%s %s (%s)\n", name, b_ver, commit[:7], date)
|
||||
_, _ = fmt.Fprintf(w, "Copyright (C) %s %s\n", licenseYear, licenseOwner)
|
||||
_, _ = fmt.Fprintf(w, "Licensed under %s\n", licenseType)
|
||||
}
|
||||
|
||||
type MainConfig struct {
|
||||
envFile string
|
||||
confDir string
|
||||
cacheDir string
|
||||
rawDir string
|
||||
token string
|
||||
once bool
|
||||
noFetch bool
|
||||
shallow bool
|
||||
eager bool
|
||||
interval time.Duration
|
||||
pageDelay time.Duration
|
||||
}
|
||||
|
||||
// WebiCache holds the configuration for the cache daemon.
|
||||
type WebiCache struct {
|
||||
ConfDir string // root directory with {pkg}/releases.conf files
|
||||
Store storage.Store // classified asset storage (fsstore)
|
||||
RawDir string // raw upstream response cache
|
||||
Client *http.Client // HTTP client for upstream calls
|
||||
Auth *githubish.Auth // GitHub API auth (optional)
|
||||
Shallow bool // fetch only the first page of releases
|
||||
NoFetch bool // skip fetching, classify from existing raw data only
|
||||
PageDelay time.Duration // delay between paginated API requests
|
||||
}
|
||||
|
||||
// delayTransport wraps an http.RoundTripper to add a delay between requests.
|
||||
type delayTransport struct {
|
||||
base http.RoundTripper
|
||||
delay time.Duration
|
||||
last time.Time
|
||||
}
|
||||
|
||||
func (t *delayTransport) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||
if !t.last.IsZero() && t.delay > 0 {
|
||||
if wait := t.delay - time.Since(t.last); wait > 0 {
|
||||
time.Sleep(wait)
|
||||
}
|
||||
}
|
||||
t.last = time.Now()
|
||||
return t.base.RoundTrip(req)
|
||||
}
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 {
|
||||
switch os.Args[1] {
|
||||
case "-V", "-version", "--version", "version":
|
||||
printVersion(os.Stdout)
|
||||
os.Exit(0)
|
||||
case "help", "-help", "--help":
|
||||
printVersion(os.Stdout)
|
||||
fmt.Fprintln(os.Stdout, "")
|
||||
fs := flag.NewFlagSet(os.Args[0], flag.ContinueOnError)
|
||||
fs.SetOutput(os.Stdout)
|
||||
registerFlags(fs, &MainConfig{})
|
||||
fs.Usage()
|
||||
os.Exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
cfg := MainConfig{}
|
||||
fs := flag.NewFlagSet(os.Args[0], flag.ContinueOnError)
|
||||
registerFlags(fs, &cfg)
|
||||
if err := fs.Parse(os.Args[1:]); err != nil {
|
||||
if errors.Is(err, flag.ErrHelp) {
|
||||
os.Exit(0)
|
||||
}
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
cfg.cacheDir = expandHome(cfg.cacheDir)
|
||||
cfg.rawDir = expandHome(cfg.rawDir)
|
||||
|
||||
if cfg.envFile != "" {
|
||||
if err := godotenv.Load(cfg.envFile); err != nil {
|
||||
log.Fatalf("envfile: %v", err)
|
||||
}
|
||||
}
|
||||
if cfg.token == "" {
|
||||
cfg.token = os.Getenv("GITHUB_TOKEN")
|
||||
}
|
||||
|
||||
fss, err := fsstore.New(cfg.cacheDir)
|
||||
if err != nil {
|
||||
log.Fatalf("fsstore: %v", err)
|
||||
}
|
||||
var store storage.Store = fss
|
||||
|
||||
var auth *githubish.Auth
|
||||
if cfg.token != "" {
|
||||
auth = &githubish.Auth{Token: cfg.token}
|
||||
}
|
||||
|
||||
client := httpclient.New()
|
||||
if cfg.pageDelay > 0 {
|
||||
client.Transport = &delayTransport{
|
||||
base: client.Transport,
|
||||
delay: cfg.pageDelay,
|
||||
}
|
||||
}
|
||||
|
||||
wc := &WebiCache{
|
||||
ConfDir: cfg.confDir,
|
||||
Store: store,
|
||||
RawDir: cfg.rawDir,
|
||||
Client: client,
|
||||
Auth: auth,
|
||||
Shallow: cfg.shallow,
|
||||
NoFetch: cfg.noFetch,
|
||||
PageDelay: cfg.pageDelay,
|
||||
}
|
||||
|
||||
filterPkgs := fs.Args()
|
||||
|
||||
if cfg.eager {
|
||||
wc.Run(filterPkgs)
|
||||
if cfg.once {
|
||||
return
|
||||
}
|
||||
} else if cfg.once {
|
||||
wc.Run(filterPkgs)
|
||||
return
|
||||
} else {
|
||||
saved := wc.NoFetch
|
||||
wc.NoFetch = true
|
||||
wc.Run(filterPkgs)
|
||||
wc.NoFetch = saved
|
||||
}
|
||||
|
||||
packages, err := discover(wc.ConfDir)
|
||||
if err != nil {
|
||||
log.Fatalf("discover: %v", err)
|
||||
}
|
||||
nameSet := make(map[string]bool, len(filterPkgs))
|
||||
for _, a := range filterPkgs {
|
||||
nameSet[a] = true
|
||||
}
|
||||
if len(filterPkgs) > 0 {
|
||||
var filtered []pkgConf
|
||||
for _, p := range packages {
|
||||
if nameSet[p.name] {
|
||||
filtered = append(filtered, p)
|
||||
}
|
||||
}
|
||||
packages = filtered
|
||||
}
|
||||
|
||||
var real []pkgConf
|
||||
for _, pkg := range packages {
|
||||
if pkg.conf.AliasOf == "" {
|
||||
real = append(real, pkg)
|
||||
}
|
||||
}
|
||||
|
||||
// rescanNew appends any conf files added since the last scan.
|
||||
// Returns true when at least one new package was added so the caller
|
||||
// can restart the batch loop and process new packages immediately.
|
||||
rescanNew := func() bool {
|
||||
discovered, err := discover(wc.ConfDir)
|
||||
if err != nil {
|
||||
log.Printf("rescan: %v", err)
|
||||
return false
|
||||
}
|
||||
known := make(map[string]bool, len(real))
|
||||
for _, p := range real {
|
||||
known[p.name] = true
|
||||
}
|
||||
added := false
|
||||
for _, p := range discovered {
|
||||
if p.conf.AliasOf != "" || known[p.name] {
|
||||
continue
|
||||
}
|
||||
if len(filterPkgs) > 0 && !nameSet[p.name] {
|
||||
continue
|
||||
}
|
||||
log.Printf("discovered new package: %s (source=%s)", p.name, p.conf.Source)
|
||||
real = append(real, p)
|
||||
added = true
|
||||
}
|
||||
return added
|
||||
}
|
||||
|
||||
log.Printf("refreshing %d packages, interval %s, batch size 20 (ctrl-c to stop)", len(real), cfg.interval)
|
||||
for {
|
||||
// Rescan before computing staleness so newly added conf files are
|
||||
// included immediately. New packages have a zero timestamp and sort
|
||||
// to the front of the stale list, so they are processed next.
|
||||
rescanNew()
|
||||
|
||||
stale := wc.stalest(real)
|
||||
if len(stale) == 0 {
|
||||
log.Printf("all packages fresh, sleeping %s", cfg.interval)
|
||||
time.Sleep(cfg.interval)
|
||||
continue
|
||||
}
|
||||
|
||||
batch := stale
|
||||
if len(batch) > 20 {
|
||||
batch = batch[:20]
|
||||
}
|
||||
rand.Shuffle(len(batch), func(i, j int) {
|
||||
batch[i], batch[j] = batch[j], batch[i]
|
||||
})
|
||||
|
||||
log.Printf("batch: %d stale, refreshing %d (most stale first)", len(stale), len(batch))
|
||||
for _, pkg := range batch {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
|
||||
if err := wc.refreshPackage(ctx, pkg); err != nil {
|
||||
log.Printf(" ERROR %s: %v", pkg.name, err)
|
||||
}
|
||||
cancel()
|
||||
time.Sleep(cfg.interval)
|
||||
// Rescan mid-batch so new packages preempt remaining batch items.
|
||||
if rescanNew() {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func registerFlags(fs *flag.FlagSet, cfg *MainConfig) {
|
||||
fs.StringVar(&cfg.envFile, "envfile", "", "path to .env file to load before running")
|
||||
fs.StringVar(&cfg.confDir, "conf", ".", "root directory containing {pkg}/releases.conf files")
|
||||
fs.StringVar(&cfg.cacheDir, "legacy", "~/.cache/webi/legacy", "legacy cache directory (fsstore root)")
|
||||
fs.StringVar(&cfg.rawDir, "raw", "~/.cache/webi/raw", "raw cache directory for upstream responses")
|
||||
fs.StringVar(&cfg.token, "token", "", "GitHub API token (or set $GITHUB_TOKEN)")
|
||||
fs.BoolVar(&cfg.once, "once", false, "run once then exit (no periodic refresh)")
|
||||
fs.BoolVar(&cfg.noFetch, "no-fetch", false, "skip fetching, classify from existing raw data only")
|
||||
fs.BoolVar(&cfg.shallow, "shallow", false, "fetch only the first page of releases (latest)")
|
||||
fs.BoolVar(&cfg.eager, "eager", false, "fetch all packages on startup (default: one per tick)")
|
||||
fs.DurationVar(&cfg.interval, "interval", 9*time.Second, "delay between individual package fetches")
|
||||
fs.DurationVar(&cfg.pageDelay, "page-delay", 2*time.Second, "delay between paginated API requests")
|
||||
}
|
||||
|
||||
func expandHome(path string) string {
|
||||
if !strings.HasPrefix(path, "~/") {
|
||||
return path
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
return path
|
||||
}
|
||||
return filepath.Join(home, path[2:])
|
||||
}
|
||||
|
||||
// stalest returns packages sorted by most stale first (oldest UpdatedAt).
|
||||
// Packages with no cache entry or empty assets are considered most stale.
|
||||
func (wc *WebiCache) stalest(packages []pkgConf) []pkgConf {
|
||||
type stamped struct {
|
||||
pkg pkgConf
|
||||
updatedAt time.Time
|
||||
}
|
||||
|
||||
var stale []stamped
|
||||
ctx := context.Background()
|
||||
for _, pkg := range packages {
|
||||
data, err := wc.Store.Load(ctx, pkg.name)
|
||||
var t time.Time
|
||||
if err == nil && data != nil {
|
||||
t = data.UpdatedAt
|
||||
}
|
||||
// Never fetched, or older than 10 minutes.
|
||||
// 0-asset results are not treated as perpetually stale — packages that
|
||||
// produce no classifiable assets (e.g. galera) respect the timestamp.
|
||||
if t.IsZero() || time.Since(t) > 10*time.Minute {
|
||||
stale = append(stale, stamped{pkg: pkg, updatedAt: t})
|
||||
}
|
||||
}
|
||||
|
||||
sort.SliceStable(stale, func(i, j int) bool {
|
||||
ti, tj := stale[i].updatedAt, stale[j].updatedAt
|
||||
if ti.Equal(tj) {
|
||||
return stale[i].pkg.name < stale[j].pkg.name
|
||||
}
|
||||
return ti.Before(tj)
|
||||
})
|
||||
|
||||
result := make([]pkgConf, len(stale))
|
||||
for i, s := range stale {
|
||||
result[i] = s.pkg
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// Run discovers packages and refreshes each one.
|
||||
func (wc *WebiCache) Run(filterPkgs []string) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
packages, err := discover(wc.ConfDir)
|
||||
if err != nil {
|
||||
log.Printf("discover: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(filterPkgs) > 0 {
|
||||
nameSet := make(map[string]bool, len(filterPkgs))
|
||||
for _, a := range filterPkgs {
|
||||
nameSet[a] = true
|
||||
}
|
||||
var filtered []pkgConf
|
||||
for _, p := range packages {
|
||||
if nameSet[p.name] {
|
||||
filtered = append(filtered, p)
|
||||
}
|
||||
}
|
||||
packages = filtered
|
||||
}
|
||||
|
||||
var real []pkgConf
|
||||
for _, pkg := range packages {
|
||||
if pkg.conf.AliasOf != "" {
|
||||
continue
|
||||
}
|
||||
real = append(real, pkg)
|
||||
}
|
||||
|
||||
log.Printf("refreshing %d packages", len(real))
|
||||
runStart := time.Now()
|
||||
|
||||
for _, pkg := range real {
|
||||
if err := wc.refreshPackage(ctx, pkg); err != nil {
|
||||
log.Printf(" ERROR %s: %v", pkg.name, err)
|
||||
}
|
||||
}
|
||||
|
||||
log.Printf("refreshed %d packages in %s", len(real), time.Since(runStart))
|
||||
}
|
||||
|
||||
type pkgConf struct {
|
||||
name string
|
||||
conf *installerconf.Conf
|
||||
}
|
||||
|
||||
func discover(dir string) ([]pkgConf, error) {
|
||||
pattern := filepath.Join(dir, "*", "releases.conf")
|
||||
matches, err := filepath.Glob(pattern)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var packages []pkgConf
|
||||
for _, path := range matches {
|
||||
pkgDir := filepath.Dir(path)
|
||||
name := filepath.Base(pkgDir)
|
||||
if strings.HasPrefix(name, "_") {
|
||||
continue
|
||||
}
|
||||
|
||||
// If the package directory is a symlink, treat it as an alias
|
||||
// of the symlink target (e.g. rust.vim → vim-rust).
|
||||
fi, err := os.Lstat(filepath.Join(dir, name))
|
||||
if err != nil {
|
||||
log.Printf("warning: %s: %v", name, err)
|
||||
continue
|
||||
}
|
||||
if fi.Mode()&os.ModeSymlink != 0 {
|
||||
target, err := os.Readlink(filepath.Join(dir, name))
|
||||
if err != nil {
|
||||
log.Printf("warning: readlink %s: %v", name, err)
|
||||
continue
|
||||
}
|
||||
packages = append(packages, pkgConf{
|
||||
name: name,
|
||||
conf: &installerconf.Conf{AliasOf: target},
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
conf, err := installerconf.Read(path)
|
||||
if err != nil {
|
||||
log.Printf("warning: %s: %v", path, err)
|
||||
continue
|
||||
}
|
||||
packages = append(packages, pkgConf{name: name, conf: conf})
|
||||
}
|
||||
|
||||
sort.Slice(packages, func(i, j int) bool {
|
||||
return packages[i].name < packages[j].name
|
||||
})
|
||||
return packages, nil
|
||||
}
|
||||
|
||||
// refreshPackage does the full pipeline for one package:
|
||||
// fetch raw → classify → write to fsstore.
|
||||
func (wc *WebiCache) refreshPackage(ctx context.Context, pkg pkgConf) error {
|
||||
pkgStart := time.Now()
|
||||
name := pkg.name
|
||||
conf := pkg.conf
|
||||
|
||||
// Step 1: Fetch raw upstream data to rawcache (unless -no-fetch).
|
||||
if !wc.NoFetch {
|
||||
shallow := wc.Shallow
|
||||
if !shallow {
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, name))
|
||||
if err == nil && d.Populated() {
|
||||
shallow = true
|
||||
}
|
||||
}
|
||||
fetchStart := time.Now()
|
||||
if err := wc.fetchRaw(ctx, pkg, shallow); err != nil {
|
||||
return fmt.Errorf("fetch: %w", err)
|
||||
}
|
||||
log.Printf(" %s: fetch %s", name, time.Since(fetchStart))
|
||||
}
|
||||
|
||||
// Step 2: Classify raw data into assets, tag variants, apply config.
|
||||
classifyStart := time.Now()
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, name))
|
||||
if err != nil {
|
||||
return fmt.Errorf("rawcache open: %w", err)
|
||||
}
|
||||
|
||||
// Open supplementary gittag raw cache if available (for packages with
|
||||
// git_url that use a non-gittag source type like servicemandist).
|
||||
var gitTagDir *rawcache.Dir
|
||||
if conf.GitURL != "" && conf.Source != "gittag" {
|
||||
gd, gdErr := rawcache.Open(filepath.Join(wc.RawDir, "_gittag", name))
|
||||
if gdErr == nil && gd.Populated() {
|
||||
gitTagDir = gd
|
||||
}
|
||||
}
|
||||
|
||||
assets, err := classifypkg.Package(name, conf, d, gitTagDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("classify: %w", err)
|
||||
}
|
||||
classifyDur := time.Since(classifyStart)
|
||||
|
||||
// Step 3: Write to fsstore.
|
||||
writeStart := time.Now()
|
||||
tx, err := wc.Store.BeginRefresh(ctx, name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin refresh: %w", err)
|
||||
}
|
||||
if err := tx.Put(assets); err != nil {
|
||||
tx.Rollback()
|
||||
return fmt.Errorf("put: %w", err)
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return fmt.Errorf("commit: %w", err)
|
||||
}
|
||||
writeDur := time.Since(writeStart)
|
||||
|
||||
log.Printf(" %s: %d assets (classify %s, write %s, total %s)",
|
||||
name, len(assets), classifyDur, writeDur, time.Since(pkgStart))
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- Fetch raw ---
|
||||
|
||||
func (wc *WebiCache) fetchRaw(ctx context.Context, pkg pkgConf, shallow bool) error {
|
||||
switch pkg.conf.Source {
|
||||
case "github", "githubsource":
|
||||
if err := wc.fetchGitHub(ctx, pkg.name, pkg.conf, shallow); err != nil {
|
||||
return err
|
||||
}
|
||||
case "nodedist":
|
||||
return wc.fetchNodeDist(ctx, pkg.name, pkg.conf)
|
||||
case "gittag":
|
||||
return wc.fetchGitTag(ctx, pkg.name, pkg.conf, shallow)
|
||||
case "gitea":
|
||||
return wc.fetchGitea(ctx, pkg.name, pkg.conf, shallow)
|
||||
case "chromedist":
|
||||
return fetchChromeDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "flutterdist":
|
||||
return fetchFlutterDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "golang":
|
||||
return fetchGolang(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "gpgdist":
|
||||
return fetchGPGDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "hashicorp":
|
||||
return fetchHashiCorp(ctx, wc.Client, wc.RawDir, pkg.name, pkg.conf)
|
||||
case "iterm2dist":
|
||||
return fetchITerm2Dist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "juliadist":
|
||||
return fetchJuliaDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "mariadbdist":
|
||||
return fetchMariaDBDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "servicemandist":
|
||||
if err := servicemandist.Fetch(ctx, wc.Client, wc.RawDir, pkg.name, wc.Auth, shallow); err != nil {
|
||||
return err
|
||||
}
|
||||
case "zigdist":
|
||||
return fetchZigDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
default:
|
||||
log.Printf(" %s: source %q not yet supported, skipping", pkg.name, pkg.conf.Source)
|
||||
return nil
|
||||
}
|
||||
|
||||
// For non-gittag sources with a git_url, also clone the repo to get
|
||||
// commit hashes. Git entries are classified from this data in
|
||||
// refreshPackage, not from the main raw cache.
|
||||
if pkg.conf.GitURL != "" && pkg.conf.Source != "gittag" {
|
||||
gitShallow := shallow
|
||||
if !wc.Shallow {
|
||||
gd, gdErr := rawcache.Open(filepath.Join(wc.RawDir, "_gittag", pkg.name))
|
||||
if gdErr == nil && !gd.Populated() {
|
||||
gitShallow = false
|
||||
}
|
||||
}
|
||||
if err := wc.fetchGitTagSupplementary(ctx, pkg.name, pkg.conf.GitURL, gitShallow); err != nil {
|
||||
log.Printf(" %s: supplementary gittag fetch: %v", pkg.name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// fetchGitTagSupplementary clones a git repo to get commit hashes for
|
||||
// packages that use a non-gittag source type (servicemandist, githubsource)
|
||||
// but also have a git_url for source installs.
|
||||
func (wc *WebiCache) fetchGitTagSupplementary(ctx context.Context, pkgName, gitURL string, shallow bool) error {
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, "_gittag", pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
repoDir := filepath.Join(wc.RawDir, "_repos")
|
||||
os.MkdirAll(repoDir, 0o755)
|
||||
|
||||
for batch, err := range gittag.Fetch(ctx, gitURL, repoDir) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range batch {
|
||||
tag := entry.Version
|
||||
if tag == "" {
|
||||
tag = "HEAD-" + entry.CommitHash
|
||||
}
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(tag, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchGitHub(ctx context.Context, pkgName string, conf *installerconf.Conf, shallow bool) error {
|
||||
owner, repo := conf.Owner, conf.Repo
|
||||
if owner == "" || repo == "" {
|
||||
return fmt.Errorf("missing owner or repo")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
tagPrefix := conf.TagPrefix
|
||||
for batch, err := range github.Fetch(ctx, wc.Client, owner, repo, wc.Auth) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("github %s/%s: %w", owner, repo, err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
if rel.Draft {
|
||||
continue
|
||||
}
|
||||
tag := rel.TagName
|
||||
if tagPrefix != "" && !strings.HasPrefix(tag, tagPrefix) {
|
||||
continue
|
||||
}
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(tag, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchNodeDist(ctx context.Context, pkgName string, conf *installerconf.Conf) error {
|
||||
baseURL := conf.BaseURL
|
||||
if baseURL == "" {
|
||||
return fmt.Errorf("missing url")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Fetch from primary URL. Tag with "official/" prefix so unofficial
|
||||
// entries for the same version don't overwrite.
|
||||
for batch, err := range nodedist.Fetch(ctx, wc.Client, baseURL) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range batch {
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge("official/"+entry.Version, data)
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch from unofficial URL if configured (e.g. Node.js unofficial builds
|
||||
// which add musl, riscv64, loong64 targets).
|
||||
if unofficialURL := conf.Extra["unofficial_url"]; unofficialURL != "" {
|
||||
for batch, err := range nodedist.Fetch(ctx, wc.Client, unofficialURL) {
|
||||
if err != nil {
|
||||
log.Printf("warning: %s unofficial fetch: %v", pkgName, err)
|
||||
break
|
||||
}
|
||||
for _, entry := range batch {
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge("unofficial/"+entry.Version, data)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchGitTag(ctx context.Context, pkgName string, conf *installerconf.Conf, shallow bool) error {
|
||||
gitURL := conf.BaseURL
|
||||
if gitURL == "" {
|
||||
return fmt.Errorf("missing url")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
repoDir := filepath.Join(wc.RawDir, "_repos")
|
||||
os.MkdirAll(repoDir, 0o755)
|
||||
|
||||
for batch, err := range gittag.Fetch(ctx, gitURL, repoDir) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range batch {
|
||||
tag := entry.Version
|
||||
if tag == "" {
|
||||
tag = "HEAD-" + entry.CommitHash
|
||||
}
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(tag, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchGitea(ctx context.Context, pkgName string, conf *installerconf.Conf, shallow bool) error {
|
||||
baseURL, owner, repo := conf.BaseURL, conf.Owner, conf.Repo
|
||||
if baseURL == "" || owner == "" || repo == "" {
|
||||
return fmt.Errorf("missing base_url, owner, or repo")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range gitea.Fetch(ctx, wc.Client, baseURL, owner, repo, nil) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, rel := range batch {
|
||||
if rel.Draft {
|
||||
continue
|
||||
}
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.TagName, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchChromeDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range chromedist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("chromedist: %w", err)
|
||||
}
|
||||
for _, ver := range batch {
|
||||
data, _ := json.Marshal(ver)
|
||||
d.Merge(ver.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchFlutterDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range flutterdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("flutterdist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
// Key by version+channel+os for uniqueness.
|
||||
key := rel.Version + "-" + rel.Channel + "-" + rel.OS
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(key, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchGolang(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range golang.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("golang: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchGPGDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range gpgdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("gpgdist: %w", err)
|
||||
}
|
||||
for _, entry := range batch {
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(entry.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchHashiCorp(ctx context.Context, client *http.Client, rawDir, pkgName string, conf *installerconf.Conf) error {
|
||||
product := conf.Repo
|
||||
if product == "" {
|
||||
product = pkgName
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for idx, err := range hashicorp.Fetch(ctx, client, product) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("hashicorp %s: %w", product, err)
|
||||
}
|
||||
for ver, vdata := range idx.Versions {
|
||||
data, _ := json.Marshal(vdata)
|
||||
d.Merge(ver, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchITerm2Dist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range iterm2dist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("iterm2dist: %w", err)
|
||||
}
|
||||
for _, entry := range batch {
|
||||
key := entry.Version
|
||||
if entry.Channel == "beta" {
|
||||
key += "-beta"
|
||||
}
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(key, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchJuliaDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range juliadist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("juliadist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchMariaDBDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range mariadbdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("mariadbdist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.ReleaseID, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchZigDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range zigdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("zigdist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -1,146 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestBootstrapCurlPipe verifies the /{pkg} route returns the curl-pipe bootstrap.
|
||||
func TestBootstrapCurlPipe(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/bat@stable")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body[:min(len(body), 200)])
|
||||
}
|
||||
|
||||
// Should contain the bootstrap env vars.
|
||||
if !strings.Contains(body, "WEBI_PKG=") {
|
||||
t.Error("missing WEBI_PKG= in bootstrap")
|
||||
}
|
||||
if !strings.Contains(body, "WEBI_HOST=") {
|
||||
t.Error("missing WEBI_HOST= in bootstrap")
|
||||
}
|
||||
if !strings.Contains(body, "WEBI_CHECKSUM=") {
|
||||
t.Error("missing WEBI_CHECKSUM= in bootstrap")
|
||||
}
|
||||
// Should NOT contain the full installer (install.sh content).
|
||||
// The bootstrap just downloads and runs webi.
|
||||
if strings.Contains(body, "pkg_install()") {
|
||||
t.Error("bootstrap should not contain pkg_install — that's the full installer")
|
||||
}
|
||||
|
||||
t.Logf("bootstrap size: %d bytes", len(body))
|
||||
}
|
||||
|
||||
// TestInstallerFull verifies /api/installers/{pkg}.sh returns the full installer.
|
||||
func TestInstallerFull(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
// Use a webi-style User-Agent so the server can detect platform.
|
||||
code, body := getWithUA(t, ts, "/api/installers/bat@stable.sh", "aarch64/unknown Darwin/24.2.0 libc")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body[:min(len(body), 500)])
|
||||
}
|
||||
|
||||
// Should contain resolved release info.
|
||||
if !strings.Contains(body, "WEBI_VERSION=") {
|
||||
t.Error("missing WEBI_VERSION= in installer")
|
||||
}
|
||||
if !strings.Contains(body, "WEBI_PKG_URL=") {
|
||||
t.Error("missing WEBI_PKG_URL= in installer")
|
||||
}
|
||||
if !strings.Contains(body, "PKG_NAME=") {
|
||||
t.Error("missing PKG_NAME= in installer")
|
||||
}
|
||||
|
||||
// Should contain the package's install.sh content (embedded).
|
||||
if !strings.Contains(body, "pkg_") {
|
||||
t.Error("installer should contain pkg_ functions from install.sh")
|
||||
}
|
||||
|
||||
t.Logf("installer size: %d bytes", len(body))
|
||||
}
|
||||
|
||||
// TestInstallerPowerShell verifies /api/installers/{pkg}.ps1 returns a PowerShell installer.
|
||||
func TestInstallerPowerShell(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "node"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := getWithUA(t, ts, "/api/installers/node@stable.ps1", "AMD64/unknown Windows/10.0.19045 msvc")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body[:min(len(body), 500)])
|
||||
}
|
||||
|
||||
if !strings.Contains(body, "$Env:WEBI_VERSION") {
|
||||
t.Error("missing $Env:WEBI_VERSION in PS1 installer")
|
||||
}
|
||||
if !strings.Contains(body, "$Env:WEBI_PKG_URL") {
|
||||
t.Error("missing $Env:WEBI_PKG_URL in PS1 installer")
|
||||
}
|
||||
if !strings.Contains(body, "$Env:PKG_NAME") {
|
||||
t.Error("missing $Env:PKG_NAME in PS1 installer")
|
||||
}
|
||||
|
||||
t.Logf("PS1 installer size: %d bytes", len(body))
|
||||
}
|
||||
|
||||
// TestInstallerSelfHosted verifies selfhosted packages get a script without resolution.
|
||||
func TestInstallerSelfHosted(t *testing.T) {
|
||||
_, ts := newTestServer(t)
|
||||
|
||||
// ssh-utils is selfhosted — has install.sh but no releases.conf.
|
||||
code, body := getWithUA(t, ts, "/api/installers/ssh-utils.sh", "aarch64/unknown Darwin/24.2.0 libc")
|
||||
if code == 404 {
|
||||
t.Skip("ssh-utils not available as installer")
|
||||
}
|
||||
if code != 200 {
|
||||
t.Skipf("status %d (selfhosted may not render without cache): %s", code, body[:min(len(body), 200)])
|
||||
}
|
||||
|
||||
t.Logf("selfhosted installer size: %d bytes", len(body))
|
||||
}
|
||||
|
||||
// TestBootstrapUnknownPackage verifies 404 for unknown packages.
|
||||
func TestBootstrapUnknownPackage(t *testing.T) {
|
||||
_, ts := newTestServer(t)
|
||||
|
||||
code, _ := get(t, ts, "/nonexistent-package-xyz")
|
||||
if code != 404 {
|
||||
t.Errorf("expected 404, got %d", code)
|
||||
}
|
||||
}
|
||||
|
||||
// getWithUA fetches a URL with a custom User-Agent header.
|
||||
func getWithUA(t *testing.T, ts *httptest.Server, path, ua string) (int, string) {
|
||||
t.Helper()
|
||||
req, err := http.NewRequest("GET", ts.URL+path, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("new request: %v", err)
|
||||
}
|
||||
req.Header.Set("User-Agent", ua)
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err != nil {
|
||||
t.Fatalf("GET %s: %v", path, err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return resp.StatusCode, string(body)
|
||||
}
|
||||
@@ -1,989 +0,0 @@
|
||||
// Command webid is the webi HTTP API server. It reads cached release
|
||||
// data from the filesystem and serves release metadata, installer
|
||||
// scripts, and bootstrap dispatches.
|
||||
//
|
||||
// It never fetches from upstream APIs — that's webicached's job.
|
||||
// This server is stateless and fast: load from cache, resolve, render.
|
||||
//
|
||||
// Usage:
|
||||
//
|
||||
// go run ./cmd/webid
|
||||
// go run ./cmd/webid -addr :3001 -cache ~/.cache/webi/legacy
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha1"
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"slices"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
"github.com/webinstall/webi-installers/internal/lexver"
|
||||
"github.com/webinstall/webi-installers/internal/render"
|
||||
"github.com/webinstall/webi-installers/internal/resolve"
|
||||
"github.com/webinstall/webi-installers/internal/resolver"
|
||||
middleware "github.com/therootcompany/golib/http/middleware/v2"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
"github.com/webinstall/webi-installers/internal/storage/fsstore"
|
||||
"github.com/webinstall/webi-installers/internal/uadetect"
|
||||
)
|
||||
|
||||
var (
|
||||
name = "webid"
|
||||
version = "0.0.0-dev"
|
||||
commit = "0000000"
|
||||
date = "0001-01-01"
|
||||
licenseYear = "2024"
|
||||
licenseOwner = "AJ ONeal"
|
||||
licenseType = "MPL-2.0"
|
||||
)
|
||||
|
||||
func printVersion(w io.Writer) {
|
||||
v := strings.TrimPrefix(version, "v")
|
||||
_, _ = fmt.Fprintf(w, "%s v%s %s (%s)\n", name, v, commit[:7], date)
|
||||
_, _ = fmt.Fprintf(w, "Copyright (C) %s %s\n", licenseYear, licenseOwner)
|
||||
_, _ = fmt.Fprintf(w, "Licensed under %s\n", licenseType)
|
||||
}
|
||||
|
||||
func main() {
|
||||
addr := flag.String("addr", ":3001", "listen address")
|
||||
cacheDir := flag.String("legacy", "~/.cache/webi/legacy", "legacy cache directory")
|
||||
installersDir := flag.String("installers", ".", "installers repo root (for install.sh/ps1)")
|
||||
|
||||
if len(os.Args) > 1 {
|
||||
switch os.Args[1] {
|
||||
case "-V", "-version", "--version", "version":
|
||||
printVersion(os.Stdout)
|
||||
os.Exit(0)
|
||||
case "help", "-help", "--help":
|
||||
printVersion(os.Stdout)
|
||||
fmt.Fprintln(os.Stdout, "")
|
||||
flag.CommandLine.SetOutput(os.Stdout)
|
||||
flag.Usage()
|
||||
os.Exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
flag.Parse()
|
||||
|
||||
cachePath := expandHome(*cacheDir)
|
||||
|
||||
fss, err := fsstore.New(cachePath)
|
||||
if err != nil {
|
||||
log.Fatalf("fsstore: %v", err)
|
||||
}
|
||||
var store storage.Store = fss
|
||||
|
||||
srv := &server{
|
||||
store: store,
|
||||
installersDir: *installersDir,
|
||||
packages: make(map[string]*packageCache),
|
||||
}
|
||||
|
||||
// Pre-load all cached packages.
|
||||
srv.loadAll()
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mmux := middleware.WithMux(mux, requestLogger)
|
||||
|
||||
// Legacy API routes (Node.js compat).
|
||||
mmux.HandleFunc("GET /api/releases/{rest...}", srv.handleReleasesAPI)
|
||||
|
||||
// New API routes (v1).
|
||||
mmux.HandleFunc("GET /v1/releases/{rest...}", srv.handleV1Releases)
|
||||
mmux.HandleFunc("GET /v1/resolve/{rest...}", srv.handleV1Resolve)
|
||||
|
||||
// Full installer script (package-install.tpl.sh + install.sh).
|
||||
mmux.HandleFunc("GET /api/installers/{rest...}", srv.handleInstaller)
|
||||
|
||||
// Debug endpoint.
|
||||
mmux.HandleFunc("GET /api/debug", srv.handleDebug)
|
||||
|
||||
// Health check (no logging — too noisy).
|
||||
mux.HandleFunc("GET /api/health", func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
fmt.Fprintln(w, "ok")
|
||||
})
|
||||
|
||||
// Bootstrap route: /{package} and /{package}@{version}
|
||||
// Detects UA and returns rendered installer script.
|
||||
mmux.HandleFunc("GET /{pkgSpec}", srv.handleBootstrap)
|
||||
|
||||
httpSrv := &http.Server{
|
||||
Addr: *addr,
|
||||
Handler: mux,
|
||||
ReadTimeout: 5 * time.Second,
|
||||
WriteTimeout: 30 * time.Second,
|
||||
IdleTimeout: 120 * time.Second,
|
||||
}
|
||||
|
||||
// Graceful shutdown.
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
|
||||
defer stop()
|
||||
|
||||
go func() {
|
||||
log.Printf("webid listening on %s", *addr)
|
||||
if err := httpSrv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
log.Fatalf("listen: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
<-ctx.Done()
|
||||
log.Println("shutting down...")
|
||||
|
||||
shutCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
httpSrv.Shutdown(shutCtx)
|
||||
}
|
||||
|
||||
// requestLogger is a middleware that logs each request with status and duration.
|
||||
func requestLogger(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
start := time.Now()
|
||||
rw := &statusWriter{ResponseWriter: w, code: http.StatusOK}
|
||||
next.ServeHTTP(rw, r)
|
||||
log.Printf("%s %s %d %s", r.Method, r.URL.Path, rw.code, time.Since(start))
|
||||
})
|
||||
}
|
||||
|
||||
// statusWriter wraps ResponseWriter to capture the HTTP status code.
|
||||
type statusWriter struct {
|
||||
http.ResponseWriter
|
||||
code int
|
||||
}
|
||||
|
||||
func (sw *statusWriter) WriteHeader(code int) {
|
||||
sw.code = code
|
||||
sw.ResponseWriter.WriteHeader(code)
|
||||
}
|
||||
|
||||
// server holds the shared state for all HTTP handlers.
|
||||
type server struct {
|
||||
store storage.Store
|
||||
installersDir string
|
||||
|
||||
mu sync.RWMutex
|
||||
packages map[string]*packageCache
|
||||
webiCksum string // cached sha1[:8] of webi.sh
|
||||
}
|
||||
|
||||
// packageCache holds a loaded package's assets and catalog.
|
||||
type packageCache struct {
|
||||
assets []storage.Asset
|
||||
dists []resolve.Dist
|
||||
catalog resolve.Catalog
|
||||
}
|
||||
|
||||
// loadAll pre-loads all packages from the store.
|
||||
func (s *server) loadAll() {
|
||||
ctx := context.Background()
|
||||
|
||||
pkgs, err := s.store.ListPackages(ctx)
|
||||
if err != nil {
|
||||
log.Printf("warn: list packages: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
count := 0
|
||||
for _, pkg := range pkgs {
|
||||
pd, err := s.store.Load(ctx, pkg)
|
||||
if err != nil {
|
||||
log.Printf("warn: load %s: %v", pkg, err)
|
||||
continue
|
||||
}
|
||||
if pd == nil || len(pd.Assets) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
pc := &packageCache{
|
||||
assets: pd.Assets,
|
||||
dists: assetsToDists(pd.Assets),
|
||||
}
|
||||
pc.catalog = resolve.Survey(pc.dists)
|
||||
|
||||
s.mu.Lock()
|
||||
s.packages[pkg] = pc
|
||||
s.mu.Unlock()
|
||||
count++
|
||||
}
|
||||
log.Printf("loaded %d packages from store", count)
|
||||
}
|
||||
|
||||
// getPackage returns the cached package data, or nil if not found.
|
||||
func (s *server) getPackage(pkg string) *packageCache {
|
||||
s.mu.RLock()
|
||||
defer s.mu.RUnlock()
|
||||
return s.packages[pkg]
|
||||
}
|
||||
|
||||
// assetsToDists converts storage.Asset slice to resolve.Dist slice.
|
||||
func assetsToDists(assets []storage.Asset) []resolve.Dist {
|
||||
dists := make([]resolve.Dist, len(assets))
|
||||
for i, a := range assets {
|
||||
dists[i] = resolve.Dist{
|
||||
Filename: a.Filename,
|
||||
Version: a.Version,
|
||||
LTS: a.LTS,
|
||||
Channel: a.Channel,
|
||||
Date: a.Date,
|
||||
OS: a.OS,
|
||||
Arch: a.Arch,
|
||||
Libc: a.Libc,
|
||||
Format: a.Format,
|
||||
Download: a.Download,
|
||||
Extra: a.Extra,
|
||||
GitTag: a.GitTag,
|
||||
GitCommitHash: a.GitCommitHash,
|
||||
Variants: a.Variants,
|
||||
}
|
||||
}
|
||||
return dists
|
||||
}
|
||||
|
||||
// handleReleasesAPI serves /api/releases/{package}@{version}.{format}
|
||||
func (s *server) handleReleasesAPI(w http.ResponseWriter, r *http.Request) {
|
||||
rest := r.PathValue("rest")
|
||||
|
||||
// Parse: {package}@{version}.{json|tab} or {package}.{json|tab}
|
||||
pkg, version, format, err := parseReleasePath(rest)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
pc := s.getPackage(pkg)
|
||||
if pc == nil {
|
||||
// Check if it's a selfhosted package.
|
||||
if s.isSelfHosted(pkg) {
|
||||
s.serveEmptyReleases(w, format)
|
||||
return
|
||||
}
|
||||
http.Error(w, fmt.Sprintf("package %q not found", pkg), http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
// Parse query parameters.
|
||||
q := r.URL.Query()
|
||||
osStr := q.Get("os")
|
||||
archStr := q.Get("arch")
|
||||
libcStr := q.Get("libc")
|
||||
ltsStr := q.Get("lts")
|
||||
channelStr := q.Get("channel")
|
||||
formatsStr := q.Get("formats")
|
||||
limitStr := q.Get("limit")
|
||||
|
||||
// Normalize wildcard "-" to empty (means "any").
|
||||
if osStr == "-" {
|
||||
osStr = ""
|
||||
}
|
||||
if archStr == "-" {
|
||||
archStr = ""
|
||||
}
|
||||
if libcStr == "-" {
|
||||
libcStr = ""
|
||||
}
|
||||
|
||||
// Map Node.js OS/arch names to our canonical names.
|
||||
osStr = normalizeQueryOS(osStr)
|
||||
archStr = normalizeQueryArch(archStr)
|
||||
|
||||
// Parse LTS.
|
||||
lts := ltsStr == "true" || ltsStr == "1"
|
||||
|
||||
// Handle channel selectors in the version field: @stable, @lts, @beta, etc.
|
||||
switch strings.ToLower(version) {
|
||||
case "stable", "latest":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "stable"
|
||||
}
|
||||
case "lts":
|
||||
version = ""
|
||||
lts = true
|
||||
case "beta", "pre", "preview":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "beta"
|
||||
}
|
||||
case "rc":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "rc"
|
||||
}
|
||||
case "alpha", "dev":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "alpha"
|
||||
}
|
||||
}
|
||||
|
||||
// Parse formats list.
|
||||
var formats []string
|
||||
if formatsStr != "" {
|
||||
formats = strings.Split(formatsStr, ",")
|
||||
}
|
||||
|
||||
// Parse limit.
|
||||
limit := 100
|
||||
if limitStr != "" {
|
||||
fmt.Sscanf(limitStr, "%d", &limit)
|
||||
}
|
||||
|
||||
// Filter matching releases, sort by specificity, then apply limit.
|
||||
filtered := filterDists(pc.dists, osStr, archStr, libcStr, channelStr, version, formats, lts)
|
||||
sortDistsDescending(filtered, osStr, archStr)
|
||||
if len(filtered) > limit {
|
||||
filtered = filtered[:limit]
|
||||
}
|
||||
|
||||
switch format {
|
||||
case "json":
|
||||
s.serveJSON(w, r, pc, filtered)
|
||||
case "tab":
|
||||
s.serveTab(w, r, filtered)
|
||||
default:
|
||||
http.Error(w, "unsupported format: "+format, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
// normalizeQueryOS maps Node.js OS names to our canonical names.
|
||||
func normalizeQueryOS(s string) string {
|
||||
switch strings.ToLower(s) {
|
||||
case "macos", "mac":
|
||||
return "darwin"
|
||||
case "win":
|
||||
return "windows"
|
||||
default:
|
||||
return s
|
||||
}
|
||||
}
|
||||
|
||||
// normalizeQueryArch maps Node.js arch names to our canonical names.
|
||||
func normalizeQueryArch(s string) string {
|
||||
switch strings.ToLower(s) {
|
||||
case "amd64":
|
||||
return string(buildmeta.ArchAMD64) // "x86_64"
|
||||
case "arm64":
|
||||
return string(buildmeta.ArchARM64) // "aarch64"
|
||||
case "armv7l":
|
||||
return string(buildmeta.ArchARMv7)
|
||||
case "armv6l":
|
||||
return string(buildmeta.ArchARMv6)
|
||||
case "x86", "i386", "i686":
|
||||
return string(buildmeta.ArchX86)
|
||||
default:
|
||||
return s
|
||||
}
|
||||
}
|
||||
|
||||
// parseReleasePath parses "{pkg}@{version}.{format}" or "{pkg}.{format}".
|
||||
func parseReleasePath(rest string) (pkg, version, format string, err error) {
|
||||
if strings.HasSuffix(rest, ".json") {
|
||||
format = "json"
|
||||
rest = strings.TrimSuffix(rest, ".json")
|
||||
} else if strings.HasSuffix(rest, ".tab") {
|
||||
format = "tab"
|
||||
rest = strings.TrimSuffix(rest, ".tab")
|
||||
} else {
|
||||
return "", "", "", fmt.Errorf("unsupported format (use .json or .tab)")
|
||||
}
|
||||
|
||||
if idx := strings.IndexByte(rest, '@'); idx >= 0 {
|
||||
pkg = rest[:idx]
|
||||
version = rest[idx+1:]
|
||||
} else {
|
||||
pkg = rest
|
||||
}
|
||||
|
||||
if pkg == "" {
|
||||
return "", "", "", fmt.Errorf("package name required")
|
||||
}
|
||||
|
||||
return pkg, version, format, nil
|
||||
}
|
||||
|
||||
// filterDists filters dists by query parameters, returning all matches
|
||||
// up to limit. This is for the API listing, not single-best resolution.
|
||||
func filterDists(dists []resolve.Dist, osStr, archStr, libcStr, channel, version string, formats []string, lts bool) []resolve.Dist {
|
||||
var result []resolve.Dist
|
||||
|
||||
archSet := make(map[string]bool)
|
||||
if archStr != "" {
|
||||
for _, a := range buildmeta.CompatArches(buildmeta.OS(osStr), buildmeta.Arch(archStr)) {
|
||||
archSet[string(a)] = true
|
||||
}
|
||||
if len(archSet) == 0 {
|
||||
archSet[archStr] = true
|
||||
}
|
||||
}
|
||||
|
||||
for _, d := range dists {
|
||||
if osStr != "" && d.OS != osStr && d.OS != "*" && d.OS != "ANYOS" && d.OS != "" &&
|
||||
!(d.OS == "posix_2017" && osStr != "windows") {
|
||||
continue
|
||||
}
|
||||
|
||||
if archStr != "" && !archSet[d.Arch] && d.Arch != "*" && d.Arch != "ANYARCH" && d.Arch != "" {
|
||||
continue
|
||||
}
|
||||
|
||||
if libcStr != "" && d.Libc != "none" && d.Libc != "" && d.Libc != libcStr {
|
||||
continue
|
||||
}
|
||||
|
||||
if lts && !d.LTS {
|
||||
continue
|
||||
}
|
||||
|
||||
if channel != "" && d.Channel != channel {
|
||||
continue
|
||||
}
|
||||
|
||||
if version != "" {
|
||||
// Match with or without "v" prefix:
|
||||
// query "0.25" should match version "v0.25.0".
|
||||
v := strings.TrimPrefix(d.Version, "v")
|
||||
vq := strings.TrimPrefix(version, "v")
|
||||
if !strings.HasPrefix(v, vq) {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
if len(formats) > 0 {
|
||||
matched := false
|
||||
for _, f := range formats {
|
||||
if strings.Contains(d.Format, f) {
|
||||
matched = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
result = append(result, d)
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// legacyRelease matches the Node.js JSON response format.
|
||||
// Production returns a bare JSON array of these objects.
|
||||
type legacyRelease struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"version"`
|
||||
GitTag string `json:"git_tag,omitempty"`
|
||||
GitCommitHash string `json:"git_commit_hash,omitempty"`
|
||||
LTS bool `json:"lts"`
|
||||
Channel string `json:"channel"`
|
||||
Date string `json:"date"`
|
||||
OS string `json:"os"`
|
||||
Arch string `json:"arch"`
|
||||
Ext string `json:"ext"`
|
||||
Download string `json:"download"`
|
||||
Libc string `json:"libc"`
|
||||
}
|
||||
|
||||
// legacyOS maps Go canonical OS names to Node.js legacy names.
|
||||
func legacyOS(s string) string {
|
||||
switch s {
|
||||
case "darwin":
|
||||
return "macos"
|
||||
case "":
|
||||
return "*"
|
||||
default:
|
||||
return s
|
||||
}
|
||||
}
|
||||
|
||||
// legacyArch maps Go canonical arch names to Node.js legacy names.
|
||||
func legacyArch(s string) string {
|
||||
switch s {
|
||||
case "x86_64":
|
||||
return "amd64"
|
||||
case "aarch64":
|
||||
return "arm64"
|
||||
case "armv7":
|
||||
return "armv7l"
|
||||
case "armv6":
|
||||
return "armv6l"
|
||||
case "armv5":
|
||||
return "arm"
|
||||
case "":
|
||||
return "*"
|
||||
default:
|
||||
return s
|
||||
}
|
||||
}
|
||||
|
||||
// legacyExt strips the leading "." from format strings.
|
||||
func legacyExt(s string) string {
|
||||
s = strings.TrimPrefix(s, ".")
|
||||
if s == "" {
|
||||
return "exe"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// legacyVersion strips the leading "v" from version strings.
|
||||
func legacyVersion(s string) string {
|
||||
return strings.TrimPrefix(s, "v")
|
||||
}
|
||||
|
||||
// legacyLibc returns "none" for empty libc values.
|
||||
func legacyLibc(s string) string {
|
||||
if s == "" {
|
||||
return "none"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func distsToLegacy(dists []resolve.Dist) []legacyRelease {
|
||||
releases := make([]legacyRelease, len(dists))
|
||||
for i, d := range dists {
|
||||
releases[i] = legacyRelease{
|
||||
Name: d.Filename,
|
||||
Version: legacyVersion(d.Version),
|
||||
GitTag: d.GitTag,
|
||||
GitCommitHash: d.GitCommitHash,
|
||||
LTS: d.LTS,
|
||||
Channel: d.Channel,
|
||||
Date: d.Date,
|
||||
OS: legacyOS(d.OS),
|
||||
Arch: legacyArch(d.Arch),
|
||||
Ext: legacyExt(d.Format),
|
||||
Download: d.Download,
|
||||
Libc: legacyLibc(d.Libc),
|
||||
}
|
||||
}
|
||||
return releases
|
||||
}
|
||||
|
||||
func (s *server) serveJSON(w http.ResponseWriter, r *http.Request, pc *packageCache, filtered []resolve.Dist) {
|
||||
// Production returns a bare JSON array, not wrapped in an object.
|
||||
releases := distsToLegacy(filtered)
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
|
||||
pretty := r.URL.Query().Get("pretty")
|
||||
if pretty == "true" || pretty == "1" {
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetIndent("", " ")
|
||||
enc.Encode(releases)
|
||||
} else {
|
||||
json.NewEncoder(w).Encode(releases)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *server) serveTab(w http.ResponseWriter, r *http.Request, filtered []resolve.Dist) {
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
|
||||
// Production only shows header row with ?pretty=true.
|
||||
pretty := r.URL.Query().Get("pretty")
|
||||
if pretty != "" && pretty != "false" {
|
||||
fmt.Fprintln(w, "VERSION\tLTS\tCHANNEL\tRELEASE_DATE\tOS\tARCH\tEXT\tHASH\tURL\t_\tLIBC")
|
||||
}
|
||||
|
||||
// Tab format matches Node.js production:
|
||||
// version \t lts \t channel \t date \t os \t arch \t ext \t hash \t download \t comment \t libc
|
||||
for _, d := range filtered {
|
||||
lts := "-"
|
||||
if d.LTS {
|
||||
lts = "lts"
|
||||
}
|
||||
channel := d.Channel
|
||||
if channel == "" {
|
||||
channel = "-"
|
||||
}
|
||||
date := d.Date
|
||||
if date == "" {
|
||||
date = "-"
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\t%s\t-\t%s\t\t%s\n",
|
||||
legacyVersion(d.Version),
|
||||
lts,
|
||||
channel,
|
||||
date,
|
||||
legacyOS(d.OS),
|
||||
legacyArch(d.Arch),
|
||||
legacyExt(d.Format),
|
||||
d.Download,
|
||||
legacyLibc(d.Libc),
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// sortDistsDescending sorts dists newest-first by version.
|
||||
func sortDistsDescending(dists []resolve.Dist, queryOS, queryArch string) {
|
||||
slices.SortStableFunc(dists, func(a, b resolve.Dist) int {
|
||||
va := lexver.Parse(strings.TrimPrefix(a.Version, "v"))
|
||||
vb := lexver.Parse(strings.TrimPrefix(b.Version, "v"))
|
||||
if cmp := lexver.Compare(vb, va); cmp != 0 {
|
||||
return cmp
|
||||
}
|
||||
if cmp := osSpecificity(a.OS, queryOS) - osSpecificity(b.OS, queryOS); cmp != 0 {
|
||||
return cmp
|
||||
}
|
||||
if cmp := archSpecificity(a.Arch, queryArch) - archSpecificity(b.Arch, queryArch); cmp != 0 {
|
||||
return cmp
|
||||
}
|
||||
return libcRank(a.Libc) - libcRank(b.Libc)
|
||||
})
|
||||
}
|
||||
|
||||
func osSpecificity(distOS, queryOS string) int {
|
||||
switch {
|
||||
case distOS == queryOS:
|
||||
return 0
|
||||
case distOS == "posix_2017":
|
||||
return 1
|
||||
default:
|
||||
return 2
|
||||
}
|
||||
}
|
||||
|
||||
func archSpecificity(distArch, queryArch string) int {
|
||||
switch {
|
||||
case distArch == queryArch:
|
||||
return 0
|
||||
case distArch == "" || distArch == "*":
|
||||
return 2
|
||||
default:
|
||||
return 1
|
||||
}
|
||||
}
|
||||
|
||||
func libcRank(libc string) int {
|
||||
switch libc {
|
||||
case "none", "":
|
||||
return 0
|
||||
default:
|
||||
return 1
|
||||
}
|
||||
}
|
||||
|
||||
// serveEmptyReleases returns an empty release list for selfhosted packages.
|
||||
func (s *server) serveEmptyReleases(w http.ResponseWriter, format string) {
|
||||
switch format {
|
||||
case "json":
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
// Production returns an empty array.
|
||||
json.NewEncoder(w).Encode([]legacyRelease{})
|
||||
case "tab":
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
}
|
||||
}
|
||||
|
||||
// isSelfHosted checks if a package has install.sh but no releases.conf.
|
||||
func (s *server) isSelfHosted(pkg string) bool {
|
||||
installPath := filepath.Join(s.installersDir, pkg, "install.sh")
|
||||
if _, err := os.Stat(installPath); err != nil {
|
||||
return false
|
||||
}
|
||||
confPath := filepath.Join(s.installersDir, pkg, "releases.conf")
|
||||
if _, err := os.Stat(confPath); err == nil {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// handleDebug returns UA detection info for the requesting client.
|
||||
func (s *server) handleDebug(w http.ResponseWriter, r *http.Request) {
|
||||
result := uadetect.FromRequest(r)
|
||||
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
json.NewEncoder(w).Encode(map[string]string{
|
||||
"user_agent": r.Header.Get("User-Agent"),
|
||||
"os": string(result.OS),
|
||||
"arch": string(result.Arch),
|
||||
"libc": string(result.Libc),
|
||||
})
|
||||
}
|
||||
|
||||
// handleBootstrap serves /{package} and /{package}@{version}.
|
||||
// This is the curl-pipe bootstrap: a minimal script that sets
|
||||
// WEBI_PKG/WEBI_HOST/WEBI_CHECKSUM and downloads+runs webi.
|
||||
func (s *server) handleBootstrap(w http.ResponseWriter, r *http.Request) {
|
||||
pkgSpec := r.PathValue("pkgSpec")
|
||||
|
||||
// Parse package@version.
|
||||
pkg, tag := pkgSpec, ""
|
||||
if idx := strings.IndexByte(pkgSpec, '@'); idx >= 0 {
|
||||
pkg = pkgSpec[:idx]
|
||||
tag = pkgSpec[idx+1:]
|
||||
}
|
||||
|
||||
if pkg == "" {
|
||||
http.Error(w, "package name required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
// Verify package exists.
|
||||
if s.getPackage(pkg) == nil && !s.isSelfHosted(pkg) {
|
||||
http.Error(w, fmt.Sprintf("package %q not found", pkg), http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
baseURL := baseURLFromRequest(r)
|
||||
webiPkg := pkg
|
||||
if tag != "" {
|
||||
webiPkg = pkg + "@" + tag
|
||||
}
|
||||
|
||||
// Read and inject the curl-pipe bootstrap template.
|
||||
tplPath := filepath.Join(s.installersDir, "_webi", "curl-pipe-bootstrap.tpl.sh")
|
||||
tpl, err := os.ReadFile(tplPath)
|
||||
if err != nil {
|
||||
log.Printf("bootstrap: read template: %v", err)
|
||||
http.Error(w, "bootstrap template not found", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
script := string(tpl)
|
||||
script = render.InjectVar(script, "WEBI_PKG", webiPkg)
|
||||
script = render.InjectVar(script, "WEBI_HOST", baseURL)
|
||||
script = render.InjectVar(script, "WEBI_CHECKSUM", s.webiChecksum())
|
||||
|
||||
// text/html so browsers see the meta redirect to cheat sheet.
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
fmt.Fprint(w, script)
|
||||
}
|
||||
|
||||
// handleInstaller serves /api/installers/{pkg}@{version}.sh
|
||||
// This is the full installer script with release resolution and
|
||||
// embedded install.sh.
|
||||
func (s *server) handleInstaller(w http.ResponseWriter, r *http.Request) {
|
||||
rest := r.PathValue("rest")
|
||||
|
||||
// Parse: {pkg}@{version}.sh or {pkg}.sh
|
||||
ext := ""
|
||||
if strings.HasSuffix(rest, ".sh") {
|
||||
ext = "sh"
|
||||
rest = strings.TrimSuffix(rest, ".sh")
|
||||
} else if strings.HasSuffix(rest, ".ps1") {
|
||||
ext = "ps1"
|
||||
rest = strings.TrimSuffix(rest, ".ps1")
|
||||
} else {
|
||||
http.Error(w, "unsupported format (use .sh or .ps1)", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
pkg, tag := rest, ""
|
||||
if idx := strings.IndexByte(rest, '@'); idx >= 0 {
|
||||
pkg = rest[:idx]
|
||||
tag = rest[idx+1:]
|
||||
}
|
||||
|
||||
if pkg == "" {
|
||||
http.Error(w, "package name required", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
// Detect platform from User-Agent.
|
||||
ua := uadetect.FromRequest(r)
|
||||
if ua.OS == "" {
|
||||
http.Error(w, "could not detect OS from User-Agent", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
isSelfHosted := s.isSelfHosted(pkg)
|
||||
pc := s.getPackage(pkg)
|
||||
|
||||
if pc == nil && !isSelfHosted {
|
||||
http.Error(w, fmt.Sprintf("package %q not found", pkg), http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
baseURL := baseURLFromRequest(r)
|
||||
|
||||
p := render.Params{
|
||||
Host: baseURL,
|
||||
PkgName: pkg,
|
||||
Tag: tag,
|
||||
OS: string(ua.OS),
|
||||
Arch: string(ua.Arch),
|
||||
Libc: string(ua.Libc),
|
||||
}
|
||||
|
||||
// Resolve the best release (if not selfhosted).
|
||||
if pc != nil {
|
||||
req := resolver.Request{
|
||||
OS: string(ua.OS),
|
||||
Arch: string(ua.Arch),
|
||||
Libc: string(ua.Libc),
|
||||
}
|
||||
|
||||
switch strings.ToLower(tag) {
|
||||
case "stable", "latest", "":
|
||||
// Default.
|
||||
case "lts":
|
||||
req.LTS = true
|
||||
case "beta", "pre", "preview":
|
||||
req.Channel = "beta"
|
||||
case "rc":
|
||||
req.Channel = "rc"
|
||||
case "alpha", "dev":
|
||||
req.Channel = "alpha"
|
||||
default:
|
||||
req.Version = tag
|
||||
}
|
||||
|
||||
res, err := resolver.Resolve(pc.assets, req)
|
||||
if err != nil {
|
||||
p.Version = "0.0.0"
|
||||
p.Channel = "error"
|
||||
p.Ext = "err"
|
||||
p.PkgURL = "https://example.com/doesntexist.ext"
|
||||
p.PkgFile = "doesntexist.ext"
|
||||
p.CSV = buildCSV(p)
|
||||
} else {
|
||||
v := strings.TrimPrefix(res.Version, "v")
|
||||
parts := splitVersion(v)
|
||||
p.Version = v
|
||||
p.Major = parts[0]
|
||||
p.Minor = parts[1]
|
||||
p.Patch = parts[2]
|
||||
p.Build = parts[3]
|
||||
if res.Asset.GitTag != "" {
|
||||
p.GitTag = res.Asset.GitTag
|
||||
} else {
|
||||
p.GitTag = "v" + v
|
||||
}
|
||||
p.GitBranch = p.GitTag
|
||||
p.GitCommitHash = res.Asset.GitCommitHash
|
||||
p.LTS = fmt.Sprintf("%v", res.Asset.LTS)
|
||||
p.Channel = res.Asset.Channel
|
||||
if p.Channel == "" {
|
||||
p.Channel = "stable"
|
||||
}
|
||||
p.Ext = strings.TrimPrefix(res.Asset.Format, ".")
|
||||
if p.Ext == "" {
|
||||
p.Ext = "exe"
|
||||
}
|
||||
p.PkgURL = res.Asset.Download
|
||||
p.PkgFile = res.Asset.Filename
|
||||
p.CSV = buildCSV(p)
|
||||
}
|
||||
|
||||
p.PkgStable = pc.catalog.Stable
|
||||
p.PkgLatest = pc.catalog.Latest
|
||||
p.PkgOSes = strings.Join(pc.catalog.OSes, " ")
|
||||
p.PkgArches = strings.Join(pc.catalog.Arches, " ")
|
||||
p.PkgLibcs = strings.Join(pc.catalog.Libcs, " ")
|
||||
p.PkgFormats = strings.Join(pc.catalog.Formats, " ")
|
||||
}
|
||||
|
||||
p.ReleasesURL = fmt.Sprintf("%s/api/releases/%s@%s.tab?os=%s&arch=%s&libc=%s&formats=tar&pretty=true",
|
||||
baseURL, pkg, tag, p.OS, p.Arch, p.Libc)
|
||||
|
||||
var script string
|
||||
var renderErr error
|
||||
if ext == "ps1" {
|
||||
tplPath := filepath.Join(s.installersDir, "_webi", "package-install.tpl.ps1")
|
||||
script, renderErr = render.PowerShell(tplPath, s.installersDir, pkg, p)
|
||||
} else {
|
||||
tplPath := filepath.Join(s.installersDir, "_webi", "package-install.tpl.sh")
|
||||
script, renderErr = render.Bash(tplPath, s.installersDir, pkg, p)
|
||||
}
|
||||
if renderErr != nil {
|
||||
log.Printf("render %s: %v", pkg, renderErr)
|
||||
http.Error(w, fmt.Sprintf("failed to render installer for %q: %v", pkg, renderErr), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
fmt.Fprint(w, script)
|
||||
}
|
||||
|
||||
// baseURLFromRequest builds the base URL from the request.
|
||||
func baseURLFromRequest(r *http.Request) string {
|
||||
if r.TLS != nil || strings.Contains(r.Host, "webinstall") || strings.Contains(r.Host, "webi.") {
|
||||
return "https://" + r.Host
|
||||
}
|
||||
return "http://" + r.Host
|
||||
}
|
||||
|
||||
// webiChecksum returns the checksum of the webi.sh bootstrap script.
|
||||
func (s *server) webiChecksum() string {
|
||||
s.mu.RLock()
|
||||
cksum := s.webiCksum
|
||||
s.mu.RUnlock()
|
||||
if cksum != "" {
|
||||
return cksum
|
||||
}
|
||||
|
||||
// Calculate checksum from webi.sh file.
|
||||
webiPath := filepath.Join(s.installersDir, "webi", "webi.sh")
|
||||
data, err := os.ReadFile(webiPath)
|
||||
if err != nil {
|
||||
return "00000000"
|
||||
}
|
||||
|
||||
h := sha1.New()
|
||||
h.Write(data)
|
||||
cksum = fmt.Sprintf("%x", h.Sum(nil))[:8]
|
||||
|
||||
s.mu.Lock()
|
||||
s.webiCksum = cksum
|
||||
s.mu.Unlock()
|
||||
return cksum
|
||||
}
|
||||
|
||||
// buildCSV creates the WEBI_CSV line in the Node.js format.
|
||||
func buildCSV(p render.Params) string {
|
||||
return strings.Join([]string{
|
||||
p.Version,
|
||||
p.LTS,
|
||||
p.Channel,
|
||||
"", // date
|
||||
p.OS,
|
||||
p.Arch,
|
||||
p.Ext,
|
||||
"-",
|
||||
p.PkgURL,
|
||||
p.PkgFile,
|
||||
"",
|
||||
}, ",")
|
||||
}
|
||||
|
||||
// splitVersion splits a version string into [major, minor, patch, build].
|
||||
func splitVersion(v string) [4]string {
|
||||
// Strip pre-release suffix for splitting.
|
||||
base := v
|
||||
build := ""
|
||||
if idx := strings.IndexByte(v, '-'); idx >= 0 {
|
||||
base = v[:idx]
|
||||
build = v[idx+1:]
|
||||
}
|
||||
|
||||
parts := strings.SplitN(base, ".", 4)
|
||||
var result [4]string
|
||||
for i := 0; i < len(parts) && i < 3; i++ {
|
||||
result[i] = parts[i]
|
||||
}
|
||||
result[3] = build
|
||||
return result
|
||||
}
|
||||
|
||||
func expandHome(path string) string {
|
||||
if !strings.HasPrefix(path, "~/") {
|
||||
return path
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
return path
|
||||
}
|
||||
return filepath.Join(home, path[2:])
|
||||
}
|
||||
@@ -1,298 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/resolve"
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
"github.com/webinstall/webi-installers/internal/storage/fsstore"
|
||||
)
|
||||
|
||||
// newTestServer creates a server backed by the real _cache directory
|
||||
// and returns an httptest.Server with proper routing (so PathValue works).
|
||||
func newTestServer(t *testing.T) (*server, *httptest.Server) {
|
||||
t.Helper()
|
||||
|
||||
cacheDir := filepath.Join("..", "..", "_cache")
|
||||
if _, err := os.Stat(cacheDir); err != nil {
|
||||
t.Skipf("no cache dir at %s", cacheDir)
|
||||
}
|
||||
|
||||
store, err := fsstore.New(cacheDir)
|
||||
if err != nil {
|
||||
t.Fatalf("fsstore: %v", err)
|
||||
}
|
||||
|
||||
srv := &server{
|
||||
store: store,
|
||||
installersDir: filepath.Join("..", ".."),
|
||||
packages: make(map[string]*packageCache),
|
||||
}
|
||||
|
||||
// Load packages.
|
||||
monthDir := time.Now().Format("2006-01")
|
||||
dir := filepath.Join(store.Root(), monthDir)
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Fatalf("readdir: %v", err)
|
||||
}
|
||||
for _, e := range entries {
|
||||
if !strings.HasSuffix(e.Name(), ".json") {
|
||||
continue
|
||||
}
|
||||
pkg := strings.TrimSuffix(e.Name(), ".json")
|
||||
pd, err := store.Load(context.Background(), pkg)
|
||||
if err != nil || pd == nil || len(pd.Assets) == 0 {
|
||||
continue
|
||||
}
|
||||
pc := &packageCache{
|
||||
assets: pd.Assets,
|
||||
dists: assetsToDists(pd.Assets),
|
||||
}
|
||||
pc.catalog = resolve.Survey(pc.dists)
|
||||
srv.packages[pkg] = pc
|
||||
}
|
||||
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /api/releases/{rest...}", srv.handleReleasesAPI)
|
||||
mux.HandleFunc("GET /v1/releases/{rest...}", srv.handleV1Releases)
|
||||
mux.HandleFunc("GET /v1/resolve/{rest...}", srv.handleV1Resolve)
|
||||
mux.HandleFunc("GET /api/installers/{rest...}", srv.handleInstaller)
|
||||
mux.HandleFunc("GET /api/debug", srv.handleDebug)
|
||||
mux.HandleFunc("GET /{pkgSpec}", srv.handleBootstrap)
|
||||
|
||||
ts := httptest.NewServer(mux)
|
||||
t.Cleanup(ts.Close)
|
||||
|
||||
return srv, ts
|
||||
}
|
||||
|
||||
// get fetches a URL from the test server and returns the body.
|
||||
func get(t *testing.T, ts *httptest.Server, path string) (int, string) {
|
||||
t.Helper()
|
||||
resp, err := http.Get(ts.URL + path)
|
||||
if err != nil {
|
||||
t.Fatalf("GET %s: %v", path, err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
return resp.StatusCode, string(body)
|
||||
}
|
||||
|
||||
// TestLegacyJSONFormat verifies our JSON output matches the production format.
|
||||
func TestLegacyJSONFormat(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
packages := []string{"bat", "node", "go", "jq"}
|
||||
for _, pkg := range packages {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/api/releases/"+pkg+".json?limit=5")
|
||||
if code != http.StatusOK {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
body = strings.TrimSpace(body)
|
||||
|
||||
// Must be a JSON array, not an object.
|
||||
if !strings.HasPrefix(body, "[") {
|
||||
t.Fatalf("expected JSON array, got: %.100s", body)
|
||||
}
|
||||
|
||||
var releases []legacyRelease
|
||||
if err := json.Unmarshal([]byte(body), &releases); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
if len(releases) == 0 {
|
||||
t.Fatal("no releases returned")
|
||||
}
|
||||
|
||||
// Check field format conventions.
|
||||
for i, r := range releases {
|
||||
if strings.HasPrefix(r.Version, "v") {
|
||||
t.Errorf("release[%d]: version %q should not have v prefix", i, r.Version)
|
||||
}
|
||||
if strings.HasPrefix(r.Ext, ".") {
|
||||
t.Errorf("release[%d]: ext %q should not have . prefix", i, r.Ext)
|
||||
}
|
||||
if r.OS == "darwin" {
|
||||
t.Errorf("release[%d]: os should be 'macos' not 'darwin'", i)
|
||||
}
|
||||
if r.Arch == "x86_64" {
|
||||
t.Errorf("release[%d]: arch should be 'amd64' not 'x86_64'", i)
|
||||
}
|
||||
if r.Arch == "aarch64" {
|
||||
t.Errorf("release[%d]: arch should be 'arm64' not 'aarch64'", i)
|
||||
}
|
||||
if r.Libc == "" {
|
||||
t.Errorf("release[%d]: libc should be 'none' not empty", i)
|
||||
}
|
||||
if r.Download == "" {
|
||||
t.Errorf("release[%d]: download URL is empty", i)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestLegacyTabFormat verifies our .tab output uses real TSV.
|
||||
func TestLegacyTabFormat(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
packages := []string{"bat", "node", "go"}
|
||||
for _, pkg := range packages {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/api/releases/"+pkg+".tab?limit=3")
|
||||
if code != http.StatusOK {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
lines := strings.Split(strings.TrimSpace(body), "\n")
|
||||
if len(lines) == 0 {
|
||||
t.Fatal("no lines returned")
|
||||
}
|
||||
|
||||
for i, line := range lines {
|
||||
fields := strings.Split(line, "\t")
|
||||
// Expect 11 tab-separated fields:
|
||||
// version, lts, channel, date, os, arch, ext, hash, download, (empty), libc
|
||||
if len(fields) != 11 {
|
||||
t.Errorf("line[%d]: expected 11 tab fields, got %d: %q", i, len(fields), line)
|
||||
continue
|
||||
}
|
||||
|
||||
version := fields[0]
|
||||
lts := fields[1]
|
||||
ext := fields[6]
|
||||
|
||||
if strings.HasPrefix(version, "v") {
|
||||
t.Errorf("line[%d]: version %q should not have v prefix", i, version)
|
||||
}
|
||||
if lts != "-" && lts != "lts" {
|
||||
t.Errorf("line[%d]: lts should be '-' or 'lts', got %q", i, lts)
|
||||
}
|
||||
if strings.HasPrefix(ext, ".") {
|
||||
t.Errorf("line[%d]: ext %q should not have . prefix", i, ext)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestLegacyJSONAgainstProduction compares our output against live production.
|
||||
// Run with: WEBI_TEST_PROD=1 go test -run TestLegacyJSONAgainstProduction
|
||||
func TestLegacyJSONAgainstProduction(t *testing.T) {
|
||||
if os.Getenv("WEBI_TEST_PROD") == "" {
|
||||
t.Skip("set WEBI_TEST_PROD=1 to compare against production")
|
||||
}
|
||||
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
packages := []string{"bat", "node", "go", "jq", "rg"}
|
||||
for _, pkg := range packages {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
// Fetch from production.
|
||||
prodURL := fmt.Sprintf("https://webinstall.dev/api/releases/%s.json?limit=3", pkg)
|
||||
prodResp, err := http.Get(prodURL)
|
||||
if err != nil {
|
||||
t.Fatalf("fetch production: %v", err)
|
||||
}
|
||||
defer prodResp.Body.Close()
|
||||
prodBody, _ := io.ReadAll(prodResp.Body)
|
||||
|
||||
var prodReleases []legacyRelease
|
||||
if err := json.Unmarshal(prodBody, &prodReleases); err != nil {
|
||||
t.Fatalf("decode production: %v\nbody: %.500s", err, string(prodBody))
|
||||
}
|
||||
|
||||
// Fetch from local.
|
||||
_, localBody := get(t, ts, "/api/releases/"+pkg+".json?limit=3")
|
||||
|
||||
var localReleases []legacyRelease
|
||||
if err := json.Unmarshal([]byte(localBody), &localReleases); err != nil {
|
||||
t.Fatalf("decode local: %v", err)
|
||||
}
|
||||
|
||||
if len(prodReleases) == 0 || len(localReleases) == 0 {
|
||||
t.Skip("empty releases")
|
||||
}
|
||||
|
||||
// Compare the first release's format.
|
||||
prod := prodReleases[0]
|
||||
local := localReleases[0]
|
||||
|
||||
if strings.HasPrefix(local.Version, "v") != strings.HasPrefix(prod.Version, "v") {
|
||||
t.Errorf("version prefix mismatch: prod=%q local=%q", prod.Version, local.Version)
|
||||
}
|
||||
if strings.HasPrefix(local.Ext, ".") != strings.HasPrefix(prod.Ext, ".") {
|
||||
t.Errorf("ext prefix mismatch: prod=%q local=%q", prod.Ext, local.Ext)
|
||||
}
|
||||
if prod.OS == "macos" && local.OS == "darwin" {
|
||||
t.Error("OS: prod uses 'macos', local uses 'darwin'")
|
||||
}
|
||||
if prod.Arch == "amd64" && local.Arch == "x86_64" {
|
||||
t.Error("Arch: prod uses 'amd64', local uses 'x86_64'")
|
||||
}
|
||||
if prod.Arch == "arm64" && local.Arch == "aarch64" {
|
||||
t.Error("Arch: prod uses 'arm64', local uses 'aarch64'")
|
||||
}
|
||||
|
||||
t.Logf("prod[0]: version=%q os=%q arch=%q ext=%q libc=%q",
|
||||
prod.Version, prod.OS, prod.Arch, prod.Ext, prod.Libc)
|
||||
t.Logf("local[0]: version=%q os=%q arch=%q ext=%q libc=%q",
|
||||
local.Version, local.OS, local.Arch, local.Ext, local.Libc)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestSortOrder verifies releases come back newest-first.
|
||||
func TestSortOrder(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
_, body := get(t, ts, "/api/releases/"+pkg+".json?limit=20")
|
||||
|
||||
var releases []legacyRelease
|
||||
if err := json.Unmarshal([]byte(body), &releases); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
|
||||
if len(releases) < 2 {
|
||||
t.Skip("need at least 2 releases")
|
||||
}
|
||||
|
||||
// First release should be newest (or equal) version.
|
||||
first := releases[0].Date
|
||||
last := releases[len(releases)-1].Date
|
||||
if first < last {
|
||||
t.Errorf("not newest-first: first=%q last=%q", first, last)
|
||||
}
|
||||
}
|
||||
|
||||
// Ensure imports are used.
|
||||
var _ = storage.Asset{}
|
||||
@@ -1,459 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/csv"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/jszwec/csvutil"
|
||||
|
||||
"github.com/webinstall/webi-installers/internal/buildmeta"
|
||||
"github.com/webinstall/webi-installers/internal/lexver"
|
||||
"github.com/webinstall/webi-installers/internal/resolver"
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
)
|
||||
|
||||
// v1Release is a single release in the new API TSV format.
|
||||
// Field order matters for csvutil — it determines column order.
|
||||
// Fields are designed to be easy to consume with cut/grep/sort.
|
||||
type v1Release struct {
|
||||
Version string `csv:"version"`
|
||||
Channel string `csv:"channel"`
|
||||
LTS string `csv:"lts"`
|
||||
Date string `csv:"date"`
|
||||
OS string `csv:"os"`
|
||||
Arch string `csv:"arch"`
|
||||
Libc string `csv:"libc"`
|
||||
Format string `csv:"format"`
|
||||
Variants string `csv:"variants"` // space-separated
|
||||
Download string `csv:"download"`
|
||||
Filename string `csv:"filename"`
|
||||
}
|
||||
|
||||
// v1ResolveResult is the response for /v1/resolve/{pkg}.
|
||||
type v1ResolveResult struct {
|
||||
Version string `csv:"version" json:"version"`
|
||||
Channel string `csv:"channel" json:"channel"`
|
||||
LTS string `csv:"lts" json:"lts"`
|
||||
Date string `csv:"date" json:"date"`
|
||||
OS string `csv:"os" json:"os"`
|
||||
Arch string `csv:"arch" json:"arch"`
|
||||
Libc string `csv:"libc" json:"libc"`
|
||||
Format string `csv:"format" json:"format"`
|
||||
Variants string `csv:"variants" json:"variants"`
|
||||
Download string `csv:"download" json:"download"`
|
||||
Filename string `csv:"filename" json:"filename"`
|
||||
Triplet string `csv:"triplet" json:"triplet"`
|
||||
}
|
||||
|
||||
// handleV1Releases serves /v1/releases/{pkg}.tsv (or .json)
|
||||
// with Go-native naming and TSV-first format.
|
||||
//
|
||||
// Query params:
|
||||
//
|
||||
// os — filter by OS (darwin, linux, windows)
|
||||
// arch — filter by arch (aarch64, x86_64, armv7l)
|
||||
// libc — filter by libc (gnu, musl, msvc)
|
||||
// channel — release channel (stable, beta, rc, alpha)
|
||||
// version — version prefix filter (e.g. "1.20")
|
||||
// lts — if "true", only LTS releases
|
||||
// format — filter by format (e.g. "tar.gz")
|
||||
// variant — filter by variant (e.g. "rocm")
|
||||
// limit — max results (default 1000)
|
||||
func (s *server) handleV1Releases(w http.ResponseWriter, r *http.Request) {
|
||||
rest := r.PathValue("rest")
|
||||
|
||||
pkg, version, format, err := parseReleasePath(rest)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
pc := s.getPackage(pkg)
|
||||
if pc == nil {
|
||||
if s.isSelfHosted(pkg) {
|
||||
s.v1ServeEmpty(w, format)
|
||||
return
|
||||
}
|
||||
http.Error(w, fmt.Sprintf("package %q not found", pkg), http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
q := r.URL.Query()
|
||||
osStr := q.Get("os")
|
||||
archStr := q.Get("arch")
|
||||
libcStr := q.Get("libc")
|
||||
channelStr := q.Get("channel")
|
||||
ltsStr := q.Get("lts")
|
||||
formatFilter := q.Get("format")
|
||||
variantStr := q.Get("variant")
|
||||
limitStr := q.Get("limit")
|
||||
|
||||
// Use version from URL path or query.
|
||||
if version == "" {
|
||||
version = q.Get("version")
|
||||
}
|
||||
|
||||
// Handle channel selectors in version field.
|
||||
switch strings.ToLower(version) {
|
||||
case "stable", "latest":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "stable"
|
||||
}
|
||||
case "lts":
|
||||
version = ""
|
||||
ltsStr = "true"
|
||||
case "beta", "pre", "preview":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "beta"
|
||||
}
|
||||
case "rc":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "rc"
|
||||
}
|
||||
case "alpha", "dev":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "alpha"
|
||||
}
|
||||
}
|
||||
|
||||
lts := ltsStr == "true" || ltsStr == "1"
|
||||
|
||||
limit := 1000
|
||||
if limitStr != "" {
|
||||
fmt.Sscanf(limitStr, "%d", &limit)
|
||||
}
|
||||
|
||||
// Filter assets directly (not via resolve.Dist).
|
||||
filtered := filterAssets(pc.assets, osStr, archStr, libcStr, channelStr, version, formatFilter, variantStr, lts, limit)
|
||||
|
||||
// Sort newest-first.
|
||||
sortAssetsDescending(filtered)
|
||||
|
||||
switch format {
|
||||
case "json":
|
||||
s.v1ServeJSON(w, filtered)
|
||||
case "tab":
|
||||
s.v1ServeTSV(w, filtered)
|
||||
default:
|
||||
http.Error(w, "unsupported format: "+format+" (use .json or .tab)", http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
// handleV1Resolve serves /v1/resolve/{pkg}.tsv (or .json)
|
||||
// It resolves the single best asset for a given platform.
|
||||
//
|
||||
// Query params:
|
||||
//
|
||||
// os — target OS (required)
|
||||
// arch — target arch (required)
|
||||
// libc — target libc
|
||||
// version — version prefix
|
||||
// channel — release channel
|
||||
// lts — if "true", only LTS
|
||||
// format — preferred formats (comma-separated, in preference order)
|
||||
// variant — preferred variant
|
||||
func (s *server) handleV1Resolve(w http.ResponseWriter, r *http.Request) {
|
||||
rest := r.PathValue("rest")
|
||||
|
||||
pkg, version, format, err := parseReleasePath(rest)
|
||||
if err != nil {
|
||||
http.Error(w, err.Error(), http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
pc := s.getPackage(pkg)
|
||||
if pc == nil {
|
||||
http.Error(w, fmt.Sprintf("package %q not found", pkg), http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
q := r.URL.Query()
|
||||
osStr := q.Get("os")
|
||||
archStr := q.Get("arch")
|
||||
libcStr := q.Get("libc")
|
||||
channelStr := q.Get("channel")
|
||||
ltsStr := q.Get("lts")
|
||||
formatsStr := q.Get("format")
|
||||
variantStr := q.Get("variant")
|
||||
|
||||
if version == "" {
|
||||
version = q.Get("version")
|
||||
}
|
||||
|
||||
// Handle channel selectors in version field.
|
||||
switch strings.ToLower(version) {
|
||||
case "stable", "latest":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "stable"
|
||||
}
|
||||
case "lts":
|
||||
version = ""
|
||||
ltsStr = "true"
|
||||
case "beta", "pre", "preview":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "beta"
|
||||
}
|
||||
case "rc":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "rc"
|
||||
}
|
||||
case "alpha", "dev":
|
||||
version = ""
|
||||
if channelStr == "" {
|
||||
channelStr = "alpha"
|
||||
}
|
||||
}
|
||||
|
||||
lts := ltsStr == "true" || ltsStr == "1"
|
||||
|
||||
var formats []string
|
||||
if formatsStr != "" {
|
||||
formats = strings.Split(formatsStr, ",")
|
||||
}
|
||||
|
||||
req := resolver.Request{
|
||||
OS: osStr,
|
||||
Arch: archStr,
|
||||
Libc: libcStr,
|
||||
Version: version,
|
||||
Channel: channelStr,
|
||||
LTS: lts,
|
||||
Formats: formats,
|
||||
Variant: variantStr,
|
||||
}
|
||||
|
||||
res, err := resolver.Resolve(pc.assets, req)
|
||||
if err != nil {
|
||||
http.Error(w, fmt.Sprintf("no match for %s: %v", pkg, err), http.StatusNotFound)
|
||||
return
|
||||
}
|
||||
|
||||
result := assetToV1Resolve(res)
|
||||
|
||||
switch format {
|
||||
case "json":
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetIndent("", " ")
|
||||
enc.Encode(result)
|
||||
case "tab":
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
data, err := marshalTSV([]v1ResolveResult{result})
|
||||
if err != nil {
|
||||
http.Error(w, "encode error: "+err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
w.Write(data)
|
||||
default:
|
||||
http.Error(w, "unsupported format: "+format, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
func assetToV1Release(a storage.Asset) v1Release {
|
||||
lts := "-"
|
||||
if a.LTS {
|
||||
lts = "lts"
|
||||
}
|
||||
channel := a.Channel
|
||||
if channel == "" {
|
||||
channel = "stable"
|
||||
}
|
||||
libc := a.Libc
|
||||
if libc == "" {
|
||||
libc = "-"
|
||||
}
|
||||
return v1Release{
|
||||
Version: a.Version,
|
||||
Channel: channel,
|
||||
LTS: lts,
|
||||
Date: a.Date,
|
||||
OS: a.OS,
|
||||
Arch: a.Arch,
|
||||
Libc: libc,
|
||||
Format: a.Format,
|
||||
Variants: strings.Join(a.Variants, " "),
|
||||
Download: a.Download,
|
||||
Filename: a.Filename,
|
||||
}
|
||||
}
|
||||
|
||||
func assetToV1Resolve(res resolver.Result) v1ResolveResult {
|
||||
a := res.Asset
|
||||
lts := "-"
|
||||
if a.LTS {
|
||||
lts = "lts"
|
||||
}
|
||||
channel := a.Channel
|
||||
if channel == "" {
|
||||
channel = "stable"
|
||||
}
|
||||
libc := a.Libc
|
||||
if libc == "" {
|
||||
libc = "-"
|
||||
}
|
||||
return v1ResolveResult{
|
||||
Version: a.Version,
|
||||
Channel: channel,
|
||||
LTS: lts,
|
||||
Date: a.Date,
|
||||
OS: a.OS,
|
||||
Arch: a.Arch,
|
||||
Libc: libc,
|
||||
Format: a.Format,
|
||||
Variants: strings.Join(a.Variants, " "),
|
||||
Download: a.Download,
|
||||
Filename: a.Filename,
|
||||
Triplet: res.Triplet,
|
||||
}
|
||||
}
|
||||
|
||||
func (s *server) v1ServeTSV(w http.ResponseWriter, assets []storage.Asset) {
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
|
||||
releases := make([]v1Release, len(assets))
|
||||
for i, a := range assets {
|
||||
releases[i] = assetToV1Release(a)
|
||||
}
|
||||
|
||||
data, err := marshalTSV(releases)
|
||||
if err != nil {
|
||||
http.Error(w, "encode error: "+err.Error(), http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
w.Write(data)
|
||||
}
|
||||
|
||||
func (s *server) v1ServeJSON(w http.ResponseWriter, assets []storage.Asset) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
|
||||
releases := make([]v1Release, len(assets))
|
||||
for i, a := range assets {
|
||||
releases[i] = assetToV1Release(a)
|
||||
}
|
||||
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetIndent("", " ")
|
||||
enc.Encode(releases)
|
||||
}
|
||||
|
||||
func (s *server) v1ServeEmpty(w http.ResponseWriter, format string) {
|
||||
switch format {
|
||||
case "json":
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.Write([]byte("[]\n"))
|
||||
case "tab":
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
// Just the header.
|
||||
data, _ := marshalTSV([]v1Release{})
|
||||
w.Write(data)
|
||||
}
|
||||
}
|
||||
|
||||
// filterAssets filters storage.Asset slices directly.
|
||||
func filterAssets(assets []storage.Asset, osStr, archStr, libcStr, channel, version, formatFilter, variant string, lts bool, limit int) []storage.Asset {
|
||||
var result []storage.Asset
|
||||
|
||||
for _, a := range assets {
|
||||
if osStr != "" && a.OS != osStr && a.OS != "ANYOS" && a.OS != "" {
|
||||
continue
|
||||
}
|
||||
if archStr != "" && a.Arch != archStr && a.Arch != "ANYARCH" && a.Arch != "" {
|
||||
continue
|
||||
}
|
||||
if libcStr != "" && a.Libc != "" && a.Libc != "none" && a.Libc != libcStr {
|
||||
continue
|
||||
}
|
||||
if lts && !a.LTS {
|
||||
continue
|
||||
}
|
||||
if channel != "" && a.Channel != channel {
|
||||
continue
|
||||
}
|
||||
if version != "" {
|
||||
v := strings.TrimPrefix(a.Version, "v")
|
||||
vq := strings.TrimPrefix(version, "v")
|
||||
if !strings.HasPrefix(v, vq) {
|
||||
continue
|
||||
}
|
||||
}
|
||||
if formatFilter != "" && !strings.Contains(a.Format, formatFilter) {
|
||||
continue
|
||||
}
|
||||
if variant != "" {
|
||||
if !hasVariant(a.Variants, variant) {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
result = append(result, a)
|
||||
if len(result) >= limit {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
// sortAssetsDescending sorts assets newest-first by version.
|
||||
func sortAssetsDescending(assets []storage.Asset) {
|
||||
slices.SortStableFunc(assets, func(a, b storage.Asset) int {
|
||||
va := lexver.Parse(strings.TrimPrefix(a.Version, "v"))
|
||||
vb := lexver.Parse(strings.TrimPrefix(b.Version, "v"))
|
||||
return lexver.Compare(vb, va) // descending
|
||||
})
|
||||
}
|
||||
|
||||
// hasVariant checks if the variant list contains the wanted variant.
|
||||
// This is a copy of resolver.hasVariant since it's unexported.
|
||||
func hasVariant(variants []string, want string) bool {
|
||||
for _, v := range variants {
|
||||
if v == want {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// marshalTSV encodes a slice of structs as tab-separated values with a header.
|
||||
// Uses csvutil for struct-to-CSV mapping, with csv.Writer set to tab delimiter.
|
||||
func marshalTSV[T any](records []T) ([]byte, error) {
|
||||
var buf bytes.Buffer
|
||||
w := csv.NewWriter(&buf)
|
||||
w.Comma = '\t'
|
||||
|
||||
enc := csvutil.NewEncoder(w)
|
||||
for _, r := range records {
|
||||
if err := enc.Encode(r); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
w.Flush()
|
||||
if err := w.Error(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
// normalizeV1Arch maps query arch names to canonical Go names.
|
||||
func normalizeV1Arch(s string) string {
|
||||
switch strings.ToLower(s) {
|
||||
case "amd64":
|
||||
return string(buildmeta.ArchAMD64) // "x86_64"
|
||||
case "arm64":
|
||||
return string(buildmeta.ArchARM64) // "aarch64"
|
||||
default:
|
||||
return s
|
||||
}
|
||||
}
|
||||
@@ -1,273 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// TestV1ReleasesTSV verifies the v1 releases endpoint returns proper TSV.
|
||||
func TestV1ReleasesTSV(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
packages := []string{"bat", "node", "go"}
|
||||
for _, pkg := range packages {
|
||||
t.Run(pkg, func(t *testing.T) {
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/v1/releases/"+pkg+".tab?limit=5")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
lines := strings.Split(strings.TrimSpace(body), "\n")
|
||||
if len(lines) < 2 {
|
||||
t.Fatal("expected header + data rows")
|
||||
}
|
||||
|
||||
// First line should be header.
|
||||
header := lines[0]
|
||||
fields := strings.Split(header, "\t")
|
||||
expectedHeaders := []string{
|
||||
"version",
|
||||
"channel",
|
||||
"lts",
|
||||
"date",
|
||||
"os",
|
||||
"arch",
|
||||
"libc",
|
||||
"format",
|
||||
"variants",
|
||||
"download",
|
||||
"filename",
|
||||
}
|
||||
if len(fields) != len(expectedHeaders) {
|
||||
t.Fatalf("expected %d columns, got %d: %q", len(expectedHeaders), len(fields), header)
|
||||
}
|
||||
for i, want := range expectedHeaders {
|
||||
if fields[i] != want {
|
||||
t.Errorf("column[%d]: want %q, got %q", i, want, fields[i])
|
||||
}
|
||||
}
|
||||
|
||||
// Data rows should have same number of fields.
|
||||
for i, line := range lines[1:] {
|
||||
dataFields := strings.Split(line, "\t")
|
||||
if len(dataFields) != len(expectedHeaders) {
|
||||
t.Errorf("row[%d]: expected %d fields, got %d: %q", i, len(expectedHeaders), len(dataFields), line)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestV1ReleasesJSON verifies the v1 releases JSON format.
|
||||
func TestV1ReleasesJSON(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/v1/releases/"+pkg+".json?limit=3")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
var releases []v1Release
|
||||
if err := json.Unmarshal([]byte(body), &releases); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
|
||||
if len(releases) == 0 {
|
||||
t.Fatal("no releases")
|
||||
}
|
||||
|
||||
// v1 API uses Go-native naming — no mapping.
|
||||
for i, r := range releases {
|
||||
if r.Version == "" {
|
||||
t.Errorf("release[%d]: empty version", i)
|
||||
}
|
||||
if r.Download == "" {
|
||||
t.Errorf("release[%d]: empty download", i)
|
||||
}
|
||||
if r.Channel == "" {
|
||||
t.Errorf("release[%d]: empty channel (should be 'stable' or similar)", i)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestV1Resolve verifies the v1 resolve endpoint.
|
||||
func TestV1Resolve(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
query string
|
||||
wantOS string
|
||||
}{
|
||||
{
|
||||
name: "linux amd64",
|
||||
query: "?os=linux&arch=x86_64",
|
||||
wantOS: "linux",
|
||||
},
|
||||
{
|
||||
name: "darwin arm64",
|
||||
query: "?os=darwin&arch=aarch64",
|
||||
wantOS: "darwin",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
code, body := get(t, ts, "/v1/resolve/"+pkg+".json"+tt.query)
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
var result v1ResolveResult
|
||||
if err := json.Unmarshal([]byte(body), &result); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
|
||||
if result.Version == "" {
|
||||
t.Error("empty version")
|
||||
}
|
||||
if result.Download == "" {
|
||||
t.Error("empty download")
|
||||
}
|
||||
if result.OS != tt.wantOS {
|
||||
t.Errorf("os: want %q, got %q", tt.wantOS, result.OS)
|
||||
}
|
||||
if result.Triplet == "" {
|
||||
t.Error("empty triplet")
|
||||
}
|
||||
|
||||
t.Logf("resolved: %s %s %s %s → %s", result.Version, result.OS, result.Arch, result.Format, result.Download)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestV1ResolveTSV verifies the TSV format for resolve.
|
||||
func TestV1ResolveTSV(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/v1/resolve/"+pkg+".tab?os=linux&arch=x86_64")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
lines := strings.Split(strings.TrimSpace(body), "\n")
|
||||
if len(lines) != 2 {
|
||||
t.Fatalf("expected 2 lines (header + result), got %d", len(lines))
|
||||
}
|
||||
|
||||
header := strings.Split(lines[0], "\t")
|
||||
data := strings.Split(lines[1], "\t")
|
||||
|
||||
if len(header) != len(data) {
|
||||
t.Fatalf("header has %d fields, data has %d", len(header), len(data))
|
||||
}
|
||||
|
||||
// Should have a "triplet" column.
|
||||
hasTriplet := false
|
||||
for _, h := range header {
|
||||
if h == "triplet" {
|
||||
hasTriplet = true
|
||||
}
|
||||
}
|
||||
if !hasTriplet {
|
||||
t.Error("missing triplet column in header")
|
||||
}
|
||||
}
|
||||
|
||||
// TestV1ResolveJQ verifies jq resolves to binaries, not git.
|
||||
func TestV1ResolveJQ(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "jq"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/v1/resolve/"+pkg+".json?os=darwin&arch=aarch64")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
var result v1ResolveResult
|
||||
if err := json.Unmarshal([]byte(body), &result); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
|
||||
if result.Format == "git" {
|
||||
t.Errorf("resolved to git instead of binary: %+v", result)
|
||||
}
|
||||
if result.OS == "" {
|
||||
t.Errorf("resolved to empty OS (git asset): %+v", result)
|
||||
}
|
||||
|
||||
t.Logf("jq resolved: version=%s os=%s arch=%s format=%s → %s",
|
||||
result.Version, result.OS, result.Arch, result.Format, result.Download)
|
||||
}
|
||||
|
||||
// TestV1ReleasesFilterOS verifies OS filtering works.
|
||||
func TestV1ReleasesFilterOS(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/v1/releases/"+pkg+".json?os=darwin&limit=10")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
var releases []v1Release
|
||||
if err := json.Unmarshal([]byte(body), &releases); err != nil {
|
||||
t.Fatalf("decode: %v", err)
|
||||
}
|
||||
|
||||
for i, r := range releases {
|
||||
if r.OS != "darwin" && r.OS != "ANYOS" && r.OS != "" {
|
||||
t.Errorf("release[%d]: os=%q, expected darwin", i, r.OS)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestV1NoQuotedFields verifies TSV output has no quoted fields.
|
||||
func TestV1NoQuotedFields(t *testing.T) {
|
||||
srv, ts := newTestServer(t)
|
||||
|
||||
pkg := "bat"
|
||||
if srv.getPackage(pkg) == nil {
|
||||
t.Skipf("package %s not in cache", pkg)
|
||||
}
|
||||
|
||||
code, body := get(t, ts, "/v1/releases/"+pkg+".tab?limit=20")
|
||||
if code != 200 {
|
||||
t.Fatalf("status %d: %s", code, body)
|
||||
}
|
||||
|
||||
lines := strings.Split(strings.TrimSpace(body), "\n")
|
||||
for i, line := range lines {
|
||||
if strings.Contains(line, "\"") {
|
||||
t.Errorf("line[%d] contains quotes: %s", i, line)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,55 +0,0 @@
|
||||
---
|
||||
title: XCode Command Line Tools
|
||||
homepage: https://webinstall.dev/commandlinetools
|
||||
tagline: |
|
||||
The XCode Command Line Tools include git, swift, make, clang, and other developer tools
|
||||
---
|
||||
|
||||
## Cheat Sheet
|
||||
|
||||
> The developer tools provided by Apple for macOS.
|
||||
|
||||
- git
|
||||
- swift
|
||||
- make
|
||||
- clang
|
||||
- etc
|
||||
|
||||
This is also part of [webi-essentials](../webi-essentials/).
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- Files
|
||||
- Manual Install
|
||||
- macOS
|
||||
- Linux
|
||||
- Alpine
|
||||
- Windows
|
||||
|
||||
### Files
|
||||
|
||||
These are the files / directories that are created and/or modified with this
|
||||
install:
|
||||
|
||||
```sh
|
||||
/Library/Developer/CommandLineTools/
|
||||
```
|
||||
|
||||
### How to Install Manually
|
||||
|
||||
It's very easy to start the installer:
|
||||
|
||||
```sh
|
||||
xcode-select --install
|
||||
```
|
||||
|
||||
The trick is to also have a mechanism to know when it has finished:
|
||||
|
||||
```sh
|
||||
while ! test -x /Library/Developer/CommandLineTools/usr/bin/git ||
|
||||
! test -x /Library/Developer/CommandLineTools/usr/bin/make; do
|
||||
sleep 0.25
|
||||
done
|
||||
|
||||
echo "Command Line Tools Installed"
|
||||
```
|
||||
@@ -1,62 +0,0 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
set -u
|
||||
|
||||
fn_install_xcode_commandlinetools() { (
|
||||
b_os="$(uname -s)"
|
||||
if test "${b_os}" != 'Darwin'; then
|
||||
echo >&2 'XCode Command Line Tools are for macOS only'
|
||||
return 1
|
||||
fi
|
||||
|
||||
# streamline the output to be pretty
|
||||
fn_check_pkg '/Library/Developer/CommandLineTools/usr/bin/clang' 'clang'
|
||||
fn_check_pkg '/Library/Developer/CommandLineTools/usr/bin/git' 'git'
|
||||
fn_check_pkg '/Library/Developer/CommandLineTools/usr/bin/make' 'make'
|
||||
echo >&2 ""
|
||||
|
||||
# git
|
||||
if xcode-select -p > /dev/null 2> /dev/null; then
|
||||
echo ""
|
||||
return 0
|
||||
fi
|
||||
|
||||
cmd_xcode_cli_tools_install="xcode-select --install"
|
||||
echo " Running $(t_cmd "${cmd_xcode_cli_tools_install}")"
|
||||
$cmd_xcode_cli_tools_install 2> /dev/null
|
||||
echo ""
|
||||
echo ">>> $(t_attn 'ACTION REQUIRED') <<<"
|
||||
echo ""
|
||||
echo " $(t_attn "Click") '$(t_bold 'Install')' $(t_attn "in the pop-up")"
|
||||
echo " (it may appear $(t_em 'under') this window)"
|
||||
echo ""
|
||||
echo "^^^ $(t_attn 'ACTION REQUIRED') ^^^"
|
||||
echo ""
|
||||
printf " waiting %s to finish installing Command Line Developer Tools ..." "$(t_em 'for you')"
|
||||
while ! test -x /Library/Developer/CommandLineTools/usr/bin/git ||
|
||||
! test -x /Library/Developer/CommandLineTools/usr/bin/make; do
|
||||
sleep 0.25
|
||||
done
|
||||
echo " $(t_info 'OK')"
|
||||
echo " Installed to $(t_path '/Library/Developer/CommandLineTools/')"
|
||||
sleep 1
|
||||
); }
|
||||
|
||||
fn_check_pkg() { (
|
||||
a_pkg="${1}"
|
||||
a_pkgname="${2:-$a_pkg}"
|
||||
|
||||
printf >&2 ' %s %s %s' \
|
||||
"$(t_dim "Checking for")" \
|
||||
"$(t_pkg "${a_pkgname}")" \
|
||||
"$(t_dim "...")"
|
||||
|
||||
if command -v "${a_pkg}" > /dev/null; then
|
||||
echo >&2 " $(t_dim 'OK')"
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo >&2 ' missing'
|
||||
); }
|
||||
|
||||
fn_install_xcode_commandlinetools
|
||||
@@ -3,13 +3,13 @@
|
||||
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
|
||||
$EXENAME = "$Env:PKG_NAME.exe"
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
|
||||
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION\bin\$VERNAME")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION\bin\$VERNAME")) {
|
||||
Write-Output "Installing $Env:PKG_NAME"
|
||||
# TODO: temp directory
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = kivikakk/comrak
|
||||
20
comrak/releases.js
Normal file
20
comrak/releases.js
Normal file
@@ -0,0 +1,20 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'kivikakk';
|
||||
var repo = 'comrak';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
all.releases = all.releases.slice(0, 10);
|
||||
//console.info(JSON.stringify(all));
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$pkg_download")) {
|
||||
IF (!(Test-Path -Path "$pkg_download")) {
|
||||
Write-Output "Downloading crabz from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing crabz"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = sstadick/crabz
|
||||
31
crabz/releases.js
Normal file
31
crabz/releases.js
Normal file
@@ -0,0 +1,31 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'sstadick';
|
||||
var repo = 'crabz';
|
||||
|
||||
module.exports = async function (request) {
|
||||
let all = await github(request, owner, repo);
|
||||
|
||||
let releases = [];
|
||||
for (let rel of all.releases) {
|
||||
let isSrc = rel.download.includes('-src.');
|
||||
if (isSrc) {
|
||||
continue;
|
||||
}
|
||||
|
||||
releases.push(rel);
|
||||
}
|
||||
all.releases = releases;
|
||||
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -3,13 +3,13 @@
|
||||
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
|
||||
$EXENAME = "$Env:PKG_NAME.exe"
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
|
||||
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\xbin\$VERNAME")) {
|
||||
Write-Output "Installing $Env:PKG_NAME"
|
||||
# TODO: temp directory
|
||||
|
||||
@@ -29,13 +29,13 @@ if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
|
||||
# Settle unpacked archive into place
|
||||
Write-Output "New Name: $VERNAME"
|
||||
Write-Output "New Location: $Env:USERPROFILE\.local\bin\$VERNAME"
|
||||
Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\bin"
|
||||
Write-Output "New Location: $Env:USERPROFILE\.local\xbin\$VERNAME"
|
||||
Move-Item -Path "$VERNAME" -Destination "$Env:USERPROFILE\.local\xbin"
|
||||
|
||||
# Exit tmp
|
||||
Pop-Location
|
||||
}
|
||||
|
||||
Write-Output "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\bin\$VERNAME'"
|
||||
Write-Output "Copying into '$Env:USERPROFILE\.local\bin\$EXENAME' from '$Env:USERPROFILE\.local\xbin\$VERNAME'"
|
||||
Remove-Item -Path "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse -ErrorAction Ignore
|
||||
Copy-Item -Path "$Env:USERPROFILE\.local\bin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse
|
||||
Copy-Item -Path "$Env:USERPROFILE\.local\xbin\$VERNAME" -Destination "$Env:USERPROFILE\.local\bin\$EXENAME" -Recurse
|
||||
|
||||
@@ -19,7 +19,7 @@ __init_curlie() {
|
||||
}
|
||||
|
||||
pkg_install() {
|
||||
# $HOME/.local/bin
|
||||
# $HOME/.local/xbin
|
||||
mkdir -p "$pkg_src_bin"
|
||||
|
||||
# mv ./curlie* "$HOME/.local/opt/curlie-v1.3.1/bin/curlie"
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = rs/curlie
|
||||
20
curlie/releases.js
Normal file
20
curlie/releases.js
Normal file
@@ -0,0 +1,20 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'rs';
|
||||
var repo = 'curlie';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
all.releases = all.releases.slice(0, 10);
|
||||
//console.info(JSON.stringify(all));
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -100,7 +100,8 @@ mkdir -p ~/.dashcore/wallets/
|
||||
mkdir -p /mnt/slc1_vol_100g/dashcore/_data
|
||||
mkdir -p /mnt/slc1_vol_100g/dashcore/_caches
|
||||
|
||||
serviceman add --name 'dashd' --daemon -- \
|
||||
sudo env PATH="$PATH" serviceman add \
|
||||
--system --user "$my_user" --path "$PATH" --name dashd --force -- \
|
||||
dashd \
|
||||
-usehd \
|
||||
-conf="$HOME/.dashcore/dash.conf" \
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user