mirror of
https://github.com/webinstall/webi-installers.git
synced 2026-05-30 12:32:48 +00:00
Compare commits
2 Commits
feat/basec
...
feat-windo
| Author | SHA1 | Date | |
|---|---|---|---|
|
6859ab4c2b | ||
|
|
f216ab69b4 |
3
.github/ISSUE_TEMPLATE/new_installer.md
vendored
3
.github/ISSUE_TEMPLATE/new_installer.md
vendored
@@ -35,18 +35,15 @@ info, and doing a find and replace on a few file system path names.
|
||||
```
|
||||
2. Copy the example template and update with info from Official Releases:
|
||||
<https://github.com/___CHANGE/ME___/releases>
|
||||
|
||||
```bash
|
||||
rsync -av _example/ CHANGE-ME/
|
||||
```
|
||||
|
||||
- [ ] update `CHANGE-ME/release.js` to use the official repo
|
||||
- [ ] Learn how `CHANGE-ME` unpacks (i.e. as a single file? as a .tar.gz? as
|
||||
a .tar.gz with a folder named CHANGE-ME?)
|
||||
- [ ] find and replace to change the name
|
||||
- [ ] update `CHANGE-ME/install.sh` (see `bat` and `jq` as examples)
|
||||
- [ ] update `CHANGE-ME/install.ps1` (see `bat` and `jq` as examples)
|
||||
|
||||
3. Needs an updated tagline and cheat sheet
|
||||
- [ ] update `CHANGE-ME/README.md`
|
||||
- [ ] official URL
|
||||
|
||||
38
.gitignore
vendored
38
.gitignore
vendored
@@ -1,41 +1,7 @@
|
||||
# generated artifacts
|
||||
.env
|
||||
node_modules
|
||||
install-*.sh
|
||||
install-*.bat
|
||||
install-*.ps1
|
||||
|
||||
# Go build outputs (from go run/build in repo root)
|
||||
/classify
|
||||
/e2etest
|
||||
/fetchraw
|
||||
/inspect
|
||||
/uaparse
|
||||
/webicached
|
||||
/zigtest
|
||||
/distributables.csv
|
||||
|
||||
# local config
|
||||
.env.*
|
||||
*.env
|
||||
.env
|
||||
!example.env
|
||||
|
||||
# caches
|
||||
_cache/
|
||||
node_modules/
|
||||
|
||||
# temporary & backup files
|
||||
.*.sw*
|
||||
*.bak
|
||||
*.bak.*
|
||||
|
||||
# agent session files
|
||||
agents/
|
||||
|
||||
# other
|
||||
.DS_Store
|
||||
desktop.ini
|
||||
.directory
|
||||
LIVE_cache
|
||||
/webid
|
||||
bin/
|
||||
.claude/
|
||||
|
||||
@@ -1,7 +1,4 @@
|
||||
{
|
||||
"globals": {
|
||||
"AbortController": false
|
||||
},
|
||||
"browser": true,
|
||||
"node": true,
|
||||
"esversion": 11,
|
||||
|
||||
492
AGENTS.md
492
AGENTS.md
@@ -1,492 +0,0 @@
|
||||
# Webi Installers — Agent Guide
|
||||
|
||||
Webi installs dev tools to `~/.local/` without sudo. Each installer is a small
|
||||
package of 3-4 files. This guide tells you how to create and modify them.
|
||||
|
||||
## Why Webi Exists
|
||||
|
||||
Webi makes tool installation trivially repeatable for people who aren't
|
||||
sysadmins — freelance clients, junior devs, anyone who shouldn't have to care
|
||||
about PATH, permissions, or platform differences. Three things matter:
|
||||
|
||||
1. **Install without friction.** No sudo, no manual PATH edits, no "necessary
|
||||
but unimportant" steps leaking into the experience.
|
||||
2. **Know where things are.** The Files section tells you exactly what got
|
||||
created or modified. Nothing should be mysterious.
|
||||
3. **Copy-paste recipes.** The cheat sheet is what you'd send someone less
|
||||
experienced than yourself instead of a project's full README — scannable,
|
||||
concrete, easy to reference by name.
|
||||
|
||||
## Quick Start: Adding a New Installer
|
||||
|
||||
1. Identify the **package type** (see [Categories](#categories) below)
|
||||
2. Find an existing installer of the same type to use as a template
|
||||
3. Create `<name>/releases.js`, `install.sh`, `install.ps1`, `README.md`
|
||||
4. Test with the command in [Testing releases.js](#testing-releasesjs)
|
||||
5. Run formatters before committing (see [Code Style](#code-style))
|
||||
|
||||
## Directory Layout
|
||||
|
||||
```
|
||||
<package-name>/
|
||||
README.md # YAML frontmatter + docs
|
||||
releases.js # Fetches release metadata (Node.js)
|
||||
install.sh # POSIX shell installer (macOS/Linux)
|
||||
install.ps1 # PowerShell installer (Windows) — optional
|
||||
```
|
||||
|
||||
Key infrastructure directories (do not modify without good reason):
|
||||
|
||||
- `_webi/` — bootstrap templates, `normalize.js` (auto-detects OS/arch/ext from
|
||||
filenames)
|
||||
- `_common/` — shared JS: `github.js`, `githubish.js`, `gitea.js`, `fetcher.js`
|
||||
- `_example/` — canonical template for new packages
|
||||
- `_examples/` — specialized templates (goreleaser, xz-compressed)
|
||||
|
||||
## Categories
|
||||
|
||||
Ref: <https://github.com/webinstall/webi-installers/issues/412>
|
||||
|
||||
| Type | Description | Template to copy |
|
||||
| ----- | -------------------------------------- | ---------------- |
|
||||
| `bin` | Single binary in tar/zip | `koji`, `delta` |
|
||||
| `bin` | Single bare binary (no archive) | `arc`, `shfmt` |
|
||||
| `bin` | Goreleaser-style archives | `keypairs` |
|
||||
| 📦 | Self-contained package (bin/man/share) | `node`, `go` |
|
||||
| 📂 | Multiple binaries/scripts | `pg` |
|
||||
| 🔗 | Alias/redirect to another package | `ripgrep` → `rg` |
|
||||
| 📝 | Bespoke / custom install | `rustlang` |
|
||||
|
||||
## releases.js
|
||||
|
||||
Fetches release metadata and returns a normalized object. Most packages use
|
||||
GitHub releases:
|
||||
|
||||
```js
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'OWNER';
|
||||
var repo = 'REPO';
|
||||
|
||||
let Releases = module.exports;
|
||||
|
||||
Releases.latest = async function () {
|
||||
let all = await github(null, owner, repo);
|
||||
return all;
|
||||
};
|
||||
|
||||
Releases.sample = async function () {
|
||||
let normalize = require('../_webi/normalize.js');
|
||||
let all = await Releases.latest();
|
||||
all = normalize(all);
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
(async function () {
|
||||
let samples = await Releases.sample();
|
||||
console.info(JSON.stringify(samples, null, 2));
|
||||
})();
|
||||
}
|
||||
```
|
||||
|
||||
### Common release transformations
|
||||
|
||||
**Strip version prefix** (monorepo or tool-prefixed tags):
|
||||
|
||||
```js
|
||||
// e.g. "tools/monorel/v0.6.5" → "v0.6.5"
|
||||
rel.version = rel.version.replace(/^tools\/monorel\//, '');
|
||||
|
||||
// e.g. "cli-v1.2.3" → "v1.2.3"
|
||||
rel.version = rel.version.replace(/^cli-/, '');
|
||||
```
|
||||
|
||||
**Filter releases** (monorepo with multiple tools, or unwanted assets):
|
||||
|
||||
```js
|
||||
all.releases = all.releases.filter(function (rel) {
|
||||
// Keep only releases for this tool
|
||||
return rel.version.startsWith('tools/monorel/');
|
||||
});
|
||||
```
|
||||
|
||||
Apply transformations inside `Releases.latest`, before returning `all`.
|
||||
|
||||
**Available sources** beyond `github.js`:
|
||||
|
||||
- `_common/gitea.js` — Gitea servers
|
||||
- `_common/git-tag.js` — Git tag listing
|
||||
- Custom fetch from any JSON API (see `go/releases.js`, `terraform/releases.js`)
|
||||
|
||||
### Testing releases.js
|
||||
|
||||
```sh
|
||||
node -e "
|
||||
let Releases = require('./<name>/releases.js');
|
||||
Releases.sample().then(function (all) {
|
||||
console.log(JSON.stringify(all, null, 2));
|
||||
});
|
||||
"
|
||||
```
|
||||
|
||||
Verify: versions are clean semver (`0.6.5` not `tools/monorel/v0.6.5`), OS/arch
|
||||
detected correctly, download URLs resolve.
|
||||
|
||||
## install.sh
|
||||
|
||||
POSIX shell (`sh`, not bash). Always wrapped in a function:
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
set -e
|
||||
set -u
|
||||
|
||||
__init_pkgname() {
|
||||
# These 6 variables are required
|
||||
pkg_cmd_name="cmd"
|
||||
|
||||
pkg_dst_cmd="$HOME/.local/bin/cmd"
|
||||
pkg_dst="$pkg_dst_cmd"
|
||||
|
||||
pkg_src_cmd="$HOME/.local/opt/cmd-v$WEBI_VERSION/bin/cmd"
|
||||
pkg_src_dir="$HOME/.local/opt/cmd-v$WEBI_VERSION"
|
||||
pkg_src="$pkg_src_cmd"
|
||||
|
||||
pkg_install() {
|
||||
mkdir -p "$(dirname "$pkg_src_cmd")"
|
||||
mv ./cmd "$pkg_src_cmd"
|
||||
}
|
||||
|
||||
pkg_get_current_version() {
|
||||
cmd --version 2> /dev/null | head -n 1 | cut -d' ' -f2
|
||||
}
|
||||
}
|
||||
|
||||
__init_pkgname
|
||||
```
|
||||
|
||||
### Framework variables available in install.sh
|
||||
|
||||
Set by the webi bootstrap (`_webi/package-install.tpl.sh`):
|
||||
|
||||
| Variable | Example | Description |
|
||||
| --------------- | ------------------- | --------------------- |
|
||||
| `WEBI_VERSION` | `1.2.3` | Selected version |
|
||||
| `WEBI_PKG_URL` | `https://...` | Download URL |
|
||||
| `WEBI_PKG_FILE` | `foo-v1.2.3.tar.gz` | Download filename |
|
||||
| `WEBI_OS` | `linux` | Detected OS |
|
||||
| `WEBI_ARCH` | `amd64` | Detected architecture |
|
||||
| `WEBI_EXT` | `tar.gz` | Archive extension |
|
||||
| `WEBI_CHANNEL` | `stable` | Release channel |
|
||||
| `PKG_NAME` | `foo` | Package name |
|
||||
|
||||
### Override functions
|
||||
|
||||
| Function | Purpose |
|
||||
| --------------------------- | --------------------------------------------- |
|
||||
| `pkg_install()` | **Required.** Move files to `$pkg_src` |
|
||||
| `pkg_get_current_version()` | Parse installed version from command output |
|
||||
| `pkg_post_install()` | Post-install setup (git config, shell config) |
|
||||
| `pkg_done_message()` | Custom completion message |
|
||||
| `pkg_link()` | Override default symlink behavior |
|
||||
| `pkg_pre_install()` | Custom pre-install logic |
|
||||
|
||||
### Framework helper functions
|
||||
|
||||
| Function | Purpose |
|
||||
| ------------------------ | ---------------------------------- |
|
||||
| `webi_download()` | Download package if not cached |
|
||||
| `webi_extract()` | Extract archive by extension |
|
||||
| `webi_path_add <dir>` | Add to PATH via envman |
|
||||
| `webi_link()` | Create versioned symlinks |
|
||||
| `webi_check_installed()` | Check if version already installed |
|
||||
|
||||
### pkg_install patterns
|
||||
|
||||
**Bare binary in archive root:**
|
||||
|
||||
```sh
|
||||
mv ./cmd "$pkg_src_cmd"
|
||||
```
|
||||
|
||||
**Binary in a subdirectory (goreleaser-style `cmd-OS-arch/cmd`):**
|
||||
|
||||
```sh
|
||||
mv ./cmd-*/cmd "$pkg_src_cmd"
|
||||
```
|
||||
|
||||
**Flexible detection (handles multiple archive layouts):**
|
||||
|
||||
```sh
|
||||
if test -f ./cmd; then
|
||||
mv ./cmd "$pkg_src_cmd"
|
||||
elif test -e ./cmd-*/cmd; then
|
||||
mv ./cmd-*/cmd "$pkg_src_cmd"
|
||||
elif test -e ./cmd-*; then
|
||||
mv ./cmd-* "$pkg_src_cmd"
|
||||
fi
|
||||
```
|
||||
|
||||
## install.ps1
|
||||
|
||||
PowerShell for Windows. Uses `$Env:` variables. See `_example/install.ps1` for
|
||||
the full template. Key differences from install.sh:
|
||||
|
||||
- Paths use backslashes, commands end in `.exe`
|
||||
- `$Env:USERPROFILE` instead of `$HOME`
|
||||
- `Test-Path`, `Move-Item`, `Copy-Item` instead of shell equivalents
|
||||
- Downloads go to `$Env:USERPROFILE\Downloads\webi\`
|
||||
- Temp work in `.local\tmp`, use `Push-Location`/`Pop-Location`
|
||||
- Symlinks done via `Copy-Item` (not actual symlinks)
|
||||
|
||||
## README.md
|
||||
|
||||
````markdown
|
||||
---
|
||||
title: toolname
|
||||
homepage: https://github.com/owner/repo
|
||||
tagline: |
|
||||
toolname: A short one-line description.
|
||||
---
|
||||
|
||||
To update or switch versions, run `webi toolname@stable` (or `@v2`, `@beta`,
|
||||
etc).
|
||||
|
||||
### Files
|
||||
|
||||
These are the files that are created and/or modified with this installer:
|
||||
|
||||
```text
|
||||
~/.config/envman/PATH.env
|
||||
~/.local/bin/toolname
|
||||
~/.local/opt/toolname-VERSION/bin/toolname
|
||||
```
|
||||
|
||||
## Cheat Sheet
|
||||
|
||||
> `toolname` does X. Brief description.
|
||||
|
||||
### How to use toolname
|
||||
|
||||
```sh
|
||||
toolname --example
|
||||
```
|
||||
````
|
||||
|
||||
Note: **Files goes above Cheat Sheet**, not inside it.
|
||||
|
||||
### Cheat Sheet tone and style
|
||||
|
||||
Webi cheat sheets are **opinionated quick-reference guides**, not comprehensive
|
||||
documentation. Think "colleague's sticky note" — not the project's official
|
||||
README.
|
||||
|
||||
The tool is the topic, but **the problem is the reason**. Cheat sheets are
|
||||
organized around tasks the reader already wants to do — the tool is how they get
|
||||
there. Headings reference the tool (the reader came to this page on purpose),
|
||||
but the content solves the underlying problem completely:
|
||||
|
||||
- "How to reverse proxy to Node" (caddy knowledge, not just node)
|
||||
- "How to run a Node app as a System Service" (serviceman knowledge)
|
||||
- "How to Enable Secure Remote Postgres Access" (openssl, pg_hba.conf, systemd)
|
||||
- "How to manually configure git to use delta" (gitconfig, not delta flags)
|
||||
- "How to make fish the default shell in iTerm2" (iTerm2 knowledge, not fish)
|
||||
|
||||
The reader's question is "how do I do X?" and the cheat sheet answers it
|
||||
completely — including configs, adjacent tools, and platform-specific
|
||||
variations. A goreleaser cheat sheet teaches you goreleaser YAML. A postgres
|
||||
cheat sheet teaches you pg_hba.conf, openssl certs, and systemd units.
|
||||
|
||||
Cheat sheets cross tool boundaries freely. Node's references caddy, serviceman,
|
||||
setcap-netbind, GitHub Actions. Postgres references serviceman, openrc, launchd.
|
||||
They link to each other's webi pages. The scope is "everything you need to
|
||||
accomplish this task," not "everything this one binary does."
|
||||
|
||||
They show the actual files and configs that matter — not documentation _about_
|
||||
configs, but the configs themselves, copy-pasteable, with inline comments
|
||||
explaining the non-obvious parts.
|
||||
|
||||
**Guidelines:**
|
||||
|
||||
- **Show the 3-5 things someone will actually do**, with copy-pasteable
|
||||
commands. Skip exhaustive flag lists and API docs.
|
||||
- **Lead with practical integration.** Show the exact `git config` lines, the
|
||||
exact hook script, the exact shell alias — don't just explain the feature and
|
||||
leave wiring up to the reader.
|
||||
- **Skip what they already know.** No need to re-explain what the tool is at
|
||||
length — the tagline and one-liner blockquote handle that. Get to the
|
||||
commands.
|
||||
- **Prefer concrete over abstract.** Instead of "you can configure X via a
|
||||
config file", show the config file contents.
|
||||
|
||||
## Shell Naming Conventions
|
||||
|
||||
**Variables:**
|
||||
|
||||
- `ALL_CAPS` — environment variables only (`PATH`, `HOME`, `WEBI_VERSION`)
|
||||
- `b_varname` — block-scoped (inside a function, loop, or conditional)
|
||||
- `g_varname` — global to the script (and sourced scripts)
|
||||
- `a_varname` — function arguments
|
||||
|
||||
**Functions and commands:**
|
||||
|
||||
- `fn_name` — helper functions (anything other than the script's main/entry
|
||||
function)
|
||||
- `cmd_name` — command aliases, e.g. `cmd_curl='curl --fail-with-body -sSL'`
|
||||
|
||||
## Code Style
|
||||
|
||||
Requires `node`, `shfmt`, `pwsh`, and `pwsh-essentials` (install all via webi).
|
||||
Run before committing:
|
||||
|
||||
```sh
|
||||
npm run fmt # prettier (JS/MD) + shfmt (sh) + pwsh-fmt (ps1)
|
||||
npm run lint # jshint + shellcheck + pwsh-lint
|
||||
```
|
||||
|
||||
Commit messages: `feat(<pkg>): add installer`, `fix(<pkg>): update install.sh`,
|
||||
`docs(<pkg>): add cheat sheet`.
|
||||
|
||||
## Naming Conventions
|
||||
|
||||
- The canonical package name is the **command name** you type: `go`, `node`,
|
||||
`rg`
|
||||
- The alternate/alias name is the project name: `golang`, `nodejs`, `ripgrep`
|
||||
- Package directories are lowercase with hyphens
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
- **Monorepo releases**: The GitHub API returns ALL releases for the repo. You
|
||||
must filter in `releases.js` and strip the tag prefix from the version.
|
||||
- **No `--version` flag**: Some tools lack version introspection. Comment out
|
||||
`pkg_get_current_version` — webi still works, it just can't skip reinstalls.
|
||||
- **normalize.js auto-detection**: OS/arch/ext are guessed from download
|
||||
filenames. If the tool uses non-standard naming, you may need to set `os`,
|
||||
`arch`, or `ext` explicitly in `releases.js`.
|
||||
- **Goreleaser archives**: Typically contain a bare binary at the archive root
|
||||
(not nested in a directory). Use `mv ./cmd "$pkg_src_cmd"`.
|
||||
|
||||
---
|
||||
|
||||
## Go Cache Daemon
|
||||
|
||||
The Go pipeline (`cmd/webicached`) replaces the Node.js release-fetching code.
|
||||
It reads `releases.conf` files, fetches upstream release metadata, classifies
|
||||
build assets, and writes to `~/.cache/webi/legacy/` in the format the Node.js server expects.
|
||||
|
||||
### Canonical Vocabulary
|
||||
|
||||
The classifier MUST use exactly these strings. They match the production API.
|
||||
|
||||
**OS**: `macos` (NOT `darwin`), `linux`, `windows`, `freebsd`, `openbsd`,
|
||||
`netbsd`, `dragonfly`, `aix`, `illumos`, `plan9`, `solaris`, `posix_2017`
|
||||
|
||||
**Arch** — exact equivalences:
|
||||
- `amd64` (NOT `x86_64`), `x86` (NOT `i386`/`i686`/`386`)
|
||||
- `arm64` (NOT `aarch64`)
|
||||
- `armv7l` (NOT `armv7`), `armv6l` (NOT `armv6`)
|
||||
- `mipsle` (NOT `mipsel`), `mips64le` (NOT `mips64el`)
|
||||
|
||||
**Arch** — compatibility downcasts:
|
||||
- `armhf` → `armv7l`, `armv7a` → `armv7l`, `armel` → `arm`
|
||||
|
||||
**Arch** — other: `arm`, `ppc64le`, `ppc64`, `loong64`, `riscv64`, `s390x`,
|
||||
`mips`, `mips64`
|
||||
|
||||
**Libc**: `none` (never empty), `gnu`, `musl`, `msvc`
|
||||
|
||||
**Ext**: `tar.gz`, `tar.xz`, `zip`, `exe`, `7z`, `pkg`, `msi`
|
||||
(no leading dot; `exe` for bare binaries)
|
||||
|
||||
### releases.conf
|
||||
|
||||
Each package directory contains a `releases.conf` that tells the daemon where
|
||||
to fetch releases. Format is `key = value`, one per line. `#` comments and
|
||||
blank lines are ignored.
|
||||
|
||||
#### Source types (mutually exclusive — pick one)
|
||||
|
||||
```ini
|
||||
# GitHub binary releases (most common)
|
||||
github_releases = sharkdp/bat
|
||||
|
||||
# GitHub source tarballs (with optional git fallback)
|
||||
github_sources = bnnanet/serviceman
|
||||
git_url = https://github.com/bnnanet/serviceman.git
|
||||
|
||||
# Git tag enumeration (vim plugins, shell scripts — git_url alone)
|
||||
git_url = https://github.com/tpope/vim-commentary.git
|
||||
|
||||
# Gitea (full URL required, or short form + base_url)
|
||||
gitea_releases = https://git.rootprojects.org/root/pathman
|
||||
|
||||
# GitLab (defaults to gitlab.com)
|
||||
gitlab_releases = owner/repo
|
||||
|
||||
# HashiCorp releases API
|
||||
hashicorp_product = terraform
|
||||
|
||||
# Custom source (servicemandist, nodedist, zigdist, etc.)
|
||||
source = nodedist
|
||||
url = https://nodejs.org/download/release
|
||||
```
|
||||
|
||||
#### Filtering, versioning, and platform
|
||||
|
||||
```ini
|
||||
tag_prefix = bun- # monorepo: strip prefix from version
|
||||
version_prefixes = jq- # strip from version string (space-separated)
|
||||
asset_filter = MinGit # filename must contain this substring
|
||||
exclude = busybox -src- -docs- # skip assets containing these (space-separated)
|
||||
os = posix_2017 # restrict ALL versions to this OS (blanket)
|
||||
alias_of = rg # mirrors another package's releases
|
||||
```
|
||||
|
||||
#### Design rules
|
||||
|
||||
- `os` is a blanket tag on ALL versions. Only use for packages that are always
|
||||
POSIX-only. For version-dependent OS tagging, use a custom `TagVariants` in
|
||||
`internal/releases/{pkg}/variants.go`.
|
||||
- `git_url` can be primary (gittag source when it's the only key) or secondary
|
||||
fallback alongside `github_sources`/`gitea_sources`.
|
||||
- Full URL forms accepted for github/gitea/gitlab (e.g.
|
||||
`github_releases = https://github.com/sharkdp/bat`).
|
||||
|
||||
### Testing
|
||||
|
||||
Test tools: `cmd/e2etest` (pipeline comparison), `cmd/comparecache` (cache diff),
|
||||
`cmd/inspect` (single-package debug). Run each with `--help` for usage.
|
||||
|
||||
### Classifier vs Per-Package Tagger
|
||||
|
||||
The general classifier (`internal/classify/`) handles patterns common across
|
||||
many projects. It MUST NOT contain one-off logic for a single package.
|
||||
|
||||
Per-package taggers (`internal/releases/{pkg}/variants.go`) handle
|
||||
project-specific knowledge. Read the existing taggers for conventions.
|
||||
|
||||
MUST: Derive arch/OS from concrete evidence — not blanket defaults.
|
||||
MUST: New general classifier patterns must apply to 2-3+ packages.
|
||||
|
||||
### Deploying
|
||||
|
||||
```sh
|
||||
./scripts/deploy-webicached.sh beta.webi.sh
|
||||
./scripts/deploy-webicached.sh next.webi.sh
|
||||
```
|
||||
|
||||
First-time setup on a new host uses `serviceman`:
|
||||
|
||||
```sh
|
||||
serviceman add --name webicached \
|
||||
--workdir ~/srv/webid/installers/ -- \
|
||||
~/bin/webicached \
|
||||
--envfile ~/srv/webid/.env.secret \
|
||||
--conf ~/srv/webid/installers/ \
|
||||
--raw ~/.cache/webi/raw
|
||||
```
|
||||
@@ -10,6 +10,7 @@
|
||||
|
||||
- You'll be asked to make changes if you don't run the code formatters and
|
||||
linters:
|
||||
|
||||
- Node / JavaScript:
|
||||
- [prettier](https://webinstall.dev/prettier)
|
||||
```sh
|
||||
@@ -20,6 +21,7 @@
|
||||
npm run lint
|
||||
```
|
||||
- Bash
|
||||
|
||||
- [shfmt](https://webinstall.dev/shfmt)
|
||||
```sh
|
||||
npm run shfmt
|
||||
|
||||
@@ -145,8 +145,8 @@ It looks like this:
|
||||
`releases.js`:
|
||||
|
||||
```js
|
||||
module.exports = function () {
|
||||
return github(null, owner, repo).then(function (all) {
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
// if you need to do something special, you can do it here
|
||||
// ...
|
||||
return all;
|
||||
|
||||
64
_common/brew.js
Normal file
64
_common/brew.js
Normal file
@@ -0,0 +1,64 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Gets a releases from 'brew'.
|
||||
*
|
||||
* @param request
|
||||
* @param {string} formula
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
function getAllReleases(request, formula) {
|
||||
if (!formula) {
|
||||
return Promise.reject('missing formula for brew');
|
||||
}
|
||||
return request({
|
||||
url: 'https://formulae.brew.sh/api/formula/' + formula + '.json',
|
||||
fail: true, // https://git.coolaj86.com/coolaj86/request.js/issues/2
|
||||
json: true,
|
||||
})
|
||||
.then(failOnBadStatus)
|
||||
.then(function (resp) {
|
||||
var ver = resp.body.versions.stable;
|
||||
var dl = (
|
||||
resp.body.bottle.stable.files.high_sierra ||
|
||||
resp.body.bottle.stable.files.catalina
|
||||
).url.replace(new RegExp(ver.replace(/\./g, '\\.'), 'g'), '{{ v }}');
|
||||
return [
|
||||
{
|
||||
version: ver,
|
||||
download: dl.replace(/{{ v }}/g, ver),
|
||||
},
|
||||
].concat(
|
||||
resp.body.versioned_formulae.map(function (f) {
|
||||
var ver = f.replace(/.*@/, '');
|
||||
return {
|
||||
version: ver,
|
||||
download: dl,
|
||||
};
|
||||
}),
|
||||
);
|
||||
})
|
||||
.catch(function (err) {
|
||||
console.error('Error fetching MariaDB versions (brew)');
|
||||
console.error(err);
|
||||
return [];
|
||||
});
|
||||
}
|
||||
|
||||
function failOnBadStatus(resp) {
|
||||
if (resp.statusCode >= 400) {
|
||||
var err = new Error('Non-successful status code: ' + resp.statusCode);
|
||||
err.code = 'ESTATUS';
|
||||
err.response = resp;
|
||||
throw err;
|
||||
}
|
||||
return resp;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(require('@root/request'), 'mariadb').then(function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
218
_common/git-tag.js
Normal file
218
_common/git-tag.js
Normal file
@@ -0,0 +1,218 @@
|
||||
'use strict';
|
||||
|
||||
require('dotenv').config({ path: '.env' });
|
||||
|
||||
var Crypto = require('crypto');
|
||||
var util = require('util');
|
||||
var exec = util.promisify(require('child_process').exec);
|
||||
var Fs = require('node:fs/promises');
|
||||
var FsSync = require('node:fs');
|
||||
var Path = require('node:path');
|
||||
|
||||
var repoBaseDir = process.env.REPO_BASE_DIR || '';
|
||||
if (!repoBaseDir) {
|
||||
repoBaseDir = Path.resolve('./_repos');
|
||||
// for stderr
|
||||
console.error(`[Warn] REPO_BASE_DIR= not set, ${repoBaseDir}`);
|
||||
}
|
||||
|
||||
var Repos = {};
|
||||
|
||||
Repos.clone = async function (repoPath, gitUrl) {
|
||||
let uuid = Crypto.randomUUID();
|
||||
let tmpPath = `${repoPath}.${uuid}.tmp`;
|
||||
let bakPath = `${repoPath}.${uuid}.dup`;
|
||||
await exec(`git clone --bare --filter=tree:0 ${gitUrl} ${tmpPath}`);
|
||||
|
||||
try {
|
||||
FsSync.accessSync(repoPath);
|
||||
return;
|
||||
} catch (e) {
|
||||
if (e.code !== 'ENOENT') {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
// sync to avoid race conditions
|
||||
try {
|
||||
FsSync.renameSync(repoPath, bakPath);
|
||||
} catch (e) {
|
||||
if (e.code !== 'ENOENT') {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
FsSync.renameSync(tmpPath, repoPath);
|
||||
|
||||
await Fs.rm(bakPath, { force: true, recursive: true });
|
||||
};
|
||||
|
||||
Repos.checkExists = async function (repoPath) {
|
||||
let err = await Fs.access(repoPath).catch(Object);
|
||||
if (!err) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if (err.code !== 'ENOENT') {
|
||||
throw err;
|
||||
}
|
||||
return false;
|
||||
};
|
||||
|
||||
Repos.fetch = async function (repoPath) {
|
||||
await exec(`git --git-dir=${repoPath} fetch`);
|
||||
};
|
||||
|
||||
Repos.getTags = async function (repoPath) {
|
||||
var { stdout } = await exec(`git --git-dir=${repoPath} tag`);
|
||||
var rawTags = stdout.trim().split('\n');
|
||||
|
||||
let tags = [];
|
||||
for (let tag of rawTags) {
|
||||
// ex: v1, v2, v1.1, 1.1.0-rc
|
||||
let maybeVersionRe = /^(v\d+|v?\d+\.\d+)/;
|
||||
let maybeVersion = maybeVersionRe.test(tag);
|
||||
if (maybeVersion) {
|
||||
tags.push(tag);
|
||||
}
|
||||
}
|
||||
|
||||
tags = tags.reverse();
|
||||
return tags;
|
||||
};
|
||||
|
||||
Repos.getTipInfo = async function (repoPath) {
|
||||
var { stdout } = await exec(
|
||||
`git --git-dir=${repoPath} rev-parse --abbrev-ref HEAD`,
|
||||
);
|
||||
var branch = stdout.trim();
|
||||
|
||||
var info = await Repos.getCommitInfo(repoPath, 'HEAD');
|
||||
info.commitish = branch;
|
||||
|
||||
return info;
|
||||
};
|
||||
|
||||
Repos.getCommitInfo = async function (repoPath, commitish) {
|
||||
var { stdout } = await exec(
|
||||
`git --git-dir=${repoPath} log -1 --format="%h %H %ad %cd" --date=iso-strict ${commitish}`,
|
||||
);
|
||||
stdout = stdout.trim();
|
||||
var commitParts = stdout.split(/\s+/g);
|
||||
return {
|
||||
commitish: commitish,
|
||||
commit_id: `${commitParts[0]}`,
|
||||
commit: `${commitParts[1]}`,
|
||||
date: commitParts[2],
|
||||
date_authored: commitParts[3],
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Lists GitHub Releases (w/ uploaded assets)
|
||||
*
|
||||
* @param request
|
||||
* @param {string} owner
|
||||
* @param {string} gitUrl
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
async function getAllReleases(gitUrl) {
|
||||
let all = {
|
||||
releases: [],
|
||||
download: '',
|
||||
};
|
||||
|
||||
let repoName = gitUrl.split('/').pop();
|
||||
repoName = repoName.replace(/\.git$/, '');
|
||||
|
||||
let repoPath = `${repoBaseDir}/${repoName}.git`;
|
||||
|
||||
let isCloned = await Repos.checkExists(repoPath);
|
||||
if (!isCloned) {
|
||||
await Repos.clone(repoPath, gitUrl);
|
||||
} else {
|
||||
await Repos.fetch(repoPath);
|
||||
}
|
||||
|
||||
let commitInfos = [];
|
||||
let tags = await Repos.getTags(repoPath);
|
||||
for (let tag of tags) {
|
||||
let commitInfo = await Repos.getCommitInfo(repoPath, tag);
|
||||
Object.assign(commitInfo, { version: tag, channel: '' });
|
||||
commitInfos.push(commitInfo);
|
||||
}
|
||||
|
||||
{
|
||||
let tipInfo = await Repos.getTipInfo(repoPath);
|
||||
// "2024-01-01T00:00:00-05:00" => "2024-01-01T05:00:00"
|
||||
let date = new Date(tipInfo.date);
|
||||
// "2024-01-01T05:00:00" => "v2024.01.01-05.00.00"
|
||||
let version = date.toISOString();
|
||||
// strip '.000Z'
|
||||
version = version.replace(/\.\d+Z/, '');
|
||||
version = version.replace(/[:\-]/g, '.');
|
||||
version = version.replace(/T/, '-');
|
||||
Object.assign(tipInfo, { version: `v${version}`, channel: '' });
|
||||
if (commitInfos.length > 1) {
|
||||
tipInfo.channel = 'beta';
|
||||
}
|
||||
commitInfos.push(tipInfo);
|
||||
}
|
||||
|
||||
let releases = [];
|
||||
for (let commitInfo of commitInfos) {
|
||||
let version = commitInfo.version.replace(/^v/, '');
|
||||
|
||||
let date = new Date(commitInfo.date);
|
||||
let isoDate = date.toISOString();
|
||||
isoDate = isoDate.replace(/\.\d+Z/, '');
|
||||
|
||||
// tags and HEAD qualify for '--branch <branchish>'
|
||||
let branch = commitInfo.commitish;
|
||||
|
||||
let rel = {
|
||||
name: `${repoName}-v${version}`,
|
||||
version: version,
|
||||
git_tag: branch,
|
||||
git_commit_hash: commitInfo.commit_id,
|
||||
lts: false,
|
||||
channel: commitInfo.channel,
|
||||
date: isoDate,
|
||||
os: '*',
|
||||
arch: '*',
|
||||
ext: 'git',
|
||||
download: gitUrl,
|
||||
};
|
||||
|
||||
releases.push(rel);
|
||||
}
|
||||
all.releases = releases;
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
(async function main() {
|
||||
let testRepos = [
|
||||
// just a few tags, and a different HEAD
|
||||
'https://github.com/tpope/vim-commentary.git',
|
||||
// no tags, just HEAD
|
||||
'https://github.com/ziglang/zig.vim.git',
|
||||
// many, many tags
|
||||
//'https://github.com/dense-analysis/ale.git',
|
||||
];
|
||||
for (let url of testRepos) {
|
||||
let all = await getAllReleases(url);
|
||||
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
}
|
||||
})()
|
||||
.then(function () {
|
||||
process.exit(0);
|
||||
})
|
||||
.catch(function (err) {
|
||||
console.error(err);
|
||||
});
|
||||
}
|
||||
50
_common/gitea.js
Normal file
50
_common/gitea.js
Normal file
@@ -0,0 +1,50 @@
|
||||
'use strict';
|
||||
|
||||
var GitHubish = require('./githubish.js');
|
||||
|
||||
/**
|
||||
* Lists Gitea Releases (w/ uploaded assets)
|
||||
*
|
||||
* @param {any} _request - deprecated
|
||||
* @param {String} owner
|
||||
* @param {String} repo
|
||||
* @param {String} baseurl
|
||||
* @param {String} [username]
|
||||
* @param {String} [token]
|
||||
*/
|
||||
async function getAllReleases(
|
||||
_request,
|
||||
owner,
|
||||
repo,
|
||||
baseurl,
|
||||
username = '',
|
||||
token = '',
|
||||
) {
|
||||
baseurl = `${baseurl}/api/v1`;
|
||||
let all = await GitHubish.getAllReleases({
|
||||
owner,
|
||||
repo,
|
||||
baseurl,
|
||||
username,
|
||||
token,
|
||||
});
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(
|
||||
null,
|
||||
'root',
|
||||
'pathman',
|
||||
'https://git.rootprojects.org',
|
||||
'',
|
||||
'',
|
||||
).then(
|
||||
//getAllReleases(require('@root/request'), 'root', 'serviceman', 'https://git.rootprojects.org').then(
|
||||
function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
},
|
||||
);
|
||||
}
|
||||
133
_common/github-source.js
Normal file
133
_common/github-source.js
Normal file
@@ -0,0 +1,133 @@
|
||||
'use strict';
|
||||
|
||||
require('dotenv').config();
|
||||
|
||||
/**
|
||||
* Gets the releases for 'ripgrep'. This function could be trimmed down and made
|
||||
* for use with any github release.
|
||||
*
|
||||
* @param request
|
||||
* @param {string} owner
|
||||
* @param {string} repo
|
||||
* @returns {PromiseLike<any> | Promise<any>}
|
||||
*/
|
||||
async function getAllReleases(
|
||||
request,
|
||||
owner,
|
||||
repo,
|
||||
oses,
|
||||
arches,
|
||||
baseurl = 'https://api.github.com',
|
||||
) {
|
||||
if (!owner) {
|
||||
return Promise.reject('missing owner for repo');
|
||||
}
|
||||
if (!repo) {
|
||||
return Promise.reject('missing repo name');
|
||||
}
|
||||
|
||||
let req = {
|
||||
url: `${baseurl}/repos/${owner}/${repo}/releases`,
|
||||
json: true,
|
||||
};
|
||||
|
||||
// TODO I really don't like global config, find a way to do better
|
||||
if (process.env.GITHUB_USERNAME) {
|
||||
req.auth = {
|
||||
user: process.env.GITHUB_USERNAME,
|
||||
pass: process.env.GITHUB_TOKEN,
|
||||
};
|
||||
}
|
||||
|
||||
let resp = await request(req);
|
||||
let gHubResp = resp.body;
|
||||
let all = {
|
||||
releases: [],
|
||||
// TODO make this ':baseurl' + ':releasename'
|
||||
download: '',
|
||||
};
|
||||
|
||||
for (let release of gHubResp) {
|
||||
// TODO tags aren't always semver / sensical
|
||||
let tag = release['tag_name'];
|
||||
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
|
||||
let channel = 'stable';
|
||||
if (release['prerelease']) {
|
||||
channel = 'beta';
|
||||
}
|
||||
let date = release['published_at'] || '';
|
||||
date = date.replace(/T.*/, '');
|
||||
|
||||
let urls = [release.tarball_url, release.zipball_url];
|
||||
for (let url of urls) {
|
||||
let resp = await request({
|
||||
method: 'HEAD',
|
||||
followRedirect: true,
|
||||
followAllRedirects: true,
|
||||
followOriginalHttpMethod: true,
|
||||
url: url,
|
||||
stream: true,
|
||||
});
|
||||
// Workaround for bug where method changes to GET
|
||||
resp.destroy();
|
||||
|
||||
// content-disposition: attachment; filename=BeyondCodeBootcamp-DuckDNS.sh-v1.0.1-0-ga2f4bde.zip
|
||||
let name = resp.headers['content-disposition'].replace(
|
||||
/.*filename=([^;]+)(;|$)/,
|
||||
'$1',
|
||||
);
|
||||
all.releases.push({
|
||||
name: name,
|
||||
version: tag,
|
||||
lts: lts,
|
||||
channel: channel,
|
||||
date: date,
|
||||
os: '', // will be guessed by download filename
|
||||
arch: '', // will be guessed by download filename
|
||||
ext: '', // will be normalized
|
||||
download: resp.request.uri.href,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (oses) {
|
||||
return combinate(all, oses, arches);
|
||||
}
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
function combinate(all, oses, arches) {
|
||||
let releases = all.releases;
|
||||
// ex: arches = ['amd64', 'arm64', 'armv7l', 'armv6l', 'x86'];
|
||||
// ex: oses = ['macos', 'linux', 'bsd', 'posix'];
|
||||
|
||||
let combos = [];
|
||||
for (let release of releases) {
|
||||
for (let arch of arches) {
|
||||
for (let os of oses) {
|
||||
let combo = {
|
||||
arch: arch,
|
||||
os: os,
|
||||
};
|
||||
let rel = Object.assign({}, release, combo);
|
||||
combos.push(rel);
|
||||
}
|
||||
}
|
||||
}
|
||||
all.releases = combos;
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(
|
||||
require('@root/request'),
|
||||
'BeyondCodeBootcamp',
|
||||
'DuckDNS.sh',
|
||||
).then(function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
41
_common/github.js
Normal file
41
_common/github.js
Normal file
@@ -0,0 +1,41 @@
|
||||
'use strict';
|
||||
|
||||
require('dotenv').config({ path: '.env' });
|
||||
|
||||
let GitHubish = require('./githubish.js');
|
||||
|
||||
/**
|
||||
* Lists GitHub Releases (w/ uploaded assets)
|
||||
*
|
||||
* @param {any} _request - deprecated
|
||||
* @param {String} owner
|
||||
* @param {String} repo
|
||||
* @param {String} [baseurl]
|
||||
* @param {String} [username]
|
||||
* @param {String} [token]
|
||||
*/
|
||||
async function getAllReleases(
|
||||
_request,
|
||||
owner,
|
||||
repo,
|
||||
baseurl = 'https://api.github.com',
|
||||
username = process.env.GITHUB_USERNAME || '',
|
||||
token = process.env.GITHUB_TOKEN || '',
|
||||
) {
|
||||
let all = await GitHubish.getAllReleases({
|
||||
owner,
|
||||
repo,
|
||||
baseurl,
|
||||
username,
|
||||
token,
|
||||
});
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = getAllReleases;
|
||||
|
||||
if (module === require.main) {
|
||||
getAllReleases(null, 'BurntSushi', 'ripgrep').then(function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
124
_common/githubish.js
Normal file
124
_common/githubish.js
Normal file
@@ -0,0 +1,124 @@
|
||||
'use strict';
|
||||
|
||||
let GitHubish = module.exports;
|
||||
|
||||
/**
|
||||
* Lists GitHub-Like Releases (w/ uploaded assets)
|
||||
*
|
||||
* @param {Object} opts
|
||||
* @param {String} opts.owner
|
||||
* @param {String} opts.repo
|
||||
* @param {String} opts.baseurl
|
||||
* @param {String} [opts.username]
|
||||
* @param {String} [opts.token]
|
||||
*/
|
||||
GitHubish.getAllReleases = async function ({
|
||||
owner,
|
||||
repo,
|
||||
baseurl,
|
||||
username = '',
|
||||
token = '',
|
||||
}) {
|
||||
if (!owner) {
|
||||
throw new Error('missing owner for repo');
|
||||
}
|
||||
if (!repo) {
|
||||
throw new Error('missing repo name');
|
||||
}
|
||||
if (!baseurl) {
|
||||
throw new Error('missing baseurl');
|
||||
}
|
||||
|
||||
let url = `${baseurl}/repos/${owner}/${repo}/releases`;
|
||||
let opts = {
|
||||
headers: {
|
||||
'Content-Type': 'appplication/json',
|
||||
},
|
||||
};
|
||||
|
||||
if (token) {
|
||||
let userpass = `${username}:${token}`;
|
||||
let basicAuth = btoa(userpass);
|
||||
Object.assign(opts.headers, {
|
||||
Authorization: `Basic ${basicAuth}`,
|
||||
});
|
||||
}
|
||||
|
||||
let resp = await fetch(url, opts);
|
||||
if (!resp.ok) {
|
||||
let headers = Array.from(resp.headers);
|
||||
console.error('Bad Resp Headers:', headers);
|
||||
let text = await resp.text();
|
||||
console.error('Bad Resp Body:', text);
|
||||
let msg = `failed to fetch releases from '${baseurl}' with user '${username}'`;
|
||||
throw new Error(msg);
|
||||
}
|
||||
|
||||
let respText = await resp.text();
|
||||
let gHubResp;
|
||||
try {
|
||||
gHubResp = JSON.parse(respText);
|
||||
} catch (e) {
|
||||
console.error('Bad Resp JSON:', respText);
|
||||
console.error(e.message);
|
||||
let msg = `failed to parse releases from '${baseurl}' with user '${username}'`;
|
||||
throw new Error(msg);
|
||||
}
|
||||
|
||||
let all = {
|
||||
releases: [],
|
||||
// todo make this ':baseurl' + ':releasename'
|
||||
download: '',
|
||||
};
|
||||
|
||||
try {
|
||||
gHubResp.forEach(transformReleases);
|
||||
} catch (e) {
|
||||
console.error(e.message);
|
||||
console.error('Error Headers:', resp.headers);
|
||||
console.error('Error Body:', resp.body);
|
||||
let msg = `failed to transform releases from '${baseurl}' with user '${username}'`;
|
||||
throw new Error(msg);
|
||||
}
|
||||
|
||||
function transformReleases(release) {
|
||||
for (let asset of release['assets']) {
|
||||
let name = asset['name'];
|
||||
let date = release['published_at']?.replace(/T.*/, '');
|
||||
let download = asset['browser_download_url'];
|
||||
|
||||
// TODO tags aren't always semver / sensical
|
||||
let version = release['tag_name'];
|
||||
let channel;
|
||||
if (release['prerelease']) {
|
||||
// -rcX, -preview, -beta, etc will be checked in _webi/normalize.js
|
||||
channel = 'beta';
|
||||
}
|
||||
let lts = /(\b|_)(lts)(\b|_)/.test(release['tag_name']);
|
||||
|
||||
all.releases.push({
|
||||
name: name,
|
||||
version: version,
|
||||
lts: lts,
|
||||
channel: channel,
|
||||
date: date,
|
||||
os: '', // will be guessed by download filename
|
||||
arch: '', // will be guessed by download filename
|
||||
ext: '', // will be normalized
|
||||
download: download,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
GitHubish.getAllReleases({
|
||||
owner: 'BurntSushi',
|
||||
repo: 'ripgrep',
|
||||
baseurl: 'https://api.github.com',
|
||||
}).then(function (all) {
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading foobar from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing foobar"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
# Example releases.conf — uses ripgrep as a sample project.
|
||||
# Copy this file into your package directory and adjust.
|
||||
github_releases = BurntSushi/ripgrep
|
||||
30
_example/releases.js
Normal file
30
_example/releases.js
Normal file
@@ -0,0 +1,30 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'BurntSushi';
|
||||
var repo = 'ripgrep';
|
||||
|
||||
/******************************************************************************/
|
||||
/** Note: Delete this Comment! **/
|
||||
/** **/
|
||||
/** Need a an example that filters out miscellaneous release files? **/
|
||||
/** See `deno`, `gitea`, or `caddy` **/
|
||||
/** **/
|
||||
/******************************************************************************/
|
||||
|
||||
module.exports = async function (request) {
|
||||
let all = await github(request, owner, repo);
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
(async function () {
|
||||
let request = require('@root/request');
|
||||
let normalize = require('../_webi/normalize.js');
|
||||
let all = await module.exports(request);
|
||||
all = normalize(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
})();
|
||||
}
|
||||
@@ -4,6 +4,7 @@
|
||||
|
||||
//var pkg = require('../package.json');
|
||||
var os = require('os');
|
||||
//var request = require('@root/request');
|
||||
//var promisify = require('util').promisify;
|
||||
//var exec = promisify(require('child_process').exec);
|
||||
var exec = require('child_process').exec;
|
||||
|
||||
@@ -15,8 +15,8 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
|
||||
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
|
||||
|
||||
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-Not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
|
||||
continue
|
||||
}
|
||||
@@ -31,7 +31,7 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_new_file | Set-Content -Path $my_ps1
|
||||
|
||||
$my_new_file = $my_new_file + "`n"
|
||||
if ($text -ne $my_new_file) {
|
||||
IF ($text -ne $my_new_file) {
|
||||
$my_status = 1
|
||||
}
|
||||
}
|
||||
|
||||
@@ -15,8 +15,8 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_ps1 = [System.IO.Path]::GetRelativePath($my_cwd, $my_file.FullName)
|
||||
$my_dir = [System.IO.Path]::GetDirectoryName($my_file.FullName)
|
||||
|
||||
if (-not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
if (-Not (Test-Path -PathType Leaf -Path $my_ps1) -or
|
||||
-Not (Test-Path -PathType Container -Path $my_dir)) {
|
||||
Write-Host (" SKIP {0} (non-regular file or parent directory)" -f $my_ps1)
|
||||
continue
|
||||
}
|
||||
@@ -39,7 +39,7 @@ foreach ($my_dir in $my_dirs) {
|
||||
$my_new_file | Set-Content -Path $my_ps1
|
||||
|
||||
$my_new_file = $my_new_file + "`n"
|
||||
if ($my_old_file -ne $my_new_file) {
|
||||
IF ($my_old_file -ne $my_new_file) {
|
||||
$my_status = 1
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env pwsh
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.vim\pack\plugins\start")) {
|
||||
New-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start" -ItemType Directory -Force | Out-Null
|
||||
}
|
||||
Remove-Item -Path "$Env:USERPROFILE\.vim\pack\plugins\start\example" -Recurse -ErrorAction Ignore
|
||||
|
||||
Submodule _webi/build-classifier updated: f9cc9f3e19...6b626a6d08
@@ -13,6 +13,7 @@ async function main() {
|
||||
caches: CACHE_DIR,
|
||||
installers: INSTALLERS_DIR,
|
||||
});
|
||||
bc.freshenRandomPackage(600 * 1000);
|
||||
|
||||
// let dirs = await bc.getProjectsByType();
|
||||
// let projNames = Object.keys(dirs.valid);
|
||||
|
||||
@@ -3,16 +3,15 @@
|
||||
var BuildsCacher = module.exports;
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Os = require('node:os');
|
||||
let Path = require('node:path');
|
||||
|
||||
let LEGACY_CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
|
||||
let HostTargets = require('./build-classifier/host-targets.js');
|
||||
let Lexver = require('./build-classifier/lexver.js');
|
||||
let Triplet = require('./build-classifier/triplet.js');
|
||||
|
||||
var ALIAS_RE = /^alias: ([\w.-]+)$/m;
|
||||
let request = require('@root/request');
|
||||
|
||||
var ALIAS_RE = /^alias: (\w+)$/m;
|
||||
|
||||
var LEGACY_ARCH_MAP = {
|
||||
'*': 'ANYARCH',
|
||||
@@ -129,8 +128,61 @@ async function readFirstBytes(path) {
|
||||
return str;
|
||||
}
|
||||
|
||||
BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
let promises = {};
|
||||
async function getLatestBuilds(Releases, installersDir, cacheDir, name, date) {
|
||||
console.info(`[INFO] getLatestBuilds: ${name}`);
|
||||
|
||||
if (!Releases) {
|
||||
Releases = require(`${installersDir}/${name}/releases.js`);
|
||||
}
|
||||
// TODO update all releases files with module.exports.xxxx = 'foo';
|
||||
if (!Releases.latest) {
|
||||
Releases.latest = Releases;
|
||||
}
|
||||
|
||||
let id = `${cacheDir}/${name}`;
|
||||
if (!promises[id]) {
|
||||
promises[id] = Promise.resolve();
|
||||
}
|
||||
|
||||
promises[id] = promises[id].then(async function () {
|
||||
return await getLatestBuildsInner(Releases, cacheDir, name, date);
|
||||
});
|
||||
|
||||
return await promises[id];
|
||||
}
|
||||
|
||||
async function getLatestBuildsInner(Releases, cacheDir, name, date) {
|
||||
let data = await Releases.latest(request);
|
||||
|
||||
if (!date) {
|
||||
date = new Date();
|
||||
}
|
||||
let isoDate = date.toISOString();
|
||||
let yearMonth = isoDate.slice(0, 7);
|
||||
|
||||
// TODO hash file
|
||||
let dataFile = `${cacheDir}/${yearMonth}/${name}.json`;
|
||||
// TODO fsstat releases.js vs require-ing time as well
|
||||
let tsFile = `${cacheDir}/${yearMonth}/${name}.updated.txt`;
|
||||
|
||||
let dirPath = Path.dirname(dataFile);
|
||||
await Fs.mkdir(dirPath, { recursive: true });
|
||||
|
||||
let json = JSON.stringify(data, null, 2);
|
||||
await Fs.writeFile(dataFile, json, 'utf8');
|
||||
|
||||
let seconds = date.valueOf();
|
||||
let ms = seconds / 1000;
|
||||
let msStr = ms.toFixed(3);
|
||||
await Fs.writeFile(tsFile, msStr, 'utf8');
|
||||
|
||||
return data;
|
||||
}
|
||||
|
||||
BuildsCacher.create = function ({ ALL_TERMS, installers, caches }) {
|
||||
let installersDir = installers;
|
||||
let cacheDir = caches;
|
||||
|
||||
if (!ALL_TERMS) {
|
||||
ALL_TERMS = Triplet.TERMS_PRIMARY_MAP;
|
||||
@@ -145,11 +197,9 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
bc._triplets = {};
|
||||
bc._targetsByBuildIdCache = {};
|
||||
bc._caches = {};
|
||||
bc._staleAge = 15 * 60 * 1000;
|
||||
bc._allFormats = {};
|
||||
bc._allTriplets = {};
|
||||
// Per-name lock: serializes cold-cache getPackages so concurrent
|
||||
// callers can't corrupt bc._caches[name] via a transformAndUpdate race.
|
||||
bc._inflight = {};
|
||||
|
||||
for (let term of TERMS_META) {
|
||||
delete bc.orphanTerms[term];
|
||||
@@ -168,6 +218,19 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
let entries = await Fs.readdir(installersDir, { withFileTypes: true });
|
||||
for (let entry of entries) {
|
||||
let meta = await bc.getProjectTypeByEntry(entry);
|
||||
if (meta.type === 'not_found') {
|
||||
let err = meta.detail;
|
||||
console.error('');
|
||||
console.error('PROBLEM');
|
||||
console.error(` ${err.message}`);
|
||||
console.error('');
|
||||
console.error('SOLUTION');
|
||||
console.error(' npm clean-install');
|
||||
console.error('');
|
||||
throw new Error(
|
||||
'[SANITY FAIL] should never have missing modules in prod',
|
||||
);
|
||||
}
|
||||
dirs[meta.type][entry.name] = meta.detail;
|
||||
}
|
||||
|
||||
@@ -236,16 +299,19 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
return { type: 'alias', detail: link };
|
||||
}
|
||||
|
||||
let cacheFile = `${LEGACY_CACHE_DIR}/${entry.name}.json`;
|
||||
let hasCacheFile = await Fs.access(cacheFile)
|
||||
.then(function () {
|
||||
return true;
|
||||
})
|
||||
.catch(function () {
|
||||
return false;
|
||||
});
|
||||
if (!hasCacheFile) {
|
||||
return { type: 'selfhosted', detail: true };
|
||||
let releasesPath = Path.join(path, 'releases.js');
|
||||
try {
|
||||
void require(releasesPath);
|
||||
} catch (err) {
|
||||
if (err.code !== 'MODULE_NOT_FOUND') {
|
||||
return { type: 'errors', detail: err };
|
||||
}
|
||||
|
||||
if (err.message.includes(`Cannot find module '${releasesPath}'`)) {
|
||||
return { type: 'selfhosted', detail: true };
|
||||
}
|
||||
|
||||
return { type: 'not_found', detail: err };
|
||||
}
|
||||
|
||||
return { type: 'valid', detail: true };
|
||||
@@ -253,26 +319,14 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
|
||||
// Typically a package is organized by release (ex: go has 1.20, 1.21, etc),
|
||||
// but we will organize by the build (ex: go1.20-darwin-arm64.tar.gz, etc).
|
||||
bc.getPackages = async function (args) {
|
||||
let name = args.name;
|
||||
let warm = bc._caches[name];
|
||||
if (warm) {
|
||||
return _doGetPackages(args);
|
||||
bc.getPackages = async function ({ Releases, name, date }) {
|
||||
if (!date) {
|
||||
date = new Date();
|
||||
}
|
||||
let inflight = bc._inflight[name];
|
||||
if (inflight) {
|
||||
return inflight;
|
||||
}
|
||||
let p = _doGetPackages(args).finally(function () {
|
||||
delete bc._inflight[name];
|
||||
});
|
||||
bc._inflight[name] = p;
|
||||
return p;
|
||||
};
|
||||
|
||||
async function _doGetPackages({ name }) {
|
||||
let dataFile = `${LEGACY_CACHE_DIR}/${name}.json`;
|
||||
let tsFile = `${LEGACY_CACHE_DIR}/${name}.updated.txt`;
|
||||
let isoDate = date.toISOString();
|
||||
let yearMonth = isoDate.slice(0, 7);
|
||||
let dataFile = `${cacheDir}/${yearMonth}/${name}.json`;
|
||||
let tsFile = `${cacheDir}/${yearMonth}/${name}.updated.txt`;
|
||||
|
||||
let tsDate;
|
||||
{
|
||||
@@ -326,7 +380,7 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
}
|
||||
}
|
||||
if (!projInfo) {
|
||||
return meta;
|
||||
projInfo = await getLatestBuilds(Releases, installersDir, cacheDir, name);
|
||||
}
|
||||
let latestProjInfo = await BuildsCacher.transformAndUpdate(
|
||||
name,
|
||||
@@ -337,8 +391,63 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
);
|
||||
bc._caches[name] = latestProjInfo;
|
||||
|
||||
return latestProjInfo;
|
||||
}
|
||||
process.nextTick(async function () {
|
||||
let now = date.valueOf();
|
||||
let age = now - projInfo.updated;
|
||||
|
||||
let fresh = age < bc._staleAge;
|
||||
if (fresh) {
|
||||
return;
|
||||
}
|
||||
|
||||
projInfo = await getLatestBuilds(Releases, installersDir, cacheDir, name);
|
||||
let latestProjInfo = BuildsCacher.transformAndUpdate(
|
||||
name,
|
||||
projInfo,
|
||||
meta,
|
||||
date,
|
||||
bc,
|
||||
);
|
||||
bc._caches[name] = latestProjInfo;
|
||||
});
|
||||
|
||||
return projInfo;
|
||||
};
|
||||
|
||||
// Makes sure that packages are updated once an hour, on average
|
||||
bc._staleNames = [];
|
||||
bc._freshenTimeout = null;
|
||||
bc.freshenRandomPackage = async function (minDelay) {
|
||||
if (!minDelay) {
|
||||
minDelay = 15 * 1000;
|
||||
}
|
||||
|
||||
if (bc._staleNames.length === 0) {
|
||||
let dirs = await bc.getProjectsByType();
|
||||
bc._staleNames = Object.keys(dirs.valid);
|
||||
bc._staleNames.sort(function () {
|
||||
return 0.5 - Math.random();
|
||||
});
|
||||
}
|
||||
|
||||
let name = bc._staleNames.pop();
|
||||
void (await bc.getPackages({
|
||||
//Releases: Releases,
|
||||
name: name,
|
||||
date: new Date(),
|
||||
}));
|
||||
console.info(`[INFO] freshenRandomPackage: ${name}`);
|
||||
|
||||
let hour = 60 * 60 * 1000;
|
||||
let delay = minDelay;
|
||||
let spread = hour / bc._staleNames.length;
|
||||
let seed = Math.random();
|
||||
delay += seed * spread;
|
||||
|
||||
clearTimeout(bc._freshenTimeout);
|
||||
bc._freshenTimeout = setTimeout(bc.freshenRandomPackage, delay);
|
||||
bc._freshenTimeout.unref();
|
||||
};
|
||||
|
||||
/**
|
||||
* Given a list of acceptable formats, get the sorted list of of formats.
|
||||
@@ -430,10 +539,7 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
exts.push('.gz');
|
||||
exts.push('.sh');
|
||||
}
|
||||
let hasGit = formats.includes('git') || formats.includes('.git');
|
||||
if (hasGit) {
|
||||
exts.push('.git');
|
||||
}
|
||||
exts.push('.git');
|
||||
|
||||
// Fallbacks
|
||||
// (we include everything to bubble an extract error over not found)
|
||||
@@ -488,8 +594,9 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
|
||||
/**
|
||||
* @param {ProjectInfo} projInfo
|
||||
* @param {HostTarget} hostTarget
|
||||
*/
|
||||
bc.enumerateLatestVersions = function (projInfo) {
|
||||
bc.enumerateLatestVersions = function (projInfo, hostTarget) {
|
||||
let lexPrefix = '';
|
||||
let matchInfo = Lexver.matchSorted(projInfo.lexvers, lexPrefix);
|
||||
let verInfo = {
|
||||
@@ -542,19 +649,29 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Version-first iteration, not triplet-first: take the newest
|
||||
// version even when its only build lives in a fallback triplet
|
||||
// (e.g. serviceman v1.0.1 only exists at posix_2017-ANYARCH-none).
|
||||
for (let lexver of projInfo.lexvers) {
|
||||
let ver = projInfo.lexversMap[lexver] || lexver;
|
||||
for (let _triplet of triplets) {
|
||||
let targetReleases = projInfo.releasesByTriplet[_triplet];
|
||||
if (!targetReleases) {
|
||||
continue;
|
||||
}
|
||||
|
||||
for (let _triplet of triplets) {
|
||||
let targetReleases = projInfo.releasesByTriplet[_triplet];
|
||||
if (!targetReleases) {
|
||||
continue;
|
||||
}
|
||||
let versions = Object.keys(targetReleases);
|
||||
//console.log('dbg: targetRelease versions', versions);
|
||||
let lexvers = [];
|
||||
for (let version of versions) {
|
||||
let lexPrefix = Lexver.parseVersion(version);
|
||||
lexvers.push(lexPrefix);
|
||||
}
|
||||
lexvers.sort();
|
||||
lexvers.reverse();
|
||||
// TODO get the other matchInfo props
|
||||
|
||||
// Make sure that these releases are the expected version
|
||||
// (ex: jq1.7 => darwin-arm64-libc, jq1.6 => darwin-x86_64-libc)
|
||||
for (let matchver of lexvers) {
|
||||
let ver = projInfo.lexversMap[matchver] || matchver;
|
||||
let packages = targetReleases[ver];
|
||||
//console.log('dbg: packages', packages);
|
||||
if (!packages) {
|
||||
continue;
|
||||
}
|
||||
@@ -605,33 +722,21 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
return triplets;
|
||||
}
|
||||
|
||||
// Prefer platform-specific matches over ANYOS/ANYARCH fallbacks.
|
||||
// This ensures e.g. darwin-aarch64-none matches before
|
||||
// ANYOS-ANYARCH-none (.git source URLs from old releases).
|
||||
let oses = [];
|
||||
if (hostTarget.os === 'windows') {
|
||||
oses = ['windows', 'ANYOS'];
|
||||
oses = ['ANYOS', 'windows'];
|
||||
} else if (hostTarget.os === 'android') {
|
||||
oses = ['android', 'linux', 'posix_2017', 'posix_2024', 'ANYOS'];
|
||||
oses = ['ANYOS', 'posix_2017', 'android', 'linux'];
|
||||
} else {
|
||||
oses = [hostTarget.os, 'posix_2017', 'posix_2024', 'ANYOS'];
|
||||
oses = ['ANYOS', 'posix_2017', hostTarget.os];
|
||||
}
|
||||
|
||||
let waterfall = HostTargets.WATERFALL[hostTarget.os] || {};
|
||||
let arches = waterfall[hostTarget.arch] ||
|
||||
HostTargets.WATERFALL.ANYOS[hostTarget.arch] || [hostTarget.arch];
|
||||
arches = arches.concat(['ANYARCH']);
|
||||
// termsToTarget omits libc for plain UAs; 'libc' → waterfall ['none','libc',...]
|
||||
let libc = hostTarget.libc || 'libc';
|
||||
let libcs = waterfall[libc] ||
|
||||
HostTargets.WATERFALL.ANYOS[libc] || [libc];
|
||||
|
||||
// Extend the glibc-host waterfall: the table only lists [none, libc]
|
||||
// but Rust projects (bat, rg) and node ship libc='gnu' builds, and
|
||||
// static musl builds also run on glibc hosts.
|
||||
if (libc === 'libc' && !libcs.includes('gnu')) {
|
||||
libcs = ['none', 'gnu', 'musl', 'libc'];
|
||||
}
|
||||
arches = ['ANYARCH'].concat(arches);
|
||||
let libcs = waterfall[hostTarget.libc] ||
|
||||
HostTargets.WATERFALL.ANYOS[hostTarget.libc] || [hostTarget.libc];
|
||||
|
||||
for (let os of oses) {
|
||||
for (let arch of arches) {
|
||||
@@ -660,17 +765,10 @@ BuildsCacher.create = function ({ ALL_TERMS, installers }) {
|
||||
};
|
||||
|
||||
BuildsCacher._classify = function (bc, projInfo, build) {
|
||||
/* jshint maxcomplexity: 30 */
|
||||
// Cache entries arrive pre-classified (os/arch/libc/ext set). Skip
|
||||
// maybeInstallable for those — it false-rejects names ending in a
|
||||
// version tag (`serviceman-v1.0.1`, `v1.0.1.zip`).
|
||||
let cacheClassified =
|
||||
build.os && build.arch && build.libc && build.ext;
|
||||
if (!cacheClassified) {
|
||||
let maybeInstallable = Triplet.maybeInstallable(projInfo, build);
|
||||
if (!maybeInstallable) {
|
||||
return null;
|
||||
}
|
||||
/* jshint maxcomplexity: 25 */
|
||||
let maybeInstallable = Triplet.maybeInstallable(projInfo, build);
|
||||
if (!maybeInstallable) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (LEGACY_OS_MAP[build.os]) {
|
||||
@@ -734,26 +832,16 @@ BuildsCacher._classify = function (bc, projInfo, build) {
|
||||
bc.unknownTerms[term] = true;
|
||||
}
|
||||
|
||||
// Skip termsToTarget for cache-classified entries: it false-flags
|
||||
// e.g. .git URLs as os=ANYOS while the cache says os=posix_2017,
|
||||
// and the mismatch check throws.
|
||||
// {NAME}.windows.x86_64v2.musl.exe
|
||||
// windows-x86_64_v2-musl
|
||||
target = { triplet: '' };
|
||||
if (cacheClassified) {
|
||||
target.os = build.os;
|
||||
target.arch = build.arch;
|
||||
target.libc = build.libc;
|
||||
target.vendor = build.vendor || 'unknown';
|
||||
target.android = false;
|
||||
target.unknownTerms = [];
|
||||
} else {
|
||||
try {
|
||||
void Triplet.termsToTarget(target, projInfo, build, terms);
|
||||
} catch (e) {
|
||||
console.error(`PACKAGE FORMAT CHANGE for '${projInfo.name}':`);
|
||||
console.error(e.message);
|
||||
console.error(build);
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
void Triplet.termsToTarget(target, projInfo, build, terms);
|
||||
} catch (e) {
|
||||
console.error(`PACKAGE FORMAT CHANGE for '${projInfo.name}':`);
|
||||
console.error(e.message);
|
||||
console.error(build);
|
||||
return null;
|
||||
}
|
||||
|
||||
target.triplet = `${target.arch}-${target.vendor}-${target.os}-${target.libc}`;
|
||||
|
||||
@@ -15,8 +15,11 @@ let bc = BuildsCacher.create({
|
||||
caches: CACHE_DIR,
|
||||
installers: INSTALLERS_DIR,
|
||||
});
|
||||
bc.freshenRandomPackage(600 * 1000);
|
||||
|
||||
Builds.init = async function () {
|
||||
bc.freshenRandomPackage(600 * 1000);
|
||||
|
||||
let dirs = await bc.getProjectsByType();
|
||||
let projNames = Object.keys(dirs.valid);
|
||||
|
||||
@@ -24,6 +27,7 @@ Builds.init = async function () {
|
||||
await Parallel.run(parallel, projNames, getAll);
|
||||
async function getAll(name) {
|
||||
void (await bc.getPackages({
|
||||
//Releases: Releases,
|
||||
name: name,
|
||||
date: new Date(),
|
||||
}));
|
||||
|
||||
@@ -1,13 +1,12 @@
|
||||
'use strict';
|
||||
|
||||
let Fs = require('node:fs/promises');
|
||||
let Os = require('node:os');
|
||||
let Path = require('node:path');
|
||||
|
||||
// let Builds = require('./builds.js');
|
||||
let BuildsCacher = require('./builds-cacher.js');
|
||||
let Triplet = require('./build-classifier/triplet.js');
|
||||
|
||||
let LEGACY_CACHE_DIR = Path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
let request = require('@root/request');
|
||||
|
||||
async function main() {
|
||||
let projName = process.argv[2];
|
||||
@@ -39,11 +38,16 @@ async function main() {
|
||||
arches: [],
|
||||
libcs: [],
|
||||
formats: [],
|
||||
// TODO channels: [],
|
||||
};
|
||||
|
||||
let dataFile = Path.join(LEGACY_CACHE_DIR, `${projName}.json`);
|
||||
let json = await Fs.readFile(dataFile, 'utf8');
|
||||
let projInfo = JSON.parse(json);
|
||||
let installersDir = Path.join(__dirname, '..');
|
||||
let Releases = require(`${installersDir}/${projName}/releases.js`);
|
||||
if (!Releases.latest) {
|
||||
Releases.latest = Releases;
|
||||
}
|
||||
|
||||
let projInfo = await Releases.latest(request);
|
||||
|
||||
// let packages = await Builds.getPackage({ name: projName });
|
||||
// console.log(packages);
|
||||
@@ -68,11 +72,9 @@ async function main() {
|
||||
console.log(`[DEBUG] transformed`);
|
||||
let sample = transformed.packages.slice(0, 20);
|
||||
console.log('packages:', sample, ':packages');
|
||||
let firstTriplet = Object.keys(transformed.releasesByTriplet)[0];
|
||||
let firstVersion = transformed.versions[0];
|
||||
console.log(
|
||||
`releasesByTriplet[${firstTriplet}][${firstVersion}]:`,
|
||||
transformed.releasesByTriplet[firstTriplet]?.[firstVersion],
|
||||
'releasesByTriplet:',
|
||||
transformed.releasesByTriplet['linux-x86_64-none'][transformed.versions[0]],
|
||||
':releasesByTriplet',
|
||||
);
|
||||
console.log('versions:', transformed.versions, ':versions');
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
############################################################
|
||||
New-Item -Path "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
New-Item -Path "$Env:USERPROFILE\.local\bin" -ItemType Directory -Force | Out-Null
|
||||
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
curl.exe -s -A "windows" "$Env:WEBI_HOST/packages/webi/webi-pwsh.ps1" -o "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1"
|
||||
Set-ExecutionPolicy -Scope Process Bypass
|
||||
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" "{{ exename }}"
|
||||
|
||||
@@ -16,7 +16,7 @@ var BAD_SH_RE = /[<>'"`$\\]/;
|
||||
Installers.renderBash = async function (
|
||||
pkgdir,
|
||||
rel,
|
||||
{ baseurl, pkg, tag, ver, os = '', arch = '', libc = '', formats },
|
||||
{ baseurl, pkg, tag, ver, os = '', arch = '', libc = '', formats, latest },
|
||||
) {
|
||||
if (!Array.isArray(formats)) {
|
||||
formats = [];
|
||||
@@ -99,12 +99,11 @@ Installers.renderBash = async function (
|
||||
['WEBI_PKG_PATHNAME', pkgFile],
|
||||
['WEBI_PKG_FILE', pkgFile], // TODO replace with pathname
|
||||
['PKG_NAME', pkg],
|
||||
['PKG_STABLE', rel.stable],
|
||||
['PKG_LATEST', rel.latest],
|
||||
['PKG_OSES', (rel.oses || []).join(' ')],
|
||||
['PKG_ARCHES', (rel.arches || []).join(' ')],
|
||||
['PKG_LIBCS', (rel.libcs || []).join(' ')],
|
||||
['PKG_FORMATS', (rel.formats || []).join(' ')],
|
||||
['PKG_LATEST', latest],
|
||||
];
|
||||
|
||||
for (let env of envReplacements) {
|
||||
|
||||
@@ -139,6 +139,8 @@ async function main() {
|
||||
console.info('');
|
||||
}
|
||||
|
||||
bc.freshenRandomPackage(600 * 1000);
|
||||
|
||||
let rows = [];
|
||||
let triples = [];
|
||||
let valids = Object.keys(dirs.valid);
|
||||
|
||||
258
_webi/normalize.js
Normal file
258
_webi/normalize.js
Normal file
@@ -0,0 +1,258 @@
|
||||
'use strict';
|
||||
|
||||
// this may need customizations between packages
|
||||
var osMap = {
|
||||
macos: /(\b|_)(apple|os(\s_-)?x\b|mac|darwin|iPhone|iOS|iPad)/i,
|
||||
linux: /(\b|_)(linux)/i,
|
||||
freebsd: /(\b|_)(freebsd)/i,
|
||||
windows: /(\b|_)(win|microsoft|msft)/i,
|
||||
sunos: /(\b|_)(sun)/i,
|
||||
aix: /(\b|_)(aix)/i,
|
||||
};
|
||||
|
||||
var maps = {
|
||||
oses: {},
|
||||
arches: {},
|
||||
libcs: {},
|
||||
formats: {},
|
||||
};
|
||||
|
||||
Object.keys(osMap).forEach(function (name) {
|
||||
maps.oses[name] = true;
|
||||
});
|
||||
|
||||
var formats = ['zip', 'xz', 'tar', 'pkg', 'msi', 'git', 'exe', 'dmg', 'git'];
|
||||
formats.forEach(function (name) {
|
||||
maps.formats[name] = true;
|
||||
});
|
||||
|
||||
// evaluation order matters
|
||||
// (i.e. otherwise x86 and x64 can cross match)
|
||||
var arches = [
|
||||
// arm64/aarch64 has very high specificity, so it comes first
|
||||
'arm64',
|
||||
// arm 7 is also generic aarch/arm/arm32
|
||||
'armv7l',
|
||||
// arm6 can run on armv7
|
||||
'armv6l',
|
||||
// amd64 is more likely and less often specified than arm64
|
||||
'amd64',
|
||||
'x86',
|
||||
'ppc64le',
|
||||
'ppc64',
|
||||
's390x',
|
||||
];
|
||||
// Used for detecting system arch from package download url, for example:
|
||||
//
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-aarch64-linux-musl.tar.gz
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-arm-linux-musleabihf.tar.gz
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-armv7-linux-musleabihf.tar.gz
|
||||
// https://git.com/org/foo/releases/v0.7.9/foo-x86_64-linux-musl.tar.gz
|
||||
//
|
||||
var archMap = {
|
||||
arm64: /(\b|_)(aarch64|arm64)/i,
|
||||
armv7l: /(\b|_)(arm32|arm[_\-]?v?7l?)/i,
|
||||
armv6l: /(\b|_)(arm|aarch32|arm[_\-]?v?6l?)(\b|_)/i,
|
||||
//amd64: /(amd.?64|x64|[_\-]64)/i,
|
||||
amd64:
|
||||
/(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)64([_\-]?bit)?(\b|_)/i,
|
||||
//x86: /(86)(\b|_)/i,
|
||||
x86: /(\b|_|amd|(dar)?win(dows)?|mac(os)?|linux|osx|x)(86|32)([_\-]?bit)(\b|_)/i,
|
||||
ppc64le: /(\b|_)(ppc64le)/i,
|
||||
ppc64: /(\b|_)(ppc64)(\b|_)/i,
|
||||
s390x: /(\b|_)(s390x)/i,
|
||||
};
|
||||
arches.forEach(function (name) {
|
||||
maps.arches[name] = true;
|
||||
});
|
||||
|
||||
var libcs = ['none', 'musl', 'gnu', 'msvc', 'libc'];
|
||||
libcs.forEach(function (name) {
|
||||
maps.libcs[name] = true;
|
||||
});
|
||||
|
||||
function normalize(all) {
|
||||
/* jshint maxcomplexity:50 */
|
||||
/* jshint maxdepth:10 */
|
||||
var supported = {
|
||||
oses: {},
|
||||
arches: {},
|
||||
libcs: {},
|
||||
formats: {},
|
||||
};
|
||||
|
||||
for (let rel of all.releases) {
|
||||
rel.version = rel.version.replace(/^v/i, '');
|
||||
|
||||
if (!rel.name) {
|
||||
rel.name = rel.download.replace(/.*\//, '');
|
||||
}
|
||||
|
||||
if (!rel.os) {
|
||||
rel.os = 'unknown';
|
||||
|
||||
let osNames = Object.keys(osMap);
|
||||
for (let osName of osNames) {
|
||||
let relName = rel.name || rel.download;
|
||||
let osRegExp = osMap[osName];
|
||||
let matches = osRegExp.test(relName);
|
||||
if (matches) {
|
||||
rel.os = osName;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
supported.oses[rel.os] = true;
|
||||
|
||||
if (!rel.arch) {
|
||||
for (let arch of arches) {
|
||||
let name = rel.name || rel.download;
|
||||
let isArch = name.match(archMap[arch]);
|
||||
if (isArch) {
|
||||
rel.arch = arch;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!rel.arch) {
|
||||
if ('macos' === rel.os) {
|
||||
rel.arch = 'amd64';
|
||||
}
|
||||
}
|
||||
supported.arches[rel.arch] = true;
|
||||
|
||||
// note: depends on rel.os
|
||||
if (!rel.libc) {
|
||||
let isMusl;
|
||||
let isMsvc;
|
||||
let isStatic;
|
||||
let isGnu;
|
||||
|
||||
// extra blocks to prevent copy pasta errors
|
||||
|
||||
{
|
||||
let muslRe = /(\b|_)(musl)(\b|_)/i;
|
||||
isMusl = muslRe.test(rel.download) || muslRe.test(rel.name);
|
||||
}
|
||||
|
||||
{
|
||||
let msvcRe = /(\b|_)(msvc)(\b|_)/i;
|
||||
isMsvc = msvcRe.test(rel.download) || msvcRe.test(rel.name);
|
||||
}
|
||||
|
||||
{
|
||||
let staticRe = /(\b|_)(static)(\b|_)/i;
|
||||
isStatic = staticRe.test(rel.download) || staticRe.test(rel.name);
|
||||
}
|
||||
|
||||
{
|
||||
let gnuRe = /(\b|_)(gnu|glibc|libc)(\b|_)/i;
|
||||
isGnu = gnuRe.test(rel.download) || gnuRe.test(rel.name);
|
||||
}
|
||||
|
||||
if (isMusl) {
|
||||
// we specifically tag things that need musl++ in their own releases
|
||||
rel.libc = 'none';
|
||||
} else if (isStatic) {
|
||||
rel.libc = 'none';
|
||||
} else if (isGnu) {
|
||||
rel.libc = 'gnu';
|
||||
if (rel.os === 'windows') {
|
||||
// windows gnu is static
|
||||
rel.libc = 'none';
|
||||
} else if (rel.os === 'darwin') {
|
||||
// if glibc is required on macos, it'll be static
|
||||
rel.libc = 'none';
|
||||
}
|
||||
} else if (isMsvc) {
|
||||
rel.libc = 'msvc';
|
||||
} else {
|
||||
// The default is no requirement for any particular libc
|
||||
// (Go, Zig, POSIX Shell, JS, etc)
|
||||
// and hopefully we never have to worry about mingw and friends
|
||||
rel.libc = 'none';
|
||||
}
|
||||
}
|
||||
supported.libcs[rel.libc] = true;
|
||||
|
||||
var tarExt;
|
||||
if (!rel.ext) {
|
||||
// pkg-v1.0.tar.gz => ['gz', 'tar', '0', 'pkg-v1']
|
||||
// pkg-v1.0.tar => ['tar', '0' ,'pkg-v1']
|
||||
// pkg-v1.0.zip => ['zip', '0', 'pkg-v1']
|
||||
var exts = (rel.name || rel.download).split('.');
|
||||
if (1 === exts.length) {
|
||||
// for bare releases in the format of foo-linux-amd64
|
||||
rel.ext = 'exe';
|
||||
}
|
||||
exts = exts.reverse().slice(0, 2);
|
||||
if ('tar' === exts[1]) {
|
||||
rel.ext = exts.reverse().join('.');
|
||||
tarExt = 'tar';
|
||||
} else if ('tgz' === exts[0]) {
|
||||
rel.ext = 'tar.gz';
|
||||
tarExt = 'tar';
|
||||
} else {
|
||||
rel.ext = exts[0];
|
||||
}
|
||||
if (/\-|linux|mac|os[_\-]?x|arm|amd|86|64|mip/i.test(rel.ext)) {
|
||||
// for bare releases in the format of foo.linux-amd64
|
||||
rel.ext = 'exe';
|
||||
}
|
||||
}
|
||||
supported.formats[tarExt || rel.ext] = true;
|
||||
|
||||
if (!rel.channel) {
|
||||
// basically like this: (+.-_)(beta|rc)(0-9)(+.-_)
|
||||
// matches:
|
||||
// - v1.0-beta
|
||||
// - v1.0-beta1.1
|
||||
// - v1.0-beta-11
|
||||
// won't match:
|
||||
// - v1.0beta
|
||||
// - v1.0-beta1b
|
||||
let isBetaRe = /(\b|_)(preview|rc|beta|alpha)(\d+)(\b|_)/;
|
||||
let isBeta = isBetaRe.test(rel.name);
|
||||
if (isBeta) {
|
||||
rel.channel = 'beta';
|
||||
} else {
|
||||
rel.channel = 'stable';
|
||||
}
|
||||
}
|
||||
|
||||
if (all.download) {
|
||||
rel.download = all.download.replace(/{{ download }}/, rel.download);
|
||||
}
|
||||
}
|
||||
|
||||
all.oses = Object.keys(supported.oses).filter(function (name) {
|
||||
return maps.oses[name];
|
||||
});
|
||||
all.arches = Object.keys(supported.arches).filter(function (name) {
|
||||
return maps.arches[name];
|
||||
});
|
||||
all.libcs = Object.keys(supported.libcs).filter(function (name) {
|
||||
return maps.libcs[name];
|
||||
});
|
||||
all.formats = Object.keys(supported.formats).filter(function (name) {
|
||||
return maps.formats[name];
|
||||
});
|
||||
|
||||
return all;
|
||||
}
|
||||
|
||||
module.exports = normalize;
|
||||
module.exports._debug = function (all) {
|
||||
all = normalize(all);
|
||||
all.releases = all.releases
|
||||
.filter(function (r) {
|
||||
return ['windows', 'macos', 'linux'].includes(r.os) && 'amd64' === r.arch;
|
||||
})
|
||||
.slice(0, 10);
|
||||
return all;
|
||||
};
|
||||
// NOT in order of priority (which would be tar, xz, zip, ...)
|
||||
module.exports.formats = formats;
|
||||
module.exports.arches = arches;
|
||||
module.exports.libcs = libcs;
|
||||
module.exports.formatsMap = maps.formats;
|
||||
@@ -45,15 +45,15 @@ $TDim = "${Esc}[2m"
|
||||
$TReset = "${Esc}[0m"
|
||||
|
||||
function Invoke-DownloadUrl {
|
||||
param (
|
||||
Param (
|
||||
[string]$URL,
|
||||
[string]$Params,
|
||||
[string]$Path,
|
||||
[switch]$Force
|
||||
)
|
||||
|
||||
if (Test-Path -Path "$Path") {
|
||||
if (-not $Force.IsPresent) {
|
||||
IF (Test-Path -Path "$Path") {
|
||||
IF (-Not $Force.IsPresent) {
|
||||
Write-Host " ${TDim}Found${TReset} $Path"
|
||||
return
|
||||
}
|
||||
@@ -65,7 +65,7 @@ function Invoke-DownloadUrl {
|
||||
|
||||
Write-Host " Downloading ${TDim}from${TReset}"
|
||||
Write-Host " ${TDim}${URL}${TReset}"
|
||||
if ($Params.Length -ne 0) {
|
||||
IF ($Params.Length -ne 0) {
|
||||
Write-Host " ?$Params"
|
||||
$URL = "${URL}?${Params}"
|
||||
}
|
||||
@@ -80,18 +80,18 @@ function Get-UserAgent {
|
||||
# This is the canonical CPU arch when the process is emulated
|
||||
$my_arch = "$Env:PROCESSOR_ARCHITEW6432"
|
||||
|
||||
if ($my_arch -eq $null -or $my_arch -eq "") {
|
||||
IF ($my_arch -eq $null -or $my_arch -eq "") {
|
||||
# This is the canonical CPU arch when the process is native
|
||||
$my_arch = "$Env:PROCESSOR_ARCHITECTURE"
|
||||
}
|
||||
|
||||
if ($my_arch -eq "AMD64") {
|
||||
IF ($my_arch -eq "AMD64") {
|
||||
# Because PowerShell is sometimes AMD64 on Windows 10 ARM
|
||||
# See https://oofhours.com/2020/02/04/powershell-on-windows-10-arm64/
|
||||
$my_os_arch = (Get-CimInstance -ClassName Win32_OperatingSystem).OSArchitecture
|
||||
$my_os_arch = wmic os get osarchitecture
|
||||
|
||||
# Using -clike because of the trailing newline
|
||||
if ($my_os_arch -clike "ARM 64*") {
|
||||
IF ($my_os_arch -clike "ARM 64*") {
|
||||
$my_arch = "ARM64"
|
||||
}
|
||||
}
|
||||
@@ -124,7 +124,7 @@ function webi_path_add($pathname) {
|
||||
$exists_in_path = $true
|
||||
}
|
||||
}
|
||||
if (-not $exists_in_path) {
|
||||
if (-Not $exists_in_path) {
|
||||
$all_user_paths = "${pathname};${all_user_paths}".Trim(';')
|
||||
[Environment]::SetEnvironmentVariable("Path", $all_user_paths, "User")
|
||||
$null = Sync-EnvPath
|
||||
|
||||
@@ -27,7 +27,6 @@ __bootstrap_webi() {
|
||||
#PKG_LIBCS=
|
||||
#PKG_FORMATS=
|
||||
#PKG_LATEST=
|
||||
#PKG_STABLE=
|
||||
WEBI_PKG_DOWNLOAD=""
|
||||
WEBI_DOWNLOAD_DIR="${HOME}/Downloads"
|
||||
if command -v xdg-user-dir > /dev/null; then
|
||||
@@ -37,7 +36,7 @@ __bootstrap_webi() {
|
||||
fi
|
||||
fi
|
||||
|
||||
WEBI_PKG_PATH="${WEBI_DOWNLOAD_DIR}/webi/${PKG_NAME:-error}/${WEBI_VERSION:-stable}"
|
||||
WEBI_PKG_PATH="${WEBI_DOWNLOAD_DIR}/webi/${PKG_NAME:-error}/${WEBI_VERSION:-latest}"
|
||||
|
||||
# get the special formatted version
|
||||
# (i.e. "go is go1.14" while node is "node v12.10.8")
|
||||
@@ -128,10 +127,7 @@ __bootstrap_webi() {
|
||||
echo ""
|
||||
echo " $(t_err "Error: no '${PKG_NAME:-"Unknown Package"}@${WEBI_TAG:-"Unknown Tag"}' release for '${WEBI_OS:-"Unknown OS"}' (${WEBI_LIBC:-"Unknown Libc"}) on '${WEBI_ARCH:-"Unknown CPU"}' as one of '${WEBI_FORMATS:-"Unknown File Type"}'")"
|
||||
echo ""
|
||||
echo " Latest Stable: ${PKG_STABLE}"
|
||||
if test "${PKG_LATEST}" != "${PKG_STABLE}"; then
|
||||
echo " Next Version: ${PKG_LATEST}"
|
||||
fi
|
||||
echo " Latest Version: ${PKG_LATEST}"
|
||||
echo " CPUs: $PKG_ARCHES"
|
||||
echo " OSes: $PKG_OSES"
|
||||
echo " libcs: $PKG_LIBCS"
|
||||
|
||||
@@ -70,20 +70,18 @@ InstallerServer.helper = async function ({
|
||||
|
||||
console.log(`dbg: Get Project Installer Type for '${projectName}':`);
|
||||
let proj = await Builds.getProjectType(projectName);
|
||||
if (proj.type === 'alias') {
|
||||
console.log(`dbg: alias`, proj);
|
||||
projectName = proj.detail;
|
||||
proj = await Builds.getProjectType(projectName); // an alias should never resolve to an alias
|
||||
}
|
||||
console.log(`dbg: proj`, proj);
|
||||
console.log(proj);
|
||||
|
||||
let validTypes = ['selfhosted', 'valid'];
|
||||
let validTypes = ['alias', 'selfhosted', 'valid'];
|
||||
if (!validTypes.includes(proj.type)) {
|
||||
let msg = `'${projectName}' doesn't have an installer: '${proj.type}': '${proj.detail}'`;
|
||||
let err = new Error(msg);
|
||||
err.code = 'ENOENT';
|
||||
throw err;
|
||||
}
|
||||
if (proj.type === 'alias') {
|
||||
projectName = proj.detail;
|
||||
}
|
||||
|
||||
let tmplParams = {
|
||||
pkg: projectName,
|
||||
@@ -122,7 +120,8 @@ InstallerServer.helper = async function ({
|
||||
name: projectName,
|
||||
date: new Date(),
|
||||
});
|
||||
let latestVersions = Builds.enumerateLatestVersions(projInfo);
|
||||
let latest = projInfo.versions[0];
|
||||
Object.assign(tmplParams, { latest });
|
||||
//console.log('projInfo', projInfo);
|
||||
|
||||
let buildTargetInfo = {
|
||||
@@ -131,27 +130,12 @@ InstallerServer.helper = async function ({
|
||||
arches: projInfo.arches,
|
||||
libcs: projInfo.libcs,
|
||||
formats: projInfo.formats,
|
||||
latest: latestVersions.latest,
|
||||
stable: latestVersions.stable,
|
||||
};
|
||||
|
||||
// TODO .findMatchingPackages() should probably account for this
|
||||
let hasOs = projInfo.oses.includes(hostTarget.os);
|
||||
let maybePosix = !hasOs && hostTarget.os !== 'windows';
|
||||
if (maybePosix) {
|
||||
let posixes = ['posix_2017', 'posix_2024'];
|
||||
for (let posixYear of posixes) {
|
||||
let hasPosix = projInfo.oses.includes(posixYear);
|
||||
if (hasPosix) {
|
||||
hasOs = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!hasOs) {
|
||||
hasOs = projInfo.oses.includes('ANYOS');
|
||||
}
|
||||
|
||||
if (!hasOs) {
|
||||
let pkg1 = Object.assign(buildTargetInfo, errPackage);
|
||||
return [pkg1, tmplParams];
|
||||
|
||||
@@ -49,7 +49,7 @@ var baseurl = 'https://webinstall.dev';
|
||||
var maxLen = 0;
|
||||
console.info('');
|
||||
console.info('Has the necessary files?');
|
||||
['README.md', 'install.sh', 'install.ps1']
|
||||
['README.md', 'releases.js', 'install.sh', 'install.ps1']
|
||||
.map(function (node) {
|
||||
maxLen = Math.max(maxLen, node.length);
|
||||
return node;
|
||||
@@ -71,14 +71,6 @@ Builds.getPackage({ name: projName }).then(async function (/*projInfo*/) {
|
||||
var nodeOs = os.platform();
|
||||
var nodeOsRelease = os.release();
|
||||
var nodeArch = os.arch();
|
||||
|
||||
// To make arch names compatible across all helpers
|
||||
if (nodeArch === 'x64') {
|
||||
nodeArch = 'amd64';
|
||||
} else if (nodeArch === 'arm64') {
|
||||
nodeArch = 'arm64';
|
||||
}
|
||||
|
||||
var nodeLibc = 'libc';
|
||||
if (process.platform === 'linux') {
|
||||
nodeLibc = 'gnu';
|
||||
|
||||
@@ -2,30 +2,72 @@
|
||||
|
||||
var Releases = module.exports;
|
||||
|
||||
var Fs = require('node:fs/promises');
|
||||
var Os = require('node:os');
|
||||
var path = require('path');
|
||||
var request = require('@root/request');
|
||||
var _normalize = require('./normalize.js');
|
||||
|
||||
var cache = {};
|
||||
//var staleAge = 5 * 1000;
|
||||
//var expiredAge = 15 * 1000;
|
||||
var staleAge = 5 * 60 * 1000;
|
||||
var expiredAge = 15 * 60 * 1000;
|
||||
|
||||
var LEGACY_CACHE_DIR = path.join(Os.homedir(), '.cache/webi/legacy');
|
||||
let installerDir = path.join(__dirname, '..');
|
||||
|
||||
// Sort releases by ext preference and libc within the same version.
|
||||
// The cache is already sorted by version (stable before beta, newest first),
|
||||
// so we only re-order within the same version string.
|
||||
Releases.get = async function (pkgdir) {
|
||||
let get;
|
||||
try {
|
||||
get = require(path.join(pkgdir, 'releases.js'));
|
||||
} catch (e) {
|
||||
let err = new Error('no releases.js for', pkgdir.split(/[\/\\]+/).pop());
|
||||
err.code = 'E_NO_RELEASE';
|
||||
throw err;
|
||||
}
|
||||
|
||||
let all = await get(request);
|
||||
|
||||
return _normalize(all);
|
||||
};
|
||||
|
||||
// TODO needs a proper test, and more accurate (though perhaps far less simple) code
|
||||
function createFormatsSorter(formats) {
|
||||
return function sortByExtLibc(a, b) {
|
||||
if (a.version !== b.version) {
|
||||
// Array.sort is stable (V8, ES2019), so returning 0 across
|
||||
// versions preserves the cache's pre-sorted version-desc order.
|
||||
return 0;
|
||||
return function sortByVerExt(a, b) {
|
||||
function lexver(semver) {
|
||||
// v1.20.156 => 00001.00020.00156.zzzzz
|
||||
// TODO BUG: v1.20.156-rc2 => 00001.00020.00156.rc2zz
|
||||
var parts = semver.split(/[+\.\-]/g);
|
||||
while (parts.length < 4) {
|
||||
parts.push('');
|
||||
}
|
||||
return parts
|
||||
.map(function (num, i) {
|
||||
if (3 === i) {
|
||||
return num.toString().padEnd(10, 'z');
|
||||
}
|
||||
return num.toString().padStart(10, '0');
|
||||
})
|
||||
.join('.');
|
||||
}
|
||||
|
||||
var aver = lexver(a.version);
|
||||
var bver = lexver(b.version);
|
||||
if (aver > bver) {
|
||||
//console.log(aver, '>', bver);
|
||||
return -1;
|
||||
}
|
||||
if (aver < bver) {
|
||||
//console.log(aver, '<', bver);
|
||||
return 1;
|
||||
}
|
||||
|
||||
var aExtPri = formats.indexOf(a.ext.replace(/tar\..*/, 'tar'));
|
||||
var bExtPri = formats.indexOf(b.ext.replace(/tar\..*/, 'tar'));
|
||||
if (aExtPri > bExtPri) {
|
||||
//console.log(a.ext, aExtPri, '>', b.ext, bExtPri);
|
||||
return -1;
|
||||
}
|
||||
if (aExtPri < bExtPri) {
|
||||
//console.log(a.ext, aExtPri, '<', b.ext, bExtPri);
|
||||
return 1;
|
||||
}
|
||||
|
||||
@@ -42,39 +84,99 @@ function createFormatsSorter(formats) {
|
||||
}
|
||||
|
||||
async function getCachedReleases(pkg) {
|
||||
// returns { download: '', releases: [{ version, date, os, arch, lts, channel, download}] }
|
||||
// returns { download: '<template string>', releases: [{ version, date, os, arch, lts, channel, download}] }
|
||||
|
||||
if (cache[pkg]) {
|
||||
return cache[pkg];
|
||||
async function chainCachePromise(fn) {
|
||||
cache[pkg].promise = cache[pkg].promise.then(fn);
|
||||
return cache[pkg].promise;
|
||||
}
|
||||
|
||||
let dataFile = `${LEGACY_CACHE_DIR}/${pkg}.json`;
|
||||
async function sleep(ms) {
|
||||
return await new Promise(function (resolve, reject) {
|
||||
setTimeout(resolve, ms);
|
||||
});
|
||||
}
|
||||
|
||||
let json = await Fs.readFile(dataFile, 'utf8').catch(function (err) {
|
||||
if (err.code === 'ENOENT') {
|
||||
return null;
|
||||
async function putCache() {
|
||||
var age = Date.now() - cache[pkg].updatedAt;
|
||||
if (age < staleAge) {
|
||||
//console.debug('NOT STALE ANYMORE - updated in previous promise');
|
||||
return cache[pkg].all;
|
||||
}
|
||||
throw err;
|
||||
});
|
||||
|
||||
if (!json) {
|
||||
let empty = { download: '', releases: [] };
|
||||
cache[pkg] = empty;
|
||||
return empty;
|
||||
//console.debug('DOWNLOADING NEW "%s" releases', pkg);
|
||||
var pkgdir = path.join(installerDir, pkg);
|
||||
|
||||
// workaround for request timeout seeming to not work
|
||||
let complete = false;
|
||||
await Promise.race([
|
||||
Releases.get(pkgdir)
|
||||
.catch(function (err) {
|
||||
if ('E_NO_RELEASE' === err.code) {
|
||||
let all = { _error: 'E_NO_RELEASE', download: '', releases: [] };
|
||||
return all;
|
||||
}
|
||||
|
||||
throw err;
|
||||
})
|
||||
.catch(function (err) {
|
||||
let hasReleases = cache[pkg].all?.releases?.length > 1;
|
||||
if (!hasReleases) {
|
||||
throw err;
|
||||
}
|
||||
|
||||
console.error(`Error: the BOOGEYMAN got us!`);
|
||||
console.error(err.stack);
|
||||
|
||||
return cache[pkg].all;
|
||||
})
|
||||
.then(function (all) {
|
||||
// Note: it is possible for slightly older data
|
||||
// to replace slightly newer data, but this is better
|
||||
// than being in a cycle where release updates _always_
|
||||
// take longer than expected.
|
||||
//console.debug('DOWNLOADED NEW "%s" releases', pkg);
|
||||
cache[pkg].updatedAt = Date.now();
|
||||
cache[pkg].all = all;
|
||||
complete = true;
|
||||
}),
|
||||
sleep(15000).then(function () {
|
||||
if (complete) {
|
||||
return;
|
||||
}
|
||||
console.error(`request timeout waiting for '${pkg}' release info`);
|
||||
}),
|
||||
]);
|
||||
|
||||
return cache[pkg].all;
|
||||
}
|
||||
|
||||
let all;
|
||||
try {
|
||||
all = JSON.parse(json);
|
||||
} catch (e) {
|
||||
console.error(`error: ${dataFile}:\n\t${e.message}`);
|
||||
let empty = { download: '', releases: [] };
|
||||
cache[pkg] = empty;
|
||||
return empty;
|
||||
if (!cache[pkg]) {
|
||||
cache[pkg] = {
|
||||
updatedAt: 0,
|
||||
all: { download: '', releases: [] },
|
||||
promise: Promise.resolve(),
|
||||
};
|
||||
}
|
||||
|
||||
cache[pkg] = all;
|
||||
return all;
|
||||
var bgRenewal;
|
||||
var age = Date.now() - cache[pkg].updatedAt;
|
||||
var fresh = age < staleAge;
|
||||
if (!fresh) {
|
||||
bgRenewal = chainCachePromise(putCache);
|
||||
}
|
||||
|
||||
var tooStale = age > expiredAge;
|
||||
if (!tooStale) {
|
||||
return await cache[pkg].all;
|
||||
}
|
||||
|
||||
return await Promise.race([
|
||||
bgRenewal,
|
||||
sleep(5000).then(function () {
|
||||
return cache[pkg].all;
|
||||
}),
|
||||
]);
|
||||
}
|
||||
|
||||
async function filterReleases(
|
||||
@@ -85,22 +187,15 @@ async function filterReleases(
|
||||
// sort the most compatible format first
|
||||
// (i.e. so that we don't do .pkg on linux except on purpose)
|
||||
var rformats = formats.slice(0).reverse();
|
||||
var sortByExtLibc = createFormatsSorter(rformats);
|
||||
var sortByVerExt = createFormatsSorter(rformats);
|
||||
var reVer = new RegExp('^' + ver + '\\b');
|
||||
|
||||
function selectMatches(rel) {
|
||||
/* jshint maxcomplexity: 25 */
|
||||
if (os) {
|
||||
// '*' = any OS (matches anything, including windows).
|
||||
// 'posix' / 'posix_20xx' = any POSIX OS (matches linux, macos,
|
||||
// freebsd, etc., but NOT windows).
|
||||
let isPosix = rel.os === 'posix' || rel.os.startsWith('posix_20');
|
||||
let osMatches =
|
||||
rel.os === '*' ||
|
||||
rel.os === os ||
|
||||
(isPosix && os !== 'windows');
|
||||
if (!osMatches) {
|
||||
return false;
|
||||
if (rel.os !== '*') {
|
||||
if (rel.os !== os) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -112,10 +207,7 @@ async function filterReleases(
|
||||
}
|
||||
}
|
||||
|
||||
// libc='libc' is serve-releases.js's default when the caller
|
||||
// didn't pin one — treat it as 'no preference', not a filter.
|
||||
let isMeaningfulLibc = libc && libc !== 'libc' && rel.libc !== 'none';
|
||||
if (isMeaningfulLibc) {
|
||||
if (rel.libc !== 'none') {
|
||||
let releaseRequiresMusl = rel.libc === 'musl';
|
||||
// goal: handle non-glibc (Alpine / Docker / musl)
|
||||
let osHasMusl = libc === 'musl';
|
||||
@@ -162,7 +254,7 @@ async function filterReleases(
|
||||
return true;
|
||||
}
|
||||
|
||||
var sortedRels = all.releases.filter(selectMatches).sort(sortByExtLibc);
|
||||
var sortedRels = all.releases.filter(selectMatches).sort(sortByVerExt);
|
||||
//console.log(sortedRels.slice(0, 4));
|
||||
|
||||
return sortedRels.slice(0, limit || 1000);
|
||||
@@ -321,8 +413,8 @@ Releases.getReleases = function ({
|
||||
};
|
||||
|
||||
if (require.main === module) {
|
||||
return Releases
|
||||
.getReleases({
|
||||
return module
|
||||
.exports({
|
||||
pkg: 'node',
|
||||
ver: '',
|
||||
os: 'macos',
|
||||
|
||||
@@ -38,8 +38,9 @@ function getOs(ua) {
|
||||
// It's the year of the Linux Desktop!
|
||||
// See also http://www.mslinux.org/
|
||||
// 'linux' must be tested before 'Microsoft' because WSL
|
||||
// (TODO: does this affect cygwin / msysgit?)
|
||||
return 'linux';
|
||||
} else if (/^ms$|Microsoft|Windows|win32|win|PowerShell|CYGWIN|MINGW/i.test(ua)) {
|
||||
} else if (/^ms$|Microsoft|Windows|win32|win|PowerShell/i.test(ua)) {
|
||||
// 'win' must be tested after 'darwin'
|
||||
return 'windows';
|
||||
} else if (/Linux|curl|wget/i.test(ua)) {
|
||||
|
||||
@@ -72,8 +72,8 @@ gc "feat: new feature"
|
||||
|
||||
### Common aliases
|
||||
|
||||
Use *alias*es to make other tools you find around webi even _more_ convenient ⚡️
|
||||
(and powerful 💪).
|
||||
Use *alias*es to make other tools you find around webi even _more_ convenient
|
||||
⚡️ (and powerful 💪).
|
||||
|
||||
```sh
|
||||
aliasman curl 'curlie'
|
||||
|
||||
@@ -1,2 +0,0 @@
|
||||
github_sources = BeyondCodeBootcamp/aliasman
|
||||
git_url = https://github.com/BeyondCodeBootcamp/aliasman.git
|
||||
30
aliasman/releases.js
Normal file
30
aliasman/releases.js
Normal file
@@ -0,0 +1,30 @@
|
||||
'use strict';
|
||||
|
||||
var githubSource = require('../_common/github-source.js');
|
||||
var owner = 'BeyondCodeBootcamp';
|
||||
var repo = 'aliasman';
|
||||
|
||||
module.exports = function (request) {
|
||||
let arches = [
|
||||
'amd64',
|
||||
'arm64',
|
||||
'armv6l',
|
||||
'armv7l',
|
||||
'ppc64le',
|
||||
'ppc64',
|
||||
's390x',
|
||||
'x86',
|
||||
];
|
||||
let oses = ['freebsd', 'linux', 'macos', 'posix'];
|
||||
return githubSource(request, owner, repo, oses, arches).then(function (all) {
|
||||
all._names = ['aliasman', 'legacy'];
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading archiver from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing archiver"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = mholt/archiver
|
||||
21
arc/releases.js
Normal file
21
arc/releases.js
Normal file
@@ -0,0 +1,21 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'mholt';
|
||||
var repo = 'archiver';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
all._names = ['archiver', 'arc'];
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/bin/pwsh
|
||||
|
||||
Write-Output "'archiver@$Env:WEBI_TAG' is an alias for 'arc@$Env:WEBI_VERSION'"
|
||||
if ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
IF ($null -eq $Env:WEBI_HOST -or $Env:WEBI_HOST -eq "") { $Env:WEBI_HOST = "https://webinstall.dev" }
|
||||
curl.exe -A MS -fsSL "$Env:WEBI_HOST/arc@$Env:WEBI_VERSION" | powershell
|
||||
|
||||
@@ -20,7 +20,7 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Checking for (or Installing) MSVC Runtime..."
|
||||
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
|
||||
|
||||
@@ -29,7 +29,7 @@ if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing AtomicParsley"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = wez/atomicparsley
|
||||
79
atomicparsley/releases.js
Normal file
79
atomicparsley/releases.js
Normal file
@@ -0,0 +1,79 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'wez';
|
||||
var repo = 'atomicparsley';
|
||||
|
||||
let targets = {
|
||||
x86win: {
|
||||
os: 'windows',
|
||||
arch: 'x86',
|
||||
libc: 'msvc',
|
||||
},
|
||||
x64win: {
|
||||
os: 'windows',
|
||||
arch: 'amd64',
|
||||
// https://github.com/wez/atomicparsley/issues/6#issuecomment-1364523028
|
||||
libc: 'msvc',
|
||||
},
|
||||
x64mac: {
|
||||
os: 'macos',
|
||||
arch: 'amd64',
|
||||
},
|
||||
x64lin: {
|
||||
os: 'linux',
|
||||
arch: 'amd64',
|
||||
libc: 'gnu',
|
||||
},
|
||||
x64musl: {
|
||||
os: 'linux',
|
||||
arch: 'amd64',
|
||||
libc: 'musl',
|
||||
},
|
||||
};
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
for (let rel of all.releases) {
|
||||
let windows32 = rel.name.includes('WindowsX86.');
|
||||
if (windows32) {
|
||||
Object.assign(rel, targets.x86win);
|
||||
continue;
|
||||
}
|
||||
|
||||
let windows64 = rel.name.includes('Windows.');
|
||||
if (windows64) {
|
||||
Object.assign(rel, targets.x64win);
|
||||
continue;
|
||||
}
|
||||
|
||||
let macos64 = rel.name.includes('MacOS');
|
||||
if (macos64) {
|
||||
Object.assign(rel, targets.x64mac);
|
||||
continue;
|
||||
}
|
||||
|
||||
let musl64 = rel.name.includes('Alpine');
|
||||
if (musl64) {
|
||||
Object.assign(rel, targets.x64musl);
|
||||
continue;
|
||||
}
|
||||
|
||||
let lin64 = rel.name.includes('Linux.');
|
||||
if (lin64) {
|
||||
Object.assign(rel, targets.x64lin);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
all._names = ['AtomicParsley', 'atomicparsley'];
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all));
|
||||
//console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading awless from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing awless"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = wallix/awless
|
||||
22
awless/releases.js
Normal file
22
awless/releases.js
Normal file
@@ -0,0 +1,22 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'wallix';
|
||||
var repo = 'awless';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
// remove checksums and .deb
|
||||
all.releases = all.releases.filter(function (rel) {
|
||||
return !/(\.txt)|(\.deb)$/i.test(rel.name);
|
||||
});
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all));
|
||||
});
|
||||
}
|
||||
@@ -1,128 +0,0 @@
|
||||
---
|
||||
title: basecamp-cli
|
||||
homepage: https://github.com/basecamp/basecamp-cli
|
||||
tagline: |
|
||||
basecamp: CLI for Basecamp 3 — manage projects, todos, messages, cards, and more from the terminal.
|
||||
---
|
||||
|
||||
To update or switch versions, run `webi basecamp-cli@stable` (or `@v0.7`,
|
||||
`@beta`, etc).
|
||||
|
||||
### Files
|
||||
|
||||
These are the files / directories that are created and/or modified with this
|
||||
install:
|
||||
|
||||
```text
|
||||
~/.config/envman/PATH.env
|
||||
~/.local/bin/basecamp
|
||||
~/.local/opt/basecamp-cli-VERSION/bin/basecamp
|
||||
~/.local/opt/basecamp-cli-VERSION/completions/
|
||||
```
|
||||
|
||||
## Cheat Sheet
|
||||
|
||||
> `basecamp` is the official CLI for Basecamp 3. It provides full API coverage
|
||||
> for projects, todos, messages, cards, schedule, files, and more — all from the
|
||||
> command line.
|
||||
|
||||
### How to authenticate
|
||||
|
||||
```sh
|
||||
basecamp auth login
|
||||
```
|
||||
|
||||
For headless environments (CI, remote servers):
|
||||
|
||||
```sh
|
||||
basecamp auth login --device-code
|
||||
```
|
||||
|
||||
Check auth status:
|
||||
|
||||
```sh
|
||||
basecamp auth status
|
||||
```
|
||||
|
||||
### How to list projects and todos
|
||||
|
||||
```sh
|
||||
basecamp projects list --md
|
||||
|
||||
basecamp todos list --assignee me --in PROJECT_ID --md
|
||||
```
|
||||
|
||||
Cross-project view of your assigned work:
|
||||
|
||||
```sh
|
||||
basecamp assignments --md
|
||||
```
|
||||
|
||||
### How to create and complete todos
|
||||
|
||||
```sh
|
||||
basecamp todo "Write release notes" --in PROJECT_ID --list TODOLIST_ID --assignee me --due tomorrow
|
||||
|
||||
basecamp done TODO_ID
|
||||
```
|
||||
|
||||
### How to post a message or comment
|
||||
|
||||
```sh
|
||||
basecamp message "Sprint Update" "Shipped v2.1 to production." --in PROJECT_ID
|
||||
|
||||
basecamp comment RECORDING_ID "Looks good." --in PROJECT_ID
|
||||
```
|
||||
|
||||
### How to move cards through a workflow
|
||||
|
||||
```sh
|
||||
basecamp cards columns --in PROJECT_ID --md
|
||||
|
||||
basecamp cards move CARD_ID --to COLUMN_ID --in PROJECT_ID
|
||||
```
|
||||
|
||||
### How to set up per-project defaults
|
||||
|
||||
Create `.basecamp/config.json` in your repo (commit it):
|
||||
|
||||
```json
|
||||
{
|
||||
"project_id": "12345",
|
||||
"todolist_id": "67890"
|
||||
}
|
||||
```
|
||||
|
||||
Then trust it once:
|
||||
|
||||
```sh
|
||||
basecamp config trust
|
||||
```
|
||||
|
||||
After that, you can omit `--in` for most commands in that repo.
|
||||
|
||||
### Shell completions
|
||||
|
||||
Completions for bash, fish, and zsh ship with the installer. Find them at:
|
||||
|
||||
```text
|
||||
~/.local/opt/basecamp-cli-VERSION/completions/
|
||||
```
|
||||
|
||||
Bash:
|
||||
|
||||
```sh
|
||||
echo "source ~/.local/opt/basecamp-cli-VERSION/completions/basecamp.bash" >> ~/.bashrc
|
||||
```
|
||||
|
||||
Fish:
|
||||
|
||||
```sh
|
||||
ln -s ~/.local/opt/basecamp-cli-VERSION/completions/basecamp.fish ~/.config/fish/completions/
|
||||
```
|
||||
|
||||
Zsh:
|
||||
|
||||
```sh
|
||||
echo "fpath+=( ~/.local/opt/basecamp-cli-VERSION/completions )" >> ~/.zshrc
|
||||
```
|
||||
@@ -1,46 +0,0 @@
|
||||
#!/usr/bin/env pwsh
|
||||
|
||||
$pkg_cmd_name = "basecamp"
|
||||
|
||||
$pkg_dst_cmd = "$Env:USERPROFILE\.local\bin\basecamp.exe"
|
||||
$pkg_dst = "$pkg_dst_cmd"
|
||||
|
||||
$pkg_src_cmd = "$Env:USERPROFILE\.local\opt\basecamp-cli-v$Env:WEBI_VERSION\bin\basecamp.exe"
|
||||
$pkg_src_bin = "$Env:USERPROFILE\.local\opt\basecamp-cli-v$Env:WEBI_VERSION\bin"
|
||||
$pkg_src_dir = "$Env:USERPROFILE\.local\opt\basecamp-cli-v$Env:WEBI_VERSION"
|
||||
$pkg_src = "$pkg_src_cmd"
|
||||
|
||||
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading basecamp from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing basecamp"
|
||||
|
||||
Push-Location .local\tmp
|
||||
|
||||
Remove-Item -Path ".\basecamp-*" -Recurse -ErrorAction Ignore
|
||||
Remove-Item -Path ".\basecamp.exe" -Recurse -ErrorAction Ignore
|
||||
|
||||
Write-Output "Unpacking $pkg_download"
|
||||
& tar xf "$pkg_download"
|
||||
|
||||
New-Item "$pkg_src_bin" -ItemType Directory -Force | Out-Null
|
||||
Move-Item -Path ".\basecamp.exe" -Destination "$pkg_src_bin"
|
||||
|
||||
New-Item "$pkg_src_dir\completions" -ItemType Directory -Force | Out-Null
|
||||
if (Test-Path -Path ".\completions") {
|
||||
Copy-Item -Path ".\completions\*" -Destination "$pkg_src_dir\completions" -Recurse
|
||||
}
|
||||
|
||||
Pop-Location
|
||||
}
|
||||
|
||||
Write-Output "Copying into '$pkg_dst_cmd' from '$pkg_src_cmd'"
|
||||
Remove-Item -Path "$pkg_dst_cmd" -Recurse -ErrorAction Ignore | Out-Null
|
||||
Copy-Item -Path "$pkg_src" -Destination "$pkg_dst" -Recurse
|
||||
@@ -1,44 +0,0 @@
|
||||
#!/bin/sh
|
||||
# shellcheck disable=SC2034
|
||||
|
||||
set -e
|
||||
set -u
|
||||
|
||||
__init_basecamp() {
|
||||
pkg_cmd_name="basecamp"
|
||||
|
||||
pkg_src_dir="$HOME/.local/opt/basecamp-cli-v$WEBI_VERSION"
|
||||
pkg_src_cmd="$pkg_src_dir/bin/basecamp"
|
||||
pkg_src="$pkg_src_cmd"
|
||||
|
||||
pkg_dst_cmd="$HOME/.local/bin/basecamp"
|
||||
pkg_dst="$pkg_dst_cmd"
|
||||
|
||||
pkg_install() {
|
||||
mkdir -p "$(dirname "$pkg_src_cmd")"
|
||||
mkdir -p "$pkg_src_dir/completions"
|
||||
|
||||
if test -f ./basecamp; then
|
||||
mv ./basecamp "$pkg_src_cmd"
|
||||
elif test -e ./basecamp-*/basecamp; then
|
||||
mv ./basecamp-*/basecamp "$pkg_src_cmd"
|
||||
elif test -e ./basecamp-*; then
|
||||
mv ./basecamp-* "$pkg_src_cmd"
|
||||
else
|
||||
echo >&2 "failed to find 'basecamp' executable"
|
||||
return 1
|
||||
fi
|
||||
|
||||
if test -d ./completions; then
|
||||
cp -a ./completions/. "$pkg_src_dir/completions/"
|
||||
fi
|
||||
}
|
||||
|
||||
pkg_get_current_version() {
|
||||
basecamp --version 2> /dev/null |
|
||||
head -n 1 |
|
||||
cut -d' ' -f3
|
||||
}
|
||||
}
|
||||
|
||||
__init_basecamp
|
||||
@@ -1,2 +0,0 @@
|
||||
github_releases = basecamp/basecamp-cli
|
||||
exclude = .deb .rpm .apk .sbom.json .pem .sig .bundle checksums.txt
|
||||
@@ -3,7 +3,7 @@
|
||||
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
|
||||
$EXENAME = "$Env:PKG_NAME.exe"
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
|
||||
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
@@ -11,11 +11,11 @@ if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
|
||||
# Fetch MSVC Runtime
|
||||
Write-Output "Checking for MSVC Runtime..."
|
||||
if (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
|
||||
IF (-not (Test-Path "\Windows\System32\vcruntime140.dll")) {
|
||||
& "$Env:USERPROFILE\.local\bin\webi-pwsh.ps1" vcruntime
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
Write-Output "Installing $Env:PKG_NAME"
|
||||
# TODO: temp directory
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = sharkdp/bat
|
||||
20
bat/releases.js
Normal file
20
bat/releases.js
Normal file
@@ -0,0 +1,20 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'sharkdp';
|
||||
var repo = 'bat';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
all.releases = all.releases.slice(0, 10);
|
||||
//console.info(JSON.stringify(all));
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -7,13 +7,13 @@ main() { (
|
||||
chmod a+x ~/.local/bin/brew-update-hourly
|
||||
|
||||
echo "Checking for serviceman..."
|
||||
~/.local/bin/webi serviceman
|
||||
if ! command -v serviceman > /dev/null; then
|
||||
"$HOME/.local/bin/webi" serviceman
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
serviceman --version
|
||||
fi
|
||||
serviceman --version
|
||||
|
||||
serviceman add --agent \
|
||||
env PATH="$PATH" serviceman add --user \
|
||||
--workdir ~/.local/opt/brew/ \
|
||||
--name sh.brew.updater -- \
|
||||
~/.local/bin/brew-update-hourly
|
||||
|
||||
188
brew/install.sh
188
brew/install.sh
@@ -4,143 +4,73 @@ set -e
|
||||
set -u
|
||||
|
||||
_install_brew() {
|
||||
# Straight from https://brew.sh
|
||||
#/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
|
||||
# Straight from https://brew.sh
|
||||
#/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
|
||||
|
||||
if test "Darwin" = "$(uname -s)"; then
|
||||
needs_xcode="$(/usr/bin/xcode-select -p > /dev/null 2> /dev/null || echo "true")"
|
||||
if test -n "${needs_xcode}"; then
|
||||
echo ""
|
||||
echo ""
|
||||
echo "ERROR: Run this command to install XCode Command Line Tools first:"
|
||||
echo ""
|
||||
echo " xcode-select --install"
|
||||
echo ""
|
||||
echo "After the install, close this terminal, open a new one, and try again."
|
||||
echo ""
|
||||
fi
|
||||
else
|
||||
if ! command -v gcc > /dev/null; then
|
||||
echo >&2 "Warning: to install 'gcc' et al on Linux use the built-in package manager."
|
||||
echo >&2 " For example, try: sudo apt install -y build-essential"
|
||||
fi
|
||||
if ! command -v git > /dev/null; then
|
||||
echo >&2 "Error: to install 'git' on Linux use the built-in package manager."
|
||||
echo >&2 " For example, try: sudo apt install -y git"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
if test "Darwin" = "$(uname -s)"; then
|
||||
needs_xcode="$(/usr/bin/xcode-select -p > /dev/null 2> /dev/null || echo "true")"
|
||||
if test -n "${needs_xcode}"; then
|
||||
echo ""
|
||||
echo ""
|
||||
echo "ERROR: Run this command to install XCode Command Line Tools first:"
|
||||
echo ""
|
||||
echo " xcode-select --install"
|
||||
echo ""
|
||||
echo "After the install, close this terminal, open a new one, and try again."
|
||||
echo ""
|
||||
fi
|
||||
else
|
||||
if ! command -v gcc > /dev/null; then
|
||||
echo >&2 "Warning: to install 'gcc' et al on Linux use the built-in package manager."
|
||||
echo >&2 " For example, try: sudo apt install -y build-essential"
|
||||
fi
|
||||
if ! command -v git > /dev/null; then
|
||||
echo >&2 "Error: to install 'git' on Linux use the built-in package manager."
|
||||
echo >&2 " For example, try: sudo apt install -y git"
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# From Straight from https://brew.sh
|
||||
if ! test -d "$HOME/.local/opt/brew"; then
|
||||
echo "Installing to '$HOME/.local/opt/brew'"
|
||||
echo ""
|
||||
echo "If you prefer to have brew installed to '/usr/local' cancel now and do the following:"
|
||||
echo " rm -rf '$HOME/.local/opt/brew'"
|
||||
# shellcheck disable=2016
|
||||
echo ' /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"'
|
||||
echo ""
|
||||
sleep 3
|
||||
#mkdir -p "$HOME/.local/opt/brew"
|
||||
#curl -fsSL https://github.com/Homebrew/brew/tarball/main |
|
||||
# tar xz --strip-components 1 -C "$HOME/.local/opt/brew"
|
||||
git clone --depth 3 https://github.com/Homebrew/brew "$HOME/.local/opt/brew"
|
||||
fi
|
||||
# From Straight from https://brew.sh
|
||||
if ! test -d "$HOME/.local/opt/brew"; then
|
||||
echo "Installing to '$HOME/.local/opt/brew'"
|
||||
echo ""
|
||||
echo "If you prefer to have brew installed to '/usr/local' cancel now and do the following:"
|
||||
echo " rm -rf '$HOME/.local/opt/brew'"
|
||||
# shellcheck disable=2016
|
||||
echo ' /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"'
|
||||
echo ""
|
||||
sleep 3
|
||||
git clone --depth=1 https://github.com/Homebrew/brew "$HOME/.local/opt/brew"
|
||||
fi
|
||||
|
||||
my_shellenv="$("$HOME/.local/opt/brew/bin/brew" shellenv)"
|
||||
eval "${my_shellenv}"
|
||||
chmod -R go-w "$(brew --prefix)/share/zsh" 2> /dev/null || true
|
||||
rm -rf "$HOME/.local/bin/brew-update-service-install"
|
||||
webi_download \
|
||||
"$WEBI_HOST/packages/brew/brew-update-service-install" \
|
||||
"$HOME/.local/bin/brew-update-service-install" \
|
||||
brew-update-service-install
|
||||
chmod a+x "$HOME/.local/bin/brew-update-service-install"
|
||||
|
||||
rm -rf "$HOME/.local/bin/brew-update-service-install"
|
||||
webi_download \
|
||||
"$WEBI_HOST/packages/brew/brew-update-service-install" \
|
||||
"$HOME/.local/bin/brew-update-service-install" \
|
||||
brew-update-service-install
|
||||
chmod a+x "$HOME/.local/bin/brew-update-service-install"
|
||||
webi_path_add "$HOME/.local/opt/brew/bin"
|
||||
export PATH="$HOME/.local/opt/brew/bin:$PATH"
|
||||
|
||||
webi_path_add "$HOME/.local/opt/brew/bin"
|
||||
webi_path_add "$HOME/.local/opt/brew/sbin"
|
||||
echo "Updating brew..."
|
||||
brew update
|
||||
|
||||
fn_brew_shell_integrate_bash
|
||||
fn_brew_shell_integrate_zsh
|
||||
fn_brew_shell_integrate_fish
|
||||
webi_path_add "$HOME/.local/opt/brew/sbin"
|
||||
export PATH="$HOME/.local/opt/brew/sbin:$PATH"
|
||||
|
||||
echo "Installed 'brew' to '$HOME/.local/opt/brew'"
|
||||
echo ""
|
||||
echo "If you prefer to have brew installed to '/usr/local' do the following:"
|
||||
echo " mv '$HOME/.local/opt/brew' '$HOME/.local/opt/brew.$(date '+%F_%H-%M-%S').bak'"
|
||||
# shellcheck disable=2016
|
||||
echo ' /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"'
|
||||
echo ""
|
||||
echo "Installed 'brew' to '$HOME/.local/opt/brew'"
|
||||
echo ""
|
||||
echo "If you prefer to have brew installed to '/usr/local' do the following:"
|
||||
echo " mv '$HOME/.local/opt/brew' '$HOME/.local/opt/brew.$(date '+%F_%H-%M-%S').bak'"
|
||||
# shellcheck disable=2016
|
||||
echo ' /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"'
|
||||
echo ""
|
||||
|
||||
echo "To register 'brew update' as a hourly system service:"
|
||||
echo " brew-update-service-install"
|
||||
echo ""
|
||||
}
|
||||
|
||||
fn_brew_shell_integrate_bash() {
|
||||
if ! command -v bash > /dev/null; then
|
||||
return 0
|
||||
fi
|
||||
if ! test -e ~/.bashrc && ! test -e ~/.bash_history; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
touch -a ~/.bashrc
|
||||
if grep -q 'brew shellenv' ~/.bashrc; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo >&2 " Edit ~/.bashrc to init brew"
|
||||
# shellcheck disable=SC2016
|
||||
{
|
||||
echo ''
|
||||
echo '# Generated by Webi. Do not edit.'
|
||||
echo 'eval "$('"$HOME/.local/opt/brew/bin/brew"' shellenv)"'
|
||||
} >> ~/.bashrc
|
||||
}
|
||||
|
||||
fn_brew_shell_integrate_zsh() {
|
||||
if ! command -v zsh > /dev/null; then
|
||||
return 0
|
||||
fi
|
||||
if ! test -e ~/.zshrc &&
|
||||
! test -e ~/.zsh_sessions &&
|
||||
! test -e ~/.zsh_history; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
touch -a ~/.zshrc
|
||||
if grep -q 'brew shellenv' ~/.zshrc; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo >&2 " Edit ~/.zshrc to init brew"
|
||||
# shellcheck disable=SC2016
|
||||
{
|
||||
echo ''
|
||||
echo '# Generated by Webi. Do not edit.'
|
||||
echo 'eval "$('"$HOME/.local/opt/brew/bin/brew"' shellenv)"'
|
||||
} >> ~/.zshrc
|
||||
}
|
||||
|
||||
fn_brew_shell_integrate_fish() {
|
||||
if ! command -v fish > /dev/null; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
mkdir -p ~/.config/fish
|
||||
touch -a ~/.config/fish/config.fish
|
||||
if grep -q 'brew shellenv' ~/.config/fish/config.fish; then
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo >&2 " Edit ~/.config/fish/config.fish to init brew"
|
||||
{
|
||||
echo ''
|
||||
echo '# Generated by Webi. Do not edit.'
|
||||
echo "$HOME/.local/opt/brew/bin/brew shellenv | source"
|
||||
} >> ~/.config/fish/config.fish
|
||||
echo "To register 'brew update' as a hourly system service:"
|
||||
echo " brew-update-service-install"
|
||||
echo ""
|
||||
}
|
||||
|
||||
_install_brew
|
||||
|
||||
109
btop/README.md
109
btop/README.md
@@ -1,109 +0,0 @@
|
||||
---
|
||||
title: btop
|
||||
homepage: https://github.com/aristocratos/btop
|
||||
tagline: |
|
||||
btop: a beautiful, interactive resource monitor
|
||||
description: |
|
||||
btop++ is a fast, feature-rich terminal resource monitor written in C++.
|
||||
It shows real-time usage and stats for CPU, memory, disks, network, and
|
||||
processes — with full mouse support, customizable themes, and an
|
||||
easy-to-use menu system. The spiritual successor to bashtop and bpytop.
|
||||
---
|
||||
|
||||
To update or switch versions, run `webi btop@stable` (or `@v1.4`, `@beta`, etc).
|
||||
|
||||
### Files
|
||||
|
||||
These are the files / directories that are created and/or modified with this
|
||||
install:
|
||||
|
||||
```
|
||||
~/.config/envman/PATH.env
|
||||
~/.local/bin/btop
|
||||
~/.local/opt/btop/
|
||||
~/.local/opt/btop-<VERSION>/
|
||||
```
|
||||
|
||||
## Cheat Sheet
|
||||
|
||||
|
||||
|
||||
> btop gives you a gorgeous, interactive view of what your system is doing —
|
||||
> CPU cores, RAM, swap, disk I/O, network throughput, and a filterable process
|
||||
> list — all in one terminal window.
|
||||
|
||||
### Launch btop
|
||||
|
||||
```sh
|
||||
btop
|
||||
```
|
||||
|
||||
### Navigation
|
||||
|
||||
| Key | Action |
|
||||
| -------------- | ----------------------------------- |
|
||||
| `Arrow keys` | Move selection in process list |
|
||||
| `Enter` | Show detailed stats for process |
|
||||
| `F` | Filter / search processes |
|
||||
| `K` | Send signal (kill, SIGTERM, etc.) |
|
||||
| `R` | Renice (change process priority) |
|
||||
| `T` | Toggle tree / flat process view |
|
||||
| `M` | Change sort field |
|
||||
| `ESC` | Open settings menu |
|
||||
| `Q` | Quit |
|
||||
|
||||
Mouse support is fully enabled by default — scroll and click anywhere in the UI.
|
||||
|
||||
### Change the color theme
|
||||
|
||||
Press `ESC` to open the menu, navigate to **Options → Color theme**, and pick
|
||||
from the built-in themes (Default, TTY, Dracula, Gruvbox, and more).
|
||||
|
||||
Custom themes can be placed in:
|
||||
|
||||
```
|
||||
~/.config/btop/themes/
|
||||
```
|
||||
|
||||
### Adjust update interval
|
||||
|
||||
In the Options menu, set **Update interval** (in milliseconds). The default is
|
||||
`2000` (2 seconds). Lower values give a more live feel; higher values reduce CPU
|
||||
overhead from btop itself.
|
||||
|
||||
### Config file location
|
||||
|
||||
btop's settings are saved automatically at:
|
||||
|
||||
```
|
||||
~/.config/btop/btop.conf
|
||||
```
|
||||
|
||||
You can edit this file directly to set options like `update_ms`, `color_theme`,
|
||||
`proc_sorting`, or `net_iface`.
|
||||
|
||||
### Run btop with a specific network interface shown
|
||||
|
||||
```sh
|
||||
btop --utf-foce # force UTF-8 box drawing
|
||||
btop --debug # verbose debug output to btop.log
|
||||
```
|
||||
|
||||
Network interface selection is done interactively inside btop via the network
|
||||
panel — press `B` / `N` to cycle interfaces.
|
||||
|
||||
### GPU monitoring (Linux x86\_64)
|
||||
|
||||
On Linux, btop supports Nvidia, AMD, and Intel GPUs out of the box provided the
|
||||
correct drivers are installed. If wattage or GPU stats are missing, you may need
|
||||
to grant extended capabilities:
|
||||
|
||||
```sh
|
||||
# Run once after install (requires sudo)
|
||||
sudo setcap cap_perfmon,cap_sys_ptrace+ep ~/.local/bin/btop
|
||||
```
|
||||
|
||||
### See also
|
||||
|
||||
- [btop releases](https://github.com/aristocratos/btop/releases)
|
||||
- [Theme gallery](https://github.com/aristocratos/btop/tree/main/themes)
|
||||
@@ -1,58 +0,0 @@
|
||||
#!/bin/sh
|
||||
__init_btop() {
|
||||
set -e
|
||||
set -u
|
||||
|
||||
################
|
||||
# Install btop #
|
||||
################
|
||||
|
||||
# Every package should define these 6 variables
|
||||
pkg_cmd_name="btop"
|
||||
|
||||
pkg_dst_cmd="$HOME/.local/bin/btop"
|
||||
pkg_dst="$pkg_dst_cmd"
|
||||
|
||||
pkg_src_cmd="$HOME/.local/opt/btop-v$WEBI_VERSION/bin/btop"
|
||||
pkg_src_dir="$HOME/.local/opt/btop-v$WEBI_VERSION"
|
||||
pkg_src="$pkg_src_cmd"
|
||||
|
||||
# pkg_install must be defined by every package
|
||||
pkg_install() {
|
||||
# ~/.local/opt/btop-v1.4.6/bin
|
||||
mkdir -p "$(dirname "${pkg_src_cmd}")"
|
||||
|
||||
# mv ./btop/bin/btop ~/.local/opt/btop-v1.4.6/bin/btop
|
||||
mv ./btop/bin/btop "${pkg_src_cmd}"
|
||||
}
|
||||
|
||||
# pkg_get_current_version is recommended, but not required
|
||||
pkg_get_current_version() {
|
||||
# 'btop --version' has output in this format:
|
||||
# btop 1.4.6 (rev abcdef0123)
|
||||
# This trims it down to just the version number:
|
||||
# 1.4.6
|
||||
btop --version 2> /dev/null |
|
||||
head -n 1 |
|
||||
cut -d ' ' -f 2
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
fn_btop_brew_install() {
|
||||
if ! command -v brew > /dev/null; then
|
||||
"$HOME/.local/bin/webi" brew
|
||||
export PATH="$HOME/.local/opt/brew/bin:$HOME/.local/opt/brew/sbin:$PATH"
|
||||
fi
|
||||
|
||||
export HOMEBREW_NO_AUTO_UPDATE=1
|
||||
export HOMEBREW_NO_ENV_HINTS=1
|
||||
brew install btop
|
||||
}
|
||||
|
||||
my_os=$(uname -s)
|
||||
if test "Darwin" = "${my_os}" && test "$WEBI_CHANNEL" = "error"; then
|
||||
fn_btop_brew_install
|
||||
return 0
|
||||
fi
|
||||
__init_btop
|
||||
@@ -1,2 +0,0 @@
|
||||
github_releases = aristocratos/btop
|
||||
exclude = -m68k -bigsur -monterey -ventura -macos
|
||||
@@ -132,7 +132,9 @@ file)
|
||||
```
|
||||
3. Add your project to the system launcher, running as the current user
|
||||
```sh
|
||||
serviceman add --name 'my-project' --daemon -- \
|
||||
sudo env PATH="$PATH" \
|
||||
serviceman add --path="$PATH" --system \
|
||||
--username "$(whoami)" --name my-project -- \
|
||||
bun run ./my-project.js
|
||||
```
|
||||
4. Restart the logging service
|
||||
@@ -153,6 +155,6 @@ For **macOS**:
|
||||
```
|
||||
3. Add your project to the system launcher, running as the current user
|
||||
```sh
|
||||
serviceman add --agent --name 'my-project' -- \
|
||||
serviceman add --path="$PATH" --user --name my-project -- \
|
||||
bun run ./my-project.js
|
||||
```
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
github_releases = oven-sh/bun
|
||||
tag_prefix = bun-
|
||||
default_x86_64 = x86_64_v3
|
||||
x86_64_v2 = baseline
|
||||
variants = profile
|
||||
39
bun/releases.js
Normal file
39
bun/releases.js
Normal file
@@ -0,0 +1,39 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'oven-sh';
|
||||
var repo = 'bun';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
all.releases = all.releases
|
||||
.filter(function (r) {
|
||||
let isDebug = r.name.includes('-profile');
|
||||
if (isDebug) {
|
||||
return false;
|
||||
}
|
||||
|
||||
let isAncient = r.name.includes('-baseline');
|
||||
if (isAncient) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
})
|
||||
.map(function (r) {
|
||||
// bun-v0.5.1 => v0.5.1
|
||||
r.version = r.version.replace(/bun-/g, '');
|
||||
return r;
|
||||
});
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -819,10 +819,10 @@ To avoid the nitty-gritty details of `launchd` plist files, you can use
|
||||
2. Use Serviceman to create a _launchd_ plist file
|
||||
|
||||
```sh
|
||||
my_username="$(id -u -n)"
|
||||
my_username="$( id -u -n )"
|
||||
|
||||
serviceman add --agent --name 'caddy' --workdir ./ -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
serviceman add --user --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
|
||||
(this will create `~/Library/LaunchAgents/caddy.plist`)
|
||||
@@ -837,8 +837,8 @@ This process creates a _User-Level_ service in `~/Library/LaunchAgents`. To
|
||||
create a _System-Level_ service in `/Library/LaunchDaemons/` instead:
|
||||
|
||||
```sh
|
||||
serviceman add --name 'caddy' --workdir ./ --daemon -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
sudo serviceman add --system --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
|
||||
### How to run Caddy as a Windows Service
|
||||
@@ -856,7 +856,7 @@ serviceman add --name 'caddy' --workdir ./ --daemon -- \
|
||||
3. Create a **Startup Registry Entry** with Serviceman.
|
||||
```sh
|
||||
serviceman.exe add --name caddy -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
4. You can manage the service directly with Serviceman. For example:
|
||||
```sh
|
||||
@@ -901,8 +901,10 @@ See the notes below to run as a **User Service** or use the JSON Config.
|
||||
```
|
||||
4. Use Serviceman to create a _systemd_ config file.
|
||||
```sh
|
||||
serviceman add --name 'caddy' --daemon -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
my_username="$( id -u -n )"
|
||||
sudo env PATH="$PATH" \
|
||||
serviceman add --system --username "${my_username}" --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
(this will create `/etc/systemd/system/caddy.service`)
|
||||
5. Manage the service with `systemctl` and `journalctl`:
|
||||
@@ -913,10 +915,10 @@ See the notes below to run as a **User Service** or use the JSON Config.
|
||||
|
||||
To create a **User Service** instead:
|
||||
|
||||
- use `--agent` when running `serviceman`:
|
||||
- don't use `sudo`, but do use `--user` when running `serviceman`:
|
||||
```sh
|
||||
serviceman add --agent --name caddy -- \
|
||||
caddy run --envfile ~/.config/caddy/env --config ./Caddyfile --adapter caddyfile
|
||||
serviceman add --user --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ~/.config/caddy/env
|
||||
```
|
||||
(this will create `~/.config/systemd/user/`)
|
||||
- user the `--user` flag to manage services and logs:
|
||||
@@ -1181,8 +1183,7 @@ To prevent search engine and browser confusion
|
||||
- _DO NOT_ prevent crawling via `robots.txt` \
|
||||
(counter-intuitive, but pages _must_ be crawled for links to _NOT_ be indexed)
|
||||
- _all_ domains using public TLS certs _will_ be indexed by default \
|
||||
(they are all linked to and crawled from various Certificate Transparency
|
||||
reports)
|
||||
(they are all linked to and crawled from various Certificate Transparency reports)
|
||||
- follow these guidelines even if the dev sites use HTTP Basic Auth
|
||||
|
||||
```Caddyfile
|
||||
@@ -1362,13 +1363,19 @@ See also: <https://caddyserver.com/docs/running>
|
||||
2. Generate the `service` file: \
|
||||
- JSON Config
|
||||
```sh
|
||||
serviceman add --name 'caddy' --daemon -- \
|
||||
caddy run --resume --envfile ./caddy.env
|
||||
my_app_user="$( id -u -n )"
|
||||
sudo env PATH="${PATH}" \
|
||||
serviceman add --system --cap-net-bind \
|
||||
--username "${my_app_user}" --name caddy -- \
|
||||
caddy run --resume --envfile ./caddy.env
|
||||
```
|
||||
- Caddyfile
|
||||
```sh
|
||||
serviceman add --name 'caddy' --daemon -- \
|
||||
caddy run --config ./Caddyfile --envfile ./caddy.env
|
||||
my_app_user="$( id -u -n )"
|
||||
sudo env PATH="${PATH}" \
|
||||
serviceman add --system --cap-net-bind \
|
||||
--username "${my_app_user}" --name caddy -- \
|
||||
caddy run --config ./Caddyfile --envfile ./caddy.env
|
||||
```
|
||||
3. Reload `systemd` config files, the logging service (it may not be started on
|
||||
a new VPS), and caddy
|
||||
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading caddy from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing caddy"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = caddyserver/caddy
|
||||
27
caddy/releases.js
Normal file
27
caddy/releases.js
Normal file
@@ -0,0 +1,27 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'caddyserver';
|
||||
var repo = 'caddy';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
// remove checksums and .deb
|
||||
all.releases = all.releases.filter(function (rel) {
|
||||
let isOneOffAsset = rel.download.includes('buildable-artifact');
|
||||
if (isOneOffAsset) {
|
||||
return false;
|
||||
}
|
||||
|
||||
return true;
|
||||
});
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all));
|
||||
});
|
||||
}
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading chromedriver from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing chromedriver"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
source = chromedist
|
||||
84
chromedriver/releases.js
Normal file
84
chromedriver/releases.js
Normal file
@@ -0,0 +1,84 @@
|
||||
'use strict';
|
||||
|
||||
// See <https://googlechromelabs.github.io/chrome-for-testing/>
|
||||
var releaseApiUrl =
|
||||
'https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json';
|
||||
|
||||
// {
|
||||
// "timestamp": "2023-11-15T21:08:56.730Z",
|
||||
// "versions": [
|
||||
// {
|
||||
// "version": "121.0.6120.0",
|
||||
// "revision": "1222902",
|
||||
// "downloads": {
|
||||
// "chrome": [],
|
||||
// "chromedriver": [
|
||||
// {
|
||||
// "platform": "linux64",
|
||||
// "url": "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/121.0.6120.0/linux64/chromedriver-linux64.zip"
|
||||
// },
|
||||
// {
|
||||
// "platform": "mac-arm64",
|
||||
// "url": "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/121.0.6120.0/mac-arm64/chromedriver-mac-arm64.zip"
|
||||
// },
|
||||
// {
|
||||
// "platform": "mac-x64",
|
||||
// "url": "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/121.0.6120.0/mac-x64/chromedriver-mac-x64.zip"
|
||||
// },
|
||||
// {
|
||||
// "platform": "win32",
|
||||
// "url": "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/121.0.6120.0/win32/chromedriver-win32.zip"
|
||||
// },
|
||||
// {
|
||||
// "platform": "win64",
|
||||
// "url": "https://edgedl.me.gvt1.com/edgedl/chrome/chrome-for-testing/121.0.6120.0/win64/chromedriver-win64.zip"
|
||||
// }
|
||||
// ],
|
||||
// "chrome-headless-shell": []
|
||||
// }
|
||||
// }
|
||||
// ]
|
||||
// }
|
||||
|
||||
module.exports = async function (request) {
|
||||
let resp = await request({
|
||||
url: releaseApiUrl,
|
||||
json: true,
|
||||
});
|
||||
|
||||
let builds = [];
|
||||
for (let release of resp.body.versions) {
|
||||
if (!release.downloads.chromedriver) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let version = release.version;
|
||||
for (let asset of release.downloads.chromedriver) {
|
||||
let build = {
|
||||
version: version,
|
||||
download: asset.url,
|
||||
// I' not sure that this is actually statically built but it
|
||||
// seems to be and at worst we'll just get bug reports for Apline
|
||||
libc: 'none',
|
||||
};
|
||||
|
||||
builds.push(build);
|
||||
}
|
||||
}
|
||||
|
||||
let all = {
|
||||
download: '',
|
||||
releases: builds,
|
||||
};
|
||||
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the latest 5 for demonstration
|
||||
all.releases = all.releases.slice(-20);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -17,13 +17,13 @@ $pkg_src = "$pkg_src_cmd"
|
||||
New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading cilium from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing cilium"
|
||||
Push-Location .local\tmp
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = cilium/cilium-cli
|
||||
22
cilium/releases.js
Normal file
22
cilium/releases.js
Normal file
@@ -0,0 +1,22 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'cilium';
|
||||
var repo = 'cilium-cli';
|
||||
|
||||
module.exports = async function (request) {
|
||||
let all = await github(request, owner, repo);
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
(async function () {
|
||||
let request = require('@root/request');
|
||||
let normalize = require('../_webi/normalize.js');
|
||||
let all = await module.exports(request);
|
||||
all = normalize(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
})();
|
||||
}
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$pkg_download")) {
|
||||
IF (!(Test-Path -Path "$pkg_download")) {
|
||||
Write-Output "Downloading cmake from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
Write-Output "Installing cmake"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = Kitware/CMake
|
||||
53
cmake/releases.js
Normal file
53
cmake/releases.js
Normal file
@@ -0,0 +1,53 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'Kitware';
|
||||
var repo = 'CMake';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
for (let rel of all.releases) {
|
||||
if (rel.version.startsWith('v')) {
|
||||
rel._version = rel.version.slice(1);
|
||||
}
|
||||
|
||||
{
|
||||
let linuxRe = /(\b|_)(linux|gnu)(\b|_)/i;
|
||||
let isLinux = linuxRe.test(rel.download) || linuxRe.test(rel.name);
|
||||
|
||||
if (isLinux) {
|
||||
let muslRe = /(\b|_)(musl|alpine)(\b|_)/i;
|
||||
let isMusl = muslRe.test(rel.download) || muslRe.test(rel.name);
|
||||
if (isMusl) {
|
||||
rel.libc = 'musl';
|
||||
} else {
|
||||
rel.libc = 'gnu';
|
||||
}
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
let windowsRe = /(\b|_)(win\d*|windows\d*)(\b|_)/i;
|
||||
let isWindows =
|
||||
windowsRe.test(rel.download) || windowsRe.test(rel.name);
|
||||
|
||||
if (isWindows) {
|
||||
rel.libc = 'msvc';
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -1,924 +0,0 @@
|
||||
// Command webicached is the release cache daemon. It fetches releases
|
||||
// from upstream sources, classifies build assets, and writes them to
|
||||
// the _cache/ directory in the format the Node.js server expects.
|
||||
//
|
||||
// This is the Go replacement for the Node.js release-fetching pipeline.
|
||||
// It reads releases.conf files to discover packages, fetches from the
|
||||
// configured source, classifies assets, and writes to fsstore.
|
||||
//
|
||||
// Default mode: classify all from existing rawcache on startup, then
|
||||
// fetch+refresh one package per tick (round-robin, 15m default).
|
||||
//
|
||||
// Usage:
|
||||
//
|
||||
// go run ./cmd/webicached # default: round-robin, one per tick
|
||||
// go run ./cmd/webicached -eager # fetch all packages on startup
|
||||
// go run ./cmd/webicached -once -no-fetch # classify from rawcache and exit
|
||||
// go run ./cmd/webicached bat goreleaser # only these packages
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"math/rand/v2"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/joho/godotenv"
|
||||
"github.com/webinstall/webi-installers/internal/classifypkg"
|
||||
"github.com/webinstall/webi-installers/internal/httpclient"
|
||||
"github.com/webinstall/webi-installers/internal/installerconf"
|
||||
"github.com/webinstall/webi-installers/internal/rawcache"
|
||||
"github.com/webinstall/webi-installers/internal/releases/chromedist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/flutterdist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/gitea"
|
||||
"github.com/webinstall/webi-installers/internal/releases/github"
|
||||
"github.com/webinstall/webi-installers/internal/releases/githubish"
|
||||
"github.com/webinstall/webi-installers/internal/releases/gittag"
|
||||
"github.com/webinstall/webi-installers/internal/releases/golang"
|
||||
"github.com/webinstall/webi-installers/internal/releases/gpgdist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/hashicorp"
|
||||
"github.com/webinstall/webi-installers/internal/releases/iterm2dist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/juliadist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/mariadbdist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/nodedist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/servicemandist"
|
||||
"github.com/webinstall/webi-installers/internal/releases/zigdist"
|
||||
"github.com/webinstall/webi-installers/internal/storage"
|
||||
"github.com/webinstall/webi-installers/internal/storage/fsstore"
|
||||
)
|
||||
|
||||
var (
|
||||
name = "webicached"
|
||||
version = "0.0.0-dev"
|
||||
commit = "0000000"
|
||||
date = "0001-01-01"
|
||||
licenseYear = "2024"
|
||||
licenseOwner = "AJ ONeal"
|
||||
licenseType = "MPL-2.0"
|
||||
)
|
||||
|
||||
func printVersion(w io.Writer) {
|
||||
b_ver := strings.TrimPrefix(version, "v")
|
||||
_, _ = fmt.Fprintf(w, "%s v%s %s (%s)\n", name, b_ver, commit[:7], date)
|
||||
_, _ = fmt.Fprintf(w, "Copyright (C) %s %s\n", licenseYear, licenseOwner)
|
||||
_, _ = fmt.Fprintf(w, "Licensed under %s\n", licenseType)
|
||||
}
|
||||
|
||||
type MainConfig struct {
|
||||
envFile string
|
||||
confDir string
|
||||
cacheDir string
|
||||
rawDir string
|
||||
token string
|
||||
once bool
|
||||
noFetch bool
|
||||
shallow bool
|
||||
eager bool
|
||||
interval time.Duration
|
||||
pageDelay time.Duration
|
||||
}
|
||||
|
||||
// WebiCache holds the configuration for the cache daemon.
|
||||
type WebiCache struct {
|
||||
ConfDir string // root directory with {pkg}/releases.conf files
|
||||
Store storage.Store // classified asset storage (fsstore)
|
||||
RawDir string // raw upstream response cache
|
||||
Client *http.Client // HTTP client for upstream calls
|
||||
Auth *githubish.Auth // GitHub API auth (optional)
|
||||
Shallow bool // fetch only the first page of releases
|
||||
NoFetch bool // skip fetching, classify from existing raw data only
|
||||
PageDelay time.Duration // delay between paginated API requests
|
||||
}
|
||||
|
||||
// delayTransport wraps an http.RoundTripper to add a delay between requests.
|
||||
type delayTransport struct {
|
||||
base http.RoundTripper
|
||||
delay time.Duration
|
||||
last time.Time
|
||||
}
|
||||
|
||||
func (t *delayTransport) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||
if !t.last.IsZero() && t.delay > 0 {
|
||||
if wait := t.delay - time.Since(t.last); wait > 0 {
|
||||
time.Sleep(wait)
|
||||
}
|
||||
}
|
||||
t.last = time.Now()
|
||||
return t.base.RoundTrip(req)
|
||||
}
|
||||
|
||||
func main() {
|
||||
if len(os.Args) > 1 {
|
||||
switch os.Args[1] {
|
||||
case "-V", "-version", "--version", "version":
|
||||
printVersion(os.Stdout)
|
||||
os.Exit(0)
|
||||
case "help", "-help", "--help":
|
||||
printVersion(os.Stdout)
|
||||
fmt.Fprintln(os.Stdout, "")
|
||||
fs := flag.NewFlagSet(os.Args[0], flag.ContinueOnError)
|
||||
fs.SetOutput(os.Stdout)
|
||||
registerFlags(fs, &MainConfig{})
|
||||
fs.Usage()
|
||||
os.Exit(0)
|
||||
}
|
||||
}
|
||||
|
||||
cfg := MainConfig{}
|
||||
fs := flag.NewFlagSet(os.Args[0], flag.ContinueOnError)
|
||||
registerFlags(fs, &cfg)
|
||||
if err := fs.Parse(os.Args[1:]); err != nil {
|
||||
if errors.Is(err, flag.ErrHelp) {
|
||||
os.Exit(0)
|
||||
}
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
cfg.cacheDir = expandHome(cfg.cacheDir)
|
||||
cfg.rawDir = expandHome(cfg.rawDir)
|
||||
|
||||
if cfg.envFile != "" {
|
||||
if err := godotenv.Load(cfg.envFile); err != nil {
|
||||
log.Fatalf("envfile: %v", err)
|
||||
}
|
||||
}
|
||||
if cfg.token == "" {
|
||||
cfg.token = os.Getenv("GITHUB_TOKEN")
|
||||
}
|
||||
|
||||
fss, err := fsstore.New(cfg.cacheDir)
|
||||
if err != nil {
|
||||
log.Fatalf("fsstore: %v", err)
|
||||
}
|
||||
var store storage.Store = fss
|
||||
|
||||
var auth *githubish.Auth
|
||||
if cfg.token != "" {
|
||||
auth = &githubish.Auth{Token: cfg.token}
|
||||
}
|
||||
|
||||
client := httpclient.New()
|
||||
if cfg.pageDelay > 0 {
|
||||
client.Transport = &delayTransport{
|
||||
base: client.Transport,
|
||||
delay: cfg.pageDelay,
|
||||
}
|
||||
}
|
||||
|
||||
wc := &WebiCache{
|
||||
ConfDir: cfg.confDir,
|
||||
Store: store,
|
||||
RawDir: cfg.rawDir,
|
||||
Client: client,
|
||||
Auth: auth,
|
||||
Shallow: cfg.shallow,
|
||||
NoFetch: cfg.noFetch,
|
||||
PageDelay: cfg.pageDelay,
|
||||
}
|
||||
|
||||
filterPkgs := fs.Args()
|
||||
|
||||
if cfg.eager {
|
||||
wc.Run(filterPkgs)
|
||||
if cfg.once {
|
||||
return
|
||||
}
|
||||
} else if cfg.once {
|
||||
wc.Run(filterPkgs)
|
||||
return
|
||||
} else {
|
||||
saved := wc.NoFetch
|
||||
wc.NoFetch = true
|
||||
wc.Run(filterPkgs)
|
||||
wc.NoFetch = saved
|
||||
}
|
||||
|
||||
packages, err := discover(wc.ConfDir)
|
||||
if err != nil {
|
||||
log.Fatalf("discover: %v", err)
|
||||
}
|
||||
nameSet := make(map[string]bool, len(filterPkgs))
|
||||
for _, a := range filterPkgs {
|
||||
nameSet[a] = true
|
||||
}
|
||||
if len(filterPkgs) > 0 {
|
||||
var filtered []pkgConf
|
||||
for _, p := range packages {
|
||||
if nameSet[p.name] {
|
||||
filtered = append(filtered, p)
|
||||
}
|
||||
}
|
||||
packages = filtered
|
||||
}
|
||||
|
||||
var real []pkgConf
|
||||
for _, pkg := range packages {
|
||||
if pkg.conf.AliasOf == "" {
|
||||
real = append(real, pkg)
|
||||
}
|
||||
}
|
||||
|
||||
// rescanNew appends any conf files added since the last scan.
|
||||
// Returns true when at least one new package was added so the caller
|
||||
// can restart the batch loop and process new packages immediately.
|
||||
rescanNew := func() bool {
|
||||
discovered, err := discover(wc.ConfDir)
|
||||
if err != nil {
|
||||
log.Printf("rescan: %v", err)
|
||||
return false
|
||||
}
|
||||
known := make(map[string]bool, len(real))
|
||||
for _, p := range real {
|
||||
known[p.name] = true
|
||||
}
|
||||
added := false
|
||||
for _, p := range discovered {
|
||||
if p.conf.AliasOf != "" || known[p.name] {
|
||||
continue
|
||||
}
|
||||
if len(filterPkgs) > 0 && !nameSet[p.name] {
|
||||
continue
|
||||
}
|
||||
log.Printf("discovered new package: %s (source=%s)", p.name, p.conf.Source)
|
||||
real = append(real, p)
|
||||
added = true
|
||||
}
|
||||
return added
|
||||
}
|
||||
|
||||
log.Printf("refreshing %d packages, interval %s, batch size 20 (ctrl-c to stop)", len(real), cfg.interval)
|
||||
for {
|
||||
// Rescan before computing staleness so newly added conf files are
|
||||
// included immediately. New packages have a zero timestamp and sort
|
||||
// to the front of the stale list, so they are processed next.
|
||||
rescanNew()
|
||||
|
||||
stale := wc.stalest(real)
|
||||
if len(stale) == 0 {
|
||||
log.Printf("all packages fresh, sleeping %s", cfg.interval)
|
||||
time.Sleep(cfg.interval)
|
||||
continue
|
||||
}
|
||||
|
||||
batch := stale
|
||||
if len(batch) > 20 {
|
||||
batch = batch[:20]
|
||||
}
|
||||
rand.Shuffle(len(batch), func(i, j int) {
|
||||
batch[i], batch[j] = batch[j], batch[i]
|
||||
})
|
||||
|
||||
log.Printf("batch: %d stale, refreshing %d (most stale first)", len(stale), len(batch))
|
||||
for _, pkg := range batch {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
|
||||
if err := wc.refreshPackage(ctx, pkg); err != nil {
|
||||
log.Printf(" ERROR %s: %v", pkg.name, err)
|
||||
}
|
||||
cancel()
|
||||
time.Sleep(cfg.interval)
|
||||
// Rescan mid-batch so new packages preempt remaining batch items.
|
||||
if rescanNew() {
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func registerFlags(fs *flag.FlagSet, cfg *MainConfig) {
|
||||
fs.StringVar(&cfg.envFile, "envfile", "", "path to .env file to load before running")
|
||||
fs.StringVar(&cfg.confDir, "conf", ".", "root directory containing {pkg}/releases.conf files")
|
||||
fs.StringVar(&cfg.cacheDir, "legacy", "~/.cache/webi/legacy", "legacy cache directory (fsstore root)")
|
||||
fs.StringVar(&cfg.rawDir, "raw", "~/.cache/webi/raw", "raw cache directory for upstream responses")
|
||||
fs.StringVar(&cfg.token, "token", "", "GitHub API token (or set $GITHUB_TOKEN)")
|
||||
fs.BoolVar(&cfg.once, "once", false, "run once then exit (no periodic refresh)")
|
||||
fs.BoolVar(&cfg.noFetch, "no-fetch", false, "skip fetching, classify from existing raw data only")
|
||||
fs.BoolVar(&cfg.shallow, "shallow", false, "fetch only the first page of releases (latest)")
|
||||
fs.BoolVar(&cfg.eager, "eager", false, "fetch all packages on startup (default: one per tick)")
|
||||
fs.DurationVar(&cfg.interval, "interval", 9*time.Second, "delay between individual package fetches")
|
||||
fs.DurationVar(&cfg.pageDelay, "page-delay", 2*time.Second, "delay between paginated API requests")
|
||||
}
|
||||
|
||||
func expandHome(path string) string {
|
||||
if !strings.HasPrefix(path, "~/") {
|
||||
return path
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
return path
|
||||
}
|
||||
return filepath.Join(home, path[2:])
|
||||
}
|
||||
|
||||
// stalest returns packages sorted by most stale first (oldest UpdatedAt).
|
||||
// Packages with no cache entry or empty assets are considered most stale.
|
||||
func (wc *WebiCache) stalest(packages []pkgConf) []pkgConf {
|
||||
type stamped struct {
|
||||
pkg pkgConf
|
||||
updatedAt time.Time
|
||||
}
|
||||
|
||||
var stale []stamped
|
||||
ctx := context.Background()
|
||||
for _, pkg := range packages {
|
||||
data, err := wc.Store.Load(ctx, pkg.name)
|
||||
var t time.Time
|
||||
if err == nil && data != nil {
|
||||
t = data.UpdatedAt
|
||||
}
|
||||
// Never fetched, or older than 10 minutes.
|
||||
// 0-asset results are not treated as perpetually stale — packages that
|
||||
// produce no classifiable assets (e.g. galera) respect the timestamp.
|
||||
if t.IsZero() || time.Since(t) > 10*time.Minute {
|
||||
stale = append(stale, stamped{pkg: pkg, updatedAt: t})
|
||||
}
|
||||
}
|
||||
|
||||
sort.SliceStable(stale, func(i, j int) bool {
|
||||
ti, tj := stale[i].updatedAt, stale[j].updatedAt
|
||||
if ti.Equal(tj) {
|
||||
return stale[i].pkg.name < stale[j].pkg.name
|
||||
}
|
||||
return ti.Before(tj)
|
||||
})
|
||||
|
||||
result := make([]pkgConf, len(stale))
|
||||
for i, s := range stale {
|
||||
result[i] = s.pkg
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
// Run discovers packages and refreshes each one.
|
||||
func (wc *WebiCache) Run(filterPkgs []string) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
packages, err := discover(wc.ConfDir)
|
||||
if err != nil {
|
||||
log.Printf("discover: %v", err)
|
||||
return
|
||||
}
|
||||
|
||||
if len(filterPkgs) > 0 {
|
||||
nameSet := make(map[string]bool, len(filterPkgs))
|
||||
for _, a := range filterPkgs {
|
||||
nameSet[a] = true
|
||||
}
|
||||
var filtered []pkgConf
|
||||
for _, p := range packages {
|
||||
if nameSet[p.name] {
|
||||
filtered = append(filtered, p)
|
||||
}
|
||||
}
|
||||
packages = filtered
|
||||
}
|
||||
|
||||
var real []pkgConf
|
||||
for _, pkg := range packages {
|
||||
if pkg.conf.AliasOf != "" {
|
||||
continue
|
||||
}
|
||||
real = append(real, pkg)
|
||||
}
|
||||
|
||||
log.Printf("refreshing %d packages", len(real))
|
||||
runStart := time.Now()
|
||||
|
||||
for _, pkg := range real {
|
||||
if err := wc.refreshPackage(ctx, pkg); err != nil {
|
||||
log.Printf(" ERROR %s: %v", pkg.name, err)
|
||||
}
|
||||
}
|
||||
|
||||
log.Printf("refreshed %d packages in %s", len(real), time.Since(runStart))
|
||||
}
|
||||
|
||||
type pkgConf struct {
|
||||
name string
|
||||
conf *installerconf.Conf
|
||||
}
|
||||
|
||||
func discover(dir string) ([]pkgConf, error) {
|
||||
pattern := filepath.Join(dir, "*", "releases.conf")
|
||||
matches, err := filepath.Glob(pattern)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var packages []pkgConf
|
||||
for _, path := range matches {
|
||||
pkgDir := filepath.Dir(path)
|
||||
name := filepath.Base(pkgDir)
|
||||
if strings.HasPrefix(name, "_") {
|
||||
continue
|
||||
}
|
||||
|
||||
// If the package directory is a symlink, treat it as an alias
|
||||
// of the symlink target (e.g. rust.vim → vim-rust).
|
||||
fi, err := os.Lstat(filepath.Join(dir, name))
|
||||
if err != nil {
|
||||
log.Printf("warning: %s: %v", name, err)
|
||||
continue
|
||||
}
|
||||
if fi.Mode()&os.ModeSymlink != 0 {
|
||||
target, err := os.Readlink(filepath.Join(dir, name))
|
||||
if err != nil {
|
||||
log.Printf("warning: readlink %s: %v", name, err)
|
||||
continue
|
||||
}
|
||||
packages = append(packages, pkgConf{
|
||||
name: name,
|
||||
conf: &installerconf.Conf{AliasOf: target},
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
conf, err := installerconf.Read(path)
|
||||
if err != nil {
|
||||
log.Printf("warning: %s: %v", path, err)
|
||||
continue
|
||||
}
|
||||
packages = append(packages, pkgConf{name: name, conf: conf})
|
||||
}
|
||||
|
||||
sort.Slice(packages, func(i, j int) bool {
|
||||
return packages[i].name < packages[j].name
|
||||
})
|
||||
return packages, nil
|
||||
}
|
||||
|
||||
// refreshPackage does the full pipeline for one package:
|
||||
// fetch raw → classify → write to fsstore.
|
||||
func (wc *WebiCache) refreshPackage(ctx context.Context, pkg pkgConf) error {
|
||||
pkgStart := time.Now()
|
||||
name := pkg.name
|
||||
conf := pkg.conf
|
||||
|
||||
// Step 1: Fetch raw upstream data to rawcache (unless -no-fetch).
|
||||
if !wc.NoFetch {
|
||||
shallow := wc.Shallow
|
||||
if !shallow {
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, name))
|
||||
if err == nil && d.Populated() {
|
||||
shallow = true
|
||||
}
|
||||
}
|
||||
fetchStart := time.Now()
|
||||
if err := wc.fetchRaw(ctx, pkg, shallow); err != nil {
|
||||
return fmt.Errorf("fetch: %w", err)
|
||||
}
|
||||
log.Printf(" %s: fetch %s", name, time.Since(fetchStart))
|
||||
}
|
||||
|
||||
// Step 2: Classify raw data into assets, tag variants, apply config.
|
||||
classifyStart := time.Now()
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, name))
|
||||
if err != nil {
|
||||
return fmt.Errorf("rawcache open: %w", err)
|
||||
}
|
||||
|
||||
// Open supplementary gittag raw cache if available (for packages with
|
||||
// git_url that use a non-gittag source type like servicemandist).
|
||||
var gitTagDir *rawcache.Dir
|
||||
if conf.GitURL != "" && conf.Source != "gittag" {
|
||||
gd, gdErr := rawcache.Open(filepath.Join(wc.RawDir, "_gittag", name))
|
||||
if gdErr == nil && gd.Populated() {
|
||||
gitTagDir = gd
|
||||
}
|
||||
}
|
||||
|
||||
assets, err := classifypkg.Package(name, conf, d, gitTagDir)
|
||||
if err != nil {
|
||||
return fmt.Errorf("classify: %w", err)
|
||||
}
|
||||
classifyDur := time.Since(classifyStart)
|
||||
|
||||
// Step 3: Write to fsstore.
|
||||
writeStart := time.Now()
|
||||
tx, err := wc.Store.BeginRefresh(ctx, name)
|
||||
if err != nil {
|
||||
return fmt.Errorf("begin refresh: %w", err)
|
||||
}
|
||||
if err := tx.Put(assets); err != nil {
|
||||
tx.Rollback()
|
||||
return fmt.Errorf("put: %w", err)
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return fmt.Errorf("commit: %w", err)
|
||||
}
|
||||
writeDur := time.Since(writeStart)
|
||||
|
||||
log.Printf(" %s: %d assets (classify %s, write %s, total %s)",
|
||||
name, len(assets), classifyDur, writeDur, time.Since(pkgStart))
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- Fetch raw ---
|
||||
|
||||
func (wc *WebiCache) fetchRaw(ctx context.Context, pkg pkgConf, shallow bool) error {
|
||||
switch pkg.conf.Source {
|
||||
case "github", "githubsource":
|
||||
if err := wc.fetchGitHub(ctx, pkg.name, pkg.conf, shallow); err != nil {
|
||||
return err
|
||||
}
|
||||
case "nodedist":
|
||||
return wc.fetchNodeDist(ctx, pkg.name, pkg.conf)
|
||||
case "gittag":
|
||||
return wc.fetchGitTag(ctx, pkg.name, pkg.conf, shallow)
|
||||
case "gitea":
|
||||
return wc.fetchGitea(ctx, pkg.name, pkg.conf, shallow)
|
||||
case "chromedist":
|
||||
return fetchChromeDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "flutterdist":
|
||||
return fetchFlutterDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "golang":
|
||||
return fetchGolang(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "gpgdist":
|
||||
return fetchGPGDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "hashicorp":
|
||||
return fetchHashiCorp(ctx, wc.Client, wc.RawDir, pkg.name, pkg.conf)
|
||||
case "iterm2dist":
|
||||
return fetchITerm2Dist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "juliadist":
|
||||
return fetchJuliaDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "mariadbdist":
|
||||
return fetchMariaDBDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
case "servicemandist":
|
||||
if err := servicemandist.Fetch(ctx, wc.Client, wc.RawDir, pkg.name, wc.Auth, shallow); err != nil {
|
||||
return err
|
||||
}
|
||||
case "zigdist":
|
||||
return fetchZigDist(ctx, wc.Client, wc.RawDir, pkg.name)
|
||||
default:
|
||||
log.Printf(" %s: source %q not yet supported, skipping", pkg.name, pkg.conf.Source)
|
||||
return nil
|
||||
}
|
||||
|
||||
// For non-gittag sources with a git_url, also clone the repo to get
|
||||
// commit hashes. Git entries are classified from this data in
|
||||
// refreshPackage, not from the main raw cache.
|
||||
if pkg.conf.GitURL != "" && pkg.conf.Source != "gittag" {
|
||||
gitShallow := shallow
|
||||
if !wc.Shallow {
|
||||
gd, gdErr := rawcache.Open(filepath.Join(wc.RawDir, "_gittag", pkg.name))
|
||||
if gdErr == nil && !gd.Populated() {
|
||||
gitShallow = false
|
||||
}
|
||||
}
|
||||
if err := wc.fetchGitTagSupplementary(ctx, pkg.name, pkg.conf.GitURL, gitShallow); err != nil {
|
||||
log.Printf(" %s: supplementary gittag fetch: %v", pkg.name, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// fetchGitTagSupplementary clones a git repo to get commit hashes for
|
||||
// packages that use a non-gittag source type (servicemandist, githubsource)
|
||||
// but also have a git_url for source installs.
|
||||
func (wc *WebiCache) fetchGitTagSupplementary(ctx context.Context, pkgName, gitURL string, shallow bool) error {
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, "_gittag", pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
repoDir := filepath.Join(wc.RawDir, "_repos")
|
||||
os.MkdirAll(repoDir, 0o755)
|
||||
|
||||
for batch, err := range gittag.Fetch(ctx, gitURL, repoDir) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range batch {
|
||||
tag := entry.Version
|
||||
if tag == "" {
|
||||
tag = "HEAD-" + entry.CommitHash
|
||||
}
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(tag, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchGitHub(ctx context.Context, pkgName string, conf *installerconf.Conf, shallow bool) error {
|
||||
owner, repo := conf.Owner, conf.Repo
|
||||
if owner == "" || repo == "" {
|
||||
return fmt.Errorf("missing owner or repo")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
tagPrefix := conf.TagPrefix
|
||||
for batch, err := range github.Fetch(ctx, wc.Client, owner, repo, wc.Auth) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("github %s/%s: %w", owner, repo, err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
if rel.Draft {
|
||||
continue
|
||||
}
|
||||
tag := rel.TagName
|
||||
if tagPrefix != "" && !strings.HasPrefix(tag, tagPrefix) {
|
||||
continue
|
||||
}
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(tag, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchNodeDist(ctx context.Context, pkgName string, conf *installerconf.Conf) error {
|
||||
baseURL := conf.BaseURL
|
||||
if baseURL == "" {
|
||||
return fmt.Errorf("missing url")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Fetch from primary URL. Tag with "official/" prefix so unofficial
|
||||
// entries for the same version don't overwrite.
|
||||
for batch, err := range nodedist.Fetch(ctx, wc.Client, baseURL) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range batch {
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge("official/"+entry.Version, data)
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch from unofficial URL if configured (e.g. Node.js unofficial builds
|
||||
// which add musl, riscv64, loong64 targets).
|
||||
if unofficialURL := conf.Extra["unofficial_url"]; unofficialURL != "" {
|
||||
for batch, err := range nodedist.Fetch(ctx, wc.Client, unofficialURL) {
|
||||
if err != nil {
|
||||
log.Printf("warning: %s unofficial fetch: %v", pkgName, err)
|
||||
break
|
||||
}
|
||||
for _, entry := range batch {
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge("unofficial/"+entry.Version, data)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchGitTag(ctx context.Context, pkgName string, conf *installerconf.Conf, shallow bool) error {
|
||||
gitURL := conf.BaseURL
|
||||
if gitURL == "" {
|
||||
return fmt.Errorf("missing url")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
repoDir := filepath.Join(wc.RawDir, "_repos")
|
||||
os.MkdirAll(repoDir, 0o755)
|
||||
|
||||
for batch, err := range gittag.Fetch(ctx, gitURL, repoDir) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range batch {
|
||||
tag := entry.Version
|
||||
if tag == "" {
|
||||
tag = "HEAD-" + entry.CommitHash
|
||||
}
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(tag, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (wc *WebiCache) fetchGitea(ctx context.Context, pkgName string, conf *installerconf.Conf, shallow bool) error {
|
||||
baseURL, owner, repo := conf.BaseURL, conf.Owner, conf.Repo
|
||||
if baseURL == "" || owner == "" || repo == "" {
|
||||
return fmt.Errorf("missing base_url, owner, or repo")
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(wc.RawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range gitea.Fetch(ctx, wc.Client, baseURL, owner, repo, nil) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, rel := range batch {
|
||||
if rel.Draft {
|
||||
continue
|
||||
}
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.TagName, data)
|
||||
}
|
||||
if shallow {
|
||||
break
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchChromeDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range chromedist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("chromedist: %w", err)
|
||||
}
|
||||
for _, ver := range batch {
|
||||
data, _ := json.Marshal(ver)
|
||||
d.Merge(ver.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchFlutterDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range flutterdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("flutterdist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
// Key by version+channel+os for uniqueness.
|
||||
key := rel.Version + "-" + rel.Channel + "-" + rel.OS
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(key, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchGolang(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range golang.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("golang: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchGPGDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range gpgdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("gpgdist: %w", err)
|
||||
}
|
||||
for _, entry := range batch {
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(entry.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchHashiCorp(ctx context.Context, client *http.Client, rawDir, pkgName string, conf *installerconf.Conf) error {
|
||||
product := conf.Repo
|
||||
if product == "" {
|
||||
product = pkgName
|
||||
}
|
||||
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for idx, err := range hashicorp.Fetch(ctx, client, product) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("hashicorp %s: %w", product, err)
|
||||
}
|
||||
for ver, vdata := range idx.Versions {
|
||||
data, _ := json.Marshal(vdata)
|
||||
d.Merge(ver, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchITerm2Dist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range iterm2dist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("iterm2dist: %w", err)
|
||||
}
|
||||
for _, entry := range batch {
|
||||
key := entry.Version
|
||||
if entry.Channel == "beta" {
|
||||
key += "-beta"
|
||||
}
|
||||
data, _ := json.Marshal(entry)
|
||||
d.Merge(key, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchJuliaDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range juliadist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("juliadist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchMariaDBDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range mariadbdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("mariadbdist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.ReleaseID, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func fetchZigDist(ctx context.Context, client *http.Client, rawDir, pkgName string) error {
|
||||
d, err := rawcache.Open(filepath.Join(rawDir, pkgName))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for batch, err := range zigdist.Fetch(ctx, client) {
|
||||
if err != nil {
|
||||
return fmt.Errorf("zigdist: %w", err)
|
||||
}
|
||||
for _, rel := range batch {
|
||||
data, _ := json.Marshal(rel)
|
||||
d.Merge(rel.Version, data)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -3,13 +3,13 @@
|
||||
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
|
||||
$EXENAME = "$Env:PKG_NAME.exe"
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
|
||||
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION\bin\$VERNAME")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\opt\$Env:PKG_NAME-v$Env:WEBI_VERSION\bin\$VERNAME")) {
|
||||
Write-Output "Installing $Env:PKG_NAME"
|
||||
# TODO: temp directory
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = kivikakk/comrak
|
||||
40
comrak/releases.js
Normal file
40
comrak/releases.js
Normal file
@@ -0,0 +1,40 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'kivikakk';
|
||||
var repo = 'comrak';
|
||||
|
||||
var ODDITIES = ['-musleabihf.1-'];
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
let builds = [];
|
||||
|
||||
loopBuilds: for (let build of all.releases) {
|
||||
let isOddity;
|
||||
for (let oddity of ODDITIES) {
|
||||
isOddity = build.name.includes(oddity);
|
||||
if (isOddity) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (isOddity) {
|
||||
continue;
|
||||
}
|
||||
|
||||
builds.push(build);
|
||||
}
|
||||
|
||||
all.releases = builds;
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
all.releases = all.releases.slice(0, 10);
|
||||
//console.info(JSON.stringify(all));
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$pkg_download")) {
|
||||
IF (!(Test-Path -Path "$pkg_download")) {
|
||||
Write-Output "Downloading crabz from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing crabz"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = sstadick/crabz
|
||||
31
crabz/releases.js
Normal file
31
crabz/releases.js
Normal file
@@ -0,0 +1,31 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'sstadick';
|
||||
var repo = 'crabz';
|
||||
|
||||
module.exports = async function (request) {
|
||||
let all = await github(request, owner, repo);
|
||||
|
||||
let releases = [];
|
||||
for (let rel of all.releases) {
|
||||
let isSrc = rel.download.includes('-src.');
|
||||
if (isSrc) {
|
||||
continue;
|
||||
}
|
||||
|
||||
releases.push(rel);
|
||||
}
|
||||
all.releases = releases;
|
||||
|
||||
return all;
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -3,13 +3,13 @@
|
||||
$VERNAME = "$Env:PKG_NAME-v$Env:WEBI_VERSION.exe"
|
||||
$EXENAME = "$Env:PKG_NAME.exe"
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading $Env:PKG_NAME from $Env:WEBI_PKG_URL to $Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part"
|
||||
& Move-Item "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE.part" "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\.local\bin\$VERNAME")) {
|
||||
Write-Output "Installing $Env:PKG_NAME"
|
||||
# TODO: temp directory
|
||||
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = rs/curlie
|
||||
21
curlie/releases.js
Normal file
21
curlie/releases.js
Normal file
@@ -0,0 +1,21 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'rs';
|
||||
var repo = 'curlie';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
all._names = ['curlie', 'curl-httpie'];
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
all.releases = all.releases.slice(0, 10);
|
||||
//console.info(JSON.stringify(all));
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -100,7 +100,8 @@ mkdir -p ~/.dashcore/wallets/
|
||||
mkdir -p /mnt/slc1_vol_100g/dashcore/_data
|
||||
mkdir -p /mnt/slc1_vol_100g/dashcore/_caches
|
||||
|
||||
serviceman add --name 'dashd' --daemon -- \
|
||||
sudo env PATH="$PATH" serviceman add \
|
||||
--system --user "$my_user" --path "$PATH" --name dashd --force -- \
|
||||
dashd \
|
||||
-usehd \
|
||||
-conf="$HOME/.dashcore/dash.conf" \
|
||||
|
||||
@@ -84,8 +84,20 @@ fn_srv_install() { (
|
||||
my_name="dashd-${my_netname}"
|
||||
fi
|
||||
|
||||
my_system_args=""
|
||||
my_kernel="$(
|
||||
uname -s
|
||||
)"
|
||||
if test "Darwin" != "${my_kernel}"; then
|
||||
my_user="$(
|
||||
id -u -n
|
||||
)"
|
||||
my_system_args="--system --username ${my_user}"
|
||||
fi
|
||||
|
||||
# shellcheck disable=SC2016,SC1090
|
||||
echo "serviceman add --name \"${my_name}\" --" \
|
||||
echo 'sudo env PATH="$PATH"' \
|
||||
"serviceman add ${my_system_args} --path \"\$PATH\" --name \"${my_name}\" --force --" \
|
||||
"dashd " \
|
||||
"${my_net_flag}" \
|
||||
-usehd \
|
||||
@@ -95,16 +107,16 @@ fn_srv_install() { (
|
||||
"-datadir=\"${my_datadir}\"" \
|
||||
"-blocksdir=\"${my_blocksdir}\""
|
||||
|
||||
echo ""
|
||||
echo "Installing latest 'serviceman'..."
|
||||
echo ""
|
||||
"$HOME/.local/bin/webi" serviceman > /dev/null
|
||||
if ! command -v serviceman > /dev/null; then
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
fi
|
||||
serviceman --version
|
||||
if ! command -v dashd > /dev/null; then
|
||||
export PATH="$HOME/.local/opt/dashcore/bin:$PATH"
|
||||
echo ""
|
||||
echo "Installing 'serviceman'..."
|
||||
echo ""
|
||||
{
|
||||
"$HOME/.local/bin/webi" serviceman
|
||||
} > /dev/null
|
||||
|
||||
# shellcheck disable=SC1090
|
||||
. ~/.config/envman/PATH.env || true
|
||||
fi
|
||||
|
||||
mkdir -p "$HOME/.dashcore/wallets/"
|
||||
@@ -119,7 +131,8 @@ fn_srv_install() { (
|
||||
cd "${my_vol}" || return 1
|
||||
# leave options unquoted so they're interpreted separately
|
||||
# shellcheck disable=SC2086
|
||||
serviceman add --name "${my_name}" -- \
|
||||
sudo env PATH="${PATH}" \
|
||||
serviceman add ${my_system_args} --path "${PATH}" --name "${my_name}" --force -- \
|
||||
dashd \
|
||||
${my_net_flag} \
|
||||
-usehd \
|
||||
|
||||
@@ -147,8 +147,8 @@ dash-qt \
|
||||
CoinJoin aids in preventing some bad actors and malicious observers being able
|
||||
to easily reconstruct details about your transactions from the publicly
|
||||
available data by creating many excess transactions. \
|
||||
(be aware, however, that dedicated bad actors can use sophisticated software
|
||||
that will reveal much of the same information over time)
|
||||
(be aware, however, that dedicated bad actors can use sophisticated software that
|
||||
will reveal much of the same information over time)
|
||||
|
||||
`dash-qt` does not enable CoinJoin mixing by default.
|
||||
|
||||
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading dashcore from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
Write-Output "Installing dashcore"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = dashpay/dash
|
||||
31
dashcore/releases.js
Normal file
31
dashcore/releases.js
Normal file
@@ -0,0 +1,31 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'dashpay';
|
||||
var repo = 'dash';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
all.releases.forEach(function (rel) {
|
||||
if (rel.name.includes('osx64')) {
|
||||
rel.os = 'macos';
|
||||
}
|
||||
|
||||
if (rel.version.startsWith('v')) {
|
||||
rel._version = rel.version.slice(1);
|
||||
}
|
||||
});
|
||||
|
||||
all._names = ['dashd', 'dashcore'];
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
// just select the first 5 for demonstration
|
||||
all.releases = all.releases.slice(0, 5);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -219,7 +219,14 @@ You can use [`serviceman`](../serviceman/):
|
||||
**Linux**
|
||||
|
||||
```sh
|
||||
serviceman add --name 'dashd' -- \
|
||||
sudo env PATH="$PATH" \
|
||||
serviceman add \
|
||||
--system \
|
||||
--username "$(id -n -u)" \
|
||||
--path "$PATH" \
|
||||
--name dashd \
|
||||
--force \
|
||||
-- \
|
||||
dashd \
|
||||
-usehd \
|
||||
-conf="$HOME/.dashcore/dash.conf" \
|
||||
@@ -232,7 +239,11 @@ serviceman add --name 'dashd' -- \
|
||||
**Mac**
|
||||
|
||||
```sh
|
||||
serviceman add --name 'dashd' -- \
|
||||
serviceman add \
|
||||
--path "$PATH" \
|
||||
--name dashd \
|
||||
--force \
|
||||
-- \
|
||||
dashd \
|
||||
-usehd \
|
||||
-conf="$HOME/.dashcore/dash.conf" \
|
||||
|
||||
@@ -20,13 +20,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading dashd from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_dir")) {
|
||||
Write-Output "Installing dashd"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
alias_of = dashcore
|
||||
3
dashd/releases.js
Normal file
3
dashd/releases.js
Normal file
@@ -0,0 +1,3 @@
|
||||
'use strict';
|
||||
|
||||
module.exports = require('../dashcore/releases.js');
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading dashmsg from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing dashmsg"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = dashhive/dashmsg
|
||||
18
dashmsg/releases.js
Normal file
18
dashmsg/releases.js
Normal file
@@ -0,0 +1,18 @@
|
||||
'use strict';
|
||||
|
||||
var github = require('../_common/github.js');
|
||||
var owner = 'dashhive';
|
||||
var repo = 'dashmsg';
|
||||
|
||||
module.exports = function (request) {
|
||||
return github(request, owner, repo).then(function (all) {
|
||||
return all;
|
||||
});
|
||||
};
|
||||
|
||||
if (module === require.main) {
|
||||
module.exports(require('@root/request')).then(function (all) {
|
||||
all = require('../_webi/normalize.js')(all);
|
||||
console.info(JSON.stringify(all, null, 2));
|
||||
});
|
||||
}
|
||||
@@ -19,13 +19,13 @@ New-Item "$Env:USERPROFILE\Downloads\webi" -ItemType Directory -Force | Out-Null
|
||||
$pkg_download = "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE"
|
||||
|
||||
# Fetch archive
|
||||
if (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
IF (!(Test-Path -Path "$Env:USERPROFILE\Downloads\webi\$Env:WEBI_PKG_FILE")) {
|
||||
Write-Output "Downloading delta from $Env:WEBI_PKG_URL to $pkg_download"
|
||||
& curl.exe -A "$Env:WEBI_UA" -fsSL "$Env:WEBI_PKG_URL" -o "$pkg_download.part"
|
||||
& Move-Item "$pkg_download.part" "$pkg_download"
|
||||
}
|
||||
|
||||
if (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
IF (!(Test-Path -Path "$pkg_src_cmd")) {
|
||||
Write-Output "Installing delta"
|
||||
|
||||
# TODO: create package-specific temp directory
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
github_releases = dandavison/delta
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user