Compare commits

..

75 Commits

Author SHA1 Message Date
Jacob Blain Christen
3406d5453d [feature] build with boring crypto where available (#344) 2024-10-04 15:09:20 -07:00
Zack Brady
991f5b6bc1 updated workflow to goreleaser builds (#341)
* updated workflow to goreleaser builds
2024-10-02 11:12:32 -07:00
Zack Brady
0595ab043a added timeout to goreleaser workflow (#340) 2024-10-01 21:18:19 -04:00
Zack Brady
73e5c1ec8b trying new workflow build processes (#337)
* trying new workflow build processes
* added last bit to new try
2024-10-01 20:07:02 -04:00
Zack Brady
25d8cb83b2 improved workflow performance (#336) 2024-10-01 16:10:37 -04:00
Adam Martin
9f7229a36b have extract use proper ref (#335)
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-10-01 16:04:06 -04:00
Zack Brady
b294b6f026 yet another workflow goreleaser fix (#334) 2024-10-01 13:21:16 -04:00
Zack Brady
ebd3fd66c8 even more workflow fixes (#333)
* reverted build hooks
* updated goreleaser arguments
2024-10-01 12:26:34 -04:00
Zack Brady
6373a476b5 added more fixes to github workflow (#332) 2024-10-01 09:23:30 -04:00
Zack Brady
2c7aacd105 fixed typo in hauler store save (#331)
* fixed typo in hauler store save
* update internal/flags/save.go

Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
Signed-off-by: Zack Brady <zackbrady123@gmail.com>

---------

Signed-off-by: Zack Brady <zackbrady123@gmail.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-09-30 18:06:10 -04:00
Zack Brady
bbcbe0239a updates to fix build processes (#330) 2024-09-30 16:51:47 -04:00
Zack Brady
8a53a26a58 added integration tests for non hauler tarballs (#325)
* added tests for tarballs
* updated tests for tarball changes
* fixed tests/build for latest changes
2024-09-27 16:38:39 -04:00
Jacob Blain Christen
41d88954c6 bump: golang >= 1.23.1 (#328)
Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
2024-09-24 11:27:28 -07:00
Adam Martin
caaed30297 add platform flag to store save (#329)
* add platform flag to store save

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* use platform parser from go-cr

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-09-24 11:18:19 -04:00
Jacob Blain Christen
aee296d48d Update feature_request.md
Remove RFE from prompt.

Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
2024-09-23 10:19:11 -07:00
Zack Brady
407ed94a0b updated/standardize command descriptions (#313)
* initial desc formatting/updates
* fixed typos
* updated commands base on feedback
* more updates based on feedback

---------

Signed-off-by: Zack Brady <zackbrady123@gmail.com>
2024-09-20 18:19:49 -04:00
Adam Martin
15a9e1a3c4 use new annotation for 'store save' manifest.json (#324)
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-09-20 17:14:27 -04:00
Jacob Blain Christen
6510947bb9 enable docker load for hauler tarballs (#320)
- fixes #276

Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
2024-09-20 13:32:49 -04:00
Adam Martin
01eebd54af bump to cosign v2.2.3-carbide.3 for new annotation (#322)
* bump cosign to v2.2.3-carbide.3
* use new annotation in 'store info' when listing images

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-09-18 09:56:22 -07:00
Adam Martin
5aa55e9eda continue on error when adding images to store (#317)
* continue on error when adding images to store

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* Update cmd/hauler/cli/store/add.go

Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
Signed-off-by: Adam Martin <42001113+amartin120@users.noreply.github.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
Signed-off-by: Adam Martin <42001113+amartin120@users.noreply.github.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-09-04 14:11:07 -04:00
Brandon
6f8cd04a32 Update README.md (#318)
Removing the weird GA "disclaimer"

Signed-off-by: Brandon <bgulla@users.noreply.github.com>
2024-09-04 11:10:29 -07:00
Zack Brady
02231d716f fixed completion commands (#312) 2024-08-29 19:10:59 -04:00
Jacob Blain Christen
16fa03fec8 github.com/rancherfederal/hauler => hauler.dev/go/hauler (#311) 2024-08-26 13:54:06 -07:00
Jacob Blain Christen
51fe531c64 pages: enable go install hauler.dev/go/hauler (#310)
should fix:
```
go install hauler.dev/go/hauler@latest
go: hauler.dev/go/hauler@latest: unrecognized import path "hauler.dev/go/hauler": reading https://hauler.dev/go/hauler?go-get=1: 404 Not Found
```

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-26 12:12:59 -07:00
Jacob Blain Christen
1a6ce4290f Create CNAME
Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
2024-08-26 11:32:55 -07:00
Jacob Blain Christen
e4ec7bed76 pages: initial workflow (#309)
Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-26 11:25:54 -07:00
Zack Brady
cb81823487 testing and linting updates (#305)
* updated makefile operations

* upgraded testdata and related tests

* initial updates for tests

* fixed errors with testdata

* last bit of updates to tests

* cleaned and commented makefile

* updated tests for latest merges

---------

Signed-off-by: Zack Brady <zackbrady123@gmail.com>
2024-08-26 12:27:18 -04:00
will
2d930b5653 feat-273: TLS Flags (#303)
* fix: move constant and flags to prevent loop

* feat: add tls cert to serve

* fix: add tls cli description

* fix: remove unnecessary code

* small updates/fixed unit test errors

* fix: migrate all flags, use exported vars

* fix: standardize to AddFlags

---------

Signed-off-by: will <30413278+wcrum@users.noreply.github.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
2024-08-25 16:16:37 -04:00
Zack Brady
bd0cd8f428 added list-repos flag (#298)
Co-authored-by: Adam Toy <atoy3731@gmail.com>
2024-08-23 09:25:02 -04:00
Zack Brady
d6b3c94920 fixed hauler login typo (#299)
* updated hauler login typo

* updated hauler login descs
2024-08-23 09:12:46 -04:00
Allen Conlon
20958826ef updated cobra function for shell completion (#304) 2024-08-22 22:06:16 -04:00
Zack Brady
d633eeffcc updated install.sh to remove github api (#293)
* updated install.sh to remove github api

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>

---------

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-14 12:27:46 -07:00
Adam Martin
c592551a37 fix image ref keys getting squashed when containing sigs/atts (#291)
* fix image ref keys getting squashed when containing sigs/atts

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
Co-authored-by: Adam Martin <adam.martin@rancherfederal.com>
2024-08-13 12:18:10 -07:00
Jacob Blain Christen
ef3eb05fce fix missing versin info in release build (#283)
Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-05 10:11:40 -07:00
dependabot[bot]
3f64914097 bump github.com/docker/docker in the go_modules group across 1 directory (#281)
Bumps the go_modules group with 1 update in the / directory: [github.com/docker/docker](https://github.com/docker/docker).

Updates `github.com/docker/docker` from 25.0.5+incompatible to 25.0.6+incompatible
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v25.0.5...v25.0.6)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
2024-08-01 17:51:34 -04:00
Zack Brady
6a74668e2c updated install script (install.sh) (#280)
* removed sudo requirement from install script
* updated install script to specify install directory
* cleaned up install script
* a bit more changes and updates to the install script
* updated install script variable syntax
* added missing logic to install script
2024-08-01 17:48:26 -04:00
Kamin Fay
0c5cf20e87 fix digest images being lost on load of hauls (Signed). (#259)
* Adding oci tests
* Fixed the oci pusher to split on the correct '@' symbol
* Reverted Pusher changes and adjusted nameMap references in Index

---------

Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-07-30 23:44:47 -04:00
will
513719bc9e feat: add readonly flag (#277)
* updated flag from writeable to readonly (default=true)

---------

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-07-30 11:26:39 -07:00
Zack Brady
047b7a7003 fixed makefile for goreleaser v2 changes (#278) 2024-07-30 13:18:17 -04:00
Zack Brady
a4685169c6 updated goreleaser versioning defaults (#279) 2024-07-30 13:17:40 -04:00
Jacob Blain Christen
47549615c4 update feature_request.md (#274)
- `[RFE]` ➡️ `[feature]`

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-07-26 14:48:08 -04:00
Zack Brady
2d725026dc merge pull request #271 from zackbradys/removed-some-references
removed some old references
2024-07-23 13:56:59 -04:00
Zack Brady
60667b7116 updated old references 2024-07-22 18:50:11 -04:00
Zack Brady
7d62a1c98e updated actions workflow user 2024-07-22 18:46:53 -04:00
Zack Brady
894ffb1533 merge pull request #269 from zackbradys/add-dockerhub-support
added dockerhub to github actions workflow
2024-07-20 01:18:08 -04:00
Zack Brady
78b3442d23 added dockerhub to github actions workflow 2024-07-20 01:12:59 -04:00
Zack Brady
cd46febb6b merge pull request #268 from zackbradys/remove-helm-chart
removed helm chart
2024-07-20 01:10:37 -04:00
Zack Brady
0957a930dd removed helm chart 2024-07-20 00:57:25 -04:00
Zack Brady
a6bc6308d9 merge pull request #267 from zackbradys/main
added debug container and workflow
2024-07-19 22:21:38 -04:00
Zack Brady
1304cf6c76 added debug container and workflow 2024-07-17 00:01:24 -04:00
Zack Brady
f2e02c80c0 merge pull request #262 from zackbradys/main
updated products flag description
2024-07-12 22:52:43 -04:00
Zack Brady
25806e993e updated products flag description 2024-07-12 01:25:05 -04:00
Zack Brady
05e67bc750 merge pull request #255 from zackbradys/main
updated chart for release
2024-06-25 23:27:24 -04:00
Zack Brady
b43ed0503a updated chart for release 2024-06-25 23:14:44 -04:00
Zack Brady
27e2fc9de0 merge pull request #254 from zackbradys/main
fixed workflow errors/warnings
2024-06-25 22:48:30 -04:00
Zack Brady
d32d75b93e fixed workflow errors/warnings 2024-06-25 22:43:31 -04:00
Zack Brady
ceb77601d0 merge pull request #252 from zackbradys/main
overhauling github actions and workflows
2024-06-24 22:52:08 -04:00
Zack Brady
d90545a9e4 fixed permissions on testdata 2024-06-17 22:23:03 -04:00
Zack Brady
bef141ab67 updated chart versions (will need to update again) 2024-06-17 19:45:18 -04:00
Zack Brady
385d767c2a last bit of fixes to workflow 2024-06-17 19:42:20 -04:00
Zack Brady
22edc77506 updated unit test workflow 2024-06-14 20:56:37 -04:00
Zack Brady
9058797bbc updated goreleaser deprecations 2024-06-14 20:43:32 -04:00
Zack Brady
35e2f655da added helm chart release job 2024-06-14 20:37:58 -04:00
Zack Brady
f5c0f6f0ae updated github template names 2024-06-14 20:04:01 -04:00
Zack Brady
0ec77b4168 merge pull request #248 from zackbradys/main
formatted all code with `go fmt`
2024-06-14 16:46:21 -04:00
Zack Brady
7a7906b8ea updated imports (and go fmt) 2024-06-13 23:44:06 -04:00
Zack Brady
f4774445f6 merge pull request #240 from pat-earl/doc_updates
added some documentation text to sync command
2024-06-10 18:13:09 -04:00
Zack Brady
d59b29bfce formatted gitignore to match dockerignore 2024-06-05 14:40:50 -04:00
Zack Brady
fd702202ac formatted all code (go fmt) 2024-06-05 08:25:45 -04:00
Adam Martin
9e9565717b Merge pull request #245 from ethanchowell/ehowell/helm-client
Configure chart commands to use helm clients for OCI and private regi…
2024-06-04 13:16:42 -04:00
Zack Brady
bfe47ae141 updated chart tests for new features 2024-06-03 23:31:50 -04:00
Zack Brady
ebab7f38a0 merge pull request #247 from kaminfay/dev/add-fileserver-timeout-flag
adding the timeout flag for fileserver command and setting default timeout to 60 seconds
2024-06-03 22:33:36 -04:00
Kamin Fay
f0cba3c2c6 Adding the timeout flag for fileserver command 2024-05-28 15:28:20 -04:00
Ethan Howell
286120da50 Configure chart commands to use helm clients for OCI and private registry support 2024-05-24 12:06:16 -04:00
Patrick Earl
dcdeb93518 Added some documentation text to sync command 2024-05-02 14:17:38 -04:00
105 changed files with 1948 additions and 1521 deletions

View File

@@ -1,8 +0,0 @@
*
!cmd
!go.mod
!go.sum
!internal
!Makefile
!pkg
!static

View File

@@ -1,6 +1,6 @@
---
name: Bug Report
about: Create a report to help us improve!
about: Submit a bug report to help us improve!
title: '[BUG]'
labels: 'bug'
assignees: ''

View File

@@ -1,14 +1,14 @@
---
name: Feature Request
about: Submit a request for us to improve!
title: '[RFE]'
about: Submit a feature request for us to improve!
title: '[feature]'
labels: 'enhancement'
assignees: ''
---
<!-- Thank you for helping us to improve Hauler! We welcome all requests for enhancements (RFEs). Please fill out each area of the template so we can better assist you. Comments like this will be hidden when you submit, but you can delete them if you wish. -->
**Is this RFE related to an Existing Problem? If so, please describe:**
**Is this Feature/Enhancement related to an Existing Problem? If so, please describe:**
<!-- Provide a clear and concise description of the problem -->

40
.github/workflows/pages.yaml vendored Normal file
View File

@@ -0,0 +1,40 @@
# Simple workflow for deploying static content to GitHub Pages
name: 📋
on:
push:
branches:
- main
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
# Single deploy job since we're just deploying
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: './static'
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

View File

@@ -7,40 +7,26 @@ on:
- '*'
jobs:
go-release:
name: Go Release Job
goreleaser:
name: GoReleaser Job
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Run Go Releaser
uses: goreleaser/goreleaser-action@v5
with:
distribution: goreleaser
version: latest
args: release --clean -p 1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
container-release:
name: Container Release Job
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
go-version-file: go.mod
check-latest: true
- name: Set Up QEMU
uses: docker/setup-qemu-action@v3
@@ -48,17 +34,27 @@ jobs:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GitHub Container Registry
- name: Authenticate to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and Push to GitHub Container Registry
uses: docker/build-push-action@v5
- name: Authenticate to DockerHub Container Registry
uses: docker/login-action@v3
with:
context: .
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.ref_name }}
registry: docker.io
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: "~> v2"
args: "release --clean --parallelism 1 --timeout 60m"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
DOCKER_CLI_EXPERIMENTAL: "enabled"

333
.github/workflows/tests.yaml vendored Normal file
View File

@@ -0,0 +1,333 @@
name: Tests Workflow
on:
workflow_dispatch:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
check-latest: true
- name: Install Go Releaser
uses: goreleaser/goreleaser-action@v6
with:
install-only: true
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt-get install -y make
sudo apt-get install -y build-essential
- name: Run Makefile Targets
run: |
make build-all
- name: Upload Hauler Binaries
uses: actions/upload-artifact@v4
with:
name: hauler-binaries
path: dist/*
- name: Upload Coverage Report
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage.out
integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
needs: [unit-tests]
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt-get install -y unzip
sudo apt-get install -y tree
- name: Download Artifacts
uses: actions/download-artifact@v4
with:
name: hauler-binaries
path: dist
- name: Prepare Hauler for Tests
run: |
pwd
ls -la
ls -la dist/
chmod -R 755 dist/ testdata/certificate-script.sh
sudo mv dist/hauler_linux_amd64_v1/hauler /usr/local/bin/hauler
./testdata/certificate-script.sh && sudo chown -R $(whoami) testdata/certs/
- name: Verify - hauler version
run: |
hauler version
- name: Verify - hauler completion
run: |
hauler completion
hauler completion bash
hauler completion fish
hauler completion powershell
hauler completion zsh
- name: Verify - hauler help
run: |
hauler help
- name: Verify - hauler login
run: |
hauler login --help
hauler login docker.io --username bob --password haulin
echo "hauler" | hauler login docker.io -u bob --password-stdin
- name: Remove Hauler Store Credentials
run: |
rm -rf ~/.docker/config.json
- name: Verify - hauler store
run: |
hauler store --help
- name: Verify - hauler store add
run: |
hauler store add --help
- name: Verify - hauler store add chart
run: |
hauler store add chart --help
# verify via helm repository
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable --version 2.8.4
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable --version 2.8.3 --verify
# verify via oci helm repository
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev --version 1.0.6
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev --version 1.0.4 --verify
# verify via local helm repository
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.2/rancher-cluster-templates-0.5.2.tgz
hauler store add chart rancher-cluster-templates-0.5.2.tgz --repo .
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.1/rancher-cluster-templates-0.5.1.tgz
hauler store add chart rancher-cluster-templates-0.5.1.tgz --repo . --version 0.5.1
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.0/rancher-cluster-templates-0.5.0.tgz
hauler store add chart rancher-cluster-templates-0.5.0.tgz --repo . --version 0.5.0 --verify
# verify via the hauler store contents
hauler store info
- name: Verify - hauler store add file
run: |
hauler store add file --help
# verify via remote file
hauler store add file https://get.rke2.io/install.sh
hauler store add file https://get.rke2.io/install.sh --name rke2-install.sh
# verify via local file
hauler store add file testdata/hauler-manifest.yaml
hauler store add file testdata/hauler-manifest.yaml --name hauler-manifest-local.yaml
# verify via the hauler store contents
hauler store info
- name: Verify - hauler store add image
run: |
hauler store add image --help
# verify via image reference
hauler store add image busybox
# verify via image reference with version and platform
hauler store add image busybox:stable --platform linux/amd64
# verify via image reference with full reference
hauler store add image gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5
# verify via the hauler store contents
hauler store info
- name: Verify - hauler store copy
run: |
hauler store copy --help
# need more tests here
- name: Verify - hauler store extract
run: |
hauler store extract --help
# verify via extracting hauler store content
hauler store extract hauler/hauler-manifest-local.yaml:latest
# view extracted content from store
cat hauler-manifest-local.yaml
- name: Verify - hauler store info
run: |
hauler store info --help
# verify via table output
hauler store info --output table
# verify via json output
hauler store info --output json
# verify via filtered output (chart)
hauler store info --type chart
# verify via filtered output (file)
hauler store info --type file
# verify via filtered output (image)
hauler store info --type image
# verify store directory structure
tree -hC store
- name: Verify - hauler store save
run: |
hauler store save --help
# verify via save
hauler store save
# verify via save with filename
hauler store save --filename store.tar.zst
# verify via save with filename and platform (amd64)
hauler store save --filename store-amd64.tar.zst --platform linux/amd64
- name: Remove Hauler Store Contents
run: |
rm -rf store
hauler store info
- name: Verify - hauler store load
run: |
hauler store load --help
# verify via load
hauler store load haul.tar.zst
# verify via load with filename and temp directory
hauler store load store.tar.zst --tempdir /opt
# verify via load with filename and platform (amd64)
hauler store load store-amd64.tar.zst
- name: Verify Hauler Store Contents
run: |
# verify store
hauler store info
# verify store directory structure
tree -hC store
- name: Verify - docker load
run: |
docker load --help
# verify via load
docker load --input store-amd64.tar.zst
- name: Verify Docker Images Contents
run: |
docker images --help
# verify images
docker images --all
- name: Remove Hauler Store Contents
run: |
rm -rf store haul.tar.zst store.tar.zst store-amd64.tar.zst
hauler store info
- name: Verify - hauler store sync
run: |
hauler store sync --help
# download local helm repository
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.2/rancher-cluster-templates-0.5.2.tgz
# verify via sync
hauler store sync --files testdata/hauler-manifest-pipeline.yaml
# need more tests here
- name: Verify - hauler store serve
run: |
hauler store serve --help
- name: Verify - hauler store serve registry
run: |
hauler store serve registry --help
# verify via registry
hauler store serve registry &
until curl -sf http://localhost:5000/v2/_catalog; do : ; done
pkill -f "hauler store serve registry"
# verify via registry with different port
hauler store serve registry --port 5001 &
until curl -sf http://localhost:5001/v2/_catalog; do : ; done
pkill -f "hauler store serve registry --port 5001"
# verify via registry with different port and readonly
hauler store serve registry --port 5001 --readonly &
until curl -sf http://localhost:5001/v2/_catalog; do : ; done
pkill -f "hauler store serve registry --port 5001 --readonly"
# verify via registry with different port with readonly with tls
# hauler store serve registry --port 5001 --readonly --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key &
# until curl -sf --cacert testdata/certs/cacerts.pem https://localhost:5001/v2/_catalog; do : ; done
# pkill -f "hauler store serve registry --port 5001 --readonly --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key"
- name: Verify - hauler store serve fileserver
run: |
hauler store serve fileserver --help
# verify via fileserver
hauler store serve fileserver &
until curl -sf http://localhost:8080; do : ; done
pkill -f "hauler store serve fileserver"
# verify via fileserver with different port
hauler store serve fileserver --port 8000 &
until curl -sf http://localhost:8000; do : ; done
pkill -f "hauler store serve fileserver --port 8000"
# verify via fileserver with different port and timeout
hauler store serve fileserver --port 8000 --timeout 120 &
until curl -sf http://localhost:8000; do : ; done
pkill -f "hauler store serve fileserver --port 8000 --timeout 120"
# verify via fileserver with different port with timeout and tls
# hauler store serve fileserver --port 8000 --timeout 120 --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key &
# until curl -sf --cacert testdata/certs/cacerts.pem https://localhost:8000; do : ; done
# pkill -f "hauler store serve fileserver --port 8000 --timeout 120 --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key"
- name: Verify Hauler Store Contents
run: |
# verify store
hauler store info
# verify store directory structure
tree -hC store
# verify registry directory structure
tree -hC registry
# verify fileserver directory structure
tree -hC fileserver
- name: Create Hauler Report
run: |
hauler version >> hauler-report.txt
hauler store info --output table >> hauler-report.txt
- name: Remove Hauler Store Contents
run: |
rm -rf store registry fileserver
hauler store info
- name: Upload Hauler Report
uses: actions/upload-artifact@v4
with:
name: hauler-report
path: hauler-report.txt

View File

@@ -1,37 +0,0 @@
name: Unit Test Workflow
on:
workflow_dispatch:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
unit-test:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Run Unit Tests
run: |
mkdir -p cmd/hauler/binaries
touch cmd/hauler/binaries/dummy.txt
go test -race -covermode=atomic -coverprofile=coverage.out ./pkg/... ./internal/... ./cmd/...
- name: On Failure, Launch Debug Session
if: ${{ failure() }}
uses: mxschmitt/action-tmate@v3
timeout-minutes: 10

21
.gitignore vendored
View File

@@ -1,9 +1,4 @@
.DS_Store
# Vagrant
.vagrant
# Editor directories and files
**/.DS_Store
.idea
.vscode
*.suo
@@ -12,16 +7,12 @@
*.sln
*.sw?
*.dir-locals.el
# old, ad-hoc ignores
artifacts
local-artifacts
airgap-scp.sh
# generated
dist/
tmp/
bin/
/store/
/registry/
store/
registry/
fileserver/
cmd/hauler/binaries
testdata/certs/
coverage.out

View File

@@ -1,19 +1,28 @@
version: 2
project_name: hauler
before:
hooks:
- rm -rf cmd/hauler/binaries
- mkdir -p cmd/hauler/binaries
- touch cmd/hauler/binaries/file
- go mod tidy
- go mod download
- go fmt ./...
- go vet ./...
- go test ./... -cover -race -covermode=atomic -coverprofile=coverage.out
- rm -rf cmd/hauler/binaries
release:
prerelease: auto
make_latest: false
env:
- vpkg=github.com/rancherfederal/hauler/internal/version
- cosign_version=v2.2.3+carbide.2
- vpkg=hauler.dev/go/hauler/internal/version
- cosign_version=v2.2.3+carbide.3
builds:
- main: cmd/hauler/main.go
- dir: ./cmd/hauler/.
goos:
- linux
- darwin
@@ -25,25 +34,97 @@ builds:
- -s -w -X {{ .Env.vpkg }}.gitVersion={{ .Version }} -X {{ .Env.vpkg }}.gitCommit={{ .ShortCommit }} -X {{ .Env.vpkg }}.gitTreeState={{if .IsGitDirty}}dirty{{else}}clean{{end}} -X {{ .Env.vpkg }}.buildDate={{ .Date }}
hooks:
pre:
- mkdir -p cmd/hauler/binaries
- wget -P cmd/hauler/binaries/ https://github.com/rancher-government-carbide/cosign/releases/download/{{ .Env.cosign_version }}/cosign-{{ .Os }}-{{ .Arch }}{{ if eq .Os "windows" }}.exe{{ end }}
- wget -P cmd/hauler/binaries/ https://github.com/hauler-dev/cosign/releases/download/{{ .Env.cosign_version }}/cosign-{{ .Os }}-{{ .Arch }}{{ if eq .Os "windows" }}.exe{{ end }}
post:
- rm -rf cmd/hauler/binaries
env:
- CGO_ENABLED=0
- GOEXPERIMENT=boringcrypto
universal_binaries:
- replace: false
changelog:
skip: false
disable: false
use: git
brews:
- name: hauler
tap:
owner: rancherfederal
repository:
owner: hauler-dev
name: homebrew-tap
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
folder: Formula
directory: Formula
description: "Hauler CLI"
dockers:
- id: hauler-amd64
goos: linux
goarch: amd64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/amd64"
- "--target=release"
image_templates:
- "docker.io/hauler/hauler-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-amd64:{{ .Version }}"
- id: hauler-arm64
goos: linux
goarch: arm64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/arm64"
- "--target=release"
image_templates:
- "docker.io/hauler/hauler-arm64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-arm64:{{ .Version }}"
- id: hauler-debug-amd64
goos: linux
goarch: amd64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/amd64"
- "--target=debug"
image_templates:
- "docker.io/hauler/hauler-debug-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-debug-amd64:{{ .Version }}"
- id: hauler-debug-arm64
goos: linux
goarch: arm64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/arm64"
- "--target=debug"
image_templates:
- "docker.io/hauler/hauler-debug-arm64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-debug-arm64:{{ .Version }}"
docker_manifests:
- id: hauler-docker
use: docker
name_template: "docker.io/hauler/hauler:{{ .Version }}"
image_templates:
- "docker.io/hauler/hauler-amd64:{{ .Version }}"
- "docker.io/hauler/hauler-arm64:{{ .Version }}"
- id: hauler-ghcr
use: docker
name_template: "ghcr.io/hauler-dev/hauler:{{ .Version }}"
image_templates:
- "ghcr.io/hauler-dev/hauler-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-arm64:{{ .Version }}"
- id: hauler-debug-docker
use: docker
name_template: "docker.io/hauler/hauler-debug:{{ .Version }}"
image_templates:
- "docker.io/hauler/hauler-debug-amd64:{{ .Version }}"
- "docker.io/hauler/hauler-debug-arm64:{{ .Version }}"
- id: hauler-debug-ghcr
use: docker
name_template: "ghcr.io/hauler-dev/hauler-debug:{{ .Version }}"
image_templates:
- "ghcr.io/hauler-dev/hauler-debug-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-debug-arm64:{{ .Version }}"

View File

@@ -1,25 +1,42 @@
FROM registry.suse.com/bci/golang:1.21 AS builder
RUN zypper --non-interactive install make bash wget ca-certificates
# builder stage
FROM registry.suse.com/bci/bci-base:15.5 AS builder
COPY . /build
WORKDIR /build
RUN make build
# fetched from goreleaser build proccess
COPY hauler /hauler
RUN echo "hauler:x:1001:1001::/home:" > /etc/passwd \
RUN echo "hauler:x:1001:1001::/home/hauler:" > /etc/passwd \
&& echo "hauler:x:1001:hauler" > /etc/group \
&& mkdir /home/hauler \
&& mkdir /store \
&& mkdir /fileserver \
&& mkdir /registry
FROM scratch
# release stage
FROM scratch AS release
COPY --from=builder /var/lib/ca-certificates/ca-bundle.pem /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group
COPY --from=builder --chown=hauler:hauler /home/. /home
COPY --from=builder --chown=hauler:hauler /home/hauler/. /home/hauler
COPY --from=builder --chown=hauler:hauler /tmp/. /tmp
COPY --from=builder --chown=hauler:hauler /store/. /store
COPY --from=builder --chown=hauler:hauler /registry/. /registry
COPY --from=builder --chown=hauler:hauler /fileserver/. /fileserver
COPY --from=builder --chown=hauler:hauler /build/bin/hauler /
COPY --from=builder --chown=hauler:hauler /hauler /hauler
USER hauler
ENTRYPOINT [ "/hauler" ]
# debug stage
FROM alpine AS debug
COPY --from=builder /var/lib/ca-certificates/ca-bundle.pem /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group
COPY --from=builder --chown=hauler:hauler /home/hauler/. /home/hauler
COPY --from=builder --chown=hauler:hauler /hauler /usr/local/bin/hauler
RUN apk --no-cache add curl
USER hauler
WORKDIR /home/hauler

View File

@@ -1,39 +1,53 @@
SHELL:=/bin/bash
GO_FILES=$(shell go list ./... | grep -v /vendor/)
# Makefile for hauler
COSIGN_VERSION=v2.2.3+carbide.2
# set shell
SHELL=/bin/bash
.SILENT:
# set go variables
GO_FILES=./...
GO_COVERPROFILE=coverage.out
all: fmt vet install test
# set build variables
BIN_DIRECTORY=bin
DIST_DIRECTORY=dist
BINARIES_DIRECTORY=cmd/hauler/binaries
# local build of hauler for current platform
# references/configuration from .goreleaser.yaml
build:
rm -rf cmd/hauler/binaries;\
mkdir -p cmd/hauler/binaries;\
wget -P cmd/hauler/binaries/ https://github.com/rancher-government-carbide/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
mkdir bin;\
CGO_ENABLED=0 go build -o bin ./cmd/...;\
goreleaser build --clean --snapshot --parallelism 1 --timeout 60m --single-target
build-all: fmt vet
goreleaser build --rm-dist --snapshot
# local build of hauler for all platforms
# references/configuration from .goreleaser.yaml
build-all:
goreleaser build --clean --snapshot --parallelism 1 --timeout 60m
# local release of hauler for all platforms
# references/configuration from .goreleaser.yaml
release:
goreleaser release --clean --snapshot --parallelism 1 --timeout 60m
# install depedencies
install:
rm -rf cmd/hauler/binaries;\
mkdir -p cmd/hauler/binaries;\
wget -P cmd/hauler/binaries/ https://github.com/rancher-government-carbide/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
CGO_ENABLED=0 go install ./cmd/...;\
vet:
go vet $(GO_FILES)
rm -rf $(BINARIES_DIRECTORY)
mkdir -p $(BINARIES_DIRECTORY)
touch cmd/hauler/binaries/file
go mod tidy
go mod download
CGO_ENABLED=0 go install ./cmd/...
# format go code
fmt:
go fmt $(GO_FILES)
# vet go code
vet:
go vet $(GO_FILES)
# test go code
test:
go test $(GO_FILES) -cover
integration_test:
go test -tags=integration $(GO_FILES)
go test $(GO_FILES) -cover -race -covermode=atomic -coverprofile=$(GO_COVERPROFILE)
# cleanup artifacts
clean:
rm -rf bin 2> /dev/null
rm -rf $(BIN_DIRECTORY) $(BINARIES_DIRECTORY) $(DIST_DIRECTORY) $(GO_COVERPROFILE)

View File

@@ -4,13 +4,11 @@
## Airgap Swiss Army Knife
> ⚠️ **Please Note:** Hauler and the Hauler Documentation are recently Generally Available (GA).
`Rancher Government Hauler` simplifies the airgap experience without requiring operators to adopt a specific workflow. **Hauler** simplifies the airgapping process, by representing assets (images, charts, files, etc...) as content and collections to allow operators to easily fetch, store, package, and distribute these assets with declarative manifests or through the command line.
`Hauler` does this by storing contents and collections as OCI Artifacts and allows operators to serve contents and collections with an embedded registry and fileserver. Additionally, `Hauler` has the ability to store and inspect various non-image OCI Artifacts.
For more information, please review the **[Hauler Documentation](https://rancherfederal.github.io/hauler-docs)!**
For more information, please review the **[Hauler Documentation](https://hauler.dev)!**
## Installation
@@ -25,7 +23,7 @@ curl -sfL https://get.hauler.dev | bash
```bash
# installs latest release
brew tap rancherfederal/homebrew-tap
brew tap hauler-dev/homebrew-tap
brew install hauler
```

View File

@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto
package main
import _ "crypto/tls/fipsonly"

View File

@@ -3,14 +3,11 @@ package cli
import (
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/log"
)
type rootOpts struct {
logLevel string
}
var ro = &rootOpts{}
var ro = &flags.CliRootOpts{}
func New() *cobra.Command {
cmd := &cobra.Command{
@@ -18,7 +15,7 @@ func New() *cobra.Command {
Short: "Airgap Swiss Army Knife",
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
l := log.FromContext(cmd.Context())
l.SetLevel(ro.logLevel)
l.SetLevel(ro.LogLevel)
l.Debugf("running cli command [%s]", cmd.CommandPath())
return nil
},
@@ -28,7 +25,7 @@ func New() *cobra.Command {
}
pf := cmd.PersistentFlags()
pf.StringVarP(&ro.logLevel, "log-level", "l", "info", "")
pf.StringVarP(&ro.LogLevel, "log-level", "l", "info", "")
// Add subcommands
addLogin(cmd)

View File

@@ -3,54 +3,48 @@ package cli
import (
"fmt"
"os"
"github.com/spf13/cobra"
)
func addCompletion(parent *cobra.Command) {
cmd := &cobra.Command{
Use: "completion",
Short: "Generates completion scripts for various shells",
Long: `The completion sub-command generates completion scripts for various shells.`,
Use: "completion",
Short: "Generate auto-completion scripts for various shells",
}
cmd.AddCommand(
addCompletionZsh(),
addCompletionBash(),
addCompletionFish(),
addCompletionPowershell(),
)
parent.AddCommand(cmd)
}
func completionError(err error) ([]string, cobra.ShellCompDirective) {
cobra.CompError(err.Error())
return nil, cobra.ShellCompDirectiveError
parent.AddCommand(cmd)
}
func addCompletionZsh() *cobra.Command {
cmd := &cobra.Command{
Use: "zsh",
Short: "Generates zsh completion scripts",
Long: `The completion sub-command generates completion scripts for zsh.`,
Short: "Generates auto-completion scripts for zsh",
Example: `To load completion run
. <(hauler completion zsh)
To configure your zsh shell to load completions for each session add to your zshrc
# ~/.zshrc or ~/.profile
command -v hauler >/dev/null && . <(hauler completion zsh)
or write a cached file in one of the completion directories in your ${fpath}:
echo "${fpath// /\n}" | grep -i completion
hauler completion zsh > _hauler
mv _hauler ~/.oh-my-zsh/completions # oh-my-zsh
mv _hauler ~/.zprezto/modules/completion/external/src/ # zprezto`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenZshCompletion(os.Stdout)
cmd.Root().GenZshCompletion(os.Stdout)
// Cobra doesn't source zsh completion file, explicitly doing it here
fmt.Println("compdef _hauler hauler")
},
@@ -61,18 +55,17 @@ func addCompletionZsh() *cobra.Command {
func addCompletionBash() *cobra.Command {
cmd := &cobra.Command{
Use: "bash",
Short: "Generates bash completion scripts",
Long: `The completion sub-command generates completion scripts for bash.`,
Short: "Generates auto-completion scripts for bash",
Example: `To load completion run
. <(hauler completion bash)
To configure your bash shell to load completions for each session add to your bashrc
# ~/.bashrc or ~/.profile
command -v hauler >/dev/null && . <(hauler completion bash)`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenBashCompletion(os.Stdout)
cmd.Root().GenBashCompletion(os.Stdout)
},
}
return cmd
@@ -81,15 +74,14 @@ func addCompletionBash() *cobra.Command {
func addCompletionFish() *cobra.Command {
cmd := &cobra.Command{
Use: "fish",
Short: "Generates fish completion scripts",
Long: `The completion sub-command generates completion scripts for fish.`,
Short: "Generates auto-completion scripts for fish",
Example: `To configure your fish shell to load completions for each session write this script to your completions dir:
hauler completion fish > ~/.config/fish/completions/hauler.fish
See http://fishshell.com/docs/current/index.html#completion-own for more details`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenFishCompletion(os.Stdout, true)
cmd.Root().GenFishCompletion(os.Stdout, true)
},
}
return cmd
@@ -98,26 +90,25 @@ func addCompletionFish() *cobra.Command {
func addCompletionPowershell() *cobra.Command {
cmd := &cobra.Command{
Use: "powershell",
Short: "Generates powershell completion scripts",
Long: `The completion sub-command generates completion scripts for powershell.`,
Short: "Generates auto-completion scripts for powershell",
Example: `To load completion run
. <(hauler completion powershell)
To configure your powershell shell to load completions for each session add to your powershell profile
Windows:
cd "$env:USERPROFILE\Documents\WindowsPowerShell\Modules"
hauler completion powershell >> hauler-completion.ps1
Linux:
cd "${XDG_CONFIG_HOME:-"$HOME/.config/"}/powershell/modules"
hauler completion powershell >> hauler-completions.ps1`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenPowerShellCompletion(os.Stdout)
cmd.Root().GenPowerShellCompletion(os.Stdout)
},
}
return cmd
}
}

View File

@@ -2,39 +2,26 @@ package cli
import (
"context"
"strings"
"os"
"io"
"fmt"
"github.com/spf13/cobra"
"io"
"os"
"strings"
"github.com/spf13/cobra"
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/cosign"
)
type Opts struct {
Username string
Password string
PasswordStdin bool
}
func (o *Opts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "Username")
f.StringVarP(&o.Password, "password", "p", "", "Password")
f.BoolVarP(&o.PasswordStdin, "password-stdin", "", false, "Take the password from stdin")
}
func addLogin(parent *cobra.Command) {
o := &Opts{}
o := &flags.LoginOpts{}
cmd := &cobra.Command{
Use: "login",
Short: "Log in to a registry",
Example: `
# Log in to reg.example.com
hauler login reg.example.com -u bob -p haulin`,
Use: "login",
Short: "Login to a registry",
Long: "Login to an OCI Compliant Registry (stored at ~/.docker/config.json)",
Example: "# login to registry.example.com\nhauler login registry.example.com -u bob -p haulin",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, arg []string) error {
ctx := cmd.Context()
@@ -47,7 +34,7 @@ hauler login reg.example.com -u bob -p haulin`,
o.Password = strings.TrimSuffix(string(contents), "\n")
o.Password = strings.TrimSuffix(o.Password, "\r")
}
if o.Username == "" && o.Password == "" {
return fmt.Errorf("username and password required")
}
@@ -55,21 +42,21 @@ hauler login reg.example.com -u bob -p haulin`,
return login(ctx, o, arg[0])
},
}
o.AddArgs(cmd)
o.AddFlags(cmd)
parent.AddCommand(cmd)
}
func login(ctx context.Context, o *Opts, registry string) error {
func login(ctx context.Context, o *flags.LoginOpts, registry string) error {
ropts := content.RegistryOptions{
Username: o.Username,
Password: o.Password,
Username: o.Username,
Password: o.Password,
}
err := cosign.RegistryLogin(ctx, nil, registry, ropts)
if err != nil {
return err
}
return nil
}
}

View File

@@ -1,25 +1,27 @@
package cli
import (
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
"fmt"
"github.com/rancherfederal/hauler/cmd/hauler/cli/store"
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
"hauler.dev/go/hauler/cmd/hauler/cli/store"
"hauler.dev/go/hauler/internal/flags"
)
var rootStoreOpts = &store.RootOpts{}
var rootStoreOpts = &flags.StoreRootOpts{}
func addStore(parent *cobra.Command) {
cmd := &cobra.Command{
Use: "store",
Aliases: []string{"s"},
Short: "Interact with hauler's embedded content store",
Short: "Interact with the content store",
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
}
rootStoreOpts.AddArgs(cmd)
rootStoreOpts.AddFlags(cmd)
cmd.AddCommand(
addStoreSync(),
@@ -29,8 +31,6 @@ func addStore(parent *cobra.Command) {
addStoreServe(),
addStoreInfo(),
addStoreCopy(),
// TODO: Remove this in favor of sync?
addStoreAdd(),
)
@@ -38,11 +38,11 @@ func addStore(parent *cobra.Command) {
}
func addStoreExtract() *cobra.Command {
o := &store.ExtractOpts{RootOpts: rootStoreOpts}
o := &flags.ExtractOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "extract",
Short: "Extract content from the store to disk",
Short: "Extract artifacts from the content store to disk",
Aliases: []string{"x"},
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
@@ -56,17 +56,17 @@ func addStoreExtract() *cobra.Command {
return store.ExtractCmd(ctx, o, s, args[0])
},
}
o.AddArgs(cmd)
o.AddFlags(cmd)
return cmd
}
func addStoreSync() *cobra.Command {
o := &store.SyncOpts{RootOpts: rootStoreOpts}
o := &flags.SyncOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "sync",
Short: "Sync content to the embedded content store",
Short: "Sync content to the content store",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -84,7 +84,7 @@ func addStoreSync() *cobra.Command {
}
func addStoreLoad() *cobra.Command {
o := &store.LoadOpts{RootOpts: rootStoreOpts}
o := &flags.LoadOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "load",
@@ -110,7 +110,7 @@ func addStoreLoad() *cobra.Command {
func addStoreServe() *cobra.Command {
cmd := &cobra.Command{
Use: "serve",
Short: "Expose the content of a local store through an OCI compliant registry or file server",
Short: "Serve the content store via an OCI Compliant Registry or Fileserver",
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
@@ -123,13 +123,13 @@ func addStoreServe() *cobra.Command {
return cmd
}
// RegistryCmd serves the embedded registry
// RegistryCmd serves the registry
func addStoreServeRegistry() *cobra.Command {
o := &store.ServeRegistryOpts{RootOpts: rootStoreOpts}
o := &flags.ServeRegistryOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "registry",
Short: "Serve the embedded registry",
RunE: func(cmd *cobra.Command, args []string) error {
Use: "registry",
Short: "Serve the OCI Compliant Registry",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
s, err := o.Store(ctx)
@@ -138,21 +138,21 @@ func addStoreServeRegistry() *cobra.Command {
}
return store.ServeRegistryCmd(ctx, o, s)
},
}
},
}
o.AddFlags(cmd)
o.AddFlags(cmd)
return cmd
return cmd
}
// FileServerCmd serves the file server
func addStoreServeFiles() *cobra.Command {
o := &store.ServeFilesOpts{RootOpts: rootStoreOpts}
o := &flags.ServeFilesOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "fileserver",
Short: "Serve the file server",
RunE: func(cmd *cobra.Command, args []string) error {
Use: "fileserver",
Short: "Serve the Fileserver",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
s, err := o.Store(ctx)
@@ -161,16 +161,16 @@ func addStoreServeFiles() *cobra.Command {
}
return store.ServeFilesCmd(ctx, o, s)
},
}
},
}
o.AddFlags(cmd)
o.AddFlags(cmd)
return cmd
return cmd
}
func addStoreSave() *cobra.Command {
o := &store.SaveOpts{RootOpts: rootStoreOpts}
o := &flags.SaveOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "save",
@@ -188,13 +188,13 @@ func addStoreSave() *cobra.Command {
return store.SaveCmd(ctx, o, o.FileName)
},
}
o.AddArgs(cmd)
o.AddFlags(cmd)
return cmd
}
func addStoreInfo() *cobra.Command {
o := &store.InfoOpts{RootOpts: rootStoreOpts}
o := &flags.InfoOpts{StoreRootOpts: rootStoreOpts}
var allowedValues = []string{"image", "chart", "file", "sigs", "atts", "sbom", "all"}
@@ -210,7 +210,7 @@ func addStoreInfo() *cobra.Command {
if err != nil {
return err
}
for _, allowed := range allowedValues {
if o.TypeFilter == allowed {
return store.InfoCmd(ctx, o, s)
@@ -225,11 +225,11 @@ func addStoreInfo() *cobra.Command {
}
func addStoreCopy() *cobra.Command {
o := &store.CopyOpts{RootOpts: rootStoreOpts}
o := &flags.CopyOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "copy",
Short: "Copy all store contents to another OCI registry",
Short: "Copy all store content to another location",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -250,7 +250,7 @@ func addStoreCopy() *cobra.Command {
func addStoreAdd() *cobra.Command {
cmd := &cobra.Command{
Use: "add",
Short: "Add content to store",
Short: "Add content to the store",
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
@@ -266,12 +266,20 @@ func addStoreAdd() *cobra.Command {
}
func addStoreAddFile() *cobra.Command {
o := &store.AddFileOpts{RootOpts: rootStoreOpts}
o := &flags.AddFileOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "file",
Short: "Add a file to the content store",
Args: cobra.ExactArgs(1),
Short: "Add a file to the store",
Example: `# fetch local file
hauler store add file file.txt
# fetch remote file
hauler store add file https://get.rke2.io/install.sh
# fetch remote file and assign new name
hauler store add file https://get.hauler.dev --name hauler-install.sh`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -289,12 +297,26 @@ func addStoreAddFile() *cobra.Command {
}
func addStoreAddImage() *cobra.Command {
o := &store.AddImageOpts{RootOpts: rootStoreOpts}
o := &flags.AddImageOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "image",
Short: "Add an image to the content store",
Args: cobra.ExactArgs(1),
Short: "Add a image to the store",
Example: `# fetch image
hauler store add image busybox
# fetch image with repository and tag
hauler store add image library/busybox:stable
# fetch image with full image reference and specific platform
hauler store add image ghcr.io/hauler-dev/hauler-debug:v1.0.7 --platform linux/amd64
# fetch image with full image reference via digest
hauler store add image gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5
# fetch image with full image reference, specific platform, and signature verification
hauler store add image rgcrprod.azurecr.us/hauler/rke2-manifest.yaml:v1.28.12-rke2r1 --platform linux/amd64 --key carbide-key.pub`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -312,27 +334,31 @@ func addStoreAddImage() *cobra.Command {
}
func addStoreAddChart() *cobra.Command {
o := &store.AddChartOpts{
RootOpts: rootStoreOpts,
ChartOpts: &action.ChartPathOptions{},
o := &flags.AddChartOpts{
StoreRootOpts: rootStoreOpts,
ChartOpts: &action.ChartPathOptions{},
}
cmd := &cobra.Command{
Use: "chart",
Short: "Add a local or remote chart to the content store",
Example: `
# add a local chart
hauler store add chart path/to/chart/directory
Short: "Add a helm chart to the store",
Example: `# fetch local helm chart
hauler store add chart path/to/chart/directory --repo .
# add a local compressed chart
hauler store add chart path/to/chart.tar.gz
# fetch local compressed helm chart
hauler store add chart path/to/chart.tar.gz --repo .
# add a remote chart
hauler store add chart longhorn --repo "https://charts.longhorn.io"
# fetch remote oci helm chart
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev
# add a specific version of a chart
hauler store add chart rancher --repo "https://releases.rancher.com/server-charts/latest" --version "2.6.2"
`,
# fetch remote oci helm chart with version
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev --version 1.0.6
# fetch remote helm chart
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable
# fetch remote helm chart with specific version
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/latest --version 2.9.1`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()

View File

@@ -4,32 +4,20 @@ import (
"context"
"github.com/google/go-containerregistry/pkg/name"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/spf13/cobra"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/content/chart"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/artifacts/file"
"hauler.dev/go/hauler/pkg/content/chart"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type AddFileOpts struct {
*RootOpts
Name string
}
func (o *AddFileOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Name, "name", "n", "", "(Optional) Name to assign to file in store")
}
func AddFileCmd(ctx context.Context, o *AddFileOpts, s *store.Layout, reference string) error {
func AddFileCmd(ctx context.Context, o *flags.AddFileOpts, s *store.Layout, reference string) error {
cfg := v1alpha1.File{
Path: reference,
}
@@ -63,20 +51,7 @@ func storeFile(ctx context.Context, s *store.Layout, fi v1alpha1.File) error {
return nil
}
type AddImageOpts struct {
*RootOpts
Name string
Key string
Platform string
}
func (o *AddImageOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Path to the key for digital signature verification")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specific platform to save. i.e. linux/amd64. Defaults to all if flag is omitted.")
}
func AddImageCmd(ctx context.Context, o *AddImageOpts, s *store.Layout, reference string) error {
func AddImageCmd(ctx context.Context, o *flags.AddImageOpts, s *store.Layout, reference string) error {
l := log.FromContext(ctx)
cfg := v1alpha1.Image{
Name: reference,
@@ -101,39 +76,21 @@ func storeImage(ctx context.Context, s *store.Layout, i v1alpha1.Image, platform
r, err := name.ParseReference(i.Name)
if err != nil {
return err
l.Warnf("unable to parse 'image' [%s], skipping...", r.Name())
return nil
}
err = cosign.SaveImage(ctx, s, r.Name(), platform)
if err != nil {
return err
l.Warnf("unable to add 'image' [%s] to store. skipping...", r.Name())
return nil
}
l.Infof("successfully added 'image' [%s]", r.Name())
return nil
}
type AddChartOpts struct {
*RootOpts
ChartOpts *action.ChartPathOptions
}
func (o *AddChartOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVar(&o.ChartOpts.RepoURL, "repo", "", "chart repository url where to locate the requested chart")
f.StringVar(&o.ChartOpts.Version, "version", "", "specify a version constraint for the chart version to use. This constraint can be a specific tag (e.g. 1.1.1) or it may reference a valid range (e.g. ^2.0.0). If this is not specified, the latest version is used")
f.BoolVar(&o.ChartOpts.Verify, "verify", false, "verify the package before using it")
f.StringVar(&o.ChartOpts.Username, "username", "", "chart repository username where to locate the requested chart")
f.StringVar(&o.ChartOpts.Password, "password", "", "chart repository password where to locate the requested chart")
f.StringVar(&o.ChartOpts.CertFile, "cert-file", "", "identify HTTPS client using this SSL certificate file")
f.StringVar(&o.ChartOpts.KeyFile, "key-file", "", "identify HTTPS client using this SSL key file")
f.BoolVar(&o.ChartOpts.InsecureSkipTLSverify, "insecure-skip-tls-verify", false, "skip tls certificate checks for the chart download")
f.StringVar(&o.ChartOpts.CaFile, "ca-file", "", "verify certificates of HTTPS-enabled servers using this CA bundle")
}
func AddChartCmd(ctx context.Context, o *AddChartOpts, s *store.Layout, chartName string) error {
func AddChartCmd(ctx context.Context, o *flags.AddChartOpts, s *store.Layout, chartName string) error {
// TODO: Reduce duplicates between api chart and upstream helm opts
cfg := v1alpha1.Chart{
Name: chartName,
@@ -147,7 +104,7 @@ func AddChartCmd(ctx context.Context, o *AddChartOpts, s *store.Layout, chartNam
func storeChart(ctx context.Context, s *store.Layout, cfg v1alpha1.Chart, opts *action.ChartPathOptions) error {
l := log.FromContext(ctx)
l.Infof("adding 'chart' [%s] to the store", cfg.Name)
// TODO: This shouldn't be necessary
opts.RepoURL = cfg.RepoURL
opts.Version = cfg.Version

View File

@@ -5,34 +5,15 @@ import (
"fmt"
"strings"
"github.com/spf13/cobra"
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
type CopyOpts struct {
*RootOpts
Username string
Password string
Insecure bool
PlainHTTP bool
}
func (o *CopyOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "Username when copying to an authenticated remote registry")
f.StringVarP(&o.Password, "password", "p", "", "Password when copying to an authenticated remote registry")
f.BoolVar(&o.Insecure, "insecure", false, "Toggle allowing insecure connections when copying to a remote registry")
f.BoolVar(&o.PlainHTTP, "plain-http", false, "Toggle allowing plain http connections when copying to a remote registry")
}
func CopyCmd(ctx context.Context, o *CopyOpts, s *store.Layout, targetRef string) error {
func CopyCmd(ctx context.Context, o *flags.CopyOpts, s *store.Layout, targetRef string) error {
l := log.FromContext(ctx)
components := strings.SplitN(targetRef, "://", 2)
@@ -55,7 +36,7 @@ func CopyCmd(ctx context.Context, o *CopyOpts, s *store.Layout, targetRef string
Insecure: o.Insecure,
PlainHTTP: o.PlainHTTP,
}
if ropts.Username != "" {
err := cosign.RegistryLogin(ctx, s, components[1], ropts)
if err != nil {

View File

@@ -2,32 +2,20 @@ package store
import (
"context"
"strings"
"encoding/json"
"fmt"
"strings"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/internal/mapper"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/internal/mapper"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type ExtractOpts struct {
*RootOpts
DestinationDir string
}
func (o *ExtractOpts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.DestinationDir, "output", "o", "", "Directory to save contents to (defaults to current directory)")
}
func ExtractCmd(ctx context.Context, o *ExtractOpts, s *store.Layout, ref string) error {
func ExtractCmd(ctx context.Context, o *flags.ExtractOpts, s *store.Layout, ref string) error {
l := log.FromContext(ctx)
r, err := reference.Parse(ref)
@@ -35,10 +23,12 @@ func ExtractCmd(ctx context.Context, o *ExtractOpts, s *store.Layout, ref string
return err
}
// use the repository from the context and the identifier from the reference
repo := r.Context().RepositoryStr() + ":" + r.Identifier()
found := false
if err := s.Walk(func(reference string, desc ocispec.Descriptor) error {
if !strings.Contains(reference, r.Name()) {
if !strings.Contains(reference, repo) {
return nil
}
found = true

View File

@@ -4,38 +4,19 @@ import (
"context"
"encoding/json"
"fmt"
"github.com/olekukonko/tablewriter"
"os"
"sort"
"github.com/olekukonko/tablewriter"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type InfoOpts struct {
*RootOpts
OutputFormat string
TypeFilter string
SizeUnit string
}
func (o *InfoOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.OutputFormat, "output", "o", "table", "Output format (table, json)")
f.StringVarP(&o.TypeFilter, "type", "t", "all", "Filter on type (image, chart, file, sigs, atts, sbom)")
// TODO: Regex/globbing
}
func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
func InfoCmd(ctx context.Context, o *flags.InfoOpts, s *store.Layout) error {
var items []item
if err := s.Walk(func(ref string, desc ocispec.Descriptor) error {
if _, ok := desc.Annotations[ocispec.AnnotationRefName]; !ok {
@@ -47,7 +28,7 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
}
defer rc.Close()
// handle multi-arch images
// handle multi-arch images
if desc.MediaType == consts.OCIImageIndexSchema || desc.MediaType == consts.DockerManifestListSchema2 {
var idx ocispec.Index
if err := json.NewDecoder(rc).Decode(&idx); err != nil {
@@ -72,13 +53,13 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
items = append(items, i)
}
}
// handle "non" multi-arch images
// handle "non" multi-arch images
} else if desc.MediaType == consts.DockerManifestSchema2 || desc.MediaType == consts.OCIManifestSchema1 {
var m ocispec.Manifest
if err := json.NewDecoder(rc).Decode(&m); err != nil {
return err
}
rc, err := s.FetchManifest(ctx, m)
if err != nil {
return err
@@ -90,7 +71,7 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
if err := json.NewDecoder(rc).Decode(&internalManifest); err != nil {
return err
}
if internalManifest.Architecture != "" {
i := newItem(s, desc, m, fmt.Sprintf("%s/%s", internalManifest.OS, internalManifest.Architecture), o)
var emptyItem item
@@ -104,8 +85,8 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
items = append(items, i)
}
}
// handle the rest
} else {
// handle the rest
} else {
var m ocispec.Manifest
if err := json.NewDecoder(rc).Decode(&m); err != nil {
return err
@@ -123,6 +104,11 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
return err
}
if o.ListRepos {
buildListRepos(items...)
return nil
}
// sort items by ref and arch
sort.Sort(byReferenceAndArch(items))
@@ -137,6 +123,30 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
return nil
}
func buildListRepos(items ...item) {
// Create map to track unique repository names
repos := make(map[string]bool)
for _, i := range items {
repoName := ""
for j := 0; j < len(i.Reference); j++ {
if i.Reference[j] == '/' {
repoName = i.Reference[:j]
break
}
}
if repoName == "" {
repoName = i.Reference
}
repos[repoName] = true
}
// Collect and print unique repository names
for repoName := range repos {
fmt.Println(repoName)
}
}
func buildTable(items ...item) {
// Create a table for the results
table := tablewriter.NewWriter(os.Stdout)
@@ -144,7 +154,7 @@ func buildTable(items ...item) {
table.SetHeaderAlignment(tablewriter.ALIGN_LEFT)
table.SetRowLine(false)
table.SetAutoMergeCellsByColumnIndex([]int{0})
totalSize := int64(0)
for _, i := range items {
if i.Type != "" {
@@ -173,11 +183,11 @@ func buildJson(item ...item) string {
}
type item struct {
Reference string
Type string
Platform string
Layers int
Size int64
Reference string
Type string
Platform string
Layers int
Size int64
}
type byReferenceAndArch []item
@@ -200,7 +210,7 @@ func (a byReferenceAndArch) Less(i, j int) bool {
return a[i].Reference < a[j].Reference
}
func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat string, o *InfoOpts) item {
func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat string, o *flags.InfoOpts) item {
var size int64 = 0
for _, l := range m.Layers {
size += l.Size
@@ -227,8 +237,12 @@ func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat
case "dev.cosignproject.cosign/sboms":
ctype = "sbom"
}
ref, err := reference.Parse(desc.Annotations[ocispec.AnnotationRefName])
refName := desc.Annotations["io.containerd.image.name"]
if refName == "" {
refName = desc.Annotations[ocispec.AnnotationRefName]
}
ref, err := reference.Parse(refName)
if err != nil {
return item{}
}
@@ -238,11 +252,11 @@ func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat
}
return item{
Reference: ref.Name(),
Type: ctype,
Platform: plat,
Layers: len(m.Layers),
Size: size,
Reference: ref.Name(),
Type: ctype,
Platform: plat,
Layers: len(m.Layers),
Size: size,
}
}

View File

@@ -5,31 +5,16 @@ import (
"os"
"github.com/mholt/archiver/v3"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/content"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
type LoadOpts struct {
*RootOpts
TempOverride string
}
func (o *LoadOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
// On Unix systems, the default is $TMPDIR if non-empty, else /tmp.
// On Windows, the default is GetTempPath, returning the first non-empty
// value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows directory.
// On Plan 9, the default is /tmp.
f.StringVarP(&o.TempOverride, "tempdir", "t", "", "overrides the default directory for temporary files, as returned by your OS.")
}
// LoadCmd
// TODO: Just use mholt/archiver for now, even though we don't need most of it
func LoadCmd(ctx context.Context, o *LoadOpts, archiveRefs ...string) error {
func LoadCmd(ctx context.Context, o *flags.LoadOpts, archiveRefs ...string) error {
l := log.FromContext(ctx)
for _, archiveRef := range archiveRefs {

View File

@@ -1,30 +1,31 @@
package store
import (
"bytes"
"context"
"encoding/json"
"os"
"path"
"path/filepath"
"slices"
referencev3 "github.com/distribution/distribution/v3/reference"
"github.com/google/go-containerregistry/pkg/name"
libv1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/layout"
"github.com/google/go-containerregistry/pkg/v1/tarball"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/mholt/archiver/v3"
"github.com/spf13/cobra"
imagev1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/log"
)
type SaveOpts struct {
*RootOpts
FileName string
}
func (o *SaveOpts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.FileName, "filename", "f", "haul.tar.zst", "Name of archive")
}
// SaveCmd
// TODO: Just use mholt/archiver for now, even though we don't need most of it
func SaveCmd(ctx context.Context, o *SaveOpts, outputFile string) error {
func SaveCmd(ctx context.Context, o *flags.SaveOpts, outputFile string) error {
l := log.FromContext(ctx)
// TODO: Support more formats?
@@ -45,6 +46,10 @@ func SaveCmd(ctx context.Context, o *SaveOpts, outputFile string) error {
return err
}
if err := writeExportsManifest(ctx, ".", o.Platform); err != nil {
return err
}
err = a.Archive([]string{"."}, absOutputfile)
if err != nil {
return err
@@ -53,3 +58,180 @@ func SaveCmd(ctx context.Context, o *SaveOpts, outputFile string) error {
l.Infof("saved store [%s] -> [%s]", o.StoreDir, absOutputfile)
return nil
}
type exports struct {
digests []string
records map[string]tarball.Descriptor
}
func writeExportsManifest(ctx context.Context, dir string, platformStr string) error {
l := log.FromContext(ctx)
// validate platform format
platform, err := libv1.ParsePlatform(platformStr)
if err != nil {
return err
}
oci, err := layout.FromPath(dir)
if err != nil {
return err
}
idx, err := oci.ImageIndex()
if err != nil {
return err
}
imx, err := idx.IndexManifest()
if err != nil {
return err
}
x := &exports{
digests: []string{},
records: map[string]tarball.Descriptor{},
}
for _, desc := range imx.Manifests {
l.Debugf("descriptor [%s] >>> %s", desc.Digest.String(), desc.MediaType)
if artifactType := types.MediaType(desc.ArtifactType); artifactType != "" && !artifactType.IsImage() && !artifactType.IsIndex() {
l.Debugf("descriptor [%s] <<< SKIPPING ARTIFACT (%q)", desc.Digest.String(), desc.ArtifactType)
continue
}
if desc.Annotations != nil {
// we only care about images that cosign has added to the layout index
if kind, hasKind := desc.Annotations[consts.KindAnnotationName]; hasKind {
if refName, hasRefName := desc.Annotations["io.containerd.image.name"]; hasRefName {
// branch on image (aka image manifest) or image index
switch kind {
case consts.KindAnnotationImage:
if err := x.record(ctx, idx, desc, refName); err != nil {
return err
}
case consts.KindAnnotationIndex:
l.Debugf("index [%s]: digest=%s, type=%s, size=%d", refName, desc.Digest.String(), desc.MediaType, desc.Size)
// when no platform is provided, warn the user of potential mismatch on import
if platform.String() == "" {
l.Warnf("index [%s]: provide an export platform to prevent potential platform mismatch on import", refName)
}
iix, err := idx.ImageIndex(desc.Digest)
if err != nil {
return err
}
ixm, err := iix.IndexManifest()
if err != nil {
return err
}
for _, ixd := range ixm.Manifests {
if ixd.MediaType.IsImage() {
// check if platform is provided, if so, skip anything that doesn't match
if platform.String() != "" {
if ixd.Platform.Architecture != platform.Architecture || ixd.Platform.OS != platform.OS {
l.Warnf("index [%s]: digest=%s, platform=%s/%s: does not match the supplied platform, skipping", refName, desc.Digest.String(), ixd.Platform.OS, ixd.Platform.Architecture)
continue
}
}
// skip 'unknown' platforms... docker hates
if ixd.Platform.Architecture == "unknown" && ixd.Platform.OS == "unknown" {
l.Warnf("index [%s]: digest=%s, platform=%s/%s: skipping 'unknown/unknown' platform", refName, desc.Digest.String(), ixd.Platform.OS, ixd.Platform.Architecture)
continue
}
if err := x.record(ctx, iix, ixd, refName); err != nil {
return err
}
}
}
default:
l.Debugf("descriptor [%s] <<< SKIPPING KIND (%q)", desc.Digest.String(), kind)
}
}
}
}
}
buf := bytes.Buffer{}
mnf := x.describe()
err = json.NewEncoder(&buf).Encode(mnf)
if err != nil {
return err
}
return oci.WriteFile("manifest.json", buf.Bytes(), 0666)
}
func (x *exports) describe() tarball.Manifest {
m := make(tarball.Manifest, len(x.digests))
for i, d := range x.digests {
m[i] = x.records[d]
}
return m
}
func (x *exports) record(ctx context.Context, index libv1.ImageIndex, desc libv1.Descriptor, refname string) error {
l := log.FromContext(ctx)
digest := desc.Digest.String()
image, err := index.Image(desc.Digest)
if err != nil {
return err
}
config, err := image.ConfigName()
if err != nil {
return err
}
xd, recorded := x.records[digest]
if !recorded {
// record one export record per digest
x.digests = append(x.digests, digest)
xd = tarball.Descriptor{
Config: path.Join(imagev1.ImageBlobsDir, config.Algorithm, config.Hex),
RepoTags: []string{},
Layers: []string{},
}
layers, err := image.Layers()
if err != nil {
return err
}
for _, layer := range layers {
xl, err := layer.Digest()
if err != nil {
return err
}
xd.Layers = append(xd.Layers[:], path.Join(imagev1.ImageBlobsDir, xl.Algorithm, xl.Hex))
}
}
ref, err := name.ParseReference(refname)
if err != nil {
return err
}
// record tags for the digest, eliminating dupes
switch tag := ref.(type) {
case name.Tag:
named, err := referencev3.ParseNormalizedNamed(refname)
if err != nil {
return err
}
named = referencev3.TagNameOnly(named)
repotag := referencev3.FamiliarString(named)
xd.RepoTags = append(xd.RepoTags[:], repotag)
slices.Sort(xd.RepoTags)
xd.RepoTags = slices.Compact(xd.RepoTags)
ref = tag.Digest(digest)
}
l.Debugf("image [%s]: type=%s, size=%d", ref.Name(), desc.MediaType, desc.Size)
// record export descriptor for the digest
x.records[digest] = xd
return nil
}

View File

@@ -2,8 +2,6 @@ package store
import (
"context"
"fmt"
"net/http"
"os"
"github.com/distribution/distribution/v3/configuration"
@@ -12,33 +10,14 @@ import (
_ "github.com/distribution/distribution/v3/registry/storage/driver/filesystem"
_ "github.com/distribution/distribution/v3/registry/storage/driver/inmemory"
"github.com/distribution/distribution/v3/version"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/internal/server"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/internal/server"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
type ServeRegistryOpts struct {
*RootOpts
Port int
RootDir string
ConfigFile string
storedir string
}
func (o *ServeRegistryOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 5000, "Port to listen on.")
f.StringVar(&o.RootDir, "directory", "registry", "Directory to use for backend. Defaults to $PWD/registry")
f.StringVarP(&o.ConfigFile, "config", "c", "", "Path to a config file, will override all other configs")
}
func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout) error {
func ServeRegistryCmd(ctx context.Context, o *flags.ServeRegistryOpts, s *store.Layout) error {
l := log.FromContext(ctx)
ctx = dcontext.WithVersion(ctx, version.Version)
@@ -47,14 +26,14 @@ func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout
return err
}
opts := &CopyOpts{}
opts := &flags.CopyOpts{}
if err := CopyCmd(ctx, opts, s, "registry://"+tr.Registry()); err != nil {
return err
}
tr.Close()
cfg := o.defaultRegistryConfig()
cfg := o.DefaultRegistryConfig()
if o.ConfigFile != "" {
ucfg, err := loadConfig(o.ConfigFile)
if err != nil {
@@ -76,44 +55,30 @@ func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout
return nil
}
type ServeFilesOpts struct {
*RootOpts
Port int
RootDir string
storedir string
}
func (o *ServeFilesOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 8080, "Port to listen on.")
f.StringVar(&o.RootDir, "directory", "fileserver", "Directory to use for backend. Defaults to $PWD/fileserver")
}
func ServeFilesCmd(ctx context.Context, o *ServeFilesOpts, s *store.Layout) error {
func ServeFilesCmd(ctx context.Context, o *flags.ServeFilesOpts, s *store.Layout) error {
l := log.FromContext(ctx)
ctx = dcontext.WithVersion(ctx, version.Version)
opts := &CopyOpts{}
opts := &flags.CopyOpts{}
if err := CopyCmd(ctx, opts, s, "dir://"+o.RootDir); err != nil {
return err
}
cfg := server.FileConfig{
Root: o.RootDir,
Port: o.Port,
}
f, err := server.NewFile(ctx, cfg)
f, err := server.NewFile(ctx, *o)
if err != nil {
return err
}
l.Infof("starting file server on port [%d]", o.Port)
if err := f.ListenAndServe(); err != nil {
return err
if o.TLSCert != "" && o.TLSKey != "" {
l.Infof("starting file server with tls on port [%d]", o.Port)
if err := f.ListenAndServeTLS(o.TLSCert, o.TLSKey); err != nil {
return err
}
} else {
l.Infof("starting file server on port [%d]", o.Port)
if err := f.ListenAndServe(); err != nil {
return err
}
}
return nil
@@ -127,27 +92,3 @@ func loadConfig(filename string) (*configuration.Configuration, error) {
return configuration.Parse(f)
}
func (o *ServeRegistryOpts) defaultRegistryConfig() *configuration.Configuration {
cfg := &configuration.Configuration{
Version: "0.1",
Storage: configuration.Storage{
"cache": configuration.Parameters{"blobdescriptor": "inmemory"},
"filesystem": configuration.Parameters{"rootdirectory": o.RootDir},
// TODO: Ensure this is toggleable via cli arg if necessary
// "maintenance": configuration.Parameters{"readonly.enabled": false},
},
}
// Add validation configuration
cfg.Validation.Manifests.URLs.Allow = []string{".+"}
cfg.Log.Level = "info"
cfg.HTTP.Addr = fmt.Sprintf(":%d", o.Port)
cfg.HTTP.Headers = http.Header{
"X-Content-Type-Options": []string{"nosniff"},
}
return cfg
}

View File

@@ -9,44 +9,23 @@ import (
"strings"
"github.com/mitchellh/go-homedir"
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
"k8s.io/apimachinery/pkg/util/yaml"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
tchart "github.com/rancherfederal/hauler/pkg/collection/chart"
"github.com/rancherfederal/hauler/pkg/collection/imagetxt"
"github.com/rancherfederal/hauler/pkg/collection/k3s"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
tchart "hauler.dev/go/hauler/pkg/collection/chart"
"hauler.dev/go/hauler/pkg/collection/imagetxt"
"hauler.dev/go/hauler/pkg/collection/k3s"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type SyncOpts struct {
*RootOpts
ContentFiles []string
Key string
Products []string
Platform string
Registry string
ProductRegistry string
}
func (o *SyncOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringSliceVarP(&o.ContentFiles, "files", "f", []string{}, "Path to content files")
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Path to the key for signature verification")
f.StringSliceVar(&o.Products, "products", []string{}, "Used for RGS Carbide customers to supply a product and version and Hauler will retrieve the images. i.e. '--product rancher=v2.7.6'")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specific platform to save. i.e. linux/amd64. Defaults to all if flag is omitted.")
f.StringVarP(&o.Registry, "registry", "r", "", "(Optional) Default pull registry for image refs that are not specifying a registry name.")
f.StringVarP(&o.ProductRegistry, "product-registry", "c", "", "(Optional) Specific Product Registry to use. Defaults to RGS Carbide Registry (rgcrprod.azurecr.us).")
}
func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
func SyncCmd(ctx context.Context, o *flags.SyncOpts, s *store.Layout) error {
l := log.FromContext(ctx)
// if passed products, check for a remote manifest to retrieve and use.
@@ -70,7 +49,7 @@ func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
if err != nil {
return err
}
err = ExtractCmd(ctx, &ExtractOpts{RootOpts: o.RootOpts}, s, fmt.Sprintf("hauler/%s-manifest.yaml:%s", parts[0],tag))
err = ExtractCmd(ctx, &flags.ExtractOpts{StoreRootOpts: o.StoreRootOpts}, s, fmt.Sprintf("hauler/%s-manifest.yaml:%s", parts[0], tag))
if err != nil {
return err
}
@@ -102,7 +81,7 @@ func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
return nil
}
func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layout) error {
func processContent(ctx context.Context, fi *os.File, o *flags.SyncOpts, s *store.Layout) error {
l := log.FromContext(ctx)
reader := yaml.NewYAMLReader(bufio.NewReader(fi))
@@ -151,19 +130,19 @@ func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layo
}
a := cfg.GetAnnotations()
for _, i := range cfg.Spec.Images {
// Check if the user provided a registry. If a registry is provided in the annotation, use it for the images that don't have a registry in their ref name.
if a[consts.ImageAnnotationRegistry] != "" || o.Registry != ""{
newRef,_ := reference.Parse(i.Name)
if a[consts.ImageAnnotationRegistry] != "" || o.Registry != "" {
newRef, _ := reference.Parse(i.Name)
newReg := o.Registry // cli flag
// if no cli flag but there was an annotation, use the annotation.
if o.Registry == "" && a[consts.ImageAnnotationRegistry] != "" {
newReg = a[consts.ImageAnnotationRegistry]
}
if newRef.Context().RegistryStr() == "" {
newRef,err = reference.Relocate(i.Name, newReg)
newRef, err = reference.Relocate(i.Name, newReg)
if err != nil {
return err
}
@@ -189,7 +168,7 @@ func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layo
}
}
l.Debugf("key for image [%s]", key)
// verify signature using the provided key.
err := cosign.VerifySignature(ctx, s, key, i.Name)
if err != nil {
@@ -209,7 +188,7 @@ func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layo
if i.Platform != "" {
platform = i.Platform
}
err = storeImage(ctx, s, i, platform)
if err != nil {
return err

View File

@@ -5,7 +5,7 @@ import (
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/internal/version"
"hauler.dev/go/hauler/internal/version"
)
func addVersion(parent *cobra.Command) {

View File

@@ -5,9 +5,9 @@ import (
"embed"
"os"
"github.com/rancherfederal/hauler/cmd/hauler/cli"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/cmd/hauler/cli"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
)
//go:embed binaries/*

View File

@@ -1,21 +0,0 @@
# HELM IGNORE OPTIONS:
# Patterns to ignore when building Helm packages.
# Supports shell glob matching, relative path matching, and negation (prefixed with !)
.DS_Store
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
*.swp
*.bak
*.tmp
*.orig
*~
.project
.idea/
*.tmproj
.vscode/

View File

@@ -1,7 +0,0 @@
apiVersion: v2
name: hauler
description: Hauler Helm Chart - Airgap Swiss Army Knife
icon: https://raw.githubusercontent.com/rancherfederal/hauler/main/static/rgs-hauler-logo-icon.svg
type: application
version: 0.1.0
appVersion: 1.0.2

View File

@@ -1,33 +0,0 @@
# Hauler Helm Chart
### Airgap Swiss Army Knife
`Rancher Government Hauler` simplifies the airgap experience without requiring operators to adopt a specific workflow. **Hauler** simplifies the airgapping process, by representing assets (images, charts, files, etc...) as content and collections to allow operators to easily fetch, store, package, and distribute these assets with declarative manifests or through the command line.
`Hauler` does this by storing contents and collections as OCI Artifacts and allows operators to serve contents and collections with an embedded registry and fileserver. Additionally, `Hauler` has the ability to store and inspect various non-image OCI Artifacts.
**GitHub Repostiory:** https://github.com/rancherfederal/hauler
**Documentation:** http://hauler.dev
---
| Type | Chart Version | App Version |
| ----------- | ------------- | ----------- |
| application | `0.1.0` | `1.0.2` |
## Installing the Chart
```bash
helm install hauler hauler/hauler -n hauler-system -f values.yaml
```
```bash
helm status hauler -n hauler-system
```
## Uninstalling the Chart
```bash
helm uninstall hauler -n hauler-system
```

View File

@@ -1,33 +0,0 @@
# Hauler Helm Chart
### Airgap Swiss Army Knife
`Rancher Government Hauler` simplifies the airgap experience without requiring operators to adopt a specific workflow. **Hauler** simplifies the airgapping process, by representing assets (images, charts, files, etc...) as content and collections to allow operators to easily fetch, store, package, and distribute these assets with declarative manifests or through the command line.
`Hauler` does this by storing contents and collections as OCI Artifacts and allows operators to serve contents and collections with an embedded registry and fileserver. Additionally, `Hauler` has the ability to store and inspect various non-image OCI Artifacts.
**GitHub Repostiory:** https://github.com/rancherfederal/hauler
**Documentation:** http://hauler.dev
---
| Type | Chart Version | App Version |
| ----------- | ------------- | ----------- |
| application | `0.1.0` | `1.0.2` |
## Installing the Chart
```bash
helm install hauler hauler/hauler -n hauler-system -f values.yaml
```
```bash
helm status hauler -n hauler-system
```
## Uninstalling the Chart
```bash
helm uninstall hauler -n hauler-system
```

View File

@@ -1,62 +0,0 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "hauler.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "hauler.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "hauler.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "hauler.labels" -}}
helm.sh/chart: {{ include "hauler.chart" . }}
{{ include "hauler.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "hauler.selectorLabels" -}}
app.kubernetes.io/name: {{ include "hauler.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "hauler.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "hauler.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

View File

@@ -1,76 +0,0 @@
{{- if .Values.haulerFileserver.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: hauler-fileserver
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.haulerFileserver.replicas }}
selector:
matchLabels:
app: hauler-fileserver
{{- include "hauler.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
app: hauler-fileserver
{{- include "hauler.selectorLabels" . | nindent 8 }}
spec:
{{- if or .Values.haulerJobs.hauls.enabled .Values.haulerJobs.manifests.enabled }}
initContainers:
{{- if .Values.haulerJobs.hauls.enabled }}
- name: wait-for-hauler-hauls-job
image: {{ .Values.hauler.initContainers.image.repository }}:{{ .Values.hauler.initContainers.image.tag }}
imagePullPolicy: {{ .Values.hauler.initContainers.imagePullPolicy }}
args: ["wait", "--for=condition=complete", "job", "hauler-hauls-job", "--namespace", "{{ .Release.Namespace }}", "--timeout={{ .Values.hauler.initContainers.timeout }}"]
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
{{- end }}
{{- if .Values.haulerJobs.manifests.enabled }}
- name: wait-for-hauler-manifests-job
image: {{ .Values.hauler.initContainers.image.repository }}:{{ .Values.hauler.initContainers.image.tag }}
imagePullPolicy: {{ .Values.hauler.initContainers.imagePullPolicy }}
args: ["wait", "--for=condition=complete", "job", "hauler-manifests-job", "--namespace", "{{ .Release.Namespace }}", "--timeout={{ .Values.hauler.initContainers.timeout }}"]
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
{{- end }}
{{- end }}
containers:
- name: hauler-fileserver
image: {{ .Values.hauler.image.repository }}:{{ .Values.hauler.image.tag }}
imagePullPolicy: {{ .Values.hauler.imagePullPolicy }}
args: ["store", "serve", "fileserver", "--port", "{{ .Values.haulerFileserver.port }}"]
ports:
- containerPort: {{ .Values.haulerFileserver.port }}
volumeMounts:
- name: hauler-data
mountPath: /store
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
restartPolicy: Always
serviceAccountName: hauler-service-account
volumes:
- name: hauler-data
persistentVolumeClaim:
claimName: hauler-data
{{- end }}

View File

@@ -1,27 +0,0 @@
{{- if and .Values.haulerFileserver.enabled .Values.haulerFileserver.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hauler-fileserver
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
rules:
- host: {{ .Values.haulerFileserver.ingress.hostname }}
http:
paths:
- backend:
service:
name: hauler-fileserver
port:
number: {{ .Values.haulerFileserver.service.ports.targetPort }}
path: /
pathType: Prefix
{{- if .Values.haulerFileserver.ingress.tls.enabled }}
tls:
- hosts:
- {{ .Values.haulerFileserver.ingress.hostname }}
secretName: {{ .Values.haulerFileserver.ingress.tls.secretName }}
{{- end }}
{{- end }}

View File

@@ -1,18 +0,0 @@
{{- if and .Values.haulerFileserver.enabled .Values.haulerFileserver.service.enabled }}
apiVersion: v1
kind: Service
metadata:
name: hauler-fileserver
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
selector:
app: hauler-fileserver
ports:
- name: hauler-fileserver
protocol: {{ .Values.haulerFileserver.service.ports.protocol }}
port: {{ .Values.haulerFileserver.service.ports.port }}
targetPort: {{ .Values.haulerFileserver.service.ports.targetPort }}
type: {{ .Values.haulerFileserver.service.type }}
{{- end }}

View File

@@ -1,76 +0,0 @@
{{- if .Values.haulerRegistry.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
name: hauler-registry
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
replicas: {{ .Values.haulerRegistry.replicas }}
selector:
matchLabels:
app: hauler-registry
{{- include "hauler.selectorLabels" . | nindent 6 }}
template:
metadata:
labels:
app: hauler-registry
{{- include "hauler.selectorLabels" . | nindent 8 }}
spec:
{{- if or .Values.haulerJobs.hauls.enabled .Values.haulerJobs.manifests.enabled }}
initContainers:
{{- if .Values.haulerJobs.hauls.enabled }}
- name: wait-for-hauler-hauls-job
image: {{ .Values.hauler.initContainers.image.repository }}:{{ .Values.hauler.initContainers.image.tag }}
imagePullPolicy: {{ .Values.hauler.initContainers.imagePullPolicy }}
args: ["wait", "--for=condition=complete", "job", "hauler-hauls-job", "--namespace", "{{ .Release.Namespace }}", "--timeout={{ .Values.hauler.initContainers.timeout }}"]
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
{{- end }}
{{- if .Values.haulerJobs.manifests.enabled }}
- name: wait-for-hauler-manifests-job
image: {{ .Values.hauler.initContainers.image.repository }}:{{ .Values.hauler.initContainers.image.tag }}
imagePullPolicy: {{ .Values.hauler.initContainers.imagePullPolicy }}
args: ["wait", "--for=condition=complete", "job", "hauler-manifests-job", "--namespace", "{{ .Release.Namespace }}", "--timeout={{ .Values.hauler.initContainers.timeout }}"]
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
{{- end }}
{{- end }}
containers:
- name: hauler-registry
image: {{ .Values.hauler.image.repository }}:{{ .Values.hauler.image.tag }}
imagePullPolicy: {{ .Values.hauler.imagePullPolicy }}
args: ["store", "serve", "registry", "--port", "{{ .Values.haulerRegistry.port }}"]
ports:
- containerPort: {{ .Values.haulerRegistry.port }}
volumeMounts:
- name: hauler-data
mountPath: /store
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
restartPolicy: Always
serviceAccountName: hauler-service-account
volumes:
- name: hauler-data
persistentVolumeClaim:
claimName: hauler-data
{{- end }}

View File

@@ -1,28 +0,0 @@
{{- if and .Values.haulerRegistry.enabled .Values.haulerRegistry.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hauler-registry
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
rules:
- host: {{ .Values.haulerRegistry.ingress.hostname }}
http:
paths:
- backend:
service:
name: hauler-registry
port:
number: {{ .Values.haulerRegistry.service.ports.targetPort }}
path: /
pathType: Prefix
{{- if .Values.haulerRegistry.ingress.tls.enabled }}
tls:
- hosts:
- {{ .Values.haulerRegistry.ingress.hostname }}
secretName: {{ .Values.haulerRegistry.ingress.tls.secretName }}
{{- end }}
{{- end }}

View File

@@ -1,18 +0,0 @@
{{- if and .Values.haulerRegistry.enabled .Values.haulerRegistry.service.enabled }}
apiVersion: v1
kind: Service
metadata:
name: hauler-registry
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
selector:
app: hauler-registry
ports:
- name: hauler-registry
protocol: {{ .Values.haulerRegistry.service.ports.protocol }}
port: {{ .Values.haulerRegistry.service.ports.port }}
targetPort: {{ .Values.haulerRegistry.service.ports.targetPort }}
type: {{ .Values.haulerRegistry.service.type }}
{{- end }}

View File

@@ -1,126 +0,0 @@
{{- if .Values.haulerJobs.hauls.enabled }}
apiVersion: batch/v1
kind: Job
metadata:
name: hauler-hauls-job
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
template:
spec:
initContainers:
- name: hauler-fetch-hauls
image: {{ .Values.haulerJobs.image.repository }}:{{ .Values.haulerJobs.image.tag }}
imagePullPolicy: {{ .Values.haulerJobs.imagePullPolicy }}
command: ["/bin/sh", "-c"]
args:
- |
{{- range .Values.haulerJobs.hauls.artifacts }}
curl -o /hauls/{{ .name }} {{ .path }} &&
{{- end }}
echo hauler fetch completed
volumeMounts:
- name: hauler-data
mountPath: /hauls
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
containers:
- name: hauler-load-hauls
image: {{ .Values.hauler.image.repository }}:{{ .Values.hauler.image.tag }}
imagePullPolicy: {{ .Values.hauler.imagePullPolicy }}
args:
- "store"
- "load"
{{- range .Values.haulerJobs.hauls.artifacts }}
- "/hauls/{{ .name }}"
{{- end }}
volumeMounts:
- name: hauler-data
mountPath: /hauls
- name: hauler-data
mountPath: /store
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
restartPolicy: OnFailure
volumes:
- name: hauler-data
persistentVolumeClaim:
claimName: hauler-data
{{- end }}
---
{{- if .Values.haulerJobs.manifests.enabled }}
apiVersion: batch/v1
kind: Job
metadata:
name: hauler-manifests-job
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
template:
spec:
initContainers:
- name: hauler-fetch-manifests
image: {{ .Values.haulerJobs.image.repository }}:{{ .Values.haulerJobs.image.tag }}
imagePullPolicy: {{ .Values.haulerJobs.imagePullPolicy }}
command: ["/bin/sh", "-c"]
args:
- |
{{- range .Values.haulerJobs.manifests.artifacts }}
curl -o /manifests/{{ .name }} {{ .path }} &&
{{- end }}
echo hauler fetch completed
volumeMounts:
- name: hauler-data
mountPath: /manifests
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
containers:
- name: hauler-load-manifests
image: {{ .Values.hauler.image.repository }}:{{ .Values.hauler.image.tag }}
imagePullPolicy: {{ .Values.hauler.imagePullPolicy }}
args:
{{- range .Values.haulerJobs.manifests.artifacts }}
- "store"
- "sync"
- "--files"
- "/manifests/{{ .name }}"
{{- end }}
volumeMounts:
- name: hauler-data
mountPath: /manifests
- name: hauler-data
mountPath: /store
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
runAsNonRoot: true
runAsUser: 1001
seccompProfile:
type: RuntimeDefault
restartPolicy: OnFailure
volumes:
- name: hauler-data
persistentVolumeClaim:
claimName: hauler-data
{{- end }}

View File

@@ -1,16 +0,0 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: hauler-data
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
spec:
accessModes:
- {{ .Values.hauler.data.pvc.accessModes }}
resources:
requests:
storage: {{ .Values.hauler.data.pvc.storageRequest }}
{{- if .Values.hauler.data.pvc.storageClass }}
storageClassName: {{ .Values.hauler.data.pvc.storageClass }}
{{- end }}

View File

@@ -1,35 +0,0 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: hauler-service-account
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: hauler-role
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
rules:
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: hauler-role-binding
namespace: {{ .Release.Namespace }}
labels:
{{- include "hauler.labels" . | nindent 4 }}
roleRef:
kind: Role
name: hauler-role
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: ServiceAccount
name: hauler-service-account
namespace: {{ .Release.Namespace }}

View File

@@ -1,94 +0,0 @@
# Helm Chart Values for Hauler
# Docs: https://hauler.dev
hauler:
image:
repository: ghcr.io/rancherfederal/hauler
tag: v1.0.2
imagePullPolicy: Always
initContainers:
image:
repository: rancher/kubectl
tag: v1.27.0 # update to your kubernetes version
imagePullPolicy: Always
timeout: 1h
data:
pvc:
accessModes: ReadWriteMany
storageClass: longhorn # optional... will use default storage class
storageRequest: 48Gi # recommended size of 3x the artifact(s)
# Helm Chart Values for the Hauler Jobs
# Docs: https://rancherfederal.github.io/hauler-docs/docs/introduction/quickstart
haulerJobs:
image:
repository: rancher/shell
tag: v0.1.22
imagePullPolicy: Always
hauls:
enabled: true
artifacts:
- path: https://raw.githubusercontent.com/rancherfederal/hauler/main/testdata/haul.tar.zst
name: haul.tar.zst
# - path: /path/to/additional-hauls.tar.zst
# name: additional-hauls.tar.zst
manifests:
enabled: true
artifacts:
- path: https://raw.githubusercontent.com/rancherfederal/hauler/main/testdata/hauler-manifest.yaml
name: hauler-manifest.yaml
# - path: /path/to/additional-manifests.yaml
# name: additional-manifests.yaml
# Helm Chart Values for the Hauler Fileserver
# Docs: https://rancherfederal.github.io/hauler-docs/docs/guides-references/command-line/hauler-store#hauler-store-serve-fileserver
haulerFileserver:
enabled: true
port: 8080 # default port for the fileserver
replicas: 1
ingress:
enabled: true
hostname: fileserver.ranchers.io
tls:
enabled: true
source: secret # only supported source
secretName: tls-certs # must be created outside of this chart
service:
enabled: true
type: ClusterIP
ports:
protocol: TCP
port: 8080 # default port for the fileserver
targetPort: 8080 # default port for the fileserver
# Helm Chart Values for the Hauler Registry
# Docs: https://rancherfederal.github.io/hauler-docs/docs/guides-references/command-line/hauler-store#hauler-store-serve-registry
haulerRegistry:
enabled: true
port: 5000 # default port for the registry
replicas: 1
ingress:
enabled: true
hostname: registry.ranchers.io
tls:
enabled: true
source: secret # only supported source
secretName: tls-certs # must be created outside of this chart
service:
enabled: true
type: ClusterIP
ports:
protocol: TCP
port: 5000 # default port for the registry
targetPort: 5000 # default port for the registry

6
go.mod
View File

@@ -1,6 +1,6 @@
module github.com/rancherfederal/hauler
module hauler.dev/go/hauler
go 1.21
go 1.23
require (
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be
@@ -54,7 +54,7 @@ require (
github.com/distribution/reference v0.5.0 // indirect
github.com/docker/cli v25.0.1+incompatible // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker v25.0.5+incompatible // indirect
github.com/docker/docker v25.0.6+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect

4
go.sum
View File

@@ -129,8 +129,8 @@ github.com/docker/cli v25.0.1+incompatible h1:mFpqnrS6Hsm3v1k7Wa/BO23oz0k121MTbT
github.com/docker/cli v25.0.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v25.0.5+incompatible h1:UmQydMduGkrD5nQde1mecF/YnSbTOaPeFIeP5C4W+DE=
github.com/docker/docker v25.0.5+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v25.0.6+incompatible h1:5cPwbwriIcsua2REJe8HqQV+6WlWc1byg2QSXzBxBGg=
github.com/docker/docker v25.0.6+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=

View File

@@ -13,15 +13,23 @@
# - curl -sfL https://get.hauler.dev | HAULER_VERSION=1.0.0 bash
# - HAULER_VERSION=1.0.0 ./install.sh
#
# Set Install Directory
# - curl -sfL https://get.hauler.dev | HAULER_INSTALL_DIR=/usr/local/bin bash
# - HAULER_INSTALL_DIR=/usr/local/bin ./install.sh
#
# Debug Usage:
# - curl -sfL https://get.hauler.dev | HAULER_DEBUG=true bash
# - HAULER_DEBUG=true ./install.sh
#
# Uninstall Usage:
# - curl -sfL https://get.hauler.dev | HAULER_UNINSTALL=true bash
# - HAULER_UNINSTALL=true ./install.sh
#
# Documentation:
# - https://hauler.dev
# - https://github.com/rancherfederal/hauler
# - https://github.com/hauler-dev/hauler
# set functions for debugging/logging
# set functions for logging
function verbose {
echo "$1"
}
@@ -36,60 +44,87 @@ function warn {
function fatal {
echo && echo "[ERROR] Hauler: $1"
exit 0
exit 1
}
# check for required dependencies
for cmd in sudo rm curl grep mkdir sed awk openssl tar; do
# debug hauler from argument or environment variable
if [ "${HAULER_DEBUG}" = "true" ]; then
set -x
fi
# start hauler preflight checks
info "Starting Preflight Checks..."
# check for required packages and dependencies
for cmd in echo curl grep sed rm mkdir awk openssl tar install source; do
if ! command -v "$cmd" &> /dev/null; then
fatal "$cmd is not installed"
fatal "$cmd is required to install Hauler"
fi
done
# set version environment variable
if [ -z "${HAULER_VERSION}" ]; then
version="${HAULER_VERSION:-$(curl -s https://api.github.com/repos/rancherfederal/hauler/releases/latest | grep '"tag_name":' | sed 's/.*"v\([^"]*\)".*/\1/')}"
else
version="${HAULER_VERSION}"
# set install directory from argument or environment variable
HAULER_INSTALL_DIR=${HAULER_INSTALL_DIR:-/usr/local/bin}
# ensure install directory exists
if [ ! -d "${HAULER_INSTALL_DIR}" ]; then
mkdir -p "${HAULER_INSTALL_DIR}" || fatal "Failed to Create Install Directory: ${HAULER_INSTALL_DIR}"
fi
# set uninstall environment variable from argument or environment
# ensure install directory is writable (by user or root privileges)
if [ ! -w "${HAULER_INSTALL_DIR}" ]; then
if [ "$(id -u)" -ne 0 ]; then
fatal "Root privileges are required to install Hauler to Directory: ${HAULER_INSTALL_DIR}"
fi
fi
# uninstall hauler from argument or environment variable
if [ "${HAULER_UNINSTALL}" = "true" ]; then
# remove the hauler binary
sudo rm -f /usr/local/bin/hauler || fatal "Failed to Remove Hauler from /usr/local/bin"
rm -rf "${HAULER_INSTALL_DIR}/hauler" || fatal "Failed to Remove Hauler from ${HAULER_INSTALL_DIR}"
# remove the installation directory
rm -rf "$HOME/.hauler" || fatal "Failed to Remove Directory: $HOME/.hauler"
# remove the working directory
rm -rf "$HOME/.hauler" || fatal "Failed to Remove Hauler Directory: $HOME/.hauler"
info "Hauler Uninstalled Successfully"
info "Successfully Uninstalled Hauler" && echo
exit 0
fi
# set version environment variable
if [ -z "${HAULER_VERSION}" ]; then
# attempt to retrieve the latest version from GitHub
HAULER_VERSION=$(curl -sI https://github.com/hauler-dev/hauler/releases/latest | grep -i location | sed -e 's#.*tag/v##' -e 's/^[[:space:]]*//g' -e 's/[[:space:]]*$//g')
# exit if the version could not be detected
if [ -z "${HAULER_VERSION}" ]; then
fatal "HAULER_VERSION is unable to be detected and/or retrieved from GitHub. Please set: HAULER_VERSION"
fi
fi
# detect the operating system
platform=$(uname -s | tr '[:upper:]' '[:lower:]')
case $platform in
PLATFORM=$(uname -s | tr '[:upper:]' '[:lower:]')
case $PLATFORM in
linux)
platform="linux"
PLATFORM="linux"
;;
darwin)
platform="darwin"
PLATFORM="darwin"
;;
*)
fatal "Unsupported Platform: $platform"
fatal "Unsupported Platform: $PLATFORM"
;;
esac
# detect the architecture
arch=$(uname -m)
case $arch in
ARCH=$(uname -m)
case $ARCH in
x86_64 | x86-32 | x64 | x32 | amd64)
arch="amd64"
ARCH="amd64"
;;
aarch64 | arm64)
arch="arm64"
ARCH="arm64"
;;
*)
fatal "Unsupported Architecture: $arch"
fatal "Unsupported Architecture: $ARCH"
;;
esac
@@ -97,85 +132,84 @@ esac
info "Starting Installation..."
# display the version, platform, and architecture
verbose "- Version: v$version"
verbose "- Platform: $platform"
verbose "- Architecture: $arch"
verbose "- Version: v${HAULER_VERSION}"
verbose "- Platform: $PLATFORM"
verbose "- Architecture: $ARCH"
verbose "- Install Directory: ${HAULER_INSTALL_DIR}"
# check if install directory exists, create it if not
# check working directory and/or create it
if [ ! -d "$HOME/.hauler" ]; then
mkdir -p "$HOME/.hauler" || fatal "Failed to Create Directory: ~/.hauler"
mkdir -p "$HOME/.hauler" || fatal "Failed to Create Directory: $HOME/.hauler"
fi
# change to install directory
cd "$HOME/.hauler" || fatal "Failed to Change Directory: ~/.hauler"
# update permissions of working directory
chmod -R 777 "$HOME/.hauler" || fatal "Failed to Update Permissions of Directory: $HOME/.hauler"
# change to working directory
cd "$HOME/.hauler" || fatal "Failed to Change Directory: $HOME/.hauler"
# start hauler artifacts download
info "Starting Download..."
# download the checksum file
if ! curl -sfOL "https://github.com/rancherfederal/hauler/releases/download/v${version}/hauler_${version}_checksums.txt"; then
fatal "Failed to Download: hauler_${version}_checksums.txt"
if ! curl -sfOL "https://github.com/hauler-dev/hauler/releases/download/v${HAULER_VERSION}/hauler_${HAULER_VERSION}_checksums.txt"; then
fatal "Failed to Download: hauler_${HAULER_VERSION}_checksums.txt"
fi
# download the archive file
if ! curl -sfOL "https://github.com/rancherfederal/hauler/releases/download/v${version}/hauler_${version}_${platform}_${arch}.tar.gz"; then
fatal "Failed to Download: hauler_${version}_${platform}_${arch}.tar.gz"
if ! curl -sfOL "https://github.com/hauler-dev/hauler/releases/download/v${HAULER_VERSION}/hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"; then
fatal "Failed to Download: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
fi
# start hauler checksum verification
info "Starting Checksum Verification..."
# verify the Hauler checksum
expected_checksum=$(awk -v version="$version" -v platform="$platform" -v arch="$arch" '$2 == "hauler_"version"_"platform"_"arch".tar.gz" {print $1}' "hauler_${version}_checksums.txt")
determined_checksum=$(openssl dgst -sha256 "hauler_${version}_${platform}_${arch}.tar.gz" | awk '{print $2}')
EXPECTED_CHECKSUM=$(awk -v HAULER_VERSION="${HAULER_VERSION}" -v PLATFORM="${PLATFORM}" -v ARCH="${ARCH}" '$2 == "hauler_"HAULER_VERSION"_"PLATFORM"_"ARCH".tar.gz" {print $1}' "hauler_${HAULER_VERSION}_checksums.txt")
DETERMINED_CHECKSUM=$(openssl dgst -sha256 "hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz" | awk '{print $2}')
if [ -z "$expected_checksum" ]; then
fatal "Failed to Locate Checksum: hauler_${version}_${platform}_${arch}.tar.gz"
elif [ "$determined_checksum" = "$expected_checksum" ]; then
verbose "- Expected Checksum: $expected_checksum"
verbose "- Determined Checksum: $determined_checksum"
verbose "- Successfully Verified Checksum: hauler_${version}_${platform}_${arch}.tar.gz"
if [ -z "${EXPECTED_CHECKSUM}" ]; then
fatal "Failed to Locate Checksum: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
elif [ "${DETERMINED_CHECKSUM}" = "${EXPECTED_CHECKSUM}" ]; then
verbose "- Expected Checksum: ${EXPECTED_CHECKSUM}"
verbose "- Determined Checksum: ${DETERMINED_CHECKSUM}"
verbose "- Successfully Verified Checksum: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
else
verbose "- Expected: $expected_checksum"
verbose "- Determined: $determined_checksum"
fatal "Failed Checksum Verification: hauler_${version}_${platform}_${arch}.tar.gz"
verbose "- Expected: ${EXPECTED_CHECKSUM}"
verbose "- Determined: ${DETERMINED_CHECKSUM}"
fatal "Failed Checksum Verification: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
fi
# uncompress the archive
tar -xzf "hauler_${version}_${platform}_${arch}.tar.gz" || fatal "Failed to Extract: hauler_${version}_${platform}_${arch}.tar.gz"
# uncompress the hauler archive
tar -xzf "hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz" || fatal "Failed to Extract: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
# install the binary
case "$platform" in
linux)
sudo install -m 755 hauler /usr/local/bin || fatal "Failed to Install Hauler to /usr/local/bin"
;;
darwin)
sudo install -m 755 hauler /usr/local/bin || fatal "Failed to Install Hauler to /usr/local/bin"
;;
*)
fatal "Unsupported Platform or Architecture: $platform/$arch"
;;
esac
# install the hauler binary
install -m 755 hauler "${HAULER_INSTALL_DIR}" || fatal "Failed to Install Hauler: ${HAULER_INSTALL_DIR}"
# add hauler to the path
if [ -f "$HOME/.bashrc" ]; then
echo "export PATH=$PATH:/usr/local/bin/" >> "$HOME/.bashrc"
source "$HOME/.bashrc"
elif [ -f "$HOME/.bash_profile" ]; then
echo "export PATH=$PATH:/usr/local/bin/" >> "$HOME/.bash_profile"
source "$HOME/.bash_profile"
elif [ -f "$HOME/.zshrc" ]; then
echo "export PATH=$PATH:/usr/local/bin/" >> "$HOME/.zshrc"
source "$HOME/.zshrc"
elif [ -f "$HOME/.profile" ]; then
echo "export PATH=$PATH:/usr/local/bin/" >> "$HOME/.profile"
source "$HOME/.profile"
else
echo "Failed to add /usr/local/bin to PATH: Unsupported Shell"
if [[ ":$PATH:" != *":${HAULER_INSTALL_DIR}:"* ]]; then
if [ -f "$HOME/.bashrc" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.bashrc"
source "$HOME/.bashrc"
elif [ -f "$HOME/.bash_profile" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.bash_profile"
source "$HOME/.bash_profile"
elif [ -f "$HOME/.zshrc" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.zshrc"
source "$HOME/.zshrc"
elif [ -f "$HOME/.profile" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.profile"
source "$HOME/.profile"
else
warn "Failed to add ${HAULER_INSTALL_DIR} to PATH: Unsupported Shell"
fi
fi
# display success message
info "Successfully Installed at /usr/local/bin/hauler"
info "Successfully Installed Hauler at ${HAULER_INSTALL_DIR}/hauler"
# display availability message
verbose "- Hauler v${version} is now available for use!"
info "Hauler v${HAULER_VERSION} is now available for use!"
# display hauler docs message
verbose "- Documentation: https://hauler.dev" && echo

49
internal/flags/add.go Normal file
View File

@@ -0,0 +1,49 @@
package flags
import (
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
)
type AddImageOpts struct {
*StoreRootOpts
Name string
Key string
Platform string
}
func (o *AddImageOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Location of public key to use for signature verification")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specifiy the platform of the image... i.e. linux/amd64 (defaults to all)")
}
type AddFileOpts struct {
*StoreRootOpts
Name string
}
func (o *AddFileOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Name, "name", "n", "", "(Optional) Rewrite the name of the file")
}
type AddChartOpts struct {
*StoreRootOpts
ChartOpts *action.ChartPathOptions
}
func (o *AddChartOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVar(&o.ChartOpts.RepoURL, "repo", "", "Location of the chart (https:// | http:// | oci://)")
f.StringVar(&o.ChartOpts.Version, "version", "", "(Optional) Specifiy the version of the chart (v1.0.0 | 2.0.0 | ^2.0.0)")
f.BoolVar(&o.ChartOpts.Verify, "verify", false, "(Optional) Verify the chart before fetching it")
f.StringVar(&o.ChartOpts.Username, "username", "", "(Optional) Username to use for authentication")
f.StringVar(&o.ChartOpts.Password, "password", "", "(Optional) Password to use for authentication")
f.StringVar(&o.ChartOpts.CertFile, "cert-file", "", "(Optional) Location of the TLS Certificate to use for client authenication")
f.StringVar(&o.ChartOpts.KeyFile, "key-file", "", "(Optional) Location of the TLS Key to use for client authenication")
f.BoolVar(&o.ChartOpts.InsecureSkipTLSverify, "insecure-skip-tls-verify", false, "(Optional) Skip TLS certificate verification")
f.StringVar(&o.ChartOpts.CaFile, "ca-file", "", "(Optional) Location of CA Bundle to enable certification verification")
}

5
internal/flags/cli.go Normal file
View File

@@ -0,0 +1,5 @@
package flags
type CliRootOpts struct {
LogLevel string
}

21
internal/flags/copy.go Normal file
View File

@@ -0,0 +1,21 @@
package flags
import "github.com/spf13/cobra"
type CopyOpts struct {
*StoreRootOpts
Username string
Password string
Insecure bool
PlainHTTP bool
}
func (o *CopyOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "(Optional) Username to use for authentication")
f.StringVarP(&o.Password, "password", "p", "", "(Optional) Password to use for authentication")
f.BoolVar(&o.Insecure, "insecure", false, "(Optional) Allow insecure connections")
f.BoolVar(&o.PlainHTTP, "plain-http", false, "(Optional) Allow plain HTTP connections")
}

14
internal/flags/extract.go Normal file
View File

@@ -0,0 +1,14 @@
package flags
import "github.com/spf13/cobra"
type ExtractOpts struct {
*StoreRootOpts
DestinationDir string
}
func (o *ExtractOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.DestinationDir, "output", "o", "", "(Optional) Specify the directory to output (defaults to current directory)")
}

20
internal/flags/info.go Normal file
View File

@@ -0,0 +1,20 @@
package flags
import "github.com/spf13/cobra"
type InfoOpts struct {
*StoreRootOpts
OutputFormat string
TypeFilter string
SizeUnit string
ListRepos bool
}
func (o *InfoOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.OutputFormat, "output", "o", "table", "(Optional) Specify the output format (table | json)")
f.StringVarP(&o.TypeFilter, "type", "t", "all", "(Optional) Filter on content type (image | chart | file | sigs | atts | sbom)")
f.BoolVar(&o.ListRepos, "list-repos", false, "(Optional) List all repository names")
}

18
internal/flags/load.go Normal file
View File

@@ -0,0 +1,18 @@
package flags
import "github.com/spf13/cobra"
type LoadOpts struct {
*StoreRootOpts
TempOverride string
}
func (o *LoadOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
// On Unix systems, the default is $TMPDIR if non-empty, else /tmp.
// On Windows, the default is GetTempPath, returning the first non-empty
// value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows directory.
// On Plan 9, the default is /tmp.
f.StringVarP(&o.TempOverride, "tempdir", "t", "", "(Optional) Override the default temporary directiory determined by the OS")
}

16
internal/flags/login.go Normal file
View File

@@ -0,0 +1,16 @@
package flags
import "github.com/spf13/cobra"
type LoginOpts struct {
Username string
Password string
PasswordStdin bool
}
func (o *LoginOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "(Optional) Username to use for authentication")
f.StringVarP(&o.Password, "password", "p", "", "(Optional) Password to use for authentication")
f.BoolVar(&o.PasswordStdin, "password-stdin", false, "(Optional) Password to use for authentication (from stdin)")
}

16
internal/flags/save.go Normal file
View File

@@ -0,0 +1,16 @@
package flags
import "github.com/spf13/cobra"
type SaveOpts struct {
*StoreRootOpts
FileName string
Platform string
}
func (o *SaveOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.FileName, "filename", "f", "haul.tar.zst", "(Optional) Specify the name of outputted archive")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specify the platform for runtime imports... i.e. linux/amd64 (unspecified implies all)")
}

87
internal/flags/serve.go Normal file
View File

@@ -0,0 +1,87 @@
package flags
import (
"fmt"
"net/http"
"github.com/distribution/distribution/v3/configuration"
"github.com/spf13/cobra"
)
type ServeRegistryOpts struct {
*StoreRootOpts
Port int
RootDir string
ConfigFile string
ReadOnly bool
TLSCert string
TLSKey string
}
func (o *ServeRegistryOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 5000, "(Optional) Specify the port to use for incoming connections")
f.StringVar(&o.RootDir, "directory", "registry", "(Optional) Directory to use for backend. Defaults to $PWD/registry")
f.StringVarP(&o.ConfigFile, "config", "c", "", "(Optional) Location of config file (overrides all flags)")
f.BoolVar(&o.ReadOnly, "readonly", true, "(Optional) Run the registry as readonly")
f.StringVar(&o.TLSCert, "tls-cert", "", "(Optional) Location of the TLS Certificate to use for server authenication")
f.StringVar(&o.TLSKey, "tls-key", "", "(Optional) Location of the TLS Key to use for server authenication")
cmd.MarkFlagsRequiredTogether("tls-cert", "tls-key")
}
func (o *ServeRegistryOpts) DefaultRegistryConfig() *configuration.Configuration {
cfg := &configuration.Configuration{
Version: "0.1",
Storage: configuration.Storage{
"cache": configuration.Parameters{"blobdescriptor": "inmemory"},
"filesystem": configuration.Parameters{"rootdirectory": o.RootDir},
"maintenance": configuration.Parameters{
"readonly": map[any]any{"enabled": o.ReadOnly},
},
},
}
if o.TLSCert != "" && o.TLSKey != "" {
cfg.HTTP.TLS.Certificate = o.TLSCert
cfg.HTTP.TLS.Key = o.TLSKey
}
cfg.HTTP.Addr = fmt.Sprintf(":%d", o.Port)
cfg.HTTP.Headers = http.Header{
"X-Content-Type-Options": []string{"nosniff"},
}
cfg.Log.Level = "info"
cfg.Validation.Manifests.URLs.Allow = []string{".+"}
return cfg
}
type ServeFilesOpts struct {
*StoreRootOpts
Port int
Timeout int
RootDir string
TLSCert string
TLSKey string
}
func (o *ServeFilesOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 8080, "(Optional) Specify the port to use for incoming connections")
f.IntVarP(&o.Timeout, "timeout", "t", 60, "(Optional) Timeout duration for HTTP Requests in seconds for both reads/writes")
f.StringVar(&o.RootDir, "directory", "fileserver", "(Optional) Directory to use for backend. Defaults to $PWD/fileserver")
f.StringVar(&o.TLSCert, "tls-cert", "", "(Optional) Location of the TLS Certificate to use for server authenication")
f.StringVar(&o.TLSKey, "tls-key", "", "(Optional) Location of the TLS Key to use for server authenication")
cmd.MarkFlagsRequiredTogether("tls-cert", "tls-key")
}

View File

@@ -1,4 +1,4 @@
package store
package flags
import (
"context"
@@ -6,28 +6,24 @@ import (
"os"
"path/filepath"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
const (
DefaultStoreName = "store"
)
type RootOpts struct {
type StoreRootOpts struct {
StoreDir string
CacheDir string
}
func (o *RootOpts) AddArgs(cmd *cobra.Command) {
func (o *StoreRootOpts) AddFlags(cmd *cobra.Command) {
pf := cmd.PersistentFlags()
pf.StringVarP(&o.StoreDir, "store", "s", DefaultStoreName, "Location to create store at")
pf.StringVarP(&o.StoreDir, "store", "s", consts.DefaultStoreName, "(Optional) Specify the directory to use for the content store")
pf.StringVar(&o.CacheDir, "cache", "", "(deprecated flag and currently not used)")
}
func (o *RootOpts) Store(ctx context.Context) (*store.Layout, error) {
func (o *StoreRootOpts) Store(ctx context.Context) (*store.Layout, error) {
l := log.FromContext(ctx)
dir := o.StoreDir

24
internal/flags/sync.go Normal file
View File

@@ -0,0 +1,24 @@
package flags
import "github.com/spf13/cobra"
type SyncOpts struct {
*StoreRootOpts
ContentFiles []string
Key string
Products []string
Platform string
Registry string
ProductRegistry string
}
func (o *SyncOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringSliceVarP(&o.ContentFiles, "files", "f", []string{}, "Location of content manifests (files)... i.e. --files ./rke2-files.yaml")
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Location of public key to use for signature verification")
f.StringSliceVar(&o.Products, "products", []string{}, "(Optional) Specify the product name to fetch collections from the product registry i.e. rancher=v2.8.5,rke2=v1.28.11+rke2r1")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specify the platform of the image... i.e linux/amd64 (defaults to all)")
f.StringVarP(&o.Registry, "registry", "r", "", "(Optional) Specify the registry of the image for images that do not alredy define one")
f.StringVarP(&o.ProductRegistry, "product-registry", "c", "", "(Optional) Specify the product registry. Defaults to RGS Carbide Registry (rgcrprod.azurecr.us)")
}

View File

@@ -2,7 +2,7 @@ package mapper
import (
"context"
"io/ioutil"
"io"
"os"
"path/filepath"
"strings"
@@ -15,7 +15,8 @@ import (
)
// NewMapperFileStore creates a new file store that uses mapper functions for each detected descriptor.
// This extends content.File, and differs in that it allows much more functionality into how each descriptor is written.
//
// This extends content.File, and differs in that it allows much more functionality into how each descriptor is written.
func NewMapperFileStore(root string, mapper map[string]Fn) *store {
fs := content.NewFile(root)
return &store{
@@ -58,7 +59,7 @@ func (s *pusher) Push(ctx context.Context, desc ocispec.Descriptor) (ccontent.Wr
// If no custom mapper found, fall back to content.File mapper
if _, ok := s.mapper[desc.MediaType]; !ok {
return content.NewIoContentWriter(ioutil.Discard, content.WithOutputHash(desc.Digest)), nil
return content.NewIoContentWriter(io.Discard, content.WithOutputHash(desc.Digest)), nil
}
filename, err := s.mapper[desc.MediaType](desc)

View File

@@ -6,7 +6,7 @@ import (
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"oras.land/oras-go/pkg/target"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
)
type Fn func(desc ocispec.Descriptor) (string, error)

View File

@@ -9,32 +9,31 @@ import (
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
"hauler.dev/go/hauler/internal/flags"
)
type FileConfig struct {
Root string
Host string
Port int
}
// NewFile returns a fileserver
// TODO: Better configs
func NewFile(ctx context.Context, cfg FileConfig) (Server, error) {
func NewFile(ctx context.Context, cfg flags.ServeFilesOpts) (Server, error) {
r := mux.NewRouter()
r.PathPrefix("/").Handler(handlers.LoggingHandler(os.Stdout, http.StripPrefix("/", http.FileServer(http.Dir(cfg.Root)))))
if cfg.Root == "" {
cfg.Root = "."
r.PathPrefix("/").Handler(handlers.LoggingHandler(os.Stdout, http.StripPrefix("/", http.FileServer(http.Dir(cfg.RootDir)))))
if cfg.RootDir == "" {
cfg.RootDir = "."
}
if cfg.Port == 0 {
cfg.Port = 8080
}
if cfg.Timeout == 0 {
cfg.Timeout = 60
}
srv := &http.Server{
Handler: r,
Addr: fmt.Sprintf(":%d", cfg.Port),
WriteTimeout: 15 * time.Second,
ReadTimeout: 15 * time.Second,
WriteTimeout: time.Duration(cfg.Timeout) * time.Second,
ReadTimeout: time.Duration(cfg.Timeout) * time.Second,
}
return srv, nil

View File

@@ -47,7 +47,7 @@ func NewTempRegistry(ctx context.Context, root string) *tmpRegistryServer {
}
// Add validation configuration
cfg.Validation.Manifests.URLs.Allow = []string{".+"}
cfg.Log.Level = "error"
cfg.HTTP.Headers = http.Header{
"X-Content-Type-Options": []string{"nosniff"},

View File

@@ -2,4 +2,5 @@ package server
type Server interface {
ListenAndServe() error
ListenAndServeTLS(string, string) error
}

View File

@@ -226,4 +226,4 @@ func (i *Info) CheckFontName(fontName string) bool {
fmt.Fprintln(os.Stderr, "font not valid, using default")
return false
}
}

View File

@@ -20,7 +20,7 @@ type ImageSpec struct {
type Image struct {
// Name is the full location for the image, can be referenced by tags or digests
Name string `json:"name"`
// Path is the path to the cosign public key used for verifying image signatures
//Key string `json:"key,omitempty"`
Key string `json:"key"`

View File

@@ -1,6 +1,8 @@
package v1alpha1
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
const K3sCollectionKind = "K3s"

View File

@@ -8,7 +8,7 @@ import (
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
)
var _ partial.Describable = (*marshallableConfig)(nil)

View File

@@ -7,9 +7,9 @@ import (
"github.com/google/go-containerregistry/pkg/v1/partial"
gtypes "github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/consts"
)
// interface guard

View File

@@ -13,9 +13,9 @@ import (
"github.com/spf13/afero"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts/file"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/consts"
)
var (

View File

@@ -13,8 +13,8 @@ import (
"github.com/opencontainers/go-digest"
"github.com/pkg/errors"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
type directory struct {

View File

@@ -7,8 +7,8 @@ import (
"os"
"path/filepath"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
type File struct{}

View File

@@ -11,9 +11,9 @@ import (
"github.com/pkg/errors"
"oras.land/oras-go/pkg/content"
content2 "github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/layer"
content2 "hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/layer"
)
type Client struct {

View File

@@ -6,7 +6,7 @@ import (
"path/filepath"
"testing"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
)
func TestClient_Detect(t *testing.T) {

View File

@@ -9,8 +9,8 @@ import (
"path/filepath"
"strings"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
type Http struct{}

View File

@@ -1,8 +1,8 @@
package file
import (
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
)
type Option func(*File)

View File

@@ -7,7 +7,7 @@ import (
gv1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/rancherfederal/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts"
)
var _ artifacts.OCI = (*Image)(nil)
@@ -54,27 +54,27 @@ func NewImage(name string, opts ...remote.Option) (*Image, error) {
}
func IsMultiArchImage(name string, opts ...remote.Option) (bool, error) {
ref, err := gname.ParseReference(name)
if err != nil {
return false, fmt.Errorf("parsing reference %q: %v", name, err)
}
ref, err := gname.ParseReference(name)
if err != nil {
return false, fmt.Errorf("parsing reference %q: %v", name, err)
}
defaultOpts := []remote.Option{
remote.WithAuthFromKeychain(authn.DefaultKeychain),
}
opts = append(opts, defaultOpts...)
desc, err := remote.Get(ref, opts...)
if err != nil {
return false, fmt.Errorf("getting image %q: %v", name, err)
}
desc, err := remote.Get(ref, opts...)
if err != nil {
return false, fmt.Errorf("getting image %q: %v", name, err)
}
_, err = desc.ImageIndex()
if err != nil {
// If the descriptor could not be converted to an image index, it's not a multi-arch image
return false, nil
}
_, err = desc.ImageIndex()
if err != nil {
// If the descriptor could not be converted to an image index, it's not a multi-arch image
return false, nil
}
// If the descriptor could be converted to an image index, it's a multi-arch image
return true, nil
}
// If the descriptor could be converted to an image index, it's a multi-arch image
return true, nil
}

View File

@@ -6,8 +6,8 @@ import (
"github.com/google/go-containerregistry/pkg/v1/static"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
var _ artifacts.OCI = (*Memory)(nil)

View File

@@ -7,7 +7,7 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/opencontainers/go-digest"
"github.com/rancherfederal/hauler/pkg/artifacts/memory"
"hauler.dev/go/hauler/pkg/artifacts/memory"
)
func TestMemory_Layers(t *testing.T) {

View File

@@ -1,6 +1,6 @@
package memory
import "github.com/rancherfederal/hauler/pkg/artifacts"
import "hauler.dev/go/hauler/pkg/artifacts"
type Option func(*Memory)

View File

@@ -3,8 +3,9 @@ package artifacts
import "github.com/google/go-containerregistry/pkg/v1"
// OCI is the bare minimum we need to represent an artifact in an oci layout
// At a high level, it is not constrained by an Image's config, manifests, and layer ordinality
// This specific implementation fully encapsulates v1.Layer's within a more generic form
//
// At a high level, it is not constrained by an Image's config, manifests, and layer ordinality
// This specific implementation fully encapsulates v1.Layer's within a more generic form
type OCI interface {
MediaType() string

View File

@@ -1,13 +1,13 @@
package chart
import (
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/content/chart"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/content/chart"
"hauler.dev/go/hauler/pkg/reference"
)
var _ artifacts.OCICollection = (*tchart)(nil)

View File

@@ -15,7 +15,7 @@ import (
"k8s.io/apimachinery/pkg/util/yaml"
"k8s.io/client-go/util/jsonpath"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
)
var defaultKnownImagePaths = []string{

View File

@@ -9,12 +9,12 @@ import (
"strings"
"sync"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/google/go-containerregistry/pkg/name"
artifact "github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
artifact "hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/log"
)
type ImageTxt struct {

View File

@@ -8,8 +8,8 @@ import (
"os"
"testing"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/image"
)
var (

View File

@@ -10,14 +10,11 @@ import (
"path"
"strings"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/reference"
)
var _ artifacts.OCICollection = (*k3s)(nil)

View File

@@ -47,11 +47,14 @@ const (
HaulerVendorPrefix = "vnd.hauler"
OCIImageIndexFile = "index.json"
KindAnnotationName = "kind"
KindAnnotation = "dev.cosignproject.cosign/image"
KindAnnotationName = "kind"
KindAnnotationImage = "dev.cosignproject.cosign/image"
KindAnnotationIndex = "dev.cosignproject.cosign/imageIndex"
CarbideRegistry = "rgcrprod.azurecr.us"
ImageAnnotationKey = "hauler.dev/key"
CarbideRegistry = "rgcrprod.azurecr.us"
ImageAnnotationKey = "hauler.dev/key"
ImageAnnotationPlatform = "hauler.dev/platform"
ImageAnnotationRegistry = "hauler.dev/registry"
DefaultStoreName = "store"
)

View File

@@ -5,8 +5,10 @@ import (
"bytes"
"compress/gzip"
"encoding/json"
"fmt"
"io"
"io/fs"
"net/url"
"os"
"path/filepath"
@@ -14,18 +16,22 @@ import (
"github.com/google/go-containerregistry/pkg/v1/partial"
gtypes "github.com/google/go-containerregistry/pkg/v1/types"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/rancherfederal/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/log"
"helm.sh/helm/v3/pkg/action"
"helm.sh/helm/v3/pkg/chart"
"helm.sh/helm/v3/pkg/chart/loader"
"helm.sh/helm/v3/pkg/cli"
"helm.sh/helm/v3/pkg/registry"
"github.com/rancherfederal/hauler/pkg/layer"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/layer"
)
var _ artifacts.OCI = (*Chart)(nil)
var (
_ artifacts.OCI = (*Chart)(nil)
settings = cli.New()
)
// Chart implements the OCI interface for Chart API objects. API spec values are
// stored into the Repo, Name, and Version fields.
@@ -36,22 +42,31 @@ type Chart struct {
// NewChart is a helper method that returns NewLocalChart or NewRemoteChart depending on v1alpha1.Chart contents
func NewChart(name string, opts *action.ChartPathOptions) (*Chart, error) {
cpo := action.ChartPathOptions{
RepoURL: opts.RepoURL,
Version: opts.Version,
CaFile: opts.CaFile,
CertFile: opts.CertFile,
KeyFile: opts.KeyFile,
InsecureSkipTLSverify: opts.InsecureSkipTLSverify,
Keyring: opts.Keyring,
Password: opts.Password,
PassCredentialsAll: opts.PassCredentialsAll,
Username: opts.Username,
Verify: opts.Verify,
chartRef := name
actionConfig := new(action.Configuration)
if err := actionConfig.Init(settings.RESTClientGetter(), settings.Namespace(), os.Getenv("HELM_DRIVER"), log.NewLogger(os.Stdout).Debugf); err != nil {
return nil, err
}
chartPath, err := cpo.LocateChart(name, cli.New())
client := action.NewInstall(actionConfig)
client.ChartPathOptions.Version = opts.Version
registryClient, err := newRegistryClient(client.CertFile, client.KeyFile, client.CaFile,
client.InsecureSkipTLSverify, client.PlainHTTP)
if err != nil {
return nil, fmt.Errorf("missing registry client: %w", err)
}
client.SetRegistryClient(registryClient)
if registry.IsOCI(opts.RepoURL) {
chartRef = opts.RepoURL + "/" + name
} else if isUrl(opts.RepoURL) { // OCI Protocol registers as a valid URL
client.ChartPathOptions.RepoURL = opts.RepoURL
} else { // Handles cases like grafana/loki
chartRef = opts.RepoURL + "/" + name
}
chartPath, err := client.ChartPathOptions.LocateChart(chartRef, settings)
if err != nil {
return nil, err
}
@@ -217,3 +232,52 @@ func (h *Chart) chartData() (gv1.Layer, error) {
return chartDataLayer, err
}
func isUrl(name string) bool {
_, err := url.ParseRequestURI(name)
return err == nil
}
func newRegistryClient(certFile, keyFile, caFile string, insecureSkipTLSverify, plainHTTP bool) (*registry.Client, error) {
if certFile != "" && keyFile != "" || caFile != "" || insecureSkipTLSverify {
registryClient, err := newRegistryClientWithTLS(certFile, keyFile, caFile, insecureSkipTLSverify)
if err != nil {
return nil, err
}
return registryClient, nil
}
registryClient, err := newDefaultRegistryClient(plainHTTP)
if err != nil {
return nil, err
}
return registryClient, nil
}
func newDefaultRegistryClient(plainHTTP bool) (*registry.Client, error) {
opts := []registry.ClientOption{
registry.ClientOptDebug(settings.Debug),
registry.ClientOptEnableCache(true),
registry.ClientOptWriter(os.Stderr),
registry.ClientOptCredentialsFile(settings.RegistryConfig),
}
if plainHTTP {
opts = append(opts, registry.ClientOptPlainHTTP())
}
// Create a new registry client
registryClient, err := registry.NewClient(opts...)
if err != nil {
return nil, err
}
return registryClient, nil
}
func newRegistryClientWithTLS(certFile, keyFile, caFile string, insecureSkipTLSverify bool) (*registry.Client, error) {
// Create a new registry client
registryClient, err := registry.NewRegistryClientWithTLS(os.Stderr, certFile, keyFile, caFile, insecureSkipTLSverify,
settings.RegistryConfig, settings.Debug,
)
if err != nil {
return nil, err
}
return registryClient, nil
}

View File

@@ -6,17 +6,11 @@ import (
"testing"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/mholt/archiver/v3"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/content/chart"
)
var (
chartpath = "../../../testdata/rancher-cluster-templates-0.4.4.tgz"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content/chart"
)
func TestNewChart(t *testing.T) {
@@ -26,10 +20,6 @@ func TestNewChart(t *testing.T) {
}
defer os.RemoveAll(tmpdir)
if err := archiver.Unarchive(chartpath, tmpdir); err != nil {
t.Fatal(err)
}
type args struct {
name string
opts *action.ChartPathOptions
@@ -43,18 +33,18 @@ func TestNewChart(t *testing.T) {
{
name: "should create from a chart archive",
args: args{
name: chartpath,
opts: &action.ChartPathOptions{},
name: "rancher-cluster-templates-0.5.2.tgz",
opts: &action.ChartPathOptions{RepoURL: "../../../testdata"},
},
want: v1.Descriptor{
MediaType: consts.ChartLayerMediaType,
Size: 13102,
Size: 14970,
Digest: v1.Hash{
Algorithm: "sha256",
Hex: "4b3bb4e474b54bf9057b298f8f11c239bb561396716d8cd5fc369c407fba2965",
Hex: "0905de044a6e57cf3cd27bfc8482753049920050b10347ae2315599bd982a0e3",
},
Annotations: map[string]string{
ocispec.AnnotationTitle: "rancher-cluster-templates-0.4.4.tgz",
ocispec.AnnotationTitle: "rancher-cluster-templates-0.5.2.tgz",
},
},
wantErr: false,
@@ -73,17 +63,17 @@ func TestNewChart(t *testing.T) {
name: "should fetch a remote chart",
args: args{
name: "cert-manager",
opts: &action.ChartPathOptions{RepoURL: "https://charts.jetstack.io", Version: "1.14.4"},
opts: &action.ChartPathOptions{RepoURL: "https://charts.jetstack.io", Version: "1.15.3"},
},
want: v1.Descriptor{
MediaType: consts.ChartLayerMediaType,
Size: 80674,
Size: 94751,
Digest: v1.Hash{
Algorithm: "sha256",
Hex: "5775fdbc1881d6e510df76d38753af54b86bd14caa8edb28fdbb79527042dede",
Hex: "016e68d9f7083d2c4fd302f951ee6490dbf4cb1ef44cfc06914c39cbfb01d858",
},
Annotations: map[string]string{
ocispec.AnnotationTitle: "cert-manager-v1.14.4.tgz",
ocispec.AnnotationTitle: "cert-manager-v1.15.3.tgz",
},
},
wantErr: false,

View File

@@ -7,7 +7,7 @@ import (
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/util/yaml"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
)
func Load(data []byte) (schema.ObjectKind, error) {

View File

@@ -4,8 +4,8 @@ import (
"context"
"encoding/json"
"fmt"
"github.com/google/go-containerregistry/pkg/name"
"io"
"io/ioutil"
"os"
"path/filepath"
"sort"
@@ -19,7 +19,8 @@ import (
"oras.land/oras-go/pkg/content"
"oras.land/oras-go/pkg/target"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/reference"
)
var _ target.Target = (*OCI)(nil)
@@ -45,8 +46,20 @@ func (o *OCI) AddIndex(desc ocispec.Descriptor) error {
if _, ok := desc.Annotations[ocispec.AnnotationRefName]; !ok {
return fmt.Errorf("descriptor must contain a reference from the annotation: %s", ocispec.AnnotationRefName)
}
key := fmt.Sprintf("%s-%s-%s", desc.Digest.String(), desc.Annotations[ocispec.AnnotationRefName], desc.Annotations[consts.KindAnnotationName])
o.nameMap.Store(key, desc)
key, err := reference.Parse(desc.Annotations[ocispec.AnnotationRefName])
if err != nil {
return err
}
if strings.TrimSpace(key.String()) != "--" {
switch key.(type) {
case name.Digest:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.Context().String(), desc.Annotations[consts.KindAnnotationName]), desc)
case name.Tag:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.String(), desc.Annotations[consts.KindAnnotationName]), desc)
}
}
return o.SaveIndex()
}
@@ -72,11 +85,21 @@ func (o *OCI) LoadIndex() error {
}
for _, desc := range o.index.Manifests {
key := fmt.Sprintf("%s-%s-%s", desc.Digest.String(), desc.Annotations[ocispec.AnnotationRefName], desc.Annotations[consts.KindAnnotationName])
if strings.TrimSpace(key) != "--" {
o.nameMap.Store(key, desc)
key, err := reference.Parse(desc.Annotations[ocispec.AnnotationRefName])
if err != nil {
return err
}
if strings.TrimSpace(key.String()) != "--" {
switch key.(type) {
case name.Digest:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.Context().String(), desc.Annotations[consts.KindAnnotationName]), desc)
case name.Tag:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.String(), desc.Annotations[consts.KindAnnotationName]), desc)
}
}
}
return nil
}
@@ -101,9 +124,9 @@ func (o *OCI) SaveIndex() error {
kindJ := descs[j].Annotations["kind"]
// Objects with the prefix of "dev.cosignproject.cosign/image" should be at the top.
if strings.HasPrefix(kindI, consts.KindAnnotation) && !strings.HasPrefix(kindJ, consts.KindAnnotation) {
if strings.HasPrefix(kindI, consts.KindAnnotationImage) && !strings.HasPrefix(kindJ, consts.KindAnnotationImage) {
return true
} else if !strings.HasPrefix(kindI, consts.KindAnnotation) && strings.HasPrefix(kindJ, consts.KindAnnotation) {
} else if !strings.HasPrefix(kindI, consts.KindAnnotationImage) && strings.HasPrefix(kindJ, consts.KindAnnotationImage) {
return false
}
return false // Default: maintain the order.
@@ -180,9 +203,11 @@ func (o *OCI) Pusher(ctx context.Context, ref string) (remotes.Pusher, error) {
var baseRef, hash string
parts := strings.SplitN(ref, "@", 2)
baseRef = parts[0]
if len(parts) > 1 {
hash = parts[1]
}
return &ociPusher{
oci: o,
ref: baseRef,
@@ -275,7 +300,7 @@ func (p *ociPusher) Push(ctx context.Context, d ocispec.Descriptor) (ccontent.Wr
if _, err := os.Stat(blobPath); err == nil {
// file already exists, discard (but validate digest)
return content.NewIoContentWriter(ioutil.Discard, content.WithOutputHash(d.Digest)), nil
return content.NewIoContentWriter(io.Discard, content.WithOutputHash(d.Digest)), nil
}
f, err := os.Create(blobPath)

View File

@@ -13,10 +13,11 @@ import (
"strings"
"time"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
"oras.land/oras-go/pkg/content"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
const maxRetries = 3
@@ -91,7 +92,7 @@ func SaveImage(ctx context.Context, s *store.Layout, ref string, platform string
// read command's stderr line by line
errors := bufio.NewScanner(stderr)
for errors.Scan() {
l.Errorf(errors.Text()) // write each line to your log, or anything you need
l.Warnf(errors.Text()) // write each line to your log, or anything you need
}
if err := errors.Err(); err != nil {
cmd.Wait()
@@ -199,7 +200,7 @@ func RetryOperation(ctx context.Context, operation func() error) error {
}
// Log the error for the current attempt.
l.Errorf("error (attempt %d/%d): %v", attempt, maxRetries, err)
l.Warnf("error (attempt %d/%d): %v", attempt, maxRetries, err)
// If this is not the last attempt, wait before retrying.
if attempt < maxRetries {
@@ -276,4 +277,4 @@ func getCosignPath() (string, error) {
binaryPath := filepath.Join(haulerDir, binaryName)
return binaryPath, nil
}
}

View File

@@ -7,7 +7,7 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts"
)
/*

View File

@@ -6,7 +6,7 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
gtypes "github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
)
type Opener func() (io.ReadCloser, error)

View File

@@ -30,13 +30,13 @@ type Fields map[string]string
// NewLogger returns a new Logger
func NewLogger(out io.Writer) Logger {
customTimeFormat := "2006-01-02 15:04:05"
zerolog.TimeFieldFormat = customTimeFormat
output := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: customTimeFormat}
l := log.Output(output)
return &logger{
zl: l.With().Timestamp().Logger(),
}
customTimeFormat := "2006-01-02 15:04:05"
zerolog.TimeFieldFormat = customTimeFormat
output := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: customTimeFormat}
l := log.Output(output)
return &logger{
zl: l.With().Timestamp().Logger(),
}
}
// FromContext returns a Logger from a context if it exists

View File

@@ -1,7 +1,7 @@
// Package reference provides general types to represent oci content within a registry or local oci layout
// Grammar (stolen mostly from containerd's grammar)
//
// reference :=
// reference :=
package reference
import (

View File

@@ -4,7 +4,7 @@ import (
"reflect"
"testing"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/reference"
)
func TestParse(t *testing.T) {

View File

@@ -15,10 +15,10 @@ import (
"oras.land/oras-go/pkg/oras"
"oras.land/oras-go/pkg/target"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/layer"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content"
"hauler.dev/go/hauler/pkg/layer"
)
type Layout struct {
@@ -118,7 +118,7 @@ func (l *Layout) AddOCI(ctx context.Context, oci artifacts.OCI, ref string) (oci
Digest: digest.FromBytes(mdata),
Size: int64(len(mdata)),
Annotations: map[string]string{
consts.KindAnnotationName: consts.KindAnnotation,
consts.KindAnnotationName: consts.KindAnnotationImage,
ocispec.AnnotationRefName: ref,
},
URLs: nil,

View File

@@ -8,8 +8,8 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/random"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/store"
)
var (

1
static/CNAME Normal file
View File

@@ -0,0 +1 @@
hauler.dev

View File

@@ -0,0 +1,9 @@
<html>
<head>
<meta name="go-import" content="hauler.dev/go/hauler git https://github.com/hauler-dev/hauler">
<meta http-equiv="refresh" content="0;URL='https://github.com/hauler-dev/hauler'">
</head>
<body>
Redirecting to the <a href="https://github.com/hauler-dev/hauler">hauler source</a>.
</body>
</html>

9
static/index.html Normal file
View File

@@ -0,0 +1,9 @@
<html>
<head>
<meta name="go-import" content="hauler.dev/go/hauler git https://github.com/hauler-dev/hauler">
<meta http-equiv="refresh" content="0;URL='https://docs.hauler.dev/docs/intro'">
</head>
<body>
Redirecting you to the <a href="https://docs.hauler.dev/docs/intro">hauler docs</a>
</body>
</html>

179
testdata/certificate-script.sh vendored Executable file
View File

@@ -0,0 +1,179 @@
#!/bin/bash
set -e
# setup directories
mkdir -p testdata/certs
cd testdata/certs
echo "<!-----------------------------------!>"
echo "<! Certificate Authority Certificate !>"
echo "<!-----------------------------------!>"
echo "Generating certificate authority private key..."
openssl genrsa -out root-ca.key 4096
echo "Generating certificate authority configuration file..."
cat <<EOF > root-ca.cnf
[ req ]
default_bits = 4096
default_keyfile = root-ca.key
distinguished_name = req_distinguished_name
req_extensions = v3_ca
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = CERTIFICATE AUTHORITY CERTIFICATE
[v3_ca]
keyUsage = critical, keyCertSign, cRLSign
extendedKeyUsage = anyExtendedKeyUsage
basicConstraints = critical, CA:TRUE
EOF
echo "Generating certificate authority certificate signing request..."
openssl req -new -sha256 -key root-ca.key -out root-ca.csr -config root-ca.cnf
echo "Generating certificate authority certificate..."
openssl x509 -req -in root-ca.csr -signkey root-ca.key -days 3650 -out root-ca.crt -extensions v3_ca -extfile root-ca.cnf
echo "Inspecting certificate authority certificate..."
openssl x509 -text -noout -in root-ca.crt > ca.txt
echo "<!------------------------------------------------!>"
echo "<! Intermediary Certificate Authority Certificate !>"
echo "<!------------------------------------------------!>"
echo "Generating intermediary certificate authority private key..."
openssl genrsa -out intermediary-ca.key 4096
echo "Generating intermediary certificate authority configuration file..."
cat <<EOF > intermediary-ca.cnf
[ req ]
default_bits = 4096
default_keyfile = intermediary-ca.key
distinguished_name = req_distinguished_name
req_extensions = v3_ca
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = INTERMEDIARY CERTIFICATE AUTHORITY CERTIFICATE
[v3_ca]
keyUsage = critical, keyCertSign, cRLSign
extendedKeyUsage = anyExtendedKeyUsage
basicConstraints = critical, CA:TRUE
EOF
echo "Generating intermediary certificate authority certificate signing request..."
openssl req -new -sha256 -key intermediary-ca.key -out intermediary-ca.csr -config intermediary-ca.cnf
echo "Generating intermediary certificate authority certificate..."
openssl x509 -req -in intermediary-ca.csr -CA root-ca.crt -CAkey root-ca.key -CAcreateserial -out intermediary-ca.crt -days 3650 -sha256 -extfile intermediary-ca.cnf -extensions v3_ca
echo "Inspecting intermediary certificate authority certificate..."
openssl x509 -text -noout -in intermediary-ca.crt > intermediary-ca.txt
echo "Verifying intermediary certificate authority certificate..."
openssl verify -CAfile root-ca.crt intermediary-ca.crt
echo "Generating full certificate chain..."
cat intermediary-ca.crt root-ca.crt > cacerts.pem
echo "<!-----------------------------------------------------------------!>"
echo "<! Server Certificate Signed by Intermediary Certificate Authority !>"
echo "<!-----------------------------------------------------------------!>"
echo "Generating server private key..."
openssl genrsa -out server-cert.key 4096
echo "Generating server certificate signing config file..."
cat <<EOF > server-cert.cnf
[ req ]
default_bits = 4096
default_keyfile = server-cert.key
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = SERVER CERTIFICATE
[v3_req]
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = localhost
DNS.2 = registry.localhost
DNS.3 = fileserver.localhost
EOF
echo "Generating server certificate signing request..."
openssl req -new -sha256 -key server-cert.key -out server-cert.csr -config server-cert.cnf
echo "Generating server certificate..."
openssl x509 -req -in server-cert.csr -CA intermediary-ca.crt -CAkey intermediary-ca.key -CAcreateserial -out server-cert.crt -days 3650 -sha256 -extfile server-cert.cnf -extensions v3_req
echo "Inspecting server certificate..."
openssl x509 -text -noout -in server-cert.crt > server-cert.txt
echo "Verifying server certificate..."
openssl verify -CAfile cacerts.pem server-cert.crt
echo "<!-----------------------------------------------------------------!>"
echo "<! Client Certificate Signed by Intermediary Certificate Authority !>"
echo "<!-----------------------------------------------------------------!>"
echo "Generating client private key..."
openssl genrsa -out client-cert.key 4096
echo "Generating client certificate signing config file..."
cat <<EOF > client-cert.cnf
[ req ]
default_bits = 4096
default_keyfile = client-cert.key
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = CLIENT CERTIFICATE
[ v3_req ]
keyUsage = digitalSignature
extendedKeyUsage = clientAuth
EOF
echo "Generating client certificate signing request..."
openssl req -new -sha256 -key client-cert.key -out client-cert.csr -config client-cert.cnf
echo "Generating client certificate..."
openssl x509 -req -in client-cert.csr -CA intermediary-ca.crt -CAkey intermediary-ca.key -CAcreateserial -out client-cert.crt -days 3650 -sha256 -extfile client-cert.cnf -extensions v3_req
echo "Inspecting client certificate..."
openssl x509 -text -noout -in client-cert.crt > client-cert.txt
echo "Verifying client certificate..."
openssl verify -CAfile cacerts.pem client-cert.crt

Some files were not shown because too many files have changed in this diff Show More