Compare commits

...

32 Commits

Author SHA1 Message Date
Jacob Blain Christen
3406d5453d [feature] build with boring crypto where available (#344) 2024-10-04 15:09:20 -07:00
Zack Brady
991f5b6bc1 updated workflow to goreleaser builds (#341)
* updated workflow to goreleaser builds
2024-10-02 11:12:32 -07:00
Zack Brady
0595ab043a added timeout to goreleaser workflow (#340) 2024-10-01 21:18:19 -04:00
Zack Brady
73e5c1ec8b trying new workflow build processes (#337)
* trying new workflow build processes
* added last bit to new try
2024-10-01 20:07:02 -04:00
Zack Brady
25d8cb83b2 improved workflow performance (#336) 2024-10-01 16:10:37 -04:00
Adam Martin
9f7229a36b have extract use proper ref (#335)
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-10-01 16:04:06 -04:00
Zack Brady
b294b6f026 yet another workflow goreleaser fix (#334) 2024-10-01 13:21:16 -04:00
Zack Brady
ebd3fd66c8 even more workflow fixes (#333)
* reverted build hooks
* updated goreleaser arguments
2024-10-01 12:26:34 -04:00
Zack Brady
6373a476b5 added more fixes to github workflow (#332) 2024-10-01 09:23:30 -04:00
Zack Brady
2c7aacd105 fixed typo in hauler store save (#331)
* fixed typo in hauler store save
* update internal/flags/save.go

Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
Signed-off-by: Zack Brady <zackbrady123@gmail.com>

---------

Signed-off-by: Zack Brady <zackbrady123@gmail.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-09-30 18:06:10 -04:00
Zack Brady
bbcbe0239a updates to fix build processes (#330) 2024-09-30 16:51:47 -04:00
Zack Brady
8a53a26a58 added integration tests for non hauler tarballs (#325)
* added tests for tarballs
* updated tests for tarball changes
* fixed tests/build for latest changes
2024-09-27 16:38:39 -04:00
Jacob Blain Christen
41d88954c6 bump: golang >= 1.23.1 (#328)
Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
2024-09-24 11:27:28 -07:00
Adam Martin
caaed30297 add platform flag to store save (#329)
* add platform flag to store save

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* use platform parser from go-cr

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-09-24 11:18:19 -04:00
Jacob Blain Christen
aee296d48d Update feature_request.md
Remove RFE from prompt.

Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
2024-09-23 10:19:11 -07:00
Zack Brady
407ed94a0b updated/standardize command descriptions (#313)
* initial desc formatting/updates
* fixed typos
* updated commands base on feedback
* more updates based on feedback

---------

Signed-off-by: Zack Brady <zackbrady123@gmail.com>
2024-09-20 18:19:49 -04:00
Adam Martin
15a9e1a3c4 use new annotation for 'store save' manifest.json (#324)
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-09-20 17:14:27 -04:00
Jacob Blain Christen
6510947bb9 enable docker load for hauler tarballs (#320)
- fixes #276

Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
2024-09-20 13:32:49 -04:00
Adam Martin
01eebd54af bump to cosign v2.2.3-carbide.3 for new annotation (#322)
* bump cosign to v2.2.3-carbide.3
* use new annotation in 'store info' when listing images

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
2024-09-18 09:56:22 -07:00
Adam Martin
5aa55e9eda continue on error when adding images to store (#317)
* continue on error when adding images to store

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* Update cmd/hauler/cli/store/add.go

Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
Signed-off-by: Adam Martin <42001113+amartin120@users.noreply.github.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
Signed-off-by: Adam Martin <42001113+amartin120@users.noreply.github.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-09-04 14:11:07 -04:00
Brandon
6f8cd04a32 Update README.md (#318)
Removing the weird GA "disclaimer"

Signed-off-by: Brandon <bgulla@users.noreply.github.com>
2024-09-04 11:10:29 -07:00
Zack Brady
02231d716f fixed completion commands (#312) 2024-08-29 19:10:59 -04:00
Jacob Blain Christen
16fa03fec8 github.com/rancherfederal/hauler => hauler.dev/go/hauler (#311) 2024-08-26 13:54:06 -07:00
Jacob Blain Christen
51fe531c64 pages: enable go install hauler.dev/go/hauler (#310)
should fix:
```
go install hauler.dev/go/hauler@latest
go: hauler.dev/go/hauler@latest: unrecognized import path "hauler.dev/go/hauler": reading https://hauler.dev/go/hauler?go-get=1: 404 Not Found
```

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-26 12:12:59 -07:00
Jacob Blain Christen
1a6ce4290f Create CNAME
Signed-off-by: Jacob Blain Christen <jacob.blain.christen@ranchergovernment.com>
2024-08-26 11:32:55 -07:00
Jacob Blain Christen
e4ec7bed76 pages: initial workflow (#309)
Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-26 11:25:54 -07:00
Zack Brady
cb81823487 testing and linting updates (#305)
* updated makefile operations

* upgraded testdata and related tests

* initial updates for tests

* fixed errors with testdata

* last bit of updates to tests

* cleaned and commented makefile

* updated tests for latest merges

---------

Signed-off-by: Zack Brady <zackbrady123@gmail.com>
2024-08-26 12:27:18 -04:00
will
2d930b5653 feat-273: TLS Flags (#303)
* fix: move constant and flags to prevent loop

* feat: add tls cert to serve

* fix: add tls cli description

* fix: remove unnecessary code

* small updates/fixed unit test errors

* fix: migrate all flags, use exported vars

* fix: standardize to AddFlags

---------

Signed-off-by: will <30413278+wcrum@users.noreply.github.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
2024-08-25 16:16:37 -04:00
Zack Brady
bd0cd8f428 added list-repos flag (#298)
Co-authored-by: Adam Toy <atoy3731@gmail.com>
2024-08-23 09:25:02 -04:00
Zack Brady
d6b3c94920 fixed hauler login typo (#299)
* updated hauler login typo

* updated hauler login descs
2024-08-23 09:12:46 -04:00
Allen Conlon
20958826ef updated cobra function for shell completion (#304) 2024-08-22 22:06:16 -04:00
Zack Brady
d633eeffcc updated install.sh to remove github api (#293)
* updated install.sh to remove github api

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>

---------

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-14 12:27:46 -07:00
80 changed files with 1550 additions and 619 deletions

View File

@@ -1,8 +0,0 @@
*
!cmd
!go.mod
!go.sum
!internal
!Makefile
!pkg
!static

View File

@@ -8,7 +8,7 @@ assignees: ''
<!-- Thank you for helping us to improve Hauler! We welcome all requests for enhancements (RFEs). Please fill out each area of the template so we can better assist you. Comments like this will be hidden when you submit, but you can delete them if you wish. -->
**Is this RFE related to an Existing Problem? If so, please describe:**
**Is this Feature/Enhancement related to an Existing Problem? If so, please describe:**
<!-- Provide a clear and concise description of the problem -->

40
.github/workflows/pages.yaml vendored Normal file
View File

@@ -0,0 +1,40 @@
# Simple workflow for deploying static content to GitHub Pages
name: 📋
on:
push:
branches:
- main
workflow_dispatch:
# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write
# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
# Single deploy job since we're just deploying
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: './static'
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

View File

@@ -7,10 +7,10 @@ on:
- '*'
jobs:
go-release:
name: Go Release Job
goreleaser:
name: GoReleaser Job
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -25,34 +25,8 @@ jobs:
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Run Go Releaser
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: "~> v2"
args: "release --clean -p 1"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
container-release:
name: Container Release Job
runs-on: ubuntu-latest
needs: [go-release]
timeout-minutes: 30
continue-on-error: true
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
go-version-file: go.mod
check-latest: true
- name: Set Up QEMU
uses: docker/setup-qemu-action@v3
@@ -74,20 +48,13 @@ jobs:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and Push Release Container to GitHub Container Registry
uses: docker/build-push-action@v5
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v6
with:
context: .
target: release
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.ref_name }}, docker.io/hauler/hauler:${{ github.ref_name }}
- name: Build and Push Debug Container to GitHub Container Registry
uses: docker/build-push-action@v5
with:
context: .
target: debug
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.repository }}-debug:${{ github.ref_name }}, docker.io/hauler/hauler-debug:${{ github.ref_name }}
distribution: goreleaser
version: "~> v2"
args: "release --clean --parallelism 1 --timeout 60m"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
DOCKER_CLI_EXPERIMENTAL: "enabled"

333
.github/workflows/tests.yaml vendored Normal file
View File

@@ -0,0 +1,333 @@
name: Tests Workflow
on:
workflow_dispatch:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version-file: go.mod
check-latest: true
- name: Install Go Releaser
uses: goreleaser/goreleaser-action@v6
with:
install-only: true
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt-get install -y make
sudo apt-get install -y build-essential
- name: Run Makefile Targets
run: |
make build-all
- name: Upload Hauler Binaries
uses: actions/upload-artifact@v4
with:
name: hauler-binaries
path: dist/*
- name: Upload Coverage Report
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage.out
integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
needs: [unit-tests]
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt-get install -y unzip
sudo apt-get install -y tree
- name: Download Artifacts
uses: actions/download-artifact@v4
with:
name: hauler-binaries
path: dist
- name: Prepare Hauler for Tests
run: |
pwd
ls -la
ls -la dist/
chmod -R 755 dist/ testdata/certificate-script.sh
sudo mv dist/hauler_linux_amd64_v1/hauler /usr/local/bin/hauler
./testdata/certificate-script.sh && sudo chown -R $(whoami) testdata/certs/
- name: Verify - hauler version
run: |
hauler version
- name: Verify - hauler completion
run: |
hauler completion
hauler completion bash
hauler completion fish
hauler completion powershell
hauler completion zsh
- name: Verify - hauler help
run: |
hauler help
- name: Verify - hauler login
run: |
hauler login --help
hauler login docker.io --username bob --password haulin
echo "hauler" | hauler login docker.io -u bob --password-stdin
- name: Remove Hauler Store Credentials
run: |
rm -rf ~/.docker/config.json
- name: Verify - hauler store
run: |
hauler store --help
- name: Verify - hauler store add
run: |
hauler store add --help
- name: Verify - hauler store add chart
run: |
hauler store add chart --help
# verify via helm repository
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable --version 2.8.4
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable --version 2.8.3 --verify
# verify via oci helm repository
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev --version 1.0.6
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev --version 1.0.4 --verify
# verify via local helm repository
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.2/rancher-cluster-templates-0.5.2.tgz
hauler store add chart rancher-cluster-templates-0.5.2.tgz --repo .
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.1/rancher-cluster-templates-0.5.1.tgz
hauler store add chart rancher-cluster-templates-0.5.1.tgz --repo . --version 0.5.1
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.0/rancher-cluster-templates-0.5.0.tgz
hauler store add chart rancher-cluster-templates-0.5.0.tgz --repo . --version 0.5.0 --verify
# verify via the hauler store contents
hauler store info
- name: Verify - hauler store add file
run: |
hauler store add file --help
# verify via remote file
hauler store add file https://get.rke2.io/install.sh
hauler store add file https://get.rke2.io/install.sh --name rke2-install.sh
# verify via local file
hauler store add file testdata/hauler-manifest.yaml
hauler store add file testdata/hauler-manifest.yaml --name hauler-manifest-local.yaml
# verify via the hauler store contents
hauler store info
- name: Verify - hauler store add image
run: |
hauler store add image --help
# verify via image reference
hauler store add image busybox
# verify via image reference with version and platform
hauler store add image busybox:stable --platform linux/amd64
# verify via image reference with full reference
hauler store add image gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5
# verify via the hauler store contents
hauler store info
- name: Verify - hauler store copy
run: |
hauler store copy --help
# need more tests here
- name: Verify - hauler store extract
run: |
hauler store extract --help
# verify via extracting hauler store content
hauler store extract hauler/hauler-manifest-local.yaml:latest
# view extracted content from store
cat hauler-manifest-local.yaml
- name: Verify - hauler store info
run: |
hauler store info --help
# verify via table output
hauler store info --output table
# verify via json output
hauler store info --output json
# verify via filtered output (chart)
hauler store info --type chart
# verify via filtered output (file)
hauler store info --type file
# verify via filtered output (image)
hauler store info --type image
# verify store directory structure
tree -hC store
- name: Verify - hauler store save
run: |
hauler store save --help
# verify via save
hauler store save
# verify via save with filename
hauler store save --filename store.tar.zst
# verify via save with filename and platform (amd64)
hauler store save --filename store-amd64.tar.zst --platform linux/amd64
- name: Remove Hauler Store Contents
run: |
rm -rf store
hauler store info
- name: Verify - hauler store load
run: |
hauler store load --help
# verify via load
hauler store load haul.tar.zst
# verify via load with filename and temp directory
hauler store load store.tar.zst --tempdir /opt
# verify via load with filename and platform (amd64)
hauler store load store-amd64.tar.zst
- name: Verify Hauler Store Contents
run: |
# verify store
hauler store info
# verify store directory structure
tree -hC store
- name: Verify - docker load
run: |
docker load --help
# verify via load
docker load --input store-amd64.tar.zst
- name: Verify Docker Images Contents
run: |
docker images --help
# verify images
docker images --all
- name: Remove Hauler Store Contents
run: |
rm -rf store haul.tar.zst store.tar.zst store-amd64.tar.zst
hauler store info
- name: Verify - hauler store sync
run: |
hauler store sync --help
# download local helm repository
curl -sfOL https://github.com/rancherfederal/rancher-cluster-templates/releases/download/rancher-cluster-templates-0.5.2/rancher-cluster-templates-0.5.2.tgz
# verify via sync
hauler store sync --files testdata/hauler-manifest-pipeline.yaml
# need more tests here
- name: Verify - hauler store serve
run: |
hauler store serve --help
- name: Verify - hauler store serve registry
run: |
hauler store serve registry --help
# verify via registry
hauler store serve registry &
until curl -sf http://localhost:5000/v2/_catalog; do : ; done
pkill -f "hauler store serve registry"
# verify via registry with different port
hauler store serve registry --port 5001 &
until curl -sf http://localhost:5001/v2/_catalog; do : ; done
pkill -f "hauler store serve registry --port 5001"
# verify via registry with different port and readonly
hauler store serve registry --port 5001 --readonly &
until curl -sf http://localhost:5001/v2/_catalog; do : ; done
pkill -f "hauler store serve registry --port 5001 --readonly"
# verify via registry with different port with readonly with tls
# hauler store serve registry --port 5001 --readonly --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key &
# until curl -sf --cacert testdata/certs/cacerts.pem https://localhost:5001/v2/_catalog; do : ; done
# pkill -f "hauler store serve registry --port 5001 --readonly --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key"
- name: Verify - hauler store serve fileserver
run: |
hauler store serve fileserver --help
# verify via fileserver
hauler store serve fileserver &
until curl -sf http://localhost:8080; do : ; done
pkill -f "hauler store serve fileserver"
# verify via fileserver with different port
hauler store serve fileserver --port 8000 &
until curl -sf http://localhost:8000; do : ; done
pkill -f "hauler store serve fileserver --port 8000"
# verify via fileserver with different port and timeout
hauler store serve fileserver --port 8000 --timeout 120 &
until curl -sf http://localhost:8000; do : ; done
pkill -f "hauler store serve fileserver --port 8000 --timeout 120"
# verify via fileserver with different port with timeout and tls
# hauler store serve fileserver --port 8000 --timeout 120 --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key &
# until curl -sf --cacert testdata/certs/cacerts.pem https://localhost:8000; do : ; done
# pkill -f "hauler store serve fileserver --port 8000 --timeout 120 --tls-cert testdata/certs/server-cert.crt --tls-key testdata/certs/server-cert.key"
- name: Verify Hauler Store Contents
run: |
# verify store
hauler store info
# verify store directory structure
tree -hC store
# verify registry directory structure
tree -hC registry
# verify fileserver directory structure
tree -hC fileserver
- name: Create Hauler Report
run: |
hauler version >> hauler-report.txt
hauler store info --output table >> hauler-report.txt
- name: Remove Hauler Store Contents
run: |
rm -rf store registry fileserver
hauler store info
- name: Upload Hauler Report
uses: actions/upload-artifact@v4
with:
name: hauler-report
path: hauler-report.txt

View File

@@ -1,43 +0,0 @@
name: Unit Test Workflow
on:
workflow_dispatch:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
unit-test:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Run Unit Tests
run: |
mkdir -p cmd/hauler/binaries
touch cmd/hauler/binaries/dummy.txt
go test -race -covermode=atomic -coverprofile=coverage.out ./...
- name: Upload Coverage Report
uses: actions/upload-artifact@v4
with:
name: coverage-report
path: coverage.out

11
.gitignore vendored
View File

@@ -1,4 +1,3 @@
.DS_Store
**/.DS_Store
.idea
.vscode
@@ -8,12 +7,12 @@
*.sln
*.sw?
*.dir-locals.el
artifacts
local-artifacts
airgap-scp.sh
dist/
tmp/
bin/
/store/
/registry/
store/
registry/
fileserver/
cmd/hauler/binaries
testdata/certs/
coverage.out

View File

@@ -3,8 +3,14 @@ version: 2
project_name: hauler
before:
hooks:
- rm -rf cmd/hauler/binaries
- mkdir -p cmd/hauler/binaries
- touch cmd/hauler/binaries/file
- go mod tidy
- go mod download
- go fmt ./...
- go vet ./...
- go test ./... -cover -race -covermode=atomic -coverprofile=coverage.out
- rm -rf cmd/hauler/binaries
release:
@@ -12,11 +18,11 @@ release:
make_latest: false
env:
- vpkg=github.com/rancherfederal/hauler/internal/version
- cosign_version=v2.2.3+carbide.2
- vpkg=hauler.dev/go/hauler/internal/version
- cosign_version=v2.2.3+carbide.3
builds:
- main: cmd/hauler/main.go
- dir: ./cmd/hauler/.
goos:
- linux
- darwin
@@ -28,12 +34,12 @@ builds:
- -s -w -X {{ .Env.vpkg }}.gitVersion={{ .Version }} -X {{ .Env.vpkg }}.gitCommit={{ .ShortCommit }} -X {{ .Env.vpkg }}.gitTreeState={{if .IsGitDirty}}dirty{{else}}clean{{end}} -X {{ .Env.vpkg }}.buildDate={{ .Date }}
hooks:
pre:
- mkdir -p cmd/hauler/binaries
- wget -P cmd/hauler/binaries/ https://github.com/hauler-dev/cosign/releases/download/{{ .Env.cosign_version }}/cosign-{{ .Os }}-{{ .Arch }}{{ if eq .Os "windows" }}.exe{{ end }}
post:
- rm -rf cmd/hauler/binaries
env:
- CGO_ENABLED=0
- GOEXPERIMENT=boringcrypto
universal_binaries:
- replace: false
@@ -50,3 +56,75 @@ brews:
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
directory: Formula
description: "Hauler CLI"
dockers:
- id: hauler-amd64
goos: linux
goarch: amd64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/amd64"
- "--target=release"
image_templates:
- "docker.io/hauler/hauler-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-amd64:{{ .Version }}"
- id: hauler-arm64
goos: linux
goarch: arm64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/arm64"
- "--target=release"
image_templates:
- "docker.io/hauler/hauler-arm64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-arm64:{{ .Version }}"
- id: hauler-debug-amd64
goos: linux
goarch: amd64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/amd64"
- "--target=debug"
image_templates:
- "docker.io/hauler/hauler-debug-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-debug-amd64:{{ .Version }}"
- id: hauler-debug-arm64
goos: linux
goarch: arm64
use: buildx
dockerfile: Dockerfile
build_flag_templates:
- "--platform=linux/arm64"
- "--target=debug"
image_templates:
- "docker.io/hauler/hauler-debug-arm64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-debug-arm64:{{ .Version }}"
docker_manifests:
- id: hauler-docker
use: docker
name_template: "docker.io/hauler/hauler:{{ .Version }}"
image_templates:
- "docker.io/hauler/hauler-amd64:{{ .Version }}"
- "docker.io/hauler/hauler-arm64:{{ .Version }}"
- id: hauler-ghcr
use: docker
name_template: "ghcr.io/hauler-dev/hauler:{{ .Version }}"
image_templates:
- "ghcr.io/hauler-dev/hauler-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-arm64:{{ .Version }}"
- id: hauler-debug-docker
use: docker
name_template: "docker.io/hauler/hauler-debug:{{ .Version }}"
image_templates:
- "docker.io/hauler/hauler-debug-amd64:{{ .Version }}"
- "docker.io/hauler/hauler-debug-arm64:{{ .Version }}"
- id: hauler-debug-ghcr
use: docker
name_template: "ghcr.io/hauler-dev/hauler-debug:{{ .Version }}"
image_templates:
- "ghcr.io/hauler-dev/hauler-debug-amd64:{{ .Version }}"
- "ghcr.io/hauler-dev/hauler-debug-arm64:{{ .Version }}"

View File

@@ -1,11 +1,8 @@
# builder stage
FROM registry.suse.com/bci/golang:1.21 AS builder
FROM registry.suse.com/bci/bci-base:15.5 AS builder
RUN zypper --non-interactive install make bash wget ca-certificates
COPY . /build
WORKDIR /build
RUN make build
# fetched from goreleaser build proccess
COPY hauler /hauler
RUN echo "hauler:x:1001:1001::/home/hauler:" > /etc/passwd \
&& echo "hauler:x:1001:hauler" > /etc/group \
@@ -25,7 +22,7 @@ COPY --from=builder --chown=hauler:hauler /tmp/. /tmp
COPY --from=builder --chown=hauler:hauler /store/. /store
COPY --from=builder --chown=hauler:hauler /registry/. /registry
COPY --from=builder --chown=hauler:hauler /fileserver/. /fileserver
COPY --from=builder --chown=hauler:hauler /build/bin/hauler /
COPY --from=builder --chown=hauler:hauler /hauler /hauler
USER hauler
ENTRYPOINT [ "/hauler" ]
@@ -37,7 +34,7 @@ COPY --from=builder /var/lib/ca-certificates/ca-bundle.pem /etc/ssl/certs/ca-cer
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group
COPY --from=builder --chown=hauler:hauler /home/hauler/. /home/hauler
COPY --from=builder --chown=hauler:hauler /build/bin/hauler /bin/hauler
COPY --from=builder --chown=hauler:hauler /hauler /usr/local/bin/hauler
RUN apk --no-cache add curl

View File

@@ -1,39 +1,53 @@
SHELL:=/bin/bash
GO_FILES=$(shell go list ./... | grep -v /vendor/)
# Makefile for hauler
COSIGN_VERSION=v2.2.3+carbide.2
# set shell
SHELL=/bin/bash
.SILENT:
# set go variables
GO_FILES=./...
GO_COVERPROFILE=coverage.out
all: fmt vet install test
# set build variables
BIN_DIRECTORY=bin
DIST_DIRECTORY=dist
BINARIES_DIRECTORY=cmd/hauler/binaries
# local build of hauler for current platform
# references/configuration from .goreleaser.yaml
build:
rm -rf cmd/hauler/binaries;\
mkdir -p cmd/hauler/binaries;\
wget -P cmd/hauler/binaries/ https://github.com/hauler-dev/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
mkdir bin;\
CGO_ENABLED=0 go build -o bin ./cmd/...;\
goreleaser build --clean --snapshot --parallelism 1 --timeout 60m --single-target
build-all: fmt vet
goreleaser build --clean --snapshot
# local build of hauler for all platforms
# references/configuration from .goreleaser.yaml
build-all:
goreleaser build --clean --snapshot --parallelism 1 --timeout 60m
# local release of hauler for all platforms
# references/configuration from .goreleaser.yaml
release:
goreleaser release --clean --snapshot --parallelism 1 --timeout 60m
# install depedencies
install:
rm -rf cmd/hauler/binaries;\
mkdir -p cmd/hauler/binaries;\
wget -P cmd/hauler/binaries/ https://github.com/hauler-dev/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
CGO_ENABLED=0 go install ./cmd/...;\
vet:
go vet $(GO_FILES)
rm -rf $(BINARIES_DIRECTORY)
mkdir -p $(BINARIES_DIRECTORY)
touch cmd/hauler/binaries/file
go mod tidy
go mod download
CGO_ENABLED=0 go install ./cmd/...
# format go code
fmt:
go fmt $(GO_FILES)
# vet go code
vet:
go vet $(GO_FILES)
# test go code
test:
go test $(GO_FILES) -cover
integration_test:
go test -tags=integration $(GO_FILES)
go test $(GO_FILES) -cover -race -covermode=atomic -coverprofile=$(GO_COVERPROFILE)
# cleanup artifacts
clean:
rm -rf bin 2> /dev/null
rm -rf $(BIN_DIRECTORY) $(BINARIES_DIRECTORY) $(DIST_DIRECTORY) $(GO_COVERPROFILE)

View File

@@ -4,8 +4,6 @@
## Airgap Swiss Army Knife
> ⚠️ **Please Note:** Hauler and the Hauler Documentation are recently Generally Available (GA).
`Rancher Government Hauler` simplifies the airgap experience without requiring operators to adopt a specific workflow. **Hauler** simplifies the airgapping process, by representing assets (images, charts, files, etc...) as content and collections to allow operators to easily fetch, store, package, and distribute these assets with declarative manifests or through the command line.
`Hauler` does this by storing contents and collections as OCI Artifacts and allows operators to serve contents and collections with an embedded registry and fileserver. Additionally, `Hauler` has the ability to store and inspect various non-image OCI Artifacts.

View File

@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto
package main
import _ "crypto/tls/fipsonly"

View File

@@ -3,14 +3,11 @@ package cli
import (
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/log"
)
type rootOpts struct {
logLevel string
}
var ro = &rootOpts{}
var ro = &flags.CliRootOpts{}
func New() *cobra.Command {
cmd := &cobra.Command{
@@ -18,7 +15,7 @@ func New() *cobra.Command {
Short: "Airgap Swiss Army Knife",
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
l := log.FromContext(cmd.Context())
l.SetLevel(ro.logLevel)
l.SetLevel(ro.LogLevel)
l.Debugf("running cli command [%s]", cmd.CommandPath())
return nil
},
@@ -28,7 +25,7 @@ func New() *cobra.Command {
}
pf := cmd.PersistentFlags()
pf.StringVarP(&ro.logLevel, "log-level", "l", "info", "")
pf.StringVarP(&ro.LogLevel, "log-level", "l", "info", "")
// Add subcommands
addLogin(cmd)

View File

@@ -10,8 +10,7 @@ import (
func addCompletion(parent *cobra.Command) {
cmd := &cobra.Command{
Use: "completion",
Short: "Generates completion scripts for various shells",
Long: `The completion sub-command generates completion scripts for various shells.`,
Short: "Generate auto-completion scripts for various shells",
}
cmd.AddCommand(
@@ -24,16 +23,10 @@ func addCompletion(parent *cobra.Command) {
parent.AddCommand(cmd)
}
func completionError(err error) ([]string, cobra.ShellCompDirective) {
cobra.CompError(err.Error())
return nil, cobra.ShellCompDirectiveError
}
func addCompletionZsh() *cobra.Command {
cmd := &cobra.Command{
Use: "zsh",
Short: "Generates zsh completion scripts",
Long: `The completion sub-command generates completion scripts for zsh.`,
Short: "Generates auto-completion scripts for zsh",
Example: `To load completion run
. <(hauler completion zsh)
@@ -51,7 +44,7 @@ func addCompletionZsh() *cobra.Command {
mv _hauler ~/.oh-my-zsh/completions # oh-my-zsh
mv _hauler ~/.zprezto/modules/completion/external/src/ # zprezto`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenZshCompletion(os.Stdout)
cmd.Root().GenZshCompletion(os.Stdout)
// Cobra doesn't source zsh completion file, explicitly doing it here
fmt.Println("compdef _hauler hauler")
},
@@ -62,8 +55,7 @@ func addCompletionZsh() *cobra.Command {
func addCompletionBash() *cobra.Command {
cmd := &cobra.Command{
Use: "bash",
Short: "Generates bash completion scripts",
Long: `The completion sub-command generates completion scripts for bash.`,
Short: "Generates auto-completion scripts for bash",
Example: `To load completion run
. <(hauler completion bash)
@@ -73,7 +65,7 @@ func addCompletionBash() *cobra.Command {
# ~/.bashrc or ~/.profile
command -v hauler >/dev/null && . <(hauler completion bash)`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenBashCompletion(os.Stdout)
cmd.Root().GenBashCompletion(os.Stdout)
},
}
return cmd
@@ -82,15 +74,14 @@ func addCompletionBash() *cobra.Command {
func addCompletionFish() *cobra.Command {
cmd := &cobra.Command{
Use: "fish",
Short: "Generates fish completion scripts",
Long: `The completion sub-command generates completion scripts for fish.`,
Short: "Generates auto-completion scripts for fish",
Example: `To configure your fish shell to load completions for each session write this script to your completions dir:
hauler completion fish > ~/.config/fish/completions/hauler.fish
See http://fishshell.com/docs/current/index.html#completion-own for more details`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenFishCompletion(os.Stdout, true)
cmd.Root().GenFishCompletion(os.Stdout, true)
},
}
return cmd
@@ -99,8 +90,7 @@ func addCompletionFish() *cobra.Command {
func addCompletionPowershell() *cobra.Command {
cmd := &cobra.Command{
Use: "powershell",
Short: "Generates powershell completion scripts",
Long: `The completion sub-command generates completion scripts for powershell.`,
Short: "Generates auto-completion scripts for powershell",
Example: `To load completion run
. <(hauler completion powershell)
@@ -117,7 +107,7 @@ func addCompletionPowershell() *cobra.Command {
cd "${XDG_CONFIG_HOME:-"$HOME/.config/"}/powershell/modules"
hauler completion powershell >> hauler-completions.ps1`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenPowerShellCompletion(os.Stdout)
cmd.Root().GenPowerShellCompletion(os.Stdout)
},
}
return cmd

View File

@@ -10,32 +10,19 @@ import (
"github.com/spf13/cobra"
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/cosign"
)
type Opts struct {
Username string
Password string
PasswordStdin bool
}
func (o *Opts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "Username")
f.StringVarP(&o.Password, "password", "p", "", "Password")
f.BoolVarP(&o.PasswordStdin, "password-stdin", "", false, "Take the password from stdin")
}
func addLogin(parent *cobra.Command) {
o := &Opts{}
o := &flags.LoginOpts{}
cmd := &cobra.Command{
Use: "login",
Short: "Log in to a registry",
Example: `
# Log in to reg.example.com
hauler login reg.example.com -u bob -p haulin`,
Args: cobra.ExactArgs(1),
Use: "login",
Short: "Login to a registry",
Long: "Login to an OCI Compliant Registry (stored at ~/.docker/config.json)",
Example: "# login to registry.example.com\nhauler login registry.example.com -u bob -p haulin",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, arg []string) error {
ctx := cmd.Context()
@@ -55,12 +42,12 @@ hauler login reg.example.com -u bob -p haulin`,
return login(ctx, o, arg[0])
},
}
o.AddArgs(cmd)
o.AddFlags(cmd)
parent.AddCommand(cmd)
}
func login(ctx context.Context, o *Opts, registry string) error {
func login(ctx context.Context, o *flags.LoginOpts, registry string) error {
ropts := content.RegistryOptions{
Username: o.Username,
Password: o.Password,

View File

@@ -6,21 +6,22 @@ import (
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/cmd/hauler/cli/store"
"hauler.dev/go/hauler/cmd/hauler/cli/store"
"hauler.dev/go/hauler/internal/flags"
)
var rootStoreOpts = &store.RootOpts{}
var rootStoreOpts = &flags.StoreRootOpts{}
func addStore(parent *cobra.Command) {
cmd := &cobra.Command{
Use: "store",
Aliases: []string{"s"},
Short: "Interact with hauler's embedded content store",
Short: "Interact with the content store",
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
}
rootStoreOpts.AddArgs(cmd)
rootStoreOpts.AddFlags(cmd)
cmd.AddCommand(
addStoreSync(),
@@ -30,8 +31,6 @@ func addStore(parent *cobra.Command) {
addStoreServe(),
addStoreInfo(),
addStoreCopy(),
// TODO: Remove this in favor of sync?
addStoreAdd(),
)
@@ -39,11 +38,11 @@ func addStore(parent *cobra.Command) {
}
func addStoreExtract() *cobra.Command {
o := &store.ExtractOpts{RootOpts: rootStoreOpts}
o := &flags.ExtractOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "extract",
Short: "Extract content from the store to disk",
Short: "Extract artifacts from the content store to disk",
Aliases: []string{"x"},
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
@@ -57,17 +56,17 @@ func addStoreExtract() *cobra.Command {
return store.ExtractCmd(ctx, o, s, args[0])
},
}
o.AddArgs(cmd)
o.AddFlags(cmd)
return cmd
}
func addStoreSync() *cobra.Command {
o := &store.SyncOpts{RootOpts: rootStoreOpts}
o := &flags.SyncOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "sync",
Short: "Sync content to the embedded content store",
Short: "Sync content to the content store",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -85,7 +84,7 @@ func addStoreSync() *cobra.Command {
}
func addStoreLoad() *cobra.Command {
o := &store.LoadOpts{RootOpts: rootStoreOpts}
o := &flags.LoadOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "load",
@@ -111,7 +110,7 @@ func addStoreLoad() *cobra.Command {
func addStoreServe() *cobra.Command {
cmd := &cobra.Command{
Use: "serve",
Short: "Expose the content of a local store through an OCI compliant registry or file server",
Short: "Serve the content store via an OCI Compliant Registry or Fileserver",
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
@@ -124,12 +123,12 @@ func addStoreServe() *cobra.Command {
return cmd
}
// RegistryCmd serves the embedded registry
// RegistryCmd serves the registry
func addStoreServeRegistry() *cobra.Command {
o := &store.ServeRegistryOpts{RootOpts: rootStoreOpts}
o := &flags.ServeRegistryOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "registry",
Short: "Serve the embedded registry",
Short: "Serve the OCI Compliant Registry",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -149,10 +148,10 @@ func addStoreServeRegistry() *cobra.Command {
// FileServerCmd serves the file server
func addStoreServeFiles() *cobra.Command {
o := &store.ServeFilesOpts{RootOpts: rootStoreOpts}
o := &flags.ServeFilesOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "fileserver",
Short: "Serve the file server",
Short: "Serve the Fileserver",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -171,7 +170,7 @@ func addStoreServeFiles() *cobra.Command {
}
func addStoreSave() *cobra.Command {
o := &store.SaveOpts{RootOpts: rootStoreOpts}
o := &flags.SaveOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "save",
@@ -189,13 +188,13 @@ func addStoreSave() *cobra.Command {
return store.SaveCmd(ctx, o, o.FileName)
},
}
o.AddArgs(cmd)
o.AddFlags(cmd)
return cmd
}
func addStoreInfo() *cobra.Command {
o := &store.InfoOpts{RootOpts: rootStoreOpts}
o := &flags.InfoOpts{StoreRootOpts: rootStoreOpts}
var allowedValues = []string{"image", "chart", "file", "sigs", "atts", "sbom", "all"}
@@ -226,11 +225,11 @@ func addStoreInfo() *cobra.Command {
}
func addStoreCopy() *cobra.Command {
o := &store.CopyOpts{RootOpts: rootStoreOpts}
o := &flags.CopyOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "copy",
Short: "Copy all store contents to another OCI registry",
Short: "Copy all store content to another location",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -251,7 +250,7 @@ func addStoreCopy() *cobra.Command {
func addStoreAdd() *cobra.Command {
cmd := &cobra.Command{
Use: "add",
Short: "Add content to store",
Short: "Add content to the store",
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
@@ -267,12 +266,20 @@ func addStoreAdd() *cobra.Command {
}
func addStoreAddFile() *cobra.Command {
o := &store.AddFileOpts{RootOpts: rootStoreOpts}
o := &flags.AddFileOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "file",
Short: "Add a file to the content store",
Args: cobra.ExactArgs(1),
Short: "Add a file to the store",
Example: `# fetch local file
hauler store add file file.txt
# fetch remote file
hauler store add file https://get.rke2.io/install.sh
# fetch remote file and assign new name
hauler store add file https://get.hauler.dev --name hauler-install.sh`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -290,12 +297,26 @@ func addStoreAddFile() *cobra.Command {
}
func addStoreAddImage() *cobra.Command {
o := &store.AddImageOpts{RootOpts: rootStoreOpts}
o := &flags.AddImageOpts{StoreRootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "image",
Short: "Add an image to the content store",
Args: cobra.ExactArgs(1),
Short: "Add a image to the store",
Example: `# fetch image
hauler store add image busybox
# fetch image with repository and tag
hauler store add image library/busybox:stable
# fetch image with full image reference and specific platform
hauler store add image ghcr.io/hauler-dev/hauler-debug:v1.0.7 --platform linux/amd64
# fetch image with full image reference via digest
hauler store add image gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5
# fetch image with full image reference, specific platform, and signature verification
hauler store add image rgcrprod.azurecr.us/hauler/rke2-manifest.yaml:v1.28.12-rke2r1 --platform linux/amd64 --key carbide-key.pub`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
@@ -313,27 +334,31 @@ func addStoreAddImage() *cobra.Command {
}
func addStoreAddChart() *cobra.Command {
o := &store.AddChartOpts{
RootOpts: rootStoreOpts,
ChartOpts: &action.ChartPathOptions{},
o := &flags.AddChartOpts{
StoreRootOpts: rootStoreOpts,
ChartOpts: &action.ChartPathOptions{},
}
cmd := &cobra.Command{
Use: "chart",
Short: "Add a local or remote chart to the content store",
Example: `
# add a local chart
hauler store add chart path/to/chart/directory
Short: "Add a helm chart to the store",
Example: `# fetch local helm chart
hauler store add chart path/to/chart/directory --repo .
# add a local compressed chart
hauler store add chart path/to/chart.tar.gz
# fetch local compressed helm chart
hauler store add chart path/to/chart.tar.gz --repo .
# add a remote chart
hauler store add chart longhorn --repo "https://charts.longhorn.io"
# fetch remote oci helm chart
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev
# add a specific version of a chart
hauler store add chart rancher --repo "https://releases.rancher.com/server-charts/latest" --version "2.6.2"
`,
# fetch remote oci helm chart with version
hauler store add chart hauler-helm --repo oci://ghcr.io/hauler-dev --version 1.0.6
# fetch remote helm chart
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/stable
# fetch remote helm chart with specific version
hauler store add chart rancher --repo https://releases.rancher.com/server-charts/latest --version 2.9.1`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()

View File

@@ -4,30 +4,20 @@ import (
"context"
"github.com/google/go-containerregistry/pkg/name"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/spf13/cobra"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/content/chart"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/artifacts/file"
"hauler.dev/go/hauler/pkg/content/chart"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type AddFileOpts struct {
*RootOpts
Name string
}
func (o *AddFileOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Name, "name", "n", "", "(Optional) Name to assign to file in store")
}
func AddFileCmd(ctx context.Context, o *AddFileOpts, s *store.Layout, reference string) error {
func AddFileCmd(ctx context.Context, o *flags.AddFileOpts, s *store.Layout, reference string) error {
cfg := v1alpha1.File{
Path: reference,
}
@@ -61,20 +51,7 @@ func storeFile(ctx context.Context, s *store.Layout, fi v1alpha1.File) error {
return nil
}
type AddImageOpts struct {
*RootOpts
Name string
Key string
Platform string
}
func (o *AddImageOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Path to the key for digital signature verification")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specific platform to save. i.e. linux/amd64. Defaults to all if flag is omitted.")
}
func AddImageCmd(ctx context.Context, o *AddImageOpts, s *store.Layout, reference string) error {
func AddImageCmd(ctx context.Context, o *flags.AddImageOpts, s *store.Layout, reference string) error {
l := log.FromContext(ctx)
cfg := v1alpha1.Image{
Name: reference,
@@ -99,39 +76,21 @@ func storeImage(ctx context.Context, s *store.Layout, i v1alpha1.Image, platform
r, err := name.ParseReference(i.Name)
if err != nil {
return err
l.Warnf("unable to parse 'image' [%s], skipping...", r.Name())
return nil
}
err = cosign.SaveImage(ctx, s, r.Name(), platform)
if err != nil {
return err
l.Warnf("unable to add 'image' [%s] to store. skipping...", r.Name())
return nil
}
l.Infof("successfully added 'image' [%s]", r.Name())
return nil
}
type AddChartOpts struct {
*RootOpts
ChartOpts *action.ChartPathOptions
}
func (o *AddChartOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVar(&o.ChartOpts.RepoURL, "repo", "", "chart repository url where to locate the requested chart")
f.StringVar(&o.ChartOpts.Version, "version", "", "specify a version constraint for the chart version to use. This constraint can be a specific tag (e.g. 1.1.1) or it may reference a valid range (e.g. ^2.0.0). If this is not specified, the latest version is used")
f.BoolVar(&o.ChartOpts.Verify, "verify", false, "verify the package before using it")
f.StringVar(&o.ChartOpts.Username, "username", "", "chart repository username where to locate the requested chart")
f.StringVar(&o.ChartOpts.Password, "password", "", "chart repository password where to locate the requested chart")
f.StringVar(&o.ChartOpts.CertFile, "cert-file", "", "identify HTTPS client using this SSL certificate file")
f.StringVar(&o.ChartOpts.KeyFile, "key-file", "", "identify HTTPS client using this SSL key file")
f.BoolVar(&o.ChartOpts.InsecureSkipTLSverify, "insecure-skip-tls-verify", false, "skip tls certificate checks for the chart download")
f.StringVar(&o.ChartOpts.CaFile, "ca-file", "", "verify certificates of HTTPS-enabled servers using this CA bundle")
}
func AddChartCmd(ctx context.Context, o *AddChartOpts, s *store.Layout, chartName string) error {
func AddChartCmd(ctx context.Context, o *flags.AddChartOpts, s *store.Layout, chartName string) error {
// TODO: Reduce duplicates between api chart and upstream helm opts
cfg := v1alpha1.Chart{
Name: chartName,

View File

@@ -5,33 +5,15 @@ import (
"fmt"
"strings"
"github.com/spf13/cobra"
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
type CopyOpts struct {
*RootOpts
Username string
Password string
Insecure bool
PlainHTTP bool
}
func (o *CopyOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "Username when copying to an authenticated remote registry")
f.StringVarP(&o.Password, "password", "p", "", "Password when copying to an authenticated remote registry")
f.BoolVar(&o.Insecure, "insecure", false, "Toggle allowing insecure connections when copying to a remote registry")
f.BoolVar(&o.PlainHTTP, "plain-http", false, "Toggle allowing plain http connections when copying to a remote registry")
}
func CopyCmd(ctx context.Context, o *CopyOpts, s *store.Layout, targetRef string) error {
func CopyCmd(ctx context.Context, o *flags.CopyOpts, s *store.Layout, targetRef string) error {
l := log.FromContext(ctx)
components := strings.SplitN(targetRef, "://", 2)

View File

@@ -7,26 +7,15 @@ import (
"strings"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/internal/mapper"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/internal/mapper"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type ExtractOpts struct {
*RootOpts
DestinationDir string
}
func (o *ExtractOpts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.DestinationDir, "output", "o", "", "Directory to save contents to (defaults to current directory)")
}
func ExtractCmd(ctx context.Context, o *ExtractOpts, s *store.Layout, ref string) error {
func ExtractCmd(ctx context.Context, o *flags.ExtractOpts, s *store.Layout, ref string) error {
l := log.FromContext(ctx)
r, err := reference.Parse(ref)
@@ -34,10 +23,12 @@ func ExtractCmd(ctx context.Context, o *ExtractOpts, s *store.Layout, ref string
return err
}
// use the repository from the context and the identifier from the reference
repo := r.Context().RepositoryStr() + ":" + r.Identifier()
found := false
if err := s.Walk(func(reference string, desc ocispec.Descriptor) error {
if !strings.Contains(reference, r.Name()) {
if !strings.Contains(reference, repo) {
return nil
}
found = true

View File

@@ -9,31 +9,14 @@ import (
"github.com/olekukonko/tablewriter"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type InfoOpts struct {
*RootOpts
OutputFormat string
TypeFilter string
SizeUnit string
}
func (o *InfoOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.OutputFormat, "output", "o", "table", "Output format (table, json)")
f.StringVarP(&o.TypeFilter, "type", "t", "all", "Filter on type (image, chart, file, sigs, atts, sbom)")
// TODO: Regex/globbing
}
func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
func InfoCmd(ctx context.Context, o *flags.InfoOpts, s *store.Layout) error {
var items []item
if err := s.Walk(func(ref string, desc ocispec.Descriptor) error {
if _, ok := desc.Annotations[ocispec.AnnotationRefName]; !ok {
@@ -121,6 +104,11 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
return err
}
if o.ListRepos {
buildListRepos(items...)
return nil
}
// sort items by ref and arch
sort.Sort(byReferenceAndArch(items))
@@ -135,6 +123,30 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
return nil
}
func buildListRepos(items ...item) {
// Create map to track unique repository names
repos := make(map[string]bool)
for _, i := range items {
repoName := ""
for j := 0; j < len(i.Reference); j++ {
if i.Reference[j] == '/' {
repoName = i.Reference[:j]
break
}
}
if repoName == "" {
repoName = i.Reference
}
repos[repoName] = true
}
// Collect and print unique repository names
for repoName := range repos {
fmt.Println(repoName)
}
}
func buildTable(items ...item) {
// Create a table for the results
table := tablewriter.NewWriter(os.Stdout)
@@ -198,7 +210,7 @@ func (a byReferenceAndArch) Less(i, j int) bool {
return a[i].Reference < a[j].Reference
}
func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat string, o *InfoOpts) item {
func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat string, o *flags.InfoOpts) item {
var size int64 = 0
for _, l := range m.Layers {
size += l.Size
@@ -226,7 +238,11 @@ func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat
ctype = "sbom"
}
ref, err := reference.Parse(desc.Annotations[ocispec.AnnotationRefName])
refName := desc.Annotations["io.containerd.image.name"]
if refName == "" {
refName = desc.Annotations[ocispec.AnnotationRefName]
}
ref, err := reference.Parse(refName)
if err != nil {
return item{}
}

View File

@@ -5,31 +5,16 @@ import (
"os"
"github.com/mholt/archiver/v3"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/content"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
type LoadOpts struct {
*RootOpts
TempOverride string
}
func (o *LoadOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
// On Unix systems, the default is $TMPDIR if non-empty, else /tmp.
// On Windows, the default is GetTempPath, returning the first non-empty
// value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows directory.
// On Plan 9, the default is /tmp.
f.StringVarP(&o.TempOverride, "tempdir", "t", "", "overrides the default directory for temporary files, as returned by your OS.")
}
// LoadCmd
// TODO: Just use mholt/archiver for now, even though we don't need most of it
func LoadCmd(ctx context.Context, o *LoadOpts, archiveRefs ...string) error {
func LoadCmd(ctx context.Context, o *flags.LoadOpts, archiveRefs ...string) error {
l := log.FromContext(ctx)
for _, archiveRef := range archiveRefs {

View File

@@ -1,30 +1,31 @@
package store
import (
"bytes"
"context"
"encoding/json"
"os"
"path"
"path/filepath"
"slices"
referencev3 "github.com/distribution/distribution/v3/reference"
"github.com/google/go-containerregistry/pkg/name"
libv1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/layout"
"github.com/google/go-containerregistry/pkg/v1/tarball"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/mholt/archiver/v3"
"github.com/spf13/cobra"
imagev1 "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/log"
)
type SaveOpts struct {
*RootOpts
FileName string
}
func (o *SaveOpts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.FileName, "filename", "f", "haul.tar.zst", "Name of archive")
}
// SaveCmd
// TODO: Just use mholt/archiver for now, even though we don't need most of it
func SaveCmd(ctx context.Context, o *SaveOpts, outputFile string) error {
func SaveCmd(ctx context.Context, o *flags.SaveOpts, outputFile string) error {
l := log.FromContext(ctx)
// TODO: Support more formats?
@@ -45,6 +46,10 @@ func SaveCmd(ctx context.Context, o *SaveOpts, outputFile string) error {
return err
}
if err := writeExportsManifest(ctx, ".", o.Platform); err != nil {
return err
}
err = a.Archive([]string{"."}, absOutputfile)
if err != nil {
return err
@@ -53,3 +58,180 @@ func SaveCmd(ctx context.Context, o *SaveOpts, outputFile string) error {
l.Infof("saved store [%s] -> [%s]", o.StoreDir, absOutputfile)
return nil
}
type exports struct {
digests []string
records map[string]tarball.Descriptor
}
func writeExportsManifest(ctx context.Context, dir string, platformStr string) error {
l := log.FromContext(ctx)
// validate platform format
platform, err := libv1.ParsePlatform(platformStr)
if err != nil {
return err
}
oci, err := layout.FromPath(dir)
if err != nil {
return err
}
idx, err := oci.ImageIndex()
if err != nil {
return err
}
imx, err := idx.IndexManifest()
if err != nil {
return err
}
x := &exports{
digests: []string{},
records: map[string]tarball.Descriptor{},
}
for _, desc := range imx.Manifests {
l.Debugf("descriptor [%s] >>> %s", desc.Digest.String(), desc.MediaType)
if artifactType := types.MediaType(desc.ArtifactType); artifactType != "" && !artifactType.IsImage() && !artifactType.IsIndex() {
l.Debugf("descriptor [%s] <<< SKIPPING ARTIFACT (%q)", desc.Digest.String(), desc.ArtifactType)
continue
}
if desc.Annotations != nil {
// we only care about images that cosign has added to the layout index
if kind, hasKind := desc.Annotations[consts.KindAnnotationName]; hasKind {
if refName, hasRefName := desc.Annotations["io.containerd.image.name"]; hasRefName {
// branch on image (aka image manifest) or image index
switch kind {
case consts.KindAnnotationImage:
if err := x.record(ctx, idx, desc, refName); err != nil {
return err
}
case consts.KindAnnotationIndex:
l.Debugf("index [%s]: digest=%s, type=%s, size=%d", refName, desc.Digest.String(), desc.MediaType, desc.Size)
// when no platform is provided, warn the user of potential mismatch on import
if platform.String() == "" {
l.Warnf("index [%s]: provide an export platform to prevent potential platform mismatch on import", refName)
}
iix, err := idx.ImageIndex(desc.Digest)
if err != nil {
return err
}
ixm, err := iix.IndexManifest()
if err != nil {
return err
}
for _, ixd := range ixm.Manifests {
if ixd.MediaType.IsImage() {
// check if platform is provided, if so, skip anything that doesn't match
if platform.String() != "" {
if ixd.Platform.Architecture != platform.Architecture || ixd.Platform.OS != platform.OS {
l.Warnf("index [%s]: digest=%s, platform=%s/%s: does not match the supplied platform, skipping", refName, desc.Digest.String(), ixd.Platform.OS, ixd.Platform.Architecture)
continue
}
}
// skip 'unknown' platforms... docker hates
if ixd.Platform.Architecture == "unknown" && ixd.Platform.OS == "unknown" {
l.Warnf("index [%s]: digest=%s, platform=%s/%s: skipping 'unknown/unknown' platform", refName, desc.Digest.String(), ixd.Platform.OS, ixd.Platform.Architecture)
continue
}
if err := x.record(ctx, iix, ixd, refName); err != nil {
return err
}
}
}
default:
l.Debugf("descriptor [%s] <<< SKIPPING KIND (%q)", desc.Digest.String(), kind)
}
}
}
}
}
buf := bytes.Buffer{}
mnf := x.describe()
err = json.NewEncoder(&buf).Encode(mnf)
if err != nil {
return err
}
return oci.WriteFile("manifest.json", buf.Bytes(), 0666)
}
func (x *exports) describe() tarball.Manifest {
m := make(tarball.Manifest, len(x.digests))
for i, d := range x.digests {
m[i] = x.records[d]
}
return m
}
func (x *exports) record(ctx context.Context, index libv1.ImageIndex, desc libv1.Descriptor, refname string) error {
l := log.FromContext(ctx)
digest := desc.Digest.String()
image, err := index.Image(desc.Digest)
if err != nil {
return err
}
config, err := image.ConfigName()
if err != nil {
return err
}
xd, recorded := x.records[digest]
if !recorded {
// record one export record per digest
x.digests = append(x.digests, digest)
xd = tarball.Descriptor{
Config: path.Join(imagev1.ImageBlobsDir, config.Algorithm, config.Hex),
RepoTags: []string{},
Layers: []string{},
}
layers, err := image.Layers()
if err != nil {
return err
}
for _, layer := range layers {
xl, err := layer.Digest()
if err != nil {
return err
}
xd.Layers = append(xd.Layers[:], path.Join(imagev1.ImageBlobsDir, xl.Algorithm, xl.Hex))
}
}
ref, err := name.ParseReference(refname)
if err != nil {
return err
}
// record tags for the digest, eliminating dupes
switch tag := ref.(type) {
case name.Tag:
named, err := referencev3.ParseNormalizedNamed(refname)
if err != nil {
return err
}
named = referencev3.TagNameOnly(named)
repotag := referencev3.FamiliarString(named)
xd.RepoTags = append(xd.RepoTags[:], repotag)
slices.Sort(xd.RepoTags)
xd.RepoTags = slices.Compact(xd.RepoTags)
ref = tag.Digest(digest)
}
l.Debugf("image [%s]: type=%s, size=%d", ref.Name(), desc.MediaType, desc.Size)
// record export descriptor for the digest
x.records[digest] = xd
return nil
}

View File

@@ -2,8 +2,6 @@ package store
import (
"context"
"fmt"
"net/http"
"os"
"github.com/distribution/distribution/v3/configuration"
@@ -12,32 +10,14 @@ import (
_ "github.com/distribution/distribution/v3/registry/storage/driver/filesystem"
_ "github.com/distribution/distribution/v3/registry/storage/driver/inmemory"
"github.com/distribution/distribution/v3/version"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/internal/server"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/internal/server"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
type ServeRegistryOpts struct {
*RootOpts
Port int
RootDir string
ConfigFile string
ReadOnly bool
}
func (o *ServeRegistryOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 5000, "Port to listen on.")
f.StringVar(&o.RootDir, "directory", "registry", "Directory to use for backend. Defaults to $PWD/registry")
f.StringVarP(&o.ConfigFile, "config", "c", "", "Path to a config file, will override all other configs")
f.BoolVar(&o.ReadOnly, "readonly", true, "Run the registry as readonly.")
}
func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout) error {
func ServeRegistryCmd(ctx context.Context, o *flags.ServeRegistryOpts, s *store.Layout) error {
l := log.FromContext(ctx)
ctx = dcontext.WithVersion(ctx, version.Version)
@@ -46,14 +26,14 @@ func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout
return err
}
opts := &CopyOpts{}
opts := &flags.CopyOpts{}
if err := CopyCmd(ctx, opts, s, "registry://"+tr.Registry()); err != nil {
return err
}
tr.Close()
cfg := o.defaultRegistryConfig()
cfg := o.DefaultRegistryConfig()
if o.ConfigFile != "" {
ucfg, err := loadConfig(o.ConfigFile)
if err != nil {
@@ -75,45 +55,30 @@ func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout
return nil
}
type ServeFilesOpts struct {
*RootOpts
Port int
Timeout int
RootDir string
}
func (o *ServeFilesOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 8080, "Port to listen on.")
f.IntVarP(&o.Timeout, "timeout", "t", 60, "Set the http request timeout duration in seconds for both reads and write.")
f.StringVar(&o.RootDir, "directory", "fileserver", "Directory to use for backend. Defaults to $PWD/fileserver")
}
func ServeFilesCmd(ctx context.Context, o *ServeFilesOpts, s *store.Layout) error {
func ServeFilesCmd(ctx context.Context, o *flags.ServeFilesOpts, s *store.Layout) error {
l := log.FromContext(ctx)
ctx = dcontext.WithVersion(ctx, version.Version)
opts := &CopyOpts{}
opts := &flags.CopyOpts{}
if err := CopyCmd(ctx, opts, s, "dir://"+o.RootDir); err != nil {
return err
}
cfg := server.FileConfig{
Root: o.RootDir,
Port: o.Port,
Timeout: o.Timeout,
}
f, err := server.NewFile(ctx, cfg)
f, err := server.NewFile(ctx, *o)
if err != nil {
return err
}
l.Infof("starting file server on port [%d]", o.Port)
if err := f.ListenAndServe(); err != nil {
return err
if o.TLSCert != "" && o.TLSKey != "" {
l.Infof("starting file server with tls on port [%d]", o.Port)
if err := f.ListenAndServeTLS(o.TLSCert, o.TLSKey); err != nil {
return err
}
} else {
l.Infof("starting file server on port [%d]", o.Port)
if err := f.ListenAndServe(); err != nil {
return err
}
}
return nil
@@ -127,27 +92,3 @@ func loadConfig(filename string) (*configuration.Configuration, error) {
return configuration.Parse(f)
}
func (o *ServeRegistryOpts) defaultRegistryConfig() *configuration.Configuration {
cfg := &configuration.Configuration{
Version: "0.1",
Storage: configuration.Storage{
"cache": configuration.Parameters{"blobdescriptor": "inmemory"},
"filesystem": configuration.Parameters{"rootdirectory": o.RootDir},
"maintenance": configuration.Parameters{
"readonly": map[any]any{"enabled": o.ReadOnly},
},
},
}
// Add validation configuration
cfg.Validation.Manifests.URLs.Allow = []string{".+"}
cfg.Log.Level = "info"
cfg.HTTP.Addr = fmt.Sprintf(":%d", o.Port)
cfg.HTTP.Headers = http.Header{
"X-Content-Type-Options": []string{"nosniff"},
}
return cfg
}

View File

@@ -9,44 +9,23 @@ import (
"strings"
"github.com/mitchellh/go-homedir"
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
"k8s.io/apimachinery/pkg/util/yaml"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
tchart "github.com/rancherfederal/hauler/pkg/collection/chart"
"github.com/rancherfederal/hauler/pkg/collection/imagetxt"
"github.com/rancherfederal/hauler/pkg/collection/k3s"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/internal/flags"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
tchart "hauler.dev/go/hauler/pkg/collection/chart"
"hauler.dev/go/hauler/pkg/collection/imagetxt"
"hauler.dev/go/hauler/pkg/collection/k3s"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/store"
)
type SyncOpts struct {
*RootOpts
ContentFiles []string
Key string
Products []string
Platform string
Registry string
ProductRegistry string
}
func (o *SyncOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringSliceVarP(&o.ContentFiles, "files", "f", []string{}, "Path(s) to local content files (Manifests). i.e. '--files ./rke2-files.yml")
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Path to the key for signature verification")
f.StringSliceVar(&o.Products, "products", []string{}, "(Optional) Feature for RGS Carbide customers to fetch collections and content from the Carbide Registry. i.e. '--product rancher=v2.8.5,rke2=v1.28.11+rke2r1'")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specific platform to save. i.e. linux/amd64. Defaults to all if flag is omitted.")
f.StringVarP(&o.Registry, "registry", "r", "", "(Optional) Default pull registry for image refs that are not specifying a registry name.")
f.StringVarP(&o.ProductRegistry, "product-registry", "c", "", "(Optional) Specific Product Registry to use. Defaults to RGS Carbide Registry (rgcrprod.azurecr.us).")
}
func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
func SyncCmd(ctx context.Context, o *flags.SyncOpts, s *store.Layout) error {
l := log.FromContext(ctx)
// if passed products, check for a remote manifest to retrieve and use.
@@ -70,7 +49,7 @@ func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
if err != nil {
return err
}
err = ExtractCmd(ctx, &ExtractOpts{RootOpts: o.RootOpts}, s, fmt.Sprintf("hauler/%s-manifest.yaml:%s", parts[0], tag))
err = ExtractCmd(ctx, &flags.ExtractOpts{StoreRootOpts: o.StoreRootOpts}, s, fmt.Sprintf("hauler/%s-manifest.yaml:%s", parts[0], tag))
if err != nil {
return err
}
@@ -102,7 +81,7 @@ func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
return nil
}
func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layout) error {
func processContent(ctx context.Context, fi *os.File, o *flags.SyncOpts, s *store.Layout) error {
l := log.FromContext(ctx)
reader := yaml.NewYAMLReader(bufio.NewReader(fi))

View File

@@ -5,7 +5,7 @@ import (
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/internal/version"
"hauler.dev/go/hauler/internal/version"
)
func addVersion(parent *cobra.Command) {

View File

@@ -5,9 +5,9 @@ import (
"embed"
"os"
"github.com/rancherfederal/hauler/cmd/hauler/cli"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/cmd/hauler/cli"
"hauler.dev/go/hauler/pkg/cosign"
"hauler.dev/go/hauler/pkg/log"
)
//go:embed binaries/*

4
go.mod
View File

@@ -1,6 +1,6 @@
module github.com/rancherfederal/hauler
module hauler.dev/go/hauler
go 1.21
go 1.23
require (
github.com/common-nighthawk/go-figure v0.0.0-20210622060536-734e95fb86be

View File

@@ -92,11 +92,11 @@ fi
# set version environment variable
if [ -z "${HAULER_VERSION}" ]; then
# attempt to retrieve the latest version from GitHub
HAULER_VERSION=$(curl -s https://api.github.com/repos/hauler-dev/hauler/releases/latest | grep '"tag_name":' | sed 's/.*"v\([^"]*\)".*/\1/')
HAULER_VERSION=$(curl -sI https://github.com/hauler-dev/hauler/releases/latest | grep -i location | sed -e 's#.*tag/v##' -e 's/^[[:space:]]*//g' -e 's/[[:space:]]*$//g')
# exit if the version could not be detected
if [ -z "${HAULER_VERSION}" ]; then
fatal "HAULER_VERSION is unable to be detected and/or retrieved from GitHub"
fatal "HAULER_VERSION is unable to be detected and/or retrieved from GitHub. Please set: HAULER_VERSION"
fi
fi

49
internal/flags/add.go Normal file
View File

@@ -0,0 +1,49 @@
package flags
import (
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
)
type AddImageOpts struct {
*StoreRootOpts
Name string
Key string
Platform string
}
func (o *AddImageOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Location of public key to use for signature verification")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specifiy the platform of the image... i.e. linux/amd64 (defaults to all)")
}
type AddFileOpts struct {
*StoreRootOpts
Name string
}
func (o *AddFileOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Name, "name", "n", "", "(Optional) Rewrite the name of the file")
}
type AddChartOpts struct {
*StoreRootOpts
ChartOpts *action.ChartPathOptions
}
func (o *AddChartOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVar(&o.ChartOpts.RepoURL, "repo", "", "Location of the chart (https:// | http:// | oci://)")
f.StringVar(&o.ChartOpts.Version, "version", "", "(Optional) Specifiy the version of the chart (v1.0.0 | 2.0.0 | ^2.0.0)")
f.BoolVar(&o.ChartOpts.Verify, "verify", false, "(Optional) Verify the chart before fetching it")
f.StringVar(&o.ChartOpts.Username, "username", "", "(Optional) Username to use for authentication")
f.StringVar(&o.ChartOpts.Password, "password", "", "(Optional) Password to use for authentication")
f.StringVar(&o.ChartOpts.CertFile, "cert-file", "", "(Optional) Location of the TLS Certificate to use for client authenication")
f.StringVar(&o.ChartOpts.KeyFile, "key-file", "", "(Optional) Location of the TLS Key to use for client authenication")
f.BoolVar(&o.ChartOpts.InsecureSkipTLSverify, "insecure-skip-tls-verify", false, "(Optional) Skip TLS certificate verification")
f.StringVar(&o.ChartOpts.CaFile, "ca-file", "", "(Optional) Location of CA Bundle to enable certification verification")
}

5
internal/flags/cli.go Normal file
View File

@@ -0,0 +1,5 @@
package flags
type CliRootOpts struct {
LogLevel string
}

21
internal/flags/copy.go Normal file
View File

@@ -0,0 +1,21 @@
package flags
import "github.com/spf13/cobra"
type CopyOpts struct {
*StoreRootOpts
Username string
Password string
Insecure bool
PlainHTTP bool
}
func (o *CopyOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "(Optional) Username to use for authentication")
f.StringVarP(&o.Password, "password", "p", "", "(Optional) Password to use for authentication")
f.BoolVar(&o.Insecure, "insecure", false, "(Optional) Allow insecure connections")
f.BoolVar(&o.PlainHTTP, "plain-http", false, "(Optional) Allow plain HTTP connections")
}

14
internal/flags/extract.go Normal file
View File

@@ -0,0 +1,14 @@
package flags
import "github.com/spf13/cobra"
type ExtractOpts struct {
*StoreRootOpts
DestinationDir string
}
func (o *ExtractOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.DestinationDir, "output", "o", "", "(Optional) Specify the directory to output (defaults to current directory)")
}

20
internal/flags/info.go Normal file
View File

@@ -0,0 +1,20 @@
package flags
import "github.com/spf13/cobra"
type InfoOpts struct {
*StoreRootOpts
OutputFormat string
TypeFilter string
SizeUnit string
ListRepos bool
}
func (o *InfoOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.OutputFormat, "output", "o", "table", "(Optional) Specify the output format (table | json)")
f.StringVarP(&o.TypeFilter, "type", "t", "all", "(Optional) Filter on content type (image | chart | file | sigs | atts | sbom)")
f.BoolVar(&o.ListRepos, "list-repos", false, "(Optional) List all repository names")
}

18
internal/flags/load.go Normal file
View File

@@ -0,0 +1,18 @@
package flags
import "github.com/spf13/cobra"
type LoadOpts struct {
*StoreRootOpts
TempOverride string
}
func (o *LoadOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
// On Unix systems, the default is $TMPDIR if non-empty, else /tmp.
// On Windows, the default is GetTempPath, returning the first non-empty
// value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows directory.
// On Plan 9, the default is /tmp.
f.StringVarP(&o.TempOverride, "tempdir", "t", "", "(Optional) Override the default temporary directiory determined by the OS")
}

16
internal/flags/login.go Normal file
View File

@@ -0,0 +1,16 @@
package flags
import "github.com/spf13/cobra"
type LoginOpts struct {
Username string
Password string
PasswordStdin bool
}
func (o *LoginOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "(Optional) Username to use for authentication")
f.StringVarP(&o.Password, "password", "p", "", "(Optional) Password to use for authentication")
f.BoolVar(&o.PasswordStdin, "password-stdin", false, "(Optional) Password to use for authentication (from stdin)")
}

16
internal/flags/save.go Normal file
View File

@@ -0,0 +1,16 @@
package flags
import "github.com/spf13/cobra"
type SaveOpts struct {
*StoreRootOpts
FileName string
Platform string
}
func (o *SaveOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.FileName, "filename", "f", "haul.tar.zst", "(Optional) Specify the name of outputted archive")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specify the platform for runtime imports... i.e. linux/amd64 (unspecified implies all)")
}

87
internal/flags/serve.go Normal file
View File

@@ -0,0 +1,87 @@
package flags
import (
"fmt"
"net/http"
"github.com/distribution/distribution/v3/configuration"
"github.com/spf13/cobra"
)
type ServeRegistryOpts struct {
*StoreRootOpts
Port int
RootDir string
ConfigFile string
ReadOnly bool
TLSCert string
TLSKey string
}
func (o *ServeRegistryOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 5000, "(Optional) Specify the port to use for incoming connections")
f.StringVar(&o.RootDir, "directory", "registry", "(Optional) Directory to use for backend. Defaults to $PWD/registry")
f.StringVarP(&o.ConfigFile, "config", "c", "", "(Optional) Location of config file (overrides all flags)")
f.BoolVar(&o.ReadOnly, "readonly", true, "(Optional) Run the registry as readonly")
f.StringVar(&o.TLSCert, "tls-cert", "", "(Optional) Location of the TLS Certificate to use for server authenication")
f.StringVar(&o.TLSKey, "tls-key", "", "(Optional) Location of the TLS Key to use for server authenication")
cmd.MarkFlagsRequiredTogether("tls-cert", "tls-key")
}
func (o *ServeRegistryOpts) DefaultRegistryConfig() *configuration.Configuration {
cfg := &configuration.Configuration{
Version: "0.1",
Storage: configuration.Storage{
"cache": configuration.Parameters{"blobdescriptor": "inmemory"},
"filesystem": configuration.Parameters{"rootdirectory": o.RootDir},
"maintenance": configuration.Parameters{
"readonly": map[any]any{"enabled": o.ReadOnly},
},
},
}
if o.TLSCert != "" && o.TLSKey != "" {
cfg.HTTP.TLS.Certificate = o.TLSCert
cfg.HTTP.TLS.Key = o.TLSKey
}
cfg.HTTP.Addr = fmt.Sprintf(":%d", o.Port)
cfg.HTTP.Headers = http.Header{
"X-Content-Type-Options": []string{"nosniff"},
}
cfg.Log.Level = "info"
cfg.Validation.Manifests.URLs.Allow = []string{".+"}
return cfg
}
type ServeFilesOpts struct {
*StoreRootOpts
Port int
Timeout int
RootDir string
TLSCert string
TLSKey string
}
func (o *ServeFilesOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 8080, "(Optional) Specify the port to use for incoming connections")
f.IntVarP(&o.Timeout, "timeout", "t", 60, "(Optional) Timeout duration for HTTP Requests in seconds for both reads/writes")
f.StringVar(&o.RootDir, "directory", "fileserver", "(Optional) Directory to use for backend. Defaults to $PWD/fileserver")
f.StringVar(&o.TLSCert, "tls-cert", "", "(Optional) Location of the TLS Certificate to use for server authenication")
f.StringVar(&o.TLSKey, "tls-key", "", "(Optional) Location of the TLS Key to use for server authenication")
cmd.MarkFlagsRequiredTogether("tls-cert", "tls-key")
}

View File

@@ -1,4 +1,4 @@
package store
package flags
import (
"context"
@@ -7,27 +7,23 @@ import (
"path/filepath"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
const (
DefaultStoreName = "store"
)
type RootOpts struct {
type StoreRootOpts struct {
StoreDir string
CacheDir string
}
func (o *RootOpts) AddArgs(cmd *cobra.Command) {
func (o *StoreRootOpts) AddFlags(cmd *cobra.Command) {
pf := cmd.PersistentFlags()
pf.StringVarP(&o.StoreDir, "store", "s", DefaultStoreName, "Location to create store at")
pf.StringVarP(&o.StoreDir, "store", "s", consts.DefaultStoreName, "(Optional) Specify the directory to use for the content store")
pf.StringVar(&o.CacheDir, "cache", "", "(deprecated flag and currently not used)")
}
func (o *RootOpts) Store(ctx context.Context) (*store.Layout, error) {
func (o *StoreRootOpts) Store(ctx context.Context) (*store.Layout, error) {
l := log.FromContext(ctx)
dir := o.StoreDir

24
internal/flags/sync.go Normal file
View File

@@ -0,0 +1,24 @@
package flags
import "github.com/spf13/cobra"
type SyncOpts struct {
*StoreRootOpts
ContentFiles []string
Key string
Products []string
Platform string
Registry string
ProductRegistry string
}
func (o *SyncOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringSliceVarP(&o.ContentFiles, "files", "f", []string{}, "Location of content manifests (files)... i.e. --files ./rke2-files.yaml")
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Location of public key to use for signature verification")
f.StringSliceVar(&o.Products, "products", []string{}, "(Optional) Specify the product name to fetch collections from the product registry i.e. rancher=v2.8.5,rke2=v1.28.11+rke2r1")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specify the platform of the image... i.e linux/amd64 (defaults to all)")
f.StringVarP(&o.Registry, "registry", "r", "", "(Optional) Specify the registry of the image for images that do not alredy define one")
f.StringVarP(&o.ProductRegistry, "product-registry", "c", "", "(Optional) Specify the product registry. Defaults to RGS Carbide Registry (rgcrprod.azurecr.us)")
}

View File

@@ -6,7 +6,7 @@ import (
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"oras.land/oras-go/pkg/target"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
)
type Fn func(desc ocispec.Descriptor) (string, error)

View File

@@ -9,22 +9,16 @@ import (
"github.com/gorilla/handlers"
"github.com/gorilla/mux"
"hauler.dev/go/hauler/internal/flags"
)
type FileConfig struct {
Root string
Host string
Port int
Timeout int
}
// NewFile returns a fileserver
// TODO: Better configs
func NewFile(ctx context.Context, cfg FileConfig) (Server, error) {
func NewFile(ctx context.Context, cfg flags.ServeFilesOpts) (Server, error) {
r := mux.NewRouter()
r.PathPrefix("/").Handler(handlers.LoggingHandler(os.Stdout, http.StripPrefix("/", http.FileServer(http.Dir(cfg.Root)))))
if cfg.Root == "" {
cfg.Root = "."
r.PathPrefix("/").Handler(handlers.LoggingHandler(os.Stdout, http.StripPrefix("/", http.FileServer(http.Dir(cfg.RootDir)))))
if cfg.RootDir == "" {
cfg.RootDir = "."
}
if cfg.Port == 0 {

View File

@@ -2,4 +2,5 @@ package server
type Server interface {
ListenAndServe() error
ListenAndServeTLS(string, string) error
}

View File

@@ -8,7 +8,7 @@ import (
"github.com/google/go-containerregistry/pkg/v1/partial"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
)
var _ partial.Describable = (*marshallableConfig)(nil)

View File

@@ -7,9 +7,9 @@ import (
"github.com/google/go-containerregistry/pkg/v1/partial"
gtypes "github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/consts"
)
// interface guard

View File

@@ -13,9 +13,9 @@ import (
"github.com/spf13/afero"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts/file"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/consts"
)
var (

View File

@@ -13,8 +13,8 @@ import (
"github.com/opencontainers/go-digest"
"github.com/pkg/errors"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
type directory struct {

View File

@@ -7,8 +7,8 @@ import (
"os"
"path/filepath"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
type File struct{}

View File

@@ -11,9 +11,9 @@ import (
"github.com/pkg/errors"
"oras.land/oras-go/pkg/content"
content2 "github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/layer"
content2 "hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/layer"
)
type Client struct {

View File

@@ -6,7 +6,7 @@ import (
"path/filepath"
"testing"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
)
func TestClient_Detect(t *testing.T) {

View File

@@ -9,8 +9,8 @@ import (
"path/filepath"
"strings"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
type Http struct{}

View File

@@ -1,8 +1,8 @@
package file
import (
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
)
type Option func(*File)

View File

@@ -7,7 +7,7 @@ import (
gv1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/rancherfederal/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts"
)
var _ artifacts.OCI = (*Image)(nil)

View File

@@ -6,8 +6,8 @@ import (
"github.com/google/go-containerregistry/pkg/v1/static"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
)
var _ artifacts.OCI = (*Memory)(nil)

View File

@@ -7,7 +7,7 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/opencontainers/go-digest"
"github.com/rancherfederal/hauler/pkg/artifacts/memory"
"hauler.dev/go/hauler/pkg/artifacts/memory"
)
func TestMemory_Layers(t *testing.T) {

View File

@@ -1,6 +1,6 @@
package memory
import "github.com/rancherfederal/hauler/pkg/artifacts"
import "hauler.dev/go/hauler/pkg/artifacts"
type Option func(*Memory)

View File

@@ -3,11 +3,11 @@ package chart
import (
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/content/chart"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/content/chart"
"hauler.dev/go/hauler/pkg/reference"
)
var _ artifacts.OCICollection = (*tchart)(nil)

View File

@@ -15,7 +15,7 @@ import (
"k8s.io/apimachinery/pkg/util/yaml"
"k8s.io/client-go/util/jsonpath"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
)
var defaultKnownImagePaths = []string{

View File

@@ -11,10 +11,10 @@ import (
"github.com/google/go-containerregistry/pkg/name"
artifact "github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/log"
artifact "hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/log"
)
type ImageTxt struct {

View File

@@ -8,8 +8,8 @@ import (
"os"
"testing"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/image"
)
var (

View File

@@ -10,11 +10,11 @@ import (
"path"
"strings"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts/file"
"hauler.dev/go/hauler/pkg/artifacts/file/getter"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/reference"
)
var _ artifacts.OCICollection = (*k3s)(nil)

View File

@@ -47,11 +47,14 @@ const (
HaulerVendorPrefix = "vnd.hauler"
OCIImageIndexFile = "index.json"
KindAnnotationName = "kind"
KindAnnotation = "dev.cosignproject.cosign/image"
KindAnnotationName = "kind"
KindAnnotationImage = "dev.cosignproject.cosign/image"
KindAnnotationIndex = "dev.cosignproject.cosign/imageIndex"
CarbideRegistry = "rgcrprod.azurecr.us"
ImageAnnotationKey = "hauler.dev/key"
ImageAnnotationPlatform = "hauler.dev/platform"
ImageAnnotationRegistry = "hauler.dev/registry"
DefaultStoreName = "store"
)

View File

@@ -16,16 +16,16 @@ import (
"github.com/google/go-containerregistry/pkg/v1/partial"
gtypes "github.com/google/go-containerregistry/pkg/v1/types"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/log"
"helm.sh/helm/v3/pkg/action"
"helm.sh/helm/v3/pkg/chart"
"helm.sh/helm/v3/pkg/chart/loader"
"helm.sh/helm/v3/pkg/cli"
"helm.sh/helm/v3/pkg/registry"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/layer"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/layer"
)
var (

View File

@@ -9,8 +9,8 @@ import (
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/content/chart"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content/chart"
)
func TestNewChart(t *testing.T) {
@@ -33,18 +33,18 @@ func TestNewChart(t *testing.T) {
{
name: "should create from a chart archive",
args: args{
name: "rancher-cluster-templates-0.4.4.tgz",
name: "rancher-cluster-templates-0.5.2.tgz",
opts: &action.ChartPathOptions{RepoURL: "../../../testdata"},
},
want: v1.Descriptor{
MediaType: consts.ChartLayerMediaType,
Size: 13102,
Size: 14970,
Digest: v1.Hash{
Algorithm: "sha256",
Hex: "4b3bb4e474b54bf9057b298f8f11c239bb561396716d8cd5fc369c407fba2965",
Hex: "0905de044a6e57cf3cd27bfc8482753049920050b10347ae2315599bd982a0e3",
},
Annotations: map[string]string{
ocispec.AnnotationTitle: "rancher-cluster-templates-0.4.4.tgz",
ocispec.AnnotationTitle: "rancher-cluster-templates-0.5.2.tgz",
},
},
wantErr: false,
@@ -63,17 +63,17 @@ func TestNewChart(t *testing.T) {
name: "should fetch a remote chart",
args: args{
name: "cert-manager",
opts: &action.ChartPathOptions{RepoURL: "https://charts.jetstack.io", Version: "1.14.4"},
opts: &action.ChartPathOptions{RepoURL: "https://charts.jetstack.io", Version: "1.15.3"},
},
want: v1.Descriptor{
MediaType: consts.ChartLayerMediaType,
Size: 80674,
Size: 94751,
Digest: v1.Hash{
Algorithm: "sha256",
Hex: "5775fdbc1881d6e510df76d38753af54b86bd14caa8edb28fdbb79527042dede",
Hex: "016e68d9f7083d2c4fd302f951ee6490dbf4cb1ef44cfc06914c39cbfb01d858",
},
Annotations: map[string]string{
ocispec.AnnotationTitle: "cert-manager-v1.14.4.tgz",
ocispec.AnnotationTitle: "cert-manager-v1.15.3.tgz",
},
},
wantErr: false,

View File

@@ -7,7 +7,7 @@ import (
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/apimachinery/pkg/util/yaml"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"hauler.dev/go/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
)
func Load(data []byte) (schema.ObjectKind, error) {

View File

@@ -19,8 +19,8 @@ import (
"oras.land/oras-go/pkg/content"
"oras.land/oras-go/pkg/target"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/reference"
)
var _ target.Target = (*OCI)(nil)
@@ -124,9 +124,9 @@ func (o *OCI) SaveIndex() error {
kindJ := descs[j].Annotations["kind"]
// Objects with the prefix of "dev.cosignproject.cosign/image" should be at the top.
if strings.HasPrefix(kindI, consts.KindAnnotation) && !strings.HasPrefix(kindJ, consts.KindAnnotation) {
if strings.HasPrefix(kindI, consts.KindAnnotationImage) && !strings.HasPrefix(kindJ, consts.KindAnnotationImage) {
return true
} else if !strings.HasPrefix(kindI, consts.KindAnnotation) && strings.HasPrefix(kindJ, consts.KindAnnotation) {
} else if !strings.HasPrefix(kindI, consts.KindAnnotationImage) && strings.HasPrefix(kindJ, consts.KindAnnotationImage) {
return false
}
return false // Default: maintain the order.

View File

@@ -15,9 +15,9 @@ import (
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/pkg/artifacts/image"
"hauler.dev/go/hauler/pkg/log"
"hauler.dev/go/hauler/pkg/store"
)
const maxRetries = 3
@@ -92,7 +92,7 @@ func SaveImage(ctx context.Context, s *store.Layout, ref string, platform string
// read command's stderr line by line
errors := bufio.NewScanner(stderr)
for errors.Scan() {
l.Errorf(errors.Text()) // write each line to your log, or anything you need
l.Warnf(errors.Text()) // write each line to your log, or anything you need
}
if err := errors.Err(); err != nil {
cmd.Wait()
@@ -200,7 +200,7 @@ func RetryOperation(ctx context.Context, operation func() error) error {
}
// Log the error for the current attempt.
l.Errorf("error (attempt %d/%d): %v", attempt, maxRetries, err)
l.Warnf("error (attempt %d/%d): %v", attempt, maxRetries, err)
// If this is not the last attempt, wait before retrying.
if attempt < maxRetries {

View File

@@ -7,7 +7,7 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/artifacts"
)
/*

View File

@@ -6,7 +6,7 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
gtypes "github.com/google/go-containerregistry/pkg/v1/types"
"github.com/rancherfederal/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/consts"
)
type Opener func() (io.ReadCloser, error)

View File

@@ -4,7 +4,7 @@ import (
"reflect"
"testing"
"github.com/rancherfederal/hauler/pkg/reference"
"hauler.dev/go/hauler/pkg/reference"
)
func TestParse(t *testing.T) {

View File

@@ -15,10 +15,10 @@ import (
"oras.land/oras-go/pkg/oras"
"oras.land/oras-go/pkg/target"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/layer"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content"
"hauler.dev/go/hauler/pkg/layer"
)
type Layout struct {
@@ -118,7 +118,7 @@ func (l *Layout) AddOCI(ctx context.Context, oci artifacts.OCI, ref string) (oci
Digest: digest.FromBytes(mdata),
Size: int64(len(mdata)),
Annotations: map[string]string{
consts.KindAnnotationName: consts.KindAnnotation,
consts.KindAnnotationName: consts.KindAnnotationImage,
ocispec.AnnotationRefName: ref,
},
URLs: nil,

View File

@@ -8,8 +8,8 @@ import (
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/random"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/store"
"hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/store"
)
var (

1
static/CNAME Normal file
View File

@@ -0,0 +1 @@
hauler.dev

View File

@@ -0,0 +1,9 @@
<html>
<head>
<meta name="go-import" content="hauler.dev/go/hauler git https://github.com/hauler-dev/hauler">
<meta http-equiv="refresh" content="0;URL='https://github.com/hauler-dev/hauler'">
</head>
<body>
Redirecting to the <a href="https://github.com/hauler-dev/hauler">hauler source</a>.
</body>
</html>

9
static/index.html Normal file
View File

@@ -0,0 +1,9 @@
<html>
<head>
<meta name="go-import" content="hauler.dev/go/hauler git https://github.com/hauler-dev/hauler">
<meta http-equiv="refresh" content="0;URL='https://docs.hauler.dev/docs/intro'">
</head>
<body>
Redirecting you to the <a href="https://docs.hauler.dev/docs/intro">hauler docs</a>
</body>
</html>

179
testdata/certificate-script.sh vendored Executable file
View File

@@ -0,0 +1,179 @@
#!/bin/bash
set -e
# setup directories
mkdir -p testdata/certs
cd testdata/certs
echo "<!-----------------------------------!>"
echo "<! Certificate Authority Certificate !>"
echo "<!-----------------------------------!>"
echo "Generating certificate authority private key..."
openssl genrsa -out root-ca.key 4096
echo "Generating certificate authority configuration file..."
cat <<EOF > root-ca.cnf
[ req ]
default_bits = 4096
default_keyfile = root-ca.key
distinguished_name = req_distinguished_name
req_extensions = v3_ca
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = CERTIFICATE AUTHORITY CERTIFICATE
[v3_ca]
keyUsage = critical, keyCertSign, cRLSign
extendedKeyUsage = anyExtendedKeyUsage
basicConstraints = critical, CA:TRUE
EOF
echo "Generating certificate authority certificate signing request..."
openssl req -new -sha256 -key root-ca.key -out root-ca.csr -config root-ca.cnf
echo "Generating certificate authority certificate..."
openssl x509 -req -in root-ca.csr -signkey root-ca.key -days 3650 -out root-ca.crt -extensions v3_ca -extfile root-ca.cnf
echo "Inspecting certificate authority certificate..."
openssl x509 -text -noout -in root-ca.crt > ca.txt
echo "<!------------------------------------------------!>"
echo "<! Intermediary Certificate Authority Certificate !>"
echo "<!------------------------------------------------!>"
echo "Generating intermediary certificate authority private key..."
openssl genrsa -out intermediary-ca.key 4096
echo "Generating intermediary certificate authority configuration file..."
cat <<EOF > intermediary-ca.cnf
[ req ]
default_bits = 4096
default_keyfile = intermediary-ca.key
distinguished_name = req_distinguished_name
req_extensions = v3_ca
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = INTERMEDIARY CERTIFICATE AUTHORITY CERTIFICATE
[v3_ca]
keyUsage = critical, keyCertSign, cRLSign
extendedKeyUsage = anyExtendedKeyUsage
basicConstraints = critical, CA:TRUE
EOF
echo "Generating intermediary certificate authority certificate signing request..."
openssl req -new -sha256 -key intermediary-ca.key -out intermediary-ca.csr -config intermediary-ca.cnf
echo "Generating intermediary certificate authority certificate..."
openssl x509 -req -in intermediary-ca.csr -CA root-ca.crt -CAkey root-ca.key -CAcreateserial -out intermediary-ca.crt -days 3650 -sha256 -extfile intermediary-ca.cnf -extensions v3_ca
echo "Inspecting intermediary certificate authority certificate..."
openssl x509 -text -noout -in intermediary-ca.crt > intermediary-ca.txt
echo "Verifying intermediary certificate authority certificate..."
openssl verify -CAfile root-ca.crt intermediary-ca.crt
echo "Generating full certificate chain..."
cat intermediary-ca.crt root-ca.crt > cacerts.pem
echo "<!-----------------------------------------------------------------!>"
echo "<! Server Certificate Signed by Intermediary Certificate Authority !>"
echo "<!-----------------------------------------------------------------!>"
echo "Generating server private key..."
openssl genrsa -out server-cert.key 4096
echo "Generating server certificate signing config file..."
cat <<EOF > server-cert.cnf
[ req ]
default_bits = 4096
default_keyfile = server-cert.key
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = SERVER CERTIFICATE
[v3_req]
keyUsage = digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = localhost
DNS.2 = registry.localhost
DNS.3 = fileserver.localhost
EOF
echo "Generating server certificate signing request..."
openssl req -new -sha256 -key server-cert.key -out server-cert.csr -config server-cert.cnf
echo "Generating server certificate..."
openssl x509 -req -in server-cert.csr -CA intermediary-ca.crt -CAkey intermediary-ca.key -CAcreateserial -out server-cert.crt -days 3650 -sha256 -extfile server-cert.cnf -extensions v3_req
echo "Inspecting server certificate..."
openssl x509 -text -noout -in server-cert.crt > server-cert.txt
echo "Verifying server certificate..."
openssl verify -CAfile cacerts.pem server-cert.crt
echo "<!-----------------------------------------------------------------!>"
echo "<! Client Certificate Signed by Intermediary Certificate Authority !>"
echo "<!-----------------------------------------------------------------!>"
echo "Generating client private key..."
openssl genrsa -out client-cert.key 4096
echo "Generating client certificate signing config file..."
cat <<EOF > client-cert.cnf
[ req ]
default_bits = 4096
default_keyfile = client-cert.key
distinguished_name = req_distinguished_name
req_extensions = v3_req
prompt = no
[ req_distinguished_name ]
C = US
ST = VIRGINIA
L = RESTON
O = HAULER
OU = HAULER DEV
CN = CLIENT CERTIFICATE
[ v3_req ]
keyUsage = digitalSignature
extendedKeyUsage = clientAuth
EOF
echo "Generating client certificate signing request..."
openssl req -new -sha256 -key client-cert.key -out client-cert.csr -config client-cert.cnf
echo "Generating client certificate..."
openssl x509 -req -in client-cert.csr -CA intermediary-ca.crt -CAkey intermediary-ca.key -CAcreateserial -out client-cert.crt -days 3650 -sha256 -extfile client-cert.cnf -extensions v3_req
echo "Inspecting client certificate..."
openssl x509 -text -noout -in client-cert.crt > client-cert.txt
echo "Verifying client certificate..."
openssl verify -CAfile cacerts.pem client-cert.crt

BIN
testdata/haul.tar.zst vendored Executable file → Normal file

Binary file not shown.

48
testdata/hauler-manifest-pipeline.yaml vendored Executable file
View File

@@ -0,0 +1,48 @@
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Images
metadata:
name: hauler-content-images-example
spec:
images:
- name: busybox
- name: busybox:stable
platform: linux/amd64
- name: gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Charts
metadata:
name: hauler-content-charts-example
spec:
charts:
- name: rancher
repoURL: https://releases.rancher.com/server-charts/stable
- name: rancher
repoURL: https://releases.rancher.com/server-charts/stable
version: 2.8.4
- name: rancher
repoURL: https://releases.rancher.com/server-charts/stable
version: 2.8.3
- name: hauler-helm
repoURL: oci://ghcr.io/hauler-dev
- name: hauler-helm
repoURL: oci://ghcr.io/hauler-dev
version: 1.0.6
- name: hauler-helm
repoURL: oci://ghcr.io/hauler-dev
version: 1.0.4
- name: rancher-cluster-templates-0.5.2.tgz
repoURL: .
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
name: hauler-content-files-example
spec:
files:
- path: https://get.rke2.io/install.sh
- path: https://get.rke2.io/install.sh
name: rke2-install.sh
- path: testdata/hauler-manifest.yaml
- path: testdata/hauler-manifest.yaml
name: hauler-manifest-local.yaml

View File

@@ -4,7 +4,10 @@ metadata:
name: hauler-content-images-example
spec:
images:
- name: busybox:latest
- name: busybox
- name: busybox:stable
platform: linux/amd64
- name: gcr.io/distroless/base@sha256:7fa7445dfbebae4f4b7ab0e6ef99276e96075ae42584af6286ba080750d6dfe5
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Charts
@@ -14,7 +17,9 @@ spec:
charts:
- name: rancher
repoURL: https://releases.rancher.com/server-charts/stable
version: 2.8.2
version: 2.8.5
- name: hauler-helm
repoURL: oci://ghcr.io/hauler-dev
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
@@ -24,3 +29,4 @@ spec:
files:
- path: https://get.rke2.io
name: install.sh
- path: testdata/hauler-manifest.yaml

Binary file not shown.

Binary file not shown.