Compare commits

...

117 Commits

Author SHA1 Message Date
Adam Martin
c592551a37 fix image ref keys getting squashed when containing sigs/atts (#291)
* fix image ref keys getting squashed when containing sigs/atts

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
Co-authored-by: Adam Martin <adam.martin@rancherfederal.com>
2024-08-13 12:18:10 -07:00
Jacob Blain Christen
ef3eb05fce fix missing versin info in release build (#283)
Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-08-05 10:11:40 -07:00
dependabot[bot]
3f64914097 bump github.com/docker/docker in the go_modules group across 1 directory (#281)
Bumps the go_modules group with 1 update in the / directory: [github.com/docker/docker](https://github.com/docker/docker).

Updates `github.com/docker/docker` from 25.0.5+incompatible to 25.0.6+incompatible
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v25.0.5...v25.0.6)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: indirect
  dependency-group: go_modules
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
2024-08-01 17:51:34 -04:00
Zack Brady
6a74668e2c updated install script (install.sh) (#280)
* removed sudo requirement from install script
* updated install script to specify install directory
* cleaned up install script
* a bit more changes and updates to the install script
* updated install script variable syntax
* added missing logic to install script
2024-08-01 17:48:26 -04:00
Kamin Fay
0c5cf20e87 fix digest images being lost on load of hauls (Signed). (#259)
* Adding oci tests
* Fixed the oci pusher to split on the correct '@' symbol
* Reverted Pusher changes and adjusted nameMap references in Index

---------

Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-07-30 23:44:47 -04:00
will
513719bc9e feat: add readonly flag (#277)
* updated flag from writeable to readonly (default=true)

---------

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
Co-authored-by: Zack Brady <zackbrady123@gmail.com>
Co-authored-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-07-30 11:26:39 -07:00
Zack Brady
047b7a7003 fixed makefile for goreleaser v2 changes (#278) 2024-07-30 13:18:17 -04:00
Zack Brady
a4685169c6 updated goreleaser versioning defaults (#279) 2024-07-30 13:17:40 -04:00
Jacob Blain Christen
47549615c4 update feature_request.md (#274)
- `[RFE]` ➡️ `[feature]`

Signed-off-by: Jacob Blain Christen <dweomer5@gmail.com>
2024-07-26 14:48:08 -04:00
Zack Brady
2d725026dc merge pull request #271 from zackbradys/removed-some-references
removed some old references
2024-07-23 13:56:59 -04:00
Zack Brady
60667b7116 updated old references 2024-07-22 18:50:11 -04:00
Zack Brady
7d62a1c98e updated actions workflow user 2024-07-22 18:46:53 -04:00
Zack Brady
894ffb1533 merge pull request #269 from zackbradys/add-dockerhub-support
added dockerhub to github actions workflow
2024-07-20 01:18:08 -04:00
Zack Brady
78b3442d23 added dockerhub to github actions workflow 2024-07-20 01:12:59 -04:00
Zack Brady
cd46febb6b merge pull request #268 from zackbradys/remove-helm-chart
removed helm chart
2024-07-20 01:10:37 -04:00
Zack Brady
0957a930dd removed helm chart 2024-07-20 00:57:25 -04:00
Zack Brady
a6bc6308d9 merge pull request #267 from zackbradys/main
added debug container and workflow
2024-07-19 22:21:38 -04:00
Zack Brady
1304cf6c76 added debug container and workflow 2024-07-17 00:01:24 -04:00
Zack Brady
f2e02c80c0 merge pull request #262 from zackbradys/main
updated products flag description
2024-07-12 22:52:43 -04:00
Zack Brady
25806e993e updated products flag description 2024-07-12 01:25:05 -04:00
Zack Brady
05e67bc750 merge pull request #255 from zackbradys/main
updated chart for release
2024-06-25 23:27:24 -04:00
Zack Brady
b43ed0503a updated chart for release 2024-06-25 23:14:44 -04:00
Zack Brady
27e2fc9de0 merge pull request #254 from zackbradys/main
fixed workflow errors/warnings
2024-06-25 22:48:30 -04:00
Zack Brady
d32d75b93e fixed workflow errors/warnings 2024-06-25 22:43:31 -04:00
Zack Brady
ceb77601d0 merge pull request #252 from zackbradys/main
overhauling github actions and workflows
2024-06-24 22:52:08 -04:00
Zack Brady
d90545a9e4 fixed permissions on testdata 2024-06-17 22:23:03 -04:00
Zack Brady
bef141ab67 updated chart versions (will need to update again) 2024-06-17 19:45:18 -04:00
Zack Brady
385d767c2a last bit of fixes to workflow 2024-06-17 19:42:20 -04:00
Zack Brady
22edc77506 updated unit test workflow 2024-06-14 20:56:37 -04:00
Zack Brady
9058797bbc updated goreleaser deprecations 2024-06-14 20:43:32 -04:00
Zack Brady
35e2f655da added helm chart release job 2024-06-14 20:37:58 -04:00
Zack Brady
f5c0f6f0ae updated github template names 2024-06-14 20:04:01 -04:00
Zack Brady
0ec77b4168 merge pull request #248 from zackbradys/main
formatted all code with `go fmt`
2024-06-14 16:46:21 -04:00
Zack Brady
7a7906b8ea updated imports (and go fmt) 2024-06-13 23:44:06 -04:00
Zack Brady
f4774445f6 merge pull request #240 from pat-earl/doc_updates
added some documentation text to sync command
2024-06-10 18:13:09 -04:00
Zack Brady
d59b29bfce formatted gitignore to match dockerignore 2024-06-05 14:40:50 -04:00
Zack Brady
fd702202ac formatted all code (go fmt) 2024-06-05 08:25:45 -04:00
Adam Martin
9e9565717b Merge pull request #245 from ethanchowell/ehowell/helm-client
Configure chart commands to use helm clients for OCI and private regi…
2024-06-04 13:16:42 -04:00
Zack Brady
bfe47ae141 updated chart tests for new features 2024-06-03 23:31:50 -04:00
Zack Brady
ebab7f38a0 merge pull request #247 from kaminfay/dev/add-fileserver-timeout-flag
adding the timeout flag for fileserver command and setting default timeout to 60 seconds
2024-06-03 22:33:36 -04:00
Kamin Fay
f0cba3c2c6 Adding the timeout flag for fileserver command 2024-05-28 15:28:20 -04:00
Ethan Howell
286120da50 Configure chart commands to use helm clients for OCI and private registry support 2024-05-24 12:06:16 -04:00
Patrick Earl
dcdeb93518 Added some documentation text to sync command 2024-05-02 14:17:38 -04:00
Adam Martin
f7c24f6129 Merge pull request #235 from rancherfederal/dependabot/go_modules/golang.org/x/net-0.23.0
Bump golang.org/x/net from 0.17.0 to 0.23.0
2024-04-23 16:40:45 -04:00
Adam Martin
fe88d7033c Merge pull request #234 from amartin120/dup-digest-bugfix
fix for dup digest smashing in cosign
2024-04-23 15:49:54 -04:00
dependabot[bot]
ef31984c97 Bump golang.org/x/net from 0.17.0 to 0.23.0
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.17.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.17.0...v0.23.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-23 19:46:05 +00:00
Adam Martin
2889f30275 fix for dup digest smashing in cosign
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-04-23 14:06:40 -04:00
Zack Brady
0674e0ab30 merge pull request #229 from zackbradys/vagrant
removed vagrant scripts
2024-04-15 09:36:24 -04:00
Zack Brady
d645c52135 removed vagrant scripts 2024-04-14 08:45:51 -04:00
Zack Brady
44baab3213 merge pull request #227 from zackbradys/helmifying
helmifying hauler
2024-04-11 22:02:37 -04:00
Zack Brady
1a317b0172 merge pull request #226 from zackbradys/testdata
updated hauler testdata
2024-04-11 21:57:37 -04:00
Zack Brady
128cb3b252 last bit of updates and formatting of chart 2024-04-07 00:23:49 -04:00
Zack Brady
91ff998634 updated hauler testdata 2024-04-06 15:19:53 -04:00
Zack Brady
8ac1ecaf29 adding functionality and cleaning up 2024-04-06 02:06:33 -04:00
Zack Brady
7447aad20a added initial helm chart 2024-04-06 00:28:59 -04:00
Zack Brady
003456d8ab merge pull request #225 from zackbradys/main
removed tag in release workflow
2024-04-05 21:36:26 -04:00
Zack Brady
f44b8b93af removed tag in release workflow 2024-04-05 21:32:58 -04:00
Zack Brady
e405840642 merge pull request #224 from zackbradys/main
fixed image ref in release workflow
2024-04-05 20:45:53 -04:00
Zack Brady
8c9aa909b0 updated/fixed image ref in release workflow 2024-04-05 20:43:36 -04:00
Zack Brady
8670489520 merge pull request #223 from zackbradys/main
fixed platforms in release workflow
2024-04-05 20:04:57 -04:00
Zack Hodgson Brady
f20d4052a4 updated/fixed platforms in release workflow 2024-04-05 20:02:00 -04:00
Zack Brady
c84bca43d2 updated/cleaned github actions (#222)
* cleaned up unit test workflow
* updated/cleaned up release workflow
* fixed workflow typo
2024-04-05 16:12:49 -07:00
Claus Albøge
6863d91f69 Make Product Registry configurable (#194)
Co-authored-by: Claus Albøge <ca@netic.dk>
2024-04-05 14:40:26 -07:00
Zack Brady
16eea6ac2a updated fileserver directory name (#219) 2024-04-05 14:36:39 -07:00
Adam Martin
f6f227567c Merge pull request #221 from amartin120/fix-file-logging
fix logging for files
2024-04-01 12:45:58 -04:00
Adam Martin
eb810c16f5 fix logging for files
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-04-01 12:23:05 -04:00
Adam Martin
b18f55ea60 Merge pull request #220 from amartin120/temp-override
temp dir override for `hauler store load`
2024-04-01 10:40:31 -04:00
Adam Martin
4bbe622073 add extra info for the tempdir override flag
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-04-01 09:39:34 -04:00
Adam Martin
ea5bcb36ae tempdir override flag for load
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-04-01 09:34:26 -04:00
Adam Martin
5c7daddfef Merge pull request #218 from amartin120/remove-cache-flag
deprecate misleading cache flag from store
2024-03-29 13:48:51 -04:00
Adam Martin
7083f3a4f3 deprecate the cache flag instead of remove
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-03-29 13:40:24 -04:00
Adam Martin
8541d73a0d Merge pull request #217 from amartin120/logging-updates
better logging when adding to store
2024-03-29 11:36:45 -04:00
Adam Martin
49d705d14c Merge pull request #216 from amartin120/cosign-update
update to v2.2.3 of RGS cosign fork
2024-03-29 11:36:29 -04:00
Clayton Castro
722851d809 Merge pull request #215 from clanktron/container
add container definition
2024-03-28 16:20:35 -07:00
clayton
82aedc867a switch to using bci-golang as builder image 2024-03-28 13:53:22 -07:00
clayton
e8fb37c6ed fix: ensure /tmp for hauler store load 2024-03-28 13:49:01 -07:00
Adam Martin
545b3f8acd added the copy back for now
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-03-28 16:16:05 -04:00
Adam Martin
3ae92fe20a remove copy at the image sync not needed with cosign update
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-03-28 15:00:00 -04:00
Adam Martin
35538bf45a removed misleading cache flag
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-03-28 14:24:37 -04:00
Adam Martin
b6701bbfbc better logging when adding to store
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-03-28 14:16:22 -04:00
Adam Martin
14738c3cd6 update to v2.2.3 of our cosign fork
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-03-28 13:11:26 -04:00
clayton
0657fd80fe add: dockerignore 2024-03-27 02:25:12 -07:00
clayton
d132e8b8e0 add: Dockerfile 2024-03-27 02:25:06 -07:00
Adam Martin
29367c152e Merge pull request #213 from rancherfederal/dependabot/go_modules/google.golang.org/protobuf-1.33.0
Bump google.golang.org/protobuf from 1.31.0 to 1.33.0
2024-03-26 14:58:08 -04:00
Adam Martin
185ae6bd74 Merge pull request #212 from rancherfederal/dependabot/go_modules/github.com/docker/docker-25.0.5incompatible
Bump github.com/docker/docker from 25.0.1+incompatible to 25.0.5+incompatible
2024-03-26 14:57:57 -04:00
dependabot[bot]
b6c78d3925 Bump google.golang.org/protobuf from 1.31.0 to 1.33.0
Bumps google.golang.org/protobuf from 1.31.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-26 13:04:41 +00:00
dependabot[bot]
e718d40744 Bump github.com/docker/docker
Bumps [github.com/docker/docker](https://github.com/docker/docker) from 25.0.1+incompatible to 25.0.5+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](https://github.com/docker/docker/compare/v25.0.1...v25.0.5)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-26 13:03:05 +00:00
Zack Brady
1505bfb3af merge pull request #207 from zackbradys/main
updated and added new logos
2024-03-20 08:17:00 -04:00
Zack Hodgson Brady
e27b5b3cd1 updated and added new logos 2024-03-19 18:22:21 -04:00
Zack Brady
0472c8fc65 merge pull request #203 from zackbradys/main
updated github files
2024-03-11 09:28:08 -04:00
Zack Hodgson Brady
70a48f2efe updated github files 2024-03-10 16:45:41 -04:00
Adam Martin
bb2a8bfbec Merge pull request #197 from fgiudici/file_name_option
Fix --name option in "store add file" command
2024-02-27 08:33:09 -05:00
Adam Martin
2779c649c2 Merge pull request #195 from rancherfederal/dependabot/go_modules/helm.sh/helm/v3-3.14.2
Bump helm.sh/helm/v3 from 3.14.1 to 3.14.2
2024-02-27 08:00:57 -05:00
Francesco Giudici
8120537af2 Fix --name option in "store add file" command
Fixes: #196

Signed-off-by: Francesco Giudici <francesco.giudici@suse.com>
2024-02-27 09:54:48 +01:00
dependabot[bot]
9cdab516f0 Bump helm.sh/helm/v3 from 3.14.1 to 3.14.2
Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.1 to 3.14.2.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.14.1...v3.14.2)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-27 00:21:10 +00:00
Adam Martin
d136d1bfd2 Merge pull request #191 from atanasdinov/cosign-error-code
Exit with status code 1 if cosign is not configured
2024-02-23 08:49:25 -05:00
Atanas Dinov
003560c3b3 Exit with status code 1 if cosign is not configured 2024-02-23 10:39:47 +02:00
Adam Martin
1b9d057f7a Merge pull request #188 from amartin120/registry-flag
add registry flag to sync cli to match annotation functionality
2024-02-22 08:57:44 -05:00
Adam Martin
2764e2d3ea Merge pull request #187 from amartin120/fix-exitcode
fix exit code on error
2024-02-22 08:57:31 -05:00
Zack Brady
360049fe19 reverting changes for logos (#189) 2024-02-21 14:35:07 -07:00
Brandon
79b240d17f Merge pull request #186 from rancherfederal/bdev
adding graphics to README.md
2024-02-20 06:50:36 -05:00
bgulla
214704bcfb adding graphics 2024-02-20 06:46:51 -05:00
Adam Martin
ef73fff01a fix exit code on error
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-02-19 16:45:04 -05:00
Adam Martin
0c6fdc86da add registry flag to cli for sync 2024-02-19 10:10:47 -05:00
Zack Brady
7fb537a31a merge pull request #184 from zackbradys/main
updated `readme` and `install.sh` for hauler `v1.0.0`
2024-02-17 22:12:44 -05:00
Zack Brady
6ca7fb6255 updated readme and removed roadmap 2024-02-17 15:00:59 -05:00
Zack Brady
d70a867283 updated/cleaned up install.sh 2024-02-17 10:53:10 -05:00
Adam Martin
46ea8b5df9 Merge pull request #180 from amartin120/remove-deprecated-cmds
remove deprecated commands
2024-02-17 09:09:05 -05:00
Adam Martin
5592ec0f88 remove deprecated commands
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-02-16 09:10:29 -05:00
Adam Martin
e8254371c0 Merge pull request #177 from amartin120/add-login
add login command
2024-02-16 08:16:58 -05:00
Adam Martin
8d2a84d27c Merge pull request #179 from rancherfederal/dependabot/go_modules/helm.sh/helm/v3-3.14.1
Bump helm.sh/helm/v3 from 3.14.0 to 3.14.1
2024-02-16 08:14:00 -05:00
Adam Martin
72734ecc76 Merge pull request #178 from amartin120/bug-file-name
bug-fix: handle complex file names
2024-02-16 08:13:28 -05:00
dependabot[bot]
4759879a5d Bump helm.sh/helm/v3 from 3.14.0 to 3.14.1
Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.0 to 3.14.1.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](https://github.com/helm/helm/compare/v3.14.0...v3.14.1)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-02-16 04:48:57 +00:00
Adam Martin
dbcfe13fb6 bug-fix: handle complex file names
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-02-15 19:55:38 -05:00
Adam Martin
cd8d4f6e46 add login command
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-02-15 15:54:23 -05:00
Adam Martin
e15c8d54fa Merge pull request #176 from amartin120/info-totals
update to add size totals and cosign artifacts to the `info` command
2024-02-15 12:39:17 -05:00
Adam Martin
ccd529ab48 update to add size totals and cosign bits to the info command
Signed-off-by: Adam Martin <adam.martin@rancherfederal.com>
2024-02-14 13:03:36 -05:00
73 changed files with 1130 additions and 1335 deletions

8
.dockerignore Normal file
View File

@@ -0,0 +1,8 @@
*
!cmd
!go.mod
!go.sum
!internal
!Makefile
!pkg
!static

View File

@@ -1,33 +1,51 @@
---
name: Bug Report
about: Create a report to help us improve!
about: Submit a bug report to help us improve!
title: '[BUG]'
labels: 'kind/bug'
labels: 'bug'
assignees: ''
---
<!-- Thank you for helping us to improve Hauler! We welcome all bug reports. Please fill out each area of the template so we can better help you. Comments like this will be hidden when you post but you can delete them if you wish. -->
<!-- Thank you for helping us to improve Hauler! We welcome all bug reports. Please fill out each area of the template so we can better assist you. Comments like this will be hidden when you submit, but you can delete them if you wish. -->
**Environmental Info:**
*
<!-- Provide the output of "uname -a" -->
-
**Hauler Version:**
*
**System CPU architecture, OS, and Version:**
* <!-- Provide the output from "uname -a" on the system where Hauler is installed -->
<!-- Provide the output of "hauler version" -->
**Describe the bug:**
* <!-- A clear and concise description of the bug. -->
-
**Steps To Reproduce:**
* <!-- A clear and concise way to reproduce the bug. -->
**Describe the Bug:**
**Expected behavior:**
* <!-- A clear and concise description of what you expected to happen, without the bug. -->
<!-- Provide a clear and concise description of the bug -->
**Actual behavior:**
* <!-- A clear and concise description of what actually happened. -->
-
**Additional context / logs:**
* <!-- Add any other context and/or logs about the problem here. -->
**Steps to Reproduce:**
<!-- Provide a clear and concise way to reproduce the bug -->
-
**Expected Behavior:**
<!-- Provide a clear and concise description of what you expected to happen -->
-
**Actual Behavior:**
<!-- Provide a clear and concise description of what actually happens -->
-
**Additional Context:**
<!-- Provide any other context and/or logs about the bug -->
-

View File

@@ -1,21 +1,33 @@
---
name: Feature Request
about: Create a report to help us improve!
title: '[RFE]'
labels: 'kind/rfe'
about: Submit a feature request for us to improve!
title: '[feature]'
labels: 'enhancement'
assignees: ''
---
<!-- Thanks for helping us to improve Hauler! We welcome all requests for enhancements (RFEs). Please fill out each area of the template so we can better help you. Comments like this will be hidden when you post but you can delete them if you wish. -->
<!-- Thank you for helping us to improve Hauler! We welcome all requests for enhancements (RFEs). Please fill out each area of the template so we can better assist you. Comments like this will be hidden when you submit, but you can delete them if you wish. -->
**Is your feature request related to a problem? Please describe.**
* <!-- A clear and concise description of the problem. -->
**Is this RFE related to an Existing Problem? If so, please describe:**
**Describe the solution you'd like**
* <!-- A clear and concise description of what you want to happen. -->
<!-- Provide a clear and concise description of the problem -->
**Describe alternatives you've considered**
* <!-- A clear and concise description of any alternative solutions or features you've considered. -->
-
**Additional context**
* <!-- Add any other context or screenshots about the feature request here. -->
**Describe Proposed Solution(s):**
<!-- Provide a clear and concise description of what you want to happen -->
-
**Describe Possible Alternatives:**
<!-- Provide a clear and concise description of any alternative solutions or features you've considered -->
-
**Additional Context:**
<!-- Provide a clear and concise description of the problem -->
-

View File

@@ -1,20 +1,36 @@
**Please check below, if the PR fulfills these requirements:**
- [ ] The commit message follows the guidelines.
- [ ] Tests for the changes have been added (for bug fixes / features).
- [ ] Docs have been added / updated (for bug fixes / features).
- [ ] Commit(s) and code follow the repositories guidelines.
- [ ] Test(s) have been added or updated to support these change(s).
- [ ] Doc(s) have been added or updated to support these change(s).
<!-- Comments like this will be hidden when you submit, but you can delete them if you wish. -->
**What kind of change does this PR introduce?**
* <!-- Bug fix, feature, docs update, ... -->
**Associated Links:**
**What is the current behavior?**
* <!-- You can also link to an open issue here -->
<!-- Provide any associated or linked related to these change(s) -->
**What is the new behavior (if this is a feature change)?**
* <!-- What changes did this PR introduce? -->
-
**Does this PR introduce a breaking change?**
* <!-- What changes might users need to make in their application due to this PR? -->
**Types of Changes:**
**Other information**:
* <!-- Any additional information -->
<!-- What is the type of change? Bugfix, Feature, Breaking Change, etc... -->
-
**Proposed Changes:**
<!-- Provide the high level and low level description of your change(s) so we can better understand these change(s) -->
-
**Verification/Testing of Changes:**
<!-- How can the changes be verified? Provide the steps necessary to reproduce and verify the proposed change(s) -->
-
**Additional Context:**
<!-- Provide any additional information, such as if this is a small or large or complex change. Feel free to kick off the discussion by explaining why you chose the solution you did and what alternatives you considered, etc... -->
-

View File

@@ -1,30 +1,93 @@
name: CI
name: Release Workflow
on:
workflow_dispatch:
push:
tags:
tags:
- '*'
jobs:
goreleaser:
go-release:
name: Go Release Job
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Go
uses: actions/setup-go@v2
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Run GoReleaser
uses: goreleaser/goreleaser-action@v2
- name: Run Go Releaser
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser
version: latest
args: release --rm-dist -p 1
version: "~> v2"
args: "release --clean -p 1"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
HOMEBREW_TAP_GITHUB_TOKEN: ${{ secrets.HOMEBREW_TAP_GITHUB_TOKEN }}
container-release:
name: Container Release Job
runs-on: ubuntu-latest
needs: [go-release]
timeout-minutes: 30
continue-on-error: true
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Set Up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Authenticate to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Authenticate to DockerHub Container Registry
uses: docker/login-action@v3
with:
registry: docker.io
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and Push Release Container to GitHub Container Registry
uses: docker/build-push-action@v5
with:
context: .
target: release
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.repository }}:${{ github.ref_name }}, docker.io/hauler/hauler:${{ github.ref_name }}
- name: Build and Push Debug Container to GitHub Container Registry
uses: docker/build-push-action@v5
with:
context: .
target: debug
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/${{ github.repository }}-debug:${{ github.ref_name }}, docker.io/hauler/hauler-debug:${{ github.ref_name }}

View File

@@ -1,41 +1,43 @@
name: Unit Test
name: Unit Test Workflow
on:
workflow_dispatch:
push:
paths-ignore:
- "**.md"
- ".github/**"
- "!.github/workflows/unittest.yaml"
branches:
- main
pull_request:
paths-ignore:
- "**.md"
- ".github/**"
- "!.github/workflows/unitcoverage.yaml"
workflow_dispatch: {}
branches:
- main
jobs:
test:
unit-test:
name: Unit Tests
runs-on: ubuntu-latest
timeout-minutes: 20
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Go
uses: actions/setup-go@v2
- name: Configure Git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Set Up Go
uses: actions/setup-go@v5
with:
go-version: 1.21.x
- name: Run Unit Tests
run: |
mkdir -p cmd/hauler/binaries
touch cmd/hauler/binaries/dummy.txt
go test -race -covermode=atomic -coverprofile=coverage.out ./pkg/... ./internal/... ./cmd/...
- name: On Failure, Launch Debug Session
if: ${{ failure() }}
uses: mxschmitt/action-tmate@v3
timeout-minutes: 5
- name: Upload Results To Codecov
uses: codecov/codecov-action@v1
go test -race -covermode=atomic -coverprofile=coverage.out ./...
- name: Upload Coverage Report
uses: actions/upload-artifact@v4
with:
files: ./coverage.out
verbose: true # optional (default = false)
name: coverage-report
path: coverage.out

16
.gitignore vendored
View File

@@ -1,9 +1,5 @@
.DS_Store
# Vagrant
.vagrant
# Editor directories and files
**/.DS_Store
.idea
.vscode
*.suo
@@ -12,20 +8,12 @@
*.sln
*.sw?
*.dir-locals.el
# old, ad-hoc ignores
artifacts
local-artifacts
airgap-scp.sh
# test artifacts
*.tar*
*.out
# generated
dist/
tmp/
bin/
/store/
/registry/
cmd/hauler/binaries
cmd/hauler/binaries

View File

@@ -1,3 +1,5 @@
version: 2
project_name: hauler
before:
hooks:
@@ -7,10 +9,11 @@ before:
release:
prerelease: auto
make_latest: false
env:
- vpkg=github.com/rancherfederal/hauler/internal/version
- cosign_version=v2.2.2+carbide.2
- cosign_version=v2.2.3+carbide.2
builds:
- main: cmd/hauler/main.go
@@ -26,7 +29,7 @@ builds:
hooks:
pre:
- mkdir -p cmd/hauler/binaries
- wget -P cmd/hauler/binaries/ https://github.com/rancher-government-carbide/cosign/releases/download/{{ .Env.cosign_version }}/cosign-{{ .Os }}-{{ .Arch }}{{ if eq .Os "windows" }}.exe{{ end }}
- wget -P cmd/hauler/binaries/ https://github.com/hauler-dev/cosign/releases/download/{{ .Env.cosign_version }}/cosign-{{ .Os }}-{{ .Arch }}{{ if eq .Os "windows" }}.exe{{ end }}
post:
- rm -rf cmd/hauler/binaries
env:
@@ -36,14 +39,14 @@ universal_binaries:
- replace: false
changelog:
skip: false
disable: false
use: git
brews:
- name: hauler
tap:
owner: rancherfederal
repository:
owner: hauler-dev
name: homebrew-tap
token: "{{ .Env.HOMEBREW_TAP_GITHUB_TOKEN }}"
folder: Formula
directory: Formula
description: "Hauler CLI"

45
Dockerfile Normal file
View File

@@ -0,0 +1,45 @@
# builder stage
FROM registry.suse.com/bci/golang:1.21 AS builder
RUN zypper --non-interactive install make bash wget ca-certificates
COPY . /build
WORKDIR /build
RUN make build
RUN echo "hauler:x:1001:1001::/home/hauler:" > /etc/passwd \
&& echo "hauler:x:1001:hauler" > /etc/group \
&& mkdir /home/hauler \
&& mkdir /store \
&& mkdir /fileserver \
&& mkdir /registry
# release stage
FROM scratch AS release
COPY --from=builder /var/lib/ca-certificates/ca-bundle.pem /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group
COPY --from=builder --chown=hauler:hauler /home/hauler/. /home/hauler
COPY --from=builder --chown=hauler:hauler /tmp/. /tmp
COPY --from=builder --chown=hauler:hauler /store/. /store
COPY --from=builder --chown=hauler:hauler /registry/. /registry
COPY --from=builder --chown=hauler:hauler /fileserver/. /fileserver
COPY --from=builder --chown=hauler:hauler /build/bin/hauler /
USER hauler
ENTRYPOINT [ "/hauler" ]
# debug stage
FROM alpine AS debug
COPY --from=builder /var/lib/ca-certificates/ca-bundle.pem /etc/ssl/certs/ca-certificates.crt
COPY --from=builder /etc/passwd /etc/passwd
COPY --from=builder /etc/group /etc/group
COPY --from=builder --chown=hauler:hauler /home/hauler/. /home/hauler
COPY --from=builder --chown=hauler:hauler /build/bin/hauler /bin/hauler
RUN apk --no-cache add curl
USER hauler
WORKDIR /home/hauler

View File

@@ -1,7 +1,7 @@
SHELL:=/bin/bash
GO_FILES=$(shell go list ./... | grep -v /vendor/)
COSIGN_VERSION=v2.2.2+carbide.2
COSIGN_VERSION=v2.2.3+carbide.2
.SILENT:
@@ -10,17 +10,17 @@ all: fmt vet install test
build:
rm -rf cmd/hauler/binaries;\
mkdir -p cmd/hauler/binaries;\
wget -P cmd/hauler/binaries/ https://github.com/rancher-government-carbide/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
wget -P cmd/hauler/binaries/ https://github.com/hauler-dev/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
mkdir bin;\
CGO_ENABLED=0 go build -o bin ./cmd/...;\
build-all: fmt vet
goreleaser build --rm-dist --snapshot
goreleaser build --clean --snapshot
install:
rm -rf cmd/hauler/binaries;\
mkdir -p cmd/hauler/binaries;\
wget -P cmd/hauler/binaries/ https://github.com/rancher-government-carbide/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
wget -P cmd/hauler/binaries/ https://github.com/hauler-dev/cosign/releases/download/$(COSIGN_VERSION)/cosign-$(shell go env GOOS)-$(shell go env GOARCH);\
CGO_ENABLED=0 go install ./cmd/...;\
vet:

View File

@@ -1,31 +1,36 @@
# Rancher Government Hauler
![rancher-government-hauler-logo](/static/rgs-hauler-logo.png)
## Airgap Swiss Army Knife
> ⚠️ This project is still in active development and *not* Generally Available (GA). Most of the core functionality and features are ready, but may have breaking changes. Please review the [Release Notes](https://github.com/rancherfederal/hauler/releases) for more information!
> ⚠️ **Please Note:** Hauler and the Hauler Documentation are recently Generally Available (GA).
`Rancher Government Hauler` simplifies the airgap experience without requiring users to adopt a specific workflow. **Hauler** simplifies the airgapping process, by representing assets (images, charts, files, etc...) as content and collections to allow users to easily fetch, store, package, and distribute these assets with declarative manifests or through the command line.
`Rancher Government Hauler` simplifies the airgap experience without requiring operators to adopt a specific workflow. **Hauler** simplifies the airgapping process, by representing assets (images, charts, files, etc...) as content and collections to allow operators to easily fetch, store, package, and distribute these assets with declarative manifests or through the command line.
`Hauler` does this by storing contents and collections as OCI Artifacts and allows users to serve contents and collections with an embedded registry and fileserver. Additionally, `Hauler` has the ability to store and inspect various non-image OCI Artifacts.
`Hauler` does this by storing contents and collections as OCI Artifacts and allows operators to serve contents and collections with an embedded registry and fileserver. Additionally, `Hauler` has the ability to store and inspect various non-image OCI Artifacts.
For more information, please review the **[Hauler Documentation](https://rancherfederal.github.io/hauler-docs)!**
For more information, please review the **[Hauler Documentation](https://hauler.dev)!**
## Installation
### Linux/Darwin
```bash
# installs latest release
curl -sfL https://get.hauler.dev | bash
```
### Homebrew
```bash
# installs latest release
brew tap rancherfederal/homebrew-tap
brew tap hauler-dev/homebrew-tap
brew install hauler
```
### Windows
```bash
# coming soon
```
@@ -33,11 +38,7 @@ brew install hauler
## Acknowledgements
`Hauler` wouldn't be possible without the open-source community, but there are a few projects that stand out:
* [go-containerregistry](https://github.com/google/go-containerregistry)
* [oras cli](https://github.com/oras-project/oras)
* [cosign](https://github.com/sigstore/cosign)
## Notices
**WARNING - Upcoming Deprecated Command(s):**
`hauler download` (alternatively, `dl`) and `hauler serve` (_not_ `hauler store serve`) commands are deprecated and will be removed in a future release.
- [oras cli](https://github.com/oras-project/oras)
- [cosign](https://github.com/sigstore/cosign)
- [go-containerregistry](https://github.com/google/go-containerregistry)

View File

@@ -1,50 +0,0 @@
# Hauler Roadmap
## \> v0.2.0
- Leverage `referrers` api to robustly link content/collection
- Support signing for all `artifact.OCI` contents
- Support encryption for `artifact.OCI` layers
- Support incremental updates to stores (some implementation of layer diffing)
- Safely embed container runtime for user created `collections` creation and transformation
- Better defaults/configuration/security around for long-lived embedded registry
- Better support multi-platform content
- Better leverage `oras` (`>=0.5.0`) for content relocation
- Store git repos as CAS in OCI format
## v0.2.0 - MVP 2
- Re-focus on cli and framework for oci content fetching and delivery
- Focus on initial key contents
- Files (local/remote)
- Charts (local/remote)
- Images
- Establish framework for `content` and `collections`
- Define initial `content` types (`file`, `chart`, `image`)
- Define initial `collection` types (`thickchart`, `k3s`)
- Define framework for manipulating OCI content (`artifact.OCI`, `artifact.Collection`)
## v0.1.0 - MVP 1
- Install single-node k3s cluster
- Support tarball and rpm installation methods
- Target narrow set of known Operating Systems to have OS-specific code if needed
- Serve container images
- Collect images from image list file
- Collect images from image archives
- Deploy docker registry
- Populate registry with all images
- Serve git repositories
- Collect repos
- Deploy git server (Caddy? NGINX?)
- Populate git server with repos
- Serve files
- Collect files from directory, including subdirectories
- Deploy caddy file server
- Populate file server with directory contents
- NOTE: "generic" option - most other use cases can be satisfied by a specially crafted file
server directory
## v0.0.x
- Install single-node k3s cluster into an Ubuntu machine using the tarball installation method

View File

@@ -1,49 +0,0 @@
## Hauler Vagrant machine
A Vagrantfile is provided to allow easy provisioning of a local air-gapped CentOS environment. Some artifacts need to be collected from the internet; below are the steps required for successfully provisioning this machine, downloading all dependencies, and installing k3s (without hauler) into this machine.
### First-time setup
1. Install vagrant, if needed: <https://www.vagrantup.com/downloads>
2. Install `vagrant-vbguest` plugin, as noted in the Vagrantfile:
```shell
vagrant plugin install vagrant-vbguest
```
3. Deploy Vagrant machine, disabling SELinux:
```shell
SELINUX=Disabled vagrant up
```
4. Access the Vagrant machine via SSH:
```shell
vagrant ssh
```
5. Run all prep scripts inside of the Vagrant machine:
> This script temporarily enables internet access from within the VM to allow downloading all dependencies. Even so, the air-gapped network configuration IS restored before completion.
```shell
sudo /opt/hauler/vagrant-scripts/prep-all.sh
```
All dependencies for all `vagrant-scripts/*-install.sh` scripts are now downloaded to the local
repository under `local-artifacts`.
### Installing k3s manually
1. Access the Vagrant machine via SSH:
```bash
vagrant ssh
```
2. Run the k3s install script inside of the Vagrant machine:
```shell
sudo /opt/hauler/vagrant-scripts/k3s-install.sh
```
### Installing RKE2 manually
1. Access the Vagrant machine via SSH:
```shell
vagrant ssh
```
2. Run the RKE2 install script inside of the Vagrant machine:
```shell
sudo /opt/hauler/vagrant-scripts/rke2-install.sh
```

65
Vagrantfile vendored
View File

@@ -1,65 +0,0 @@
##################################
# The vagrant-vbguest plugin is required for CentOS 7.
# Run the following command to install/update this plugin:
# vagrant plugin install vagrant-vbguest
##################################
Vagrant.configure("2") do |config|
config.vm.box = "centos/8"
config.vm.hostname = "airgap"
config.vm.network "private_network", type: "dhcp"
config.vm.synced_folder ".", "/vagrant"
config.vm.provider "virtualbox" do |vb|
vb.memory = "2048"
vb.cpus = "2"
config.vm.provision "airgap", type: "shell", run: "always",
inline: "/vagrant/vagrant-scripts/airgap.sh airgap"
end
# SELinux is Enforcing by default.
# To set SELinux as Disabled on a VM that has already been provisioned:
# SELINUX=Disabled vagrant up --provision-with=selinux
# To set SELinux as Permissive on a VM that has already been provsioned
# SELINUX=Permissive vagrant up --provision-with=selinux
config.vm.provision "selinux", type: "shell", run: "once" do |sh|
sh.upload_path = "/tmp/vagrant-selinux"
sh.env = {
'SELINUX': ENV['SELINUX'] || "Enforcing"
}
sh.inline = <<~SHELL
#!/usr/bin/env bash
set -eux -o pipefail
if ! type -p getenforce setenforce &>/dev/null; then
echo SELinux is Disabled
exit 0
fi
case "${SELINUX}" in
Disabled)
if mountpoint -q /sys/fs/selinux; then
setenforce 0
umount -v /sys/fs/selinux
fi
;;
Enforcing)
mountpoint -q /sys/fs/selinux || mount -o rw,relatime -t selinuxfs selinuxfs /sys/fs/selinux
setenforce 1
;;
Permissive)
mountpoint -q /sys/fs/selinux || mount -o rw,relatime -t selinuxfs selinuxfs /sys/fs/selinux
setenforce 0
;;
*)
echo "SELinux mode not supported: ${SELINUX}" >&2
exit 1
;;
esac
echo SELinux is $(getenforce)
SHELL
end
end

View File

@@ -31,9 +31,8 @@ func New() *cobra.Command {
pf.StringVarP(&ro.logLevel, "log-level", "l", "info", "")
// Add subcommands
addDownload(cmd)
addLogin(cmd)
addStore(cmd)
addServe(cmd)
addVersion(cmd)
addCompletion(cmd)

View File

@@ -3,23 +3,24 @@ package cli
import (
"fmt"
"os"
"github.com/spf13/cobra"
)
func addCompletion(parent *cobra.Command) {
cmd := &cobra.Command{
Use: "completion",
Short: "Generates completion scripts for various shells",
Use: "completion",
Short: "Generates completion scripts for various shells",
Long: `The completion sub-command generates completion scripts for various shells.`,
}
cmd.AddCommand(
addCompletionZsh(),
addCompletionBash(),
addCompletionFish(),
addCompletionPowershell(),
)
parent.AddCommand(cmd)
}
@@ -34,19 +35,19 @@ func addCompletionZsh() *cobra.Command {
Short: "Generates zsh completion scripts",
Long: `The completion sub-command generates completion scripts for zsh.`,
Example: `To load completion run
. <(hauler completion zsh)
To configure your zsh shell to load completions for each session add to your zshrc
# ~/.zshrc or ~/.profile
command -v hauler >/dev/null && . <(hauler completion zsh)
or write a cached file in one of the completion directories in your ${fpath}:
echo "${fpath// /\n}" | grep -i completion
hauler completion zsh > _hauler
mv _hauler ~/.oh-my-zsh/completions # oh-my-zsh
mv _hauler ~/.zprezto/modules/completion/external/src/ # zprezto`,
Run: func(cmd *cobra.Command, args []string) {
@@ -64,11 +65,11 @@ func addCompletionBash() *cobra.Command {
Short: "Generates bash completion scripts",
Long: `The completion sub-command generates completion scripts for bash.`,
Example: `To load completion run
. <(hauler completion bash)
To configure your bash shell to load completions for each session add to your bashrc
# ~/.bashrc or ~/.profile
command -v hauler >/dev/null && . <(hauler completion bash)`,
Run: func(cmd *cobra.Command, args []string) {
@@ -84,9 +85,9 @@ func addCompletionFish() *cobra.Command {
Short: "Generates fish completion scripts",
Long: `The completion sub-command generates completion scripts for fish.`,
Example: `To configure your fish shell to load completions for each session write this script to your completions dir:
hauler completion fish > ~/.config/fish/completions/hauler.fish
See http://fishshell.com/docs/current/index.html#completion-own for more details`,
Run: func(cmd *cobra.Command, args []string) {
cmd.GenFishCompletion(os.Stdout, true)
@@ -101,18 +102,18 @@ func addCompletionPowershell() *cobra.Command {
Short: "Generates powershell completion scripts",
Long: `The completion sub-command generates completion scripts for powershell.`,
Example: `To load completion run
. <(hauler completion powershell)
To configure your powershell shell to load completions for each session add to your powershell profile
Windows:
cd "$env:USERPROFILE\Documents\WindowsPowerShell\Modules"
hauler completion powershell >> hauler-completion.ps1
Linux:
cd "${XDG_CONFIG_HOME:-"$HOME/.config/"}/powershell/modules"
hauler completion powershell >> hauler-completions.ps1`,
Run: func(cmd *cobra.Command, args []string) {
@@ -120,4 +121,4 @@ func addCompletionPowershell() *cobra.Command {
},
}
return cmd
}
}

View File

@@ -1,45 +0,0 @@
package cli
import (
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/cmd/hauler/cli/download"
)
func addDownload(parent *cobra.Command) {
o := &download.Opts{}
cmd := &cobra.Command{
Use: "download",
Short: "Download OCI content from a registry and populate it on disk",
Long: `*** WARNING: Deprecated Command ***
The 'download (dl)' command is deprecated and will be removed in a future release of Hauler.
Locate OCI content based on it's reference in a compatible registry and download the contents to disk.
Note that the content type determines it's format on disk. Hauler's built in content types act as follows:
- File: as a file named after the pushed contents source name (ex: my-file.yaml:latest --> my-file.yaml)
- Image: as a .tar named after the image (ex: alpine:latest --> alpine:latest.tar)
- Chart: as a .tar.gz named after the chart (ex: loki:2.0.2 --> loki-2.0.2.tar.gz)`,
Example: `
# Download a file
hauler dl localhost:5000/my-file.yaml:latest
# Download an image
hauler dl localhost:5000/rancher/k3s:v1.22.2-k3s2
# Download a chart
hauler dl localhost:5000/hauler/longhorn:1.2.0`,
Aliases: []string{"dl"},
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, arg []string) error {
ctx := cmd.Context()
return download.Cmd(ctx, o, arg[0])
},
}
o.AddArgs(cmd)
parent.AddCommand(cmd)
}

View File

@@ -1,87 +0,0 @@
package download
import (
"context"
"encoding/json"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/v1/remote"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"oras.land/oras-go/pkg/content"
"oras.land/oras-go/pkg/oras"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/internal/mapper"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
)
type Opts struct {
DestinationDir string
Username string
Password string
Insecure bool
PlainHTTP bool
}
func (o *Opts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.DestinationDir, "output", "o", "", "Directory to save contents to (defaults to current directory)")
f.StringVarP(&o.Username, "username", "u", "", "Username when copying to an authenticated remote registry")
f.StringVarP(&o.Password, "password", "p", "", "Password when copying to an authenticated remote registry")
f.BoolVar(&o.Insecure, "insecure", false, "Toggle allowing insecure connections when copying to a remote registry")
f.BoolVar(&o.PlainHTTP, "plain-http", false, "Toggle allowing plain http connections when copying to a remote registry")
}
func Cmd(ctx context.Context, o *Opts, ref string) error {
l := log.FromContext(ctx)
ropts := content.RegistryOptions{
Username: o.Username,
Password: o.Password,
Insecure: o.Insecure,
PlainHTTP: o.PlainHTTP,
}
rs, err := content.NewRegistry(ropts)
if err != nil {
return err
}
r, err := reference.Parse(ref)
if err != nil {
return err
}
desc, err := remote.Get(r, remote.WithAuthFromKeychain(authn.DefaultKeychain), remote.WithContext(ctx))
if err != nil {
return err
}
manifestData, err := desc.RawManifest()
if err != nil {
return err
}
var manifest ocispec.Manifest
if err := json.Unmarshal(manifestData, &manifest); err != nil {
return err
}
mapperStore, err := mapper.FromManifest(manifest, o.DestinationDir)
if err != nil {
return err
}
pushedDesc, err := oras.Copy(ctx, rs, r.Name(), mapperStore, "",
oras.WithAdditionalCachedMediaTypes(consts.DockerManifestSchema2))
if err != nil {
return err
}
l.Infof("downloaded [%s] with digest [%s]", pushedDesc.MediaType, pushedDesc.Digest.String())
return nil
}

75
cmd/hauler/cli/login.go Normal file
View File

@@ -0,0 +1,75 @@
package cli
import (
"context"
"fmt"
"io"
"os"
"strings"
"github.com/spf13/cobra"
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
)
type Opts struct {
Username string
Password string
PasswordStdin bool
}
func (o *Opts) AddArgs(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Username, "username", "u", "", "Username")
f.StringVarP(&o.Password, "password", "p", "", "Password")
f.BoolVarP(&o.PasswordStdin, "password-stdin", "", false, "Take the password from stdin")
}
func addLogin(parent *cobra.Command) {
o := &Opts{}
cmd := &cobra.Command{
Use: "login",
Short: "Log in to a registry",
Example: `
# Log in to reg.example.com
hauler login reg.example.com -u bob -p haulin`,
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, arg []string) error {
ctx := cmd.Context()
if o.PasswordStdin {
contents, err := io.ReadAll(os.Stdin)
if err != nil {
return err
}
o.Password = strings.TrimSuffix(string(contents), "\n")
o.Password = strings.TrimSuffix(o.Password, "\r")
}
if o.Username == "" && o.Password == "" {
return fmt.Errorf("username and password required")
}
return login(ctx, o, arg[0])
},
}
o.AddArgs(cmd)
parent.AddCommand(cmd)
}
func login(ctx context.Context, o *Opts, registry string) error {
ropts := content.RegistryOptions{
Username: o.Username,
Password: o.Password,
}
err := cosign.RegistryLogin(ctx, nil, registry, ropts)
if err != nil {
return err
}
return nil
}

View File

@@ -1,57 +0,0 @@
package cli
import (
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/cmd/hauler/cli/serve"
)
func addServe(parent *cobra.Command) {
cmd := &cobra.Command{
Use: "serve",
Short: "Run one or more of hauler's embedded servers types",
Long: `*** WARNING: Deprecated Command ***
The 'serve' command is deprecated and will be removed in a future release of Hauler.`,
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
}
cmd.AddCommand(
addServeFiles(),
addServeRegistry(),
)
parent.AddCommand(cmd)
}
func addServeFiles() *cobra.Command {
o := &serve.FilesOpts{}
cmd := &cobra.Command{
Use: "files",
Short: "Start a fileserver",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
return serve.FilesCmd(ctx, o)
},
}
o.AddFlags(cmd)
return cmd
}
func addServeRegistry() *cobra.Command {
o := &serve.RegistryOpts{}
cmd := &cobra.Command{
Use: "registry",
Short: "Start a registry",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
return serve.RegistryCmd(ctx, o)
},
}
o.AddFlags(cmd)
return cmd
}

View File

@@ -1,37 +0,0 @@
package serve
import (
"context"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/internal/server"
)
type FilesOpts struct {
Root string
Port int
}
func (o *FilesOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Root, "root", "r", ".", "Path to root of the directory to serve")
f.IntVarP(&o.Port, "port", "p", 8080, "Port to listen on")
}
func FilesCmd(ctx context.Context, o *FilesOpts) error {
cfg := server.FileConfig{
Root: o.Root,
Port: o.Port,
}
s, err := server.NewFile(ctx, cfg)
if err != nil {
return err
}
if err := s.ListenAndServe(); err != nil {
return err
}
return nil
}

View File

@@ -1,81 +0,0 @@
package serve
import (
"context"
"fmt"
"net/http"
"os"
"github.com/distribution/distribution/v3/configuration"
dcontext "github.com/distribution/distribution/v3/context"
"github.com/distribution/distribution/v3/version"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/internal/server"
)
type RegistryOpts struct {
Root string
Port int
ConfigFile string
}
func (o *RegistryOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.Root, "root", "r", ".", "Path to root of the directory to serve")
f.IntVarP(&o.Port, "port", "p", 5000, "Port to listen on")
f.StringVarP(&o.ConfigFile, "config", "c", "", "Path to a config file, will override all other configs")
}
func RegistryCmd(ctx context.Context, o *RegistryOpts) error {
ctx = dcontext.WithVersion(ctx, version.Version)
cfg := o.defaultConfig()
if o.ConfigFile != "" {
ucfg, err := loadConfig(o.ConfigFile)
if err != nil {
return err
}
cfg = ucfg
}
s, err := server.NewRegistry(ctx, cfg)
if err != nil {
return err
}
if err := s.ListenAndServe(); err != nil {
return err
}
return nil
}
func loadConfig(filename string) (*configuration.Configuration, error) {
f, err := os.Open(filename)
if err != nil {
return nil, err
}
return configuration.Parse(f)
}
func (o *RegistryOpts) defaultConfig() *configuration.Configuration {
cfg := &configuration.Configuration{
Version: "0.1",
Storage: configuration.Storage{
"cache": configuration.Parameters{"blobdescriptor": "inmemory"},
"filesystem": configuration.Parameters{"rootdirectory": o.Root},
// TODO: Ensure this is toggleable via cli arg if necessary
// "maintenance": configuration.Parameters{"readonly.enabled": false},
},
}
cfg.Log.Level = "info"
cfg.HTTP.Addr = fmt.Sprintf(":%d", o.Port)
cfg.HTTP.Headers = http.Header{
"X-Content-Type-Options": []string{"nosniff"},
"Accept": []string{"application/vnd.dsse.envelope.v1+json, application/json"},
}
return cfg
}

View File

@@ -1,9 +1,10 @@
package cli
import (
"fmt"
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
"fmt"
"github.com/rancherfederal/hauler/cmd/hauler/cli/store"
)
@@ -125,11 +126,11 @@ func addStoreServe() *cobra.Command {
// RegistryCmd serves the embedded registry
func addStoreServeRegistry() *cobra.Command {
o := &store.ServeRegistryOpts{RootOpts: rootStoreOpts}
o := &store.ServeRegistryOpts{RootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "registry",
Short: "Serve the embedded registry",
RunE: func(cmd *cobra.Command, args []string) error {
Use: "registry",
Short: "Serve the embedded registry",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
s, err := o.Store(ctx)
@@ -138,21 +139,21 @@ func addStoreServeRegistry() *cobra.Command {
}
return store.ServeRegistryCmd(ctx, o, s)
},
}
},
}
o.AddFlags(cmd)
o.AddFlags(cmd)
return cmd
return cmd
}
// FileServerCmd serves the file server
func addStoreServeFiles() *cobra.Command {
o := &store.ServeFilesOpts{RootOpts: rootStoreOpts}
o := &store.ServeFilesOpts{RootOpts: rootStoreOpts}
cmd := &cobra.Command{
Use: "fileserver",
Short: "Serve the file server",
RunE: func(cmd *cobra.Command, args []string) error {
Use: "fileserver",
Short: "Serve the file server",
RunE: func(cmd *cobra.Command, args []string) error {
ctx := cmd.Context()
s, err := o.Store(ctx)
@@ -161,12 +162,12 @@ func addStoreServeFiles() *cobra.Command {
}
return store.ServeFilesCmd(ctx, o, s)
},
}
},
}
o.AddFlags(cmd)
o.AddFlags(cmd)
return cmd
return cmd
}
func addStoreSave() *cobra.Command {
@@ -196,7 +197,7 @@ func addStoreSave() *cobra.Command {
func addStoreInfo() *cobra.Command {
o := &store.InfoOpts{RootOpts: rootStoreOpts}
var allowedValues = []string{"image", "chart", "file", "all"}
var allowedValues = []string{"image", "chart", "file", "sigs", "atts", "sbom", "all"}
cmd := &cobra.Command{
Use: "info",
@@ -210,7 +211,7 @@ func addStoreInfo() *cobra.Command {
if err != nil {
return err
}
for _, allowed := range allowedValues {
if o.TypeFilter == allowed {
return store.InfoCmd(ctx, o, s)

View File

@@ -8,15 +8,13 @@ import (
"github.com/spf13/cobra"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/content/chart"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
)
type AddFileOpts struct {
@@ -33,7 +31,9 @@ func AddFileCmd(ctx context.Context, o *AddFileOpts, s *store.Layout, reference
cfg := v1alpha1.File{
Path: reference,
}
if len(o.Name) > 0 {
cfg.Name = o.Name
}
return storeFile(ctx, s, cfg)
}
@@ -50,12 +50,14 @@ func storeFile(ctx context.Context, s *store.Layout, fi v1alpha1.File) error {
return err
}
desc, err := s.AddOCI(ctx, f, ref.Name())
l.Infof("adding 'file' [%s] to the store as [%s]", fi.Path, ref.Name())
_, err = s.AddOCI(ctx, f, ref.Name())
if err != nil {
return err
}
l.Infof("added 'file' to store at [%s], with digest [%s]", ref.Name(), desc.Digest.String())
l.Infof("successfully added 'file' [%s]", ref.Name())
return nil
}
@@ -93,6 +95,7 @@ func AddImageCmd(ctx context.Context, o *AddImageOpts, s *store.Layout, referenc
func storeImage(ctx context.Context, s *store.Layout, i v1alpha1.Image, platform string) error {
l := log.FromContext(ctx)
l.Infof("adding 'image' [%s] to the store", i.Name)
r, err := name.ParseReference(i.Name)
if err != nil {
@@ -104,7 +107,7 @@ func storeImage(ctx context.Context, s *store.Layout, i v1alpha1.Image, platform
return err
}
l.Infof("added 'image' to store at [%s]", r.Name())
l.Infof("successfully added 'image' [%s]", r.Name())
return nil
}
@@ -141,6 +144,7 @@ func AddChartCmd(ctx context.Context, o *AddChartOpts, s *store.Layout, chartNam
func storeChart(ctx context.Context, s *store.Layout, cfg v1alpha1.Chart, opts *action.ChartPathOptions) error {
l := log.FromContext(ctx)
l.Infof("adding 'chart' [%s] to the store", cfg.Name)
// TODO: This shouldn't be necessary
opts.RepoURL = cfg.RepoURL
@@ -160,11 +164,11 @@ func storeChart(ctx context.Context, s *store.Layout, cfg v1alpha1.Chart, opts *
if err != nil {
return err
}
desc, err := s.AddOCI(ctx, chrt, ref.Name())
_, err = s.AddOCI(ctx, chrt, ref.Name())
if err != nil {
return err
}
l.Infof("added 'chart' to store at [%s], with digest [%s]", ref.Name(), desc.Digest.String())
l.Infof("successfully added 'chart' [%s]", ref.Name())
return nil
}

View File

@@ -9,9 +9,8 @@ import (
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/cosign"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
)
type CopyOpts struct {
@@ -55,7 +54,7 @@ func CopyCmd(ctx context.Context, o *CopyOpts, s *store.Layout, targetRef string
Insecure: o.Insecure,
PlainHTTP: o.PlainHTTP,
}
if ropts.Username != "" {
err := cosign.RegistryLogin(ctx, s, components[1], ropts)
if err != nil {

View File

@@ -2,18 +2,17 @@ package store
import (
"context"
"strings"
"encoding/json"
"fmt"
"strings"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/internal/mapper"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
)
type ExtractOpts struct {
@@ -37,7 +36,7 @@ func ExtractCmd(ctx context.Context, o *ExtractOpts, s *store.Layout, ref string
found := false
if err := s.Walk(func(reference string, desc ocispec.Descriptor) error {
if !strings.Contains(reference, r.Name()) {
return nil
}

View File

@@ -6,16 +6,14 @@ import (
"os"
"path/filepath"
"github.com/rancherfederal/hauler/pkg/layer"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
)
const (
DefaultStoreName = "store"
DefaultCacheDir = "hauler"
)
type RootOpts struct {
@@ -25,8 +23,8 @@ type RootOpts struct {
func (o *RootOpts) AddArgs(cmd *cobra.Command) {
pf := cmd.PersistentFlags()
pf.StringVar(&o.CacheDir, "cache", "", "Location of where to store cache data (defaults to $XDG_CACHE_DIR/hauler)")
pf.StringVarP(&o.StoreDir, "store", "s", DefaultStoreName, "Location to create store at")
pf.StringVar(&o.CacheDir, "cache", "", "(deprecated flag and currently not used)")
}
func (o *RootOpts) Store(ctx context.Context) (*store.Layout, error) {
@@ -48,37 +46,9 @@ func (o *RootOpts) Store(ctx context.Context) (*store.Layout, error) {
return nil, err
}
// TODO: Do we want this to be configurable?
c, err := o.Cache(ctx)
if err != nil {
return nil, err
}
s, err := store.NewLayout(abs, store.WithCache(c))
s, err := store.NewLayout(abs)
if err != nil {
return nil, err
}
return s, nil
}
func (o *RootOpts) Cache(ctx context.Context) (layer.Cache, error) {
dir := o.CacheDir
if dir == "" {
// Default to $XDG_CACHE_HOME
cachedir, err := os.UserCacheDir()
if err != nil {
return nil, err
}
abs, _ := filepath.Abs(filepath.Join(cachedir, DefaultCacheDir))
if err := os.MkdirAll(abs, os.ModePerm); err != nil {
return nil, err
}
dir = abs
}
c := layer.NewFilesystemCache(dir)
return c, nil
}

View File

@@ -4,18 +4,16 @@ import (
"context"
"encoding/json"
"fmt"
"github.com/olekukonko/tablewriter"
"os"
"sort"
"github.com/olekukonko/tablewriter"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/pkg/reference"
"github.com/rancherfederal/hauler/pkg/store"
)
type InfoOpts struct {
@@ -30,7 +28,7 @@ func (o *InfoOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringVarP(&o.OutputFormat, "output", "o", "table", "Output format (table, json)")
f.StringVarP(&o.TypeFilter, "type", "t", "all", "Filter on type (image, chart, file)")
f.StringVarP(&o.TypeFilter, "type", "t", "all", "Filter on type (image, chart, file, sigs, atts, sbom)")
// TODO: Regex/globbing
}
@@ -47,7 +45,7 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
}
defer rc.Close()
// handle multi-arch images
// handle multi-arch images
if desc.MediaType == consts.OCIImageIndexSchema || desc.MediaType == consts.DockerManifestListSchema2 {
var idx ocispec.Index
if err := json.NewDecoder(rc).Decode(&idx); err != nil {
@@ -72,13 +70,13 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
items = append(items, i)
}
}
// handle "non" multi-arch images
// handle "non" multi-arch images
} else if desc.MediaType == consts.DockerManifestSchema2 || desc.MediaType == consts.OCIManifestSchema1 {
var m ocispec.Manifest
if err := json.NewDecoder(rc).Decode(&m); err != nil {
return err
}
rc, err := s.FetchManifest(ctx, m)
if err != nil {
return err
@@ -90,7 +88,7 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
if err := json.NewDecoder(rc).Decode(&internalManifest); err != nil {
return err
}
if internalManifest.Architecture != "" {
i := newItem(s, desc, m, fmt.Sprintf("%s/%s", internalManifest.OS, internalManifest.Architecture), o)
var emptyItem item
@@ -104,8 +102,8 @@ func InfoCmd(ctx context.Context, o *InfoOpts, s *store.Layout) error {
items = append(items, i)
}
}
// handle the rest
} else {
// handle the rest
} else {
var m ocispec.Manifest
if err := json.NewDecoder(rc).Decode(&m); err != nil {
return err
@@ -145,6 +143,7 @@ func buildTable(items ...item) {
table.SetRowLine(false)
table.SetAutoMergeCellsByColumnIndex([]int{0})
totalSize := int64(0)
for _, i := range items {
if i.Type != "" {
row := []string{
@@ -152,11 +151,14 @@ func buildTable(items ...item) {
i.Type,
i.Platform,
fmt.Sprintf("%d", i.Layers),
i.Size,
byteCountSI(i.Size),
}
totalSize += i.Size
table.Append(row)
}
}
table.SetFooter([]string{"", "", "", "Total", byteCountSI(totalSize)})
table.Render()
}
@@ -169,11 +171,11 @@ func buildJson(item ...item) string {
}
type item struct {
Reference string
Type string
Platform string
Layers int
Size string
Reference string
Type string
Platform string
Layers int
Size int64
}
type byReferenceAndArch []item
@@ -182,22 +184,24 @@ func (a byReferenceAndArch) Len() int { return len(a) }
func (a byReferenceAndArch) Swap(i, j int) { a[i], a[j] = a[j], a[i] }
func (a byReferenceAndArch) Less(i, j int) bool {
if a[i].Reference == a[j].Reference {
return a[i].Platform < a[j].Platform
if a[i].Type == "image" && a[j].Type == "image" {
return a[i].Platform < a[j].Platform
}
if a[i].Type == "image" {
return true
}
if a[j].Type == "image" {
return false
}
return a[i].Type < a[j].Type
}
return a[i].Reference < a[j].Reference
}
func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat string, o *InfoOpts) item {
// skip listing cosign items
if desc.Annotations["kind"] == "dev.cosignproject.cosign/atts" ||
desc.Annotations["kind"] == "dev.cosignproject.cosign/sigs" ||
desc.Annotations["kind"] == "dev.cosignproject.cosign/sboms" {
return item{}
}
var size int64 = 0
for _, l := range m.Layers {
size = +l.Size
size += l.Size
}
// Generate a human-readable content type
@@ -213,6 +217,15 @@ func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat
ctype = "image"
}
switch desc.Annotations["kind"] {
case "dev.cosignproject.cosign/sigs":
ctype = "sigs"
case "dev.cosignproject.cosign/atts":
ctype = "atts"
case "dev.cosignproject.cosign/sboms":
ctype = "sbom"
}
ref, err := reference.Parse(desc.Annotations[ocispec.AnnotationRefName])
if err != nil {
return item{}
@@ -223,11 +236,11 @@ func newItem(s *store.Layout, desc ocispec.Descriptor, m ocispec.Manifest, plat
}
return item{
Reference: ref.Name(),
Type: ctype,
Platform: plat,
Layers: len(m.Layers),
Size: byteCountSI(size),
Reference: ref.Name(),
Type: ctype,
Platform: plat,
Layers: len(m.Layers),
Size: size,
}
}

View File

@@ -5,20 +5,26 @@ import (
"os"
"github.com/mholt/archiver/v3"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/content"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
)
type LoadOpts struct {
*RootOpts
TempOverride string
}
func (o *LoadOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
_ = f
// On Unix systems, the default is $TMPDIR if non-empty, else /tmp.
// On Windows, the default is GetTempPath, returning the first non-empty
// value from %TMP%, %TEMP%, %USERPROFILE%, or the Windows directory.
// On Plan 9, the default is /tmp.
f.StringVarP(&o.TempOverride, "tempdir", "t", "", "overrides the default directory for temporary files, as returned by your OS.")
}
// LoadCmd
@@ -28,7 +34,7 @@ func LoadCmd(ctx context.Context, o *LoadOpts, archiveRefs ...string) error {
for _, archiveRef := range archiveRefs {
l.Infof("loading content from [%s] to [%s]", archiveRef, o.StoreDir)
err := unarchiveLayoutTo(ctx, archiveRef, o.StoreDir)
err := unarchiveLayoutTo(ctx, archiveRef, o.StoreDir, o.TempOverride)
if err != nil {
return err
}
@@ -38,8 +44,8 @@ func LoadCmd(ctx context.Context, o *LoadOpts, archiveRefs ...string) error {
}
// unarchiveLayoutTo accepts an archived oci layout and extracts the contents to an existing oci layout, preserving the index
func unarchiveLayoutTo(ctx context.Context, archivePath string, dest string) error {
tmpdir, err := os.MkdirTemp("", "hauler")
func unarchiveLayoutTo(ctx context.Context, archivePath string, dest string, tempOverride string) error {
tmpdir, err := os.MkdirTemp(tempOverride, "hauler")
if err != nil {
return err
}

View File

@@ -14,10 +14,9 @@ import (
"github.com/distribution/distribution/v3/version"
"github.com/spf13/cobra"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/internal/server"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
)
type ServeRegistryOpts struct {
@@ -26,8 +25,7 @@ type ServeRegistryOpts struct {
Port int
RootDir string
ConfigFile string
storedir string
ReadOnly bool
}
func (o *ServeRegistryOpts) AddFlags(cmd *cobra.Command) {
@@ -36,6 +34,7 @@ func (o *ServeRegistryOpts) AddFlags(cmd *cobra.Command) {
f.IntVarP(&o.Port, "port", "p", 5000, "Port to listen on.")
f.StringVar(&o.RootDir, "directory", "registry", "Directory to use for backend. Defaults to $PWD/registry")
f.StringVarP(&o.ConfigFile, "config", "c", "", "Path to a config file, will override all other configs")
f.BoolVar(&o.ReadOnly, "readonly", true, "Run the registry as readonly.")
}
func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout) error {
@@ -68,7 +67,7 @@ func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout
if err != nil {
return err
}
if err = r.ListenAndServe(); err != nil {
return err
}
@@ -79,17 +78,17 @@ func ServeRegistryCmd(ctx context.Context, o *ServeRegistryOpts, s *store.Layout
type ServeFilesOpts struct {
*RootOpts
Port int
RootDir string
storedir string
Port int
Timeout int
RootDir string
}
func (o *ServeFilesOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.IntVarP(&o.Port, "port", "p", 8080, "Port to listen on.")
f.StringVar(&o.RootDir, "directory", "store-files", "Directory to use for backend. Defaults to $PWD/store-files")
f.IntVarP(&o.Timeout, "timeout", "t", 60, "Set the http request timeout duration in seconds for both reads and write.")
f.StringVar(&o.RootDir, "directory", "fileserver", "Directory to use for backend. Defaults to $PWD/fileserver")
}
func ServeFilesCmd(ctx context.Context, o *ServeFilesOpts, s *store.Layout) error {
@@ -100,17 +99,18 @@ func ServeFilesCmd(ctx context.Context, o *ServeFilesOpts, s *store.Layout) erro
if err := CopyCmd(ctx, opts, s, "dir://"+o.RootDir); err != nil {
return err
}
cfg := server.FileConfig{
Root: o.RootDir,
Port: o.Port,
Root: o.RootDir,
Port: o.Port,
Timeout: o.Timeout,
}
f, err := server.NewFile(ctx, cfg)
if err != nil {
return err
}
l.Infof("starting file server on port [%d]", o.Port)
if err := f.ListenAndServe(); err != nil {
return err
@@ -134,9 +134,9 @@ func (o *ServeRegistryOpts) defaultRegistryConfig() *configuration.Configuration
Storage: configuration.Storage{
"cache": configuration.Parameters{"blobdescriptor": "inmemory"},
"filesystem": configuration.Parameters{"rootdirectory": o.RootDir},
// TODO: Ensure this is toggleable via cli arg if necessary
// "maintenance": configuration.Parameters{"readonly.enabled": false},
"maintenance": configuration.Parameters{
"readonly": map[any]any{"enabled": o.ReadOnly},
},
},
}

View File

@@ -27,19 +27,23 @@ import (
type SyncOpts struct {
*RootOpts
ContentFiles []string
Key string
Products []string
Platform string
ContentFiles []string
Key string
Products []string
Platform string
Registry string
ProductRegistry string
}
func (o *SyncOpts) AddFlags(cmd *cobra.Command) {
f := cmd.Flags()
f.StringSliceVarP(&o.ContentFiles, "files", "f", []string{}, "Path to content files")
f.StringSliceVarP(&o.ContentFiles, "files", "f", []string{}, "Path(s) to local content files (Manifests). i.e. '--files ./rke2-files.yml")
f.StringVarP(&o.Key, "key", "k", "", "(Optional) Path to the key for signature verification")
f.StringSliceVar(&o.Products, "products", []string{}, "Used for RGS Carbide customers to supply a product and version and Hauler will retrieve the images. i.e. '--product rancher=v2.7.6'")
f.StringSliceVar(&o.Products, "products", []string{}, "(Optional) Feature for RGS Carbide customers to fetch collections and content from the Carbide Registry. i.e. '--product rancher=v2.8.5,rke2=v1.28.11+rke2r1'")
f.StringVarP(&o.Platform, "platform", "p", "", "(Optional) Specific platform to save. i.e. linux/amd64. Defaults to all if flag is omitted.")
f.StringVarP(&o.Registry, "registry", "r", "", "(Optional) Default pull registry for image refs that are not specifying a registry name.")
f.StringVarP(&o.ProductRegistry, "product-registry", "c", "", "(Optional) Specific Product Registry to use. Defaults to RGS Carbide Registry (rgcrprod.azurecr.us).")
}
func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
@@ -50,7 +54,14 @@ func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
l.Infof("processing content file for product: '%s'", product)
parts := strings.Split(product, "=")
tag := strings.ReplaceAll(parts[1], "+", "-")
manifestLoc := fmt.Sprintf("%s/hauler/%s-manifest.yaml:%s", consts.CarbideRegistry, parts[0], tag)
ProductRegistry := o.ProductRegistry // cli flag
// if no cli flag use CarbideRegistry.
if o.ProductRegistry == "" {
ProductRegistry = consts.CarbideRegistry
}
manifestLoc := fmt.Sprintf("%s/hauler/%s-manifest.yaml:%s", ProductRegistry, parts[0], tag)
l.Infof("retrieving product manifest from: '%s'", manifestLoc)
img := v1alpha1.Image{
Name: manifestLoc,
@@ -59,7 +70,7 @@ func SyncCmd(ctx context.Context, o *SyncOpts, s *store.Layout) error {
if err != nil {
return err
}
err = ExtractCmd(ctx, &ExtractOpts{RootOpts: o.RootOpts}, s, fmt.Sprintf("hauler/%s-manifest.yaml:%s", parts[0],tag))
err = ExtractCmd(ctx, &ExtractOpts{RootOpts: o.RootOpts}, s, fmt.Sprintf("hauler/%s-manifest.yaml:%s", parts[0], tag))
if err != nil {
return err
}
@@ -140,12 +151,22 @@ func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layo
}
a := cfg.GetAnnotations()
for _, i := range cfg.Spec.Images {
// Check if the user provided a registry. If a registry is provided in the annotation, use it for the images that don't have a registry in their ref name.
if a[consts.ImageAnnotationRegistry] != "" {
newRef,_ := reference.Parse(i.Name)
if a[consts.ImageAnnotationRegistry] != "" || o.Registry != "" {
newRef, _ := reference.Parse(i.Name)
newReg := o.Registry // cli flag
// if no cli flag but there was an annotation, use the annotation.
if o.Registry == "" && a[consts.ImageAnnotationRegistry] != "" {
newReg = a[consts.ImageAnnotationRegistry]
}
if newRef.Context().RegistryStr() == "" {
newRef,_ = reference.Relocate(i.Name, a[consts.ImageAnnotationRegistry])
newRef, err = reference.Relocate(i.Name, newReg)
if err != nil {
return err
}
}
i.Name = newRef.Name()
}
@@ -156,13 +177,19 @@ func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layo
// if no cli flag but there was an annotation, use the annotation.
if o.Key == "" && a[consts.ImageAnnotationKey] != "" {
key, err = homedir.Expand(a[consts.ImageAnnotationKey])
if err != nil {
return err
}
}
// the individual image key trumps all
if i.Key != "" {
key, err = homedir.Expand(i.Key)
if err != nil {
return err
}
}
l.Debugf("key for image [%s]", key)
// verify signature using the provided key.
err := cosign.VerifySignature(ctx, s, key, i.Name)
if err != nil {
@@ -182,8 +209,7 @@ func processContent(ctx context.Context, fi *os.File, o *SyncOpts, s *store.Layo
if i.Platform != "" {
platform = i.Platform
}
l.Debugf("platform for image [%s]", platform)
err = storeImage(ctx, s, i, platform)
if err != nil {
return err

View File

@@ -2,8 +2,8 @@ package main
import (
"context"
"os"
"embed"
"os"
"github.com/rancherfederal/hauler/cmd/hauler/cli"
"github.com/rancherfederal/hauler/pkg/cosign"
@@ -23,9 +23,12 @@ func main() {
// ensure cosign binary is available
if err := cosign.EnsureBinaryExists(ctx, binaries); err != nil {
logger.Errorf("%v", err)
os.Exit(1)
}
if err := cli.New().ExecuteContext(ctx); err != nil {
logger.Errorf("%v", err)
cancel()
os.Exit(1)
}
}

14
go.mod
View File

@@ -21,7 +21,7 @@ require (
github.com/spf13/afero v1.10.0
github.com/spf13/cobra v1.8.0
golang.org/x/sync v0.6.0
helm.sh/helm/v3 v3.14.0
helm.sh/helm/v3 v3.14.2
k8s.io/apimachinery v0.29.0
k8s.io/client-go v0.29.0
oras.land/oras-go v1.2.5
@@ -54,7 +54,7 @@ require (
github.com/distribution/reference v0.5.0 // indirect
github.com/docker/cli v25.0.1+incompatible // indirect
github.com/docker/distribution v2.8.3+incompatible // indirect
github.com/docker/docker v25.0.1+incompatible // indirect
github.com/docker/docker v25.0.6+incompatible // indirect
github.com/docker/docker-credential-helpers v0.7.0 // indirect
github.com/docker/go-connections v0.5.0 // indirect
github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c // indirect
@@ -144,17 +144,17 @@ require (
go.opentelemetry.io/otel/metric v1.19.0 // indirect
go.opentelemetry.io/otel/trace v1.19.0 // indirect
go.starlark.net v0.0.0-20230525235612-a134d8f9ddca // indirect
golang.org/x/crypto v0.18.0 // indirect
golang.org/x/net v0.17.0 // indirect
golang.org/x/crypto v0.21.0 // indirect
golang.org/x/net v0.23.0 // indirect
golang.org/x/oauth2 v0.10.0 // indirect
golang.org/x/sys v0.16.0 // indirect
golang.org/x/term v0.16.0 // indirect
golang.org/x/sys v0.18.0 // indirect
golang.org/x/term v0.18.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.3.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d // indirect
google.golang.org/grpc v1.58.3 // indirect
google.golang.org/protobuf v1.31.0 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect

28
go.sum
View File

@@ -129,8 +129,8 @@ github.com/docker/cli v25.0.1+incompatible h1:mFpqnrS6Hsm3v1k7Wa/BO23oz0k121MTbT
github.com/docker/cli v25.0.1+incompatible/go.mod h1:JLrzqnKDaYBop7H2jaqPtU4hHvMKP+vjCwu2uszcLI8=
github.com/docker/distribution v2.8.3+incompatible h1:AtKxIZ36LoNK51+Z6RpzLpddBirtxJnzDrHLEKxTAYk=
github.com/docker/distribution v2.8.3+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v25.0.1+incompatible h1:k5TYd5rIVQRSqcTwCID+cyVA0yRg86+Pcrz1ls0/frA=
github.com/docker/docker v25.0.1+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v25.0.6+incompatible h1:5cPwbwriIcsua2REJe8HqQV+6WlWc1byg2QSXzBxBGg=
github.com/docker/docker v25.0.6+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker-credential-helpers v0.7.0 h1:xtCHsjxogADNZcdv1pKUHXryefjlVRqWqIhk/uXJp0A=
github.com/docker/docker-credential-helpers v0.7.0/go.mod h1:rETQfLdHNT3foU5kuNkFR1R1V12OJRRO5lzt2D1b5X0=
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
@@ -564,8 +564,8 @@ golang.org/x/crypto v0.0.0-20210421170649-83a5a9bb288b/go.mod h1:T9bdIzuCu7OtxOm
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220722155217-630584e8d5aa/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.3.0/go.mod h1:hebNnKkNXi2UzZN1eVRvBB7co0a+JxK6XbPiWVs/3J4=
golang.org/x/crypto v0.18.0 h1:PGVlW0xEltQnzFZ55hkuX5+KLyrMYhHld1YHO4AKcdc=
golang.org/x/crypto v0.18.0/go.mod h1:R0j02AL6hcrfOiy9T4ZYp/rcWeMxM3L6QYxlOuEG1mg=
golang.org/x/crypto v0.21.0 h1:X31++rzVUdKhX5sWmSOFZxx8UW/ldWx55cbf08iNAMA=
golang.org/x/crypto v0.21.0/go.mod h1:0BP7YvVV9gBbVKyeTG0Gyn+gZm94bibOW5BjDEYAOMs=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190306152737-a1d7652674e8/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190510132918-efd6b22b2522/go.mod h1:ZjyILWgesfNpC6sMxTJOJm9Kp84zZh5NQWvqDGG3Qr8=
@@ -638,8 +638,8 @@ golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.2.0/go.mod h1:KqCZLdyyvdV855qA2rE3GC2aiw5xGR5TEjj8smXukLY=
golang.org/x/net v0.17.0 h1:pVaXccu2ozPjCXewfr1S7xza/zcXTity9cCdXQYSjIM=
golang.org/x/net v0.17.0/go.mod h1:NxSsAGuq816PNPmqtQdLE42eU2Fs7NoRIZrHJAlaCOE=
golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
golang.org/x/net v0.23.0/go.mod h1:JKghWKKOSdJwpW2GEx0Ja7fmaKnMsbu+MWVZTokSYmg=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@@ -713,14 +713,14 @@ golang.org/x/sys v0.0.0-20220906165534-d0df966e6959/go.mod h1:oPkhp1MJrh7nUepCBc
golang.org/x/sys v0.2.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.16.0 h1:xWw16ngr6ZMtmxDyKyIgsE93KNKz5HKmMa3b8ALHidU=
golang.org/x/sys v0.16.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
golang.org/x/sys v0.18.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.0.0-20220526004731-065cf7ba2467/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.2.0/go.mod h1:TVmDHMZPmdnySmBfhjOoOdhjzdE1h4u1VwSiw2l1Nuc=
golang.org/x/term v0.16.0 h1:m+B6fahuftsE9qjo0VWp2FW0mB3MTJvR0BaMQrq0pmE=
golang.org/x/term v0.16.0/go.mod h1:yn7UURbUtPyrVJPGPq404EukNFxcm/foM+bV/bfcDsY=
golang.org/x/term v0.18.0 h1:FcHjZXDMxI8mM3nwhX9HlKop4C0YQvCVCdwYl2wOtE8=
golang.org/x/term v0.18.0/go.mod h1:ILwASektA3OnRv7amZ1xhE/KTR+u50pbXfZ03+6Nx58=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@@ -888,8 +888,8 @@ google.golang.org/protobuf v1.24.0/go.mod h1:r/3tXBNzIEhYS9I1OUVjXDlt8tc493IdKGj
google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
google.golang.org/protobuf v1.33.0/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -910,8 +910,8 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.4.0 h1:ZazjZUfuVeZGLAmlKKuyv3IKP5orXcwtOwDQH6YVr6o=
gotest.tools/v3 v3.4.0/go.mod h1:CtbdzLSsqVhDgMtKsx03ird5YTGB3ar27v0u/yKBW5g=
helm.sh/helm/v3 v3.14.0 h1:TaZIH6uOchn7L27ptwnnuHJiFrT/BsD4dFdp/HLT2nM=
helm.sh/helm/v3 v3.14.0/go.mod h1:2itvvDv2WSZXTllknfQo6j7u3VVgMAvm8POCDgYH424=
helm.sh/helm/v3 v3.14.2 h1:V71fv+NGZv0icBlr+in1MJXuUIHCiPG1hW9gEBISTIA=
helm.sh/helm/v3 v3.14.2/go.mod h1:2itvvDv2WSZXTllknfQo6j7u3VVgMAvm8POCDgYH424=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
honnef.co/go/tools v0.0.0-20190418001031-e561f6794a2a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=

View File

@@ -2,28 +2,42 @@
# Usage:
# - curl -sfL... | ENV_VAR=... bash
# - ENV_VAR=... bash ./install.sh
# - ./install.sh ENV_VAR=...
# Example:
# - ENV_VAR=... ./install.sh
#
# Install Usage:
# Install Latest Release
# - curl -sfL https://get.hauler.dev | bash
# - ./install.sh
#
# Install Specific Release
# - curl -sfL https://get.hauler.dev | HAULER_VERSION=0.4.2 bash
# - curl -sfL https://get.hauler.dev | HAULER_VERSION=1.0.0 bash
# - HAULER_VERSION=1.0.0 ./install.sh
#
# Set Install Directory
# - curl -sfL https://get.hauler.dev | HAULER_INSTALL_DIR=/usr/local/bin bash
# - HAULER_INSTALL_DIR=/usr/local/bin ./install.sh
#
# Debug Usage:
# - curl -sfL https://get.hauler.dev | HAULER_DEBUG=true bash
# - HAULER_DEBUG=true ./install.sh
#
# Uninstall Usage:
# - curl -sfL https://get.hauler.dev | HAULER_UNINSTALL=true bash
# - HAULER_UNINSTALL=true ./install.sh
#
# Documentation:
# - https://hauler.dev
# - https://github.com/rancherfederal/hauler
# set functions for debugging/logging
function info {
echo && echo "[INFO] Hauler: $1"
}
# - https://github.com/hauler-dev/hauler
# set functions for logging
function verbose {
echo "$1"
}
function info {
echo && echo "[INFO] Hauler: $1"
}
function warn {
echo && echo "[WARN] Hauler: $1"
}
@@ -33,124 +47,169 @@ function fatal {
exit 1
}
# check for required dependencies
for cmd in curl sed awk openssl tar rm; do
# debug hauler from argument or environment variable
if [ "${HAULER_DEBUG}" = "true" ]; then
set -x
fi
# start hauler preflight checks
info "Starting Preflight Checks..."
# check for required packages and dependencies
for cmd in echo curl grep sed rm mkdir awk openssl tar install source; do
if ! command -v "$cmd" &> /dev/null; then
fatal "$cmd is not installed"
fatal "$cmd is required to install Hauler"
fi
done
# start hauler installation
info "Starting Installation..."
# set install directory from argument or environment variable
HAULER_INSTALL_DIR=${HAULER_INSTALL_DIR:-/usr/local/bin}
# set version with an environment variable
version=${HAULER_VERSION:-$(curl -s https://api.github.com/repos/rancherfederal/hauler/releases/latest | grep '"tag_name":' | sed 's/.*"v\([^"]*\)".*/\1/')}
# ensure install directory exists
if [ ! -d "${HAULER_INSTALL_DIR}" ]; then
mkdir -p "${HAULER_INSTALL_DIR}" || fatal "Failed to Create Install Directory: ${HAULER_INSTALL_DIR}"
fi
# set verision with an argument
while [[ $# -gt 0 ]]; do
case "$1" in
HAULER_VERSION=*)
version="${1#*=}"
shift
;;
*)
shift
;;
esac
done
# ensure install directory is writable (by user or root privileges)
if [ ! -w "${HAULER_INSTALL_DIR}" ]; then
if [ "$(id -u)" -ne 0 ]; then
fatal "Root privileges are required to install Hauler to Directory: ${HAULER_INSTALL_DIR}"
fi
fi
# uninstall hauler from argument or environment variable
if [ "${HAULER_UNINSTALL}" = "true" ]; then
# remove the hauler binary
rm -rf "${HAULER_INSTALL_DIR}/hauler" || fatal "Failed to Remove Hauler from ${HAULER_INSTALL_DIR}"
# remove the working directory
rm -rf "$HOME/.hauler" || fatal "Failed to Remove Hauler Directory: $HOME/.hauler"
info "Successfully Uninstalled Hauler" && echo
exit 0
fi
# set version environment variable
if [ -z "${HAULER_VERSION}" ]; then
# attempt to retrieve the latest version from GitHub
HAULER_VERSION=$(curl -s https://api.github.com/repos/hauler-dev/hauler/releases/latest | grep '"tag_name":' | sed 's/.*"v\([^"]*\)".*/\1/')
# exit if the version could not be detected
if [ -z "${HAULER_VERSION}" ]; then
fatal "HAULER_VERSION is unable to be detected and/or retrieved from GitHub"
fi
fi
# detect the operating system
platform=$(uname -s | tr '[:upper:]' '[:lower:]')
case $platform in
PLATFORM=$(uname -s | tr '[:upper:]' '[:lower:]')
case $PLATFORM in
linux)
platform="linux"
PLATFORM="linux"
;;
darwin)
platform="darwin"
PLATFORM="darwin"
;;
*)
fatal "Unsupported Platform: $platform"
fatal "Unsupported Platform: $PLATFORM"
;;
esac
# detect the architecture
arch=$(uname -m)
case $arch in
ARCH=$(uname -m)
case $ARCH in
x86_64 | x86-32 | x64 | x32 | amd64)
arch="amd64"
ARCH="amd64"
;;
aarch64 | arm64)
arch="arm64"
ARCH="arm64"
;;
*)
fatal "Unsupported Architecture: $arch"
fatal "Unsupported Architecture: $ARCH"
;;
esac
# start hauler installation
info "Starting Installation..."
# display the version, platform, and architecture
verbose "- Version: v$version"
verbose "- Platform: $platform"
verbose "- Architecture: $arch"
verbose "- Version: v${HAULER_VERSION}"
verbose "- Platform: $PLATFORM"
verbose "- Architecture: $ARCH"
verbose "- Install Directory: ${HAULER_INSTALL_DIR}"
# check working directory and/or create it
if [ ! -d "$HOME/.hauler" ]; then
mkdir -p "$HOME/.hauler" || fatal "Failed to Create Directory: $HOME/.hauler"
fi
# update permissions of working directory
chmod -R 777 "$HOME/.hauler" || fatal "Failed to Update Permissions of Directory: $HOME/.hauler"
# change to working directory
cd "$HOME/.hauler" || fatal "Failed to Change Directory: $HOME/.hauler"
# start hauler artifacts download
info "Starting Download..."
# download the checksum file
if ! curl -sOL "https://github.com/rancherfederal/hauler/releases/download/v${version}/hauler_${version}_checksums.txt"; then
fatal "Failed to Download: hauler_${version}_checksums.txt"
if ! curl -sfOL "https://github.com/hauler-dev/hauler/releases/download/v${HAULER_VERSION}/hauler_${HAULER_VERSION}_checksums.txt"; then
fatal "Failed to Download: hauler_${HAULER_VERSION}_checksums.txt"
fi
# download the archive file
if ! curl -sOL "https://github.com/rancherfederal/hauler/releases/download/v${version}/hauler_${version}_${platform}_${arch}.tar.gz"; then
fatal "Failed to Download: hauler_${version}_${platform}_${arch}.tar.gz"
if ! curl -sfOL "https://github.com/hauler-dev/hauler/releases/download/v${HAULER_VERSION}/hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"; then
fatal "Failed to Download: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
fi
# start hauler checksum verification
info "Starting Checksum Verification..."
# Verify the Hauler checksum
expected_checksum=$(awk -v version="$version" -v platform="$platform" -v arch="$arch" '$2 == "hauler_"version"_"platform"_"arch".tar.gz" {print $1}' "hauler_${version}_checksums.txt")
determined_checksum=$(openssl dgst -sha256 "hauler_${version}_${platform}_${arch}.tar.gz" | awk '{print $2}')
# verify the Hauler checksum
EXPECTED_CHECKSUM=$(awk -v HAULER_VERSION="${HAULER_VERSION}" -v PLATFORM="${PLATFORM}" -v ARCH="${ARCH}" '$2 == "hauler_"HAULER_VERSION"_"PLATFORM"_"ARCH".tar.gz" {print $1}' "hauler_${HAULER_VERSION}_checksums.txt")
DETERMINED_CHECKSUM=$(openssl dgst -sha256 "hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz" | awk '{print $2}')
if [ -z "$expected_checksum" ]; then
fatal "Failed to Locate Checksum: hauler_${version}_${platform}_${arch}.tar.gz"
elif [ "$determined_checksum" = "$expected_checksum" ]; then
verbose "- Expected Checksum: $expected_checksum"
verbose "- Determined Checksum: $determined_checksum"
verbose "- Successfully Verified Checksum: hauler_${version}_${platform}_${arch}.tar.gz"
if [ -z "${EXPECTED_CHECKSUM}" ]; then
fatal "Failed to Locate Checksum: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
elif [ "${DETERMINED_CHECKSUM}" = "${EXPECTED_CHECKSUM}" ]; then
verbose "- Expected Checksum: ${EXPECTED_CHECKSUM}"
verbose "- Determined Checksum: ${DETERMINED_CHECKSUM}"
verbose "- Successfully Verified Checksum: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
else
verbose "- Expected: $expected_checksum"
verbose "- Determined: $determined_checksum"
fatal "Failed Checksum Verification: hauler_${version}_${platform}_${arch}.tar.gz"
verbose "- Expected: ${EXPECTED_CHECKSUM}"
verbose "- Determined: ${DETERMINED_CHECKSUM}"
fatal "Failed Checksum Verification: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
fi
# uncompress the archive
tar -xzf "hauler_${version}_${platform}_${arch}.tar.gz" || fatal "Failed to Extract: hauler_${version}_${platform}_${arch}.tar.gz"
# uncompress the hauler archive
tar -xzf "hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz" || fatal "Failed to Extract: hauler_${HAULER_VERSION}_${PLATFORM}_${ARCH}.tar.gz"
# install the binary
case "$platform" in
linux)
install hauler /usr/local/bin || fatal "Failed to Install Hauler to /usr/local/bin"
;;
darwin)
install hauler /usr/local/bin || fatal "Failed to Install Hauler to /usr/local/bin"
;;
*)
fatal "Unsupported Platform or Architecture: $platform/$arch"
;;
esac
# install the hauler binary
install -m 755 hauler "${HAULER_INSTALL_DIR}" || fatal "Failed to Install Hauler: ${HAULER_INSTALL_DIR}"
# clean up checksum(s)
rm -rf "hauler_${version}_checksums.txt" || warn "Failed to Remove: hauler_${version}_checksums.txt"
# clean up archive file(s)
rm -rf "hauler_${version}_${platform}_${arch}.tar.gz" || warn "Failed to Remove: hauler_${version}_${platform}_${arch}.tar.gz"
# clean up other files
rm -rf LICENSE README.md hauler
# add hauler to the path
if [[ ":$PATH:" != *":${HAULER_INSTALL_DIR}:"* ]]; then
if [ -f "$HOME/.bashrc" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.bashrc"
source "$HOME/.bashrc"
elif [ -f "$HOME/.bash_profile" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.bash_profile"
source "$HOME/.bash_profile"
elif [ -f "$HOME/.zshrc" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.zshrc"
source "$HOME/.zshrc"
elif [ -f "$HOME/.profile" ]; then
echo "export PATH=\$PATH:${HAULER_INSTALL_DIR}" >> "$HOME/.profile"
source "$HOME/.profile"
else
warn "Failed to add ${HAULER_INSTALL_DIR} to PATH: Unsupported Shell"
fi
fi
# display success message
info "Successfully Installed at /usr/local/bin/hauler"
info "Successfully Installed Hauler at ${HAULER_INSTALL_DIR}/hauler"
# display availability message
verbose "- Hauler v${version} is now available for use!"
info "Hauler v${HAULER_VERSION} is now available for use!"
# display hauler docs message
verbose "- Documentation: https://hauler.dev" && echo

View File

@@ -2,7 +2,7 @@ package mapper
import (
"context"
"io/ioutil"
"io"
"os"
"path/filepath"
"strings"
@@ -15,7 +15,8 @@ import (
)
// NewMapperFileStore creates a new file store that uses mapper functions for each detected descriptor.
// This extends content.File, and differs in that it allows much more functionality into how each descriptor is written.
//
// This extends content.File, and differs in that it allows much more functionality into how each descriptor is written.
func NewMapperFileStore(root string, mapper map[string]Fn) *store {
fs := content.NewFile(root)
return &store{
@@ -58,7 +59,7 @@ func (s *pusher) Push(ctx context.Context, desc ocispec.Descriptor) (ccontent.Wr
// If no custom mapper found, fall back to content.File mapper
if _, ok := s.mapper[desc.MediaType]; !ok {
return content.NewIoContentWriter(ioutil.Discard, content.WithOutputHash(desc.Digest)), nil
return content.NewIoContentWriter(io.Discard, content.WithOutputHash(desc.Digest)), nil
}
filename, err := s.mapper[desc.MediaType](desc)

View File

@@ -12,9 +12,10 @@ import (
)
type FileConfig struct {
Root string
Host string
Port int
Root string
Host string
Port int
Timeout int
}
// NewFile returns a fileserver
@@ -30,11 +31,15 @@ func NewFile(ctx context.Context, cfg FileConfig) (Server, error) {
cfg.Port = 8080
}
if cfg.Timeout == 0 {
cfg.Timeout = 60
}
srv := &http.Server{
Handler: r,
Addr: fmt.Sprintf(":%d", cfg.Port),
WriteTimeout: 15 * time.Second,
ReadTimeout: 15 * time.Second,
WriteTimeout: time.Duration(cfg.Timeout) * time.Second,
ReadTimeout: time.Duration(cfg.Timeout) * time.Second,
}
return srv, nil

View File

@@ -47,7 +47,7 @@ func NewTempRegistry(ctx context.Context, root string) *tmpRegistryServer {
}
// Add validation configuration
cfg.Validation.Manifests.URLs.Allow = []string{".+"}
cfg.Log.Level = "error"
cfg.HTTP.Headers = http.Header{
"X-Content-Type-Options": []string{"nosniff"},

View File

@@ -226,4 +226,4 @@ func (i *Info) CheckFontName(fontName string) bool {
fmt.Fprintln(os.Stderr, "font not valid, using default")
return false
}
}

View File

@@ -20,7 +20,7 @@ type ImageSpec struct {
type Image struct {
// Name is the full location for the image, can be referenced by tags or digests
Name string `json:"name"`
// Path is the path to the cosign public key used for verifying image signatures
//Key string `json:"key,omitempty"`
Key string `json:"key"`

View File

@@ -1,6 +1,8 @@
package v1alpha1
import metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
const K3sCollectionKind = "K3s"

View File

@@ -25,6 +25,11 @@ func (h Http) Name(u *url.URL) string {
return ""
}
name, _ := url.PathUnescape(u.String())
if err != nil {
return ""
}
contentType := resp.Header.Get("Content-Type")
for _, v := range strings.Split(contentType, ",") {
t, _, err := mime.ParseMediaType(v)
@@ -36,7 +41,7 @@ func (h Http) Name(u *url.URL) string {
}
// TODO: Not this
return filepath.Base(u.String())
return filepath.Base(name)
}
func (h Http) Open(ctx context.Context, u *url.URL) (io.ReadCloser, error) {

View File

@@ -1,6 +1,7 @@
package image
import (
"fmt"
"github.com/google/go-containerregistry/pkg/authn"
gname "github.com/google/go-containerregistry/pkg/name"
gv1 "github.com/google/go-containerregistry/pkg/v1"
@@ -51,3 +52,29 @@ func NewImage(name string, opts ...remote.Option) (*Image, error) {
Image: img,
}, nil
}
func IsMultiArchImage(name string, opts ...remote.Option) (bool, error) {
ref, err := gname.ParseReference(name)
if err != nil {
return false, fmt.Errorf("parsing reference %q: %v", name, err)
}
defaultOpts := []remote.Option{
remote.WithAuthFromKeychain(authn.DefaultKeychain),
}
opts = append(opts, defaultOpts...)
desc, err := remote.Get(ref, opts...)
if err != nil {
return false, fmt.Errorf("getting image %q: %v", name, err)
}
_, err = desc.ImageIndex()
if err != nil {
// If the descriptor could not be converted to an image index, it's not a multi-arch image
return false, nil
}
// If the descriptor could be converted to an image index, it's a multi-arch image
return true, nil
}

View File

@@ -3,8 +3,9 @@ package artifacts
import "github.com/google/go-containerregistry/pkg/v1"
// OCI is the bare minimum we need to represent an artifact in an oci layout
// At a high level, it is not constrained by an Image's config, manifests, and layer ordinality
// This specific implementation fully encapsulates v1.Layer's within a more generic form
//
// At a high level, it is not constrained by an Image's config, manifests, and layer ordinality
// This specific implementation fully encapsulates v1.Layer's within a more generic form
type OCI interface {
MediaType() string

View File

@@ -1,11 +1,11 @@
package chart
import (
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/content/chart"
"github.com/rancherfederal/hauler/pkg/reference"
)

View File

@@ -9,12 +9,12 @@ import (
"strings"
"sync"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/google/go-containerregistry/pkg/name"
artifact "github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/log"
)
type ImageTxt struct {

View File

@@ -11,12 +11,9 @@ import (
"strings"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/artifacts/file"
"github.com/rancherfederal/hauler/pkg/artifacts/file/getter"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/reference"
)

View File

@@ -50,8 +50,8 @@ const (
KindAnnotationName = "kind"
KindAnnotation = "dev.cosignproject.cosign/image"
CarbideRegistry = "rgcrprod.azurecr.us"
ImageAnnotationKey = "hauler.dev/key"
CarbideRegistry = "rgcrprod.azurecr.us"
ImageAnnotationKey = "hauler.dev/key"
ImageAnnotationPlatform = "hauler.dev/platform"
ImageAnnotationRegistry = "hauler.dev/registry"
)

View File

@@ -5,8 +5,10 @@ import (
"bytes"
"compress/gzip"
"encoding/json"
"fmt"
"io"
"io/fs"
"net/url"
"os"
"path/filepath"
@@ -15,17 +17,21 @@ import (
gtypes "github.com/google/go-containerregistry/pkg/v1/types"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/rancherfederal/hauler/pkg/artifacts"
"github.com/rancherfederal/hauler/pkg/log"
"helm.sh/helm/v3/pkg/action"
"helm.sh/helm/v3/pkg/chart"
"helm.sh/helm/v3/pkg/chart/loader"
"helm.sh/helm/v3/pkg/cli"
"github.com/rancherfederal/hauler/pkg/layer"
"helm.sh/helm/v3/pkg/registry"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/layer"
)
var _ artifacts.OCI = (*Chart)(nil)
var (
_ artifacts.OCI = (*Chart)(nil)
settings = cli.New()
)
// Chart implements the OCI interface for Chart API objects. API spec values are
// stored into the Repo, Name, and Version fields.
@@ -36,22 +42,31 @@ type Chart struct {
// NewChart is a helper method that returns NewLocalChart or NewRemoteChart depending on v1alpha1.Chart contents
func NewChart(name string, opts *action.ChartPathOptions) (*Chart, error) {
cpo := action.ChartPathOptions{
RepoURL: opts.RepoURL,
Version: opts.Version,
CaFile: opts.CaFile,
CertFile: opts.CertFile,
KeyFile: opts.KeyFile,
InsecureSkipTLSverify: opts.InsecureSkipTLSverify,
Keyring: opts.Keyring,
Password: opts.Password,
PassCredentialsAll: opts.PassCredentialsAll,
Username: opts.Username,
Verify: opts.Verify,
chartRef := name
actionConfig := new(action.Configuration)
if err := actionConfig.Init(settings.RESTClientGetter(), settings.Namespace(), os.Getenv("HELM_DRIVER"), log.NewLogger(os.Stdout).Debugf); err != nil {
return nil, err
}
chartPath, err := cpo.LocateChart(name, cli.New())
client := action.NewInstall(actionConfig)
client.ChartPathOptions.Version = opts.Version
registryClient, err := newRegistryClient(client.CertFile, client.KeyFile, client.CaFile,
client.InsecureSkipTLSverify, client.PlainHTTP)
if err != nil {
return nil, fmt.Errorf("missing registry client: %w", err)
}
client.SetRegistryClient(registryClient)
if registry.IsOCI(opts.RepoURL) {
chartRef = opts.RepoURL + "/" + name
} else if isUrl(opts.RepoURL) { // OCI Protocol registers as a valid URL
client.ChartPathOptions.RepoURL = opts.RepoURL
} else { // Handles cases like grafana/loki
chartRef = opts.RepoURL + "/" + name
}
chartPath, err := client.ChartPathOptions.LocateChart(chartRef, settings)
if err != nil {
return nil, err
}
@@ -217,3 +232,52 @@ func (h *Chart) chartData() (gv1.Layer, error) {
return chartDataLayer, err
}
func isUrl(name string) bool {
_, err := url.ParseRequestURI(name)
return err == nil
}
func newRegistryClient(certFile, keyFile, caFile string, insecureSkipTLSverify, plainHTTP bool) (*registry.Client, error) {
if certFile != "" && keyFile != "" || caFile != "" || insecureSkipTLSverify {
registryClient, err := newRegistryClientWithTLS(certFile, keyFile, caFile, insecureSkipTLSverify)
if err != nil {
return nil, err
}
return registryClient, nil
}
registryClient, err := newDefaultRegistryClient(plainHTTP)
if err != nil {
return nil, err
}
return registryClient, nil
}
func newDefaultRegistryClient(plainHTTP bool) (*registry.Client, error) {
opts := []registry.ClientOption{
registry.ClientOptDebug(settings.Debug),
registry.ClientOptEnableCache(true),
registry.ClientOptWriter(os.Stderr),
registry.ClientOptCredentialsFile(settings.RegistryConfig),
}
if plainHTTP {
opts = append(opts, registry.ClientOptPlainHTTP())
}
// Create a new registry client
registryClient, err := registry.NewClient(opts...)
if err != nil {
return nil, err
}
return registryClient, nil
}
func newRegistryClientWithTLS(certFile, keyFile, caFile string, insecureSkipTLSverify bool) (*registry.Client, error) {
// Create a new registry client
registryClient, err := registry.NewRegistryClientWithTLS(os.Stderr, certFile, keyFile, caFile, insecureSkipTLSverify,
settings.RegistryConfig, settings.Debug,
)
if err != nil {
return nil, err
}
return registryClient, nil
}

View File

@@ -6,19 +6,13 @@ import (
"testing"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/mholt/archiver/v3"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"helm.sh/helm/v3/pkg/action"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/content/chart"
)
var (
chartpath = "../../../testdata/podinfo-6.0.3.tgz"
)
func TestNewChart(t *testing.T) {
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
@@ -26,10 +20,6 @@ func TestNewChart(t *testing.T) {
}
defer os.RemoveAll(tmpdir)
if err := archiver.Unarchive(chartpath, tmpdir); err != nil {
t.Fatal(err)
}
type args struct {
name string
opts *action.ChartPathOptions
@@ -43,18 +33,18 @@ func TestNewChart(t *testing.T) {
{
name: "should create from a chart archive",
args: args{
name: chartpath,
opts: &action.ChartPathOptions{},
name: "rancher-cluster-templates-0.4.4.tgz",
opts: &action.ChartPathOptions{RepoURL: "../../../testdata"},
},
want: v1.Descriptor{
MediaType: consts.ChartLayerMediaType,
Size: 13524,
Size: 13102,
Digest: v1.Hash{
Algorithm: "sha256",
Hex: "e30b95a08787de69ffdad3c232d65cfb131b5b50c6fd44295f48a078fceaa44e",
Hex: "4b3bb4e474b54bf9057b298f8f11c239bb561396716d8cd5fc369c407fba2965",
},
Annotations: map[string]string{
ocispec.AnnotationTitle: "podinfo-6.0.3.tgz",
ocispec.AnnotationTitle: "rancher-cluster-templates-0.4.4.tgz",
},
},
wantErr: false,
@@ -72,18 +62,18 @@ func TestNewChart(t *testing.T) {
// TODO: Use a mock helm server
name: "should fetch a remote chart",
args: args{
name: "ingress-nginx",
opts: &action.ChartPathOptions{RepoURL: "https://kubernetes.github.io/ingress-nginx", Version: "4.0.16"},
name: "cert-manager",
opts: &action.ChartPathOptions{RepoURL: "https://charts.jetstack.io", Version: "1.14.4"},
},
want: v1.Descriptor{
MediaType: consts.ChartLayerMediaType,
Size: 38591,
Size: 80674,
Digest: v1.Hash{
Algorithm: "sha256",
Hex: "b0ea91f7febc6708ad9971871d2de6e8feb2072110c3add6dd7082d90753caa2",
Hex: "5775fdbc1881d6e510df76d38753af54b86bd14caa8edb28fdbb79527042dede",
},
Annotations: map[string]string{
ocispec.AnnotationTitle: "ingress-nginx-4.0.16.tgz",
ocispec.AnnotationTitle: "cert-manager-v1.14.4.tgz",
},
},
wantErr: false,

View File

@@ -4,8 +4,8 @@ import (
"context"
"encoding/json"
"fmt"
"github.com/google/go-containerregistry/pkg/name"
"io"
"io/ioutil"
"os"
"path/filepath"
"sort"
@@ -20,6 +20,7 @@ import (
"oras.land/oras-go/pkg/target"
"github.com/rancherfederal/hauler/pkg/consts"
"github.com/rancherfederal/hauler/pkg/reference"
)
var _ target.Target = (*OCI)(nil)
@@ -45,8 +46,20 @@ func (o *OCI) AddIndex(desc ocispec.Descriptor) error {
if _, ok := desc.Annotations[ocispec.AnnotationRefName]; !ok {
return fmt.Errorf("descriptor must contain a reference from the annotation: %s", ocispec.AnnotationRefName)
}
key := fmt.Sprintf("%s-%s-%s", desc.Digest.String(), desc.Annotations[ocispec.AnnotationRefName], desc.Annotations[consts.KindAnnotationName])
o.nameMap.Store(key, desc)
key, err := reference.Parse(desc.Annotations[ocispec.AnnotationRefName])
if err != nil {
return err
}
if strings.TrimSpace(key.String()) != "--" {
switch key.(type) {
case name.Digest:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.Context().String(), desc.Annotations[consts.KindAnnotationName]), desc)
case name.Tag:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.String(), desc.Annotations[consts.KindAnnotationName]), desc)
}
}
return o.SaveIndex()
}
@@ -72,11 +85,21 @@ func (o *OCI) LoadIndex() error {
}
for _, desc := range o.index.Manifests {
key := fmt.Sprintf("%s-%s-%s", desc.Digest.String(), desc.Annotations[ocispec.AnnotationRefName], desc.Annotations[consts.KindAnnotationName])
if strings.TrimSpace(key) != "--" {
o.nameMap.Store(key, desc)
key, err := reference.Parse(desc.Annotations[ocispec.AnnotationRefName])
if err != nil {
return err
}
if strings.TrimSpace(key.String()) != "--" {
switch key.(type) {
case name.Digest:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.Context().String(), desc.Annotations[consts.KindAnnotationName]), desc)
case name.Tag:
o.nameMap.Store(fmt.Sprintf("%s-%s", key.String(), desc.Annotations[consts.KindAnnotationName]), desc)
}
}
}
return nil
}
@@ -180,9 +203,11 @@ func (o *OCI) Pusher(ctx context.Context, ref string) (remotes.Pusher, error) {
var baseRef, hash string
parts := strings.SplitN(ref, "@", 2)
baseRef = parts[0]
if len(parts) > 1 {
hash = parts[1]
}
return &ociPusher{
oci: o,
ref: baseRef,
@@ -275,7 +300,7 @@ func (p *ociPusher) Push(ctx context.Context, d ocispec.Descriptor) (ccontent.Wr
if _, err := os.Stat(blobPath); err == nil {
// file already exists, discard (but validate digest)
return content.NewIoContentWriter(ioutil.Discard, content.WithOutputHash(d.Digest)), nil
return content.NewIoContentWriter(io.Discard, content.WithOutputHash(d.Digest)), nil
}
f, err := os.Create(blobPath)

View File

@@ -1,21 +1,23 @@
package cosign
import (
"bufio"
"context"
"embed"
"fmt"
"os"
"os/exec"
"os/user"
"path/filepath"
"runtime"
"context"
"time"
"bufio"
"embed"
"strings"
"time"
"oras.land/oras-go/pkg/content"
"github.com/rancherfederal/hauler/pkg/store"
"github.com/rancherfederal/hauler/pkg/artifacts/image"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/store"
)
const maxRetries = 3
@@ -24,7 +26,7 @@ const retryDelay = time.Second * 5
// VerifyFileSignature verifies the digital signature of a file using Sigstore/Cosign.
func VerifySignature(ctx context.Context, s *store.Layout, keyPath string, ref string) error {
operation := func() error {
cosignBinaryPath, err := getCosignPath(ctx)
cosignBinaryPath, err := getCosignPath()
if err != nil {
return err
}
@@ -45,30 +47,62 @@ func VerifySignature(ctx context.Context, s *store.Layout, keyPath string, ref s
func SaveImage(ctx context.Context, s *store.Layout, ref string, platform string) error {
l := log.FromContext(ctx)
operation := func() error {
cosignBinaryPath, err := getCosignPath(ctx)
cosignBinaryPath, err := getCosignPath()
if err != nil {
return err
}
// check to see if the image is multi-arch
isMultiArch, err := image.IsMultiArchImage(ref)
if err != nil {
return err
}
l.Debugf("multi-arch image: %v", isMultiArch)
cmd := exec.Command(cosignBinaryPath, "save", ref, "--dir", s.Root)
// Conditionally add platform.
if platform != "" {
if platform != "" && isMultiArch {
l.Debugf("platform for image [%s]", platform)
cmd.Args = append(cmd.Args, "--platform", platform)
}
output, err := cmd.CombinedOutput()
stdout, err := cmd.StdoutPipe()
if err != nil {
if strings.Contains(string(output), "specified reference is not a multiarch image") {
l.Debugf(fmt.Sprintf("specified image [%s] is not a multiarch image. (choosing default)", ref))
// Rerun the command without the platform flag
cmd = exec.Command(cosignBinaryPath, "save", ref, "--dir", s.Root)
output, err = cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("error adding image to store: %v, output: %s", err, output)
}
} else {
return fmt.Errorf("error adding image to store: %v, output: %s", err, output)
}
return err
}
stderr, err := cmd.StderrPipe()
if err != nil {
return err
}
// start the command after having set up the pipe
if err := cmd.Start(); err != nil {
return err
}
// read command's stdout line by line
output := bufio.NewScanner(stdout)
for output.Scan() {
l.Debugf(output.Text()) // write each line to your log, or anything you need
}
if err := output.Err(); err != nil {
cmd.Wait()
return err
}
// read command's stderr line by line
errors := bufio.NewScanner(stderr)
for errors.Scan() {
l.Errorf(errors.Text()) // write each line to your log, or anything you need
}
if err := errors.Err(); err != nil {
cmd.Wait()
return err
}
// Wait for the command to finish
err = cmd.Wait()
if err != nil {
return err
}
return nil
@@ -81,7 +115,7 @@ func SaveImage(ctx context.Context, s *store.Layout, ref string, platform string
func LoadImages(ctx context.Context, s *store.Layout, registry string, ropts content.RegistryOptions) error {
l := log.FromContext(ctx)
cosignBinaryPath, err := getCosignPath(ctx)
cosignBinaryPath, err := getCosignPath()
if err != nil {
return err
}
@@ -108,7 +142,7 @@ func LoadImages(ctx context.Context, s *store.Layout, registry string, ropts con
if err := cmd.Start(); err != nil {
return err
}
// read command's stdout line by line
output := bufio.NewScanner(stdout)
for output.Scan() {
@@ -140,17 +174,18 @@ func LoadImages(ctx context.Context, s *store.Layout, registry string, ropts con
// RegistryLogin - performs cosign login
func RegistryLogin(ctx context.Context, s *store.Layout, registry string, ropts content.RegistryOptions) error {
cosignBinaryPath, err := getCosignPath(ctx)
log := log.FromContext(ctx)
cosignBinaryPath, err := getCosignPath()
if err != nil {
return err
}
cmd := exec.Command(cosignBinaryPath, "login", registry, "-u", ropts.Username, "-p", ropts.Password)
output, err := cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("error logging into registry: %v, output: %s", err, output)
}
log.Infof(strings.Trim(string(output), "\n"))
return nil
}
@@ -165,7 +200,7 @@ func RetryOperation(ctx context.Context, operation func() error) error {
}
// Log the error for the current attempt.
l.Errorf("Error (attempt %d/%d): %v", attempt, maxRetries, err)
l.Errorf("error (attempt %d/%d): %v", attempt, maxRetries, err)
// If this is not the last attempt, wait before retrying.
if attempt < maxRetries {
@@ -177,14 +212,13 @@ func RetryOperation(ctx context.Context, operation func() error) error {
return fmt.Errorf("operation failed after %d attempts", maxRetries)
}
func EnsureBinaryExists(ctx context.Context, bin embed.FS) (error) {
func EnsureBinaryExists(ctx context.Context, bin embed.FS) error {
// Set up a path for the binary to be copied.
binaryPath, err := getCosignPath(ctx)
binaryPath, err := getCosignPath()
if err != nil {
return fmt.Errorf("Error: %v\n", err)
return fmt.Errorf("error: %v", err)
}
// Determine the architecture so that we pull the correct embedded binary.
arch := runtime.GOARCH
rOS := runtime.GOOS
@@ -198,25 +232,24 @@ func EnsureBinaryExists(ctx context.Context, bin embed.FS) (error) {
// retrieve the embedded binary
f, err := bin.ReadFile(fmt.Sprintf("binaries/%s", binaryName))
if err != nil {
return fmt.Errorf("Error: %v\n", err)
return fmt.Errorf("error: %v", err)
}
// write the binary to the filesystem
err = os.WriteFile(binaryPath, f, 0755)
if err != nil {
return fmt.Errorf("Error: %v\n", err)
return fmt.Errorf("error: %v", err)
}
return nil
}
// getCosignPath returns the binary path
func getCosignPath(ctx context.Context) (string, error) {
func getCosignPath() (string, error) {
// Get the current user's information
currentUser, err := user.Current()
if err != nil {
return "", fmt.Errorf("Error: %v\n", err)
return "", fmt.Errorf("error: %v", err)
}
// Get the user's home directory
@@ -224,14 +257,14 @@ func getCosignPath(ctx context.Context) (string, error) {
// Construct the path to the .hauler directory
haulerDir := filepath.Join(homeDir, ".hauler")
// Create the .hauler directory if it doesn't exist
if _, err := os.Stat(haulerDir); os.IsNotExist(err) {
// .hauler directory does not exist, create it
if err := os.MkdirAll(haulerDir, 0755); err != nil {
return "", fmt.Errorf("Error creating .hauler directory: %v\n", err)
}
}
// Create the .hauler directory if it doesn't exist
if _, err := os.Stat(haulerDir); os.IsNotExist(err) {
// .hauler directory does not exist, create it
if err := os.MkdirAll(haulerDir, 0755); err != nil {
return "", fmt.Errorf("error creating .hauler directory: %v", err)
}
}
// Determine the binary name.
rOS := runtime.GOOS
@@ -241,7 +274,7 @@ func getCosignPath(ctx context.Context) (string, error) {
}
// construct path to binary
binaryPath := filepath.Join(haulerDir, binaryName)
binaryPath := filepath.Join(haulerDir, binaryName)
return binaryPath, nil
}
}

View File

@@ -30,7 +30,9 @@ type Fields map[string]string
// NewLogger returns a new Logger
func NewLogger(out io.Writer) Logger {
output := zerolog.ConsoleWriter{Out: os.Stdout}
customTimeFormat := "2006-01-02 15:04:05"
zerolog.TimeFieldFormat = customTimeFormat
output := zerolog.ConsoleWriter{Out: os.Stdout, TimeFormat: customTimeFormat}
l := log.Output(output)
return &logger{
zl: l.With().Timestamp().Logger(),

View File

@@ -1,7 +1,7 @@
// Package reference provides general types to represent oci content within a registry or local oci layout
// Grammar (stolen mostly from containerd's grammar)
//
// reference :=
// reference :=
package reference
import (
@@ -25,6 +25,7 @@ type Reference interface {
// NewTagged will create a new docker.NamedTagged given a path-component
func NewTagged(n string, tag string) (gname.Reference, error) {
n = strings.Replace(strings.ToLower(n), "+", "-", -1)
repo, err := Parse(n)
if err != nil {
return nil, err

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

View File

@@ -0,0 +1,38 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- Generator: Adobe Illustrator 28.3.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="358.739px" height="301.861px" viewBox="0 0 358.739 301.861" enable-background="new 0 0 358.739 301.861"
xml:space="preserve">
<path fill="#F17203" d="M317.817,295.13H40.925c-19.036,0-34.612-15.575-34.612-34.612V44.196c0-19.036,15.575-34.612,34.612-34.612
h276.892c19.036,0,34.612,15.575,34.612,34.612v216.322C352.428,279.555,336.853,295.13,317.817,295.13z"/>
<g>
<path fill="#FFFFFF" d="M299.929,116.41h2.311c-5.574,0-10.366-3.95-11.431-9.422l-1.704-8.759
c-2.178-11.2-11.988-19.286-23.399-19.286h-33.5c-0.246,0-0.484,0.031-0.729,0.037v-2.649c0-17.332-14.051-31.383-31.383-31.383
H75.91c-17.333,0-31.383,14.051-31.383,31.383v52.419c0,0.254,0.032,0.499,0.038,0.752c-8.418,0.062-15.226,6.895-15.226,15.327
c0,8.472,6.867,15.339,15.339,15.339h31.175c0.119,0,0.229-0.032,0.347-0.035h25.26c21.78,0,41.302,13.571,48.841,34.004
l72.408,0.279c1.713,0.544,3.496,0.919,5.322,1.162c-2.776,4.551-4.824,9.589-5.9,14.987h-54.474
c-8.472,0-15.339,6.867-15.339,15.339c0,8.472,6.867,15.339,15.339,15.339h58.919c7.786,14.77,23.277,24.875,41.102,24.875
c25.615,0,46.454-20.839,46.454-46.455c0-9.042-2.637-17.461-7.124-24.607c13.918-3.213,24.305-15.657,24.305-30.552v-16.712
C331.312,130.461,317.262,116.41,299.929,116.41z M232.207,109.622h27.86l0.627,3.224c3.711,19.082,20.013,33.132,39.234,34.181
v0.063c0.388,0,0.705,0.316,0.705,0.704v16.712c0,0.388-0.316,0.704-0.705,0.704h-67.722c-0.35,0-0.609-0.271-0.663-0.605
c-0.014-0.326-0.036-0.649-0.041-0.978v-53.3C231.503,109.938,231.819,109.622,232.207,109.622z M168.358,163.528
c-15.333-20.984-40.12-34.074-66.899-34.074H75.91c-0.388,0-0.704-0.316-0.704-0.704V76.331c0-0.388,0.316-0.704,0.704-0.704
h124.186c0.388,0,0.704,0.316,0.704,0.704v86.717c0,0.202,0.001,0.404,0.003,0.605L168.358,163.528z M267.677,233.009
c-7.359,0-13.345-5.987-13.345-13.345s5.987-13.345,13.345-13.345s13.345,5.987,13.345,13.345S275.036,233.009,267.677,233.009z"/>
<path fill="#FFFFFF" d="M97.53,173.209c-25.615,0-46.454,20.839-46.454,46.454c0,25.615,20.839,46.455,46.454,46.455
s46.454-20.839,46.454-46.455C143.985,194.049,123.146,173.209,97.53,173.209z M97.53,233.009c-7.359,0-13.345-5.987-13.345-13.345
s5.987-13.345,13.345-13.345s13.345,5.987,13.345,13.345S104.889,233.009,97.53,233.009z"/>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 2.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

View File

@@ -0,0 +1,52 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- Generator: Adobe Illustrator 28.3.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="477.791px" height="498.742px" viewBox="0 0 477.791 498.742" enable-background="new 0 0 477.791 498.742"
xml:space="preserve">
<path fill="#F17203" d="M375.348,301.805H98.456c-19.036,0-34.612-15.575-34.612-34.612V50.871
c0-19.036,15.575-34.612,34.612-34.612h276.892c19.036,0,34.612,15.575,34.612,34.612v216.322
C409.96,286.23,394.385,301.805,375.348,301.805z"/>
<g>
<path fill="#FFFFFF" d="M357.46,123.085h2.311c-5.574,0-10.366-3.95-11.431-9.422l-1.704-8.759
c-2.178-11.2-11.988-19.286-23.399-19.286h-33.5c-0.246,0-0.484,0.031-0.729,0.037v-2.649c0-17.332-14.051-31.383-31.383-31.383
H133.441c-17.333,0-31.383,14.051-31.383,31.383v52.419c0,0.254,0.032,0.499,0.038,0.752c-8.418,0.062-15.226,6.895-15.226,15.327
c0,8.472,6.867,15.339,15.339,15.339h31.175c0.119,0,0.229-0.032,0.347-0.035h25.26c21.78,0,41.302,13.571,48.841,34.004
l72.408,0.279c1.713,0.544,3.496,0.919,5.322,1.162c-2.776,4.551-4.824,9.589-5.9,14.987h-54.474
c-8.472,0-15.339,6.867-15.339,15.339c0,8.472,6.867,15.339,15.339,15.339h58.919c7.786,14.77,23.277,24.875,41.102,24.875
c25.615,0,46.454-20.839,46.454-46.455c0-9.042-2.637-17.461-7.124-24.607c13.918-3.213,24.305-15.657,24.305-30.552v-16.712
C388.844,137.136,374.793,123.085,357.46,123.085z M289.739,116.297h27.86l0.627,3.224c3.711,19.082,20.013,33.132,39.234,34.181
v0.063c0.388,0,0.705,0.316,0.705,0.704v16.712c0,0.388-0.316,0.704-0.705,0.704h-67.722c-0.35,0-0.609-0.271-0.663-0.605
c-0.014-0.326-0.036-0.649-0.041-0.978v-53.3C289.034,116.613,289.351,116.297,289.739,116.297z M225.889,170.203
c-15.333-20.984-40.12-34.074-66.899-34.074h-25.549c-0.388,0-0.704-0.316-0.704-0.704V83.006c0-0.388,0.316-0.704,0.704-0.704
h124.186c0.388,0,0.704,0.316,0.704,0.704v86.717c0,0.202,0.001,0.404,0.003,0.605L225.889,170.203z M325.209,239.684
c-7.359,0-13.345-5.987-13.345-13.345s5.987-13.345,13.345-13.345s13.345,5.987,13.345,13.345S332.567,239.684,325.209,239.684z"/>
<path fill="#FFFFFF" d="M155.062,179.884c-25.615,0-46.454,20.839-46.454,46.454c0,25.615,20.839,46.455,46.454,46.455
s46.454-20.839,46.454-46.455C201.516,200.724,180.677,179.884,155.062,179.884z M155.062,239.684
c-7.359,0-13.345-5.987-13.345-13.345s5.987-13.345,13.345-13.345s13.345,5.987,13.345,13.345S162.42,239.684,155.062,239.684z"/>
</g>
<path fill="#384745" d="M173.809,393.336h21.499v55.743c0,11.072,4.635,15.707,13.518,15.707c8.755,0,14.033-4.505,14.033-16.093
v-55.358h20.855v54.972c0,19.955-13.002,33.473-35.531,33.473c-21.114,0-34.374-12.359-34.374-31.541V393.336z"/>
<path fill="#384745" d="M117.429,393.336h21.757l29.353,86.898h-21.757l-5.408-16.479h-29.095l-5.276,16.479H88.206L117.429,393.336
z M117.429,447.793h18.666l-5.922-18.281c-2.06-6.179-2.961-14.932-2.961-14.932h-0.515c0,0-1.287,8.881-3.217,14.802
L117.429,447.793z"/>
<path fill="#384745" d="M329.804,393.336h57.547v15.963h-36.048v18.668h30.254v15.963h-30.254v20.34h37.335v15.963h-58.834V393.336z
"/>
<path fill="#384745" d="M399.84,393.336h35.79c16.994,0,28.452,8.239,28.452,24.202c0,13.004-8.885,20.858-17.51,23.561
c2.575,2.188,4.505,5.149,6.052,8.239c3.604,7.338,6.05,15.448,13.646,15.448c1.932,0,3.476-0.643,3.476-0.643l-1.674,15.32
c0,0-4.635,1.159-8.625,1.159c-10.299,0-16.221-3.992-22.273-17.767c-2.575-6.177-6.179-16.994-10.942-16.994h-4.893v34.374H399.84
V393.336z M421.339,408.914v21.369h7.724c6.182,0,13.39-1.93,13.39-11.198c0-7.596-4.893-10.171-10.815-10.171H421.339z"/>
<path fill="#384745" d="M259.227,393.072h21.565v71.151h36.286v16.012h-57.851V393.072z"/>
<path fill="#384745" d="M9.837,393.072h21.565v32.927h26.473v-32.927h21.565v87.163H57.874v-36.929H31.402v36.929H9.837V393.072z"/>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.8 KiB

BIN
static/rgs-hauler-logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

View File

@@ -0,0 +1,51 @@
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- Generator: Adobe Illustrator 28.3.0, SVG Export Plug-In . SVG Version: 6.00 Build 0) -->
<svg version="1.1" id="Layer_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
width="633.948px" height="187.484px" viewBox="0 0 633.948 187.484" enable-background="new 0 0 633.948 187.484"
xml:space="preserve">
<path fill="#384745" d="M378.324,56.272h17.803v46.159c0,9.169,3.838,13.007,11.194,13.007c7.25,0,11.62-3.731,11.62-13.326v-45.84
h17.269v45.52c0,16.524-10.767,27.718-29.422,27.718c-17.484,0-28.464-10.234-28.464-26.118V56.272z"/>
<path fill="#384745" d="M331.638,56.272h18.016l24.306,71.957h-18.016l-4.478-13.645h-24.093l-4.369,13.645H307.44L331.638,56.272z
M331.638,101.366h15.457l-4.904-15.138c-1.706-5.117-2.452-12.365-2.452-12.365h-0.427c0,0-1.065,7.354-2.663,12.257
L331.638,101.366z"/>
<path fill="#384745" d="M507.499,56.272h47.653v13.219h-29.85v15.459h25.052v13.219h-25.052v16.843h30.916v13.219h-48.718V56.272z"
/>
<path fill="#384745" d="M565.493,56.272h29.637c14.072,0,23.56,6.823,23.56,20.041c0,10.768-7.357,17.272-14.499,19.51
c2.133,1.812,3.731,4.263,5.011,6.823c2.984,6.077,5.009,12.792,11.299,12.792c1.6,0,2.879-0.533,2.879-0.533l-1.386,12.686
c0,0-3.838,0.96-7.142,0.96c-8.528,0-13.432-3.305-18.443-14.713c-2.133-5.115-5.117-14.072-9.061-14.072h-4.052v28.464h-17.803
V56.272z M583.296,69.172v17.695h6.396c5.119,0,11.088-1.598,11.088-9.273c0-6.29-4.052-8.423-8.955-8.423H583.296z"/>
<path fill="#384745" d="M449.056,56.053h17.857v58.918h30.047v13.259h-47.904V56.053z"/>
<path fill="#384745" d="M242.545,56.053h17.857v27.266h21.921V56.053h17.857v72.177h-17.857V97.65h-21.921v30.58h-17.857V56.053z"/>
<path fill="#F17203" d="M183.245,172.234H29.368c-10.579,0-19.235-8.656-19.235-19.235V32.783c0-10.579,8.656-19.235,19.235-19.235
h153.877c10.579,0,19.235,8.656,19.235,19.235V153C202.48,163.579,193.825,172.234,183.245,172.234z"/>
<g>
<path fill="#FFFFFF" d="M173.305,72.915h1.284c-3.098,0-5.761-2.195-6.352-5.236l-0.947-4.868
c-1.21-6.224-6.662-10.718-13.003-10.718H135.67c-0.137,0-0.269,0.017-0.405,0.02v-1.472c0-9.632-7.808-17.441-17.441-17.441H48.81
c-9.632,0-17.441,7.808-17.441,17.441v29.131c0,0.141,0.018,0.278,0.021,0.418c-4.678,0.035-8.462,3.832-8.462,8.518
c0,4.708,3.816,8.524,8.524,8.524h17.325c0.066,0,0.127-0.018,0.193-0.019h14.038c12.104,0,22.953,7.542,27.143,18.897
l40.239,0.155c0.952,0.302,1.943,0.511,2.958,0.646c-1.542,2.529-2.681,5.329-3.279,8.328H99.797c-4.708,0-8.524,3.816-8.524,8.524
s3.816,8.524,8.524,8.524h32.743c4.327,8.208,12.936,13.824,22.842,13.824c14.235,0,25.816-11.581,25.816-25.816
c0-5.025-1.466-9.704-3.959-13.675c7.735-1.786,13.507-8.701,13.507-16.979v-9.287C190.745,80.723,182.937,72.915,173.305,72.915z
M135.67,69.142h15.483l0.349,1.791c2.062,10.604,11.122,18.412,21.804,18.995v0.035c0.216,0,0.392,0.176,0.392,0.391v9.287
c0,0.216-0.176,0.391-0.392,0.391H135.67c-0.194,0-0.339-0.15-0.369-0.336c-0.008-0.181-0.02-0.361-0.023-0.544v-29.62
C135.278,69.317,135.454,69.142,135.67,69.142z M100.187,99.099c-8.521-11.661-22.296-18.936-37.178-18.936H48.81
c-0.216,0-0.391-0.176-0.391-0.391V50.641c0-0.216,0.176-0.391,0.391-0.391h69.014c0.216,0,0.391,0.176,0.391,0.391v48.191
c0,0.112,0,0.224,0.001,0.336L100.187,99.099z M155.382,137.712c-4.089,0-7.416-3.327-7.416-7.416c0-4.089,3.327-7.416,7.416-7.416
c4.089,0,7.416,3.327,7.416,7.416C162.798,134.385,159.471,137.712,155.382,137.712z"/>
<path fill="#FFFFFF" d="M60.826,104.479c-14.235,0-25.816,11.581-25.816,25.816c0,14.235,11.581,25.816,25.816,25.816
s25.816-11.581,25.816-25.816C86.642,116.061,75.061,104.479,60.826,104.479z M60.826,137.712c-4.089,0-7.416-3.327-7.416-7.416
c0-4.089,3.327-7.416,7.416-7.416s7.416,3.327,7.416,7.416C68.242,134.385,64.915,137.712,60.826,137.712z"/>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
<g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.8 KiB

View File

@@ -1,23 +0,0 @@
---
apiVersion: collection.hauler.cattle.io/v1alpha1
kind: ThickCharts
metadata:
name: mythickchart
spec:
charts:
# charts are also fetched and served as OCI content (currently experimental in helm)
# HELM_EXPERIMENTAL_OCI=1 helm chart pull <hauler-registry>/loki:2.6.2
# - name: loki
# repoURL: https://grafana.github.io/helm-charts
# - name: longhorn
# repoURL: https://charts.longhorn.io
# - name: cert-manager
# repoURL: https://charts.jetstack.io
# version: v1.6.1
# extraImages:
# - ref: quay.io/jetstack/cert-manager-cainjector:v1.6.1
- name: podinfo
repoURL: https://stefanprodan.github.io/podinfo

View File

@@ -1,56 +0,0 @@
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
name: myfile
spec:
files:
# hauler can save/redistribute files on disk (be careful! paths are relative)
- path: testdata/contents.yaml
# when directories are specified, the directory contents will be archived and stored
- path: testdata/
# hauler can also fetch remote content, and will "smartly" identify filenames _when possible_
# filename below = "k3s-images.txt"
- path: "https://github.com/k3s-io/k3s/releases/download/v1.22.2%2Bk3s2/k3s-images.txt"
# when discovered filenames are not desired, a file name can be specified
- path: https://get.k3s.io
name: k3s-init.sh
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Images
metadata:
name: myimage
spec:
images:
# images can be referenced shorthanded without a tag
- name: hello-world
# or namespaced with a tag
- name: rancher/cowsay:latest
# or by their digest:
- name: registry@sha256:42043edfae481178f07aa077fa872fcc242e276d302f4ac2026d9d2eb65b955f
# or fully qualified from any OCI compliant registry registry
- name: ghcr.io/fluxcd/flux-cli:v0.22.0
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Charts
metadata:
name: mychart
spec:
charts:
# charts are also fetched and served as OCI content (currently experimental in helm)
# HELM_EXPERIMENTAL_OCI=1 helm chart pull <hauler-registry>/loki:2.6.2
- name: loki
repoURL: https://grafana.github.io/helm-charts
# version: latest # the latest version will be used when version is empty
# specific versions can also be used
- name: rancher
repoURL: https://releases.rancher.com/server-charts/latest
version: 2.6.2

BIN
testdata/haul.tar.zst vendored Executable file

Binary file not shown.

26
testdata/hauler-manifest.yaml vendored Executable file
View File

@@ -0,0 +1,26 @@
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Images
metadata:
name: hauler-content-images-example
spec:
images:
- name: busybox:latest
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Charts
metadata:
name: hauler-content-charts-example
spec:
charts:
- name: rancher
repoURL: https://releases.rancher.com/server-charts/stable
version: 2.8.2
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Files
metadata:
name: hauler-content-files-example
spec:
files:
- path: https://get.rke2.io
name: install.sh

View File

@@ -1,13 +0,0 @@
---
apiVersion: collection.hauler.cattle.io/v1alpha1
kind: K3s
metadata:
name: myk3s
spec:
# version can be exact (as listed on https://github.com/k3s-io/k3s/releases)
version: v1.22.2+k3s2
# or can point to a channel, in which case the latest version in the channel is used: https://update.k3s.io/v1-release/channels
# version: stable
# version: latest
# version: v1.22

Binary file not shown.

BIN
testdata/rancher-cluster-templates-0.4.4.tgz vendored Executable file

Binary file not shown.

View File

@@ -1,25 +0,0 @@
#!/bin/sh
set -x
if [ "$#" -ne 1 ] || ( [ "$1" != "internet" ] && [ "$1" != "airgap" ] ); then
echo \
"Enable or disable internet access in hauler's CentOS Vagrant machine.
Usage: $0 internet
$0 airgap" >&2
exit 1
fi
if [ "$1" = "internet" ]; then
# internet: set default gateway to NAT network interface
default_iface="eth0"
gw_ip="10.0.2.2"
else
# airgap: set default gateway to private network interface
default_iface="eth1"
gw_ip=$(ip -f inet a show "${default_iface}" | awk 'match($0, /inet ([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})/, arr) { print arr[1] }')
fi
ip r delete default
ip r add default via ${gw_ip} dev ${default_iface} proto dhcp metric 100

View File

@@ -1,41 +0,0 @@
#!/bin/sh
################################################################################
# RUN IN VAGRANT MACHINE
# Install a default, bare k3s cluster into the Vagrant machine
################################################################################
if [ -f "/usr/local/bin/k3s-uninstall.sh" ]; then
/usr/local/bin/k3s-uninstall.sh
else
echo "k3s is not installed"
fi
if pgrep -x "firewalld" >/dev/null
then
echo "[FATAL] disable firewalld first"
fi
SELINUXSTATUS=$(getenforce)
if [ "$SELINUXSTATUS" == "Permissive" ]; then
echo "[FATAL] disable selinux"
exit 1
else
echo "SELINUX disabled. continuing"
fi
LOCAL_IMAGES_FILEPATH=/var/lib/rancher/k3s/agent/images
ARTIFACT_DIR=/opt/hauler/local-artifacts/k3s
mkdir -p ${LOCAL_IMAGES_FILEPATH}
cp ${ARTIFACT_DIR}/images/* ${LOCAL_IMAGES_FILEPATH}
cp ${ARTIFACT_DIR}/bin/k3s /usr/local/bin/k3s
chmod +x /usr/local/bin/k3s
yum install -y ${ARTIFACT_DIR}/rpm/*
INSTALL_K3S_SKIP_DOWNLOAD=true ${ARTIFACT_DIR}/bin/k3s-install.sh
chmod +r /etc/rancher/k3s/k3s.yaml

View File

@@ -1,50 +0,0 @@
#!/bin/sh
################################################################################
# RUN IN VAGRANT MACHINE
# Download all required dependencies for an air-gapped k3s install, saving them
# to the folder shared with the host machine.
################################################################################
BASE_SHARED_DIR="/opt/hauler"
VAGRANT_SCRIPTS_DIR="${BASE_SHARED_DIR}/vagrant-scripts"
ARTIFACTS_DIR="${BASE_SHARED_DIR}/local-artifacts/k3s"
K3S_VERSION='v1.18.8+k3s1'
K3S_VERSION_URL='v1.18.8%2Bk3s1'
LOCAL_IMAGES="${ARTIFACTS_DIR}/images"
LOCAL_BIN="${ARTIFACTS_DIR}/bin"
LOCAL_RPM="${ARTIFACTS_DIR}/rpm"
mkdir -p ${LOCAL_IMAGES}
mkdir -p ${LOCAL_BIN}
mkdir -p ${LOCAL_RPM}
# temporarily allow internet access
${VAGRANT_SCRIPTS_DIR}/airgap.sh internet
pushd ${LOCAL_IMAGES}
curl -LO https://github.com/rancher/k3s/releases/download/${K3S_VERSION_URL}/k3s-airgap-images-amd64.tar
popd
pushd ${LOCAL_BIN}
curl -LO https://github.com/rancher/k3s/releases/download/${K3S_VERSION_URL}/k3s
curl -L https://raw.githubusercontent.com/rancher/k3s/${K3S_VERSION_URL}/install.sh -o k3s-install.sh
chmod +x ./*
popd
pushd ${LOCAL_RPM}
curl -LO https://rpm.rancher.io/k3s-selinux-0.1.1-rc1.el7.noarch.rpm
yum install -y yum-utils
yumdownloader --destdir=. --resolve container-selinux selinux-policy-base
popd
# restore air-gap configuration
${VAGRANT_SCRIPTS_DIR}/airgap.sh airgap

View File

@@ -1,12 +0,0 @@
#!/bin/sh
BASE_SHARED_DIR="/opt/hauler"
VAGRANT_SCRIPTS_DIR="${BASE_SHARED_DIR}/vagrant-scripts"
for script in ${VAGRANT_SCRIPTS_DIR}/*-prep.sh ; do
echo "---"
echo "Running ${script} ..."
echo "---"
sh "${script}"
done

View File

@@ -1,73 +0,0 @@
#!/bin/sh
################################################################################
# RUN IN VAGRANT MACHINE
# Install a default, bare rke2 cluster into the Vagrant machine
################################################################################
BASE_SHARED_DIR="/opt/hauler"
VAGRANT_SCRIPTS_DIR="${BASE_SHARED_DIR}/vagrant-scripts"
RKE2_VERSION_DOCKER='v1.18.4-beta16-rke2'
if pgrep -x "firewalld" >/dev/null
then
echo "[FATAL] disable firewalld first"
fi
mkdir -p /etc/rancher/rke2/
# TODO - allow using selinux
SELINUXSTATUS="$(getenforce)"
if [ "$SELINUXSTATUS" = "Permissive" ] || [ "$SELINUXSTATUS" = "Enforcing" ]
then
echo "selinux: true" | sudo tee -a /etc/rancher/rke2/config.yaml > /dev/null
else
echo "SELINUX disabled. continuing"
fi
LOCAL_IMAGES_FILEPATH=/var/lib/rancher/rke2/agent/images
ARTIFACT_DIR="${BASE_SHARED_DIR}/local-artifacts/rke2"
mkdir -p ${LOCAL_IMAGES_FILEPATH}
cp ${ARTIFACT_DIR}/images/* ${LOCAL_IMAGES_FILEPATH}
# TODO - add ability to use local binary with yum install
# ----------------------------------------------------------
# uncomment to use a specific local binary for the install
# ----------------------------------------------------------
# LOCAL_RKE2_BIN='rke2-beta13-dev'
#if [ -n "${LOCAL_RKE2_BIN}" ] && [ -f "${ARTIFACT_DIR}/bin/${LOCAL_RKE2_BIN}" ] ; then
# echo "Use "${ARTIFACT_DIR}/bin/${LOCAL_RKE2_BIN}" for rke2 binary"
#
# INSTALL_RKE2_SKIP_START=true \
# RKE2_RUNTIME_IMAGE="rancher/rke2-runtime:${RKE2_VERSION_DOCKER}" \
# ${ARTIFACT_DIR}/bin/rke2-installer.run
#
# rm -f /usr/local/bin/rke2
#
# cp "${ARTIFACT_DIR}/bin/${LOCAL_RKE2_BIN}" /usr/local/bin/rke2
#
# systemctl start rke2
#else
# ${ARTIFACT_DIR}/bin/rke2-installer.run
#fi
yum install -y ${ARTIFACT_DIR}/rpm/*
systemctl enable rke2-server && systemctl start rke2-server
while [ -f "/etc/rancher/rke2/rke2.yaml" ] ; do
echo "Waiting for /etc/rancher/rke2/rke2.yaml to exist..."
sleep 10
done
chmod +r /etc/rancher/rke2/rke2.yaml
echo "RKE2 cluster is wrapping up installation, run the following commands to allow kubectl access:
export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
export PATH=/var/lib/rancher/rke2/bin/:\${PATH}"

View File

@@ -1,72 +0,0 @@
#!/bin/sh
################################################################################
# RUN IN VAGRANT MACHINE
# Download all required dependencies for an air-gapped rke2 install, saving them
# to the folder shared with the host machine.
################################################################################
BASE_SHARED_DIR="/opt/hauler"
VAGRANT_SCRIPTS_DIR="${BASE_SHARED_DIR}/vagrant-scripts"
ARTIFACTS_DIR="${BASE_SHARED_DIR}/local-artifacts/rke2"
RKE2_VERSION='v1.18.13+rke2r1'
RKE2_VERSION_URL='v1.18.13%2Brke2r1'
RKE2_VERSION_DOCKER='v1.18.13-rke2r1'
LOCAL_IMAGES="${ARTIFACTS_DIR}/images"
LOCAL_BIN="${ARTIFACTS_DIR}/bin"
LOCAL_RPM="${ARTIFACTS_DIR}/rpm"
mkdir -p ${LOCAL_IMAGES}
mkdir -p ${LOCAL_BIN}
mkdir -p ${LOCAL_RPM}
# temporarily allow internet access
${VAGRANT_SCRIPTS_DIR}/airgap.sh internet
pushd ${LOCAL_IMAGES}
curl -LO https://github.com/rancher/rke2/releases/download/${RKE2_VERSION_URL}/rke2-images.linux-amd64.tar.gz
gunzip rke2-images.linux-amd64.tar.gz
popd
#pushd ${LOCAL_BIN}
#
#curl -L https://github.com/rancher/rke2/releases/download/${RKE2_VERSION_URL}/rke2-installer.linux-amd64.run -o rke2-installer.run
#chmod +x ./*
#
#popd
pushd ${LOCAL_RPM}
yum install -y yum-plugin-downloadonly
rke2_rpm_channel='stable'
rpm_site='rpm.rancher.io'
maj_ver='7'
rke2_majmin=$(echo "${RKE2_VERSION}" | sed -E -e "s/^v([0-9]+\.[0-9]+).*/\1/")
cat <<-EOF >"/etc/yum.repos.d/rancher-rke2.repo"
[rancher-rke2-common-${rke2_rpm_channel}]
name=Rancher RKE2 Common (${RKE2_VERSION})
baseurl=https://${rpm_site}/rke2/${rke2_rpm_channel}/common/centos/${maj_ver}/noarch
enabled=1
gpgcheck=1
gpgkey=https://${rpm_site}/public.key
[rancher-rke2-${rke2_majmin}-${rke2_rpm_channel}]
name=Rancher RKE2 ${rke2_majmin} (${RKE2_VERSION})
baseurl=https://${rpm_site}/rke2/${rke2_rpm_channel}/${rke2_majmin}/centos/${maj_ver}/x86_64
enabled=1
gpgcheck=1
gpgkey=https://${rpm_site}/public.key
EOF
yum install --downloadonly --downloaddir=./ rke2-server
popd
# restore air-gap configuration
${VAGRANT_SCRIPTS_DIR}/airgap.sh airgap