Merge pull request #22 from rancherfederal/dev

initial alpha with functional packaging/relocating
This commit is contained in:
joshrwolf
2021-06-18 07:16:52 -06:00
committed by GitHub
45 changed files with 2084 additions and 1268 deletions

1
.gitignore vendored
View File

@@ -24,3 +24,4 @@ dist/
./bundle/
tmp/
bin/
pkg.yaml

View File

@@ -1,6 +1,6 @@
project_name: hauler
builds:
- main: cmd/haulerctl/main.go
- main: cmd/hauler/main.go
goos:
- linux
- darwin

111
README.md
View File

@@ -1,65 +1,70 @@
# Hauler - Kubernetes Air Gap Migration
```bash
# _ _
# | |__ __ _ _ _| | ___ _ __
# | '_ \ / _` | | | | |/ _ \ '__|
# | | | | (_| | |_| | | __/ |
# |_| |_|\__,_|\__,_|_|\___|_|
#
# , , _______________________________
# ,-----------|'------'| | |
# /. '-' |-' |_____________________________|
# |/| | |
# | .________.'----' _______________________________
# | || | || | |
# \__|' \__|' |_____________________________|
#
# __________________________________________________________
# | |
# |________________________________________________________|
#
# __________________________________________________________
# | |
# |________________________________________________________|
```
## WARNING - Work In Progress
Hauler is a tool designed to ease the burden of working with containers and kubernetes in an airgap. Several components of hauler are used in unison to provide airgap utilities.
Hauler's utility is split into a few commands intended to solve increasingly complex airgapped use cases.
__Portable self contained clusters__:
Within the `hauler package` subset of commands, `Packages` (name to be finalized) can be created, updated, and ran.
A `Package` is a hauler specific, configurable, self-contained, compressed archive (`*.tar.zst`) that contains all dependencies needed to 1) create a kubernetes cluster, 2) deploy resources into the cluster.
```bash
# Build a minimal portable k8s cluster
hauler package build
# Build a package that deploys resources when deployed
hauler package build -p path/to/chart -p path/to/manifests -i extra/image:latest -i busybox:musl
# Build a package that deploys a cluster, oci registry, and sample app on boot
# Note the aliases introduced
hauler pkg b -p testdata/docker-registry -p testdata/rawmanifests
```
Hauler packages at their core stand on the shoulders of other technologies (`k3s`, `rke2`, and `fleet`), and as such, are designed to be extremely flexible.
Common use cases are to build turn key, appliance like clusters designed to boot on disconnected or low powered devices. Or portable "utility" clusters that can act as a stepping stone for further downstream deployable infrastructure. Since ever `Package` is built as an entirely self contained archive, disconnected environments are _always_ a first class citizen.
__Image Relocation__:
For disconnected workloads that don't require a cluster to be created first, images can be efficiently packaged and relocated with `hauler relocate`.
Images are stored as a compressed archive of an `oci` layout, ensuring only the required de-duplicated image layers are packaged and transferred.
## Installation
Hauler is and will always be a statically compiled binary, we strongly believe in a zero dependency tool is key to reducing operational complexity in airgap environments.
Before GA, hauler can be downloaded from the releases page for every tagged release
## Dev
A `Vagrant` file is provided as a testing ground. The boot scripts at `vagrant-scripts/*.sh` will be ran on boot to ensure the dev environment is airgapped.
```bash
vagrant up
vagrant ssh
```
More info can be found in the [vagrant docs](VAGRANT.md).
## WIP Warnings
API stability (including as a code library and as a network endpoint) is NOT guaranteed before `v1` API definitions and a 1.0 release. The following recommendations are made regarding usage patterns of hauler:
- `alpha` (`v1alpha1`, `v1alpha2`, ...) API versions: use **_only_** through `haulerctl`
- `beta` (`v1beta1`, `v1beta2`, ...) API versions: use as an **_experimental_** library and/or API endpoint
- `stable` (`v1`, `v2`, ...) API versions: use as stable CLI tool, library, and/or API endpoint
## Purpose: collect, transfer, and self-host cloud-native artifacts
Kubernetes-focused software usually relies on executables, archives, container images, helm charts, and more for installation. Collecting the dependencies, standing up tools for serving these artifacts, and mirroring them into the self-hosting solutions is usually a manual process with minimal automation.
Hauler aims to fill this gap by standardizing low-level components of this stack and automating the collection and transfer of artifacts.
## Additional Details
- [Roadmap](./ROADMAP.md)
- [Vagrant](./VAGRANT.md)
## Go CLI
The initial MVP for a hauler CLI used to streamline the packaging and deploying processes is in the `cmd/` and `pkg/` folders, along with `go.mod` and `go.sum`. Currently only a `package` subcommand is supported, which generates a `.tar.gz` archive used in the future `deploy` subcommand.
### Build
To build hauler, the Go CLI v1.14 or higher is required. See <https://golang.org/dl/> for downloads and see <https://golang.org/doc/install> for installation instructions.
```bash
# Current arch build
make build
To build hauler for your local machine (usually for the `package` step), run the following:
```shell
mkdir bin
go build -o bin ./cmd/...
```
To build hauler for linux amd64 (required for the `deploy` step in an air-gapped environment), run the following:
```shell
mkdir bin-linux-amd64
GOOS=linux GOARCH=amd64 go build -o bin-linux-amd64 ./cmd/...
```
# Multiarch dev build
goreleaser build --rm-dist --snapshot
```

View File

@@ -4,25 +4,38 @@ import (
"context"
"github.com/rancherfederal/hauler/pkg/oci"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
var (
copyLong = `hauler copies artifacts stored on a registry to local disk`
copyExample = `
# Run Hauler
hauler copy locahost:5000/artifacts:latest
`
)
type copyOpts struct {
*rootOpts
dir string
sourceRef string
}
// NewCopyCommand creates a new sub command under
// haulerctl for coping files to local disk
// hauler for coping files to local disk
func NewCopyCommand() *cobra.Command {
opts := &copyOpts{}
opts := &copyOpts{
rootOpts: &ro,
}
cmd := &cobra.Command{
Use: "copy",
Short: "Download artifacts from OCI registry to local disk",
Long: copyLong,
Example: copyExample,
Aliases: []string{"c", "cp"},
//Args: cobra.MinimumNArgs(1),
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
opts.sourceRef = args[0]
return opts.Run(opts.sourceRef)
@@ -41,7 +54,7 @@ func (o *copyOpts) Run(src string) error {
defer cancel()
if err := oci.Get(ctx, src, o.dir); err != nil {
logrus.Error(err)
o.logger.Errorf("error copy artifact %s to local directory %s: %v", src, o.dir, err)
}
return nil

25
cmd/hauler/app/pkg.go Normal file
View File

@@ -0,0 +1,25 @@
package app
import "github.com/spf13/cobra"
type pkgOpts struct{}
func NewPkgCommand() *cobra.Command {
opts := &pkgOpts{}
//TODO
_ = opts
cmd := &cobra.Command{
Use: "pkg",
Short: "Interact with packages",
Aliases: []string{"p", "package"},
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
}
cmd.AddCommand(NewPkgBuildCommand())
cmd.AddCommand(NewPkgRunCommand())
return cmd
}

202
cmd/hauler/app/pkg_build.go Normal file
View File

@@ -0,0 +1,202 @@
package app
import (
"context"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/driver"
"github.com/rancherfederal/hauler/pkg/packager"
"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"os"
"sigs.k8s.io/yaml"
)
type pkgBuildOpts struct {
*rootOpts
cfgFile string
name string
dir string
driver string
driverVersion string
fleetVersion string
images []string
paths []string
}
func NewPkgBuildCommand() *cobra.Command {
opts := pkgBuildOpts{
rootOpts: &ro,
}
cmd := &cobra.Command{
Use: "build",
Short: "Build a self contained compressed archive of manifests and images",
Long: `
Compressed archives created with this command can be extracted and run anywhere the underlying 'driver' can be run.
Archives are built by collecting all the dependencies (images and manifests) required.
Examples:
# Build a package containing a helm chart with images autodetected from the generated helm chart
hauler package build -p path/to/helm/chart
# Build a package, sourcing from multiple manifest sources and additional images not autodetected
hauler pkg build -p path/to/raw/manifests -p path/to/kustomize -i busybox:latest -i busybox:musl
# Build a package using a different version of k3s
hauler p build -p path/to/chart --driver-version "v1.20.6+k3s1"
# Build a package from a config file (if ./pkg.yaml does not exist, one will be created)
hauler package build -c ./pkg.yaml
`,
Aliases: []string{"b"},
PreRunE: func(cmd *cobra.Command, args []string) error {
return opts.PreRun()
},
RunE: func(cmd *cobra.Command, args []string) error {
return opts.Run()
},
}
f := cmd.PersistentFlags()
f.StringVarP(&opts.name, "name", "n", "pkg",
"name of the pkg to create, will dicate file name")
f.StringVarP(&opts.cfgFile, "config", "c", "",
"path to config file")
f.StringVar(&opts.dir, "directory", "",
"Working directory for building package, if empty, an ephemeral temporary directory will be used. Set this to persist package artifacts between builds.")
f.StringVarP(&opts.driver, "driver", "d", "k3s",
"")
f.StringVar(&opts.driverVersion, "driver-version", "v1.21.1+k3s1",
"")
f.StringVar(&opts.fleetVersion, "fleet-version", "v0.3.5",
"")
f.StringSliceVarP(&opts.paths, "path", "p", []string{},
"")
f.StringSliceVarP(&opts.images, "image", "i", []string{},
"")
return cmd
}
func (o *pkgBuildOpts) PreRun() error {
_, err := os.Stat(o.cfgFile)
if os.IsNotExist(err) {
if o.cfgFile == "" {
return nil
}
o.logger.Warnf("Did not find an existing %s, creating one", o.cfgFile)
p := o.toPackage()
data, err := yaml.Marshal(p)
if err != nil {
return err
}
if err := os.WriteFile(o.cfgFile, data, 0644); err != nil {
return err
}
} else if err != nil {
return err
}
return nil
}
func (o *pkgBuildOpts) Run() error {
o.logger.Infof("Building package")
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
var p v1alpha1.Package
if o.cfgFile != "" {
o.logger.Infof("Config file '%s' specified, attempting to load existing package config", o.cfgFile)
cfgData, err := os.ReadFile(o.cfgFile)
if err != nil {
return err
}
if err := yaml.Unmarshal(cfgData, &p); err != nil {
return err
}
} else {
o.logger.Infof("No config file specified, strictly using cli arguments")
p = o.toPackage()
}
var wdir string
if o.dir != "" {
if _, err := os.Stat(o.dir); err != nil {
o.logger.Errorf("Failed to use specified working directory: %s\n%v", err)
return err
}
wdir = o.dir
} else {
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
return err
}
defer os.RemoveAll(tmpdir)
wdir = tmpdir
}
pkgr := packager.NewPackager(wdir, o.logger)
d := driver.NewDriver(p.Spec.Driver)
if _, bErr := pkgr.PackageBundles(ctx, p.Spec.Paths...); bErr != nil {
return bErr
}
if iErr := pkgr.PackageImages(ctx, o.images...); iErr != nil {
return iErr
}
if dErr := pkgr.PackageDriver(ctx, d); dErr != nil {
return dErr
}
if fErr := pkgr.PackageFleet(ctx, p.Spec.Fleet); fErr != nil {
return fErr
}
a := packager.NewArchiver()
if aErr := pkgr.Archive(a, p, o.name); aErr != nil {
return aErr
}
o.logger.Successf("Finished building package")
return nil
}
func (o *pkgBuildOpts) toPackage() v1alpha1.Package {
p := v1alpha1.Package{
TypeMeta: metav1.TypeMeta{
Kind: "",
APIVersion: "",
},
ObjectMeta: metav1.ObjectMeta{
Name: o.name,
},
Spec: v1alpha1.PackageSpec{
Fleet: v1alpha1.Fleet{
Version: o.fleetVersion,
},
Driver: v1alpha1.Driver{
Type: o.driver,
Version: o.driverVersion,
},
Paths: o.paths,
Images: o.images,
},
}
return p
}

View File

@@ -0,0 +1,84 @@
package app
import (
"os"
"testing"
)
func Test_pkgBuildOpts_Run(t *testing.T) {
l, _ := setupCliLogger(os.Stdout, "debug")
tro := rootOpts{l}
type fields struct {
rootOpts *rootOpts
cfgFile string
name string
driver string
driverVersion string
fleetVersion string
images []string
paths []string
}
tests := []struct {
name string
fields fields
wantErr bool
}{
{
name: "should package all types of local manifests",
fields: fields{
rootOpts: &tro,
cfgFile: "pkg.yaml",
name: "k3s",
driver: "k3s",
driverVersion: "v1.21.1+k3s1",
fleetVersion: "v0.3.5",
images: nil,
paths: []string{
"../../../testdata/docker-registry",
"../../../testdata/rawmanifests",
},
},
wantErr: false,
},
{
name: "should package using fleet.yaml",
fields: fields{
rootOpts: &tro,
cfgFile: "pkg.yaml",
name: "k3s",
driver: "k3s",
driverVersion: "v1.21.1+k3s1",
fleetVersion: "v0.3.5",
images: nil,
paths: []string{
"../../../testdata/custom",
},
},
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
o := &pkgBuildOpts{
rootOpts: tt.fields.rootOpts,
cfgFile: tt.fields.cfgFile,
name: tt.fields.name,
driver: tt.fields.driver,
driverVersion: tt.fields.driverVersion,
fleetVersion: tt.fields.fleetVersion,
images: tt.fields.images,
paths: tt.fields.paths,
}
if err := o.PreRun(); err != nil {
t.Errorf("PreRun() error = %v", err)
}
defer os.Remove(o.cfgFile)
if err := o.Run(); (err != nil) != tt.wantErr {
t.Errorf("Run() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}

91
cmd/hauler/app/pkg_run.go Normal file
View File

@@ -0,0 +1,91 @@
package app
import (
"context"
"github.com/rancherfederal/hauler/pkg/bootstrap"
"github.com/rancherfederal/hauler/pkg/driver"
"github.com/rancherfederal/hauler/pkg/packager"
"github.com/spf13/cobra"
"os"
)
type pkgRunOpts struct {
*rootOpts
cfgFile string
}
func NewPkgRunCommand() *cobra.Command {
opts := pkgRunOpts{
rootOpts: &ro,
}
cmd := &cobra.Command{
Use: "run",
Short: "Run a compressed archive",
Long: `
Run a compressed archive created from a 'hauler package build'.
Examples:
# Run a package
hauler package run pkg.tar.zst
`,
Aliases: []string{"r"},
Args: cobra.MinimumNArgs(1),
PreRunE: func(cmd *cobra.Command, args []string) error {
return opts.PreRun()
},
RunE: func(cmd *cobra.Command, args []string) error {
return opts.Run(args[0])
},
}
return cmd
}
func (o *pkgRunOpts) PreRun() error {
return nil
}
func (o *pkgRunOpts) Run(pkgPath string) error {
o.logger.Infof("Running from '%s'", pkgPath)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
return err
}
o.logger.Debugf("Using temporary working directory: %s", tmpdir)
a := packager.NewArchiver()
if err := packager.Unpackage(a, pkgPath, tmpdir); err != nil {
return err
}
o.logger.Debugf("Unpackaged %s", pkgPath)
b, err := bootstrap.NewBooter(tmpdir, o.logger)
if err != nil {
return err
}
d := driver.NewDriver(b.Package.Spec.Driver)
if preErr := b.PreBoot(ctx, d); preErr != nil {
return preErr
}
if bErr := b.Boot(ctx, d); bErr != nil {
return bErr
}
if postErr := b.PostBoot(ctx, d); postErr != nil {
return postErr
}
o.logger.Successf("Access the cluster with '/opt/hauler/bin/kubectl'")
return nil
}

View File

@@ -6,12 +6,15 @@ import (
type relocateOpts struct {
inputFile string
*rootOpts
}
// NewRelocateCommand creates a new sub command under
// haulterctl for relocating images and artifacts
func NewRelocateCommand() *cobra.Command {
opts := &relocateOpts{}
opts := &relocateOpts{
rootOpts: &ro,
}
cmd := &cobra.Command{
Use: "relocate",
@@ -23,10 +26,6 @@ func NewRelocateCommand() *cobra.Command {
},
}
f := cmd.PersistentFlags()
f.StringVarP(&opts.inputFile, "input", "i", "haul.tar.zst",
"package output location relative to the current directory (haul.tar.zst)")
cmd.AddCommand(NewRelocateArtifactsCommand(opts))
cmd.AddCommand(NewRelocateImagesCommand(opts))

View File

@@ -0,0 +1,56 @@
package app
import (
"context"
"github.com/rancherfederal/hauler/pkg/oci"
"github.com/spf13/cobra"
)
type relocateArtifactsOpts struct {
*relocateOpts
destRef string
}
var (
relocateArtifactsLong = `hauler relocate artifacts process an archive with files to be pushed to a registry`
relocateArtifactsExample = `
# Run Hauler
hauler relocate artifacts artifacts.tar.zst locahost:5000/artifacts:latest
`
)
// NewRelocateArtifactsCommand creates a new sub command of relocate for artifacts
func NewRelocateArtifactsCommand(relocate *relocateOpts) *cobra.Command {
opts := &relocateArtifactsOpts{
relocateOpts: relocate,
}
cmd := &cobra.Command{
Use: "artifacts",
Short: "Use artifact from bundle artifacts to populate a target file server with the artifact's contents",
Long: relocateArtifactsLong,
Example: relocateArtifactsExample,
Args: cobra.MinimumNArgs(2),
Aliases: []string{"a", "art", "af"},
RunE: func(cmd *cobra.Command, args []string) error {
opts.inputFile = args[0]
opts.destRef = args[1]
return opts.Run(opts.destRef, opts.inputFile)
},
}
return cmd
}
func (o *relocateArtifactsOpts) Run(dst string, input string) error {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
if err := oci.Put(ctx, input, dst); err != nil {
o.logger.Errorf("error pushing artifact to registry %s: %v", dst, err)
}
return nil
}

View File

@@ -0,0 +1,103 @@
package app
import (
"os"
"path/filepath"
"strings"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/layout"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/rancherfederal/hauler/pkg/oci"
"github.com/rancherfederal/hauler/pkg/packager"
"github.com/spf13/cobra"
)
var (
relocateImagesLong = `hauler relocate images processes a bundle provides by hauler package build and copies all of
the collected images to a registry`
relocateImagesExample = `
# Run Hauler
hauler relocate images pkg.tar.zst locahost:5000
`
)
type relocateImagesOpts struct {
*relocateOpts
destRef string
}
// NewRelocateImagesCommand creates a new sub command of relocate for images
func NewRelocateImagesCommand(relocate *relocateOpts) *cobra.Command {
opts := &relocateImagesOpts{
relocateOpts: relocate,
}
cmd := &cobra.Command{
Use: "images",
Short: "Use artifact from bundle images to populate a target registry with the artifact's images",
Long: relocateImagesLong,
Example: relocateImagesExample,
Args: cobra.MinimumNArgs(2),
Aliases: []string{"i", "img", "imgs"},
RunE: func(cmd *cobra.Command, args []string) error {
opts.inputFile = args[0]
opts.destRef = args[1]
return opts.Run(opts.destRef, opts.inputFile)
},
}
return cmd
}
func (o *relocateImagesOpts) Run(dst string, input string) error {
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
return err
}
o.logger.Debugf("Using temporary working directory: %s", tmpdir)
a := packager.NewArchiver()
if err := packager.Unpackage(a, input, tmpdir); err != nil {
o.logger.Errorf("error unpackaging input %s: %v", input, err)
}
o.logger.Debugf("Unpackaged %s", input)
path := filepath.Join(tmpdir, "layout")
ly, err := layout.FromPath(path)
if err != nil {
o.logger.Errorf("error creating OCI layout: %v", err)
}
for nm, hash := range oci.ListImages(ly) {
n := strings.SplitN(nm, "/", 2)
img, err := ly.Image(hash)
o.logger.Infof("Copy %s to %s", n[1], dst)
if err != nil {
o.logger.Errorf("error creating image from layout: %v", err)
}
dstimg := dst + "/" + n[1]
tag, err := name.ParseReference(dstimg)
if err != nil {
o.logger.Errorf("err parsing destination image %s: %v", dstimg, err)
}
if err := remote.Write(tag, img); err != nil {
o.logger.Errorf("error writing image to destination registry %s: %v", dst, err)
}
}
return nil
}

81
cmd/hauler/app/root.go Normal file
View File

@@ -0,0 +1,81 @@
package app
import (
"io"
"os"
"time"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/spf13/cobra"
)
var (
loglevel string
timeout time.Duration
getLong = `hauler provides CLI-based air-gap migration assistance using k3s.
Choose your functionality and new a package when internet access is available,
then deploy the package into your air-gapped environment.
`
getExample = `
hauler pkg build
hauler pkg run pkg.tar.zst
hauler relocate artifacts artifacts.tar.zst
hauler relocate images pkg.tar.zst locahost:5000
hauler copy localhost:5000/artifacts:latest
`
)
type rootOpts struct {
logger log.Logger
}
var ro rootOpts
// NewRootCommand defines the root hauler command
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "hauler",
Short: "hauler provides CLI-based air-gap migration assistance",
Long: getLong,
Example: getExample,
SilenceUsage: true,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
l, err := setupCliLogger(os.Stdout, loglevel)
if err != nil {
return err
}
ro.logger = l
return nil
},
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
}
cobra.OnInitialize()
cmd.AddCommand(NewRelocateCommand())
cmd.AddCommand(NewCopyCommand())
cmd.AddCommand(NewPkgCommand())
f := cmd.PersistentFlags()
f.StringVarP(&loglevel, "loglevel", "l", "debug",
"Log level (debug, info, warn, error, fatal, panic)")
f.DurationVar(&timeout, "timeout", 1*time.Minute,
"TODO: timeout for operations")
return cmd
}
func setupCliLogger(out io.Writer, level string) (log.Logger, error) {
l := log.NewLogger(out)
return l, nil
}

View File

@@ -3,7 +3,7 @@ package main
import (
"log"
"github.com/rancherfederal/hauler/cmd/haulerctl/app"
"github.com/rancherfederal/hauler/cmd/hauler/app"
)
func main() {

View File

@@ -1,109 +0,0 @@
package app
import (
"context"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/bootstrap"
"github.com/rancherfederal/hauler/pkg/packager"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"os"
"path/filepath"
"sigs.k8s.io/yaml"
)
type deployOpts struct {
*rootOpts
haulerDir string
}
// NewBootstrapCommand new a new sub command of haulerctl that bootstraps a cluster
func NewBootstrapCommand() *cobra.Command {
opts := &deployOpts{
rootOpts: &ro,
}
cmd := &cobra.Command{
Use: "bootstrap",
Short: "Single-command install of a k3s cluster with known tools running inside of it",
Long: `Single-command install of a k3s cluster with known tools running inside of it. Tools
include an OCI registry and Git server`,
Aliases: []string{"b", "boot"},
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
return opts.Run(args[0])
},
}
f := cmd.Flags()
f.StringVarP(&opts.haulerDir, "hauler-dir", "", "/opt/hauler", "Directory to install hauler components in")
return cmd
}
// Run performs the operation.
func (o *deployOpts) Run(packagePath string) error {
o.logger.Infof("Bootstrapping from '%s'", packagePath)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
return err
}
defer os.Remove(tmpdir)
o.logger.Debugf("Using temporary working directory: %s", tmpdir)
a := packager.NewArchiver()
err = packager.Unpackage(a, packagePath, tmpdir)
if err != nil {
return err
}
bundleData, err := os.ReadFile(filepath.Join(tmpdir, "package.json"))
if err != nil {
return err
}
var p v1alpha1.Package
if err := yaml.Unmarshal(bundleData, &p); err != nil {
return err
}
d := v1alpha1.NewDriver(p.Spec.Driver.Kind)
bootLogger := o.logger.WithFields(logrus.Fields{
"driver": p.Spec.Driver.Kind,
})
b, err := bootstrap.NewBooter(tmpdir)
if err != nil {
return err
}
o.logger.Infof("Initializing package for driver: %s", p.Spec.Driver.Kind)
if err := b.Init(); err != nil {
return err
}
o.logger.Infof("Performing pre %s boot steps", p.Spec.Driver.Kind)
if err := b.PreBoot(ctx, d, bootLogger); err != nil {
return err
}
o.logger.Infof("Booting %s", p.Spec.Driver.Kind)
if err := b.Boot(ctx, d, bootLogger); err != nil {
return err
}
o.logger.Infof("Performing post %s boot steps", p.Spec.Driver.Kind)
if err := b.PostBoot(ctx, d, bootLogger); err != nil {
return err
}
o.logger.Infof("Success! You can access the cluster with '/opt/hauler/bin/kubectl'")
return nil
}

View File

@@ -1,36 +0,0 @@
package app
import (
"github.com/spf13/cobra"
"github.com/spf13/viper"
)
type bundleOpts struct {
bundleDir string
}
// NewBundleCommand creates a new sub command under
// haulterctl for bundling images and artifacts
func NewBundleCommand() *cobra.Command {
opts := &bundleOpts{}
cmd := &cobra.Command{
Use: "bundle",
Short: "bundle images or artifact for relocation",
Long: "",
Aliases: []string{"b"},
RunE: func(cmd *cobra.Command, args []string) error {
return cmd.Help()
},
}
f := cmd.PersistentFlags()
f.StringVarP(&opts.bundleDir, "bundledir", "b", "./bundle",
"directory locating a bundle, if one exists we will append (./bundle)")
cmd.AddCommand(NewBundleArtifactsCommand(opts))
viper.AutomaticEnv()
return cmd
}

View File

@@ -1,51 +0,0 @@
package app
import (
"context"
"fmt"
"github.com/spf13/cobra"
)
type bundleArtifactsOpts struct {
bundle *bundleOpts
}
// NewBundleArtifactsCommand creates a new sub command of bundle for artifacts
func NewBundleArtifactsCommand(bundle *bundleOpts) *cobra.Command {
opts := &bundleArtifactsOpts{bundle: bundle}
cmd := &cobra.Command{
Use: "artifacts",
Short: "Choose a folder on disk, new artifact containing all of folder's contents",
RunE: func(cmd *cobra.Command, args []string) error {
return opts.Run()
},
}
return cmd
}
func (o *bundleArtifactsOpts) Run() error {
//TODO
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
//b := bundle.NewLayoutStore(o.bundleDir)
//
//images := []string{"alpine:latest", "registry:2.7.1"}
//
//for _, i := range images {
// if err := b.Add(ctx, i); err != nil {
// return err
// }
//}
_ = ctx
fmt.Println("bundle artifacts")
fmt.Println(o.bundle.bundleDir)
return nil
}

View File

@@ -1,119 +0,0 @@
package app
import (
"context"
"os"
"github.com/pterm/pterm"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/packager"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
"sigs.k8s.io/yaml"
)
type createOpts struct {
*rootOpts
driver string
outputFile string
configFile string
}
// NewCreateCommand creates a new sub command under
// haulerctl for creating dependency artifacts for bootstraps
func NewCreateCommand() *cobra.Command {
opts := &createOpts{
rootOpts: &ro,
}
cmd := &cobra.Command{
Use: "create",
Short: "package all dependencies into a compressed archive",
Long: `package all dependencies into a compressed archive used by deploy.
Container images, git repositories, and more, packaged and ready to be served within an air gap.`,
Aliases: []string{"c"},
PreRunE: func(cmd *cobra.Command, args []string) error {
return opts.PreRun()
},
RunE: func(cmd *cobra.Command, args []string) error {
return opts.Run()
},
}
f := cmd.Flags()
f.StringVarP(&opts.driver, "driver", "d", "k3s",
"Driver type to use for package (k3s or rke2)")
f.StringVarP(&opts.outputFile, "output", "o", "haul",
"package output location relative to the current directory (haul.tar.zst)")
f.StringVarP(&opts.configFile, "config", "c", "./package.yaml",
"config file")
return cmd
}
func (o *createOpts) PreRun() error {
return nil
}
// Run performs the operation.
func (o *createOpts) Run() error {
o.logger.Infof("Creating new deployable bundle using driver: %s", o.driver)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
if _, err := os.Stat(o.configFile); err != nil {
logrus.Error(err)
}
bundleData, err := os.ReadFile(o.configFile)
if err != nil {
logrus.Error(err)
}
var p v1alpha1.Package
err = yaml.Unmarshal(bundleData, &p)
if err != nil {
logrus.Error(err)
}
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
logrus.Error(err)
}
defer os.RemoveAll(tmpdir)
pkgr := packager.NewPackager(tmpdir)
pb, _ := pterm.DefaultProgressbar.WithTotal(4).WithTitle("Start Packaging").Start()
o.logger.Infof("Packaging driver (%s %s) artifacts...", p.Spec.Driver.Version, p.Spec.Driver.Kind)
d := v1alpha1.NewDriver(p.Spec.Driver.Kind)
if err = pkgr.Driver(ctx, d); err != nil {
logrus.Error(err)
}
pb.Increment()
o.logger.Infof("Packaging fleet artifacts...")
if err = pkgr.Fleet(ctx, p.Spec.Fleet); err != nil {
logrus.Error(err)
}
pb.Increment()
o.logger.Infof("Packaging images and manifests defined in specified paths...")
if _, err = pkgr.Bundles(ctx, p.Spec.Paths...); err != nil {
logrus.Error(err)
}
pb.Increment()
a := packager.NewArchiver()
o.logger.Infof("Archiving and compressing package to: %s.%s", o.outputFile, a.String())
if err = pkgr.Archive(a, p, o.outputFile); err != nil {
logrus.Error(err)
}
pb.Increment()
return nil
}

View File

@@ -1,78 +0,0 @@
package app
import (
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"io/ioutil"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"os"
"sigs.k8s.io/yaml"
"testing"
)
func Test_createOpts_Run(t *testing.T) {
l, _ := setupCliLogger(os.Stdout, "debug")
tro := rootOpts{l}
p := v1alpha1.Package{
TypeMeta: metav1.TypeMeta{
Kind: "Package",
APIVersion: "hauler.cattle.io/v1alpha1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
},
Spec: v1alpha1.PackageSpec{
Fleet: v1alpha1.Fleet{Version: "0.3.5"},
Driver: v1alpha1.Driver{
Kind: "k3s",
Version: "v1.21.1+k3s1",
},
Paths: []string{
"../../../testdata/docker-registry",
"../../../testdata/rawmanifests",
},
Images: []string{},
},
}
data, _ := yaml.Marshal(p)
if err := ioutil.WriteFile("create_test.package.yaml", data, 0644); err != nil {
t.Fatalf("failed to write test config file: %v", err)
}
defer os.Remove("create_test.package.yaml")
type fields struct {
rootOpts *rootOpts
driver string
outputFile string
configFile string
}
tests := []struct {
name string
fields fields
wantErr bool
}{
{
name: "should work",
fields: fields{
rootOpts: &tro,
driver: "k3s",
outputFile: "package",
configFile: "./create_test.package.yaml",
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
o := &createOpts{
rootOpts: tt.fields.rootOpts,
driver: tt.fields.driver,
outputFile: tt.fields.outputFile,
configFile: tt.fields.configFile,
}
if err := o.Run(); (err != nil) != tt.wantErr {
t.Errorf("Run() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}

View File

@@ -1,64 +0,0 @@
package app
import (
"context"
"io/ioutil"
"os"
"path/filepath"
"github.com/rancherfederal/hauler/pkg/oci"
"github.com/rancherfederal/hauler/pkg/packager"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
type relocateArtifactsOpts struct {
relocate *relocateOpts
destRef string
}
// NewRelocateArtifactsCommand creates a new sub command of relocate for artifacts
func NewRelocateArtifactsCommand(relocate *relocateOpts) *cobra.Command {
opts := &relocateArtifactsOpts{relocate: relocate}
cmd := &cobra.Command{
Use: "artifacts",
Short: "Use artifact from bundle artifacts to populate a target file server with the artifact's contents",
RunE: func(cmd *cobra.Command, args []string) error {
opts.destRef = args[0]
return opts.Run(opts.destRef)
},
}
return cmd
}
func (o *relocateArtifactsOpts) Run(dst string) error {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
ar := packager.NewArchiver()
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
logrus.Error(err)
}
packager.Unpackage(ar, o.relocate.inputFile, tmpdir)
files, err := ioutil.ReadDir(tmpdir)
if err != nil {
logrus.Error(err)
}
for _, f := range files {
if err := oci.Put(ctx, filepath.Join(tmpdir, f.Name()), dst); err != nil {
logrus.Error(err)
}
}
return nil
}

View File

@@ -1,89 +0,0 @@
package app
import (
"fmt"
"os"
"path/filepath"
"strings"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/layout"
"github.com/google/go-containerregistry/pkg/v1/remote"
"github.com/rancherfederal/hauler/pkg/oci"
"github.com/rancherfederal/hauler/pkg/packager"
"github.com/spf13/cobra"
)
type relocateImagesOpts struct {
relocate *relocateOpts
destRef string
}
// NewRelocateImagesCommand creates a new sub command of relocate for images
func NewRelocateImagesCommand(relocate *relocateOpts) *cobra.Command {
opts := &relocateImagesOpts{relocate: relocate}
cmd := &cobra.Command{
Use: "images",
Short: "Use artifact from bundle images to populate a target registry with the artifact's images",
RunE: func(cmd *cobra.Command, args []string) error {
opts.destRef = args[0]
return opts.Run(opts.destRef)
},
}
return cmd
}
func (o *relocateImagesOpts) Run(dst string) error {
ar := packager.NewArchiver()
tmpdir, err := os.MkdirTemp("", "hauler")
if err != nil {
return err
}
packager.Unpackage(ar, o.relocate.inputFile, tmpdir)
if err != nil {
return err
}
path := filepath.Join(tmpdir, "layout")
ly, err := layout.FromPath(path)
if err != nil {
return err
}
for nm, hash := range oci.ListImages(ly) {
n := strings.SplitN(nm, "/", 2)
img, err := ly.Image(hash)
fmt.Printf("Copy %s to %s", n[1], dst)
fmt.Println()
if err != nil {
return err
}
dstimg := dst + "/" + n[1]
tag, err := name.ParseReference(dstimg)
if err != nil {
return err
}
if err := remote.Write(tag, img); err != nil {
return err
}
}
return nil
}

View File

@@ -1,120 +0,0 @@
package app
import (
"fmt"
"io"
"os"
"time"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/sirupsen/logrus"
"github.com/spf13/cobra"
homedir "github.com/mitchellh/go-homedir"
"github.com/spf13/viper"
)
var (
cfgFile string
loglevel string
timeout time.Duration
getLong = `haulerctl provides CLI-based air-gap migration assistance using k3s.
Choose your functionality and new a package when internet access is available,
then deploy the package into your air-gapped environment.
`
getExample = `
# Run Hauler
haulerctl bundle images <images>
haulerctl bundle artifacts <artfiacts>
haulerctl relocate artifacts -i <package-name>
haulerctl relocate images -i <package-name> locahost:5000
haulerctl copy
haulerctl create
haulerctl bootstrap`
)
type rootOpts struct {
logger log.Logger
}
var ro rootOpts
// NewRootCommand defines the root haulerctl command
func NewRootCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "haulerctl",
Short: "haulerctl provides CLI-based air-gap migration assistance",
Long: getLong,
Example: getExample,
SilenceUsage: true,
PersistentPreRunE: func(cmd *cobra.Command, args []string) error {
l, err := setupCliLogger(os.Stdout, loglevel)
if err != nil {
return err
}
ro.logger = l
return nil
},
RunE: func(cmd *cobra.Command, _ []string) error {
return cmd.Help()
},
}
cobra.OnInitialize(initConfig)
cmd.AddCommand(NewRelocateCommand())
cmd.AddCommand(NewCreateCommand())
cmd.AddCommand(NewBundleCommand())
cmd.AddCommand(NewCopyCommand())
cmd.AddCommand(NewBootstrapCommand())
f := cmd.PersistentFlags()
f.StringVarP(&loglevel, "loglevel", "l", "info",
"Log level (debug, info, warn, error, fatal, panic)")
f.StringVarP(&cfgFile, "config", "c", "./hauler.yaml",
"config file (./hauler.yaml)")
f.DurationVar(&timeout, "timeout", 1*time.Minute,
"timeout for operations")
return cmd
}
// initConfig reads in config file and ENV variables if set.
func initConfig() {
if cfgFile != "" {
// Use config file from the flag.
viper.SetConfigFile(cfgFile)
} else {
// Find home directory.
home, err := homedir.Dir()
cobra.CheckErr(err)
// Search config in home directory with name ".hauler" (without extension).
viper.AddConfigPath(home)
viper.SetConfigName(".hauler")
}
viper.AutomaticEnv() // read in environment variables that match
// If a config file is found, read it in.
if err := viper.ReadInConfig(); err == nil {
fmt.Fprintln(os.Stderr, "Using config file:", viper.ConfigFileUsed())
}
}
func setupCliLogger(out io.Writer, level string) (log.Logger, error) {
l := log.NewLogger(out)
lvl, err := logrus.ParseLevel(level)
if err != nil {
return nil, err
}
l.SetLevel(lvl)
return l, nil
}

15
go.mod
View File

@@ -5,29 +5,28 @@ go 1.16
require (
cloud.google.com/go/storage v1.8.0 // indirect
github.com/Microsoft/go-winio v0.5.0 // indirect
github.com/containerd/cgroups v1.0.1 // indirect
github.com/containerd/containerd v1.5.0-beta.4
github.com/containerd/continuity v0.1.0 // indirect
github.com/containers/image/v5 v5.12.0
github.com/deislabs/oras v0.11.1
github.com/docker/docker v20.10.6+incompatible // indirect
github.com/google/go-containerregistry v0.4.1
github.com/google/uuid v1.2.0 // indirect
github.com/docker/libtrust v0.0.0-20160708172513-aabc10ec26b7 // indirect
github.com/google/go-containerregistry v0.5.1
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/imdario/mergo v0.3.12
github.com/klauspost/compress v1.13.0 // indirect
github.com/klauspost/pgzip v1.2.5 // indirect
github.com/mholt/archiver/v3 v3.5.0
github.com/mitchellh/go-homedir v1.1.0
github.com/opencontainers/image-spec v1.0.2-0.20190823105129-775207bd45b6
github.com/otiai10/copy v1.6.0
github.com/pelletier/go-toml v1.8.1 // indirect
github.com/pterm/pterm v0.12.23
github.com/pterm/pterm v0.12.24
github.com/rancher/fleet v0.3.5
github.com/rancher/fleet/pkg/apis v0.0.0
github.com/sirupsen/logrus v1.8.1
github.com/spf13/afero v1.6.0
github.com/spf13/cobra v1.1.3
github.com/spf13/viper v1.7.0
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 // indirect
github.com/ulikunitz/xz v0.5.10 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190809123943-df4f5c81cb3b // indirect
golang.org/x/net v0.0.0-20210525063256-abc453219eb5 // indirect
google.golang.org/genproto v0.0.0-20210524171403-669157292da3 // indirect
google.golang.org/grpc v1.38.0 // indirect

80
go.sum
View File

@@ -38,8 +38,6 @@ cloud.google.com/go/storage v1.8.0 h1:86K1Gel7BQ9/WmNWn7dTKMvTLFzwtBe5FNqYbi9X35
cloud.google.com/go/storage v1.8.0/go.mod h1:Wv1Oy7z6Yz3DshWRJFhqM/UCfaWIRTdp0RXyy7KQOVs=
contrib.go.opencensus.io/exporter/ocagent v0.4.12/go.mod h1:450APlNTSR6FrvC3CTRqYosuDstRB9un7SOx2k/9ckA=
dmitri.shuralyov.com/gpu/mtl v0.0.0-20190408044501-666a987793e9/go.mod h1:H6x//7gZCb22OMCxBHrMx7a5I7Hp++hsVxbQ4BYO7hU=
github.com/14rcole/gopopulate v0.0.0-20180821133914-b175b219e774 h1:SCbEWT58NSt7d2mcFdvxC9uyrdcTfvBbPLThhkDmXzg=
github.com/14rcole/gopopulate v0.0.0-20180821133914-b175b219e774/go.mod h1:6/0dYRLLXyJjbkIPeeGyoJ/eKOSI0eU6eTlCBYibgd0=
github.com/360EntSecGroup-Skylar/excelize v1.4.1/go.mod h1:vnax29X2usfl7HHkBrX5EvSCJcmH3dT9luvxzu8iGAE=
github.com/Azure/azure-sdk-for-go v16.2.1+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
github.com/Azure/azure-sdk-for-go v31.2.0+incompatible/go.mod h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
@@ -137,9 +135,8 @@ github.com/Microsoft/hcsshim v0.8.7/go.mod h1:OHd7sQqRFrYd3RmSgbgji+ctCwkbq2wbEY
github.com/Microsoft/hcsshim v0.8.9/go.mod h1:5692vkUqntj1idxauYlpoINNKeqCiG6Sg38RRsjT5y8=
github.com/Microsoft/hcsshim v0.8.10-0.20200715222032-5eafd1556990/go.mod h1:ay/0dTb7NsG8QMDfsRfLHgZo/6xAJShLe1+ePPflihk=
github.com/Microsoft/hcsshim v0.8.14/go.mod h1:NtVKoYxQuTLx6gEq0L96c9Ju4JbRJ4nY2ow3VK6a9Lg=
github.com/Microsoft/hcsshim v0.8.15 h1:Aof83YILRs2Vx3GhHqlvvfyx1asRJKMFIMeVlHsZKtI=
github.com/Microsoft/hcsshim v0.8.15/go.mod h1:x38A4YbHbdxJtc0sF6oIz+RG0npwSCAvn69iY6URG00=
github.com/Microsoft/hcsshim v0.8.16 h1:8/auA4LFIZFTGrqfKhGBSXwM6/4X1fHa/xniyEHu8ac=
github.com/Microsoft/hcsshim v0.8.16/go.mod h1:o5/SZqmR7x9JNKsW3pu+nqHm0MF8vbA+VxGOoXdC600=
github.com/Microsoft/hcsshim/test v0.0.0-20201218223536-d3e5debf77da/go.mod h1:5hlzMzRKMLyo42nCZ9oml8AdTlq/0cvIaBv6tK1RehU=
github.com/Microsoft/hcsshim/test v0.0.0-20210227013316-43a75bb4edd3/go.mod h1:mw7qgWloBUl75W/gVH3cQszUg1+gUITj7D6NY7ywVnY=
github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ=
@@ -162,11 +159,7 @@ github.com/Shopify/sarama v1.19.0/go.mod h1:FVkBWblsNy7DGZRfXLU0O9RCGt5g3g3yEuWX
github.com/Shopify/toxiproxy v2.1.4+incompatible/go.mod h1:OXgGpZ6Cli1/URJOF1DMxUHB2q5Ap20/P/eIdh4G0pI=
github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6/go.mod h1:3eOhrUMpNV+6aFIbp5/iudMxNCF27Vw2OZgy4xEx0Fg=
github.com/TomOnTime/utfutil v0.0.0-20180511104225-09c41003ee1d/go.mod h1:WML6KOYjeU8N6YyusMjj2qRvaPNUEvrQvaxuFcMRFJY=
github.com/VividCortex/ewma v1.1.1 h1:MnEK4VOv6n0RSY4vtRe3h11qjxL3+t0B8yOL8iMXdcM=
github.com/VividCortex/ewma v1.1.1/go.mod h1:2Tkkvm3sRDVXaiyucHiACn4cqf7DpdyLvmxzcbUokwA=
github.com/VividCortex/gohistogram v1.0.0/go.mod h1:Pf5mBqqDxYaXu3hDrrU+w6nw50o/4+TcAqDqk/vUH7g=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d h1:licZJFw2RwpHMqeKTCYkitsPqHNxTmd4SNR5r94FGM8=
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d/go.mod h1:asat636LX7Bqt5lYEZ27JNDcqxfjdBQuJ/MM4CN/Lzo=
github.com/afex/hystrix-go v0.0.0-20180502004556-fa1af6a1f4f5/go.mod h1:SkGFH1ia65gfNATL8TAiHDNxPzPdmEL5uirI2Uyuz6c=
github.com/agnivade/levenshtein v1.0.1/go.mod h1:CURSv5d9Uaml+FovSIICkLbAUZ9S4RqaHDIsdSBg7lM=
github.com/ajstarks/svgo v0.0.0-20180226025133-644b8db467af/go.mod h1:K08gAheRH3/J6wwsYMMT4xOr94bZjxIelGM0+d/wbFw=
@@ -291,7 +284,6 @@ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMn
github.com/cilium/ebpf v0.0.0-20200110133405-4032b1d8aae3/go.mod h1:MA5e5Lr8slmEg9bt0VpxxWqJlO4iwu3FBdHUzV7wQVg=
github.com/cilium/ebpf v0.0.0-20200702112145-1c8d4c9ef775/go.mod h1:7cR51M8ViRLIdUjrmSXlK9pkrsDlLHbO8jiB8X8JnOc=
github.com/cilium/ebpf v0.2.0/go.mod h1:To2CFviqOWL/M0gIMsvSMlqe7em/l1ALkX1PyjrX2Qs=
github.com/cilium/ebpf v0.4.0/go.mod h1:4tRaxcgiL706VnOzHOdBlY8IEAIdxINsQBcU4xJJXRs=
github.com/clbanning/x2j v0.0.0-20191024224557-825249438eec/go.mod h1:jMjuTZXRI4dUb/I5gc9Hdhagfvm9+RyrPryS/auMzxE=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/clusterhq/flocker-go v0.0.0-20160920122132-2b8b7259d313/go.mod h1:P1wt9Z3DP8O6W3rvwCt0REIlshg1InHImaLW0t3ObY0=
@@ -311,9 +303,8 @@ github.com/containerd/cgroups v0.0.0-20190919134610-bf292b21730f/go.mod h1:OApqh
github.com/containerd/cgroups v0.0.0-20200531161412-0dbf7f05ba59/go.mod h1:pA0z1pT8KYB3TCXK/ocprsh7MAkoW8bZVzPdih9snmM=
github.com/containerd/cgroups v0.0.0-20200710171044-318312a37340/go.mod h1:s5q4SojHctfxANBDvMeIaIovkq29IP48TKAxnhYRxvo=
github.com/containerd/cgroups v0.0.0-20200824123100-0b889c03f102/go.mod h1:s5q4SojHctfxANBDvMeIaIovkq29IP48TKAxnhYRxvo=
github.com/containerd/cgroups v0.0.0-20210114181951-8a68de567b68 h1:hkGVFjz+plgr5UfxZUTPFbUFIF/Km6/s+RVRIRHLrrY=
github.com/containerd/cgroups v0.0.0-20210114181951-8a68de567b68/go.mod h1:ZJeTFisyysqgcCdecO57Dj79RfL0LNeGiFUqLYQRYLE=
github.com/containerd/cgroups v1.0.1 h1:iJnMvco9XGvKUvNQkv88bE4uJXxRQH18efbKo9w5vHQ=
github.com/containerd/cgroups v1.0.1/go.mod h1:0SJrPIenamHDcZhEcJMNBB85rHcUsw4f25ZfBiPYRkU=
github.com/containerd/console v0.0.0-20180822173158-c12b1e7919c1/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw=
github.com/containerd/console v0.0.0-20181022165439-0650fd9eeb50/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw=
github.com/containerd/console v0.0.0-20191206165004-02ecf6a7291e/go.mod h1:8Pf4gM6VEbTNRIT26AyyU7hxdQU3MvAvxVI0sc00XBE=
@@ -339,9 +330,8 @@ github.com/containerd/continuity v0.0.0-20191127005431-f65d91d395eb/go.mod h1:GL
github.com/containerd/continuity v0.0.0-20200107194136-26c1120b8d41/go.mod h1:Dq467ZllaHgAtVp4p1xUQWBrFXR9s/wyoTpG8zOJGkY=
github.com/containerd/continuity v0.0.0-20200710164510-efbc4488d8fe/go.mod h1:cECdGN1O8G9bgKTlLhuPJimka6Xb/Gg7vYzCTNVxhvo=
github.com/containerd/continuity v0.0.0-20201208142359-180525291bb7/go.mod h1:kR3BEg7bDFaEddKm54WSmrol1fKWDU1nKYkgrcgZT7Y=
github.com/containerd/continuity v0.0.0-20210208174643-50096c924a4e h1:6JKvHHt396/qabvMhnhUZvWaHZzfVfldxE60TK8YLhg=
github.com/containerd/continuity v0.0.0-20210208174643-50096c924a4e/go.mod h1:EXlVlkqNba9rJe3j7w3Xa924itAMLgZH4UD/Q4PExuQ=
github.com/containerd/continuity v0.1.0 h1:UFRRY5JemiAhPZrr/uE0n8fMTLcZsUvySPr1+D7pgr8=
github.com/containerd/continuity v0.1.0/go.mod h1:ICJu0PwR54nI0yPEnJ6jcS+J7CZAUXrLh8lPo2knzsM=
github.com/containerd/fifo v0.0.0-20180307165137-3d5202aec260/go.mod h1:ODA38xgv3Kuk8dQz2ZQXpnv/UZZUHUCL7pnLehbXgQI=
github.com/containerd/fifo v0.0.0-20190226154929-a9fb20d87448/go.mod h1:ODA38xgv3Kuk8dQz2ZQXpnv/UZZUHUCL7pnLehbXgQI=
github.com/containerd/fifo v0.0.0-20200410184934-f15a3290365b/go.mod h1:jPQ2IAeZRCYxpS/Cm1495vGFww6ecHmMk1YJH2Q5ln0=
@@ -373,15 +363,8 @@ github.com/containerd/zfs v0.0.0-20210315114300-dde8f0fda960/go.mod h1:m+m51S1Dv
github.com/containernetworking/cni v0.7.1/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY=
github.com/containernetworking/cni v0.8.0/go.mod h1:LGwApLUm2FpoOfxTDEeq8T9ipbpZ61X79hmU3w8FmsY=
github.com/containernetworking/plugins v0.8.6/go.mod h1:qnw5mN19D8fIwkqW7oHHYDHVlzhJpcY6TQxn/fUyDDM=
github.com/containers/image/v5 v5.12.0 h1:1hNS2QkzFQ4lH3GYQLyAXB0acRMhS1Ubm6oV++8vw4w=
github.com/containers/image/v5 v5.12.0/go.mod h1:VasTuHmOw+uD0oHCfApQcMO2+36SfyncoSahU7513Xs=
github.com/containers/libtrust v0.0.0-20190913040956-14b96171aa3b h1:Q8ePgVfHDplZ7U33NwHZkrVELsZP5fYj9pM5WBZB2GE=
github.com/containers/libtrust v0.0.0-20190913040956-14b96171aa3b/go.mod h1:9rfv8iPl1ZP7aqh9YA68wnZv2NUDbXdcdPHVz0pFbPY=
github.com/containers/ocicrypt v1.0.1/go.mod h1:MeJDzk1RJHv89LjsH0Sp5KTY3ZYkjXO/C+bKAeWFIrc=
github.com/containers/ocicrypt v1.1.0 h1:A6UzSUFMla92uxO43O6lm86i7evMGjTY7wTKB2DyGPY=
github.com/containers/ocicrypt v1.1.0/go.mod h1:b8AOe0YR67uU8OqfVNcznfFpAzu3rdgUV4GP9qXPfu4=
github.com/containers/storage v1.30.1 h1:+87sZDoUp0uNsP45dWypHTWTEoy0eNDgFYjTU1XIRVQ=
github.com/containers/storage v1.30.1/go.mod h1:NDJkiwxnSHD1Is+4DGcyR3SIEYSDOa0xnAW+uGQFx9E=
github.com/coredns/corefile-migration v1.0.10/go.mod h1:RMy/mXdeDlYwzt0vdMEJvT2hGJ2I86/eO0UdXmH9XNI=
github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
@@ -442,7 +425,6 @@ github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible
github.com/docker/distribution v2.7.1+incompatible h1:a5mlkVzth6W5A4fOsS3D2EO5BUmsJpcB+cRlLU7cSug=
github.com/docker/distribution v2.7.1+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w=
github.com/docker/docker v1.4.2-0.20190924003213-a8608b5b67c7/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v1.4.2-0.20191219165747-a9416c67da9f/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v1.4.2-0.20200203170920-46ec8731fbce/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v17.12.0-ce-rc1.0.20200618181300-9dc6525e6118+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
github.com/docker/docker v17.12.0-ce-rc1.0.20200916142827-bd33bbf0497b+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
@@ -516,7 +498,6 @@ github.com/form3tech-oss/jwt-go v3.2.2+incompatible h1:TcekIExNqud5crz4xD2pavyTg
github.com/form3tech-oss/jwt-go v3.2.2+incompatible/go.mod h1:pbq4aXjuKjdthFRnoDwaVPLA+WlJuPGy+QneDUgJi2k=
github.com/franela/goblin v0.0.0-20200105215937-c9ffbefa60db/go.mod h1:7dvUGVsVBjqR7JHJk0brhHOZYGmfBYOrK0ZhYMEtBr4=
github.com/franela/goreq v0.0.0-20171204163338-bcd34c9993f8/go.mod h1:ZhphrRTfi2rbfLwlschooIH4+wKKDR4Pdxhh+TRoA20=
github.com/frankban/quicktest v1.11.3/go.mod h1:wRf/ReqHper53s+kmmSZizM8NamnL3IM0I9ntUbOk+k=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/fsnotify/fsnotify v1.4.9 h1:hsms1Qyu0jgnwNXIxa+/V/PDsU6CfLf6CNO8H7IWoS4=
github.com/fsnotify/fsnotify v1.4.9/go.mod h1:znqG4EE+3YCdAaPaxE2ZRY/06pZUdp0tY4IgpuI1SZQ=
@@ -738,14 +719,11 @@ github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-containerregistry v0.4.1 h1:Lrcj2AOoZ7WKawsoKAh2O0dH0tBqMW2lTEmozmK4Z3k=
github.com/google/go-containerregistry v0.4.1/go.mod h1:Ct15B4yir3PLOP5jsy0GNeYVaIZs/MK/Jz5any1wFW0=
github.com/google/go-containerregistry v0.5.1 h1:/+mFTs4AlwsJ/mJe8NDtKb7BxLtbZFpcn8vDsneEkwQ=
github.com/google/go-containerregistry v0.5.1/go.mod h1:Ct15B4yir3PLOP5jsy0GNeYVaIZs/MK/Jz5any1wFW0=
github.com/google/go-github/v28 v28.0.0/go.mod h1:+5GboIspo7F0NG2qsvfYh7en6F3EK37uyqv+c35AR3s=
github.com/google/go-github/v29 v29.0.2 h1:opYN6Wc7DOz7Ku3Oh4l7prmkOMwEcQxpFtxdU8N8Pts=
github.com/google/go-github/v29 v29.0.2/go.mod h1:CHKiKKPHJ0REzfwc14QMklvtHwCveD0PxlMjLlzAM5E=
github.com/google/go-intervals v0.0.2 h1:FGrVEiUnTRKR8yE04qzXYaJMtnIYqobR5QbblK3ixcM=
github.com/google/go-intervals v0.0.2/go.mod h1:MkaR3LNRfeKLPmqgJYs4E66z5InYjmCjbbr4TQlcT6Y=
github.com/google/go-jsonnet v0.17.0/go.mod h1:sOcuej3UW1vpPTZOr8L7RQimqai1a57bt5j22LzGZCw=
github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
@@ -767,9 +745,8 @@ github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaU
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.0.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.2.0 h1:qJYtXnJRWmpe7m/3XlyhrsLrEURqHRM2kxzoxXqyUDs=
github.com/google/uuid v1.2.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
github.com/googleapis/gax-go/v2 v2.0.5 h1:sjZBwGj9Jlw33ImPtvFviGYvseOtDM7hkSKB7+Tv3SM=
github.com/googleapis/gax-go/v2 v2.0.5/go.mod h1:DWXyrwAJ9X0FpwwEdw+IPEYBICEFu5mhpdKc/us6bOk=
@@ -793,7 +770,6 @@ github.com/gorilla/mux v1.7.1/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2z
github.com/gorilla/mux v1.7.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
@@ -936,7 +912,6 @@ github.com/jstemmer/go-junit-report v0.9.1 h1:6QPYqodiu3GuPL+7mfx+NwDdp2eTkp9IfE
github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7C0MuV77Wo=
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/juju/ansiterm v0.0.0-20180109212912-720a0952cc2a/go.mod h1:UJSiEoRfvx3hP73CvoARgeLjaIOjybY9vj8PUPPFGeU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/jung-kurt/gofpdf v1.0.3-0.20190309125859-24315acbbda5/go.mod h1:7Id9E/uU8ce6rXgefFLlgrJj/GYY22cpxn+r32jIOes=
@@ -954,7 +929,6 @@ github.com/klauspost/compress v1.4.1/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0
github.com/klauspost/compress v1.10.10/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.1/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.12.2/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg=
github.com/klauspost/compress v1.13.0 h1:2T7tUoQrQT+fQWdaY5rjWztFGAFwbGD04iPJg90ZiOs=
github.com/klauspost/compress v1.13.0/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg=
github.com/klauspost/cpuid v0.0.0-20180405133222-e7e905edc00e/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
@@ -999,7 +973,6 @@ github.com/lucas-clemente/aes12 v0.0.0-20171027163421-cd47fb39b79f/go.mod h1:JpH
github.com/lucas-clemente/quic-clients v0.1.0/go.mod h1:y5xVIEoObKqULIKivu+gD/LU90pL73bTdtQjPBvtCBk=
github.com/lucas-clemente/quic-go v0.10.2/go.mod h1:hvaRS9IHjFLMq76puFJeWNfmn+H70QZ/CXoxqw9bzao=
github.com/lucas-clemente/quic-go-certificates v0.0.0-20160823095156-d2f86524cced/go.mod h1:NCcRLrOTZbzhZvixZLlERbJtDtYsmMw8Jc4vS8Z0g58=
github.com/lunixbochs/vtclean v0.0.0-20180621232353-2d01aacdc34a/go.mod h1:pHhQNgMf3btfWnGBVipUOjRYhoOsdGqdm/+2c2E2WMI=
github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/magiconair/properties v1.8.1 h1:ZC2Vc7/ZFkGmsVC9KvOjumD+G5lXy2RtTKyzRKO2BQ4=
@@ -1012,7 +985,6 @@ github.com/mailru/easyjson v0.0.0-20190626092158-b2ccc519800e/go.mod h1:C1wdFJiN
github.com/mailru/easyjson v0.7.0 h1:aizVhC/NAAcKWb+5QsU1iNOZb4Yws5UO2I+aIprQITM=
github.com/mailru/easyjson v0.7.0/go.mod h1:KAzv3t3aY1NaHWoQz1+4F1ccyAH66Jk7yos7ldAVICs=
github.com/malexdev/utfutil v0.0.0-20180510171754-00c8d4a8e7a8/go.mod h1:UtpLyb/EupVKXF/N0b4NRe1DNg+QYJsnsHQ038romhM=
github.com/manifoldco/promptui v0.8.0/go.mod h1:n4zTdgP0vr0S3w7/O/g98U+e0gwLScEXGwov2nIKuGQ=
github.com/markbates/pkger v0.17.1/go.mod h1:0JoVlrol20BSywW79rN3kdFFsE5xYM+rSCQDXbLhiuI=
github.com/marstr/guid v1.1.0/go.mod h1:74gB1z2wpxxInTG6yaqA7KrtM0NZ+RbrcqDvYHefzho=
github.com/marstr/guid v1.1.0/go.mod h1:74gB1z2wpxxInTG6yaqA7KrtM0NZ+RbrcqDvYHefzho=
@@ -1030,13 +1002,12 @@ github.com/mattn/go-oci8 v0.0.7/go.mod h1:wjDx6Xm9q7dFtHJvIlrI99JytznLw5wQ4R+9mN
github.com/mattn/go-runewidth v0.0.2/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.4/go.mod h1:LwmH8dsx7+W8Uxz3IHJYH5QSwggIsqBzpuz5H//U1FU=
github.com/mattn/go-runewidth v0.0.10/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU=
github.com/mattn/go-runewidth v0.0.13 h1:lTGmDsbAYt5DmK6OnoV7EuIF1wEIFAcxld6ypU4OSgU=
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-runewidth v0.0.13/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-shellwords v1.0.3/go.mod h1:3xCvwCdWdlDJUrvuMn7Wuy9eWs4pE8vqg+NOMyg4B2o=
github.com/mattn/go-shellwords v1.0.10/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/mattn/go-shellwords v1.0.11 h1:vCoR9VPpsk/TZFW2JwK5I9S0xdrtUq2bph6/YjEPnaw=
github.com/mattn/go-shellwords v1.0.11/go.mod h1:EZzvwXDESEeg03EKmM+RmDnNOPKG4lLtQsUlTZDWQ8Y=
github.com/mattn/go-sqlite3 v1.9.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
github.com/mattn/go-sqlite3 v1.12.0 h1:u/x3mp++qUxvYfulZ4HKOvVO0JWhk7HtE8lWhbGz/Do=
github.com/mattn/go-sqlite3 v1.12.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsOqkbpncsNc=
@@ -1052,13 +1023,11 @@ github.com/mholt/certmagic v0.6.2-0.20190624175158-6a42ef9fe8c2/go.mod h1:g4cOPx
github.com/miekg/dns v1.0.14/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/miekg/dns v1.1.3/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/miekg/dns v1.1.4/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/miekg/pkcs11 v1.0.3 h1:iMwmD7I5225wv84WxIG/bmxz9AXjWvTWIbM/TYHvWtw=
github.com/miekg/pkcs11 v1.0.3/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs=
github.com/mindprince/gonvml v0.0.0-20190828220739-9ebdce4bb989/go.mod h1:2eu9pRWp8mo84xCg6KswZ+USQHjwgRhNp06sozOdsTY=
github.com/minio/md5-simd v1.1.0/go.mod h1:XpBqgZULrMYD3R+M28PcmP0CkI7PEMzB3U77ZrKZ0Gw=
github.com/minio/minio-go/v7 v7.0.2/go.mod h1:dJ80Mv2HeGkYLH1sqS/ksz07ON6csH3S6JUMSQ2zAns=
github.com/minio/sha256-simd v0.1.1/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM=
github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible h1:aKW/4cBs+yK6gpqU3K/oIwk9Q/XICqd3zOX/UFuvqmk=
github.com/mistifyio/go-zfs v2.1.2-0.20190413222219-f784269be439+incompatible/go.mod h1:8AuVvqP/mXw1px98n46wfvcGfQ4ci2FwoAjKYxuo3Z4=
github.com/mitchellh/cli v1.0.0/go.mod h1:hNIlj7HEI86fIcpObd7a0FcrxTWetlwJDGcceTlRvqc=
github.com/mitchellh/copystructure v1.0.0 h1:Laisrj+bAB6b/yJwB5Bt3ITZhGJdqmxquMKeZ+mmkFQ=
@@ -1109,8 +1078,6 @@ github.com/mozilla-services/yaml v0.0.0-20191106225358-5c216288813c/go.mod h1:Is
github.com/mozilla/tls-observatory v0.0.0-20190404164649-a3c1b6cfecfd/go.mod h1:SrKMQvPiws7F7iqYp8/TX+IhxCYhzr6N/1yb8cwHsGk=
github.com/mrunalp/fileutils v0.0.0-20200520151820-abd8a0e76976/go.mod h1:x8F1gnqOkIEiO4rqoeEEEqQbo7HjGMTvyoq3gej4iT0=
github.com/mrunalp/fileutils v0.5.0/go.mod h1:M1WthSahJixYnrXQl/DFQuteStB1weuxD2QJNHXfbSQ=
github.com/mtrmac/gpgme v0.1.2 h1:dNOmvYmsrakgW7LcgiprD0yfRuQQe8/C8F6Z+zogO3s=
github.com/mtrmac/gpgme v0.1.2/go.mod h1:GYYHnGSuS7HK3zVS2n3y73y0okK/BeKzwnn5jgiVFNI=
github.com/munnerz/goautoneg v0.0.0-20120707110453-a547fc61f48d/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/mvdan/xurls v1.1.0/go.mod h1:tQlNn3BED8bE/15hnSL2HLkDeLWpNPAwtw7wkEq44oU=
@@ -1191,11 +1158,9 @@ github.com/opencontainers/runtime-spec v1.0.1/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/
github.com/opencontainers/runtime-spec v1.0.2-0.20190207185410-29686dbc5559/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.3-0.20200728170252-4d89ac9fbff6/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-spec v1.0.3-0.20200929063507-e6143ca7d51d h1:pNa8metDkwZjb9g4T8s+krQ+HRgZAkqnXml+wNir/+s=
github.com/opencontainers/runtime-spec v1.0.3-0.20200929063507-e6143ca7d51d/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0=
github.com/opencontainers/runtime-tools v0.0.0-20181011054405-1d69bd0f9c39/go.mod h1:r3f7wjNzSs2extwzU3Y+6pKfobzPh+kKFJ3ofN+3nfs=
github.com/opencontainers/selinux v1.6.0/go.mod h1:VVGKuOLlE7v4PJyT6h7mNWvq1rzqiriPsEqVhc+svHE=
github.com/opencontainers/selinux v1.8.0 h1:+77ba4ar4jsCbL1GLbFL8fFM57w6suPfSS9PDLDY7KM=
github.com/opencontainers/selinux v1.8.0/go.mod h1:RScLhm78qiWa2gbVCcGkC7tCGdgk3ogry1nUQF8Evvo=
github.com/opentracing-contrib/go-observer v0.0.0-20170622124052-a52f23424492/go.mod h1:Ngi6UdF0k5OKD5t5wlmGhe/EDKPoUM3BXZSSfIuJbis=
github.com/opentracing/basictracer-go v1.0.0/go.mod h1:QfBfYuafItcjQuMwinw9GhYKwFXS9KnPs5lxoYwgW74=
@@ -1207,8 +1172,6 @@ github.com/openzipkin/zipkin-go v0.2.1/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnh
github.com/openzipkin/zipkin-go v0.2.2/go.mod h1:NaW6tEwdmWMaCDZzg8sh+IBNOxHMPnhQw8ySjnjRyN4=
github.com/ory/dockertest v3.3.4+incompatible h1:VrpM6Gqg7CrPm3bL4Wm1skO+zFWLbh7/Xb5kGEbJRh8=
github.com/ory/dockertest v3.3.4+incompatible/go.mod h1:1vX4m9wsvi00u5bseYwXaSnhNrne+V0E6LAcBILJdPs=
github.com/ostreedev/ostree-go v0.0.0-20190702140239-759a8c1ac913 h1:TnbXhKzrTOyuvWrjI8W6pcoI9XPbLHFXCdN2dtUw7Rw=
github.com/ostreedev/ostree-go v0.0.0-20190702140239-759a8c1ac913/go.mod h1:J6OG6YJVEWopen4avK3VNQSnALmmjvniMmni/YFYAwc=
github.com/otiai10/copy v1.6.0 h1:IinKAryFFuPONZ7cm6T6E2QX/vcJwSnlaA5lfoaXIiQ=
github.com/otiai10/copy v1.6.0/go.mod h1:XWfuS3CrI0R6IE0FbgHsEazaXO8G0LpMp9o8tos0x4E=
github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95/go.mod h1:9qAhocn7zKJG+0mI8eUu6xqkFDYS2kb2saOteoSB3cE=
@@ -1224,9 +1187,8 @@ github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTK
github.com/paulmach/orb v0.1.3/go.mod h1:VFlX/8C+IQ1p6FTRRKzKoOPJnvEtA5G0Veuqwbu//Vk=
github.com/pborman/uuid v1.2.0/go.mod h1:X/NO0urCmaxf9VXbdlT7C2Yzkj2IKimNn4k+gtPdI/k=
github.com/pelletier/go-buffruneio v0.2.0/go.mod h1:JkE26KsDizTr40EUHkXVtNPvgGtbSNq5BcowyYOWdKo=
github.com/pelletier/go-toml v1.2.0 h1:T5zMGML61Wp+FlcbWjRDT7yAxhJNAiPPLOFECq181zc=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pelletier/go-toml v1.8.1 h1:1Nf83orprkJyknT6h7zbuEGUEjcyVlCxSUGTENmNCRM=
github.com/pelletier/go-toml v1.8.1/go.mod h1:T2/BmBdy8dvIRq1a/8aqjN41wvWlN4lrapLU/GW4pbc=
github.com/performancecopilot/speed v3.0.0+incompatible/go.mod h1:/CLtqpZ5gBg1M9iaPbIdPPGyKcA8hKdoy6hAWba7Yac=
github.com/peterbourgon/diskv v2.0.1+incompatible h1:UBdAOUP5p4RWqPBg048CAvpKN+vxiaj6gdUUzhl4XmI=
github.com/peterbourgon/diskv v2.0.1+incompatible/go.mod h1:uqqh8zWWbv1HBMNONnaR/tNboyR3/BZd58JJSHlUSCU=
@@ -1250,9 +1212,6 @@ github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZN
github.com/posener/complete v1.1.1/go.mod h1:em0nMJCgc9GFtwrmVmEMR/ZL6WyhyjMBndrE9hABlRI=
github.com/pquerna/cachecontrol v0.0.0-20171018203845-0dec1b30a021/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/pquerna/cachecontrol v0.0.0-20180306154005-525d0eb5f91d/go.mod h1:prYjPmNq4d1NPVmpShWobRqXY3q7Vp+80DqgxxUrUIA=
github.com/pquerna/ffjson v0.0.0-20181028064349-e517b90714f7/go.mod h1:YARuvh7BUWHNhzDq2OM5tzR2RiCcN2D7sapiKyCel/M=
github.com/pquerna/ffjson v0.0.0-20190813045741-dac163c6c0a9 h1:kyf9snWXHvQc+yxE9imhdI8YAm4oKeZISlaAR+x73zs=
github.com/pquerna/ffjson v0.0.0-20190813045741-dac163c6c0a9/go.mod h1:YARuvh7BUWHNhzDq2OM5tzR2RiCcN2D7sapiKyCel/M=
github.com/prometheus/client_golang v0.0.0-20180209125602-c332b6f63c06/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.2/go.mod h1:OsXs2jCmiKlQ1lTBmv21f2mNfw4xf/QclQDMrYNZzcM=
@@ -1298,8 +1257,8 @@ github.com/prometheus/procfs v0.2.0/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4O
github.com/prometheus/procfs v0.6.0 h1:mxy4L2jP6qMonqmq+aTtOx1ifVWUgG/TAmntgbh3xv4=
github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
github.com/pterm/pterm v0.12.23 h1:+PL0YqmmT0QiDLOpevE3e2HPb5UIDBxh6OlLm8jDhxg=
github.com/pterm/pterm v0.12.23/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
github.com/pterm/pterm v0.12.24 h1:VM23UV0UddFxHaEN14SCIIYcr4SE5VnoK80AnoeuGbg=
github.com/pterm/pterm v0.12.24/go.mod h1:PhQ89w4i95rhgE+xedAoqous6K9X+r6aSOI2eFF7DZI=
github.com/qri-io/starlib v0.4.2-0.20200213133954-ff2e8cd5ef8d h1:K6eOUihrFLdZjZnA4XlRp864fmWXv9YTIk7VPLhRacA=
github.com/qri-io/starlib v0.4.2-0.20200213133954-ff2e8cd5ef8d/go.mod h1:7DPO4domFU579Ga6E61sB9VFNaniPVwJP5C4bBCu3wA=
github.com/quasilyte/go-consistent v0.0.0-20190521200055-c6f3937de18c/go.mod h1:5STLWrekHfjyYwxBRVRXNOSewLJ3PWfDJd1VyTS21fI=
@@ -1325,7 +1284,6 @@ github.com/rancher/wrangler v0.7.3-0.20210224225730-5ed69efb6ab9/go.mod h1:9du9G
github.com/rancher/wrangler-cli v0.0.0-20200815040857-81c48cf8ab43/go.mod h1:KxpGNhk/oVL6LCfyxESTD1sb8eXRlUxtkbNm06+7dZU=
github.com/rcrowley/go-metrics v0.0.0-20181016184325-3113b8401b8a/go.mod h1:bCqnVzQkZxMG4s8nGwiZ5l3QUCyqpo9Y+/ZMZ9VjZe4=
github.com/remyoudompheng/bigfft v0.0.0-20170806203942-52369c62f446/go.mod h1:uYEyJGbgTkfkS4+E/PavXkNJcbFIpEtjt2B0KDQ5+9M=
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.2.0 h1:S1pD9weZBuJdFmowNwbpi7BJ8TNftyUImj/0WQi72jY=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/robfig/cron v1.1.0 h1:jk4/Hud3TTdcrJgUOBgsqrZBarcxl6ADIjSC2iniwLY=
@@ -1413,7 +1371,6 @@ github.com/spf13/viper v1.7.0 h1:xVKxvI7ouOI5I+U9s2eeiUfMaWBVoXA3AWskkrqK0VM=
github.com/spf13/viper v1.7.0/go.mod h1:8WkrPz2fc9jxqZNCJI/76HCieCp4Q8HaLFoCha5qpdg=
github.com/src-d/gcfg v1.4.0 h1:xXbNR5AlLSA315x2UO+fTSSAXCDf+Ar38/6oyGbDKQ4=
github.com/src-d/gcfg v1.4.0/go.mod h1:p/UMsR43ujA89BJY9duynAwIpvqEujIH/jFlfL7jWoI=
github.com/stefanberger/go-pkcs11uri v0.0.0-20201008174630-78d3cae3a980 h1:lIOOHPEbXzO3vnmx2gok1Tfs31Q8GQqKLc8vVqyQq/I=
github.com/stefanberger/go-pkcs11uri v0.0.0-20201008174630-78d3cae3a980/go.mod h1:AO3tvPzVZ/ayst6UlUKUv6rcPQInYe3IknH3jYhAKu8=
github.com/stoewer/go-strcase v1.2.0/go.mod h1:IBiWB2sKIp3wVVQ3Y035++gc+knqhUQag1KpM8ahLw8=
github.com/storageos/go-api v2.2.0+incompatible/go.mod h1:ZrLn+e0ZuF3Y65PNF6dIwbJPZqfmtCXxFm9ckv0agOY=
@@ -1439,11 +1396,8 @@ github.com/subosito/gotenv v1.2.0 h1:Slr1R9HxAlEKefgq5jn9U+DnETlIUa6HfgEzj0g5d7s
github.com/subosito/gotenv v1.2.0/go.mod h1:N0PQaV/YGNqwC0u51sEeR/aUtSLEXKX9iv69rRypqCw=
github.com/syndtr/gocapability v0.0.0-20170704070218-db04d3cc01c8/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
github.com/syndtr/gocapability v0.0.0-20180916011248-d98352740cb2/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635 h1:kdXcSzyDtseVEc4yCz2qF8ZrQvIDBJLl4S1c3GCXmoI=
github.com/syndtr/gocapability v0.0.0-20200815063812-42c35b437635/go.mod h1:hkRG7XYTFWNJGYcbNJQlaLq0fg1yr4J4t/NcTQtrfww=
github.com/tchap/go-patricia v2.2.6+incompatible/go.mod h1:bmLyhP68RS6kStMGxByiQ23RP/odRBOTVjwp2cDyi6I=
github.com/tchap/go-patricia v2.3.0+incompatible h1:GkY4dP3cEfEASBPPkWd+AmjYxhmDkqO9/zg7R0lSQRs=
github.com/tchap/go-patricia v2.3.0+incompatible/go.mod h1:bmLyhP68RS6kStMGxByiQ23RP/odRBOTVjwp2cDyi6I=
github.com/thecodeteam/goscaleio v0.1.0/go.mod h1:68sdkZAsK8bvEwBlbQnlLS+xU+hvLYM/iQ8KXej1AwM=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/timakin/bodyclose v0.0.0-20190930140734-f7f2e9bca95e/go.mod h1:Qimiffbc6q9tBWlVV6x0P9sat/ao1xEkREYPPj9hphk=
@@ -1471,10 +1425,6 @@ github.com/valyala/bytebufferpool v1.0.0/go.mod h1:6bBcMArwyJ5K/AmCkWv1jt77kVWyC
github.com/valyala/fasthttp v1.2.0/go.mod h1:4vX61m6KN+xDduDNwXrhIAVZaZaZiQ1luJk8LWSxF3s=
github.com/valyala/quicktemplate v1.2.0/go.mod h1:EH+4AkTd43SvgIbQHYu59/cJyxDoOVRUAfrukLPuGJ4=
github.com/valyala/tcplisten v0.0.0-20161114210144-ceec8f93295a/go.mod h1:v3UYOV9WzVtRmSR+PDvWpU/qWl4Wa5LApYYX4ZtKbio=
github.com/vbatts/tar-split v0.11.1 h1:0Odu65rhcZ3JZaPHxl7tCI3V/C/Q9Zf82UFravl02dE=
github.com/vbatts/tar-split v0.11.1/go.mod h1:LEuURwDEiWjRjwu46yU3KVGuUdVv/dcnpcEPSzR8z6g=
github.com/vbauerster/mpb/v6 v6.0.3 h1:j+twHHhSUe8aXWaT/27E98G5cSBeqEuJSVCMjmLg0PI=
github.com/vbauerster/mpb/v6 v6.0.3/go.mod h1:5luBx4rDLWxpA4t6I5sdeeQuZhqDxc+wr5Nqf35+tnM=
github.com/vektah/gqlparser v1.1.2/go.mod h1:1ycwN7Ij5njmMkPPAOaRFY4rET2Enx7IkVv3vaXspKw=
github.com/vektra/mockery v1.1.2/go.mod h1:VcfZjKaFOPO+MpN4ZvwPjs4c48lkq1o3Ym8yHZJu0jU=
github.com/vishvananda/netlink v0.0.0-20181108222139-023a6dafdcdf/go.mod h1:+SR5DhBJrl6ZM7CoCKvpw5BKroDKQ+PJqOg65H/2ktk=
@@ -1490,7 +1440,6 @@ github.com/vmihailenco/tagparser v0.1.2/go.mod h1:OeAg3pn3UbLjkWt+rN9oFYB6u/cQgq
github.com/vmware/govmomi v0.20.3/go.mod h1:URlwyTFZX72RmxtxuaFL2Uj3fD1JTvZdx59bHWk6aFU=
github.com/whilp/git-urls v0.0.0-20191001220047-6db9661140c0/go.mod h1:2rx5KE5FLD0HRfkkpyn8JwbVLBdhgeiOb2D2D9LLKM4=
github.com/willf/bitset v1.1.11-0.20200630133818-d5bec3311243/go.mod h1:RjeCKbqT1RxIR/KWY6phxZiaY1IyutSBfGjNPySAYV4=
github.com/willf/bitset v1.1.11 h1:N7Z7E9UvjW+sGsEl7k/SJrvY2reP1A07MrGuCjIOjRE=
github.com/willf/bitset v1.1.11/go.mod h1:83CECat5yLh5zVOf4P1ErAgKA5UDvKtgyUABdr3+MjI=
github.com/xanzy/ssh-agent v0.2.1 h1:TCbipTQL2JiiCprBWx9frJ2eJlCYT00NmctrHxVAr70=
github.com/xanzy/ssh-agent v0.2.1/go.mod h1:mLlQY/MoOhWBj+gOGMQkOeiEvkx+8pJSI+0Bx9h2kr4=
@@ -1529,7 +1478,6 @@ github.com/ziutek/mymysql v1.5.4 h1:GB0qdRGsTwQSBVYuVShFBKaXSnSnYYC2d9knnE1LHFs=
github.com/ziutek/mymysql v1.5.4/go.mod h1:LMSpPZ6DbqWFxNCHW77HeMg9I646SAhApZ/wKdgO/C0=
go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.etcd.io/bbolt v1.3.5 h1:XAzx9gjCb0Rxj7EoqcClPD1d5ZBxZJk0jbuoPHenBt0=
go.etcd.io/bbolt v1.3.5/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
go.etcd.io/etcd v0.0.0-20191023171146-3cf2f69b5738/go.mod h1:dnLIgRNXwCJa5e+c6mIZCrds/GIG4ncV9HhK5PX7jPg=
go.etcd.io/etcd v0.5.0-alpha.5.0.20200910180754-dd1b699fc489/go.mod h1:yVHk9ub3CSBatqGNg7GRmsnfLWtoW60w4eDYfh7vHDg=
@@ -1539,7 +1487,6 @@ go.mongodb.org/mongo-driver v1.1.1/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qL
go.mongodb.org/mongo-driver v1.1.2/go.mod h1:u7ryQJ+DOzQmeO7zB6MHyr8jkEQvC8vH7qLUO4lqsUM=
go.mozilla.org/gopgagent v0.0.0-20170926210634-4d7ea76ff71a h1:N7VD+PwpJME2ZfQT8+ejxwA4Ow10IkGbU0MGf94ll8k=
go.mozilla.org/gopgagent v0.0.0-20170926210634-4d7ea76ff71a/go.mod h1:YDKUvO0b//78PaaEro6CAPH6NqohCmL2Cwju5XI2HoE=
go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1 h1:A/5uWzF44DlIgdm/PQFwfMkW0JX+cIcQi/SwLAmZP5M=
go.mozilla.org/pkcs7 v0.0.0-20200128120323-432b2356ecb1/go.mod h1:SNgMg+EgDFwmvSmLRTNKC5fegJjB7v23qTQ0XLGUNHk=
go.mozilla.org/sops/v3 v3.6.1 h1:SQXX2hXcZHBw4ZIPpvFbEns6hA7f2rE6kbN6E0heEkM=
go.mozilla.org/sops/v3 v3.6.1/go.mod h1:3KLncZfyE0cG/28CriTo0JJMARroeKToDIISBgN93xw=
@@ -1601,10 +1548,8 @@ golang.org/x/crypto v0.0.0-20200604202706-70a84ac30bf9/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20200728195943-123391ffb6de/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201002170205-7f63de1d35b0/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad h1:DN0cp81fZ3njFcrLCytUHRSUkqBjfTo4Tx9RJTWs0EY=
golang.org/x/crypto v0.0.0-20201221181555-eec23a3978ad/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2 h1:It14KIkyBFYkHkwZ7k45minvA9aorojkyjGk9KJ5B/w=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2/go.mod h1:T9bdIzuCu7OtxOm1hfPfRQxPLYneinmdGuTeoZ9dtd4=
golang.org/x/exp v0.0.0-20180321215751-8460e604b9de/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20180807140117-3d87b88a115f/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
@@ -1698,8 +1643,6 @@ golang.org/x/net v0.0.0-20201022231255-08b38378de70/go.mod h1:sp8m0HH+o8qH0wwXwY
golang.org/x/net v0.0.0-20201024042810-be3efd7ff127/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=
golang.org/x/net v0.0.0-20210316092652-d523dce5a7f4/go.mod h1:RBQZq4jEuRlivfhVLdyRGr576XBO4/greRjx4P4O3yc=
golang.org/x/net v0.0.0-20210428140749-89ef3d95e781/go.mod h1:OJAsFXCWl8Ukc7SiCT/9KSuxbyM7479/AVlXFRxuMCk=
@@ -1815,7 +1758,6 @@ golang.org/x/sys v0.0.0-20210315160823-c6e025ad8005/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210320140829-1e4c9ba3b0c4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210324051608-47abb6519492/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40 h1:JWgyZ1qgdTaF3N3oxC+MdTV7qvEEgHo3otj+HB5CM7Q=

128
k3ama.sh
View File

@@ -1,128 +0,0 @@
#!/bin/bash
# , , _______________________________
# ,-----------|'------'| | |
# /. '-' |-' |_____________________________|
# |/| | |
# | .________.'----' _______________________________
# | || | || | |
# \__|' \__|' |_____________________________|
#
# |‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾|
# |________________________________________________________|
# |
# |‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾|
# |________________________________________________________|
#
# k3ama - airgap migration assistant
LOCAL_IMAGES_FILEPATH=/var/lib/rancher/k3s/agent/images
ADDL_IMAGES=./artifacts/images
copy_images(){
cp -rvf ${ADDL_IMAGES}/* ${LOCAL_IMAGES_FILEPATH}
}
install_k3s(){
AIRGAP_IMAGES_TAR="$1"
## Note: currently requires root
mkdir -p ${LOCAL_IMAGES_FILEPATH}
echo "copying ${AIRGAP_IMAGES_TAR} -> ${LOCAL_IMAGES_FILEPATH}"
cp artifacts/k3s-airgap-images-amd64.tar /var/lib/rancher/k3s/agent/images
# copy over the k3s binary
cp ./artifacts/k3s /usr/local/bin/k3s
chmod +x /usr/local/bin/k3s
INSTALL_K3S_SKIP_DOWNLOAD=true ./scripts/k3s-install.sh
}
uninstall_k3s(){
if [ -f "/usr/local/bin/k3s-uninstall.sh" ]; then
/usr/local/bin/k3s-uninstall.sh
else
echo "k3s is not installed"
fi
}
check_deps(){
#TODO
echo "TODO: check to ensure that the dependencies are in place."
#rpm -qa | grep k3s-selinux
}
#gather_selinux_rpms(){
# if ! yum list installed yum-utils >/dev/null 2>&1; then
# yum install -y yum-utils
# fi
#wget -O ./https://rpm.rancher.io/k3s-selinux-0.1.1-rc1.el7.noarch.rpm
#yumdownloader --destdir=. --resolve container-selinux selinux-policy-base
#}
usage () {
echo "USAGE: $0 [--image-list rancher-images.txt] [--images rancher-images.tar.gz]"
echo " [-l|--image-list path] text file with list of images; one image per line."
echo " [-i|--images path] tar.gz generated by docker save."
echo " [-h|--help] Usage message"
}
check_firewalld(){
if pgrep -x "firewalld" >/dev/null
then
echo "[FATAL] disable firewalld first"
fi
}
check_selinux(){
# yes i know we want selinux, but it's a pain in the ass right now and i will come back to it
SELINUXSTATUS=$(getenforce)
if [ "$SELINUXSTATUS" == "Permissive" ]; then
echo "[FATAL] disable selinux"
exit 1
else
echo "SELINUX disabled. continuing"
fi
}
copy_yaml_manifests(){
cp -r ./yaml/* /var/lib/rancher/k3s/server/manifests
}
copy_local_bins(){
if [ -f "./artifacts/k9s" ]; then
cp -v ./artifacts/k9s /usr/local/bin/
fi
}
copy_local_kubectl(){
echo "TODO"
}
iptable_block_docker_io() {
# iptables -A OUTPUT -p tcp -m string --string "docker.io" --algo kmp -j REJECT
echo "iptable_block_docker_io() disabled"
}
## TODO: Make this interactive with case statements
uninstall_k3s
copy_local_bins
iptable_block_docker_io
check_deps
check_firewalld
#check_selinux
install_k3s ./artifacts/k3s-airgap-images-amd64.tar
copy_images
copy_yaml_manifests
/usr/local/bin/k3s kubectl get pods -A -w
#######
# Notes:
# - workaround: busybox is not included in the main images.txt list and therefor the pvcs cannot create.
# - VAGRANT FAIL: INFO[0000] Preparing data dir /var/lib/rancher/k3s/data ... for some reason local-path provisioner cannot create vols in vagrant
# - bug: RunContainerError results you try to reinstall k3s on top of an old instance WHEN RUNNING SELINUX
#######

View File

@@ -1,7 +1,6 @@
package v1alpha1
import (
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/cli-utils/pkg/object"
)
@@ -15,82 +14,78 @@ type Drive interface {
SystemObjects() (objs []object.ObjMetadata)
}
//Driver
type Driver struct {
Kind string `json:"kind"`
Type string `json:"type"`
Version string `json:"version"`
}
type k3s struct {
dataDir string
etcDir string
}
//TODO: Don't hardcode this
func (k k3s) BinURL() string {
return "https://github.com/k3s-io/k3s/releases/download/v1.21.1%2Bk3s1/k3s"
}
func (k k3s) Images() ([]string, error) {
//TODO: Replace this with a query to images.txt on release page
return []string{
"docker.io/rancher/coredns-coredns:1.8.3",
"docker.io/rancher/klipper-helm:v0.5.0-build20210505",
"docker.io/rancher/klipper-lb:v0.2.0",
"docker.io/rancher/library-busybox:1.32.1",
"docker.io/rancher/library-traefik:2.4.8",
"docker.io/rancher/local-path-provisioner:v0.0.19",
"docker.io/rancher/metrics-server:v0.3.6",
"docker.io/rancher/pause:3.1",
}, nil
}
func (k k3s) Config() (*map[string]interface{}, error) {
// TODO: This should be typed
c := make(map[string]interface{})
c["write-kubeconfig-mode"] = "0644"
//TODO: Add uid or something to ensure this works for multi-node setups
c["node-name"] = "hauler"
return &c, nil
}
func (k k3s) SystemObjects() (objs []object.ObjMetadata) {
//TODO: Make sure this matches up with specified config disables
for _, dep := range []string{"coredns", "local-path-provisioner", "metrics-server"} {
objMeta, _ := object.CreateObjMetadata("kube-system", dep, schema.GroupKind{Kind: "Deployment", Group: "apps"})
objs = append(objs, objMeta)
}
return objs
}
func (k k3s) LibPath() string { return "/var/lib/rancher/k3s" }
func (k k3s) EtcPath() string { return "/etc/rancher/k3s" }
//TODO: Implement rke2 as a driver
type rke2 struct{}
func (r rke2) Images() ([]string, error) { return []string{}, nil }
func (r rke2) BinURL() string { return "" }
func (r rke2) LibPath() string { return "" }
func (r rke2) EtcPath() string { return "" }
func (r rke2) Config() (*map[string]interface{}, error) { return nil, nil }
func (r rke2) SystemObjects() (objs []object.ObjMetadata) { return objs }
//NewDriver will return the appropriate driver given a kind, defaults to k3s
func NewDriver(kind string) Drive {
var d Drive
switch kind {
case "rke2":
//TODO
d = rke2{}
default:
d = k3s{
dataDir: "/var/lib/rancher/k3s",
etcDir: "/etc/rancher/k3s",
}
}
return d
}
////TODO: Don't hardcode this
//func (k k3s) BinURL() string {
// return "https://github.com/k3s-io/k3s/releases/download/v1.21.1%2Bk3s1/k3s"
//}
//
//func (k k3s) PackageImages() ([]string, error) {
// //TODO: Replace this with a query to images.txt on release page
// return []string{
// "docker.io/rancher/coredns-coredns:1.8.3",
// "docker.io/rancher/klipper-helm:v0.5.0-build20210505",
// "docker.io/rancher/klipper-lb:v0.2.0",
// "docker.io/rancher/library-busybox:1.32.1",
// "docker.io/rancher/library-traefik:2.4.8",
// "docker.io/rancher/local-path-provisioner:v0.0.19",
// "docker.io/rancher/metrics-server:v0.3.6",
// "docker.io/rancher/pause:3.1",
// }, nil
//}
//
//func (k k3s) Config() (*map[string]interface{}, error) {
// // TODO: This should be typed
// c := make(map[string]interface{})
// c["write-kubeconfig-mode"] = "0644"
//
// //TODO: Add uid or something to ensure this works for multi-node setups
// c["node-name"] = "hauler"
//
// return &c, nil
//}
//
//func (k k3s) SystemObjects() (objs []object.ObjMetadata) {
// //TODO: Make sure this matches up with specified config disables
// for _, dep := range []string{"coredns", "local-path-provisioner", "metrics-server"} {
// objMeta, _ := object.CreateObjMetadata("kube-system", dep, schema.GroupKind{Kind: "Deployment", Group: "apps"})
// objs = append(objs, objMeta)
// }
// return objs
//}
//
//func (k k3s) LibPath() string { return "/var/lib/rancher/k3s" }
//func (k k3s) EtcPath() string { return "/etc/rancher/k3s" }
//
////TODO: Implement rke2 as a driver
//type rke2 struct{}
//
//func (r rke2) PackageImages() ([]string, error) { return []string{}, nil }
//func (r rke2) BinURL() string { return "" }
//func (r rke2) LibPath() string { return "" }
//func (r rke2) EtcPath() string { return "" }
//func (r rke2) Config() (*map[string]interface{}, error) { return nil, nil }
//func (r rke2) SystemObjects() (objs []object.ObjMetadata) { return objs }
//
////NewDriver will return the appropriate driver given a kind, defaults to k3s
//func NewDriver(kind string) Drive {
// var d Drive
// switch kind {
// case "rke2":
// //TODO
// d = rke2{}
//
// default:
// d = k3s{
// dataDir: "/var/lib/rancher/k3s",
// etcDir: "/etc/rancher/k3s",
// }
// }
//
// return d
//}

View File

@@ -1,6 +1,9 @@
package v1alpha1
import "fmt"
import (
"fmt"
"strings"
)
//Fleet is used as the deployment engine for all things Hauler
type Fleet struct {
@@ -10,12 +13,20 @@ type Fleet struct {
//TODO: These should be identified from the chart version
func (f Fleet) Images() ([]string, error) {
return []string{"rancher/gitjob:v0.1.15", "rancher/fleet:v0.3.5", "rancher/fleet-agent:v0.3.5"}, nil
return []string{
fmt.Sprintf("rancher/gitjob:v0.1.15"),
fmt.Sprintf("rancher/fleet:%s", f.Version),
fmt.Sprintf("rancher/fleet-agent:%s", f.Version),
}, nil
}
func (f Fleet) CRDChart() string {
return fmt.Sprintf("https://github.com/rancher/fleet/releases/download/v0.3.5/fleet-crd-%s.tgz", f.Version)
return fmt.Sprintf("https://github.com/rancher/fleet/releases/download/%s/fleet-crd-%s.tgz", f.Version, f.VLess())
}
func (f Fleet) Chart() string {
return fmt.Sprintf("https://github.com/rancher/fleet/releases/download/v0.3.5/fleet-%s.tgz", f.Version)
return fmt.Sprintf("https://github.com/rancher/fleet/releases/download/%s/fleet-%s.tgz", f.Version, f.VLess())
}
func (f Fleet) VLess() string {
return strings.ReplaceAll(f.Version, "v", "")
}

View File

@@ -5,35 +5,33 @@ import (
"context"
"fmt"
"github.com/google/go-containerregistry/pkg/v1/tarball"
"github.com/imdario/mergo"
"github.com/otiai10/copy"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/driver"
"github.com/rancherfederal/hauler/pkg/fs"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/sirupsen/logrus"
"helm.sh/helm/v3/pkg/chart/loader"
"io"
"k8s.io/cli-runtime/pkg/genericclioptions"
"os"
"os/exec"
"path/filepath"
"sigs.k8s.io/yaml"
)
type Booter interface {
Init() error
PreBoot(context.Context) error
Boot(context.Context, v1alpha1.Drive) error
PostBoot(context.Context, v1alpha1.Drive) error
Boot(context.Context, driver.Driver) error
PostBoot(context.Context, driver.Driver) error
}
type booter struct {
Package v1alpha1.Package
fs fs.PkgFs
logger log.Logger
}
//NewBooter will build a new booter given a path to a directory containing a hauler package.json
func NewBooter(pkgPath string) (*booter, error) {
func NewBooter(pkgPath string, logger log.Logger) (*booter, error) {
pkg, err := v1alpha1.LoadPackageFromDir(pkgPath)
if err != nil {
return nil, err
@@ -44,116 +42,99 @@ func NewBooter(pkgPath string) (*booter, error) {
return &booter{
Package: pkg,
fs: fsys,
logger: logger,
}, nil
}
func (b booter) Init() error {
d := v1alpha1.NewDriver(b.Package.Spec.Driver.Kind)
func (b booter) PreBoot(ctx context.Context, d driver.Driver) error {
b.logger.Infof("Beginning pre boot")
//TODO: Feel like there's a better way to do all this dir creation
if err := os.MkdirAll(d.DataPath(), os.ModePerm); err != nil {
return err
}
//TODO: Feel like there's a better way to do this
if err := b.moveBin(); err != nil {
return err
}
if err := b.moveImages(d); err != nil {
return err
}
if err := b.moveBundles(d); err != nil {
return err
}
if err := b.moveCharts(d); err != nil {
return err
}
return nil
}
func (b booter) PreBoot(ctx context.Context, d v1alpha1.Drive, logger log.Logger) error {
l := logger.WithFields(logrus.Fields{
"phase": "preboot",
})
l.Infof("Creating driver configuration")
if err := b.writeConfig(d); err != nil {
b.logger.Debugf("Writing %s config", d.Name())
if err := d.WriteConfig(); err != nil {
return err
}
b.logger.Successf("Completed pre boot")
return nil
}
func (b booter) Boot(ctx context.Context, d v1alpha1.Drive, logger log.Logger) error {
l := logger.WithFields(logrus.Fields{
"phase": "boot",
})
//TODO: Generic
cmd := exec.Command("/bin/sh", "/opt/hauler/bin/k3s-init.sh")
cmd.Env = append(os.Environ(), []string{
"INSTALL_K3S_SKIP_DOWNLOAD=true",
"INSTALL_K3S_SELINUX_WARN=true",
"INSTALL_K3S_SKIP_SELINUX_RPM=true",
"INSTALL_K3S_BIN_DIR=/opt/hauler/bin",
//TODO: Provide a real dryrun option
//"INSTALL_K3S_SKIP_START=true",
}...)
func (b booter) Boot(ctx context.Context, d driver.Driver) error {
b.logger.Infof("Beginning boot")
var stdoutBuf, stderrBuf bytes.Buffer
cmd.Stdout = io.MultiWriter(os.Stdout, &stdoutBuf)
cmd.Stderr = io.MultiWriter(os.Stderr, &stderrBuf)
out := io.MultiWriter(os.Stdout, &stdoutBuf, &stderrBuf)
err := cmd.Run()
err := d.Start(out)
if err != nil {
return err
}
l.Infof("Driver successfully started!")
l.Infof("Waiting for driver core components to provision...")
b.logger.Infof("Waiting for driver core components to provision...")
waitErr := waitForDriver(ctx, d)
if waitErr != nil {
return err
}
b.logger.Successf("Completed boot")
return nil
}
func (b booter) PostBoot(ctx context.Context, d v1alpha1.Drive, logger log.Logger) error {
l := logger.WithFields(logrus.Fields{
"phase": "postboot",
})
func (b booter) PostBoot(ctx context.Context, d driver.Driver) error {
b.logger.Infof("Beginning post boot")
cf := genericclioptions.NewConfigFlags(true)
cf.KubeConfig = stringptr(fmt.Sprintf("%s/k3s.yaml", d.EtcPath()))
cf := NewBootConfig("fleet-system", d.KubeConfigPath())
fleetCrdChartPath := b.fs.Chart().Path(fmt.Sprintf("fleet-crd-%s.tgz", b.Package.Spec.Fleet.Version))
fleetCrdChartPath := b.fs.Chart().Path(fmt.Sprintf("fleet-crd-%s.tgz", b.Package.Spec.Fleet.VLess()))
fleetCrdChart, err := loader.Load(fleetCrdChartPath)
if err != nil {
return err
}
l.Infof("Installing fleet crds")
fleetCrdRelease, fleetCrdErr := installChart(cf, fleetCrdChart, "fleet-crd", "fleet-system", nil)
b.logger.Infof("Installing fleet crds")
fleetCrdRelease, fleetCrdErr := installChart(cf, fleetCrdChart, "fleet-crd", nil, b.logger)
if fleetCrdErr != nil {
return fleetCrdErr
}
l.Infof("Successfully installed '%s' to namespace '%s'", fleetCrdRelease.Name, fleetCrdRelease.Namespace)
b.logger.Infof("Installed '%s' to namespace '%s'", fleetCrdRelease.Name, fleetCrdRelease.Namespace)
fleetChartPath := b.fs.Chart().Path(fmt.Sprintf("fleet-%s.tgz", b.Package.Spec.Fleet.Version))
fleetChartPath := b.fs.Chart().Path(fmt.Sprintf("fleet-%s.tgz", b.Package.Spec.Fleet.VLess()))
fleetChart, err := loader.Load(fleetChartPath)
if err != nil {
return err
}
l.Infof("Installing fleet")
fleetRelease, fleetErr := installChart(cf, fleetChart, "fleet", "fleet-system", nil)
b.logger.Infof("Installing fleet")
fleetRelease, fleetErr := installChart(cf, fleetChart, "fleet", nil, b.logger)
if fleetErr != nil {
return fleetErr
}
l.Infof("Successfully installed '%s' to namespace '%s'", fleetRelease.Name, fleetRelease.Namespace)
b.logger.Infof("Installed '%s' to namespace '%s'", fleetRelease.Name, fleetRelease.Namespace)
b.logger.Successf("Completed post boot")
return nil
}
@@ -167,9 +148,9 @@ func (b booter) moveBin() error {
return copy.Copy(b.fs.Bin().Path(), path)
}
func (b booter) moveImages(d v1alpha1.Drive) error {
func (b booter) moveImages(d driver.Driver) error {
//NOTE: archives are not recursively searched, this _must_ be at the images dir
path := filepath.Join(d.LibPath(), "agent/images")
path := d.DataPath("agent/images")
if err := os.MkdirAll(path, 0700); err != nil {
return err
}
@@ -182,49 +163,18 @@ func (b booter) moveImages(d v1alpha1.Drive) error {
return tarball.MultiRefWriteToFile(filepath.Join(path, "hauler.tar"), refs)
}
func (b booter) moveBundles(d v1alpha1.Drive) error {
path := filepath.Join(d.LibPath(), "server/manifests/hauler")
if err := os.MkdirAll(d.LibPath(), 0700); err != nil {
return err
}
return copy.Copy(b.fs.Bundle().Path(), path)
}
func (b booter) moveCharts(d v1alpha1.Drive) error {
path := filepath.Join(d.LibPath(), "server/static/charts/hauler")
func (b booter) moveBundles(d driver.Driver) error {
path := d.DataPath("server/manifests/hauler")
if err := os.MkdirAll(path, 0700); err != nil {
return err
}
return copy.Copy(b.fs.Bundle().Path(), path)
}
func (b booter) moveCharts(d driver.Driver) error {
path := d.DataPath("server/static/charts/hauler")
if err := os.MkdirAll(path, 0700); err != nil {
return err
}
return copy.Copy(b.fs.Chart().Path(), path)
}
func (b booter) writeConfig(d v1alpha1.Drive) error {
if err := os.MkdirAll(d.EtcPath(), os.ModePerm); err != nil {
return err
}
c, err := d.Config()
if err != nil {
return err
}
var uc map[string]interface{}
path := filepath.Join(d.EtcPath(), "config.yaml")
if data, err := os.ReadFile(path); err != nil {
err := yaml.Unmarshal(data, &uc)
if err != nil {
return err
}
}
//Merge with user defined configs taking precedence
if err := mergo.Merge(c, uc); err != nil {
return err
}
data, err := yaml.Marshal(c)
return os.WriteFile(path, data, 0644)
}

29
pkg/bootstrap/config.go Normal file
View File

@@ -0,0 +1,29 @@
package bootstrap
import (
"k8s.io/cli-runtime/pkg/genericclioptions"
)
type BootSettings struct {
config *genericclioptions.ConfigFlags
Namespace string
KubeConfig string
}
func NewBootConfig(ns, kubepath string) *BootSettings {
env := &BootSettings{
Namespace: ns,
KubeConfig: kubepath,
}
env.config = &genericclioptions.ConfigFlags{
Namespace: &env.Namespace,
KubeConfig: &env.KubeConfig,
}
return env
}
// RESTClientGetter gets the kubeconfig from BootSettings
func (s *BootSettings) RESTClientGetter() genericclioptions.RESTClientGetter {
return s.config
}

View File

@@ -0,0 +1,20 @@
package bootstrap
import (
"testing"
)
func TestBootSettings(t *testing.T) {
ns := "test"
kpath := "somepath"
settings := NewBootConfig(ns, kpath)
if settings.Namespace != ns {
t.Errorf("expected namespace %q, got %q", ns, settings.Namespace)
}
if settings.KubeConfig != kpath {
t.Errorf("expected kube-config %q, got %q", kpath, settings.KubeConfig)
}
}

View File

@@ -3,26 +3,23 @@ package bootstrap
import (
"context"
"errors"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/driver"
"github.com/rancherfederal/hauler/pkg/kube"
log "github.com/sirupsen/logrus"
"github.com/rancherfederal/hauler/pkg/log"
"helm.sh/helm/v3/pkg/action"
"helm.sh/helm/v3/pkg/chart"
"helm.sh/helm/v3/pkg/release"
"k8s.io/cli-runtime/pkg/genericclioptions"
"os"
"path/filepath"
"time"
)
func waitForDriver(ctx context.Context, d v1alpha1.Drive) error {
func waitForDriver(ctx context.Context, d driver.Driver) error {
ctx, cancel := context.WithTimeout(ctx, 2*time.Minute)
defer cancel()
//TODO: This is a janky way of waiting for file to exist
path := filepath.Join(d.EtcPath(), "k3s.yaml")
for {
_, err := os.Stat(path)
_, err := os.Stat(d.KubeConfigPath())
if err == nil {
break
}
@@ -48,20 +45,19 @@ func waitForDriver(ctx context.Context, d v1alpha1.Drive) error {
}
//TODO: This is likely way too fleet specific
func installChart(cf *genericclioptions.ConfigFlags, chart *chart.Chart, releaseName, namespace string, vals map[string]interface{}) (*release.Release, error) {
func installChart(cf *BootSettings, chart *chart.Chart, releaseName string, vals map[string]interface{}, logger log.Logger) (*release.Release, error) {
actionConfig := new(action.Configuration)
if err := actionConfig.Init(cf, namespace, os.Getenv("HELM_DRIVER"), log.Debugf); err != nil {
if err := actionConfig.Init(cf.RESTClientGetter(), cf.Namespace, os.Getenv("HELM_DRIVER"), logger.Debugf); err != nil {
return nil, err
}
client := action.NewInstall(actionConfig)
client.ReleaseName = releaseName
client.Namespace, cf.Namespace = namespace, stringptr(namespace) // TODO: Not sure why this needs to be set twice
client.CreateNamespace = true
client.Wait = true
//TODO: Do this better
client.Namespace = cf.Namespace
return client.Run(chart, vals)
}
//still can't figure out why helm does it this way
func stringptr(val string) *string { return &val }

49
pkg/driver/driver.go Normal file
View File

@@ -0,0 +1,49 @@
package driver
import (
"context"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"io"
"sigs.k8s.io/cli-utils/pkg/object"
)
type Driver interface {
Name() string
//TODO: Really want this to just return a usable client
KubeConfigPath() string
Images(ctx context.Context) (map[name.Reference]v1.Image, error)
Binary() (io.ReadCloser, error)
SystemObjects() []object.ObjMetadata
Start(io.Writer) error
DataPath(...string) string
WriteConfig() error
}
//NewDriver will return a new concrete Driver type given a kind
func NewDriver(driver v1alpha1.Driver) (d Driver) {
switch driver.Type {
case "rke2":
// TODO
default:
d = K3s{
Version: driver.Version,
Config: K3sConfig{
DataDir: "/var/lib/rancher/k3s",
KubeConfig: "/etc/rancher/k3s/k3s.yaml",
KubeConfigMode: "0644",
Disable: nil,
},
}
}
return
}

141
scripts/k3s-install.sh → pkg/driver/embed/k3s-init.sh Executable file → Normal file
View File

@@ -1,5 +1,6 @@
#!/bin/sh
set -e
set -o noglob
# Usage:
# curl ... | ENV_VAR=... sh -
@@ -22,6 +23,9 @@ set -e
# - INSTALL_K3S_SKIP_DOWNLOAD
# If set to true will not download k3s hash or binary.
#
# - INSTALL_K3S_FORCE_RESTART
# If set to true will always restart the K3s service
#
# - INSTALL_K3S_SYMLINK
# If set to 'skip' will not create symlinks, 'force' will overwrite,
# default will symlink if command does not exist in path.
@@ -76,6 +80,9 @@ set -e
# - INSTALL_K3S_SELINUX_WARN
# If set to true will continue if k3s-selinux policy is not found.
#
# - INSTALL_K3S_SKIP_SELINUX_RPM
# If set to true will skip automatic installation of the k3s RPM.
#
# - INSTALL_K3S_CHANNEL_URL
# Channel URL for fetching k3s download URL.
# Defaults to 'https://update.k3s.io/v1-release/channels'.
@@ -84,7 +91,7 @@ set -e
# Channel to use for fetching k3s download URL.
# Defaults to 'stable'.
GITHUB_URL=https://github.com/rancher/k3s/releases
GITHUB_URL=https://github.com/k3s-io/k3s/releases
STORAGE_URL=https://storage.googleapis.com/k3s-ci-builds
DOWNLOADER=
@@ -163,8 +170,8 @@ setup_env() {
if [ -z "${K3S_URL}" ]; then
CMD_K3S=server
else
if [ -z "${K3S_TOKEN}" ] && [ -z "${K3S_CLUSTER_SECRET}" ]; then
fatal "Defaulted k3s exec command to 'agent' because K3S_URL is defined, but K3S_TOKEN or K3S_CLUSTER_SECRET is not defined."
if [ -z "${K3S_TOKEN}" ] && [ -z "${K3S_TOKEN_FILE}" ] && [ -z "${K3S_CLUSTER_SECRET}" ]; then
fatal "Defaulted k3s exec command to 'agent' because K3S_URL is defined, but K3S_TOKEN, K3S_TOKEN_FILE or K3S_CLUSTER_SECRET is not defined."
fi
CMD_K3S=agent
fi
@@ -221,7 +228,13 @@ setup_env() {
if [ -n "${INSTALL_K3S_BIN_DIR}" ]; then
BIN_DIR=${INSTALL_K3S_BIN_DIR}
else
# --- use /usr/local/bin if root can write to it, otherwise use /opt/bin if it exists
BIN_DIR=/usr/local/bin
if ! $SUDO sh -c "touch ${BIN_DIR}/k3s-ro-test && rm -rf ${BIN_DIR}/k3s-ro-test"; then
if [ -d /opt/bin ]; then
BIN_DIR=/opt/bin
fi
fi
fi
# --- use systemd directory if defined or create default ---
@@ -266,7 +279,7 @@ can_skip_download() {
fi
}
# --- verify an executabe k3s binary is installed ---
# --- verify an executable k3s binary is installed ---
verify_k3s_is_executable() {
if [ ! -x ${BIN_DIR}/k3s ]; then
fatal "Executable k3s binary not found at ${BIN_DIR}/k3s"
@@ -307,14 +320,14 @@ setup_verify_arch() {
# --- verify existence of network downloader executable ---
verify_downloader() {
# Return failure if it doesn't exist or is no executable
[ -x "$(which $1)" ] || return 1
[ -x "$(command -v $1)" ] || return 1
# Set verified executable as our downloader program and return success
DOWNLOADER=$1
return 0
}
# --- create tempory directory and cleanup when done ---
# --- create temporary directory and cleanup when done ---
setup_tmp() {
TMP_DIR=$(mktemp -d -t k3s-install.XXXXXXXXXX)
TMP_HASH=${TMP_DIR}/k3s.hash
@@ -429,15 +442,38 @@ setup_binary() {
# --- setup selinux policy ---
setup_selinux() {
case ${INSTALL_K3S_CHANNEL} in
*testing)
rpm_channel=testing
;;
*latest)
rpm_channel=latest
;;
*)
rpm_channel=stable
;;
esac
rpm_site="rpm.rancher.io"
if [ "${rpm_channel}" = "testing" ]; then
rpm_site="rpm-testing.rancher.io"
fi
policy_hint="please install:
yum install -y container-selinux selinux-policy-base
rpm -i https://rpm.rancher.io/k3s-selinux-0.1.1-rc1.el7.noarch.rpm
yum install -y https://${rpm_site}/k3s/${rpm_channel}/common/centos/7/noarch/k3s-selinux-0.2-1.el7_8.noarch.rpm
"
policy_error=fatal
if [ "$INSTALL_K3S_SELINUX_WARN" = true ]; then
if [ "$INSTALL_K3S_SELINUX_WARN" = true ] || grep -q 'ID=flatcar' /etc/os-release; then
policy_error=warn
fi
if [ "$INSTALL_K3S_SKIP_SELINUX_RPM" = true ] || can_skip_download; then
info "Skipping installation of SELinux RPM"
else
install_selinux_rpm ${rpm_site} ${rpm_channel}
fi
if ! $SUDO chcon -u system_u -r object_r -t container_runtime_exec_t ${BIN_DIR}/k3s >/dev/null 2>&1; then
if $SUDO grep '^\s*SELINUX=enforcing' /etc/selinux/config >/dev/null 2>&1; then
$policy_error "Failed to apply container_runtime_exec_t to ${BIN_DIR}/k3s, ${policy_hint}"
@@ -449,6 +485,41 @@ setup_selinux() {
fi
}
# --- if on an el7/el8 system, install k3s-selinux
install_selinux_rpm() {
if [ -r /etc/redhat-release ] || [ -r /etc/centos-release ] || [ -r /etc/oracle-release ]; then
dist_version="$(. /etc/os-release && echo "$VERSION_ID")"
maj_ver=$(echo "$dist_version" | sed -E -e "s/^([0-9]+)\.?[0-9]*$/\1/")
set +o noglob
$SUDO rm -f /etc/yum.repos.d/rancher-k3s-common*.repo
set -o noglob
if [ -r /etc/redhat-release ]; then
case ${maj_ver} in
7)
$SUDO yum -y install yum-utils
$SUDO yum-config-manager --enable rhel-7-server-extras-rpms
;;
8)
:
;;
*)
return
;;
esac
fi
$SUDO tee /etc/yum.repos.d/rancher-k3s-common.repo >/dev/null << EOF
[rancher-k3s-common-${2}]
name=Rancher K3s Common (${2})
baseurl=https://${1}/k3s/${2}/common/centos/${maj_ver}/noarch
enabled=1
gpgcheck=1
gpgkey=https://${1}/public.key
EOF
$SUDO yum -y install "k3s-selinux"
fi
return
}
# --- download and verify k3s ---
download_and_verify() {
if can_skip_download; then
@@ -480,7 +551,7 @@ create_symlinks() {
for cmd in kubectl crictl ctr; do
if [ ! -e ${BIN_DIR}/${cmd} ] || [ "${INSTALL_K3S_SYMLINK}" = force ]; then
which_cmd=$(which ${cmd} 2>/dev/null || true)
which_cmd=$(command -v ${cmd} 2>/dev/null || true)
if [ -z "${which_cmd}" ] || [ "${INSTALL_K3S_SYMLINK}" = force ]; then
info "Creating ${BIN_DIR}/${cmd} symlink to k3s"
$SUDO ln -sf k3s ${BIN_DIR}/${cmd}
@@ -545,25 +616,18 @@ getshims() {
killtree $({ set +x; } 2>/dev/null; getshims; set -x)
do_unmount() {
{ set +x; } 2>/dev/null
MOUNTS=
while read ignore mount ignore; do
MOUNTS="$mount\n$MOUNTS"
done </proc/self/mounts
MOUNTS=$(printf $MOUNTS | grep "^$1" | sort -r)
if [ -n "${MOUNTS}" ]; then
set -x
umount ${MOUNTS}
else
set -x
fi
do_unmount_and_remove() {
awk -v path="$1" '$2 ~ ("^" path) { print $2 }' /proc/self/mounts | sort -r | xargs -r -t -n 1 sh -c 'umount "$0" && rm -rf "$0"'
}
do_unmount '/run/k3s'
do_unmount '/var/lib/rancher/k3s'
do_unmount '/var/lib/kubelet/pods'
do_unmount '/run/netns/cni-'
do_unmount_and_remove '/run/k3s'
do_unmount_and_remove '/var/lib/rancher/k3s'
do_unmount_and_remove '/var/lib/kubelet/pods'
do_unmount_and_remove '/var/lib/kubelet/plugins'
do_unmount_and_remove '/run/netns/cni-'
# Remove CNI namespaces
ip netns show 2>/dev/null | grep cni- | xargs -r -t -n 1 ip netns delete
# Delete network interface(s) that match 'master cni0'
ip link show 2>/dev/null | grep 'master cni0' | while read ignore iface ignore; do
@@ -590,12 +654,12 @@ set -x
${KILLALL_K3S_SH}
if which systemctl; then
if command -v systemctl; then
systemctl disable ${SYSTEM_NAME}
systemctl reset-failed ${SYSTEM_NAME}
systemctl daemon-reload
fi
if which rc-update; then
if command -v rc-update; then
rc-update delete ${SYSTEM_NAME} default
fi
@@ -625,6 +689,11 @@ rm -rf /var/lib/rancher/k3s
rm -rf /var/lib/kubelet
rm -f ${BIN_DIR}/k3s
rm -f ${KILLALL_K3S_SH}
if type yum >/dev/null 2>&1; then
yum remove -y k3s-selinux
rm -f /etc/yum.repos.d/rancher-k3s-common*.repo
fi
EOF
$SUDO chmod 755 ${UNINSTALL_K3S_SH}
$SUDO chown root:root ${UNINSTALL_K3S_SH}
@@ -632,19 +701,18 @@ EOF
# --- disable current service if loaded --
systemd_disable() {
$SUDO systemctl disable ${SYSTEM_NAME} >/dev/null 2>&1 || true
$SUDO rm -f /etc/systemd/system/${SERVICE_K3S} || true
$SUDO rm -f /etc/systemd/system/${SERVICE_K3S}.env || true
$SUDO systemctl disable ${SYSTEM_NAME} >/dev/null 2>&1 || true
}
# --- capture current env and create file containing k3s_ variables ---
create_env_file() {
info "env: Creating environment file ${FILE_K3S_ENV}"
UMASK=$(umask)
umask 0377
$SUDO touch ${FILE_K3S_ENV}
$SUDO chmod 0600 ${FILE_K3S_ENV}
env | grep '^K3S_' | $SUDO tee ${FILE_K3S_ENV} >/dev/null
env | egrep -i '^(NO|HTTP|HTTPS)_PROXY' | $SUDO tee -a ${FILE_K3S_ENV} >/dev/null
umask $UMASK
env | grep -Ei '^(NO|HTTP|HTTPS)_PROXY' | $SUDO tee -a ${FILE_K3S_ENV} >/dev/null
}
# --- write systemd service file ---
@@ -655,13 +723,16 @@ create_systemd_service_file() {
Description=Lightweight Kubernetes
Documentation=https://k3s.io
Wants=network-online.target
After=network-online.target
[Install]
WantedBy=multi-user.target
[Service]
Type=${SYSTEMD_TYPE}
EnvironmentFile=${FILE_K3S_ENV}
EnvironmentFile=-/etc/default/%N
EnvironmentFile=-/etc/sysconfig/%N
EnvironmentFile=-${FILE_K3S_ENV}
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
@@ -772,7 +843,7 @@ service_enable_and_start() {
[ "${INSTALL_K3S_SKIP_START}" = true ] && return
POST_INSTALL_HASHES=$(get_installed_hashes)
if [ "${PRE_INSTALL_HASHES}" = "${POST_INSTALL_HASHES}" ]; then
if [ "${PRE_INSTALL_HASHES}" = "${POST_INSTALL_HASHES}" ] && [ "${INSTALL_K3S_FORCE_RESTART}" != true ]; then
info 'No change detected so skipping service start'
return
fi

View File

@@ -0,0 +1,507 @@
#!/bin/sh
set -e
if [ "${DEBUG}" = 1 ]; then
set -x
fi
# Usage:
# curl ... | ENV_VAR=... sh -
# or
# ENV_VAR=... ./install.sh
#
# Environment variables:
#
# - INSTALL_RKE2_CHANNEL
# Channel to use for fetching rke2 download URL.
# Defaults to 'latest'.
#
# - INSTALL_RKE2_METHOD
# The installation method to use.
# Default is on RPM-based systems is "rpm", all else "tar".
#
# - INSTALL_RKE2_TYPE
# Type of rke2 service. Can be either "server" or "agent".
# Default is "server".
#
# - INSTALL_RKE2_EXEC
# This is an alias for INSTALL_RKE2_TYPE, included for compatibility with K3s.
# If both are set, INSTALL_RKE2_TYPE is preferred.
#
# - INSTALL_RKE2_VERSION
# Version of rke2 to download from github.
#
# - INSTALL_RKE2_RPM_RELEASE_VERSION
# Version of the rke2 RPM release to install.
# Format would be like "1.el7" or "2.el8"
#
# - INSTALL_RKE2_TAR_PREFIX
# Installation prefix when using the tar installation method.
# Default is /usr/local, unless /usr/local is read-only or has a dedicated mount point,
# in which case /opt/rke2 is used instead.
#
# - INSTALL_RKE2_COMMIT
# Commit of RKE2 to download from temporary cloud storage.
# If set, this forces INSTALL_RKE2_METHOD=tar.
# * (for developer & QA use)
#
# - INSTALL_RKE2_AGENT_IMAGES_DIR
# Installation path for airgap images when installing from CI commit
# Default is /var/lib/rancher/rke2/agent/images
#
# - INSTALL_RKE2_ARTIFACT_PATH
# If set, the install script will use the local path for sourcing the rke2.linux-$SUFFIX and sha256sum-$ARCH.txt files
# rather than the downloading the files from the internet.
# Default is not set.
#
# info logs the given argument at info log level.
info() {
echo "[INFO] " "$@"
}
# warn logs the given argument at warn log level.
warn() {
echo "[WARN] " "$@" >&2
}
# fatal logs the given argument at fatal log level.
fatal() {
echo "[ERROR] " "$@" >&2
if [ -n "${SUFFIX}" ]; then
echo "[ALT] Please visit 'https://github.com/rancher/rke2/releases' directly and download the latest rke2.${SUFFIX}.tar.gz" >&2
fi
exit 1
}
# check_target_mountpoint return success if the target directory is on a dedicated mount point
check_target_mountpoint() {
mountpoint -q "${INSTALL_RKE2_TAR_PREFIX}"
}
# check_target_ro returns success if the target directory is read-only
check_target_ro() {
touch "${INSTALL_RKE2_TAR_PREFIX}"/.rke2-ro-test && rm -rf "${INSTALL_RKE2_TAR_PREFIX}"/.rke2-ro-test
test $? -ne 0
}
# setup_env defines needed environment variables.
setup_env() {
STORAGE_URL="https://storage.googleapis.com/rke2-ci-builds"
INSTALL_RKE2_GITHUB_URL="https://github.com/rancher/rke2"
DEFAULT_TAR_PREFIX="/usr/local"
# --- bail if we are not root ---
if [ ! $(id -u) -eq 0 ]; then
fatal "You need to be root to perform this install"
fi
# --- make sure install channel has a value
if [ -z "${INSTALL_RKE2_CHANNEL}" ]; then
INSTALL_RKE2_CHANNEL="stable"
fi
# --- make sure install type has a value
if [ -z "${INSTALL_RKE2_TYPE}" ]; then
INSTALL_RKE2_TYPE="${INSTALL_RKE2_EXEC:-server}"
fi
# --- use yum install method if available by default
if [ -z "${INSTALL_RKE2_ARTIFACT_PATH}" ] && [ -z "${INSTALL_RKE2_COMMIT}" ] && [ -z "${INSTALL_RKE2_METHOD}" ] && command -v yum >/dev/null 2>&1; then
INSTALL_RKE2_METHOD="yum"
fi
# --- install tarball to /usr/local by default, except if /usr/local is on a separate partition or is read-only
# --- in which case we go into /opt/rke2.
if [ -z "${INSTALL_RKE2_TAR_PREFIX}" ]; then
INSTALL_RKE2_TAR_PREFIX=${DEFAULT_TAR_PREFIX}
if check_target_mountpoint || check_target_ro; then
INSTALL_RKE2_TAR_PREFIX="/opt/rke2"
warn "${DEFAULT_TAR_PREFIX} is read-only or a mount point; installing to ${INSTALL_RKE2_TAR_PREFIX}"
fi
fi
if [ -z "${INSTALL_RKE2_AGENT_IMAGES_DIR}" ]; then
INSTALL_RKE2_AGENT_IMAGES_DIR="/var/lib/rancher/rke2/agent/images"
fi
}
# check_method_conflict will exit with an error if the user attempts to install
# via tar method on a host with existing RPMs.
check_method_conflict() {
case ${INSTALL_RKE2_METHOD} in
yum | rpm | dnf)
return
;;
*)
if rpm -q rke2-common >/dev/null 2>&1; then
fatal "Cannot perform ${INSTALL_RKE2_METHOD:-tar} install on host with existing RKE2 RPMs - please run rke2-uninstall.sh first"
fi
;;
esac
}
# setup_arch set arch and suffix,
# fatal if architecture not supported.
setup_arch() {
case ${ARCH:=$(uname -m)} in
amd64)
ARCH=amd64
SUFFIX=$(uname -s | tr '[:upper:]' '[:lower:]')-${ARCH}
;;
x86_64)
ARCH=amd64
SUFFIX=$(uname -s | tr '[:upper:]' '[:lower:]')-${ARCH}
;;
*)
fatal "unsupported architecture ${ARCH}"
;;
esac
}
# verify_downloader verifies existence of
# network downloader executable.
verify_downloader() {
cmd="$(command -v "${1}")"
if [ -z "${cmd}" ]; then
return 1
fi
if [ ! -x "${cmd}" ]; then
return 1
fi
# Set verified executable as our downloader program and return success
DOWNLOADER=${cmd}
return 0
}
# setup_tmp creates a temporary directory
# and cleans up when done.
setup_tmp() {
TMP_DIR=$(mktemp -d -t rke2-install.XXXXXXXXXX)
TMP_CHECKSUMS=${TMP_DIR}/rke2.checksums
TMP_TARBALL=${TMP_DIR}/rke2.tarball
TMP_AIRGAP_CHECKSUMS=${TMP_DIR}/rke2-images.checksums
TMP_AIRGAP_TARBALL=${TMP_DIR}/rke2-images.tarball
cleanup() {
code=$?
set +e
trap - EXIT
rm -rf "${TMP_DIR}"
exit $code
}
trap cleanup INT EXIT
}
# --- use desired rke2 version if defined or find version from channel ---
get_release_version() {
if [ -n "${INSTALL_RKE2_COMMIT}" ]; then
version="commit ${INSTALL_RKE2_COMMIT}"
elif [ -n "${INSTALL_RKE2_VERSION}" ]; then
version=${INSTALL_RKE2_VERSION}
else
info "finding release for channel ${INSTALL_RKE2_CHANNEL}"
INSTALL_RKE2_CHANNEL_URL=${INSTALL_RKE2_CHANNEL_URL:-'https://update.rke2.io/v1-release/channels'}
version_url="${INSTALL_RKE2_CHANNEL_URL}/${INSTALL_RKE2_CHANNEL}"
case ${DOWNLOADER} in
*curl)
version=$(${DOWNLOADER} -w "%{url_effective}" -L -s -S "${version_url}" -o /dev/null | sed -e 's|.*/||')
;;
*wget)
version=$(${DOWNLOADER} -SqO /dev/null "${version_url}" 2>&1 | grep -i Location | sed -e 's|.*/||')
;;
*)
fatal "Unsupported downloader executable '${DOWNLOADER}'"
;;
esac
INSTALL_RKE2_VERSION="${version}"
fi
}
# check_download performs a HEAD request to see if a file exists at a given url
check_download() {
case ${DOWNLOADER} in
*curl)
curl -o "/dev/null" -fsLI -X HEAD "$1"
;;
*wget)
wget -q --spider "$1"
;;
*)
fatal "downloader executable not supported: '${DOWNLOADER}'"
;;
esac
}
# download downloads a file from a url using either curl or wget
download() {
if [ $# -ne 2 ]; then
fatal "download needs exactly 2 arguments"
fi
case ${DOWNLOADER} in
*curl)
curl -o "$1" -fsSL "$2"
;;
*wget)
wget -qO "$1" "$2"
;;
*)
fatal "downloader executable not supported: '${DOWNLOADER}'"
;;
esac
# Abort if download command failed
if [ $? -ne 0 ]; then
fatal "download failed"
fi
}
# download_checksums downloads hash from github url.
download_checksums() {
if [ -n "${INSTALL_RKE2_COMMIT}" ]; then
CHECKSUMS_URL=${STORAGE_URL}/rke2.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz.sha256sum
else
CHECKSUMS_URL=${INSTALL_RKE2_GITHUB_URL}/releases/download/${INSTALL_RKE2_VERSION}/sha256sum-${ARCH}.txt
fi
info "downloading checksums at ${CHECKSUMS_URL}"
download "${TMP_CHECKSUMS}" "${CHECKSUMS_URL}"
CHECKSUM_EXPECTED=$(grep "rke2.${SUFFIX}.tar.gz" "${TMP_CHECKSUMS}" | awk '{print $1}')
}
# download_tarball downloads binary from github url.
download_tarball() {
if [ -n "${INSTALL_RKE2_COMMIT}" ]; then
TARBALL_URL=${STORAGE_URL}/rke2.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz
else
TARBALL_URL=${INSTALL_RKE2_GITHUB_URL}/releases/download/${INSTALL_RKE2_VERSION}/rke2.${SUFFIX}.tar.gz
fi
info "downloading tarball at ${TARBALL_URL}"
download "${TMP_TARBALL}" "${TARBALL_URL}"
}
# stage_local_checksums stages the local checksum hash for validation.
stage_local_checksums() {
info "staging local checksums from ${INSTALL_RKE2_ARTIFACT_PATH}/sha256sum-${ARCH}.txt"
cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/sha256sum-${ARCH}.txt" "${TMP_CHECKSUMS}"
CHECKSUM_EXPECTED=$(grep "rke2.${SUFFIX}.tar.gz" "${TMP_CHECKSUMS}" | awk '{print $1}')
if [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst" ]; then
AIRGAP_CHECKSUM_EXPECTED=$(grep "rke2-images.${SUFFIX}.tar.zst" "${TMP_CHECKSUMS}" | awk '{print $1}')
elif [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz" ]; then
AIRGAP_CHECKSUM_EXPECTED=$(grep "rke2-images.${SUFFIX}.tar.gz" "${TMP_CHECKSUMS}" | awk '{print $1}')
fi
}
# stage_local_tarball stages the local tarball.
stage_local_tarball() {
info "staging tarball from ${INSTALL_RKE2_ARTIFACT_PATH}/rke2.${SUFFIX}.tar.gz"
cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2.${SUFFIX}.tar.gz" "${TMP_TARBALL}"
}
# stage_local_airgap_tarball stages the local checksum hash for validation.
stage_local_airgap_tarball() {
if [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst" ]; then
info "staging zst airgap image tarball from ${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst"
cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.zst" "${TMP_AIRGAP_TARBALL}"
AIRGAP_TARBALL_FORMAT=zst
elif [ -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz" ]; then
info "staging gzip airgap image tarball from ${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz"
cp -f "${INSTALL_RKE2_ARTIFACT_PATH}/rke2-images.${SUFFIX}.tar.gz" "${TMP_AIRGAP_TARBALL}"
AIRGAP_TARBALL_FORMAT=gz
fi
}
# verify_tarball verifies the downloaded installer checksum.
verify_tarball() {
info "verifying tarball"
CHECKSUM_ACTUAL=$(sha256sum "${TMP_TARBALL}" | awk '{print $1}')
if [ "${CHECKSUM_EXPECTED}" != "${CHECKSUM_ACTUAL}" ]; then
fatal "download sha256 does not match ${CHECKSUM_EXPECTED}, got ${CHECKSUM_ACTUAL}"
fi
}
# unpack_tarball extracts the tarball, correcting paths and moving systemd units as necessary
unpack_tarball() {
info "unpacking tarball file to ${INSTALL_RKE2_TAR_PREFIX}"
mkdir -p ${INSTALL_RKE2_TAR_PREFIX}
tar xzf "${TMP_TARBALL}" -C "${INSTALL_RKE2_TAR_PREFIX}"
if [ "${INSTALL_RKE2_TAR_PREFIX}" != "${DEFAULT_TAR_PREFIX}" ]; then
info "updating tarball contents to reflect install path"
sed -i "s|${DEFAULT_TAR_PREFIX}|${INSTALL_RKE2_TAR_PREFIX}|" ${INSTALL_RKE2_TAR_PREFIX}/lib/systemd/system/rke2-*.service ${INSTALL_RKE2_TAR_PREFIX}/bin/rke2-uninstall.sh
info "moving systemd units to /etc/systemd/system"
mv -f ${INSTALL_RKE2_TAR_PREFIX}/lib/systemd/system/rke2-*.service /etc/systemd/system/
info "install complete; you may want to run: export PATH=\$PATH:${INSTALL_RKE2_TAR_PREFIX}/bin"
fi
}
# download_airgap_checksums downloads the checksum file for the airgap image tarball
# and prepares the checksum value for later validation.
download_airgap_checksums() {
if [ -z "${INSTALL_RKE2_COMMIT}" ]; then
return
fi
AIRGAP_CHECKSUMS_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.zst.sha256sum
# try for zst first; if that fails use gz for older release branches
if ! check_download "${AIRGAP_CHECKSUMS_URL}"; then
AIRGAP_CHECKSUMS_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz.sha256sum
fi
info "downloading airgap checksums at ${AIRGAP_CHECKSUMS_URL}"
download "${TMP_AIRGAP_CHECKSUMS}" "${AIRGAP_CHECKSUMS_URL}"
AIRGAP_CHECKSUM_EXPECTED=$(grep "rke2-images.${SUFFIX}.tar" "${TMP_AIRGAP_CHECKSUMS}" | awk '{print $1}')
}
# download_airgap_tarball downloads the airgap image tarball.
download_airgap_tarball() {
if [ -z "${INSTALL_RKE2_COMMIT}" ]; then
return
fi
AIRGAP_TARBALL_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.zst
# try for zst first; if that fails use gz for older release branches
if ! check_download "${AIRGAP_TARBALL_URL}"; then
AIRGAP_TARBALL_URL=${STORAGE_URL}/rke2-images.${SUFFIX}-${INSTALL_RKE2_COMMIT}.tar.gz
fi
info "downloading airgap tarball at ${AIRGAP_TARBALL_URL}"
download "${TMP_AIRGAP_TARBALL}" "${AIRGAP_TARBALL_URL}"
}
# verify_airgap_tarball compares the airgap image tarball checksum to the value
# calculated by CI when the file was uploaded.
verify_airgap_tarball() {
if [ -z "${AIRGAP_CHECKSUM_EXPECTED}" ]; then
return
fi
info "verifying airgap tarball"
AIRGAP_CHECKSUM_ACTUAL=$(sha256sum "${TMP_AIRGAP_TARBALL}" | awk '{print $1}')
if [ "${AIRGAP_CHECKSUM_EXPECTED}" != "${AIRGAP_CHECKSUM_ACTUAL}" ]; then
fatal "download sha256 does not match ${AIRGAP_CHECKSUM_EXPECTED}, got ${AIRGAP_CHECKSUM_ACTUAL}"
fi
}
# install_airgap_tarball moves the airgap image tarball into place.
install_airgap_tarball() {
if [ -z "${AIRGAP_CHECKSUM_EXPECTED}" ]; then
return
fi
mkdir -p "${INSTALL_RKE2_AGENT_IMAGES_DIR}"
# releases that provide zst artifacts can read from the compressed archive; older releases
# that produce only gzip artifacts need to have the tarball decompressed ahead of time
if grep -qF '.tar.zst' "${TMP_AIRGAP_CHECKSUMS}" || [ "${AIRGAP_TARBALL_FORMAT}" = "zst" ]; then
info "installing airgap tarball to ${INSTALL_RKE2_AGENT_IMAGES_DIR}"
mv -f "${TMP_AIRGAP_TARBALL}" "${INSTALL_RKE2_AGENT_IMAGES_DIR}/rke2-images.${SUFFIX}.tar.zst"
else
info "decompressing airgap tarball to ${INSTALL_RKE2_AGENT_IMAGES_DIR}"
gzip -dc "${TMP_AIRGAP_TARBALL}" > "${INSTALL_RKE2_AGENT_IMAGES_DIR}/rke2-images.${SUFFIX}.tar"
fi
}
# do_install_rpm builds a yum repo config from the channel and version to be installed,
# and calls yum to install the required packates.
do_install_rpm() {
maj_ver="7"
if [ -r /etc/redhat-release ] || [ -r /etc/centos-release ] || [ -r /etc/oracle-release ]; then
dist_version="$(. /etc/os-release && echo "$VERSION_ID")"
maj_ver=$(echo "$dist_version" | sed -E -e "s/^([0-9]+)\.?[0-9]*$/\1/")
case ${maj_ver} in
7|8)
:
;;
*) # In certain cases, like installing on Fedora, maj_ver will end up being something that is not 7 or 8
maj_ver="7"
;;
esac
fi
case "${INSTALL_RKE2_CHANNEL}" in
v*.*)
# We are operating with a version-based channel, so we should parse our version out
rke2_majmin=$(echo "${INSTALL_RKE2_CHANNEL}" | sed -E -e "s/^v([0-9]+\.[0-9]+).*/\1/")
rke2_rpm_channel=$(echo "${INSTALL_RKE2_CHANNEL}" | sed -E -e "s/^v[0-9]+\.[0-9]+-(.*)/\1/")
# If our regex fails to capture a "sane" channel out of the specified channel, fall back to `stable`
if [ "${rke2_rpm_channel}" = ${INSTALL_RKE2_CHANNEL} ]; then
info "using stable RPM repositories"
rke2_rpm_channel="stable"
fi
;;
*)
get_release_version
rke2_majmin=$(echo "${INSTALL_RKE2_VERSION}" | sed -E -e "s/^v([0-9]+\.[0-9]+).*/\1/")
rke2_rpm_channel=${1}
;;
esac
info "using ${rke2_majmin} series from channel ${rke2_rpm_channel}"
rpm_site="rpm.rancher.io"
if [ "${rke2_rpm_channel}" = "testing" ]; then
rpm_site="rpm-${rke2_rpm_channel}.rancher.io"
fi
rm -f /etc/yum.repos.d/rancher-rke2*.repo
cat <<-EOF >"/etc/yum.repos.d/rancher-rke2.repo"
[rancher-rke2-common-${rke2_rpm_channel}]
name=Rancher RKE2 Common (${1})
baseurl=https://${rpm_site}/rke2/${rke2_rpm_channel}/common/centos/${maj_ver}/noarch
enabled=1
gpgcheck=1
gpgkey=https://${rpm_site}/public.key
[rancher-rke2-${rke2_majmin}-${rke2_rpm_channel}]
name=Rancher RKE2 ${rke2_majmin} (${1})
baseurl=https://${rpm_site}/rke2/${rke2_rpm_channel}/${rke2_majmin}/centos/${maj_ver}/x86_64
enabled=1
gpgcheck=1
gpgkey=https://${rpm_site}/public.key
EOF
if [ -z "${INSTALL_RKE2_VERSION}" ]; then
yum -y install "rke2-${INSTALL_RKE2_TYPE}"
else
rke2_rpm_version=$(echo "${INSTALL_RKE2_VERSION}" | sed -E -e "s/[\+-]/~/g" | sed -E -e "s/v(.*)/\1/")
if [ -n "${INSTALL_RKE2_RPM_RELEASE_VERSION}" ]; then
yum -y install "rke2-${INSTALL_RKE2_TYPE}-${rke2_rpm_version}-${INSTALL_RKE2_RPM_RELEASE_VERSION}"
else
yum -y install "rke2-${INSTALL_RKE2_TYPE}-${rke2_rpm_version}"
fi
fi
}
do_install_tar() {
setup_tmp
if [ -n "${INSTALL_RKE2_ARTIFACT_PATH}" ]; then
stage_local_checksums
stage_local_airgap_tarball
stage_local_tarball
else
get_release_version
info "using ${INSTALL_RKE2_VERSION:-commit $INSTALL_RKE2_COMMIT} as release"
download_airgap_checksums
download_airgap_tarball
download_checksums
download_tarball
fi
verify_airgap_tarball
install_airgap_tarball
verify_tarball
unpack_tarball
systemctl daemon-reload
}
do_install() {
setup_env
check_method_conflict
setup_arch
if [ -z "${INSTALL_RKE2_ARTIFACT_PATH}" ]; then
verify_downloader curl || verify_downloader wget || fatal "can not find curl or wget for downloading files"
fi
case ${INSTALL_RKE2_METHOD} in
yum | rpm | dnf)
do_install_rpm "${INSTALL_RKE2_CHANNEL}"
;;
*)
do_install_tar "${INSTALL_RKE2_CHANNEL}"
;;
esac
}
do_install
exit 0

173
pkg/driver/k3s.go Normal file
View File

@@ -0,0 +1,173 @@
package driver
import (
"bufio"
"context"
_ "embed"
"fmt"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/imdario/mergo"
"github.com/rancherfederal/hauler/pkg/packager/images"
"io"
"k8s.io/apimachinery/pkg/runtime/schema"
"net/http"
"net/url"
"os"
"os/exec"
"path/filepath"
"sigs.k8s.io/cli-utils/pkg/object"
"sigs.k8s.io/yaml"
)
const (
k3sReleaseUrl = "https://github.com/k3s-io/k3s/releases/download"
)
//go:embed embed/k3s-init.sh
var k3sInit string
type K3s struct {
Version string
Config K3sConfig
}
//TODO: Would be nice if these just pointed to k3s/pkg/cli/cmds
type K3sConfig struct {
DataDir string `json:"data-dir,omitempty"`
KubeConfig string `json:"write-kubeconfig,omitempty"`
KubeConfigMode string `json:"write-kubeconfig-mode,omitempty"`
Disable []string `json:"disable,omitempty"`
}
//NewK3s returns a new k3s driver
func NewK3s() K3s {
//TODO: Allow for configuration overrides
return K3s{
Config: K3sConfig{
DataDir: "/var/lib/rancher/k3s",
KubeConfig: "/etc/rancher/k3s/k3s.yaml",
KubeConfigMode: "0644",
Disable: []string{},
},
}
}
func (k K3s) Name() string { return "k3s" }
func (k K3s) KubeConfigPath() string { return k.Config.KubeConfig }
func (k K3s) DataPath(elem ...string) string {
base := []string{k.Config.DataDir}
return filepath.Join(append(base, elem...)...)
}
func (k K3s) WriteConfig() error {
kCfgPath := filepath.Dir(k.Config.KubeConfig)
if err := os.MkdirAll(kCfgPath, os.ModePerm); err != nil {
return err
}
data, err := yaml.Marshal(k.Config)
c := make(map[string]interface{})
if err := yaml.Unmarshal(data, &c); err != nil {
return err
}
var uc map[string]interface{}
path := filepath.Join(kCfgPath, "config.yaml")
if data, err := os.ReadFile(path); err != nil {
err := yaml.Unmarshal(data, &uc)
if err != nil {
return err
}
}
//Merge with user defined configs taking precedence
if err := mergo.Merge(&c, uc); err != nil {
return err
}
mergedData, err := yaml.Marshal(&c)
if err != nil {
return err
}
return os.WriteFile(path, mergedData, 0644)
}
func (k K3s) Images(ctx context.Context) (map[name.Reference]v1.Image, error) {
imgs, err := k.listImages()
if err != nil {
return nil, err
}
return images.ResolveRemoteRefs(imgs...)
}
func (k K3s) Binary() (io.ReadCloser, error) {
u, err := url.Parse(fmt.Sprintf("%s/%s/%s", k3sReleaseUrl, k.Version, k.Name()))
if err != nil {
return nil, err
}
resp, err := http.Get(u.String())
if err != nil || resp.StatusCode != 200 {
return nil, fmt.Errorf("failed to return executable for k3s %s from %s", k.Version, u.String())
}
return resp.Body, nil
}
//SystemObjects returns a slice of object.ObjMetadata required for driver to be functional and accept new resources
//hauler's bootstrapping sequence will always wait for SystemObjects to be in a Ready status before proceeding
func (k K3s) SystemObjects() (objs []object.ObjMetadata) {
for _, dep := range []string{"coredns"} {
objMeta, _ := object.CreateObjMetadata("kube-system", dep, schema.GroupKind{Kind: "Deployment", Group: "apps"})
objs = append(objs, objMeta)
}
return objs
}
func (k K3s) Start(out io.Writer) error {
if err := os.WriteFile("/opt/hauler/bin/k3s-init.sh", []byte(k3sInit), 0755); err != nil {
return err
}
cmd := exec.Command("/bin/sh", "/opt/hauler/bin/k3s-init.sh")
cmd.Env = append(os.Environ(), []string{
"INSTALL_K3S_SKIP_DOWNLOAD=true",
"INSTALL_K3S_SELINUX_WARN=true",
"INSTALL_K3S_SKIP_SELINUX_RPM=true",
"INSTALL_K3S_BIN_DIR=/opt/hauler/bin",
//TODO: Provide a real dryrun option
//"INSTALL_K3S_SKIP_START=true",
}...)
cmd.Stdout = out
return cmd.Run()
}
func (k K3s) listImages() ([]string, error) {
u, err := url.Parse(fmt.Sprintf("%s/%s/k3s-images.txt", k3sReleaseUrl, k.Version))
if err != nil {
return nil, err
}
resp, err := http.Get(u.String())
if err != nil || resp.StatusCode != 200 {
return nil, fmt.Errorf("failed to return images for k3s %s from %s", k.Version, u.String())
}
defer resp.Body.Close()
var imgs []string
scanner := bufio.NewScanner(resp.Body)
for scanner.Scan() {
imgs = append(imgs, scanner.Text())
}
return imgs, nil
}

View File

@@ -70,33 +70,26 @@ func (p PkgFs) Chart() PkgFs {
}
//AddBundle will add a bundle to a package and all images that are autodetected from it
func (p PkgFs) AddBundle(b *fleetapi.Bundle) error {
func (p PkgFs) AddBundle(b *fleetapi.Bundle) (map[name.Reference]v1.Image, error) {
if err := p.mkdirIfNotExists(v1alpha1.BundlesDir, os.ModePerm); err != nil {
return err
return nil, err
}
data, err := json.Marshal(b)
if err != nil {
return err
return nil, err
}
if err := p.Bundle().WriteFile(fmt.Sprintf("%s.json", b.Name), data, 0644); err != nil {
return err
return nil, err
}
imgs, err := images.ImageMapFromBundle(b)
if err != nil {
return err
return nil, err
}
for k, v := range imgs {
err := p.AddImage(k, v)
if err != nil {
return err
}
}
return nil
return imgs, nil
}
func (p PkgFs) AddBin(r io.Reader, name string) error {

View File

@@ -1,7 +1,7 @@
package log
import (
"github.com/sirupsen/logrus"
"github.com/pterm/pterm"
"io"
)
@@ -10,12 +10,12 @@ type Logger interface {
Infof(string, ...interface{})
Warnf(string, ...interface{})
Debugf(string, ...interface{})
WithFields(logrus.Fields) *logrus.Entry
Successf(string, ...interface{})
}
type standardLogger struct {
*logrus.Logger
//TODO: Actually check this
level string
}
type Event struct {
@@ -28,10 +28,44 @@ var (
)
func NewLogger(out io.Writer) *standardLogger {
logger := logrus.New()
logger.SetOutput(out)
return &standardLogger{}
}
return &standardLogger{logger}
func (l *standardLogger) Errorf(format string, args ...interface{}) {
l.logf("error", format, args...)
}
func (l *standardLogger) Infof(format string, args ...interface{}) {
l.logf("info", format, args...)
}
func (l *standardLogger) Warnf(format string, args ...interface{}) {
l.logf("warn", format, args...)
}
func (l *standardLogger) Debugf(format string, args ...interface{}) {
l.logf("debug", format, args...)
}
func (l *standardLogger) Successf(format string, args ...interface{}) {
l.logf("success", format, args...)
}
func (l *standardLogger) logf(level string, format string, args ...interface{}) {
switch level {
case "debug":
pterm.Debug.Printfln(format, args...)
case "info":
pterm.Info.Printfln(format, args...)
case "warn":
pterm.Warning.Printfln(format, args...)
case "success":
pterm.Success.Printfln(format, args...)
case "error":
pterm.Error.Printfln(format, args...)
default:
pterm.Error.Printfln("%s is not a valid log level", level)
}
}
func (l *standardLogger) InvalidArg(arg string) {

53
pkg/oci/layout_test.go Normal file
View File

@@ -0,0 +1,53 @@
package oci
import (
"fmt"
"testing"
v1 "github.com/google/go-containerregistry/pkg/v1"
"github.com/google/go-containerregistry/pkg/v1/layout"
"github.com/google/go-containerregistry/pkg/v1/random"
)
func Test_ListImages(t *testing.T) {
img, err := random.Image(1024, 5)
if err != nil {
fmt.Printf("error creating test image: %v", err)
}
ly := createLayout(img, ".")
dg := getDigest(img)
m := ListImages(ly)
for _, hash := range m {
if hash != dg {
t.Errorf("error got %v want %v", hash, dg)
}
}
}
func createLayout(img v1.Image, path string) layout.Path {
p, err := layout.FromPath(path)
if err != nil {
fmt.Printf("error creating layout: %v", err)
}
p.AppendImage(img)
return p
}
func getDigest(img v1.Image) v1.Hash {
digest, err := img.Digest()
if err != nil {
fmt.Printf("error getting digest: %v", err)
}
return digest
}

View File

@@ -1 +1,59 @@
package oci
import (
"context"
"fmt"
"io/ioutil"
"net/http/httptest"
"net/url"
"os"
"testing"
"time"
"github.com/google/go-containerregistry/pkg/registry"
)
const timeout = 1 * time.Minute
func Test_Get_Put(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
// Set up a fake registry.
s := httptest.NewServer(registry.New())
defer s.Close()
u, err := url.Parse(s.URL)
if err != nil {
t.Fatal(err)
}
file, err := ioutil.TempFile(".", "artifact.txt")
if err != nil {
t.Fatal(err)
}
text := []byte("Some stuff!")
if _, err = file.Write(text); err != nil {
t.Fatal(err)
}
img := fmt.Sprintf("%s/artifact:latest", u.Host)
if err := Put(ctx, file.Name(), img); err != nil {
t.Fatal(err)
}
dir, err := ioutil.TempDir(".", "tmp")
if err != nil {
t.Fatal(err)
}
if err := Get(ctx, img, dir); err != nil {
t.Fatal(err)
}
defer os.Remove(file.Name())
defer os.RemoveAll(dir)
}

View File

@@ -50,14 +50,28 @@ func MapImager(imager ...Imager) (map[name.Reference]v1.Image, error) {
}
func ImageMapFromBundle(b *fleetapi.Bundle) (map[name.Reference]v1.Image, error) {
opts := fleetapi.BundleDeploymentOptions{
DefaultNamespace: "default",
opts := fleetapi.BundleDeploymentOptions{}
//TODO: Why doesn't fleet do this...
if b.Spec.Helm != nil {
opts.Helm = b.Spec.Helm
}
m := &manifest.Manifest{Resources: b.Spec.Resources}
if b.Spec.Kustomize != nil {
opts.Kustomize = b.Spec.Kustomize
}
if b.Spec.YAML != nil {
opts.YAML = b.Spec.YAML
}
m, err := manifest.New(&b.Spec)
if err != nil {
return nil, err
}
//TODO: I think this is right?
objs, err := helmdeployer.Template("anything", m, opts)
objs, err := helmdeployer.Template(b.Name, m, opts)
if err != nil {
return nil, err
}
@@ -74,32 +88,6 @@ func ImageMapFromBundle(b *fleetapi.Bundle) (map[name.Reference]v1.Image, error)
return ResolveRemoteRefs(di...)
}
func IdentifyImages(b *fleetapi.Bundle) (discoveredImages, error) {
opts := fleetapi.BundleDeploymentOptions{
DefaultNamespace: "default",
}
m := &manifest.Manifest{Resources: b.Spec.Resources}
//TODO: I think this is right?
objs, err := helmdeployer.Template("anything", m, opts)
if err != nil {
return nil, err
}
var di discoveredImages
for _, o := range objs {
imgs, err := imageFromRuntimeObject(o.(*unstructured.Unstructured))
if err != nil {
return nil, err
}
di = append(di, imgs...)
}
return di, err
}
//ResolveRemoteRefs will return a slice of remote images resolved from their fully qualified name
func ResolveRemoteRefs(images ...string) (map[name.Reference]v1.Image, error) {
m := make(map[name.Reference]v1.Image)
@@ -137,7 +125,7 @@ var knownImagePaths = []string{
"{.spec.containers[*].image}",
}
////imageFromRuntimeObject will return any images found in known obj specs
//imageFromRuntimeObject will return any images found in known obj specs
func imageFromRuntimeObject(obj *unstructured.Unstructured) (images []string, err error) {
objData, _ := obj.MarshalJSON()
@@ -170,6 +158,7 @@ func parseJSONPath(input interface{}, parser *jsonpath.JSONPath, template string
return nil, err
}
r := strings.Split(buf.String(), " ")
f := func(s rune) bool { return s == ' ' }
r := strings.FieldsFunc(buf.String(), f)
return r, nil
}

View File

@@ -2,32 +2,42 @@ package packager
import (
"context"
"github.com/google/go-containerregistry/pkg/name"
v1 "github.com/google/go-containerregistry/pkg/v1"
fleetapi "github.com/rancher/fleet/pkg/apis/fleet.cattle.io/v1alpha1"
"github.com/rancher/fleet/pkg/bundle"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/driver"
"github.com/rancherfederal/hauler/pkg/fs"
"github.com/rancherfederal/hauler/pkg/log"
"github.com/rancherfederal/hauler/pkg/packager/images"
"io"
"k8s.io/apimachinery/pkg/util/json"
"net/http"
"path/filepath"
)
type Packager interface {
Bundles(context.Context, ...string) ([]*fleetapi.Bundle, error)
Driver(context.Context, v1alpha1.Drive) error
Fleet(context.Context, v1alpha1.Fleet) error
Archive(Archiver, v1alpha1.Package, string) error
PackageBundles(context.Context, ...string) ([]*fleetapi.Bundle, error)
PackageDriver(context.Context, driver.Driver) error
PackageFleet(context.Context, v1alpha1.Fleet) error
PackageImages(context.Context, ...string) error
}
type pkg struct {
fs fs.PkgFs
logger log.Logger
}
//NewPackager loads a new packager given a path on disk
func NewPackager(path string) Packager {
func NewPackager(path string, logger log.Logger) Packager {
return pkg{
fs: fs.NewPkgFS(path),
fs: fs.NewPkgFS(path),
logger: logger,
}
}
@@ -44,11 +54,19 @@ func (p pkg) Archive(a Archiver, pkg v1alpha1.Package, output string) error {
return Package(a, p.fs.Path(), output)
}
func (p pkg) Bundles(ctx context.Context, path ...string) ([]*fleetapi.Bundle, error) {
opts := &bundle.Options{Compress: true}
func (p pkg) PackageBundles(ctx context.Context, path ...string) ([]*fleetapi.Bundle, error) {
p.logger.Infof("Packaging %d bundle(s)", len(path))
opts := &bundle.Options{
Compress: true,
}
var cImgs int
var bundles []*fleetapi.Bundle
for _, pth := range path {
p.logger.Infof("Creating bundle from path: %s", pth)
bundleName := filepath.Base(pth)
fb, err := bundle.Open(ctx, bundleName, pth, "", opts)
if err != nil {
@@ -57,85 +75,103 @@ func (p pkg) Bundles(ctx context.Context, path ...string) ([]*fleetapi.Bundle, e
//TODO: Figure out why bundle.Open doesn't return with GVK
bn := fleetapi.NewBundle("fleet-local", bundleName, *fb.Definition)
if err := p.fs.AddBundle(bn); err != nil {
imgs, err := p.fs.AddBundle(bn)
if err != nil {
return nil, err
}
if err := p.pkgImages(ctx, imgs); err != nil {
return nil, err
}
bundles = append(bundles, bn)
cImgs += len(imgs)
}
p.logger.Successf("Finished packaging %d bundle(s) along with %d autodetected image(s)", len(path), cImgs)
return bundles, nil
}
func (p pkg) Driver(ctx context.Context, d v1alpha1.Drive) error {
if err := writeURL(p.fs, d.BinURL(), "k3s"); err != nil {
return err
}
func (p pkg) PackageDriver(ctx context.Context, d driver.Driver) error {
p.logger.Infof("Packaging %s components", d.Name())
//TODO: Stop hardcoding
if err := writeURL(p.fs, "https://get.k3s.io", "k3s-init.sh"); err != nil {
return err
}
imgMap, err := images.MapImager(d)
p.logger.Infof("Adding %s executable to package", d.Name())
rc, err := d.Binary()
if err != nil {
return err
}
for ref, im := range imgMap {
err := p.fs.AddImage(ref, im)
if err != nil {
return err
}
if err := p.fs.AddBin(rc, d.Name()); err != nil {
return err
}
rc.Close()
p.logger.Infof("Adding required images for %s to package", d.Name())
imgMap, err := d.Images(ctx)
if err != nil {
return err
}
err = p.pkgImages(ctx, imgMap)
if err != nil {
return err
}
p.logger.Successf("Finished packaging %s components", d.Name())
return nil
}
//TODO: Add this to Driver?
func (p pkg) Fleet(ctx context.Context, fl v1alpha1.Fleet) error {
func (p pkg) PackageImages(ctx context.Context, imgs ...string) error {
p.logger.Infof("Packaging %d user defined images", len(imgs))
imgMap, err := images.ResolveRemoteRefs(imgs...)
if err != nil {
return err
}
if err := p.pkgImages(ctx, imgMap); err != nil {
return err
}
p.logger.Successf("Finished packaging %d user defined images", len(imgs))
return nil
}
//TODO: Add this to PackageDriver?
func (p pkg) PackageFleet(ctx context.Context, fl v1alpha1.Fleet) error {
p.logger.Infof("Packaging fleet components")
imgMap, err := images.MapImager(fl)
if err != nil {
return err
}
for ref, im := range imgMap {
err := p.fs.AddImage(ref, im)
if err != nil {
return err
}
if err := p.pkgImages(ctx, imgMap); err != nil {
return err
}
p.logger.Infof("Adding fleet crds to package")
if err := p.fs.AddChart(fl.CRDChart(), fl.Version); err != nil {
return err
}
p.logger.Infof("Adding fleet to package")
if err := p.fs.AddChart(fl.Chart(), fl.Version); err != nil {
return err
}
p.logger.Successf("Finished packaging fleet components")
return nil
}
func writeURL(fsys fs.PkgFs, rawURL string, name string) error {
rc, err := fetchURL(rawURL)
if err != nil {
return err
//pkgImages is a helper function to loop through an image map and add it to a layout
func (p pkg) pkgImages(ctx context.Context, imgMap map[name.Reference]v1.Image) error {
var i int
for ref, im := range imgMap {
p.logger.Infof("Packaging image (%d/%d): %s", i+1, len(imgMap), ref.Name())
if err := p.fs.AddImage(ref, im); err != nil {
return err
}
i++
}
defer rc.Close()
return fsys.AddBin(rc, name)
}
func fetchURL(rawURL string) (io.ReadCloser, error) {
resp, err := http.Get(rawURL)
if err != nil {
return nil, err
}
if resp.StatusCode != 200 {
return nil, err
}
return resp.Body, nil
return nil
}

View File

@@ -0,0 +1,37 @@
package packager
import (
"context"
"testing"
"github.com/rancherfederal/hauler/pkg/apis/hauler.cattle.io/v1alpha1"
"github.com/rancherfederal/hauler/pkg/fs"
)
func Test_pkg_driver(t *testing.T) {
type fields struct {
fs fs.PkgFs
}
type args struct {
ctx context.Context
d v1alpha1.Driver
}
tests := []struct {
name string
fields fields
args args
wantErr bool
}{
// TODO: Add test cases.
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
p := pkg{
fs: tt.fields.fs,
}
if err := p.driver(tt.args.ctx, tt.args.d); (err != nil) != tt.wantErr {
t.Errorf("driver() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}

8
testdata/custom/fleet.yaml vendored Normal file
View File

@@ -0,0 +1,8 @@
#defaultNamespace: fleet-system
#helm:
# chart: https://github.com/rancher/fleet/releases/download/v0.3.5/fleet-agent-0.3.5.tgz
# releaseName: fleet-agent
helm:
repo: https://charts.longhorn.io
chart: longhorn