Files
hauler/pkg/getter/getter.go
Zack Brady c0294c733b update release/2.0 from main (#546)
* fix: handling of file referenced dependencies without repository field (#514)

co-authored-by: devleitner <devleitner@protonmail.com>

* bump go.opentelemetry.io/otel/sdk (#520)

bumps the go_modules group with 1 update in the / directory: [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go).

updates `go.opentelemetry.io/otel/sdk` from 1.39.0 to 1.40.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.39.0...v1.40.0)

---

updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.40.0
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* dev.md file (#521)

* smaller changes and updates for v1.4.2 release (#524)

* smaller changes and updates for v1.4.2 release
* removed unused env variable

* over-"haul": replace oras v1 and cosign fork with native containerd-based implementation (#515)

* remove oras from hauler

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* remove cosign fork and use upstream cosign for verification

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* added support for oci referrers

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* updated README.md projects list

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* updates for copilot PR review

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for unsafe type assertions

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for http getter and dead code

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* fixes for more clarity and better error handling

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for resource leaks and unchecked errors

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for rewrite logic for docker.io images due to cosign removal

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for sigs and referrers

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for index.json missing mediatype

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix to make sure manifest.json doesnt include anything other than actual container images

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bump github.com/docker/cli in the go_modules group across 1 directory (#526)

bumps the go_modules group with 1 update in the / directory: [github.com/docker/cli](https://github.com/docker/cli).


updates `github.com/docker/cli` from 29.0.3+incompatible to 29.2.0+incompatible
- [Commits](https://github.com/docker/cli/compare/v29.0.3...v29.2.0)

---

updated-dependencies:
- dependency-name: github.com/docker/cli
  dependency-version: 29.2.0+incompatible
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* removed deprecated code (#528)

* removed deprecated code
* removed all supported for v1alpha1

* fix extract for oci files (#529)

* fix extract for oci files

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* have extract guard against path traversal

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* improved test coverage (#530)

* improved test coverage

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* adjusted mapper_test for oddball oci files

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* adjust extract to handle an image index appropriately (#531)

* adjust extract to handle images and image indices appropriately

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* updates for review feedback

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* fix dockerhub default host bug (#534)

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* adjust hauler's kind annotation to not reflect cosign (#535)

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bump google.golang.org/grpc in the go_modules group across 1 directory (#536)

bumps the go_modules group with 1 update in the / directory: [google.golang.org/grpc](https://github.com/grpc/grpc-go).

updates `google.golang.org/grpc` from 1.78.0 to 1.79.3
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.78.0...v1.79.3)

---

updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add cherry-pick workflow for release branches (#533)

this workflow automates cherry-picking changes from merged pull requests to specified release branches based on comments... it handles permission checks, version parsing, and conflict resolution during the cherry-pick process.

Signed-off-by: Camryn Carter <camryn.carter@ranchergovernment.com>

* images.txt testdata file (#539)

* fix keep registry logic (#537)

* fixed keep registry logic
* trim library/
* updated test
* test updates

* option to sync images.txt files natively (#538)

* sync images.txt files
* test worklflow sync w image list
* images.txt

* chunk the haul (#519)

* chunk the haul
* validate numeric suffix on join
* enforce valid chunk size
* containerd warning
* updated test.go files

* bump github.com/go-jose/go-jose/v4 (#542)

bumps the go_modules group with 1 update in the / directory: [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose).


updates `github.com/go-jose/go-jose/v4` from 4.1.3 to 4.1.4

- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4)

---

updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.1.4
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* allow multiple prefix references (#532)

* allow multiple prefix references
* fixed some duplications

* add optional flag for excluding extra artifacts when pulling from a registry (#541)

* add optional flag for excluding extra artifacts when pulling from a registry

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* add optional flag to charts for excluding extra artifacts when pulling from a registry

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
Signed-off-by: Camryn Carter <camryn.carter@ranchergovernment.com>
Co-authored-by: devLeitner <87783219+devLeitner@users.noreply.github.com>
Co-authored-by: devleitner <devleitner@protonmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Camryn Carter <camryn.carter@ranchergovernment.com>
Co-authored-by: Adam Martin <adam.martin@ranchergovernment.com>
2026-04-08 12:09:23 -04:00

149 lines
2.9 KiB
Go

package getter
import (
"context"
"fmt"
"io"
"net/url"
v1 "github.com/google/go-containerregistry/pkg/v1"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"github.com/pkg/errors"
content2 "hauler.dev/go/hauler/pkg/artifacts"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content"
"hauler.dev/go/hauler/pkg/layer"
)
type Client struct {
Getters map[string]Getter
Options ClientOptions
}
// ClientOptions provides options for the client
type ClientOptions struct {
NameOverride string
}
var (
ErrGetterTypeUnknown = errors.New("no getter type found matching reference")
)
type Getter interface {
Open(context.Context, *url.URL) (io.ReadCloser, error)
Detect(*url.URL) bool
Name(*url.URL) string
Config(*url.URL) content2.Config
}
func NewClient(opts ClientOptions) *Client {
defaults := map[string]Getter{
"file": NewFile(),
"directory": NewDirectory(),
"http": NewHttp(),
}
c := &Client{
Getters: defaults,
Options: opts,
}
return c
}
func (c *Client) LayerFrom(ctx context.Context, source string) (v1.Layer, error) {
u, err := url.Parse(source)
if err != nil {
return nil, err
}
g, err := c.getterFrom(u)
if err != nil {
if errors.Is(err, ErrGetterTypeUnknown) {
return nil, err
}
return nil, fmt.Errorf("create getter: %w", err)
}
opener := func() (io.ReadCloser, error) {
return g.Open(ctx, u)
}
annotations := make(map[string]string)
annotations[ocispec.AnnotationTitle] = c.Name(source)
switch g.(type) {
case *directory:
annotations[content.AnnotationUnpack] = "true"
}
l, err := layer.FromOpener(opener,
layer.WithMediaType(consts.FileLayerMediaType),
layer.WithAnnotations(annotations))
if err != nil {
return nil, err
}
return l, nil
}
func (c *Client) ContentFrom(ctx context.Context, source string) (io.ReadCloser, error) {
u, err := url.Parse(source)
if err != nil {
return nil, fmt.Errorf("parse source %s: %w", source, err)
}
g, err := c.getterFrom(u)
if err != nil {
if errors.Is(err, ErrGetterTypeUnknown) {
return nil, err
}
return nil, fmt.Errorf("create getter: %w", err)
}
return g.Open(ctx, u)
}
func (c *Client) getterFrom(srcUrl *url.URL) (Getter, error) {
for _, g := range c.Getters {
if g.Detect(srcUrl) {
return g, nil
}
}
return nil, errors.Wrapf(ErrGetterTypeUnknown, "source %s", srcUrl.String())
}
func (c *Client) Name(source string) string {
if c.Options.NameOverride != "" {
return c.Options.NameOverride
}
u, err := url.Parse(source)
if err != nil {
return source
}
for _, g := range c.Getters {
if g.Detect(u) {
return g.Name(u)
}
}
return source
}
func (c *Client) Config(source string) content2.Config {
u, err := url.Parse(source)
if err != nil {
return nil
}
for _, g := range c.Getters {
if g.Detect(u) {
return g.Config(u)
}
}
return nil
}
type config struct {
Reference string `json:"reference"`
Annotations map[string]string `json:"annotations,omitempty"`
}