Files
hauler/internal/mapper/mappers.go
Zack Brady c0294c733b update release/2.0 from main (#546)
* fix: handling of file referenced dependencies without repository field (#514)

co-authored-by: devleitner <devleitner@protonmail.com>

* bump go.opentelemetry.io/otel/sdk (#520)

bumps the go_modules group with 1 update in the / directory: [go.opentelemetry.io/otel/sdk](https://github.com/open-telemetry/opentelemetry-go).

updates `go.opentelemetry.io/otel/sdk` from 1.39.0 to 1.40.0
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.39.0...v1.40.0)

---

updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/sdk
  dependency-version: 1.40.0
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* dev.md file (#521)

* smaller changes and updates for v1.4.2 release (#524)

* smaller changes and updates for v1.4.2 release
* removed unused env variable

* over-"haul": replace oras v1 and cosign fork with native containerd-based implementation (#515)

* remove oras from hauler

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* remove cosign fork and use upstream cosign for verification

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* added support for oci referrers

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* updated README.md projects list

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* updates for copilot PR review

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for unsafe type assertions

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for http getter and dead code

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* fixes for more clarity and better error handling

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for resource leaks and unchecked errors

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for rewrite logic for docker.io images due to cosign removal

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for sigs and referrers

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix for index.json missing mediatype

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bug fix to make sure manifest.json doesnt include anything other than actual container images

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bump github.com/docker/cli in the go_modules group across 1 directory (#526)

bumps the go_modules group with 1 update in the / directory: [github.com/docker/cli](https://github.com/docker/cli).


updates `github.com/docker/cli` from 29.0.3+incompatible to 29.2.0+incompatible
- [Commits](https://github.com/docker/cli/compare/v29.0.3...v29.2.0)

---

updated-dependencies:
- dependency-name: github.com/docker/cli
  dependency-version: 29.2.0+incompatible
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* removed deprecated code (#528)

* removed deprecated code
* removed all supported for v1alpha1

* fix extract for oci files (#529)

* fix extract for oci files

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* have extract guard against path traversal

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* improved test coverage (#530)

* improved test coverage

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* adjusted mapper_test for oddball oci files

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* adjust extract to handle an image index appropriately (#531)

* adjust extract to handle images and image indices appropriately

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* updates for review feedback

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* fix dockerhub default host bug (#534)

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* adjust hauler's kind annotation to not reflect cosign (#535)

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* bump google.golang.org/grpc in the go_modules group across 1 directory (#536)

bumps the go_modules group with 1 update in the / directory: [google.golang.org/grpc](https://github.com/grpc/grpc-go).

updates `google.golang.org/grpc` from 1.78.0 to 1.79.3
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.78.0...v1.79.3)

---

updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-version: 1.79.3
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* add cherry-pick workflow for release branches (#533)

this workflow automates cherry-picking changes from merged pull requests to specified release branches based on comments... it handles permission checks, version parsing, and conflict resolution during the cherry-pick process.

Signed-off-by: Camryn Carter <camryn.carter@ranchergovernment.com>

* images.txt testdata file (#539)

* fix keep registry logic (#537)

* fixed keep registry logic
* trim library/
* updated test
* test updates

* option to sync images.txt files natively (#538)

* sync images.txt files
* test worklflow sync w image list
* images.txt

* chunk the haul (#519)

* chunk the haul
* validate numeric suffix on join
* enforce valid chunk size
* containerd warning
* updated test.go files

* bump github.com/go-jose/go-jose/v4 (#542)

bumps the go_modules group with 1 update in the / directory: [github.com/go-jose/go-jose/v4](https://github.com/go-jose/go-jose).


updates `github.com/go-jose/go-jose/v4` from 4.1.3 to 4.1.4

- [Release notes](https://github.com/go-jose/go-jose/releases)
- [Commits](https://github.com/go-jose/go-jose/compare/v4.1.3...v4.1.4)

---

updated-dependencies:
- dependency-name: github.com/go-jose/go-jose/v4
  dependency-version: 4.1.4
  dependency-type: indirect
  dependency-group: go_modules

...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* allow multiple prefix references (#532)

* allow multiple prefix references
* fixed some duplications

* add optional flag for excluding extra artifacts when pulling from a registry (#541)

* add optional flag for excluding extra artifacts when pulling from a registry

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

* add optional flag to charts for excluding extra artifacts when pulling from a registry

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Adam Martin <adam.martin@ranchergovernment.com>
Signed-off-by: Camryn Carter <camryn.carter@ranchergovernment.com>
Co-authored-by: devLeitner <87783219+devLeitner@users.noreply.github.com>
Co-authored-by: devleitner <devleitner@protonmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Camryn Carter <camryn.carter@ranchergovernment.com>
Co-authored-by: Adam Martin <adam.martin@ranchergovernment.com>
2026-04-08 12:09:23 -04:00

145 lines
4.8 KiB
Go

package mapper
import (
"fmt"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
"hauler.dev/go/hauler/pkg/consts"
"hauler.dev/go/hauler/pkg/content"
)
type Fn func(desc ocispec.Descriptor) (string, error)
// FromManifest will return the appropriate content store given a reference and source type adequate for storing the results on disk
func FromManifest(manifest ocispec.Manifest, root string) (content.Target, error) {
// First, switch on config mediatype to identify known types.
switch manifest.Config.MediaType {
case consts.ChartLayerMediaType, consts.ChartConfigMediaType:
return NewMapperFileStore(root, Chart())
case consts.FileLocalConfigMediaType, consts.FileDirectoryConfigMediaType, consts.FileHttpConfigMediaType:
return NewMapperFileStore(root, Files())
case consts.DockerConfigJSON, ocispec.MediaTypeImageConfig:
// Standard OCI/Docker image config. OCI artifacts that distribute files
// (e.g. rke2-binary) reuse this config type but set AnnotationTitle on their
// layers. When title annotations are present prefer Files() so the title is
// used as the output filename; otherwise treat as a container image.
for _, layer := range manifest.Layers {
if _, ok := layer.Annotations[ocispec.AnnotationTitle]; ok {
return NewMapperFileStore(root, Files())
}
}
return NewMapperFileStore(root, Images())
}
// Unknown config type: title annotation indicates a file artifact; otherwise use
// a catch-all mapper that writes blobs by digest.
for _, layer := range manifest.Layers {
if _, ok := layer.Annotations[ocispec.AnnotationTitle]; ok {
return NewMapperFileStore(root, Files())
}
}
return NewMapperFileStore(root, Default())
}
func Images() map[string]Fn {
m := make(map[string]Fn)
manifestMapperFn := Fn(func(desc ocispec.Descriptor) (string, error) {
return consts.ImageManifestFile, nil
})
for _, l := range []string{consts.DockerManifestSchema2, consts.DockerManifestListSchema2, consts.OCIManifestSchema1} {
m[l] = manifestMapperFn
}
layerMapperFn := Fn(func(desc ocispec.Descriptor) (string, error) {
return fmt.Sprintf("%s.tar.gz", desc.Digest.String()), nil
})
for _, l := range []string{consts.OCILayer, consts.DockerLayer} {
m[l] = layerMapperFn
}
configMapperFn := Fn(func(desc ocispec.Descriptor) (string, error) {
return consts.ImageConfigFile, nil
})
for _, l := range []string{consts.DockerConfigJSON} {
m[l] = configMapperFn
}
return m
}
func Chart() map[string]Fn {
m := make(map[string]Fn)
chartMapperFn := Fn(func(desc ocispec.Descriptor) (string, error) {
f := "chart.tar.gz"
if _, ok := desc.Annotations[ocispec.AnnotationTitle]; ok {
f = desc.Annotations[ocispec.AnnotationTitle]
}
return f, nil
})
provMapperFn := Fn(func(desc ocispec.Descriptor) (string, error) {
return "prov.json", nil
})
m[consts.ChartLayerMediaType] = chartMapperFn
m[consts.ProvLayerMediaType] = provMapperFn
return m
}
// DefaultCatchAll is the sentinel key used in a mapper map to match any media type
// not explicitly registered. Push checks for this key as a fallback.
const DefaultCatchAll = ""
// Default returns a catch-all mapper that extracts any layer blob using its title
// annotation as the filename, falling back to a digest-based name. Used when the
// manifest config media type is not a known hauler type.
func Default() map[string]Fn {
m := make(map[string]Fn)
m[DefaultCatchAll] = Fn(func(desc ocispec.Descriptor) (string, error) {
if title, ok := desc.Annotations[ocispec.AnnotationTitle]; ok {
return title, nil
}
return fmt.Sprintf("%s.bin", desc.Digest.String()), nil
})
return m
}
func Files() map[string]Fn {
m := make(map[string]Fn)
fileMapperFn := Fn(func(desc ocispec.Descriptor) (string, error) {
// Use the title annotation to determine the filename
if title, ok := desc.Annotations[ocispec.AnnotationTitle]; ok {
return title, nil
}
// Fallback to digest-based filename if no title
return fmt.Sprintf("%s.file", desc.Digest.String()), nil
})
// Match the media type that's actually used in the manifest
// (set by getter.LayerFrom in pkg/getter/getter.go)
m[consts.FileLayerMediaType] = fileMapperFn
m[consts.OCILayer] = fileMapperFn // Also handle standard OCI layers that have title annotation
m["application/vnd.oci.image.layer.v1.tar"] = fileMapperFn // And the tar variant
// Catch-all for OCI artifacts that use custom layer media types (e.g. rke2-binary).
// Write the blob if it carries an AnnotationTitle; silently discard everything else
// (config blobs, metadata) by returning an empty filename.
m[DefaultCatchAll] = Fn(func(desc ocispec.Descriptor) (string, error) {
if title, ok := desc.Annotations[ocispec.AnnotationTitle]; ok {
return title, nil
}
return "", nil // No title → discard (config blob or unrecognised metadata)
})
return m
}