8 Commits

Include renames

Author	SHA1	Message	Date
3 people	5a44b5f10c	Feat: auto remediate cue issues (#7199) ... * feat(cue/upgrade): auto-remediate legacy CUE syntax at render time Transparently rewrite CUE templates that use deprecated list arithmetic (+, *) and conflicting field names (error) so that older definitions continue to work with CUE ≥ v0.14 (KubeVela ≥ 1.11). - CUEUpgradeFunc registry with ID, CUE/KubeVela version guards, precheck, and upgrade function fields - upgradeListConcatenation: rewrites list1+list2 → list.Concat([list1,list2]) and list*n → list.Repeat(list, n); adds "list" import as needed - collectAddChain + extractListConcatArgs: flatten left-associative + chains and existing list.Concat([...]) leaves into a single flat call, so both fresh chains (a+b+c+d) and partially-upgraded chains produce one list.Concat([a,b,c,d]) with no nesting across repeated passes - upgradeErrorFieldLabel: rewrites unquoted `error` field labels to "error" to avoid conflict with the CUE 0.14 built-in; precheck uses a tighter \berror\s*: regex to avoid false positives on identifiers like errorMessage - EnsureCueVersionCompatibility: single entry point used at render time; LRU cache with TTL eviction, Prometheus metrics, feature flag - ParseVersion: regex anchored to reject garbage suffixes (e.g. "1.11foo") while accepting pre-release+build metadata (e.g. "v1.13.0-alpha.1+dev") - template.go: call EnsureCueVersionCompatibility for every template area (main, health, custom status, status detail) with correct DefinitionKind derived from which definition pointer is non-nil - validate.go: upgrade policy templates before compiling in validateNoRequiredParameters - `vela def upgrade FILE [-o OUTPUT]`: upgrades a single .cue file - `vela def upgrade FILE --validate [--quiet]`: exit 1 if upgrade needed - `vela def compat definitions` / `vela def compat applications`: scan cluster definitions/apps for compat issues; output as table or YAML - Cyclomatic complexity kept below threshold by extracting scanDefinitions, scanDefRevisions, buildDefCompatReport, scanApplications, scanAppRevision as standalone functions with options structs - revisionNum() helper for numeric vN comparison (avoids lexicographic bugs) - mergeImports() dedup helper shared by ToCUEString and formatCUEString - ANSI escape sequences replaced with fatih/color for portability - goconst: "yaml" → outputFormatYAML named constant throughout - Component, trait, and policy definition validating handlers: removed spurious obj.Name argument from fmt.Sprintf in warning messages - FromCUEString: only prepend importString to the stored template when imports are non-empty; empty importString ("\n") was causing a leading newline that made yaml.v3 use |2 block scalar on every generated YAML - gen_sdk testdata: removed unused imports (vela/op, encoding/base64) from one_of.cue that were exposed by our importString+templateString change - e2e test: fix flaky trait-order assertion using ContainElements instead of index-based equality Upgraded all built-in .cue files that used deprecated list arithmetic: - vela-templates/definitions/internal/component/cron-task.cue - vela-templates/definitions/internal/trait/command.cue - vela-templates/definitions/internal/trait/container-ports.cue - vela-templates/definitions/internal/trait/env.cue - vela-templates/definitions/internal/trait/init-container.cue Removed unused stdlib imports that caused `def gen-api` to fail: - vela-templates/definitions/internal/workflowstep/apply-deployment.cue - vela-templates/definitions/internal/workflowstep/apply-terraform-provider.cue - vela-templates/definitions/internal/workflowstep/build-push-image.cue Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Brian Kane <briankane1@gmail.com> * fix(cue/upgrade): address PR review comments - sync.atomic.Pointer for compatCache to fix data race on reinit - SummaryVec → HistogramVec for both duration metrics (aggregatable across HA replicas); buckets tuned to sub-millisecond upgrade path and millisecond render path respectively - errorFieldLabelRe: extend to match optional (?) and required (!) field constraint markers before the colon - cue-compatibility-cache-size: clamp negative values to 0 (disabled) with warning log; document 0=disabled in flag help; cache put is no-op when capacity <= 0 - webhook: replace RequiresUpgrade+EnsureCueVersionCompatibility double parse with single EnsureCueVersionCompatibility call; use string comparison to detect upgrade and emit warning - def compat: log warning when ApplicationRevision fetch fails instead of silently skipping (partial results are preserved) - e2e: only delete definitions in DeferCleanup if this test created them (avoid deleting pre-existing shared resources) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Brian Kane <briankane1@gmail.com> * fix(cue/upgrade): address further PR review comments - EnsureCueVersionCompatibility: return (string, bool) where bool indicates semantic upgrades were applied (len(applied)>0), not string inequality — prevents false-positive warnings from formatting-only normalisation; update all call sites - webhook handlers (component, trait, policy): switch from RequiresUpgrade+EnsureCueVersionCompatibility double-call to single EnsureCueVersionCompatibility call using wasUpgraded bool; remove now-unused strings imports - cache: skip eviction goroutine when capacity==0 (disabled); set compatCacheCancel=nil on disabled path to avoid stale cancel on next InitCompatibilityCache call - e2e: replace boolean ownership tracking with createAndTrack helper that checks pre-existence via Get before Create, eliminating both the ambiguous-create leak and the boilerplate booleans Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Brian Kane <briankane1@gmail.com> * fix: address reviewer comments — cache determinism, e2e ownership race - cache: store normalised string in compatEntry.upgraded even when no semantic fixes were applied, so cache-hit and cache-miss paths return identical output (fixes non-deterministic behaviour flagged in review) - upgrade: return entry.upgraded on the requiresUpgrade=false cache-hit path instead of the raw input cueStr - e2e: replace GET-then-CREATE ownership inference with atomic CREATE- first pattern; err==nil means we created it (register DeferCleanup), IsAlreadyExists means it pre-existed (skip cleanup), eliminating the GET/CREATE race window that could misattribute ownership Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: Brian Kane <briankane1@gmail.com> --------- Signed-off-by: Brian Kane <briankane1@gmail.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-06-26 05:57:23 -07:00
3 people	bcf6af9747	Feat: helm chart auth (#7148) ... * feat: kv native helm auth implementation Signed-off-by: Ayush Kumar <aykumar@guidewire.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * feat: add e2e test cases for helm auth Signed-off-by: Ayush Kumar <aykumar@guidewire.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * test: remove the env auth gate and improve test cases Signed-off-by: Ayush Kumar <aykumar@guidewire.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * lint: drop unused []byte return from non-docker secret dispatchers unparam flagged dispatchBasicAuthSecret, dispatchTLSSecret, and dispatchOpaqueSecret because the second return value (raw config bytes) was always nil. Only dispatchDockerConfigJSONSecret actually needs that slot for the OCI temp credfile. Drop it from the other three and update resolveAuthOptions plus the unit tests to match. Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * feat: pr review changes Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * style: gofmt/goimports auth.go doc comments Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * chore: remove e2e-auth-images-load-local target The dedicated target is gone; e2e-test-local now reads the same .vscode/k3d-preload.txt image list that the VS Code setup task uses so both flows pull from one source. Empty or missing file is a no-op. Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * chore: inline auth-test image list in e2e-test-local `.vscode/k3d-preload.txt` is gitignored so it would not exist on a fresh clone. Inline the three auth-test registry images directly in the Make target so `make e2e-test-local` is self-contained. Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * feat: address PR review comments - Webhook ValidateCuexTemplate now wraps the context in helm.WithDryRun so a ComponentDefinition with concrete helm.#Render arguments cannot trigger a real chart fetch or install during admission validation. - helmchart audit ConfigMap keeps `helm.oam.dev/chart` as a label when the source string is a valid Kubernetes label value (alphanumeric + `.-_`, 1-63 chars); URLs containing `://` or `/` only live in the annotation. Preserves existing label selectors for repo-style sources without breaking long OCI/HTTPS URLs. - helm_test.go capture-and-restore singleton.KubeClient in fetchURLChart and fetchRepoChart auth Describes so fake clients do not leak into later tests in the package. - utils_test.go swap the defer order so singleton.ReloadClients runs before WorkloadCompiler.Reload, otherwise the compiler reload would pick up the fake dynamic client and leak fake state. - e2e.mk pre-load loop separates docker pull and k3d image import with `;` rather than `&&`. `set -e` does not abort a for-loop body when a command inside an `&&` chain fails, so the old form could continue after a failed pull. Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * style: gofmt AfterAll indentation in helmchart_test.go Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * feat: enhance Docker Hub credential handling in auth configuration Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * feat: enhance Docker Hub credential handling in auth configuration Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * fix(helm-auth): normalize Docker Hub alias for dockerconfigjson Secrets Cubic flagged that the Docker Hub alias fix in writeOCIRegistryConfigFile only covered synthesized basic-auth credentials. For verbatim kubernetes.io/dockerconfigjson Secrets, a user-supplied config keyed under "registry-1.docker.io" (the OCI pull host) would not be found by ORAS/Helm, which normalizes to "https://index.docker.io/v1/". normalizeDockerHubAliases parses the verbatim JSON, and when any of the three Docker Hub host aliases ("registry-1.docker.io", "index.docker.io", "docker.io") is keyed but the canonical v1 key is absent, copies the entry under the canonical key. No-op when: - the canonical key is already present - no Docker Hub host is involved - the JSON is malformed or missing the auths field Includes 7 new unit specs for the helper covering each alias, the no-op cases, and malformed input. Also brings in collateral local-fix changes: - pkg/webhook/utils/utils_test.go: register the cue.oam.dev/v1alpha1 Package GVK in the fake DynamicClient scheme so TestValidateCuexTemplate/withCuexPackageImports can resolve the test/ext package after the WorkloadCompiler switch. - gofmt field-alignment normalization on auth_test.go and auth_registry_helpers_test.go (drift caught by check-diff CI). Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * fix: namespace-default rendered resources, bind chart cache to credentials, surface HTTP 401s clearly, inject auth-test CA in e2e Four fixes from stress testing the helmchart component: 1. Rendered resources without metadata.namespace landed in vela-system. Both code paths now default to releaseNamespace for namespaced kinds: the velaLabelPostRenderer (before helm SDK apply) and parseManifestResources (before KubeVela's tracker re-applies). isClusterScopedKind covers CRDs, ClusterRoles, Namespaces, etc. 2. Cached chart bytes survived credential rotation. computeAuthCacheTag hashes Secret.Type + sorted Data into a 16-char tag suffixed onto the cache key when auth.secretRef is declared. Any Secret edit (or different Secret reference) invalidates the cache and forces a fresh registry call that exercises the new credentials at the wire. Public charts (no auth) are unaffected. 3. HTTP 401/403 on chart fetch surfaced as YAML/JSON parse errors. HTTPGetWithOption now rejects non-2xx responses with 'HTTP <status>: <body>' instead of returning the raw body for downstream parsers to choke on. 4. E2E HTTPS tests against chartmuseum's self-signed cert failed at the admission webhook. injectAuthTestCA patches vela-core in BeforeSuite with an init container that combines /etc/ssl/certs/ca-certificates.crt with testdata/auth/certs/ca.crt into a shared volume and points SSL_CERT_FILE at it. Also normalize Docker Hub host aliases on verbatim kubernetes.io/dockerconfigjson Secrets (cubic comment), and revert TestValidateCuexTemplate's fake DynamicClient to the empty-scheme form that lets it wrap every GVK as Unstructured (CI unit test fix). Finding 3 (options.wait default) is deferred: schema-level defaults don't materialize for optional fields carried into _options as a structural copy. Tracked separately. Issue 2 from the stress matrix (cache + credential rotation) is also filed upstream as kubevela/kubevela#7150 for a longer-term credential-bound cache invalidation design. Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * fix(helm-auth): RESTMapper scope detection + auth tag binds source URL + drop flaky webhook test case Address cubic P1 review comments and the unit-test CI failure. isClusterScopedKind only matched well-known built-in kinds; isClusterScopedGVK now asks the RESTMapper first so third-party cluster-scoped CRDs are recognised, with the static allowlist as a fallback. computeAuthCacheTag now folds params.Source and params.RepoURL into the hash so a multi-host dockerconfigjson Secret cannot reuse cached bytes across different registries. TestValidateCuexTemplate/withCuexPackageImports relied on cuex.DefaultCompiler.Reload picking up a fake-client-served Package CRD; since ValidateCuexTemplate now uses velacuex.WorkloadCompiler the fake-client setup does not surface the test/ext package, so the case is dropped (covered transitively by the helm provider unit tests and the e2e suite under helm.WithDryRun). Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * style: gofmt import order in helm.go (lint + check-diff) Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * fix(e2e): correct vela-core selector, alpine bundler, harden rollout wait The label selector app.kubernetes.io/name=vela-core also matches the cluster-gateway Deployment, so the init container was being appended to the wrong workload and the controller pod never picked up the test CA. Switching to the unique controller.oam.dev/name label fixes the routing, and the diagnostic message at the empty-list branch is updated to point operators at the new label. addVolume and addInitContainer now update in place when an entry with the same name already exists. Previously they silently skipped, which meant the first patch on a cluster stuck for the lifetime of that cluster and a corrected image or args could not be picked up by a re-run. busybox:1.36 ships without /etc/ssl/certs/ca-certificates.crt, so the bundler was silently producing an empty combined.crt. alpine:3.18 ships the public CA roots, so concatenating the auth-test CA onto the system bundle produces a usable trust store that the main container reads via SSL_CERT_FILE. waitForDeploymentsAvailable used to return as soon as DeploymentAvailable flipped to True and the new RS reached the desired replica count, which left a window where old-RS pods still satisfied Available while the controller container was still running with the previous image. The check now also requires Status.Replicas == Status.UpdatedReplicas, AvailableReplicas >= specReplicas and UnavailableReplicas == 0, so the helper only returns once the rollout is fully complete. Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> * test(e2e): gate auth-test setup behind KUBEVELA_E2E_AUTH so other suites stay green webhook-upgrade-check runs the same test/e2e-test suite as the full e2e job but with ginkgo --focus-file requiredparam_validation_test.go. The unconditional setupAuthRegistries in BeforeSuite was bringing up the ChartMuseum / zot / nginx-bearer stack and patching vela-core for a focus that did not need any of it, and the patched controller never finished rolling on the upgrade-check cluster (the helm upgrade in that job is expected to fail, so vela-core is in a stretched state when our BeforeSuite tries to patch it). Re-introduce KUBEVELA_E2E_AUTH=1 as the explicit opt-in. BeforeSuite and AfterSuite only touch the auth registries when the env var is set, the Helmchart Auth Describe block skips otherwise, and the make e2e-test and e2e-test-local targets export the env so the full suite still runs the auth specs. CI workflows that run a focused subset (like webhook-upgrade-check) inherit the default off behavior and stop paying the registry-setup cost. Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com> --------- Signed-off-by: Ayush Kumar <aykumar@guidewire.com> Signed-off-by: Vishal Kumar <vishal210893@gmail.com> Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com> Co-authored-by: Vishal Kumar <vishal210893@gmail.com>	2026-05-22 09:09:55 -07:00
Brian KaneandGitHub	c79f03fe92	Fix: corrects validating webhook behaviour with cuex compilers (#6799) ... Signed-off-by: Brian Kane <briankane1@gmail.com>	2025-07-17 15:18:36 -07:00
6 people	bc15e5b359	Feat: Semantic versioning support for Definitions (#6648) ... * feature: Add Semantic versioning to KubeVela Definitions Fixes https://github.com/kubevela/kubevela/issues/6435 Fixes https://github.com/kubevela/kubevela/issues/6534 Changes: - Adds an optional "Version" field for all Definition Specs. - Adds the following new validations to Webhooks for Definitions: - Validate the "Version" field follows Semantic versioning. - Dis-allow conflicting versioning fields ( Name annotation, Spec.Version) - Adds the following new validations to Webhooks for Application: - Dis-allow the use of both the "publishVersion" & "autoUpdate" annotations. - Enahnce "multiStageComponentApply" feature to support auto updates. Boy Scout Changes: - Fixes Plugin e2e tests broken by the fix for 6534. - Fixes the dryRun and livediff commands to respect the "-n" namespace flag. - Fixes the Application ValidationWebhook to respect the "-n" namespace flag. Co-authored-by: Rahul Kumar <35751394+bugbounce@users.noreply.github.com> Co-authored-by: Chaitanya Reddy <chaitanyareddy0702@gmail.com> Co-authored-by: Vibhor Chinda <vibhorchinda@gmail.com> Co-authored-by: Shivin Gopalani <gopalanishivin@gmail.com> Signed-off-by: kanchan-dhamane <74534570+kanchan-dhamane@users.noreply.github.com> * feature: Add KEP to define the proposal Signed-off-by: kanchan-dhamane <74534570+kanchan-dhamane@users.noreply.github.com> * fix: Rebase and fix merge conflicts Signed-off-by: kanchan-dhamane <74534570+kanchan-dhamane@users.noreply.github.com> * Fix: Adds unit test cases Signed-off-by: kanchan-dhamane <74534570+kanchan-dhamane@users.noreply.github.com> --------- Signed-off-by: kanchan-dhamane <74534570+kanchan-dhamane@users.noreply.github.com> Co-authored-by: bugbounce <35751394+bugbounce@users.noreply.github.com>	2025-02-03 11:09:28 +08:00
Tianxin DongandGitHub	82dad1ebbb	Chore: update go version to 1.22 (#6560) ... * chore: update go version to 1.22 Signed-off-by: FogDong <fog@bentoml.com> * fix: fix lint Signed-off-by: FogDong <fog@bentoml.com> * fix: fix unit test Signed-off-by: FogDong <fog@bentoml.com> * fix: update static check tool Signed-off-by: FogDong <fog@bentoml.com> * fix: add debug Signed-off-by: FogDong <fog@bentoml.com> * fix: use ghcr to fix dockerhub rate limit Signed-off-by: FogDong <fog@bentoml.com> * fix: use ghcr for addons Signed-off-by: FogDong <fog@bentoml.com> * fix: add more timeout for e2e multicluster test Signed-off-by: FogDong <fog@bentoml.com> * fix: use ghcr Signed-off-by: FogDong <fog@bentoml.com> * fix: fix e2e addon image Signed-off-by: FogDong <fog@bentoml.com> * fix: test addon terraform version Signed-off-by: FogDong <fog@bentoml.com> * fix: fix admission image Signed-off-by: FogDong <fog@bentoml.com> * fix: fix terraform version Signed-off-by: FogDong <fog@bentoml.com> --------- Signed-off-by: FogDong <fog@bentoml.com>	2024-07-03 16:10:56 +08:00
caiqi1111andGitHub	30fb7a56a0	Fix: validate cue template in webhook handler and add policyvaladatin… (#6355)	2023-09-22 08:42:12 -05:00
SomefiveandGitHub	dd899c2b39	Chore: remove outdated apis(v1alppha2 ApplicationConfiguration Component, and HealthScope, Rollout) (#6041) ... * remove outdated api Signed-off-by: Somefive <yd219913@alibaba-inc.com> * fix rt test: no component rt Signed-off-by: Somefive <yd219913@alibaba-inc.com> * recover context.revision to component hash Signed-off-by: Somefive <yd219913@alibaba-inc.com> --------- Signed-off-by: Somefive <yd219913@alibaba-inc.com>	2023-06-01 09:32:49 +08:00
yangsoonandGitHub	abbec2c36b	definitions support specify the revision name (#2044) ... * support named defRev * add test	2021-08-10 20:03:45 +08:00