mirror of
https://github.com/kubevela/kubevela.git
synced 2026-05-20 00:07:27 +00:00
* Docs(KEP): Go SDK for X-Definition Authoring (defkit) Introduces KEP proposal for defkit, a Go SDK that enables platform engineers to author X-Definitions using native Go code instead of CUE. Key proposed features: - Fluent builder API for Component, Trait, Policy, and WorkflowStep definitions - Transparent Go-to-CUE compilation - IDE support with autocomplete and type checking - Schema-agnostic resource construction - Collection operations (map, filter, dedupe) - Composable health and status expressions - Addon integration with godef/ folder support - Module dependencies for definition sharing via go get Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(KEP): Examples and minor api changes given in the document Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(KEP): align defkit examples - Fix golang version in CI - Fix variable declaration in example for testing - Add Is() comparison method to status check Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Docs(KEP): add security considerations section - Add goal #7 for secure code execution model - Add Security Considerations section covering: - Code execution model (compile-time only, not runtime) - Security benefits over CUE (static analysis, dependency scanning) - Threat model with mitigations Addresses PR feedback about code execution safety. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Docs(KEP): add module versioning and definition placement sections - Add Module Versioning section explaining git-based version derivation - Add Definition Placement section covering: - Motivation for placement constraints in multi-cluster environments - Fluent API for placement (RunOn, NotRunOn, label conditions) - Logical combinators (And, Or, Not) - Module-level placement defaults - Placement evaluation logic - CLI experience for managing cluster labels - Add Module Hooks section for lifecycle callbacks - Minor fixes and clarifications throughout Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Docs(KEP): add module hooks and update addon integration sections - Add Module Hooks section covering: - Use cases (CRD installation, setup scripts, post-install samples) - Hook configuration in module.yaml (pre-apply, post-apply) - Hook types (path for manifests, script for shell scripts) - waitFor field with condition names and CUE expressions - CLI usage (--skip-hooks, --dry-run) - Update Addon Integration section with implementation details: - godef/ folder structure with module.yaml - CLI flags (--godef, --components, --traits, --policies, --workflowsteps) - Conflict detection and --override-definitions flag - Development workflow Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Docs(KEP): address PR review comments and clarify placement labels - Fix misleading "Sandboxed Compilation" claim (cubic-ai feedback) - renamed to "Isolated Compilation" and clarified that security relies on trust model, not technical sandboxing - Fix inconsistent apiVersion in module hooks example (defkit.oam.dev/v1 → core.oam.dev/v1beta1) - Clarify that placement uses vela-cluster-identity ConfigMap directly, not the vela cluster labels command (which is planned for future) - Add --stats flag to apply-module CLI documentation Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Docs(KEP): fix API documentation Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add core fluent API types for Go-based definitions Introduce the defkit package providing a fluent Go API for defining KubeVela X-Definitions (components, traits, policies, workflow steps). Core types added: - types.go: Value, Condition, Param interfaces - base.go: Base definition types and interfaces - param.go: Parameter builders (String, Int, Bool, Array, Map, Struct, Enum) - expr.go: Expression builders for conditions and comparisons - resource.go: Resource operations (Set, SetIf, Spread) - context.go: KubeVela context references (appName, namespace, etc.) - test_context.go: Test utilities for definition validation This enables writing type-safe Go definitions that compile to CUE. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add collection operations and helper builders Add fluent API for array/collection transformations: - CollectionOp with Filter, Map, Pick, Wrap, Dedupe operations - From() and Each() entry points for collection pipelines - FieldRef, FieldEquals, FieldMap for field-level operations - MultiSource for complex multi-array comprehensions - Add helper builders for template variables - Add value transformation utilities Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add CUE code generator Implement CUEGenerator that transforms Go definitions into CUE code Added helper methods and writers for conversion Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add status and health policy builders Add fluent builders for customStatus and healthPolicy CUE generation Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add definition type builders Add fluent builders for all four KubeVela X-Definition types: - ComponentDefinition - TraitDefinition - PolicyDefinition - WorkflowStepDefinition Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(goloader): add Go module loader for definitions - Definition interface and registry for runtime discovery - Discover and parse Go-based definition files - Compile Go definitions to CUE at runtime - Module environment for batch processing - Parallel generation for better performance Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(cli): add vela def commands for Go-based definitions - init-module: scaffold a new Go definition module - apply-module: compile and apply definitions to cluster - list-module: show definitions in a module - validate-module: validate definitions without applying - Also support the cue commands for xdefintions for go code Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add testing utilities and matchers - CUE comparison matchers for Ginkgo/Gomega tests - Test helpers for definition validation Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add patch container helpers for container mod operations Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(cli): update the go module to 1.23.8 for defkit init-module command Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Refactor: Add grouped help output for vela def command Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add definition placement for cluster-aware deployments Enable definitions to specify which clusters they should run on based on cluster identity labels stored in a well-known ConfigMap. Also derives module version from git tags and improves init-module to create directories from --name flag. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add RunOn/NotRunOn fluent API for placement constraints Add placement methods to all definition builders allowing definitions to specify cluster eligibility using the placement package's fluent API. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Docs(defkit): add commented placement example to module.yaml template Show users the placement syntax in generated module.yaml without setting actual values. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add module-level placement support Add placement constraints at the module level in module.yaml that apply to all definitions unless overridden at definition level. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add CLI placement enforcement in apply-module Add placement constraint checking to `vela def apply-module` command. Definitions are skipped if cluster labels don't match module placement. - Add --ignore-placement flag to bypass placement checks - Display placement status during apply with clear skip reasons - Track placement-skipped count in summary output Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(defkit): show all flags in subcommand help output Fix custom help function to properly display flags for def subcommands like init-module and apply-module instead of only showing parent flags. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(defkit): apply name prefix to definitions in apply-module The --prefix flag was not being applied to definition names. The prefix was set in module loader metadata but not used when creating Kubernetes objects from parsed CUE. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Chore(defkit): align module command help with standard vela pattern Remove argument placeholders from command Use field to align with other vela commands (addon, cluster, workflow). Arguments are shown in examples and individual --help output instead of the listing. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(goloader): use json.Unmarshal for go mod download output The downloadGoModule function parses JSON output from 'go mod download -json' but was incorrectly using yaml.Unmarshal with json struct tags. The yaml.v3 library ignores json tags, resulting in empty field values. This would cause remote Go module loading (e.g., github.com/foo/bar@v1.0.0) to fail with "go mod download did not return a directory" because result.Dir would be empty. Fix: Use json.Unmarshal instead since the data is JSON from the Go toolchain. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(goloader): use semver for MinVelaVersion comparison String comparison of version numbers is incorrect for cases like "v1.10.0" > "v1.9.0" which returns false due to lexicographic ordering. Use the Masterminds/semver library (already a dependency) for proper semantic version comparison in ValidateModule(). Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(placement): validate operator in module placement conditions Add validation to catch invalid placement operators at module load time instead of silently failing at runtime evaluation. - Add Operator.IsValid() method to check for valid operators - Add ValidOperators() helper function - Add validatePlacementConditions() in ValidateModule() - Provides clear error message with valid operator list Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(cli): validate conflict strategy in apply-module Invalid --conflict values like "invalid" were silently accepted and would fall through the switch statement, behaving like "overwrite". Add ConflictStrategy.IsValid() method and validation at flag parsing to provide clear error message for invalid values. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(placement): support definition-level placement constraints Previously only module-level placement was enforced. Now individual definitions can specify their own placement constraints that override module defaults. Changes: - Add Placement field to DefinitionInfo and DefinitionPlacement types - Add GetPlacement/HasPlacement to Definition interface - Update registry ToJSON to include placement in output - Update goloader to capture definition placement from registry - Update CLI apply-module to use GetEffectivePlacement() for combining module-level and definition-level placement - Add comprehensive tests for definition placement Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Chore(defkit): remove dead PatchTemplate code PatchTemplate, PatchOp, SetPatchOp, and SetIfPatchOp were defined but never used anywhere in the codebase. The PatchResource type already provides the same functionality and is the one actually being used through Template.Patch(). Removed: - PatchTemplate struct and its methods (ToCue, SetIf, Set) - PatchOp interface - SetPatchOp struct and its ToCue method - SetIfPatchOp struct and its ToCue method - NewPatchTemplate constructor This cleanup reduces maintenance burden without affecting any functionality. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(cli): pass actual VelaVersion to validate-module command The help text for `vela def validate-module` promised to check minVelaVersion requirements but ValidateModule() was called with an empty string, causing the check to be silently skipped. Now passes velaversion.VelaVersion so modules specifying a minimum KubeVela version will be properly validated against the current CLI version. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): implement WithDetails() and FromTyped() APIs WithDetails(): - Adds WithDetails(message, details...) method to StatusBuilder - Allows adding structured key-value details alongside status messages - Uses existing StatusDetail and statusWithDetailsExpr infrastructure - Example: s.WithDetails(s.Format("Ready: %v", ...), s.Detail("endpoint", ...)) FromTyped(): - Converts typed Kubernetes objects (runtime.Object) to Resource - Provides compile-time type safety for building resources - Requires TypeMeta to be set on the object - Includes MustFromTyped() variant that panics on error - Example: defkit.FromTyped(&appsv1.Deployment{...}) Both APIs were documented in the KEP but not implemented. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Style(defkit): apply gofmt formatting Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(defkit): fix remote module download with @latest version When downloading a Go module without an explicit version, always append @latest to ensure go mod download fetches from the remote repository instead of skipping the download. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(defkit): support running def commands from any directory Previously, module commands like `vela def list-module` only worked when run from within the kubevela repository. Now they work from any directory by honoring replace directives in the source module's go.mod. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): generate doc.go files in init-module Create doc.go files with package documentation in each definition directory (components, traits, policies, workflowsteps). This ensures go mod tidy works correctly by making each directory a valid Go package, and provides helpful examples for users creating new definitions. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(defkit): deduplicate definitions from overlapping directory scans The module loader scans both conventional directories (components/, traits/, etc.) and the root directory. Since DiscoverDefinitions uses recursive filepath.Walk, files in subdirectories were found twice. Added file tracking to skip already-processed files. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(defkit): validate placement constraints and fix GOWORK interference Add validation for conflicting placement constraints at registration time. Definitions with logically impossible placement (e.g., same condition in both RunOn and NotRunOn) now fail fast with a clear error message. Also fix placement loading when parent directories contain go.work files by setting GOWORK=off when running the registry generator. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add parameter schema constraints and runtime condition methods Extend the parameter fluent API with comprehensive validation and conditional logic support: - Schema constraints for input validation (Min/Max, Pattern, MinLen/MaxLen, MinItems/MaxItems) - Runtime conditions for template logic (In, Contains, Matches, StartsWith/EndsWith, Len*, IsEmpty/IsNotEmpty, HasKey, IsFalse) Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(defkit): add waitFor support with CUE expressions for module hooks Add the ability to specify custom readiness conditions for module hooks using the new `waitFor` field. This allows users to define precise conditions for when resources should be considered ready. The waitFor field supports two formats: - Simple condition name (e.g., "Ready", "Established") - checks status.conditions for the named condition with status "True" - CUE expression (e.g., "status.replicas == status.readyReplicas") - evaluated against the full resource for flexible readiness checks Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Feat(addon): add godef support for Go-based definitions in addons Add support for a godef/ folder in addons that allows writing definitions in Go instead of CUE. When an addon is enabled, Go definitions are automatically compiled to CUE and deployed alongside traditional CUE definitions. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: lint issues and make reviewable Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: lint and build failure Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: lint and ci errors Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: golangci-lint errors for defkit package - Use standard library errors (errors.Is/As) instead of pkg/errors - Fix ineffassign issues by scoping variables correctly - Add nolint comments for intentional nilerr, makezero patterns - Combine chained appends in addon init.go - Add gosec nolint for CLI file operations and permissions - Increase gocyclo threshold to 35, nolint complex CLI commands Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: kubectl installation with retry and fallback version in github actions Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix(ci): hardcode kubectl version to avoid flaky CDN endpoint Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Chore: improve test coverage for codecov Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Chore: add more tests for codecov and CI to pass Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: ci failure on style Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: OperatorNotEquals to fail closed with empty values Change NotEquals operator to return false when Values slice is empty, matching the fail-closed behavior of Equals operator. This prevents silent widening of placement eligibility when a malformed constraint is created. Following Kubernetes label selector semantics where In/NotIn operators require non-empty values, we apply a fail-closed approach for safety in placement decisions. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: OpenArrayParam field shadowing and remove redundant GetName() Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: path traversal vulnerability in Go definition scaffolding Validate Go definition names before using them in file paths to prevent creation of files outside the addon directory. Unsanitized names could contain path traversal segments (e.g., "../../../etc/passwd") allowing arbitrary file writes. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: unescaped string interpolation in health_expr CUE generation Use %q format verb in formatValue() to properly escape quotes and special characters when generating CUE strings. Update fieldContainsExpr to use formatValue() instead of raw string interpolation. This prevents invalid CUE when substring values contain quotes or backslashes. Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: Guard against typed nil in Gomega matchers to prevent panic Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: Guard against malformed bracket path in parseBracketAccess Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: incomplete AppRevision test to actually verify resolution Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Fix: apply fail-closed behavior to NotIn with empty values Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> * Doc: Added note about RawCUE and some alignment style Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in> --------- Signed-off-by: Anoop Gopalakrishnan <anoop2811@aol.in>
1025 lines
31 KiB
Go
1025 lines
31 KiB
Go
/*
|
|
Copyright 2025 The KubeVela Authors.
|
|
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
*/
|
|
|
|
// Package goloader provides functionality to load and process Go-based KubeVela definitions.
|
|
// It supports loading definitions from Go source files and converting them to CUE format.
|
|
package goloader
|
|
|
|
import (
|
|
"bytes"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"go/ast"
|
|
"go/parser"
|
|
"go/token"
|
|
"os"
|
|
"os/exec"
|
|
"path/filepath"
|
|
"runtime"
|
|
"strings"
|
|
"sync"
|
|
|
|
"github.com/oam-dev/kubevela/pkg/definition/defkit"
|
|
"github.com/oam-dev/kubevela/pkg/definition/defkit/placement"
|
|
)
|
|
|
|
// GoExtension is the file extension for Go files
|
|
const GoExtension = ".go"
|
|
|
|
// DefinitionInfo holds information about a discovered Go definition
|
|
type DefinitionInfo struct {
|
|
// Name is the definition name (e.g., "webservice", "daemon")
|
|
Name string
|
|
// Type is the definition type (e.g., "component", "trait", "policy", "workflow-step")
|
|
Type string
|
|
// FunctionName is the Go function that returns the definition
|
|
FunctionName string
|
|
// FilePath is the path to the Go file containing the definition
|
|
FilePath string
|
|
// PackageName is the Go package name
|
|
PackageName string
|
|
// Placement contains the definition-level placement constraints (if any)
|
|
Placement *DefinitionPlacement
|
|
}
|
|
|
|
// DefinitionPlacement holds placement constraints for a definition.
|
|
// This is extracted from the definition's Go code.
|
|
type DefinitionPlacement struct {
|
|
// RunOn specifies conditions that must be satisfied for the definition to run.
|
|
RunOn []PlacementCondition `json:"runOn,omitempty"`
|
|
// NotRunOn specifies conditions that exclude clusters from running the definition.
|
|
NotRunOn []PlacementCondition `json:"notRunOn,omitempty"`
|
|
}
|
|
|
|
// PlacementCondition represents a single placement condition.
|
|
type PlacementCondition struct {
|
|
Key string `json:"key"`
|
|
Operator string `json:"operator"`
|
|
Values []string `json:"values,omitempty"`
|
|
}
|
|
|
|
// ToPlacementSpec converts DefinitionPlacement to placement.PlacementSpec.
|
|
func (p *DefinitionPlacement) ToPlacementSpec() placement.PlacementSpec {
|
|
if p == nil {
|
|
return placement.PlacementSpec{}
|
|
}
|
|
|
|
spec := placement.PlacementSpec{}
|
|
|
|
for _, cond := range p.RunOn {
|
|
spec.RunOn = append(spec.RunOn, &placement.LabelCondition{
|
|
Key: cond.Key,
|
|
Operator: placement.Operator(cond.Operator),
|
|
Values: cond.Values,
|
|
})
|
|
}
|
|
|
|
for _, cond := range p.NotRunOn {
|
|
spec.NotRunOn = append(spec.NotRunOn, &placement.LabelCondition{
|
|
Key: cond.Key,
|
|
Operator: placement.Operator(cond.Operator),
|
|
Values: cond.Values,
|
|
})
|
|
}
|
|
|
|
return spec
|
|
}
|
|
|
|
// IsEmpty returns true if no placement constraints are defined.
|
|
func (p *DefinitionPlacement) IsEmpty() bool {
|
|
return p == nil || (len(p.RunOn) == 0 && len(p.NotRunOn) == 0)
|
|
}
|
|
|
|
// LoadResult contains the result of loading Go definitions
|
|
type LoadResult struct {
|
|
// CUE is the generated CUE string
|
|
CUE string
|
|
// YAML is the generated YAML string (Kubernetes CR format)
|
|
YAML []byte
|
|
// Definition contains metadata about the definition
|
|
Definition DefinitionInfo
|
|
// Error is set if loading failed for this definition
|
|
Error error
|
|
}
|
|
|
|
// GeneratorEnvironment manages a reusable temp directory for CUE generation.
|
|
// This avoids running `go mod tidy` for each definition - it runs only ONCE per module.
|
|
type GeneratorEnvironment struct {
|
|
// TempDir is the temporary directory with go.mod already set up
|
|
TempDir string
|
|
// ModuleRoot is the root directory of the user's Go module
|
|
ModuleRoot string
|
|
// ModuleName is the Go module name from go.mod
|
|
ModuleName string
|
|
// mu protects concurrent access during parallel generation
|
|
mu sync.Mutex
|
|
}
|
|
|
|
// NewGeneratorEnvironment creates a new generator environment for a module.
|
|
// It sets up the temp directory and runs `go mod tidy` ONCE.
|
|
func NewGeneratorEnvironment(moduleRoot string) (*GeneratorEnvironment, error) {
|
|
// Get the module name from go.mod
|
|
moduleName, err := getModuleNameFromRoot(moduleRoot)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to get module name for %s: %w", moduleRoot, err)
|
|
}
|
|
|
|
// Create a temporary directory for the generator
|
|
tempDir, err := os.MkdirTemp("", "vela-def-gen-*")
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to create temp directory: %w", err)
|
|
}
|
|
|
|
// Discover all Go packages in the module (components, traits, etc.)
|
|
subPackages := discoverSubPackages(moduleRoot, moduleName)
|
|
|
|
// Create go.mod that references the original module
|
|
// First, try to copy replace directives from the source module's go.mod
|
|
// This handles cases where the user has a local replace for kubevela
|
|
sourceReplaces := getReplacesFromGoMod(moduleRoot)
|
|
|
|
// If no kubevela replace in source, try to find it locally
|
|
kubeVelaReplace := ""
|
|
if !strings.Contains(sourceReplaces, "github.com/oam-dev/kubevela") {
|
|
kubeVelaRoot := findKubeVelaRoot()
|
|
if kubeVelaRoot != "" {
|
|
kubeVelaReplace = fmt.Sprintf("replace github.com/oam-dev/kubevela => %s\n", kubeVelaRoot)
|
|
}
|
|
}
|
|
|
|
goMod := fmt.Sprintf(`module vela-def-gen
|
|
|
|
go 1.21
|
|
|
|
require github.com/oam-dev/kubevela v0.0.0
|
|
require %s v0.0.0
|
|
|
|
%s%sreplace %s => %s
|
|
`, moduleName, sourceReplaces, kubeVelaReplace, moduleName, moduleRoot)
|
|
|
|
if err := os.WriteFile(filepath.Join(tempDir, "go.mod"), []byte(goMod), 0600); err != nil {
|
|
_ = os.RemoveAll(tempDir)
|
|
return nil, fmt.Errorf("failed to write go.mod: %w", err)
|
|
}
|
|
|
|
// Write a placeholder main.go that imports ALL subpackages
|
|
// This ensures go mod tidy properly sets up requirements
|
|
var imports strings.Builder
|
|
imports.WriteString(fmt.Sprintf("\t_ \"%s\"\n", moduleName))
|
|
for _, pkg := range subPackages {
|
|
imports.WriteString(fmt.Sprintf("\t_ \"%s\"\n", pkg))
|
|
}
|
|
|
|
placeholderMain := fmt.Sprintf(`package main
|
|
|
|
import (
|
|
"fmt"
|
|
%s)
|
|
|
|
func main() {
|
|
fmt.Println("placeholder")
|
|
}
|
|
`, imports.String())
|
|
|
|
if err := os.WriteFile(filepath.Join(tempDir, "main.go"), []byte(placeholderMain), 0600); err != nil {
|
|
_ = os.RemoveAll(tempDir)
|
|
return nil, fmt.Errorf("failed to write placeholder main.go: %w", err)
|
|
}
|
|
|
|
// Run go mod tidy ONCE for the entire module
|
|
tidyCmd := exec.Command("go", "mod", "tidy", "-e")
|
|
tidyCmd.Dir = tempDir
|
|
tidyCmd.Env = append(os.Environ(), "GO111MODULE=on")
|
|
if output, err := tidyCmd.CombinedOutput(); err != nil {
|
|
_ = os.RemoveAll(tempDir)
|
|
return nil, fmt.Errorf("go mod tidy failed: %s: %w", string(output), err)
|
|
}
|
|
|
|
return &GeneratorEnvironment{
|
|
TempDir: tempDir,
|
|
ModuleRoot: moduleRoot,
|
|
ModuleName: moduleName,
|
|
}, nil
|
|
}
|
|
|
|
// discoverSubPackages finds all Go packages under a module root
|
|
func discoverSubPackages(moduleRoot, moduleName string) []string {
|
|
var packages []string
|
|
seen := make(map[string]bool)
|
|
|
|
_ = filepath.Walk(moduleRoot, func(path string, info os.FileInfo, err error) error {
|
|
if err != nil {
|
|
return nil //nolint:nilerr // Intentionally skip errors to continue walking
|
|
}
|
|
if info.IsDir() {
|
|
// Skip hidden directories and common non-Go directories
|
|
name := info.Name()
|
|
if strings.HasPrefix(name, ".") || name == "vendor" || name == "testdata" {
|
|
return filepath.SkipDir
|
|
}
|
|
return nil
|
|
}
|
|
// Check if this is a Go file (not test)
|
|
if !strings.HasSuffix(path, ".go") || strings.HasSuffix(path, "_test.go") {
|
|
return nil
|
|
}
|
|
// Get the directory and compute import path
|
|
dir := filepath.Dir(path)
|
|
relPath, err := filepath.Rel(moduleRoot, dir)
|
|
if err != nil {
|
|
return nil //nolint:nilerr // Skip files where we can't compute relative path
|
|
}
|
|
// Build the import path
|
|
var importPath string
|
|
if relPath == "." {
|
|
importPath = moduleName
|
|
} else {
|
|
importPath = moduleName + "/" + filepath.ToSlash(relPath)
|
|
}
|
|
if !seen[importPath] {
|
|
seen[importPath] = true
|
|
if importPath != moduleName { // Don't duplicate the root
|
|
packages = append(packages, importPath)
|
|
}
|
|
}
|
|
return nil
|
|
})
|
|
|
|
return packages
|
|
}
|
|
|
|
// Close cleans up the temporary directory
|
|
func (env *GeneratorEnvironment) Close() error {
|
|
if env.TempDir != "" {
|
|
return os.RemoveAll(env.TempDir)
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// GenerateCUE generates CUE for a single definition, reusing the pre-configured environment.
|
|
// This is much faster than GenerateCUEFromGoFile because it doesn't run `go mod tidy`.
|
|
// Note: This method is thread-safe but serializes access. For parallel processing,
|
|
// use GenerateCUEParallel which creates separate files for each worker.
|
|
func (env *GeneratorEnvironment) GenerateCUE(filePath string, defInfo DefinitionInfo) (string, error) {
|
|
env.mu.Lock()
|
|
defer env.mu.Unlock()
|
|
|
|
return env.generateCUEInternal(filePath, defInfo, "main.go")
|
|
}
|
|
|
|
// generateCUEInternal is the internal implementation that generates CUE.
|
|
// The filename parameter allows parallel workers to use different files.
|
|
func (env *GeneratorEnvironment) generateCUEInternal(filePath string, defInfo DefinitionInfo, filename string) (string, error) {
|
|
// Get the import path for this file
|
|
absPath, err := filepath.Abs(filePath)
|
|
if err != nil {
|
|
return "", fmt.Errorf("failed to get absolute path for %s: %w", filePath, err)
|
|
}
|
|
|
|
_, importPath, err := findModuleInfo(filepath.Dir(absPath))
|
|
if err != nil {
|
|
return "", fmt.Errorf("failed to find module info for %s: %w", filePath, err)
|
|
}
|
|
|
|
// Generate the Go program for this definition
|
|
genProgram := generateCUEGeneratorProgram(importPath, defInfo)
|
|
|
|
// Write the generator program
|
|
genFile := filepath.Join(env.TempDir, filename)
|
|
if err := os.WriteFile(genFile, []byte(genProgram), 0600); err != nil {
|
|
return "", fmt.Errorf("failed to write generator program: %w", err)
|
|
}
|
|
|
|
// Run the generator (no go mod tidy needed - already done)
|
|
runCmd := exec.Command("go", "run", filename)
|
|
runCmd.Dir = env.TempDir
|
|
runCmd.Env = append(os.Environ(), "GO111MODULE=on")
|
|
var stdout, stderr bytes.Buffer
|
|
runCmd.Stdout = &stdout
|
|
runCmd.Stderr = &stderr
|
|
if err := runCmd.Run(); err != nil {
|
|
return "", fmt.Errorf("generator failed: %s: %w", stderr.String(), err)
|
|
}
|
|
|
|
return stdout.String(), nil
|
|
}
|
|
|
|
// GenerateCUEParallel generates CUE for multiple definitions in parallel.
|
|
// It uses worker goroutines to process definitions concurrently.
|
|
// Each worker uses a unique filename to avoid conflicts.
|
|
func (env *GeneratorEnvironment) GenerateCUEParallel(definitions []DefinitionInfo) []LoadResult {
|
|
results := make([]LoadResult, len(definitions))
|
|
|
|
// Use number of CPUs as worker count, but cap at number of definitions
|
|
numWorkers := runtime.NumCPU()
|
|
if numWorkers > len(definitions) {
|
|
numWorkers = len(definitions)
|
|
}
|
|
if numWorkers < 1 {
|
|
numWorkers = 1
|
|
}
|
|
|
|
// Create work channel
|
|
type workItem struct {
|
|
index int
|
|
defInfo DefinitionInfo
|
|
}
|
|
workChan := make(chan workItem, len(definitions))
|
|
var wg sync.WaitGroup
|
|
|
|
// Start workers - each worker gets a unique ID for its filename
|
|
for workerID := 0; workerID < numWorkers; workerID++ {
|
|
wg.Add(1)
|
|
go func(id int) {
|
|
defer wg.Done()
|
|
// Each worker uses a unique filename to enable true parallelism
|
|
filename := fmt.Sprintf("worker_%d.go", id)
|
|
for work := range workChan {
|
|
result := LoadResult{Definition: work.defInfo}
|
|
cue, err := env.generateCUEInternal(work.defInfo.FilePath, work.defInfo, filename)
|
|
if err != nil {
|
|
result.Error = err
|
|
} else {
|
|
result.CUE = cue
|
|
}
|
|
results[work.index] = result
|
|
}
|
|
}(workerID)
|
|
}
|
|
|
|
// Send work
|
|
for i, def := range definitions {
|
|
workChan <- workItem{index: i, defInfo: def}
|
|
}
|
|
close(workChan)
|
|
|
|
// Wait for all workers to finish
|
|
wg.Wait()
|
|
|
|
return results
|
|
}
|
|
|
|
// IsGoFile checks if a file is a Go source file (not a test file)
|
|
func IsGoFile(path string) bool {
|
|
return strings.HasSuffix(path, GoExtension) && !strings.HasSuffix(path, "_test.go")
|
|
}
|
|
|
|
// IsGoDefinitionFile checks if a Go file likely contains defkit definitions
|
|
// by looking for defkit imports
|
|
func IsGoDefinitionFile(path string) (bool, error) {
|
|
if !IsGoFile(path) {
|
|
return false, nil
|
|
}
|
|
|
|
content, err := os.ReadFile(path) //nolint:gosec // G304: Path is from trusted module directory walk
|
|
if err != nil {
|
|
return false, fmt.Errorf("failed to read file %s: %w", path, err)
|
|
}
|
|
|
|
// Quick check for defkit import
|
|
return strings.Contains(string(content), "github.com/oam-dev/kubevela/pkg/definition/defkit"), nil
|
|
}
|
|
|
|
// DiscoverDefinitions finds Go files that contain defkit definitions in a directory
|
|
func DiscoverDefinitions(dir string) ([]string, error) {
|
|
var files []string
|
|
|
|
err := filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if info.IsDir() {
|
|
return nil
|
|
}
|
|
|
|
isDefFile, err := IsGoDefinitionFile(path)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
if isDefFile {
|
|
files = append(files, path)
|
|
}
|
|
return nil
|
|
})
|
|
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to walk directory %s: %w", dir, err)
|
|
}
|
|
|
|
return files, nil
|
|
}
|
|
|
|
// AnalyzeGoFile analyzes a Go file and extracts definition function information
|
|
func AnalyzeGoFile(filePath string) ([]DefinitionInfo, error) {
|
|
fset := token.NewFileSet()
|
|
node, err := parser.ParseFile(fset, filePath, nil, parser.ParseComments)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("failed to parse Go file %s: %w", filePath, err)
|
|
}
|
|
|
|
var definitions []DefinitionInfo
|
|
packageName := node.Name.Name
|
|
|
|
// Look for functions that return *defkit.ComponentDefinition or similar
|
|
for _, decl := range node.Decls {
|
|
fn, ok := decl.(*ast.FuncDecl)
|
|
if !ok {
|
|
continue
|
|
}
|
|
|
|
// Skip methods (we only want package-level functions)
|
|
if fn.Recv != nil {
|
|
continue
|
|
}
|
|
|
|
// Check return type
|
|
if fn.Type.Results == nil || len(fn.Type.Results.List) != 1 {
|
|
continue
|
|
}
|
|
|
|
retType := fn.Type.Results.List[0].Type
|
|
defType := getDefinitionType(retType)
|
|
if defType == "" {
|
|
continue
|
|
}
|
|
|
|
// Extract definition name from function name
|
|
defName := extractDefinitionName(fn.Name.Name, defType)
|
|
|
|
definitions = append(definitions, DefinitionInfo{
|
|
Name: defName,
|
|
Type: defType,
|
|
FunctionName: fn.Name.Name,
|
|
FilePath: filePath,
|
|
PackageName: packageName,
|
|
})
|
|
}
|
|
|
|
return definitions, nil
|
|
}
|
|
|
|
// getDefinitionType extracts the definition type from a return type AST
|
|
func getDefinitionType(expr ast.Expr) string {
|
|
// Handle pointer type: *defkit.ComponentDefinition
|
|
if star, ok := expr.(*ast.StarExpr); ok {
|
|
expr = star.X
|
|
}
|
|
|
|
// Handle selector expression: defkit.ComponentDefinition
|
|
sel, ok := expr.(*ast.SelectorExpr)
|
|
if !ok {
|
|
return ""
|
|
}
|
|
|
|
// Check if it's from defkit package
|
|
ident, ok := sel.X.(*ast.Ident)
|
|
if !ok || ident.Name != "defkit" {
|
|
return ""
|
|
}
|
|
|
|
switch sel.Sel.Name {
|
|
case "ComponentDefinition":
|
|
return "component"
|
|
case "TraitDefinition":
|
|
return "trait"
|
|
case "PolicyDefinition":
|
|
return "policy"
|
|
case "WorkflowStepDefinition":
|
|
return "workflow-step"
|
|
default:
|
|
return ""
|
|
}
|
|
}
|
|
|
|
// extractDefinitionName extracts a definition name from a function name
|
|
// e.g., "WebserviceComponent" -> "webservice", "Daemon" -> "daemon"
|
|
func extractDefinitionName(funcName, defType string) string {
|
|
// Map definition type to expected suffix
|
|
typeToSuffix := map[string]string{
|
|
"component": "Component",
|
|
"trait": "Trait",
|
|
"policy": "Policy",
|
|
"workflow-step": "WorkflowStep",
|
|
}
|
|
|
|
name := funcName
|
|
|
|
// First, try the suffix matching the definition type
|
|
if suffix, ok := typeToSuffix[defType]; ok {
|
|
if strings.HasSuffix(name, suffix) {
|
|
return strings.ToLower(strings.TrimSuffix(name, suffix))
|
|
}
|
|
}
|
|
|
|
// Fallback: try common suffixes
|
|
suffixes := []string{"Component", "Trait", "Policy", "WorkflowStep", "Definition"}
|
|
for _, suffix := range suffixes {
|
|
if strings.HasSuffix(name, suffix) {
|
|
name = strings.TrimSuffix(name, suffix)
|
|
break
|
|
}
|
|
}
|
|
|
|
return strings.ToLower(name)
|
|
}
|
|
|
|
// GenerateCUEFromGoFile generates CUE from a Go definition file
|
|
// This uses a code generation approach: it creates a temporary Go program
|
|
// that imports the definition and calls ToCue() on it
|
|
func GenerateCUEFromGoFile(filePath string, defInfo DefinitionInfo) (string, error) {
|
|
// Get the absolute path and module information
|
|
absPath, err := filepath.Abs(filePath)
|
|
if err != nil {
|
|
return "", fmt.Errorf("failed to get absolute path for %s: %w", filePath, err)
|
|
}
|
|
|
|
// Find the module root and import path
|
|
moduleRoot, importPath, err := findModuleInfo(filepath.Dir(absPath))
|
|
if err != nil {
|
|
return "", fmt.Errorf("failed to find module info for %s: %w", filePath, err)
|
|
}
|
|
|
|
// Get the module name from the import path (the base module, not subpackage)
|
|
moduleName, err := getModuleNameFromRoot(moduleRoot)
|
|
if err != nil {
|
|
return "", fmt.Errorf("failed to get module name for %s: %w", moduleRoot, err)
|
|
}
|
|
|
|
// Create a temporary directory for the generator
|
|
tempDir, err := os.MkdirTemp("", "vela-def-gen-*")
|
|
if err != nil {
|
|
return "", fmt.Errorf("failed to create temp directory: %w", err)
|
|
}
|
|
defer func() { _ = os.RemoveAll(tempDir) }()
|
|
|
|
// Generate the temporary Go program
|
|
genProgram := generateCUEGeneratorProgram(importPath, defInfo)
|
|
|
|
// Write the generator program
|
|
genFile := filepath.Join(tempDir, "main.go")
|
|
if err := os.WriteFile(genFile, []byte(genProgram), 0600); err != nil {
|
|
return "", fmt.Errorf("failed to write generator program: %w", err)
|
|
}
|
|
|
|
// Create go.mod that references the original module
|
|
// We need to add replace directives for:
|
|
// 1. github.com/oam-dev/kubevela -> the kubevela repo (for defkit) - only if found locally
|
|
// 2. The user's module -> their local path (so go mod tidy doesn't try to fetch from remote)
|
|
kubeVelaRoot := findKubeVelaRoot()
|
|
var kubeVelaReplace string
|
|
if kubeVelaRoot != "" {
|
|
kubeVelaReplace = fmt.Sprintf("replace github.com/oam-dev/kubevela => %s\n", kubeVelaRoot)
|
|
}
|
|
goMod := fmt.Sprintf(`module vela-def-gen
|
|
|
|
go 1.21
|
|
|
|
require github.com/oam-dev/kubevela v0.0.0
|
|
require %s v0.0.0
|
|
|
|
%sreplace %s => %s
|
|
`, moduleName, kubeVelaReplace, moduleName, moduleRoot)
|
|
|
|
if err := os.WriteFile(filepath.Join(tempDir, "go.mod"), []byte(goMod), 0600); err != nil {
|
|
return "", fmt.Errorf("failed to write go.mod: %w", err)
|
|
}
|
|
|
|
// Run go mod tidy with -e flag to ignore errors for missing modules
|
|
// This is needed because the user's module may have dependencies we can't resolve
|
|
tidyCmd := exec.Command("go", "mod", "tidy", "-e")
|
|
tidyCmd.Dir = tempDir
|
|
tidyCmd.Env = append(os.Environ(), "GO111MODULE=on")
|
|
if output, err := tidyCmd.CombinedOutput(); err != nil {
|
|
return "", fmt.Errorf("go mod tidy failed: %s: %w", string(output), err)
|
|
}
|
|
|
|
// Run the generator
|
|
runCmd := exec.Command("go", "run", "main.go")
|
|
runCmd.Dir = tempDir
|
|
runCmd.Env = append(os.Environ(), "GO111MODULE=on")
|
|
var stdout, stderr bytes.Buffer
|
|
runCmd.Stdout = &stdout
|
|
runCmd.Stderr = &stderr
|
|
if err := runCmd.Run(); err != nil {
|
|
return "", fmt.Errorf("generator failed: %s: %w", stderr.String(), err)
|
|
}
|
|
|
|
return stdout.String(), nil
|
|
}
|
|
|
|
// getModuleNameFromRoot reads the module name from go.mod in the given directory
|
|
func getModuleNameFromRoot(moduleRoot string) (string, error) {
|
|
goModContent, err := os.ReadFile(filepath.Join(moduleRoot, "go.mod")) //nolint:gosec // G304: Reading go.mod from user-specified module
|
|
if err != nil {
|
|
return "", fmt.Errorf("failed to read go.mod: %w", err)
|
|
}
|
|
|
|
lines := strings.Split(string(goModContent), "\n")
|
|
for _, line := range lines {
|
|
line = strings.TrimSpace(line)
|
|
if strings.HasPrefix(line, "module ") {
|
|
return strings.TrimPrefix(line, "module "), nil
|
|
}
|
|
}
|
|
return "", errors.New("could not find module name in go.mod")
|
|
}
|
|
|
|
// getReplacesFromGoMod reads replace directives from a go.mod file
|
|
// and returns them as a string that can be included in another go.mod.
|
|
// This allows copying replace directives from the source module to the temp module.
|
|
func getReplacesFromGoMod(moduleRoot string) string {
|
|
goModContent, err := os.ReadFile(filepath.Join(moduleRoot, "go.mod")) //nolint:gosec // G304: Reading go.mod from user-specified module
|
|
if err != nil {
|
|
return ""
|
|
}
|
|
|
|
var replaces strings.Builder
|
|
lines := strings.Split(string(goModContent), "\n")
|
|
inReplaceBlock := false
|
|
|
|
for _, line := range lines {
|
|
trimmed := strings.TrimSpace(line)
|
|
|
|
// Handle replace block: replace ( ... )
|
|
if trimmed == "replace (" {
|
|
inReplaceBlock = true
|
|
continue
|
|
}
|
|
if inReplaceBlock {
|
|
if trimmed == ")" {
|
|
inReplaceBlock = false
|
|
continue
|
|
}
|
|
// Each line in the block is a replace directive
|
|
if trimmed != "" && !strings.HasPrefix(trimmed, "//") {
|
|
replaces.WriteString("replace " + trimmed + "\n")
|
|
}
|
|
continue
|
|
}
|
|
|
|
// Handle single-line replace: replace foo => bar
|
|
if strings.HasPrefix(trimmed, "replace ") {
|
|
replaces.WriteString(trimmed + "\n")
|
|
}
|
|
}
|
|
|
|
return replaces.String()
|
|
}
|
|
|
|
// findKubeVelaRoot attempts to find the kubevela repository root
|
|
// It first checks if we're running from within kubevela, then falls back to GOPATH
|
|
func findKubeVelaRoot() string {
|
|
// First, try to find kubevela in the current working directory ancestry
|
|
cwd, err := os.Getwd()
|
|
if err == nil {
|
|
current := cwd
|
|
for {
|
|
goModPath := filepath.Join(current, "go.mod")
|
|
if content, err := os.ReadFile(goModPath); err == nil { //nolint:gosec // G304: Searching for kubevela root in parent directories
|
|
if strings.Contains(string(content), "module github.com/oam-dev/kubevela") {
|
|
return current
|
|
}
|
|
}
|
|
parent := filepath.Dir(current)
|
|
if parent == current {
|
|
break
|
|
}
|
|
current = parent
|
|
}
|
|
}
|
|
|
|
// Try GOPATH
|
|
gopath := os.Getenv("GOPATH")
|
|
if gopath == "" {
|
|
gopath = filepath.Join(os.Getenv("HOME"), "go")
|
|
}
|
|
kubevelaPath := filepath.Join(gopath, "src", "github.com", "oam-dev", "kubevela")
|
|
if _, err := os.Stat(kubevelaPath); err == nil {
|
|
return kubevelaPath
|
|
}
|
|
|
|
// Return empty string - kubevela will be resolved from module cache by go mod tidy
|
|
return ""
|
|
}
|
|
|
|
// generateCUEGeneratorProgram creates a Go program that generates CUE from a definition
|
|
func generateCUEGeneratorProgram(importPath string, defInfo DefinitionInfo) string {
|
|
return fmt.Sprintf(`package main
|
|
|
|
import (
|
|
"fmt"
|
|
|
|
def "%s"
|
|
)
|
|
|
|
func main() {
|
|
component := def.%s()
|
|
fmt.Print(component.ToCue())
|
|
}
|
|
`, importPath, defInfo.FunctionName)
|
|
}
|
|
|
|
// findModuleInfo finds the Go module root and import path for a directory
|
|
func findModuleInfo(dir string) (moduleRoot, importPath string, err error) {
|
|
// Walk up to find go.mod
|
|
current := dir
|
|
for {
|
|
goModPath := filepath.Join(current, "go.mod")
|
|
if _, err := os.Stat(goModPath); err == nil {
|
|
moduleRoot = current
|
|
break
|
|
}
|
|
parent := filepath.Dir(current)
|
|
if parent == current {
|
|
return "", "", errors.New("could not find go.mod in parent directories")
|
|
}
|
|
current = parent
|
|
}
|
|
|
|
// Read go.mod to get module name
|
|
goModContent, err := os.ReadFile(filepath.Join(moduleRoot, "go.mod")) //nolint:gosec // G304: Reading go.mod from user-specified module
|
|
if err != nil {
|
|
return "", "", fmt.Errorf("failed to read go.mod: %w", err)
|
|
}
|
|
|
|
// Parse module name from go.mod
|
|
lines := strings.Split(string(goModContent), "\n")
|
|
var moduleName string
|
|
for _, line := range lines {
|
|
line = strings.TrimSpace(line)
|
|
if strings.HasPrefix(line, "module ") {
|
|
moduleName = strings.TrimPrefix(line, "module ")
|
|
break
|
|
}
|
|
}
|
|
if moduleName == "" {
|
|
return "", "", errors.New("could not find module name in go.mod")
|
|
}
|
|
|
|
// Calculate import path relative to module root
|
|
relPath, err := filepath.Rel(moduleRoot, dir)
|
|
if err != nil {
|
|
return "", "", fmt.Errorf("failed to calculate relative path: %w", err)
|
|
}
|
|
|
|
if relPath == "." {
|
|
importPath = moduleName
|
|
} else {
|
|
importPath = moduleName + "/" + filepath.ToSlash(relPath)
|
|
}
|
|
|
|
return moduleRoot, importPath, nil
|
|
}
|
|
|
|
// LoadFromFile loads a Go definition file and returns the generated CUE.
|
|
// If the file contains no definitions (e.g., helper files with shared types),
|
|
// it returns an empty slice without error.
|
|
func LoadFromFile(filePath string) ([]LoadResult, error) {
|
|
// Analyze the file to find definitions
|
|
definitions, err := AnalyzeGoFile(filePath)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// If no definitions found, return empty slice (not an error)
|
|
// This handles helper files like schemas.go that import defkit but don't define components
|
|
if len(definitions) == 0 {
|
|
return []LoadResult{}, nil
|
|
}
|
|
|
|
var results []LoadResult
|
|
for _, defInfo := range definitions {
|
|
result := LoadResult{Definition: defInfo}
|
|
|
|
cue, err := GenerateCUEFromGoFile(filePath, defInfo)
|
|
if err != nil {
|
|
result.Error = err
|
|
} else {
|
|
result.CUE = cue
|
|
}
|
|
|
|
results = append(results, result)
|
|
}
|
|
|
|
return results, nil
|
|
}
|
|
|
|
// LoadFromDirectory loads all Go definition files from a directory
|
|
func LoadFromDirectory(dir string) ([]LoadResult, error) {
|
|
files, err := DiscoverDefinitions(dir)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
var allResults []LoadResult
|
|
for _, file := range files {
|
|
results, err := LoadFromFile(file)
|
|
if err != nil {
|
|
// Add error result for the file (only for actual errors, not "no definitions")
|
|
allResults = append(allResults, LoadResult{
|
|
Definition: DefinitionInfo{FilePath: file},
|
|
Error: err,
|
|
})
|
|
continue
|
|
}
|
|
// Only add results if there are actual definitions
|
|
// (empty slice means the file is a helper file with no definitions)
|
|
allResults = append(allResults, results...)
|
|
}
|
|
|
|
return allResults, nil
|
|
}
|
|
|
|
// LoadFromFileWithEnv loads a Go definition file using a pre-configured generator environment.
|
|
// This is more efficient than LoadFromFile when loading multiple files from the same module.
|
|
func LoadFromFileWithEnv(env *GeneratorEnvironment, filePath string) ([]LoadResult, error) {
|
|
// Analyze the file to find definitions
|
|
definitions, err := AnalyzeGoFile(filePath)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// If no definitions found, return empty slice (not an error)
|
|
if len(definitions) == 0 {
|
|
return []LoadResult{}, nil
|
|
}
|
|
|
|
var results []LoadResult
|
|
for _, defInfo := range definitions {
|
|
result := LoadResult{Definition: defInfo}
|
|
|
|
cue, err := env.GenerateCUE(filePath, defInfo)
|
|
if err != nil {
|
|
result.Error = err
|
|
} else {
|
|
result.CUE = cue
|
|
}
|
|
|
|
results = append(results, result)
|
|
}
|
|
|
|
return results, nil
|
|
}
|
|
|
|
// LoadFromDirectoryWithEnv loads all Go definition files from a directory using a pre-configured environment.
|
|
// This is much faster than LoadFromDirectory because it runs `go mod tidy` only once.
|
|
func LoadFromDirectoryWithEnv(env *GeneratorEnvironment, dir string) ([]LoadResult, error) {
|
|
files, err := DiscoverDefinitions(dir)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
var allResults []LoadResult
|
|
for _, file := range files {
|
|
results, err := LoadFromFileWithEnv(env, file)
|
|
if err != nil {
|
|
allResults = append(allResults, LoadResult{
|
|
Definition: DefinitionInfo{FilePath: file},
|
|
Error: err,
|
|
})
|
|
continue
|
|
}
|
|
allResults = append(allResults, results...)
|
|
}
|
|
|
|
return allResults, nil
|
|
}
|
|
|
|
// LoadFromDirectoryOptimized loads all Go definition files from a directory with optimized performance.
|
|
// It creates a generator environment once, runs `go mod tidy` once, then processes all definitions.
|
|
// This is the recommended function for loading multiple definitions from a module.
|
|
func LoadFromDirectoryOptimized(dir string) ([]LoadResult, error) {
|
|
// Find module root
|
|
moduleRoot, _, err := findModuleInfo(dir)
|
|
if err != nil {
|
|
// Fall back to non-optimized loading if we can't find module root
|
|
return LoadFromDirectory(dir)
|
|
}
|
|
|
|
// Create generator environment (runs go mod tidy once)
|
|
env, err := NewGeneratorEnvironment(moduleRoot)
|
|
if err != nil {
|
|
// Fall back to non-optimized loading on error
|
|
return LoadFromDirectory(dir)
|
|
}
|
|
defer func() { _ = env.Close() }()
|
|
|
|
return LoadFromDirectoryWithEnv(env, dir)
|
|
}
|
|
|
|
// registryMainPath is the conventional path for the registry main program.
|
|
// Definition modules should have this file to enable efficient registry-based loading.
|
|
const registryMainPath = "cmd/register/main.go"
|
|
|
|
// LoadFromModuleWithRegistry loads definitions from a Go module using the registry pattern.
|
|
// This is the simplest and most efficient approach - it requires:
|
|
// 1. Each definition file has init() that calls defkit.Register()
|
|
// 2. Module has cmd/register/main.go that imports all packages and outputs JSON
|
|
//
|
|
// How it works:
|
|
// 1. Run `go run ./cmd/register` in the module directory
|
|
// 2. The main.go imports all definition packages (triggering init() functions)
|
|
// 3. init() functions call defkit.Register() for each definition
|
|
// 4. main() calls defkit.ToJSON() to output all registered definitions
|
|
// 5. CLI parses the JSON output
|
|
//
|
|
// This approach:
|
|
// - No temp directory needed
|
|
// - No go mod tidy needed (module is already set up)
|
|
// - Runs `go run` exactly ONCE
|
|
// - No AST parsing required
|
|
// - Fastest possible loading
|
|
func LoadFromModuleWithRegistry(moduleRoot string) ([]LoadResult, error) {
|
|
// Check if module has cmd/register/main.go
|
|
mainPath := filepath.Join(moduleRoot, registryMainPath)
|
|
if _, err := os.Stat(mainPath); os.IsNotExist(err) {
|
|
return nil, fmt.Errorf("module does not have %s - run 'vela def init-module' to create it", registryMainPath)
|
|
}
|
|
|
|
// Run go run ./cmd/register directly in the module
|
|
// GOWORK=off ensures consistent behavior when parent directories have go.work files
|
|
runCmd := exec.Command("go", "run", "./cmd/register")
|
|
runCmd.Dir = moduleRoot
|
|
runCmd.Env = append(os.Environ(), "GO111MODULE=on", "GOWORK=off")
|
|
var stdout, stderr bytes.Buffer
|
|
runCmd.Stdout = &stdout
|
|
runCmd.Stderr = &stderr
|
|
if err := runCmd.Run(); err != nil {
|
|
return nil, fmt.Errorf("registry generator failed: %s: %w", stderr.String(), err)
|
|
}
|
|
|
|
// Parse JSON output
|
|
var registryOutput defkit.RegistryOutput
|
|
if err := json.Unmarshal(stdout.Bytes(), ®istryOutput); err != nil {
|
|
return nil, fmt.Errorf("failed to parse registry output %s: %w", stdout.String(), err)
|
|
}
|
|
|
|
// Convert to LoadResults
|
|
results := make([]LoadResult, 0, len(registryOutput.Definitions))
|
|
for _, def := range registryOutput.Definitions {
|
|
result := LoadResult{
|
|
CUE: def.CUE,
|
|
Definition: DefinitionInfo{
|
|
Name: def.Name,
|
|
Type: string(def.Type),
|
|
},
|
|
}
|
|
|
|
// Convert placement if present
|
|
if def.Placement != nil {
|
|
result.Definition.Placement = &DefinitionPlacement{}
|
|
for _, cond := range def.Placement.RunOn {
|
|
result.Definition.Placement.RunOn = append(result.Definition.Placement.RunOn, PlacementCondition{
|
|
Key: cond.Key,
|
|
Operator: cond.Operator,
|
|
Values: cond.Values,
|
|
})
|
|
}
|
|
for _, cond := range def.Placement.NotRunOn {
|
|
result.Definition.Placement.NotRunOn = append(result.Definition.Placement.NotRunOn, PlacementCondition{
|
|
Key: cond.Key,
|
|
Operator: cond.Operator,
|
|
Values: cond.Values,
|
|
})
|
|
}
|
|
}
|
|
|
|
results = append(results, result)
|
|
}
|
|
|
|
return results, nil
|
|
}
|
|
|
|
// SupportsRegistry checks if a module uses the registry pattern.
|
|
// It checks for the presence of cmd/register/main.go which is the conventional
|
|
// entry point for registry-based definition loading.
|
|
func SupportsRegistry(moduleRoot string) bool {
|
|
mainPath := filepath.Join(moduleRoot, registryMainPath)
|
|
_, err := os.Stat(mainPath)
|
|
return err == nil
|
|
}
|
|
|
|
// LoadFromModuleAuto automatically selects the best loading strategy:
|
|
// - If the module uses defkit.Register(), use the registry approach
|
|
// - Otherwise, fall back to the AST-based approach
|
|
func LoadFromModuleAuto(moduleRoot string) ([]LoadResult, error) {
|
|
if SupportsRegistry(moduleRoot) {
|
|
return LoadFromModuleWithRegistry(moduleRoot)
|
|
}
|
|
return LoadFromDirectoryOptimized(moduleRoot)
|
|
}
|