diff --git a/docs/en/platform-engineers/trait.md b/docs/en/platform-engineers/trait.md index f324c59da..f8dc6cce5 100644 --- a/docs/en/platform-engineers/trait.md +++ b/docs/en/platform-engineers/trait.md @@ -1,84 +1,166 @@ # Extending Traits in KubeVela +> WARNINIG: you are now reading a platform builder/administrator oriented documentation. + In the following tutorial, you will learn how to add a new trait and expose it to users via Appfile. +We will use [KubeWatch](https://github.com/wonderflow/kubewatch) as example, this is a forked version +that we make it work as CRD controller. User could use CRD(`kubewatches.labs.bitnami.com`) to describe K8s resources they +want to watch including any types of CRD. -### Step 1: Install KubeWatch via Cap Center +## Step 1: Create Trait Definition -Add cap center that contains KubeWatch: +To register [KubeWatch](https://github.com/wonderflow/kubewatch) as a new trait in KubeVela, +the only thing needed is to create an OAM `TraitDefinition` object for it. +A full example can be found in this [kubewatch.yaml](https://github.com/oam-dev/catalog/blob/master/registry/kubewatch.yaml). +Several highlights are list below. -```bash -$ vela cap center config my-center https://github.com/oam-dev/catalog/tree/master/registry -successfully sync 2/2 from my-center remote center -Successfully configured capability center my-center and sync from remote +### 1. Describe The Trait Usage -$ vela cap center sync my-center -successfully sync 2/2 from my-center remote center -sync finished -``` - -Install KubeWatch: - -```bash -$ vela cap install my-center/kubewatch -Installing trait capability kubewatch +```yaml +... + name: kubewatch + annotations: + definition.oam.dev/description: "Add a watch for resource" ... -Successfully installed chart (kubewatch) with release name (kubewatch) -Successfully installed capability kubewatch from my-center - - ``` -### Step 2: Verify Kubewatch Trait Added +We use label `definition.oam.dev/description` to add one line description for this trait. +It will be shown in helper commands such as `$ vela traits`. +### 2. Register API Resource + +```yaml +... +spec: + definitionRef: + name: kubewatches.labs.bitnami.com +... +``` + +This is how you register Kubewatch's API resource (`kubewatches.labs.bitnami.com`) as the Trait. + + +KubeVela uses Kubernetes API resource discovery mechanism to manage all registered capabilities. + + +### 3. Configure Installation Dependency + +```yaml +... + extension: + install: + helm: + repo: my-repo + name: kubewatch + url: https://wonderflow.info/kubewatch/archives/ + version: 0.1.0 + ... +``` + +The `extension.install` field is used by KubeVela to automatically install the dependency (if any) when the new workload +type added to KubeVela. The dependency is described by a Helm chart custom resource. +We highly recommend you to configure this field since otherwise, +users will have to install dependencies like this kubewatch controller manually later to user your new trait. + +### 4. Define Workloads this trait can apply to + +```yaml +... +spec: + ... + appliesToWorkloads: + - "*" +... +``` + +A trait can work on specified workload or any kinds of workload, that deponds on what you describe here. +Use `"*"` to represent your trait can work on any workloads. + +You can also specify the trait can only work on K8s Deployment and Statefulset by describe like below: + +```yaml +... +spec: + ... + appliesToWorkloads: + - "deployments.apps" + - "statefulsets.apps" +... +``` + +### 5. Define the field if the trait can receive workload reference + +```yaml +... +spec: + workloadRefPath: spec.workloadRef +... +``` + +Once registered, the OAM framework can inject workload reference information automatically to trait CR object during creation or update. +The workload reference will include group, version, kind and name. Then, the trait can get the whole workload information +from this reference. + +With the help of the OAM framework, end users will never bother writing the relationship info such like `targetReference`. +Platform builders only need to declare this info here once, then the OAM framework will help glue them together. + +### 6. Define User Parameters + +```yaml +... + template: | + output: { + apiVersion: "labs.bitnami.com/v1alpha1" + kind: "KubeWatch" + spec: handler: webhook: url: parameter.webhook + } + parameter: { + webhook: string + } + ``` + +For a given capability, KubeVela leverages [CUElang](https://github.com/cuelang/cue/blob/master/doc/tutorial/kubernetes/README.md) +to define the parameters that the end users could configure in the Appfile. +In nutshell, `parameter.*` expected to be filled by users. + +> In the upcoming release, we will publish a detailed guide about defining CUE templates in KubeVela. +> For now, the best samples to learn about this section is the [built-in templates](https://github.com/oam-dev/kubevela/tree/master/hack/vela-templates) of KubeVela. + +Note that in this example, we only need to give the webhook url as parameter for using KubeWatch. + +## Step 2: Register New Trait to KubeVela + +As long as the definition file is ready, you just need to apply it to Kubernetes. + +```bash +$ kubectl apply -f https://raw.githubusercontent.com/oam-dev/catalog/master/registry/kubewatch.yaml +``` + +And the new trait will immediately become available for developers to use in KubeVela. +It may take some time to be available as the dependency(helm chart) need to install. ```bash $ vela traits -Synchronizing capabilities from cluster⌛ ... -Sync capabilities successfully ✅ (no changes) -TYPE CATEGORY DESCRIPTION -kubewatch trait Add a watch for resource -... +"my-repo" has been added to your repositories +Successfully installed chart (kubewatch) with release name (kubewatch) +Automatically discover capabilities successfully ✅ Add(1) Update(0) Delete(0) + +TYPE CATEGORY DESCRIPTION ++kubewatch trait Add a watch for resource + +NAME DESCRIPTION APPLIES TO +autoscale Automatically scale the app following certain triggers or metrics webservice + worker +kubewatch Add a watch for resource +metrics Configure metrics targets to be monitored for the app webservice + task +rollout Configure canary deployment strategy to release the app webservice +route Configure route policy to the app webservice +scaler Manually scale the app webservice + worker ``` -### Step 3: Adding Kubewatch Trait to The App - -Write an Appfile: - -```bash -$ cat << EOF > vela.yaml -name: testapp -services: - testsvc: - type: webservice - image: crccheck/hello-world - port: 8000 - route: - domain: testsvc.example.com -EOF -``` - -Deploy it: - -```bash -$ vela up -``` - -Now add `kubewatch` config to Appfile: - -```bash -$ cat << EOF >> vela.yaml - kubewatch: - webhook: https://hooks.slack.com/ -EOF -``` - -Update deployment: - -``` -$ vela up -``` - -Check your Slack channel to verify the nofitications: - -![Image of Kubewatch](../../resources/kubewatch-notif.jpg) +### Step 3: Using the new extended trait +Using the extended trait is just the same as the trait installed from Capability center, please [refer there to see how +to use](../developers/cap-center.md#Use-the-newly-installed-capability). diff --git a/docs/en/platform-engineers/workload-type.md b/docs/en/platform-engineers/workload-type.md index 09958bfd3..b5a722b31 100644 --- a/docs/en/platform-engineers/workload-type.md +++ b/docs/en/platform-engineers/workload-type.md @@ -6,7 +6,7 @@ In the following tutorial, you will learn how to add OpenFaaS Function a new wor ## Step 1: Create Workload Definition -To register OpenFaaS as a new workload type in KubeVela, the only thing needed is to create a OAM `WorkloadDefinition` object for it. A full example can be found in this [openfaas.yaml](https://github.com/oam-dev/catalog/blob/master/registry/openfaas.yaml). Several highlights are list below. +To register OpenFaaS as a new workload type in KubeVela, the only thing needed is to create an OAM `WorkloadDefinition` object for it. A full example can be found in this [openfaas.yaml](https://github.com/oam-dev/catalog/blob/master/registry/openfaas.yaml). Several highlights are list below. ### 1. Describe The Workload Type @@ -94,21 +94,26 @@ $ kubectl apply -f https://raw.githubusercontent.com/openfaas/faas-netes/master/ As long as the definition file is ready, you just need to apply it to Kubernetes. ```bash -$ kubectl apply -f openfaas.yaml +$ kubectl apply -f https://raw.githubusercontent.com/oam-dev/catalog/master/registry/openfaas.yaml ``` And the new workload type will immediately become available for developers to use in KubeVela. +It may take some time to be available as the dependency(helm chart) need to install. ```bash $ vela workloads +Successfully installed chart (openfaas) with release name (openfaas) +"my-repo" has been added to your repositories Automatically discover capabilities successfully ✅ Add(1) Update(0) Delete(0) -TYPE CATEGORY DESCRIPTION -*openfaas workload OpenFaaS function workload + +TYPE CATEGORY DESCRIPTION ++openfaas workload OpenFaaS function workload + NAME DESCRIPTION openfaas OpenFaaS function workload -task One-time task/job -webservice Long running service with network routes -worker Backend worker without ports exposed +task One-off task to run a piece of code or script to completion +webservice Long-running scalable service with stable endpoint to receive external traffic +worker Long-running scalable backend worker without network endpoint ``` ## (Optional) Step 3: Deploy OpenFaaS Function via Appfile diff --git a/pkg/commands/cli.go b/pkg/commands/cli.go index da6406179..bcb166c1c 100644 --- a/pkg/commands/cli.go +++ b/pkg/commands/cli.go @@ -6,11 +6,12 @@ import ( "os" "runtime" + "github.com/oam-dev/kubevela/pkg/utils/common" + "github.com/gosuri/uitable" "github.com/oam-dev/kubevela/api/types" "github.com/oam-dev/kubevela/cmd/vela/fake" "github.com/oam-dev/kubevela/pkg/commands/util" - "github.com/oam-dev/kubevela/pkg/oam" "github.com/oam-dev/kubevela/pkg/utils/system" "github.com/oam-dev/kubevela/version" "github.com/spf13/cobra" @@ -50,7 +51,7 @@ func NewCommand() *cobra.Command { commandArgs := types.Args{ Config: restConf, - Schema: oam.Scheme, + Schema: common.Scheme, } if err := system.InitDirs(); err != nil { diff --git a/pkg/controller/dependency/install.go b/pkg/controller/dependency/install.go index 002d94f31..a49b7d8e4 100644 --- a/pkg/controller/dependency/install.go +++ b/pkg/controller/dependency/install.go @@ -33,7 +33,6 @@ import ( "github.com/oam-dev/kubevela/api/types" cmdutil "github.com/oam-dev/kubevela/pkg/commands/util" - "github.com/oam-dev/kubevela/pkg/oam" "github.com/oam-dev/kubevela/pkg/utils/helm" ) @@ -48,7 +47,7 @@ var ( ) func init() { - helmInstallFunc = oam.InstallHelmChart + helmInstallFunc = helm.InstallHelmChart } // Setup vela dependency. @@ -66,7 +65,7 @@ func Install(kubecli client.Client) { return } for key, chart := range velaConfig.Data { - err := installHelmChart(kubecli, []byte(chart), log) + err := installHelmChart([]byte(chart), log) if err != nil { log.Error(err, fmt.Sprintf("failed to install helm chart for %s", key)) } @@ -103,7 +102,7 @@ func fetchVelaConfig(kubecli client.Client) (*v1.ConfigMap, error) { return velaConfig, nil } -func installHelmChart(client client.Client, chart []byte, log logr.Logger) error { +func installHelmChart(chart []byte, log logr.Logger) error { ioStreams := cmdutil.IOStreams{In: os.Stdin, Out: os.Stdout, ErrOut: os.Stderr} var helmChart types.Chart err := json.Unmarshal(chart, &helmChart) @@ -111,20 +110,6 @@ func installHelmChart(client client.Client, chart []byte, log logr.Logger) error return errors.Wrap(err, "failed to unmarshal the helm chart data") } log.Info("installing helm chart", "chart name", helmChart.Name) - // create the namespace - if helmChart.Namespace != types.DefaultAppNamespace { - if len(helmChart.Namespace) > 0 { - exist, err := cmdutil.DoesNamespaceExist(client, helmChart.Namespace) - if err != nil { - return err - } - if !exist { - if err = cmdutil.NewNamespace(client, helmChart.Namespace); err != nil { - return fmt.Errorf("create namespace (%s) failed for chart %s", helmChart.Namespace, helmChart.Name) - } - } - } - } if err = helmInstallFunc(ioStreams, helmChart); err != nil { return err } diff --git a/pkg/controller/dependency/install_test.go b/pkg/controller/dependency/install_test.go index cbe12cce0..5f70c84f6 100644 --- a/pkg/controller/dependency/install_test.go +++ b/pkg/controller/dependency/install_test.go @@ -5,12 +5,9 @@ import ( "testing" "github.com/pkg/errors" - v1 "k8s.io/api/core/v1" crdv1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" "k8s.io/apimachinery/pkg/runtime" clientgoscheme "k8s.io/client-go/kubernetes/scheme" - "sigs.k8s.io/controller-runtime/pkg/client/fake" "github.com/oam-dev/kubevela/api/types" "github.com/oam-dev/kubevela/api/v1alpha1" @@ -18,29 +15,8 @@ import ( ) var ( - scheme *runtime.Scheme - errHelm = fmt.Errorf("err") - velaConfigBase = v1.ConfigMap{ - ObjectMeta: metav1.ObjectMeta{ - Name: VelaConfigName, - Namespace: types.DefaultOAMNS, - Labels: map[string]string{"vela": "dependency"}, - }, - Data: map[string]string{ - "certificates.cert-manager.io": `{ - "repo": "jetstack", - "urL": "https://charts.jetstack.io", - "name": "cert-manager", - "version": "v1.0.0" - }`, - "prometheuses.monitoring.coreos.com": `{ - "repo": "jetstack", - "urL": "https://charts.jetstack.io", - "name": "cert-manager", - "version": "v1.0.0" - }`, - }, - } + scheme *runtime.Scheme + errHelm = fmt.Errorf("err") ) func init() { @@ -52,23 +28,14 @@ func init() { func TestSuccessfulInstall(t *testing.T) { helmInstallFunc = successHelmInstall - velaConfig := velaConfigBase.DeepCopy() - crd := crdv1.CustomResourceDefinition{ - ObjectMeta: metav1.ObjectMeta{ - Name: "certificates.cert-manager.io", - }, - } - client := fake.NewFakeClientWithScheme(scheme, velaConfig, &crd) - if err := installHelmChart(client, []byte("{}"), log); err != nil { + if err := installHelmChart([]byte("{}"), log); err != nil { t.Errorf("failed to install dependency error: %v", err) } } func TestFailedInstall(t *testing.T) { helmInstallFunc = failedHelmInstall - velaConfig := velaConfigBase.DeepCopy() - client := fake.NewFakeClientWithScheme(scheme, velaConfig) - if err := installHelmChart(client, []byte("{}"), log); errors.Cause(err) != errHelm { + if err := installHelmChart([]byte("{}"), log); errors.Cause(err) != errHelm { t.Errorf("failed to get install dependency error: %v", err) } } diff --git a/pkg/oam/capability.go b/pkg/oam/capability.go index 985686362..1819731ef 100644 --- a/pkg/oam/capability.go +++ b/pkg/oam/capability.go @@ -98,7 +98,7 @@ func InstallCapability(client client.Client, mapper discoverymapper.DiscoveryMap ioStreams.Info("Installing workload capability " + wd.Name) if tp.Install != nil { tp.Source.ChartName = tp.Install.Helm.Name - if err = InstallHelmChart(ioStreams, tp.Install.Helm); err != nil { + if err = helm.InstallHelmChart(ioStreams, tp.Install.Helm); err != nil { return err } } @@ -126,7 +126,7 @@ func InstallCapability(client client.Client, mapper discoverymapper.DiscoveryMap ioStreams.Info("Installing trait capability " + td.Name) if tp.Install != nil { tp.Source.ChartName = tp.Install.Helm.Name - if err = InstallHelmChart(ioStreams, tp.Install.Helm); err != nil { + if err = helm.InstallHelmChart(ioStreams, tp.Install.Helm); err != nil { return err } } @@ -167,10 +167,6 @@ func GetSyncedCapabilities(repoName, addonName string) (types.Capability, error) return types.Capability{}, fmt.Errorf("%s/%s not exist, try vela cap:center:sync %s to sync from remote", repoName, addonName, repoName) } -func InstallHelmChart(ioStreams cmdutil.IOStreams, c types.Chart) error { - return helm.Install(ioStreams, c.Repo, c.URL, c.Name, c.Version, c.Namespace, c.Name, c.Values) -} - func ListCapabilityCenters() ([]apis.CapabilityCenterMeta, error) { var capabilityCenterList []apis.CapabilityCenterMeta centers, err := plugins.LoadRepos() diff --git a/pkg/plugins/cluster.go b/pkg/plugins/cluster.go index 01b719a4c..714343187 100644 --- a/pkg/plugins/cluster.go +++ b/pkg/plugins/cluster.go @@ -5,8 +5,13 @@ import ( "fmt" "io/ioutil" "net/http" + "os" "path/filepath" + "github.com/oam-dev/kubevela/pkg/utils/helm" + + util2 "github.com/oam-dev/kubevela/pkg/commands/util" + "github.com/crossplane/oam-kubernetes-runtime/pkg/oam/util" "github.com/crossplane/oam-kubernetes-runtime/pkg/oam/discoverymapper" @@ -61,6 +66,13 @@ func GetWorkloadsFromCluster(ctx context.Context, namespace string, c types.Args templateErrors = append(templateErrors, errors.Wrapf(err, "handle workload template `%s` failed", wd.Name)) continue } + if tmp.Install != nil { + tmp.Source = &types.Source{ChartName: tmp.Install.Helm.Name} + ioStream := util2.IOStreams{In: os.Stdin, Out: os.Stdout, ErrOut: os.Stderr} + if err = helm.InstallHelmChart(ioStream, tmp.Install.Helm); err != nil { + return nil, nil, fmt.Errorf("unable to install helm chart dependency %s(%s from %s) for this workload '%s': %v ", tmp.Install.Helm.Name, tmp.Install.Helm.Version, tmp.Install.Helm.URL, wd.Name, err) + } + } gvk, err := util.GetGVKFromDefinition(dm, wd.Spec.Reference) if err != nil { return nil, nil, fmt.Errorf("make sure you have installed CRD(controller) for this capability '%s': %v ", wd.Name, err) @@ -97,6 +109,13 @@ func GetTraitsFromCluster(ctx context.Context, namespace string, c types.Args, s templateErrors = append(templateErrors, errors.Wrapf(err, "handle trait template `%s` failed\n", td.Name)) continue } + if tmp.Install != nil { + tmp.Source = &types.Source{ChartName: tmp.Install.Helm.Name} + ioStream := util2.IOStreams{In: os.Stdin, Out: os.Stdout, ErrOut: os.Stderr} + if err = helm.InstallHelmChart(ioStream, tmp.Install.Helm); err != nil { + return nil, nil, fmt.Errorf("unable to install helm chart dependency %s(%s from %s) for this trait '%s': %v ", tmp.Install.Helm.Name, tmp.Install.Helm.Version, tmp.Install.Helm.URL, td.Name, err) + } + } gvk, err := util.GetGVKFromDefinition(dm, td.Spec.Reference) if err != nil { return nil, nil, fmt.Errorf("make sure you have installed CRD(controller) for this capability '%s': %v ", td.Name, err) diff --git a/pkg/server/main/startAPIServer.go b/pkg/server/main/startAPIServer.go index e0227a784..074761b9a 100644 --- a/pkg/server/main/startAPIServer.go +++ b/pkg/server/main/startAPIServer.go @@ -8,7 +8,8 @@ import ( "syscall" "time" - "github.com/oam-dev/kubevela/pkg/oam" + "github.com/oam-dev/kubevela/pkg/utils/common" + "github.com/oam-dev/kubevela/pkg/server" "github.com/oam-dev/kubevela/pkg/server/util" @@ -27,7 +28,7 @@ func main() { o.DestWritter = w })) - c, err := oam.InitArgs() + c, err := common.InitBaseRestConfig() if err != nil { ctrl.Log.Error(err, "failed to init Kubernetes Config") os.Exit(1) diff --git a/pkg/server/util/middleware.go b/pkg/server/util/middleware.go index c586332f2..08da80658 100644 --- a/pkg/server/util/middleware.go +++ b/pkg/server/util/middleware.go @@ -7,7 +7,6 @@ import ( "github.com/gin-gonic/gin" uuid "github.com/satori/go.uuid" "go.uber.org/zap/zapcore" - "sigs.k8s.io/controller-runtime/pkg/client" ) // Header Keys @@ -130,10 +129,3 @@ func ValidateHeaders() gin.HandlerFunc { } } } - -func StoreClient(kubeClient client.Client) gin.HandlerFunc { - return func(c *gin.Context) { - c.Set("KubeClient", kubeClient) - c.Next() - } -} diff --git a/pkg/oam/common.go b/pkg/utils/common/common.go similarity index 92% rename from pkg/oam/common.go rename to pkg/utils/common/common.go index 0d28757f6..291b53c7d 100644 --- a/pkg/oam/common.go +++ b/pkg/utils/common/common.go @@ -1,4 +1,4 @@ -package oam +package common import ( "fmt" @@ -26,7 +26,7 @@ func init() { // +kubebuilder:scaffold:scheme } -func InitArgs() (types.Args, error) { +func InitBaseRestConfig() (types.Args, error) { restConf, err := config.GetConfig() if err != nil { fmt.Println("get kubeConfig err", err) diff --git a/pkg/utils/helm/helm.go b/pkg/utils/helm/helm.go index 4aeece643..79bb986f6 100644 --- a/pkg/utils/helm/helm.go +++ b/pkg/utils/helm/helm.go @@ -7,6 +7,8 @@ import ( "os" "strings" + "github.com/oam-dev/kubevela/pkg/utils/common" + "github.com/pkg/errors" "helm.sh/helm/v3/pkg/action" "helm.sh/helm/v3/pkg/chart" @@ -15,17 +17,41 @@ import ( "helm.sh/helm/v3/pkg/getter" "helm.sh/helm/v3/pkg/release" "helm.sh/helm/v3/pkg/repo" + "sigs.k8s.io/controller-runtime/pkg/client" "github.com/oam-dev/kubevela/api/types" cmdutil "github.com/oam-dev/kubevela/pkg/commands/util" ) +const VelaDebugLog = "VELA_DEBUG" + var ( settings = cli.New() ) func Install(ioStreams cmdutil.IOStreams, repoName, repoURL, chartName, version, namespace, releaseName string, vals map[string]interface{}) error { + + if len(namespace) > 0 { + args, err := common.InitBaseRestConfig() + if err != nil { + return err + } + kubeClient, err := client.New(args.Config, client.Options{Scheme: args.Schema}) + if err != nil { + return err + } + exist, err := cmdutil.DoesNamespaceExist(kubeClient, namespace) + if err != nil { + return err + } + if !exist { + if err = cmdutil.NewNamespace(kubeClient, namespace); err != nil { + return fmt.Errorf("create namespace (%s) failed for chart %s: %s", namespace, chartName, err) + } + } + } + if !IsHelmRepositoryExist(repoName, repoURL) { err := AddHelmRepository(repoName, repoURL, "", "", "", "", "", false, ioStreams.Out) @@ -195,7 +221,9 @@ func GetHelmRelease(ns string) ([]*release.Release, error) { } func GetChart(client *action.Install, name string) (*chart.Chart, error) { - settings.Debug = true + if os.Getenv(VelaDebugLog) != "" { + settings.Debug = true + } chartPath, err := client.ChartPathOptions.LocateChart(name, settings) if err != nil { @@ -208,3 +236,7 @@ func GetChart(client *action.Install, name string) (*chart.Chart, error) { } return chartRequested, nil } + +func InstallHelmChart(ioStreams cmdutil.IOStreams, c types.Chart) error { + return Install(ioStreams, c.Repo, c.URL, c.Name, c.Version, c.Namespace, c.Name, c.Values) +}