diff --git a/charts/vela-core/templates/defwithtemplate/manualscale.yaml b/charts/vela-core/templates/defwithtemplate/manualscale.yaml index 3fe214c37..03d4b5b74 100644 --- a/charts/vela-core/templates/defwithtemplate/manualscale.yaml +++ b/charts/vela-core/templates/defwithtemplate/manualscale.yaml @@ -13,7 +13,7 @@ spec: name: manualscalertraits.core.oam.dev workloadRefPath: spec.workloadRef template: | - output: { + outputs: scaler: { apiVersion: "core.oam.dev/v1alpha2" kind: "ManualScalerTrait" spec: { diff --git a/docs/en/_sidebar.md b/docs/en/_sidebar.md index 5c95e3656..038429d3e 100644 --- a/docs/en/_sidebar.md +++ b/docs/en/_sidebar.md @@ -5,6 +5,13 @@ - Platform Team Guide - [What is KubeVela?](/en/platform-engineers/overview.md) + - Designing Abstraction + - [Application CRD](/en/application.md) + - CUE + - [CUE Basic](/en/cue/basic.md) + - [Workload Type](/en/cue/workload-type.md) + - [Trait](/en/cue/trait.md) + - [Status](/en/cue/status.md) - Register Capability Modules - [Workload Type](/en/platform-engineers/workload-type.md) - [Trait](/en/platform-engineers/trait.md) diff --git a/docs/en/application.md b/docs/en/application.md new file mode 100644 index 000000000..d4b8c65dd --- /dev/null +++ b/docs/en/application.md @@ -0,0 +1,162 @@ +# Application CRD + +Application CRD describes the components and configurations of an application deployment. +It captures all of the definitions of an application in a single object and acts as a declarative anchor to +avoid configuration-drift. + +Application CRD provides an abstraction layer on top of infrastructure (e.g. Kubernetes) capabilities to +simplify APIs by hiding low level details. For instance, it enables developers to model the "web service" workload +without defining detailed `Deployment` + `Service` combo each time, or claim the auto-scaling requirements +without referring to the underlying [KEDA](https://keda.sh/) ScaleObject. + +## Spec of an Application + +The following is an example of an Application, it will create a backend service with workload type worker, +and a frontend service with workload type webservice. +The frontend service will have a sidecar and auto scale policy trait. + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: Application +metadata: + name: website +spec: + components: + - name: backend + type: worker + settings: + image: busybox + cmd: + - sleep + - '1000' + - name: frontend + type: webservice + settings: + image: nginx + traits: + - name: autoscaler + properties: + min: 1 + max: 10 + - name: sidecar + properties: + name: "sidecar-test" + image: "fluentd" +``` + +Let's explain more details of the Application spec with the example: + +1. An application named `website` is created, indicated by the `.metadata.name` field. +2. The application created two different components, indicated by the `.spec.components` field. +3. The name of the first component is `backend`, indicated by the `.spec.components[0].name` field. +4. The workload type of the first component is `worker`, indicated by the `.spec.components[0].type` field. +The workload type refer to a WorkloadDefinition object named `worker` which will contain the schema of the component. +5. The `settings` field of the first component is a schema free field, the schema depends on the workload type. In this +example, it has two fields `image` and `cmd` which is defined in the `parameter` of the `.spec.template` field of the `worker` like below. + ```yaml + apiVersion: core.oam.dev/v1alpha2 + kind: WorkloadDefinition + metadata: + name: worker + spec: + template: | + output: { + apiVersion: "apps/v1" + kind: "Deployment" + spec: { + selector: matchLabels: { + "app.oam.dev/component": context.name + } + template: { + metadata: labels: { + "app.oam.dev/component": context.name + } + spec: { + containers: [{ + name: context.name + image: parameter.image + + if parameter["cmd"] != _|_ { + command: parameter.cmd + } + }] + } + } + } + } + parameter: { + image: string + cmd?: [...string] + } + ``` +6. The second component named `frontend` has a type `webservice`, the schema of the `settings` field is defined by the `webservice` +workload type. +7. The second component has two traits, indicated by the `.spec.components[0].traits` field. +8. The first trait in the second component named `autoscaler`, indicated by the `.spec.components[0].traits[0].name` field. +The trait name refer to a TraitDefinition object named `autoscaler` which will define the trait schema. +9. The `properties` field in the first trait in the second component is also a schema free field, its schema defends on the trait. +10. The second trait in the second component named `sidecar` follows the same rule, refer to a TraitDefinition named `scaler` which will +define the schema of the `properties` field. + + +## Application Object and K8s Resources + +As you can see in the above example, the relationships of all these concepts are: + +* An application object composed by multiple components. +* A component in the application composed by name, workload and multiple traits. +* A workload composed by type and settings. The type refer to a WorkloadDefinition object which is the template of the workload type. +The settings are parameter values which can fill with the template and generate the K8s resource. +* A trait composed by name and properties. The name refer to a TraitDefinition object which is the template of the trait. +The properties are parameter values which can fill with the template and generate the auxiliary K8s resources. + +Beside the relationships, there are some standard contracts between application object and K8s resourcs. + +* The K8s resource generated by the workload will be tagged the following KubeVela system labels: + - `workload.oam.dev/type=` represents the name of the WorkloadDefinition object referred. + In the example above, it is `worker` to the first component. + - `app.oam.dev/name=` represents the name of the application. In the example above, it is `website`. + - `app.oam.dev/component=` represents the name of the component in the application. In the example above, + it is `backend` to the first component. + - `trait.oam.dev/resource=` represents the auxiliary K8s object resource name. + In the example above, we don't have this kind of resource, so this label could be empty. + It's an advanced usage for composed K8s resources in a workload type. + - revision will also be added, this part is still under development with revision mechanism. + +* The K8s resource generated by the trait will be tagged the following KubeVela system labels: + - `trait.oam.dev/type=` represents the name of the TraitDefinition object referred. + In the example above, it is `autoscaler` to the first trait in the second component. + - `app.oam.dev/name=` represents the name of the application. In the example above, it is `website`. + - `app.oam.dev/component=` represents the name of the component in the application. In the example above, + it is `backend` to the first component. + - `trait.oam.dev/resource=` represents the auxiliary K8s object resource name. + Look at the example below, the `scaler` in the template inside the outputs is the resource name of this trait. + ```yaml + apiVersion: core.oam.dev/v1alpha2 + kind: TraitDefinition + metadata: + name: scaler + spec: + appliesToWorkloads: + - webservice + - worker + definitionRef: + name: manualscalertraits.core.oam.dev + workloadRefPath: spec.workloadRef + template: | + outputs: scaler: { + apiVersion: "core.oam.dev/v1alpha2" + kind: "ManualScalerTrait" + spec: { + replicaCount: parameter.replicas + } + } + parameter: { + //+short=r + //+usage=Replicas of the workload + replicas: *1 | int + } + ``` + - revision will also be added, this part is still under development with revision mechanism. + +We will introduce more about the CUE template in the `CUE` section. \ No newline at end of file diff --git a/docs/en/concepts.md b/docs/en/concepts.md index f9a9ccfa7..bdc0661e6 100644 --- a/docs/en/concepts.md +++ b/docs/en/concepts.md @@ -37,15 +37,15 @@ spec: type: webservice settings: image: nginx - traits: - - name: autoscaler - properties: - min: 1 - max: 10 - - name: sidecar - properties: - name: "sidecar-test" - image: "fluentd" + traits: + - name: autoscaler + properties: + min: 1 + max: 10 + - name: sidecar + properties: + name: "sidecar-test" + image: "fluentd" ``` ### Workload Type @@ -74,42 +74,39 @@ metadata: spec: definitionRef: name: deployments.apps - extension: - template: | - output: { - apiVersion: "apps/v1" - kind: "Deployment" - spec: { - selector: matchLabels: { - "app.oam.dev/component": context.name - } + template: | + output: { + apiVersion: "apps/v1" + kind: "Deployment" + spec: { + selector: matchLabels: { + "app.oam.dev/component": context.name + } + template: { + metadata: labels: { + "app.oam.dev/component": context.name + } + spec: { + containers: [{ + name: context.name + image: parameter.image - template: { - metadata: labels: { - "app.oam.dev/component": context.name - } + if parameter["cmd"] != _|_ { + command: parameter.cmd + } + }] + } + } + } + } - spec: { - containers: [{ - name: context.name - image: parameter.image + parameter: { + // +usage=Which image would you like to use for your service + // +short=i + image: string - if parameter["cmd"] != _|_ { - command: parameter.cmd - } - }] - } - } - } - } - - parameter: { - // +usage=Which image would you like to use for your service - // +short=i - image: string - - cmd?: [...string] - } + cmd?: [...string] + } ``` Once this definition is applied to the cluster, the end users will be able to claim a component with workload type of `worker`, and fill in the properties as below: diff --git a/docs/en/cue/basic.md b/docs/en/cue/basic.md new file mode 100644 index 000000000..137a86f41 --- /dev/null +++ b/docs/en/cue/basic.md @@ -0,0 +1,401 @@ +# CUE Basic + +KubeVela use [CUE](https://cuelang.org/) as its template DSL. With the help of CUE, we can define simple but powerful +template in Definition Objects and build abstraction for applications. + +## Why CUE ? + +Why does KubeVela choose CUE? [Points of Cedric Charly](https://blog.cedriccharly.com/post/20191109-the-configuration-complexity-curse/) speaks well, +let me conclude here. + +* **CUE is designed for large scale configuration.** + CUE deliberately opts for the graph unification model used in computational linguistics instead of the traditional + inheritance model. The graph model can help KubeVela to build a clear view of resources' relationships and dependency. + For large scale infrastructure, that will be a complex, tightly interconnected graph of + resources that describes an organization's entire computing environment. In this case, CUE has the ability to understand a + configuration worked on by engineers across a whole company and to safely change a value that modifies thousands of + objects in a configuration. +* **CUE supports first-class code generation and automation.** + A design goal of CUE is to have code that is straightfoward for humans to write, but is also simple for machines to + generate. This goal highly consistent with KubeVela which wants to offer abstractions to bridge the gap between concepts + used by app developers and kubernetes. CUE can integrate with existing tools and workflows naturally while other tools + would have to build complex custom solutions. CUE can generate Kubernetes definitions from Go code and OpenAPI schemas + and immediately work with resources directly or build higher level libraries. +* **CUE integrates very well with Go.** + KubeVela is built with GO just like most projects of the while Kubernetes system. CUE is also implemented in and + exposes a rich client API in Go. KubeVela integrates with CUE as its core library and works as a Kubernetes CRD controller. + With the help of CUE, KubeVela can easily handle data constraint problems. + +If you want to go deeper I recommend reading [The Logic of CUE](https://cuelang.org/docs/concepts/logic/) +to understand the theoretical foundation and what makes CUE different from other configuration languages. + +## CUE in KubeVela + +Let's go back to discuss how does CUE be used in KubeVela. As you know, KubeVela helps platform builder to build abstraction +from Kubernetes resources to an application. We will use a [K8s Deployment](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/) +as example to explain how was it implemented. + +In KubeVela we usually build a WorkloadDefinition to generate K8s Deployment as it's workload-like resources. +A complete WorkloadDefinition example like below: + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: WorkloadDefinition +metadata: + name: mydeploy +spec: + definitionRef: + name: deployments.apps + template: | + parameter: { + name: string + image: string + } + output: { + apiVersion: "apps/v1" + kind: "Deployment" + spec: { + selector: matchLabels: { + "app.oam.dev/component": parameter.name + } + template: { + metadata: labels: { + "app.oam.dev/component": parameter.name + } + spec: { + containers: [{ + name: parameter.name + image: parameter.image + }] + }}} + } +``` + +In this example, the `template` field is totally CUE, it contains two keywords, `output` and `parameter`. +The `output` defines what will be rendering out by the template. The `parameter` defines the input parameters which can +be part of the application. + +Let's try to write the CUE template step by step. + +As you see, below is a Deployment YAML, most of the fields can be hidden and only expose `image` and `env` field for end user. + +```yaml +apiVersion: apps/v1 +kind: Deployment +meadata: + name: mytest +spec: + template: + spec: + containers: + - name: mytest + env: + - name: a + value: b + image: nginx:v1 + metadata: + labels: + app.oam.dev/component: mytest + selector: + matchLabels: + app.oam.dev/component: mytest +``` + +The first step is to convert the YAML to JSON and put the whole json object into the `output` keyword field. +CUE is a superset of JSON: any valid JSON file is a valid CUE file. It provides some conveniences such as you can omit +some quotes from field names without special characters. + +Here is the converted result: + +```cue +output: { + apiVersion: "apps/v1" + kind: "Deployment" + metadata: name: "mytest" + spec: { + selector: matchLabels: { + "app.oam.dev/component": "mytest" + } + template: { + metadata: labels: { + "app.oam.dev/component": "mytest" + } + spec: { + containers: [{ + name: "mytest" + image: "nginx:v1" + env: [{name:"a",value:"b"}] + }] + }}} +} +``` + +Here are all conveniences add by CUE as a superset of JSON: + +* C-style comments, +* quotes may be omitted from field names without special characters, +* commas at the end of fields are optional, +* comma after last element in list is allowed, +* outer curly braces are optional. + +After that we add `parameter` keyword into the template, and use it as a variable reference, this is basic CUE grammar. + +Fields of keyword `parameter` will be detected by KubeVela and be exposed to users using in application. + +```cue +parameter: { + name: string + image: string +} +output: { + apiVersion: "apps/v1" + kind: "Deployment" + spec: { + selector: matchLabels: { + "app.oam.dev/component": parameter.name + } + template: { + metadata: labels: { + "app.oam.dev/component": parameter.name + } + spec: { + containers: [{ + name: parameter.name + image: parameter.image + }] + }}} +} +``` + +Finally, you can put the whole CUE template into the `template` field of WorkloadDefinition object. That's all you need +to know to create a basic KubeVela capability. + +## More Advanced Usage of CUE Grammar in Definition + +In this section, we will introduce some more advanced CUE grammar to use in KubeVela. + +### Structural parameter + +If you have some complex type of parameters in your template, and want to define a struct or embed struct as parameters, +then you could use structural parameter. + +1. Define a struct type in, it includes a struct, a string and an integer. + ``` + #Config: { + name: string + value: int + other: { + key: string + value: string + } + } + ``` + +2. Use the struct defined in the `parameter` keyword, and use it as an array list. + ``` + parameter: { + name: string + image: string + configSingle: #Config + config: [...#Config] + } + ``` + +3. In `output` keyword, it's referenced the same way with other normal field s. + ``` + output: { + ... + spec: { + containers: [{ + name: parameter.name + image: parameter.image + env: parameter.config + }] + } + ... + } + ``` + +4. The structural field `config` can be easily used in Application like below: + ``` + apiVersion: core.oam.dev/v1alpha2 + kind: Application + metadata: + name: website + spec: + components: + - name: backend + type: mydeploy + settings: + image: crccheck/hello-world + name: mysvc + config: + - name: a + value: 1 + other: + key: mykey + value: myvalue + ``` + +### Conditional Parameter + +Conditional parameter can be used to decide template condition logic. +Below is an example that when `useENV=true`, it will render env section, otherwise, it will not. + +``` +parameter: { + name: string + image: string + useENV: bool +} +output: { + ... + spec: { + containers: [{ + name: parameter.name + image: parameter.image + if parameter.useENV == true { + env: [{name: "my-env", value: "my-value"}] + } + }] + } + ... +} +``` + +### Optional Parameter and Default Value + +Optional parameter can be optional, that usually works with conditional logic. If some field does not exit, the CUE +grammar is `if _variable_ != _|_`, the example is like below: + +``` +parameter: { + name: string + image: string + config?: [...#Config] +} +output: { + ... + spec: { + containers: [{ + name: parameter.name + image: parameter.image + if parameter.config != _|_ { + config: parameter.config + } + }] + } + ... +} +``` + +Default Value is marked with a `*` prefix. It's used like + +``` +parameter: { + name: string + image: *"nginx:v1" | string + port: *80 | int + number: *123.4 | float +} +output: { + ... + spec: { + containers: [{ + name: parameter.name + image: parameter.image + }] + } + ... +} +``` + +So if a parameter field is neither a parameter with default value nor a conditional field, it's a required value. + +### Loop + +#### Loop for map type + +```cue +parameter: { + name: string + image: string + env: [string]: string +} +output: { + spec: { + containers: [{ + name: parameter.name + image: parameter.image + env: [ + for k, v in parameter.env { + name: k + value: v + }, + ] + }] + } +} +``` + +#### Loop for slice + +```cue +parameter: { + name: string + image: string + env: [...{name:string,value:string}] +} +output: { + ... + spec: { + containers: [{ + name: parameter.name + image: parameter.image + env: [ + for _, v in parameter.env { + name: v.name + value: v.value + }, + ] + }] + } +} +``` + +### Import CUE internal packages + +CUE has [lots of internal packages](https://pkg.go.dev/cuelang.org/go@v0.2.2/pkg) which also can be used in KubeVela. + +Below is an example that use `strings.Join` to concat string list to one string. + +```cue +import ("strings") + +parameter: { + outputs: [{ip: "1.1.1.1", hostname: "xxx.com"}, {ip: "2.2.2.2", hostname: "yyy.com"}] +} +output: { + spec: { + if len(parameter.outputs) > 0 { + _x: [ for _, v in parameter.outputs { + "\(v.ip) \(v.hostname)" + }] + message: "Visiting URL: " + strings.Join(_x, "") + } + } +} +``` + + + + + + + + + + + + + diff --git a/docs/en/cue/status.md b/docs/en/cue/status.md new file mode 100644 index 000000000..f0dd60822 --- /dev/null +++ b/docs/en/cue/status.md @@ -0,0 +1,139 @@ +# Status Loop Back + +In the previous sections, we use CUE as template to render K8s resources. After an application deployed, the status +loop back can also use CUE. + +KubeVela use CUE to define health check and custom status message for an application in workload type and trait. + +## Health Check + +The spec of health check is `spec.status.healthPolicy`, they are the same for both Workload Type and Trait. + +If not defined, the health result will always be `true`. + +The keyword in CUE is `isHealth`, the result of CUE expression must be `bool` type. +Application CRD controller will evaluate the CUE expression periodically until it +becomes healthy. Every time the controller will get all the K8s resources and fill them into the context field. + +So the context will contain following information: + +```cue +context:{ + name: + appName: + output: + outputs: { + : + : + } +} +``` + +Trait will not have the `context.ouput`, other fields are the same. + +The example of health check likes below: + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: WorkloadDefinition +spec: + status: + healthPolicy: | + isHealth: (context.output.status.readyReplicas > 0) && (context.output.status.readyReplicas == context.output.status.replicas) + ... +``` + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: TraitDefinition +spec: + status: + healthPolicy: | + isHealth: len(context.outputs.service.spec.clusterIP) > 0 + ... +``` + +Refer to [this doc](https://github.com/oam-dev/kubevela/blob/master/config/samples/app-with-status/template.yaml) for the complete example. + +The health check result will be recorded into the Application CRD resource. + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: Application +spec: + components: + - name: myweb + settings: + cmd: + - sleep + - "1000" + enemies: alien + image: busybox + lives: "3" + traits: + - name: ingress + properties: + domain: www.example.com + http: + /: 80 + type: worker +status: + ... + services: + - healthy: true + message: "type: busybox,\t enemies:alien" + name: myweb + traits: + - healthy: true + message: 'Visiting URL: www.example.com, IP: 47.111.233.220' + type: ingress + status: running +``` + +## Custom Status + +The spec of custom status is `spec.status.customStatus`, they are the same for both Workload Type and Trait. + +The keyword in CUE is `message`, the result of CUE expression must be `string` type. + +The custom status has the same mechanism with health check. +Application CRD controller will evaluate the CUE expression after the health check succeed. + +The context will contain following information: + +```cue +context:{ + name: + appName: + output: + outputs: { + : + : + } +} +``` + +Trait will not have the `context.ouput`, other fields are the same. + + +Refer to [this doc](https://github.com/oam-dev/kubevela/blob/master/config/samples/app-with-status/template.yaml) for the complete example. + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: WorkloadDefinition +spec: + status: + customStatus: |- + message: "type: " + context.output.spec.template.spec.containers[0].image + ",\t enemies:" + context.outputs.gameconfig.data.enemies + ... +``` + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: TraitDefinition +spec: + status: + customStatus: |- + message: "type: "+ context.outputs.service.spec.type +",\t clusterIP:"+ context.outputs.service.spec.clusterIP+",\t ports:"+ "\(context.outputs.service.spec.ports[0].port)"+",\t domain"+context.outputs.ingress.spec.rules[0].host + ... +``` diff --git a/docs/en/platform-engineers/advanced-cue.md b/docs/en/cue/trait.md similarity index 80% rename from docs/en/platform-engineers/advanced-cue.md rename to docs/en/cue/trait.md index 61c56713c..861be4687 100644 --- a/docs/en/platform-engineers/advanced-cue.md +++ b/docs/en/cue/trait.md @@ -1,17 +1,17 @@ # Using CUE to Extend Trait in advanced way -> WARNINIG: you are now reading a platform builder/administrator oriented documentation. +In this section we will introduce how to define a Trait with CUE template. -In the following tutorial, you will learn how to add a trait in a more advanced way without writing any CRD controller. -In general, the advanced way can help you build abstraction by composition or decomposition. - -## Trait generate multiple resources - -With the help of CUE template, we can combine multiple K8s resources into one trait. +## Basic Usage -You can use the keyword `outputs` to create multiple K8s objects. The format MUST be `outputs::`. +Defining a Trait with CUE template is a little different from Workload Type. +The different part is a trait must use `outputs` keyword instead of `output` to define resources that will be generated. -Let's look at an example, assume you hope to make a combo for K8s service and ingress, naming it as `ingress`. +With the help of CUE template, it is very nature to compose multiple K8s resources in one trait. + +The format MUST be `outputs::`. + +Below is an example that make a combo for K8s service and ingress, naming this trait as `ingress`. ```yaml apiVersion: core.oam.dev/v1alpha2 @@ -66,13 +66,7 @@ spec: } ``` -Apply this newly defined TraitDefinition into our system: - -```shell script -kubectl apply -f https://raw.githubusercontent.com/oam-dev/kubevela/master/docs/examples/registry/ingress.yaml -``` - -You can check it by using the application object like below: +It can be used in the application object like below: ```yaml apiVersion: core.oam.dev/v1alpha2 @@ -82,8 +76,6 @@ metadata: spec: components: - name: express-server - scopes: - healthscopes.core.oam.dev: testapp-default-health settings: cmd: - node @@ -99,13 +91,8 @@ spec: type: webservice ``` -Apply it: - -```shell script -kubectl apply -f https://raw.githubusercontent.com/oam-dev/kubevela/master/docs/examples/advanced-cue/app1.yaml -``` - -Then you will see the deployment behind webservice along with the K8s service and ingress behind the ingress trait created. +After the application deployed, you will see the deployment(belong to webservice) along with the K8s service and +ingress(belong to ingress trait) created. ### Generate multiple resources by using for loop @@ -145,14 +132,7 @@ spec: } ``` - -Apply this newly defined TraitDefinition into our system: - -```shell script -kubectl apply -f https://raw.githubusercontent.com/oam-dev/kubevela/master/docs/examples/registry/for-loop.yaml -``` - -Use the newly created trait like below: +The usage of this trait could be: ```yaml apiVersion: core.oam.dev/v1alpha2 @@ -164,11 +144,7 @@ spec: - name: express-server type: webservice settings: - cmd: - - node - - server.js - image: oamdev/testapp:v1 - port: 8080 + ... traits: - name: expose properties: @@ -177,25 +153,15 @@ spec: myservice2: 8081 ``` -Apply it: - -```shell script -kubectl apply -f https://raw.githubusercontent.com/oam-dev/kubevela/master/docs/examples/advanced-cue/app2.yaml -``` - -Then you will see the deployment behind webservice along with two K8s services created. - - ## Patch Trait -For the purpose of separate of concerns, we usually won't do decomposition some fields out as trait from the underlying workload. +For the purpose of separate of concerns, we usually want to decompose some fields out as trait from the underlying workload. -For example the [webservice workload] is implemented by K8s Deployment, but the workload doesn't care about the `replicas` field. -In this case, you can write a [ManualScalerTrait](https://github.com/oam-dev/kubevela/tree/master/pkg/controller/core.oam.dev/v1alpha2/core/traits/manualscalertrait) -CRD controller to control the `replicas` field after the deployment created. - -But now, you are more encouraged to use patch trait in KubeVela. With the help of patch trait, you don't need to write CRD -controller for this case anymore. +For example the [webservice workload] is implemented by K8s Deployment, but the workload doesn't care about the node affinity related fields. + +Users should be able to run this app without node affinity and they can add it as a trait later when they want. +This is exactly what patch trait do. +With the help of CUE template, you don't need to write a CRD controller for this case. The keyword is `patch`, object describe after the keyword will be patched into the workload. @@ -206,8 +172,8 @@ apiVersion: core.oam.dev/v1alpha2 kind: TraitDefinition metadata: annotations: - definition.oam.dev/description: "Manually scale the app" - name: scaler + definition.oam.dev/description: "affinity specify node affinity and toleration" + name: node-affinity spec: appliesToWorkloads: - webservice @@ -215,33 +181,82 @@ spec: extension: template: |- patch: { - spec: replicas: parameter.replicas + spec: template: spec: { + if parameter.affinity != _|_ { + affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: [{ + matchExpressions: [ + for k, v in parameter.affinity { + key: k + operator: "In" + values: v + }, + ]}] + } + if parameter.tolerations != _|_ { + tolerations: [ + for k, v in parameter.tolerations { + effect: "NoSchedule" + key: k + operator: "Equal" + value: v + }] + } + } } + parameter: { - replicas: *1 | int + affinity?: [string]: [...string] + tolerations?: [string]: string } ``` -The patch trait rely on the workload object will always match the structure `spec.replicas` and the type of the field. -So we usually use it with the field `appliesToWorkloads` which can limit the trait can only be used by these specified workloads. +You can use it like: + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: Application +metadata: + name: testapp +spec: + components: + - name: express-server + type: webservice + settings: + image: oamdev/testapp:v1 + traits: + - name: "node-affinity" + properties: + affinity: + server-owner: ["owner1","owner2"] + resource-pool: ["pool1","pool2","pool3"] + tolerations: + resource-pool: "broken-pool1" + server-owner: "old-owner" +``` + +The patch trait rely on the workload object will always match the structure `spec.template.spec.affinity` and +the type of these fields. +So we usually use it with the field `appliesToWorkloads` which can limit the trait only to be used +by these specified workloads. By default, the patch implemented in KubeVela relies on the CUE merge operation. It has these constraints: * New field will be added only when the schema doesn't conflict with each other, and the value not finalized. -For example, if the workload already define the `spec.replicas` is `5`, then the patch trait replicas value `1` will fail to patch. +For example, if a field already has a final value `replicas=5`, then the patch trait will conflict when patches `replicas=1`. +It only works when `replica` is not finalized before patch. * Array list in the patch will be merged into the workload by the order of index, you need to use strategy patch. ### Strategy Patch Trait -`strategy patch` is a special patch logic for patching array list supported in KubeVela, it's not native CUElang feature, -so you need to write annotation for using it. +`strategy patch` is a special patch logic for patching array list supported **only** in KubeVela, +it's not native CUE feature, so you need to write annotation for using it. -The annotation keyword is `//+patchKey=`. +The annotation grammar is `//+patchKey=`. -By adding this annotation, merging logic of two array list will not follow the CUE rule, instead of that, it will +By adding this annotation, merging logic of two lists will not follow the CUE rule, instead of that, it will regard the element type of the array list will always be object, and compare the object field with the specified key name. If the value of the key name equal, then the patch data will merge into that, if no equal found, the patch will append into the array list. @@ -277,7 +292,7 @@ with same name, it will be a sidecar container append into the `spec.template.sp ### Patch works with output -Patch can also work with output, if patch and output both exist in one trait, the patch part will execute first and then +Patch can also work with outputs, if patch and outputs both exist in one trait, the patch part will execute first and then the output object will be rendered out. ```yaml @@ -296,7 +311,7 @@ spec: extension: template: |- patch: {spec: template: metadata: labels: app: context.name} - output: { + outputs: service: { apiVersion: "v1" kind: "Service" metadata: name: context.name @@ -624,74 +639,3 @@ spec: appMountPath: "/usr/share/nginx/html" initMountPath: "/work-dir" ``` - -### Node affinity and anti-affinity - -Node affinity and anti-affinity is also common trait: - -```yaml -apiVersion: core.oam.dev/v1alpha2 -kind: TraitDefinition -metadata: - annotations: - definition.oam.dev/description: "affinity specify node affinity and toleration" - name: node-affinity -spec: - appliesToWorkloads: - - webservice - - worker - extension: - template: |- - patch: { - spec: template: spec: { - if parameter.affinity != _|_ { - affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: [{ - matchExpressions: [ - for k, v in parameter.affinity { - key: k - operator: "In" - values: v - }, - ]}] - } - if parameter.tolerations != _|_ { - tolerations: [ - for k, v in parameter.tolerations { - effect: "NoSchedule" - key: k - operator: "Equal" - value: v - }] - } - } - } - - parameter: { - affinity?: [string]: [...string] - tolerations?: [string]: string - } -``` - -You can use it like: - -```yaml -apiVersion: core.oam.dev/v1alpha2 -kind: Application -metadata: - name: testapp -spec: - components: - - name: express-server - type: webservice - settings: - image: oamdev/testapp:v1 - traits: - - name: "node-affinity" - properties: - affinity: - server-owner: ["owner1","owner2"] - resource-pool: ["pool1","pool2","pool3"] - tolerations: - resource-pool: "broken-pool1" - server-owner: "old-owner" -``` \ No newline at end of file diff --git a/docs/en/cue/workload-type.md b/docs/en/cue/workload-type.md new file mode 100644 index 000000000..4548cddf0 --- /dev/null +++ b/docs/en/cue/workload-type.md @@ -0,0 +1,227 @@ +# Workload Type with CUE + +In the [CUE basic section](./basic.md), we have explained how CUE works as template of Workload Type and Trait. +In this section, we will introduce more details about workload type. + +## Basic Usage + +The very basic usage of CUE in workload is to extend a K8s Resource as a workload type(WorkloadDefinition). + +A K8s Deployment as a workload: + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: WorkloadDefinition +metadata: + name: worker +spec: + definitionRef: + name: deployments.apps + template: | + parameter: { + name: string + image: string + } + output: { + apiVersion: "apps/v1" + kind: "Deployment" + spec: { + selector: matchLabels: { + "app.oam.dev/component": parameter.name + } + template: { + metadata: labels: { + "app.oam.dev/component": parameter.name + } + spec: { + containers: [{ + name: parameter.name + image: parameter.image + }] + }}} + } +``` + +A K8s Job as workload: + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: WorkloadDefinition +metadata: + name: task + annotations: + definition.oam.dev/description: "Describes jobs that run code or a script to completion." +spec: + definitionRef: + name: jobs.batch + template: | + output: { + apiVersion: "batch/v1" + kind: "Job" + spec: { + parallelism: parameter.count + completions: parameter.count + template: spec: { + restartPolicy: parameter.restart + containers: [{ + image: parameter.image + if parameter["cmd"] != _|_ { + command: parameter.cmd + } + }] + } + } + } + parameter: { + count: *1 | int + image: string + restart: *"Never" | string + cmd?: [...string] + } +``` + +Other resources are the same, you can define all K8s resources include CRD in this way. + +## Context + +When you want to reference the runtime instance name for an app, you can use the `conext` keyword instead of define a new parameter. + +KubeVela will provide a `context` struct including app name(`context.appName`) and component name(`context.name`). + +```cue +context: { + appName: string + name: string +} +``` + +Values of the context will be injected automatically when an application is deploying. +So you can reference the context variable to use this information. + +```yaml +parameter: { + image: string +} +output: { + ... + spec: { + containers: [{ + name: context.name + image: parameter.image + }] + } + ... +} +``` + +## Composition + +A workload type can contain multiple K8s resources, for example, a webserver workload type may be composed by +K8s Deployment and Service. + +The main workload resource MUST be defined in keyword `output` while the auxiliary workload resources MUST be defined +in keyword `outputs` with a resource name inside. + +The format MUST be `outputs::`. + +In the underlying OAM model, the `output` resource will become the `workload` object while the `outputs` resources will +become traits. + +Below is the example: + +```yaml +apiVersion: core.oam.dev/v1alpha2 +kind: WorkloadDefinition +metadata: + name: webserver + annotations: + definition.oam.dev/description: "webserver was composed by deployment and service" +spec: + definitionRef: + name: deployments.apps + template: | + output: { + apiVersion: "apps/v1" + kind: "Deployment" + spec: { + selector: matchLabels: { + "app.oam.dev/component": context.name + } + template: { + metadata: labels: { + "app.oam.dev/component": context.name + } + spec: { + containers: [{ + name: context.name + image: parameter.image + + if parameter["cmd"] != _|_ { + command: parameter.cmd + } + + if parameter["env"] != _|_ { + env: parameter.env + } + + if context["config"] != _|_ { + env: context.config + } + + ports: [{ + containerPort: parameter.port + }] + + if parameter["cpu"] != _|_ { + resources: { + limits: + cpu: parameter.cpu + requests: + cpu: parameter.cpu + } + } + }] + } + } + } + } + // workload can have extra object composition by using 'outputs' keyword + outputs: service: { + apiVersion: "v1" + kind: "Service" + spec: { + selector: { + "app.oam.dev/component": context.name + } + ports: [ + { + port: parameter.port + targetPort: parameter.port + }, + ] + } + } + parameter: { + image: string + cmd?: [...string] + port: *80 | int + env?: [...{ + name: string + value?: string + valueFrom?: { + secretKeyRef: { + name: string + key: string + } + } + }] + cpu?: string + } +``` + +The main workload inside the `output` keyword is a K8s Deployment. The auxiliary resources inside the `outputs` field +is a K8s service, the resource name in the CUE template is `service` after the `outputs` keyword. + +The resource name will also be labeled on the K8s resource when it is deployed. In this example, the K8s Service will have +a label(`trait.oam.dev/resource=service`). + diff --git a/docs/examples/registry/autoscale.yaml b/docs/examples/registry/autoscale.yaml index d15cde6c0..671abbe10 100644 --- a/docs/examples/registry/autoscale.yaml +++ b/docs/examples/registry/autoscale.yaml @@ -22,7 +22,7 @@ spec: template: | import "strconv" - output: { + outputs: autoscaler: { apiVersion: "standard.oam.dev/v1alpha1" kind: "Autoscaler" spec: { diff --git a/docs/examples/registry/metrics.yaml b/docs/examples/registry/metrics.yaml index b1c2107f7..8110b1022 100644 --- a/docs/examples/registry/metrics.yaml +++ b/docs/examples/registry/metrics.yaml @@ -22,7 +22,7 @@ spec: url: https://prometheus-community.github.io/helm-charts version: 9.4.4 template: |- - output: { + outputs: metrics: { apiVersion: "standard.oam.dev/v1alpha1" kind: "MetricsTrait" spec: { diff --git a/docs/examples/registry/patch-scaler.yaml b/docs/examples/registry/patch-scaler.yaml new file mode 100644 index 000000000..e7775e00e --- /dev/null +++ b/docs/examples/registry/patch-scaler.yaml @@ -0,0 +1,18 @@ +apiVersion: core.oam.dev/v1alpha2 +kind: TraitDefinition +metadata: + annotations: + definition.oam.dev/description: "Manually scale the app" + name: patch-scaler +spec: + appliesToWorkloads: + - webservice + - worker + extension: + template: |- + patch: { + spec: replicas: parameter.replicas + } + parameter: { + replicas: *1 | int + } \ No newline at end of file diff --git a/docs/examples/registry/rollout.yaml b/docs/examples/registry/rollout.yaml index 8b273e04a..301d34e70 100644 --- a/docs/examples/registry/rollout.yaml +++ b/docs/examples/registry/rollout.yaml @@ -21,7 +21,7 @@ spec: url: https://oam.dev/flagger/archives/ version: 1.1.0 template: |- - output: { + outputs: canary: { apiVersion: "flagger.app/v1beta1" kind: "Canary" spec: { diff --git a/docs/examples/registry/route.yaml b/docs/examples/registry/route.yaml index cba56f0ba..03579bd39 100644 --- a/docs/examples/registry/route.yaml +++ b/docs/examples/registry/route.yaml @@ -19,7 +19,7 @@ spec: url: https://kubernetes-charts.storage.googleapis.com/ version: 1.41.2 template: | - output: { + outputs: route: { apiVersion: "standard.oam.dev/v1alpha1" kind: "Route" spec: { diff --git a/docs/examples/advanced-cue/webserver.yaml b/docs/examples/registry/webserver.yaml similarity index 100% rename from docs/examples/advanced-cue/webserver.yaml rename to docs/examples/registry/webserver.yaml diff --git a/hack/vela-templates/cue/manualscale.cue b/hack/vela-templates/cue/manualscale.cue index 5e0fd5c9a..9b6f3ca33 100644 --- a/hack/vela-templates/cue/manualscale.cue +++ b/hack/vela-templates/cue/manualscale.cue @@ -1,4 +1,4 @@ -output: { +outputs: scaler: { apiVersion: "core.oam.dev/v1alpha2" kind: "ManualScalerTrait" spec: {