github/kubevela
1
0
Fork 0
mirror of https://github.com/kubevela/kubevela.git synced 2026-02-14 18:10:21 +00:00
Code Issues Packages Projects Releases Wiki Activity

Support cloud resource provisiong and consuming (#1264)

Browse Source 
* Support cloud resource provisioning and consuming (Crossplane)

Provided a way to store secret for cloud resource generated by
Crossplane and to consume the secret

Refer to #1128

* Separate cloud resource producer and consumer in two applications

* add unit test to check whether application can consume cloud resource secret

* update Cloud Resource doc

* Provisioning and consuming cloud resource in different applications v1 (one cloud resource)

* one component consumes two cloud resources

Co-authored-by: Jianbo Sun <wonderflow.sun@gmail.com>
This commit is contained in:
Zheng Xi Zhou
2021-03-29 17:20:33 +08:00
committed by GitHub
parent 14635b6f2d
commit 9e40b77b60
18 changed files with 1089 additions and 118 deletions
Download Patch File Download Diff File Expand all files Collapse all files

521
docs/en/platform-engineers/cloud-services.md
View File

@@ -1,5 +1,5 @@
---
title: Defining Cloud Database as Component
title: Define and Consume Cloud Resource
---
KubeVela provides unified abstraction even for cloud services.
@@ -11,104 +11,417 @@ The following practice could be considered:
- you want to allow your end users explicitly claim a "instance" of the cloud service and consume it, and release the "instance" when deleting the application.
- Use `TraitDefinition` if:
- you don't want to give your end users any control/workflow of claiming or releasing the cloud service, you only want to give them a way to consume a cloud service which could even be managed by some other system. A `Service Binding` trait is widely used in this case.
In this documentation, we will define an Alibaba Cloud's RDS (Relational Database Service), and an Alibaba Cloud's OSS (Object Storage System) as example. This mechanism works the same with other cloud providers.
In a single application, they are in form of Traits, and in multiple applications, they are in form of Components.
In this documentation, we will add a Alibaba Cloud's RDS (Relational Database Service) as a component.
## Install and Configure Crossplane
## Step 1: Install and Configure Crossplane
KubeVela uses [Crossplane](https://crossplane.io/) as the cloud service operator. Please Refer to [Installation](https://github.com/crossplane/provider-alibaba/releases/tag/v0.5.0)
to install Crossplane Alibaba provider v0.5.0.
KubeVela uses [Crossplane](https://crossplane.io/) as the cloud service operator.
If you'd like to configure any other Crossplane providers, please refer to [Crossplane Select a Getting Started Configuration](https://crossplane.io/docs/v1.1/getting-started/install-configure.html#select-a-getting-started-configuration).
> This tutorial has been tested with Crossplane version `0.14`. Please follow the [Crossplane documentation](https://crossplane.io/docs/), especially the `Install & Configure` and `Compose Infrastructure` sections to configure
Crossplane with your cloud account.
```
$ kubectl crossplane install provider crossplane/provider-alibaba:v0.5.0
**Note: When installing Crossplane via Helm chart, please DON'T set `alpha.oam.enabled=true` as all OAM features are already installed by KubeVela.**
# Note the xxx and yyy here is your own AccessKey and SecretKey to the cloud resources.
$ kubectl create secret generic alibaba-account-creds -n crossplane-system --from-literal=accessKeyId=xxx --from-literal=accessKeySecret=yyy
## Step 2: Add Component Definition
$ kubectl apply -f provider.yaml
```
Register the `rds` component to KubeVela.
`provider.yaml` is as below.
```bash
$ cat << EOF | kubectl apply -f -
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: crossplane-system
---
apiVersion: alibaba.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
name: default
spec:
credentials:
source: Secret
secretRef:
namespace: crossplane-system
name: alibaba-account-creds
key: credentials
region: cn-beijing
```
Note: We currently just use Crossplane Alibaba provider. But we are about to use [Crossplane](https://crossplane.io/) as the
cloud resource operator for Kubernetes in the near future.
## Provisioning and consuming cloud resource in a single application v1 (one cloud resource)
### Step 1: Register ComponentDefinition `alibaba-rds` as RDS cloud resource producer
First, register the `alibaba-rds` workload type to KubeVela.
```yaml
apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
metadata:
name: rds
name: alibaba-rds
namespace: vela-system
annotations:
definition.oam.dev/apiVersion: "database.example.org/v1alpha1"
definition.oam.dev/kind: "PostgreSQLInstance"
definition.oam.dev/description: "RDS on Ali Cloud"
definition.oam.dev/description: "Alibaba Cloud RDS Resource"
spec:
workload:
definition:
apiVersion: database.example.org/v1alpha1
kind: PostgreSQLInstance
apiVersion: database.alibaba.crossplane.io/v1alpha1
kind: RDSInstance
schematic:
cue:
template: |
output: {
apiVersion: "database.example.org/v1alpha1"
kind: "PostgreSQLInstance"
metadata:
name: context.name
apiVersion: "database.alibaba.crossplane.io/v1alpha1"
kind: "RDSInstance"
spec: {
parameters:
storageGB: parameter.storage
compositionSelector: {
matchLabels:
provider: parameter.provider
forProvider: {
engine: parameter.engine
engineVersion: parameter.engineVersion
dbInstanceClass: parameter.instanceClass
dbInstanceStorageInGB: 20
securityIPList: "0.0.0.0/0"
masterUsername: parameter.username
}
writeConnectionSecretToRef:
name: parameter.secretname
writeConnectionSecretToRef: {
namespace: context.namespace
name: context.outputSecretName
}
providerConfigRef: {
name: "default"
}
deletionPolicy: "Delete"
}
}
parameter: {
engine: *"mysql" | string
engineVersion: *"8.0" | string
instanceClass: *"rds.mysql.c1.large" | string
username: string
}
```
Noted: In application, application developers need to use property `outputSecretName` as the secret name which is used to store all connection
items of cloud resource connections information.
### Step 2: Prepare TraitDefinition `service-binding` to do env-secret mapping
As for data binding in Application, KubeVela recommends defining a trait to finish the job. We have prepared a common
trait for convenience. This trait works well for binding resources' info into pod spec Env.
```yaml
apiVersion: core.oam.dev/v1beta1
kind: TraitDefinition
metadata:
annotations:
definition.oam.dev/description: "binding cloud resource secrets to pod env"
name: service-binding
spec:
appliesToWorkloads:
- webservice
- worker
schematic:
cue:
template: |
patch: {
spec: template: spec: {
// +patchKey=name
containers: [{
name: context.name
// +patchKey=name
env: [
for envName, v in parameter.envMappings {
name: envName
valueFrom: {
secretKeyRef: {
name: v.secret
if v["key"] != _|_ {
key: v.key
}
if v["key"] == _|_ {
key: envName
}
}
}
},
]
}]
}
}
parameter: {
secretname: *"db-conn" | string
provider: *"alibaba" | string
storage: *20 | int
envMappings: [string]: [string]: string
}
EOF
```
## Step 3: Verify
With the help of this `service-binding` trait, developers can explicitly set parameter `envMappings` to mapping all environment names with secret key. Here is an example.
Instantiate RDS component in an [Application](../application) to provide cloud resources.
```yaml
...
traits:
- type: service-binding
properties:
envMappings:
# environments refer to db-conn secret
DB_PASSWORD:
secret: db-conn
key: password # 1) If the env name is different from secret key, secret key has to be set.
endpoint:
secret: db-conn # 2) If the env name is the same as the secret key, secret key can be omitted.
username:
secret: db-conn
# environments refer to oss-conn secret
BUCKET_NAME:
secret: oss-conn
key: Bucket
...
```
### Step 3: Create an application to provision and consume cloud resource
Create an application with a cloud resource provisioning component and a consuming component as below.
```yaml
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: mydatabase
name: webapp
spec:
components:
- name: myrds
type: rds
- name: express-server
type: webservice
properties:
name: "alibaba-rds"
storage: 20
secretname: "myrds-conn"
image: zzxwill/flask-web-application:v0.3.1-crossplane
ports: 80
traits:
- type: service-binding
properties:
envMappings:
# environments refer to db-conn secret
DB_PASSWORD:
secret: db-conn
key: password # 1) If the env name is different from secret key, secret key has to be set.
endpoint:
secret: db-conn # 2) If the env name is the same as the secret key, secret key can be omitted.
username:
secret: db-conn
- name: sample-db
type: alibaba-rds
properties:
name: sample-db
engine: mysql
engineVersion: "8.0"
instanceClass: rds.mysql.c1.large
username: oamtest
outputSecretName: db-conn
```
Apply above application to Kubernetes and a RDS instance will be automatically provisioned (may take some time, ~5 mins).
Apply it and verify the application.
> TBD: add status check , show database create result.
```shell
$ kubectl get application
NAME AGE
webapp 46m
$ kubectl port-forward deployment/express-server 80:80
Forwarding from 127.0.0.1:80 -> 80
Forwarding from [::1]:80 -> 80
Handling connection for 80
Handling connection for 80
```
## Step 4: Consuming The Cloud Service
![](../../resources/crossplane-visit-application.jpg)
In this section, we will show how another component consumes the RDS instance.
## Provisioning and consuming cloud resource in a single application v2 (two cloud resources)
> Note: we recommend to define the cloud resource claiming to an independent application if that cloud resource has standalone lifecycle. Otherwise, it could be defined in the same application of the consumer component.
### `ComponentDefinition` With Secret Reference
Based on the section `Provisioning and consuming cloud resource in a single application v1 (one cloud resource)`, register
one more cloud resource workload type `alibaba-oss` to KubeVela.
```yaml
apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
metadata:
name: webserver
name: alibaba-oss
namespace: vela-system
annotations:
definition.oam.dev/description: "webserver to consume cloud resources"
definition.oam.dev/description: "Alibaba Cloud RDS Resource"
spec:
workload:
definition:
apiVersion: oss.alibaba.crossplane.io/v1alpha1
kind: Bucket
schematic:
cue:
template: |
output: {
apiVersion: "oss.alibaba.crossplane.io/v1alpha1"
kind: "Bucket"
spec: {
name: parameter.name
acl: parameter.acl
storageClass: parameter.storageClass
dataRedundancyType: parameter.dataRedundancyType
writeConnectionSecretToRef: {
namespace: context.namespace
name: context.outputSecretName
}
providerConfigRef: {
name: "default"
}
deletionPolicy: "Delete"
}
}
parameter: {
name: string
acl: *"private" | string
storageClass: *"Standard" | string
dataRedundancyType: *"LRS" | string
}
```
Update the application to also consume cloud resource OSS.
```yaml
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: webapp
spec:
components:
- name: express-server
type: webservice
properties:
image: zzxwill/flask-web-application:v0.3.1-crossplane
ports: 80
traits:
- type: service-binding
properties:
envMappings:
# environments refer to db-conn secret
DB_PASSWORD:
secret: db-conn
key: password # 1) If the env name is different from secret key, secret key has to be set.
endpoint:
secret: db-conn # 2) If the env name is the same as the secret key, secret key can be omitted.
username:
secret: db-conn
# environments refer to oss-conn secret
BUCKET_NAME:
secret: oss-conn
key: Bucket
- name: sample-db
type: alibaba-rds
properties:
name: sample-db
engine: mysql
engineVersion: "8.0"
instanceClass: rds.mysql.c1.large
username: oamtest
outputSecretName: db-conn
- name: sample-oss
type: alibaba-oss
properties:
name: velaweb
outputSecretName: oss-conn
```
Apply it and verify the application.
```shell
$ kubectl port-forward deployment/express-server 80:80
Forwarding from 127.0.0.1:80 -> 80
Forwarding from [::1]:80 -> 80
Handling connection for 80
Handling connection for 80
```
![](../../resources/crossplane-visit-application-v2.jpg)
## Provisioning and consuming cloud resource in different applications
In this section, cloud resource will be provisioned in one application and consumed in another application.
### Provision Cloud Resource
Instantiate RDS component with `alibaba-rds` workload type in an [Application](../application.md) to provide cloud resources.
As we have claimed an RDS instance with ComponentDefinition name `alibaba-rds`.
The component in the application should refer to this type.
```yaml
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: baas-rds
spec:
components:
- name: sample-db
type: alibaba-rds
properties:
name: sample-db
engine: mysql
engineVersion: "8.0"
instanceClass: rds.mysql.c1.large
username: oamtest
outputSecretName: db-conn
```
Apply the application to Kubernetes and a RDS instance will be automatically provisioned (may take some time, ~2 mins).
A secret `db-conn` will also be created in the same namespace as that of the application.
```shell
$ kubectl get application
NAME AGE
baas-rds 9h
$ kubectl get rdsinstance
NAME READY SYNCED STATE ENGINE VERSION AGE
sample-db-v1 True True Running mysql 8.0 9h
$ kubectl get secret
NAME TYPE DATA AGE
db-conn connection.crossplane.io/v1alpha1 4 9h
$ ✗ kubectl get secret db-conn -o yaml
apiVersion: v1
data:
endpoint: xxx==
password: yyy
port: MzMwNg==
username: b2FtdGVzdA==
kind: Secret
```
### Consuming the Cloud Resource
In this section, we will show how another component consumes the RDS instance.
> Note: we recommend defining the cloud resource claiming to an independent application if that cloud resource has
> standalone lifecycle.
#### Step 1: Define a ComponentDefinition with Secret Reference
```yaml
apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
metadata:
name: webconsumer
annotations:
definition.oam.dev/description: A Deployment provides declarative updates for Pods and ReplicaSets
spec:
workload:
definition:
@@ -124,10 +437,12 @@ spec:
selector: matchLabels: {
"app.oam.dev/component": context.name
}
template: {
metadata: labels: {
"app.oam.dev/component": context.name
}
spec: {
containers: [{
name: context.name
@@ -136,46 +451,106 @@ spec:
if parameter["cmd"] != _|_ {
command: parameter.cmd
}
env: [{
name: "DB_NAME"
value: mySecret.dbName
}, {
name: "DB_PASSWORD"
value: mySecret.password
if parameter["dbSecret"] != _|_ {
env: [
{
name: "username"
value: dbConn.username
},
{
name: "endpoint"
value: dbConn.endpoint
},
{
name: "DB_PASSWORD"
value: dbConn.password
},
]
}
ports: [{
containerPort: parameter.port
}]
if parameter["cpu"] != _|_ {
resources: {
limits:
cpu: parameter.cpu
requests:
cpu: parameter.cpu
}
}
}]
}
}
}
}
}
mySecret: {
dbName: string
parameter: {
// +usage=Which image would you like to use for your service
// +short=i
image: string
// +usage=Commands to run in the container
cmd?: [...string]
// +usage=Which port do you want customer traffic sent to
// +short=p
port: *80 | int
// +usage=Referred db secret
// +insertSecretTo=dbConn
dbSecret?: string
// +usage=Number of CPU units for the service, like `0.5` (0.5 CPU core), `1` (1 CPU core)
cpu?: string
}
dbConn: {
username: string
endpoint: string
password: string
}
parameter: {
image: string
//+InsertSecretTo=mySecret
dbConnection: string
cmd?: [...string]
}
```
With the `//+InsertSecretTo=mySecret` annotation, KubeVela knows this parameter value comes from a Kubernetes Secret (whose name is set by user), so it will inject its data to `mySecret` which is referenced as environment variable in the template.
The key point is the annotation `//+insertSecretTo=dbConn`, KubeVela will know the parameter is a K8s secret, it will parse
the secret and bind the data into the CUE struct `dbConn`.
Then declare an application to consume the RDS instance.
Then the `output` can reference the `dbConn` struct for the data value. The name `dbConn` can be any name.
It's just an example in this case. The `+insertSecretTo` is keyword, it defines the data binding mechanism.
Now create the Application to consume the data.
```yaml
apiVersion: core.oam.dev/v1alpha2
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: data-consumer
name: webapp
spec:
components:
- name: myweb
type: webserver
- name: express-server
type: webconsumer
properties:
image: "nginx"
dbConnection: "mydb-outputs"
image: zzxwill/flask-web-application:v0.3.1-crossplane
ports: 80
dbSecret: db-conn
```
// TBD show the result
```shell
$ kubectl get application
NAME AGE
baas-rds 10h
webapp 14h
$ kubectl get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
express-server-v1 1/1 1 1 9h
$ kubectl port-forward deployment/express-server 80:80
```
We can see the cloud resource is successfully consumed by the application.
![](../../resources/crossplane-visit-application.jpg)

BIN
docs/resources/crossplane-visit-application-v2.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 82 KiB

BIN
docs/resources/crossplane-visit-application.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 94 KiB

136
pkg/appfile/parser.go
View File

@@ -20,6 +20,8 @@ import (
"context"
"encoding/json"
"fmt"
"regexp"
"strings"
"cuelang.org/go/cue"
"cuelang.org/go/cue/format"
@@ -27,6 +29,7 @@ import (
"github.com/crossplane/crossplane-runtime/apis/core/v1alpha1"
"github.com/crossplane/crossplane-runtime/pkg/fieldpath"
"github.com/pkg/errors"
v1 "k8s.io/api/core/v1"
kerrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime"
@@ -39,6 +42,7 @@ import (
"github.com/oam-dev/kubevela/apis/types"
"github.com/oam-dev/kubevela/pkg/appfile/config"
"github.com/oam-dev/kubevela/pkg/appfile/helm"
"github.com/oam-dev/kubevela/pkg/controller/utils"
"github.com/oam-dev/kubevela/pkg/dsl/definition"
"github.com/oam-dev/kubevela/pkg/dsl/process"
"github.com/oam-dev/kubevela/pkg/oam"
@@ -75,6 +79,10 @@ type Workload struct {
FullTemplate *util.Template
engine definition.AbstractEngine
// OutputSecretName is the secret name which this workload will generate after it successfully generate a cloud resource
OutputSecretName string
// RequiredSecrets stores secret names which the workload needs from cloud resource component and its context
RequiredSecrets []process.RequiredSecrets
}
// GetUserConfigName get user config from AppFile, it will contain config file in it.
@@ -271,10 +279,23 @@ func (p *Parser) GenerateApplicationConfiguration(app *Appfile, ns string) (*v1a
appconfig.Labels[oam.LabelAppName] = app.Name
var components []*v1alpha2.Component
ctx := context.Background()
for _, wl := range app.Workloads {
var comp *v1alpha2.Component
var acComp *v1alpha2.ApplicationConfigurationComponent
var err error
var (
comp *v1alpha2.Component
acComp *v1alpha2.ApplicationConfigurationComponent
err error
)
if wl.IsCloudResourceConsumer() {
requiredSecrets, err := parseWorkloadInsertSecretTo(ctx, p.client, ns, wl)
if err != nil {
return nil, nil, err
}
wl.RequiredSecrets = requiredSecrets
}
switch wl.CapabilityCategory {
case types.HelmCategory:
comp, acComp, err = generateComponentFromHelmModule(p.client, wl, app.Name, app.RevisionName, ns)
@@ -299,6 +320,17 @@ func (p *Parser) GenerateApplicationConfiguration(app *Appfile, ns string) (*v1a
}
func generateComponentFromCUEModule(c client.Client, wl *Workload, appName, revision, ns string) (*v1alpha2.Component, *v1alpha2.ApplicationConfigurationComponent, error) {
var (
outputSecretName string
err error
)
if wl.IsCloudResourceProducer() {
outputSecretName, err = GetOutputSecretNames(wl)
if err != nil {
return nil, nil, err
}
wl.OutputSecretName = outputSecretName
}
pCtx, err := PrepareProcessContext(c, wl, appName, revision, ns)
if err != nil {
return nil, nil, err
@@ -537,8 +569,9 @@ func evalWorkloadWithContext(pCtx process.Context, wl *Workload, appName, compNa
}
// PrepareProcessContext prepares a DSL process Context
func PrepareProcessContext(k8sClient client.Client, wl *Workload, applicationName, revision string, namespace string) (process.Context, error) {
pCtx := process.NewContext(wl.Name, applicationName, revision)
func PrepareProcessContext(k8sClient client.Client, wl *Workload, applicationName, revision, namespace string) (process.Context, error) {
pCtx := process.NewContext(namespace, wl.Name, applicationName, revision)
pCtx.InsertSecrets(wl.OutputSecretName, wl.RequiredSecrets)
userConfig := wl.GetUserConfigName()
if userConfig != "" {
cg := config.Configmap{Client: k8sClient}
@@ -555,3 +588,96 @@ func PrepareProcessContext(k8sClient client.Client, wl *Workload, applicationNam
}
return pCtx, nil
}
// GetOutputSecretNames set all secret names, which are generated by cloud resource, to context
func GetOutputSecretNames(workloads *Workload) (string, error) {
secretName, err := getComponentSetting(process.OutputSecretName, workloads.Params)
if err != nil {
return "", err
}
return fmt.Sprint(secretName), nil
}
func parseWorkloadInsertSecretTo(ctx context.Context, c client.Client, namespace string, wl *Workload) ([]process.RequiredSecrets, error) {
var requiredSecret []process.RequiredSecrets
api, err := utils.GenerateOpenAPISchemaFromDefinition(wl.Name, wl.Template)
if err != nil {
if !errors.Is(err, errors.Errorf(utils.ErrNoSectionParameterInCue, wl.Name)) {
return nil, nil
}
return nil, err
}
schema, err := utils.ConvertOpenAPISchema2SwaggerObject(api)
if err != nil {
return nil, err
}
for k, v := range schema.Properties {
description := v.Value.Description
if strings.Contains(description, utils.InsertSecretToTag) {
contextName := strings.Split(description, utils.InsertSecretToTag)[1]
contextName = strings.TrimSpace(contextName)
secretNameInterface, err := getComponentSetting(k, wl.Params)
if err != nil {
return nil, err
}
secretName, ok := secretNameInterface.(string)
if !ok {
return nil, fmt.Errorf("failed to convert secret name %v to string", secretNameInterface)
}
secretData, err := extractSecret(ctx, c, namespace, secretName)
if err != nil {
return nil, err
}
requiredSecret = append(requiredSecret, process.RequiredSecrets{
Name: secretName,
ContextName: contextName,
Namespace: namespace,
Data: secretData,
})
}
}
return requiredSecret, nil
}
func extractSecret(ctx context.Context, c client.Client, namespace, name string) (map[string]interface{}, error) {
secretData := make(map[string]interface{})
var secret v1.Secret
if err := c.Get(ctx, client.ObjectKey{Name: name, Namespace: namespace}, &secret); err != nil {
return nil, fmt.Errorf("failed to get secret %s from namespace %s which is required by the component: %w",
name, namespace, err)
}
for k, v := range secret.Data {
secretData[k] = string(v)
}
if len(secretData) == 0 {
return nil, fmt.Errorf("data in secret %s from namespace %s isn't available", name, namespace)
}
return secretData, nil
}
func getComponentSetting(settingParamName string, params map[string]interface{}) (interface{}, error) {
if secretName, ok := params[settingParamName]; ok {
return secretName, nil
}
return nil, fmt.Errorf("failed to get the value of component setting %s", settingParamName)
}
// IsCloudResourceProducer checks whether a workload is cloud resource producer role
func (wl *Workload) IsCloudResourceProducer() bool {
var existed bool
_, existed = wl.Params[process.OutputSecretName]
return existed
}
// IsCloudResourceConsumer checks whether a workload is cloud resource consumer role
func (wl *Workload) IsCloudResourceConsumer() bool {
requiredSecretTag := strings.TrimRight(utils.InsertSecretToTag, "=")
matched, err := regexp.Match(regexp.QuoteMeta(requiredSecretTag), []byte(wl.Template))
if err != nil || !matched {
return false
}
return true
}

180
pkg/appfile/parser_test.go
View File

@@ -42,6 +42,7 @@ import (
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
oamtypes "github.com/oam-dev/kubevela/apis/types"
"github.com/oam-dev/kubevela/pkg/dsl/definition"
"github.com/oam-dev/kubevela/pkg/dsl/process"
"github.com/oam-dev/kubevela/pkg/oam"
"github.com/oam-dev/kubevela/pkg/oam/util"
)
@@ -848,3 +849,182 @@ spec:
Expect(diff).Should(BeEmpty())
})
})
var _ = Describe("Test Get OutputSecretNames", func() {
Context("Workload will generate cloud resource secret", func() {
It("", func() {
var targetSecretName = "db-conn"
wl := &Workload{
Params: map[string]interface{}{
"outputSecretName": targetSecretName,
},
}
name, err := GetOutputSecretNames(wl)
Expect(err).Should(BeNil())
Expect(name).Should(Equal(targetSecretName))
})
})
Context("Workload will not generate cloud resource secret", func() {
It("", func() {
wl := &Workload{}
name, err := GetOutputSecretNames(wl)
Expect(err).ShouldNot(BeNil())
Expect(name).Should(Equal(""))
})
})
})
var _ = Describe("Test parsing Workload's insertSecretTo tag", func() {
var (
ctx = context.Background()
ns = "default"
targetSecretName = "db-conn"
data = map[string][]byte{
"endpoint": []byte("aaa"),
"password": []byte("bbb"),
"username": []byte("ccc"),
}
)
Context("Workload template is not valid", func() {
It("", func() {
var (
template = `
settings: {
// +usage=Which image would you like to use for your service
// +short=i
image: string
// +usage=Commands to run in the container
cmd?: [...string]
// +usage=Which port do you want customer traffic sent to
// +short=p
port: *80 | int
// +usage=Referred db secret
// +insertSecretTo=dbConn
dbSecret?: string
// +usage=Number of CPU units for the service
cpu?: string
}
`
)
wl := &Workload{
Name: "abc",
Template: template,
}
By("call target function")
secrets, err := parseWorkloadInsertSecretTo(ctx, k8sClient, ns, wl)
Expect(err).Should(BeNil())
Expect(secrets).Should(BeNil())
})
})
Context("Workload will generate cloud resource secret", func() {
It("", func() {
var (
template = `
parameter: {
// +usage=Which image would you like to use for your service
// +short=i
image: string
// +usage=Commands to run in the container
cmd?: [...string]
// +usage=Which port do you want customer traffic sent to
// +short=p
port: *80 | int
// +usage=Referred db secret
// +insertSecretTo=dbConn
dbSecret?: string
// +usage=Number of CPU units for the service
cpu?: string
}
`
)
wl := &Workload{
Name: "abc",
Params: map[string]interface{}{
"dbSecret": targetSecretName,
},
Template: template,
}
By("create secret")
s := &corev1.Secret{
TypeMeta: metav1.TypeMeta{APIVersion: "v1", Kind: "Secret"},
ObjectMeta: metav1.ObjectMeta{
Name: "db-conn",
Namespace: "default",
},
Data: data,
}
targetRequiredSecret := []process.RequiredSecrets{
{
Name: targetSecretName,
ContextName: "dbConn",
Namespace: ns,
Data: map[string]interface{}{
"endpoint": "aaa",
"password": "bbb",
"username": "ccc",
},
},
}
err := k8sClient.Create(ctx, s)
Expect(err).Should(BeNil())
By("call target function")
secrets, err := parseWorkloadInsertSecretTo(ctx, k8sClient, ns, wl)
Expect(err).Should(BeNil())
Expect(secrets).Should(Equal(targetRequiredSecret))
})
})
})
var _ = Describe("Test IsCloudResourceProducer", func() {
Context("Workload is a Cloud Resource producer", func() {
It("", func() {
var targetSecretName = "db-conn"
wl := &Workload{
Params: map[string]interface{}{
"outputSecretName": targetSecretName,
},
}
Expect(wl.IsCloudResourceProducer()).Should(Equal(true))
})
})
Context("Workload is a Cloud Resource producer", func() {
It("", func() {
wl := &Workload{}
Expect(wl.IsCloudResourceProducer()).Should(Equal(false))
})
})
})
var _ = Describe("Test IsCloudResourceConsumer", func() {
Context("Workload is a Cloud Resource consumer", func() {
It("", func() {
wl := &Workload{
Template: "// +insertSecretTo=dbConn",
}
Expect(wl.IsCloudResourceConsumer()).Should(Equal(true))
})
})
Context("Workload is a Cloud Resource consumer", func() {
It("", func() {
wl := &Workload{
Template: "// +useage=dbConn",
}
Expect(wl.IsCloudResourceProducer()).Should(Equal(false))
})
})
})

181
pkg/controller/core.oam.dev/v1alpha2/application/application_controller_test.go
View File

@@ -233,12 +233,101 @@ var _ = Describe("Test Application Controller", func() {
Expect(json.Unmarshal(webserverwdJson, webserverwd)).Should(BeNil())
Expect(k8sClient.Create(ctx, webserverwd.DeepCopy())).Should(SatisfyAny(BeNil(), &util.AlreadyExistMatcher{}))
var deployDef v1alpha2.WorkloadDefinition
Expect(yaml.Unmarshal([]byte(deploymentWorkloadDefinition), &deployDef)).Should(BeNil())
Expect(k8sClient.Create(ctx, &deployDef)).Should(SatisfyAny(BeNil()))
})
AfterEach(func() {
var tobeDeletedDeployDef v1alpha2.WorkloadDefinition
Expect(k8sClient.Get(ctx, client.ObjectKey{Name: "deployment", Namespace: "default"}, &tobeDeletedDeployDef)).Should(SatisfyAny(BeNil()))
Expect(k8sClient.Delete(ctx, &tobeDeletedDeployDef)).Should(SatisfyAny(BeNil()))
By("[TEST] Clean up resources after an integration test")
})
It("app can consume db secret generated by other application", func() {
var (
appName = "webapp"
ns = "default"
componentName = "express-server-test"
targetSecretName = "db-conn"
secretData = map[string][]byte{
"endpoint": []byte("aaa"),
"password": []byte("bbb"),
"username": []byte("ccc"),
}
businessApplication = `
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: webapp
namespace: default
spec:
components:
- name: express-server-test
type: deployment
properties:
image: "nignx:latest"
ports: 80
dbSecret: "db-conn"
`
appKey = client.ObjectKey{
Name: appName,
Namespace: ns,
}
)
By("Check WorkloadDefinition")
var wd v1alpha2.WorkloadDefinition
err := k8sClient.Get(ctx, client.ObjectKey{Namespace: ns, Name: "deployment"}, &wd)
Expect(err).Should(BeNil())
By("create secret")
s := &corev1.Secret{
TypeMeta: metav1.TypeMeta{APIVersion: "v1", Kind: "Secret"},
ObjectMeta: metav1.ObjectMeta{
Name: targetSecretName,
Namespace: ns,
},
Data: secretData,
}
err = k8sClient.Create(ctx, s)
Expect(err).Should(BeNil())
By("apply business application which needs consume db")
var app v1beta1.Application
err = yaml.Unmarshal([]byte(businessApplication), &app)
Expect(err).Should(BeNil())
err = k8sClient.Create(ctx, &app)
Expect(err).Should(BeNil())
reconcileRetry(reconciler, reconcile.Request{NamespacedName: appKey})
By("checking application")
var a v1beta1.Application
err = k8sClient.Get(ctx, appKey, &a)
Expect(err).Should(BeNil())
By("Check ApplicationContext Created")
var appContext v1alpha2.ApplicationContext
Expect(k8sClient.Get(ctx, appKey, &appContext)).Should(BeNil())
By("Check Component Created with the expected workload spec")
var component v1alpha2.Component
Expect(k8sClient.Get(ctx, client.ObjectKey{Namespace: ns, Name: componentName}, &component)).Should(BeNil())
var deploy v1.Deployment
Expect(json.Unmarshal(component.Spec.Workload.Raw, &deploy)).Should(BeNil())
containers := deploy.Spec.Template.Spec.Containers
Expect(len(containers)).Should(Equal(1))
envs := containers[0].Env
Expect(len(envs)).Should(Equal(1))
Expect(envs[0].Value).Should(Equal("ccc"))
})
It("app-without-trait will only create workload", func() {
expDeployment := getExpDeployment("myweb2", appwithNoTrait.Name)
ns := &corev1.Namespace{
@@ -2033,6 +2122,98 @@ spec:
}
}
`
deploymentWorkloadDefinition = `
apiVersion: core.oam.dev/beta1
kind: WorkloadDefinition
metadata:
name: deployment
namespace: default
annotations:
definition.oam.dev/description: A Deployment provides declarative updates for Pods and ReplicaSets
spec:
workload:
definition:
apiVersion: apps/v1
kind: Deployment
extension:
template: |
output: {
apiVersion: "apps/v1"
kind: "Deployment"
metadata: name: "business-deploy"
spec: {
selector: matchLabels: {
"app.oam.dev/component": context.name
}
template: {
metadata: labels: {
"app.oam.dev/component": context.name
}
spec: {
containers: [{
name: "business-deploy"
image: parameter.image
if parameter["cmd"] != _|_ {
command: parameter.cmd
}
if parameter["dbSecret"] != _|_ {
env: [
{
name: "username"
value: dbConn.username
},
]
}
ports: [{
containerPort: parameter.port
}]
if parameter["cpu"] != _|_ {
resources: {
limits:
cpu: parameter.cpu
requests:
cpu: parameter.cpu
}
}
}]
}
}
}
}
parameter: {
// +usage=Which image would you like to use for your service
// +short=i
image: string
// +usage=Commands to run in the container
cmd?: [...string]
// +usage=Which port do you want customer traffic sent to
// +short=p
port: *80 | int
// +usage=Referred db secret
// +insertSecretTo=dbConn
dbSecret?: string
// +usage=Number of CPU units for the service
cpu?: string
}
dbConn: {
username: string
endpoint: string
port: string
}
`
)
func NewMock() *httptest.Server {

30
pkg/controller/core.oam.dev/v1alpha2/application/apply.go
View File

@@ -170,27 +170,39 @@ func (h *appHandler) createOrUpdateAppRevision(ctx context.Context, appRev *v1be
return h.r.Update(ctx, appRev)
}
func (h *appHandler) statusAggregate(appfile *appfile.Appfile) ([]common.ApplicationComponentStatus, bool, error) {
func (h *appHandler) statusAggregate(appFile *appfile.Appfile) ([]common.ApplicationComponentStatus, bool, error) {
var appStatus []common.ApplicationComponentStatus
var healthy = true
for _, wl := range appfile.Workloads {
for _, wl := range appFile.Workloads {
var status = common.ApplicationComponentStatus{
Name: wl.Name,
Healthy: true,
}
pCtx := process.NewContext(wl.Name, appfile.Name, appfile.RevisionName)
var (
outputSecretName string
err error
)
pCtx := process.NewContext(h.app.Namespace, wl.Name, appFile.Name, appFile.RevisionName)
if wl.IsCloudResourceProducer() {
outputSecretName, err = appfile.GetOutputSecretNames(wl)
if err != nil {
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, setting outputSecretName error", appFile.Name, wl.Name)
}
pCtx.InsertSecrets(outputSecretName, wl.RequiredSecrets)
}
if err := wl.EvalContext(pCtx); err != nil {
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, evaluate context error", appfile.Name, wl.Name)
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, evaluate context error", appFile.Name, wl.Name)
}
for _, tr := range wl.Traits {
if err := tr.EvalContext(pCtx); err != nil {
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, trait=%s, evaluate context error", appfile.Name, wl.Name, tr.Name)
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, trait=%s, evaluate context error", appFile.Name, wl.Name, tr.Name)
}
}
workloadHealth, err := wl.EvalHealth(pCtx, h.r, h.app.Namespace)
if err != nil {
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, check health error", appfile.Name, wl.Name)
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, check health error", appFile.Name, wl.Name)
}
if !workloadHealth {
// TODO(wonderflow): we should add a custom way to let the template say why it's unhealthy, only a bool flag is not enough
@@ -200,7 +212,7 @@ func (h *appHandler) statusAggregate(appfile *appfile.Appfile) ([]common.Applica
status.Message, err = wl.EvalStatus(pCtx, h.r, h.app.Namespace)
if err != nil {
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, evaluate workload status message error", appfile.Name, wl.Name)
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, evaluate workload status message error", appFile.Name, wl.Name)
}
var traitStatusList []common.ApplicationTraitStatus
for _, trait := range wl.Traits {
@@ -210,7 +222,7 @@ func (h *appHandler) statusAggregate(appfile *appfile.Appfile) ([]common.Applica
}
traitHealth, err := trait.EvalHealth(pCtx, h.r, h.app.Namespace)
if err != nil {
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, trait=%s, check health error", appfile.Name, wl.Name, trait.Name)
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, trait=%s, check health error", appFile.Name, wl.Name, trait.Name)
}
if !traitHealth {
// TODO(wonderflow): we should add a custom way to let the template say why it's unhealthy, only a bool flag is not enough
@@ -219,7 +231,7 @@ func (h *appHandler) statusAggregate(appfile *appfile.Appfile) ([]common.Applica
}
traitStatus.Message, err = trait.EvalStatus(pCtx, h.r, h.app.Namespace)
if err != nil {
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, trait=%s, evaluate status message error", appfile.Name, wl.Name, trait.Name)
return nil, false, errors.WithMessagef(err, "app=%s, comp=%s, trait=%s, evaluate status message error", appFile.Name, wl.Name, trait.Name)
}
traitStatusList = append(traitStatusList, traitStatus)
}

40
pkg/controller/utils/capability.go
View File

@@ -46,12 +46,17 @@ const (
UsageTag = "+usage="
// ShortTag is the short alias annotation
ShortTag = "+short"
// InsertSecretToTag marks the value should be set as an context
InsertSecretToTag = "+insertSecretTo="
)
// ErrNoSectionParameterInCue means there is not parameter section in Cue template of a workload
const ErrNoSectionParameterInCue = "capability %s doesn't contain section `parameter`"
// CapabilityDefinitionInterface is the interface for Capability (WorkloadDefinition and TraitDefinition)
type CapabilityDefinitionInterface interface {
GetCapabilityObject(ctx context.Context, k8sClient client.Client, namespace, name string) (types.Capability, error)
GetOpenAPISchema(ctx context.Context, k8sClient client.Client, objectKey client.ObjectKey) ([]byte, error)
GetCapabilityObject(ctx context.Context, k8sClient client.Client, namespace, name string) (*types.Capability, error)
GetOpenAPISchema(ctx context.Context, k8sClient client.Client, namespace, name string) ([]byte, error)
}
// CapabilityComponentDefinition is the struct for ComponentDefinition
@@ -287,15 +292,10 @@ func getOpenAPISchema(capability types.Capability) ([]byte, error) {
if err != nil {
return nil, err
}
swagger, err := openapi3.NewSwaggerLoader().LoadSwaggerFromData(openAPISchema)
schema, err := ConvertOpenAPISchema2SwaggerObject(openAPISchema)
if err != nil {
return nil, err
}
schemaRef := swagger.Components.Schemas["parameter"]
if schemaRef == nil {
return nil, fmt.Errorf(util.ErrGenerateOpenAPIV2JSONSchemaForCapability, capability.Name, nil)
}
schema := schemaRef.Value
fixOpenAPISchema("", schema)
parameter, err := schema.MarshalJSON()
@@ -307,8 +307,12 @@ func getOpenAPISchema(capability types.Capability) ([]byte, error) {
// generateOpenAPISchemaFromCapabilityParameter returns the parameter of a definition in cue.Value format
func generateOpenAPISchemaFromCapabilityParameter(capability types.Capability) ([]byte, error) {
name := capability.Name
template, err := prepareParameterCue(name, capability.CueTemplate)
return GenerateOpenAPISchemaFromDefinition(capability.Name, capability.CueTemplate)
}
// GenerateOpenAPISchemaFromDefinition returns the parameter of a definition
func GenerateOpenAPISchemaFromDefinition(definitionName, cueTemplate string) ([]byte, error) {
template, err := prepareParameterCue(definitionName, cueTemplate)
if err != nil {
return nil, err
}
@@ -340,7 +344,7 @@ func prepareParameterCue(capabilityName, capabilityTemplate string) (string, err
}
if !withParameterFlag {
return "", fmt.Errorf("capability %s doesn't contain section `parmeter`", capabilityName)
return "", fmt.Errorf(ErrNoSectionParameterInCue, capabilityName)
}
return template, nil
}
@@ -371,3 +375,17 @@ func fixOpenAPISchema(name string, schema *openapi3.Schema) {
}
schema.Description = description
}
// ConvertOpenAPISchema2SwaggerObject converts OpenAPI v2 JSON schema to Swagger Object
func ConvertOpenAPISchema2SwaggerObject(data []byte) (*openapi3.Schema, error) {
swagger, err := openapi3.NewSwaggerLoader().LoadSwaggerFromData(data)
if err != nil {
return nil, err
}
schemaRef, ok := swagger.Components.Schemas[mycue.ParameterTag]
if !ok {
return nil, errors.New(util.ErrGenerateOpenAPIV2JSONSchemaForCapability)
}
return schemaRef.Value, nil
}

7
pkg/controller/utils/capability_test.go
View File

@@ -31,6 +31,7 @@ import (
"github.com/oam-dev/kubevela/apis/core.oam.dev/common"
"github.com/oam-dev/kubevela/apis/types"
mycue "github.com/oam-dev/kubevela/pkg/cue"
"github.com/oam-dev/kubevela/pkg/oam/util"
"github.com/oam-dev/kubevela/pkg/utils/system"
)
@@ -61,7 +62,7 @@ func TestGetOpenAPISchema(t *testing.T) {
name: "invalidWorkload",
fileDir: TestDir,
fileName: "workloadNoParameter.cue",
want: want{data: "", err: fmt.Errorf("capability invalidWorkload doesn't contain section `parmeter`")},
want: want{data: "", err: fmt.Errorf("capability invalidWorkload doesn't contain section `parameter`")},
},
}
@@ -103,7 +104,7 @@ func TestFixOpenAPISchema(t *testing.T) {
for name, tc := range cases {
t.Run(name, func(t *testing.T) {
swagger, _ := openapi3.NewSwaggerLoader().LoadSwaggerFromFile(filepath.Join(TestDir, tc.inputFile))
schema := swagger.Components.Schemas["parameter"].Value
schema := swagger.Components.Schemas[mycue.ParameterTag].Value
fixOpenAPISchema("", schema)
fixedSchema, _ := schema.MarshalJSON()
expectedSchema, _ := ioutil.ReadFile(filepath.Join(TestDir, tc.fixedFile))
@@ -132,7 +133,7 @@ func TestGenerateOpenAPISchemaFromCapabilityParameter(t *testing.T) {
"GenerateOpenAPISchemaFromInvalidCapability": {
reason: "generate OpenAPI schema for an invalid Workload/Trait",
capability: types.Capability{Name: invalidWorkloadName},
want: want{data: nil, err: fmt.Errorf("capability IAmAnInvalidWorkloadDefinition doesn't contain section `parmeter`")},
want: want{data: nil, err: fmt.Errorf("capability IAmAnInvalidWorkloadDefinition doesn't contain section `parameter`")},
},
}
for name, tc := range cases {

6
pkg/cue/convert.go
View File

@@ -26,8 +26,8 @@ import (
"github.com/oam-dev/kubevela/apis/types"
)
// para struct contains the parameter
const specValue = "parameter"
// ParameterTag is the keyword in CUE template to define users' input
var ParameterTag = "parameter"
// GetParameters get parameter from cue template
func GetParameters(templateStr string) ([]types.Parameter, error) {
@@ -45,7 +45,7 @@ func GetParameters(templateStr string) ([]types.Parameter, error) {
var found bool
for i := 0; i < tempStruct.Len(); i++ {
paraDef = tempStruct.Field(i)
if paraDef.Name == specValue {
if paraDef.Name == ParameterTag {
found = true
break
}

9
pkg/dsl/definition/template.go
View File

@@ -27,6 +27,7 @@ import (
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"sigs.k8s.io/controller-runtime/pkg/client"
mycue "github.com/oam-dev/kubevela/pkg/cue"
"github.com/oam-dev/kubevela/pkg/dsl/model"
"github.com/oam-dev/kubevela/pkg/dsl/process"
"github.com/oam-dev/kubevela/pkg/dsl/task"
@@ -92,14 +93,14 @@ func (wd *workloadDef) Complete(ctx process.Context, abstractTemplate string, pa
return errors.WithMessagef(err, "marshal parameter of workload %s", wd.name)
}
if string(bt) != "null" {
paramFile = fmt.Sprintf("parameter: %s", string(bt))
paramFile = fmt.Sprintf("%s: %s", mycue.ParameterTag, string(bt))
}
}
if err := bi.AddFile("parameter", paramFile); err != nil {
return errors.WithMessagef(err, "invalid parameter of workload %s", wd.name)
}
if err := bi.AddFile("-", ctx.BaseContextFile()); err != nil {
if err := bi.AddFile("-", ctx.ExtendedContextFile()); err != nil {
return err
}
wd.pd.ImportBuiltinPackagesFor(bi)
@@ -275,13 +276,13 @@ func (td *traitDef) Complete(ctx process.Context, abstractTemplate string, param
return errors.WithMessagef(err, "marshal parameter of trait %s", td.name)
}
if string(bt) != "null" {
paramFile = fmt.Sprintf("parameter: %s", string(bt))
paramFile = fmt.Sprintf("%s: %s", mycue.ParameterTag, string(bt))
}
}
if err := bi.AddFile("parameter", paramFile); err != nil {
return errors.WithMessagef(err, "invalid parameter of trait %s", td.name)
}
if err := bi.AddFile("context", ctx.BaseContextFile()); err != nil {
if err := bi.AddFile("context", ctx.ExtendedContextFile()); err != nil {
return errors.WithMessagef(err, "invalid context of trait %s", td.name)
}
td.pd.ImportBuiltinPackagesFor(bi)

4
pkg/dsl/definition/template_test.go
View File

@@ -139,7 +139,7 @@ parameter: {
}
for _, v := range testCases {
ctx := process.NewContext("test", "myapp", "myapp-v1")
ctx := process.NewContext("default", "test", "myapp", "myapp-v1")
wt := NewWorkloadAbstractEngine("testworkload", &PackageDiscover{})
assert.NoError(t, wt.Complete(ctx, v.workloadTemplate, v.params))
base, assists := ctx.Output()
@@ -638,7 +638,7 @@ parameter: {
}
`
ctx := process.NewContext("test", "myapp", "myapp-v1")
ctx := process.NewContext("default", "test", "myapp", "myapp-v1")
wt := NewWorkloadAbstractEngine("-", &PackageDiscover{})
if err := wt.Complete(ctx, baseTemplate, map[string]interface{}{
"replicas": 2,

5
pkg/dsl/model/instance_test.go
View File

@@ -17,12 +17,15 @@ limitations under the License.
package model
import (
"fmt"
"testing"
"cuelang.org/go/cue"
"github.com/stretchr/testify/assert"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/runtime/schema"
mycue "github.com/oam-dev/kubevela/pkg/cue"
)
func TestIndexMatchLine(t *testing.T) {
@@ -164,7 +167,7 @@ metadata: name: parameter.name
`,
}
_, err = ins.Unstructured()
assert.Equal(t, err.Error(), `metadata.name: reference "parameter" not found`)
assert.Equal(t, err.Error(), fmt.Sprintf(`metadata.name: reference "%s" not found`, mycue.ParameterTag))
ins = &instance{
v: `
apiVersion: "apps/v1"

63
pkg/dsl/process/handle.go
View File

@@ -38,6 +38,10 @@ const (
ContextAppName = "appName"
// ContextAppRevision is the revision name of app of context
ContextAppRevision = "appRevision"
// ContextNamespace is the namespace of the app
ContextNamespace = "namespace"
// OutputSecretName is used to store all secret names which are generated by cloud resource components
OutputSecretName = "outputSecretName"
)
// Context defines Rendering Context Interface
@@ -47,7 +51,9 @@ type Context interface {
SetConfigs(configs []map[string]string)
Output() (model.Instance, []Auxiliary)
BaseContextFile() string
ExtendedContextFile() string
BaseContextLabels() map[string]string
InsertSecrets(outputSecretName string, requiredSecrets []RequiredSecrets)
}
// Auxiliary are objects rendered by definition template.
@@ -72,16 +78,32 @@ type templateContext struct {
configs []map[string]string
base model.Instance
auxiliaries []Auxiliary
// namespace is the namespace of Application which is used to set the namespace for Crossplane connection secret,
// ComponentDefinition/TratiDefinition OpenAPI v3 schema
namespace string
// outputSecretName is used to store all secret names which are generated by cloud resource components
outputSecretName string
// requiredSecrets is used to store all secret names which are generated by cloud resource components and required by current component
requiredSecrets []RequiredSecrets
}
// RequiredSecrets is used to store all secret names which are generated by cloud resource components and required by current component
type RequiredSecrets struct {
Namespace string
Name string
ContextName string
Data map[string]interface{}
}
// NewContext create render templateContext
func NewContext(name, appName, appRevision string) Context {
func NewContext(namespace, name, appName, appRevision string) Context {
return &templateContext{
name: name,
appName: appName,
appRevision: appRevision,
configs: []map[string]string{},
auxiliaries: []Auxiliary{},
namespace: namespace,
}
}
@@ -106,6 +128,7 @@ func (ctx *templateContext) BaseContextFile() string {
buff += fmt.Sprintf(ContextName+": \"%s\"\n", ctx.name)
buff += fmt.Sprintf(ContextAppName+": \"%s\"\n", ctx.appName)
buff += fmt.Sprintf(ContextAppRevision+": \"%s\"\n", ctx.appRevision)
buff += fmt.Sprintf(ContextNamespace+": \"%s\"\n", ctx.namespace)
if ctx.base != nil {
buff += fmt.Sprintf(OutputFieldName+": %s\n", structMarshal(ctx.base.String()))
@@ -126,11 +149,37 @@ func (ctx *templateContext) BaseContextFile() string {
buff += ConfigFieldName + ": " + string(bt)
}
if len(ctx.requiredSecrets) > 0 {
for _, s := range ctx.requiredSecrets {
data, _ := json.Marshal(s.Data)
buff += s.ContextName + ":" + string(data) + "\n"
}
}
if ctx.outputSecretName != "" {
buff += fmt.Sprintf("%s:\"%s\"", OutputSecretName, ctx.outputSecretName)
}
return fmt.Sprintf("context: %s", structMarshal(buff))
}
func (ctx *templateContext) BaseContextLabels() map[string]string {
// ExtendedContextFile return cue format string of templateContext and extended secret context
func (ctx *templateContext) ExtendedContextFile() string {
context := ctx.BaseContextFile()
var bareSecret string
if len(ctx.requiredSecrets) > 0 {
for _, s := range ctx.requiredSecrets {
data, _ := json.Marshal(s.Data)
bareSecret += s.ContextName + ":" + string(data) + "\n"
}
}
if bareSecret != "" {
return context + "\n" + bareSecret
}
return context
}
func (ctx *templateContext) BaseContextLabels() map[string]string {
return map[string]string{
// appName is oam.LabelAppName
ContextAppName: ctx.appName,
@@ -146,6 +195,16 @@ func (ctx *templateContext) Output() (model.Instance, []Auxiliary) {
return ctx.base, ctx.auxiliaries
}
// InsertSecrets will add cloud resource secret stuff to context
func (ctx *templateContext) InsertSecrets(outputSecretName string, requiredSecrets []RequiredSecrets) {
if outputSecretName != "" {
ctx.outputSecretName = outputSecretName
}
if requiredSecrets != nil {
ctx.requiredSecrets = requiredSecrets
}
}
func structMarshal(v string) string {
skip := false
v = strings.TrimFunc(v, func(r rune) bool {

17
pkg/dsl/process/handle_test.go
View File

@@ -63,12 +63,17 @@ image: "myserver"
Ins: svcIns,
Name: "service",
}
targetRequiredSecrets := []RequiredSecrets{{
ContextName: "conn1",
Data: map[string]interface{}{"password": "123"},
}}
ctx := NewContext("mycomp", "myapp", "myapp-v1")
ctx := NewContext("myns", "mycomp", "myapp", "myapp-v1")
ctx.InsertSecrets("db-conn", targetRequiredSecrets)
ctx.SetBase(base)
ctx.AppendAuxiliaries(svcAux)
ctxInst, err := r.Compile("-", ctx.BaseContextFile())
ctxInst, err := r.Compile("-", ctx.ExtendedContextFile())
if err != nil {
t.Error(err)
return
@@ -93,4 +98,12 @@ image: "myserver"
outputsJs, err := ctxInst.Lookup("context", OutputsFieldName, "service").MarshalJSON()
assert.Equal(t, nil, err)
assert.Equal(t, "{\"apiVersion\":\"v1\",\"kind\":\"ConfigMap\"}", string(outputsJs))
ns, err := ctxInst.Lookup("context", ContextNamespace).String()
assert.Equal(t, nil, err)
assert.Equal(t, "myns", ns)
requiredSecrets, err := ctxInst.Lookup("context", "conn1").MarshalJSON()
assert.Equal(t, nil, err)
assert.Equal(t, "{\"password\":\"123\"}", string(requiredSecrets))
}

4
pkg/dsl/task/process_test.go
View File

@@ -27,6 +27,8 @@ import (
"cuelang.org/go/cue"
cueJson "cuelang.org/go/pkg/encoding/json"
"github.com/bmizerany/assert"
mycue "github.com/oam-dev/kubevela/pkg/cue"
)
const TaskTemplate = `
@@ -69,7 +71,7 @@ func TestProcess(t *testing.T) {
}
taskTemplate, _ = taskTemplate.Fill(map[string]interface{}{
"serviceURL": "http://127.0.0.1:8090/api/v1/token?val=test-token",
}, "parameter")
}, mycue.ParameterTag)
inst, err := Process(taskTemplate)
if err != nil {

2
pkg/utils/common/common.go
View File

@@ -110,7 +110,7 @@ func GetCUEParameterValue(cueStr string) (cue.Value, error) {
var found bool
for i := 0; i < tempStruct.Len(); i++ {
paraDef = tempStruct.Field(i)
if paraDef.Name == "parameter" {
if paraDef.Name == mycue.ParameterTag {
found = true
break
}

2
references/appfile/addon.go
View File

@@ -198,7 +198,7 @@ func generateSecretFromTerraformOutput(k8sClient client.Client, outputList []str
// getTerraformJSONFiles gets Terraform JSON files or modules from workload
func getTerraformJSONFiles(k8sClient client.Client, wl *appfile.Workload, applicationName, revisionName string, namespace string) ([]byte, error) {
pCtx, err := appfile.PrepareProcessContext(k8sClient, wl, applicationName, namespace, revisionName)
pCtx, err := appfile.PrepareProcessContext(k8sClient, wl, applicationName, revisionName, namespace)
if err != nil {
return nil, err
}