Compare commits

...

21 Commits

Author SHA1 Message Date
github-actions[bot]
8ef2513b2e fix: fix --cluster when addon enable (#5343)
Signed-off-by: zhaowei.wang <zhaowei.wang@metabit-trading.com>
(cherry picked from commit 021ca69cfd)

Co-authored-by: zhaowei.wang <zhaowei.wang@metabit-trading.com>
2023-01-13 17:09:26 +08:00
github-actions[bot]
657e3b1bde Fix: optimize skip reconcile and expose error if the traits patch an invalid workload like terraform (#5342)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
(cherry picked from commit 3730291eff)

Co-authored-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2023-01-13 17:08:52 +08:00
github-actions[bot]
004e2a814d Feat: upgrade the workflow version to v0.4.0 (#5337)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
(cherry picked from commit e0b8c9e4c9)

Co-authored-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2023-01-13 16:57:03 +08:00
github-actions[bot]
6dcba5ef11 [Backport release-1.7] Fix: conflict while using gc policy and shared-resource policy concurrently (#5333)
* Fix: conflict while using gc policy and shared-resource policy concurrently

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
(cherry picked from commit f8239be21e)

* Fix: github ci

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
(cherry picked from commit 72e54e5f90)

Co-authored-by: Somefive <yd219913@alibaba-inc.com>
2023-01-13 15:55:05 +08:00
github-actions[bot]
fd39804dc9 Fix: maintain compatibility with old project data (#5331)
Signed-off-by: barnettZQG <barnett.zqg@gmail.com>
(cherry picked from commit fef55b9b1b)

Co-authored-by: barnettZQG <barnett.zqg@gmail.com>
2023-01-13 13:59:35 +08:00
github-actions[bot]
ce57fbb752 Feat: need one Trait to set Rollout strategy of Workload (#5327)
Signed-off-by: StevenLeiZhang <zhangleiic@163.com>
(cherry picked from commit 5d00b2ac73)

Co-authored-by: StevenLeiZhang <zhangleiic@163.com>
2023-01-12 17:19:17 +08:00
github-actions[bot]
80b2b2c2d3 velaql support indexing into exported array field (#5323)
Signed-off-by: hnd4r7 <307365651@qq.com>
(cherry picked from commit 441d8f0a66)

Co-authored-by: hnd4r7 <307365651@qq.com>
2023-01-12 10:13:57 +08:00
github-actions[bot]
49d97e5c2b Fix typo in the long cli description of vela system command (#5322)
Signed-off-by: Girish Ramnani <girishramnani95@gmail.com>
(cherry picked from commit 273b91c41f)

Co-authored-by: Girish <girishramnani95@gmail.com>
2023-01-12 10:10:15 +08:00
github-actions[bot]
1ee5915546 Fix: the developer user can't load the definition (#5319)
Signed-off-by: barnettZQG <barnett.zqg@gmail.com>
(cherry picked from commit 78c9a1a370)

Co-authored-by: barnettZQG <barnett.zqg@gmail.com>
2023-01-11 18:28:36 +08:00
github-actions[bot]
3f6b38cc7f small optimzie for addon (#5318)
Signed-off-by: 楚岳 <wangyike.wyk@alibaba-inc.com>

name the compoennt

Signed-off-by: 楚岳 <wangyike.wyk@alibaba-inc.com>
(cherry picked from commit 3a020041d9)

Co-authored-by: 楚岳 <wangyike.wyk@alibaba-inc.com>
2023-01-11 16:33:16 +08:00
Jianbo Sun
c2c7ab91f9 Feat: vela dry-run render results should be affected by override policy and deploy workflowstep (#4815) (#5314)
* [Feature] vela dry-run render results should be affected by override policy and deploy workflowstep

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

* multiple input files support; policy,workflow support; new flag: merge orphan policy or workflow

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

* add more tests

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

* fix comment issues

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

* add tests

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

* fix e2e

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

* fix tests

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

Signed-off-by: cezhang <c1zhang.dev@gmail.com>
Co-authored-by: cezhang <c1zhang.dev@gmail.com>
2023-01-11 15:39:45 +08:00
github-actions[bot]
866ffb9689 Chore: vela delete doc (#5315)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
(cherry picked from commit ce79ab1691)

Co-authored-by: Somefive <yd219913@alibaba-inc.com>
2023-01-11 15:08:08 +08:00
github-actions[bot]
61afb366ee Fix: fix vela debug cli to find id for step (#5313)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
(cherry picked from commit 2f77353cbe)

Co-authored-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2023-01-11 14:57:02 +08:00
github-actions[bot]
a2a8d73a58 Fix: don't return err if subresource type is not found when listing application resources (#5312)
Signed-off-by: hnd4r7 <307365651@qq.com>
(cherry picked from commit 58a150d1d4)

Co-authored-by: hnd4r7 <307365651@qq.com>
2023-01-11 14:56:22 +08:00
github-actions[bot]
fc3b428788 fix: errorMsg when uninstall vela (#5310)
Signed-off-by: bitliu <bitliu@tencent.com>
(cherry picked from commit 771e0b5429)

Co-authored-by: bitliu <bitliu@tencent.com>
2023-01-11 14:49:14 +08:00
github-actions[bot]
500dc52b34 [Backport release-1.7] Fix: create a config with the same name reported an incorrect error (#5307)
* Fix: create a config with the same name reported an incorrect error

Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
(cherry picked from commit 3c1759896b)

* Fix: create a config with the same name reported an incorrect error

Signed-off-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
(cherry picked from commit 0aa456642b)

Co-authored-by: wuzhongjian <wuzhongjian_yewu@cmss.chinamobile.com>
2023-01-11 14:25:13 +08:00
github-actions[bot]
789aa38476 [Backport release-1.7] Feat: enhance the application synchronizer (#5305)
* Feat: enhance the application synchronizer

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>
(cherry picked from commit e5d112b04a)

* Fix: e2e test case

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>
(cherry picked from commit 940ffb30f5)

* Fix: the unit test case

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>
(cherry picked from commit c3e896fbc1)

Co-authored-by: barnettZQG <barnett.zqg@gmail.com>
2023-01-11 14:24:25 +08:00
github-actions[bot]
17adf35717 Fix: Index structure map[string]string,Mongo resulting in inconsistent results obtained by filtering non-string type by index. (#5303)
Signed-off-by: old.prince <di7zhang@gmail.com>
(cherry picked from commit a763d3da54)

Co-authored-by: old.prince <di7zhang@gmail.com>
2023-01-11 13:06:15 +08:00
github-actions[bot]
586f266798 [Backport release-1.7] Fix: more explicit error when addon package hasn't a metadata.yaml (#5302)
* more explicit error when addon package hasn't a metadata.yaml

Signed-off-by: 楚岳 <wangyike.wyk@alibaba-inc.com>

fix checkdiff

Signed-off-by: 楚岳 <wangyike.wyk@alibaba-inc.com>
(cherry picked from commit 6953fe872d)

* fix commets

Signed-off-by: 楚岳 <wangyike.wyk@alibaba-inc.com>
(cherry picked from commit 7cacb50976)

* fix test

Signed-off-by: 楚岳 <wangyike.wyk@alibaba-inc.com>
(cherry picked from commit 4d35cd58f9)

Co-authored-by: 楚岳 <wangyike.wyk@alibaba-inc.com>
2023-01-11 11:53:00 +08:00
github-actions[bot]
b8dcbe4964 Fix: Delete appplication fails if status.workflow.endTime not specified. (#5297)
Error Details:
E0106 08:12:02.807341       1 controller.go:317] controller/application "msg"="Reconciler error" "error"="Application.core.oam.dev \"test\" is invalid: status.workflow.endTime: Invalid value: \"null\": status.workflow.endTime in body must be of type string: \"null\"" "name"="test" "namespace"="test" "reconciler group"="core.oam.dev" "reconciler kind"="Application"

If the workflow is not completed, the endtime should be null, and the deletion of the application will fail

Signed-off-by: old.prince <di7zhang@gmail.com>
(cherry picked from commit 16b6a02018)

Co-authored-by: old.prince <di7zhang@gmail.com>
2023-01-10 15:16:26 +08:00
github-actions[bot]
778579c79b Test: let addon helper tests use local helm server (#5290)
Signed-off-by: Charlie Chiang <charlie_c_0129@outlook.com>
(cherry picked from commit 44eaa1d004)

Co-authored-by: Charlie Chiang <charlie_c_0129@outlook.com>
2023-01-09 13:30:04 +08:00
110 changed files with 2063 additions and 386 deletions

View File

@@ -64,6 +64,8 @@ jobs:
- name: Install ginkgo
run: |
sudo sed -i 's/azure\.//' /etc/apt/sources.list
sudo apt-get update
sudo apt-get install -y golang-ginkgo-dev
- name: Start MongoDB

View File

@@ -61,6 +61,8 @@ jobs:
- name: Install ginkgo
run: |
sudo sed -i 's/azure\.//' /etc/apt/sources.list
sudo apt-get update
sudo apt-get install -y golang-ginkgo-dev
- name: Setup K3d

View File

@@ -336,7 +336,8 @@ type WorkflowStatus struct {
Steps []workflowv1alpha1.WorkflowStepStatus `json:"steps,omitempty"`
StartTime metav1.Time `json:"startTime,omitempty"`
EndTime metav1.Time `json:"endTime,omitempty"`
// +nullable
EndTime metav1.Time `json:"endTime,omitempty"`
}
// DefinitionType describes the type of DefinitionRevision.

View File

@@ -848,6 +848,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -2806,6 +2807,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -4942,6 +4944,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean

View File

@@ -774,6 +774,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -1535,6 +1536,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean

View File

@@ -0,0 +1,77 @@
# Code generated by KubeVela templates. DO NOT EDIT. Please edit the original cue file.
# Definition source cue file: vela-templates/definitions/internal/k8s-update-strategy.cue
apiVersion: core.oam.dev/v1beta1
kind: TraitDefinition
metadata:
annotations:
definition.oam.dev/alias: ""
definition.oam.dev/description: Set k8s update strategy for Deployment/DaemonSet/StatefulSet
name: k8s-update-strategy
namespace: {{ include "systemDefinitionNamespace" . }}
spec:
appliesToWorkloads:
- deployments.apps
- statefulsets.apps
- daemonsets.apps
conflictsWith: []
podDisruptive: false
schematic:
cue:
template: |
patch: spec: {
if parameter.targetKind == "Deployment" && parameter.strategy.type != "OnDelete" {
// +patchStrategy=retainKeys
strategy: {
type: parameter.strategy.type
if parameter.strategy.type == "RollingUpdate" {
rollingUpdate: {
maxSurge: parameter.strategy.rollingStrategy.maxSurge
maxUnavailable: parameter.strategy.rollingStrategy.maxUnavailable
}
}
}
}
if parameter.targetKind == "StatefulSet" && parameter.strategy.type != "Recreate" {
// +patchStrategy=retainKeys
updateStrategy: {
type: parameter.strategy.type
if parameter.strategy.type == "RollingUpdate" {
rollingUpdate: partition: parameter.strategy.rollingStrategy.partition
}
}
}
if parameter.targetKind == "DaemonSet" && parameter.strategy.type != "Recreate" {
// +patchStrategy=retainKeys
updateStrategy: {
type: parameter.strategy.type
if parameter.strategy.type == "RollingUpdate" {
rollingUpdate: {
maxSurge: parameter.strategy.rollingStrategy.maxSurge
maxUnavailable: parameter.strategy.rollingStrategy.maxUnavailable
}
}
}
}
}
parameter: {
// +usage=Specify the apiVersion of target
targetAPIVersion: *"apps/v1" | string
// +usage=Specify the kind of target
targetKind: *"Deployment" | "StatefulSet" | "DaemonSet"
// +usage=Specify the strategy of update
strategy: {
// +usage=Specify the strategy type
type: *"RollingUpdate" | "Recreate" | "OnDelete"
// +usage=Specify the parameters of rollong update strategy
rollingStrategy?: {
maxSurge: *"25%" | string
maxUnavailable: *"25%" | string
partition: *0 | int
}
}
}
workloadRefPath: ""

View File

@@ -848,6 +848,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -2806,6 +2807,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -4942,6 +4944,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean

View File

@@ -774,6 +774,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -1535,6 +1536,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean

View File

@@ -0,0 +1,77 @@
# Code generated by KubeVela templates. DO NOT EDIT. Please edit the original cue file.
# Definition source cue file: vela-templates/definitions/internal/k8s-update-strategy.cue
apiVersion: core.oam.dev/v1beta1
kind: TraitDefinition
metadata:
annotations:
definition.oam.dev/alias: ""
definition.oam.dev/description: Set k8s update strategy for Deployment/DaemonSet/StatefulSet
name: k8s-update-strategy
namespace: {{ include "systemDefinitionNamespace" . }}
spec:
appliesToWorkloads:
- deployments.apps
- statefulsets.apps
- daemonsets.apps
conflictsWith: []
podDisruptive: false
schematic:
cue:
template: |
patch: spec: {
if parameter.targetKind == "Deployment" && parameter.strategy.type != "OnDelete" {
// +patchStrategy=retainKeys
strategy: {
type: parameter.strategy.type
if parameter.strategy.type == "RollingUpdate" {
rollingUpdate: {
maxSurge: parameter.strategy.rollingStrategy.maxSurge
maxUnavailable: parameter.strategy.rollingStrategy.maxUnavailable
}
}
}
}
if parameter.targetKind == "StatefulSet" && parameter.strategy.type != "Recreate" {
// +patchStrategy=retainKeys
updateStrategy: {
type: parameter.strategy.type
if parameter.strategy.type == "RollingUpdate" {
rollingUpdate: partition: parameter.strategy.rollingStrategy.partition
}
}
}
if parameter.targetKind == "DaemonSet" && parameter.strategy.type != "Recreate" {
// +patchStrategy=retainKeys
updateStrategy: {
type: parameter.strategy.type
if parameter.strategy.type == "RollingUpdate" {
rollingUpdate: {
maxSurge: parameter.strategy.rollingStrategy.maxSurge
maxUnavailable: parameter.strategy.rollingStrategy.maxUnavailable
}
}
}
}
}
parameter: {
// +usage=Specify the apiVersion of target
targetAPIVersion: *"apps/v1" | string
// +usage=Specify the kind of target
targetKind: *"Deployment" | "StatefulSet" | "DaemonSet"
// +usage=Specify the strategy of update
strategy: {
// +usage=Specify the strategy type
type: *"RollingUpdate" | "Recreate" | "OnDelete"
// +usage=Specify the parameters of rollong update strategy
rollingStrategy?: {
maxSurge: *"25%" | string
maxUnavailable: *"25%" | string
partition: *0 | int
}
}
}
workloadRefPath: ""

2
go.mod
View File

@@ -57,7 +57,7 @@ require (
github.com/koding/websocketproxy v0.0.0-20181220232114-7ed82d81a28c
github.com/kubevela/pkg v0.0.0-20230105054759-263dc191bf51
github.com/kubevela/prism v1.7.0-alpha.1
github.com/kubevela/workflow v0.3.6-0.20221230102636-6ae0c5cbc40f
github.com/kubevela/workflow v0.4.0
github.com/kyokomi/emoji v2.2.4+incompatible
github.com/mitchellh/hashstructure/v2 v2.0.1
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd

4
go.sum
View File

@@ -1289,8 +1289,8 @@ github.com/kubevela/pkg v0.0.0-20230105054759-263dc191bf51 h1:xrcNNaAjqC6tr1leSY
github.com/kubevela/pkg v0.0.0-20230105054759-263dc191bf51/go.mod h1:ZRnxY/gOcg/8FilZA+eYr+rtVXb1ijT5HFTe8zrv9zo=
github.com/kubevela/prism v1.7.0-alpha.1 h1:oeZFn1Oy6gxSSFzMTfsWjLOCKaaooMVm1JGNK4j4Mlo=
github.com/kubevela/prism v1.7.0-alpha.1/go.mod h1:AJSDfdA+RkRSnWx3xEcogbmOTpX+l7RSIwqVHxwUtaI=
github.com/kubevela/workflow v0.3.6-0.20221230102636-6ae0c5cbc40f h1:7EZWIfcTOgMlLgHkdDlf++hSjBTulfr4DYhZjeQbiJI=
github.com/kubevela/workflow v0.3.6-0.20221230102636-6ae0c5cbc40f/go.mod h1:AX/WL3G/YBkpmNpA/SKKm9M3Y0T9y95gZA8mFWylkyM=
github.com/kubevela/workflow v0.4.0 h1:Zzb9wPOvLUGEArdnC6EsfNM5j3VNJj0/v2iDKq1JT3k=
github.com/kubevela/workflow v0.4.0/go.mod h1:AX/WL3G/YBkpmNpA/SKKm9M3Y0T9y95gZA8mFWylkyM=
github.com/kulti/thelper v0.4.0/go.mod h1:vMu2Cizjy/grP+jmsvOFDx1kYP6+PD1lqg4Yu5exl2U=
github.com/kunwardeep/paralleltest v1.0.3/go.mod h1:vLydzomDFpk7yu5UX02RmP0H8QfRPOV/oFhWN85Mjb4=
github.com/kylelemons/godebug v0.0.0-20160406211939-eadb3ce320cb/go.mod h1:B69LEHPfb2qLo0BaaOLcbitczOKLWTsrBG9LczfCD4k=

View File

@@ -848,6 +848,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -2806,6 +2807,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -4942,6 +4944,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean

View File

@@ -774,6 +774,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean
@@ -1535,6 +1536,7 @@ spec:
x-kubernetes-map-type: atomic
endTime:
format: date-time
nullable: true
type: string
finished:
type: boolean

View File

@@ -911,7 +911,6 @@ func NewAddonInstaller(ctx context.Context, cli client.Client, discoveryClient *
func (h *Installer) enableAddon(addon *InstallPackage) (string, error) {
var err error
h.addon = addon
if !h.skipVersionValidate {
err = checkAddonVersionMeetRequired(h.ctx, addon.SystemRequirements, h.cli, h.dc)
if err != nil {
@@ -1076,6 +1075,8 @@ func (h *Installer) checkDependency(addon *InstallPackage) ([]string, error) {
// createOrUpdate will return true if updated
func (h *Installer) createOrUpdate(app *v1beta1.Application) (bool, error) {
// Set the publish version for the addon application
oam.SetPublishVersion(app, apiutils.GenerateVersion("addon"))
var existApp v1beta1.Application
err := h.cli.Get(h.ctx, client.ObjectKey{Name: app.Name, Namespace: app.Namespace}, &existApp)
if apierrors.IsNotFound(err) {
@@ -1087,8 +1088,6 @@ func (h *Installer) createOrUpdate(app *v1beta1.Application) (bool, error) {
existApp.Spec = app.Spec
existApp.Labels = app.Labels
existApp.Annotations = app.Annotations
// Set the publish version for the addon application
oam.SetPublishVersion(&existApp, apiutils.GenerateVersion("addon"))
err = h.cli.Update(h.ctx, &existApp)
if err != nil {
klog.Errorf("fail to create application: %v", err)
@@ -1211,6 +1210,9 @@ func (h *Installer) dispatchAddonResource(addon *InstallPackage) error {
}
func (h *Installer) renderNotes(addon *InstallPackage) (string, error) {
if len(addon.Notes.Data) == 0 {
return "", nil
}
r := addonCueTemplateRender{
addon: addon,
inputArgs: h.args,

View File

@@ -58,6 +58,9 @@ func EnableAddon(ctx context.Context, name string, version string, cli client.Cl
if err != nil {
return "", err
}
if err := validateAddonPackage(pkg); err != nil {
return "", errors.Wrap(err, fmt.Sprintf("failed to enable addon: %s", name))
}
return h.enableAddon(pkg)
}
@@ -106,6 +109,9 @@ func EnableAddonByLocalDir(ctx context.Context, name string, dir string, cli cli
if err != nil {
return "", err
}
if err := validateAddonPackage(pkg); err != nil {
return "", errors.Wrap(err, fmt.Sprintf("failed to enable addon by local dir: %s", dir))
}
h := NewAddonInstaller(ctx, cli, dc, applicator, config, &Registry{Name: LocalAddonRegistryName}, args, nil, nil, opts...)
needEnableAddonNames, err := h.checkDependency(pkg)
if err != nil {

View File

@@ -17,9 +17,12 @@ limitations under the License.
package addon
import (
"bytes"
"context"
"errors"
"net/http"
"net/http/httptest"
"os"
"strings"
v1 "k8s.io/api/core/v1"
@@ -29,6 +32,38 @@ import (
. "github.com/onsi/gomega"
)
func setupMockServer() *httptest.Server {
var listenURL string
s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
fileList := []string{
"index.yaml",
"fluxcd-test-version-1.0.0.tgz",
"fluxcd-test-version-2.0.0.tgz",
"vela-workflow-v0.3.5.tgz",
"foo-v1.0.0.tgz",
"bar-v1.0.0.tgz",
"bar-v2.0.0.tgz",
"mock-be-dep-addon-v1.0.0.tgz",
}
for _, f := range fileList {
if strings.Contains(req.URL.Path, f) {
file, err := os.ReadFile("../../e2e/addon/mock/testrepo/helm-repo/" + f)
if err != nil {
_, _ = w.Write([]byte(err.Error()))
}
if f == "index.yaml" {
// in index.yaml, url is hardcoded to 127.0.0.1:9098,
// so we need to replace it with the real random listen url
file = bytes.ReplaceAll(file, []byte("http://127.0.0.1:9098"), []byte(listenURL))
}
_, _ = w.Write(file)
}
}
}))
listenURL = s.URL
return s
}
var _ = Describe("test FindAddonPackagesDetailFromRegistry", func() {
Describe("when no registry is added, no matter what you do, it will just return error", func() {
Context("when empty addonNames and registryNames is supplied", func() {
@@ -50,12 +85,15 @@ var _ = Describe("test FindAddonPackagesDetailFromRegistry", func() {
})
Describe("one versioned registry is added", func() {
var s *httptest.Server
BeforeEach(func() {
// Prepare KubeVela registry
s = setupMockServer()
// Prepare registry
reg := &Registry{
Name: "KubeVela",
Name: "addon_helper_test",
Helm: &HelmSource{
URL: "https://addons.kubevela.net",
URL: s.URL,
},
}
ds := NewRegistryDataStore(k8sClient)
@@ -63,38 +101,36 @@ var _ = Describe("test FindAddonPackagesDetailFromRegistry", func() {
})
AfterEach(func() {
// Clean up KubeVela registry
// Clean up registry
ds := NewRegistryDataStore(k8sClient)
Expect(ds.DeleteRegistry(context.Background(), "KubeVela")).To(Succeed())
Expect(ds.DeleteRegistry(context.Background(), "addon_helper_test")).To(Succeed())
s.Close()
})
Context("when empty addonNames and registryNames is supplied", func() {
It("should return error, empty addonNames are not allowed", func() {
_, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{}, []string{"KubeVela"})
_, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{}, []string{"addon_helper_test"})
Expect(err).To(HaveOccurred())
})
It("should return error, empty addonNames are not allowed", func() {
_, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, nil, []string{"KubeVela"})
_, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, nil, []string{"addon_helper_test"})
Expect(err).To(HaveOccurred())
})
})
Context("one existing addon name provided", func() {
It("should return one valid result, matching all registries", func() {
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"velaux"}, nil)
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"foo"}, nil)
Expect(err).To(Succeed())
Expect(res).To(HaveLen(1))
Expect(res[0].Name).To(Equal("velaux"))
Expect(res[0].InstallPackage).ToNot(BeNil())
Expect(res[0].APISchema).ToNot(BeNil())
Expect(res[0].Name).To(Equal("foo"))
})
It("should return one valid result, matching one registry", func() {
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"velaux"}, []string{"KubeVela"})
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"foo"}, []string{"addon_helper_test"})
Expect(err).To(Succeed())
Expect(res).To(HaveLen(1))
Expect(res[0].Name).To(Equal("velaux"))
Expect(res[0].InstallPackage).ToNot(BeNil())
Expect(res[0].APISchema).ToNot(BeNil())
Expect(res[0].Name).To(Equal("foo"))
})
})
@@ -108,26 +144,20 @@ var _ = Describe("test FindAddonPackagesDetailFromRegistry", func() {
Context("two existing addon names provided", func() {
It("should return two valid result", func() {
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"velaux", "traefik"}, nil)
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"foo", "bar"}, nil)
Expect(err).To(Succeed())
Expect(res).To(HaveLen(2))
Expect(res[0].Name).To(Equal("velaux"))
Expect(res[0].InstallPackage).ToNot(BeNil())
Expect(res[0].APISchema).ToNot(BeNil())
Expect(res[1].Name).To(Equal("traefik"))
Expect(res[1].InstallPackage).ToNot(BeNil())
Expect(res[1].APISchema).ToNot(BeNil())
Expect(res[0].Name).To(Equal("foo"))
Expect(res[1].Name).To(Equal("bar"))
})
})
Context("one existing addon name and one non-existent addon name provided", func() {
It("should return only one valid result", func() {
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"velaux", "non-existent-addon"}, nil)
res, err := FindAddonPackagesDetailFromRegistry(context.Background(), k8sClient, []string{"foo", "non-existent-addon"}, nil)
Expect(err).To(Succeed())
Expect(res).To(HaveLen(1))
Expect(res[0].Name).To(Equal("velaux"))
Expect(res[0].InstallPackage).ToNot(BeNil())
Expect(res[0].APISchema).ToNot(BeNil())
Expect(res[0].Name).To(Equal("foo"))
})
})
})

View File

@@ -196,9 +196,13 @@ func (cmd *InitCmd) createHelmComponent() error {
tmpl := helmComponentTmpl{}
tmpl.Type = "helm"
tmpl.Properties.RepoType = "helm"
if strings.HasPrefix(cmd.HelmRepoURL, "oci") {
tmpl.Properties.RepoType = "oci"
}
tmpl.Properties.URL = cmd.HelmRepoURL
tmpl.Properties.Chart = cmd.HelmChartName
tmpl.Properties.Version = cmd.HelmChartVersion
tmpl.Name = "addon-" + cmd.AddonName
str, err := toCUEResourceString(tmpl)
if err != nil {
@@ -383,6 +387,7 @@ func (cmd *InitCmd) writeFiles() error {
// helmComponentTmpl is a template for a helm component .cue in an addon
type helmComponentTmpl struct {
Name string `json:"name"`
Type string `json:"type"`
Properties struct {
RepoType string `json:"repoType"`

View File

@@ -21,6 +21,7 @@ import (
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
"github.com/pkg/errors"
@@ -503,6 +504,19 @@ func checkBondComponentExist(u unstructured.Unstructured, app v1beta1.Applicatio
return false
}
func validateAddonPackage(addonPkg *InstallPackage) error {
if reflect.DeepEqual(addonPkg.Meta, Meta{}) {
return fmt.Errorf("the addon package doesn't have `metadata.yaml`")
}
if addonPkg.Name == "" {
return fmt.Errorf("`matadata.yaml` must define the name of addon")
}
if addonPkg.Version == "" {
return fmt.Errorf("`matadata.yaml` must define the version of addon")
}
return nil
}
// FilterDependencyRegistries will return all registries besides the target registry itself
func FilterDependencyRegistries(i int, rs []Registry) []Registry {
if i >= len(rs) {

View File

@@ -17,9 +17,11 @@ limitations under the License.
package addon
import (
"fmt"
"net/http/httptest"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
@@ -402,6 +404,30 @@ func TestFilterDependencyRegistries(t *testing.T) {
}
}
func TestCheckAddonPackageValid(t *testing.T) {
testCases := []struct {
testCase Meta
err error
}{{
testCase: Meta{},
err: fmt.Errorf("the addon package doesn't have `metadata.yaml`"),
}, {
testCase: Meta{Version: "v1.4.0"},
err: fmt.Errorf("`matadata.yaml` must define the name of addon"),
}, {
testCase: Meta{Name: "test-addon"},
err: fmt.Errorf("`matadata.yaml` must define the version of addon"),
}, {
testCase: Meta{Name: "test-addon", Version: "1.4.5"},
err: nil,
},
}
for _, testCase := range testCases {
err := validateAddonPackage(&InstallPackage{Meta: testCase.testCase})
assert.Equal(t, reflect.DeepEqual(err, testCase.err), true)
}
}
const (
compDefYaml = `
apiVersion: core.oam.dev/v1beta1

View File

@@ -59,8 +59,8 @@ func (a *Application) PrimaryKey() string {
}
// Index return custom index
func (a *Application) Index() map[string]string {
index := make(map[string]string)
func (a *Application) Index() map[string]interface{} {
index := make(map[string]interface{})
if a.Name != "" {
index["name"] = a.Name
}
@@ -154,8 +154,8 @@ func (a *ApplicationComponent) PrimaryKey() string {
}
// Index return custom index
func (a *ApplicationComponent) Index() map[string]string {
index := make(map[string]string)
func (a *ApplicationComponent) Index() map[string]interface{} {
index := make(map[string]interface{})
if a.Name != "" {
index["name"] = a.Name
}
@@ -202,8 +202,8 @@ func (a *ApplicationPolicy) PrimaryKey() string {
}
// Index return custom index
func (a *ApplicationPolicy) Index() map[string]string {
index := make(map[string]string)
func (a *ApplicationPolicy) Index() map[string]interface{} {
index := make(map[string]interface{})
if a.Name != "" {
index["name"] = a.Name
}
@@ -348,8 +348,8 @@ func (a *ApplicationRevision) PrimaryKey() string {
}
// Index return custom index
func (a *ApplicationRevision) Index() map[string]string {
index := make(map[string]string)
func (a *ApplicationRevision) Index() map[string]interface{} {
index := make(map[string]interface{})
if a.Version != "" {
index["version"] = a.Version
}
@@ -434,8 +434,8 @@ func (w *ApplicationTrigger) PrimaryKey() string {
}
// Index return custom index
func (w *ApplicationTrigger) Index() map[string]string {
index := make(map[string]string)
func (w *ApplicationTrigger) Index() map[string]interface{} {
index := make(map[string]interface{})
if w.AppPrimaryKey != "" {
index["appPrimaryKey"] = w.AppPrimaryKey
}

View File

@@ -93,8 +93,8 @@ func (c *Cluster) PrimaryKey() string {
}
// Index set to nil for list
func (c *Cluster) Index() map[string]string {
index := make(map[string]string)
func (c *Cluster) Index() map[string]interface{} {
index := make(map[string]interface{})
if c.Name != "" {
index["name"] = c.Name
}

View File

@@ -53,8 +53,8 @@ func (p *Env) PrimaryKey() string {
}
// Index return custom index
func (p *Env) Index() map[string]string {
index := make(map[string]string)
func (p *Env) Index() map[string]interface{} {
index := make(map[string]interface{})
if p.Name != "" {
index["name"] = p.Name
}

View File

@@ -62,8 +62,8 @@ func (e *EnvBinding) PrimaryKey() string {
}
// Index return custom index
func (e *EnvBinding) Index() map[string]string {
index := make(map[string]string)
func (e *EnvBinding) Index() map[string]interface{} {
index := make(map[string]interface{})
if e.Name != "" {
index["name"] = e.Name
}

View File

@@ -61,8 +61,8 @@ func (p Pipeline) ShortTableName() string {
}
// Index return custom index
func (p Pipeline) Index() map[string]string {
var index = make(map[string]string)
func (p Pipeline) Index() map[string]interface{} {
var index = make(map[string]interface{})
if p.Project != "" {
index["project"] = p.Project
}
@@ -102,8 +102,8 @@ func (c *PipelineContext) PrimaryKey() string {
}
// Index return custom index
func (c *PipelineContext) Index() map[string]string {
index := make(map[string]string)
func (c *PipelineContext) Index() map[string]interface{} {
index := make(map[string]interface{})
if c.ProjectName != "" {
index["project_name"] = c.ProjectName
}

View File

@@ -54,8 +54,8 @@ func (p *Project) PrimaryKey() string {
}
// Index return custom index
func (p *Project) Index() map[string]string {
index := make(map[string]string)
func (p *Project) Index() map[string]interface{} {
index := make(map[string]interface{})
if p.Name != "" {
index["name"] = p.Name
}

View File

@@ -146,8 +146,8 @@ func (u *SystemInfo) PrimaryKey() string {
}
// Index return custom index
func (u *SystemInfo) Index() map[string]string {
index := make(map[string]string)
func (u *SystemInfo) Index() map[string]interface{} {
index := make(map[string]interface{})
if u.InstallID != "" {
index["installID"] = u.InstallID
}

View File

@@ -48,8 +48,8 @@ func (d *Target) PrimaryKey() string {
}
// Index return custom index
func (d *Target) Index() map[string]string {
index := make(map[string]string)
func (d *Target) Index() map[string]interface{} {
index := make(map[string]interface{})
if d.Name != "" {
index["name"] = d.Name
}

View File

@@ -67,8 +67,8 @@ func (u *User) PrimaryKey() string {
}
// Index return custom index
func (u *User) Index() map[string]string {
index := make(map[string]string)
func (u *User) Index() map[string]interface{} {
index := make(map[string]interface{})
if u.Name != "" {
index["name"] = u.Name
}
@@ -106,8 +106,8 @@ func (u *ProjectUser) PrimaryKey() string {
}
// Index return custom index
func (u *ProjectUser) Index() map[string]string {
index := make(map[string]string)
func (u *ProjectUser) Index() map[string]interface{} {
index := make(map[string]interface{})
if u.Username != "" {
index["username"] = u.Username
}
@@ -177,8 +177,8 @@ func (r *Role) PrimaryKey() string {
}
// Index return custom index
func (r *Role) Index() map[string]string {
index := make(map[string]string)
func (r *Role) Index() map[string]interface{} {
index := make(map[string]interface{})
if r.Name != "" {
index["name"] = r.Name
}
@@ -207,8 +207,8 @@ func (p *Permission) PrimaryKey() string {
}
// Index return custom index
func (p *Permission) Index() map[string]string {
index := make(map[string]string)
func (p *Permission) Index() map[string]interface{} {
index := make(map[string]interface{})
if p.Name != "" {
index["name"] = p.Name
}
@@ -250,8 +250,8 @@ func (p *PermissionTemplate) PrimaryKey() string {
}
// Index return custom index
func (p *PermissionTemplate) Index() map[string]string {
index := make(map[string]string)
func (p *PermissionTemplate) Index() map[string]interface{} {
index := make(map[string]interface{})
if p.Name != "" {
index["name"] = p.Name
}

View File

@@ -38,6 +38,8 @@ const (
// LabelSyncGeneration describes the generation synced from
LabelSyncGeneration = "ux.oam.dev/synced-generation"
// LabelSyncRevision describes the revision name synced from
LabelSyncRevision = "ux.oam.dev/synced-revision"
// LabelSyncNamespace describes the namespace synced from
LabelSyncNamespace = "ux.oam.dev/from-namespace"
)

View File

@@ -18,7 +18,6 @@ package model
import (
"fmt"
"strconv"
"time"
workflowv1alpha1 "github.com/kubevela/workflow/api/v1alpha1"
@@ -88,8 +87,8 @@ func (w *Workflow) PrimaryKey() string {
}
// Index return custom primary key
func (w *Workflow) Index() map[string]string {
index := make(map[string]string)
func (w *Workflow) Index() map[string]interface{} {
index := make(map[string]interface{})
if w.Name != "" {
index["name"] = w.Name
}
@@ -100,7 +99,7 @@ func (w *Workflow) Index() map[string]string {
index["envName"] = w.EnvName
}
if w.Default != nil {
index["default"] = strconv.FormatBool(*w.Default)
index["default"] = *w.Default
}
return index
@@ -161,8 +160,8 @@ func (w *WorkflowRecord) PrimaryKey() string {
}
// Index return custom primary key
func (w *WorkflowRecord) Index() map[string]string {
index := make(map[string]string)
func (w *WorkflowRecord) Index() map[string]interface{} {
index := make(map[string]interface{})
if w.Name != "" {
index["name"] = w.Name
}

View File

@@ -106,7 +106,7 @@ func NewDatastore(cfg datastore.Config) (ds datastore.DataStore, err error) {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}
case "kubeapi":
ds, err = kubeapi.New(context.Background(), cfg)
ds, err = kubeapi.New(context.Background(), cfg, k8sClient)
if err != nil {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}

View File

@@ -607,7 +607,8 @@ func (c *applicationServiceImpl) DetailComponent(ctx context.Context, app *model
return nil, err
}
var cd v1beta1.ComponentDefinition
if err := c.KubeClient.Get(ctx, types.NamespacedName{Name: component.Type, Namespace: velatypes.DefaultKubeVelaNS}, &cd); err != nil {
loadCtx := utils.WithProject(ctx, "")
if err := c.KubeClient.Get(loadCtx, types.NamespacedName{Name: component.Type, Namespace: velatypes.DefaultKubeVelaNS}, &cd); err != nil {
klog.Warningf("component definition %s get failure. %s", pkgUtils.Sanitize(component.Type), err.Error())
}
@@ -1072,7 +1073,8 @@ func (c *applicationServiceImpl) UpdateComponent(ctx context.Context, app *model
func (c *applicationServiceImpl) createComponent(ctx context.Context, app *model.Application, com apisv1.CreateComponentRequest, main bool) (*apisv1.ComponentBase, error) {
var cd v1beta1.ComponentDefinition
if err := c.KubeClient.Get(ctx, types.NamespacedName{Name: com.ComponentType, Namespace: velatypes.DefaultKubeVelaNS}, &cd); err != nil {
loadCtx := utils.WithProject(ctx, "")
if err := c.KubeClient.Get(loadCtx, types.NamespacedName{Name: com.ComponentType, Namespace: velatypes.DefaultKubeVelaNS}, &cd); err != nil {
klog.Warningf("component definition %s get failure. %s", pkgUtils.Sanitize(com.ComponentType), err.Error())
return nil, bcode.ErrComponentTypeNotSupport
}

View File

@@ -135,6 +135,14 @@ func (u *configServiceImpl) CreateConfig(ctx context.Context, project string, re
}
ns = pro.GetNamespace()
}
exist, err := u.Factory.IsExist(ctx, ns, req.Name)
if err != nil {
klog.Errorf("check config name is exist failure %s", err.Error())
return nil, bcode.ErrConfigExist
}
if exist {
return nil, bcode.ErrConfigExist
}
var properties = make(map[string]interface{})
if err := json.Unmarshal([]byte(req.Properties), &properties); err != nil {
return nil, err
@@ -154,9 +162,6 @@ func (u *configServiceImpl) CreateConfig(ctx context.Context, project string, re
return nil, err
}
if err := u.Factory.CreateOrUpdateConfig(ctx, configItem, ns); err != nil {
if errors.Is(err, config.ErrConfigExist) {
return nil, bcode.ErrConfigExist
}
return nil, err
}
return convertConfig(project, *configItem), nil
@@ -194,9 +199,12 @@ func (u *configServiceImpl) UpdateConfig(ctx context.Context, project string, na
return nil, err
}
if err := u.Factory.CreateOrUpdateConfig(ctx, configItem, ns); err != nil {
if errors.Is(err, config.ErrConfigExist) {
if errors.Is(err, config.ErrChangeTemplate) {
return nil, bcode.ErrChangeTemplate
}
if errors.Is(err, config.ErrChangeSecretType) {
return nil, bcode.ErrChangeSecretType
}
return nil, err
}
return convertConfig(project, *configItem), nil

View File

@@ -104,7 +104,7 @@ func NewDatastore(cfg datastore.Config) (ds datastore.DataStore, err error) {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}
case "kubeapi":
ds, err = kubeapi.New(context.Background(), cfg)
ds, err = kubeapi.New(context.Background(), cfg, k8sClient)
if err != nil {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}

View File

@@ -550,10 +550,10 @@ func (w *workflowServiceImpl) syncWorkflowStatus(ctx context.Context,
record.Message = status.Message
record.Mode = status.Mode
if cb := app.Status.Workflow.ContextBackend; cb != nil && workflowContext == nil {
if cb := app.Status.Workflow.ContextBackend; cb != nil && workflowContext == nil && cb.Namespace != "" && cb.Name != "" {
var cm corev1.ConfigMap
if err := w.KubeClient.Get(ctx, types.NamespacedName{Namespace: cb.Namespace, Name: cb.Name}, &cm); err != nil {
klog.Error(err, "failed to load the context values", "Application", app.Name)
klog.Errorf("failed to load the context values of the application %s:%s", app.Name, err.Error())
}
record.ContextValue = cm.Data
}
@@ -866,8 +866,8 @@ func (w *workflowServiceImpl) RollbackRecord(ctx context.Context, appModel *mode
if len(revisions) == 0 {
return nil, bcode.ErrApplicationNoReadyRevision
}
revisionVersion = revisions[0].Index()["version"]
klog.Infof("select lastest complete revision %s", revisions[0].Index()["version"])
revisionVersion = pkgUtils.ToString(revisions[0].Index()["version"])
klog.Infof("select lastest complete revision %s", revisionVersion)
}
var record = &model.WorkflowRecord{

View File

@@ -89,7 +89,7 @@ func NewDatastore(cfg datastore.Config) (ds datastore.DataStore, err error) {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}
case "kubeapi":
ds, err = kubeapi.New(context.Background(), cfg)
ds, err = kubeapi.New(context.Background(), cfg, k8sClient)
if err != nil {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}

View File

@@ -21,14 +21,13 @@ import (
"sort"
"time"
client2 "sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/oam-dev/kubevela/apis/core.oam.dev/common"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
"github.com/oam-dev/kubevela/apis/types"
"github.com/oam-dev/kubevela/pkg/oam"
"github.com/oam-dev/kubevela/pkg/apiserver/infrastructure/clients"
"github.com/oam-dev/kubevela/pkg/multicluster"
"github.com/oam-dev/kubevela/pkg/apiserver/domain/model"
@@ -56,8 +55,9 @@ var waitBackOff = wait.Backoff{
// InfoCalculateCronJob is the cronJob to calculate the system info store in db
type InfoCalculateCronJob struct {
Store datastore.DataStore `inject:"datastore"`
cron *cron.Cron
KubeClient client.Client `inject:"kubeClient"`
Store datastore.DataStore `inject:"datastore"`
cron *cron.Cron
}
// Start start the worker
@@ -230,12 +230,8 @@ func (i InfoCalculateCronJob) calculateAppInfo(ctx context.Context) (int, []stri
}
func (i InfoCalculateCronJob) calculateAddonInfo(ctx context.Context) (map[string]string, error) {
client, err := clients.GetKubeClient()
if err != nil {
return nil, err
}
apps := &v1beta1.ApplicationList{}
if err := client.List(ctx, apps, client2.InNamespace(types.DefaultKubeVelaNS), client2.HasLabels{oam.LabelAddonName}); err != nil {
if err := i.KubeClient.List(ctx, apps, client.InNamespace(types.DefaultKubeVelaNS), client.HasLabels{oam.LabelAddonName}); err != nil {
return nil, err
}
res := map[string]string{}
@@ -257,11 +253,7 @@ func (i InfoCalculateCronJob) calculateAddonInfo(ctx context.Context) (map[strin
}
func (i InfoCalculateCronJob) calculateClusterInfo(ctx context.Context) (int, error) {
client, err := clients.GetKubeClient()
if err != nil {
return 0, err
}
cs, err := multicluster.ListVirtualClusters(ctx, client)
cs, err := multicluster.ListVirtualClusters(ctx, i.KubeClient)
if err != nil {
return 0, err
}

View File

@@ -83,7 +83,8 @@ var _ = Describe("Test calculate cronJob", func() {
testProject = "test-cronjob-project"
mockDataInDs()
i = InfoCalculateCronJob{
Store: ds,
Store: ds,
KubeClient: k8sClient,
}
systemInfo := model.SystemInfo{InstallID: "test-id", EnableCollection: true}
Expect(ds.Add(ctx, &systemInfo)).Should(SatisfyAny(BeNil(), DataExistMatcher{}))

View File

@@ -30,8 +30,8 @@ import (
)
type cached struct {
resourceVersion string
targets int64
revision string
targets int64
}
// initCache will initialize the cache
@@ -48,9 +48,8 @@ func (c *CR2UX) initCache(ctx context.Context) error {
if !ok {
continue
}
// Change the generation to resource version
gen, ok := app.Labels[model.LabelSyncGeneration]
if !ok || gen == "" {
revision, ok := app.Labels[model.LabelSyncRevision]
if !ok {
continue
}
namespace := app.Labels[model.LabelSyncNamespace]
@@ -60,7 +59,7 @@ func (c *CR2UX) initCache(ctx context.Context) error {
}
// we should check targets if we synced from app status
c.syncCache(key, gen, 0)
c.syncCache(key, revision, 0)
}
return nil
}
@@ -78,8 +77,10 @@ func (c *CR2UX) shouldSync(ctx context.Context, targetApp *v1beta1.Application,
}
// if no LabelSourceOfTruth label, it means the app is existing ones, check the existing labels and annotations
if _, appName := targetApp.Annotations[oam.AnnotationAppName]; appName {
return false
if targetApp.Annotations != nil {
if _, exist := targetApp.Annotations[oam.AnnotationAppName]; exist {
return false
}
}
key := formatAppComposedName(targetApp.Name, targetApp.Namespace)
@@ -90,16 +91,15 @@ func (c *CR2UX) shouldSync(ctx context.Context, targetApp *v1beta1.Application,
_, _, err := c.getApp(ctx, targetApp.Name, targetApp.Namespace)
if del || err != nil {
c.cache.Delete(key)
} else if cd.resourceVersion == targetApp.ResourceVersion {
klog.Infof("app %s/%s with resource version(%v) hasn't updated, ignore the sync event..", targetApp.Name, targetApp.Namespace, targetApp.ResourceVersion)
} else if cd.revision == getRevision(*targetApp) {
klog.V(5).Infof("app %s/%s with resource revision(%v) hasn't updated, ignore the sync event..", targetApp.Name, targetApp.Namespace, targetApp.ResourceVersion)
return false
}
}
return true
}
func (c *CR2UX) syncCache(key string, resourceVersion string, targets int64) {
func (c *CR2UX) syncCache(key string, revision string, targets int64) {
// update cache
c.cache.Store(key, &cached{resourceVersion: resourceVersion, targets: targets})
c.cache.Store(key, &cached{revision: revision, targets: targets})
}

View File

@@ -68,14 +68,18 @@ var _ = Describe("Test Cache", func() {
app1 := &v1beta1.Application{}
app1.Name = "app1"
app1.Namespace = "app1-ns"
app1.ResourceVersion = "1"
Expect(cr2ux.shouldSync(ctx, app1, false)).Should(BeEquivalentTo(true))
app2 := &v1beta1.Application{}
app2.Name = "app2"
app2.Namespace = "app2-ns"
app2.ResourceVersion = "1"
app2.Generation = 1
app2.Status.LatestRevision = &common.Revision{Name: "v1"}
Expect(cr2ux.shouldSync(ctx, app2, false)).Should(BeEquivalentTo(true))
// Only need to sync once.
cr2ux.syncCache(formatAppComposedName(app2.Name, app2.Namespace), "v1", 1)
Expect(cr2ux.shouldSync(ctx, app2, false)).Should(BeEquivalentTo(false))
app3 := &v1beta1.Application{}
@@ -83,18 +87,17 @@ var _ = Describe("Test Cache", func() {
app3.Namespace = "app3-ns"
app3.ResourceVersion = "3"
app3.Labels = map[string]string{
model.LabelSyncGeneration: "1",
model.LabelSyncNamespace: "app3-ns",
model.LabelSourceOfTruth: model.FromUX,
model.LabelSourceOfTruth: model.FromUX,
}
Expect(cr2ux.shouldSync(ctx, app3, false)).Should(BeEquivalentTo(false))
Expect(ds.Put(ctx, &model.Application{Name: "app1", Labels: map[string]string{
model.LabelSyncGeneration: "1",
model.LabelSyncNamespace: "app1-ns",
model.LabelSyncRevision: "v1",
model.LabelSyncNamespace: "app1-ns",
}})).Should(BeNil())
cr2ux.syncCache(formatAppComposedName(app1.Name, app1.Namespace), "1", 0)
cr2ux.syncCache(formatAppComposedName(app1.Name, app1.Namespace), "v1", 0)
app1.Status.LatestRevision = &common.Revision{Name: "v1"}
Expect(cr2ux.shouldSync(ctx, app1, false)).Should(BeEquivalentTo(false))
Expect(cr2ux.shouldSync(ctx, app1, true)).Should(BeEquivalentTo(true))
Expect(ds.Delete(ctx, &model.Application{Name: "app1"})).Should(BeNil())

View File

@@ -46,11 +46,7 @@ func (c *CR2UX) ConvertApp2DatastoreApp(ctx context.Context, targetApp *v1beta1.
}
sourceOfTruth := model.FromCR
if _, ok := targetApp.Labels[oam.LabelAddonName]; ok && strings.HasPrefix(targetApp.Name, "addon-") && targetApp.Namespace == apitypes.DefaultKubeVelaNS {
project = v1.CreateProjectRequest{
Name: model.DefaultSystemProject,
Alias: model.DefaultSystemProjectAlias,
Namespace: targetApp.Namespace,
}
project = c.generateSystemProject(ctx, targetApp.Namespace)
sourceOfTruth = model.FromInner
}
@@ -62,6 +58,7 @@ func (c *CR2UX) ConvertApp2DatastoreApp(ctx context.Context, targetApp *v1beta1.
Labels: map[string]string{
model.LabelSyncNamespace: targetApp.Namespace,
model.LabelSyncGeneration: strconv.FormatInt(targetApp.Generation, 10),
model.LabelSyncRevision: getRevision(*targetApp),
model.LabelSourceOfTruth: sourceOfTruth,
},
}
@@ -87,7 +84,7 @@ func (c *CR2UX) ConvertApp2DatastoreApp(ctx context.Context, targetApp *v1beta1.
if err != nil {
return nil, err
}
klog.Infof("generate the environment %s for the application %s", env.Name, targetApp.Name)
klog.V(5).Infof("generate the environment %s for the application %s", env.Name, targetApp.Name)
dsApp.Env = env
if newProject != "" {
project = v1.CreateProjectRequest{
@@ -149,6 +146,8 @@ func (c *CR2UX) ConvertApp2DatastoreApp(ctx context.Context, targetApp *v1beta1.
// 7. convert the revision
if revision := convert.FromCRApplicationRevision(ctx, cli, targetApp, *dsApp.Workflow, dsApp.Env.Name); revision != nil {
dsApp.Revision = revision
} else {
klog.Warningf("can't generate the application revision(%s) for the app %s", getRevision(*targetApp), targetApp.Name)
}
// 8. convert the workflow record
if record := convert.FromCRWorkflowRecord(targetApp, *dsApp.Workflow, dsApp.Revision); record != nil {
@@ -157,6 +156,27 @@ func (c *CR2UX) ConvertApp2DatastoreApp(ctx context.Context, targetApp *v1beta1.
return dsApp, nil
}
// In order to maintain compatibility with old data,
// if there is a project named addons, continue to use it, but change the alias to System.
func (c *CR2UX) generateSystemProject(ctx context.Context, ns string) v1.CreateProjectRequest {
var pro = model.Project{Name: "addons"}
if err := c.ds.Get(ctx, &pro); err == nil {
if pro.Alias == "Addons" {
pro.Alias = model.DefaultSystemProjectAlias
pro.Namespace = ns
if err := c.ds.Put(ctx, &pro); err != nil {
klog.Warningf("failed to update the project alias to System:%s", err.Error())
}
}
return v1.CreateProjectRequest{Name: pro.Name, Alias: pro.Alias, Namespace: pro.Namespace}
}
return v1.CreateProjectRequest{
Name: model.DefaultSystemProject,
Alias: model.DefaultSystemProjectAlias,
Namespace: ns,
}
}
func (c *CR2UX) generateEnv(ctx context.Context, defaultProject string, envNamespace string, envTargetNames map[string]string) (*model.Env, string, error) {
existEnv := &model.Env{Namespace: envNamespace}
existEnvs, err := c.ds.List(ctx, existEnv, nil)
@@ -212,3 +232,17 @@ func (c *CR2UX) generateEnv(ctx context.Context, defaultProject string, envNames
}
return env, "", nil
}
func getRevision(app v1beta1.Application) string {
if app.Status.LatestRevision == nil {
return ""
}
return app.Status.LatestRevision.Name
}
func getSyncedRevision(rev *model.ApplicationRevision) string {
if rev == nil {
return ""
}
return rev.Version
}

View File

@@ -133,7 +133,10 @@ func (c *CR2UX) AddOrUpdate(ctx context.Context, targetApp *v1beta1.Application)
}
// update cache
c.syncCache(dsApp.AppMeta.PrimaryKey(), targetApp.ResourceVersion, int64(len(dsApp.Targets)))
key := formatAppComposedName(targetApp.Name, targetApp.Namespace)
syncedVersion := getSyncedRevision(dsApp.Revision)
c.syncCache(key, syncedVersion, int64(len(dsApp.Targets)))
klog.Infof("application %s/%s revision %s synced successful", targetApp.Name, targetApp.Namespace, syncedVersion)
return nil
}

View File

@@ -26,6 +26,7 @@ import (
corev1 "k8s.io/api/core/v1"
"github.com/oam-dev/kubevela/apis/core.oam.dev/common"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
"github.com/oam-dev/kubevela/pkg/apiserver/domain/model"
"github.com/oam-dev/kubevela/pkg/apiserver/domain/service"
@@ -138,7 +139,7 @@ var _ = Describe("Test CR convert to ux", func() {
app2 := &v1beta1.Application{}
Expect(common2.ReadYamlToObject("testdata/test-app2.yaml", app2)).Should(BeNil())
app1.Namespace = appNS1
app1.Generation = 2
app1.Status.LatestRevision = &common.Revision{Name: "v2"}
app1.Spec = app2.Spec
Expect(cr2ux.AddOrUpdate(context.Background(), app1)).Should(BeNil())
comp3 := model.ApplicationComponent{AppPrimaryKey: apName1, Name: "blog"}

View File

@@ -87,7 +87,7 @@ func NewDatastore(cfg datastore.Config) (ds datastore.DataStore, err error) {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}
case "kubeapi":
ds, err = kubeapi.New(context.Background(), cfg)
ds, err = kubeapi.New(context.Background(), cfg, k8sClient)
if err != nil {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}

View File

@@ -101,7 +101,7 @@ func (a *ApplicationSync) Start(ctx context.Context, errorChan chan error) {
app := getApp(obj)
if app.DeletionTimestamp == nil {
a.Queue.Add(app)
klog.Infof("watched update/add app event, namespace: %s, name: %s", app.Namespace, app.Name)
klog.V(4).Infof("watched update/add app event, namespace: %s, name: %s", app.Namespace, app.Name)
}
}
@@ -114,7 +114,7 @@ func (a *ApplicationSync) Start(ctx context.Context, errorChan chan error) {
},
DeleteFunc: func(obj interface{}) {
app := getApp(obj)
klog.Infof("watched delete app event, namespace: %s, name: %s", app.Namespace, app.Name)
klog.V(4).Infof("watched delete app event, namespace: %s, name: %s", app.Namespace, app.Name)
a.Queue.Forget(app)
a.Queue.Done(app)
err = cu.DeleteApp(ctx, app)

View File

@@ -26,6 +26,7 @@ import (
corev1 "k8s.io/api/core/v1"
"k8s.io/client-go/util/workqueue"
"github.com/oam-dev/kubevela/apis/core.oam.dev/common"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
"github.com/oam-dev/kubevela/pkg/apiserver/domain/model"
"github.com/oam-dev/kubevela/pkg/apiserver/domain/repository"
@@ -133,6 +134,9 @@ var _ = Describe("Test Worker CR sync to datastore", func() {
app2.Spec = newapp2.Spec
Expect(k8sClient.Update(context.TODO(), app2)).Should(BeNil())
app2.Status.LatestRevision = &common.Revision{Name: "v3"}
Expect(k8sClient.Status().Update(context.TODO(), app2)).Should(BeNil())
Eventually(func() error {
appm := model.ApplicationComponent{AppPrimaryKey: formatAppComposedName(app2.Name, app2.Namespace), Name: "nginx2"}
return ds.Get(ctx, &appm)

View File

@@ -74,7 +74,7 @@ type Entity interface {
PrimaryKey() string
TableName() string
ShortTableName() string
Index() map[string]string
Index() map[string]interface{}
}
// NewEntity Create a new object based on the input type

View File

@@ -35,8 +35,8 @@ import (
"k8s.io/klog/v2"
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/oam-dev/kubevela/pkg/apiserver/infrastructure/clients"
"github.com/oam-dev/kubevela/pkg/apiserver/infrastructure/datastore"
pkgUtils "github.com/oam-dev/kubevela/pkg/utils"
)
type kubeapi struct {
@@ -46,17 +46,13 @@ type kubeapi struct {
// New new kubeapi datastore instance
// Data is stored using ConfigMap.
func New(ctx context.Context, cfg datastore.Config) (datastore.DataStore, error) {
kubeClient, err := clients.GetKubeClient()
if err != nil {
return nil, err
}
func New(ctx context.Context, cfg datastore.Config, client client.Client) (datastore.DataStore, error) {
if cfg.Database == "" {
cfg.Database = "kubevela_store"
}
var namespace corev1.Namespace
if err := kubeClient.Get(ctx, types.NamespacedName{Name: cfg.Database}, &namespace); apierrors.IsNotFound(err) {
if err := kubeClient.Create(ctx, &corev1.Namespace{
if err := client.Get(ctx, types.NamespacedName{Name: cfg.Database}, &namespace); apierrors.IsNotFound(err) {
if err := client.Create(ctx, &corev1.Namespace{
ObjectMeta: metav1.ObjectMeta{
Name: cfg.Database,
Annotations: map[string]string{"description": "For KubeVela API Server metadata storage."},
@@ -64,9 +60,9 @@ func New(ctx context.Context, cfg datastore.Config) (datastore.DataStore, error)
return nil, fmt.Errorf("create namespace failure %w", err)
}
}
migrate(cfg.Database)
migrate(cfg.Database, client)
return &kubeapi{
kubeClient: kubeClient,
kubeClient: client,
namespace: cfg.Database,
}, nil
}
@@ -81,7 +77,7 @@ func generateName(entity datastore.Entity) string {
func (m *kubeapi) generateConfigMap(entity datastore.Entity) *corev1.ConfigMap {
data, _ := json.Marshal(entity)
labels := entity.Index()
labels := convertIndex2Labels(entity.Index())
if labels == nil {
labels = make(map[string]string)
}
@@ -176,7 +172,7 @@ func (m *kubeapi) Put(ctx context.Context, entity datastore.Entity) error {
return datastore.ErrTableNameEmpty
}
// update labels
labels := entity.Index()
labels := convertIndex2Labels(entity.Index())
if labels == nil {
labels = make(map[string]string)
}
@@ -350,8 +346,8 @@ func (m *kubeapi) List(ctx context.Context, entity datastore.Entity, op *datasto
rq, _ := labels.NewRequirement(MigrateKey, selection.DoesNotExist, []string{"ok"})
selector = selector.Add(*rq)
for k, v := range entity.Index() {
metedataLabels := convertIndex2Labels(entity.Index())
for k, v := range metedataLabels {
rq, err := labels.NewRequirement(k, selection.Equals, []string{verifyValue(v)})
if err != nil {
return nil, datastore.ErrIndexInvalid
@@ -441,7 +437,8 @@ func (m *kubeapi) Count(ctx context.Context, entity datastore.Entity, filterOpti
if err != nil {
return 0, datastore.NewDBError(err)
}
for k, v := range entity.Index() {
metedataLabels := convertIndex2Labels(entity.Index())
for k, v := range metedataLabels {
rq, err := labels.NewRequirement(k, selection.Equals, []string{verifyValue(v)})
if err != nil {
return 0, datastore.ErrIndexInvalid
@@ -494,3 +491,19 @@ func verifyValue(v string) string {
s = strings.ReplaceAll(s, " ", "-")
return strings.ToLower(s)
}
func convertIndex2Labels(index map[string]interface{}) map[string]string {
if index == nil {
return nil
}
ret := make(map[string]string, len(index))
for k, v := range index {
value := pkgUtils.ToString(v)
if value == "" {
klog.Warningf("unable to cast %#v of type %T to string", v, v)
continue
}
ret[k] = pkgUtils.ToString(v)
}
return ret
}

View File

@@ -73,7 +73,7 @@ var _ = BeforeSuite(func(done Done) {
By("new kube client success")
clients.SetKubeClient(k8sClient)
kubeStore, err = New(context.TODO(), datastore.Config{Database: "test"})
kubeStore, err = New(context.TODO(), datastore.Config{Database: "test"}, k8sClient)
Expect(err).Should(BeNil())
Expect(kubeStore).ToNot(BeNil())
close(done)

View File

@@ -72,20 +72,38 @@ var _ = Describe("Test kubeapi datastore driver", func() {
err := kubeStore.Put(context.TODO(), &model.Application{Name: "kubevela-app", Description: "this is demo"})
Expect(err).ToNot(HaveOccurred())
})
It("Test index", func() {
It("Test application index", func() {
var app = model.Application{
Name: "test",
}
selector, err := labels.Parse(fmt.Sprintf("table=%s", app.TableName()))
Expect(err).ToNot(HaveOccurred())
Expect(cmp.Diff(app.Index()["name"], "test")).Should(BeEmpty())
for k, v := range app.Index() {
index := convertIndex2Labels(app.Index())
for k, v := range index {
rq, err := labels.NewRequirement(k, selection.Equals, []string{v})
Expect(err).ToNot(HaveOccurred())
selector = selector.Add(*rq)
}
Expect(cmp.Diff(selector.String(), "name=test,table=vela_application")).Should(BeEmpty())
})
It("Test workflow index", func() {
defaultPtr := false
var workflow = model.Workflow{
Name: "test",
Default: &defaultPtr,
}
selector, err := labels.Parse(fmt.Sprintf("table=%s", workflow.TableName()))
Expect(err).ToNot(HaveOccurred())
Expect(cmp.Diff(workflow.Index()["name"], "test")).Should(BeEmpty())
index := convertIndex2Labels(workflow.Index())
for k, v := range index {
rq, err := labels.NewRequirement(k, selection.Equals, []string{v})
Expect(err).ToNot(HaveOccurred())
selector = selector.Add(*rq)
}
Expect(cmp.Diff(selector.String(), "default=false,name=test,table=vela_workflow")).Should(BeEmpty())
})
It("Test list function", func() {
var app model.Application
list, err := kubeStore.List(context.TODO(), &app, &datastore.ListOptions{Page: -1})

View File

@@ -27,7 +27,6 @@ import (
"sigs.k8s.io/controller-runtime/pkg/client"
"github.com/oam-dev/kubevela/pkg/apiserver/domain/model"
"github.com/oam-dev/kubevela/pkg/apiserver/infrastructure/clients"
)
// MigrateKey marks the label key of the migrated data
@@ -36,17 +35,13 @@ const MigrateKey = "db.oam.dev/migrated"
// migrate will migrate the configmap to new short table name, it won't delete the configmaps:
// users can delete by the following commands:
// kubectl -n kubevela delete cm -l db.oam.dev/migrated=ok
func migrate(dbns string) {
kubeClient, err := clients.GetKubeClient()
if err != nil {
panic(err)
}
func migrate(dbns string, c client.Client) {
models := model.GetRegisterModels()
for _, k := range models {
var configMaps corev1.ConfigMapList
table := k.TableName()
selector, _ := labels.Parse(fmt.Sprintf("table=%s", table))
if err = kubeClient.List(context.Background(), &configMaps, &client.ListOptions{Namespace: dbns, LabelSelector: selector}); err != nil {
if err := c.List(context.Background(), &configMaps, &client.ListOptions{Namespace: dbns, LabelSelector: selector}); err != nil {
err = client.IgnoreNotFound(err)
if err != nil {
klog.Errorf("migrate db for kubeapi storage err: %v", err)
@@ -72,14 +67,14 @@ func migrate(dbns string) {
}
cm.Labels[MigrateKey] = "ok"
err = kubeClient.Update(context.Background(), &cm)
err := c.Update(context.Background(), &cm)
if err != nil {
klog.Errorf("update migrated record %s for kubeapi storage err: %v", cm.Name, err)
}
cm.Name = strings.ReplaceAll(k.ShortTableName()+strings.TrimPrefix(cm.Name, checkprefix), "_", "-")
cm.ResourceVersion = ""
delete(cm.Labels, MigrateKey)
err = kubeClient.Create(context.Background(), &cm)
err = c.Create(context.Background(), &cm)
if err != nil {
klog.Errorf("migrate record %s for kubeapi storage err: %v", cm.Name, err)
}

View File

@@ -47,7 +47,7 @@ var _ = Describe("Test Migrate", func() {
cm.Namespace = nsName
Expect(ds.kubeClient.Create(context.Background(), cm)).Should(BeNil())
migrate(nsName)
migrate(nsName, k8sClient)
cmList := v1.ConfigMapList{}
Expect(k8sClient.List(context.Background(), &cmList, client.InNamespace(nsName))).Should(BeNil())
Expect(len(cmList.Items)).Should(BeEquivalentTo(2))

View File

@@ -90,7 +90,7 @@ func NewDatastore(cfg datastore.Config) (ds datastore.DataStore, err error) {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}
case "kubeapi":
ds, err = kubeapi.New(context.Background(), cfg)
ds, err = kubeapi.New(context.Background(), cfg, k8sClient)
if err != nil {
return nil, fmt.Errorf("create mongodb datastore instance failure %w", err)
}

View File

@@ -121,7 +121,7 @@ func (dt *Target) createTarget(req *restful.Request, res *restful.Response) {
// Call the domain layer code
TargetDetail, err := dt.TargetService.CreateTarget(req.Request.Context(), createReq)
if err != nil {
klog.Errorf("create -target failure %s", err.Error())
klog.Errorf("create target failure %s", err.Error())
bcode.ReturnError(req, res, err)
return
}

View File

@@ -43,6 +43,7 @@ import (
"github.com/oam-dev/kubevela/pkg/apiserver/utils"
"github.com/oam-dev/kubevela/pkg/apiserver/utils/container"
pkgconfig "github.com/oam-dev/kubevela/pkg/config"
"github.com/oam-dev/kubevela/pkg/features"
pkgUtils "github.com/oam-dev/kubevela/pkg/utils"
"github.com/oam-dev/kubevela/pkg/utils/apply"
)
@@ -86,6 +87,8 @@ func (s *restServer) buildIoCContainer() error {
if err != nil {
return err
}
authClient := utils.NewAuthClient(kubeClient)
var ds datastore.DataStore
switch s.cfg.Datastore.Type {
case "mongodb":
@@ -94,7 +97,7 @@ func (s *restServer) buildIoCContainer() error {
return fmt.Errorf("create mongodb datastore instance failure %w", err)
}
case "kubeapi":
ds, err = kubeapi.New(context.Background(), s.cfg.Datastore)
ds, err = kubeapi.New(context.Background(), s.cfg.Datastore, kubeClient)
if err != nil {
return fmt.Errorf("create kubeapi datastore instance failure %w", err)
}
@@ -106,23 +109,22 @@ func (s *restServer) buildIoCContainer() error {
return fmt.Errorf("fail to provides the datastore bean to the container: %w", err)
}
kubeClient = utils.NewAuthClient(kubeClient)
if err := s.beanContainer.ProvideWithName("kubeClient", kubeClient); err != nil {
if err := s.beanContainer.ProvideWithName("kubeClient", authClient); err != nil {
return fmt.Errorf("fail to provides the kubeClient bean to the container: %w", err)
}
if err := s.beanContainer.ProvideWithName("kubeConfig", kubeConfig); err != nil {
return fmt.Errorf("fail to provides the kubeConfig bean to the container: %w", err)
}
if err := s.beanContainer.ProvideWithName("apply", apply.NewAPIApplicator(kubeClient)); err != nil {
if err := s.beanContainer.ProvideWithName("apply", apply.NewAPIApplicator(authClient)); err != nil {
return fmt.Errorf("fail to provides the apply bean to the container: %w", err)
}
factory := pkgconfig.NewConfigFactory(kubeClient)
factory := pkgconfig.NewConfigFactory(authClient)
if err := s.beanContainer.ProvideWithName("configFactory", factory); err != nil {
return fmt.Errorf("fail to provides the config factory bean to the container: %w", err)
}
addonStore := pkgaddon.NewRegistryDataStore(kubeClient)
addonStore := pkgaddon.NewRegistryDataStore(authClient)
if err := s.beanContainer.ProvideWithName("registryDatastore", addonStore); err != nil {
return fmt.Errorf("fail to provides the registry datastore bean to the container: %w", err)
}
@@ -149,6 +151,12 @@ func (s *restServer) buildIoCContainer() error {
func (s *restServer) Run(ctx context.Context, errChan chan error) error {
for feature := range features.APIServerMutableFeatureGate.GetAll() {
if features.APIServerFeatureGate.Enabled(feature) {
klog.Infof("The %s feature enabled", feature)
}
}
// build the Ioc Container
if err := s.buildIoCContainer(); err != nil {
return err

View File

@@ -86,6 +86,12 @@ func (c *authClient) Status() client.StatusWriter {
return &authAppStatusClient{StatusWriter: c.Client.Status()}
}
// Get .
func (c *authClient) Get(ctx context.Context, key client.ObjectKey, obj client.Object) error {
ctx = ContextWithUserInfo(ctx)
return c.Client.Get(ctx, key, obj)
}
// List .
func (c *authClient) List(ctx context.Context, obj client.ObjectList, opts ...client.ListOption) error {
ctx = ContextWithUserInfo(ctx)

View File

@@ -37,4 +37,7 @@ var (
// ErrNotFoundDistribution means the distribution is not exist
ErrNotFoundDistribution = NewBcode(404, 16007, "the distribution is not exist")
// ErrChangeSecretType the secret type of the config can not be change
ErrChangeSecretType = NewBcode(400, 16008, "the secret type of the config can not be change")
)

View File

@@ -23,8 +23,6 @@ import (
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/yaml"
@@ -167,7 +165,6 @@ var _ = Describe("Test Live-Diff", func() {
Expect(k8sClient.Create(context.Background(), un)).Should(Succeed())
}
ctx := context.Background()
Expect(k8sClient.Create(ctx, &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: "vela-system"}})).Should(Succeed())
applyFile("diff-input-app-with-externals.yaml", "default")
applyFile("diff-apprevision.yaml", "default")
app := &v1beta1.Application{}
@@ -190,7 +187,6 @@ var _ = Describe("Test Live-Diff", func() {
Expect(runDiff()).Should(ContainSubstring("\"myworker\" not found"))
applyFile("td-myingress.yaml", "vela-system")
applyFile("cd-myworker.yaml", "vela-system")
applyFile("wd-deploy.yaml", "vela-system")
applyFile("wd-ref-objects.yaml", "vela-system")
Expect(runDiff()).Should(ContainSubstring("\"deploy-livediff-demo\" not found"))
applyFile("external-workflow.yaml", "default")

View File

@@ -37,14 +37,19 @@ import (
"github.com/kubevela/workflow/pkg/cue/packages"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1alpha1"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
"github.com/oam-dev/kubevela/apis/types"
apiutils "github.com/oam-dev/kubevela/pkg/apiserver/utils"
"github.com/oam-dev/kubevela/pkg/appfile"
"github.com/oam-dev/kubevela/pkg/cue/definition"
"github.com/oam-dev/kubevela/pkg/oam"
"github.com/oam-dev/kubevela/pkg/oam/discoverymapper"
oamutil "github.com/oam-dev/kubevela/pkg/oam/util"
"github.com/oam-dev/kubevela/pkg/policy/envbinding"
"github.com/oam-dev/kubevela/pkg/utils"
"github.com/oam-dev/kubevela/pkg/utils/apply"
"github.com/oam-dev/kubevela/pkg/workflow/step"
)
// DryRun executes dry-run on an application
@@ -136,13 +141,15 @@ func (d *Option) ExecuteDryRun(ctx context.Context, application *v1beta1.Applica
if app.Namespace != "" {
ctx = oamutil.SetNamespaceInCtx(ctx, app.Namespace)
}
appFile, err := parser.GenerateAppFileFromApp(ctx, app)
generateCtx := apiutils.WithProject(ctx, "")
appFile, err := parser.GenerateAppFileFromApp(generateCtx, app)
if err != nil {
return nil, nil, errors.WithMessage(err, "cannot generate appFile from application")
}
if appFile.Namespace == "" {
appFile.Namespace = corev1.NamespaceDefault
}
comps, err := appFile.GenerateComponentManifests()
if err != nil {
return nil, nil, errors.WithMessage(err, "cannot generate manifests from components and traits")
@@ -206,3 +213,120 @@ func (d *Option) PrintDryRun(buff *bytes.Buffer, appName string, comps []*types.
}
return nil
}
// ExecuteDryRunWithPolicies is similar to ExecuteDryRun func, but considers deploy workflow step and topology+override policies
func (d *Option) ExecuteDryRunWithPolicies(ctx context.Context, application *v1beta1.Application, buff *bytes.Buffer) error {
app := application.DeepCopy()
if app.Namespace == "" {
app.Namespace = corev1.NamespaceDefault
} else {
ctx = oamutil.SetNamespaceInCtx(ctx, app.Namespace)
}
parser := appfile.NewDryRunApplicationParser(d.Client, d.DiscoveryMapper, d.PackageDiscover, d.Auxiliaries)
af, err := parser.GenerateAppFileFromApp(ctx, app)
if err != nil {
return err
}
deployWorkflowCount := 0
for _, wfs := range af.WorkflowSteps {
if wfs.Type == step.DeployWorkflowStep {
deployWorkflowCount++
deployWorkflowStepSpec := &step.DeployWorkflowStepSpec{}
if err := utils.StrictUnmarshal(wfs.Properties.Raw, deployWorkflowStepSpec); err != nil {
return err
}
topologyPolicies, overridePolicies, err := filterPolicies(af.Policies, deployWorkflowStepSpec.Policies)
if err != nil {
return err
}
if len(topologyPolicies) > 0 {
for _, tp := range topologyPolicies {
patchedApp, err := patchApp(app, overridePolicies)
if err != nil {
return err
}
comps, pms, err := d.ExecuteDryRun(ctx, patchedApp)
if err != nil {
return err
}
err = d.PrintDryRun(buff, fmt.Sprintf("%s with topology %s", patchedApp.Name, tp.Name), comps, pms)
if err != nil {
return err
}
}
} else {
patchedApp, err := patchApp(app, overridePolicies)
if err != nil {
return err
}
comps, pms, err := d.ExecuteDryRun(ctx, patchedApp)
if err != nil {
return err
}
err = d.PrintDryRun(buff, fmt.Sprintf("%s only with override policies", patchedApp.Name), comps, pms)
if err != nil {
return err
}
}
}
}
if deployWorkflowCount == 0 {
comps, pms, err := d.ExecuteDryRun(ctx, app)
if err != nil {
return err
}
err = d.PrintDryRun(buff, app.Name, comps, pms)
if err != nil {
return err
}
}
return nil
}
func filterPolicies(policies []v1beta1.AppPolicy, policyNames []string) ([]v1beta1.AppPolicy, []v1beta1.AppPolicy, error) {
policyMap := make(map[string]v1beta1.AppPolicy)
for _, policy := range policies {
policyMap[policy.Name] = policy
}
var topologyPolicies []v1beta1.AppPolicy
var overridePolicies []v1beta1.AppPolicy
for _, policyName := range policyNames {
if policy, found := policyMap[policyName]; found {
switch policy.Type {
case v1alpha1.TopologyPolicyType:
topologyPolicies = append(topologyPolicies, policy)
case v1alpha1.OverridePolicyType:
overridePolicies = append(overridePolicies, policy)
}
} else {
return nil, nil, errors.Errorf("policy %s not found", policyName)
}
}
return topologyPolicies, overridePolicies, nil
}
func patchApp(application *v1beta1.Application, overridePolicies []v1beta1.AppPolicy) (*v1beta1.Application, error) {
app := application.DeepCopy()
for _, policy := range overridePolicies {
if policy.Properties == nil {
return nil, fmt.Errorf("override policy %s must not have empty properties", policy.Name)
}
overrideSpec := &v1alpha1.OverridePolicySpec{}
if err := utils.StrictUnmarshal(policy.Properties.Raw, overrideSpec); err != nil {
return nil, errors.Wrapf(err, "failed to parse override policy %s", policy.Name)
}
overrideComps, err := envbinding.PatchComponents(app.Spec.Components, overrideSpec.Components, overrideSpec.Selector)
if err != nil {
return nil, errors.Wrapf(err, "failed to apply override policy %s", policy.Name)
}
app.Spec.Components = overrideComps
}
return app, nil
}

View File

@@ -17,8 +17,11 @@ limitations under the License.
package dryrun
import (
"bytes"
"context"
"encoding/json"
"os"
"strings"
"github.com/oam-dev/kubevela/apis/types"
@@ -59,3 +62,108 @@ var _ = Describe("Test DryRun", func() {
Expect(diff).Should(BeEmpty())
})
})
var _ = Describe("Test dry run with policies", func() {
It("Test dry run with override policy", func() {
webservice, err := os.ReadFile("../../../charts/vela-core/templates/defwithtemplate/webservice.yaml")
Expect(err).Should(BeNil())
webserviceYAML := strings.Replace(string(webservice), "{{ include \"systemDefinitionNamespace\" . }}", types.DefaultKubeVelaNS, 1)
wwd := v1beta1.ComponentDefinition{}
Expect(yaml.Unmarshal([]byte(webserviceYAML), &wwd)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &wwd)).Should(BeNil())
scaler, err := os.ReadFile("../../../charts/vela-core/templates/defwithtemplate/scaler.yaml")
Expect(err).Should(BeNil())
scalerYAML := strings.Replace(string(scaler), "{{ include \"systemDefinitionNamespace\" . }}", types.DefaultKubeVelaNS, 1)
var td v1beta1.TraitDefinition
Expect(yaml.Unmarshal([]byte(scalerYAML), &td)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &td)).Should(BeNil())
appYAML := readDataFromFile("./testdata/testing-dry-run-1.yaml")
app := &v1beta1.Application{}
Expect(yaml.Unmarshal([]byte(appYAML), &app)).Should(BeNil())
var buff = bytes.Buffer{}
err = dryrunOpt.ExecuteDryRunWithPolicies(context.TODO(), app, &buff)
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app with topology target-default)"))
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app with topology target-prod)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
Expect(buff.String()).Should(ContainSubstring("replicas: 1"))
Expect(buff.String()).Should(ContainSubstring("replicas: 3"))
Expect(buff.String()).Should(ContainSubstring("kind: Service"))
})
It("Test dry run only with override policy", func() {
appYAML := readDataFromFile("./testdata/testing-dry-run-2.yaml")
app := &v1beta1.Application{}
Expect(yaml.Unmarshal([]byte(appYAML), &app)).Should(BeNil())
var buff = bytes.Buffer{}
err := dryrunOpt.ExecuteDryRunWithPolicies(context.TODO(), app, &buff)
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app only with override policies)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
Expect(buff.String()).Should(ContainSubstring("replicas: 3"))
Expect(buff.String()).Should(ContainSubstring("kind: Service"))
})
It("Test dry run without deploy workflow", func() {
appYAML := readDataFromFile("./testdata/testing-dry-run-3.yaml")
app := &v1beta1.Application{}
Expect(yaml.Unmarshal([]byte(appYAML), &app)).Should(BeNil())
var buff = bytes.Buffer{}
err := dryrunOpt.ExecuteDryRunWithPolicies(context.TODO(), app, &buff)
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
})
It("Test dry run without custom policy", func() {
topo, err := os.ReadFile("./testdata/pd-mypolicy.yaml")
Expect(err).Should(BeNil())
var pd v1beta1.PolicyDefinition
Expect(yaml.Unmarshal([]byte(topo), &pd)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &pd)).Should(BeNil())
appYAML := readDataFromFile("./testdata/testing-dry-run-4.yaml")
app := &v1beta1.Application{}
Expect(yaml.Unmarshal([]byte(appYAML), &app)).Should(BeNil())
var buff = bytes.Buffer{}
err = dryrunOpt.ExecuteDryRunWithPolicies(context.TODO(), app, &buff)
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app) -- Component(testing-dryrun)"))
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app) -- Policy(mypolicy)"))
Expect(buff.String()).Should(ContainSubstring("name: my-policy"))
})
It("Test dry run with trait", func() {
nocalhost, err := os.ReadFile("../../../charts/vela-core/templates/defwithtemplate/nocalhost.yaml")
Expect(err).Should(BeNil())
nocalhostYAML := strings.Replace(string(nocalhost), "{{ include \"systemDefinitionNamespace\" . }}", types.DefaultKubeVelaNS, 1)
var td v1beta1.TraitDefinition
Expect(yaml.Unmarshal([]byte(nocalhostYAML), &td)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &td)).Should(BeNil())
appYAML := readDataFromFile("./testdata/testing-dry-run-5.yaml")
app := &v1beta1.Application{}
Expect(yaml.Unmarshal([]byte(appYAML), &app)).Should(BeNil())
var buff = bytes.Buffer{}
err = dryrunOpt.ExecuteDryRunWithPolicies(context.TODO(), app, &buff)
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app) -- Component(testing-dryrun)"))
Expect(buff.String()).Should(ContainSubstring("## From the trait nocalhost"))
Expect(buff.String()).Should(ContainSubstring("trait.oam.dev/type: nocalhost"))
})
})

View File

@@ -17,11 +17,19 @@ limitations under the License.
package dryrun
import (
"context"
"os"
"path/filepath"
"testing"
"time"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/yaml"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
"github.com/oam-dev/kubevela/apis/types"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
@@ -103,6 +111,16 @@ var _ = BeforeSuite(func(done Done) {
tdMyIngress, err := oamutil.Object2Unstructured(myingressDef)
Expect(err).Should(BeNil())
// create vela-system ns
Expect(k8sClient.Create(context.TODO(), &corev1.Namespace{ObjectMeta: metav1.ObjectMeta{Name: types.DefaultKubeVelaNS}})).Should(Succeed())
// create deploy workflow step definition
deploy, err := os.ReadFile("./testdata/wd-deploy.yaml")
Expect(err).Should(BeNil())
var wfsd v1beta1.WorkflowStepDefinition
Expect(yaml.Unmarshal([]byte(deploy), &wfsd)).Should(BeNil())
wfsd.SetNamespace(types.DefaultKubeVelaNS)
Expect(k8sClient.Create(context.TODO(), &wfsd)).Should(BeNil())
dryrunOpt = NewDryRunOption(k8sClient, cfg, dm, pd, []oam.Object{cdMyWorker, tdMyIngress}, false)
diffOpt = &LiveDiffOption{DryRun: dryrunOpt, Parser: appfile.NewApplicationParser(k8sClient, dm, pd)}

View File

@@ -0,0 +1,22 @@
# Code generated by KubeVela templates. DO NOT EDIT. Please edit the original cue file.
# Definition source cue file: vela-templates/definitions/internal/topology.cue
apiVersion: core.oam.dev/v1beta1
kind: PolicyDefinition
metadata:
name: mypolicy
namespace: vela-system
spec:
schematic:
cue:
template: |
output: {
apiVersion: "testing/v1"
kind: "Policy"
policy: {
name: parameter.name
}
}
parameter: {
name: string
}

View File

@@ -0,0 +1,48 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
namespace: default
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
ports:
- port: 8000
expose: true
traits:
- type: scaler
properties:
replicas: 1
policies:
- name: target-default
type: topology
properties:
clusters: [ "local" ]
namespace: "default"
- name: target-prod
type: topology
properties:
clusters: [ "local" ]
namespace: "prod"
- name: deploy-ha
type: override
properties:
components:
- type: webservice
traits:
- type: scaler
properties:
replicas: 3
workflow:
steps:
- name: deploy2default
type: deploy
properties:
policies: [ "target-default" ]
- name: deploy2prod
type: deploy
properties:
policies: [ "target-prod", "deploy-ha" ]

View File

@@ -0,0 +1,27 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
ports:
- port: 8000
expose: true
traits:
- type: scaler
properties:
replicas: 1
policies:
- name: deploy-ha
type: override
properties:
components:
- type: webservice
traits:
- type: scaler
properties:
replicas: 3

View File

@@ -0,0 +1,15 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
workflow:
steps:
- name: suspend
type: suspend

View File

@@ -0,0 +1,16 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
policies:
- name: mypolicy
type: mypolicy
properties:
name: "my-policy"

View File

@@ -0,0 +1,48 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
traits:
- type: nocalhost
properties:
port: 9080
gitUrl: https://github.com/nocalhost/bookinfo-productpage.git
image: nocalhost-docker.pkg.coding.net/nocalhost/dev-images/python:3.7.7-slim-productpage-with-pydevd
shell: "bash"
workDir: "/opt/work"
resources:
limits:
memory: 1Gi
cpu: "1"
requests:
memory: 512Mi
cpu: "0.5"
debug:
remoteDebugPort: 9009
hotReload: true
sync:
type: send
filePattern:
- ./
ignoreFilePattern:
- .git
- .idea
command:
run:
- sh
- run.sh
debug:
- sh
- debug.sh
env:
- name: "foo"
value: "bar"
portForward:
- 39080:9080

View File

@@ -24,6 +24,7 @@ import (
utilnet "k8s.io/apimachinery/pkg/util/net"
"k8s.io/apiserver/pkg/endpoints/request"
"k8s.io/client-go/transport"
"k8s.io/klog/v2"
"github.com/oam-dev/kubevela/pkg/utils"
)
@@ -50,6 +51,7 @@ func (rt *impersonatingRoundTripper) RoundTrip(req *http.Request) (*http.Respons
ctx := req.Context()
req = req.Clone(ctx)
userInfo, exists := request.UserFrom(ctx)
klog.V(7).Infof("impersonation request log. path: %s method: %s user info: %+v", req.URL.String(), req.Method, userInfo)
if exists && userInfo != nil {
if name := userInfo.GetName(); name != "" {
req.Header.Set(transport.ImpersonateUserHeader, name)

View File

@@ -88,6 +88,12 @@ var ErrConfigNotFound = errors.New("the config is not exist")
// ErrTemplateNotFound means the template is not exist
var ErrTemplateNotFound = errors.New("the template is not exist")
// ErrChangeTemplate means the template of the config can not be change
var ErrChangeTemplate = errors.New("the template of the config can not be change")
// ErrChangeSecretType means the secret type of the config can not be change
var ErrChangeSecretType = errors.New("the secret type of the config can not be change")
// NamespacedName the namespace and name model
type NamespacedName struct {
Name string `json:"name"`
@@ -192,6 +198,7 @@ type Factory interface {
ListConfigs(ctx context.Context, namespace, template, scope string, withStatus bool) ([]*Config, error)
DeleteConfig(ctx context.Context, namespace, name string) error
CreateOrUpdateConfig(ctx context.Context, i *Config, ns string) error
IsExist(ctx context.Context, namespace, name string) (bool, error)
CreateOrUpdateDistribution(ctx context.Context, ns, name string, ads *CreateDistributionSpec) error
ListDistributions(ctx context.Context, ns string) ([]*Distribution, error)
@@ -549,7 +556,10 @@ func (k *kubeConfigFactory) CreateOrUpdateConfig(ctx context.Context, i *Config,
var secret v1.Secret
if err := k.cli.Get(ctx, pkgtypes.NamespacedName{Namespace: i.Namespace, Name: i.Name}, &secret); err == nil {
if secret.Labels[types.LabelConfigType] != i.Template.Name {
return ErrConfigExist
return ErrChangeTemplate
}
if secret.Type != i.Secret.Type {
return ErrChangeSecretType
}
}
if err := k.apiApply.Apply(ctx, i.Secret, apply.Quiet()); err != nil {
@@ -571,6 +581,17 @@ func (k *kubeConfigFactory) CreateOrUpdateConfig(ctx context.Context, i *Config,
return nil
}
func (k *kubeConfigFactory) IsExist(ctx context.Context, namespace, name string) (bool, error) {
var secret v1.Secret
if err := k.cli.Get(ctx, pkgtypes.NamespacedName{Namespace: namespace, Name: name}, &secret); err != nil {
if apierrors.IsNotFound(err) {
return false, nil
}
return false, err
}
return true, nil
}
func (k *kubeConfigFactory) ListConfigs(ctx context.Context, namespace, template, scope string, withStatus bool) ([]*Config, error) {
var list = &v1.SecretList{}
requirement := fmt.Sprintf("%s=%s", types.LabelConfigCatalog, types.VelaCoreConfig)

View File

@@ -144,6 +144,12 @@ var _ = Describe("test config factory", func() {
Expect(len(config.Targets)).Should(Equal(1))
})
It("check if the config exist", func() {
exist, err := fac.IsExist(context.TODO(), "default", "db-config")
Expect(err).Should(BeNil())
Expect(exist).Should(BeTrue())
})
It("list the distributions", func() {
distributions, err := fac.ListDistributions(context.TODO(), "default")
Expect(err).Should(BeNil())

View File

@@ -254,7 +254,7 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu
}
}
case workflowv1alpha1.WorkflowStateSkipped:
return ctrl.Result{}, nil
return r.result(nil).requeue(executor.GetBackoffWaitTime()).ret()
default:
}

View File

@@ -351,13 +351,16 @@ func (td *traitDef) Complete(ctx process.Context, abstractTemplate string, param
patcher := val.LookupPath(value.FieldPath(PatchFieldName))
base, auxiliaries := ctx.Output()
if base != nil && patcher.Exists() {
if patcher.Exists() {
if base == nil {
return fmt.Errorf("patch trait %s into an invalid workload", td.name)
}
if err := base.Unify(patcher, sets.CreateUnifyOptionsForPatcher(patcher)...); err != nil {
return errors.WithMessagef(err, "invalid patch trait %s into workload", td.name)
}
}
outputsPatcher := val.LookupPath(value.FieldPath(PatchOutputsFieldName))
if base != nil && outputsPatcher.Exists() {
if outputsPatcher.Exists() {
for _, auxiliary := range auxiliaries {
target := outputsPatcher.LookupPath(value.FieldPath(auxiliary.Name))
if !target.Exists() {

View File

@@ -25,6 +25,7 @@ import (
"k8s.io/apimachinery/pkg/runtime"
"github.com/kubevela/workflow/pkg/cue/packages"
wfprocess "github.com/kubevela/workflow/pkg/cue/process"
"github.com/oam-dev/kubevela/apis/types"
"github.com/oam-dev/kubevela/pkg/cue/process"
@@ -1534,3 +1535,34 @@ func TestTraitPatchSingleOutput(t *testing.T) {
r.True(ok)
r.Equal("val", val)
}
func TestTraitCompleteErrorCases(t *testing.T) {
cases := map[string]struct {
ctx wfprocess.Context
traitName string
template string
params map[string]interface{}
err string
}{
"patch trait": {
ctx: process.NewContext(process.ContextData{}),
template: `
patch: {
// +patchKey=name
spec: template: spec: containers: [parameter]
}
parameter: {
name: string
image: string
command?: [...string]
}`,
err: "patch trait patch trait into an invalid workload",
},
}
for k, v := range cases {
td := NewTraitAbstractEngine(k, &packages.PackageDiscover{})
err := td.Complete(v.ctx, v.template, v.params)
assert.Error(t, err)
assert.Contains(t, err.Error(), v.err)
}
}

View File

@@ -371,6 +371,7 @@ func DeleteManagedResourceInApplication(ctx context.Context, cli client.Client,
}
return nil
}
util.RemoveAnnotations(obj, []string{oam.AnnotationAppSharedBy})
}
if mr.SkipGC || hasOrphanFinalizer(app) {
if labels := obj.GetLabels(); labels != nil {

View File

@@ -21,6 +21,7 @@ import (
"path"
"reflect"
"sort"
"strconv"
"strings"
)
@@ -80,6 +81,55 @@ func SliceIncludeSlice(a, b []string) bool {
return true
}
// ToString convery an interface to a string type.
func ToString(i interface{}) string {
if i == nil {
return ""
}
v := reflect.ValueOf(i)
if v.Kind() == reflect.Ptr && !v.IsNil() {
v = v.Elem()
}
i = v.Interface()
switch s := i.(type) {
case string:
return s
case bool:
return strconv.FormatBool(s)
case float64:
return strconv.FormatFloat(s, 'f', -1, 64)
case float32:
return strconv.FormatFloat(float64(s), 'f', -1, 32)
case int:
return strconv.Itoa(s)
case int64:
return strconv.FormatInt(s, 10)
case int32:
return strconv.Itoa(int(s))
case int16:
return strconv.FormatInt(int64(s), 10)
case int8:
return strconv.FormatInt(int64(s), 10)
case uint:
return strconv.FormatUint(uint64(s), 10)
case uint64:
// nolint
return strconv.FormatUint(uint64(s), 10)
case uint32:
return strconv.FormatUint(uint64(s), 10)
case uint16:
return strconv.FormatUint(uint64(s), 10)
case uint8:
return strconv.FormatUint(uint64(s), 10)
case []byte:
return string(s)
case nil:
return ""
default:
return ""
}
}
// MapKey2Array convery map keys to array
func MapKey2Array(source map[string]string) []string {
var list []string

View File

@@ -137,3 +137,40 @@ func TestJoinURL(t *testing.T) {
}
}
func TestToString(t *testing.T) {
type Obj struct {
k string
}
obj := &Obj{"foo"}
boolPtr := true
caseA := []struct {
input interface{}
expect string
}{
{int(666), "666"},
{int8(6), "6"},
{int16(6), "6"},
{int32(6), "6"},
{int64(6), "6"},
{uint(6), "6"},
{uint8(6), "6"},
{uint16(6), "6"},
{uint32(6), "6"},
{uint64(6), "6"},
{float32(3.14), "3.14"},
{float64(3.14), "3.14"},
{true, "true"},
{false, "false"},
{&boolPtr, "true"},
{nil, ""},
{[]byte("one time"), "one time"},
{"one more time", "one more time"},
{obj, ""},
}
for _, test := range caseA {
v := ToString(test.input)
assert.Equal(t, test.expect, v)
}
}

View File

@@ -37,7 +37,7 @@ type QueryView struct {
const (
// PatternQL is the pattern string of velaQL, velaQL's query syntax is `ViewName{key1=value1 ,key2="value2",}.Export`
PatternQL = `(?P<view>[a-z0-9](?:[a-z0-9\-]{0,61}[a-z0-9])?)(?P<parameter>{.*?})?\.?(?P<export>[_a-zA-Z][\._a-zA-Z0-9]*)?`
PatternQL = `(?P<view>[a-z0-9](?:[a-z0-9\-]{0,61}[a-z0-9])?)(?P<parameter>{.*?})?\.?(?P<export>[_a-zA-Z][\._a-zA-Z0-9\[\]]*)?`
// PatternKV is the pattern string of parameter
PatternKV = `(?P<key>[^=]+)=(?P<value>[^=]*?)(?:,|$)`
// KeyWordView represent view keyword

View File

@@ -58,6 +58,13 @@ func TestParseVelaQL(t *testing.T) {
Export: "output.value.spec",
},
err: nil,
}, {
ql: `view{test=true}.output.value[0].spec`,
query: QueryView{
View: "view",
Export: "output.value[0].spec",
},
err: nil,
}}
for _, testcase := range testcases {

View File

@@ -41,6 +41,7 @@ import (
"github.com/kubevela/workflow/pkg/types"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
"github.com/oam-dev/kubevela/pkg/apiserver/utils"
"github.com/oam-dev/kubevela/pkg/multicluster"
querytypes "github.com/oam-dev/kubevela/pkg/velaql/providers/query/types"
)
@@ -229,7 +230,7 @@ func (h *provider) CollectLogsInPod(ctx monitorContext.Context, wfCtx wfContext.
if err = val.UnmarshalTo(opts); err != nil {
return errors.Wrapf(err, "invalid log options content")
}
cliCtx := multicluster.ContextWithClusterName(ctx, cluster)
cliCtx := utils.ContextWithUserInfo(multicluster.ContextWithClusterName(ctx, cluster))
h.cfg.Wrap(pkgmulticluster.NewTransportWrapper())
clientSet, err := kubernetes.NewForConfig(h.cfg)
if err != nil {

View File

@@ -25,6 +25,7 @@ import (
appsv1 "k8s.io/api/apps/v1"
v12 "k8s.io/api/core/v1"
kerrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/api/meta"
v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@@ -908,7 +909,7 @@ func iterateListSubResources(ctx context.Context, cluster string, k8sClient clie
clusterCTX := multicluster.ContextWithClusterName(ctx, cluster)
items, err := listItemByRule(clusterCTX, k8sClient, resource, *parentObject, specifiedFunc, rule.DefaultGenListOptionFunc, rule.DisableFilterByOwnerReference)
if err != nil {
if meta.IsNoMatchError(err) || runtime.IsNotRegisteredError(err) {
if meta.IsNoMatchError(err) || runtime.IsNotRegisteredError(err) || kerrors.IsNotFound(err) {
klog.Warningf("ignore list resources: %s as %v", resource.Kind, err)
continue
}

View File

@@ -49,19 +49,6 @@ import (
cmdutil "github.com/oam-dev/kubevela/pkg/utils/util"
)
const (
// DescAnnotation records the description of addon
DescAnnotation = "addons.oam.dev/description"
// DependsOnWorkFlowStepName is workflow step name which is used to check dependsOn app
DependsOnWorkFlowStepName = "depends-on-app"
// AddonTerraformProviderNamespace is the namespace of addon terraform provider
AddonTerraformProviderNamespace = "default"
// AddonTerraformProviderNameArgument is the argument name of addon terraform provider
AddonTerraformProviderNameArgument = "providerName"
)
const (
statusEnabled = "enabled"
statusDisabled = "disabled"
@@ -185,7 +172,7 @@ func NewAddonEnableCommand(c common.Args, ioStream cmdutil.IOStreams) *cobra.Com
if !file.IsDir() {
return fmt.Errorf("%s is not addon dir", addonOrDir)
}
ioStream.Infof("enable addon by local dir: %s \n", addonOrDir)
ioStream.Infof(color.New(color.FgYellow).Sprintf("enabling addon by local dir: %s \n", addonOrDir))
// args[0] is a local path install with local dir, use base dir name as addonName
abs, err := filepath.Abs(addonOrDir)
if err != nil {
@@ -314,7 +301,7 @@ non-empty new arg
if !file.IsDir() {
return fmt.Errorf("%s is not addon dir", addonOrDir)
}
ioStream.Infof("enable addon by local dir: %s \n", addonOrDir)
ioStream.Infof(color.New(color.FgYellow).Sprintf("enabling addon by local dir: %s \n", addonOrDir))
// args[0] is a local path install with local dir
abs, err := filepath.Abs(addonOrDir)
if err != nil {
@@ -1172,12 +1159,12 @@ func hasAddon(addons []*pkgaddon.UIData, name string) bool {
return false
}
func transClusters(cstr string) []string {
func transClusters(cstr string) []interface{} {
if len(cstr) == 0 {
return nil
}
cstr = strings.TrimPrefix(strings.TrimSuffix(cstr, "}"), "{")
var clusterL []string
var clusterL []interface{}
clusterList := strings.Split(cstr, ",")
for _, v := range clusterList {
clusterL = append(clusterL, strings.TrimSpace(v))

View File

@@ -191,19 +191,19 @@ func TestAddonUpgradeCmdWithErrLocalPath(t *testing.T) {
func TestTransCluster(t *testing.T) {
testcase := []struct {
str string
res []string
res []interface{}
}{
{
str: "{cluster1, cluster2}",
res: []string{"cluster1", "cluster2"},
res: []interface{}{"cluster1", "cluster2"},
},
{
str: "{cluster1,cluster2}",
res: []string{"cluster1", "cluster2"},
res: []interface{}{"cluster1", "cluster2"},
},
{
str: "{cluster1, cluster2 }",
res: []string{"cluster1", "cluster2"},
res: []interface{}{"cluster1", "cluster2"},
},
}
for _, s := range testcase {

View File

@@ -143,8 +143,10 @@ func (d *debugOpts) debugWorkflow(ctx context.Context, wargs *WorkflowArgs, cli
if err != nil {
return fmt.Errorf("failed to select workflow step: %w", err)
}
d.step = unwrapStepName(step)
d.step = unwrapStepID(step, wargs.WorkflowInstance)
d.errMsg = d.errMap[d.step]
} else {
d.step = unwrapStepID(d.step, wargs.WorkflowInstance)
}
// debug workflow steps
@@ -261,6 +263,21 @@ func unwrapStepName(step string) string {
}
}
func unwrapStepID(step string, instance *wfTypes.WorkflowInstance) string {
step = unwrapStepName(step)
for _, status := range instance.Status.Steps {
if status.Name == step {
return status.ID
}
for _, sub := range status.SubStepsStatus {
if sub.Name == step {
return sub.ID
}
}
}
return step
}
func (d *debugOpts) getDebugRawValue(ctx context.Context, cli client.Client, pd *packages.PackageDiscover, instance *wfTypes.WorkflowInstance) (*value.Value, string, error) {
debugCM := &corev1.ConfigMap{}
if err := cli.Get(ctx, client.ObjectKey{Name: debug.GenerateContextName(instance.Name, d.step, string(instance.UID)), Namespace: instance.Namespace}, debugCM); err != nil {

View File

@@ -330,7 +330,7 @@ var (
that you can use to configure customized recycle rules.
This command supports delete application in various modes.
Natively, you can use it like "kubectl delete app <app-name>".
Natively, you can use it like "kubectl delete app [app-name]".
In the cases you only want to delete the application but leave the
resources there, you can use the --orphan parameter.
In the cases the server-side controller is uninstalled, or you want to

View File

@@ -20,10 +20,17 @@ import (
"bytes"
"context"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
wfv1alpha1 "github.com/kubevela/workflow/api/v1alpha1"
"k8s.io/client-go/kubernetes/scheme"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1alpha1"
"github.com/oam-dev/kubevela/pkg/workflow/step"
"github.com/pkg/errors"
"github.com/spf13/cobra"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
@@ -45,9 +52,10 @@ import (
// DryRunCmdOptions contains dry-run cmd options
type DryRunCmdOptions struct {
cmdutil.IOStreams
ApplicationFile string
DefinitionFile string
OfflineMode bool
ApplicationFiles []string
DefinitionFile string
OfflineMode bool
MergeStandaloneFiles bool
}
// NewDryRunCommand creates `dry-run` command
@@ -62,8 +70,26 @@ func NewDryRunCommand(c common.Args, ioStreams cmdutil.IOStreams) *cobra.Command
You can also specify a remote url for app:
vela dry-run -d /definition/directory/or/file/ -f https://remote-host/app.yaml
And more, you can specify policy and workflow with application file:
vela dry-run -d /definition/directory/or/file/ -f /path/to/app.yaml -f /path/to/policy.yaml -f /path/to/workflow.yaml, OR
vela dry-run -d /definition/directory/or/file/ -f /path/to/app.yaml,/path/to/policy.yaml,/path/to/workflow.yaml
Additionally, if the provided policy and workflow files are not referenced by application file, warning message will show up
and those files will be ignored. You can use "merge" flag to make those standalone files effective:
vela dry-run -d /definition/directory/or/file/ -f /path/to/app.yaml,/path/to/policy.yaml,/path/to/workflow.yaml --merge
Limitation:
1. Only support one object per file(yaml) for "-f" flag. More support will be added in the future improvement.
2. Dry Run with policy and workflow will only take override/topology policies and deploy workflow step into considerations. Other workflow step will be ignored.
`,
Example: `
# dry-run application
vela dry-run -f app.yaml
# dry-run application with policy and workflow
vela dry-run -f app.yaml -f policy.yaml -f workflow.yaml
`,
Example: "vela dry-run",
Annotations: map[string]string{
types.TagCommandType: types.TypeApp,
},
@@ -88,9 +114,10 @@ You can also specify a remote url for app:
},
}
cmd.Flags().StringVarP(&o.ApplicationFile, "file", "f", "./app.yaml", "application file name")
cmd.Flags().StringSliceVarP(&o.ApplicationFiles, "file", "f", []string{"app.yaml"}, "application related file names")
cmd.Flags().StringVarP(&o.DefinitionFile, "definition", "d", "", "specify a definition file or directory, it will only be used in dry-run rather than applied to K8s cluster")
cmd.Flags().BoolVar(&o.OfflineMode, "offline", false, "Run `dry-run` in offline / local mode, all validation steps will be skipped")
cmd.Flags().BoolVar(&o.MergeStandaloneFiles, "merge", false, "Merge standalone files to produce dry-run results")
addNamespaceAndEnvArg(cmd)
cmd.SetOut(ioStreams.Out)
return cmd
@@ -140,23 +167,21 @@ func DryRunApplication(cmdOption *DryRunCmdOptions, c common.Args, namespace str
// Perform validation only if not in offline mode
if !cmdOption.OfflineMode {
err = dryRunOpt.ValidateApp(ctx, cmdOption.ApplicationFile)
if err != nil {
return buff, errors.WithMessagef(err, "validate application: %s by dry-run", cmdOption.ApplicationFile)
for _, applicationFile := range cmdOption.ApplicationFiles {
err = dryRunOpt.ValidateApp(ctx, applicationFile)
if err != nil {
return buff, errors.WithMessagef(err, "validate application: %s by dry-run", applicationFile)
}
}
}
app, err := readApplicationFromFile(cmdOption.ApplicationFile)
app, err := readApplicationFromFiles(cmdOption, &buff)
if err != nil {
return buff, errors.WithMessagef(err, "read application file: %s", cmdOption.ApplicationFile)
return buff, errors.WithMessagef(err, "read application files: %s", cmdOption.ApplicationFiles)
}
comps, policies, err := dryRunOpt.ExecuteDryRun(ctx, app)
err = dryRunOpt.ExecuteDryRunWithPolicies(ctx, app, &buff)
if err != nil {
return buff, errors.WithMessage(err, "generate OAM objects")
}
if err = dryRunOpt.PrintDryRun(&buff, app.Name, comps, policies); err != nil {
return buff, err
}
return buff, nil
@@ -241,3 +266,140 @@ func readApplicationFromFile(filename string) (*corev1beta1.Application, error)
err = json.Unmarshal(fileContent, app)
return app, err
}
func readApplicationFromFiles(cmdOption *DryRunCmdOptions, buff *bytes.Buffer) (*corev1beta1.Application, error) {
var app *corev1beta1.Application
var policies []*v1alpha1.Policy
var wf *wfv1alpha1.Workflow
policyNameMap := make(map[string]struct{})
for _, filename := range cmdOption.ApplicationFiles {
fileContent, err := utils.ReadRemoteOrLocalPath(filename, true)
if err != nil {
return nil, err
}
fileType := filepath.Ext(filename)
switch fileType {
case ".yaml", ".yml":
// only support one object in one yaml file
fileContent, err = yaml.YAMLToJSON(fileContent)
if err != nil {
return nil, err
}
decode := scheme.Codecs.UniversalDeserializer().Decode
// cannot guarantee get the object, but gkv is enough
_, gkv, _ := decode(fileContent, nil, nil)
jsonFileContent, err := yaml.YAMLToJSON(fileContent)
if err != nil {
return nil, err
}
switch *gkv {
case corev1beta1.ApplicationKindVersionKind:
if app != nil {
return nil, errors.New("more than one applications provided")
}
app = new(corev1beta1.Application)
err = json.Unmarshal(jsonFileContent, app)
if err != nil {
return nil, err
}
case v1alpha1.PolicyGroupVersionKind:
policy := new(v1alpha1.Policy)
err = json.Unmarshal(jsonFileContent, policy)
if err != nil {
return nil, err
}
policies = append(policies, policy)
case v1alpha1.WorkflowGroupVersionKind:
if wf != nil {
return nil, errors.New("more than one external workflow provided")
}
wf = new(wfv1alpha1.Workflow)
err = json.Unmarshal(jsonFileContent, wf)
if err != nil {
return nil, err
}
default:
return nil, fmt.Errorf("file %s is not application, policy or workflow", filename)
}
}
}
// only allow one application
if app == nil {
return nil, errors.New("no application provided")
}
// workflow not referenced by application
if !cmdOption.MergeStandaloneFiles {
if wf != nil &&
((app.Spec.Workflow != nil && app.Spec.Workflow.Ref != wf.Name) || app.Spec.Workflow == nil) {
buff.WriteString(fmt.Sprintf("WARNING: workflow %s not referenced by application\n\n", wf.Name))
}
} else {
if wf != nil {
app.Spec.Workflow = &corev1beta1.Workflow{
Ref: "",
Steps: wf.Steps,
}
}
err := getPolicyNameFromWorkflow(wf, policyNameMap)
if err != nil {
return nil, err
}
}
for _, policy := range policies {
// check standalone policies
if _, exist := policyNameMap[policy.Name]; !exist && !cmdOption.MergeStandaloneFiles {
buff.WriteString(fmt.Sprintf("WARNING: policy %s not referenced by application\n\n", policy.Name))
continue
}
app.Spec.Policies = append(app.Spec.Policies, corev1beta1.AppPolicy{
Name: policy.Name,
Type: policy.Type,
Properties: policy.Properties,
})
}
return app, nil
}
func getPolicyNameFromWorkflow(wf *wfv1alpha1.Workflow, policyNameMap map[string]struct{}) error {
checkPolicy := func(wfsb wfv1alpha1.WorkflowStepBase, policyNameMap map[string]struct{}) error {
workflowStepSpec := &step.DeployWorkflowStepSpec{}
if err := utils.StrictUnmarshal(wfsb.Properties.Raw, workflowStepSpec); err != nil {
return err
}
for _, p := range workflowStepSpec.Policies {
policyNameMap[p] = struct{}{}
}
return nil
}
if wf == nil {
return nil
}
for _, wfs := range wf.Steps {
if wfs.Type == step.DeployWorkflowStep {
err := checkPolicy(wfs.WorkflowStepBase, policyNameMap)
if err != nil {
return err
}
for _, sub := range wfs.SubSteps {
if sub.Type == step.DeployWorkflowStep {
err = checkPolicy(sub, policyNameMap)
if err != nil {
return err
}
}
}
}
}
return nil
}

View File

@@ -17,24 +17,27 @@
package cli
import (
"bytes"
"context"
"os"
"strings"
wfv1alpha1 "github.com/kubevela/workflow/api/v1alpha1"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1alpha1"
. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
"sigs.k8s.io/yaml"
"github.com/oam-dev/kubevela/apis/core.oam.dev/v1beta1"
"github.com/oam-dev/kubevela/apis/types"
"github.com/oam-dev/kubevela/pkg/appfile/dryrun"
"github.com/oam-dev/kubevela/pkg/oam/discoverymapper"
common2 "github.com/oam-dev/kubevela/pkg/utils/common"
)
var _ = Describe("Test dry run with policy", func() {
It("Test dry run with normal policy", func() {
var _ = Describe("Testing dry-run", func() {
It("Testing dry-run", func() {
webservice, err := os.ReadFile("../../charts/vela-core/templates/defwithtemplate/webservice.yaml")
Expect(err).Should(BeNil())
webserviceYAML := strings.Replace(string(webservice), "{{ include \"systemDefinitionNamespace\" . }}", types.DefaultKubeVelaNS, 1)
@@ -42,100 +45,195 @@ var _ = Describe("Test dry run with policy", func() {
Expect(yaml.Unmarshal([]byte(webserviceYAML), &wwd)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &wwd)).Should(BeNil())
plcd := v1beta1.PolicyDefinition{}
Expect(yaml.Unmarshal([]byte(plcdef), &plcd)).Should(BeNil())
plcd.Namespace = types.DefaultKubeVelaNS
Expect(k8sClient.Create(context.TODO(), &plcd)).Should(BeNil())
app := v1beta1.Application{}
Expect(yaml.Unmarshal([]byte(plcapp), &app)).Should(BeNil())
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
pd, err := c.GetPackageDiscover()
scaler, err := os.ReadFile("../../charts/vela-core/templates/defwithtemplate/scaler.yaml")
Expect(err).Should(BeNil())
dm, err := discoverymapper.New(cfg)
Expect(err).Should(BeNil())
dryRunOpt := dryrun.NewDryRunOption(k8sClient, cfg, dm, pd, nil, false)
comps, plcs, err := dryRunOpt.ExecuteDryRun(context.TODO(), &app)
Expect(err).Should(BeNil())
speci := plcs[0].Object["spec"].(map[string]interface{})
Expect(speci["service"].(string)).Should(BeEquivalentTo("unified"))
buff := bytes.NewBufferString("")
Expect(dryRunOpt.PrintDryRun(buff, app.Name, comps, plcs)).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring(`backends:
- service: server-v1
weight: 80
- service: server-v2
weight: 20
service: unified`))
Expect(buff.String()).Should(ContainSubstring("- image: oamdev/hello-world:v1\n name: server-v1"))
Expect(buff.String()).Should(ContainSubstring("- image: oamdev/hello-world:v2\n name: server-v2"))
})
It("Test dry run with cue component format", func() {
scalerYAML := strings.Replace(string(scaler), "{{ include \"systemDefinitionNamespace\" . }}", types.DefaultKubeVelaNS, 1)
var td v1beta1.TraitDefinition
Expect(yaml.Unmarshal([]byte(scalerYAML), &td)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &td)).Should(BeNil())
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFile: "test-data/dry-run/app.yaml", DefinitionFile: "test-data/dry-run/my-comp.cue", OfflineMode: true}
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-1.yaml"}, OfflineMode: false}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("name: hello-world"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
Expect(buff.String()).Should(ContainSubstring("name: hello-world-service"))
Expect(buff.String()).Should(ContainSubstring("kind: Service"))
Expect(buff.String()).Should(ContainSubstring("replicas: 1"))
})
It("Testing dry-run with policy", func() {
deploy, err := os.ReadFile("../../charts/vela-core/templates/defwithtemplate/deploy.yaml")
Expect(err).Should(BeNil())
deployYAML := strings.Replace(string(deploy), "{{ include \"systemDefinitionNamespace\" . }}", types.DefaultKubeVelaNS, 1)
var wfsd v1beta1.WorkflowStepDefinition
Expect(yaml.Unmarshal([]byte(deployYAML), &wfsd)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &wfsd)).Should(BeNil())
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-2.yaml"}, OfflineMode: false}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app with topology target-default)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
Expect(buff.String()).Should(ContainSubstring("replicas: 1"))
})
It("Testing dry-run with workflow", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-3.yaml"}, OfflineMode: false}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app with topology target-default)"))
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app with topology target-prod)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
Expect(buff.String()).Should(ContainSubstring("replicas: 1"))
Expect(buff.String()).Should(ContainSubstring("replicas: 3"))
})
It("Testing dry-run with ref workflow", func() {
policy, err := os.ReadFile("test-data/dry-run/testing-policy.yaml")
Expect(err).Should(BeNil())
var p v1alpha1.Policy
Expect(yaml.Unmarshal([]byte(policy), &p)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &p)).Should(BeNil())
workflow, err := os.ReadFile("test-data/dry-run/testing-wf.yaml")
Expect(err).Should(BeNil())
var wf wfv1alpha1.Workflow
Expect(yaml.Unmarshal([]byte(workflow), &wf)).Should(BeNil())
Expect(k8sClient.Create(context.TODO(), &wf)).Should(BeNil())
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-4.yaml"}, OfflineMode: false}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app with topology deploy-somewhere)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
})
It("Testing dry-run without application provided", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-policy.yaml"}, OfflineMode: false}
_, err := DryRunApplication(&opt, c, "")
Expect(err).ShouldNot(BeNil())
Expect(err.Error()).Should(ContainSubstring("no application provided"))
})
It("Testing dry-run with more than one applications provided", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-1.yaml", "test-data/dry-run/testing-dry-run-2.yaml"}, OfflineMode: false}
_, err := DryRunApplication(&opt, c, "")
Expect(err).ShouldNot(BeNil())
Expect(err.Error()).Should(ContainSubstring("more than one applications provided"))
})
It("Testing dry-run with more than one workflow provided", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-1.yaml", "test-data/dry-run/testing-wf.yaml", "test-data/dry-run/testing-wf.yaml"}, OfflineMode: false}
_, err := DryRunApplication(&opt, c, "")
Expect(err).ShouldNot(BeNil())
Expect(err.Error()).Should(ContainSubstring("more than one external workflow provided"))
})
It("Testing dry-run with unexpected file", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-trait.yaml"}, OfflineMode: false}
_, err := DryRunApplication(&opt, c, "")
Expect(err).ShouldNot(BeNil())
Expect(err.Error()).Should(ContainSubstring("is not application, policy or workflow"))
})
It("Testing dry-run with unexpected file", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-trait.yaml"}, OfflineMode: false}
_, err := DryRunApplication(&opt, c, "")
Expect(err).ShouldNot(BeNil())
Expect(err.Error()).Should(ContainSubstring("is not application, policy or workflow"))
})
It("Testing dry-run merging with external workflow and policy", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-5.yaml", "test-data/dry-run/testing-wf.yaml", "test-data/dry-run/testing-policy.yaml"}, OfflineMode: false, MergeStandaloneFiles: true}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app with topology deploy-somewhere)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
})
It("Testing dry-run with standalone policy", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-5.yaml", "test-data/dry-run/testing-policy.yaml"}, OfflineMode: false, MergeStandaloneFiles: false}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("WARNING: policy deploy-somewhere not referenced by application"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
})
It("Testing dry-run with standalone workflow", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-5.yaml", "test-data/dry-run/testing-wf.yaml"}, OfflineMode: false, MergeStandaloneFiles: false}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("WARNING: workflow testing-wf not referenced by application"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
})
It("Testing dry-run offline", func() {
c := common2.Args{}
c.SetConfig(cfg)
c.SetClient(k8sClient)
opt := DryRunCmdOptions{ApplicationFiles: []string{"test-data/dry-run/testing-dry-run-6.yaml"}, DefinitionFile: "test-data/dry-run/testing-worker-def.yaml", OfflineMode: true}
buff, err := DryRunApplication(&opt, c, "")
Expect(err).Should(BeNil())
Expect(buff.String()).Should(ContainSubstring("# Application(testing-app)"))
Expect(buff.String()).Should(ContainSubstring("name: testing-dryrun"))
Expect(buff.String()).Should(ContainSubstring("kind: Deployment"))
Expect(buff.String()).Should(ContainSubstring("workload.oam.dev/type: myworker"))
})
})
var plcapp = `apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: my-test-2
spec:
components:
- name: server-v1
type: webservice
properties:
image: oamdev/hello-world:v1
- name: server-v2
type: webservice
properties:
image: oamdev/hello-world:v2
policies:
- type: my-plc
name: unified
properties:
weights:
- service: server-v1
weight: 80
- service: server-v2
weight: 20
`
var plcdef = `apiVersion: core.oam.dev/v1beta1
kind: PolicyDefinition
metadata:
annotations:
definition.oam.dev/description: My ingress route policy.
name: my-plc
spec:
schematic:
cue:
template: |
#ServerWeight: {
service: string
weight: int
}
parameter: weights: [...#ServerWeight]
output: {
apiVersion: "split.smi-spec.io/v1alpha3"
kind: "TrafficSplit"
metadata: name: context.name
spec: {
service: context.name
backends: parameter.weights
}
}`

View File

@@ -37,7 +37,9 @@ import (
// LiveDiffCmdOptions contains the live-diff cmd options
type LiveDiffCmdOptions struct {
DryRunCmdOptions
cmdutil.IOStreams
ApplicationFile string
DefinitionFile string
AppName string
Namespace string
Revision string
@@ -47,10 +49,8 @@ type LiveDiffCmdOptions struct {
// NewLiveDiffCommand creates `live-diff` command
func NewLiveDiffCommand(c common.Args, order string, ioStreams cmdutil.IOStreams) *cobra.Command {
o := &LiveDiffCmdOptions{
DryRunCmdOptions: DryRunCmdOptions{
IOStreams: ioStreams,
}}
o := &LiveDiffCmdOptions{IOStreams: ioStreams}
cmd := &cobra.Command{
Use: "live-diff",
DisableFlagsInUseLine: true,

View File

@@ -58,7 +58,7 @@ func NewSystemCommand(c common.Args) *cobra.Command {
cmd := &cobra.Command{
Use: "system",
Short: "Manage system.",
Long: "Manage system, incluing printing the system deployment information in vela-system namespace and diagnosing the system's health.",
Long: "Manage system, including printing the system deployment information in vela-system namespace and diagnosing the system's health.",
Example: "# Check all deployments information in all namespaces with label app.kubernetes.io/name=vela-core :\n" +
"> vela system info\n" +
"# Specify a deployment name with a namespace to check detail information:\n" +

View File

@@ -1,8 +0,0 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: vela-app
spec:
components:
- name: express-server
type: my-comp

View File

@@ -1,50 +0,0 @@
"my-comp": {
annotations: {}
attributes: workload: definition: {
apiVersion: "apps/v1"
kind: "Deployment"
}
description: "My component."
labels: {}
type: "component"
}
template: {
output: {
metadata: name: "hello-world"
spec: {
replicas: 1
selector: matchLabels: "app.kubernetes.io/name": "hello-world"
template: {
metadata: labels: "app.kubernetes.io/name": "hello-world"
spec: containers: [{
name: "hello-world"
image: "somefive/hello-world"
ports: [{
name: "http"
containerPort: 80
protocol: "TCP"
}]
}]
}
}
apiVersion: "apps/v1"
kind: "Deployment"
}
outputs: "hello-world-service": {
metadata: name: "hello-world-service"
spec: {
ports: [{
name: "http"
protocol: "TCP"
port: 80
targetPort: 8080
}]
selector: app: "hello-world"
type: "LoadBalancer"
}
apiVersion: "v1"
kind: "Service"
}
parameter: {}
}

View File

@@ -0,0 +1,16 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
traits:
- type: scaler
properties:
replicas: 1

View File

@@ -0,0 +1,20 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
traits:
- type: scaler
properties:
replicas: 1
policies:
- name: target-default
type: topology
properties:
clusters: [ "local" ]
namespace: "default"

View File

@@ -0,0 +1,44 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
traits:
- type: scaler
properties:
replicas: 1
policies:
- name: target-default
type: topology
properties:
clusters: [ "local" ]
namespace: "default"
- name: target-prod
type: topology
properties:
clusters: [ "local" ]
namespace: "prod"
- name: deploy-ha
type: override
properties:
components:
- type: webservice
traits:
- type: scaler
properties:
replicas: 3
workflow:
steps:
- name: deploy2default
type: deploy
properties:
policies: [ "target-default" ]
- name: deploy2prod
type: deploy
properties:
policies: [ "target-prod", "deploy-ha" ]

View File

@@ -0,0 +1,12 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1
workflow:
ref: testing-wf

View File

@@ -0,0 +1,11 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: webservice
properties:
image: oamdev/hello-world:v1

View File

@@ -0,0 +1,12 @@
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
name: testing-app
spec:
components:
- name: testing-dryrun
type: myworker
properties:
image: oamdev/hello-world:v1

View File

@@ -0,0 +1,8 @@
apiVersion: core.oam.dev/v1alpha1
kind: Policy
metadata:
name: deploy-somewhere
namespace: default
type: topology
properties:
clusters: ["anywhere"]

View File

@@ -0,0 +1,8 @@
apiVersion: core.oam.dev/v1alpha2
kind: TraitDefinition
metadata:
name: bars.example.com
namespace: default
spec:
definitionRef:
name: bars.example.com

View File

@@ -0,0 +1,10 @@
apiVersion: core.oam.dev/v1alpha1
kind: Workflow
metadata:
name: testing-wf
namespace: default
steps:
- type: deploy
name: deploy-somewhere
properties:
policies: ["deploy-somewhere"]

View File

@@ -0,0 +1,49 @@
apiVersion: core.oam.dev/v1beta1
kind: ComponentDefinition
metadata:
name: myworker
spec:
workload:
definition:
apiVersion: apps/v1
kind: Deployment
schematic:
cue:
template: |
output: {
apiVersion: "apps/v1"
kind: "Deployment"
}
output: {
spec: {
selector: matchLabels: {
"app.oam.dev/component": context.name
}
template: {
metadata: labels: {
"app.oam.dev/component": context.name
}
spec: {
containers: [{
name: context.name
image: parameter.image
if parameter["cmd"] != _|_ {
command: parameter.cmd
}
}]
}
}
}
}
parameter: {
// +usage=Which image would you like to use for your service
// +short=i
image: string
// +usage=Commands to run in the container
cmd?: [...string]
}

Some files were not shown because too many files have changed in this diff Show More