73 Commits

Author SHA1 Message Date
github-actions[bot]
6bbce07a21 Feat: add mode in workflow template (#4960)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
(cherry picked from commit 19a30371dd)

Co-authored-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-10-31 11:09:54 +08:00
Somefive
cea9ef5c97 Chore: use functions from kubevela/pkg (#4693)
* Chore: use functions from kubevela/pkg

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: rename multiclusterpkg to pkgmulticluster

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-09-09 15:10:18 +08:00
Tianxin Dong
239c5474dd Chore: refactor workflow from workflow engine (#4631)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-09-02 12:55:03 +08:00
qiaozp
924d55381e Feat: component replication (#4449) 2022-08-17 14:04:57 +08:00
Charlie Chiang
2d8cb1278c Fix: reject applications with empty policy properties (#4563)
* Fix: reject applications with empty policies

Signed-off-by: Charlie Chiang <charlie_c_0129@outlook.com>

* Style: change err msg

Signed-off-by: Charlie Chiang <charlie_c_0129@outlook.com>

* Fix: use 400 instead of 422 to show err msg

Signed-off-by: Charlie Chiang <charlie_c_0129@outlook.com>

* Test: fix tests

Signed-off-by: Charlie Chiang <charlie_c_0129@outlook.com>
2022-08-05 15:03:52 +08:00
Somefive
e595879da9 Feat: add featuregates to disallow url in ref-objects (#4446)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-07-25 22:03:10 +08:00
Somefive
92fa67cd69 Feat: support url in ref-objects (#4240)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-06-24 19:34:51 +08:00
Somefive
9c7d3f408d Feat: support shared resource (#4213)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-06-24 10:56:27 +08:00
Tianxin Dong
e10928d96f Feat: add mode and meta in workflow (#4206)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-06-23 10:55:34 +08:00
Somefive
383a2bd695 Feat: minimize controller privileges & enforce authentication in multicluster e2e test (#4031)
* Feat: enable auth in multicluster test & restrict controller privileges while enabling authentication

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: fix statekeep permission leak & comprev cleanup leak

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: use user info in ref-object select

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: set legacy-rt-gc to disabled by default

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: pending healthscope with authentication test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-05-28 01:26:06 +08:00
zq200618
7d9c647b1a Feat: support step group with substep in the workflow (#3772)
Signed-off-by: zhengq2006 <zhengq200618@cmbchina.com>

workflow substep develop

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>

Feat: support step group with substep in the workflow

Signed-off-by: Qiang Zheng <zhengq20018@cmbchina.com>
2022-05-23 10:26:21 +08:00
cezhang
877ab3909d Fix: update latest version Fix: 1.2 upgrade 1.3 workflowstep XXX not found (#3804)
* Fix: 1.2 upgrade 1.3 workflowstep XXX not found

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

handle publishversion case

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

* add test

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

add test

Signed-off-by: cezhang <c1zhang.dev@gmail.com>

lint code

Signed-off-by: cezhang <c1zhang.dev@gmail.com>
2022-05-07 12:07:26 +08:00
Tianxin Dong
21216055fb Feat: add vela debug command (#3580)
* Feat: add debug configmap if debug policy is specified

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* Feat: add vela debug command

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* make code reviewable

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix sonartype lift

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix cue string

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* Feat: display better for debug

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* tidy the go mod

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* Feat: add debug test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* change uitable vendor

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* add more tests

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* pass resource keeper from handler

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix lint

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix rebase

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* Pending test temporary

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-04-18 11:06:14 +08:00
Somefive
160ef64855 Feat: support livediff for referred object (#3525)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-28 17:52:33 +08:00
Somefive
c4a8fcf29c Feat: rework referred objects for version (#3446)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-28 16:09:12 +08:00
Somefive
741544c00c Feat: rework apprev (#3498)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-24 14:40:19 +08:00
Somefive
aeaa199982 Fix: remove useless envbinding crd (#3379)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-08 13:40:05 +08:00
Somefive
de81c24f42 Feat: support standalone style multi-cluster (#3223)
* Feat: ref component

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support topology and override

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add support for external policy and workflow

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add admission control

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: disable cross namespace ref object

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support labelSelector in ref-objects

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add pre approve for deploy step

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support comp/trait type in override policy even not used by prototype

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support regex match for patch component name

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: labelSelector not work for cluster

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: ref workflow contains external policy

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: revision test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: parallel apply components

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add test for oam provider

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: service ref-comp & indirect trait ns

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: align namespace setting for chart

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add strict unmarshal and reformat

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: merge with cluster rework

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: patch trait-def

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: apply components + load dynamic component

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for loadPoliciesInOrder

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add test for open merge

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: reformat & add test for step generator

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for parse override policy related defs

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for multicluster provider (expandTopology and overrideConfiguration)

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add admission test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: revert trait status pass in component status

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for dependency in workflowstep & standalone multicluster test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add check for ref and steps in WorkflowStep & enhance ref-objects scheme check

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-07 10:21:00 +08:00
yangsoon
6cee468743 Feat: add more performance optimization and prometheus metrics for controller (#3086)
* Feat: add more prometheus metrics

Signed-off-by: yangsoon <yangsoonlx@gmail.com>

* Feat: add detail gc rt duration metrics

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add monitor to client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add all cache object

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: watch job

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add deleg client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for rt list and disable controllerrevision

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add apprev disable optimize

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: optimize log

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add time recorder for app ctrl

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add in-memory workflow context

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add reconcile-reduction & random-pick-gc & optimize rt record

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for healthcheck & resourcetracker trigger

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: record the resource-tracker number by informer event-handler

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Feat: add promethus collect annotation in template

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Fix: command line comment bug

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: rename args and remove legacy controller metrics

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: make code reviewable

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Chore: optimize flag descriptions

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: break optimize package

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: gc policy test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

Co-authored-by: Yin Da <yd219913@alibaba-inc.com>
Co-authored-by: yangsoon <songyang.song@alibaba-inc.com>
2022-01-14 15:18:02 +08:00
Tianxin Dong
6c4b7699b9 Fix: nit fix for workflow (#2957)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2021-12-20 11:00:07 +08:00
Somefive
b622cbdb7f Feat: ResourceTracker new architecture (#2849)
* Feat: new rt

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: add publish version

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>
2021-12-10 15:00:03 +08:00
Somefive
dd84700657 Fix: health check appRev label bug (#2816)
* Fix: healthscope failed since appRevision is not set && rollout controller amend appRevision to workload since compRev does not have this one

* Fix: test
2021-11-26 17:47:43 +08:00
yangsoon
3d2fcacb5a Feat: built in gc policy to application (#2575) 2021-11-24 15:38:13 +08:00
Somefive
64ab932c6d Feat: support apiserver-related multicluster features (#2625)
* Feat: remove envbinding policy into workflow

Feat: add support for env change (env gc)

Fix: fix rollout timeout setting bug

* Feat: support disable trait and env without workflow

* Fix: add hint for replaced value
2021-11-08 10:11:50 -05:00
Tianxin Dong
85c887d3e4 Fix: change raw extension to pointer (#2451)
* Fix: change raw extension to pointer

* Test: fix ut
2021-10-13 16:16:53 +08:00
Jianbo Sun
cad21c5138 Fix: refactor workflow and fix some e2e tests (#2272)
* Fix: add e2e plugin test back

* Fix: e2e rollout test and try change port to 37081

* Fix: add rollout plan test back

* Refactor: change the workflow state machine and add workflow succeed state

* Refactor: refine definition controller and fix e2e of app revision

* Refactor: unlock all the normal cases

* Fix: add helm schematic logic back into workflow
2021-09-13 14:17:12 +08:00
Jian.Li
6cbdbe84b2 Refactor application code to make it run as Dag workflow (#2236)
* Refactor: remove use of AppConfig in AppRevision

* Refactor: remove insert secret and configmap

* Feat(workflow): upgrade

* Fix(conflict): workflow cherry

* Feat(workflow): support DAG mode

* Feat(workflow): prepare steps in step

* Feat(tools): random string

* Fix(rand): gosec weak random

* Fix(ci): test passing

* Feat(workflow): generate steps

* Fix: fix rebase from master

* Fix: fix workflow ut

* Feat(test): add test cases

* Fix: fix lint and rebase from master

* Refactor: application code

* Fix: fix ci lint

* Fix: make code reviewable

* Fix: workflow_test.go

* Feat: collect services

* Fix(ci): unit tests

* Feat: make one

* Test: application with input/output and workflow

* Fix: trace test

* Fix: update step index falied

* Feat: refactor op.#Load

* Fix: delete dead code

* Refactor: op.xxx

* Fix: patch component

* Test: add generator test

* Fix: add license

* Fix: pending e2e plugin test

* Fix: disable test/e2e-test

* Fix: patch by script

Co-authored-by: 天元 <jianbo.sjb@alibaba-inc.com>
Co-authored-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2021-09-12 10:12:46 +08:00
yangsoon
cec16cc1a4 Fix(webhook): only check the uniqueness of component names under the same namespace (#2222) 2021-09-02 14:09:45 +08:00
Jianbo Sun
0781937100 Fix: align all CUE template keyword to use parameter (#2181) 2021-08-30 11:43:20 +08:00
Yue Wang
324fc1ac7c add CUE-based health check in HealthScope controller (#1956)
add e2e test

Signed-off-by: Yue Wang <seiwy2010@gmail.com>
2021-08-20 15:55:47 +08:00
LeoLiuYan
a715564147 Feat allow users to specify component revision name (#1929)
* Feat allow users to specify component revision name

* Add unit tests

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add ExternalRevision field to ComponentManifest

* Add handleComponentRevisionNameUnspecified func

* Add unit tests for webhook

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add more unit tests

* Feat allow users to specify component revision name

* Add unit tests

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add ExternalRevision field to ComponentManifest

* Add handleComponentRevisionNameUnspecified func

* Add unit tests for webhook

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add more unit tests

* Try to fix webhook unit tests

* fix race (#2040)

* fix test bug

* More unit tst

* More unit test and trigger ci

Co-authored-by: yangsoon <yangsoonlx@gmail.com>
Co-authored-by: Yin Da <yd219913@alibaba-inc.com>
2021-08-08 12:11:24 +08:00
yangsoon
0752244d0b enable policy (#2011) 2021-08-04 18:58:57 +08:00
Jianbo Sun
b88ac35779 don't filter not found error in load template to avoid panic (#1994)
* should return error if type not found to avoid panic

* don't filter not found error in load template to avoid panic
2021-07-31 07:11:07 +08:00
yangsoon
7161959aca add EnvBinding Controller for Multi cluster deploy and Configuration Management (#1968)
* patch env configuration

* add e2e-test
2021-07-30 10:02:51 +08:00
Hongchao Deng
ed0039e1ee [Feature] CUE Based Workflow Step (#1935)
* Workflow WIP (#1934)

* interface

* add model value

* walk fields

* provider

* stdlin

* tasks

* add stdlib

* Adjust TaskGenerator

* Unified configmap format

* development completed

* fix import cycle

* demo passing

* upgrade action

* add value test cases

* format

* apis

* add context test cases

* fmt

* fmt

* add context to taskGenerator (#1942)

* fix configmap test

* add workflow integration test

* upgrade code

* initial workflow commit

* interface

* add model value

* walk fields

* provider

* stdlin

* tasks

* add stdlib

* Adjust TaskGenerator

* Unified configmap format

* development completed

* fix import cycle

* demo passing

* upgrade action

* add value test cases

* format

* apis

* add context test cases

* fmt

* fmt

* add context to taskGenerator (#1942)

* fix configmap test

* add workflow integration test

* lint code

* improve pkg[value、context] testing coverage

* Improve testing coverage(#1957)

* add license

* add test cases

Co-authored-by: Jian.Li <74582607+leejanee@users.noreply.github.com>
Co-authored-by: Jian.Li <lj176172@alibaba-inc.com>
2021-07-22 18:53:30 +08:00
yangsoon
01ed9a647c TraitDefinition support use insertSecretTo in template to consumer cloud resource (#1849)
* traitdef support insertSecretTo

* add test
2021-07-04 21:40:43 +08:00
天元
facd56fd4f improve workflow docs and update workflow API 2021-07-02 18:06:37 +08:00
Jianbo Sun
bd41d49311 allow component to defer insert secret and config (#1869) 2021-07-02 15:09:45 +08:00
yangsoon
aec9913557 fix invalid metadata.labels error when specify the version of the trait in the app (#1855)
* fix trait name

* add test
2021-06-29 15:53:57 +08:00
Chwetion
b582ccc060 add application component name duplication check in webhook (#1811)
* add application component name duplication check in webhook

* Helm test will not clean up the test data, and the application test of the component with the same name created failed during e2e.

* fix the problems encountered during the test:

* ignore the parsing failure caused by the existing application without definition when checking;

* after the addition is successful, the resources left by the helm test are automatically cleaned up;

Co-authored-by: chwetion <chwetion@foxmail.com>
2021-06-24 16:47:17 +08:00
Hongchao Deng
e5c8f259f3 app controller: reconcile workflow and create res configmap (#1788) 2021-06-15 13:40:12 +08:00
Jian.Li
4f47bec238 refactor cue related packages in vela (#1734)
* refactor-cue

* import group

* readme

* fmt
2021-06-02 15:37:06 +08:00
Hongchao Deng
fa8b3ef763 refactor appfile pkg for adding workflow (#1732) 2021-05-30 22:57:03 +08:00
yangsoon
1660930ed3 applicaiton supports specifying different versions of Definition (#1597)
* app support specify the version of definition

* add e2e-test

* add docs

* add helm related test

* fix doc

* add more test

* fix docs
2021-05-07 17:52:44 +08:00
Yue Wang
63b348f4d8 implement cli vela system live-diff (#1419)
move dry-run and live-diff into an independent pkg

WIP add sample and doc

WIP unit test for live-diff

add unit test

Signed-off-by: roy wang <seiwy2010@gmail.com>
2021-04-13 12:29:25 +08:00
LeoLiuYan
389722a94e Remove duplicated logic (#1465)
Signed-off-by: LeoLiuYan <929908264@qq.com>
2021-04-12 10:24:34 +08:00
Yue Wang
2d6f2083db refactor pkg/appfile (#1447)
Signed-off-by: roy wang <seiwy2010@gmail.com>
2021-04-11 14:10:16 +08:00
Herman Zhu
5b1b054cca refactor(workload) remove all the duplicate fields (#1410)
remove all the duplicate fields in parser above as template contains

fix #1388

Signed-off-by: zhuhuijun <zhuhuijunzhj@gmail.com>
2021-04-02 10:24:27 +08:00
天元
05c0fb8f9a fix test 2021-03-29 19:45:29 +08:00
Zheng Xi Zhou
9e40b77b60 Support cloud resource provisiong and consuming (#1264)
* Support cloud resource provisioning and consuming (Crossplane)

Provided a way to store secret for cloud resource generated by
Crossplane and to consume the secret

Refer to #1128

* Separate cloud resource producer and consumer in two applications

* add unit test to check whether application can consume cloud resource secret

* update Cloud Resource doc

* Provisioning and consuming cloud resource in different applications v1 (one cloud resource)

* one component consumes two cloud resources

Co-authored-by: Jianbo Sun <wonderflow.sun@gmail.com>
2021-03-29 17:20:33 +08:00