141 Commits

Author SHA1 Message Date
github-actions[bot]
e9f7cf7c23 Fix: add context parameters into the error message (#3159)
Signed-off-by: zeed-w-beez <zeed.w.zhao@gmail.com>
(cherry picked from commit 170888063d)

Co-authored-by: zeed-w-beez <zeed.w.zhao@gmail.com>
2022-01-25 13:28:01 +08:00
yangsoon
6cee468743 Feat: add more performance optimization and prometheus metrics for controller (#3086)
* Feat: add more prometheus metrics

Signed-off-by: yangsoon <yangsoonlx@gmail.com>

* Feat: add detail gc rt duration metrics

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add monitor to client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add all cache object

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: watch job

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add deleg client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for rt list and disable controllerrevision

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add apprev disable optimize

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: optimize log

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add time recorder for app ctrl

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add in-memory workflow context

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add reconcile-reduction & random-pick-gc & optimize rt record

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for healthcheck & resourcetracker trigger

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: record the resource-tracker number by informer event-handler

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Feat: add promethus collect annotation in template

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Fix: command line comment bug

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: rename args and remove legacy controller metrics

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: make code reviewable

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Chore: optimize flag descriptions

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: break optimize package

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: gc policy test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

Co-authored-by: Yin Da <yd219913@alibaba-inc.com>
Co-authored-by: yangsoon <songyang.song@alibaba-inc.com>
2022-01-14 15:18:02 +08:00
wyike
5b1c47c918 cue template health check skipStandard workload and fix test diverge (#3087)
Signed-off-by: wangyike <wangyike_wyk@163.com>

add annotation

Signed-off-by: wangyike <wangyike_wyk@163.com>

set skipStandardworkload in wl

Signed-off-by: wangyike <wangyike_wyk@163.com>

add annotation

Signed-off-by: wangyike <wangyike_wyk@163.com>
2022-01-13 19:08:43 +08:00
Jianbo Sun
12a106fcfe Fix: migrate test from crd v1beta1 to v1 (#3061)
Signed-off-by: Jianbo Sun <jianbo.sjb@alibaba-inc.com>
2022-01-10 10:25:35 +08:00
Zheng Xi Zhou
96485ea0db Fix: when a provider is set in ComponentDefinition, it should look for the provider (#3047)
* Fix: when a provider is set in ComponentDefinition, it should look for the provider

Fixed the issue "The error message is not correct when AWS provider is not available"

Fix #3046

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-01-06 19:02:03 +08:00
Tianxin Dong
6c4b7699b9 Fix: nit fix for workflow (#2957)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2021-12-20 11:00:07 +08:00
yangsoon
3b1f097807 Fix: change catlog helm repo path to https://charts.kubevela.net/example (#2942)
Signed-off-by: yangsoon <yangsoonlx@gmail.com>
2021-12-16 15:17:00 +08:00
Somefive
b622cbdb7f Feat: ResourceTracker new architecture (#2849)
* Feat: new rt

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: add publish version

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>
2021-12-10 15:00:03 +08:00
Zheng Xi Zhou
7d341c4bd3 Fix: pass on properties of Terraform typed application to Configuration (#2847)
* Fix: pass on properties of Terraform typed application to Configuration

Pass on properties like `writeConnectionSecretRef`, `Region` and `providerRef`
to Configuration.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* revert definitions

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2021-12-02 15:45:50 +08:00
Zheng Xi Zhou
92335eca17 Fix: support exposing cloud resource identifier and console url (#2825)
* Fix: support exposing cloud resource identifier and console url

To help users quickly navigate to the instance in the web console, exposing
necessary information in ComponentDefinition and convert them to Configuration

* fix ci

* Check whether wl.FullTemplate.ComponentDefinition is nil before getting annotations
2021-11-28 17:21:12 +08:00
Somefive
dd84700657 Fix: health check appRev label bug (#2816)
* Fix: healthscope failed since appRevision is not set && rollout controller amend appRevision to workload since compRev does not have this one

* Fix: test
2021-11-26 17:47:43 +08:00
yangsoon
3d2fcacb5a Feat: built in gc policy to application (#2575) 2021-11-24 15:38:13 +08:00
Somefive
64ab932c6d Feat: support apiserver-related multicluster features (#2625)
* Feat: remove envbinding policy into workflow

Feat: add support for env change (env gc)

Fix: fix rollout timeout setting bug

* Feat: support disable trait and env without workflow

* Fix: add hint for replaced value
2021-11-08 10:11:50 -05:00
Jianbo Sun
4ca68f2030 Fix: health check will check for multiclusters (#2645) 2021-11-07 11:50:27 +08:00
Somefive
bf3a1cddf9 Feat: remove envbinding policy into workflow (#2556)
Fix: add more test
2021-11-05 17:29:05 +08:00
Zheng Xi Zhou
e9ac4bb62f Fix: add another output CLUSTER_ID for Alibaba Cloud ack (#2543)
To support sub-path of a git repo which stores HCLs, also update the dependency
of Terraform controller
2021-10-24 10:02:39 +08:00
Tianxin Dong
85c887d3e4 Fix: change raw extension to pointer (#2451)
* Fix: change raw extension to pointer

* Test: fix ut
2021-10-13 16:16:53 +08:00
qiaozp
a574fc0fbf Refactor(cli): Refactor vela env, deprecate vela config (#2037)
* Refactor(cli): cut env to namespace, use application to save

Signed-off-by: qiaozp <chivalry.pp@gmail.com>

* Fix: test

* Fix: typo
2021-10-08 13:10:47 +08:00
Zheng Xi Zhou
b8894b7a44 Fix: support terraform/provider-azure addon (#2402)
Also added some componentdefinitions for Azure cloud resource
2021-10-08 11:35:52 +08:00
Zheng Xi Zhou
305403ff8b Support remote git repo for Terraform configuration (#2337)
* Fix: support remote git repo for Terraform configuration

Besides inline HCL Configuration, supprted remote git repo Configuration

* Fix: add unit-tests

* Fix: update all manifests in vela-minimal
2021-09-29 14:58:27 +08:00
Tianxin Dong
e6668ac390 Fix: do not override the workload name if its specified (#2336)
* Fix: do not override the workload name if its specified

* Fix: resolve comments
2021-09-23 15:05:25 +08:00
Tianxin Dong
8e172091f6 Test: fix resource tracker e2e (#2281) 2021-09-14 10:26:57 +08:00
Jianbo Sun
cad21c5138 Fix: refactor workflow and fix some e2e tests (#2272)
* Fix: add e2e plugin test back

* Fix: e2e rollout test and try change port to 37081

* Fix: add rollout plan test back

* Refactor: change the workflow state machine and add workflow succeed state

* Refactor: refine definition controller and fix e2e of app revision

* Refactor: unlock all the normal cases

* Fix: add helm schematic logic back into workflow
2021-09-13 14:17:12 +08:00
Jian.Li
445db3b276 Fix: E2E Test (#2274)
* Fix: value.lookUpByScript bug

* Fix: apply-component patch trait resource
2021-09-12 22:45:14 +08:00
Jian.Li
6cbdbe84b2 Refactor application code to make it run as Dag workflow (#2236)
* Refactor: remove use of AppConfig in AppRevision

* Refactor: remove insert secret and configmap

* Feat(workflow): upgrade

* Fix(conflict): workflow cherry

* Feat(workflow): support DAG mode

* Feat(workflow): prepare steps in step

* Feat(tools): random string

* Fix(rand): gosec weak random

* Fix(ci): test passing

* Feat(workflow): generate steps

* Fix: fix rebase from master

* Fix: fix workflow ut

* Feat(test): add test cases

* Fix: fix lint and rebase from master

* Refactor: application code

* Fix: fix ci lint

* Fix: make code reviewable

* Fix: workflow_test.go

* Feat: collect services

* Fix(ci): unit tests

* Feat: make one

* Test: application with input/output and workflow

* Fix: trace test

* Fix: update step index falied

* Feat: refactor op.#Load

* Fix: delete dead code

* Refactor: op.xxx

* Fix: patch component

* Test: add generator test

* Fix: add license

* Fix: pending e2e plugin test

* Fix: disable test/e2e-test

* Fix: patch by script

Co-authored-by: 天元 <jianbo.sjb@alibaba-inc.com>
Co-authored-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2021-09-12 10:12:46 +08:00
Eng Zer Jun
426aa7af34 Refactor: move from io/ioutil to io and os package (#2234)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <zerjun@beatchain.co>
2021-09-06 18:33:42 +08:00
yangsoon
cec16cc1a4 Fix(webhook): only check the uniqueness of component names under the same namespace (#2222) 2021-09-02 14:09:45 +08:00
Jianbo Sun
0781937100 Fix: align all CUE template keyword to use parameter (#2181) 2021-08-30 11:43:20 +08:00
Yue Wang
9780ce3bef Feat(cue): support access components artifacts in cue template context (#2161)
Signed-off-by: roywang <seiwy2010@gmail.com>

Co-authored-by: roywang <royyuewang@tencent.com>
2021-08-29 09:11:10 +08:00
Yue Wang
324fc1ac7c add CUE-based health check in HealthScope controller (#1956)
add e2e test

Signed-off-by: Yue Wang <seiwy2010@gmail.com>
2021-08-20 15:55:47 +08:00
Tianxin Dong
bd91cf762d Fix(workflow): set the namespace to app's ns if it's not specified (#2133)
* Fix(workflow): set the namespace to app's ns if it's not specified

* fix dryrun test
2021-08-20 13:43:04 +08:00
Zhizhen He
1e294b3164 chore: replace github.com/ghodss/yaml with sigs.k8s.io/yaml (#2089) 2021-08-15 10:53:05 +08:00
LeoLiuYan
a715564147 Feat allow users to specify component revision name (#1929)
* Feat allow users to specify component revision name

* Add unit tests

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add ExternalRevision field to ComponentManifest

* Add handleComponentRevisionNameUnspecified func

* Add unit tests for webhook

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add more unit tests

* Feat allow users to specify component revision name

* Add unit tests

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add ExternalRevision field to ComponentManifest

* Add handleComponentRevisionNameUnspecified func

* Add unit tests for webhook

Signed-off-by: LeoLiuYan <929908264@qq.com>

* Add more unit tests

* Try to fix webhook unit tests

* fix race (#2040)

* fix test bug

* More unit tst

* More unit test and trigger ci

Co-authored-by: yangsoon <yangsoonlx@gmail.com>
Co-authored-by: Yin Da <yd219913@alibaba-inc.com>
2021-08-08 12:11:24 +08:00
Gesa Stupperich
2ee3256b20 Store component parameters in context (#2030) 2021-08-06 10:48:00 +08:00
yangsoon
0752244d0b enable policy (#2011) 2021-08-04 18:58:57 +08:00
Jianbo Sun
aa87d3da24 upgrade K8s dependency lib to v0.21 (#1985)
* upgrade K8s dependency to v0.21

* update CRD for new version of controller-runtime

* fix ci component revision create must fill Raw in runtime.RawExtension

* try fix test

* start control plane timeout set to 1min and fix tests

* add timeout for env test start and stop

* longer timeout time for BeforeSuit function

* upgrade kubebuilder and k8s cluster version to match with v1.21.2 in github action

* fix test

* fix resource tracker ownerRef override

* update developer guides
2021-08-03 11:30:02 +08:00
Jianbo Sun
b88ac35779 don't filter not found error in load template to avoid panic (#1994)
* should return error if type not found to avoid panic

* don't filter not found error in load template to avoid panic
2021-07-31 07:11:07 +08:00
yangsoon
7161959aca add EnvBinding Controller for Multi cluster deploy and Configuration Management (#1968)
* patch env configuration

* add e2e-test
2021-07-30 10:02:51 +08:00
Jian.Li
a736b1f7b0 Workflow Support Resource GC (#1970)
* gc

* add test cases

* test case
2021-07-27 19:22:05 +08:00
Hongchao Deng
ed0039e1ee [Feature] CUE Based Workflow Step (#1935)
* Workflow WIP (#1934)

* interface

* add model value

* walk fields

* provider

* stdlin

* tasks

* add stdlib

* Adjust TaskGenerator

* Unified configmap format

* development completed

* fix import cycle

* demo passing

* upgrade action

* add value test cases

* format

* apis

* add context test cases

* fmt

* fmt

* add context to taskGenerator (#1942)

* fix configmap test

* add workflow integration test

* upgrade code

* initial workflow commit

* interface

* add model value

* walk fields

* provider

* stdlin

* tasks

* add stdlib

* Adjust TaskGenerator

* Unified configmap format

* development completed

* fix import cycle

* demo passing

* upgrade action

* add value test cases

* format

* apis

* add context test cases

* fmt

* fmt

* add context to taskGenerator (#1942)

* fix configmap test

* add workflow integration test

* lint code

* improve pkg[value、context] testing coverage

* Improve testing coverage(#1957)

* add license

* add test cases

Co-authored-by: Jian.Li <74582607+leejanee@users.noreply.github.com>
Co-authored-by: Jian.Li <lj176172@alibaba-inc.com>
2021-07-22 18:53:30 +08:00
yangsoon
0611c0a79a fix bug: failed to aggregate status if application use a kube/helm type component (#1866)
* fix aggregate status

* add test
2021-07-08 15:16:55 +08:00
yangsoon
01ed9a647c TraitDefinition support use insertSecretTo in template to consumer cloud resource (#1849)
* traitdef support insertSecretTo

* add test
2021-07-04 21:40:43 +08:00
Yue Wang
9adf269869 support helm repo secret (#1872)
Signed-off-by: roywang <seiwy2010@gmail.com>
2021-07-04 21:39:48 +08:00
天元
facd56fd4f improve workflow docs and update workflow API 2021-07-02 18:06:37 +08:00
Jianbo Sun
bd41d49311 allow component to defer insert secret and config (#1869) 2021-07-02 15:09:45 +08:00
yangsoon
aec9913557 fix invalid metadata.labels error when specify the version of the trait in the app (#1855)
* fix trait name

* add test
2021-06-29 15:53:57 +08:00
Chwetion
b582ccc060 add application component name duplication check in webhook (#1811)
* add application component name duplication check in webhook

* Helm test will not clean up the test data, and the application test of the component with the same name created failed during e2e.

* fix the problems encountered during the test:

* ignore the parsing failure caused by the existing application without definition when checking;

* after the addition is successful, the resources left by the helm test are automatically cleaned up;

Co-authored-by: chwetion <chwetion@foxmail.com>
2021-06-24 16:47:17 +08:00
Yue Wang
5a778b4732 add API v1beta1.ComponentManifest (#1814)
rfc appfile pkg: remove appConfig&component

fix unit test

rfc vela cli dryrun & livediff

fix unit test & e2e test

rfc app controller: remove appConfig&component

fix unit test & e2e test

fix rollout and webhook

fix use AppRevision as others' owner

move ComponentManifest to oam.types

add a util to make conversion between AppConfig and ComponentManifest

complete Component revision mechanism in app controller

add test for component revision

rename

fix unit tests

add clean up for component revision

fix unit test

Signed-off-by: roywang <seiwy2010@gmail.com>
2021-06-24 15:06:58 +08:00
Hongchao Deng
e5c8f259f3 app controller: reconcile workflow and create res configmap (#1788) 2021-06-15 13:40:12 +08:00
Zheng Xi Zhou
8dcbb61cb8 Generate OpenAPI JSON schema for Terraform Component (#1738)
Fix #1736
2021-06-03 11:23:20 +08:00