Commit Graph

482 Commits

Author SHA1 Message Date
Sunghoon Kang
1300a980f0 Feat: reconcile app with scoped permissions (#3434)
* Refactor: refactor multi cluster round trippers

Before adding more RoundTrippers, it would be better to expose common
logic in the utility package.

This commit exports `tryCancelRequest` at `utils` package, and make
`secretMultiClusterRoundTripper` implement `RoundTripperWrapper`
interface to allow chaining multiple round trippers.

Refs #3432

Signed-off-by: Sunghoon Kang <hoon@linecorp.com>

* Feat: reconcile app with scoped permissions

Currently, all Application resources are reconciled by the Roles bound
to the controller service account. This behavior gives us the power to
manage resources across multiple namespaces. However, this behavior can
be problematic in the soft-multitenancy environment.

This commit adds `serviceAccountName` to ApplicationSepc to reconcile
Application with the given service account for reconciling Application
with scoped permissions.

Refs #3432

Signed-off-by: Sunghoon Kang <hoon@linecorp.com>

* Refactor: extract context setter as method

https://github.com/oam-dev/kubevela/pull/3434#discussion_r825561603

Signed-off-by: Sunghoon Kang <hoon@linecorp.com>

* Feat: use annotation instead of spec

https://github.com/oam-dev/kubevela/issues/3432#issuecomment-1066460269

Signed-off-by: Sunghoon Kang <hoon@linecorp.com>

* Refactor: unify service account setter caller

https://github.com/oam-dev/kubevela/pull/3434#discussion_r825853612

Signed-off-by: Sunghoon Kang <hoon@linecorp.com>

* Refactor: rename GetServiceAccountName

https://github.com/oam-dev/kubevela/pull/3434#discussion_r826514565

Signed-off-by: Sunghoon Kang <hoon@linecorp.com>
2022-03-15 11:55:50 +08:00
Somefive
19424cfaa4 Fix: add state keep for healthy suspend (#3426)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-14 14:38:43 +08:00
Somefive
d8f95c9cf8 Feat: support multicluster service in application status (#3387)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-10 10:21:56 +08:00
Tianxin Dong
bef24bdf22 Fix: separate waiting and failed backoff time and make them configurable (#3362)
* Fix: separate waiting and failed backoff time and make them configurable

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* change var name in chart values

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix ci

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-03-08 16:04:19 +08:00
Somefive
de81c24f42 Feat: support standalone style multi-cluster (#3223)
* Feat: ref component

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support topology and override

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add support for external policy and workflow

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add admission control

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: disable cross namespace ref object

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support labelSelector in ref-objects

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add pre approve for deploy step

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support comp/trait type in override policy even not used by prototype

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: support regex match for patch component name

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: labelSelector not work for cluster

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: ref workflow contains external policy

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: revision test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: parallel apply components

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add test for oam provider

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: service ref-comp & indirect trait ns

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: align namespace setting for chart

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add strict unmarshal and reformat

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: merge with cluster rework

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: patch trait-def

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: apply components + load dynamic component

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for loadPoliciesInOrder

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: add test for open merge

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: reformat & add test for step generator

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for parse override policy related defs

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for multicluster provider (expandTopology and overrideConfiguration)

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add admission test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: revert trait status pass in component status

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test for dependency in workflowstep & standalone multicluster test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add check for ref and steps in WorkflowStep & enhance ref-objects scheme check

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-03-07 10:21:00 +08:00
Tianxin Dong
d6cf2cdd2f Fix: add process context in workflow (#3318)
* Fix: add process context in workflow

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* add context data in process context

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* delete usesless func

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix ut

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* use multi cluster ctx in process ctx

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* remove debug log

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-02-28 10:30:55 +08:00
Zheng Xi Zhou
37a523cbf7 Fix: not steady unit test (#3293)
The UT `TestGetOpenAPISchemaFromTerraformComponentDefinition` won't
succeed 100%. The required variables in the generated isn't in the expected
order all the time.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-18 15:15:10 +08:00
Zheng Xi Zhou
46967daf11 Fix: fixed required items for a Terraform ComponentDefinition (#3257)
If a Terraform variable is required, the item in OpenAPI schema
is required.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-15 18:53:40 +08:00
yangs
65522cabdb Fix: fix panic when user disable create apprevision (#3244)
Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

Co-authored-by: yangsoon <songyang.song@alibaba-inc.com>
2022-02-14 17:54:41 +08:00
Somefive
d7261cd965 Fix: disable cochange for apprev when def changes (#3217)
* Fix: disable cochange for apprev when def changes

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: add test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-02-10 11:40:46 +08:00
Zheng Xi Zhou
24f147a72c Fix: support more Terraform variable types (#3204)
Support Any, set and some complicated variable types

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-08 18:48:38 +08:00
yangs
91125c6fec Feat: dispatch manifests in concurrent (#3060)
* Feat: dispatch manifests in concurrent

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Fix: merge workflow pkg convert to pkg util

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

Co-authored-by: yangsoon <songyang.song@alibaba-inc.com>
2022-02-08 16:25:57 +08:00
Jian.Li
68ada429f0 Feat: application support controller requirement (#3192)
* application controller version control

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>

* modify command arg name

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>
2022-02-08 15:44:38 +08:00
Zheng Xi Zhou
2c41ee9c3c Fix: add unit test for getting Terraform Configuration from remote git (#3193)
Add another unit test when the configuration of Terraform locates
in a subpath of a git remote repository

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-08 10:16:13 +08:00
Zheng Xi Zhou
7895560675 Fix: retrieve Terraform variables from variables.tf (#3149)
* Fix: retrieve Terraform variables from variables.tf

If Terraform modules/resources are stored in remote git repos, get
variables from file `varialbes.tf`, or from `main.tf`.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-01-24 11:52:07 +08:00
Tianxin Dong
cb7bc0e7f4 Fix: handle workflow cache reconcile (#3128)
* Fix: handle workflow cache reconcile

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix return and move backoff to memory

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* handle failed to patch case

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* add store in err case

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* make reviewable

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix ut

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* do cleanup in ut

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-01-22 15:54:17 +08:00
Jian.Li
354e92cfbd Fix: workflow skip executing all steps occasionally (#3025)
* fix asi

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>

* fix lint

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>

* add trace tag

* add args for this feature

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>

* enable-asi-compatibility

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>
2022-01-19 16:07:19 +08:00
chwetion
951506b79b fix revision will change when add new trait with skiprevisionaffect to application (#3032)
Signed-off-by: chwetion <chwetion@foxmail.com>

Co-authored-by: chwetion <chwetion@foxmail.com>
2022-01-18 13:40:30 +08:00
Somefive
12f392cd92 Fix: rollout workload namespace not aligned with rollout (#3107)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-01-17 20:06:27 +08:00
yangsoon
6cee468743 Feat: add more performance optimization and prometheus metrics for controller (#3086)
* Feat: add more prometheus metrics

Signed-off-by: yangsoon <yangsoonlx@gmail.com>

* Feat: add detail gc rt duration metrics

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add monitor to client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add all cache object

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: watch job

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add deleg client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for rt list and disable controllerrevision

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add apprev disable optimize

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: optimize log

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add time recorder for app ctrl

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add in-memory workflow context

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add reconcile-reduction & random-pick-gc & optimize rt record

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for healthcheck & resourcetracker trigger

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: record the resource-tracker number by informer event-handler

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Feat: add promethus collect annotation in template

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Fix: command line comment bug

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: rename args and remove legacy controller metrics

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: make code reviewable

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Chore: optimize flag descriptions

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: break optimize package

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: gc policy test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

Co-authored-by: Yin Da <yd219913@alibaba-inc.com>
Co-authored-by: yangsoon <songyang.song@alibaba-inc.com>
2022-01-14 15:18:02 +08:00
Somefive
8b5d7ed395 Feat: reset re-sync time (#3008)
* Feat: set resync

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: rename re-sync period

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor return value

Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-01-13 20:12:54 +08:00
wyike
5b1c47c918 cue template health check skipStandard workload and fix test diverge (#3087)
Signed-off-by: wangyike <wangyike_wyk@163.com>

add annotation

Signed-off-by: wangyike <wangyike_wyk@163.com>

set skipStandardworkload in wl

Signed-off-by: wangyike <wangyike_wyk@163.com>

add annotation

Signed-off-by: wangyike <wangyike_wyk@163.com>
2022-01-13 19:08:43 +08:00
Minglong Song
b30ca0c8a2 Fix: Do not fiter event triggered by resync for application controller (#3064)
Signed-off-by: songminglong <1247768069@qq.com>
2022-01-10 14:48:58 +08:00
Zheng Xi Zhou
1a50dd76b5 Fix: generate docs for Terraform ComponentDefinition (#3051)
* Fix: generate docs for Terraform ComponentDefinition

Generated kubevela.io docs for Terraform typed ComponentDefinition
with `vela def gen-doc xxx` cli.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* Refactor code

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* add ut

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* add ut

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* refine cloud resource title

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-01-10 14:08:48 +08:00
wyike
770894412a Chore: deprecate appDeployment related logic (#3050)
* deperecated appDeployment

Signed-off-by: wangyike <wangyike_wyk@163.com>

* delete more code

Signed-off-by: wangyike <wangyike_wyk@163.com>
2022-01-06 16:24:09 +08:00
wyike
53081d043b Chore: deprecate and delete approllout related code (#3040)
* Fex: delete useles approllout related code

Signed-off-by: wangyike <wangyike_wyk@163.com>

fix failed test

Signed-off-by: wangyike <wangyike_wyk@163.com>

Fix: failed e2e test

Signed-off-by: wangyike <wangyike_wyk@163.com>

fix-test

Signed-off-by: wangyike <wangyike_wyk@163.com>

fix test

Signed-off-by: wangyike <wangyike_wyk@163.com>

fix e2e-test

Signed-off-by: wangyike <wangyike_wyk@163.com>

fix e2e rollout test

Signed-off-by: wangyike <wangyike_wyk@163.com>

* delete useless test code

Signed-off-by: wangyike <wangyike_wyk@163.com>

* fix failed unit-test

Signed-off-by: wangyike <wangyike_wyk@163.com>
2022-01-06 14:33:11 +08:00
Zheng Xi Zhou
8033279751 Fix: Generate doc for Terraform typed ComponentDefinition (#3041)
* Fix: Generate doc for Terraform typed ComponentDefinition

When the configuration of a Terraform Typed ComponentDefinition is in
remote Git, generate the properties for it. And generated outputs for
all Terraform ComponentDefinition

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-01-05 19:45:30 +08:00
Somefive
acac0554f3 Fix: filter managedfields update events (#3043)
Signed-off-by: Yin Da <yd219913@alibaba-inc.com>
2022-01-05 13:30:58 +08:00
Jian.Li
133a89be3f Fix: deprecate CRD discovery for CUE import in Definition to prevent memory leak and OOM crash (#2925)
* Pause test case

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>

* fix lint

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>

* diable discover the open api of the CRD

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>

* fix definition test cases

Signed-off-by: Jian.Li <lj176172@alibaba-inc.com>
2021-12-16 17:53:15 +08:00
yangsoon
3b1f097807 Fix: change catlog helm repo path to https://charts.kubevela.net/example (#2942)
Signed-off-by: yangsoon <yangsoonlx@gmail.com>
2021-12-16 15:17:00 +08:00
Tianxin Dong
655c2615e1 Feat: add workflow reconciling backoff time and failed limit times (#2881)
* Feat: add workflow failed after retries

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* Feat: add workflow reconcile backoff time

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix lint

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* make reviewable

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* resolve some comments

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* add tests

* fix rebase

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix status

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix requeue time interval

* resolve comments

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* change time to pointer

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix pointer test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* change time to cm

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* resolve comments and add e2e test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2021-12-15 11:33:33 +08:00
Somefive
4dc213469a Feat: add compatibility code for new rt (#2920)
Signed-off-by: Yin Da <yd219913@alibaba-inc.com>
2021-12-14 21:27:47 +08:00
Somefive
a89bb69a62 Fix: add design docs for ResourceTracker (#2909)
* Fix: enhance rt logic and add docs

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: test conflict

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>
2021-12-13 19:41:42 +08:00
Somefive
b622cbdb7f Feat: ResourceTracker new architecture (#2849)
* Feat: new rt

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: add publish version

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>
2021-12-10 15:00:03 +08:00
Jian.Li
3f2e76c75a Feat: HTTP Provider requires adding certificate chain from secret (#2771)
* http support load tls certs

* fix: TLS MinVersion too low
2021-12-08 11:39:04 +08:00
Tianxin Dong
4b07f1d000 Feat: support generate definition schema from terraform remote (#2860) 2021-12-07 15:50:04 +08:00
Zheng Xi Zhou
5630c02d7f Fix: add more fields for Terraform types definitions in api schema (#2836)
* Fix: add more fields for Terraform types definitions in api schema

Added `writeConnectionStringSecrecto`, `region` and `deleteResoruce`
fields for Terraform types definitions in OpenAPI schema

* fix ci issue

* fix header
2021-12-01 10:03:08 +08:00
Zheng Xi Zhou
c588f85438 Fix: sync status of Terraform typed components to application (#2844)
Synchronized the status of an Terraform typed components to .status.
services of the application

Signed-off-by: zzxwill <zzxwill@gmail.com>
2021-11-30 17:49:59 +08:00
Tianxin Dong
52ac570faa Feat: add filter in list definitions (#2835)
* Feat: add filter in list definitions

* add workload type
2021-11-30 16:53:01 +08:00
Jian.Li
ec105cbb02 Fix: application conditions confusion. (#2834)
* fix condition confusion

* fix test cases
2021-11-30 16:43:27 +08:00
Somefive
f9ecc02417 Feat: rework rt (#2797) 2021-11-29 16:14:30 +08:00
Somefive
f5fc822a3e Fix legacy resourcetracker not garbage collect while previousRT is empty or equals currentRT (#2817) 2021-11-26 17:48:17 +08:00
Somefive
dd84700657 Fix: health check appRev label bug (#2816)
* Fix: healthscope failed since appRevision is not set && rollout controller amend appRevision to workload since compRev does not have this one

* Fix: test
2021-11-26 17:47:43 +08:00
Somefive
782025507c Fix: health check do not check resource in correct namespace while ns is override by envbinding (#2818) 2021-11-26 16:04:44 +08:00
Somefive
d1f70def0e Fix: healthscope support envbinding (namespace) (#2783)
* Fix: healthscope support envbinding (namespace)

* Fix: ensure healthscope not patch status when application workflow is still running
2021-11-24 21:26:06 +08:00
yangsoon
3d2fcacb5a Feat: built in gc policy to application (#2575) 2021-11-24 15:38:13 +08:00
yangsoon
6dff9ab679 Fix: fix compatibility test 2021-11-22 23:21:17 +08:00
Hongchao Deng
e5aba94cd1 Merge branch 'apiserver' into merge 2021-11-22 17:25:50 +08:00
wyike
4ad27e9bcd Fix: (#2761)
1. load component in arrary, so apply them in order addon's needNamespace will be apply firstly
2. apply application  in controle plane will be first workflowStep
3. bigger application reconcile timeout context get avoid of time out
2021-11-21 21:03:41 +08:00
Somefive
34aa74ff48 Feat: envbinding support cloud resource deploy and share (#2734)
* Feat: envbinding support cloud resource deploy and bind

* Fix: change bind-cloud-resource to share-cloud-resource
2021-11-20 13:07:27 +08:00