Commit Graph

13 Commits

Author SHA1 Message Date
Tianxin Dong
cb7bc0e7f4 Fix: handle workflow cache reconcile (#3128)
* Fix: handle workflow cache reconcile

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix return and move backoff to memory

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* handle failed to patch case

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* add store in err case

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* make reviewable

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix ut

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* do cleanup in ut

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-01-22 15:54:17 +08:00
yangsoon
6cee468743 Feat: add more performance optimization and prometheus metrics for controller (#3086)
* Feat: add more prometheus metrics

Signed-off-by: yangsoon <yangsoonlx@gmail.com>

* Feat: add detail gc rt duration metrics

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add monitor to client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add all cache object

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: watch job

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add deleg client

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for rt list and disable controllerrevision

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add apprev disable optimize

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Fix: optimize log

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add time recorder for app ctrl

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add in-memory workflow context

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add reconcile-reduction & random-pick-gc & optimize rt record

Signed-off-by: Yin Da <yd219913@alibaba-inc.com>

* Feat: add optimize for healthcheck & resourcetracker trigger

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: refactor

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Feat: record the resource-tracker number by informer event-handler

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Feat: add promethus collect annotation in template

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Fix: command line comment bug

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: rename args and remove legacy controller metrics

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: make code reviewable

Signed-off-by: yangsoon <songyang.song@alibaba-inc.com>

* Chore: optimize flag descriptions

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Chore: break optimize package

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

* Fix: gc policy test

Signed-off-by: Somefive <yd219913@alibaba-inc.com>

Co-authored-by: Yin Da <yd219913@alibaba-inc.com>
Co-authored-by: yangsoon <songyang.song@alibaba-inc.com>
2022-01-14 15:18:02 +08:00
Tianxin Dong
655c2615e1 Feat: add workflow reconciling backoff time and failed limit times (#2881)
* Feat: add workflow failed after retries

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* Feat: add workflow reconcile backoff time

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix lint

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* make reviewable

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* resolve some comments

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* add tests

* fix rebase

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix status

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix requeue time interval

* resolve comments

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* change time to pointer

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix pointer test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* change time to cm

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* resolve comments and add e2e test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2021-12-15 11:33:33 +08:00
Tianxin Dong
844479c7b5 Fix: add owner reference in workflow context cm (#2573)
* Fix: add owner reference in workflow context cm

* fix ci

* delete useless test case
2021-11-04 14:19:35 +08:00
Jian.Li
dfbe65c9c0 Feat: Commit step-generate data without success (#2539)
* Feat: commit without success

* Feat: add test case
2021-10-23 11:25:50 +08:00
Jian.Li
b66f2995a5 Fix: Bugs (as follows) (#2278)
* Fix: suspend step id

* Feat: add test cases

* Fix: reslove conflict

* Fix: reduce reconclie duration for test
2021-09-13 20:03:49 +08:00
Jian.Li
6cbdbe84b2 Refactor application code to make it run as Dag workflow (#2236)
* Refactor: remove use of AppConfig in AppRevision

* Refactor: remove insert secret and configmap

* Feat(workflow): upgrade

* Fix(conflict): workflow cherry

* Feat(workflow): support DAG mode

* Feat(workflow): prepare steps in step

* Feat(tools): random string

* Fix(rand): gosec weak random

* Fix(ci): test passing

* Feat(workflow): generate steps

* Fix: fix rebase from master

* Fix: fix workflow ut

* Feat(test): add test cases

* Fix: fix lint and rebase from master

* Refactor: application code

* Fix: fix ci lint

* Fix: make code reviewable

* Fix: workflow_test.go

* Feat: collect services

* Fix(ci): unit tests

* Feat: make one

* Test: application with input/output and workflow

* Fix: trace test

* Fix: update step index falied

* Feat: refactor op.#Load

* Fix: delete dead code

* Refactor: op.xxx

* Fix: patch component

* Test: add generator test

* Fix: add license

* Fix: pending e2e plugin test

* Fix: disable test/e2e-test

* Fix: patch by script

Co-authored-by: 天元 <jianbo.sjb@alibaba-inc.com>
Co-authored-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2021-09-12 10:12:46 +08:00
Zhizhen He
1e294b3164 chore: replace github.com/ghodss/yaml with sigs.k8s.io/yaml (#2089) 2021-08-15 10:53:05 +08:00
Jianbo Sun
aa87d3da24 upgrade K8s dependency lib to v0.21 (#1985)
* upgrade K8s dependency to v0.21

* update CRD for new version of controller-runtime

* fix ci component revision create must fill Raw in runtime.RawExtension

* try fix test

* start control plane timeout set to 1min and fix tests

* add timeout for env test start and stop

* longer timeout time for BeforeSuit function

* upgrade kubebuilder and k8s cluster version to match with v1.21.2 in github action

* fix test

* fix resource tracker ownerRef override

* update developer guides
2021-08-03 11:30:02 +08:00
Jian.Li
9d6aaa6044 Workflow Support op.# ApplyComponent and op.#ApplyRemaining (#1993)
* add cap

* workflow

* add docs

* upgrade op.#ApplyComponent

* lint

* bytes to string
2021-07-31 07:14:00 +08:00
Jian.Li
4481e7637d Optimize Workflow (#1978)
* open op.#Steps

* export

* fmt code

* fix bug & upgrade workflow doc

* fmt

* upgrade test coverage

* upgrade test coverage
2021-07-29 11:35:22 +08:00
Jianbo Sun
e71d71c6b7 use corev1.ObjectReference insteadof crossplane-runtime TypedReference (#1974)
* refactor crossplane-runtime TypedReference to ObjectReference and make condtion locally

* fix test
2021-07-28 13:59:13 +08:00
Hongchao Deng
ed0039e1ee [Feature] CUE Based Workflow Step (#1935)
* Workflow WIP (#1934)

* interface

* add model value

* walk fields

* provider

* stdlin

* tasks

* add stdlib

* Adjust TaskGenerator

* Unified configmap format

* development completed

* fix import cycle

* demo passing

* upgrade action

* add value test cases

* format

* apis

* add context test cases

* fmt

* fmt

* add context to taskGenerator (#1942)

* fix configmap test

* add workflow integration test

* upgrade code

* initial workflow commit

* interface

* add model value

* walk fields

* provider

* stdlin

* tasks

* add stdlib

* Adjust TaskGenerator

* Unified configmap format

* development completed

* fix import cycle

* demo passing

* upgrade action

* add value test cases

* format

* apis

* add context test cases

* fmt

* fmt

* add context to taskGenerator (#1942)

* fix configmap test

* add workflow integration test

* lint code

* improve pkg[value、context] testing coverage

* Improve testing coverage(#1957)

* add license

* add test cases

Co-authored-by: Jian.Li <74582607+leejanee@users.noreply.github.com>
Co-authored-by: Jian.Li <lj176172@alibaba-inc.com>
2021-07-22 18:53:30 +08:00