Commit Graph
33 Commits
Author SHA1 Message Date
Ayush Kumar 65dedda40a [Backport release-1.10] Fix: prevent unbounded read in Terraform remote configuration loader (GHSA-fmgp-q6jx-gg3x) (#7191)
* Merge commit from fork

* fix: prevent unbounded read in Terraform remote configuration loader (GHSA-fmgp-q6jx-gg3x)

* fix: bound remote Terraform clone and invalidate cache on rejection

Follow-up hardening for GHSA-fmgp-q6jx-gg3x.

Bound the clone of the attacker-supplied repository: shallow Depth:1, a
2-minute fetch timeout via PlainCloneContext, and post-clone caps on the
retained tree size (64 MiB) and file count, rejecting and removing a clone
that exceeds them.

Invalidate the clone cache: re-clone when the recorded remote URL changes,
and remove the cache on a failed clone or a rejected read so a corrected
repository is re-fetched instead of a poisoned or stale tree being reused.

Validate the module name before building the cache path, and log clone,
rejection, and eviction events.

Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com>

* Fix: resolve gosec G304 lint failure in Terraform module cache check (#7190)

Wrap the cache remote marker read in filepath.Clean, the same pattern
other os.ReadFile call sites in this repo use to satisfy gosec. The
path is built from filepath.Join and a constant suffix, with the module
name validated beforehand, so behavior is unchanged.

The finding surfaced on master after the GHSA-fmgp-q6jx-gg3x merge
because the advisory workflow did not run the full lint job.

Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com>

* Fix: retain resource creator when rebuilding appliedResources from ResourceTracker

The release-1.10 change that rebuilds Application status.appliedResources from
the ResourceTracker (#7086) dropped the per-resource creator, because the
ResourceTracker does not persist the creator field. As a result
status.appliedResources reported an empty creator and the "applied resource in
workflow step status" controller test failed, since it expects the creator
"workflow".

Re-attach the creator recorded during dispatch to the resources rebuilt from
the ResourceTracker. The ResourceTracker stays authoritative for which
resources exist; only the creator attribution that it does not persist is
restored.

Signed-off-by: Ayush Kumar <65535504+roguepikachu@users.noreply.github.com>

* Fix: do not log or persist credentials embedded in Terraform module remote URLs

The remote URL of a Terraform module can embed credentials (for example
https://user:token@host/repo.git). The cache-reuse logic logged the raw URL
and wrote it to the .remote-url cache marker, which could leak those
credentials into controller logs and onto disk.

Strip the userinfo from the URL before logging it and before writing the cache
marker. The marker now stores a credential-free URL and the reuse check
compares the same stripped form, so cache reuse and re-clone-on-change behave
as before while no secret is persisted.

Signed-off-by: Ayush Kumar <65535504+roguepikachu@users.noreply.github.com>

---------

Signed-off-by: Ayush Kumar <ayushshyamkumar888@gmail.com>
Signed-off-by: Ayush Kumar <65535504+roguepikachu@users.noreply.github.com>
2026-06-25 09:49:49 +01:00
Tianxin Dong 4f8bf44684 Refactor: use cuex engine (#6575)
* refactor: use cuex engine

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix lint

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix unit test

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix static check and sdk tests

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix testdata

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix velaql unit test

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix docgen parser

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix cuegen

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix velaql

Signed-off-by: FogDong <fog@bentoml.com>

* fix: delete useless print

Signed-off-by: FogDong <fog@bentoml.com>

* fix: set client for ql

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix mt tests

Signed-off-by: FogDong <fog@bentoml.com>

* fix: set kubeclient in generator

Signed-off-by: FogDong <fog@bentoml.com>

* fix: use pass kube client

Signed-off-by: FogDong <fog@bentoml.com>

* fix: simplify ql

Signed-off-by: FogDong <fog@bentoml.com>

* fix: fix lint

Signed-off-by: FogDong <fog@bentoml.com>

* fix: add wf debug back

Signed-off-by: FogDong <fog@bentoml.com>

* fix: add loader

Signed-off-by: FogDong <fog@bentoml.com>

---------

Signed-off-by: FogDong <fog@bentoml.com>
2024-07-27 17:44:20 +08:00
Somefive c255d21ad6 Chore: remove unnecessary go mod (#6093)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2023-06-08 15:32:57 +08:00
Joshua Agboola a9bc43a9dd Feat: Support SSH for retrieving terraform modules in private git repo (#5059)
* Feat: Support SSH for retrieving terraform modules in private git repo

Signed-off-by: motilayo <44736801+motilayo@users.noreply.github.com>

* fix lint errors

Signed-off-by: motilayo <44736801+motilayo@users.noreply.github.com>

* fix gofmt lint error

Signed-off-by: motilayo <joshuaagboola@live.ca>

* update gitCredentialsReference to gitCredentialsSecretReference, validate secret, improve known_hosts logic

Signed-off-by: motilayo <joshuaagboola@live.ca>

* SImplify logic to get publickey

Signed-off-by: motilayo <joshuaagboola@live.ca>

* Set known_hosts and export SSH_KNOWN_HOSTS

Signed-off-by: motilayo <joshuaagboola@live.ca>

* Call GetGitSSHPublicKey for cli

Signed-off-by: motilayo <joshuaagboola@live.ca>

* fix parser.go - nil check for ref.Client

Signed-off-by: motilayo <joshuaagboola@live.ca>

* Unit test for GetGitSSHPublicKey

Signed-off-by: motilayo <joshuaagboola@live.ca>

* improve test coverage

Signed-off-by: motilayo <joshuaagboola@live.ca>

* make reviewable

Signed-off-by: motilayo <joshuaagboola@live.ca>

* minor improvements & cleanup

Signed-off-by: motilayo <joshuaagboola@live.ca>

* update secret name in test

Signed-off-by: motilayo <joshuaagboola@live.ca>

Signed-off-by: motilayo <44736801+motilayo@users.noreply.github.com>
Signed-off-by: motilayo <joshuaagboola@live.ca>
2022-12-23 16:27:27 +08:00
Tianxin Dong 56f200fb59 Fix: deprecated resolve reference in gen open API (#4913)
* Fix: depracted resolve reference in gen open API

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix test

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-10-25 11:13:41 +08:00
barnettZQG 49ed837f97 Feat: support to manage the configs by the CLI, UI, and workflow. (#4794)
* Feat: support to manage the integrations by the CLI and the workflow

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: remove the xml

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: add the unit test for the nacos writer

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Feat: add the integration API

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Feat: make the provider commands to be deprecated

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: make the unit test work

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Feat: rename the integration to the config

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: make the unit test cases work

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Feat: refactor the config commands

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Feat: add the distribution status for the config

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: sort the import packages

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: refine the code style

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: refine the code style

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: get the content format before render the content

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Feat: add some examples

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: the command test cases

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Feat: add the definitions of the workflow step

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: add some tests

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: add some tests

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: change the name

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: retry the CI

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

* Fix: refine some words

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>

Signed-off-by: barnettZQG <barnett.zqg@gmail.com>
2022-10-17 17:15:45 +08:00
Tianxin Dong 239c5474dd Chore: refactor workflow from workflow engine (#4631)
Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-09-02 12:55:03 +08:00
Tianxin Dong c3ca308489 Chore: update cue version to 0.4.3 (#4425)
* Chore: update cue version to 0.4.3

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* resolve some comments

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* fix lint

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* add more tests

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* rebase and add more tests

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>

* resolve comments

Signed-off-by: FogDong <dongtianxin.tx@alibaba-inc.com>
2022-08-08 17:57:48 +08:00
Jianbo Sun eefccfb565 Chore: seprate test into linux env which can't be execute in darwin (#3823)
Signed-off-by: Jianbo Sun <jianbo.sjb@alibaba-inc.com>
2022-05-07 18:14:12 +08:00
Somefive 3cde2337c6 Feat: upgrade dependencies (#3776)
Signed-off-by: Somefive <yd219913@alibaba-inc.com>
2022-05-05 14:37:50 +08:00
Zheng Xi Zhou 37a523cbf7 Fix: not steady unit test (#3293)
The UT `TestGetOpenAPISchemaFromTerraformComponentDefinition` won't
succeed 100%. The required variables in the generated isn't in the expected
order all the time.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-18 15:15:10 +08:00
Zheng Xi Zhou 46967daf11 Fix: fixed required items for a Terraform ComponentDefinition (#3257)
If a Terraform variable is required, the item in OpenAPI schema
is required.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-15 18:53:40 +08:00
Zheng Xi Zhou 24f147a72c Fix: support more Terraform variable types (#3204)
Support Any, set and some complicated variable types

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-08 18:48:38 +08:00
Zheng Xi Zhou 2c41ee9c3c Fix: add unit test for getting Terraform Configuration from remote git (#3193)
Add another unit test when the configuration of Terraform locates
in a subpath of a git remote repository

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-02-08 10:16:13 +08:00
Zheng Xi Zhou 7895560675 Fix: retrieve Terraform variables from variables.tf (#3149)
* Fix: retrieve Terraform variables from variables.tf

If Terraform modules/resources are stored in remote git repos, get
variables from file `varialbes.tf`, or from `main.tf`.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-01-24 11:52:07 +08:00
Zheng Xi Zhou 1a50dd76b5 Fix: generate docs for Terraform ComponentDefinition (#3051)
* Fix: generate docs for Terraform ComponentDefinition

Generated kubevela.io docs for Terraform typed ComponentDefinition
with `vela def gen-doc xxx` cli.

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* Refactor code

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* add ut

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* add ut

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>

* refine cloud resource title

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-01-10 14:08:48 +08:00
Zheng Xi Zhou 8033279751 Fix: Generate doc for Terraform typed ComponentDefinition (#3041)
* Fix: Generate doc for Terraform typed ComponentDefinition

When the configuration of a Terraform Typed ComponentDefinition is in
remote Git, generate the properties for it. And generated outputs for
all Terraform ComponentDefinition

Signed-off-by: Zheng Xi Zhou <zzxwill@gmail.com>
2022-01-05 19:45:30 +08:00
Tianxin Dong 4b07f1d000 Feat: support generate definition schema from terraform remote (#2860) 2021-12-07 15:50:04 +08:00
qiaozp 1c7e0c054d Fix: add detail cache, fix uiSchema (#2716)
* add detail cache, fix uischema

* remove
2021-11-15 17:17:07 +08:00
Eng Zer Jun 426aa7af34 Refactor: move from io/ioutil to io and os package (#2234)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <zerjun@beatchain.co>
2021-09-06 18:33:42 +08:00
Zheng Xi Zhou 4e010d7bae Fix: support more Terraform variable types (#2194)
Including null, list(...), map(...) Terraform types
2021-08-30 20:16:24 +08:00
Jianbo Sun 0781937100 Fix: align all CUE template keyword to use parameter (#2181) 2021-08-30 11:43:20 +08:00
yangsoon 66644de597 fix func fixOpenAPISchema (#2003) 2021-08-02 13:04:46 +08:00
Kinsoandkinsolee e494516593 fix(openapi): only parse the parameter field (#1790)
Co-authored-by: kinsolee <lijingzhao@forchange.tech>
2021-06-15 13:36:23 +08:00
Zheng Xi Zhou 8dcbb61cb8 Generate OpenAPI JSON schema for Terraform Component (#1738)
Fix #1736
2021-06-03 11:23:20 +08:00
Jian.Li 4f47bec238 refactor cue related packages in vela (#1734)
* refactor-cue

* import group

* readme

* fmt
2021-06-02 15:37:06 +08:00
yangsoon 7929f9fe09 fix store openapi (#1689) 2021-05-19 20:08:51 +08:00
Yue Wang 2d6f2083db refactor pkg/appfile (#1447)
Signed-off-by: roy wang <seiwy2010@gmail.com>
2021-04-11 14:10:16 +08:00
yangsoon 08a1dc5a22 Add package discover refresh when component/trait definition are registered (#1402)
* add discover refresh

* add test

* fix store openapi schema

* fix discovermapper
2021-04-08 20:35:21 +08:00
Zheng Xi ZhouandJianbo Sun 9e40b77b60 Support cloud resource provisiong and consuming (#1264)
* Support cloud resource provisioning and consuming (Crossplane)

Provided a way to store secret for cloud resource generated by
Crossplane and to consume the secret

Refer to #1128

* Separate cloud resource producer and consumer in two applications

* add unit test to check whether application can consume cloud resource secret

* update Cloud Resource doc

* Provisioning and consuming cloud resource in different applications v1 (one cloud resource)

* one component consumes two cloud resources

Co-authored-by: Jianbo Sun <wonderflow.sun@gmail.com>
2021-03-29 17:20:33 +08:00
yangsoon 4b9e933ca5 add v1beta1 types (#1268) 2021-03-24 13:27:29 +08:00
zzxwill 9afe577087 Add or update copyright for workload/trait definition controllers
Added or updated copyright in newly added files for
orkloaddefinition and traitdefinition controllers and
its dependencies
2021-03-10 14:18:57 +08:00
Zheng Xi ZhouandJianbo Sun c2b6e6efdd Store capability OpenAPI V3 JSON schema in ConfigMap (#1106)
* Store capability OpenAPI V3 JSON schema in ConfigMap

Store the OpenAPI v3 JSON Schema generated from the parameters of
a capability which could be used by api server

To fix #899

address comments
try to fix data race issue

Co-authored-by: Jianbo Sun <wonderflow.sun@gmail.com>

* address comments: generate cue instance from cue json instead of local file

* update ConfigMap data at any time

* add more unit-test

* fix conflicts

* adapt to new spec.schematic.cue.template

* adapt to new spec.schematic.cue.template for controller unittest cases

* fix ci issues

* adapt to namespaced scope definition

* add more testcases on namespaced definitions with and without namespaces

Co-authored-by: Jianbo Sun <wonderflow.sun@gmail.com>
2021-03-10 10:44:58 +08:00