From 5eb2ae491910ef5aae2248fc12452a6196e2883a Mon Sep 17 00:00:00 2001 From: techmaharaj Date: Mon, 18 Apr 2022 13:46:09 +0530 Subject: [PATCH 1/3] Adding First Draft of Contributing.md --- CONTRIBUTING.md | 197 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 CONTRIBUTING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..67231ab --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,197 @@ +We πŸ’š Opensource! + +Yes, because we feel that it’s the best way to build and improve a product. It allows people like you from across the globe to contribute and improve a product over time. And we’re super happy to see that you’d like to contribute to ZTKA. + +We are always on the lookout for anything that can improve the product. Be it feature requests, issues/bugs, code or content, we’d love to see what you’ve got to make this better. If you’ve got anything exciting and would love to contribute, this is the right place to begin your journey as a contributor to ZTKA and the larger open source community. + +**How to get started?** + +The easiest way to start is to look at existing issues and see if there’s something there that you’d like to work on. You can filter issues with the label β€œGood first issue” which are relatively self sufficient issues and great for first time contributors. + +Once you decide on an issue, please comment on it so that all of us know that you’re on it. + +If you’re looking to add a new feature, raise a new issue and start a discussion with the community. Engage with the maintainers of the project and work your way through. + +Below are all the details you need to know about the `RCloud Base` repo and get started with the development. + +## RCloud Base + +This repository contains all the rcloud-system components that are the backbone for ztka and gitops. + +### Prerequisites + +- Postgres: Primary database +- Ory Kratos: API for user management +- Elasticsearch: Storage for audit logs + +You can use the bitnami/charts for postgres and elastic/helm-charts for elasticsearch. + +### Development setup + +#### Using docker-compose + +Run following Docker Compose command to setup all requirements like Postgres db, Kratos etc. for the rcloud-base. + +This will start up postgres and elasticsearch as well as kratos and run the kratos migrations. It will also run all the necessary migrations. It also starts up a mail slurper for you to use Kratos. + +`docker-compose up -d` + +Start rcloud-base: + +`go run github.com/RafayLabs/rcloud-base` + +**Manual** + +**Start databases** + +**Postgres** + +```bash +docker run --network host \ + --env POSTGRES_HOST_AUTH_METHOD=trust \ + -v pgdata:/var/lib/postgresql/data \ + -it postgres +``` + +**Elasticsearch** + +```bash +docker run --network host \ + -v elastic-data:/usr/share/elasticsearch/data \ + -e "discovery.type=single-node" \ + -e "xpack.security.enabled=false" \ + -it docker.elastic.co/elasticsearch/elasticsearch:8.0.0 +``` + +**Create the initial db/user** + +Scripts for admindb: + +```SQL +create database admindb; +CREATE ROLE admindbuser WITH LOGIN PASSWORD ''; +GRANT ALL PRIVILEGES ON DATABASE admindb to admindbuser; +``` + +Now in the newly created db: + +```sql +CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; +grant execute on function uuid_generate_v4() to admindbuser; +``` + +Scripts for clusterdb: + +```sql +CREATE database clusterdb; +CREATE ROLE clusterdbuser WITH LOGIN PASSWORD ''; +GRANT ALL PRIVILEGES ON DATABASE clusterdb to clusterdbuser; +``` + +Now in the newly created db: + +```sql +CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; +grant execute on function uuid_generate_v4() to clusterdbuser; +``` + +This will grant the necessary permission to the newly created user to run uuid_generate_v4() + +**Run application migrations** + +We use golang-migrate to perform migrations. +Install golang-migrate + +`go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest` + +-tags `postgres` is important as otherwise it compiles without postgres support + +You can refer to the guide for full details. +Run migrations + +Example for admindb: + +```bash +export POSTGRESQL_URL='postgres://:@:/admindb?sslmode=disable' +migrate -path ./persistence/migrations/admindb -database "$POSTGRESQL_URL" up +``` + +See cli-usage for more info. + +### Development setup + +Copy env.example to .env: + +`cp env.example .env` + +Run following Docker Compose command to setup all requirements like Postgres db, Kratos etc. for the rcloud-base: + +`docker-compose up -d` + +Start rcloud-base server: + +`go run github.com/RafayLabs/rcloud-base` + +**Code Structure** + +The following section lists out the code structure for each of the 4 repos. Mention the folder structure along with its importance and what it is for. + +``` +components +β”œβ”€β”€ adminsrv +β”‚ β”œβ”€β”€ proto +β”‚ β”œβ”€β”€ server +β”‚ β”‚ └── organization.go +β”‚ β”‚ └── project.go +β”‚ β”œβ”€β”€ internal +β”‚ β”œβ”€β”€ pkg +β”‚ β”‚ └── service +β”‚ β”œβ”€β”€ Dockerfile.adminsrv +β”‚ └── main.go +β”‚ └── buf.yaml +β”‚ └── buf.gen.yaml +β”œβ”€β”€ authz +β”‚ β”œβ”€β”€ proto +β”‚ β”œβ”€β”€ server +β”‚ β”‚ └── authz.go +β”‚ β”œβ”€β”€ pkg +β”‚ β”‚ └── service +β”‚ β”œβ”€β”€ Dockerfile.authz +β”‚ └── main.go +β”‚ └── buf.yaml +β”‚ └── buf.gen.yaml +β”œβ”€β”€ usermgmt +β”‚ β”œβ”€β”€ proto +β”‚ β”œβ”€β”€ server +β”‚ β”‚ └── user.go +β”‚ β”‚ └── role.go +β”‚ β”œβ”€β”€ internal +β”‚ β”œβ”€β”€ pkg +β”‚ β”‚ └── service +β”‚ β”œβ”€β”€ Dockerfile.usermgmt +β”‚ └── main.go +β”‚ └── buf.yaml +β”‚ └── buf.gen.yaml +β”œβ”€β”€ cluster-scheduler +β”‚ β”œβ”€β”€ proto +β”‚ β”œβ”€β”€ server +β”‚ β”‚ └── cluster.go +β”‚ β”œβ”€β”€ internal +β”‚ β”œβ”€β”€ pkg +β”‚ β”‚ └── service +β”‚ β”œβ”€β”€ Dockerfile.cluster-scheduler +β”‚ └── main.go +β”‚ └── buf.yaml +β”‚ └── buf.gen.yaml +β”œβ”€β”€ common +β”‚ β”œβ”€β”€ proto +β”‚ β”œβ”€β”€ pkg +β”‚ └── buf.yaml +β”‚ └── buf.gen.yaml +``` + +**Need Help?** + +We’re there for you - the best part of being a part of an open source community. If you are stuck somewhere or are facing an issue or just don’t know how to get started, feel free to let us know. + +You can reach out to us via our Slack Channel, Twitter, Discord etc. From 0980f1e4e55714477d066e629fb62f1f30c97bf0 Mon Sep 17 00:00:00 2001 From: techmaharaj Date: Wed, 20 Apr 2022 15:18:13 +0530 Subject: [PATCH 2/3] Updating contribution draft --- CONTRIBUTING.md | 191 +++++++++++++++++------------------------------- 1 file changed, 66 insertions(+), 125 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 67231ab..be62aa5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -14,37 +14,48 @@ If you’re looking to add a new feature, raise a new issue and start a discussi Below are all the details you need to know about the `RCloud Base` repo and get started with the development. -## RCloud Base +# Rcloud Base This repository contains all the rcloud-system components that are the backbone for ztka and gitops. -### Prerequisites +## Prerequisites -- Postgres: Primary database -- Ory Kratos: API for user management -- Elasticsearch: Storage for audit logs +- [Postgres](https://github.com/postgres/postgres): Primary database +- [Ory Kratos](https://www.ory.sh/kratos): API for user management +- [Elasticsearch](https://www.elastic.co/elasticsearch/): Storage for audit logs -You can use the bitnami/charts for postgres and elastic/helm-charts for elasticsearch. +> You can use the +> [bitnami/charts](https://github.com/bitnami/charts/tree/master/bitnami/postgresql/#installing-the-chart) +> for postgres and +> [elastic/helm-charts](https://github.com/elastic/helm-charts) for +> elasticsearch. -### Development setup +## Development setup -#### Using docker-compose +### Using `docker-compose` -Run following Docker Compose command to setup all requirements like Postgres db, Kratos etc. for the rcloud-base. +Run following Docker Compose command to setup all requirements like +Postgres db, Kratos etc. for the rcloud-base. -This will start up postgres and elasticsearch as well as kratos and run the kratos migrations. It will also run all the necessary migrations. It also starts up a mail slurper for you to use Kratos. +_This will start up postgres and elasticsearch as well as kratos and +run the kratos migrations. It will also run all the necessary +migrations. It also starts up a mail slurper for you to use Kratos._ -`docker-compose up -d` +```bash +docker-compose --env-file ./env.example up -d +``` Start rcloud-base: -`go run github.com/RafayLabs/rcloud-base` +```bash +go run github.com/RafayLabs/rcloud-base +``` -**Manual** +### Manual -**Start databases** +#### Start databases -**Postgres** +##### Postgres ```bash docker run --network host \ @@ -53,7 +64,7 @@ docker run --network host \ -it postgres ``` -**Elasticsearch** +##### Elasticsearch ```bash docker run --network host \ @@ -63,134 +74,64 @@ docker run --network host \ -it docker.elastic.co/elasticsearch/elasticsearch:8.0.0 ``` -**Create the initial db/user** +#### Create the initial db and user -Scripts for admindb: - -```SQL +```sql create database admindb; CREATE ROLE admindbuser WITH LOGIN PASSWORD ''; GRANT ALL PRIVILEGES ON DATABASE admindb to admindbuser; ``` -Now in the newly created db: +#### Ory Kratos -```sql -CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; -grant execute on function uuid_generate_v4() to admindbuser; -``` +Install Ory Kratos using the [installation +guide](https://www.ory.sh/docs/kratos/install) from Kratos +documentation. -Scripts for clusterdb: - -```sql -CREATE database clusterdb; -CREATE ROLE clusterdbuser WITH LOGIN PASSWORD ''; -GRANT ALL PRIVILEGES ON DATABASE clusterdb to clusterdbuser; -``` - -Now in the newly created db: - -```sql -CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; -grant execute on function uuid_generate_v4() to clusterdbuser; -``` - -This will grant the necessary permission to the newly created user to run uuid_generate_v4() - -**Run application migrations** - -We use golang-migrate to perform migrations. -Install golang-migrate - -`go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest` - --tags `postgres` is important as otherwise it compiles without postgres support - -You can refer to the guide for full details. -Run migrations - -Example for admindb: +Perform the Kratos migrations: ```bash +export DSN='postgres://:@:/admindb?sslmode=disable' +kratos -c migrate sql -e --yes +``` + +Start the Ory Kratos server using kratos config provided in +[_kratos](./_kratos) directory. + +#### Run application migrations + +We use [`golang-migrate`](https://github.com/golang-migrate/migrate) to perform migrations. + +##### Install [`golang-migrate`](https://github.com/golang-migrate/migrate) + +```shell +go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest +``` + +_`-tags 'postgres'` is important as otherwise it compiles without postgres support_ + +You can refer to the [guide](https://github.com/golang-migrate/migrate/tree/master/cmd/migrate) for full details. + +##### Run migrations + +_It is required to perform Kratos migrations before this step._ + +```shell export POSTGRESQL_URL='postgres://:@:/admindb?sslmode=disable' migrate -path ./persistence/migrations/admindb -database "$POSTGRESQL_URL" up ``` -See cli-usage for more info. +See [cli-usage](https://github.com/golang-migrate/migrate#cli-usage) for more info. -### Development setup +#### Start application -Copy env.example to .env: +Start rcloud-base: -`cp env.example .env` - -Run following Docker Compose command to setup all requirements like Postgres db, Kratos etc. for the rcloud-base: - -`docker-compose up -d` - -Start rcloud-base server: - -`go run github.com/RafayLabs/rcloud-base` - -**Code Structure** - -The following section lists out the code structure for each of the 4 repos. Mention the folder structure along with its importance and what it is for. - -``` -components -β”œβ”€β”€ adminsrv -β”‚ β”œβ”€β”€ proto -β”‚ β”œβ”€β”€ server -β”‚ β”‚ └── organization.go -β”‚ β”‚ └── project.go -β”‚ β”œβ”€β”€ internal -β”‚ β”œβ”€β”€ pkg -β”‚ β”‚ └── service -β”‚ β”œβ”€β”€ Dockerfile.adminsrv -β”‚ └── main.go -β”‚ └── buf.yaml -β”‚ └── buf.gen.yaml -β”œβ”€β”€ authz -β”‚ β”œβ”€β”€ proto -β”‚ β”œβ”€β”€ server -β”‚ β”‚ └── authz.go -β”‚ β”œβ”€β”€ pkg -β”‚ β”‚ └── service -β”‚ β”œβ”€β”€ Dockerfile.authz -β”‚ └── main.go -β”‚ └── buf.yaml -β”‚ └── buf.gen.yaml -β”œβ”€β”€ usermgmt -β”‚ β”œβ”€β”€ proto -β”‚ β”œβ”€β”€ server -β”‚ β”‚ └── user.go -β”‚ β”‚ └── role.go -β”‚ β”œβ”€β”€ internal -β”‚ β”œβ”€β”€ pkg -β”‚ β”‚ └── service -β”‚ β”œβ”€β”€ Dockerfile.usermgmt -β”‚ └── main.go -β”‚ └── buf.yaml -β”‚ └── buf.gen.yaml -β”œβ”€β”€ cluster-scheduler -β”‚ β”œβ”€β”€ proto -β”‚ β”œβ”€β”€ server -β”‚ β”‚ └── cluster.go -β”‚ β”œβ”€β”€ internal -β”‚ β”œβ”€β”€ pkg -β”‚ β”‚ └── service -β”‚ β”œβ”€β”€ Dockerfile.cluster-scheduler -β”‚ └── main.go -β”‚ └── buf.yaml -β”‚ └── buf.gen.yaml -β”œβ”€β”€ common -β”‚ β”œβ”€β”€ proto -β”‚ β”œβ”€β”€ pkg -β”‚ └── buf.yaml -β”‚ └── buf.gen.yaml +```bash +go run github.com/RafayLabs/rcloud-base ``` -**Need Help?** +## Need Help? We’re there for you - the best part of being a part of an open source community. If you are stuck somewhere or are facing an issue or just don’t know how to get started, feel free to let us know. From 8e12fdc8f92d399a3ee1272e832b2808c93c3f0b Mon Sep 17 00:00:00 2001 From: techmaharaj Date: Wed, 20 Apr 2022 16:29:47 +0530 Subject: [PATCH 3/3] Updated help section --- CONTRIBUTING.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index be62aa5..388dea5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -133,6 +133,4 @@ go run github.com/RafayLabs/rcloud-base ## Need Help? -We’re there for you - the best part of being a part of an open source community. If you are stuck somewhere or are facing an issue or just don’t know how to get started, feel free to let us know. - -You can reach out to us via our Slack Channel, Twitter, Discord etc. +If you are interested to contribute to rcloud-base but are stuck with any of the steps, feel free to reach out to us. Please create an issue in this repository describing your issue and we'll take it up from there.