diff --git a/prepare-tf/README.md b/prepare-tf/README.md index 8ceb11cb..1463a2ab 100644 --- a/prepare-tf/README.md +++ b/prepare-tf/README.md @@ -1,29 +1,22 @@ +⚠️ This is work in progress. The UX needs to be improved, +and the docs could be better. + This directory contains a Terraform configuration to deploy a bunch of Kubernetes clusters on various cloud providers, using their respective managed Kubernetes products. -To use it: +## With shell wrapper -1. Select the provider you wish to use. +This is the recommended use. It makes it easy to start N clusters +on any provider. It will create a directory with a name like +`tag-YYYY-MM-DD-HH-MM-SS-SEED-PROVIDER`, copy the Terraform configuration +to that directory, then create the clusters using that configuration. -Change the `source` attribute of the `module "clusters"` section. -Check the content of the `modules` directory to see available choices. - -```bash -vim main.tf -``` - -2. Initialize the provider. - -```bash -terraform init -``` - -3. Configure provider authentication. +1. One-time setup: configure provider authentication for the provider(s) that you wish to use. - Digital Ocean: ```bash - export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ") + doctl auth init ``` - Google Cloud Platform: you will need to create a project named `prepare-tf` @@ -33,25 +26,88 @@ terraform init - Linode: ```bash - export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ") + linode-cli configure ``` -- Oracle Cloud: if you have setup the OCI CLI (and have a `~/.oci/config` config file), - Terraform will use it by default +- Oracle Cloud: FIXME + (set up `oci` through the `oci-cli` Python package) - Scaleway: run `scw init` -4. Decide how many clusters and how many nodes per clusters you want. +2. Optional: set number of clusters, cluster size, and region. + +By default, 1 cluster will be configured, with 2 nodes, and auto-scaling up to 5 nodes. + +If you want, you can override these parameters, with the following variables. ```bash export TF_VAR_how_many_clusters=5 export TF_VAR_min_nodes_per_pool=2 -# Optional (will enable autoscaler when available) export TF_VAR_max_nodes_per_pool=4 -# Optional (will only work on some providers) -export TF_VAR_enable_arm_pool=true +export TF_VAR_location=xxx ``` +The `location` variable is optional. Each provider should have a default value. +The value of the `location` variable is provider-specific. Examples: + +| Provider | Example value | How to see possible values +|---------------|-------------------|--------------------------- +| Digital Ocean | `ams3` | `doctl compute region list` +| Google Cloud | `europe-north1-a` | `gcloud compute zones list` +| Linode | `eu-central` | `linode-cli regions list` +| Oracle Cloud | `eu-stockholm-1` | `oci iam region list` + +3. Run! + +```bash +./run.sh +``` + +(If you don't specify a provider name, it will list available providers.) + +4. Shutting down + +Go to the directory that was created by the previous step (`tag-YYYY-MM...`) +and run `terraform destroy`. + +You can also run `./clean.sh` which will destroy ALL clusters deployed by the previous run script. + +## Without shell wrapper + +Expert mode. + +Useful to run steps sperarately, and/or when working on the Terraform configurations. + +1. Select the provider you wish to use. + +Go to the `source` directory and edit `main.tf`. + +Change the `source` attribute of the `module "clusters"` section. + +Check the content of the `modules` directory to see available choices. + +2. Initialize the provider. + +```bash +terraform init +``` + +3. Configure provider authentication. + +See steps above, and add the following extra steps: + +- Digital Coean: + ```bash + export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ") + ``` + +- Linode: + ```bash + export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ") + ``` + +4. Decide how many clusters and how many nodes per clusters you want. + 5. Provision clusters. ```bash @@ -60,7 +116,7 @@ terraform apply 6. Perform second stage provisioning. -This will install a SSH server on the clusters. +This will install an SSH server on the clusters. ```bash cd stage2 diff --git a/prepare-tf/cleanup.sh b/prepare-tf/cleanup.sh new file mode 100755 index 00000000..58d73f7e --- /dev/null +++ b/prepare-tf/cleanup.sh @@ -0,0 +1,9 @@ +#!/bin/sh +export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ") +export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ") +for T in tag-*; do +( + cd $T + terraform apply -destroy -auto-approve && mv ../$T ../deleted$T +) +done diff --git a/prepare-tf/run.sh b/prepare-tf/run.sh new file mode 100755 index 00000000..8dc06036 --- /dev/null +++ b/prepare-tf/run.sh @@ -0,0 +1,49 @@ +#!/bin/sh +set -e + +TIME=$(which time) + +PROVIDER=$1 +[ "$PROVIDER" ] || { + echo "Please specify a provider as first argument, or 'ALL' for parallel mode." + echo "Available providers:" + ls -1 source/modules + exit 1 +} + +[ "$TAG" ] || { + TIMESTAMP=$(date +%Y-%m-%d-%H-%M-%S) + RANDOMTAG=$(base64 /dev/urandom | tr A-Z a-z | tr -d /+ | head -c5) + export TAG=tag-$TIMESTAMP-$RANDOMTAG +} + +[ "$PROVIDER" = "ALL" ] && { + for PROVIDER in $(ls -1 source/modules); do + $TERMINAL -T $TAG-$PROVIDER -e sh -c " + export TAG=$TAG-$PROVIDER + $0 $PROVIDER + cd $TAG-$PROVIDER + bash + " & + done + exit 0 +} + +[ -d "source/modules/$PROVIDER" ] || { + echo "Provider '$PROVIDER' not found." + echo "Available providers:" + ls -1 source/modules + exit 1 +} + +export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ") +export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ") + +cp -a source $TAG +cd $TAG +cp -r modules/$PROVIDER modules/PROVIDER +$TIME -o time.1.init terraform init +$TIME -o time.2.stage1 terraform apply -auto-approve +cd stage2 +$TIME -o ../time.3.init terraform init +$TIME -o ../time.4.stage2 terraform apply -auto-approve diff --git a/prepare-tf/locals.tf b/prepare-tf/source/locals.tf similarity index 100% rename from prepare-tf/locals.tf rename to prepare-tf/source/locals.tf diff --git a/prepare-tf/main.tf b/prepare-tf/source/main.tf similarity index 96% rename from prepare-tf/main.tf rename to prepare-tf/source/main.tf index 67dec20e..08d0e57f 100644 --- a/prepare-tf/main.tf +++ b/prepare-tf/source/main.tf @@ -1,5 +1,5 @@ module "clusters" { - source = "./modules/linode" + source = "./modules/PROVIDER" for_each = local.clusters cluster_name = each.value.cluster_name min_nodes_per_pool = var.min_nodes_per_pool @@ -7,6 +7,7 @@ module "clusters" { enable_arm_pool = var.enable_arm_pool node_size = var.node_size common_tags = local.common_tags + location = var.location } locals { diff --git a/prepare-tf/modules/digitalocean/main.tf b/prepare-tf/source/modules/digitalocean/main.tf similarity index 64% rename from prepare-tf/modules/digitalocean/main.tf rename to prepare-tf/source/modules/digitalocean/main.tf index 27d23bda..c315acf9 100644 --- a/prepare-tf/modules/digitalocean/main.tf +++ b/prepare-tf/source/modules/digitalocean/main.tf @@ -1,12 +1,13 @@ resource "digitalocean_kubernetes_cluster" "_" { name = var.cluster_name - tags = local.common_tags - region = var.region + tags = var.common_tags + # Region is mandatory, so let's provide a default value. + region = var.location!=null ? var.location : "nyc1" version = var.k8s_version node_pool { name = "x86" - tags = local.common_tags + tags = var.common_tags size = local.node_type auto_scale = true min_nodes = var.min_nodes_per_pool diff --git a/prepare-tf/modules/digitalocean/outputs.tf b/prepare-tf/source/modules/digitalocean/outputs.tf similarity index 77% rename from prepare-tf/modules/digitalocean/outputs.tf rename to prepare-tf/source/modules/digitalocean/outputs.tf index 69fbcd9f..6c6567f3 100644 --- a/prepare-tf/modules/digitalocean/outputs.tf +++ b/prepare-tf/source/modules/digitalocean/outputs.tf @@ -5,3 +5,7 @@ output "kubeconfig" { output "cluster_id" { value = digitalocean_kubernetes_cluster._.id } + +output "has_metrics_server" { + value = false +} diff --git a/prepare-tf/modules/digitalocean/providers.tf b/prepare-tf/source/modules/digitalocean/providers.tf similarity index 100% rename from prepare-tf/modules/digitalocean/providers.tf rename to prepare-tf/source/modules/digitalocean/providers.tf diff --git a/prepare-tf/modules/digitalocean/variables.tf b/prepare-tf/source/modules/digitalocean/variables.tf similarity index 87% rename from prepare-tf/modules/digitalocean/variables.tf rename to prepare-tf/source/modules/digitalocean/variables.tf index ad829379..19a3f3a6 100644 --- a/prepare-tf/modules/digitalocean/variables.tf +++ b/prepare-tf/source/modules/digitalocean/variables.tf @@ -8,10 +8,6 @@ variable "common_tags" { default = [] } -locals { - common_tags = [for tag in var.common_tags : replace(tag, "=", "-")] -} - variable "node_size" { type = string default = "M" @@ -48,9 +44,9 @@ locals { # To view supported regions, run: # doctl compute region list -variable "region" { +variable "location" { type = string - default = "nyc1" + default = null } # To view supported versions, run: diff --git a/prepare-tf/modules/googlecloud/main.tf b/prepare-tf/source/modules/googlecloud/main.tf similarity index 82% rename from prepare-tf/modules/googlecloud/main.tf rename to prepare-tf/source/modules/googlecloud/main.tf index 259270a2..3a3a562f 100644 --- a/prepare-tf/modules/googlecloud/main.tf +++ b/prepare-tf/source/modules/googlecloud/main.tf @@ -1,7 +1,8 @@ resource "google_container_cluster" "_" { name = var.cluster_name project = "prepare-tf" - location = "europe-north1-a" + # "location" is mandatory, so let's provide a default value if none was given. + location = var.location!=null ? var.location : "europe-north1-a" min_master_version = var.k8s_version node_pool { diff --git a/prepare-tf/modules/googlecloud/outputs.tf b/prepare-tf/source/modules/googlecloud/outputs.tf similarity index 100% rename from prepare-tf/modules/googlecloud/outputs.tf rename to prepare-tf/source/modules/googlecloud/outputs.tf diff --git a/prepare-tf/modules/googlecloud/providers.tf b/prepare-tf/source/modules/googlecloud/providers.tf similarity index 100% rename from prepare-tf/modules/googlecloud/providers.tf rename to prepare-tf/source/modules/googlecloud/providers.tf diff --git a/prepare-tf/modules/googlecloud/variables.tf b/prepare-tf/source/modules/googlecloud/variables.tf similarity index 84% rename from prepare-tf/modules/googlecloud/variables.tf rename to prepare-tf/source/modules/googlecloud/variables.tf index 02e20386..fcd27cad 100644 --- a/prepare-tf/modules/googlecloud/variables.tf +++ b/prepare-tf/source/modules/googlecloud/variables.tf @@ -42,7 +42,14 @@ locals { node_type = var.node_types[var.node_size] } -# See supported versions with: +# To view supported locations, run: +# gcloud compute zones list +variable "location" { + type = string + default = null +} + +# To view supported versions, run: # gcloud container get-server-config --region=europe-north1 '--format=flattened(channels)' # But it's also possible to just specify e.g. "1.20" and it figures it out. variable "k8s_version" { diff --git a/prepare-tf/modules/linode/main.tf b/prepare-tf/source/modules/linode/main.tf similarity index 69% rename from prepare-tf/modules/linode/main.tf rename to prepare-tf/source/modules/linode/main.tf index ae871267..b273a556 100644 --- a/prepare-tf/modules/linode/main.tf +++ b/prepare-tf/source/modules/linode/main.tf @@ -1,7 +1,8 @@ resource "linode_lke_cluster" "_" { label = var.cluster_name tags = var.common_tags - region = var.region + # "region" is mandatory, so let's provide a default value if none was given. + region = var.location!=null ? var.location : "eu-central" k8s_version = var.k8s_version pool { diff --git a/prepare-tf/modules/linode/outputs.tf b/prepare-tf/source/modules/linode/outputs.tf similarity index 100% rename from prepare-tf/modules/linode/outputs.tf rename to prepare-tf/source/modules/linode/outputs.tf diff --git a/prepare-tf/modules/linode/providers.tf b/prepare-tf/source/modules/linode/providers.tf similarity index 100% rename from prepare-tf/modules/linode/providers.tf rename to prepare-tf/source/modules/linode/providers.tf diff --git a/prepare-tf/modules/linode/variables.tf b/prepare-tf/source/modules/linode/variables.tf similarity index 91% rename from prepare-tf/modules/linode/variables.tf rename to prepare-tf/source/modules/linode/variables.tf index 2ee3edad..83d6237d 100644 --- a/prepare-tf/modules/linode/variables.tf +++ b/prepare-tf/source/modules/linode/variables.tf @@ -42,11 +42,11 @@ locals { node_type = var.node_types[var.node_size] } -# To view supported versions, run: +# To view supported regions, run: # linode-cli regions list -variable "region" { +variable "location" { type = string - default = "us-east" + default = null } # To view supported versions, run: diff --git a/prepare-tf/modules/oraclecloud/main.tf b/prepare-tf/source/modules/oraclecloud/main.tf similarity index 100% rename from prepare-tf/modules/oraclecloud/main.tf rename to prepare-tf/source/modules/oraclecloud/main.tf diff --git a/prepare-tf/modules/oraclecloud/network.tf b/prepare-tf/source/modules/oraclecloud/network.tf similarity index 100% rename from prepare-tf/modules/oraclecloud/network.tf rename to prepare-tf/source/modules/oraclecloud/network.tf diff --git a/prepare-tf/modules/oraclecloud/outputs.tf b/prepare-tf/source/modules/oraclecloud/outputs.tf similarity index 100% rename from prepare-tf/modules/oraclecloud/outputs.tf rename to prepare-tf/source/modules/oraclecloud/outputs.tf diff --git a/prepare-tf/modules/oraclecloud/providers.tf b/prepare-tf/source/modules/oraclecloud/providers.tf similarity index 100% rename from prepare-tf/modules/oraclecloud/providers.tf rename to prepare-tf/source/modules/oraclecloud/providers.tf diff --git a/prepare-tf/modules/oraclecloud/variables.tf b/prepare-tf/source/modules/oraclecloud/variables.tf similarity index 90% rename from prepare-tf/modules/oraclecloud/variables.tf rename to prepare-tf/source/modules/oraclecloud/variables.tf index 73ae8a82..5d3e5858 100644 --- a/prepare-tf/modules/oraclecloud/variables.tf +++ b/prepare-tf/source/modules/oraclecloud/variables.tf @@ -70,6 +70,13 @@ locals { node_type = var.node_types[var.node_size] } +# To view supported regions, run: +# oci iam region list | jq .data[].name +variable "location" { + type = string + default = null +} + # To view supported versions, run: # oci ce cluster-options get --cluster-option-id all | jq -r '.data["kubernetes-versions"][]' variable "k8s_version" { diff --git a/prepare-tf/modules/scaleway/main.tf b/prepare-tf/source/modules/scaleway/main.tf similarity index 93% rename from prepare-tf/modules/scaleway/main.tf rename to prepare-tf/source/modules/scaleway/main.tf index cf0cabc6..67a1a067 100644 --- a/prepare-tf/modules/scaleway/main.tf +++ b/prepare-tf/source/modules/scaleway/main.tf @@ -1,5 +1,6 @@ resource "scaleway_k8s_cluster" "_" { name = var.cluster_name + region = var.location tags = var.common_tags version = var.k8s_version cni = var.cni diff --git a/prepare-tf/modules/scaleway/outputs.tf b/prepare-tf/source/modules/scaleway/outputs.tf similarity index 76% rename from prepare-tf/modules/scaleway/outputs.tf rename to prepare-tf/source/modules/scaleway/outputs.tf index f09f50d8..9c8fa4f6 100644 --- a/prepare-tf/modules/scaleway/outputs.tf +++ b/prepare-tf/source/modules/scaleway/outputs.tf @@ -7,5 +7,5 @@ output "cluster_id" { } output "has_metrics_server" { - value = var.k8s_version >= 1.22 + value = sort([var.k8s_version, "1.22"])[0] == "1.22" } diff --git a/prepare-tf/modules/scaleway/providers.tf b/prepare-tf/source/modules/scaleway/providers.tf similarity index 100% rename from prepare-tf/modules/scaleway/providers.tf rename to prepare-tf/source/modules/scaleway/providers.tf diff --git a/prepare-tf/modules/scaleway/variables.tf b/prepare-tf/source/modules/scaleway/variables.tf similarity index 89% rename from prepare-tf/modules/scaleway/variables.tf rename to prepare-tf/source/modules/scaleway/variables.tf index 4753a15e..26e0e905 100644 --- a/prepare-tf/modules/scaleway/variables.tf +++ b/prepare-tf/source/modules/scaleway/variables.tf @@ -47,7 +47,12 @@ variable "cni" { default = "cilium" } -# See supported versions with: +variable "location" { + type = string + default = null +} + +# To view supported versions, run: # scw k8s version list -o json | jq -r .[].name variable "k8s_version" { type = string diff --git a/prepare-tf/providers.tf b/prepare-tf/source/providers.tf similarity index 100% rename from prepare-tf/providers.tf rename to prepare-tf/source/providers.tf diff --git a/prepare-tf/stage2.tmpl b/prepare-tf/source/stage2.tmpl similarity index 100% rename from prepare-tf/stage2.tmpl rename to prepare-tf/source/stage2.tmpl diff --git a/prepare-tf/variables.tf b/prepare-tf/source/variables.tf similarity index 89% rename from prepare-tf/variables.tf rename to prepare-tf/source/variables.tf index b101d535..da7b559c 100644 --- a/prepare-tf/variables.tf +++ b/prepare-tf/source/variables.tf @@ -28,3 +28,9 @@ variable "enable_arm_pool" { type = bool default = false } + +variable "location" { + type = string + default = null +} +