🏭️ Add wrapper script for 'prepare-tf'

This should make it easy to start a bunch of clusters
(using the new Terraform provisioning method) on various
providers.

prepare-tf/README.md

@@ -1,29 +1,22 @@
+⚠️ This is work in progress. The UX needs to be improved,
+and the docs could be better.
+
 This directory contains a Terraform configuration to deploy
 a bunch of Kubernetes clusters on various cloud providers,
 using their respective managed Kubernetes products.
 
-To use it:
+## With shell wrapper
 
-1. Select the provider you wish to use.
+This is the recommended use. It makes it easy to start N clusters
+on any provider. It will create a directory with a name like
+`tag-YYYY-MM-DD-HH-MM-SS-SEED-PROVIDER`, copy the Terraform configuration
+to that directory, then create the clusters using that configuration.
 
-Change the `source` attribute of the `module "clusters"` section.
-Check the content of the `modules` directory to see available choices.
-
-```bash
-vim main.tf
-```
-
-2. Initialize the provider.
-
-```bash
-terraform init
-```
-
-3. Configure provider authentication.
+1. One-time setup: configure provider authentication for the provider(s) that you wish to use.
 
 - Digital Ocean:
   ```bash
-  export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ")
+  doctl auth init
   ```
 
 - Google Cloud Platform: you will need to create a project named `prepare-tf`
@@ -33,25 +26,88 @@ terraform init
 
 - Linode:
   ```bash
-  export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ")
+  linode-cli configure
   ```
 
-- Oracle Cloud: if you have setup the OCI CLI (and have a `~/.oci/config` config file),
-  Terraform will use it by default
+- Oracle Cloud: FIXME
+  (set up `oci` through the `oci-cli` Python package)
 
 - Scaleway: run `scw init`
 
-4. Decide how many clusters and how many nodes per clusters you want.
+2. Optional: set number of clusters, cluster size, and region.
+
+By default, 1 cluster will be configured, with 2 nodes, and auto-scaling up to 5 nodes.
+
+If you want, you can override these parameters, with the following variables.
 
 ```bash
 export TF_VAR_how_many_clusters=5
 export TF_VAR_min_nodes_per_pool=2
-# Optional (will enable autoscaler when available)
 export TF_VAR_max_nodes_per_pool=4
-# Optional (will only work on some providers)
-export TF_VAR_enable_arm_pool=true
+export TF_VAR_location=xxx
 ```
 
+The `location` variable is optional. Each provider should have a default value.
+The value of the `location` variable is provider-specific. Examples:
+
+| Provider      | Example value     | How to see possible values
+|---------------|-------------------|---------------------------
+| Digital Ocean | `ams3`            | `doctl compute region list`
+| Google Cloud  | `europe-north1-a` | `gcloud  compute zones list`
+| Linode        | `eu-central`      | `linode-cli regions list`
+| Oracle Cloud  | `eu-stockholm-1`  | `oci iam region list`
+
+3. Run!
+
+```bash
+./run.sh <providername>
+```
+
+(If you don't specify a provider name, it will list available providers.)
+
+4. Shutting down
+
+Go to the directory that was created by the previous step (`tag-YYYY-MM...`)
+and run `terraform destroy`.
+
+You can also run `./clean.sh` which will destroy ALL clusters deployed by the previous run script.
+
+## Without shell wrapper
+
+Expert mode.
+
+Useful to run steps sperarately, and/or when working on the Terraform configurations.
+
+1. Select the provider you wish to use.
+
+Go to the `source` directory and edit `main.tf`.
+
+Change the `source` attribute of the `module "clusters"` section.
+
+Check the content of the `modules` directory to see available choices.
+
+2. Initialize the provider.
+
+```bash
+terraform init
+```
+
+3. Configure provider authentication.
+
+See steps above, and add the following extra steps:
+
+- Digital Coean:
+  ```bash
+  export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ")
+  ```
+
+- Linode:
+  ```bash
+  export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ")
+  ```
+
+4. Decide how many clusters and how many nodes per clusters you want.
+
 5. Provision clusters.
 
 ```bash
@@ -60,7 +116,7 @@ terraform apply
 
 6. Perform second stage provisioning.
 
-This will install a SSH server on the clusters.
+This will install an SSH server on the clusters.
 
 ```bash
 cd stage2

prepare-tf/cleanup.sh

@@ -0,0 +1,9 @@
+#!/bin/sh
+export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ")
+export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ")
+for T in  tag-*; do
+(
+  cd $T
+  terraform apply -destroy -auto-approve && mv ../$T ../deleted$T
+)
+done

prepare-tf/run.sh

@@ -0,0 +1,49 @@
+#!/bin/sh
+set -e
+
+TIME=$(which time)
+
+PROVIDER=$1
+[ "$PROVIDER" ] || {
+  echo "Please specify a provider as first argument, or 'ALL' for parallel mode."
+  echo "Available providers:"
+  ls -1 source/modules
+  exit 1
+}
+
+[ "$TAG" ] || {
+  TIMESTAMP=$(date +%Y-%m-%d-%H-%M-%S)
+  RANDOMTAG=$(base64 /dev/urandom | tr A-Z a-z | tr -d /+ | head -c5)
+  export TAG=tag-$TIMESTAMP-$RANDOMTAG
+}
+
+[ "$PROVIDER" = "ALL" ] && {
+  for PROVIDER in $(ls -1 source/modules); do
+    $TERMINAL -T $TAG-$PROVIDER -e sh -c "
+      export TAG=$TAG-$PROVIDER
+      $0 $PROVIDER
+      cd $TAG-$PROVIDER
+      bash
+      " &
+  done
+  exit 0
+}
+
+[ -d "source/modules/$PROVIDER" ] || {
+  echo "Provider '$PROVIDER' not found."
+  echo "Available providers:"
+  ls -1 source/modules
+  exit 1  
+}
+
+export LINODE_TOKEN=$(grep ^token ~/.config/linode-cli | cut -d= -f2 | tr -d " ")
+export DIGITALOCEAN_ACCESS_TOKEN=$(grep ^access-token ~/.config/doctl/config.yaml | cut -d: -f2 | tr -d " ")
+
+cp -a source $TAG
+cd $TAG
+cp -r modules/$PROVIDER modules/PROVIDER
+$TIME -o time.1.init terraform init
+$TIME -o time.2.stage1 terraform apply -auto-approve
+cd stage2
+$TIME -o ../time.3.init terraform init
+$TIME -o ../time.4.stage2 terraform apply -auto-approve

prepare-tf/source/main.tf

@@ -1,5 +1,5 @@
 module "clusters" {
-  source             = "./modules/linode"
+  source             = "./modules/PROVIDER"
   for_each           = local.clusters
   cluster_name       = each.value.cluster_name
   min_nodes_per_pool = var.min_nodes_per_pool
@@ -7,6 +7,7 @@ module "clusters" {
   enable_arm_pool    = var.enable_arm_pool
   node_size          = var.node_size
   common_tags        = local.common_tags
+  location           = var.location
 }
 
 locals {

prepare-tf/source/modules/digitalocean/main.tf

@@ -1,12 +1,13 @@
 resource "digitalocean_kubernetes_cluster" "_" {
   name    = var.cluster_name
-  tags    = local.common_tags
-  region  = var.region
+  tags    = var.common_tags
+  # Region is mandatory, so let's provide a default value.
+  region  = var.location!=null ? var.location : "nyc1"
   version = var.k8s_version
 
   node_pool {
     name       = "x86"
-    tags       = local.common_tags
+    tags       = var.common_tags
     size       = local.node_type
     auto_scale = true
     min_nodes  = var.min_nodes_per_pool

prepare-tf/source/modules/digitalocean/outputs.tf

@@ -5,3 +5,7 @@ output "kubeconfig" {
 output "cluster_id" {
   value = digitalocean_kubernetes_cluster._.id
 }
+
+output "has_metrics_server" {
+  value = false
+}

prepare-tf/source/modules/digitalocean/variables.tf

@@ -8,10 +8,6 @@ variable "common_tags" {
   default = []
 }
 
-locals {
-  common_tags = [for tag in var.common_tags : replace(tag, "=", "-")]
-}
-
 variable "node_size" {
   type    = string
   default = "M"
@@ -48,9 +44,9 @@ locals {
 
 # To view supported regions, run:
 # doctl compute region list
-variable "region" {
+variable "location" {
   type    = string
-  default = "nyc1"
+  default = null
 }
 
 # To view supported versions, run:

prepare-tf/source/modules/googlecloud/main.tf

@@ -1,7 +1,8 @@
 resource "google_container_cluster" "_" {
   name               = var.cluster_name
   project            = "prepare-tf"
-  location           = "europe-north1-a"
+  # "location" is mandatory, so let's provide a default value if none was given.
+  location           = var.location!=null ? var.location : "europe-north1-a"
   min_master_version = var.k8s_version
 
   node_pool {

prepare-tf/source/modules/googlecloud/variables.tf

@@ -42,7 +42,14 @@ locals {
   node_type = var.node_types[var.node_size]
 }
 
-# See supported versions with:
+# To view supported locations, run:
+# gcloud compute zones list
+variable "location" {
+  type    = string
+  default = null
+}
+
+# To view supported versions, run:
 # gcloud container get-server-config --region=europe-north1 '--format=flattened(channels)'
 # But it's also possible to just specify e.g. "1.20" and it figures it out.
 variable "k8s_version" {

prepare-tf/source/modules/linode/main.tf

@@ -1,7 +1,8 @@
 resource "linode_lke_cluster" "_" {
   label       = var.cluster_name
   tags        = var.common_tags
-  region      = var.region
+  # "region" is mandatory, so let's provide a default value if none was given.
+  region      = var.location!=null ? var.location : "eu-central"
   k8s_version = var.k8s_version
 
   pool {

prepare-tf/source/modules/linode/variables.tf

@@ -42,11 +42,11 @@ locals {
   node_type = var.node_types[var.node_size]
 }
 
-# To view supported versions, run:
+# To view supported regions, run:
 # linode-cli regions list
-variable "region" {
+variable "location" {
   type    = string
-  default = "us-east"
+  default = null
 }
 
 # To view supported versions, run:

prepare-tf/source/modules/oraclecloud/variables.tf

@@ -70,6 +70,13 @@ locals {
   node_type = var.node_types[var.node_size]
 }
 
+# To view supported regions, run:
+# oci iam region list | jq .data[].name
+variable "location" {
+  type    = string
+  default = null
+}
+
 # To view supported versions, run:
 # oci ce cluster-options get --cluster-option-id all | jq -r '.data["kubernetes-versions"][]'
 variable "k8s_version" {

prepare-tf/source/modules/scaleway/main.tf

@@ -1,5 +1,6 @@
 resource "scaleway_k8s_cluster" "_" {
   name                        = var.cluster_name
+  region                      = var.location
   tags                        = var.common_tags
   version                     = var.k8s_version
   cni                         = var.cni

prepare-tf/source/modules/scaleway/outputs.tf

@@ -7,5 +7,5 @@ output "cluster_id" {
 }
 
 output "has_metrics_server" {
-  value = var.k8s_version >= 1.22
+  value = sort([var.k8s_version, "1.22"])[0] == "1.22"
 }

prepare-tf/source/modules/scaleway/variables.tf

@@ -47,7 +47,12 @@ variable "cni" {
   default = "cilium"
 }
 
-# See supported versions with:
+variable "location" {
+  type    = string
+  default = null
+}
+
+# To view supported versions, run:
 # scw k8s version list -o json | jq -r .[].name
 variable "k8s_version" {
   type    = string

prepare-tf/source/variables.tf

@@ -28,3 +28,9 @@ variable "enable_arm_pool" {
   type    = bool
   default = false
 }
+
+variable "location" {
+  type    = string
+  default = null
+}
+