github/container.training

Fork 0

mirror of https://github.com/jpetazzo/container.training.git synced 2026-05-21 16:22:51 +00:00

Files

Jérôme Petazzoni 12d9f06f8a ➕ Add YTT content

2022-06-23 08:37:50 +02:00

11 KiB

Raw Blame History

YTT

YAML Templating Tool
Part of Carvel

(a set of tools for Kubernetes application building, configuration, and deployment)
Can be used for any YAML

(Kubernetes, Compose, CI pipelines...)

Features

Manipulate data structures, not text (≠ Helm)
Deterministic, hermetic execution
Define variables, blocks, functions
Write code in Starlark (dialect of Python)
Define and override values (Helm-style)
Patch resources arbitrarily (Kustomize-style)

Getting started

Install ytt (binary download)
Start with one (or multiple) Kubernetes YAML files

(without comments; no # allowed at this point!)
ytt -f one.yaml -f two.yaml | kubectl apply -f-
ytt -f. | kubectl apply -f-

No comments?!?

Replace # with #!
#@ is used by ytt

It's a kind of template tag, for instance:

#! This is a comment
#@ a = 42
#@ b = "*"
a: #@ a
b: #@ b
operation: multiply
result: #@ a*b

#@ at the beginning of a line = instruction
#@ somewhere else = value

Building strings

Concatenation:

  #@ repository = "dockercoins"
  #@ tag = "v0.1"
  containers:
  - name: worker
    image: #@ repository + "/worker:" + tag

Formatting:

  #@ repository = "dockercoins"
  #@ tag = "v0.1"
  containers:
  - name: worker
    image: #@ "{}/worker:{}".format(repository, tag)

Defining functions

Reusable functions can be written in Starlark (=Python)
Blocks (def, if, for...) must be terminated with #@ end

Example:

  #@ def image(component, repository="dockercoins", tag="v0.1"):
  #@   return "{}/{}:{}".format(repository, component, tag)
  #@ end
  containers:
  - name: worker
    image: #@ image("worker")
  - name: hasher
    image: #@ image("hasher")

Structured data

Functions can return complex types

Example: defining a common set of labels

  #@ name = "worker"
  #@ def labels(component):
  #@   return {
  #@     "app": component,
  #@     "container.training/generated-by": "ytt",
  #@   }
  #@ end
  kind: Pod
  apiVersion: v1
  metadata:
    name: #@ name
    labels: #@ labels(name)

YAML functions

Function body can also be straight YAML:

  #@ name = "worker"

  #@ def labels(component):
  app: #@ component
  container.training/generated-by: ytt
  #@ end

  kind: Pod
  apiVersion: v1
  metadata:
    name: #@ name
    labels: #@ labels(name)

The return type of the function is then a YAML fragment

More YAML functions

We can load library functions:
```
#@ load("@ytt:sha256", "sha256")
```
This is (sort of) equivalent fo from ytt.sha256 import sha256

Functions can contain a mix of code and YAML fragment:

  #@ load("@ytt:sha256", "sha256")

  #@ def annotations():
  #@ author = "Jérôme Petazzoni"
  author: #@ author
  author_hash: #@ sha256.sum(author)[:8]
  #@ end

  annotations: #@ annotations()

Data values

We can define a schema in a separate file:

  #@data/values-schema
  --- #! there must be a "---" here!
  repository: dockercoins
  tag: v0.1

This defines the data values (=customizable parameters),

as well as their types and default values
Technically, #@data/values-schema is an annotation, and it applies to a YAML document; so the following element must be a YAML document
This is conceptually similar to Helm's values file
(but with type enforcement as a bonus)

Using data values

Requires loading @ytt:data
Values are then available in data.values

Example:

  #@ load("@ytt:data", "data")
  #@ def image(component):
  #@   return "{}/{}:{}".format(data.values.repository, component, data.values.tag)
  #@ end
  #@ name = "worker"
  containers:
  - name: #@ name
    image: #@ image(name)

Overriding data values

There are many ways to set and override data values:
- plain YAML files
- data value overlays
- environment variables
- command-line flags
Precedence of the different methods is defined in the docs

Values in plain YAML files

Content of values.yaml:
```
tag: latest
```

Values get merged with --data-values-file:

ytt -f config/ --data-values-file values.yaml

Multiple files can be specified
These files can also be URLs!

Data value overlay

Content of values.yaml:

#@data/values
--- #! must have --- here
tag: latest

Values get merged by being specified like "normal" files:
```
ytt -f config/ -f values.yaml
```
Multiple files can be specified

Set a value with a flag

Set a string value:
```
ytt -f config/ --data-value tag=latest
```
Set a YAML value (useful to parse it as e.g. integer, boolean...):
```
ytt -f config/ --data-value-yaml replicas=10
```

Read a string value from a file:

ytt -f config/ --data-value-file ca_cert=cert.pem

Set values from environment variables

Set environment variables with a prefix:

export VAL_tag=latest
export VAL_repository=ghcr.io/dockercoins

Use the variables as strings:
```
ytt -f config/ --data-values-env VAL
```

Or parse them as YAML:

ytt -f config/ --data-values-env-yaml VAL

Lines starting with `#@`

This generates an empty document:

#@ def hello():
hello: world
#@ end

#@ hello()

Do this instead:

#@ def hello():
hello: world
#@ end

--- #@ hello()

Generating multiple documents, take 1

This won't work:

#@ def app():
kind: Deployment
apiVersion: apps/v1
--- #! separate from next document
kind: Service
apiVersion: v1
#@ end

--- #@ app()

Generating multiple documents, take 2

This won't work either:

#@ def app():
--- #! the initial separator indicates "this is a Document Set"
kind: Deployment
apiVersion: apps/v1
--- #! separate from next document
kind: Service
apiVersion: v1
#@ end

--- #@ app()

Generating multiple documents, take 3

We must use the template module:

#@ load("@ytt:template", "template")

#@ def app():
--- #! the initial separator indicates "this is a Document Set"
kind: Deployment
apiVersion: apps/v1
--- #! separate from next document
kind: Service
apiVersion: v1
#@ end

--- #@ template.replace(app())

template.replace(...) is the only way (?) to replace one element with many

Libraries

A reusable ytt configuration can be transformed into a library

Put it in a subdirectory named _ytt_lib/whatever, then:

#@ load("@ytt:library", "library")
#@ load("@ytt:template", "template")
#@ whatever = library.get("whatever")
#@ my_values = {"tag": "latest", "registry": "..."}
#@ output = whatever.with_data_values(my_values).eval()
--- #@ template.replace(output)

The with_data_values() step is optional, but useful to "configure" the library

Note the whole combo:

template.replace(library.get("...").with_data_values(...).eval())

Overlays

Powerful, but complex, but powerful! 💥
Define transformations that are applied after generating the whole document set
General idea:
- select YAML nodes to be transformed with an #@overlay/match decorator
- write a YAML snippet with the modifications to be applied
  (a bit like a strategic merge patch)

Example

#@ load("@ytt:overlay", "overlay")

#@ selector = {"kind": "Deployment", "metadata": {"name": "worker"}}
#@overlay/match by=overlay.subset(selector)
--- 
spec:
  replicas: 10

By default, #@overlay/match must find exactly one match

(that can be changed by specifying expects=..., missing_ok=True... see docs)
By default, the specified fields (here, spec.replicas) must exist

(that can also be changed by annotating the optional fields)

Matching using a YAML document

#@ load("@ytt:overlay", "overlay")

#@ def match():
kind: Deployment
metadata:
  name: worker
#@ end

#@overlay/match by=overlay.subset(match())
--- 
spec:
  replicas: 10

This is equivalent to the subset match of the previous slide
It will find YAML nodes having all the listed fields

Removing a field

#@ load("@ytt:overlay", "overlay")

#@ def match():
kind: Deployment
metadata:
  name: worker
#@ end

#@overlay/match by=overlay.subset(match())
--- 
spec:
  #@overlay/remove
  replicas:

This would remove the replicas: field from a specific Deployment spec
This could be used e.g. when enabling autoscaling

Selecting multiple nodes

#@ load("@ytt:overlay", "overlay")

#@ def match():
kind: Deployment
#@ end

#@overlay/match by=overlay.subset(match()), expects="1+"
--- 
spec:
  #@overlay/remove
  replicas:

This would match all Deployments
(assuming that at least one exists)
It would remove the replicas: field from their spec
(the field must exist!)

Adding a field

#@ load("@ytt:overlay", "overlay")

#@overlay/match by=overlay.all, expects="1+"
--- 
metadata:
  #@overlay/match missing_ok=True
  annotations:
    #@overlay/match expects=0
    rainbow: 🌈

#@overlay/match missing_ok=True
will match whether our resources already have annotations or not
#@overlay/match expects=0
will only match if the rainbow annotation doesn't exist
(to make sure that we don't override/replace an existing annotation)

Overlays vs data values

The documentation has a detailed discussion about this question
In short:
- values = for parameters that are exposed to the user
- overlays = for arbitrary extra modifications
Values are easier to use (use them when possible!)
Fallback to overlays when values don't expose what you need

(keeping in mind that overlays are harder to write/understand/maintain)

Gotchas

Reminder: put your #@ at the right place!

#! This will generate "hello, world!"
--- #@ "{}, {}!".format("hello", "world")

#! But this will generate an empty document
--- 
#@ "{}, {}!".format("hello", "world")

Also, don't use YAML anchors (*foo and &foo)
They don't mix well with ytt
Remember to use template.render(...) when generating multiple nodes

(or to update lists or arrays without replacing them entirely)

Next steps with ytt

Read this documentation page about injecting secrets
Check the FAQ, it gives some insights about what's possible with ytt
Exercise idea: write an overlay that will find all ConfigMaps mounted in Pods...

...and annotate the Pod with a hash of the ConfigMap

???

:EN:- YTT :FR:- YTT

11 KiB Raw Blame History

YTT