github/container.training
1
0
Fork 0
mirror of https://github.com/jpetazzo/container.training.git synced 2026-02-14 09:39:56 +00:00
Code Issues Packages Projects Releases Wiki Activity

Revamp operator example

Browse Source 
Use Elastic Cloud for Kubernetes instead of the
UPMC Enterprises operator.
This commit is contained in:
Jerome Petazzoni
2020-01-19 11:32:04 -06:00
parent ba323cb4e6
commit a32df01165
6 changed files with 2342 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
k8s/eck-cerebro.yaml Normal file
View File

@@ -0,0 +1,69 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: cerebro
name: cerebro
spec:
selector:
matchLabels:
app: cerebro
template:
metadata:
labels:
app: cerebro
spec:
volumes:
- name: conf
configMap:
name: cerebro
containers:
- image: lmenezes/cerebro
name: cerebro
volumeMounts:
- name: conf
mountPath: /conf
args:
- -Dconfig.file=/conf/application.conf
env:
- name: ELASTICSEARCH_PASSWORD
valueFrom:
secretKeyRef:
name: demo-es-elastic-user
key: elastic
---
apiVersion: v1
kind: Service
metadata:
labels:
app: cerebro
name: cerebro
spec:
ports:
- port: 9000
protocol: TCP
targetPort: 9000
selector:
app: cerebro
type: NodePort
---
apiVersion: v1
kind: ConfigMap
metadata:
name: cerebro
data:
application.conf: |
secret = "ki:s:[[@=Ag?QI`W2jMwkY:eqvrJ]JqoJyi2axj3ZvOv^/KavOT4ViJSv?6YY4[N"
hosts = [
{
host = "http://demo-es-http.eck-demo.svc.cluster.local:9200"
name = "demo"
auth = {
username = "elastic"
password = ${?ELASTICSEARCH_PASSWORD}
}
}
]

19
k8s/eck-elasticsearch.yaml Normal file
View File

@@ -0,0 +1,19 @@
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: demo
namespace: eck-demo
spec:
http:
tls:
selfSignedCertificate:
disabled: true
nodeSets:
- name: default
count: 1
config:
node.data: true
node.ingest: true
node.master: true
node.store.allow_mmap: false
version: 7.5.1

168
k8s/eck-filebeat.yaml Normal file
View File

@@ -0,0 +1,168 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-config
namespace: eck-demo
labels:
k8s-app: filebeat
data:
filebeat.yml: |-
filebeat.inputs:
- type: container
paths:
- /var/log/containers/*.log
processors:
- add_kubernetes_metadata:
host: ${NODE_NAME}
matchers:
- logs_path:
logs_path: "/var/log/containers/"
# To enable hints based autodiscover, remove `filebeat.inputs` configuration and uncomment this:
#filebeat.autodiscover:
# providers:
# - type: kubernetes
# node: ${NODE_NAME}
# hints.enabled: true
# hints.default_config:
# type: container
# paths:
# - /var/log/containers/*${data.kubernetes.container.id}.log
processors:
- add_cloud_metadata:
- add_host_metadata:
cloud.id: ${ELASTIC_CLOUD_ID}
cloud.auth: ${ELASTIC_CLOUD_AUTH}
output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
username: ${ELASTICSEARCH_USERNAME}
password: ${ELASTICSEARCH_PASSWORD}
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: filebeat
namespace: eck-demo
labels:
k8s-app: filebeat
spec:
selector:
matchLabels:
k8s-app: filebeat
template:
metadata:
labels:
k8s-app: filebeat
spec:
serviceAccountName: filebeat
terminationGracePeriodSeconds: 30
hostNetwork: true
dnsPolicy: ClusterFirstWithHostNet
containers:
- name: filebeat
image: docker.elastic.co/beats/filebeat:7.5.1
args: [
"-c", "/etc/filebeat.yml",
"-e",
]
env:
- name: ELASTICSEARCH_HOST
value: demo-es-http
- name: ELASTICSEARCH_PORT
value: "9200"
- name: ELASTICSEARCH_USERNAME
value: elastic
- name: ELASTICSEARCH_PASSWORD
valueFrom:
secretKeyRef:
name: demo-es-elastic-user
key: elastic
- name: ELASTIC_CLOUD_ID
value:
- name: ELASTIC_CLOUD_AUTH
value:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
securityContext:
runAsUser: 0
# If using Red Hat OpenShift uncomment this:
#privileged: true
resources:
limits:
memory: 200Mi
requests:
cpu: 100m
memory: 100Mi
volumeMounts:
- name: config
mountPath: /etc/filebeat.yml
readOnly: true
subPath: filebeat.yml
- name: data
mountPath: /usr/share/filebeat/data
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
- name: varlog
mountPath: /var/log
readOnly: true
volumes:
- name: config
configMap:
defaultMode: 0600
name: filebeat-config
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
- name: varlog
hostPath:
path: /var/log
# data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
- name: data
hostPath:
path: /var/lib/filebeat-data
type: DirectoryOrCreate
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: filebeat
subjects:
- kind: ServiceAccount
name: filebeat
namespace: eck-demo
roleRef:
kind: ClusterRole
name: filebeat
apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: filebeat
labels:
k8s-app: filebeat
rules:
- apiGroups: [""] # "" indicates the core API group
resources:
- namespaces
- pods
verbs:
- get
- watch
- list
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: filebeat
namespace: eck-demo
labels:
k8s-app: filebeat
---

17
k8s/eck-kibana.yaml Normal file
View File

@@ -0,0 +1,17 @@
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
name: demo
spec:
version: 7.5.1
count: 1
elasticsearchRef:
name: demo
namespace: eck-demo
http:
service:
spec:
type: NodePort
tls:
selfSignedCertificate:
disabled: true

1802
k8s/eck-operator.yaml Normal file
View File

File diff suppressed because it is too large Load Diff

306
slides/k8s/operators.md
View File

@@ -121,7 +121,7 @@ Examples:
## One operator in action
- We will install the UPMC Enterprises ElasticSearch operator
- We will install [Elastic Cloud on Kubernetes](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-quickstart.html), an ElasticSearch operator
- This operator requires PersistentVolumes
@@ -206,51 +206,92 @@ Now, the StorageClass should have `(default)` next to its name.
## Install the ElasticSearch operator
- The operator needs:
- The operator provides:
- a Deployment for its controller
- a few CustomResourceDefinitions
- a Namespace for its other resources
- a ValidatingWebhookConfiguration for type checking
- a StatefulSet for its controller and webhook code
- a ServiceAccount, ClusterRole, ClusterRoleBinding for permissions
- a Namespace
- We have grouped all the definitions for these resources in a YAML file
- All these resources are grouped in a convenient YAML file
.exercise[
- Install the operator:
```bash
kubectl apply -f ~/container.training/k8s/elasticsearch-operator.yaml
kubectl apply -f ~/container.training/k8s/eck-operator.yaml
```
]
---
## Wait for the operator to be ready
## Check our new custom resources
- Some operators require to create their CRDs separately
- This operator will create its CRD itself
(i.e. the CRD is not listed in the YAML that we applied earlier)
- Let's see which CRDs were created
.exercise[
- Wait until the `elasticsearchclusters` CRD shows up:
- List all CRDs:
```bash
kubectl get crds
```
]
This operator supports ElasticSearch, but also Kibana and APM. Cool!
---
## Create the `eck-demo` namespace
- For clarity, we will create everything in a new namespace, `eck-demo`
- This namespace is hard-coded in the YAML files that we are going to use
- We need to create that namespace
.exercise[
- Create the `eck-demo` namespace:
```bash
kubectl create namespace eck-demo
```
- Switch to that namespace:
```bash
kns eck-demo
```
]
---
class: extra-details
## Can we use a different namespace?
Yes, but then we need to update all the YAML manifests that we
are going to apply in the next slides.
The `eck-demo` namespace is hard-coded in these YAML manifests.
Why?
Because when defining a ClusterRoleBinding that references a
ServiceAccount, we have to indicate in which namespace the
ServiceAccount is located.
---
## Create an ElasticSearch resource
- We can now create a resource with `kind: ElasticsearchCluster`
- We can now create a resource with `kind: ElasticSearch`
- The YAML for that resource will specify all the desired parameters:
- how many nodes do we want of each type (client, master, data)
- how many nodes we want
- image to use
- add-ons (kibana, cerebro, ...)
- whether to use TLS or not
@@ -260,7 +301,7 @@ Now, the StorageClass should have `(default)` next to its name.
- Create our ElasticSearch cluster:
```bash
kubectl apply -f ~/container.training/k8s/elasticsearch-cluster.yaml
kubectl apply -f ~/container.training/k8s/eck-elasticsearch.yaml
```
]
@@ -269,49 +310,88 @@ Now, the StorageClass should have `(default)` next to its name.
## Operator in action
- Over the next minutes, the operator will create:
- Over the next minutes, the operator will create our ES cluster
- StatefulSets (one for master nodes, one for data nodes)
- Deployments (for client nodes; and for add-ons like cerebro and kibana)
- Services (for all these pods)
- It will report our cluster status through the CRD
.exercise[
- Wait for all the StatefulSets to be fully up and running:
- Check the logs of the operator:
```bash
kubectl get statefulsets -w
stern --namespace=elastic-system operator
```
<!--
```wait elastic-operator-0```
```tmux split-pane -v```
--->
- Watch the status of the cluster through the CRD:
```bash
kubectl get es -w
```
<!--
```longwait green```
```key ^C```
```key ^D```
```key ^C```
-->
]
---
## Connecting to our cluster
- Since connecting directly to the ElasticSearch API is a bit raw,
<br/>we'll connect to the cerebro frontend instead
- It's not easy to use the ElasticSearch API from the shell
- But let's check at least if ElasticSearch is up!
.exercise[
- Edit the cerebro service to change its type from ClusterIP to NodePort:
- Get the ClusterIP of our ES instance:
```bash
kubectl patch svc cerebro-es -p "spec: { type: NodePort }"
kubectl get services
```
- Retrieve the NodePort that was allocated:
- Issue a request with `curl`:
```bash
kubectl get svc cerebro-es
curl http://`CLUSTERIP`:9200
```
- Connect to that port with a browser
]
We get an authentication error. Our cluster is protected!
---
## (Bonus) Setup filebeat
## Obtaining the credentials
- The operator creates a user named `elastic`
- It generates a random password and stores it in a Secret
.exercise[
- Extract the password:
```bash
kubectl get secret demo-es-elastic-user \
-o go-template="{{ .data.elastic | base64decode }} "
```
- Use it to connect to the API:
```bash
curl -u elastic:`PASSWORD` http://`CLUSTERIP`:9200
```
]
We should see a JSON payload with the `"You Know, for Search"` tagline.
---
## Sending data to the cluster
- Let's send some data to our brand new ElasticSearch cluster!
@@ -321,22 +401,170 @@ Now, the StorageClass should have `(default)` next to its name.
- Deploy filebeat:
```bash
kubectl apply -f ~/container.training/k8s/filebeat.yaml
kubectl apply -f ~/container.training/k8s/eck-filebeat.yaml
```
- Wait until some pods are up:
```bash
watch kubectl get pods -l k8s-app=filebeat
```
<!--
```wait Running```
```key ^C```
-->
- Check that a filebeat index was created:
```bash
curl -u elastic:`PASSWORD` http://`CLUSTERIP`:9200/_cat/indices
```
]
We should see at least one index being created in cerebro.
---
## Deploying an instance of Kibana
- Kibana can visualize the logs injected by filebeat
- The ECK operator can also manage Kibana
- Let's give it a try!
.exercise[
- Deploy a Kibana instance:
```bash
kubectl apply -f ~/container.training/k8s/eck-kibana.yaml
```
- Wait for it to be ready:
```bash
kubectl get kibana -w
```
<!--
```longwait green```
```key ^C```
-->
]
---
## (Bonus) Access log data with kibana
## Connecting to Kibana
- Let's expose kibana (by making kibana-es a NodePort too)
- Kibana is automatically set up to conect to ElasticSearch
- Then access kibana
(this is arranged by the YAML that we're using)
- We'll need to configure kibana indexes
- However, it will ask for authentication
- It's using the same user/password as ElasticSearch
.exercise[
- Get the NodePort allocated to Kibana:
```bash
kubectl get services
```
- Connect to it with a web browser
- Use the same user/password as before
]
---
## Setting up Kibana
After the Kibana UI loads, we need to click around a bit
.exercise[
- Pick "explore on my own"
- Click on Use Elasticsearch data / Connect to your Elasticsearch index"
- Enter `filebeat-*` for the index pattern and click "Next step"
- Select `@timestamp` as time filter field name
- Click on "discover" (the small icon looking like a compass on the left bar)
- Play around!
]
---
## Scaling up the cluster
- At this point, we have only one node
- We are going to scale up
- But first, we'll deploy Cerebro, an UI for ElasticSearch
- This will let us see the state of the cluster, how indexes are sharded, etc.
---
## Deploying Cerebro
- Cerebro is stateless, so it's fairly easy to deploy
(one Deployment + one Service)
- However, it needs the address and credentials for ElasticSearch
- We prepared yet another manifest for that!
.exercise[
- Deploy Cerebro:
```bash
kubectl apply -f ~/container.training/k8s/eck-cerebro.yaml
```
- Lookup the NodePort number and connect to it:
```bash
kuebctl get services
```
]
---
## Scaling up the cluster
- We can see on Cerebro that the cluster is "yellow"
(because our index is not replicated)
- Let's change that!
.exercise[
- Edit the ElasticSearch cluster manifest:
```bash
kubectl edit es demo
```
- Find the field `count: 1` and change it to 3
- Save and quit
<!--
```wait Please edit```
```keys /count:```
```key ^J```
```keys $r3:x```
```key ^J```
-->
]
---