Opentelemetry and a few things on prometheus

slides/3.yml

@@ -49,6 +49,7 @@ chapters:
   - k8s/monitoring-intro.md
   #- k8s/prometheus-intro.md
   - k8s/prometheus-endpoint.md
+  - k8s/opentelemetry.md
 
 
     #-

slides/k8s/opentelemetry.md

@@ -0,0 +1,72 @@
+# OpenTelemetry
+
+*Opentelemetry* is a "tracing" framework. It's a fusion of two other frameworks:
+*opentracing* and *opencensus*.
+
+The goal is to have deep integration with software languages and application framework to
+enable deep dive tracing of different events accross different components
+
+---
+## Span ! span ! span !
+
+- A unit a tracing is called a `span`.
+
+- A span has a start time and and stop time and an ID.
+
+- It represent an action that took some time to complete
+
+    ex: call to function `B`, DB transation, REST call to a backend...
+
+- A span could have a parent and could be parent of multiple child spans.
+
+    ex: during the call to function `B`, a sub call to `C` and `D` has been issued
+
+- Think of it as a "tree" of calls
+
+---
+## Distributed tracing
+
+- This could be applied to multiple components
+
+    ex: If microservice `A` send REST call to microservice `B`
+
+    - `A` will have a span for the call to `B`
+    - `B` will have a span for the call from `A`
+       (that normally starts shortly after, and finishes shortly before)
+    - the span of `A` will be the parent of the span of `B`,
+       so that they join the same "tree" of call
+
+details: `A` will send headers (depends of the protocol used) to tag the span ID,
+so that `B` can generate child span and joining the same tree of call
+
+---
+## Centrally stored
+
+- We do have "spans", ok. But what do we do with that ?
+
+- We store them.
+
+- In the previous exemple:
+
+    - `A` will send trace to it's local agent
+    - `B` will do the same
+    - Every span will ends up in the same DB so that we can reconstruct the "tree" of call
+      later on and analyze it.
+
+- there is multiple implementation of those agents + DB + WebUI. The most famous opensource ones:
+
+    - Zipkin
+    - Jaeger
+
+---
+## Distributed sampled
+
+- Huh, we store all of them ? (that could be a lot of storage)
+
+- No, we could apply sampling, to reduce storage/network footprint.
+
+- Smart sampling is applied directly in the application to save CPU if span is not needed.
+
+- It also insures that if a span is mark as sampled, all child-span are sampled together
+
+    (so that the tree of call is complete)

slides/k8s/prometheus-endpoint.md

@@ -1,3 +1,10 @@
+# Prometheus
+
+Prometheus is monitoring system with small storage io footprint. It's quite ubiquitous
+
+in the kubernetes world. This section is not a description
+
+
 ## Prometheus endpoint
 
 The goal here is to expose an HTTP endoint for prometheus. Sample response:
@@ -53,3 +60,27 @@ Links (do you see a pattern ?):
 
 - Try to not expose monitoring channel more than needed. Often localhost is enough
     (sidecars run in the same network namespace as other containers)
+
+---
+## Ok! and then change prometheus conf ?
+
+- Well, not really. It achievable this way, but...
+
+- Prometheus has good service discovery paired with kubernetes.
+
+- Depending on how we installed prometheus, we just need:
+
+    - pods annotations:
+
+       ```
+        annotations:
+          prometheus.io/port: 9090
+          prometheus.io/path: /metrics
+       ```
+
+    - *service monitor* custom resource object
+.small[
+        https://github.com/coreos/prometheus-operator/blob/master/Documentation/api.md#servicemonitor
+]
+
+*Note: More on prometheus next day*

slides/k8s/software-dev-banalities.md

@@ -1,4 +1,4 @@
-# Software development
+## Software development
 
 From years, decades, (centuries !), software development has followed the same principles: