New Relic Gerry version (day 1)

This commit is contained in:
Gerry S
2022-08-22 10:28:39 -04:00
parent b46dcd5157
commit 6d0849eebb
38 changed files with 22045 additions and 50 deletions

View File

@@ -100,7 +100,11 @@ _We will give more details about namespaces and cgroups later._
* But it is easier to use `docker exec`.
```bash
$ docker exec -ti ticktock sh
$ docker ps -lq # Get Last Container ID
17e4e95e2702
$ docker exec 17
$ docker exec -ti $(docker ps -lq) sh # bash-fu version
```
* This creates a new process (running `sh`) _inside_ the container.

View File

@@ -0,0 +1,20 @@
class: title
# High Level Discussion
![image](images/title-understanding-docker-images.png)
---
## White Board Topics
* What is the real problem that containers solve?
* What are the inputs to a Unix Process?
* What is the init Process?
* Userland vs Kernel
* The Root File System
* What is an Overlay File System?
* Wrapping it all up to represent a container image
* Deploying Container images

View File

@@ -0,0 +1,318 @@
class: title
# A Macroscopic View
---
## Macroscopic Items
* The business case for containers
* The problem containers are solving
* What applications need
* What is the OS doing provides?
---
## What do CIOs worry about?
Who are the CIO's customers?
* Business Units: Need Computers to Run Applications
* Peak Capacity
* CFO: Demanding Budget Justifications
* Spend Less
---
## History of Solutions
For Each Business Application Buy a Machine
* Buy a machine for each application
* Big enough for Peak Load (CPU, Memory, Disk)
The Age of VMs
* Buy bigger machines and chop them up into logical machines
* Distribute your applications as VMs theses machines
* Observe what and when the application load actually is
* Possibly rebalance be to inform possibly moving
But Maintaining Machines (Bare Metal or VM) is hard (Patches, Packages, Drivers, etc)
---
## What Developers and Ops worry about
* Getting Software deployed
* Mysterious reasons why deployed application doesn't work
* Developer to Ops:
* "Hey it works on my development machine..."
* "I don't know why it isn't working for ***you***"
* "Everything ***looks*** the same"
* "I have no idea what could be different"
---
## The History of Software Deployment
Software Deployment is just a reproducible way to install files:
* Cards
* Tapes
* Floppy Disks
* Zip/Tar Files
* Installation "Files" (rpm/deb/msi)
* VM Images
---
## What is the Problem Containers are Solving?
It depends on who you are:
* For the CIO: Better resource utilization
* For Ops: Software Distribution
* For the Developer & Ops: Reproducible Environment
<BR><BR>
Ummm, but what exactly are containers....
* Wait a few more slides...
---
## Macroscopic view: Applications and the OS
Applications:
* What are the inputs/outputs to a program?
The OS:
* What does the OS provide?
---
## What are the inputs/outputs to a program?
Explicitly:
* Command Line Arguments
* Environment Variables
* Standard In
* Standard Out/Err
Implicitly (via the File System):
* Configuration Files
* Other Installed Applications
* Any other files
Also Implicitly
* Memory
* Network
---
## What does the OS provide?
* OS Kernel
* Kernel loded at boot time
* Sets up disk drives, network cards, other hardware, etc
* Manages all hardware, processes, memory, etc
* Kernel Space
* Low level innards of Kernel (fluid internal API)
* No direct access by applications of most Kernel functionality
* User Space (userland) Processes
* Code running outside the Kernel
* Very stable shim library access from User Space to Kernel Space (Think "fopen")
* The "init" Process
* User Space Process run after Kernel has booted
* Always PID 1
---
## OS Processes
* Created when an application is launched
* Each has a unique Process ID (PID)
* Provides it its own logical 'view' of all implicit inputs/output when launching app
* File System ( root directory, / )
* Memory
* Network Adaptors
* Other running processes
---
## What do we mean by "The OS"
Different Linux's
* Ubuntu / Debian; Centos / RHEL; Raspberry Pi; etc
What do they have in common?
* They all have a kernel that provides access to Userland (ie fopen)
* They typically have all the commands (bash, sh, ls, grep, ...)
What may be different?
* May use different versions of the Kernel (4.18, 5.4, ...)
* Internally different, but providing same Userland API
* Many other bundled commands, packages and package management tools
* Namely what makes it 'Debian' vs 'Centos'
---
## What might a 'Minimal' Linux be?
You could actually just have:
* A Linux Kernel
* An application (for simplicity a statically linked C program)
* The kernel configured to run that application as its 'init' process
Would you ever do this?
* Why not?
* It certainly would be very secure
---
## So Finally... What are Containers?
Containers just a Linux process that 'thinks' it is it's own machine
* With its own 'view' of things like:
* File System ( root directory, / ), Memory, Network Adaptors, Other running processes
* Leverages our understanding that a (logical) Linux Machine is
* A kernel
* A bunch of files ( Maybe a few Environment Variables )
Since it is a process running on a host machine
* It uses the kernel of the host machine
* And of course you need some tools to create the running container process
---
## Container Runtimes and Container Images
The Linux kernel actually has no concept of a container.
* There have been many 'container' technologies
* See [A Brief History of containers: From the 1970's till now](https://blog.aquasec.com/a-brief-history-of-containers-from-1970s-chroot-to-docker-2016)
* Over the years more capabilities have been added to the kernel to make it easier
<BR>
A 'Container technology' is:
* A Container Image Format of the unit of software deployment
* A bundle of all the files and miscellaneous configuration
* A Container Runtime Engine
* Software that takes a Container Image and creates a running container
---
## The Container Runtime War is now Over
The Cloud Native Computing Foundation (CNCF) has standardized containers
* A standard container image format
* A standard for building and configuring container runtimes
* A standard REST API for loading/downloading container image to a registries
There primary Container Runtimes are:
* containerd: using the 'docker' Command Line Interface (or Kubernetes)
* CRI-O: using the 'podman' Command Line Interface (or Kubernetes/OpenShift)
* Others exists, for example Singularity which has a history in HPC
---
## Linux Namespaces Makes Containers Possible
- Provide processes with their own isolated view of the system.
- Namespaces limit what you can see (and therefore, what you can use).
- These namespaces are available in modern kernels:
- pid: processes
- net: network
- mnt: root file system (ie chroot)
- uts: hostname
- ipc
- user: UID/GID mapping
- time: time
- cgroup: Resource Monitoring and Limiting
- Each process belongs to one namespace of each type.
---
## Namespaces are always active
- Namespaces exist even when you don't use containers.
- This is a bit similar to the UID field in UNIX processes:
- all processes have the UID field, even if no user exists on the system
- the field always has a value / the value is always defined
<br/>
(i.e. any process running on the system has some UID)
- the value of the UID field is used when checking permissions
<br/>
(the UID field determines which resources the process can access)
- You can replace "UID field" with "namespace" above and it still works!
- In other words: even when you don't use containers,
<br/>there is one namespace of each type, containing all the processes on the system.

View File

@@ -0,0 +1,224 @@
class: title
# Our training environment
![SSH terminal](images/title-our-training-environment.jpg)
---
class: in-person
## Connecting to your Virtual Machine
You need an SSH client.
* On OS X, Linux, and other UNIX systems, just use `ssh`:
```bash
$ ssh <login>@<ip-address>
```
* On Windows, if you don't have an SSH client, you can download:
* Putty (www.putty.org)
* Git BASH (https://git-for-windows.github.io/)
* MobaXterm (https://mobaxterm.mobatek.net/)
---
class: in-person
## Connecting to our lab environment
.lab[
- Log into your VM with your SSH client:
```bash
ssh `user`@`A.B.C.D`
```
(Replace `user` and `A.B.C.D` with the user and IP address provided to you)
]
You should see a prompt looking like this:
```
[A.B.C.D] (...) user@node1 ~
$
```
If anything goes wrong — ask for help!
---
## Our Docker VM
About the Lab VM
- The VM is created just before the training.
- It will stay up during the whole training.
- It will be destroyed shortly after the training.
- It comes pre-loaded with Docker and some other useful tools.
---
## Why don't we run Docker locally?
- I can log into your VMs to help you with labs
- Installing docker is out of the scope of this class (lots of online docs)
- It's better to spend time learning containers than fiddling with the installer!
---
class: in-person
## `tailhist`
- The shell history of the instructor is available online in real time
- Note the IP address of the instructor's virtual machine (A.B.C.D)
- Open http://A.B.C.D:1088 in your browser and you should see the history
- The history is updated in real time (using a WebSocket connection)
- It should be green when the WebSocket is connected
(if it turns red, reloading the page should fix it)
- If you want to play with it on your lab machine, tailhist is installed
- sudo apt install firewalld
- sudo firewall-cmd --add-port=1088/tcp
---
## Checking your Virtual Machine
Once logged in, make sure that you can run a basic Docker command:
.small[
```bash
$ docker version
Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:10:06 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:08:35 2018
OS/Arch: linux/amd64
Experimental: false
```
]
If this doesn't work, raise your hand so that an instructor can assist you!
???
:EN:Container concepts
:FR:Premier contact avec les conteneurs
:EN:- What's a container engine?
:FR:- Qu'est-ce qu'un *container engine* ?
---
## Doing or re-doing the workshop on your own?
- Use something like
[Play-With-Docker](http://play-with-docker.com/) or
[Play-With-Kubernetes](https://training.play-with-kubernetes.com/)
Zero setup effort; but environment are short-lived and
might have limited resources
- Create your own cluster (local or cloud VMs)
Small setup effort; small cost; flexible environments
- Create a bunch of clusters for you and your friends
([instructions](https://@@GITREPO@@/tree/master/prepare-vms))
Bigger setup effort; ideal for group training
---
class: self-paced
## Get your own Docker nodes
- If you already have some Docker nodes: great!
- If not: let's get some thanks to Play-With-Docker
.lab[
- Go to http://www.play-with-docker.com/
- Log in
- Create your first node
<!-- ```open http://www.play-with-docker.com/``` -->
]
You will need a Docker ID to use Play-With-Docker.
(Creating a Docker ID is free.)
---
## Terminals
Once in a while, the instructions will say:
<br/>"Open a new terminal."
There are multiple ways to do this:
- create a new window or tab on your machine, and SSH into the VM;
- use screen or tmux on the VM and open a new window from there.
You are welcome to use the method that you feel the most comfortable with.
---
## Tmux cheat sheet
[Tmux](https://en.wikipedia.org/wiki/Tmux) is a terminal multiplexer like `screen`.
*You don't have to use it or even know about it to follow along.
<br/>
But some of us like to use it to switch between terminals.
<br/>
It has been preinstalled on your workshop nodes.*
- Ctrl-b c → creates a new window
- Ctrl-b n → go to next window
- Ctrl-b p → go to previous window
- Ctrl-b " → split window top/bottom
- Ctrl-b % → split window left/right
- Ctrl-b Alt-1 → rearrange windows in columns
- Ctrl-b Alt-2 → rearrange windows in rows
- Ctrl-b arrows → navigate to other windows
- Ctrl-b d → detach session
- tmux attach → re-attach to session

View File

@@ -0,0 +1,27 @@
```bash
$ docker run -it debian
root@ef22f9437171:/# apt-get update
root@ef22f9437171:/# apt-get install skopeo
root@ef22f9437171:/# apt-get wget curl jq
root@ef22f9437171:/# skopeo login docker.io -u containertraining -p testaccount
$ docker commit $(docker ps -lq) skop
```
```bash
root@0ab665194c4f:~# skopeo copy docker://docker.io/containertraining/test-image-0 dir:/root/test-image-0
root@0ab665194c4f:~# cd /root/test-image-0
root@0ab665194c4f:~# jq <manifest.json .layers[].digest
```
Stuff in Exploring-images
image-test-0/1/2 + jpg

View File

@@ -0,0 +1,20 @@
FROM busybox
ADD verifyImageFiles.sh /
WORKDIR /play
RUN echo "== LAYER 0 ==" && \
echo "A is for Aardvark" >A && \
echo "B is for Beetle" >B && \
mkdir C/ && \
echo "A is for Cowboy Allan" >C/CA && \
mkdir -p C/CB && \
echo "A is for Cowboy Buffalo Alex" >C/CB/CBA && \
echo "B is for Cowboy Buffalo Bill" >C/CB/CBB && \
echo "Z is for Cowboy Zeke" >> C/CZ && \
mkdir D/ && \
echo "A is for Detective Alisha" >D/DA && \
echo "B is for Detective Betty" >D/DB && \
echo "E is for Elephant" >E && \
find . >../state.layer-0

View File

@@ -0,0 +1,17 @@
FROM test-image-0
WORKDIR /play
RUN echo "== LAYER 1 == Change File B, Create File C/CC, Add Dir C/CD, Remove File E, Create Dir F, Add File G, Create Empty Dir H" && \
echo "B is for Butterfly" >B && \
echo "C is for Cowboy Chuck">C/CC && \
mkdir -p C/CD && \
echo "A is for Cowboy Dandy Austin" >C/CD/CDA && \
rm E && \
mkdir F && \
echo "A is for Ferret Albert" >F/FA && \
echo "G is for Gorilla" >G && \
mkdir H && \
find . >../state.layer-1

View File

@@ -0,0 +1,18 @@
FROM test-image-1
WORKDIR /play
RUN echo "== LAYER 2 == Remove File C/CA, Remove Dir G, Remove Dir D / Replace with new Dir D, Remove Dir C/CB, Remove Dir C/CB, Remove Dir F, Add File G, Remove Dir H / Create File H" && \
rm C/CA && \
rm -rf C/CB && \
echo "Z is for Cowboy Zoe" >> CZ && \
rm -rf D && \
mkdir -p D && \
echo "A is for Duplicitous Albatros" >D/DA && \
rm -rf F && \
rm G && \
echo "G is for Geccos" >G && \
rmdir H \
echo "H is for Human" >H && \
find . >../state.layer-2

View File

@@ -0,0 +1,87 @@
clear
baseDir=$(pwd)
rm -rf /tmp/exploringImags
mkdir -p /tmp/exploringImags
cd /tmp/exploringImags
echo "== LAYER 0 =="
echo "A is for Aardvark" >A
echo "B is for Beetle" >B
mkdir C/
echo "A is for Cowboy Allan" >C/CA
mkdir -p C/CB
echo "A is for Cowboy Buffalo Alex" >C/CB/CBA
echo "B is for Cowboy Buffalo Bill" >C/CB/CBB
echo "Z is for Cowboy Zeke" >C/CZ
mkdir D/
echo "A is for Detective Alisha" >D/DA
echo "B is for Detective Betty" >D/DB
echo "E is for Elephant" >E
find . >../state.layer-0
tree | grep -v directories | tee ../tree.layer-0
$baseDir/verifyImageFiles.sh 0 $(pwd)
echo "== LAYER 1 == Change File B, Create File C/CC, Add Dir C/CD, Remove File E, Create Dir F, Add File G, Create Empty Dir H"
echo "B is for Butterfly" >B
echo "C is for Cowboy Chuck">C/CC
mkdir -p C/CD
echo "A is for Cowboy Dandy Austin" >C/CD/CDA
rm E
mkdir F
echo "A is for Ferret Albert" >F/FA
echo "G is for Gorilla" >G
mkdir H
find . >../state.layer-1
tree | grep -v directories | tee ../tree.layer-1
$baseDir/verifyImageFiles.sh 1 $(pwd)
echo "== LAYER 2 == Remove File C/CA, Remove Dir G, Remove Dir D Replace with new Dir D, Remove Dir C/CB, Remove Dir C/CB, Add File H/HA, Add File, Create Dir I"
rm C/CA
rm -rf C/CB
echo "Z is for Cowboy Zoe" >C/CZ
rm -rf D
mkdir -p D
echo "A is for Duplicitous Albatros" >D/DA
rm -rf F
rm -rf G
echo "G is for Geccos" >G
rmdir H
echo "H is for Human" >H
find . >../state.layer-2
tree | grep -v directories | tee ../tree.layer-2
$baseDir/verifyImageFiles.sh 2 $(pwd)

View File

@@ -0,0 +1,88 @@
fileContentsCompare() {
layer=$1
text=$2
file=$(pwd)/$3
if [ -f "$file" ]; then
fileContents=$(cat $file)
if [ "$fileContents" != "$text" ]; then
echo In Layer $layer Unexpected contents in file: $file
echo -- Contents: $fileContents
echo -- Expected: $text
fi
else
echo Missing File $file in Layer $layer
fi
}
checkLayer() {
layer=$1
find . >/tmp/state
if [[ $(diff /tmp/state $targetDir/../state.layer-$layer) ]]; then
echo Directory Structure mismatch in layer: $layer
diff /tmp/state $targetDir/../state.layer-$layer
fi
case $layer in
0)
fileContentsCompare $layer "A is for Aardvark" A
fileContentsCompare $layer "B is for Beetle" B
fileContentsCompare $layer "A is for Cowboy Allan" C/CA
fileContentsCompare $layer "A is for Cowboy Buffalo Alex" C/CB/CBA
fileContentsCompare $layer "B is for Cowboy Buffalo Bill" C/CB/CBB
fileContentsCompare $layer "Z is for Cowboy Zeke" C/CZ
fileContentsCompare $layer "A is for Detective Alisha" D/DA
fileContentsCompare $layer "B is for Detective Betty" D/DB
fileContentsCompare $layer "E is for Elephant" E
;;
# echo "== LAYER 1 == Change File B, Create File C/CC, Add Dir C/CD, Remove File E, Create Dir F, Add File G, Create Empty Dir H"
1)
fileContentsCompare $layer "A is for Aardvark" A
fileContentsCompare $layer "B is for Butterfly" B ## CHANGED FILE B
fileContentsCompare $layer "A is for Cowboy Allan" C/CA
fileContentsCompare $layer "A is for Cowboy Buffalo Alex" C/CB/CBA
fileContentsCompare $layer "B is for Cowboy Buffalo Bill" C/CB/CBB
fileContentsCompare $layer "C is for Cowboy Chuck" C/CC ## ADDED FILE C/CC
fileContentsCompare $layer "A is for Cowboy Dandy Austin" C/CD/CDA ## ADDED DIR C/CD, ADDED FILE C/CD/CDA
fileContentsCompare $layer "Z is for Cowboy Zeke" C/CZ
fileContentsCompare $layer "A is for Detective Alisha" D/DA
fileContentsCompare $layer "B is for Detective Betty" D/DB
## REMOVED FILE E
fileContentsCompare $layer "A is for Ferret Albert" F/FA ## ADDED DIR F, ADDED FILE F/A
fileContentsCompare $layer "G is for Gorilla" G ## ADDED G
## CREATED EMPTY DIR H
;;
# echo "== LAYER 2 == Remove File C/CA, Remove Dir C/CB, Remove Dir C/CB, Remove Dir D Replace with new Dir D, Delete and Recreatee File G, Add File H/HA Create Dir I"
2)
fileContentsCompare $layer "A is for Aardvark" A
fileContentsCompare $layer "B is for Butterfly" B
## REMOVED FILE C/CA
## REMOVED DIR C/CB
fileContentsCompare $layer "C is for Cowboy Chuck" C/CC
fileContentsCompare $layer "A is for Cowboy Dandy Austin" C/CD/CDA
fileContentsCompare $layer "Z is for Cowboy Zoe" C/CZ ## CHANGED FILE C/CZ
## REMOVE DIR D
fileContentsCompare $layer "A is for Duplicitous Albatros" D/DA ## RECREATE DIR D, ADD FILE D/DA
fileContentsCompare $layer "G is for Geccos" G ## DELETED FILE G, ADDED FILE G (Implicit CHANGED)
fileContentsCompare $layer "H is for Human" H ## ADDED FILE H
;;
esac
}
layer=$1
targetDir=$2
echo VERIFYING LAYER $layer
checkLayer $layer

Binary file not shown.

After

Width:  |  Height:  |  Size: 219 KiB

View File

@@ -0,0 +1,120 @@
# Container Based Software Deployment
---
class: pic
![dummmy](containers/software-deployment/slide-1.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-2.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-3.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-4.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-5.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-6.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-7.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-8.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-9.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-10.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-11.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-12.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-13.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-14.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-15.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-16.jpg)
---
class: pic
![dummmy](containers/software-deployment/slide-17.jpg)

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 129 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 123 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 154 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 110 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 138 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 152 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 62 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 109 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 139 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 148 KiB