github/container.training
1
0
Fork 0
mirror of https://github.com/jpetazzo/container.training.git synced 2026-02-14 17:49:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

more trainer info

Browse Source
This commit is contained in:
Bret Fisher
2016-09-28 01:26:56 -04:00
parent dfd2bf4aeb
commit 75caa65973
2 changed files with 183 additions and 85 deletions
Download Patch File Download Diff File Expand all files Collapse all files

128
README.md
View File

@@ -14,25 +14,40 @@ at multiple conferences and events like:
- Zenika, Paris (2016, February)
- Container Solutions, Amsterdam (2016, February)
## Content
- Chapter 1: Getting Started: running apps with docker-compose
- Chapter 2: Scaling out with Swarm Mode
- Chapter 3: Operating the Swarm (networks, updates, logging)
- Chapter 4: Deeper in Swarm (stateful services, scripting, DAB's)
## How This Repo is Organized
- **dockercoins**
- compose files and source code for the dockercoins sample apps used throughout the workshop
- Sample App: compose files and source code for the dockercoins sample apps
used throughout the workshop
- **docs**
- presentation slide deck, using https://remarkjs.com
- Slide Deck: presentation slide deck, works out-of-box with GitHub Pages,
uses https://remarkjs.com
- **prepare-local**
- untested scripts for automating the creation of local virtualbox VM's
(could use your help validating)
- **prepare-vms**
- scripts for automating the creation of AWS instances for students
## Slide Deck
- The slides are in the `docs` directory.
- To view them locally open `docs/index.html` in your browser. It works offline too.
- To view them online open https://jpetazzo.github.io/orchestration-workshop/ in your browser.
- They use https://remarkjs.com to allow simple markdown in a html file that remark will transform into a presentation in the browser.
- To view them locally open `docs/index.html` in your browser. It works
offline too.
- To view them online open https://jpetazzo.github.io/orchestration-workshop/
in your browser.
- When you fork this repo, be sure GitHub Pages is enabled in repo Settings
for "master branch /docs folder" and you'll have your own website for them.
- They use https://remarkjs.com to allow simple markdown in a html file that
remark will transform into a presentation in the browser.
## Sample App: Dockercoins!
The sample app is in the `dockercoins` directory. It's used during all chapters for explaining different concepts of orchestration.
The sample app is in the `dockercoins` directory. It's used during all chapters
for explaining different concepts of orchestration.
To see it in action:
@@ -40,56 +55,93 @@ To see it in action:
- this will build and start all the services
- the web UI will be available on port 8000
## Running the Workshop
General timeline of planning a workshop
- Fork repo and run through slides, doing the hands-on to be sure you understand the different `dockercoins` repo's and the steps we go through to get to a full Swarm Mode cluster of many containers. You'll update the first few slides and last slide at a minimum.
- Your docs directory can use GitHub Pages
- This workshop expects 5 servers per student. You can get away with as little as 2 servers per student, but you'll need to change the slide deck to accommodate. More is more fun.
- If you have more then ~30 students, try to get an assistant (TA) to help people with issues, so you don't have to stop the workshop to help someone with ssh etc.
- AWS is our most tested process for generating student machines. In `prepare-vms` you'll find scripts to create EC2 instances, install docker, pre-pull images, and even print "cards" to place at each students seat with IP's and username/password.
- Be sure to test creating *all* your needed servers a week before workshop. You'll likely hit AWS limits in the region closest to your class, and it sometimes takes days to get AWS to raise those limits with a support ticket.
- Create a https://gitter.im chat room for your workshop and update slides with url. Also useful for TA to monitor this during workshop. You can use before/after to answer questions.
- If you can send an email to students ahead of time, mention how they should get ssh, and test it. If they can `ssh github.com` and get `permission denied (publickey)` then you know it worked.
- Typically we create the servers the day before or morning of workshop, and leave them up the rest of day after workshop. If creating hundreds of servers, you'll likely want to run all these `trainer` commands from a dedicated instance you have in same region as instances you want to create. Much faster this way. Also, create 2 sets of servers for yourself.
- Remember you'll need to print the "cards" for students, so you'll need to create instances while you have a way to print them.
### Final touches
- Set two groups of machines for instructor's use.
- You will use the first group during the workshop.
- Log into the first machine of the second group.
- Put up the web server as instructed above.
- Use cli53 to add an A record for e.g. `view.dckr.info`.
### General timeline of planning a workshop
- Fork repo and run through slides, doing the hands-on to be sure you
understand the different `dockercoins` repo's and the steps we go through to
get to a full Swarm Mode cluster of many containers. You'll update the first
few slides and last slide at a minimum, with your info.
- Your docs directory can use GitHub Pages.
- This workshop expects 5 servers per student. You can get away with as little
as 2 servers per student, but you'll need to change the slide deck to
accommodate. More servers = more fun.
- If you have more then ~20 students, try to get an assistant (TA) to help
people with issues, so you don't have to stop the workshop to help someone
with ssh etc.
- AWS is our most tested process for generating student machines. In
`prepare-vms` you'll find scripts to create EC2 instances, install docker,
pre-pull images, and even print "cards" to place at each students seat with
IP's and username/password.
- Test AWS Scripts: Be sure to test creating *all* your needed servers a week
before workshop (just for a few minutes). You'll likely hit AWS limits in the
region closest to your class, and it sometimes takes days to get AWS to raise
those limits with a support ticket.
- Create a https://gitter.im chat room for your workshop and update slides
with url. Also useful for TA to monitor this during workshop. You can use it
before/after to answer questions, and generally works as a better answer then
"email me that question".
- If you can send an email to students ahead of time, mention how they should
get SSH, and test that SSH works. If they can `ssh github.com` and get
`permission denied (publickey)` then they know it worked, and SSH is properly
installed and they don't have anything blocking it. SSH and a browser are all
they need for class.
- Typically you create the servers the day before or morning of workshop, and
leave them up the rest of day after workshop. If creating hundreds of servers,
you'll likely want to run all these `trainer` commands from a dedicated
instance you have in same region as instances you want to create. Much faster
this way if you're on poor internet. Also, create 2 sets of servers for
yourself, and use one during workshop and the 2nd is a backup.
- Remember you'll need to print the "cards" for students, so you'll need to
create instances while you have a way to print them.
### Things That Could Go Wrong
- Creating AWS instances ahead of time, and you hit its limits in region and didn't plan enough time to wait on support to increase your limits. :(
- Students have technical issues during workshop. Can't get ssh working, locked down computer, host firewall, etc.
- Horrible wifi, or ssh port TCP/22 not open on network! If wifi sucks you can try using MOSH https://mosh.org. TMUX can prevent you from loosing your place if you get disconnected from servers. https://tmux.github.io
- Creating AWS instances ahead of time, and you hit its limits in region and
didn't plan enough time to wait on support to increase your limits. :(
- Students have technical issues during workshop. Can't get ssh working,
locked-down computer, host firewall, etc.
- Horrible wifi, or ssh port TCP/22 not open on network! If wifi sucks you
can try using MOSH https://mosh.org which handles SSH over UDP. TMUX can also
prevent you from loosing your place if you get disconnected from servers.
https://tmux.github.io
- Forget to print "cards" and cut them up for handing out IP's.
- Forget to have fun and focus on your students!
### Creating the VMs
`prepare-vms/trainer` is the script that gets you most of what you need for setting up instances. See `prepare-vms/README.md` for all the info on tools and scripts.
`prepare-vms/trainer` is the script that gets you most of what you need for
setting up instances. See
[prepare-vms/README.md](/jpetazzo/orchestration-workshop/tree/master/prepare-vms)
for all the info on tools and scripts.
### Content for Different Workshop Durations
With all the slides, this workshop is a full day long. If you need to deliver it in shorter timelines, here's some recommendations on what to cut out. You can replace `---` with `???` which will hide slides. Or leave them there and add something like `(EXTRA CREDIT)` to title so they can still view the content but you know to skip during presentation.
With all the slides, this workshop is a full day long. If you need to deliver
it in shorter timelines, here's some recommendations on what to cut out. You
can replace `---` with `???` which will hide slides. Or leave them there and
add something like `(EXTRA CREDIT)` to title so students can still view the
content but you also know to skip during presentation.
#### 3 Hour Version
- Limit time on debug tools, maybe skip a few. *"Chapter 1: Identifying bottlenecks"*
- Limit time on Compose, try to have them building the Swarm Mode by 30 minutes in.
- Limit time on debug tools, maybe skip a few. *"Chapter 1:
Identifying bottlenecks"*
- Limit time on Compose, try to have them building the Swarm Mode by 30
minutes in
- Skip most of Chapter 3, Centralized Logging and ELK
- Skip most of Chapter 4, but keep stateful services and DAB's if possible
- Mention what DAB's are, but make this part option in case you run out of time.
- Mention what DAB's are, but make this part optional in case you run out
of time
#### 2 Hour Version
- Skip the story arc of debugging dockercoins all together, skipping the troubleshooting tools. Just focus on getting it from single-host to multi-host and multi-container.
- Goal is first 30min on Compose and what dockercoins is, and getting it up on one node in docker-compose.
- Next 60-75 minutes is getting dockercoins in Swarm Mode services across servers.
- Skip all the above, and:
- Skip the story arc of debugging dockercoins all together, skipping the
troubleshooting tools. Just focus on getting them from single-host to
multi-host and multi-container.
- Goal is first 30min on intro and Docker Compose and what dockercoins is,
and getting it up on one node in docker-compose.
- Next 60-75 minutes is getting dockercoins in Swarm Mode services across
servers. Big Win.
- Last 15-30 minutes is for stateful services, DAB files, and questions.

140
prepare-vms/README.md
View File

@@ -1,67 +1,113 @@
# Trainer tools to prepare VMs for Docker workshops
# Trainer tools to create and prepare VMs for Docker workshops on AWS
## 1. Prerequisites
## Prerequisites
* [Docker](https://docs.docker.com/engine/installation/)
* [Docker Compose](https://docs.docker.com/compose/install/)
- [Docker](https://docs.docker.com/engine/installation/)
- [Docker Compose](https://docs.docker.com/compose/install/)
## 2. Clone the repo
## General Workflow
- fork/clone repo
- set required environment variables for AWS
- create your own setting file from `settings/example.yaml`
- run `./trainer` commands to create instances, install docker, setup each users environment in node1, other management tasks
- run `./trainer cards` command to generate PDF for printing handouts of each users host IP's and login info
## Clone/Fork the Repo, and Build the Tools Image
The Docker Compose file here is used to build a image with all the dependencies to run the `./trainer` commands and optional tools. Each run of the script will check if you have those dependencies locally on your host, and will only use the container if you're [missing a dependency](jpetazzo/orchestration-workshop/blob/master/prepare-vms/trainer#L5).
$ git clone https://github.com/jpetazzo/orchestration-workshop.git
$ cd orchestration-workshop/prepare-vms
$ docker-compose build
$ ./trainer <commands> # See "Summary of commands" section below
## 3. Preparing the environment
## Preparing to Run `./trainer`
Required environment variables:
### Required AWS Permissions/Info
* `AWS_ACCESS_KEY_ID`
* `AWS_SECRET_ACCESS_KEY`
* `AWS_DEFAULT_REGION`
- Initial assumptions are you're using a root account. If you'd like to use a IAM user, it will need `AmazonEC2FullAccess` and `IAMReadOnlyAccess`.
- Using a non-default VPC or Security Group isn't supported out of box yet, but until then you can [customize the `trainer-cli` script](jpetazzo/orchestration-workshop/blob/master/prepare-vms/scripts/trainer-cli#L396-L401).
- These instances will assign the default VPC Security Group, which does not open any ports from Internet by default. So you'll need to add Inbound rules for `SSH | TCP | 22 | 0.0.0.0/0` and `Custom TCP Rule | TCP | 8000 - 8002 | 0.0.0.0/0`, or run `./trainer opensg` which opens up all ports.
### 4. Update settings.yaml
### Required Environment Variables
Then pass `settings/YOUR_WORKSHOP_NAME-settings.yaml` as an argument to `deploy`, `cards`, etc.
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_DEFAULT_REGION`
## Usage
### Update/copy `settings/example.yaml`
### Summary of commands
Then pass `settings/YOUR_WORKSHOP_NAME-settings.yaml` as an argument to `trainer deploy`, `trainer cards`, etc.
The `trainer` script can be executed directly.
./trainer cards 2016-09-28-00-33-bret settings/orchestration.yaml
Summary of steps to launch a batch of instances for a workshop:
## `./trainer` Usage
* Export the environment variables needed by the AWS CLI (see **2. Preparing the environment** above)
* `./trainer start N` (where `N` is the number of AWS instances to create)
* `./trainer list` to view the list of tags
* `./trainer list TAG` to view the instances with a given `TAG`
* `./trainer deploy TAG settings/somefile.yaml` to run `scripts/postprep.rc` via parallel-ssh
* `./trainer pull-images TAG` to pre-pull a bunch of Docker images to the instances
* `./trainer test TAG`
* `./trainer cards TAG settings/somefile.yaml` to generate a PDF and an HTML file you can print and cut to hand out cards with connection information to attendees
```
./trainer <command> [n-instances|tag] [settings/file.yaml]
`./trainer` will run locally if all its dependencies are fulfilled; otherwise it will run in a Docker container.
Core commands:
start n Start n instances
list [TAG] If a tag is provided, list its VMs. Otherwise, list tags.
deploy TAG Deploy all instances with a given tag
pull-images TAG Pre-pull docker images. Run only after deploying.
stop TAG Stop and delete instances tagged TAG
It will check for the necessary environment variables. Then, if all its dependencies are installed
locally, it will execute `trainer-cli`. If not, it will look for a local Docker image
tagged `preparevms_prepare-vms` (created automatically when you run `docker-compose build`).
If found, it will run in a container. If not found, the user will be prompted to
either install the missing dependencies or run `docker-compose build`.
Extras:
ips TAG List all IPs of instances with a given tag (updates ips.txt)
ids TAG/TOKEN List all instance IDs with a given tag
shell Get a shell in the trainer container
status TAG Print information about this tag and its VMs
tags List all tags (per-region)
retag TAG/TOKEN TAG Retag instances with a new tag
## Detailed usage
Beta:
ami Look up Amazon Machine Images
cards FILE Generate cards
opensg Modify AWS security groups
```
### Start some VMs
### Summary of What `./trainer` Does For You
$ ./trainer start 10
- Used to manage bulk AWS instances for you without needing to use AWS cli or gui.
- Can manage multiple "tags" or groups of instances, which are tracked in `prepare-vms/tags/`
- Can also create PDF/HTML for printing student info for instance IP's and login.
- The `./trainer` script can be executed directly.
- It will run locally if all its dependencies are fulfilled; otherwise it will run in the Docker container you created with `docker-compose build` (preparevms_prepare-vms).
- During `start` it will add your default local SSH key to all instances under the `ubuntu` user.
- During `deploy` it will create the `docker` user with password `training`, which is printing on the cards for students. For now, this is hard coded.
A few things will happen:
### Example Steps to Launch a Batch of Instances for a Workshop
* Your local SSH key will be synced
* AWS instances will be created and tagged
* A directory will be created
- Export the environment variables needed by the AWS CLI (see **Required Environment Variables** above)
- Run `./trainer start N` Creates `N` EC2 instances
- Your local SSH key will be synced to instances under `ubuntu` user
- AWS instances will be created and tagged based on date, and IP's stored in `prepare-vms/tags/`
- Run `./trainer deploy TAG settings/somefile.yaml` to run `scripts/postprep.rc` via parallel-ssh
- If it errors or times out, you should be able to rerun
- Requires good connection to run all the parallel SSH connections, up to 100 parallel (ProTip: create dedicated management instance in same AWS region where you run all these utils from)
- Run `./trainer pull-images TAG` to pre-pull a bunch of Docker images to the instances
- Run `./trainer cards TAG settings/somefile.yaml` generates PDF/HTML files to print and cut and hand out to students
- *Have a great workshop*
- Run `./trainer stop TAG` to terminate instances.
Details below.
## Other Tools
### Deploying your SSH key to all the machines
- Make sure that you have SSH keys loaded (`ssh-add -l`).
- Source `rc`.
- Run `pcopykey`.
### Installing extra packages
- Source `postprep.rc`.
(This will install a few extra packages, add entries to
/etc/hosts, generate SSH keys, and deploy them on all hosts.)
## Even More Details
#### Sync of SSH keys
@@ -83,7 +129,7 @@ This ips.txt file will be created in the $TAG/ directory and a symlink will be p
If you create new VMs, the symlinked file will be overwritten.
## Deployment
#### Deployment
Instances can be deployed manually using the `deploy` command:
@@ -91,29 +137,29 @@ Instances can be deployed manually using the `deploy` command:
The `postprep.rc` file will be copied via parallel-ssh to all of the VMs and executed.
### Pre-pull images
#### Pre-pull images
$ ./trainer pull-images TAG
### Generate cards
#### Generate cards
$ ./trainer cards TAG settings/somefile.yaml
### List tags
#### List tags
$ ./trainer list
### List VMs
#### List VMs
$ ./trainer list TAG
This will print a human-friendly list containing some information about each instance.
### Stop and destroy VMs
#### Stop and destroy VMs
$ ./trainer stop TAG
## ToDo
* Don't write to bash history in system() in postprep
* compose, etc version inconsistent (int vs str)
- Don't write to bash history in system() in postprep
- compose, etc version inconsistent (int vs str)