DOCKER

FUNDAMENTALS

17.06-v1.3 © 2017 Docker, Inc.


 A NOTE ON PEDAGOGY
Docker believes in learning by doing, with support.
The course is lab driven with lecture.
Work together
Ask questions at any time

17.06-v1.3 © 2017 Docker, Inc.


 SESSION LOGISTICS
2 days duration
mostly exercises
regular breaks

17.06-v1.3 © 2017 Docker, Inc.


 ASSUMED KNOWLEDGE AND REQUIREMENTS
Familiarity with using the Linux command line
Linux Cheat Sheet: http://bit.ly/2mTQr8l

17.06-v1.3 © 2017 Docker, Inc.


 YOUR LAB ENVIRONMENT
You have been given several instances for use in exercises.
Ask instructor for username and password if you don't have them already.

17.06-v1.3 © 2017 Docker, Inc.


 AGENDA
Fundamental Containerization

Fundamental Orchestration

...plus other odds and ends.

17.06-v1.3 © 2017 Docker, Inc.


 INTRODUCING DOCKER

17.06-v1.3 © 2017 Docker, Inc.


 FRICTION IN THE SOFTWARE SUPPLY CHAIN

17.06-v1.3 © 2017 Docker, Inc.


 Devops circa 1912

17.06-v1.3 © 2017 Docker, Inc.


 Photo Roel Hemkes, CC-BY 2.0 Photo Jay Phagan, CC-BY 2.0 Photo Roy Luck, CC-BY 2.0

Encapsulation eliminates friction across infrastructure, and standardization

facilitates scale.

17.06-v1.3 © 2017 Docker, Inc.


 Dependency conflicts, infrastructure mismatches, and lack of scalability are all
examples of problems with

STANDARDIZATION AND ENCAPSULATION.

17.06-v1.3 © 2017 Docker, Inc.


 DEPLOYMENT NIGHTMARE

17.06-v1.3 © 2017 Docker, Inc.


 ANY APP, ANYWHERE

17.06-v1.3 © 2017 Docker, Inc.


 SECURITY
“Gartner asserts that applications deployed in containers are more secure than
applications deployed on the bare OS.”
http://blogs.gartner.com/joerg-fritsch/can-you-operationalize-docker-containers/

17.06-v1.3 © 2017 Docker, Inc.


 SECURITY BREACHES

17.06-v1.3 © 2017 Docker, Inc.


 WHAT DOES THIS HAVE TO DO WITH DOCKER...?

17.06-v1.3 © 2017 Docker, Inc.


 WHAT DOES THIS HAVE TO DO WITH DOCKER...?

17.06-v1.3 © 2017 Docker, Inc.


 WHAT DOES THIS HAVE TO DO WITH DOCKER...?

17.06-v1.3 © 2017 Docker, Inc.


 SAFER APPLICATIONS

17.06-v1.3 © 2017 Docker, Inc.


 ENCAPSULATION I: PHYSICAL SERVERS
One application, one physical server

17.06-v1.3 © 2017 Docker, Inc.


 ENCAPSULATION I: PHYSICAL SERVERS
LIMITS OF PHYSICAL
ENCAPSULATION
Slow deployment
Huge costs
Provisioning speed limited by physical
logistics
Difficult to scale
Difficult to migrate
Vendor lock-in

17.06-v1.3 © 2017 Docker, Inc.


 ENCAPSULATION II: VIRTUAL MACHINES
Multiple apps on one server
Elastic, real time provisioning
Scalable pay-per-use cloud models
viable

17.06-v1.3 © 2017 Docker, Inc.


 ENCAPSULATION II: VIRTUAL MACHINES
VIRTUAL MACHINE
LIMITATIONS
Virtual Machines require CPU & memory
allocation
Significant overhead from guest OS

17.06-v1.3 © 2017 Docker, Inc.


 ENCAPSULATION III: CONTAINERS
Containers leverage kernel features to
create extremely light-weight
encapsulation:
Kernel namespaces
Network namespaces
Linux containers
cgroups & security tools
Results in faster spool-up and denser
servers

17.06-v1.3 © 2017 Docker, Inc.


 The most basic thing Docker provides is a

FRAMEWORK FOR SERVICE ENCAPSULATION

But what are the implications of this for developers, ops, and orgs?

17.06-v1.3 © 2017 Docker, Inc.


 DISTRIBUTED APPLICATION ARCHITECTURE
Encapsulation supercharges:

Monolith Densification
Service-Based Architecture
Devops

17.06-v1.3 © 2017 Docker, Inc.


 MODERNIZE TRADITIONAL APPS

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER PRODUCT OFFERINGS

17.06-v1.3 © 2017 Docker, Inc.


 COMPONENTS OF CE/EE

17.06-v1.3 © 2017 Docker, Inc.


 EE SUBSCRIPTION TIERS
Subscription Tier
Basic Standard Advanced
Container engine and built-in X X X
orchestration
Image management (private registry, X X
caching)
Integrated container app management X X
Multitennancy, RBAC, LDAP/AD X X
Integrated secrets management, image X X
signing, policy
Security Scanning X
 Node RBAC 17.06-v1.3 © 2017 Docker, Inc.
X
 CE/EE RELEASE CADENCE

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER CERTIFIED PRODUCTS
Infrastructure

Plugins

Certified Plugins & Containers available

Containers
at Docker Store, store.docker.com

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER STORE, CLOUD & HUB
Docker Store

Docker Hub

Docker Cloud

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER TRAINING
Enablement for all these products
Deep dives on special topics
Docker Distributed Application Engineer & Ops Certifications (coming soon!)

e: training@docker.com

w: training.docker.com

17.06-v1.3 © 2017 Docker, Inc.


 PART 1: CONTAINERS

17.06-v1.3 © 2017 Docker, Inc.


 CONTAINERIZATION FUNDAMENTALS

17.06-v1.3 © 2017 Docker, Inc.


 TOPICS
Containers under the hood
Starting, stopping and deleting containers
Inspecting containers
Executing processes inside running containers

17.06-v1.3 © 2017 Docker, Inc.


 CONTAINERS ARE PROCESSES
Containers are processes sandboxed by:

Kernel namespaces
Root privilege management
System call restrictions
Private network stacks
etc

17.06-v1.3 © 2017 Docker, Inc.


 KERNEL NAMESPACES
Kernel Namespaces virtualize system
resources for groups of processes.
clone() flags:

17.06-v1.3 © 2017 Docker, Inc.


 ROOT & SYSCALL PERMISSIONS

17.06-v1.3 © 2017 Docker, Inc.


 CONTAINER LOGS
STDOUT and STDERR for a containerized process
docker container logs <container name>

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: CONTAINER BASICS
Work through:

Running and Inspecting a Container

Interactive Containers
Detached Containers and Logging
Starting, Stopping, Inspecting and Deleting Containers

In the Docker Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 CONTAINER BASICS TAKEAWAYS
Single process with PID 1
Private & ephemeral filesystem and data

New Syntax:

docker container run

docker container rm
docker container start
docker container stop & docker container kill
docker container exec
docker container attach
docker container logs
docker container inspect
docker container ls 17.06-v1.3 © 2017 Docker, Inc.

 FURTHER READING
List of container commands: http://dockr.ly/2iLBV2I
Getting started with containers: http://dockr.ly/2gmxKWB
Start containers automatically: http://dockr.ly/2xB8sMl
Limit a container's resources: http://dockr.ly/2wqN5Nn
Keep containers alive during daemon downtime: http://dockr.ly/2emLwb5
Isolate containers with a user namespace: http://dockr.ly/2gmyKdf

17.06-v1.3 © 2017 Docker, Inc.


 CREATING IMAGES

17.06-v1.3 © 2017 Docker, Inc.


 TOPICS
Layered filesystems
Creating images
Dockerfiles & best practice
Tagging, Namespacing & Sharing images

17.06-v1.3 © 2017 Docker, Inc.


 WHAT ARE IMAGES?
A filesystem for container process
Made of a stack of immutable layers
Start with a base image
New layer for each change

17.06-v1.3 © 2017 Docker, Inc.


 SHARING LAYERS

17.06-v1.3 © 2017 Docker, Inc.


 THE WRITABLE CONTAINER LAYER

17.06-v1.3 © 2017 Docker, Inc.


 COPY ON WRITE

17.06-v1.3 © 2017 Docker, Inc.


 CREATING IMAGES
Three methods:

Commit the R/W container layer as a new R/O image layer.

Define new layers to add to a starting image in a Dockerfile.
Import a tarball into Docker as a standalone base layer.

17.06-v1.3 © 2017 Docker, Inc.


 COMMITTING CONTAINER CHANGES
docker container commit
saves container layer as new R/O image layer
Pro: build images interactively
Con: hard to reproduce or audit

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: INTERACTIVE IMAGE CREATION
Work through the 'Interactive Image Creation' exercise in the Docker
Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILES
Content manifest
Provides image layer documentation
Enables automation (CI/CD)

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILES
FROM command defines base image.
Each subsequent command adds a layer
docker image build ... builds image from Dockerfile
# Comments begin with the pound sign
FROM ubuntu:16.04
RUN apt-get update
ADD /data /myapp/data
...

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: DOCKERFILES (1/2)
Work through the 'Creating Images with Dockerfiles (1/2)' exercise in the Docker
Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 BUILD OUTPUT
$ docker image build -t demo .
Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM ubuntu:16.04
---> 7b9b13f7b9c0
Step 2/3 : RUN apt-get update
---> Running in 2e8333703768
...
Reading package lists...
---> 0fac7902d4d4
Removing intermediate container 2e8333703768
Step 3/3 : RUN apt-get install -y iputils-ping
---> Running in 3277108034a6
Reading package lists...
---> 9469b454b516
Removing intermediate container 3277108034a6
Successfully built 9469b454b516
Successfully tagged demo:latest

17.06-v1.3 © 2017 Docker, Inc.


 BUILD CONTEXT
$ docker image build -t demo .
Sending build context to Docker daemon 2.048kB

Directory archive
Must contain all local files necessary for image
Will omit anything listed in .dockerignore

17.06-v1.3 © 2017 Docker, Inc.


 EXAMINING THE BUILD PROCESS
For each command:
Launch a new container based on the image thus far
Execute command in that container
Step 2/3 : RUN apt-get update
---> Running in 2e8333703768

Commit R/W layer to image

---> 0fac7902d4d4
Delete intermediate container Removing intermediate container 2e8333703768

17.06-v1.3 © 2017 Docker, Inc.


 EXAMINING THE BUILD PROCESS
This:

RUN cd /src
RUN bash setup.sh

is different than this:

RUN cd /src && bash setup.sh

because every Dockerfile command runs in a different container, and only the
filesystem, not the in-memory state, is persisted from layer to layer.

17.06-v1.3 © 2017 Docker, Inc.


 BUILD CACHE
After completion, the resulting image layer is labeled
with a hash of the content of all current image layers
in the stack.

17.06-v1.3 © 2017 Docker, Inc.


 CMD AND ENTRYPOINT
Recall all containers run a process as their PID 1
CMD and ENTRYPOINT allow us to specify default processes.

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: DOCKERFILES (2/2)
Work through the 'Creating Images with Dockerfiles (2/2)' exercise in the Docker
Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 CMD AND ENTRYPOINT
CMD alone: default command and list of parameters.
CMD + ENTRYPOINT: ENTRYPOINT provides command, CMD provides
default parameters.
CMD overridden by command arguments to docker container run
ENTRYPOINT overridden via
--entrypoint flag to docker container run.

17.06-v1.3 © 2017 Docker, Inc.


 SHELL VS. EXEC FORMAT
# Shell form
ENTRYPOINT sudo -u ${USER} java ...

# Exec form
ENTRYPOINT ["sudo", "-u", "jdoe", "java", ...]

17.06-v1.3 © 2017 Docker, Inc.


 COPY AND ADD COMMANDS
COPY copies files from build context to image:

COPY <src> <dest>

ADD can also untar and fetch URLs.

In both cases
create checksum for files added
log checksum in build cache
Cache invalidated if checksum changed

17.06-v1.3 © 2017 Docker, Inc.


 ENV AND ARG COMMANDS
ENV sets environment variables inside container:
ENV APP_PORT 8080

ARG defines arguments that can be passed in from the command line during
build:
ARG DB_MOUNT=/my/default/path
ENV DB_PATH $DB_MOUNT

docker image build --build-arg DB_MOUNT=/my/custom/path

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILE COMMAND ROUNDUP
FROM: base image to start from (usually OS)
RUN: run a command in the environment defined so far
CMD and ENTRYPOINT: define default behavior
COPY and ADD: copy files into container
ENV: define environment variables inside container
ARG: define build time argument (set with --build-arg)

Many more Dockerfile commands are available; see the docs at

https://docs.docker.com/engine/reference/builder/

17.06-v1.3 © 2017 Docker, Inc.


 MULTI-STAGE BUILDS
Hello World, in C:

FROM alpine:3.5
RUN apk update && \
apk add --update alpine-sdk
RUN mkdir /app
WORKDIR /app
ADD hello.c /app
RUN mkdir bin
RUN gcc -Wall hello.c -o bin/hello
CMD /app/bin/hello

builds to:

Bills-MBP:demo billmills$ docker image ls hwc

REPOSITORY TAG IMAGE ID CREATED SIZE
hwc latest 142c29686b6a 15 hours ago 184 MB

17.06-v1.3 © 2017 Docker, Inc.


 MULTI-STAGE BUILDS
Hello World, lightweight:

# Full SDK version (built and discarded)

FROM alpine:3.5 AS build
RUN apk update && \
apk add --update alpine-sdk
RUN mkdir /app
WORKDIR /app
ADD hello.c /app
RUN mkdir bin
RUN gcc -Wall hello.c -o bin/hello

# Lightweight image returned as final product

FROM alpine:3.5 Bills-MBP:demo billmills$ docker image ls hwc
COPY --from=build /app/bin/hello /app/hello
CMD /app/hello REPOSITORY TAG IMAGE ID CREATED SIZE
hwc latest 5d925cfc9c96 39 seconds ago 4MB

17.06-v1.3 © 2017 Docker, Inc.


 BUILD TARGETS
Dockerfile

FROM <base image> as base

...

FROM <foo image> as foo

...

FROM <bar image> as bar

...

FROM alpine:3.4
...
COPY --from foo ...
COPY --from bar ...
...

Building the image

docker image build --target <name> ...

17.06-v1.3 © 2017 Docker, Inc.

 EXERCISE: MULTI-STAGE BUILDS
Work through the 'Multi-Stage Builds' exercise in the Docker Fundamentals
Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILE BEST PRACTICES: LAYERS
Start with official images
Combine commands
Single ENTRYPOINT
Use multi-stage builds to drop compilers, SDKs...

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILE BEST PRACTICES: CACHING
The cache is busted from the point in the Dockerfile where:
Anything explicitly changes
Anything implicitly changes:

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILE BEST PRACTICES: CACHING
Bad: Good:

FROM alpine:3.5 FROM alpine:3.5

RUN apk add <experimental deps> RUN apk update && \
RUN apk update && \ apk add --update alpine-sdk
apk add --update alpine-sdk RUN apk add <experimental deps>
RUN gcc -Wall hello-dev.c -o hello RUN gcc -Wall hello-dev.c -o hello
CMD /hello CMD /hello

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILE BEST PRACTICES: CACHING
Bad: Good:

FROM python:3.5-alpine FROM python:3.5-alpine

RUN mkdir /app RUN mkdir /app
COPY /mypy /app/ COPY /mypy/reqs.txt /app/
RUN pip install -r app/reqs.txt RUN pip install -r app/reqs.txt
... COPY /mypy /app/
...

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILE BEST PRACTICES: CACHING
If the build ARG changes by use of the --build-arg flag, at what line is the
following Dockerfile's cache invalidated?

...
ARG MYDEP=default
RUN gcc -Wall hello.c -o bin/hello
RUN apt-get install $MYDEP
...

17.06-v1.3 © 2017 Docker, Inc.


 DOCKERFILE BEST PRACTICES: CACHING
Bad: Good:

... ...
ARG MYDEP=default RUN gcc -Wall hello.c -o bin/hello
RUN gcc -Wall hello.c -o bin/hello ARG MYDEP=default
RUN apt-get install $MYDEP RUN apt-get install $MYDEP
... ...

17.06-v1.3 © 2017 Docker, Inc.


 IMAGE TAGS
Optional string after image name, separated by :
:latest by default
Same image with two tags shares same ID, image layers:

$ docker image ls centos*

REPOSITORY TAG IMAGE ID CREATED SIZE
centos 7 8140d0c64310 7 days ago 193 MB
$ docker image tag centos:7 centos:mytag
$ docker image ls centos*
REPOSITORY TAG IMAGE ID CREATED SIZE
centos 7 8140d0c64310 7 days ago 193 MB
centos mytag 8140d0c64310 7 days ago 193 MB

17.06-v1.3 © 2017 Docker, Inc.


 IMAGE NAMESPACES
Images exist in one of three namespaces:

Root (ubuntu, nginx, mongo, mysql, ...)

User / Org (jdoe/myapp:1.1, tutum/mongodb:latest, ...)
Registry (FQDN/jdoe/myapp:1.1, ...)

Root and User/Org indicate images distributed on store.docker.com; Registry

namespacing indicates a Docker Trusted Registry image.

17.06-v1.3 © 2017 Docker, Inc.


 IMAGE TAGGING & NAMESPACING
Tag on build: docker image build -t myapp:1.0
Retag an existing image: docker image tag myapp:1.0
me/myapp:2.0
Note docker image tag can set both tag and namespace.

17.06-v1.3 © 2017 Docker, Inc.


 SHARING IMAGES
Log in to store.docker.com: docker login
Share an image: docker image push <image name>
Public repos available for anyone to docker pull.

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: MANAGING IMAGES
Work through the 'Managing Images' exercise in the Docker Fundamentals
Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 IMAGE CREATION TAKEAWAYS
Images are built out of read-only layers.
Dockerfiles specify image layer contents.
Key Dockerfile commands: FROM, RUN, COPY and ENTRYPOINT
Images must be namespaced according to where you intend on sharing them.

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
Best practices for writing Dockerfiles: http://dockr.ly/22WiJiO
Use multi-stage builds: http://dockr.ly/2ewcUY3
More about images, containers, and storage drivers: http://dockr.ly/1TuWndC
Graphdriver plugins: http://dockr.ly/2eIVCab
Docker Reference Architecture: An Intro to Storage Solutions for Docker CaaS:
http://dockr.ly/2x8sBw2
How to select a storage driver: http://dockr.ly/2eDu8yO
Use the AUFS storage driver: http://dockr.ly/2jVc1Zz
User guided caching in Docker: http://dockr.ly/2xKafPf

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER SYSTEM COMMANDS

17.06-v1.3 © 2017 Docker, Inc.


 CLEAN-UP COMMANDS
docker system df
TYPE TOTAL ACTIVE SIZ E RECLAIMABLE
Images 39 2 9.01 GB 7.269 GB (80%)
Containers 2 2 69.36 MB 0 B (0%)
Local Volumes 0 0 0 B 0 B

docker system prune

more limited...

docker image prune [--filter "foo=bar"]

docker container prune [--filter "foo=bar"]
docker volume prune [--filter "foo=bar"]
docker network prune [--filter "foo=bar"]
17.06-v1.3 © 2017 Docker, Inc.

 INSPECT THE SYSTEM
docker system info
Containers: 2
Running: 2
Paused: 0
Stopped: 0
Images: 105
Server Version: 17.03.0-ee
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Swarm: active
NodeID: ybmqksh6fm627armruq0e8id1
Is Manager: true
ClusterID: 2rbf1dv6t5ntro2fxbry6ikr3
Managers: 1
Nodes: 1
Orchestration:
Task History Retention Limit: 5
Raft:
Snapshot Interval: 10000
Number of Old Snapshots to Retain: 0
Heartbeat Tick: 1
17.06-v1.3 © 2017 Docker, Inc.
 ...
 SYSTEM EVENTS
Start observing with ...
docker system events
Generate events with ...
docker container run --rm alpine echo 'Hello World!'

2017-01-25T16:57:48.553596179-06:00 container create 30eb630790d44052f26c1081...

2017-01-25T16:57:48.556718161-06:00 container attach 30eb630790d44052f26c1081...
2017-01-25T16:57:48.698190608-06:00 network connect de1b2b40f522e69318847ada3...
2017-01-25T16:57:49.062631155-06:00 container start 30eb630790d44052f26c1081d...
2017-01-25T16:57:49.164526268-06:00 container die 30eb630790d44052f26c1081dbf...
2017-01-25T16:57:49.613422740-06:00 network disconnect de1b2b40f522e69318847a...
2017-01-25T16:57:49.815845051-06:00 container destroy 30eb630790d44052f26c108...

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: SYSTEM COMMANDS
Work through:

Cleaning up Docker Resources

Inspection Commands

in the Docker Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
System commands reference: http://dockr.ly/2eMR53i

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER VOLUMES

17.06-v1.3 © 2017 Docker, Inc.


 TOPICS
Creating & deleting volumes
Mounting volumes
Inspecting volumes
Sharing volumes

17.06-v1.3 © 2017 Docker, Inc.


 VOLUMES
Persist when a container is deleted
Can be shared between containers
Separate from the union file system

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER VOLUME COMMAND
docker volume sub-commands:

docker volume create --name demo

docker volume ls
docker volume inspect demo
docker volume rm demo
docker volume prune

17.06-v1.3 © 2017 Docker, Inc.


 MOUNT A VOLUME
Mounted at container startup
docker container run -v [name]:[path in container FS] …
Example:
# Execute a new container and mount the volume test1 in the folder /www/test1
docker container run -it -v test1:/www/test1 ubuntu:16.04 bash

17.06-v1.3 © 2017 Docker, Inc.


 WHERE ARE OUR VOLUMES?
docker container inspect, look for the "source" field as shown below:

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER VOLUME INSPECT COMMAND

17.06-v1.3 © 2017 Docker, Inc.


 DELETING A VOLUME
# Delete the volume called test1
> docker volume rm test1

# Delete a container and remove its associated volumes
> docker container rm -v <container ID>

Note the -v option: volumes not automatically deleted when deleting a container.

17.06-v1.3 © 2017 Docker, Inc.


 DELETING VOLUMES
Cannot delete a volume if it is being used by a container (running or stopped)
docker container rm -v <container ID> will not delete a volume
associated with the container if that volume is mounted in another container

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: CREATING AND MOUNTING
VOLUMES
Work through the 'Creating and Mounting Volumes' exercise in the Docker
Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 MOUNTING HOST DIRECTORIES
Can map directories on the host to a container path
Changes made on the host are reflected inside the container
Syntax:
docker container run -v [host path]:[container path]:
[rw|ro]
rw or ro controls the write status of the directory inside the container
# Mount the contents of the public_html directory on the
# hosts to the container volume at /data/www
> docker container run -d \
-v /home/user/public_html:/data/www ubuntu

17.06-v1.3 © 2017 Docker, Inc.


 INSPECTING THE MAPPED DIRECTORY
docker container inspect

17.06-v1.3 © 2017 Docker, Inc.


 USE CASES FOR MOUNTING HOST DIRECTORIES
Storage management
Rapid updates (ex code development)

17.06-v1.3 © 2017 Docker, Inc.


 SHARING DATA BETWEEN CONTAINERS
Volumes can be mounted into multiple containers
Allows data to be shared between containers
Example use cases

Note: Be aware of potential conflicts and security breaches!

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: RECORDING LOGS
Work through the Volumes Usecase: 'Recording Logs' exercise in the Docker
Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 VOLUMES IN DOCKERFILES
VOLUME instruction creates a mount point
Can specify arguments in a JSON array or string
Cannot map volumes to host directories
Volumes are initialized when the container is executed

# String example
VOLUME /myvol

# String example with multiple volumes

VOLUME /www/website1 /www/website2

# JSON example
VOLUME [“/myvol”, “/myvol2”]

17.06-v1.3 © 2017 Docker, Inc.


 EXAMPLE DOCKERFILE WITH VOLUMES
Volume initialized along with data on docker container run ...
FROM ubuntu:16.04

RUN apt-get update

RUN apt-get install -y vim wget

RUN mkdir /data/myvol -p && echo "hello world" > /data/myvol/testfile

VOLUME ["/data/myvol"]

17.06-v1.3 © 2017 Docker, Inc.


 INSPECTING AN IMAGE FOR VOLUMES
docker image inspect <Image ID>

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER VOLUME TAKEAWAYS
Volumes are for persistent data
Volumes bypass the copy on write system
A volume persists even after its container has been deleted

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
How to use volumes: http://dockr.ly/2vRZBDG
Troubleshoot volume errors: http://dockr.ly/2vyjvbP
Docker volume reference: http://dockr.ly/2ewrlew

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER PLUGINS

17.06-v1.3 © 2017 Docker, Inc.


 PLUGINS
Extend the Docker platform
Distributed as Docker images
Hosted on store.docker.com

List plugins on system:

$ docker plugin ls
ID NAME DESCRIPTION ENABLED
bee424413706 vieux/sshfs:latest sshFS plugin for Docker true

17.06-v1.3 © 2017 Docker, Inc.


 INSTALL A PLUGIN
$ docker plugin install vieux/sshfs
Plugin "vieux/sshfs" is requesting the following privileges:
- network: [host]
- mount: [/var/lib/docker/plugins/]
- device: [/dev/fuse]
- capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
latest: Pulling from vieux/sshfs
a23658ccfda2: Download complete
Digest: sha256:d33ffa08df2a4fa87f83fded4ac3ac2e3a96d296c55aa1a0afce33fb2cc7b9b0
Status: Downloaded newer image for vieux/sshfs:latest
Installed plugin vieux/sshfs

17.06-v1.3 © 2017 Docker, Inc.


 USING THE PLUGIN
Create a volume...

docker volume create -d vieux/sshfs \

-o sshcmd=<user@host:path> \
-o password=<password> \
sshvolume

Use the volume...

docker container run -v sshvolume:/demo -it centos:7 bash

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: PLUGINS
Work through the 'Docker Plugins' exercise in the Docker Fundamentals Exercises
book.

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
How to implement a plugin: http://dockr.ly/2vRIEcG
Create an authorization plugin: http://dockr.ly/2wuxfBB
List of Docker Engine plugins: http://dockr.ly/2vyXERe
Docker network driver plugins: http://dockr.ly/2wqiill
Volume plugins: http://dockr.ly/2vSbU2W
Plugin configuration reference: http://dockr.ly/2er4bm2
Plugin API reference: http://dockr.ly/2ewofHi

17.06-v1.3 © 2017 Docker, Inc.


 CONTAINERIZATION FUNDAMENTALS
CONCLUSION: ANY APP, ANYWHERE.
Containers are isolated processes
Images provide filesystem for containers
Volumes persist data

17.06-v1.3 © 2017 Docker, Inc.


 PART 2: ORCHESTRATION

Image CC-BY Phil Roeder

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER NETWORKING BASICS

17.06-v1.3 © 2017 Docker, Inc.


 TOPICS
Bridge networks
Overlay Networks
Network firewalling
Port management

17.06-v1.3 © 2017 Docker, Inc.


 THE CONTAINER NETWORK MODEL

17.06-v1.3 © 2017 Docker, Inc.


 SINGLE HOST NETWORKING
Sandbox: Kernel Namespace
Endpoint: Virtual ETHernet port
Network: Linux Bridge, an L2 (ie MAC address) packet
router
docker0 is the default container network.

17.06-v1.3 © 2017 Docker, Inc.


 SINGLE HOST NETWORKING

17.06-v1.3 © 2017 Docker, Inc.


 NETWORK FIREWALLS

17.06-v1.3 © 2017 Docker, Inc.


 NETWORK FIREWALLS

docker network connect my_bridge u2

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: INTRODUCTION TO CONTAINER
NETWORKING
Work through the 'Introduction to Container Networking' exercise in the Docker
Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 SECURITY WARNINGS
Do not use the host network in production.
Do not connect containers to the same network unecessarily.

17.06-v1.3 © 2017 Docker, Inc.


 EXPOSING CONTAINER PORTS
Containers have no public IP address by default; reachable only locally via their
host's linux bridge.
Can map a container port to a host port to allow container reachability.
Ports can be mapped manually or automatically.
Port mappings visible via
docker container ls or
docker container port

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: CONTAINER PORT MAPPING
Work through the 'Container Port Mapping' exercise in the Docker Fundamentals
Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER NETWORKING TAKEAWAYS
Single host: linux bridge
Multi-host: VXLAN overlay
Separate Docker networks are firewalled from each other by default
Containers are firewalled from the outside world by default, but can expose ports
on the host
Advanced networking: http://dockr.ly/2eTRNdy

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
Docker Reference Architecture: Designing Scalable, Portable Docker Container
Networks: http://dockr.ly/2tbvh5G
Network containers: http://dockr.ly/2x1BYgW
Docker container networking: http://dockr.ly/1QnT6y8
Get started with multi-host networking: http://dockr.ly/2gtqRms
Understand container communication: http://dockr.ly/2iSrHO0

17.06-v1.3 © 2017 Docker, Inc.


 INTRODUCTION TO DOCKER COMPOSE

17.06-v1.3 © 2017 Docker, Inc.


 TOPICS
Services
Defining Application with Docker Compose
Scaling Applications

17.06-v1.3 © 2017 Docker, Inc.


 DISTRIBUTED APPLICATION ARCHITECTURE
Applications consisting of one or more containers across one or more nodes
Docker Compose facilitates multi-container design on a single node.

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER SERVICES
Goal: declare and (re)configure many similar containers all at once
Goal: scale apps by adding containers seamlessly
A service defines the desired state of a group of identically configured containers.
Docker provides transparent service discovery for Services.

17.06-v1.3 © 2017 Docker, Inc.


 SERVICE DISCOVERY

Services are assigned a Virtual IP which spreads traffic out across the underlying
containers automatically. 17.06-v1.3 © 2017 Docker, Inc.

 OUR APPLICATION: DOCKERCOINS
It is a DockerCoin miner!
Dockercoins consists of 5 services
working together:

(DockerCoins 2016 logo courtesy of @XtlCnslt and @ndeloof. Thanks!)

17.06-v1.3 © 2017 Docker, Inc.


 OUR SAMPLE APPLICATION
https://github.com/docker-training/orchestration-workshop/tree/17.06
The application is in the dockercoins subdirectory
Each service has its own subdirectory & Dockerfile

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER-COMPOSE.YML

version: "3.3"

services:
rng:
image: user/dockercoins_rng:1.0
networks:
- dockercoins
ports:
- "8001:80"

hasher:
image: user/dockercoins_hasher:1.0
networks:
- dockercoins
ports:
- "8002:80"
...

17.06-v1.3 © 2017 Docker, Inc.


 SERVICE DISCOVERY: WORKER.PY

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: STARTING A COMPOSE APP
Work through the 'Starting a Compose App' exercise in the Docker Fundamentals
Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 CONNECTING TO THE WEB UI
webui dashboard: http://[IP]:8000/
Looks like about 3.33 coins/second.

17.06-v1.3 © 2017 Docker, Inc.


 SCALING UP THE APPLICATION
Want higher performance
Need to determine bottlenecks
Common UNIX tools to the rescue!

17.06-v1.3 © 2017 Docker, Inc.


 LOOKING AT RESOURCE USAGE
top
(you should see idle cycles)
vmstat 3
(the 4 numbers should be almost zero, except bo for logging)

We have available resources; how can we use them?

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: SCALING A COMPOSE APP
Work through the 'Scaling a Compose App' exercise in the Docker Fundamentals
Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER COMPOSE TAKEAWAYS
Docker Compose makes single node orchestration easy
Docker Compose makes scaling services easy
Bottleneck identification important
Syntactically: docker-compose.yml + API

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
Docker compose examples: http://dockr.ly/1FL2VQ6
Overview of docker-compose CLI: http://dockr.ly/2wtQlZT
Docker compose reference: http://dockr.ly/2iHUpeX

17.06-v1.3 © 2017 Docker, Inc.


 INTRODUCTION TO SWARM MODE

17.06-v1.3 © 2017 Docker, Inc.


 TOPICS
Creating Swarms
Services and Tasks
Multi-host networking
Routing Mesh
Service Upgrades

17.06-v1.3 © 2017 Docker, Inc.


 DISTRIBUTED APPLICATION ARCHITECTURE
Applications consisting of one or more containers across one or more nodes
Docker Swarm facilitates multi-node design.
Also supports multiple interacting services (like Compose)

17.06-v1.3 © 2017 Docker, Inc.


 SWARMS & SERVICES

17.06-v1.3 © 2017 Docker, Inc.


 SWARMS & SERVICES

17.06-v1.3 © 2017 Docker, Inc.


 RECOVERING FROM NODE FAILURE

docker service create --replicas 3 --name myapp --network mynet --publish 80:80 myapp:1.0

17.06-v1.3 © 2017 Docker, Inc.


 RECOVERING FROM NODE FAILURE

Swarm will schedule a new task in order to create the new container so that we
once again have 3 replicas.

17.06-v1.3 © 2017 Docker, Inc.


 NETWORKING REQUIREMENTS
Control plane: enable service discovery for containers across hosts
Data plane: enable moving a packet from host A to host B
Management plane: decide where containers should be scheduled

17.06-v1.3 © 2017 Docker, Inc.


 AN UNSCALABLE CONTROL PLANE
Heartbeat-style: all nodes register their containers with all other nodes
Network traffic would scale like n2
Unacceptable, want multi-host applications with O(1000) nodes

17.06-v1.3 © 2017 Docker, Inc.


 CONTROL PLANE

SWIM PROTOCOL
1. Ask neighbor A if they're still alive
2. Ask three other neighbors if they can reach neighbor A
3. No to both: conclude neighbor A is unhealthy / dead

Original ref: http://bit.ly/2eSzcTM

17.06-v1.3 © 2017 Docker, Inc.


 GOSSIP CONTROL PLANE
Docker control plane: piggyback DNS info on SWIM traffic
DNS entries spred 'infectiously' through cluster.
Gossip control plane scales like O(0)

17.06-v1.3 © 2017 Docker, Inc.


 OVERLAY DATA PLANE

17.06-v1.3 © 2017 Docker, Inc.


 MANAGEMENT PLANE
Need a manager to decide where to schedule containers
Manager should be highly available
Use a Raft consensus to reconcile managers.
Essentially a majority election triggered when the current leader fails.

17.06-v1.3 © 2017 Docker, Inc.


 SWARM NETWORK TOPOLOGY

Note: fixed IPs for managers, dynamic IPs OK for workers

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: SWARMS & SERVICES
Work through:

Creating a Swarm
Starting a Service
Node Failure Recovery

in the Docker Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 UNDER THE HOOD - SWARM INIT
When we first run docker swarm init:

current node enters Swarm Mode as manager-leader

listens for workers on :2377
creates scheduler and internal distributed state store
generates self-signed root CA
generates join tokens
creates an overlay network named ingress for external traffic inbound to services

17.06-v1.3 © 2017 Docker, Inc.


 KEY CONCEPTS - SERVICES
Service: desired state of containers based on a given image
Primary point of user interaction with the swarm
Scheduler tries to automatically maintains desired state of all services

17.06-v1.3 © 2017 Docker, Inc.


 KEY CONCEPTS - TASKS
A Task represents a unit of work assigned to a node
One task, one container
Atomic scheduling unit of swarm

17.06-v1.3 © 2017 Docker, Inc.


 BEHIND THE SCENES: SERVICE MANAGEMENT

17.06-v1.3 © 2017 Docker, Inc.


 SECURE BY DEFAULT
All manager communication is automatically mutually TLS encrypted.
First Swarm manager creates root CA
Root CA signs certificates for all subsequent nodes
Keys and certs rotated every 90 days

17.06-v1.3 © 2017 Docker, Inc.


 SWARMKIT
Open source toolkit to build multi-node systems
SwarmKit comes with two components:

Project repository: https://github.com/docker/swarmkit

Full features list: https://docs.docker.com/engine/swarm/
100% Docker native orchestration

17.06-v1.3 © 2017 Docker, Inc.


 SERVICES & THE OUTSIDE WORLD

Problem: how does my load balancer know which nodes to send external traffic
to?
17.06-v1.3 © 2017 Docker, Inc.

 THE ROUTING MESH

docker service create --replicas 2 --publish 80:80 
--name webapp webapp:1.0

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: LOAD BALANCING & THE ROUTING
MESH
Work through the 'Load Balancing & the Routing Mesh' exercise in the Docker
Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 OUR APPLICATION: DOCKERCOINS
It is a DockerCoin miner!
Dockercoins consists of 5 services
working together:

(DockerCoins 2016 logo courtesy of @XtlCnslt and @ndeloof. Thanks!)

17.06-v1.3 © 2017 Docker, Inc.


 SWARMING OUR APP: STACKS
Collection of services
Uses docker-compose.yml v3.1 as manifest
Can specify replicas, networks, volumes...

17.06-v1.3 © 2017 Docker, Inc.


 SCALING & SCHEDULING SERVICES
Default: each service will run exactly one container
Improve performance by adding more containers
VIPs load balance across tasks
Works best with stateless containers
Scheduling strategies: replicated or global

17.06-v1.3 © 2017 Docker, Inc.


 UPDATING SERVICES
Apps get periodic updates
Want updates with minimal service interruption
Swarm mode provides tooling for:

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: APPLICATION DEPLOYMENT
Work through:

Dockercoins On Swarm
Scaling and Scheduling Services
Updating a Service

in the Docker Fundamentals Exercises book.

17.06-v1.3 © 2017 Docker, Inc.


 SWARM DETAILS: NODE CONSTRAINTS

docker service create --replicas 3 --name myapp \

--network mynet --publish 80:80 \
–-constraint com.example.storage="ssd" myapp:1.0

17.06-v1.3 © 2017 Docker, Inc.


 SWARM DETAILS: SERVICE LOAD BALANCING
Network requests for service names resolve to a VIP, internally load balanced by
IPVS.
DNSRR also available
Default is now VIP
Specify on service creation:
docker service create --endpoint-mode [VIP|DNSRR]

17.06-v1.3 © 2017 Docker, Inc.


 SWARM MODE ROBUSTNESS
It doesn't matter:
which node a container runs on
if a few nodes die
if interacting processes are on different nodes
which nodes are running user-facing containers

... everything will still work.

17.06-v1.3 © 2017 Docker, Inc.


 SWARM MODE TAKEAWAYS
Distributed applications across infrastructure
High availability
Self healing service definitions
Default security via mutual TLS encryption & certificate rotation.
Simple service discovery & load balancing

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
Getting started with Docker swarm: http://dockr.ly/2vUFWTA
Swarm mode overview: http://dockr.ly/2jh11Vd

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER SECRETS

17.06-v1.3 © 2017 Docker, Inc.


 WHAT IS A SECRET?

17.06-v1.3 © 2017 Docker, Inc.


 MOTIVATION FOR SECRETS
Challenges in distributed systems

Secrets can be embed in source code in GitHub

Secrets can be distributed to other nodes in container orchestration systems
Secrets could be tampered with in transit

17.06-v1.3 © 2017 Docker, Inc.


 SECRETS USE CASE
Manage services with sensitive information in Docker Swarm such as
Passwords
TLS certificates
Private keys
And more...

17.06-v1.3 © 2017 Docker, Inc.


 SECRETS WORKFLOW 1: CREATION & STORAGE
Transmitted over mutual TLS
Encrypted at rest
Part of the Raft datastore (therefore HA)
Label-based access control

17.06-v1.3 © 2017 Docker, Inc.


 SECRETS WORKFLOW 2: DISTRIBUTION
Secret access is per service
Managers propagate secrets (TLS) to
only the containers that need them
("Least Privilege")
tmpfs-mounted unencrypted in
container at
/run/secrets/<secret_name>
Deleted when service loses access to a
secret

17.06-v1.3 © 2017 Docker, Inc.


 SECRETS WORKFLOW 3: SECRET USAGE

17.06-v1.3 © 2017 Docker, Inc.


 EXERCISE: SECRETS
Work through the 'Docker Secrets' exercise in the Docker Fundamentals Exercise
book.

17.06-v1.3 © 2017 Docker, Inc.


 FURTHER READING
Manage sensitive data with Docker secrets: http://dockr.ly/2vUNbuH
Docker secret reference: http://dockr.ly/2iSsNJC
Introducing Docker secrets management: http://dockr.ly/2k7zwzE
Securing the AtSea app with Docker secrets: http://dockr.ly/2wx5MyV
Example of using Docker secrets in UCP: http://dockr.ly/2gtszUY

17.06-v1.3 © 2017 Docker, Inc.


 FUNDAMENTAL ORCHESTRATION TAKEAWAYS
Distributed Application Architecture orchestrates one or more containers across
one or more nodes
Docker Swarm and Docker Compose provide native node and container
orchestration support
Services, Swarms and Stacks enhance scalability and stability

17.06-v1.3 © 2017 Docker, Inc.


 DOCKER FUNDAMENTALS
Please take our feedback survey:

https://dockertraining.typeform.com/to/gjciUl
Get in touch: training@docker.com

training.docker.com

17.06-v1.3 © 2017 Docker, Inc.

