path: root/README.md

---
title: Developing with Docker
author: gitlab.com/xlgmokha/developing-with-docker
date: 2020-10-24
---

# Developing with Docker
Mo Khan | Software Developer | GitLab

```text
      Building
        smaller
          Docker
            images?
                              ##         .
                        ## ## ##        ==
                    ## ## ## ## ##    ===
                /"""""""""""""""""\___/ ===
                {                       /  ===-
                \______ O           __/
                  \    \         __/
                    \____\_______/
```

# whoami

Software developer from Calgary, AB, Canada.

* GitLab
* Cisco
* ThoughtWorks

# Why?

License scanning at GitLab.

1. Scan target project for lock files (Gemfile.lock, Pipfile.lock etc)
2. Install project tools (Ruby 2.7.2, Python 3.8.4)
3. Install project dependencies (Rails, Django)

```bash
        モ ls ~/development/gitlab | grep lock
        Gemfile.lock
        Pipfile.lock
        yarn.lock
```

Ship a Docker image:

* Multiple versions of Ruby, Python etc
* Package managers for different languages
* Omnibus package of scanner code
* System packages/dependencies (libpq-dev, libsqlite3-dev etc)

Multiple Languages/Versions:

* Dotnet Core
* Golang
* Java
* Mono
* NodeJS
* PHP
* Python
* Ruby
* Rust

Multiple Package Managers:

* Bundler
* pip
* pipenv
* gradle
* maven

Large Docker images

* slow downloads
* more disk space is required
* more bandwidth is consumed

# Overview

```text
  -------------
  | git       |
  -------------     ----------------
  | main      | --> | gitlab-runner |
  | feature-a |     ----------------
  | feature-b |           |
  -------------           | (launch container)
                          V
                    ---------------
                    |             | <----- 10GB ------
                    | Docker Host | ---------        |
                    |             |         |        |
                    ---------------         |        |
                          |              download    |
                          V                 |        |
                    --------------          V        |
                    | License    |     ------------  |
                    | scanner    |     | registry | -|
                    --------------     ------------
```

License scanner

1. Search for lockfiles
2. Install desired version of language tools
3. Install packages via package manager
4. Scan for licenses
5. Export JSON report

# Zoom in

How did you shrink the image from 10GB down to 1GB?

```text
      ---------------
      |             | <----- 10GB ------
      | Docker Host | ---------        |
      |             |         |        |
      ---------------         |        |
            |              download    |
            V                 |        |
      --------------          V        |
      | License    |     ------------  |
      | scanner    |     | registry | -|
      --------------     ------------
```

Also added

* Support for more languages and versions.
* Support for limited connectivity environments.

# Agenda

* Definitions
* Ecosystem
* Build
* Analyze
* Optimize

```text
        < What are we going to talk about? >
        ----------------------------------
            \
             \
              \
                            ##        .
                      ## ## ##       ==
                  ## ## ## ##      ===
              /""""""""""""""""___/ ===
          ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ /  ===- ~~~
              \______ o          __/
                \    \        __/
                  \____\______/
```

# Definitions

* Image
* Container
* Registry

# Definitions - Image

A Docker image is similar to a Ruby class.
A class defines the behaviour and data associated with the class.

```ruby
class Person
  attr_reader :name

  def initialize(name)
    @name = name
  end

  def hug(other)
    puts "#{name} 🤗 #{other.name}"
  end
end
```

# Definitions

You can't do much with a class until you create
an instance of one.

```ruby
                      mo = Person.new("mo")
                      me = Person.new("me")
```

# Definitions - Objects

Once a class is instantiated you can invoke
methods on the object. An object can interact
with other objects.

```ruby
                      mo.hugs(me)
```

# Definitions - Container

A container is a running instance of an image.
Similar to how an object is an instance of a class.

| Ruby | Docker |
| -- | -- |
| Class | Image |
| Object | Container |

# Definitions - Registry

Registry: stores images and makes them available to others

This include metadata about images and blobs for each layer in the image.

For example:

* https://registry-1.docker.io
* https://registry.gitlab.com

```bash
curl -s -i https://registry-1.docker.io/v2/alpine/tags/list
```

# Architecture

```text
                      ----------
                      | Client |
                      ----------
                      | build  |
                      | pull   |
                      | run    |
                      ----------
                          | (tcp/unix socket)
                          V
                    ---------------
                    | Docker Host |
                    ---------------
                    | Daemon     |
                    | Containers |
                    | Images     |
                    --------------
                          |  A
                          V  |
                     ------------
                     | Registry |
                     ------------
                     | Images   |
                     ------------
```

https://docs.docker.com/get-started/overview/#docker-architecture

# $ docker run -it alpine:latest cat /etc/os-release

```terminal8
docker run -it alpine:latest cat /etc/os-release
```

1. check if "alpine:latest" is on docker host
1. download "alpine:latest" from registry to docker host
1. start a container using the "alpine:latest" image

# Dockerfile

```file
path: examples/001/Dockerfile
relative: true
lang: docker
```

https://docs.docker.com/engine/reference/builder/

# FROM alpine:latest

Initializes a build stage and sets a Base Image.

```file
path: examples/001/Dockerfile
relative: true
lang: docker
```

# COPY "hello.rb"

Copy "hello.rb" from the host to "/usr/local/bin/hello" in the Docker image.

```bash
$ cat examples/001/hello.rb
```
```file
path: examples/001/hello.rb
relative: true
lang: ruby
```

# Dockerfile - RUN

RUN a command from within the image and make "hello" executable.

```file
path: examples/001/Dockerfile
relative: true
lang: docker
lines:
  start: 2
  end: 3
```

# Dockerfile - CMD

Set the default command to run when the docker image
is launched as a container.

```file
path: examples/001/Dockerfile
relative: true
lang: docker
lines:
  start: 3
  end: 4
```

# docker build -t developing-with-docker:latest examples/001/

```terminal32
docker build --network=host -t developing-with-docker:latest examples/001/
```

# docker run developing-with-docker:latest

```terminal32
docker run developing-with-docker:latest
```

# dive

Useful for identifying bloat.
https://github.com/wagoodman/dive

* Displays each layer
* Allows investigating files that are added/removed/changed in each layer

# dive original

```bash
│ Layers ├──────────────────────────────────────────────────────────────────────────── ┃ ● Current Layer Contents ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Cmp   Size  Command                                                                    Permission     UID:GID       Size  Filetree
    5.6 MB  FROM 5a96ef02e9cab83                                                       drwxr-xr-x         0:0     841 kB  ├─⊕ bin
      40 B  #(nop) COPY file:253d38e67af26b201caf4e271248576ba6a7da0431f5ddee20c5c6df5 drwxr-xr-x         0:0        0 B  ├── dev
     16 MB  apk add ruby                                                               drwxr-xr-x         0:0     383 kB  ├─⊕ etc
      40 B  chmod +x /usr/local/bin/hello                                              drwxr-xr-x         0:0        0 B  ├── home
                                                                                       drwxr-xr-x         0:0     3.9 MB  ├─⊕ lib
│ Layer Details ├───────────────────────────────────────────────────────────────────── drwxr-xr-x         0:0        0 B  ├─⊕ media
                                                                                       drwxr-xr-x         0:0        0 B  ├── mnt
Tags:   (unavailable)                                                                  drwxr-xr-x         0:0        0 B  ├── opt
Id:     e19b1e8ac9df6c9165f82c5e278f4d8b4839c8e6727001a12477e3947ffd83ea               dr-xr-xr-x         0:0        0 B  ├── proc
Digest: sha256:aebe0431f34491da1f2785934e4190cfb9fbba01fc1aeff69f13c139a6ccac65        drwx------         0:0        0 B  ├── root
Command:                                                                               drwxr-xr-x         0:0        0 B  ├── run
apk add ruby                                                                           drwxr-xr-x         0:0     226 kB  ├─⊕ sbin
                                                                                       drwxr-xr-x         0:0        0 B  ├── srv
│ Image Details ├───────────────────────────────────────────────────────────────────── drwxr-xr-x         0:0        0 B  ├── sys
                                                                                       drwxrwxrwx         0:0        0 B  ├── tmp
                                                                                       drwxr-xr-x         0:0      14 MB  ├─⊕ usr
Total Image size: 21 MB                                                                drwxr-xr-x         0:0     1.8 MB  └─⊕ var
Potential wasted space: 532 kB
Image efficiency score: 98 %

Count   Total Space  Path
    2        428 kB  /etc/ssl/certs/ca-certificates.crt
    2         79 kB  /lib/apk/db/installed
    2         25 kB  /lib/apk/db/scripts.tar
    2         288 B  /lib/apk/db/triggers
    2         123 B  /etc/apk/world
    2          80 B  /usr/local/bin/hello
    2           0 B  /lib/apk/db/lock
    2           0 B  /var/cache/misc
```

# dive - layers

```bash
│ Layers ├─────────────────────────────────────────────────────────
Cmp   Size  Command
    5.6 MB  FROM 5a96ef02e9cab83
      40 B  #(nop) COPY file:253d38e67af26b201caf4e271248576ba6a7da
     16 MB  apk add ruby
      40 B  chmod +x /usr/local/bin/hello

│ Layer Details ├──────────────────────────────────────────────────

Tags:   (unavailable)
Id:     e19b1e8ac9df6c9165f82c5e278f4d8b4839c8e6727001a12477e3947ff
Digest: sha256:aebe0431f34491da1f2785934e4190cfb9fbba01fc1aeff69f13
Command:
apk add ruby

│ Image Details ├──────────────────────────────────────────────────


Total Image size: 21 MB
Potential wasted space: 532 kB
Image efficiency score: 98 %

Count   Total Space  Path
    2        428 kB  /etc/ssl/certs/ca-certificates.crt
    2         79 kB  /lib/apk/db/installed
    2         25 kB  /lib/apk/db/scripts.tar
    2         288 B  /lib/apk/db/triggers
    2         123 B  /etc/apk/world
    2          80 B  /usr/local/bin/hello
    2           0 B  /lib/apk/db/lock
    2           0 B  /var/cache/misc
```

# dive - layer details

```bash
┃ ● Current Layer Contents ┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Permission     UID:GID       Size  Filetree
drwxr-xr-x         0:0     841 kB  ├─⊕ bin
drwxr-xr-x         0:0        0 B  ├── dev
drwxr-xr-x         0:0     383 kB  ├─⊕ etc
drwxr-xr-x         0:0        0 B  ├── home
drwxr-xr-x         0:0     3.9 MB  ├─⊕ lib
drwxr-xr-x         0:0        0 B  ├─⊕ media
drwxr-xr-x         0:0        0 B  ├── mnt
drwxr-xr-x         0:0        0 B  ├── opt
dr-xr-xr-x         0:0        0 B  ├── proc
drwx------         0:0        0 B  ├── root
drwxr-xr-x         0:0        0 B  ├── run
drwxr-xr-x         0:0     226 kB  ├─⊕ sbin
drwxr-xr-x         0:0        0 B  ├── srv
drwxr-xr-x         0:0        0 B  ├── sys
drwxrwxrwx         0:0        0 B  ├── tmp
drwxr-xr-x         0:0      14 MB  ├─⊕ usr
drwxr-xr-x         0:0     1.8 MB  └─⊕ var
```

# docker pull - back then

```bash
2.8.0: Pulling from gitlab-org/security-products/license-management
0a01a72a686c: Pull complete
cc899a5544da: Pull complete
19197c550755: Pull complete
716d454e56b6: Pull complete
4a00dd5b28fc: Pull complete
e039d25729bf: Pull complete
930b12354a74: Pull complete
5c0b45be82b1: Pull complete
2174b0b17785: Pull complete
79e1fdae2dfc: Extracting [================>       ]    322MB/396.1MB
2c46852653ff: Download complete
40e535af8764: Download complete
0c1954047133: Download complete
080be37ae17e: Download complete
4179fc4ef96a: Download complete
87d7bc66884f: Download complete
581b5a43e64d: Download complete
a754f2766934: Download complete
c1e41c3670fe: Download complete
e782eb3070d9: Download complete
50d1fb4b55b2: Download complete
276696690bcf: Download complete
5eec42d5363b: Download complete
2296aa2193e9: Download complete
5fe4c102c0bc: Download complete
97390612da81: Downloading [=====>                ]  81.05MB/174MB
311b1e270e29: Downloading [===========>          ]  42.12MB/189.4MB
53dfbd975f60: Downloading [=>                    ]  25.36MB/843.4MB
3dd2acdebe0f: Waiting
d548f098494f: Waiting
da1cc42017ff: Waiting
cfc3cd025ca9: Waiting
69ea647e6c07: Waiting
1e27d5f85aa2: Waiting
94cf5e06627d: Waiting
30e1f788589d: Waiting
d9238ec317d1: Waiting
e17797fa5e82: Waiting
9003b36c1e4e: Waiting
```

# Start with a minimal base image

```Dockerfile
FROM licensefinder/license_finder:5.6.2
```

```bash
REPOSITORY                   TAG                 SIZE
debian                       stable-slim         69.2MB
licensefinder/license_finder 5.6.2               3.63GB
```

```Dockerfile
FROM debian:stable-slim
```

# Be picky

*Warning: The content below may be considered offensive.*

```Dockerfile
RUN apt-get update && apt-get install -y \
  build-essential \
  curl \
  git-core \
  sudo \
  unzip \
  wget
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash - && \
    apt-get -y install nodejs
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - && \
  echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list && \
  apt-get update && \
  apt-get install yarn
RUN apt-get install -y python rebar
RUN apt-get install -y python-pip && \
    pip install --upgrade pip==$PIP_INSTALL_VERSION
RUN apt-get install -y locales
RUN wget https://packages.erlang-solutions.com/erlang-solutions_${MIX_VERSION}_all.deb && \
    sudo dpkg -i erlang-solutions_${MIX_VERSION}_all.deb && \
    sudo rm -f erlang-solutions_${MIX_VERSION}_all.deb && \
    sudo apt-get update && \
    sudo apt-get install -y esl-erlang && \
    sudo apt-get install -y elixir
RUN apt-get install -y python-dev && \
  pip install --ignore-installed six --ignore-installed colorama --ignore-installed requests --ignore-installed chardet --ignore-installed urllib3 --upgrade setuptools && \
  pip install conan
RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF &&\
  echo "deb https://download.mono-project.com/repo/ubuntu stable-xenial main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list &&\
  apt-get update &&\
  apt-get install -y mono-complete &&\
  curl -o /usr/local/bin/nuget.exe https://dist.nuget.org/win-x86-commandline/latest/nuget.exe &&\
  echo "alias nuget=\"mono /usr/local/bin/nuget.exe\"" >> ~/.bash_aliases
RUN wget -q https://packages.microsoft.com/config/ubuntu/16.04/packages-microsoft-prod.deb &&\
  sudo dpkg -i packages-microsoft-prod.deb &&\
  sudo apt-get update &&\
  sudo apt-get install -y dotnet-runtime-2.1
```

# Group 'apt-get install' together

```Dockerfile
COPY config/install.sh /opt/install.sh
RUN bash /opt/install.sh
```

```bash
#!/bin/bash
apt-get install -y --no-install-recommends \
  apt-transport-https \
  autoconf \
  automake \
  bsdmainutils \
  bzip2 \
  ca-certificates \
  cmake \
  curl \
  gnupg2 \
  make \
  pkg-config \
  re2c \
  rebar \
  zstd
```

*note: avoid installing build tools if you can*

# Deflate

Compress directories that are large during build time.

```bash
#!/bin/bash

function deflate() {
  local file=$1
  local dir=$2
  local zstd_command="/usr/bin/zstd -19 -T0"
  tar --use-compress-program "$zstd_command" -cf "$file" "$dir"
}

cd /opt
deflate /opt/asdf.tar.zst asdf

cd /usr/lib
deflate /usr/lib/gcc.tar.zst gcc
deflate /usr/lib/mono.tar.zst mono
deflate /usr/lib/rustlib.tar.zst rustlib

cd /usr/share
deflate /usr/share/dotnet.tar.zst dotnet
```

# Inflate

Decompress directories when the container is launched
by hooking into the ENTRYPOINT.

```Dockerfile
ENTRYPOINT ["/run.sh"]
```

```bash
#!/bin/bash

function inflate() {
  local file=$1
  local to_dir=$2
  if [ -f "$file" ]; then
    tar --use-compress-program zstd -xf "$file" -C "$to_dir"
    rm "$file"
  fi
}

inflate /opt/asdf.tar.zst /opt
inflate /usr/lib/gcc.tar.zst /usr/lib
inflate /usr/lib/mono.tar.zst /usr/lib
inflate /usr/lib/rustlib.tar.zst /usr/lib
inflate /usr/share/dotnet.tar.zst /usr/share

sh "$@"
```

# Build specialized packages

```ruby
build do
  env = with_standard_compiler_flags(with_embedded_path)
  configure_command = [
    "--disable-debug-env",
    "--disable-dtrace",
    "--disable-install-capi",
    "--disable-install-doc",
    "--disable-install-rdoc",
    "--disable-jit-support",
    "--enable-shared",
    "--prefix=#{install_dir}",
    "--with-out-ext=coverage,dbm,readline,rdoc,win32,win32ole,sdbm",
    "--without-gdbm",
    "--without-gmp",
    "--without-jemalloc",
    "--without-tk",
    "--without-valgrind"
  ]
  configure(*configure_command, env: env)
  make "-j #{workers}", env: env
  make "-j #{workers} install", env: env
end
```

```bash
# ls -lh /opt/toolcache/ruby-* | awk '{ print $5 " " $9 }'
5.3M /opt/toolcache/ruby-2.4.10_2.4.10-1_amd64.deb
5.3M /opt/toolcache/ruby-2.4.5_2.4.5-1_amd64.deb
5.3M /opt/toolcache/ruby-2.4.9_2.4.9-1_amd64.deb
5.4M /opt/toolcache/ruby-2.5.8_2.5.8-1_amd64.deb
5.6M /opt/toolcache/ruby-2.6.0_2.6.0-1_amd64.deb
5.6M /opt/toolcache/ruby-2.6.1_2.6.1-1_amd64.deb
5.6M /opt/toolcache/ruby-2.6.2_2.6.2-1_amd64.deb
5.6M /opt/toolcache/ruby-2.6.3_2.6.3-1_amd64.deb
5.6M /opt/toolcache/ruby-2.6.4_2.6.4-1_amd64.deb
5.6M /opt/toolcache/ruby-2.6.5_2.6.5-1_amd64.deb
5.6M /opt/toolcache/ruby-2.6.6_2.6.6-1_amd64.deb
5.7M /opt/toolcache/ruby-2.7.0_2.7.0-1_amd64.deb
5.7M /opt/toolcache/ruby-2.7.1_2.7.1-1_amd64.deb
5.7M /opt/toolcache/ruby-2.7.2_2.7.2-1_amd64.deb
```

# Results

```bash
REPOSITORY                 TAG                 SIZE
analyzers/license-finder   3.28.1              1.4GB
analyzers/license-finder   2.8.0               9.83GB
analyzers/license-finder   1.5.0               4.06GB
```

# Summary

* Keep each layer small
* More layers provides opportunity for more parallel downloads
* Download will block on the largest layer.
* Too many layers can cause too many parallel downloads
* Try to collapse layers by grouping logical things together
* Cleanup unnecessary artifacts in each layer
* Deflate files when building layers
* Inflate files when container is launched

Thank you

[gitlab.com/xlgmokha/developing-with-docker](https://gitlab.com/xlgmokha/developing-with-docker)

# Feedback

* people will know what Docker is
* watch the screen from different sizes
  * increase size and not make it black background. hard to see.
  * white background
* Take into account:
  * Not native English speakers
    * Don't use fancy English words
  * Talk slower: adjust talking speed based on the audience.
* What is the story?
  * Explain the problem and context of the situation and why are we interested in layers and this talk.
    * Why should I watch this talk?
    * Visitors may not be engaged so do something to promote and get people interested in what you will share.
    * explain why we had this problem and explain how the scanner works in the context of a CI build.
  * What is the problem that this person is solving?
* Mostly Ruby On Rails focused audience there are some devops interested people.
  * Some people are interested in learning how to shrink Docker images and this can be useful to people.
* I want you to learn about:
  * zstandard
  * debian/rpm/apk packaging  (omnibus)

Best talk:

* Idea
* drop everything that does not support this idea around this talk.
* Pattern: humans get bored, get distracted and return.
  * If I don't understand when I return then I don't like it anymore.
  * goal: If you get distracted for a second you can still understand what is going on.