Docker outside of Docker no longer works in EKS

Article ID:360028151031
2 minute readKnowledge base

Issue

  • I am using docker agents that mount the /var/run/docker.sock (i.e. Docker outside of Docker) and the docker commands inside that agent fail with connection issues. Example of a failed docker build:

+ docker build -t test-image:latest .
Sending build context to Docker daemon  2.048kB
[...]
Step 4/5 : RUN apk --update --no-cache add   openjdk8-jre=$JRE_VERSION   curl
 ---> Running in a394ce75098c
fetch https://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz: temporary error (try again later)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz: temporary error (try again later)
ERROR: unsatisfiable constraints:
  curl (missing):
    required by: world[curl]
The command '/bin/sh -c apk --update --no-cache add   openjdk8-jre=$JRE_VERSION   curl' returned a non-zero code: 7

Environment

Related Issue

Explanation

The docker bridge network is disabled by default in the AWS EKS AMI since the release v20190211. If you do not not specify any network when creating a docker container, the container has no network interface other than loopback:

$ ifconfig
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

For that reason, containers started using docker outside of docker - that is to say mounting the /var/run/docker.sock from a jenkins kubernetes agent for example - will not be able to communicate to the outside world.

Resolution

Use container image build tools in Kubernetes, as per option 1 under How to build my own container images in CloudBees CI (CloudBees Core) on Modern Cloud Platforms.

Workaround

Use Docker in Docker instead of Docker outside of Docker (i.e. mounting /var/run/docker.sock). Since Docker in Docker runs its own docker daemon inside the container, it is not impacted by this issue.

To learn how to configure a "DinD" agent, review option 3 under How to build my own container images in CloudBees CI (CloudBees Core) on Modern Cloud Platforms.