Docker outside of Docker no longer works in EKS

Article ID:360028151031
2 minute readKnowledge base

Issue

  • I am using docker agents that mount the /var/run/docker.sock (i.e. Docker outside of Docker) and the docker commands inside that agent fail with connection issues. Example of a failed docker build:

+ docker build -t test-image:latest .
Sending build context to Docker daemon  2.048kB
[...]
Step 4/5 : RUN apk --update --no-cache add   openjdk8-jre=$JRE_VERSION   curl
 ---> Running in a394ce75098c
fetch https://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz: temporary error (try again later)
fetch https://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz: temporary error (try again later)
ERROR: unsatisfiable constraints:
  curl (missing):
    required by: world[curl]
The command '/bin/sh -c apk --update --no-cache add   openjdk8-jre=$JRE_VERSION   curl' returned a non-zero code: 7

Environment

Related Issue

Explanation

The docker bridge network is disabled by default in the AWS EKS AMI since the release v20190211. If you do not not specify any network when creating a docker container, the container has no network interface other than loopback:

$ ifconfig
lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

For that reason, containers started using docker outside of docker - that is to say mounting the /var/run/docker.sock from a jenkins kubernetes agent for example - will not be able to communicate to the outside world.

Resolution

Use container image build tools in Kubernetes, as per option 1 under How to build my own container images in CloudBees CI on modern cloud platforms.

Workaround

Use Docker in Docker instead of Docker outside of Docker (i.e. mounting /var/run/docker.sock). Since Docker in Docker runs its own docker daemon inside the container, it is not impacted by this issue.

To learn how to configure a "DinD" agent, review option 3 under How to build my own container images in CloudBees CI on modern cloud platforms.

This article is part of our Knowledge Base and is provided for guidance-based purposes only. The solutions or workarounds described here are not officially supported by CloudBees and may not be applicable in all environments. Use at your own discretion, and test changes in a safe environment before applying them to production systems.