Issue
-
Docker image builds (Docker-in-Docker approach) are randomly failing when accessing external resources
$ docker build . ... Get:1 https://security.debian.org/debian-security/ jessie/updates/main libssl1.0.0 amd64 1.0.1t-1+deb8u11 [1047 kB] Get:2 https://deb.debian.org/debian/ jessie/main libkeyutils1 amd64 1.5.9-5+b1 [12.0 kB] Get:3 https://security.debian.org/debian-security/ jessie/updates/main libkrb5support0 amd64 1.12.1+dfsg-19+deb8u5 [59.5 kB] Get:4 https://security.debian.org/debian-security/ jessie/updates/main libk5crypto3 amd64 1.12.1+dfsg-19+deb8u5 [115 kB] Get:5 https://security.debian.org/debian-security/ jessie/updates/main libkrb5-3 amd64 1.12.1+dfsg-19+deb8u5 [303 kB] Get:6 https://security.debian.org/debian-security/ jessie/updates/main libgssapi-krb5-2 amd64 1.12.1+dfsg-19+deb8u5 [152 kB] Get:7 https://security.debian.org/debian-security/ jessie/updates/main libidn11 amd64 1.29-1+deb8u3 [137 kB] Get:8 https://security.debian.org/debian-security/ jessie/updates/main libssh2-1 amd64 1.4.3-4.1+deb8u3 [127 kB] Get:9 https://security.debian.org/debian-security/ jessie/updates/main libcurl3 amd64 7.38.0-4+deb8u15 [259 kB] Get:10 https://security.debian.org/debian-security/ jessie/updates/main krb5-locales all 1.12.1+dfsg-19+deb8u5 [2649 kB] Get:11 https://security.debian.org/debian-security/ jessie/updates/main openssl amd64 1.0.1t-1+deb8u11 [665 kB] Get:12 https://security.debian.org/debian-security/ jessie/updates/main ca-certificates all 20141019+deb8u4 [185 kB] Get:13 https://security.debian.org/debian-security/ jessie/updates/main curl amd64 7.38.0-4+deb8u15 [204 kB] Err https://deb.debian.org/debian/ jessie/main libkeyutils1 amd64 1.5.9-5+b1 Connection failed Get:14 https://deb.debian.org/debian/ jessie/main libsasl2-modules-db amd64 2.1.26.dfsg1-13+deb8u1 [67.1 kB] Get:15 https://deb.debian.org/debian/ jessie/main libsasl2-modules-db amd64 2.1.26.dfsg1-13+deb8u1 [67.1 kB] Get:16 https://deb.debian.org/debian/ jessie/main libsasl2-2 amd64 2.1.26.dfsg1-13+deb8u1 [105 kB] Get:17 https://deb.debian.org/debian/ jessie/main libldap-2.4-2 amd64 2.4.40+dfsg-1+deb8u4 [218 kB] Get:18 https://deb.debian.org/debian/ jessie/main librtmp1 amd64 2.4+20150115.gita107cef-1+deb8u1 [60.0 kB] Get:19 https://deb.debian.org/debian/ jessie/main libsasl2-modules amd64 2.1.26.dfsg1-13+deb8u1 [101 kB] Fetched 6456 kB in 8min 6s (13.3 kB/s) E: Failed to fetch https://deb.debian.org/debian/pool/main/k/keyutils/libkeyutils1_1.5.9-5+b1_amd64.deb Connection failed
Environment
-
Kubernetes 1.12 with Calico > 3.1
-
Builds using Docker-in-Docker approach to build Docker images
Explanation
The default maximum transmission unit (MTU) value of Calico might have been changed to 1440 if you have upgraded from a version prior to v3.1 of Calico,
while the Docker bridge (docker0) MTU is 1500 by default.
That may lead to network connectivity issues/instabilities.
Resolution
The MTU needs to be the same between the docker0 and calico network interfaces.
The MTU value depends on your environment and setup.
See the Calico guide.
Possible solutions:
-
Change the default MTU value of the
docker0bridge to match the Calico MTU. See the Docker guide. -
Change the default MTU value of Calico to match the default
docker0bridge. If using kops, the Calico configuration is located in thecalico-configconfigmap as the propertyveth_mtu. After updating it, you need to rotate all Calico containers.