Build container images in GitLab CI (iptables-legacy at the rescue)Wed 18 January 2023
It's 2023 and these days, building a container image in a CI pipeline should be straightforward. So let's try.
For this blog post we'll focus on GitLab SaaS only, that is, gitlab.com, as it's what I use for work and for personal projects.
To get started, we just need two files in our Git repository:
Dockerfileif you prefer to name it this way) that defines how to build a container image.
.gitlab-ci.ymlfile that defines what the CI should do. In the example below, we want to build a container image and push it to the GitLab registry associated with the GitLab repo.
Here is our Git tree:
$ ls -A Containerfile .git .gitlab-ci.yml $ cat Containerfile FROM debian:stable RUN apt-get update CMD echo hello world $ cat .gitlab-ci.yml build-container-image: stage: build image: debian:testing before_script: - apt-get update - apt-get install -y buildah ca-certificates script: - buildah build -t $CI_REGISTRY_IMAGE . - buildah login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY - buildah push $CI_REGISTRY_IMAGE
A few remarks:
- We use
buildah, but we could have used
- However we don't use
docker, because its client/server design makes it cumbersome to use in a CI environment: it requires a separate container to run the Docker daemon, plus setting the
DOCKER_HOSTvariable. Why bother?
Now let's push that. Does the CI pass? No, of course, otherwise I wouldn't be writing this blog post ;)
The CI fails at the
buildah build command, with a rather cryptic error:
$ buildah build --tag $CI_REGISTRY_IMAGE . [...] STEP 2/3: RUN apt-get update error running container: did not get container start message from parent: EOF Error: building at STEP "RUN apt-get update": netavark: code: 4, msg: iptables v1.8.8 (nf_tables): Could not fetch rule set generation id: Invalid argument
The hint here is
nf_tables... Back in July 2021, GitLab did a major update of
their shared runners infrastructure, and broke nftables support in the process,
as it seems.
So we have to use iptables instead.
Let's fix our
.gitlab-ci.yml, which now looks like that:
$ cat .gitlab-ci.yml build-container-image: stage: build image: debian:testing before_script: - apt-get update - apt-get install -y buildah ca-certificates - | # Switch to iptables legacy, as GitLab CI doesn't support nftables. apt-get install -y --no-install-recommends iptables update-alternatives --set iptables /usr/sbin/iptables-legacy script: - buildah build -t $CI_REGISTRY_IMAGE . - buildah login -u $CI_REGISTRY_USER -p $CI_JOB_TOKEN $CI_REGISTRY - buildah push $CI_REGISTRY_IMAGE
And push again. Does that work? Yes!
If you're interested in this issue, feel free to fork https://gitlab.com/arnaudr/gitlab-build-container-image and try it by yourself.
It's been more than a year since this change, and I'm surprised that I didn't find much about it on the Internet, neither mentions of the issue, nor of a workaround. Maybe nobody builds container images in GitLab CI, or maybe they do it another way, I don't know. In any case, now it's documented in this blog, hopefully some will find it useful.