As I'm moving everything into Kubernetes I needed a way to run debmirror
in a Docker container to synchronize my apt repositories.
There wasn't much guidance on how to do this, so here it is.
Some basic background on the image, I used a Debian base image and installed the debmirror
and gpg
packages. Brought in the correct GPG keys for the repositories I'm mirroring. Finally I set the CMD
directive to run debmirror
with the correct arguments. Since I wanted to use environment variables for the arguments I used sh -c
as the first part which will process the environment variables, without it they are not processed and debmirror is executed with the name of the variable instead of the actual value.
I wanted to use the same image to mirror the Debian, Ubuntu and Docker repositories without building separate images for each one so I included all of the keys in the single image. You could however pull those into separate images, or install through a script, etc, but that's more overhead I didn't want to deal with.
Here is my Dockerfile
:
FROM debian:10.5-slim
RUN apt update
RUN apt install -y debmirror gpg
# DEBIAN
RUN gpg -q --keyserver keys.gnupg.net --no-default-keyring --keyring trustedkeys.gpg --recv-keys DC30D7C23CBBABEE 4DFAB270CAA96DFA DCC9EFBF77E11517
# DOCKER
RUN gpg -q --keyserver keys.gnupg.net --no-default-keyring --keyring trustedkeys.gpg --recv-keys 7EA0A9C3F273FCD8
# UBUNTU
RUN gpg -q --keyserver keys.gnupg.net --no-default-keyring --keyring trustedkeys.gpg --recv-keys 871920D1991BC93C
CMD [ "sh", "-c", \
"debmirror ${DEST} --nosource --host=${HOST} --root=${ROOT} --dist=${DIST} --section=${SECTION} --i18n --arch=${ARCH} --passive --cleanup --method=${METHOD} --progress ${OTHERARGS} --rsync-extra=${RSYNCEXTRA:=trace}" ]
To build it, I used a simple docker command:
docker image build -t debmirror .
A note on which mirror to choose. The mirror that you choose should support rsync
. If it doesn't, you'll need to add another environment variable that tells it to ignore the trace
files. -e RSYNCEXTRA=none
. The standard Debian mirrors do not support rsync
and it'll sit at Updating remote trace files (using rsync) ...
for a few minutes before finally erroring out. Because of this, for Debian, I chose to use the Oregon State University mirror which is close to me, fast and allows rsync
. For Ubuntu I chose to use the Arizona State University mirror. The Docker repository doesn't seem to support rsync
so I had to specify the RSYNCEXTRA=none
environment variable.
Execution
To run it you can use the following
Debian
Powershell:
docker run --rm -it -e DEST=/mnt/debmirror/debian `
-e HOST=debian.oregonstate.edu `
-e ROOT=debian/ `
-e DIST=buster,buster-updates `
-e SECTION=main,contrib,non-free,main/debian-installer `
-e ARCH=amd64,arm64,armhf `
-e METHOD=http `
debmirror
Bash:
docker run --rm -it -e DEST=/mnt/debmirror/debian \
-e HOST=debian.oregonstate.edu \
-e ROOT=debian/ \
-e DIST=buster,buster-updates \
-e SECTION=main,contrib,non-free,main/debian-installer \
-e ARCH=amd64,arm64,armhf \
-e METHOD=http \
debmirror
Docker
PowerShell:
docker run --rm -it -e DEST=/mnt/debmirror/docker/debian `
-e HOST=download.docker.com `
-e ROOT=/linux/debian `
-e DIST=buster `
-e SECTION=stable `
-e ARCH=amd64,arm64,armhf `
-e METHOD=https `
-e RSYNCEXTRA=none `
debmirror
Bash:
docker run --rm -it -e DEST=/mnt/debmirror/docker/debian \
-e HOST=download.docker.com \
-e ROOT=/linux/debian \
-e DIST=buster \
-e SECTION=stable \
-e ARCH=amd64,arm64,armhf \
-e METHOD=https \
-e RSYNCEXTRA=none \
debmirror
Ubuntu
Powershell:
docker run --rm -it -e DEST=/mnt/debmirror/ubuntu `
-e HOST=mirror.arizona.edu `
-e ROOT=ubuntu/ `
-e DIST=groovy `
-e SECTION=main,restricted,universe,multiverse `
-e ARCH=amd64 `
-e METHOD=http `
debmirror
Bash:
docker run --rm -it -e DEST=/mnt/debmirror/ubuntu \
-e HOST=mirror.arizona.edu \
-e ROOT=ubuntu/ \
-e DIST=groovy \
-e SECTION=main,restricted,universe,multiverse \
-e ARCH=amd64 \
-e METHOD=http \
debmirror
Conclusion
Using a single container will now allow me to setup any number of Kubernetes jobs that will sync any apt repository with ease. I just need to update the image with the correct GPG keys.
If I need to mirror too many more of them I will probably put the installation of the GPG keys and execution of debmirror into a script which would then become the executed command. That would allow me to pass in the key id's as an environment variable instead of installing them in the image itself. I just didn't feel the need to add that complexity for this particular purpose. I still may do it as I work on getting this to run in Kubernetes.
I have a post on how to run the site and debmirror all in Kubernetes. It is at https://www.frakkingsweet.com/debian-mirror-in-kubernetes/