I've been building a 3d printer for about 6 months now and one of my biggest pain points have been the firmware and configuration management.

I'm not going into the guts of configuring Marlin. That's way out of the scope of this (or any single) blog post. This is only about an easy way of keeping track of your configurations between Marlin changes and increasing the build speed of the firmware.

Two pre-requisites is that you will need to be able to create and run Docker images. Generally this is done by have Docker Desktop running on your local computer. You will also need to have PowerShell. In theory this should work on a Linux and Mac using PowerShell Core but I haven't tested it (yet). I've only used it on Windows.

When you first build your configuration with the most frequently documented way of using the Arduino IDE it's fine waiting the 5 minutes to build plus the other few minutes to flash. After your 20th build and forgetting everything you've changed, you want to throw your printer out the window. Or just find a better way. And there is, and it's finally going to be documented somewhere. By somewhere I mean here.

You can use the arduino-cli to do the compiling of the firmware, this is where the speed comes from. The IDE is slow, the CLI is fast. On my computer it took the compile time from about 5 minutes down to about 45 seconds. That's a really big improvement. Because it's a standalone executable we can run this in Docker. Because there's a Linux version of arduino-cli and with the use of Docker and PowerShell Core it's portable.

There is one caveat to this method. Flashing the firmware. To do that I use an OctoPi that is connected to my printer with the Firmware Updater plugin. If you kill your connection to your board by passing in some bad configuration, like serial ports, then you'll need to find a way of flashing the firmware. You can (and this is how I did it) clone the code to your computer and setup the Arduino IDE to flash the board. I haven't had to do that for a long time though.

Now for the meat of this post.

Primary reasons for running in Docker:

• Speed
• More speed
• Portability
• No extra needed software (Arduino IDE for example)
• Easy configuration management (more on that below)

Here's the folder layout and the purpose of the 6 files. Each file and their contents is detailed below.

• .gitignore Git's ignore file so we don't check in the outputs
• Configuration.h Main Marlin configuration file
• Configuration_adv.h Advanced Marlin configuration file
• Dockerfile the dockerfile that will build the Marlin Firmware
• Dockerfile-Configs the dockerfile that makes it easy to pull out specific config files from source
• run.ps the PowerShell script to glue everything together

.gitignore

.gitignore ignores the files that are built by this tool. There's not much sense in keeping them in source control, you can if you want though, but I wouldn't.

The contents of .gitignore is:

build/
temp/
configs/

Configuration.h and Configuration_adv.h

The configuration.h and configuration_adv.h files are the files that you modify and want to keep track of. You will base them off of the original configuration.h and configuration_adv.h files from the Marlin source code.

Dockerfile-Configs

The Dockerfile-Configs is used to get the configs from marlin source code. I don't want to do all of the pulls, checkouts and copies by hand so I used docker and my PowerShell script to do it for me.

The contents of Dockerfile-Configs is:

FROM debian:10.2 AS build

ARG MARLIN_BRANCH=2.0.x

RUN apt update && \
    apt install -y git

# cache busting with run.ps1 only when the latest release changes
COPY temp/marlinlatest /marlinlatest
RUN git clone https://github.com/MarlinFirmware/Marlin.git /marlin
WORKDIR /marlin
RUN git checkout $MARLIN_BRANCH

COPY Configuration.h /mine/Configuration.h
COPY Configuration_adv.h /mine/Configuration_adv.h

FROM scratch
COPY --from=build /marlin/Marlin/Configuration.h /marlin/Original.Configuration.h
COPY --from=build /marlin/Marlin/Configuration_adv.h /marlin/Original.Configuration_adv.h
COPY --from=build /mine/Configuration.h /marlin/Mine.Configuration.h
COPY --from=build /mine/Configuration_adv.h /marlin/Mine.Configuration_adv.h

What it does is uses a temporary image named build. It uses a Debian base image, installs git, does a git clone of the Marlin Firmware repository. Then a checkout of the correct branch.  After that image is built it creates an empty container with nothing in it, scratch and copies the config files from build into it.

To get the config files out of it you have to create a container from that image, then run docker cp to get the files out. I didn't want to do that every time by hand so I put it in my PowerShell script.

Dockerfile

`Dockerfile` is used for actually doing the build of the Marlin firmware. It's where all of the magic happens. I use an Archim 2 controller board, so I have some things in there that are related specific to that.

Here's the file:

FROM debian:10.2 AS build

ARG MARLIN_BRANCH=2.0.x

RUN apt update && \
    apt install -y git

WORKDIR /arduino-cli
COPY temp/arduino-cli.tar.gz  arduino-cli.tar.gz
RUN tar -xvf arduino-cli.tar.gz
RUN ls -la

RUN ./arduino-cli \
    core \
    --additional-urls https://raw.githubusercontent.com/ultimachine/ArduinoAddons/master/package_ultimachine_index.json \
    update-index

RUN ./arduino-cli \
    core \
    --additional-urls https://raw.githubusercontent.com/ultimachine/ArduinoAddons/master/package_ultimachine_index.json \
    install ultimachine:sam

RUN ./arduino-cli \
    lib \
    install TMCStepper

# cache busting with run.ps1 only when the latest release changes
COPY temp/marlinlatest /marlinlatest
RUN git clone https://github.com/MarlinFirmware/Marlin.git /marlin
WORKDIR /marlin
RUN git checkout $MARLIN_BRANCH

RUN mkdir /build && \
    cp /marlin/Marlin/Configuration.h /build/Configuration.original.h && \
    cp /marlin/Marlin/Configuration_adv.h /build/Configuration_adv.original.h

COPY Configuration.h Marlin/Configuration.h
COPY Configuration_adv.h Marlin/Configuration_adv.h

COPY Configuration.h /build/Configuration.h
COPY Configuration_adv.h /build/Configuration_adv.h

RUN /arduino-cli/arduino-cli \
    compile \
    --fqbn ultimachine:sam:archim \
    --build-path /build/temp \
    --build-cache-path /build/cache \
    -v \
    /marlin/Marlin/Marlin.ino

FROM scratch
COPY --from=build /build/temp/Marlin.ino.bin /marlin/marlin.bin
COPY --from=build /build/Configuration.h /marlin/Configuration.h
COPY --from=build /build/Configuration_adv.h /marlin/Configuration_adv.h
COPY --from=build /build/Configuration.original.h /marlin/Configuration.original.h
COPY --from=build /build/Configuration_adv.original.h /marlin/Configuration_adv.original.h

It's works very similarly to Dockerfile-Configs in that it uses a temporary container based on Debian, and installs git. It also copies in the arduino-cli that is downloaded through the PowerShell script, installs some of my printer specific modules (Archim2 and TMCStepper) and then does the build. After it actually builds it will copy the built firmware, the original config files, and our config files and copy them to an empty image.

Run.ps1

run.ps1 is the glue that binds it all together. This is what will call all of the docker commands, build images, copies files, etc.

Here it is:

param(
    [string]
    $branch = "2.0.x",

    [string]
    $commit = "",

    [Switch]
    $configsOnly
)
$MARLIN_BRANCH = $branch

function Compute-Sha([string] $file)
{
    if (Test-Path $file)
    {
        $hash = Get-FileHash -Algorithm SHA256 -Path $file
        return $hash.Hash
    }

    return ""
}

function Download-File([string] $file, [string] $url)
{
    Invoke-WebRequest -Uri $url -OutFile download.temp
    if ((Compute-Sha -file $file) -ne (Compute-Sha -file download.temp))
    {
        Copy-Item download.temp $file -Force
    }
    Remove-Item download.temp
}

Download-File -file "./temp/marlinlatest" -url "https://api.github.com/repos/MarlinFirmware/Marlin/branches/${MARLIN_BRANCH}"

if ($commit -ne "")
{
    $MARLIN_BRANCH = $commit
}

if ($configsOnly -eq $true)
{
    docker image build --build-arg MARLIN_BRANCH=${MARLIN_BRANCH} -t marlinconfigs -f ./Dockerfile-Configs .
    $containerId = & docker container create -it marlinconfigs /bin/sh
    mkdir configs -ErrorAction SilentlyContinue | Out-Null

    docker cp ${containerId}:/marlin/. configs

    docker rm $containerId
}
else {
    Download-File -file "./temp/arduino-cli.tar.gz" -url "https://downloads.arduino.cc/arduino-cli/arduino-cli_latest_Linux_64bit.tar.gz"
    
    docker image build --build-arg MARLIN_BRANCH=${MARLIN_BRANCH} -t marlin .
    $containerId = & docker container create -it marlin /bin/sh
    mkdir build -ErrorAction SilentlyContinue | Out-Null

    docker cp ${containerId}:/marlin/. build

    docker rm $containerId
}

It has 2 things it can do, build the firmware or just copy out the configs into a directory for easy comparison. It defaults to using the latest commit of the stable (ish) 2.0.x branch of Marlin and to do the build.

If all you want is to build with the 2.0.x branch and the latest commit with your configs, just use ./run.ps1 from within PowerShell. It'll build and you'll get your firmware and the configs in the ./build directory.

If you want the configs from the latest commit in the 2.0.x branch, use ./run.ps1 -configsOnly.

If you want to use a different branch, like the popular 2.0.x-bugfix, add -branch "2.0.x-bugfix" on to the end. For example: ./run.ps1 -configsOnly -branch "2.0.x-bugfix".

If you want to use a specific commit on the branch. Add -commit "<commit hash or tag>". For example: ./run.ps1 -configsOnly -commit "2.0.1".

The output directory for the building of the firmware is ./build and the output directory when getting the config files is ./config.

Conclusion

Doing it this way I am now able to easily version and keep track of my configurations, compare with the latest Marlin configurations, and compile the firmware. I'm pretty pleased with how it has turned out and wanted to share it with the world.

My GitHub repo that uses this is at https://github.com/veccsolutions/3DPrinterConfig