Posts tagged: autotest

  • Testing Docker images with Autotest

    For awhile now, I’ve had a docker image which blindly gets rebuilt via dockerhub every time the upstream image changes. As the docker image I’m building just pulls bits and pieces from other sources, I didn’t really have anything to unit test as part of the CI process. Since it was working fine at the time, I just left it.

    Fast-forward a few years and sure enough, one of the things I bundle into the image shipped a broken version. The docker build still succeeded, but the final image didn’t work as expected. Rather than let that happen again, I thought I should find a testing solution.

    Docker’s Autotest looked like a good fit. It can run tests on new commits, PRs and crucially on upstream image changes. However, I was a bit confused how I could use it without bloating my image with tests embedded inside. Turns out it’s really simple using a multi-stage build.

    Suppose we have the following trivial docker image, where we simploy add a dummy example-app to the alpine image.

    FROM alpine:latest
    COPY app/ /app
    WORKDIR /app
    RUN ln -s /app/example-app /usr/local/bin/example-app

    We can turn it into a multi-stage image with a testing layer.

    # Create release layer and label it
    FROM alpine:latest as release
    COPY app/ /app
    WORKDIR /app
    RUN ln -s /app/example-app /usr/local/bin/example-app
    # Create testing layer
    FROM alpine:latest as test
    COPY --from=release / /
    COPY test/ /test
    WORKDIR /test
    ENTRYPOINT ["/bin/sh", "-c"]
    CMD ["./"]
    # Output release as the final image
    FROM release

    First we build the regular image and label it release. We then build a testing layer using release as a base and add in our tests. Here we just copy in a directory containing a bash script of arbitrary tests. The only requirement is that it exits with a 0 when tests pass, or anything else when they fail.

    Finally we call FROM release to ensure that on a regular build, we only build the app without the testing layer.

    For Autotest to work, we need to create a docker-compose.test.yml file with a sut service.

    version: "3.8"
          context: .
          target: "test"

    We set the target to test so that it builds and runs the test layer from our image. We can test it locally (with a deliberate test failure):

    $ docker-compose -f docker-compose.test.yml up sut
    sut_1  | Executing tests...
    sut_1  | OK - example-app should run fine
    sut_1  | FAIL - example-app should run fine
    sut_1  | PASSED: 1 FAILED: 1
    docker-autotest-example_sut_1 exited with code 1

    Now that it works locally we just need to enable Autotest in the Dockerhub settings.

    Full example repository: Autotest-example.

    April 16, 2021 - 2 minute read - docker autotest testing