Cloud Kube | Docker Build and Registry

Now that we’ve got a pipeline, we’re going to start working on deploying our service into the cloud. The first step is to create a docker image and store it somewhere that our pipeline as well as our cloud provider can access.

Create a Docker Image Registry

Canister.io provides a convenient and free (for small needs) docker container registry.

Create an account there you should get a default namespace (mine’s bullyrooks). Once you’re in create a new repo

Name it cloud_application and submit the repo creation form

Now we need to add the registry credentials to our github repo. Navigate to your github repo and click on the settings tab. Find secrets in the nav bar.

We’re going to make two secrets CANISTER_USERNAME and CANISTER_PASSWORD populating those values with the credentials we just created for canister.io

Create a Dockerfile

Create a new file in the root of the project directory called Dockerfile. Add this code

# syntax=docker/dockerfile:experimental
FROM openjdk:11-slim-buster as build

COPY .mvn .mvn
COPY mvnw .
COPY pom.xml .
COPY src src

# Setup the maven cache
RUN --mount=type=cache,target=/root/.m2,rw ./mvnw -B package -DskipTests

FROM openjdk:11-jre-slim-buster
COPY --from=build target/*.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

Here’s what we’re doing

• download a build image that has java 11 already installed on it
• copy over our build files one by one
• build the jar using a cache of the maven .m2 directory
• download a clean build image
• copy the .jar we just created into that image
• tell docker to start our jar on startup

Docker has pretty advanced caching techniques built in. By defining our build in this way we’re limiting the number of things that need to be rebuild for small changes (like a code change). Larger changes (like adding a dependency) can’t use previously cached stages, so they’ll have to rebuild from an earlier stage (but not from the earliest phases such as downloading the build image).

you can build locally to test it out

$ docker build .

Add Docker Build to our Pipeline.

Add the following lines to our feature.yaml github actions file (you’ll have to change out the image name for your registry name)

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

      - name: Login to canister.io
        uses: docker/login-action@v1
        with:
          registry: cloud.canister.io:5000
          username: ${{ secrets.CANISTER_USERNAME }}
          password: ${{ secrets.CANISTER_PASSWORD }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v3
        with:
          images: cloud.canister.io:5000/bullyrooks/cloud_application

      - name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          # Key is named differently to avoid collision
          key: ${{ runner.os }}-multi-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-multi-buildx

      - name: Build and push
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-new
        # This ugly bit is necessary if you don't want your cache to grow forever
        # till it hits GitHub's limit of 5GB.
        # Temp fix
        # https://github.com/docker/build-push-action/issues/252
        # https://github.com/moby/buildkit/issues/1896
      - name: Move cache
        run: |
          rm -rf /tmp/.buildx-cache
          mv /tmp/.buildx-cache-new /tmp/.buildx-cache

Here’s what we’re doing with this pipeline:

• Configure docker build to use buildx. This has enhanced features such as caching which we’ll utilize.
• Login to our container registry
• Connect to our registry to access our image metadata (such as tags and labels). This allows us to access them in the build/push action.
• Restore a cache of our previous docker build so that we can leverage it in this build
• Build and push our docker image using our previous login.
• Move the build cache as a workaround for a github action bug. (via Documentation)

Git Commit and Push

Lets push this up on a feature branch to see if it builds:

$ git checkout -b docker_build
Switched to a new branch 'docker_build'

$ git add .

$ git commit -m "docker build on feature"
[docker_build e1cfbec] docker build on feature
 2 files changed, 48 insertions(+), 1 deletion(-)
 create mode 100644 Dockerfile

$ git push --set-upstream origin docker_build
Enumerating objects: 10, done.
Counting objects: 100% (10/10), done.
Delta compression using up to 4 threads
Compressing objects: 100% (5/5), done.
Writing objects: 100% (6/6), 1.30 KiB | 1.30 MiB/s, done.
Total 6 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
remote: 
remote: Create a pull request for 'docker_build' on GitHub by visiting:
remote:      https://github.com/bullyrooks/cloud_application/pull/new/docker_build
remote:
To github.com-bullyrook:bullyrooks/cloud_application.git
 * [new branch]      docker_build -> docker_build
Branch 'docker_build' set up to track remote branch 'docker_build' from 'origin'.

Its going to take a long time the first time because EVERYTHING has to be pulled down, cached and built the first time. Subsequent builds shouldn’t take as long.

Create the Main Build Pipeline

We’re going to merge this to main, but we have to add the docker build to the main pipeline as well. I’m going to do something a little different here though. When we build on a feature branch we’re tagging the image with a build tag by default. When we build off of main we’re going to want a semantic version. There’s a bunch of ways to do this, but I want it to be automated and as hands off as possible.

Add this code to the main.yaml github action file. Again, make sure to change out your image name for your registry name.

      - name: Bump version
        run: |
          git config --global user.email "[email protected]"
          git config --global user.name "Actions"
          git fetch --tags
          wget -O - https://raw.githubusercontent.com/treeder/bump/master/gitbump.sh | bash
          echo "VERSION=$(git tag --sort=-v:refname --list "v[0-9]*" | head -n 1 | cut -c 2-)" >> $GITHUB_ENV

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v1

      - name: Login to canister.io
        uses: docker/login-action@v1
        with:
          registry: cloud.canister.io:5000
          username: ${{ secrets.CANISTER_USERNAME }}
          password: ${{ secrets.CANISTER_PASSWORD }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v3
        with:
          images: cloud.canister.io:5000/bullyrooks/cloud_application
          #setting value manually, but could come from git tag
          tags: |
            type=ref,event=tag

      - name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          # Key is named differently to avoid collision
          key: ${{ runner.os }}-multi-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-multi-buildx

      - name: Build and push
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: |
            cloud.canister.io:5000/bullyrooks/cloud_application:${{ env.VERSION }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-new
        # This ugly bit is necessary if you don't want your cache to grow forever
        # till it hits GitHub's limit of 5GB.
        # Temp fix
        # https://github.com/docker/build-push-action/issues/252
        # https://github.com/moby/buildkit/issues/1896
      - name: Move cache
        run: |
          rm -rf /tmp/.buildx-cache
          mv /tmp/.buildx-cache-new /tmp/.buildx-cache

You can see that we’re pulling in a script from a github repo, setting an environment variable and then using that as our tag when we push up the image. This script automatically revs the version (based on a git tag) and then pushes that tag back onto the git commit. I am pretty sure that it only revs patch version though, if you want to rev major or minor version, you’ll have to push up a git tag onto the main branch that reflects your desired version.

Lets try it out.

$ git add .

$ git commit -m "main pipeline docker update"
[docker_build 350e053] main pipeline docker update
 1 file changed, 37 insertions(+), 1 deletion(-)

$ git push
Enumerating objects: 9, done.
Counting objects: 100% (9/9), done.
Delta compression using up to 4 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 1.16 KiB | 1.17 MiB/s, done.
Total 5 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To github.com-bullyrook:bullyrooks/cloud_application.git
   fd00054..350e053  docker_build -> docker_build

$ git merge --squash docker_build
Updating cf0e328..350e053
Fast-forward
Squash commit -- not updating HEAD
 .github/workflows/features.yaml                    | 27 ++++++++++++++-
 .github/workflows/main.yaml                        | 38 +++++++++++++++++++++-
 Dockerfile                                         | 22 +++++++++++++
 .../controller/HelloWorldController.java           |  1 -
 4 files changed, 85 insertions(+), 3 deletions(-)
 create mode 100644 Dockerfile

$ git commit -m "docker build pipeline"
[main f4e91b4] docker build pipeline
 4 files changed, 85 insertions(+), 3 deletions(-)
 create mode 100644 Dockerfile

$ git push
Enumerating objects: 28, done.
Counting objects: 100% (28/28), done.
Delta compression using up to 4 threads
Compressing objects: 100% (11/11), done.
Writing objects: 100% (15/15), 2.19 KiB | 2.19 MiB/s, done.
Total 15 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 2 local objects.
To github.com-bullyrook:bullyrooks/cloud_application.git
   cf0e328..f4e91b4  main -> main

Versioning

Switch back over to your github actions and watch the main pipeline build. You should see something like this in your build logs:

#18 DONE 15.8s
Setting outputs
  digest=sha256:d75d79c29e3975afec55478aa44f34f3c986da53d18d3afda4671c953699ef6c
  metadata={
    "containerimage.config.digest": "sha256:773003bc0ecd2b1fce4174b048664c06c4a942aa0ca2ff639aebad5978ecf823",
    "containerimage.digest": "sha256:d75d79c29e3975afec55478aa44f34f3c986da53d18d3afda4671c953699ef6c",
    "image.name": "cloud.canister.io:5000/***/cloud_application:0.0.1"
  }

And you can see that the version was calculated (defaults to 0.0.1 when no previous version is available) and pushed up an image tagged with that version.

You can confirm this in git

$ git pull
remote: Enumerating objects: 1, done.
remote: Counting objects: 100% (1/1), done.
remote: Total 1 (delta 0), reused 1 (delta 0), pack-reused 0
Unpacking objects: 100% (1/1), 142 bytes | 47.00 KiB/s, done.
From github.com-bullyrook:bullyrooks/cloud_application
 * [new tag]         v0.0.1     -> v0.0.1
Already up to date.

$ git tag 
v0.0.1

And you can push up a new major version

$ git tag -a v1.0.0 -m "pushing major version"

$ git push origin v1.0.0
Enumerating objects: 1, done.
Counting objects: 100% (1/1), done.
Writing objects: 100% (1/1), 164 bytes | 164.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0), pack-reused 0
To github.com-bullyrook:bullyrooks/cloud_application.git
 * [new tag]         v1.0.0 -> v1.0.0

make a minor change (a newline will work) and push that up as well

$ git add .

$ git commit -m "no op change"
[main a060e27] no op change
 1 file changed, 1 insertion(+)

$ git push
Enumerating objects: 19, done.
Counting objects: 100% (19/19), done.
Delta compression using up to 4 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (10/10), 749 bytes | 749.00 KiB/s, done.
Total 10 (delta 2), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To github.com-bullyrook:bullyrooks/cloud_application.git
   f4e91b4..a060e27  main -> main

And you’ll see that the version change was picked up and incorporated

Setting outputs
  digest=sha256:f69dc33471f047f3f60b7ecb0d1ab561dba3e95ef30e7be655777e9d507079f1
  metadata={
    "containerimage.config.digest": "sha256:d77cb98a1e750d2a6a8aa3cad1858ab04eb1c37e6dfb19ed327fdaa34c0c7dea",
    "containerimage.digest": "sha256:f69dc33471f047f3f60b7ecb0d1ab561dba3e95ef30e7be655777e9d507079f1",
    "image.name": "cloud.canister.io:5000/***/cloud_application:1.0.1"
  }