Automated Terraform Deploy Using Github Actions

Now we just need to build and deploy our application automatically so that whenever we push changes to the main branch, they’ll automatically become available through our endpoint.

Create Our Artifact Storage

We’re going to use a ‘poor man’s artifactory’ to store our compiled .jar files. With our current build process our .jar files are lost every time we rebuild. This might not be a good idea in case we need to rollback. We’ll use an S3 bucket to store the artifacts. And we’ll create that manually.

First, create a generic S3 bucket. Mine’s called bullyrooks-zipped-artifacts

Update the Terraform to Use the Artifact

Let’s start a new branch

$ git checkout -b "deploy-workflow"

Now update our terraform to use the artifact from the bucket

Add variables to variables.tf

variable "s3_artifact_bucket" {
  default = "bullyrooks-zipped-artifacts"
}
#this key should never exist
variable "s3_artifact_key" {
  default = "bullyrooks/helloworld-lambda/helloworld-lambda-0.0.1-SNAPSHOT-aws.jar"
}

and update the main.tf to pull the artifact from the bucket

resource "aws_lambda_function" "tf-helloWorld" {
  function_name = "helloWorld-lambda"
  s3_bucket = var.s3_artifact_bucket
  s3_key = var.s3_artifact_key
  role = aws_iam_role.iam_for_helloWorld_lambda.arn
  handler = "org.springframework.cloud.function.adapter.aws.SpringBootApiGatewayRequestHandler::handleRequest"
  memory_size = 512
  timeout = 15

  runtime = "java11"

  depends_on = [
    aws_iam_role_policy_attachment.helloWorld-log-attach,
    aws_cloudwatch_log_group.helloWorld-logs,
  ]

  environment {
    variables = {
      FUNCTION_NAME = "apiFunction"
    }
  }
}

Create the Github Action

In your project root, create a directory called .github/workflows and add a file there called main.yml with this content:

# This is a basic workflow to help you get started with Actions

name: helloworld-lambda-cicd

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the develop branch
on:
  push:
    branches: [ main ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    timeout-minutes: 30

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2
      - name: Set up JDK 11
        uses: actions/setup-java@v2
        with:
          java-version: '11'
          distribution: 'adopt'
      - name: Cache Maven packages
        uses: actions/cache@v2
        with:
          path: ~/.m2
          key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
          restore-keys: ${{ runner.os }}-m2
      - name: Build with Maven
        run: mvn --batch-mode --update-snapshots package
      - name: Upload shaded jar
        uses: actions/upload-artifact@v2
        with:
          name: jar
          path: target/helloworld-lambda-0.0.1-SNAPSHOT-aws.jar
  upload:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Make artifact directory
        run: mkdir -p ./artifacts/${{ github.repository }}

      - name: Download jar
        uses: actions/download-artifact@v2
        with:
          name: jar
          path: artifacts/${{ github.repository }}

      - name: Display structure of downloaded files
        run: ls -R
        working-directory: artifacts/${{ github.repository }}

      - name: Rename jar
        run: |
          mv artifacts/${{ github.repository }}/helloworld-lambda-0.0.1-SNAPSHOT-aws.jar artifacts/${{ github.repository }}/${{ github.sha }}.jar

      - name: Push Zip to S3
        uses: jakejarvis/[email protected]
        env:
          SOURCE_DIR: './artifacts'
          AWS_REGION: 'us-east-1'
          AWS_S3_BUCKET: ${{ secrets.AWS_BUCKET_NAME }}
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  deploy:
    runs-on: ubuntu-latest
    needs: upload
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Install Terraform
        env:
          TERRAFORM_VERSION: "1.0.2"
        run: |
          tf_version=$TERRAFORM_VERSION
          wget https://releases.hashicorp.com/terraform/"$tf_version"/terraform_"$tf_version"_linux_amd64.zip
          unzip terraform_"$tf_version"_linux_amd64.zip
          sudo mv terraform /usr/local/bin/
      - name: Verify Terraform version
        run: terraform --version
        working-directory: tf
      - name: Terraform Init
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        run: terraform init -input=false
        working-directory: tf
      - name: Terraform Apply
        env:
          AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
          AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          TF_VAR_s3_artifact_bucket: ${{ secrets.AWS_BUCKET_NAME }}
          TF_VAR_s3_artifact_key: ${{ github.repository }}/${{ github.sha }}.jar
        run: terraform apply -auto-approve -input=false
        working-directory: tf

I’ll walk you through the code:

Setup Secrets

As you can see, we’re using a bunch of secrets which we’ll need to setup now. Log into your Github account, navigate to the project repository and click on Settings

Open the Secrets tab and click on New repository secret

We need to make 3 secrets AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_BUCKET_NAME

AWS_BUCKET_NAME will be the name of the bucket you created (bullyrooks-zipped-artifacts) and access and secret key will be from your credentials file.

Commit and Merge

$ git status
$ git add .
$ git commit -m "build and deploy"
$ git push --set-upstream origin deploy-workflow
$ git checkout main
$ git merge deploy-workflow
$ git push

Review the Actions

This should have kicked off a workflow

Eventually, it should succeed (I had to make a few changes to get it working correctly)

Now go grab the API Gateway URL from the lambda configuration and confirm this new endpoint works