Cloud Kube Pt2 | Setting Up a Datastore

Cloud Kube Pt2 | Setting Up a Datastore

Software Development

Kubernetes Application Hosted in the Cloud (Part 2)

full course
  1. Cloud Kube Pt2 | Setting Up a Datastore

    written by Bullyrook on January 23, 2022

  2. Kube Cloud Pt2 | Automated Testing with TestContainers

    written by Bullyrook on January 23, 2022

  3. Kube Cloud Pt2 | Database Configuration in Kubernetes

    written by Bullyrook on January 23, 2022

The first part of these courses was all about setting up an build and deploy pipeline so that we could automate building a helm chart and deploying to our cloud hosted environment at Okteto.

In this course we’re going to start adding more functionality so that we can demonstrate best practices around testing, logging, messaging and tracing.

Updates

At this point, you may need to make some upates in order to get mongodb working with springboot. It appears that you need java sdk 13.0.2 (at least) in order to get spring boot to communicate with mongodb. You’ll also need to update your intellij and dockerfile as well. Here’s the new dockerfile configuration

# syntax=docker/dockerfile:experimental
FROM openjdk:15.0.2-slim-buster as build

COPY .mvn .mvn
COPY mvnw .
COPY pom.xml .

# download dependencies
RUN ./mvnw dependency:go-offline

COPY src src

# Setup the maven cache
RUN --mount=type=cache,target=/root/.m2,rw ./mvnw -B package -DskipTests

FROM openjdk:15.0.2-slim-buster
COPY --from=build target/*.jar app.jar
ENTRYPOINT ["java", "-jar", "app.jar"]

I updated all the way up to 15.0.2, but I only use java 11 features so the maven pom doesn’t need to be updated.

You can see that I also added a layer to retrieve dependencies in offline mode. I am not sure this helps because the cache should already do this, but doesn’t hurt.

Additionally, I didn’t like the image name as cloud_application instead of cloud-application. So I updated the github workflows to use my new name. If you make this change, you’ll need to update your helm values.yaml to pull the correct image down (and create the repo in canister.io with the preferred name)

main.yml

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v3
        with:
          images: cloud.canister.io:5000/bullyrooks/cloud-application
          #setting value manually, but could come from git tag
          tags: |
            type=ref,event=tag
...
      - name: Build and push
        id: docker_build
        uses: docker/build-push-action@v2
        with:
          context: .
          push: true
          tags: |
            cloud.canister.io:5000/bullyrooks/cloud-application:${{ env.VERSION }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-new
...
      - name: Deploy
        uses: WyriHaximus/github-action-helm3@v2
        with:
          exec: |
            helm repo add bullyrooks https://bullyrooks.github.io/helm-charts/
            helm repo update
            helm upgrade cloud-application bullyrooks/cloud-application --install
          kubeconfig: '${{ secrets.KUBECONFIG }}'

values.yml

image:
  repository: cloud.canister.io:5000/bullyrooks/cloud-application
  pullPolicy: Always

Create a Datastore

We’re going to setup a free mongodb cloud instance. Go to the mongodb site and create an account. As you verify and create your account, make sure that you select Shared cloud database.

Choose AWS, use-east-1 and the rest of the defaults. I change the cluster name to bullyrooks.

For Security, use username and password. Choose a username and password and click Create User

For connection, leave My Local Environment and click Add My Current IP Address. Also, add 0.0.0.0/0 (should allow all access), which will be needed for okteto to connect. Then hit Finish and Close. You’ll have to wait for the cluster to be created. If you click on Connect, choose Connect your Application, then change Driver to Java and Version to 4.3 or later, you’ll see the connection string that you’ll need to connect to this instance.

Create the Repository Code

Checkout a new branch from main

git checkout -b database

Add the mongodb database starter to your pom.xml in the dependencies section

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>

Create these new packages under com.bullyrooks.cloud_application 

com.bullyrooks.cloud_application.repository
com.bullyrooks.cloud_application.repository.document
com.bullyrooks.cloud_application.repository.mapper

Create a new class under document called MessageDocument. Add this content

package com.bullyrooks.cloud_application.repository.document;

import lombok.Data;
import org.springframework.data.annotation.Id;
import org.springframework.data.mongodb.core.mapping.Document;

@Document("messages")
@Data
public class MessageDocument {
    @Id
    private String messageId;
    private String firstName;
    private String lastName;
    private String message;
}

Now create an interface called MessageRepository under com.bullyrooks.cloud_application.repository. Add this content

package com.bullyrooks.cloud_application.repository;

import com.bullyrooks.cloud_application.repository.document.MessageDocument;
import org.springframework.data.mongodb.repository.MongoRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface MessageRepository extends MongoRepository<MessageDocument, String> {

}

Create the Controller Code

We’re going to be using mapstruct again to help create the mappers needed to support a ports and adapters design. Add this to your pom.xml
/code

    <properties>
        <java.version>11</java.version>
        <org.mapstruct.version>1.4.1.Final</org.mapstruct.version>
    </properties>

...

<!-- mapstruct -->
<dependency>
    <groupId>org.mapstruct</groupId>
    <artifactId>mapstruct</artifactId>
    <version>${org.mapstruct.version}</version>
</dependency>
<dependency>
    <groupId>org.mapstruct</groupId>
    <artifactId>mapstruct-processor</artifactId>
    <version>${org.mapstruct.version}</version>
</dependency>

Create a dto package in the controller package and put two classes in there.

package com.bullyrooks.cloud_application.controller.dto;

import lombok.Data;

@Data
public class CreateMessageRequestDTO {
    private String firstName;
    private String lastName;
    private String message;
}

package com.bullyrooks.cloud_application.controller.dto;

import lombok.Data;

@Data
public class CreateMessageResponseDTO {
    private String firstName;
    private String lastName;
    private String message;
}

Create a package called mapper under controller and put these two classes there

package com.bullyrooks.cloud_application.controller.mapper;

import com.bullyrooks.cloud_application.controller.dto.CreateMessageRequestDTO;
import com.bullyrooks.cloud_application.service.model.Message;
import org.mapstruct.Mapper;
import org.mapstruct.factory.Mappers;

@Mapper
public interface CreateMessageRequestDTOMapper {
    CreateMessageRequestDTOMapper INSTANCE = Mappers.getMapper(CreateMessageRequestDTOMapper.class);

    Message dtoToModel(CreateMessageRequestDTO userAccountEntity);

}

package com.bullyrooks.cloud_application.controller.mapper;

import com.bullyrooks.cloud_application.controller.dto.CreateMessageRequestDTO;
import com.bullyrooks.cloud_application.controller.dto.CreateMessageResponseDTO;
import com.bullyrooks.cloud_application.service.model.Message;
import org.mapstruct.Mapper;
import org.mapstruct.factory.Mappers;

@Mapper
public interface CreateMessageResponseDTOMapper {
    CreateMessageResponseDTOMapper INSTANCE = Mappers.getMapper(CreateMessageResponseDTOMapper.class);

    CreateMessageResponseDTO modelToDTO(Message message);

}

Create a new class under controller and put this code there

package com.bullyrooks.cloud_application.controller;

import com.bullyrooks.cloud_application.controller.dto.CreateMessageRequestDTO;
import com.bullyrooks.cloud_application.controller.dto.CreateMessageResponseDTO;
import com.bullyrooks.cloud_application.controller.dto.HelloWorldResponse;
import com.bullyrooks.cloud_application.controller.mapper.CreateMessageRequestDTOMapper;
import com.bullyrooks.cloud_application.controller.mapper.CreateMessageResponseDTOMapper;
import com.bullyrooks.cloud_application.service.MessageService;
import com.bullyrooks.cloud_application.service.model.Message;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
@Slf4j
public class MessageController {

    @Autowired
    MessageService messageService;

    @PostMapping("/message")
    public CreateMessageResponseDTO createMessage(@RequestBody CreateMessageRequestDTO request){
        log.info("createMessage : {}", request);
        Message message = CreateMessageRequestDTOMapper.INSTANCE.dtoToModel(request);
        Message response = messageService.saveMessage(message);
        return CreateMessageResponseDTOMapper.INSTANCE.modelToDTO(response);
    }

}

What we’re doing here is defining a request and response data transformation object (dto) and mapping the dto into our internal representation to align with ports and adapters design.

Create the Service Code

Make this package:

com.bullyrooks.cloud_application.service.model

In the model package create this class

package com.bullyrooks.cloud_application.service.model;

import lombok.Data;

@Data
public class Message {
    private String messageId;
    private String firstName;
    private String lastName;
    private String message;
}

Create this class in the service package

package com.bullyrooks.cloud_application.service;

import com.bullyrooks.cloud_application.repository.MessageRepository;
import com.bullyrooks.cloud_application.repository.document.MessageDocument;
import com.bullyrooks.cloud_application.repository.mapper.MessageDocumentMapper;
import com.bullyrooks.cloud_application.service.model.Message;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
@Slf4j
public class MessageService {

    @Autowired
    MessageRepository messageRepository;

    public Message saveMessage(Message message){
        MessageDocument msgDoc = MessageDocumentMapper.INSTANCE.modelToDocument(message);
        log.info("saving document: {}", msgDoc);
        MessageDocument returnDoc = messageRepository.save(msgDoc);
        return MessageDocumentMapper.INSTANCE.documentToModel(returnDoc);
    }
}

Here we’re creating our actual service logic, but we’ll need to also create our mappers to the documents

Create this class in a package called com.bullyrooks.cloud_application.repository.mapper

package com.bullyrooks.cloud_application.repository.mapper;

import com.bullyrooks.cloud_application.repository.document.MessageDocument;
import com.bullyrooks.cloud_application.service.model.Message;
import org.mapstruct.Mapper;
import org.mapstruct.factory.Mappers;

@Mapper
public interface MessageDocumentMapper {
    MessageDocumentMapper INSTANCE = Mappers.getMapper(MessageDocumentMapper.class);

    MessageDocument modelToDocument(Message model);

    Message documentToModel(MessageDocument returnDoc);
}

Set Configuration

Create an application.yml file and add this code (or whatever matches your database configuration)

spring:
  application:
    name: cloud_application
  data:
    mongodb:
      uri: mongodb+srv://bullyrooks:${mongodb.password}@bullyrooks.4zqpz.mongodb.net/bullyrooks?retryWrites=true&w=majority

And update your application run configuration in Intellij to insert your password.

you will also need -Djdk.tls.client.protocols=TLSv1.2 to connect to mongodb

Start up your application. You should see it start and connect to mongodb

: Tomcat started on port(s): 8080 (http) with context path ''
2022-01-17 14:51:12.388  INFO 23840 --- [  restartedMain] c.b.cloud_application.CloudApplication   : Started CloudApplication in 2.109 seconds (JVM running for 2.993)
2022-01-17 14:51:12.618  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:4, serverValue:305244}] to bullyrooks-shard-00-01.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.618  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:5, serverValue:7709}] to bullyrooks-shard-00-02.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.618  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:3, serverValue:305292}] to bullyrooks-shard-00-01.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.618  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:2, serverValue:300367}] to bullyrooks-shard-00-00.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.618  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:1, serverValue:300367}] to bullyrooks-shard-00-00.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.618  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.connection            : Opened connection [connectionId{localValue:6, serverValue:7709}] to bullyrooks-shard-00-02.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.619  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.cluster               : Monitor thread successfully connected to server with description ServerDescription{address=bullyrooks-shard-00-02.4zqpz.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=270532600, setName='atlas-x20br4-shard-0', canonicalAddress=bullyrooks-shard-00-02.4zqpz.mongodb.net:27017, hosts=[bullyrooks-shard-00-02.4zqpz.mongodb.net:27017, bullyrooks-shard-00-01.4zqpz.mongodb.net:27017, bullyrooks-shard-00-00.4zqpz.mongodb.net:27017], passives=[], arbiters=[], primary='bullyrooks-shard-00-01.4zqpz.mongodb.net:27017', tagSet=TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='US_EAST_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=10, topologyVersion=TopologyVersion{processId=61e5b12ff56b6bdfb1dc18fa, counter=3}, lastWriteDate=Mon Jan 17 14:51:12 MST 2022, lastUpdateTimeNanos=330476231174200}
2022-01-17 14:51:12.621  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.cluster               : Monitor thread successfully connected to server with description ServerDescription{address=bullyrooks-shard-00-01.4zqpz.mongodb.net:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=279366100, setName='atlas-x20br4-shard-0', canonicalAddress=bullyrooks-shard-00-01.4zqpz.mongodb.net:27017, hosts=[bullyrooks-shard-00-02.4zqpz.mongodb.net:27017, bullyrooks-shard-00-01.4zqpz.mongodb.net:27017, bullyrooks-shard-00-00.4zqpz.mongodb.net:27017], passives=[], arbiters=[], primary='bullyrooks-shard-00-01.4zqpz.mongodb.net:27017', tagSet=TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='US_EAST_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=7fffffff00000000000000cc, setVersion=10, topologyVersion=TopologyVersion{processId=61dc74fe739b2f0c61413767, counter=13}, lastWriteDate=Mon Jan 17 14:51:12 MST 2022, lastUpdateTimeNanos=330476231480400}
2022-01-17 14:51:12.621  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.cluster               : Monitor thread successfully connected to server with description ServerDescription{address=bullyrooks-shard-00-00.4zqpz.mongodb.net:27017, type=REPLICA_SET_SECONDARY, state=CONNECTED, ok=true, minWireVersion=0, maxWireVersion=9, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=30, roundTripTimeNanos=278135600, setName='atlas-x20br4-shard-0', canonicalAddress=bullyrooks-shard-00-00.4zqpz.mongodb.net:27017, hosts=[bullyrooks-shard-00-02.4zqpz.mongodb.net:27017, bullyrooks-shard-00-01.4zqpz.mongodb.net:27017, bullyrooks-shard-00-00.4zqpz.mongodb.net:27017], passives=[], arbiters=[], primary='bullyrooks-shard-00-01.4zqpz.mongodb.net:27017', tagSet=TagSet{[Tag{name='nodeType', value='ELECTABLE'}, Tag{name='provider', value='AWS'}, Tag{name='region', value='US_EAST_1'}, Tag{name='workloadType', value='OPERATIONAL'}]}, electionId=null, setVersion=10, topologyVersion=TopologyVersion{processId=61dc74e45b583b963580aabc, counter=14}, lastWriteDate=Mon Jan 17 14:51:12 MST 2022, lastUpdateTimeNanos=330476230523800}
2022-01-17 14:51:12.624  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.cluster               : Setting max election id to 7fffffff00000000000000cc from replica set primary bullyrooks-shard-00-01.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.625  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.cluster               : Setting max set version to 10 from replica set primary bullyrooks-shard-00-01.4zqpz.mongodb.net:27017
2022-01-17 14:51:12.625  INFO 23840 --- [ngodb.net:27017] org.mongodb.driver.cluster               : Discovered replica set primary bullyrooks-shard-00-01.4zqpz.mongodb.net:27017

Test and Verification

Now head over to postman and hit the service endpoint. Just manual testing for now.

You can also log into mongo and see the record that was created

Save to Git

We’re in a good spot. Save to git and push to remote

$ git add .
warning: LF will be replaced by CRLF in pom.xml.
The file will have its original line endings in your working directory

$ git commit -m "message endpoint"
[database f6b7f59] message endpoint
 12 files changed, 175 insertions(+)
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/controller/MessageController.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/controller/dto/CreateMessageRequestDTO.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/controller/dto/CreateMessageResponseDTO.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/controller/mapper/CreateMessageRequestDTOMapper.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/controller/mapper/CreateMessageResponseDTOMapper.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/repository/MessageRepository.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/repository/mapper/MessageDocumentMapper.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/service/MessageService.java
 create mode 100644 src/main/java/com/bullyrooks/cloud_application/service/model/Message.java
 create mode 100644 src/main/resources/application.yml

$ git push
Enumerating objects: 43, done.
Counting objects: 100% (43/43), done.
Delta compression using up to 4 threads
Compressing objects: 100% (24/24), done.
Writing objects: 100% (29/29), 3.71 KiB | 950.00 KiB/s, done.
Total 29 (delta 6), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (6/6), completed with 2 local objects.
To github.com-bullyrook:bullyrooks/cloud_application.git
   e24001a..f6b7f59  database -> database

0 comments on “Cloud Kube Pt2 | Setting Up a Datastore”Add yours →

Leave a Reply

Your email address will not be published. Required fields are marked *