Kube Cloud Pt6 | Synchronous Contract Testing Conclusion

This course absolutely warrants a final thoughts page. Contract testing can be quite powerful, but it has to be done correctly and that means that you need quite a bit of engineering experience not only from a software development side, but also from an operations CICD side as well.

First the downsides

Support

Pact has been in active development for awhile. Its implementations are sprawling. There are several libraries that have overlapping functionality that are in various states of maturity. The documentation is okay, but its not clear what best practices are. Additionally, there are loads of proof of concept demonstrations available online (and I’m probably adding to the noise) with differing levels of ‘good’ implementation and depth. A lot of the demos out there are out of date and don’t work well with the more recently libraries. This has safely been the most time I’ve spent (2 weekends) trying to get something working and documented.

I’ve worked with the pact team before and they’re knowledgeable and supportive, but this project feels like its a little of ‘everything for everyone’: lots of configuration but little in the way of convention (defined best practices and the framework guiding you to them).

Engineering Maturity

If you work in an organization where the CICD pipeline is owned by another team (usually called devOps, but there’s no interest in software development), you’re going to struggle being successful implementing contract testing with pact. I had to backup and rethink my workflow a few times. If you have siloed software engineering from operations, you’re going to get frustrated and it wouldn’t surprise me if an operations team that is working with engineergs trying to implement pact says something like ‘go off and figure it out and then tell us what you want to do when you have a plan’.

It didn’t help that pact’s guidance (tags vs environments and releases) has evolved as well and some of the tools support their new model (stick with the docker cli) and some tools don’t (maven plugin).

I literally kicked off probably 100 github action workflows tinkering with the pipelines working on this.

Production Readiness

You will probably get to a point with pact that you are literally blocked. You can’t deploy a consumer because the provider doesn’t support the contract and the provider can’t deploy because it can’t fulfill a contract. Will you have the experience to untie that knot? Will you have to ‘delete the integration’ and start over (I’ll admit, I did it a few times)? Will you have people in your team supporting you (devOps) or people demanding that you tear down contract testing (usually engineering leadership)? Can you slowly roll it out and harden process against the edge cases? Will you be able to implement rollback? If not, you could bring your engineering organization to a standstill.

Now the positive side

Proactive vs Reactive

I prefer contract testing so that we can prevent issues in production before we have to react to them. If I can prevent a team from delivering a breaking change and also inform them of why its a breaking change and which service it impacts, I can then point them to the right people to talk to to see how to fix the issue.

This actually saves time in the long run and the costs of multi team war rooms trying to figure out why we’re losing money by the minute. Those get expensive really fast.

Improved Communication

Its a little counterintuitive since a system is sitting in the middle of your CICD pipeline ‘replacing’ a conversation that should have happened. However, in my experience software teams (especially those empowered by microservice development to release early and often) are not doing the due diligence they should have been doing to understand systemic impacts of service releases.

Pact will probably trigger conversations about service governance best practices like

• What is a breaking change
• Semantic versioning
• Dependency mapping
• Service level objectives
• The purpose of your microservice (put an end to people making your service do more than it was intended to do)

Reduced Dependency on End to End Testing

The entire goal of contract testing is to make sure that if your services meet their defined specifications then everything ‘should work’. The only way to ‘prove’ this before was to write an end to end test that did just that. However, end to end test suites begin to balloon and eventually become monsters of their own within your organization.

Contract testing may slow down an individual build and slightly increase build dependencies (the webhook is an example), but this is peanuts compared to multi-hour end to end tests that eat up your acceptance testing environments. If I had to make the choice, I would choose contract tests.

When Should you use Contract Testing

Pact has thoughts on this. So do I.

Use contract testing sparingly. Use it when you have externally available services (write your own consumer contracts and confirm that you don’t break them). Use them when you have service interactions already in place (don’t start with pact, unless you’re an expert). Don’t use them for intra-domain service interactions, use them for inter-domain service interactions (usually interfaces that are used between teams).

Pact will (slightly) slow you down in the short term, so don’t do this to yourself when you’re writing greenfield services that are all within the same domain. Its a very good tool to use when you want to make sure that you don’t break something that will be noticed by someone on another team.