Continuous Testing with Microservices (with Examples)

When is Microservice Architecture the Way to Go?

Choosing and designing the correct architecture for a system is critical. One must ensure the quality of service requirements and the handling of non-functional requirements, such as maintainability, extensibility, and testability.

Microservice architecture is quite a recurrent choice in the latest ecosystems after companies adopted Agile and DevOps. While not being a de facto choice, when dealing with systems that are extensively growing and where a monolith architecture is no longer feasible to maintain, it is one of the preferred options. Keeping components service-oriented and loosely coupled allows continuous development and release cycles ongoing. This drives businesses to constantly test and upgrade their software.

The main prerequisites that call for such an architecture are:

Domain-Driven Design
Continuous Delivery and DevOps Culture
Failure Isolation
Decentralization

It has the following benefits:

Team ownership
Frequent releases
Easier maintenance
Easier upgrades to newer versions
Technology agnostic

It has the following cons:

microservice-to-microservice communication mechanisms
Increasing the number of services increases the overall system complexity

The more distributed and complex the architecture is, the more challenging it is to ensure that the system can be expanded and maintained while controlling cost and risk. One business transaction might involve multiple combinations of protocols and technologies. It is not just about the use cases but also about its operations. When adopting Agile and DevOps approaches, one should find a balance between flexibility versus functionality aiming to achieve continuous revision and testing.

The Importance of Testing Strategies in Relation to Microservices

Adopting DevOps in an organization aims to eliminate the various isolated departments and move towards one overall team. This move seeks to specifically improve the relationships and processes between the software team and the operations team. Delivering at a faster rate also means ensuring that there is continuous testing as part of the software delivery pipeline. Deploying daily (and in some cases even every couple of hours) is one of the main targets for fast end-to-end business solution delivery. Reliability and security must be kept in mind here, and this is where testing comes in.

The inclusion of test-driven development is the only way to achieve genuine confidence that code is production-ready. Valid test cases add value to the system since they validate and document the system itself. Apart from that, good code coverage encourages improvements and assists during refactoring.

Microservices architecture decentralizes communication channels, which makes testing more complicated. It’s not an insurmountable problem. A team owning a microservice should not be afraid to introduce changes because they might break existing client applications. Manual testing is very inefficient, considering that continuous integration and continuous deployment is the current best practice. DevOps engineers should ensure to include automation tests in their development workflow: write tests, add/refactor code, and run tests.

Common Microservice Testing Methods

The test pyramid is an easy concept that helps us identify the effort required when writing tests, and where the number of tests should decrease if granularity decreases. It also applies when considering continuous testing for microservices.

To make the topic more concrete, we will tackle the testing of a sample microservice using Spring Boot and Java. Microservice architectures, by construct, are more complicated than monolithic architecture. Nonetheless, we will keep the focus on the type of tests and not on the architecture. Our snippets are based on a minimal project composed of one API-driven microservice owning a data store using MongoDB.

Unit tests

Unit tests should be the majority of tests since they are fast, reliable, and easy to maintain. These tests are also called white-box tests. The engineer implementing them is familiar with the logic and is writing the test to validate the module specifications and check the quality of code.

The focus of these tests is a small part of the system in isolation, i.e., the Class Under Test (CUT). The Single Responsibility Principle is a good guideline on how to manage code relating to functionality.

The most common form of a unit test is a “solitary unit test.” It does not cross any boundaries and does not need any collaborators apart from the CUT.

As outlined by Bill Caputo, databases, messaging channels, or other systems are the boundaries; any additional class used or required by the CUT is a collaborator. A unit test should never cross a boundary. When making use of collaborators, one is writing a “sociable unit tests.” Using mocks for dependencies used by the CUT is a way to test sociable code with a “solitary unit test.”

In traditional software development models, developer testing was not yet wildly adopted, having to test completely off-sync from development. Achieving a high code coverage rating was considered a key indicator of test suite confidence.

With the introduction of Agile and short iterative cycles, it’s evident now that previous test models no longer work. Frequent changes are expected continuously. It is much more critical to test observable behavior rather than having all code paths covered. Unit tests should be more about assertions than code coverage because the aim is to verify that the logic is working as expected.

It is useless to have a component with loads of tests and a high percentage of code coverage when tests do not have proper assertions. Applying a more Behavior-Driven Development (BDD) approach ensures that tests are verifying the end state and that the behavior matches the requirements set by the business. The advantage of having focused tests with a well-defined scope is that it becomes easier to identify the cause of failure. BDD tests give us higher confidence that failure was a consequence of a change in feature behavior. Tests that otherwise focus more on code coverage cannot offer much confidence since there would be a higher risk that failure is a repercussion for changes done in the tests themselves due to code paths implementation details.

Tests should follow Martin Fowler’s suggestion when he stated the following (in Refactoring: Improving the Design of Existing Code, Second Edition. Kent Beck, and Martin Fowler. Addison-Wesley. 2018):

Another reason to focus less on minor implementation details is refactoring. During refactoring, unit tests should be there to give us confidence and not slow down work. A change in the implementation of the collaborator might result in a test failure, which might make tests harder to maintain. It is highly recommended to keep a minimum of sociable unit tests. This is especially true when such tests might slow down the development life cycle with the possibility that tests end up ignored. An excellent situation to include a sociable unit test is negative testing, especially when dealing with behavior verification.

Integration tests

One of the most significant challenges with microservices is testing their interaction with the rest of the infrastructure services, i.e., the boundaries that the particular CUT depends on, such as databases or other services. The test pyramid clearly shows that integration tests should be less than unit tests but more than component and end-to-end tests. These other types of tests might be slower, harder to write, and maintain, and be quite fragile when compared to unit tests. Crossing boundaries might have an impact on performance and execution time due to network and database access; still, they are indispensable, especially in the DevOps culture.

In a Continuous Deployment scope, narrow integration tests are favored instead of broad integration tests. The latter is very close to end-to-end tests where it requires the actual service running rather than the use of a test double of those services to test the code interactions. The main goal to achieve is to build manageable operative tests in a fast, easy, and resilient fashion. Integration tests focus on the interaction of the CUT to one service at a time. Our focus is on narrow integration tests. Verification of the interaction between a pair of services can be confirmed to be as expected, where services can be either an infrastructure service or any other service.

Persistence tests

A controversial type of test is when testing the persistence layer, with the primary aim to test the queries and the effect on test data. One option is the use of in-memory databases. Some might consider the use of in-memory databases as a sociable unit test since it is a self-contained test, idempotent, and fast. The test runs against the database created with the desired configuration. After the test runs and assertions are verified, the data store is automatically scrubbed once the JVM exits due to its ephemeral nature. Keep in mind that there is still a connection happening to a different service and is considered a narrow integration test. In a Test-Driven Development (TDD) approach, such tests are essential since test suites should run within seconds. In-memory databases are a valid trade-off to ensure that tests are kept as fast as possible and not ignored in the long run.

@Before
public void setup() throws Exception {
   try {
	// this will download the version of mongo marked as production. One should
	// always mention the version that is currently being used by the SUT
	String ip = "localhost";
	int port = 27017;

	IMongodConfig mongodConfig = new MongodConfigBuilder().version(Version.Main. PRODUCTION)
		.net(new Net(ip, port, Network.localhostIsIPv6())).build();

	MongodStarter starter = MongodStarter.getDefaultInstance();
mongodExecutable = starter.prepare(mongodConfig);
	mongodExecutable.start();

   } catch (IOException e) {
	e.printStackTrace();
   }
}

Snippet 1: Installation and startup of the In-memory MongoDB

The above is not a full integration test since an in-memory database does not behave exactly as the production database server. Therefore, it is not a replica for the “real” mongo server, which would be the case if one opts for broad integration tests.

Another option for persistence integration tests is to have broad tests running connected to an actual database server or with the use of containers. Containers ease the pain since, on request, one provisions the database, compared to having a fixed server. Keep in mind such tests are time-consuming, and categorizing tests is a possible solution. Since these tests depend on another service running apart from the CUT, it’s considered a system test. These tests are still essential, and by using categories, one can better determine when specific tests should run to get the best balance between cost and value. For example, during the development cycle, one might run only the narrow integration tests using the in-memory database. Nightly builds could also run tests falling under a category such as broad integration tests.

@Category(FastIntegration.class)
@RunWith(SpringRunner.class)
@DataMongoTest
public class DailyTaskRepositoryInMemoryIntegrationTest {
	. . . 
}

@Category(SlowIntegration.class)
@RunWith(SpringRunner.class)
@DataMongoTest(excludeAutoConfiguration = EmbeddedMongoAutoConfiguration.class)
public class DailyTaskRepositoryIntegrationTest {
   ...
}

Snippet 2: Using categories to differentiate the types of integration tests

Consumer-driven tests

Inter-Process Communication (IPC) mechanisms are one central aspect of distributed systems based on a microservices architecture. This setup raises various complications during the creation of test suites. In addition to that, in an Agile team, changes are continuously in progress, including changes in APIs or events. No matter which IPC mechanism the system is using, there is the presence of a contract between any two services. There are various types of contracts, depending on which mechanism one chooses to use in the system. When using APIs, the contract is the HTTP request and response, while in the case of an event-based system, the contract is the domain event itself.

A primary goal when testing microservices is to ensure those contracts are well defined and stable at any point in time. In a TDD top-down approach, these are the first tests to be covered. A fundamental integration test ensures that the consumer has quick feedback as soon as a client does not match the real state of the producer to whom it is talking.

These tests should be part of the regular deployment pipeline. Their failure would allow the consumers to become aware that a change on the producer side has occurred, and that changes are required to achieve consistency again. Without the need to write intricate end-to-end tests, ‘consumer-driven contract testing’ would target this use case.

The following is a sample of a contract verifier generated by the spring-cloud-contract plugin.

@Test
public void validate_add_New_Task() throws Exception {
  // given:
   MockMvcRequestSpecification request = given()
.header("Content-Type", "application/json;charset=UTF-8")
	.body("{"taskName":"newTask","taskDescription":"newDescription","isComplete":false,"isUrgent":true}");

  // when:
   ResponseOptions response = given().spec(request).post("/tasks");

  // then:
   assertThat(response.statusCode()).isEqualTo(200);
   assertThat(response.header("Content-Type")).isEqualTo("application/json;charset=UTF-8");
  // and:
   DocumentContext parsedJson = JsonPath.parse(response.getBody().asString());
   assertThatJson(parsedJson).field("['taskName']").isEqualTo("newTask");
   assertThatJson(parsedJson).field("['isUrgent']").isEqualTo(true);
   assertThatJson(parsedJson).field("['isComplete']").isEqualTo(false);
   assertThatJson(parsedJson).field("['id']").isEqualTo("3");
   assertThatJson(parsedJson).field("['taskDescription']").isEqualTo("newDescription");
}

Snippet 3: Contract Verifier auto-generated by the spring-cloud-contract plugin

A BaseClass written in the producer is instructing what kind of response to expect on the various types of requests by using the standalone setup. The packaged collection of stubs is available to all consumers to be able to pull them in their implementation. Complexity arises when multiple consumers make use of the same contract; therefore, the producer needs to have a global view of the service contracts required.

@RunWith(SpringRunner.class)
@SpringBootTest
public class ContractBaseClass {

	@Autowired
	private DailyTaskController taskController;

	@MockBean
	private DailyTaskRepository dailyTaskRepository;

	@Before
	public void before() {
		RestAssuredMockMvc.standaloneSetup(this.taskController);
		Mockito.when(this.dailyTaskRepository.findById("1")).thenReturn(
Optional.of(new DailyTask("1", "Test", "Description", false, null)));
		
		. . . 
				
		Mockito.when(this.dailyTaskRepository.save(
new DailyTask(null, "newTask", "newDescription", false, true))).thenReturn(
new DailyTask("3", "newTask", "newDescription", false, true));
		
	}

Snippet 4: The producer’s BaseClass defining the response expected for each request

On the consumer side, with the inclusion of the spring-cloud-starter-contract-stub-runner dependency, we configured the test to use the stubs binary. This test would run using the stubs generated by the producer as per configuration having version specified or always the latest. The stub artifact links the client with the producer to ensure that both are working on the same contract. Any change that occurs would reflect in those tests, and thus, the consumer would identify whether the producer has changed or not.

@SpringBootTest(classes = TodayAskApplication.class)
@RunWith(SpringRunner.class)
@AutoConfigureStubRunner(ids = "com.cwie.arch:today:+:stubs:8080", stubsMode = StubRunnerProperties.StubsMode.LOCAL)
public class TodayClientStubTest {
 	 . . .
	@Test
	public void addTask_expectNewTaskResponse () {
		Task newTask = todayClient.createTask(
new Task(null, "newTask", "newDescription", false, true));
		BDDAssertions.then(newTask).isNotNull();
		BDDAssertions.then(newTask.getId()).isEqualTo("3");
		. . . 
		
	}
}

Snippet 5: Consumer injecting the stub version defined by the producer

Such integration tests verify that a provider’s API is still in line with the consumers’ expectations. When using mocked unit tests for APIs, we would have stubbed APIs and mocked the behavior. From a consumer point of view, these types of tests will ensure that the client is matching our expectations. It is essential to note that if the producer side changes the API, those tests will not fail. And it is imperative to define what the test is covering.

// the response we expect is represented in the task1.json file
private Resource taskOne = new ClassPathResource("task1.json");

@Autowired
private TodayClient todayClient;

@Test
public void createNewTask_expectTaskIsCreated() {
WireMock.stubFor(WireMock.post(WireMock.urlMatching("/tasks"))
		.willReturn(WireMock.aResponse()
		.withHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_UTF8_VALUE)
		.withStatus(HttpStatus.OK.value())
		.withBody(transformResourceJsonToString(taskOne))));
		
	Task tasks = todayClient.createTask(new Task(null, "runUTest", "Run Test", false, true));
	BDDAssertions.then(tasks.getId()).isEqualTo("1");

Snippet 6: A consumer test doing assertions on its own defined response

Component tests

Microservice architecture can grow fast, and so the component under test might be integrating with multiple other components and multiple infrastructure services. Until now, we have covered white-box testing with unit tests and narrow integration tests to test the CUT crossing the boundary to integrate with another service.

The fastest type of component testing is the in-process approach, where, alongside the use of test doubles and in-memory data stores, testing remains within boundaries. The main disadvantage of this approach is that the deployable production service is not fully tested; on the contrary, the component requires changes to wire the application differently. The preferred method is out-of-process component testing. These are like end-to-end tests, but with all external collaborators changed out with test doubles, by doing so, it exercises the fully deployed artifact making use of real network calls. The test would be responsible for properly configuring any externals services as stubs.

@Ignore
@RunWith(SpringRunner.class)
@SpringBootTest(classes = { TodayConfiguration.class, TodayIntegrationApplication.class,
   CloudFoundryClientConfiguration.class })
public class BaseFunctionalitySteps {

   @Autowired
   private CloudFoundryOperations cf;

   private static File manifest = new File(".manifest.yml");

   @Autowired
   private TodayClient client;

   // Any stubs required 
   . . . 

   public void setup() {

cf.applications().pushManifest(PushApplicationManifestRequest.builder() 
 
 .manifest(ApplicationManifestUtils.read(manifest.toPath()).get(0)).build()).block();
}
. . .
// Any calls required by tests
public void requestForAllTasks() {
this.client.getTodoTasks();
}
}

Snippet 7: Deployment of the manifest on CloudFoundry and any calls required by tests

Cloud Foundry is one of the options used for container-based testing architectures. “It is an open-source cloud application platform that makes it faster and easier to build, test, deploy, and scale applications.” The following is the manifest.yml, a file that defines the configuration of all applications in the system. This file is used to deploy the actual service in the production-ready format on the Pivotal organization’s space where the MongoDB service is already set up, matching the production version.

---
applications:
- name: today
  instances: 1
  path: ../today/target/today-0.0.1-SNAPSHOT.jar 
  memory: 1024M
  routes:
  - route: today.cfapps.io
  services:
  - mongo-it

Snippet 8: Deployment of one instance of the service depending on mongo service

When opting for the out-of-process approach, keep in mind that actual boundaries are under test, and thus, tests end up being slower since there are network and database interactions. It would be ideal to have those test suites written in a separate module. To be able to run them separately at a different maven stage instead of the usual ‘test’ phase.

Since the emphasis of the tests is on the component itself, tests cover the primary responsibilities of the component while purposefully neglecting any other part of the system.

Cucumber, a software tool that supports Behavior-Driven Development, is an option to define such behavioral tests. With its plain language parser, Gherkin, it ensures that customers can easily understand all tests described. The following Cucumber feature file is ensuring that our component implementation is matching the business requirements for that particular feature.

Feature: Tasks

Scenario: Retrieving one task from list
 Given the component is running
 And the data consists of one or more tasks
 When user requests for task x
 Then the correct task x is returned

Scenario: Retrieving all lists
 Given the data consists of one or more tasks
 When user requests for all tasks
 Then all tasks in database are returned

Scenario: Negative Test
 Given the component is not running
 When user requests for task x it fails with response 404

Snippet 9: A feature file defining BDD tests

End-to-end tests

Similar to component tests, the aim of these end-to-end tests is not to perform code coverage but to ensure that the system meets the business scenarios requested. The difference is that in end-to-end testing, all components are up and running during the test.

As per the testing pyramid diagram, the number of end-to-end tests decreases further, taking into consideration the slowness they might cause. The first step is to have the setup running, and for this example, we will be leveraging docker.

version: '3.7'
services:
    today-app:
        image: today-app:1
        container_name: "today-app"
        build:
          context: ./
          dockerfile: DockerFile
        environment:
           - SPRING_DATA_MONGODB_HOST=mongodb
        volumes:
          - /data/today-app
        ports:
          - "8082:8080"
        links:
          - mongodb
        depends_on:
          - mongodb

    mongodb:
        image: mongo:3.2
        container_name: "mongodb"
        restart: always
        environment:
           - AUTH=no
           - MONGO_DATA_DIR=/data/db
           - MONGO_LOG_DIR=/dev/log
        volumes:
           - ./data:/data
        ports:
           - 27017:27017
        command: mongod --smallfiles --logpath=/dev/null # --quiet

Snippet 10: The docker.yml definition used to deploy the defined service and the specified version of mongo as containers

As per component tests, it makes sense to keep end-to-end tests in a separate module and different phases. The exec-maven-plugin was used to deploy all required components, exec our tests, and finally clean and teardown our test environment.

Microservices snippet 11 — **Snippet 11:** Using exec-maven-plugin executions with docker commands to prepare for tests and clean-up after tests

Since this is a broad-stack test, a smaller selection of tests per feature will be executed. Tests are selected based on perceived business risk. The previous types of tests covered low-level details. That means whether a user story matches the Acceptance Criteria. These tests should also immediately stop a release, as a failure here might cause severe business repercussions.

Conclusion

Handoff-centric testing often ends up being a very long process, taking up to weeks until all bugs are identified, fixed, and a new deployment readied. Feedback is only received after a release is made, making the lifespan of a version of our quickest possible turnaround time.

The continuous testing approach ensures immediate feedback. Meaning the DevOps engineer is immediately aware of whether the feature implemented is production-ready or not, depending on the outcome of the tests run. From unit tests up to end-to-end tests, they all assist in speeding up the assessment process.

Microservices architecture helps create faster rollouts to production since it is domain-driven. It ensures failure isolation and increases ownership. When multiple teams are working on the same project, it’s another reason to adopt such an architecture: To ensure that teams are independent and do not interfere with each other’s work.

Improve testability by moving toward continuous testing. Each microservice has a well-defined domain, and its scope should be limited to one actor. The test cases applied are specific and more concise, and tests are isolated, facilitating releases and faster deployments.

Following the TDD approach, there is no coding unless a failed test returns. This process increases confidence once an iterative implementation results in a successful trial. This process implies that testing happens in parallel with the actual implementation, and all the tests mentioned above are executed before changes reach a staging environment. Continuous testing keeps evolving until it enters the next release stage, that is, a staging environment, where the focus switches to more exhaustive testing such as load testing.

Agile, DevOps, and continuous delivery require continuous testing. The key benefit is the immediate feedback produced from automated tests. The possible repercussions could influence user experience but also have high-risk business consequences. For more information about continuous testing, Contact phoenixNAP today.

Twitter Facebook LinkedIn Email