Maybe you’ve heard what test automation is but you’re unsure about how to fit it all together in practice. If that’s you, then you feel just like I did when I was first getting my head around how test automation works. So many conversations about testing quickly become abstract and sometimes we simply want to be practical—just show me the code!
If you feel the same way, then you’re in luck! Today we’re going to cover how automated tests work in detail. To do this, we’ll cover the test pyramid and how it relates to testing, which will set a foundation so we can go through and break down each category of testing—from unit tests to integration tests and finally to journey tests. And we did promise to keep things practical, so we’ll talk through some real-world code examples too.
By the end of this post, you should know what the test pyramid is and how it relates to automation testing. You will also learn in detail what unit, integration, and journey tests are and how they’re written.
What Are Automated Tests?
Before we begin, let’s start with a quick definition:
Automated tests perform machine-based assertions on an application to determine the state of the current application against the assertions.
In simple terms, we ask questions of our systems through testing, and the current state of the system (how it’s coded) informs our test results.
Don’t worry if it still feels abstract, as we’ll be breaking it down.
What Is the Test Pyramid?
Before we dive into our examples on each of our testing steps, let’s define them. A lot of the nuance of how automated tests should be written hinges on the theory around test pyramids, so it’s important to have a strong grasp before continuing.
At its heart, the testing pyramid is about feedback loops, and it forms a recommendation on how to structure your tests to achieve the fastest possible feedback loop.
But, what do I mean by feedback loop? I mean the time it takes from modifying some code until you know that it’s wrong (and, importantly, where it’s wrong). The notion of the pyramid is important as it represents how we should heavily load the bottom layer while keeping the top layer thin in order to retain our pyramid shape (more on balancing testing layers soon!).
Test pyramids can come in slightly different formats. For today, let’s use a simplified version, which breaks down into unit, integration, and journey tests.
Unit Tests
Unit tests are the base of our pyramid—they run fast and validate individual functions work as we expect. However, since we’re only testing at the function level, these tests don’t tell us whether our application works as a whole. But importantly, unit tests run blazingly fast. And when they fail, they tell us exactly what went wrong to the precision of a single function. So while unit tests don’t tell us if the application is broken as a user would deem it, they give us the fastest feedback possible.
Integration Tests
Integration tests form the middle of the testing pyramid. If unit tests together sum up to a single software component, then our integration tests ensure that each software component can work with another. Usually, integration tests run over a network boundary of some form. Integration tests essentially validate whether all the aforementioned functions are put together correctly and that other software components can successfully communicate with the component in question. Integration tests are less about what the functions do (that’s our unit tests), but more that our functions are put together correctly.
Journey Tests
These tests form the last layer and the top of our pyramid. Journey tests typically execute a journey as our user would see it. We don’t tend to cover edge cases in journey tests as they should be covered at base levels of the pyramid (either unit or integration).
And journey tests are merely a few, short tests that cover a lot of code. Because we’re writing so few of them and they cover so much code, they give us a lot of bang-for-your-buck for establishing if our application is working as expected since they are covering core journeys. But—frustratingly—they won’t really tell you what is broken (at least, not to the same degree that a unit test would). Additionally, journey tests can be very slow to run and brittle to maintain (but more on why this is later).
Putting the Pyramid Into Practice
Hopefully, by now you have a decent understanding of the test pyramid and how the layers fit together. Don’t worry if you don’t completely understand at this point. We’re going to break down each layer and show you how it works in practice.
Unit Tests: An Example
Unit tests run against individual pieces of code. So let’s take an example.
The following test is written using the Jest unit test library. Jest gives us the “test” and the “expect” function. With these tools, we can then describe the scenarios we want to run for our tests and also our expected output. Tests can have any number of assertions, but typically you’ll have only one (or a small amount). Keeping assertions small means we can fit them to our descriptions easier.
function isOdd(a) {
if (typeof a != “number”) {
throw new Error(“Must pass a number”);
}
return a % 2 != 0
}
test(“Passing an odd number returns true”, () => {
expect(isOdd(1)).toBe(true);
});
test(“Passing an even number returns false”, () => {
expect(isOdd(1)).toBe(true);
});
test(“Passing a string throws error”, () => {
expect(() => isOdd(“Hello”)).toThrow(“Must pass a number”);
});
Here we can see that we’ve written:
- A happy path test
- An unhappy path test
- A dynamic type checking and error handling test
The test coverage is therefore 100 percent. However, we may want to write additional cases in the future for edge cases if we believe they add more value. For instance: what should the function do if we pass superfluous arguments? And what about edge cases? Negative numbers? Passing a zero value? Or passing a very large value? All of these are valid tests, but adding more tests has diminishing value over time. If we’re incredibly thorough with our testing at all levels, we likely won’t ship any code. It’s a balance. Choosing whether to add these additional cases will depend on factors such as: how much business impact would there be if the code were faulty? Are there other tests you could write that deliver more value?
Integration Tests: An Example
Building on top of our unit tests we have our integration tests. Below is an example of a test written with SuperTest. SuperTest is a small helper library around Jest (which we used for the unit test example). SuperTest gives us the request functionality (and its chained assertion methods).
The following code takes an endpoint, in our case “/user” makes a request to it, asserting the response to be a 200 success. SuperTest also allows us to simply test HTTP endpoints. Remember when we said integration tests often happen over a network boundary? This is precisely what we meant. We tell our test what our setup criteria are, and we can then assert that we received the correct response, error code, headers, etc.
describe(‘GET /user’, function() {
it(‘responds with json’, function(done) {
request(app)
.get(‘/user’)
.set(‘Accept’, ‘application/json’)
.expect(‘Content-Type’, /json/)
.expect(200, done);
});
});
But you might be thinking: in the above example are we simply testing that the endpoint responds? Not that it has expected values?
At first, this may seem like a low-value test. However, since we have already covered a lot of the functional testing in our unit tests at the integration test level, we are really only testing that our application links up all of those functions and responds without errors. Of course, we can write more detailed scenarios, but we want to be careful that we’re not replicating unit test code in integration tests. Unit tests run faster, so we should ideally write our tests there when possible.
Journey Tests: An Example
Last up is the journey test. The following is an example of a journey test using the Cypress testing framework. Cypress gives us a test runner and an assertion library. The “describe”, “it”, and “cy” commands all come from the Cypress library.
describe(“When at checkout”, () => {
it(“Can buy”, () => {
cy
.get(“.buy”)
.click();
cy
.get(“.summary”)
.should(“have.text”, “Thanks for purchasing!”);
});
});
In our journey test, we are testing the physical interface of the application, which should not only include what the user sees, but also behave as a user would when conducting a task (we call this a journey). In our example, we are navigating to our route, finding a button by its class name, clicking it, and then asserting that we can see our summary.
Can you spot why we said these tests could be brittle? For instance, if a developer decides to change an element class name (which in theory might not break functionality!), this change may result in a test failure. Oh no! We can use clever ways of structuring our code to reduce the fragility of our tests, but we cannot remove the brittleness completely—it’s inherent in the nature of the journey test.
The End of Our Tour of Automated Testing
And that concludes our foray into automated testing today. I hope this post gave you a clearer picture of how automated testing works in practice. We’ve seen how we need to have a balanced pyramid and also some examples of how our pyramid could start to be implemented.
In the real world, automated testing requires constant collaboration communication and continual improvement of our codebase. As we find bugs in our code, we work to ensure our coverage is updated. But in updating our codebase we need to ensure we balance our pyramid to keep our feedback fast.
It’s not easy. But, do it right and automated testing yields incredible results that are far superior to their manual testing counterparts. Learn how to identify the best automation platform for your needs with our simple, five step process.
And remember, the best way to learn is to get your hands dirty and have a go! So take the examples and attempt to implement your own small application with each layer of the test pyramid!
Author bio: This post was written by Lou Bichard. Lou is a JavaScript full stack engineer with a passion for culture, approach, and delivery. He believes the best products emerge from high performing teams and practices. Lou is a fan and advocate of old-school lean and systems thinking, XP, continuous delivery, and DevOps.
