Regression testing is a specific form of testing that verifies whether a given piece of software suffered regressions after undergoing changes. “Regression” here means “going back to a previous undesired state.” So automated regression testing is nothing more than the process of automatically verifying that the application has not regressed to a previous undesired state.
Ideally, regression testing—whether automated or not—should be performed every time a software application is changed in some way, whether by receiving a new feature, an improvement, or a bug fix.
In today’s post, we’ll define and explain what automated regression testing means. We’ll start by explaining what a regression is, how costly they are for software teams, and why you should employ regression testing to avoid them. Since manual regression testing would be both time-consuming and error-prone, we’ll then proceed to cover the need for automation, offering advice and tips on how to actually implement the technique for your teams.
Let’s get started!
Automated Regression Testing: Let’s Come Up With a Definition
Before we dive into the “how” of automated regression testing, we need to clarify its “what” and “why.” What is it, after all? What does automated regression testing mean, and why should we use it?
This section will answer the questions above and more.
What Does “Regression” Mean in “Automated Regression Testing”?
The first step in order to understand “automated regression testing” is to get rid of the “automated” part, at least for now. First, we need to understand what “X” means, then “Automated X” will be self-evident. Or, at the very worst, easier to grok.
If we want to understand what “regression testing” means, the first step is to define “regression.” In this context, is “regression” good or bad? Should we welcome it or fear it? The answer to the questions above is straightforward. Here, regression means essentially the same thing it means in everyday conversation—to go back to a previous state. So, in the context of software development, we say we’ve got a regression when our application unintentionally reverted to a previous state.
Why Do We Need Regression Testing?
As we’ve just defined in the previous section, “regression” means an unintentional reversion to a previous state. However, something really important was left implicit in the definition: software regression, more often than not, means the application went back to a bad previous state.
Have you ever had the experience of making some existing feature stop working after implementing a new one? What about that nasty bug returning months after you thought you had completely eliminated it? These are examples of regression. The application has regressed to a previous inferior state.
Regression Testing Is Vital in Software
The field of software development is disproportionately prone to regression problems. Each addition or change made by any developer has the potential to cause unexpected problems in areas (supposedly) unrelated to the spot where it was performed. Any non-trivial software project, maintained by a team with more than, say, five people, has an incredibly high number of potential regressions during each release.
Developers should always keep in mind that all the changes they make, no matter how small, simple, or insignificant they seem, have the potential to cause surprising side effects. They can break functionalities that don’t have anything to do with the changes being made. By performing regression tests, the developer checks that not only does their change behave as it’s supposed to, but also that it plays well with all of the code that was written up until that point.
That’s where regression testing comes in handy. Regression testing is nothing more than the execution (partial or total) of a test suite in order to verify that a given application hasn’t returned to a previous undesired state. If manually done, though, regression testing can be extremely time-consuming and error-prone, which leads us to our next point.
Regression Testing: Why Automate It?
I often say that everything that’s automatable should be automated. Meaning, if you can automate a process, then you probably should do so. If you can automate a process but you’re still doing it manually, I’m afraid you’re leaving money on the table. Chances are the manual process is slow, time-consuming, tedious, and error-prone. That implies you’re losing money in at least three different ways.
For starters, you waste money by having well-paid professional performing tasks that could be automated. Then, there’s always the opportunity cost. The people who are performing the tests could be doing more valuable tasks. Such tasks could have the potential to generate way more value. Finally, since the manual process is error-prone, then people are bound to make mistakes, which will result in losses.
Automated Regression Testing: How to Put It Into Practice
Now that we’re done defining concepts, it’s time for some practical tips on how to actually implement regression testing. There’s good news. If you already write some kind of automated tests for your application—e.g. unit tests—then you’re already performing regression testing without even knowing about it.
Regression testing is not a “new” category of automated tests. It’s not an alternative to unit tests or integration tests. On the contrary, your automated tests —unit tests, integration tests, and similar—written from day one in the project can and should act as regression tests. After each new change to the codebase, just re-run all of the relevant tests on the suite to ensure they’re not failing.
Automated Regression Testing Best Practices
Additionally, there are some best practices you should implement when adopting regression testing, regardless of the specifics of your implementation:
- Adopt a Test Management Software: This is vital for all projects that are larger than a one-developer toy project. You’ll have, realistically, way more tests cases than you’d be able to track and manage individually. Luckily for you, there are lots of test management tools at your disposal.
- Keep a Testing Schedule: You should maintain a strict testing schedule throughout the entire project. This will ensure the final project is thoroughly tested. Additionally, the schedule will encourage the team to adapt to a frequent testing regimen.
- Write a New Failing Test for Every New Bug Found: Imagine your code has an unequivocal, reproducible bug. But all of your tests are passing. That means that either the current tests are wrong, or your test suite is lacking tests. If you find yourself in this scenario, write a new failing test to document the bug.
- Categorize All of Your Tests: You should split your test suite into smaller categories. Your test management tool will most likely provide you with the ability to categorize your tests. That way, your team members can easily identify each kind of test.
Automated Regression Testing: Put It to Work for Your Team ASAP
Software development is a creative endeavor like no other, but the path to reaping its rewards is full of risks. One of the dangers when writing code is to break existing functionality while adding new ones. Even when fixing bugs, we can make older ones—that we thought were dead—come back to life. Each small and seemingly insignificant change poses the risk of a dreaded regression.
If you want to get rid of regressions—and you should want to, believe me—then a suite of regression tests is the solution you seek.
Author bio: This post was written by Carlos Schults.Carlos is a .NET software developer with experience in both desktop and web development, and he’s now trying his hand at mobile. He has a passion for writing clean and concise code, and he’s interested in practices that help you improve app health, such as code review, automated testing, and continuous build.
