Once, I was planning to make mashed potatoes for dinner. I got some fresh potatoes, garlic, cream, butter, salt, and milk. Each ingredient was thoroughly checked. After preparing the dish, I took a scoop and tasted it. The flavor was fine, but I forgot to add the salt. So I added a few pinches of kosher salt and served it. My parents were also having dinner with me. And guess what? After having a spoonful, my mom set the mashed potatoes aside.
Regrettably, I added more salt than needed. That totally ruined the dish. So what did I do wrong? If we think about this in terms of testing, I individually unit tested the ingredients. After preparing the dish, I did integration testing as well. But after adding salt, I forgot to execute regression testing.
Not only do we need to test things in cooking and other daily activities, but we also need regression testing when we’re developing software. Building on that, in this post, we’re going to discuss the things you need to include in your regression test plan in the form of a checklist for carrying out regression testing. But before that, let’s find out what regression testing is.
What Is Regression Testing?
We all know what testing means in the software industry. Testing is checking whether a specific feature or the entire application is working as expected or not. But what is regression testing?
The term “regression” in software means unwanted bugs caused by code changes. Often, we have to make changes in a certain piece of code. This can happen in multiple scenarios like the following:
- When we have to write a new function to fix a bug
- When the client requests for some change in an already-working feature
Because of the code changes we make, a defect may arise.
Regression testing is a type of repetitive testing of a pretested code. The goal is to detect the sudden defects that arise because of code changes.
Now that we know what regression testing is, let’s find out about the things you need to include in your regression test plan.
Checklist to Follow During Regression Testing
It’s time to discuss the things to check while carrying out regression testing. We shall divide the regression test plan checklist into three sections. Let’s get started.
Checklist Before Execution of Test Cases
The following are the items to check before you begin executing the test cases.
Did You Set Up All the Preconditions?
What do we check before development?
- Is the PC working properly?
- Do we have all the essentials like the IDE and database configured?
These are known as preconditions. Before regression testing, check the following:
- Your test environment should be set up properly to produce accurate results.
- You should have a process for defect management. Check if you have a software for bug tracking.
Do You Have a How-To or Step-by-Step Guide?
What do we do before using a new electronic device? Obviously, we read the instruction manual. Similarly, if you’re testing manually, you must have an Excel file with the steps for running test cases. And for automated regression testing, a similar guide needs to exist. What if you don’t have one? Write a guide that you can refer to for finding out which script to run in a certain situation.
Did You Identify All Inputs and Outputs?
In the how-to guide, check if you have a list of all the inputs and their expected outputs. For instance, suppose you need to test the log-in page of a web app after some code changes. If you have a guide that states which script to run while testing the log-in page and what the expected outcome is of that script, your job becomes easier. If the actual output doesn’t match the expected one, it’s time to log a defect.
Did You Validate All the Test Cases?
Make sure your test lead or manager validates all the test cases. Ever wondered what would happen when you report a defect and later the developer or your QA lead finds out that the test case along with the defect you reported is invalid? Now that would really be embarrassing. To prevent such a situation, after writing test cases, ask your QA lead or the concerned person to check and validate the test cases.
Do You Have a Defect Raising Procedure?
It’s very important to have a defect raising procedure. Finding a defect and mailing it to the developer doesn’t make sense at all in the case of large code. The developer, as well as the tester, will soon lose track, and many critical bugs will stay unfixed. In the current age of agile, use tools like Bugzilla, Jira, etc.
You can report a defect as soon as you find one with these tools. Not only that, you can mark the defects as per their severity. As soon as the developer fixes a defect, you’ll get a notification. You can then run the test again and mark the defect as fixed. Sounds a lot simpler, doesn’t it?
Things to Check After You Get the Test Data
There are also some things to consider after you get the test data but before you begin testing. Especially with the latest data compliance rules, a lot of restrictions are imposed on how you can use test data. So let’s discuss what you need to check in this part of your regression test plan.
Did You Identify the Test Data?
For a huge application, you’ll get loads of data. Identify and sort the data according to your testing criteria. For instance, suppose you’re testing an e-commerce app. To test the log-in feature, you’ll need dummy user IDs and passwords. For testing the add-to-cart feature, you’ll need dummy product details and so on. Sort and categorize the data to make the test cycle easier.
Does Your Regression Testing Perform Field-Level Validation?
How will you ensure that the test cases that you wrote provide 100% coverage? The answer is field-level validation. A tester needs to validate every field in software to ensure that all the defects are identified. Field-level validation is a test case design procedure that ensures not a single element in the software remains untested. Once you get the test data, check whether you can use the data to validate the software at the field level. You can consider using Testim’s automated solution to increase your test coverage. This will save the work of rewriting test cases later if anything goes unnoticed.
Did You Perform Test Data Maintenance/Version Control?
Regression testing means performing repeated test cases on the software or application. For repetitive test cases, you cannot use the same data if you want to avoid redundant test results. So, in your regression test plan, be sure to include a step where you sort the data and categorize it into versions. Use the data accordingly in each test cycle.
For example, suppose you’re testing a table that filters data by hourly, daily, and monthly entries. Now, if you’re testing the table with the same data that you used yesterday, you’ll get a repetitive result. But if you’re using data that’s created currently, you’ll get accurate results in each test cycle.
Are You Using Normal Data or Test-Specific Data?
For testing any product, the test data varies according to the type of test performed. For instance, suppose you’re giving a test ride of the new Ducati Panigale. If you’re testing the motorcycle on a street, the normal stock tires will do the job. But what if you’re testing the motorcycle on a racetrack? To get optimum performance, you’ll need track-specific tires. The same is applicable for testing as well. During regression testing, check if you’re using normal data or synthetic data created only for testing. Obviously, synthetic data is preferred nowadays in order to comply with data privacy regulations and data security policies.
Does Your Test Data Comply With Data Privacy Regulations?
Nowadays, all customers are concerned about data. Data privacy regulations like GDPR must be followed. These compliances state that you cannot use any real user’s data for testing. When you get the test data, ensure that your data is compliant with data privacy regulations. If any data isn’t compliant with required protocols, the result may be a huge fine along with loss of reputation.
Test Validation
Now that you have everything you need to begin regression testing, get started. But what do you check once you’re done with testing? Let’s find out.
Did You Execute All the Test Cases?
Check and double-check if you executed all the test cases. Also, check if you logged all the defects. If you accidentally missed anything, there’s a high chance that a critical defect may go unnoticed and unfixed in production. Now that will cause some serious damage!
Is There Any High-Priority or Critical Defect Still Open?
A tester always holds a position of power. They have the right to decide whether a developed module is ready for release or not. In your bug reporting tool, you’ll find all the defects are sorted as per their priority. Check if any high-priority defect is still open. Don’t mark the module to be ready for release unless all the defects are fixed.
Did You Capture and Document Expected and Actual Test Results?
Once you finish running the test cases, document the test results along with the expected results. This will come in handy especially during a regression test. After carrying out a second or third testing cycle, you can compare the outcomes with the expected or previous ones. Report as soon as you find an anomaly.
Did You Collect Test Metrics?
Metrics are parameters that define the health of a test report. After finishing regression testing, find out if you have all the required metrics, such as the below:
- Test progress percentage
- Test cases that you planned to run per day vs. actual test cases executed per day
- Percentage of defects that you detected
- Passed and failed test case ratio
- The average time the developer took to fix a defect
- The efficiency of test design and review
- Accepted and deferred defects count or percentage
Did You Complete the Test Closure Report?
What happens once you finish the final build’s regression testing phase? Your manager will want a detailed report of the test cases executed, bugs found or removed, etc. That’s where a test closure report will come in handy. A test closure report will have a summarized report of all the pretest and post-test details that the management will need before release.
Do You Really Need a Checklist?
What do you do while packing your luggage before going on a trip? Or before you go grocery shopping? You prepare a checklist, don’t you? Before carrying out a large task, we need many items. A checklist helps us to gather all the items before starting the task, so that once we begin, we don’t run around to gather items that we missed.
The same theory is applicable for regression testing as well. If you have a regression test plan, your testing process will be smooth and fast. So why wait? Start preparing a checklist and run regression testing of your application.
This post was written by Arnab Roy Chowdhury. Arnab is a UI developer by profession and a blogging enthusiast. He has strong expertise in the latest UI/UX trends, project methodologies, testing, and scripting.
