Before you initiate the QA automation process, you need to answer a very important question: what is the purpose of automation on your current project?

In our previous article, QA automation: when and why, we identified the following list of tasks that could be completed with the help of QA automation: 

• setup of testing environments,
• generation of test data,
• generation of test reports,
• monitoring of connections between artefacts,
• execution of tests and checks. 

In order to tackle such tasks more effectively, QA engineers first have to decide which specific test and user scenarios should be automated. There are several criteria that need to be taken into consideration when selecting the exact scenarios; there’s also a so-called test pyramid that you can use to make the final decision:

As you can see in the image, the cost and execution time of tests, as well as the possibility of getting false negative errors, are lower for unit testing and higher for GUI testing. During automation, QA engineers mostly focus on the coverage of the Service / API layer, followed by the User Interface (GUI) layer. The latter usually represents a range of end-to-end scenarios that check whether the functionalities are working correctly and are compliant with the required business logic.

Generally, if the project’s budget and/or resources are limited, it’s recommended to prioritise the API layer, since writing API test scenarios takes less time in comparison to the preparation of GUI tests, and on top of that, it allows the team to ensure the quality of the product’s business logic. The same rule applies for sitations when you need to meet tight deadlines and release the product to the market as soon as possible. 

In other cases, such as having no restrictions in regards to the project’s budget or release schedule, it is possible to provide a near-full coverage of the application’s business logic thanks to the end-to-end tests that mimic the actions of a real end-user via UI. 

Here are a couple of tips that you can follow to identify which scenarios (no matter whether it’s API or GUI tests) you should automate: 

1. Define critical functionalities 

You need to determine which functionalities are the most critical from the user’s point of view – if there’s an error on production related to any of them, its cost will be quite high. Test coverage of such functionalities must be your top priority, as any disruption in their operation will make it practically impossible to use the product. Test scenarios of this kind require the utmost attention and increased number of resources. 

2. Find cases where errors may cause major financial setbacks

These test scenarios might not be used as frequently, however, possible financial losses are sure to have a negative impact on the project’s development, as well as on the company’s growth in general. 

3. Identify important integration scenarios 

Another useful practice is identifying integration scenarios that might affect the operation of adjacent commands and/or services. Errors during integration may disrupt the work of multiple departments, or even the company as a whole. 

4. Identify which test scenarios would require the most resources in order to be performed manually 

5. Make a list of the most “cost-efficient” tests

Investigate the cases in which automated tests would take significantly less time than their manual counterparts (e.g. several minutes instead of several hours). Also, don’t forget to consider the time needed to implement such autotests: if the development of one test scenario takes you a month whereas the same thing could’ve been checked manually in a minute or two, this sort of investment is neither beneficial nor reasonable in regards to the team and the project. 

Let’s take a look at some examples. 

If the development team is working on a project in the medical field:

• First and foremost, define a list of scenarios that may cause permanent damage to the person’s day-to-day life, their physical and/or mental health. 
• Then, ensure the confidentiality of transmitted and collected data – it must be encrypted and protected from any, and all, third parties. 
• It’s also recommended to determine which cases may lead to economic losses for the healthcare organisation in question, as well as the most frequent / routine operations performed by the medical staff. 

If the project is related to finances, the team should pay attention to the company’s gains / losses:

• To start with, identify which user scenarios involve the biggest number of financial risks.
• After that, make a list of the most visited pages across the entire application, and define the users’ actions associated with each of these pages. 
• Plus, it’s important to mention various high-stakes integrations with bank accounts and financial tools – thorough testing of these will help you to avoid economic losses with every release. 

To sum up, the selection of scenarios for QA automation is quite far from being a trivial task. When analysing the requirements for the project’s test coverage, you not just should, but must involve QA engineers, clients, product managers, developers – and ultimately, any other member of the team. Every participant could make an equally valuable contribution to the short-listing process and also share their knowledge about the technical side of things or, for example, about the importance of certain functionalities from the business perspective. After all, it is the end-users who will be interacting with your product the most and will therefore expect the best quality of service to be provided to them. 

Noveo provides a wide range of professional QA automation services: our QA engineers can determine and prioritise the most significant test scenarios for your business, as well as implement and maintain automated testing tailored to every aspect of your project.