As test automation has been introduced to software delivery process, test results have exploded. Test execution agents or robots could run round the clock without taking breaks. In addition, the number of test cases could accumulate with every sprint. It would not be wrong to suggest that more results would be produced for management and analysis. However, you would be required to take the right approach.
In case, you exceed the time saved by running automated tests by spending time in investigating test results, automation would not be improving output quality. It would definitely not be worth the time, effort and cost spent on it. In order you look forward to reaping the benefits of automation, it would be in your best interest to understand proper handling of increasing number of test results.
Find below few important aspects to consider when handling and analyzing test results as generated by the automation process. By following the below mentioned tests, you would be able to make the most of 3x Faster Test Automation accelQ ROI analysys.
- Setting up of automated monitoring
In case, any team has been looking after several tasks in the software delivery process, addition of another task of monitoring the result log would not imply improvement in quality. You would be required to have a test team that looks forward to monitor test results constantly independently.
- Try to determine reason for failure of cases
Apparently, if the tester has been spending adequate time on analyzing failure of automated test case rather than executing the case, you should be rest assured that very purpose of automation would be lost. Investigation on failing test case along with pointing out the reason for failure would be easy and quick. These product owners, testers and developers would require swift feedback for catching inconsistencies in a quick manner.
- Sharing or results
Most would be using release platforms for handling bugs and managing tests. They would be using test teams as essential tools for tracking test strategies, bugs, test case descriptions etc. You should integrate test automation platform by pushing results to test management system or by pulling the results from test automation platform with use of API.
- Using dashboard for results
Usage of visual dashboards on shared monitors would prove to be an effective mode for sharing results in and between different teams and their workplace. It would enable development team to react quickly to issues and fix them before bug actually releases into production environment.