On overcoming challenges with GUI-based test automation

Automated testing is widely used in modern software development to check if the software meets its requirements, including its graphical user interface (GUI). GUI-based test automation, like other automation, aims to save time and money compared to manual testing. While automation has successfully reduced costs for other types of testing (e.g., unit- or integration tests), GUI-based testing has faced technical challenges for over a decade, with limited progress in solving them.

The objective is to contribute to the software engineering body of knowledge by (1) identifying the main challenges in GUI-based test automation and (2) finding technical solutions to mitigate one or some of the main challenges.We conducted several studies to reach this objective. We used a systematic literature review to identify the main challenges in GUI-based test automation, followed by several studies that propose and evaluate novel approaches to mitigate the main challenges.

Our first contribution is mapping the challenges in GUI-based test automation reported in academic literature. We mapped the main challenges (i.e., most reported) on a timeline and classified them from essential to accidental. This classification is valuable since we can focus our research on the main challenges today that we are more likely to mitigate using a technical solution (i.e., accidental). Our second contribution is several novel approaches that explore new concepts or advance state-of-the-art techniques to mitigate some of the main accidental challenges.

Our results provide alternative approaches and concepts that can mitigate some of the main challenges in GUI-based test automation, thereby lowering the cost of creating and maintaining automated GUI tests. With a reduced cost of automation, testers can focus more on other tasks like requirements, test design, and exploratory testing.