A Survey on Web Testing: On the Rise of AI and Applications in Industry

2025-03-07

AI Summary

This study addresses the fragmentation and academia-industry disconnect in web testing research from 2014 to 2024. We conduct a systematic literature review (SLR), covering top-tier venues (e.g., ICST) and tool ecosystems (e.g., Selenium). Methodologically, we integrate quantitative bibliometric analysis with thematic clustering, thereby providing the first empirical quantification of the rising trend in AI-driven test case generation. Key contributions include: (1) confirming black-box automation as the dominant paradigm and ICST as the central academic forum; (2) identifying critical research gaps—particularly industrial deployment bottlenecks (e.g., scarcity of mature open-source tools, lack of real-world validation) and severe underrepresentation of human-subject studies; and (3) establishing weak academia-industry collaboration as the principal barrier to technology transfer. Our findings offer evidence-based guidance for strategic research prioritization and tool development in web testing.

Technology Category

• Data Mining & Knowledge Management: Web
• Application Domains: Humanities & Computational Social Science
• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving

Application Category

• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Web application testing is an essential practice to ensure the reliability, security, and performance of web systems in an increasingly digital world. This paper presents a systematic literature survey focusing on web testing methodologies, tools, and trends from 2014 to 2024. By analyzing otalPapersIncluded research papers, the survey identifies key trends, demographics, contributions, tools, challenges, and innovations in this domain. In addition, the survey analyzes the experimental setups adopted by the studies, including the number of participants involved and the outcomes of the experiments. Our results show that web testing research has been highly active, with ICST as the leading venue. Most studies focus on novel techniques, emphasizing automation in black-box testing. Selenium is the most widely used tool, while industrial adoption and human studies remain comparatively limited. The findings provide a detailed overview of trends, advancements, and challenges in web testing research, the evolution of automated testing methods, the role of artificial intelligence in test case generation, and gaps in current research. Special attention was given to the level of collaboration and engagement with the industry. A positive trend in using industrial systems is observed, though many tools lack open-source availability.

Problem

• Analyzes web testing methodologies, tools, and trends from 2014 to 2024.
• Identifies key trends, challenges, and innovations in web testing research.
• Explores AI's role in test case generation and industrial collaboration gaps.

Innovation

• Systematic literature survey on web testing
• Focus on automation in black-box testing
• AI role in test case generation