DragonCrawl: A Generative, Intent-Based Framework for Scalable Mobile End-to-End Testing

2026-07-30

AI Summary

Traditional end-to-end testing on mobile platforms faces significant challenges, including frequent UI changes, high maintenance costs, and poor cross-platform scalability. This work proposes DragonCrawl, a regression testing system powered by large language models that replaces conventional semantic matching with generative intent reasoning. By integrating multimodal visual recognition and backend state instrumentation, DragonCrawl enables robust validation through coordinated UI interactions and system state checks. The system leverages the GPT-4o multimodal model and seamlessly integrates into CI/CD pipelines, achieving test pass rates of 91.6% on iOS and 92.2% on Android. It reduces test integration time from 96–120 hours to under four hours and has cumulatively saved approximately 27 person-years in maintenance effort.

Abstract

As mobile applications grow in complexity, traditional End-to-End (E2E) testing frameworks struggle with UI volatility, maintenance overhead, and cross-platform scalability. This paper presents DragonCrawl, an AI-driven mobile testing system for continuous regression testing that has evolved from embedding-based similarity matching to generative intent-based reasoning using large language models. Unlike prior LLM-based testing research focused on exploratory testing and crash detection, DragonCrawl validates specific user flows on every code change, blocking commits that break critical functionality. By leveraging GPT-4o's multimodal capabilities, DragonCrawl achieves 91.6% pass rate on iOS and 92.2% on Android across 1,013 automated tests running continuously in CI/CD pipelines. The system reduces test onboarding time from 96-120 hours to under 4 hours and has saved an estimated 27 developer years in test maintenance effort. We present the architectural evolution from V1 (semantic embedding matching) to V2 (generative intent-based reasoning), discuss implementation challenges including token explosion and memory constraints, and report operational experience from production deployment. The integration of multimodal vision for end-state detection and tool calling for backend state transitions enables comprehensive regression testing that bridges UI interactions with system state. Our results demonstrate that AI-driven testing can maintain stability while eliminating the brittleness of traditional automated tests, enabling continuous quality assurance at scale.

Problem

• mobile end-to-end testing
• UI volatility
• test maintenance
• cross-platform scalability
• regression testing

Innovation

• generative testing
• intent-based reasoning
• multimodal LLM
• mobile E2E testing
• CI/CD regression