Trace2Script: Framework for Web UI Test Script Generation from Natural Language Test Cases
Authors: J. M. Chowdhury*, S. M. R. Pial*, J. Y. Khan, M. R. Z. Ratul, & A. Iqbal
Under Preparation
Trace2Script is an AI Agent-powered framework that translates natural language test cases into executable and robust Playwright test scripts for automated Web UI testing. The framework proposes and engineers a Web Agent architecture for simulating realistic human actions during autonomous Web UI traversal with comprehensive trace logging of element interactions. We constructed and validated a comprehensive benchmark dataset of 445 natural language test cases spanning 19 production web applications, ranging from basic UI interactions to complex workflow scenarios. Our evaluation demonstrates that software-aware constraints with explicit behavior modeling significantly outperform unconstrained LLM-based test script generation across production applications.
* denotes equal contribution