Write a Blog >>
Thu 12 Nov 2020 01:30 - 01:32 at Virtual room 2 - Testing 2

For large industrial applications, system test cases are still often described in natural language (NL), and their number can reach thousands. Test automation is to automatically execute the test cases. Achieving test automation typically requires substantial manual effort for creating executable test scripts from these NL test cases. In particular, given that each NL test case consists of a sequence of NL test steps, testers first implement a test API method for each test step and then write a test script for invoking these test API methods sequentially for test automation. Across different test cases, multiple test steps can share semantic similarities, supposedly mapped to the same API method. However, due to numerous test steps in various NL forms under manual inspection, testers may not realize those semantically similar test steps and thus waste effort to implement duplicate test API methods for them. To address this issue, in this paper, we propose a new approach based on natural language processing to cluster similar NL test steps together such that the test steps in each cluster can be mapped to the same test API method. Our approach includes domain-specific word embedding training along with measurement based on Relaxed Word Mover’sDistance to analyze the similarity of test steps. Our approach also includes a technique to combine hierarchical agglomerative clustering and K-means clustering post-refinement to derive high-quality and manually-adjustable clustering results. The evaluation results of our approach on a large industrial mobile app, WeChat, show that our approach can cluster the test steps with high accuracy, substantially reducing the number of clusters and thus reducing the downstream manual effort. In particular, compared with the baseline approach, our approach achieves 79.8% improvement on cluster quality, reducing 65.9% number of clusters, i.e., the number of test API methods to be implemented.

Thu 12 Nov

Displayed time zone: (UTC) Coordinated Universal Time change

01:30 - 02:00
Clustering Test Steps in Natural Language toward Automating Test Automation
Industry Papers
Linyi Li University of Illinois at Urbana-Champaign, Zhenwen Li Peking University, China, Weijie Zhang Tencent, China, Jun Zhou Tencent, China, Pengcheng Wang Tencent, China, Jing Wu Tencent, China, Guanghua He Tencent, China, Xia Zeng Tencent, China, Yuetang Deng Tencent, Inc., Tao Xie Peking University
PRF: A Framework for Building Automatic Program Repair Prototypes for JVM-Based Languages
Tool Demos
Ali Ghanbari Iowa State University, Andrian Marcus University of Texas at Dallas
DOI Pre-print
SOSRepair: Expressive Semantic Search for Real-World Program Repair
Journal First
Afsoon Afzal Carnegie Mellon University, Manish Motwani University of Massachusetts, Amherst, Kathryn Stolee North Carolina State University, Yuriy Brun University of Massachusetts Amherst, Claire Le Goues Carnegie Mellon University
Link to publication DOI Pre-print Media Attached
tsDetect: An Open Source Test Smells Detection Tool
Tool Demos
Anthony Peruma Rochester Institute of Technology, Khalid Almalki Rochester Institute of Technology, USA, Christian D. Newman Rochester Institute of Technology, Mohamed Wiem Mkaouer Rochester Institute of Technology, Ali Ouni ETS Montreal, University of Quebec, Fabio Palomba University of Salerno
DOI Pre-print Media Attached
Understanding and Automatically Detecting Conflicting Interactions between Smart Home IoT Applications
Research Papers
Rahmadi Trimananda University of California at Irvine, USA, Seyed Amir Hossein Aqajari University of California at Irvine, USA, Jason Chuang University of California at Irvine, USA, Brian Demsky University of California at Irvine, Guoqing Harry Xu University of California at Los Angeles, Shan Lu University of Chicago, USA
DOI Pre-print Media Attached File Attached
Conversations on Testing 2
Paper Presentations
Afsoon Afzal Carnegie Mellon University, Anthony Peruma Rochester Institute of Technology, Linyi Li University of Illinois at Urbana-Champaign, Rahmadi Trimananda University of California at Irvine, USA, M: Corina S. Păsăreanu Carnegie Mellon University Silicon Valley, NASA Ames Research Center