Write a Blog >>
Thu 12 Nov 2020 01:30 - 01:32 at Virtual room 2 - Testing 2

For large industrial applications, system test cases are still often described in natural language (NL), and their number can reach thousands. Test automation is to automatically execute the test cases. Achieving test automation typically requires substantial manual effort for creating executable test scripts from these NL test cases. In particular, given that each NL test case consists of a sequence of NL test steps, testers first implement a test API method for each test step and then write a test script for invoking these test API methods sequentially for test automation. Across different test cases, multiple test steps can share semantic similarities, supposedly mapped to the same API method. However, due to numerous test steps in various NL forms under manual inspection, testers may not realize those semantically similar test steps and thus waste effort to implement duplicate test API methods for them. To address this issue, in this paper, we propose a new approach based on natural language processing to cluster similar NL test steps together such that the test steps in each cluster can be mapped to the same test API method. Our approach includes domain-specific word embedding training along with measurement based on Relaxed Word Mover’sDistance to analyze the similarity of test steps. Our approach also includes a technique to combine hierarchical agglomerative clustering and K-means clustering post-refinement to derive high-quality and manually-adjustable clustering results. The evaluation results of our approach on a large industrial mobile app, WeChat, show that our approach can cluster the test steps with high accuracy, substantially reducing the number of clusters and thus reducing the downstream manual effort. In particular, compared with the baseline approach, our approach achieves 79.8% improvement on cluster quality, reducing 65.9% number of clusters, i.e., the number of test API methods to be implemented.

Conference Day
Thu 12 Nov

Displayed time zone: (UTC) Coordinated Universal Time change

01:30 - 02:00
Clustering Test Steps in Natural Language toward Automating Test Automation
Industry Papers
Linyi LiUniversity of Illinois at Urbana-Champaign, Zhenwen LiPeking University, China, Weijie ZhangTencent, China, Jun ZhouTencent, China, Pengcheng WangTencent, China, Jing WuTencent, China, Guanghua HeTencent, China, Xia ZengTencent, China, Yuetang DengTencent, Inc., Tao XiePeking University
PRF: A Framework for Building Automatic Program Repair Prototypes for JVM-Based Languages
Tool Demos
Ali GhanbariThe University of Texas at Dallas, Andrian MarcusUniversity of Texas at Dallas
DOI Pre-print
SOSRepair: Expressive Semantic Search for Real-World Program Repair
Journal First
Afsoon AfzalCarnegie Mellon University, Manish MotwaniUniversity of Massachusetts, Amherst, Kathryn StoleeNorth Carolina State University, Yuriy BrunUniversity of Massachusetts Amherst, Claire Le GouesCarnegie Mellon University
Link to publication DOI Pre-print Media Attached
tsDetect: An Open Source Test Smells Detection Tool
Tool Demos
Anthony PerumaRochester Institute of Technology, Khalid AlmalkiRochester Institute of Technology, USA, Christian D. NewmanRochester Institute of Technology, Mohamed Wiem MkaouerRochester Institute of Technology, Ali OuniETS Montreal, University of Quebec, Fabio PalombaUniversity of Salerno
DOI Pre-print Media Attached
Understanding and Automatically Detecting Conflicting Interactions between Smart Home IoT Applications
Research Papers
Rahmadi TrimanandaUniversity of California at Irvine, USA, Seyed Amir Hossein AqajariUniversity of California at Irvine, USA, Jason ChuangUniversity of California at Irvine, USA, Brian DemskyUniversity of California at Irvine, Guoqing Harry XuUniversity of California at Los Angeles, Shan LuUniversity of Chicago, USA
DOI Pre-print Media Attached File Attached
Conversations on Testing 2
Paper Presentations
Afsoon AfzalCarnegie Mellon University, Anthony PerumaRochester Institute of Technology, Linyi LiUniversity of Illinois at Urbana-Champaign, Rahmadi TrimanandaUniversity of California at Irvine, USA, M: Corina S. PăsăreanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center