Write a Blog >>
Thu 12 Nov 2020 01:05 - 01:06 at Virtual room 2 - Testing 1

Detecting Graphical User Interface (GUI) elements in GUI images is a domain-specific object detection task. It supports many software engineering tasks, such as GUI animation and testing, GUI search and code generation. Existing studies for GUI element detection directly borrow the mature methods from computer vision (CV) domain, including old fashioned ones that rely on traditional image processing features (e.g., canny edge, contours), and deep learning models that learn to detect from large-scale GUI data. Unfortunately, these CV methods are not originally designed with the awareness of the unique characteristics of GUIs and GUI elements and the high localization accuracy of the GUI element detection task. We conduct the first large-scale empirical study of seven representative GUI element detection methods on over 50k GUI images to understand the capabilities, limitations and effective designs of these methods. This study not only sheds the light on the technical challenges to be addressed but also informs the design of new GUI element detection methods. We accordingly design a new GUI-specific old-fashioned method for non-text GUI element detection which adopts a novel top-down coarse-to-fine strategy, and incorporate it with the mature deep learning model for GUI text detection.Our evaluation on 25,000 GUI images shows that our method significantly advances the start-of-the-art performance in GUI element detection.

Thu 12 Nov
Times are displayed in time zone: (UTC) Coordinated Universal Time change

01:00 - 01:02
Long-paper
FrUITeR: A Framework for Evaluating UI Test Reuse
Research Papers
Yixue ZhaoUniversity of Southern California, USA, Justin ChenColumbia University, USA, Adriana SejfiaUniversity of Southern California, USA, Marcelo LaserUniversity of Southern California, USA, Jie M. ZhangUniversity College London, UK, Federica SarroUniversity College London, UK, Mark HarmanUniversity College London, UK, Nenad MedvidovićUniversity of Southern California, USA
DOI Pre-print Media Attached
01:03 - 01:04
Talk
ModCon: A Model-Based Testing Platform for Smart Contracts
Tool Demos
Ye LiuNanyang Technological University, Singapore, Yi LiNanyang Technological University, Singapore, Shang-Wei LinNanyang Technological University, Singapore, Qiang YanWeBank, n.n.
DOI Pre-print Media Attached
01:05 - 01:06
Talk
Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination?
Research Papers
Jieshan ChenAustralian National University, Australia, Mulong XieAustralian National University, Australia, Zhenchang XingAustralian National University, Australia, Chunyang ChenMonash University, Australia, Xiwei XuData61 at CSIRO, Australia, Liming ZhuData61 at CSIRO, Australia / UNSW, Australia, Guoqiang LiShanghai Jiao Tong University, China
DOI
01:07 - 01:08
Talk
UIED: A Hybrid Tool for GUI Element Detection
Tool Demos
Mulong XieAustralian National University, Australia, Sidong FengAustralian National University, Australia, Zhenchang XingAustralian National University, Australia, Jieshan ChenAustralian National University, Australia, Chunyang ChenMonash University, Australia
DOI
01:09 - 01:10
Talk
WebRR: Self-Replay Enhanced Robust Record/Replay for Web Application Testing
Industry Papers
Zhenyue LongChina Southern Power Grid, China, Guoquan WuInstitute of Software at Chinese Academy of Sciences, China, Xiaojiang ChenChina Southern Power Grid, China, Wei ChenInstitute of Software at Chinese Academy of Sciences, China, Jun WeiState Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences
DOI
01:11 - 01:30
Talk
Conversations on Testing 1
Paper Presentations
Guoquan WuInstitute of Software at Chinese Academy of Sciences, China, Jieshan ChenAustralian National University, Australia, Sidong FengAustralian National University, Australia, Ye LiuNanyang Technological University, Singapore, Yixue ZhaoUniversity of Southern California, USA, Mulong XieAustralian National University, Australia, M: Corina S PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center