Graphical User Interface (GUI) elements detection is critical for many GUI automation and GUI testing tasks.
Acquiring the accurate positions and classes of GUI elements is also the very first step to conduct GUI reverse engineering or perform GUI testing.
In this paper, we implement a \textbf{U}ser \textbf{I}terface \textbf{E}lement \textbf{D}etection (UIED), a toolkit designed to provide user with a simple and easy-to-use platform to achieve accurate GUI element detection.
UIED integrates multiple detection methods including old-fashioned computer vision (CV) approaches and deep learning models to handle diverse and complicated GUI images.
Besides, it equips with a novel customized GUI element detection methods to produce state-of-the-art detection results.
Our tool enables the user to change and edit the detection result in an interactive dashboard.
Finally, it exports the detected UI elements in the GUI image to design files that can be further edited in popular UI design tools such as Sketch and Photoshop.
UIED is evaluated to be capable of accurate detection and useful for downstream works.
Tool URL: \url{http://uied.online}
Github Link: \url{https://github.com/MulongXie/UIED}
Thu 12 Nov Times are displayed in time zone: (UTC) Coordinated Universal Time change
01:00 - 01:30: Testing 1Paper Presentations / Industry Papers / Research Papers / Tool Demos at Virtual room 2 | |||
| 01:00 - 01:02 Long-paper | FrUITeR: A Framework for Evaluating UI Test Reuse Research Papers Yixue ZhaoUniversity of Southern California, USA, Justin ChenColumbia University, USA, Adriana SejfiaUniversity of Southern California, USA, Marcelo LaserUniversity of Southern California, USA, Jie M. ZhangUniversity College London, UK, Federica SarroUniversity College London, UK, Mark HarmanUniversity College London, UK, Nenad MedvidovićUniversity of Southern California, USA DOI Pre-print Media Attached | ||
| 01:03 - 01:04 Talk | ModCon: A Model-Based Testing Platform for Smart Contracts Tool Demos Ye LiuNanyang Technological University, Singapore, Yi LiNanyang Technological University, Singapore, Shang-Wei LinNanyang Technological University, Singapore, Qiang YanWeBank, n.n. DOI Pre-print Media Attached | ||
| 01:05 - 01:06 Talk | Object Detection for Graphical User Interface: Old Fashioned or Deep Learning or a Combination? Research Papers Jieshan ChenAustralian National University, Australia, Mulong XieAustralian National University, Australia, Zhenchang XingAustralian National University, Australia, Chunyang ChenMonash University, Australia, Xiwei XuData61 at CSIRO, Australia, Liming ZhuData61 at CSIRO, Australia / UNSW, Australia, Guoqiang LiShanghai Jiao Tong University, China DOI | ||
| 01:07 - 01:08 Talk | UIED: A Hybrid Tool for GUI Element Detection Tool Demos Mulong XieAustralian National University, Australia, Sidong FengAustralian National University, Australia, Zhenchang XingAustralian National University, Australia, Jieshan ChenAustralian National University, Australia, Chunyang ChenMonash University, Australia DOI | ||
| 01:09 - 01:10 Talk | WebRR: Self-Replay Enhanced Robust Record/Replay for Web Application Testing Industry Papers Zhenyue LongChina Southern Power Grid, China, Guoquan WuInstitute of Software at Chinese Academy of Sciences, China, Xiaojiang ChenChina Southern Power Grid, China, Wei ChenInstitute of Software at Chinese Academy of Sciences, China, Jun WeiState Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences & University of Chinese Academy of Sciences DOI | ||
| 01:11 - 01:30 Talk | Conversations on Testing 1 Paper Presentations Guoquan WuInstitute of Software at Chinese Academy of Sciences, China, Jieshan ChenAustralian National University, Australia, Sidong FengAustralian National University, Australia, Ye LiuNanyang Technological University, Singapore, Yixue ZhaoUniversity of Southern California, USA, Mulong XieAustralian National University, Australia, M: Corina S PasareanuCarnegie Mellon University Silicon Valley, NASA Ames Research Center | ||