Write a Blog >>
Tue 10 Nov 2020 01:39 - 01:40 at Virtual room 1 - ML Testing 1

The massive progress of machine learning has seen its application over a variety of domains in the past decade. But how do we develop a systematic, scalable and modular strategy to validate machine-learning systems? We present, to the best of our knowledge, the first approach, which provides a systematic test framework for machine-learning systems that accepts grammar-based inputs. Our OGMA approach automatically discovers erroneous behaviours in classifiers and leverages these erroneous behaviours to improve the respective models. OGMA leverages inherent robustness properties present in any well trained machine-learning model to direct test generation and thus, implementing a scalable test generation methodology. To evaluate our OGMA approach, we have tested it on three real world natural language processing (NLP) classifiers. We have found thousands of erroneous behaviours in these systems. We also compare OGMA with a random test generation approach and observe that OGMA is more effective than such random test generation by up to 489%.

Tue 10 Nov
Times are displayed in time zone: (UTC) Coordinated Universal Time change

01:30 - 01:32
Talk
Research Papers
Shenao YanRutgers University, USA, Guanhong TaoPurdue University, USA, Xuwei LiuPurdue University, USA, Juan ZhaiRutgers University, USA, Shiqing MaRutgers University, USA, Lei XuNanjing University, China, Xiangyu ZhangPurdue University
DOI
01:33 - 01:34
Talk
Research Papers
Zan WangTianjin University, China, Ming YanTianjin University, China, Junjie ChenTianjin University, China, Shuang LiuTianjin University, China, Dongdi ZhangTianjin University, China
DOI
01:35 - 01:36
Talk
Research Papers
Yuhao ZhangPeking University, Luyao RenPeking University, China, Liqian ChenNational University of Defense Technology, China, Yingfei XiongPeking University, Shing-Chi CheungHong Kong University of Science and Technology, China, Tao XiePeking University
DOI
01:37 - 01:38
Talk
Research Papers
Ziqi ZhangPeking University, China, Yuanchun LiMicrosoft Research, China, Yao GuoPeking University, Xiangqun ChenPeking University, Yunxin LiuMicrosoft Research, China
DOI
01:39 - 01:40
Talk
Journal First
Sakshi UdeshiSingapore University of Technology and Design, Sudipta ChattopadhyaySingapore University of Technology and Design
01:41 - 01:42
Talk
Research Papers
Fabrice Harel-CanadaUniversity of California at Los Angeles, USA, Lingxiao WangUniversity of California at Los Angeles, USA, Muhammad Ali GulzarUniversity of California at Los Angeles, USA, Quanquan GuUniversity of California at Los Angeles, USA, Miryung KimUniversity of California at Los Angeles, USA
DOI
01:43 - 01:44
Talk
Research Papers
Zenan LiNanjing University, China, Xiaoxing MaNanjing University, China, Chang XuNanjing University, China, Jingwei XuNanjing University, China, Chun CaoNanjing University, China, Jian LuNanjing University, China
DOI
01:45 - 02:00
Talk
Research Papers
Fabrice Harel-CanadaUniversity of California at Los Angeles, USA, Ming YanTianjin University, China, Sakshi UdeshiSingapore University of Technology and Design, Shenao YanRutgers University, USA, Yuhao ZhangPeking University, Zenan LiNanjing University, China, Ziqi ZhangPeking University, China, M: Hamid BagheriUniversity of Nebraska-Lincoln, USA