Write a Blog >>
Tue 10 Nov 2020 01:09 - 01:10 at Virtual room 2 - Analysis 1

We envision visual semantics learning (VSL), a novel methodology that derives high-level functional description of given software from its visual (graphical) outputs. By visual semantics, we mean the semantic description about the software’s behaviors that are exhibited in its visual outputs. VSL works by composing this description based on visual element labels extracted from these outputs through image/video understanding and natural language generation. The result of VSL can then support tasks that may benefit from the high-level functional description. Just like a developer relies on program understanding to conduct many of such tasks, automatically understanding software (i.e., by machine rather than by human developers) is necessary to eventually enable fully automated software engineering. Apparently, VSL only works with software that does produce visual outputs that meaningfully demonstrate the software’s behaviors. Nevertheless, learning visual semantics would be a useful first step towards automated software understanding. We outline the design of our approach to VSL and present early results demonstrating its merits.

Tue 10 Nov
Times are displayed in time zone: (UTC) Coordinated Universal Time change

01:00 - 01:02
Talk
Research Papers
Changjian ZhangCarnegie Mellon University, USA, David GarlanCarnegie Mellon University, USA, Eunsuk KangCarnegie Mellon University, USA
Link to publication DOI Media Attached
01:03 - 01:04
Talk
Research Papers
Juan ZhaiRutgers University, USA, Yu ShiPurdue University, USA, Minxue PanNanjing University, China, Guian ZhouNanjing University, China, Yongxiang LiuNanjing University, China, Chunrong FangNanjing University, China, Shiqing MaRutgers University, USA, Lin TanPurdue University, USA, Xiangyu ZhangPurdue University
DOI
01:05 - 01:06
Talk
Research Papers
Mingxue ZhangChinese University of Hong Kong, China, Wei MengChinese University of Hong Kong, China
DOI
01:07 - 01:08
Talk
Tool Demos
Rebecca BrunnerBowling Green State University, USA, Robert DyerUniversity of Nebraska - Lincoln, Maria PaquinBoise State University, Elena ShermanBoise State University
DOI
01:09 - 01:10
Talk
Visions and Reflections
Haipeng CaiWashington State University, USA, Shiv Raj PantWashington State University, USA, Wen Li
DOI
01:11 - 01:12
Talk
Tool Demos
Konner MaciasUniversity of California at Los Angeles, USA, Mihir MathurUniversity of California, Los Angeles, Bobby BruceUniversity of California at Davis, USA, Tianyi ZhangHarvard University, USA, Miryung KimUniversity of California at Los Angeles, USA
DOI
01:13 - 01:30
Talk
Research Papers
Juan ZhaiRutgers University, USA, Changjian ZhangCarnegie Mellon University, USA, Konner MaciasUniversity of California at Los Angeles, USA, Haipeng CaiWashington State University, USA, Mingxue ZhangChinese University of Hong Kong, China, Robert DyerUniversity of Nebraska - Lincoln, M: Shin Hwei TanSouthern University of Science and Technology