We envision visual semantics learning (VSL), a novel methodology that derives high-level functional description of given software from its visual (graphical) outputs. By visual semantics, we mean the semantic description about the software’s behaviors that are exhibited in its visual outputs. VSL works by composing this description based on visual element labels extracted from these outputs through image/video understanding and natural language generation. The result of VSL can then support tasks that may benefit from the high-level functional description. Just like a developer relies on program understanding to conduct many of such tasks, automatically understanding software (i.e., by machine rather than by human developers) is necessary to eventually enable fully automated software engineering. Apparently, VSL only works with software that does produce visual outputs that meaningfully demonstrate the software’s behaviors. Nevertheless, learning visual semantics would be a useful first step towards automated software understanding. We outline the design of our approach to VSL and present early results demonstrating its merits.
Tue 10 NovDisplayed time zone: (UTC) Coordinated Universal Time change
01:00 - 01:30 | |||
01:00 2mTalk | A Behavioral Notion of Robustness for Software Systems Research Papers Changjian Zhang Carnegie Mellon University, USA, David Garlan Carnegie Mellon University, USA, Eunsuk Kang Carnegie Mellon University, USA Link to publication DOI Media Attached | ||
01:03 1mTalk | C2S: Translating Natural Language Comments to Formal Program Specifications Research Papers Juan Zhai Rutgers University, USA, Yu Shi Purdue University, USA, Minxue Pan Nanjing University, China, Guian Zhou Nanjing University, China, Yongxiang Liu Nanjing University, China, Chunrong Fang Nanjing University, China, Shiqing Ma Rutgers University, USA, Lin Tan Purdue University, USA, Xiangyu Zhang Purdue University DOI | ||
01:05 1mTalk | Detecting and Understanding JavaScript Global Identifier Conflicts on the Web Research Papers Mingxue Zhang Chinese University of Hong Kong, China, Wei Meng Chinese University of Hong Kong, China DOI | ||
01:07 1mTalk | PAClab: A Program Analysis Collaboratory Tool Demos Rebecca Brunner Bowling Green State University, USA, Robert Dyer University of Nebraska - Lincoln, Maria Paquin Boise State University, Elena Sherman Boise State University DOI | ||
01:09 1mTalk | Towards Learning Visual Semantics Visions and Reflections Haipeng Cai Washington State University, USA, Shiv Raj Pant Washington State University, USA, Wen Li DOI | ||
01:11 1mTalk | WebJShrink: A Web Service for Debloating Java Bytecode Tool Demos Konner Macias University of California at Los Angeles, USA, Mihir Mathur University of California, Los Angeles, Bobby Bruce University of California at Davis, USA, Tianyi Zhang Harvard University, USA, Miryung Kim University of California at Los Angeles, USA DOI | ||
01:13 17mTalk | Conversations on Analysis 1 Research Papers Juan Zhai Rutgers University, USA, Changjian Zhang Carnegie Mellon University, USA, Konner Macias University of California at Los Angeles, USA, Haipeng Cai Washington State University, USA, Mingxue Zhang Chinese University of Hong Kong, China, Robert Dyer University of Nebraska - Lincoln, M: Shin Hwei Tan Southern University of Science and Technology | ||