Write a Blog >>
Fri 13 Nov 2020 01:37 - 01:38 at Virtual room 1 - Documentation

The typical software tutorial includes step-by-step instructions for installing developer tools, editing files and code, and running commands. When these software tutorials are not executable, either due to missing instructions, ambiguous steps, or simply broken commands, their value is diminished. Non-executable tutorials impact developers in several ways, including frustrating learning experiences, and limiting usability of developer tools.

To understand to what extent software tutorials are executable—and why they may fail—we conduct an empirical study on over 600 tutorials, including nearly 15,000 code blocks. We find a naive execution strategy achieves an overall executability rate of only 26%. Even a human-annotation-based execution strategy—while doubling executability—still yields no tutorial that can successfully execute all steps. We identify several common executability barriers, ranging from potentially innocuous causes, such as interactive prompts requiring human responses, to insidious errors, such as missing steps and inaccessible resources. We validate our findings with major stakeholders in technical documentation and discuss possible strategies for improving software tutorials, such as providing accessible alternatives for tutorial takers, and investing in automated tutorial testing to ensure continuous quality of software tutorials.

Fri 13 Nov
Times are displayed in time zone: (UTC) Coordinated Universal Time change

01:30 - 01:32
Talk
Beyond Accuracy: Assessing Software Documentation Quality
Visions and Reflections
Christoph TreudeUniversity of Adelaide, Australia, Justin Middleton North Carolina State University, USA, Thushari AtapattuThe University of Adelaide
DOI
01:33 - 01:34
Talk
Contextual Documentation Referencing on Stack Overflow
Journal First
Sebastian Baltes QAware GmbH and The University of Adelaide, Christoph TreudeUniversity of Adelaide, Australia, Martin RobillardMcGill University
Pre-print
01:35 - 01:36
Talk
DeepCommenter: A Deep Code Comment Generation Tool with Hybrid Lexical and Syntactical Information
Tool Demos
Boao LiZhejiang University, China, Meng YanChongqing University, Xin XiaMonash University, Xing HuPeking University, Ge LiPeking University, David LoSingapore Management University
DOI
01:37 - 01:38
Talk
Docable: Evaluating the Executability of Software Tutorials
Research Papers
Samim MirhosseiniNorth Carolina State University, USA, Chris ParninNorth Carolina State University, USA
DOI Pre-print
01:39 - 01:40
Talk
RulePad: Interactive Authoring of Checkable Design Rules
Research Papers
Sahar MehrpourGeorge Mason University, USA, Thomas LaTozaGeorge Mason University, USA, Hamed SarvariGeorge Mason University, USA
DOI Pre-print
01:41 - 01:42
Talk
Software Documentation and Augmented Reality: Love or Arranged Marriage?
Visions and Reflections
Sridhar ChimalakondaIndian Institute of Technology Tirupati, Akhila Sri Manasa VenigallaIIT Tirupati, India
DOI
01:43 - 02:00
Talk
Conversations on Documentation
Paper Presentations
Akhila Sri Manasa Venigalla, Christoph TreudeUniversity of Adelaide, Australia, Sahar MehrpourGeorge Mason University, USA, Samim MirhosseiniNorth Carolina State University, USA, Sridhar ChimalakondaIndian Institute of Technology Tirupati, M: Venera ArnaoudovaWashington State University