Do Code Review Measures Explain the Incidence of Post-Release Defects? Case Study Replications and Bayesian Networks
Aim: In contrast to studies of defects found during code review, we aim to clarify whether code reviews measures can explain the prevalence of post-release defects.
Method: We replicate McIntosh \etal’s (EMSE 2016) study that uses additive regression to model the relationship between defects and code reviews. To increase external validity, we apply the same methodology on a new software project. We discuss our findings with the first author of the original study, McIntosh. We then investigate how to reduce the impact of correlated predictors in the variable selection process and how to increase understanding of the inter-relationships among the predictors by employing Bayesian Network (BN) models.
Context: As in the original study, we use the same measures authors obtained for Qt project in the original study. We mine data from version control and issue tracker of Google Chrome and operationalize measures that are close analogs to the large collection of code, process, and code review measures used in the replicated the study.
Results: Both the data from the original study and the Chrome data showed high instability of the influence of code review measures on defects with the results being highly sensitive to variable selection procedure. Models without code review predictors had as good or better fit than those with review predictors. Replication, however, confirms with the bulk of prior work showing that prior defects, module size, and authorship have the strongest relationship to post-release defects. The application of BN models helped explain the observed instability by demonstrating that the review-related predictors do {\it not} affect post-release defects directly and showed indirect effects. For example, changes that have \emph{no review discussion} tend to be associated with files that have had many \emph{prior defects} which in turn increase the number of post-release defects.
We hope that similar analyses of other software engineering techniques may also yield a more nuanced view of their impact. Our replication package including our data and scripts is publicly available.
Fri 13 NovDisplayed time zone: (UTC) Coordinated Universal Time change
01:00 - 01:30 | |||
01:00 2mTalk | A Randomized Controlled Trial on the Effects of Embedded Computer Language Switching Research Papers P. Merlin Uesbeck University of Nevada at Las Vegas, USA, Cole S. Peterson University of Nebraska-Lincoln, USA, Bonita Sharif University of Nebraska-Lincoln, USA, Andreas Stefik University of Nevada at Las Vegas, USA DOI | ||
01:03 1mTalk | BugsInPy: A Database of Existing Bugs in Python Programs to Enable Controlled Testing and Debugging Studies Tool Demos Ratnadira Widyasari Singapore Management University, Singapore, Sheng Qin Sim Singapore Management University, Singapore, Camellia Lok Singapore Management University, Singapore, Haodi Qi Singapore Management University, Singapore, Jack Phan Singapore Management University, Singapore, Qijin Tay Singapore Management University, Singapore, Constance Tan Singapore Management University, Singapore, Fiona Wee Singapore Management University, Singapore, Jodie Ethelda Tan Singapore Management University, Singapore, Yuheng Yieh Singapore Management University, Singapore, Brian Goh Singapore Management University, Singapore, Ferdian Thung Singapore Management University, Hong Jin Kang Singapore Management University, Singapore, Thong Hoang Singapore Management University, Singapore, David Lo Singapore Management University, Eng Lieh Ouh Singapore Management University, Singapore DOI | ||
01:05 1mTalk | Do Code Review Measures Explain the Incidence of Post-Release Defects? Case Study Replications and Bayesian Networks Journal First Andrey Krutauz Concordia University, Tapajit Dey Lero - The Irish Software Research Centre and University of Limerick, Peter Rigby Concordia University, Montreal, Canada, Audris Mockus University of Tennessee - Knoxville | ||
01:07 1mTalk | On the Naturalness of Hardware Descriptions Research Papers Jaeseong Lee University of Texas at Austin, USA, Pengyu Nie University of Texas at Austin, USA, Junyi Jessy Li University of Texas at Austin, USA, Milos Gligoric University of Texas at Austin DOI | ||
01:09 1mTalk | Understanding Build Issue Resolution in Practice: Symptoms and Fix Patterns Research Papers Yiling Lou Peking University, China, Zhenpeng Chen Peking University, China, Yanbin Cao Peking University, China, Dan Hao Peking University, China, Lu Zhang Peking University, China DOI | ||
01:11 1mTalk | Understanding Type Changes in Java Research Papers Ameya Ketkar Oregon State University, USA, Nikolaos Tsantalis Concordia University, Canada, Danny Dig University of Colorado Boulder, USA DOI Pre-print Media Attached | ||
01:13 17mTalk | Conversations on Empirical 2 Paper Presentations Cole S. Peterson University of Nebraska-Lincoln, USA, Pengyu Nie University of Texas at Austin, USA, Ratnadira Widyasari Singapore Management University, Singapore, Peter Rigby Concordia University, Montreal, Canada, Yiling Lou Peking University, China, M: Kelly Blincoe University of Auckland | ||