Write a Blog >>
Fri 13 Nov 2020 08:33 - 08:34 at Virtual room 1 - Testing 4

Database Management Systems (DBMS) are used ubiquitously. To efficiently access data, they apply sophisticated optimizations. Incorrect optimizations can result in logic bugs, which cause a query to compute an incorrect result set. We propose Non-Optimizing Reference Engine Construction (NoREC), a fully-automatic approach to detect optimization bugs in DBMS. Conceptually, this approach aims to evaluate a query by an optimizing and a non-optimizing version of a DBMS, to then detect differences in their returned result set, which would indicate a bug in the DBMS. Obtaining a non-optimizing version of a DBMS is challenging, because DBMS typically provide limited control over optimizations. Our core insight is that a given, potentially randomly-generated optimized query can be rewritten to one that the DBMS cannot optimize. Evaluating this unoptimized query effectively corresponds to a non-optimizing reference engine executing the original query. We evaluated NoREC in an extensive testing campaign on four widely-used DBMS, namely PostgreSQL, MariaDB, SQLite, and CockroachDB. We found 159 previously unknown bugs in the latest versions of these systems, 141 of which have been fixed by the developers. Of these, 51 were optimization bugs, while the remaining were error and crash bugs. Our results suggest that NoREC is effective, general and requires little implementation effort, which makes the technique widely applicable in practice.

Fri 13 Nov

Displayed time zone: (UTC) Coordinated Universal Time change

08:30 - 09:00
08:30
2m
Talk
A Taxonomy to Assess and Tailor Risk-based Testing in Recent Testing Standards
Journal First
Juergen Grossmann Fraunhofer, Michael Felderer University of Innsbruck, Johannes Viehmann Fraunhofer FOKUS, Germany, Ina Schieferdecker Fraunhofer FOKUS & TU Berlin, Germany
08:33
1m
Talk
Detecting Optimization Bugs in Database Engines via Non-optimizing Reference Engine Construction
Research Papers
Manuel Rigger ETH Zurich, Zhendong Su ETH Zurich
DOI Pre-print Media Attached
08:35
1m
Talk
Evolutionary Improvement of Assertion Oracles
Research Papers
Valerio Terragni USI Lugano, Switzerland, Gunel Jahangirova USI Lugano, Switzerland, Paolo Tonella USI Lugano, Switzerland, Mauro Pezze USI Lugano, Switzerland
DOI
08:37
1m
Talk
Precise Learn-to-Rank Fault Localization Using Dynamic and Static Features of Target Programs
Journal First
Yunho Kim KAIST, SEOKHYEON MUN KAIST, Shin Yoo Korea Advanced Institute of Science and Technology, Moonzoo Kim KAIST and V+Lab
08:39
1m
Talk
When Does My Program Do This? Learning Circumstances of Software Behavior
Research Papers
Alexander Kampmann CISPA, Germany, Nikolas Havrikov CISPA, Germany, Ezekiel O. Soremekun CISPA, Germany, Andreas Zeller CISPA, Germany
DOI
08:41
19m
Talk
Conversations on Testing 4
Paper Presentations
Manuel Rigger ETH Zurich, Valerio Terragni USI Lugano, Switzerland, Gunel Jahangirova USI Lugano, Switzerland, Alexander Kampmann CISPA, Germany, M: Marcel Böhme Monash University, Australia