Write a Blog >>
Fri 13 Nov 2020 08:33 - 08:34 at Virtual room 1 - Testing 4

Database Management Systems (DBMS) are used ubiquitously. To efficiently access data, they apply sophisticated optimizations. Incorrect optimizations can result in logic bugs, which cause a query to compute an incorrect result set. We propose Non-Optimizing Reference Engine Construction (NoREC), a fully-automatic approach to detect optimization bugs in DBMS. Conceptually, this approach aims to evaluate a query by an optimizing and a non-optimizing version of a DBMS, to then detect differences in their returned result set, which would indicate a bug in the DBMS. Obtaining a non-optimizing version of a DBMS is challenging, because DBMS typically provide limited control over optimizations. Our core insight is that a given, potentially randomly-generated optimized query can be rewritten to one that the DBMS cannot optimize. Evaluating this unoptimized query effectively corresponds to a non-optimizing reference engine executing the original query. We evaluated NoREC in an extensive testing campaign on four widely-used DBMS, namely PostgreSQL, MariaDB, SQLite, and CockroachDB. We found 159 previously unknown bugs in the latest versions of these systems, 141 of which have been fixed by the developers. Of these, 51 were optimization bugs, while the remaining were error and crash bugs. Our results suggest that NoREC is effective, general and requires little implementation effort, which makes the technique widely applicable in practice.

Fri 13 Nov
Times are displayed in time zone: (UTC) Coordinated Universal Time change

08:30 - 08:32
Talk
A Taxonomy to Assess and Tailor Risk-based Testing in Recent Testing Standards
Journal First
Juergen GrossmannFraunhofer, Michael FeldererUniversity of Innsbruck, Johannes ViehmannFraunhofer FOKUS, Germany, Ina SchieferdeckerFraunhofer FOKUS & TU Berlin, Germany
08:33 - 08:34
Talk
Detecting Optimization Bugs in Database Engines via Non-optimizing Reference Engine Construction
Research Papers
Manuel RiggerETH Zurich, Zhendong SuETH Zurich
DOI Pre-print Media Attached
08:35 - 08:36
Talk
Evolutionary Improvement of Assertion Oracles
Research Papers
Valerio TerragniUSI Lugano, Switzerland, Gunel JahangirovaUSI Lugano, Switzerland, Paolo TonellaUSI Lugano, Switzerland, Mauro PezzeUSI Lugano, Switzerland
DOI
08:37 - 08:38
Talk
Precise Learn-to-Rank Fault Localization Using Dynamic and Static Features of Target Programs
Journal First
Yunho KimKAIST, SEOKHYEON MUNKAIST, Shin YooKorea Advanced Institute of Science and Technology, Moonzoo KimKAIST and V+Lab
08:39 - 08:40
Talk
When Does My Program Do This? Learning Circumstances of Software Behavior
Research Papers
Alexander KampmannCISPA, Germany, Nikolas HavrikovCISPA, Germany, Ezekiel O. SoremekunCISPA, Germany, Andreas ZellerCISPA, Germany
DOI
08:41 - 09:00
Talk
Conversations on Testing 4
Paper Presentations
Manuel RiggerETH Zurich, Valerio TerragniUSI Lugano, Switzerland, Gunel JahangirovaUSI Lugano, Switzerland, Alexander KampmannCISPA, Germany, M: Marcel BöhmeMonash University, Australia