ITU

How Does the Degree of Variability Affect Bug-Finding?

Research output: Conference Article in Proceeding or Book/Report chapterArticle in proceedingsResearchpeer-review

View graph of relations

Software projects embrace variability to increase adaptability and to lower cost; however, others blame variability for increasing complexity and making reasoning about programs more difficult. We carry out a controlled experiment to quantify the impact of variability on debugging of preprocessor-based programs. We measure speed and precision for bug finding tasks defined at three different degrees of variability on several subject programs derived from real systems.
The results show that the speed of bug finding decreases linearly with the number of features, while effectiveness of finding bugs is relatively independent of the degree of variability. Still, identifying the set of configurations in which the bug manifests itself is difficult already for a low number of features. Surprisingly, identifying the exact set of affected configurations appears to be harder than finding the bug in the first place. The difficulty in reasoning about several configurations is a likely reason why the variability bugs are actually introduced in configurable programs.
We hope that the detailed findings presented here will inspire the creation of programmer support tools addressing the challenges faced by developers when reasoning about configurations, contributing to more effective debugging and, ultimately, fewer bugs in highly-configurable systems.
Original languageEnglish
Title of host publicationICSE '16 Proceedings of the 38th International Conference on Software Engineering
Number of pages12
PublisherAssociation for Computing Machinery
Publication date14 May 2016
Pages679-690
ISBN (Print)978-1-4503-3900-1
DOIs
Publication statusPublished - 14 May 2016

ID: 81023632