The Impact of Continuous Code Quality Assessment on Defects

Continuous Code Quality Assessment (CCQA) tools
promise that increasing code quality leads to fewer defects, i.e.,
that software quality from the user view can be increased by
increasing quality of the product. Currently, there is limited
evidence on that application of CCQA tools, such as, SonarCloud
(SC), during software development actually reduces the amount
of defects over time. In this paper we study five open-source
projects that adopt SonarCloud (SC) for CCQA and we compare
frequencies of defect reports before and after adoption of SC. For
only one project (Apache Ratis), we find a statistically significant
decrease of defects after adoption of the tool. After closer
investigation we find, that this decrease is likely just a coincidence
and not caused by the adoption of SC and adherence to its code
quality recommendations. In general, we find no evidence for
that application of a CCQA tool increases product quality.