Data-efficient performance learning for configurable systems

Jianmei Guo, Dingyu Yang, Norbert Siegmund, Sven Apel, Atri Sarkar, Pavel Valov, Krzysztof Czarnecki, Andrzej Wasowski, Huiqun Yu

Publikation: Artikel i tidsskrift og konference artikel i tidsskriftTidsskriftartikelForskningpeer review

Abstract

Many software systems today are configurable, offering customization of functionality by feature selection. Understanding how performance varies in terms of feature selection is key for selecting appropriate configurations that meet a set of given requirements. Due to a huge configuration space and the possibly high cost of performance measurement, it is usually not feasible to explore the entire configuration space of a configurable system exhaustively. It is thus a major challenge to accurately predict performance based on a small sample of measured system variants. To address this challenge, we propose a data-efficient learning approach, called DECART, that combines several techniques of machine learning and statistics for performance prediction of configurable systems. DECART builds, validates, and determines a prediction model based on an available sample of measured system variants. Empirical results on 10 real-world configurable systems demonstrate the effectiveness and practicality of DECART. In particular, DECART achieves a prediction accuracy of 90% or higher based on a small sample, whose size is linear in the number of features. In addition, we propose a sample quality metric and introduce a quantitative analysis of the quality of a sample for performance prediction.
OriginalsprogEngelsk
TidsskriftEmpirical Software Engineering
Antal sider42
ISSN1382-3256
DOI
StatusUdgivet - nov. 2017

Emneord

  • Parameter tuning
  • Model selection
  • Regression
  • Configurable systems
  • Performance prediction

Fingeraftryk

Dyk ned i forskningsemnerne om 'Data-efficient performance learning for configurable systems'. Sammen danner de et unikt fingeraftryk.

Citationsformater