Beyond Continuous Delivery: An Empirical Investigation of Continuous Deployment Challenges

Mojtaba Shahin, Muhammad Ali Babar, Mansooreh Zahedi, Liming Zhu

Research output: Conference Article in Proceeding or Book/Report chapterArticle in proceedingsResearchpeer-review


Context: A growing number of software organizations have been adopting Continuous DElivery (CDE) and Continuous Deployment (CD) practices. Researchers have started investing significant efforts in studying different aspects of CDE and CD. Many studies refer to CDE (i.e., where an application is potentially capable of being deployed) and CD (i.e., where an application is automatically deployed to production on every update) as synonyms and do not distinguish them from each other. Despite CDE being successfully adopted by a large number of organizations, it is not empirically known why organizations still are unable or demotivated to have automatic and continuous deployment (i.e., CD practice). Goal: This study aims at empirically investigating and classifying the factors that may impact on adopting and implementing CD practice. Method: We conducted a mixed-method empirical study consisting of interviewing 21 software practitioners, followed by a survey with 98 respondents. Results: Our study reveals 11 confounding factors that limit or demotivate software organizations to push changes automatically and continuously to production. The most important ones are "lack of automated (user) acceptance test", "manual quality check", "deployment as business decision", "insufficient level of automated test coverage", and "highly bureaucratic deployment process". Conclusion: Our findings highlight several areas for future research and provide suggestions for practitioners to streamline deployment process.
Original languageEnglish
Title of host publication2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)
Number of pages10
Publication date2017
ISBN (Print)978-1-5090-4039-1/17
Publication statusPublished - 2017


  • continuous delivery
  • Continuous Deployment
  • DevOps
  • empirical study


Dive into the research topics of 'Beyond Continuous Delivery: An Empirical Investigation of Continuous Deployment Challenges'. Together they form a unique fingerprint.

Cite this