Resource-Aware Data Science

Data scientists achieve more accurate predictions than ever, even in real-time, for the benefit of our society. Their productivity is fueled by the exponential evolution of hardware and the surge of machine learning tools, which hide the complexities of hardware. However, there is a widening performance gap between the software tools and modern hardware. A modern hardware infrastructure, 10X the cost of prior generation, does not even halve the time to reach an accurate deep learning model. Such poor hardware utilization makes the current surge of data science unsustainable. There is a pressing need for a Resource-Aware Data Science (RAD) infrastructure. Our goal is to address this need. To achieve this goal, we research novel techniques for collaborative scheduling of deep learning tasks onto the resources of modern commodity hardware. Our results will benefit data scientists through optimizing performance and sustainability of their tools while preserving their productivity.

Its year 2025. Sabina is a data scientist and starting a new project in NLP group of ITU about speech recognition for Danish language to be used in virtual assistants. The lab built a new server infrastructure with state-of-the-art hardware and larger scale to be able to process more data and achieve more accurate language models for this project. However, she realizes the new, and very expensive and more energy consuming, server infrastructure does not improve her results compared to the old infrastructure they have. In addition, she has to compete for the time on this infrastructure with other researchers in the NLP group. They are discussing buying more hardware, which means more money and energy consumption. Sabina is upset because of the unsustainable infrastructure they built. She is unsure how to utilize it better. Then, she remembers the Resource-Aware Data Science (RAD) project one of her colleagues, Pinar, has been working on. She goes to her for help. By adopting the tools and methodology developed in RAD project, Sabina manages to double the efficiency of her data science processing pipeline over the new server infrastructure. In addition, NLP group starts to share this server infrastructure more efficiently across different researchers obviating the need to buy more hardware.