Machine Learning on Tiny Hardware

Today many data sources are small low-powered and hardware-constrained devices such as mobile phones, wearable or self-driving smart platforms, etc. Edge computing is a broad term that refers to computations performed on such edge devices. A powerful natural language processing model (for speech recognition, text analysis, etc.) to be deployed at an edge device can cost almost one million DKK to train in the cloud without accounting for the electricity cost and carbon footprint. To achieve sustainable progress in machine learning, it becomes increasingly important to enable techniques that get more value out of data at the edge rather than always sending the data to a remote and more powerful hardware device in the cloud for further data processing and training machine learning models. Exploiting the data closer to the source would reduce data movement. This, in turn, would reduce latency, costs, and power required to deploy machine learning models at the edge in addition to more secure data processing. However, the challenge is operating on edge devices that are way more resource-constrained compared to the servers in the cloud with CPU-GPU co-processors that sparked the machine learning advancements.
Our goal is to enable more powerful Machine learning On Tiny Hardware (MOTH) devices such as the ones found in edge settings. To reach that goal, we research novel techniques for creating dynamic and collaborative end-to-end machine learning pipelines, from data loading and pre-processing to model training and serving, that leverage the hardware heterogeneity at the edge. The outcome will be methodologies and software components that make end-to-end machine learning more secure and sustainable on resource-constrained hardware. This outcome will benefit any institution that collects data at the edge, and data scientists at those institutions creating models for various use cases (air quality monitors, hearing aids, etc.) to be deployed at the edge.

Today real-world applications such as speech recognition in virtual assistants (e.g., mobile phones) or image recognition at satellites are powered by machine learning models. Training one such model in the cloud, however, can cost almost 1 million DKK even without the electricity cost and carbon footprint. These models are then deployed on the small devices running one or more of such applications. The goal of the MOTH project is to develop novel mechanisms to get more value out of data using the computing power at these devices. In contrast to the computing resources in the cloud, these devices are more resource-constrained (one or two orders of magnitude), which creates a challenge. However, enabling more operations at these devices would reduce latency, costs, and power required to deploy machine learning models on them in addition to more secure data processing. This would benefit all institutions that collect and process data on such resource-constrained devices.