Efficient Deep Learning deployments for Edge AI on resource-constrained environments using hls4ml

Abstract

The present thesis was carried out within the framework of the CERN Technical Student program under the guidance of Mr. Dionysios Pnevmatikatos, Professor of the Faculty of Electrical and Computer Engineering at the National Technical University of Athens and Drs. Sioni Paris Summers and Maurizio Pierini, CERN researchers in the CMS experiment.

Edge Machine Learning (EML) involves running Machine Learning algorithms on devices located near the edges of a data network. This technique allows data to be processed close to the source, thus reducing network latency, power consumption, and the amount of data that needs to be transferred to central systems.

This paper focuses on two EML applications, involving the implementation and execution of Neural Networks on FPGA (Field-Programmable Gate Arrays) and ASIC (Application-Specific Integrated Circuit) devices, respectively.

In the context of the Edge SpAIce project, the objective is to run Convolutional Neural Networks on an Earth observation nano-satellite for marine pollution detection. By using an Image Segmentation system, images containing pollution are detected, and only the relevant images are sent to Earth. This reduces the necessary bandwidth of the satellite antennas, thus reducing the cost, system complexity and energy consumption. To deploy Neural Networks at the Edge on FPGAs and ASICs, CERN is developing the hls4ml (High-level Synthesis for Machine Learning) library. In order to perform highly accurate Image Segmentation from satellite data, models with millions of parameters are required, thus hls4ml needs to be enhanced and repurposed from Physics applications to Earth observation. Results on accuracy, resource and energy consumption, latency and processing rate are presented in detail.Within the PixESL project, the aim is to integrate a Neural Network directly into the silicon of pixel detectors used in CERN experiments, medical applications and space dosimetry. The Neural Network receives multiple signals from the pixel hits and detects whether they come from one or more particles and extracts the angles of incidence and charge of each particle. The main work analyzed in the scope of this thesis is the proposed approaches to solve this task, including the design of a Neural Network architecture, as well as the hardware implementation of the network and synthesis results.