Design and Evaluation of Approximation Techniques using Approximate Multipliers on Deep Neural Networks

Abstract

Over the past decade Convolutional Neural Networks (CNNs) emerged as the state-of-the-art approach to tackle certain Computer Vision problems such as image classification and object detection. The state-of-the-art works clearly indicate that neural networks feature an intrinsic error-resilience property. Since they often process noisy or redundant data and their users are willing to accept certain errors in many cases, the principles of approximate computing can be employed in the design of their energy efficient implementations. In this thesis, we target the development of novel approximation techniques that provide a good trade-off between between energy consumption, and inference accuracy, by performing an in-depth design space exploration to find optimal solutions. We extended the Tensorflow Approximate Layers library, which provides convolutional layers with reduced precision implemented using approximate multipliers, by designing and developing four new approximation techniques in an effort to find the optimal solutions. In the first technique we followed a non-uniform structure per layer. In the second technique, we split the number of filters in each layer into k equivalent parts and assign in each of these parts a different approximate multiplier. In the third technique, we performed approximations inside the filters by either replacing the multiplications i.e., partial products, with diverse approximate components or simply skipping this operations i.e, not executing them at all. In the fourth and final approximation technique we observed that the filter weights of each layer follow a normal distribution and based on this we proposed to execute only the multiplications that have filter weights that belong in either this range [μ-σ,μ+σ] or this [μ-2σ,μ+2σ]. The evaluation of our proposed techniques is performed in Tensorflow with Resnet-8 using the validation set from CIFAR-10 and three inexact multipliers with different perforation, by examining the inference accuracy and energy for the inference of one input image. The final results show that the third and second technique are the best since they provide a significant energy saving up to 33.5% and 30.4% compared to the accurate implementation respectively with a negligible drop in the inference accuracy.