The ParalOS Framework for Heterogeneous VPUs: Scheduling, Memory Management & Application Development

Abstract

Embedded systems are presented today with the challenge of a very rapidly evolving application diversity followed by increased programming and computational complexity. As Moore’s Law is reaching a, physics induced, cul-de-sac, customised heterogeneous System-on-Chip (SoC) and more specifically Vision Processing Units (VPUs) emerge as an attractive HW solution in various application domains. However, these platforms still require sophisticated monolithic SW development to provide efficient implementations. In this context, a framework for accelerating the SW development of computationally intensive applications on VPUs, while still enabling the exploitation of their full HW potential via low-level kernel optimisations is proposed in this thesis. This framework is tailored for heterogeneous architectures and integrates a dynamic task scheduler with a high-level transparent API, a novel scratchpad memory management scheme, I/O standardisation, inter-process communication (IPC) techniques, and an insightful visual profiler. The Intel Movidius Myriad family of VPUs is used as an evaluation platform employing both synthetic benchmarks and real-world applications, which vary from Convolutional Neural Networks (CNNs) to complex computer vision algorithms for Visual Based Navigation (VBN) targeting the space industry. The results are very promising, showcasing in terms of execution time, a limited ∼8% performance overhead vs manually optimised CNN programs while achieving up to 4.2x performance gain in content-dependent applications. Regarding the Scratchpad Memory usage, a reduction of up to 33% is recorded compared to well-established memory allocators and finally the IPC cost is decreased up to 6x vs the default vendor implementation.