Exploring Core Heterogeneity for GPU Accelerators

Abstract

Initially proposed as dedicated graphics processing accelerators, GPUs now find applicability in an ever-growing range of domains, including machine learning, high performance computing, automotive, and bioinformatics. Analyzing a plethora of GPU workloads has revealed that GPU resources are often underutilized. Evidently, homogeneous hardware accelerators cannot cope with the vast variety of computing requirements dictated by GPU application domains. Historically, multi-core CPUs have addressed a similar issue by introducing heterogeneity within the chip in the form of heterogeneous cores. This provides an additional degree of flexibility, allowing applications to execute on the hardware that best fits their demands. Such architectures have already been implemented and gained popularity in the domain of CPUs where they provide better power efficiency without sacrificing performance. Motivated by the aforementioned observations, this paper extends the state-of-the-art, cycle-accurate GPU simulator to support heterogeneous Streaming Multiprocessors (SMs) within the GPU chip. The proposed version provides support for tweaking the vast majority of the pipeline the original simulator already provided between the SMs unlocking a very wide design space. Modifications are also applied to the corresponding power model to simulate heterogeneous architectures. The simulator extension is coupled with an API extension which exposes programmatic control over the preferable SM type for executing each kernel. Additionally, the potential impact of heterogeneous GPU architectures on specialized domains is demonstrated through a detailed case study targeting the co-location of resource-sensitive and -insensitive applications belonging to the High Performance Computing domain. Finally, the heterogeneous GPU architecture is evaluated against homogeneous GPU baselines, demonstrating a 24.1% average speedup accompanied by a 2.6% energy gain across a set of 50 different application combinations.