Container Storage Interface (CSI) Volume Plugin for Multi-Terabyte, Unified Filesystem Hierarchies over Local Storage Kubernetes Clusters

Abstract

Fast, local NVMe storage, coupled with advanced data services through a lightweight critical data path, is what truly accelerates and secures business-critical stateful applications today. This diploma thesis endeavors to address some of these storage requirements. Motivated by the growing demand for large-scale and high-performance storage systems, this research proposes a novel solution designed to aggregate storage across different local devices and nodes into single, unified namespaces. These namespaces are exposed to and consumed by containerized workloads as persistent volumes within Kubernetes clusters through the Container Storage Interface (CSI) standard. In this thesis, we employ the MergerFS union filesystem in Kubernetes through a CSI volume plugin to combine multiple local storage spaces into a FUSE-based filesystem. We proceed to assess its performance, comparing it with existing enterprise-grade storage solutions, to determine if it is a viable option for supporting large, petabyte-level, storage pools in the cloud.