OS mechanisms for efficient memory management and filesystems for serverless functions in virtual machines

Abstract

The Function-as-a-Service (FaaS) execution model has dominated cloud computing. FaaS applications are developed as standalone functions for which users provide the source code, while providers secure the dependencies (e.g., runtime) and the necessary resources (e.g., memory, CPU) for their execution. Programming is simplified: (i) developers are alleviated from the burden to manage an entire system and (ii) resource management becomes easier, i.e., providers can scale dynamically and transparently the resources allocated per function based on the incoming load of requests. Typically, resource allocation is controlled by increasing or decreasing the number of function instances that serve requests. Thus, resource elasticity is one of the key defining characteristics of FaaS. Moreover, sharing common resources among multiple instances further improves resource efficiency. This thesis focuses on the OS mechanisms related to the filesystems and the memory management of functions executed inside Virtual Machines (VMs). An initial examination of sandboxing runtimes used in serverless environments revealed overhead in function cold starts, caused by the virtualization barrier arising from differing filesystem semantics between the host and the guest. To that end, this thesis introduces and implements a storage driver for functions executed as containers inside Virtual Machines, while maintaining compatibility with host-native execution. The approach narrows the gap between guest containers and host containers by efficiently sharing function dependencies through the host page cache. This is achieved by constructing a block-based layered container view on the host, in which each layer is pluggable and shareable across multiple VMs. As a result, each guest can prepare a union filesystem view of the container without inheriting the host’s filesystem semantics, while still benefiting from shared cached data at the host level. Evaluation results show that, in contrast to other block-based drivers, the proposed solution can boot 80 lightweight VM containers in under 4 seconds, owing to shared host cache buffers and a 97.51% reduction in disk I/O. Furthermore, the multi-container-per-VM model (N:1) is identified as the most suitable approach for improving shareability between function instances. In this model, compared to the single-container-per-VM (1:1), more than one function instances share the same VM and initialized runtime and thereby achieve better latencies and more efficient resource usage. However this model naturally requires larger VMs, that will be able to accommodate the deployment of multiple concurrent function instances. These resources remain allocated –albeit idle– even when the load is low, as function instances terminate and get reclaimed. While compute resources assigned to VM sandboxed functions can be seamlessly adjusted on the fly, memory elasticity remains challenging. Hot(un)plugging memory resources suffers from long reclamation latencies and occupies valuable CPU resources. The obliviousness of the OS memory manager to hot-plugged memory is identified as the key factor hindering hot-unplug performance. To address this challenge, a novel mechanism is introduced for fast and efficient VM memory hot-(un)plug, specifically targeting VM-sandboxed serverless functions. The central insight behind the new mechanism is the segregation of hot-plugged memory regions from regular VM memory, thereby bounding the lifetime of allocations within these regions and enabling their rapid reclamation. The mechanism is implemented as an extension to the OS memory manager. Experimental evaluation demonstrates that it achieves an order-of-magnitude improvement over the state of the art, maintaining bounded tail latencies during memory reclamation and enabling sub-second reclamation of multiple GiBs of memory under realistic FaaS load. Overall, this thesis revisits today’s OS mechanisms involved in FaaS sandboxes filesystem preparation and memory management and proposes techniques that extend them in order to (i) scale up faster when the incoming load suggests it, (ii) share resources more efficiently among multiple sandboxes and (iii) reclaim resources faster when scaling down.