Deep Reinforcement Learning for Tail-Latency Regulation in Co-located Applications through Cooperative Core and Cache Allocation

Abstract

The amount of workloads ran on the Cloud is growing all the time. Data center operators and cloud providers have embraced workload co-location and multi- tenancy as first-class system design concerns to efficiently service and manage these massive computing needs. Current state-of-the-art resource managers place applications on the available pool of resources using standard metrics such as CPU or memory usage. As a result, current state-of-the-art resource managers fail to achieve adequate resource utilization. In this thesis, we design a resource manager that leverages deep reinforce- ment learning for its policy and uses performance monitoring counters which are a more complex metric that is able to determine a machine's current state. We showcase the impact of applying stress on different server resources and the need for a better scheduler that considers the correct metrics. We integrate our solution with OpenAI Gym, one of the most widely used tool-kits for devel- oping and comparing reinforcement learning algorithms, and we show that we can achieve higher resource usage compared to the default scheduler as well as other state-of-the-art schedulers.