Deep Reinforcement Learning Mechanisms for Efficient Dynamic Resource Management in Cloud-Native Applications

Abstract

Effectively managing computational resources in modern cloud-native infrastructures is a challenging task due to their elastic scalability and the heterogeneous nature of their workloads. This thesis introduces a two-phase framework for intelligent resource allocation in hierarchical edge-cloud systems, integrating deep reinforcement learning (DRL) with predictive modeling using deep learning techniques. In the first phase, neural sequence models—specifically Long Short-Term Memory (LSTM) and Transformer architectures—are trained on the Alibaba Cloud Trace dataset to forecast workload telemetry, such as CPU utilization, in containerized batch workloads. Experimental comparisons across standard regression metrics (RMSE, MAE, and R2) consistently show that the Transformer model outperforms LSTM in both accuracy and temporal consistency. With a forecasting accuracy of R2 = 0.850 and minimal variance across test scenarios, the Transformer demonstrates strong capabilities in capturing complex temporal patterns, making it highly suitable for proactive autoscaling strategies. The second phase formulates the resource allocation task as a Markov Decision Process (MDP) and utilizes a Deep Q-Network (DQN) agent to learn optimal job placement policies. A custom simulation environment— developed using Gymnasium and Ray RLlib—models real-world infrastructure with near-edge, far-edge, and cloud clusters, each characterized by unique latency, cost, and energy profiles. The environment incorporates dynamic job arrivals, priority-aware scheduling, and latency-sensitive reward shaping. To ensure scalability and reflect distributed system conditions, the framework is extended into a multi-agent system where independent gateways operate concurrently over shared infrastructure. Comprehensive evaluation of the system under both real and Transformer-generated synthetic workloads demonstrates high allocation success rates, efficient edge resource utilization, low average job cost, and strong energy performance. Notably, Transformer-generated workloads yield enhanced policy stability (20–25% improvement) and more realistic workload characteristics compared to other synthetic generation methods. Performance indicators such as reward-to-cost ratio and reward-per-kWh validate the economic and operational advantages of the DRL-based approach under synthetic testing. Overall, the proposed framework showcases the potential of integrating advanced Transformer-based workload forecasting with DRL for real-time, adaptive resource management in cloud-native environments. The results support the development of intelligent autoscaling mechanisms capable of satisfying latency, cost, and energy requirements in emerging edge-cloud applications through accurate prediction and stable policy execution.