LOC-SAC: A Lyapunov-Guided Online Collaborative Offloading Strategy with the SAC Algorithm

초록

Lyapunov optimization plays a pivotal role in stabilizing systems and reducing energy consumption for online collaborative computing offloading. However, existing studies have encountered the issue of instantaneous greediness during the transformation process based on Lyapunov optimization, primarily due to the continuous operations involved in queue length calculations and local optimizations. This study investigates the application of soft actor-critic in optimizing the objective function of Lyapunov-guided queue models, aiming to achieve long-term and globally optimal offloading decisions. Firstly, the Lyapunov optimization framework is employed to build the service queue model and derive the upper bound of the system objective function. Secondly, the system action of resource allocation and the state space are constructed based on the queue model. Moreover, a deep reinforcement learning problem involving queue change and a Lyapunov drift-plus-penalty is formulated to minimize the system energy consumption. Thirdly, a queue-based internal discount factor is integrated into the system reward function of the SAC algorithm, enabling the system to achieve long-term gains more efficiently. Finally, an adaptive cloud-edge service offloading strategy is proposed to ensure queue stability and minimize energy consumption without necessitating a locally optimal solution at each time slot. Experimental results demonstrate that the proposed strategy exhibits superior scalability and robustness compared to other state-of the-art and baseline approaches in online collaborative service offloading systems. © 2013 IEEE.

키워드