Multi-Agent Beam Tracking for Multi-RIS Hybrid Beamforming System Using Reinforcement Learning

초록

Reconfigurable intelligent surfaces (RISs) are transformative for next-generation wireless communications, addressing propagation losses, enhancing spectral efficiency, and extending coverage. This study proposes two multi-agent deep reinforcement learning (DRL) based algorithms for millimeter-wave (mmWave) beam tracking in multi-RIS-assisted hybrid beamforming systems: a multi-agent deep deterministic policy gradient (MADDPG) approach for continuous beam angles and a multi-agent deep Q-network (MADQN) approach for discrete angles. Both maximize sum rates by optimizing hybrid beamforming and RIS phase shifts using historical data, without real-time user or separate channel information. Simulations demonstrate their adaptability to mobility and superior performance over benchmarks.