Towards a 3-tiered space-air-ground network with reinforcement learning

A Space-Air-Ground Integrated Network (SAGIN) is a novel networking concept that integrates satellite networks, aerial networks and terrestrial networks into a 3-tiered network. It has been developed as an adaptable computing and traffic model in the present decade. In addition to various benefits, there are some unprecedented challenges in SAGIN, and reliability is one among them. The wireless communication network requires stable communication between Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs). For ensuring reliable communication links, Massive Multiple-Input Multiple-Output (MaMIMO) is used with the deployed aerial vehicles, and the mobility of UGVs can be controlled by UAV, providing device-to-device (D2D) communication between vehicular nodes, so that no interruption can occur. This work involves developing a 3-Tier D2D Architecture, comprised of network links UGVs, network links UAVs, and the combined model of both and also links of the satellite groups at lower orbits. The UAV senses the environment and transmits the data to its operator for appropriate decision-making. This work focuses on incorporating D2D communications in SAGIN for ensuring reliability via Reinforcement Learning (RL) based on Markov process. We derive the optimal numbers of transmitting nodes used for communication links, represent the Markov State transition and provide the Bellman equation for our model. The main objective is to maximize Energy Efficiency (EE), under the limitations of Spectral Efficiency (SE). Performance evaluations are employed to assess reliability over the links in the corresponding proposed architecture. The simulation results with water-filling optimization show that the proposed model achieves enhanced EE and SE per D2D link.