Journal of Astronautics ›› 2022, Vol. 43 ›› Issue (1): 111-121.doi: 10.3873/j.issn.1000-1328.2022.01.013

Previous Articles     Next Articles

Incremental Reinforcement Learning Flight Control with Adaptive Learning Rate

LIU Jun hui, SHAN Jia yuan, RONG Ji li, ZHENG Xiong   

  1. 1.School of Aerospace Engineering,Beijing Institute of Technology, Beijing 100081, China; 2. Key Laboratory of Dynamics and Control of Flight Vehicle, Ministry of Education, Beijing Institute of Technology, Beijing 100081, China; 3. China Academy of Launch Vehicle Technology, Beijing 100076, China
  • Received:2021-02-05 Revised:2021-06-15 Online:2022-01-15 Published:2021-12-15

Abstract: The existing incremental reinforcement learning (IRL) flight control with preset learning rate has a high failure rate under autonomous learning and can not adapt to control flight vehicle stably with wide range of dynamic variation. An online adaptive learning rate based incremental reinforcement learning (ALRIRL) control method is proposed. First of all, based on the wavelet analysis, a cost function is constructed to evaluate the stability of the controller. Then, utilizing gradient descent method, an online iterative method of learning rate is designed to improve the convergence of IRL. Finally,the nonlinear numerical simulation and Monte Carlo shooting test are developed under random initial state and random dynamic pressure variation to demonstrate the effectiveness of the proposed ALRIRL. The simulation results show that the proposed method can adaptively adjust the learning rate according to the control performance of real time monitoring, maintain attitude stability of flight vehicle, and improve the success rate of IRL. The proposed method can reduce the dependence of IRL flight control algorithm on the preset learning rate, and broaden the application of IRL in the case of large scale dynamic parameters variation of flight vehicle.

Key words: Adaptive learning rate, Wavelet analysis, Flight control, Incremental reinforcement learning

CLC Number: