Near optimal reinforcement learning control of linear time-delay systems

Abstract

The control of time-delay linear systems is a common challenge in real-world autonomous system applications. Time delays can negatively affect the stability and performance of controllers, necessitating the exploration of alternative approaches. In this context, this paper proposes the implementation of a reinforcement learning (RL)-based policy iteration (PI) algorithm by transforming a time-delay system into an augmented state approximate linear system. This transformation is achieved by segmenting the delay into discrete delays, which allows for the application of RL algorithms to solve the optimal control problem. Through simulation studies in scenarios such as chemical plants and regenerative chatter systems, the effectiveness of the proposed methodology is demonstrated and associated challenges are identified. This approach offers a solution to address the complexity of controller design for time-delay systems, facilitating system management through approximations.