## Adaptive Dynamic Programming for Control: Algorithms and StabilityThere are many methods of stable controller design for nonlinear systems. In seeking to go beyond the minimum requirement of stability, Adaptive Dynamic Programming in Discrete Time approaches the challenging topic of optimal control for nonlinear systems using the tools of adaptive dynamic programming (ADP). The range of systems treated is extensive; affine, switched, singularly perturbed and time-delay nonlinear systems are discussed as are the uses of neural networks and techniques of value and policy iteration. The text features three main aspects of ADP in which the methods proposed for stabilization and for tracking and games benefit from the incorporation of optimal control methods: • infinite-horizon control for which the difficulty of solving partial differential Hamilton–Jacobi–Bellman equations directly is overcome, and proof provided that the iterative value function updating sequence converges to the infimum of all the value functions obtained by admissible control law sequences; • finite-horizon control, implemented in discrete-time nonlinear systems showing the reader how to obtain suboptimal control solutions within a fixed number of control steps and with results more easily applied in real systems than those usually gained from infinite-horizon control; • nonlinear games for which a pair of mixed optimal policies are derived for solving games both when the saddle point does not exist, and, when it does, avoiding the existence conditions of the saddle point. Non-zero-sum games are studied in the context of a single network scheme in which policies are obtained guaranteeing system stability and minimizing the individual performance function yielding a Nash equilibrium. In order to make the coverage suitable for the student as well as for the expert reader, Adaptive Dynamic Programming in Discrete Time: • establishes the fundamental theory involved clearly with each chapter devoted to a clearly identifiable control paradigm; • demonstrates convergence proofs of the ADP algorithms to deepen understanding of the derivation of stability and convergence with the iterative computational methods used; and • shows how ADP methods can be put to use both in simulation and in real applications. This text will be of considerable interest to researchers interested in optimal control and its applications in operations research, applied mathematics computational intelligence and engineering. Graduate students working in control and operations research will also find the ideas presented here to be a source of powerful methods for furthering their study. |

### What people are saying - Write a review

We haven't found any reviews in the usual places.

### Contents

1 | |

Optimal State Feedback Control for DiscreteTime Systems | 27 |

Optimal Tracking Control for DiscreteTime Systems | 108 |

Optimal State Feedback Control of Nonlinear Systems with Time Delays | 161 |

Optimal Tracking Control of Nonlinear Systems with Time Delays | 201 |

Optimal Feedback Control for ContinuousTime Systems via ADP | 222 |

### Other editions - View all

Adaptive Dynamic Programming for Control: Algorithms and Stability Huaguang Zhang,Derong Liu,Yanhong Luo,Ding Wang No preview available - 2015 |

Adaptive Dynamic Programming for Control: Algorithms and Stability Huaguang Zhang,Derong Liu,Yanhong Luo,Ding Wang No preview available - 2012 |

### Common terms and phrases

action network action NN adaptive dynamic programming ADP method affine nonlinear approximation error call admission control compute continuous-time control input control law sequence control pair control policy control scheme convergence cost functional costate critic network deﬁned deﬁnite denoted desired trajectory differential games discrete-time discrete-time systems finite GDHP HJB equation IEEE Trans Autom IEEE Trans Neural IEEE Trans Syst implement initial iteration step iterative ADP algorithm learning rate Lemma Lewis FL linear Lipschitz continuous Lyapunov function mathematical induction matrix minimize model network neural networks nonlinear systems optimal control law optimal control problem optimal feedback control optimal state feedback optimal tracking control optimal value function output parameters player Proof Q-learning reinforcement learning saddle point satisﬁes shown in Fig simulation solve stability Syst Man Cybern tracking control problem tracking error Trans Neural Netw utility function value function sequence vector xT(k zero-sum games Zhang HG