# CDS 110b: Linear Quadratic Optimal Control

 WARNING: This page is for a previous year. See current course homepage to find most recent page available.

This Wednesday lecture provides an overview of optimal control theory. Beginning with a review of optimization, we introduce the notion of Lagrange multipliers and provide a summary of the Pontryagin's maximum principle.

## Frequently Asked Questions

Q: In the example on Bang-Bang control discussed in the lecture, how is the control law for ${\displaystyle u}$ obtained?

Pontryagin's Maximum Principle says that ${\displaystyle u}$ has to be chosen to minimise the Hamiltonian ${\displaystyle H(x,u,\lambda )}$ for given values of ${\displaystyle x}$ and ${\displaystyle \lambda }$. In the example, ${\displaystyle H=1+({\lambda }^{T}A)x+({\lambda }^{T}B)u}$. At first glance, it seems that the more negative ${\displaystyle u}$ is the more ${\displaystyle H}$ will be minimised. And since the most negative value of ${\displaystyle u}$ allowed is ${\displaystyle -1}$, ${\displaystyle u=-1}$. However, the co-efficient of ${\displaystyle u}$ may be of either sign. Therefore, the sign of ${\displaystyle u}$ has to be chosen such that the sign of the term ${\displaystyle ({\lambda }^{T}B)u}$ is negative. That's how we come up with ${\displaystyle u=-sign({\lambda }^{T}B)}$.

Shaunak Sen, 12 Jan 06

Q: Notation question for you: In the Lecture notes from Wednesday, I'm assuming that ${\displaystyle T}$ is the final time and ${\displaystyle T}$ (superscript T) is a transpose operation. Am I correct in my assumption?

Yes, you are correct.

Jeremy Gillula, 07 Jan 05

Q: What do you mean by penalizing something, from Q>=0 "penalizes" state error?

According to the form of the quadratic cost function ${\displaystyle J}$, there are three quadratic terms such as ${\displaystyle x^{T}Qx}$, ${\displaystyle u^{T}Ru}$, and ${\displaystyle x(T)^{T}P_{1}x(T)}$. When ${\displaystyle Q\geq 0}$ and if ${\displaystyle Q}$ is relative big, the value of ${\displaystyle x}$ will have bigger contribution to the value of ${\displaystyle J}$. In order to keep ${\displaystyle J}$ small, ${\displaystyle x}$ must be relatively small. So selecting a big ${\displaystyle Q}$ can keep ${\displaystyle x}$ in small value regions. This is what the "penalizing" means.

So in the optimal control design, the relative values of ${\displaystyle Q}$, ${\displaystyle R}$, and ${\displaystyle P_{1}}$ represent how important ${\displaystyle X}$, ${\displaystyle U}$, and ${\displaystyle X(T)}$ are in the designer's concerns.

Zhipu Jin,13 Jan 03