CDS 110b: Linear Quadratic Optimal Control

 WARNING: This page is for a previous year. See current course homepage to find most recent page available.

This Wednesday lecture provides an overview of optimal control theory. Beginning with a review of optimization, we introduce the notion of Lagrange multipliers and provide a summary of the Pontryagin's maximum principle.

Course Materials

Q: In the example on Bang-Bang control discussed in the lecture, how is the control law for $$u$$ obtained?

Pontryagin's Maximum Principle says that $$u$$ has to be chosen to minimise the Hamiltonian $$H(x,u,\lambda)$$ for given values of $$x$$ and $$\lambda$$. In the example, $$H = 1 + ({\lambda}^TA)x + ({\lambda}^TB)u$$. At first glance, it seems that the more negative $$u$$ is the more $$H$$ will be minimised. And since the most negative value of $$u$$ allowed is $$-1$$, $$u=-1$$. However, the co-efficient of $$u$$ may be of either sign. Therefore, the sign of $$u$$ has to be chosen such that the sign of the term $$({\lambda}^TB)u$$ is negative. That's how we come up with $$u = -sign({\lambda}^TB)$$.

Shaunak Sen, 12 Jan 06

Q: Notation question for you: In the Lecture notes from Wednesday, I'm assuming that $$T$$ is the final time and $$T$$ (superscript T) is a transpose operation. Am I correct in my assumption?

Yes, you are correct.

Jeremy Gillula, 07 Jan 05

Q: What do you mean by penalizing something, from Q>=0 "penalizes" state error?

According to the form of the quadratic cost function $$J$$, there are three quadratic terms such as $$x^T Q x$$, $$u^T R u$$, and $$x(T)^T P_1 x(T)$$. When $$Q \geq 0$$ and if $$Q$$ is relative big, the value of $$x$$ will have bigger contribution to the value of $$J$$. In order to keep $$J$$ small, $$x$$ must be relatively small. So selecting a big $$Q$$ can keep $$x$$ in small value regions. This is what the "penalizing" means.

So in the optimal control design, the relative values of $$Q$$, $$R$$, and $$P_1$$ represent how important $$X$$, $$U$$, and $$X(T)$$ are in the designer's concerns.

Zhipu Jin,13 Jan 03