In this short tutorial, we explain how to use Riccati’s theory to solve an LQ control problem with targets. The related MATLAB code is downloadable freely. We start considering the case of finite time horizon to later address the case of infinite time horizon.

## Finite time horizon

We consider the optimal control problem:

$\min_{u\in L^2(0,T)}J(u)=\frac12 \left[ \int_0^T \|u(t)-q(t)\|^2 dt+\beta\int_0^T \|C(x(t)-z(t))\|^2 dt+\gamma \|D(x(T)-z(T))\|^2\right],$

where

$\begin{cases} \frac{d}{dt}x(t)+Ax(t)=Bu(t)\hspace{0.6 cm} & t\in (0,T)\\ x(0)=x_0. \end{cases}$

In the above control problem, $A\in\mathcal{M}_{n\times n}$, $B\in \mathcal{M}_{n\times m}$, $C\in \mathcal{M}_{r\times n}$ and $D\in\mathcal{M}_{r\times n}$. The control $u:[0,T]\longrightarrow \mathbb{R}^m$, while the state $x:[0,T]\longrightarrow \mathbb{R}^n$. The control target is $q\in C^1([0,T];\mathbb{R}^m)$ and the state target is $z\in C^1([0,T];\mathbb{R}^n)$. $\beta\geq 0$ and $\gamma\geq 0$ are positive parameters.

By the Direct Methods in the Calculus of Variations and strict convexity, the above problem admits an unique optimal control.

We compute the optimal pair (optimal control, optimal state) by using the well-known Riccati’s theory (see, for instance, [1, Lemma 2.6] and [2, section 4.3]).

For further details regarding the algorithm, we refer to RiccatiAlgorithm.pdf.

### Example

Take

$A= \begin{pmatrix} 2&-1\\ -1&2 \end{pmatrix},\hspace{0.2 cm}B= \begin{pmatrix} 1\\ 0 \end{pmatrix},\hspace{0.2 cm}C= \begin{pmatrix} 1&0\\ 0&1 \end{pmatrix},\hspace{0.2 cm}\mbox{and}\hspace{0.2 cm}D= \begin{pmatrix} 0&0\\ 0&0 \end{pmatrix}.$

Choose $\beta=26$, $\gamma=0$, $x_0=[1.4;1.4]$, $q\equiv 0$, $z(t)=[\sin(t);\sin(t)]$ and T=10. We obtain figures state_1.png, state_2.png and control.png.

Since the parameter $\beta$ is large enough and the control acts only on the first component of the state equation

• the first component of the state is close to the target;
• the second component of the state is less close to the target;
• the control is far from its target.

The algorithm described in this guide can be employed to test the fulfillment of the turnpike property (see, e.g., [1] and [3]). In agreement with the theory, the turnpike effect is evident if:

• the targets are constants;
• (A,B) is controllable;
• (A,C) is observable, $\beta>0$ and $\gamma=0$;
• the time horizon T is large enough.

## Infinite time horizon

We consider the optimal control problem

$\min_{u\in&space;L^2_{loc}(0,\infty)}J^{\infty}(u)=\frac12&space;\left[&space;\int_0^{\infty}&space;\|u(t)-q(t)\|^2&space;dt+\beta\int_0^{\infty}&space;\|C(x(t)-z(t))\|^2&space;dt\right],$

where:

$\begin{cases}&space;\frac{d}{dt}x(t)+Ax(t)=Bu(t)\hspace{0.6&space;cm}&space;&&space;t\in&space;(0,+\infty)\\&space;x(0)=x_0.&space;\end{cases}$

In the above control problem, $A\in\mathcal{M}_{n\times n}$, $B\in \mathcal{M}_{n\times m}$ and $C\in \mathcal{M}_{r\times n}$ The control $u:(0,+\infty)\longrightarrow&space;\mathbb{R}^m$, while the state $x:[0,+\infty)\longrightarrow&space;\mathbb{R}^n$. The control target is $q\in&space;L^2_{loc}((0,+\infty);\mathbb{R}^m)$ and the state target is $z\in&space;W^{1,2}_{loc}((0,+\infty);\mathbb{R}^n)$. $\beta\geq 0$ and $\gamma\geq 0$ are positive parameters.

Assumptions

• the targets $q$ and $z$ satisfies the equation

$\frac{d}{dt}z(t)+Az(t)=Bq(t)\hspace{0.6&space;cm}&space;t\in&space;(0,+\infty);$

• (A,B) is controllable and (A,C) is observable.

The above assumptions guarantee the existence of a control $u\in&space;L^2_{loc}((0,+\infty);\mathbb{R}^m)$ such that $J^{\infty}(u)<+\infty$. By the Direct Methods in the Calculus of Variations and strict convexity, the above problem admits an unique optimal control.

Further details are available in the second section of RiccatiAlgorithm.pdf.

• Dario Pighin

## References

[1] A. PORRETTA and E. ZUAZUA, Long time versus steady state optimal control, SIAM Journal on Control and Optimization, 51 (2013), pp. 4242–4273.

[2] E. TRÉLAT, Contrôle optimal: théorie & applications, Vuibert, 2008.

[3] E. TRÉLAT and E. ZUAZUA, The turnpike property in finite-dimensional nonlinear optimal control, Journal of Differential Equations, 258 (2015), pp. 81–114.

## Acknowledgments

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No. 694126-DyCon).

DyCon Webpage