Singular values and vectors

For any $M\times N$ matrix $\mathbf{E}$, if we construct a matrix

$$\mathbf{B} = \left[ \begin{array}{cc} \mathbf{0} & \mathbf{E}^T \mathbf{E} & \mathbf{0} \end{array} \right] ,$$ (7)

then the eigenvector equations $\mathbf{B}\mathbf{q_i}=\lambda_i\mathbf{q}_i$ can be written as

$$\mathbf{E}\mathbf{v_i} = \lambda_i \mathbf{u}_i , \quad \mathbf{E}^T\mathbf{u_i} = \lambda_i \mathbf{v}_i , \quad \mathbf{q}_i= \left[ \begin{array}{c} \mathbf{\mathbf{v}_i} \mathbf{\mathbf{u}_i} \end{array} \right] ,$$ (8)

which are equivalent to $\mathbf{E}^T\mathbf{E}\mathbf{v}_i=\lambda_i^2\mathbf{v}_i$ and $\mathbf{E}\mathbf{E}^T\mathbf{u}_i=\lambda_i^2\mathbf{u}_i$. If $\mathbf{U}$ and $\mathbf{V}$ are, respectively, the symmetric $M\times M$ and $N\times N$ matrices with columns $\mathbf{u}_i$ and $\mathbf{v}_i$ and $\mathbf{\Lambda}$ is the diagonal matrix with diagonal elements $\lambda_i$ (with zeros filling up the necessary gaps), then it follows that

$$\mathbf{E} = \mathbf{U}\mathbf{\Lambda}\mathbf{V}^T .$$ (9)

All this is relevant to the minimization problem because in the general case, the ``optimal'' solution to that problem can be written as

$$\tilde{\mathbf{x}} = \mathbf{V}'\mathbf{\Lambda}^{'-1}\mathbf{U}'\mathbf{y} ,$$ (10)

where the $'$ indicates that we have taken only the parts of these matrices that correspond to the non-zero $\lambda_i$ values.

The vectors $\mathbf{u}_i$ and $\mathbf{v}_i$ are called (backward and forward, see [2]) singular vectors while $\lambda_i$ are called the singular values of $\mathbf{E}$.

The relevance of all this to data assimilation is that the singular values and vectors of the tangent linear model provide one of the methods for generating ensembles. It can be shown that these vectors sample the ``relevant'' directions in the phase space and the proof (either mathematical, numerical, or simply by assertion) is left as an exercise to the reader. We will fearlessly move on to the next topic which has something to do with Lyapunov vectors, bred vectors and their relation to each other and to singular vectors.

(In the remaining time, Amit presented the linearization of the dynamics and will continue the discussion on 1 March, 2005. Some background material will be posted when we return from IPAM on February 27.)