\begin{document}
\subsection{Unsupervised pre training, basis extraction/manifold learning}
\par The first stage to learn the reduced order model will be to learn the manifold in which the solution will be moving. We now that this manifold is going to be flat for solutions of linear systems, and the best order reduction achievable in those cases will be correspondent pod system.
\par The loss function  to minimize in the pre training stage will be given by
\begin{equation}\label{latent_probability_fun_to_maximize_pre}
\underset{argmax \quad \theta}{\mathcal{L}(X,\theta)}=\underset{argmax \quad \theta}{\frac{1}{N}}\sum log(P(h_{t}\mid X_{t})=\underset{argmin \quad \theta}{\frac{1}{N}}\sum \|X_{t}-\hat{X}_{t}(h_{t}(\theta,X_{t})\|^{2}
\end{equation}\par In the shallow architecture of \ref{pod_vs_net} , having a linear activation function in the hidden layer with \textit{k-neurons} basically turns \ref{latent_probability_fun_to_maximize_pre} into a \textit{principal component} loss function \footnote{Orthogonality is not ensured here, but the spanned subspace of the k learned modes is going to be the same as the one spaned by the k principal components attained by SVD} .\\
\end{document}
{"config": "arxiv", "file": "2006.12599/equations/order_reduction/pre_training.tex"}