\setcounter{figure}{0}

\section{The choice of wavelet basis and the data extracted} \label{sec:appx_data}


\subsection{CIFAR-10 dataset in the wavelet space} 

About the choice for wavelet basis, it seems natural to choose a 2D basis for the images typically used for classification. A 1D wavelet basis will not be as capable to extract all the information we need from images. A 3D wavelets did not show an advantage over 2D wavelets in our numerical experiments, but it might be advantageous in certain datasets.

For the CIFAR-10 dataset, we experimented with Haar and also the first 5 Daubechies wavelets in 2D. We observed that the data extracted with Daubechies-2 was slightly more informative (higher rank), compared to Haar and Daubechies-1. But, we did not find the information extracted with Daubechies-3 to 5 to be more informative. So, our experiments on this dataset are with Daubechies-2 wavelets. This wavelet basis extracts 3,468 features from each image in this dataset, but not all of these features are influential in our pattern analysis for classification. Hence, we chose 3,000 of those coefficients using RR-QR algorithm.

We note that the end result of our analysis, e.g., the patterns extracted in the tensor decomposition were not noticeably sensitive to the choice of wavelet basis.

Figure~\ref{fig:cifar10_cd_matrix} shows only the first 1,500 columns for each $\mathcal{D}_i$, to show how the data generally looks like. But, note that we actually used 3,000 wavelet coefficients in our experiments.
 
\begin{figure}[H]
  \centering
   \includegraphics[width=0.45\linewidth]{fig_cd_wcat.jpg}
   \hspace{.2cm}
   \includegraphics[width=0.45\linewidth]{fig_cd_wdog.jpg}
  \caption{Matrix of wavelet coefficients for training images for classes of Cats (left) and Dogs (right) in CIFAR-10 dataset. This figure shows the first 1,500 most influential wavelet coefficients of images, chosen by the RR-QR algorithm.}
  \label{fig:cifar10_cd_matrix}
\end{figure}

\subsection{MNIST dataset in the wavelet space}

For MNIST, again we did not find the choice of wavelet basis to be noticeable in the patterns extracted. We found the Daubechies-1 to transform the images similar to higher order Daubechies wavelets. We note that the data extracted from this dataset has significant redundancies and linear dependencies compared to the data extracted from CIFAR-10. This is to be expected because the images of MNIST are very simple compared to images of CIFAR-10.

Figure~\ref{fig:mnist_wmatrix} shows each of the $\mathcal{D}_i$'s for the 10 classes of MNIST.

\begin{figure}[H]
  \centering
   \includegraphics[width=.25\linewidth]{mnist_tr_w0.jpg}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w1.jpg}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w2.jpg}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w3.jpg}\\
   \vspace{.2cm}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w4.jpg}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w5.jpg}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w6.jpg}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w7.jpg}\\
   \vspace{.2cm}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w8.jpg}
   \includegraphics[width=0.22\linewidth]{mnist_tr_w9.jpg}
  \caption{Matrix of wavelet coefficients for training images of 10 classes of MNIST dataset. Columns represent 100 of the most influential wavelet coefficients of images, chosen by the RR-QR algorithm. The same color scale is used for all figures.}
  \label{fig:mnist_wmatrix}
\end{figure}

Figure~\ref{fig:mnist_right} shows the right basis $V$ which is common among all classes.

\begin{figure}[H]
  \centering
   \includegraphics[width=0.4\linewidth]{mnist_right.jpg}
  \caption{The right basis $V$ obtained from Higher Order GSVD, which is common among all classes of MNIST. Images in Figure~\ref{fig:mnist_rightbasis} are reconstruction of specific columns of this matrix, back into the pixel space.}
  \label{fig:mnist_right}
\end{figure}
{"config": "arxiv", "file": "2006.09879/a1-wavelet-basis.tex"}