\subsection{Numerical experiments}

\label{subsec:Numerical-experiments}

To validate our theory and demonstrate the practical applicability
of our inferential procedures, we perform a series of numerical experiments
for a variety of settings. Specifically, we set $d=100$, $p=0.2$,
and generate the ground-truth tensor $\bm{T}^{\star}=\sum_{l=1}^{r}\bm{u}_{l}^{\star}$
in a random fashion such that $\bm{u}_{l}^{\star}\overset{\mathsf{i.i.d.}}{\sim}\mathcal{N}(\bm{0},\bm{I}_{d})$.
Regarding the algorithmic parameters for nonconvex optimization (i.e.~Algorithm~\ref{alg:gd}),
we choose $L=r^{2}$, $\epsilon_{\mathsf{th}}=0.4$, $\eta_{t}\equiv3\times10^{-5}/p$,
and $t_{0}=100$. The noise components are independently drawn from
Gaussian distributions, obeying $E_{i,j,k}\sim\mathcal{N}(0,\sigma_{i,j,k}^{2}),1\leq i\leq j\leq k\leq d$
with variance $\sigma_{i,j,k}^{2}$ constructed as follows. We generate
$w_{i,j,k}\overset{\mathsf{i.i.d.}}{\sim}\mathsf{Unif}[0,1],1\leq i,j,k\leq d$
and let
\[
\sigma_{i,j,k}^{2}=\frac{\sigma^{2}w_{i,j,k}^{\beta}}{\sum_{1\leq i\leq j\leq k\leq d}w_{i,j,k}^{\beta}}\frac{d^{3}}{6},
\]
where $\beta$ represents the degree of heteroscedasticity. The noise
becomes more heteroscedastic as $\beta$ increases, and setting $\beta=0$
reduces to the homoscedastic case where the noise variances are identical
across all entries. In what follows, we set $\beta=5$.

\begin{comment}
Simulation results are reported over 100 independent trials in all
experiments.
\end{comment}


\paragraph{Tensor factor entries.}

We begin with inference for the entries of the tensor factors of interest.
Consider the construction of $95\%$ confidence intervals (i.e.~$\alpha=0.05$).
Define the normalized estimation error as follows
\[
R_{l,k}^{\mathsf{U}}:=\frac{1}{\sqrt{\left(\bm{\Sigma}_{k}\right)_{l,l}}}\left(u_{l,k}-u_{l,k}^{\star}\right),\qquad1\leq l\leq r,\,1\leq k\leq d.
\]
For each $1\leq l\leq r$ and $1\leq k\leq d$, we denote by $\mathsf{CR}_{l,k}$
the empirical coverage rate for the tensor factor entry $u_{l,k}^{\star}$
over 100 independent trials. Let $\mathsf{Mean}(\mathsf{CR})$ (resp.~$\mathsf{Std}\left(\mathsf{CR}\right)$)
denote the average (resp.~the standard deviation) of $\{\mathsf{CR}_{l,k}\}$
over all tensor factor entries. Figure~\ref{fig:QQ-u-entry} displays
the Q-Q (quantile-quantile) plots of $R_{1,1}^{\mathsf{F}}$ vs.~a
standard Gaussian random variable, and Table~\ref{table:coverage-u-entry}
summarizes the numerical results for varying $p,r$ and $\sigma$.
Encouragingly, the empirical coverage rates are all very close to
$95\%$, and the empirical distributions of the normalized estimation
errors are all well approximated by a standard Gaussian distribution.

\begin{figure}[t]
\centering

\begin{tabular}{ccc}
\includegraphics[width=0.31\textwidth]{QQ-U-11.eps} & \includegraphics[width=0.31\textwidth]{QQ-U-12.eps} & \includegraphics[width=0.31\textwidth]{QQ-U-13.eps}\tabularnewline
(a) & (b) & (c)\tabularnewline
\end{tabular}

\caption{Q-Q (quantile-quantile) plots of $R_{1,1}^{\mathsf{U}}$, $R_{1,2}^{\mathsf{U}}$
and $R_{1,3}^{\mathsf{U}}$ vs.~a standard Gaussian distribution
(where $r=4$, $p=0.2$, $\sigma=0.1$ and $\beta=5$).\label{fig:QQ-u-entry}}
\end{figure}

\begin{table}
\centering \caption{Empirical coverage rates of tensor factor entries for varying $r$
and $\sigma$.\label{table:coverage-u-entry}}
\begin{tabular}{c|c|c}
\hline 
$(r,\sigma)$ & $\mathsf{Mean}(\mathsf{CR})$ & $\mathsf{Std}(\mathsf{CR})$\tabularnewline
\hline 
$(2,10^{-2})$ & $0.9481$ & $0.0201$\tabularnewline
\hline 
$(2,10^{-1})$ & $0.9477$ & $0.0228$\tabularnewline
\hline 
$(2,1)$ & $0.9478$ & $0.0215$\tabularnewline
\hline 
$(4,10^{-2})$ & $0.9450$ & $0.0218$\tabularnewline
\hline 
$(4,10^{-1})$ & $0.9472$ & $0.0231$\tabularnewline
\hline 
$(4,1)$ & $0.9462$ & $0.0234$\tabularnewline
\hline 
\end{tabular}
\end{table}


\paragraph{Tensor entries.}

Next, we turn to inference for tensor entries. Similar to the above
case, we intend to construct $95\%$ confidence intervals. Define
\[
R_{i,j,k}^{\mathsf{T}}:=\frac{1}{\sqrt{v_{i,j,k}}}\left(T_{i,j,k}-T_{i,j,k}^{\star}\right),\qquad1\leq i\leq j\leq k\leq d.
\]
For each $1\leq i\leq j\leq k\leq d$, we record the empirical coverage
rate $\mathsf{CR}_{i,j,k}$ for the tensor entry $T_{i,j,k}^{\star}$
over 100 Monte Carlo trials. Denote by $\mathsf{Mean}(\mathsf{CR})$
(resp.~$\mathsf{Std}(\mathsf{CR})$) the average (resp.~the standard
deviation) of $\{\mathsf{CR}_{i,j,k}\}$ over entries $1\leq i\leq j\leq k\leq d$.
Figure~\ref{fig:QQ-T-entry} depicts the Q-Q (quantile-quantile)
plots of $R_{1,1,1}^{\mathsf{T}}$, $R_{1,1,2}^{\mathsf{T}}$ and
$R_{1,2,3}^{\mathsf{T}}$ vs.~a standard Gaussian random variable.
Table~\ref{table:coverage-T-entry} collects the numerical results
$\mathsf{Mean}(\mathsf{CR})$ and $\mathsf{Std}(\mathsf{CR})$ for
a variety of settings. Similar to previous experiments, the confidence
intervals and the Q-Q plots match our theoretical prediction in a
reasonably well manner.

\begin{figure}[t]
\centering

\begin{tabular}{ccc}
\includegraphics[width=0.31\textwidth]{QQ-T-111.eps} & \includegraphics[width=0.31\textwidth]{QQ-T-112.eps} & \includegraphics[width=0.31\textwidth]{QQ-T-123.eps}\tabularnewline
(a) & (b) & (c)\tabularnewline
\end{tabular}

\caption{Q-Q (quantile-quantile) plots of $R_{1,1,1}^{\mathsf{T}}$, $R_{1,1,2}^{\mathsf{T}}$
and $R_{1,2,3}^{\mathsf{T}}$ vs.~a standard Gaussian distribution
(where $r=4$, $p=0.2$, $\sigma=0.1$ and $\beta=5$).\label{fig:QQ-T-entry}}
\end{figure}

\begin{table}
\centering\caption{Empirical coverage rates of tensor entries for different $r$ and
$\sigma$.\label{table:coverage-T-entry}}

\vspace{0.5em}
\begin{tabular}{c|c|c}
\hline 
$(r,\sigma)$ & $\mathsf{Mean}(\mathsf{CR})$ & $\mathsf{Std}(\mathsf{CR})$\tabularnewline
\hline 
$(2,10^{-2})$ & $0.9494$ & $0.0218$\tabularnewline
\hline 
$(2,10^{-1})$ & $0.9513$ & $0.0218$\tabularnewline
\hline 
$(2,1)$ & $0.9475$ & $0.0222$\tabularnewline
\hline 
$(4,10^{-2})$ & $0.9434$ & $0.0225$\tabularnewline
\hline 
$(4,10^{-1})$ & $0.9494$ & $0.0220$\tabularnewline
\hline 
$(4,1)$ & $0.9494$ & $0.0219$\tabularnewline
\hline 
\end{tabular}
\end{table}


\paragraph{$\ell_{2}$ estimation accuracy.}

Finally, let us verify the Euclidean estimation guarantees we develop
for Algorithm \ref{alg:gd}. Figure~\ref{fig:l2-loss} plots the
Euclidean estimation errors of the tensor factor estimate $\bm{u}_{1}$
(resp.~the tensor estimate $\bm{T}$). In this series of experiments,
we focus on the homoskedastic case, i.e.~ $\beta=0$. As one can
see, the empirical $\ell_{2}$ risks are exceedingly close to the
Cramér--Rao lower bounds supplied in Theorem \ref{thm:optimality-L2}.

\begin{figure}[t]
\centering

\begin{tabular}{cc}
\includegraphics[width=0.31\textwidth]{loss-u.eps} & \includegraphics[width=0.31\textwidth]{loss-T.eps}\tabularnewline
(a) & (b)\tabularnewline
\end{tabular}

\caption{(a) $\ell_{2}$ estimation error of $\bm{u}_{1}$ vs.~the Cramér--Rao
lower bound; (b) Euclidean estimation errors of $\bm{T}$ vs.~the
Cramér--Rao lower bound (where $r=4$, $p=0.2$ and $\beta=0$).\label{fig:l2-loss}}
\end{figure}
{"config": "arxiv", "file": "2006.08580/experiment.tex"}