Online Orthogonal Dictionary Learning Based on Frank-Wolfe Method
  Dictionary learning is a widely used unsupervised learning method in signal
processing and machine learning. Most existing works of dictionary learning are
in an offline manner. There are mainly two offline ways for dictionary
learning. One is to do an alternative optimization of both the dictionary and
the sparse code; the other way is to optimize the dictionary by restricting it
over the orthogonal group. The latter one is called orthogonal dictionary
learning which has a lower complexity implementation, hence, it is more
favorable for lowcost devices. However, existing schemes on orthogonal
dictionary learning only work with batch data and can not be implemented
online, which is not applicable for real-time applications. This paper proposes
a novel online orthogonal dictionary scheme to dynamically learn the dictionary
from streaming data without storing the historical data. The proposed scheme
includes a novel problem formulation and an efficient online algorithm design
with convergence analysis. In the problem formulation, we relax the orthogonal
constraint to enable an efficient online algorithm. In the algorithm design, we
propose a new Frank-Wolfe-based online algorithm with a convergence rate of
O(ln t/t^(1/4)). The convergence rate in terms of key system parameters is also
derived. Experiments with synthetic data and real-world sensor readings
demonstrate the effectiveness and efficiency of the proposed online orthogonal
dictionary learning scheme.
