Rubik's Cube Operator: A Plug And Play Permutation Module for Better
  Arranging High Dimensional Industrial Data in Deep Convolutional Processes
  The convolutional neural network (CNN) has been widely applied to process the
industrial data based tensor input, which integrates data records of
distributed industrial systems from the spatial, temporal, and system dynamics
aspects. However, unlike images, information in the industrial data based
tensor is not necessarily spatially ordered. Thus, directly applying CNN is
ineffective. To tackle such issue, we propose a plug and play module, the
Rubik's Cube Operator (RCO), to adaptively permutate the data organization of
the industrial data based tensor to an optimal or suboptimal order of
attributes before being processed by CNNs, which can be updated with subsequent
CNNs together via the gradient-based optimizer. The proposed RCO maintains K
binary and right stochastic permutation matrices to permutate attributes of K
axes of the input industrial data based tensor. A novel learning process is
proposed to enable learning permutation matrices from data, where the
Gumbel-Softmax is employed to reparameterize elements of permutation matrices,
and the soft regularization loss is proposed and added to the task-specific
loss to ensure the feature diversity of the permuted data. We verify the
effectiveness of the proposed RCO via considering two representative learning
tasks processing industrial data via CNNs, the wind power prediction (WPP) and
the wind speed prediction (WSP) from the renewable energy domain. Computational
experiments are conducted based on four datasets collected from different wind
farms and the results demonstrate that the proposed RCO can improve the
performance of CNN based networks significantly.
