Single Unit Status in Deep Convolutional Neural Network Codes for Face
  Identification: Sparseness Redefined
  Deep convolutional neural networks (DCNNs) trained for face identification
develop representations that generalize over variable images, while retaining
subject (e.g., gender) and image (e.g., viewpoint) information. Identity,
gender, and viewpoint codes were studied at the "neural unit" and ensemble
levels of a face-identification network. At the unit level, identification,
gender classification, and viewpoint estimation were measured by deleting units
to create variably-sized, randomly-sampled subspaces at the top network layer.
Identification of 3,531 identities remained high (area under the ROC
approximately 1.0) as dimensionality decreased from 512 units to 16 (0.95), 4
(0.80), and 2 (0.72) units. Individual identities separated statistically on
every top-layer unit. Cross-unit responses were minimally correlated,
indicating that units code non-redundant identity cues. This "distributed" code
requires only a sparse, random sample of units to identify faces accurately.
Gender classification declined gradually and viewpoint estimation fell steeply
as dimensionality decreased. Individual units were weakly predictive of gender
and viewpoint, but ensembles proved effective predictors. Therefore,
distributed and sparse codes co-exist in the network units to represent
different face attributes. At the ensemble level, principal component analysis
of face representations showed that identity, gender, and viewpoint information
separated into high-dimensional subspaces, ordered by explained variance.
Identity, gender, and viewpoint information contributed to all individual unit
responses, undercutting a neural tuning analogy for face attributes.
Interpretation of neural-like codes from DCNNs, and by analogy, high-level
visual codes, cannot be inferred from single unit responses. Instead, "meaning"
is encoded by directions in the high-dimensional space.
