Design Smells in Deep Learning Programs: An Empirical Study
  Nowadays, we are witnessing an increasing adoption of Deep Learning (DL)
based software systems in many industries. Designing a DL program requires
constructing a deep neural network (DNN) and then training it on a dataset.
This process requires that developers make multiple architectural (e.g., type,
size, number, and order of layers) and configuration (e.g., optimizer,
regularization methods, and activation functions) choices that affect the
quality of the DL models, and consequently software quality. An under-specified
or poorly-designed DL model may train successfully but is likely to perform
poorly when deployed in production. Design smells in DL programs are poor
design and-or configuration decisions taken during the development of DL
components, that are likely to have a negative impact on the performance (i.e.,
prediction accuracy) and then quality of DL based software systems. In this
paper, we present a catalogue of 8 design smells for a popular DL architecture,
namely deep Feedforward Neural Networks which is widely employed in industrial
applications. The design smells were identified through a review of the
existing literature on DL design and a manual inspection of 659 DL programs
with performance issues and design inefficiencies. The smells are specified by
describing their context, consequences, and recommended refactorings. To
provide empirical evidence on the relevance and perceived impact of the
proposed design smells, we conducted a survey with 81 DL developers. In
general, the developers perceived the proposed design smells as reflective of
design or implementation problems, with agreement levels varying between 47\%
and 68\%.
