HARK Side of Deep Learning -- From Grad Student Descent to Automated
  Machine Learning
  Recent advancements in machine learning research, i.e., deep learning,
introduced methods that excel conventional algorithms as well as humans in
several complex tasks, ranging from detection of objects in images and speech
recognition to playing difficult strategic games. However, the current
methodology of machine learning research and consequently, implementations of
the real-world applications of such algorithms, seems to have a recurring
HARKing (Hypothesizing After the Results are Known) issue. In this work, we
elaborate on the algorithmic, economic and social reasons and consequences of
this phenomenon. We present examples from current common practices of
conducting machine learning research (e.g. avoidance of reporting negative
results) and failure of generalization ability of the proposed algorithms and
datasets in actual real-life usage. Furthermore, a potential future trajectory
of machine learning research and development from the perspective of
accountable, unbiased, ethical and privacy-aware algorithmic decision making is
discussed. We would like to emphasize that with this discussion we neither
claim to provide an exhaustive argumentation nor blame any specific institution
or individual on the raised issues. This is simply a discussion put forth by
us, insiders of the machine learning field, reflecting on us.
