Occams Razor for Big Data? On Detecting Quality in Large Unstructured
  Datasets
  Detecting quality in large unstructured datasets requires capacities far
beyond the limits of human perception and communicability and, as a result,
there is an emerging trend towards increasingly complex analytic solutions in
data science to cope with this problem. This new trend towards analytic
complexity represents a severe challenge for the principle of parsimony or
Occams Razor in science. This review article combines insight from various
domains such as physics, computational science, data engineering, and cognitive
science to review the specific properties of big data. Problems for detecting
data quality without losing the principle of parsimony are then highlighted on
the basis of specific examples. Computational building block approaches for
data clustering can help to deal with large unstructured datasets in minimized
computation time, and meaning can be extracted rapidly from large sets of
unstructured image or video data parsimoniously through relatively simple
unsupervised machine learning algorithms. Why we still massively lack in
expertise for exploiting big data wisely to extract relevant information for
specific tasks, recognize patterns, generate new information, or store and
further process large amounts of sensor data is then reviewed; examples
illustrating why we need subjective views and pragmatic methods to analyze big
data contents are brought forward. The review concludes on how cultural
differences between East and West are likely to affect the course of big data
analytics, and the development of increasingly autonomous artificial
intelligence aimed at coping with the big data deluge in the near future.
