Entropy, Perception, and Relativity
  In this paper, I expand Shannon's definition of entropy into a new form of
entropy that allows integration of information from different random events.
Shannon's notion of entropy is a special case of my more general definition of
entropy. I define probability using a so-called performance function, which is
de facto an exponential distribution. Assuming that my general notion of
entropy reflects the true uncertainty about a probabilistic event, I understand
that our perceived uncertainty differs. I claim that our perception is the
result of two opposing forces similar to the two famous antagonists in Chinese
philosophy: Yin and Yang. Based on this idea, I show that our perceived
uncertainty matches the true uncertainty in points determined by the golden
ratio. I demonstrate that the well-known sigmoid function, which we typically
employ in artificial neural networks as a non-linear threshold function,
describes the actual performance. Furthermore, I provide a motivation for the
time dilation in Einstein's Special Relativity, basically claiming that
although time dilation conforms with our perception, it does not correspond to
reality. At the end of the paper, I show how to apply this theoretical
framework to practical applications. I present recognition rates for a pattern
recognition problem, and also propose a network architecture that can take
advantage of general entropy to solve complex decision problems.
