Applying Deutsch's concept of good explanations to artificial
  intelligence and neuroscience -- an initial exploration
  Artificial intelligence has made great strides since the deep learning
revolution, but AI systems still struggle to extrapolate outside of their
training data and adapt to new situations. For inspiration we look to the
domain of science, where scientists have been able to develop theories which
show remarkable ability to extrapolate and sometimes predict the existence of
phenomena which have never been observed before. According to David Deutsch,
this type of extrapolation, which he calls "reach", is due to scientific
theories being hard to vary. In this work we investigate Deutsch's hard-to-vary
principle and how it relates to more formalized principles in deep learning
such as the bias-variance trade-off and Occam's razor. We distinguish internal
variability, how much a model/theory can be varied internally while still
yielding the same predictions, with external variability, which is how much a
model must be varied to accurately predict new, out-of-distribution data. We
discuss how to measure internal variability using the size of the Rashomon set
and how to measure external variability using Kolmogorov complexity. We explore
what role hard-to-vary explanations play in intelligence by looking at the
human brain and distinguish two learning systems in the brain. The first system
operates similar to deep learning and likely underlies most of perception and
motor control while the second is a more creative system capable of generating
hard-to-vary explanations of the world. We argue that figuring out how
replicate this second system, which is capable of generating hard-to-vary
explanations, is a key challenge which needs to be solved in order to realize
artificial general intelligence. We make contact with the framework of
Popperian epistemology which rejects induction and asserts that knowledge
generation is an evolutionary process which proceeds through conjecture and
refutation.
