Maximum Probability Theorem: A Framework for Probabilistic Learning
  We present a theoretical framework of probabilistic learning derived by
Maximum Probability (MP) Theorem shown in the current paper. In this
probabilistic framework, a model is defined as an event in the probability
space, and a model or the associated event -- either the true underlying model
or the parameterized model -- have a quantified probability measure. This
quantification of a model's probability measure is derived by the MP Theorem,
in which we have shown that an event's probability measure has an upper-bound
given its conditional distribution on an arbitrary random variable. Through
this alternative framework, the notion of model parameters is encompassed in
the definition of the model or the associated event. Therefore, this framework
deviates from the conventional approach of assuming a prior on the model
parameters. Instead, the regularizing effects of assuming prior over parameters
is seen through maximizing probabilities of models or according to information
theory, minimizing the information content of a model. The probability of a
model in our framework is invariant to reparameterization and is solely
dependent on the model's likelihood function. Also, rather than maximizing the
posterior in a conventional Bayesian setting, the objective function in our
alternative framework is defined as the probability of set operations (e.g.
intersection) on the event of the true underlying model and the event of the
model at hand. Our theoretical framework, as a derivation of MP theorem, adds
clarity to probabilistic learning through solidifying the definition of
probabilistic models, quantifying their probabilities, and providing a visual
understanding of objective functions.
