The Random Conditional Distribution for Higher-Order Probabilistic
  Inference
  The need to condition distributional properties such as expectation,
variance, and entropy arises in algorithmic fairness, model simplification,
robustness and many other areas. At face value however, distributional
properties are not random variables, and hence conditioning them is a semantic
error and type error in probabilistic programming languages. On the other hand,
distributional properties are contingent on other variables in the model,
change in value when we observe more information, and hence in a precise sense
are random variables too. In order to capture the uncertain over distributional
properties, we introduce a probability construct -- the random conditional
distribution -- and incorporate it into a probabilistic programming language
Omega. A random conditional distribution is a higher-order random variable
whose realizations are themselves conditional random variables. In Omega we
extend distributional properties of random variables to random conditional
distributions, such that for example while the expectation a real valued random
variable is a real value, the expectation of a random conditional distribution
is a distribution over expectations. As a consequence, it requires minimal
syntax to encode inference problems over distributional properties, which so
far have evaded treatment within probabilistic programming systems and
probabilistic modeling in general. We demonstrate our approach case studies in
algorithmic fairness and robustness.
