Symbolic Network: Generalized Neural Policies for Relational MDPs
  A Relational Markov Decision Process (RMDP) is a first-order representation
to express all instances of a single probabilistic planning domain with
possibly unbounded number of objects. Early work in RMDPs outputs generalized
(instance-independent) first-order policies or value functions as a means to
solve all instances of a domain at once. Unfortunately, this line of work met
with limited success due to inherent limitations of the representation space
used in such policies or value functions. Can neural models provide the missing
link by easily representing more complex generalized policies, thus making them
effective on all instances of a given domain?
  We present SymNet, the first neural approach for solving RMDPs that are
expressed in the probabilistic planning language of RDDL. SymNet trains a set
of shared parameters for an RDDL domain using training instances from that
domain. For each instance, SymNet first converts it to an instance graph and
then uses relational neural models to compute node embeddings. It then scores
each ground action as a function over the first-order action symbols and node
embeddings related to the action. Given a new test instance from the same
domain, SymNet architecture with pre-trained parameters scores each ground
action and chooses the best action. This can be accomplished in a single
forward pass without any retraining on the test instance, thus implicitly
representing a neural generalized policy for the whole domain. Our experiments
on nine RDDL domains from IPPC demonstrate that SymNet policies are
significantly better than random and sometimes even more effective than
training a state-of-the-art deep reactive policy from scratch.
