Synthetic Ground Truth Generation for Evaluating Generative Policy
  Models
  Generative Policy-based Models aim to enable a coalition of systems, be they
devices or services to adapt according to contextual changes such as
environmental factors, user preferences and different tasks whilst adhering to
various constraints and regulations as directed by a managing party or the
collective vision of the coalition. Recent developments have proposed new
architectures to realize the potential of GPMs but as the complexity of systems
and their associated requirements increases, there is an emerging requirement
to have scenarios and associated datasets to realistically evaluate GPMs with
respect to the properties of the operating environment, be it the future
battlespace or an autonomous organization. In order to address this
requirement, in this paper, we present a method of applying an agile knowledge
representation framework to model requirements, both individualistic and
collective that enables synthetic generation of ground truth data such that
advanced GPMs can be evaluated robustly in complex environments. We also
release conceptual models, annotated datasets, as well as means to extend the
data generation approach so that similar datasets can be developed for varying
complexities and different situations.
