IReEn: Reverse-Engineering of Black-Box Functions via Iterative Neural
  Program Synthesis
  In this work, we investigate the problem of revealing the functionality of a
black-box agent. Notably, we are interested in the interpretable and formal
description of the behavior of such an agent. Ideally, this description would
take the form of a program written in a high-level language. This task is also
known as reverse engineering and plays a pivotal role in software engineering,
computer security, but also most recently in interpretability. In contrast to
prior work, we do not rely on privileged information on the black box, but
rather investigate the problem under a weaker assumption of having only access
to inputs and outputs of the program. We approach this problem by iteratively
refining a candidate set using a generative neural program synthesis approach
until we arrive at a functionally equivalent program. We assess the performance
of our approach on the Karel dataset. Our results show that the proposed
approach outperforms the state-of-the-art on this challenge by finding an
approximately functional equivalent program in 78% of cases -- even exceeding
prior work that had privileged information on the black-box.
