Towards General Purpose Vision Systems
  Computer vision systems today are primarily N-purpose systems, designed and
trained for a predefined set of tasks. Adapting such systems to new tasks is
challenging and often requires non-trivial modifications to the network
architecture (e.g. adding new output heads) or training process (e.g. adding
new losses). To reduce the time and expertise required to develop new
applications, we would like to create general purpose vision systems that can
learn and perform a range of tasks without any modification to the architecture
or learning process.
  In this paper, we propose GPV-1, a task-agnostic vision-language architecture
that can learn and perform tasks that involve receiving an image and producing
text and/or bounding boxes, including classification, localization, visual
question answering, captioning, and more. We also propose evaluations of
generality of architecture, skill-concept transfer, and learning efficiency
that may inform future work on general purpose vision. Our experiments indicate
GPV-1 is effective at multiple tasks, reuses some concept knowledge across
tasks, can perform the Referring Expressions task zero-shot, and further
improves upon the zero-shot performance using a few training samples.
