Visual and Semantic Prototypes-Jointly Guided CNN for Generalized
  Zero-shot Learning
  In the process of exploring the world, the curiosity constantly drives humans
to cognize new things. Supposing you are a zoologist, for a presented animal
image, you can recognize it immediately if you know its class. Otherwise, you
would more likely attempt to cognize it by exploiting the side-information
(e.g., semantic information, etc.) you have accumulated. Inspired by this, this
paper decomposes the generalized zero-shot learning (G-ZSL) task into an open
set recognition (OSR) task and a zero-shot learning (ZSL) task, where OSR
recognizes seen classes (if we have seen (or known) them) and rejects unseen
classes (if we have never seen (or known) them before), while ZSL identifies
the unseen classes rejected by the former. Simultaneously, without violating
OSR's assumptions (only known class knowledge is available in training), we
also first attempt to explore a new generalized open set recognition (G-OSR) by
introducing the accumulated side-information from known classes to OSR. For
G-ZSL, such a decomposition effectively solves the class overfitting problem
with easily misclassifying unseen classes as seen classes. The problem is
ubiquitous in most existing G-ZSL methods. On the other hand, for G-OSR,
introducing such semantic information of known classes not only improves the
recognition performance but also endows OSR with the cognitive ability of
unknown classes. Specifically, a visual and semantic prototypes-jointly guided
convolutional neural network (VSG-CNN) is proposed to fulfill these two tasks
(G-ZSL and G-OSR) in a unified end-to-end learning framework. Extensive
experiments on benchmark datasets demonstrate the advantages of our learning
framework.
