Efficient Human-in-the-loop System for Guiding DNNs Attention
Attention guidance is an approach to addressing dataset bias in deep
learning, where the model relies on incorrect features to make decisions.
Focusing on image classification tasks, we propose an efficient
human-in-the-loop system to interactively direct the attention of classifiers
to the regions specified by users, thereby reducing the influence of
co-occurrence bias and improving the transferability and interpretability of a
DNN. Previous approaches for attention guidance require the preparation of
pixel-level annotations and are not designed as interactive systems. We present
a new interactive method to allow users to annotate images with simple clicks,
and study a novel active learning strategy to significantly reduce the number
of annotations. We conducted both a numerical evaluation and a user study to
evaluate the proposed system on multiple datasets. Compared to the existing
non-active-learning approach which usually relies on huge amounts of
polygon-based segmentation masks to fine-tune or train the DNNs, our system can
save lots of labor and money and obtain a fine-tuned network that works better
even when the dataset is biased. The experiment results indicate that the
proposed system is efficient, reasonable, and reliable.