PETA: Photo Albums Event Recognition using Transformers Attention
  In recent years the amounts of personal photos captured increased
significantly, giving rise to new challenges in multi-image understanding and
high-level image understanding. Event recognition in personal photo albums
presents one challenging scenario where life events are recognized from a
disordered collection of images, including both relevant and irrelevant images.
Event recognition in images also presents the challenge of high-level image
understanding, as opposed to low-level image object classification. In absence
of methods to analyze multiple inputs, previous methods adopted temporal
mechanisms, including various forms of recurrent neural networks. However,
their effective temporal window is local. In addition, they are not a natural
choice given the disordered characteristic of photo albums. We address this gap
with a tailor-made solution, combining the power of CNNs for image
representation and transformers for album representation to perform global
reasoning on image collection, offering a practical and efficient solution for
photo albums event recognition. Our solution reaches state-of-the-art results
on 3 prominent benchmarks, achieving above 90\% mAP on all datasets. We further
explore the related image-importance task in event recognition, demonstrating
how the learned attentions correlate with the human-annotated importance for
this subjective task, thus opening the door for new applications.
