Learning from pandemics: using extraordinary events can improve disease
  now-casting models
  Online searches have been used to study different health-related behaviours,
including monitoring disease outbreaks. An obvious caveat is that several
reasons can motivate individuals to seek online information and models that are
blind to people's motivations are of limited use and can even mislead. This is
particularly true during extraordinary public health crisis, such as the
ongoing pandemic, when fear, curiosity and many other reasons can lead
individuals to search for health-related information, masking the
disease-driven searches. However, health crisis can also offer an opportunity
to disentangle between different drivers and learn about human behavior. Here,
we focus on the two pandemics of the 21st century (2009-H1N1 flu and Covid-19)
and propose a methodology to discriminate between search patterns linked to
general information seeking (media driven) and search patterns possibly more
associated with actual infection (disease driven). We show that by learning
from such pandemic periods, with high anxiety and media hype, it is possible to
select online searches and improve model performance both in pandemic and
seasonal settings. Moreover, and despite the common claim that more data is
always better, our results indicate that lower volume of the right data can be
better than including large volumes of apparently similar data, especially in
the long run. Our work provides a general framework that can be applied beyond
specific events and diseases, and argues that algorithms can be improved simply
by using less (better) data. This has important consequences, for example, to
solve the accuracy-explainability trade-off in machine-learning.
