Evaluating Memento Service Optimizations
  Services and applications based on the Memento Aggregator can suffer from
slow response times due to the federated search across web archives performed
by the Memento infrastructure. In an effort to decrease the response times, we
established a cache system and experimented with machine learning models to
predict archival holdings. We reported on the experimental results in previous
work and can now, after these optimizations have been in production for two
years, evaluate their efficiency, based on long-term log data. During our
investigation we find that the cache is very effective with a 70-80% cache hit
rate for human-driven services. The machine learning prediction operates at an
acceptable average recall level of 0.727 but our results also show that a more
frequent retraining of the models is needed to further improve prediction
accuracy.
