Outperforming Good-Turing: Preliminary Report
  Estimating a large alphabet probability distribution from a limited number of
samples is a fundamental problem in machine learning and statistics. A variety
of estimation schemes have been proposed over the years, mostly inspired by the
early work of Laplace and the seminal contribution of Good and Turing. One of
the basic assumptions shared by most commonly-used estimators is the unique
correspondence between the symbol's sample frequency and its estimated
probability. In this work we tackle this paradigmatic assumption; we claim that
symbols with "similar" frequencies shall be assigned the same estimated
probability value. This way we regulate the number of parameters and improve
generalization. In this preliminary report we show that by applying an ensemble
of such regulated estimators, we introduce a dramatic enhancement in the
estimation accuracy (typically up to 50%), compared to currently known methods.
An implementation of our suggested method is publicly available at the first
author's web-page.
