lojix/learning/src/main/com/thesett/aima/math/InformationTheory.java
public static double expectedI(double[] probabilities)
    {
        double result = 0.0d;

        // Loop over the probabilities for all the symbols calculating the contribution of each to the expected value.
        for (double p : probabilities)
        {
            // Calculate the information in each symbol. I(p) = - ln p and weight this by its probability of occurence
            // to get that symbols contribution to the expected value over all symbols.
            if (p > 0.0d)
            {
                result -= p * Math.log(p);
            }
        }

        // Convert the result from nats to bits by dividing by ln 2.
        return result / LN2;
    }
public static double expectedI(double[] probabilities)
    {
        double result = 0.0d;

        // Loop over the probabilities for all the symbols calculating the contribution of each to the expected value.
        for (double p : probabilities)
        {
            // Calculate the information in each symbol. I(p) = - ln p and weight this by its probability of occurence
            // to get that symbols contribution to the expected value over all symbols.
            if (p > 0.0d)
            {
                result -= p * Math.log(p);
            }
        }

        // Convert the result from nats to bits by dividing by ln 2.
        return result / LN2;
    }
For a given set of probabilities of the occurences of symbols this function calculates the expected information
content of a set of symbols given its probability distribution. The answer is expressed as a positive number of
bits.

<p>The probabilities should add up to one. This method does not provide any kind of check on this condition.

@param  probabilities The probability distribution of the symbols. The probabilities should add to one.

@return The expected information content, in bits, learned from being told a symbol drawn from the distribution.