Is the Information Entropy the Same as the Statistical Mechanical   Entropy?
  It is shown that the standard expression for the information entropy, originally due to Shannon, is only valid for a particular set of states. For the general case of statistical mechanics, one needs to include an additional term in the expression for the entropy as a function of the probability. A simple derivation of the general formula is given. 
Phil Attard
cond-mat.stat-mech physics.chem-ph physics.data-an