A Light Discussion and Derivation of Entropy
  The expression for entropy sometimes appears mysterious - as it often is asserted without justification. This short manuscript contains a discussion of the underlying assumptions behind entropy as well as simple derivation of this ubiquitous quantity. 
Jonathon Shlens
cs.IT math.IT