property entropy maximized messages message space equiprobable ie unpredictable case special case information entropy random variable two outcomes binary entropy function usually taken logarithmic base thus shannon sh unit math h mathrm b p p log p p log p math joint entropy two discrete random variables merely entropy pairing implies independent joint
Formal science Computer science Computer communications (networks) Information theory