title: What does entropy mean in this context?
body: <p>I'm reading an image segmentation paper in which the problem is approached using the  paradigm "signal separation", the idea that a signal (in this case, an image) is composed of several signals (objects in the image) as well as noise, and the task is to separate out the signals (segment the image).</p>
  
  <p>The output of the algorithm is a  matrix,  <a href="https://i.stack.imgur.com/NA4yk.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/NA4yk.gif" alt="S \in R^{MxT}"></a> which represents a segmentation of an image into M components. T is the total number of pixels in the image, <a href="https://i.stack.imgur.com/HDoi7.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/HDoi7.gif" alt="s_{ij}"></a>  is the value of the source component (/signal/object) i at pixel j</p>
  
  <p>In the paper I'm reading, the authors wish to select a component m for <a href="https://i.stack.imgur.com/1iSbL.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/1iSbL.gif" alt="m \in [1,M]"></a>  which matches certain smoothness and entropy criteria. But I'm failing to understand what entropy is in this case.</p>
  
  <p>Entropy is defined as the following:</p>
  
  <p><a href="https://i.stack.imgur.com/yqW9s.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/yqW9s.gif" alt="H(s_m) = - \sum_{n=1}^{256} p_n (s_m) \cdot log_2 (p_n (s_m)), m= 1,..., M"></a></p>
  
  <p>and they say that ''<a href="https://i.stack.imgur.com/9EGap.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/9EGap.gif" alt="{p_n(s_m)}_​{n=1}^{256}"></a> are probabilities associated with the bins of the histogram of <a href="https://i.stack.imgur.com/cNT3R.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/cNT3R.gif" alt="s_m"></a>''</p>
  
  <p>The target component is a tumor and the paper reads: "the tumor related component <a href="https://i.stack.imgur.com/KU63L.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/KU63L.gif" alt="s_m"></a> with "almost" constant values is expected to have the lowest value of entropy."</p>
  
  <p>But what does low entropy mean in this context? What does each bin represent? What does a vector with low entropy look like?</p>
  
  <p><a href="https://drive.google.com/open?id=0B2C7oTadbXbjdjdTaUI0S29VVVU" rel="noreferrer">link to paper</a></p>