title: Resize an image with offset in Python
body: <p>In my computer vision application, I have been doing image operations in this order:</p>
  
  <ol>
  <li>resize with bilinear interpolation</li>
  <li>crop</li>
  </ol>
  
  <p>This turns out to be quite inefficient when there are multiple areas of interest (AoI) in each image, with each AoI needing to be resized at a different scale.</p>
  
  <p>E.g.:</p>
  
  <pre class="lang-py prettyprint-override"><code>from PIL import Image
  import numpy as np
  from scipy.misc import imresize
  
  img_array = np.array(Image.open('foo.png').convert('L'))
  
  # [top, left, bottom, right]
  areas_of_interest = np.array([
      [10.0, 14.0, 55.0, 70.0],
      [81.0, 33.0, 170.0, 88.0],
  ])
  
  # [vertical scale, horizontal scale]
  scales = np.array([
      [1.5, 2.3],
      [1.2, 1.9],
  ])
  
  for aoi, scale in zip(areas_of_interest, scales):
      new_shape = np.array(img_array.shape) * scale
      resized_img = imresize(
          img_array,
          new_shape,
          interp='bilinear',
      )
  
      scaled_aoi = aoi * np.tile(scale, reps=2)
      cropped_img = resized_img[
          scaled_aoi[0]:scaled_aoi[2],
          scaled_aoi[1]:scaled_aoi[3],
      ]
  
      do_something_with_img(cropped_img)
  </code></pre>
  
  <p>Is there a way I can invert the order of the crop and resize operations <strong>while maintaining byte-perfect identity</strong> of the resulting image?</p>
  
  <p>If I naïvely just crop and then resize, the resulting image is not identical to the old method, for two reasons:</p>
  
  <ol>
  <li>the border is missing pixel data to perform the interpolation;</li>
  <li>the offset of the AoI w.r.t the original image is not taken into account.</li>
  </ol>
  
  <p>The result is that I get images that look very similar to the original method, but the dimensions are not necessarily identical, and the pixel values are also different due to the pixel aliasing.</p>
  
  <p>I believe that, <em>in theory</em>, it should be possible to get results identical to the original method, because of the way bilinear interpolation works.</p>
  
  <p>I <em>think</em> I need three ingredients for the solution:</p>
  
  <ol>
  <li>include a margin on each side of the AoI when cropping;</li>
  <li><em>account for the offset</em> of the AoI w.r.t. the original image when performing the interpolation;</li>
  <li>correctly crop the margin after resizing;</li>
  </ol>
  
  <p>Numbers 1 and 3 should be easy enough, but I'm not aware of any library that does number 2 for me, and I really don't feel like rolling my own bilinear image scaling in numpy or cython.</p>
  
  <p>Does anybody know of a way to get number 2 without jumping through too many hoops? I have access to PIL, the usual numpy/scipy stack, and OpenCV (cv2).</p>
  
  <p>EDIT: Adding a concrete example, run this in a Jupyter notebook</p>
  
  <pre class="lang-py prettyprint-override"><code>from PIL import Image
  import numpy as np
  from scipy.misc import imresize
  
  def nbimage(data, mode):
      '''Display raw data as a notebook inline image.
  
      Parameters:
      data: array-like object, two or three dimensions. If three dimensional,
            first or last dimension must have length 3 or 4 and will be
            interpreted as color (RGB or RGBA).
      mode: the PIL image mode
      '''
      from IPython.display import display, Image as IpyImage
      from PIL.Image import fromarray
      from StringIO import StringIO
  
      s = StringIO()
      fromarray(data, mode=mode).save(s, 'png')
      display(IpyImage(s.getvalue()))
  
  # This is http://www.pdphoto.org/PictureDetail.php?mat=pdef&amp;pg=7917
  img =Image.open('zoo_1_bg_012604.jpg').convert('L')
  
  img_array = np.array(img)
  nbimage(img_array, mode='L')
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/E6y4W.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/E6y4W.png" alt="Original image converted to grayscale"></a></p>
  
  <p>Define the areas of interest and the scales:</p>
  
  <pre class="lang-py prettyprint-override"><code>areas_of_interest = np.array([
      [155, 623, 155 + 114, 623 + 134],
      [567, 605, 136 + 567, 605 + 190],
  ], dtype=np.float)
  
  scales = np.array([
      [1.5, 2.3],
      [1.2, 1.9],
  ])
  </code></pre>
  
  <p>Resize then crop:</p>
  
  <pre class="lang-py prettyprint-override"><code>resize_then_crop = []
  
  for aoi, scale in zip(areas_of_interest, scales):
      new_shape = np.round(np.array(img_array.shape) * scale).astype(np.int)
      resized_img = imresize(img_array, new_shape, interp='bilinear')
  
      scaled_aoi = np.round(aoi * np.tile(scale, reps=2)).astype(np.int)
      cropped_img = resized_img[
          scaled_aoi[0]:scaled_aoi[2],
          scaled_aoi[1]:scaled_aoi[3],
      ]
  
      resize_then_crop.append(cropped_img)
  </code></pre>
  
  <p>Crop then resize:</p>
  
  <pre class="lang-py prettyprint-override"><code>crop_then_resize = []
  for aoi, scale in zip(areas_of_interest, scales):
      aoi = aoi.astype(np.int)
  
      cropped_img = img_array[
          aoi[0]:aoi[2],
          aoi[1]:aoi[3],
      ]
  
      new_shape = np.round(np.array(cropped_img.shape) * scale).astype(np.int)
      resized_img = imresize(cropped_img, new_shape, interp='bilinear')
  
      crop_then_resize.append(resized_img)
  </code></pre>
  
  <p>Compare the results:</p>
  
  <pre class="lang-py prettyprint-override"><code>for img1, img2 in zip(resize_then_crop, crop_then_resize):
      nbimage(img1, mode='L')
      nbimage(img2, mode='L')
      print 'Shape of resize-then-crop:', img1.shape
      print 'Shape of crop-then-resize:', img2.shape
      print 'Are they equal?', np.array_equal(img1, img2)
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/fJqxl.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/fJqxl.png" alt="Eye, resized then cropped"></a></p>
  
  <p><a href="https://i.stack.imgur.com/SeOXC.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/SeOXC.png" alt="Eye, cropped then resized"></a></p>
  
  <pre>
  Shape of resize-then-crop: (172, 308)
  Shape of crop-then-resize: (171, 308)
  Are they equal? False
  </pre>
  
  <p><a href="https://i.stack.imgur.com/NQTR4.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/NQTR4.png" alt="Claws, resized then cropped"></a></p>
  
  <p><a href="https://i.stack.imgur.com/VdGsd.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/VdGsd.png" alt="Claws, cropped then resized"></a></p>
  
  <pre>
  Shape of resize-then-crop: (164, 360)
  Shape of crop-then-resize: (163, 361)
  Are they equal? False
  </pre>