title: Error using OpenCV perspectiveTransform and cropping an image [ Python ]
body: <p>EDIT: updated post. Minimum reproducible example code included, with images that produce the error mentioned in this post.</p>
  <p>I'm new and unexperienced with openCV. I don't have a deep understanding of the openCV data structures, or what data structures are returned from an OpenCV function. So far I've been able to get by piecing together different code segments found with google searches. I've run into a problem where this is biting me, because I don't understand what's going on behind the scenes with cv2.findHomography and cv2.perspectiveTransform.</p>
  <p><strong>Goal of the Code</strong>:
  take image1 and image2 as arguments
  if it exists, find the region in image1 that image2 matches. (using SIFT, not template matching)
  crop image1 to the region that matches image2</p>
  <p><strong>Current Status</strong>:
  the code works for most images, but occasionally I get an error thrown where the cropped image size
  dimensions are 0 in either height or width. I attempt to check for this condition, by checking the
  width and height from the perspectiveTransform coordinates but not catching
  all the invalid cropped images.</p>
  <p><strong>Problem</strong>:
  An invalid cropped image is created, which causes an error when the image is displayed with
  cv2.imshow()</p>
  <p>Code:</p>
  <pre><code>import cv2
  import numpy as np
  import matplotlib.pyplot as plt
  import time
  from time import time
  from timeit import Timer
  from Stat_File import FileHelper
  import os
  import win32api, win32con
  import win32gui
  import ctypes
  
  
  user32 = ctypes.windll.user32
  SCREEN_WIDTH =  user32.GetSystemMetrics(0)
  SCREEN_HEIGHT =  user32.GetSystemMetrics(1)
  
  MIN_MATCH_COUNT = 10
  
  cv2.imread(&quot;./img/verify_page/single/Captcha_Picture/button.png&quot;)
  
  original = &quot;./img/verify_page/single/opencv/01B.png&quot;
  match_image = cv2.imread(&quot;./img/verify_page/single/opencv/01A.png&quot;)
  
  find_image = cv2.imread(original)
  
  scale_width = (SCREEN_WIDTH / find_image.shape[1])*.9
  scale_height = (SCREEN_HEIGHT / find_image.shape[0])*.9
  scale = min(scale_width, scale_height)
  
  #resized window width and height
  window_width = int(find_image.shape[1] * scale)
  window_height = int(find_image.shape[0] * scale)
  cv2.namedWindow('ORIGINAL', cv2.WINDOW_NORMAL)
  cv2.resizeWindow('ORIGINAL', window_width, window_height)
  cv2.imshow(&quot;ORIGINAL&quot;, find_image)
  cv2.waitKey(0)
  
  
  sift = cv2.SIFT.create()
  keypoint1, descr1 = sift.detectAndCompute(match_image, None)
  keypoint2, descr2 = sift.detectAndCompute(find_image, None)
  good = []
  good2 = []
  
  find_h, find_w, _ = match_image.shape
  match_h, match_w, _ = find_image.shape
  
  bf = cv2.BFMatcher()
  matches = bf.knnMatch(descr1, descr2, k=2)
  for m,n in matches:
      if m.distance &lt; 0.75*n.distance:
          good.append([m])
          good2.append(m)
  
  if len(good)&gt;MIN_MATCH_COUNT:
      src_pts = np.float32([ keypoint1[m.queryIdx].pt for m in good2 ]).reshape(-1,1,2)
      dst_pts = np.float32([ keypoint2[m.trainIdx].pt for m in good2 ]).reshape(-1,1,2)
  
      M, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC,5.0)
      matchesMask = mask.ravel().tolist()
  
      pts = np.float32([ [0,0],[0,find_h-1],[find_w-1,find_h-1],[find_w-1,0] ]).reshape(-1,1,2)
      dst = None
      try: 
          dst = cv2.perspectiveTransform(pts,M)
  
          isClosed = True
          color = (255, 0, 0)     # Blue color in BGR 
          thickness = 2           # Line thickness of 2 px 
          img2 = cv2.polylines(find_image,[np.int32(dst)],isClosed,color,thickness, 
          cv2.LINE_AA)
  
          scale_width = (SCREEN_WIDTH / img2.shape[1])*.9
          scale_height = (SCREEN_HEIGHT / img2.shape[0])*.9
          scale = min(scale_width, scale_height)
  
          #resized window width and height
          window_width = int(img2.shape[1] * scale)
          window_height = int(img2.shape[0] * scale)
          cv2.namedWindow('POLYLINES', cv2.WINDOW_NORMAL)
          cv2.resizeWindow('POLYLINES', window_width, window_height)
          cv2.imshow(&quot;POLYLINES&quot;, img2)
          cv2.waitKey(0)
  
          dst = np.int32(dst)
          upper_left,lower_left,lower_right,upper_right = dst
          x1 = upper_left[0][0]
          x2 = upper_right[0][0]
          y1 = upper_left[0][1]
          y2 = lower_left[0][1]
          print(&quot;x1: &quot;, x1)
          print(&quot;x2: &quot;, x2)
          print(&quot;y1: &quot;, y1)
          print(&quot;y2: &quot;, y2)
  
          width = x2-x1
          height = y2-y1
          top_left = (x1,y1)
          #print(&quot;\ttop_left: &quot;, top_left)
          print(&quot;\t\tcalculated width: &quot;, width)
          print(&quot;\t\tcalculated height: &quot;, height)
  
          orig = cv2.imread(original)
          cropped_image = orig[y1:y2, x1:x2]
          actual_height, actual_width, _ = cropped_image.shape
          print(&quot;actual width: &quot;, actual_width)
          print(&quot;actual height: &quot;, actual_height)
          cv2.imshow('ORIGINAL CROPPED', cropped_image)
          cv2.waitKey(0)
          cv2.destroyAllWindows()
  
      except cv2.error as e:
          pass
  </code></pre>
  <p>Use this as the find_image:
  <a href="https://i.stack.imgur.com/MVVoA.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/MVVoA.png" alt="whole_img" /></a></p>
  <p>Use this as the match image:</p>
  <p><a href="https://i.stack.imgur.com/Kra2n.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Kra2n.png" alt="match_img" /></a></p>
  <p>Using the above two images produces the desired result. I found I get an error when the entire match image is not found in the larger image, only part of it. That is, part of the &quot;match&quot; is offscreen. Using below image as the find image will produce an invalid cropped image:
  <a href="https://i.stack.imgur.com/XyC03.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XyC03.png" alt="error_img" /></a></p>
  <p>If I execute this code, it works for the most part. Every now and then however I'll receive an error where x1,x2 y1,y2 suggest a nonzero cropped image size. If I do the crop however with the line:</p>
  <pre><code>cropped_image = orig[y1:y2, x1:x2]
  </code></pre>
  <p>and check the result size, I'll get a zero dimension in either height or width. That is:</p>
  <pre><code>x2-x1 != cropped_image.shape[1]
  </code></pre>
  <p>If I run the above program on a problematic image, the console output I get is:</p>
  <pre><code>x1:  446
  x2:  542
  y1:  -51
  y2:  130
          calculated width:  96
          calculated height:  181
          actual width:  96
          actual height:  0
  </code></pre>
  <p>Sometimes an error will be thrown, and the error is:</p>
  <pre><code>cv2.error: OpenCV(4.4.0) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-h4wtvo23\opencv\modules\features2d\src\sift.dispatch.cpp:465: error: (-5:Bad argument) image is empty or has incorrect depth (!=CV_8U) in function 'cv::SIFT_Impl::detectAndCompute'
  </code></pre>
  <p>Any help would be appreciated. Thank you very much-</p>