title: Unwarping a set of rectangles to form a square grid in OpenCV
body: <p>I am trying to map an image which contains these points, drawn as rectangles using top left point and the bottom right points, though I do have the points as quadrilaterals </p>
  
  <p><img src="https://i.stack.imgur.com/0iToc.png" alt="enter image description here"></p>
  
  <p>I am trying to map the image with the above squares to the squares below:</p>
  
  <p><img src="https://i.stack.imgur.com/ZGJ9h.png" alt="enter image description here"></p>
  
  <p>I believe that I am supposed to obtain a perspective transform from each quad.  Here is some code that I wrote to try and facilitate this:</p>
  
  <pre><code>cv::Mat output = cv::Mat::zeros(outputSize,CV_32F);
  
  
      // Uses the size of the quadOutput to
      std::vector&lt;std::vector&lt;cv::Point2f&gt;&gt; quadOutputImage = quadralateralsFromImageOutput(imageToWarp.size().height);
  
      cv::Mat perspectiveMatrix = cv::Mat(3,3,CV_32F);
  
      cv::Mat warp = cv::Mat::zeros(outputSize,CV_32F);
  
      for (int i = 0; i &lt; quadOutputImage.size(); i ++) {
  
              // get the mapping from the quadtriangles
              perspectiveMatrix = cv::getPerspectiveTransform(quadCentroids[i],quadOutputImage[i]);
  
              // perform the warp with the current quadtriangles
              cv::warpPerspective(imageToWarp,warp,perspectiveMatrix,output.size());
  
              // copy roi to output
  
      }
  
      return warp;
  </code></pre>
  
  <p>I do this quad by quad and the image does not appear to be correct at all.  It appears that it is slightly skewed.</p>
  
  <h1>Edit</h1>
  
  <p>I also tried finding the homography matrix, but I ended up with an odd result.  Here's the code that I wrote</p>
  
  <pre><code>    cv::Mat warpedImageGeneration(const cv::Mat&amp; imageToWarp,cv::Size outputSize, std::vector&lt;std::vector&lt;cv::Point2f&gt;&gt; quadCentroids) {        
  
      // Uses the size of the quadOutput 50 by 50 squares
      std::vector&lt;std::vector&lt;cv::Point2f&gt;&gt; quadOutputImage = quadralateralsFromImageOutput(450);
  
      // flatten the matrices
      std::vector&lt;cv::Point2f&gt; flattenedQuads = flattenMatrix(quadOutputImage);
      std::vector&lt;cv::Point2f&gt; flattenedCentroids = flattenMatrix(quadCentroids);
  
      cv::Mat perspectiveMatrix = cv::Mat(3,3,CV_32F);
  
      cv::Mat warp = cv::Mat::zeros(450,450,CV_32F);
  
      perspectiveMatrix = cv::findHomography(flattenedCentroids, flattenedQuads);
  
      cv::warpPerspective(imageToWarp, warp, perspectiveMatrix, warp.size());
  
      return warp;
   }
  </code></pre>
  
  <p>On an example image, this is the unwarped result I get:</p>
  
  <p><img src="https://i.stack.imgur.com/Q6vsp.png" alt="enter image description here"></p>
  
  <p>I think the issue could be that the vertices for the source are out of quadrilateral order i.e top left, top right, bottom left and bottom right.</p>