title: How can I improve the accuracy of the character segmentation of my OCR system?
body: <p>I am currently working on a Optical Character Recognition system for Japanese letters. I am already able to identify individual characters if the letter in question is seperated and in the right size. (The deep learning part of the work)</p>
  
  <p>As a next step I am trying to segmentate individual characters in an image to make a prediction about which letter it is. (For now it is only about black characters on white background, scanned PDFs and such)</p>
  
  <p>So far, the most promising results I got was using the function "cv2.findContours" from OpenCV.
  Here are 3 examples:</p>
  
  <p><a href="https://i.stack.imgur.com/pAoKY.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/pAoKY.jpg" alt="Example Image 1"></a></p>
  
  <p><a href="https://i.stack.imgur.com/Osnc2.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Osnc2.jpg" alt="Example Image 2"></a></p>
  
  <p><a href="https://i.stack.imgur.com/dEgIf.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dEgIf.jpg" alt="Example Image 3"></a></p>
  
  <p>While the results are not entirely horrible, there are still many cases where two or more characters are treated as one or where one characters is split up into multiple boxes. I cannot seem to make the code work for all fonts and character sizes.
  While the first image is still pretty close to being perfect, the second and third are not nearly as accurate. (I hope it is clear where the mistakes are)</p>
  
  <p>I tried completely different approaches, such as hough transform, but I couldn't achieve anything nearly as good as this approach.</p>
  
  <p>This by the way is my current code:</p>
  
  <pre><code>import cv2
  import numpy as np
  
  file_name = '../data/test.jpg'
  
  img  = cv2.imread(file_name)
  
  img_final = cv2.imread(file_name)    
  img_final = cv2.resize(img_final, (img_final.shape[1], img_final.shape[0]))    
  img2gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
  blur = cv2.GaussianBlur(img2gray,(7,7),0)
  #     thresh = cv2.adaptiveThreshold(blur,255,1,1,11,2)Y)    
  ret, mask = cv2.threshold(blur, 180, 255, cv2.THRESH_BINARY)    
  image_final = cv2.bitwise_and(img2gray , img2gray , mask = mask)
  
  ret, new_img = cv2.threshold(image_final, 180 , 255, cv2.THRESH_BINARY_INV)
  
  kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(2,2))
  dilated = cv2.dilate(new_img,kernel,iterations = 1)
  
  _, contours, hierarchy = cv2.findContours(dilated,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE) 
  
  index = 0     
  for contour in contours: 
      [x,y,w,h] = cv2.boundingRect(contour)
  
      if w &lt;1 and h&lt;1:    
          continue
  
      cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,255),2)
  
      cropped = image_final[y :y +  h , x : x + w]
  
      s = '../output/crop_' + str(index) + '.jpg'     
      cv2.imwrite(s , cropped)    
      index = index + 1
  
  cv2.imshow('captcha_result' , img)    
  cv2.waitKey()
  s2 = '../data/output.jpg'     
  cv2.imwrite(s2 , img)
  </code></pre>
  
  <p>Now questions are the following:</p>
  
  <ul>
  <li><p>Does anybody have an idea how to improve the accuracy of my code?
  Is it better to take a whole new approach?
  Can a sliding window help me here?</p></li>
  <li><p>Where do I go from here?
  Can I maybe use the sliding window to send the individual characters to the prediction? 
  With all the false positives (e.g. characters being split in two, despite trying to limit it) I am uncertain whether or not I can simply use the cropped images of characters as they are and how to further filter the results.</p></li>
  </ul>
  
  <p>As I am new to all this, I would really appreciate any help or hint I can get!</p>
  
  <p>I am looking forward to your replies! :)</p>