Handwriting recognition. Can't recognize multiline words
I expect the modeltrocr-base-handwrittento extract all the text from the picture.16e9e061da2.9e37232443debf531136×2047 34.7 KBBut the result is got from it issentiment.Full code:from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image

p = 'picture.png'
processor = TrOCRProcessor.from_pretrained("trocr-base-handwritten/")
model = VisionEncoderDecoderModel.from_pretrained("trocr-base-handwritten/")
image = Image.open(p)
image_rgb = image.convert('RGB')
pixels = processor(image_rgb, return_tensors="pt").pixel_values
generated_ids = model.generate(pixels)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)

# generated_text output 
# ['sentiment']How to configure, improve the model to get at least the similar phrasesense of humor?
2023-05-10T22:51:48Z
Was thinking the exact same thing today, and posted a topic at:How to do full page analysis with TrOCR (integrating with text segmentation analysis)I am a complete novice, but believe this is because the TrOCR model is based on single line input?@nielsrmentioned we could use a separate text line segmentation program to then feed those line-by-line, or bounding box results into TrOCR. Don’t have a clue how to do that, and am hoping he or the other fine folks@huggingfaceforums might offer some helpful suggestions…
2023-05-11T07:07:40Z
Hey, thx for sharing your concern.i noticed that the oneline pictures are recognized quite good in comparison to the multiliners.From the muultiliners i often get results like1953 54meaningless numbers.Suggestion of a workaround would be great. Let’s wait for an answer.
2023-05-11T07:08:45Z
@jhhfHave you considered other models, solutions for handwriting recognition by the way?
2023-05-12T18:29:46Z
@kvdm-dev- yes, I’ve been looking at LayoutMLV2,LayoutML3andDonut; but agree with you that the one-line transformers trained to a specific language seem to be quite good!@nielsr→ any suggestions?
2023-05-13T14:19:43Z
And what is your experience withLayoutML3andDonut? IsTrOCRbetter in your opinion?
2023-05-13T18:20:58Z
Hoping to try LayoutML3 (N.C.) and Donut (MIT) this week. Came across these two yesterday, and think they may work for us:Craft-Text-Detector (Updated 2022)craft-hw-ocrCaft-HW-OCR (Updated 2021) - Based on Craft-Text-Detector, and sends results to TrOCR!craft-text-detectorIf you test any of the above, let me know your results please!
2023-05-14T13:01:56Z
Yeah, sure.For the time being, i’m busy with other stuff, but later i’m gonna get back to improving recognition quality as my goal to understand the multiline handwritings.