Update README.md

README.md

@@ -1,3 +1,71 @@
 ---
-license: gpl
+license: gpl-3.0
+tags:
+- DocVQA
+- Document Question Answering
+- Document Visual Question Answering
+datasets:
+- rubentito/mp-docvqa
+language:
+- en
 ---
+
+# LongT5 base with transient-global attention fine-tuned on MP-DocVQA
+
+This is LongT5 trained on SQuADv2, CoQA and TryoCoQA datasets from [Tryolabs hub](https://huggingface.co/tryolabs/long-t5-tglobal-base-blogpost-cqa-onnx) and fine-tuned on Multipage DocVQA (MP-DocVQA) dataset.
+
+
+## How to use
+
+Here is how to use this model to get the features of a given text in PyTorch:
+
+```python
+import torch
+from transformers import AutoTokenizer, LongT5ForConditionalGeneration
+
+tokenizer = AutoTokenizer.from_pretrained("rubentito/longt5-tglobal-base-mpdocvqa")
+model = LongT5ForConditionalGeneration.from_pretrained("rubentito/longt5-tglobal-base-mpdocvqa")
+
+context = "Huggingface has democratized NLP. Huge thanks to Huggingface for this."
+question = "What has Huggingface done?"
+input_text = "question: {:s}  context: {:s}".format(question, context)
+
+encoding = tokenizer(input_text, return_tensors="pt")
+output = self.model.generate(**encoding)
+answer = tokenizer.decode(output['sequences'], skip_special_tokens=True)
+```
+
+## Metrics
+**Average Normalized Levenshtein Similarity (ANLS)**
+
+The standard metric for text-based VQA tasks (ST-VQA and DocVQA). It evaluates the method's reasoning capabilities while smoothly penalizes OCR recognition errors.
+Check [Scene Text Visual Question Answering](https://arxiv.org/abs/1905.13648) for detailed information.
+
+**Answer Page Prediction Accuracy (APPA)**
+
+In the MP-DocVQA task, the models can provide the index of the page where the information required to answer the question is located. For this subtask accuracy is used to evaluate the predictions: i.e. if the predicted page is correct or not.
+Check [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/abs/2212.05935) for detailed information.
+
+## Model results
+
+Extended experimentation can be found in Table 2 of [Hierarchical multimodal transformers for Multi-Page DocVQA](https://arxiv.org/pdf/2212.05935.pdf).
+You can also check the live leaderboard at the [RRC Portal](https://rrc.cvc.uab.es/?ch=17&com=evaluation&task=4).
+| Model 		 																	| HF name								| Parameters 	|	ANLS 		| APPA		|
+|-----------------------------------------------------------------------------------|:--------------------------------------|:-------------:|:-------------:|:---------:|
+| [Bert large](https://huggingface.co/rubentito/bert-large-mpdocvqa)	            | rubentito/bert-large-mpdocvqa			| 334M 			| 0.4183 		| 51.6177 	|
+| [Longformer base](https://huggingface.co/rubentito/longformer-base-mpdocvqa)		| rubentito/longformer-base-mpdocvqa	| 148M			| 0.5287		| 71.1696 	|
+| [BigBird ITC base](https://huggingface.co/rubentito/bigbird-base-itc-mpdocvqa) 	| rubentito/bigbird-base-itc-mpdocvqa	| 131M			| 0.4929		| 67.5433 	|
+| [LayoutLMv3 base](https://huggingface.co/rubentito/layoutlmv3-base-mpdocvqa)		| rubentito/layoutlmv3-base-mpdocvqa	| 125M 			| 0.4538		| 51.9426 	|
+| [**T5 base**](https://huggingface.co/rubentito/t5-base-mpdocvqa)					| rubentito/t5-base-mpdocvqa			| 223M 			| 0.5050		| 0.0000 	|
+| Hi-VT5 																			| TBA 									| 316M 			| 0.6201		| 79.23		|
+
+## Citation Information 
+
+```tex
+@article{tito2022hierarchical,
+  title={Hierarchical multimodal transformers for Multi-Page DocVQA},
+  author={Tito, Rub{\`e}n and Karatzas, Dimosthenis and Valveny, Ernest},
+  journal={arXiv preprint arXiv:2212.05935},
+  year={2022}
+}
+```