Improve model card: Add metadata, usage example, and comprehensive content

This PR significantly enhances the model card for ChartCoder by:
- Adding key metadata: `pipeline_tag: image-text-to-text`, `library_name: transformers`, `license: cc-by-nc-4.0`, and `tags: - code-generation`. This improves discoverability on the Hub (e.g., at https://huggingface.co/models?pipeline_tag=image-text-to-text) and indicates compatibility with the Hugging Face Transformers library.
- Enriching the introductory section with an "About ChartCoder" summary derived from the paper's abstract.
- Integrating comprehensive details from the project's GitHub repository, including "Notes", "News", "Overview", "Models", and "Data" sections, along with updated image links to ensure proper rendering.
- Providing a clear "Installation" and "Training" guide for local setup.
- Adding a practical "Inference (Sample Usage)" code snippet using the `transformers` library, enabling users to easily load and run the model.
- Updating the top links with badges for datasets, the model itself, the arXiv paper, and the GitHub repository.

These changes provide a much richer and more actionable resource for users exploring ChartCoder on the Hugging Face Hub.

README.md

@@ -1,46 +1,167 @@
-# ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (ACL25 Main)
+---
+pipeline_tag: image-text-to-text
+library_name: transformers
+license: cc-by-nc-4.0
+tags:
+  - code-generation
+---
 
-<a href="https://huggingface.co/datasets/xxxllz/Chart2Code-160k" target="_blank">🤗 Dataset(HuggingFace)</a> | <a href="https://modelscope.cn/datasets/Noct25/Chart2Code-160k" target="_blank">🤖 Dataset(ModelScope)</a> | <a href="https://arxiv.org/abs/2501.06598" target="_blank">📑 Paper </a>
+# ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation (ACL 2025 Main)
 
-This repository contains the code to train and infer ChartCoder. **Our Github repository [ChartCoder](https://github.com/thunlp/ChartCoder) updates with more details and news.**
+[![🤗 Dataset (HuggingFace)](https://img.shields.io/badge/Dataset-HuggingFace-FFD21E.svg?logo=huggingface&logoColor=yellow)](https://huggingface.co/datasets/xxxllz/Chart2Code-160k)  [![🤖 Dataset (ModelScope)](https://img.shields.io/badge/Dataset-ModelScope-00A0E9.svg)](https://modelscope.cn/datasets/Noct25/Chart2Code-160k)  [![🤗 Model (HuggingFace)](https://img.shields.io/badge/Model-HuggingFace-FFD21E.svg?logo=huggingface&logoColor=yellow)](https://huggingface.co/xxxllz/ChartCoder) [![📑 Paper (arXiv:2501.06598)](https://img.shields.io/badge/arXiv-2501.06598-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2501.06598) [![GitHub Repo](https://img.shields.io/badge/GitHub-Repo-181717.svg?logo=github)](https://github.com/thunlp/ChartCoder)
 
+This repository is the official implementation of [ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation](https://arxiv.org/abs/2501.06598).
+
+## About ChartCoder
+
+ChartCoder is the first dedicated multimodal large language model (MLLM) designed for **chart-to-code generation**. It leverages Code LLMs as its language backbone to significantly enhance the executability of generated code. This model addresses two key challenges in chart interpretation:
+
+1.  **Low executability and poor detail restoration** in generated code from existing MLLMs.
+2.  **Lack of large-scale and diverse training data** for chart-to-code tasks.
+
+To overcome these, ChartCoder introduces:
+-   **Chart2Code-160k**: The first large-scale and diverse dataset for chart-to-code generation.
+-   **Snippet-of-Thought (SoT)**: A method that transforms direct chart-to-code generation into a step-by-step process.
+
+With only 7B parameters, ChartCoder surpasses existing open-source MLLMs on chart-to-code benchmarks, achieving superior chart restoration and code executability.
+
+## Notes
+1. ChartCoder is tested on the new version of Chartmimic, which contains 600 samples. The iclr version of ChartMimic is https://huggingface.co/datasets/ChartMimic/ChartMimic/blob/main/dataset-iclr.tar.gz.
+2. The code we utilize for evaluating is the Supplementary Material of https://openreview.net/forum?id=sGpCzsfd1K.
+
+All the results (including the baseline and our models) in the Table 3 in the paper is evaluated based on above two settings. When conducting the assessment in other settings, there may be performance differences. If you want to replicate the performance in the paper, it is recommended to achieve it under the aforementioned settings.
+
+## News
+
+**[2025.5.17]** ChartCoder has been accepted by **ACL 2025 Main**.
+
+**[2025.3.13]** We have upload our dataset [Chart2Code-160k(HF)](https://huggingface.co/datasets/xxxllz/Chart2Code-160k) to Huggingface.
+
+**[2025.2.19]** We have released our dataset [Chart2Code-160k](https://modelscope.cn/datasets/Noct25/Chart2Code-160k) to ModelScope.
+
+**[2025.1.16]** We have updated our data generation code [data_generator](https://github.com/thunlp/ChartCoder/tree/main/data_generator), built on [Multi-modal-Self-instruct](https://github.com/zwq2018/Multi-modal-Self-instruct). Please follow their instructions and our code to generate the <chart, code> data pairs.
+
+## Overview
+
+![main](https://github.com/thunlp/ChartCoder/raw/main/fig/main.png)
 
 ## Installation
-1. Clone this repo
-```
+
+To get started with ChartCoder, clone the repository and set up the environment:
+
+```bash
 git clone https://github.com/thunlp/ChartCoder.git
-```
-2.  Create environment
-```
+cd ChartCoder
 conda create -n chartcoder python=3.10 -y
 conda activate chartcoder
 pip install --upgrade pip  # enable PEP 660 support
 pip install -e .
 ```
-3. Additional packages required for training
-```
+
+For training, additional packages are required:
+
+```bash
 pip install -e ".[train]"
 pip install flash-attn --no-build-isolation
 ```
 
-## Train
-The whole training process consists of two stages. To train the ChartCoder, ```siglip-so400m-patch14-384``` and ```deepseek-coder-6.7b-instruct``` should be downloaded first.
+## Models
 
-For **Pre-training**, run
-```
+| Model | Download Link |
+|---|---|
+| MLP Connector | [projector](https://drive.google.com/file/d/1S_LwG65TIz_miW39rFPhuEAb5ClgopYi/view?usp=drive_link) |
+| ChartCoder | [ChartCoder](https://huggingface.co/xxxllz/ChartCoder) |
+
+The MLP Connector is our pre-trained MLP weights, which you could directly use for SFT.
+
+## Data
+
+| Dataset | Download Link |
+|---|---|
+| Chart2Code-160k | [HuggingFace](https://huggingface.co/datasets/xxxllz/Chart2Code-160k) |
+| Chart2Code-160k | [ModelScope](https://modelscope.cn/datasets/Noct25/Chart2Code-160k) |
+
+## Training
+
+The whole training process consists of two stages. To train the ChartCoder, `siglip-so400m-patch14-384` and `deepseek-coder-6.7b-instruct` should be downloaded first.
+
+For **Pre-training**, run:
+
+```bash
 bash scripts/train/pretrain_siglip.sh
 ```
-For **SFT**, run 
-```
+
+For **SFT**, run:
+
+```bash
 bash scripts/train/finetune_siglip_a4.sh
 ```
-Please change the model path to your local path. See the corresponding ```.sh ``` file for details. 
-We also provide other training scripts, such as using CLIP ```_clip``` and multiple machines ```_m```. See ``` scripts/train ``` for further information.
 
+Please change the model path to your local path. See the corresponding `.sh` file for details. We also provide other training scripts, such as using CLIP `_clip` and multiple machines `_m`. See `scripts/train` for further information.
+
+## Inference (Sample Usage)
+
+You can easily use ChartCoder with the Hugging Face `transformers` library. Ensure you have `transformers` and `torch` installed.
+
+```python
+from transformers import AutoProcessor, AutoModelForCausalLM
+import torch
+from PIL import Image
+import requests
+from io import BytesIO
+
+# Load model and processor
+model_id = "xxxllz/ChartCoder" # The model's Hugging Face ID
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16,
+    low_cpu_mem_usage=True,
+    trust_remote_code=True
+)
+processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
+
+# Example image (replace with a real chart image path or URL)
+# For demonstration, let's use a placeholder image. In a real scenario, load your chart image.
+# Example: image = Image.open("path/to/your/chart_image.png").convert("RGB")
+# Or from a URL:
+image_url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/chart_example.png"
+image = Image.open(BytesIO(requests.get(image_url).content)).convert("RGB")
+
+# Define your prompt for chart-to-code generation
+prompt = "Generate Python code to recreate the given chart. Provide only the code, no explanations."
+
+# Prepare messages in the chat template format
+messages = [
+    {"role": "user", "content": f"<image>\
+{prompt}"}
+]
+text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+# Move inputs to GPU if available
+inputs = processor(text=text, images=image, return_tensors="pt").to(model.device)
+
+# Generate code
+output_ids = model.generate(**inputs, max_new_tokens=512)
+output_text = processor.batch_decode(output_ids, skip_special_tokens=True)[0].strip()
+
+# Print the generated code
+print(output_text)
+```
+
+## Results
+
+Please refer to our paper for detailed performance on ChartMimic, Plot2Code and ChartX benchmarks. Thanks for these contributions to the chart-to-code field.
+![results](https://github.com/thunlp/ChartCoder/raw/main/fig/results.png)
+
+## Contact
+
+For any questions, you can contact [[email protected]](mailto:[email protected]).
 
 ## Citation
+
 If you find this work useful, consider giving this repository a star ⭐️ and citing 📝 our paper as follows:
-```
+
+```bibtex
 @misc{zhao2025chartcoderadvancingmultimodallarge,
       title={ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation}, 
       author={Xuanle Zhao and Xianzhen Luo and Qi Shi and Chi Chen and Shuo Wang and Wanxiang Che and Zhiyuan Liu and Maosong Sun},
@@ -50,4 +171,8 @@ If you find this work useful, consider giving this repository a star ⭐️ and
       primaryClass={cs.AI},
       url={https://arxiv.org/abs/2501.06598}, 
 }
-```
+```
+
+## Acknowledgement
+
+The code is based on the [LLaVA-NeXT](https://github.com/LLaVA-VL/LLaVA-NeXT). Thanks for these great works and open sourcing!