Update README.md

README.md

@@ -1,3 +1,152 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language: en
+library_name: transformers
+tags:
+  - sdgs
+  - sustainability
+  - multi-label-classification
+  - text-classification
+  - luke
+datasets:
+  - osdg/osdg-community
+  - SDG-AI-Lab/sdgi_corpus
+pipeline_tag: text-classification
+---
+
+# SDG Classifier: A Fine-Tuned LUKE Model for Multi-Label SDG Classification
+
+This repository contains the pre-trained model weights (`best_model.pt`) for the paper: **"Bridging the Sustainable Development Goals: A Multi-Label Text Classification Approach for Mapping and Visualizing Nexuses in Sustainability Research"**.
+
+➡️ **GitHub Repository (Code):** [Insert Link to your GitHub Repository Here]  
+➡️ **Paper Link:** [Link to Published Paper will be added upon publication]
+
+## 📝 Model Description
+
+This model is a fine-tuned version of `studio-ousia/luke-large-lite` for multi-label text classification of the 17 UN Sustainable Development Goals (SDGs). It has been trained on a uniquely diverse, multi-sectoral, and multilingual corpus designed to achieve high generalization performance across various domains (academic, policy, civil society, etc.).
+
+The model takes a text input (up to 512 tokens) and outputs a probability score for each of the 17 SDGs, indicating the relevance of the text to each goal.
+
+## 🚀 How to Use
+
+This model was trained with a custom classification head in PyTorch. To use it, you need to define the model architecture first and then load the downloaded weights (`best_model.pt`).
+
+Below is a complete example of how to load the model and perform a prediction.
+
+```python
+import torch
+from torch import nn
+from transformers import AutoTokenizer, AutoModel
+from huggingface_hub import hf_hub_download
+from pathlib import Path
+
+# --- 1. Define the Model Architecture ---
+# This class must match the architecture used during training.
+# You can copy this class from the original training script.
+class SDGClassifier(nn.Module):
+    def __init__(self, model_path, pooler_dropout, class_number):
+        super(SDGClassifier, self).__init__()
+        self.bert = AutoModel.from_pretrained(model_path)
+        self.dropout = nn.Dropout(pooler_dropout)
+        self.pooler = nn.Sequential(nn.Linear(in_features=self.bert.config.hidden_size, out_features=self.bert.config.hidden_size))
+        self.tanh = nn.Tanh()
+        self.cls = nn.Linear(in_features=self.bert.config.hidden_size, out_features=class_number)
+
+    def forward(self, input_ids, attention_mask, token_type_ids, position, labels):
+        # Note: 'position' and 'labels' are dummy inputs required by the forward signature,
+        # but are not used for inference if labels are not provided.
+        bert_output = self.bert(input_ids, attention_mask, token_type_ids=token_type_ids, output_attentions=True, output_hidden_states=True)
+        average_hidden_state = (bert_output.last_hidden_state * attention_mask.unsqueeze(-1)).sum(1) / attention_mask.sum(1, keepdim=True)
+        pooler_output = self.tanh(self.pooler(self.dropout(average_hidden_state)))
+        logits = self.cls(pooler_output)
+        return logits, average_hidden_state, bert_output.attentions
+
+# --- 2. Setup and Load Model ---
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+
+# Model configuration
+BASE_MODEL = 'studio-ousia/luke-large-lite'
+NUM_CLASSES = 17
+DROPOUT_RATE = 0.26 # This is the optimized dropout rate from the paper's training
+
+# Instantiate the model
+model = SDGClassifier(model_path=BASE_MODEL, pooler_dropout=DROPOUT_RATE, class_number=NUM_CLASSES).to(device)
+model.eval() # Set to evaluation mode
+
+# Download the fine-tuned weights from this Hub
+model_weights_path = hf_hub_download(
+    repo_id="GE-Lab/SDGs-classifier",
+    filename="best_model.pt"
+)
+
+# Load the weights into the model
+model.load_state_dict(torch.load(model_weights_path, map_location=device))
+
+print("Model loaded successfully!")
+
+# --- 3. Prepare Input ---
+tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
+text = "Our research focuses on renewable energy solutions to combat climate change and ensure a sustainable future for all."
+
+inputs = tokenizer.encode_plus(
+    text,
+    None,
+    add_special_tokens=True,
+    max_length=512,
+    padding='max_length',
+    return_token_type_ids=True,
+    truncation=True,
+    return_tensors='pt'
+).to(device)
+
+# The model's forward pass requires these additional dummy inputs
+inputs['position'] = torch.arange(0, inputs['input_ids'].shape[1]).unsqueeze(0).to(device)
+inputs['labels'] = torch.zeros(1, NUM_CLASSES).to(device) # Dummy labels for inference
+
+# --- 4. Get Predictions ---
+with torch.no_grad():
+    logits, _, _ = model(**inputs)
+    probabilities = torch.sigmoid(logits).cpu().numpy()[0]
+    predictions = (probabilities > 0.5).astype(int)
+
+# --- 5. Interpret the Results ---
+goal_contents = ['Goal 1: No Poverty','Goal 2: Zero Hunger','Goal 3: Good Health and Well-being','Goal 4: Quality Education','Goal 5: Gender Equality','Goal 6: Clean Water and Sanitation','Goal 7: Affordable and Clean Energy','Goal 8: Decent Work and Economic Growth','Goal 9: Industry, Innovation and Infrastructure','Goal 10: Reduced Inequalities','Goal 11: Sustainable Cities and Communities','Goal 12: Responsible Consumption and Production','Goal 13: Climate Action','Goal 14: Life Below Water','Goal 15: Life on Land','Goal 16: Peace, Justice and Strong Institutions','Goal 17: Partnerships for the Goals']
+
+print(f"\nText: '{text}'")
+print("\n--- Predicted SDGs (Threshold > 0.5) ---")
+predicted_goals = [goal_contents[i] for i, pred in enumerate(predictions) if pred == 1]
+if predicted_goals:
+    for goal in predicted_goals:
+        print(goal)
+else:
+    print("No SDGs detected with a probability > 0.5")
+
+print("\n--- All SDG Probabilities ---")
+for i, prob in enumerate(probabilities):
+    print(f"{goal_contents[i]:<55}: {prob:.2%}")
+
+```
+
+## 📈 Training and Evaluation
+
+### Training Data
+The model was trained on a novel, heterogeneous corpus of 23,969 multi-labeled documents from 11 diverse sources, including government, academia, industry, and civil society, with some sources translated from Japanese. This approach was designed to address the "interpretive diversity" of SDG-related language.
+
+For full details on reconstructing the training corpus, please refer to **Supplementary Information S4** in our paper.
+
+### Evaluation
+This model was selected based on its superior generalization performance (especially recall) on external datasets like the OSDG Community Dataset and the SDGi Corpus. On a human-coded sample of scientific articles, the model achieved a macro-averaged **F1-score of 0.623**. For a full breakdown of performance metrics, please see the paper.
+
+## 📜 Citation
+
+If you use this model in your research, please cite our paper:
+
+```bibtex
+@article{Miyashita2025,
+  author    = {Naoto Miyashita and Takanori Matsui and Chihiro Haga and Naoki Masuhara and Shun Kawakubo},
+  title     = {Bridging the Sustainable Development Goals: A Multi-Label Text Classification Approach for Mapping and Visualizing Nexuses in Sustainability Research},
+  journal   = {Sustainability Science},
+  year      = {2025},
+  % TODO: Add Volume, Pages, DOI upon publication
+}
+```