Sefika/semeval_prompt_tuning_5

Model Card for Model ID

Model Details

Model Description

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

• Developed by: Sefika Efeoglu

• Model type: text-to-text

• Language(s) (NLP): [More Information Needed]

• License: [More Information Needed]

• Finetuned from model [optional]: https://huggingface.co/google/flan-t5-base

Uses

import json
import torch
from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM
from datetime import datetime
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-base")
model_id = "Sefika/semeval_prompt_tuning_5"
model = T5ForConditionalGeneration.from_pretrained(model_id,
                                                    device_map="auto",
                                                    load_in_8bit=False,
                                                    torch_dtype=torch.float16)
prompt = """Example Sentence:The purpose of the <e1>audit</e1> was to report on the <e2>financial statements</e2>.\n"""+\
         """Sentence: Query Sentence:The most common <e1>audits</e1> were about <e2>waste</e2> and recycling.\n"""+\
         """What is the relation type between e1: audits. and e2 : waste.  according to given relation types below in the sentence?\n"""+\
         """Relation types: Relation types: Cause-Effect(e2,e1), Content-Container(e1,e2), Member-Collection(e1,e2), Instrument-Agency(e1,e2), Product-Producer(e2,e1), Member-Collection(e2,e1), Message-Topic(e1,e2), Entity-Origin(e2,e1), Message-Topic(e2,e1), Instrument-Agency(e2,e1), Content-Container(e2,e1), Product-Producer(e1,e2), Entity-Origin(e1,e2), Component-Whole(e1,e2), Entity-Destination(e1,e2), Other, Cause-Effect(e1,e2), Component-Whole(e2,e1), Entity-Destination(e2,e1). \n"""
inputs = self.tokenizer(prompt, add_special_tokens=True, max_length=526,return_tensors="pt").input_ids.to("cuda")

outputs = self.model.generate(inputs, max_new_tokens=length,  pad_token_id=self.tokenizer.eos_token_id)

response = self.tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(response[0])
#"Cause-Effect(e1,e2)"

Training Details

Training Data

semeval-2010-task8 [More Information Needed]

Training Procedure

5 fold cross validation with sentence and relation types. Input is sentence and the output is relation types

Training Hyperparameters

Epoch:5, BS:16 and others are default.

Hardware

Colab Pro+ A100.

Citation [optional]

Efeoglu, Sefika, and Adrian Paschke. "Retrieval-Augmented Generation-based Relation Extraction." arXiv preprint arXiv:2404.13397 (2024).