How to use the "feature-extraction" pipleine on "facebook/galactica" model
I would like to extract the features for a text using the “facebook/galactica-6.7b” model for use as features in a downstream prediction model. Following the pipeline example, I’m able to extract embeddings using the “allenai/scibert_scivocab_uncased” without issue:from transformers import pipeline 
extractor = pipeline(model="allenai/scibert_scivocab_uncased", task="feature-extraction")

input_text = """Here is some text. It has a few sentences."""
result = extractor(input_text, return_tensors=True)I get a Tensor tape of size [1,13,768] just as expected.However, if I try the same with the “facebook/galactica-6.7b”  model I get an error:from transformers import pipeline 
extractor = pipeline(model="facebook/galactica-6.7b", task="feature-extraction")

input_text = """Here is some text. It has a few sentences."""
result = extractor(input_text, return_tensors=True)TypeError: forward() got an unexpected keyword argument ‘token_type_ids’Something is different with the galactica model but I’m not sure how to troubleshoot. I’ve looked at the model card and the original github repo but I can’t find instructions on extracting the text embeddings.
2023-11-10T18:35:57Z
This is an old problem, but still relevant with version 4.34.The problem is that Galactica uses its own tokenizer but the OPT model implementation. This tokenizer returnstoken_type_ids, but OPT does not expect them.I patched the forward method of the pipeline model. Afterwards the pipeline works.from transformers import pipeline
pipe = pipeline(task="feature-extraction", model="facebook/galactica-6.7b")
forward_method = pipe.model.forward

def patched_forward(*args, token_type_ids, **kwargs):
    return forward_method(*args, **kwargs)

pipe.model.forward = patched_forward