Different prediction tensors on single item vs a list of items
I am doing sentiment analysis on tweets. I used the roberta-base model. I trained the model on a dataset containing around 90,000 entries. When I predict using the saved model, the results are different when predicting single sentence and when the same sentence is one of the items in a list and all sentences are looped through to predict.E.g., “hi there” will give a tensor value and a different tensor value when passed in a list like:[“hi there”, “let’s go out”, “how are you?”]The difference is so high that a sentence which is positive and correctly predicted as positive when predicted for the single string is predicted as negative when passed in a list.Is it something that is expected? Or is there anything I need to make sure to avoid this?
2021-02-11T22:08:33Z
Hi@rashubcan you post the code you’re using to generate the predictions (seeherefor general advice on getting help)? Even better would be a Google Colab notebook to be able to inspect the inputs / outputs
2021-02-12T14:54:34Z
Hi@lewtunI followed this colab notebook:colab.research.google.comGoogle ColaboratoryWhen predicting, if I do the following, this will be correct prediction:learn.predict(“This was a really good movie, i loved it”)But if I pass a list for the prediction like the following, the tensor values change:test = [“This was a really good movie, i loved it”, “Wowwwwww, about an hour ago I finally finishedwatching this terrible movie!!!”, “Im still a big IMDb fan, but seriously rethink this rating processbecause this movie should be rated no higher than maaaybbbeee like a 3”, “I am disappointed in thedirector, Sydney Pollack who gave us the classic Tootsie and other films. This one is a waste of timeand energy.”, "Wow, this movie really sucked down below the normal scale of dull, boring, andunimaginative films I’ve seen recently. ", “Sorry. Someone has to say it. This really is/was a dullmovie. Worthy perhaps, but dull nonetheless.”, “This is a truly hilarious film and one that I have seenmany times. This is a film you could watch again and again, with a fabulous sound track! One for allthose at school in the 90’s to watch!”]df = pd.DataFrame(test,columns=[‘information’])final_res =for txt in df[‘information’]:result = learn.predict(txt)final_res.append(result)
2021-02-12T16:24:23Z
Hi@rashub, thanks for sharing the Colab notebook. This seems to be a problem with thefasthugslibrary, nottransformersso perhaps@morgan(the creator) can shed some light on this?For what it’s worth, I checked that it’s not afastaiproblem by feeding your inputs to a ULMFiT classifier from this tutorial:fastai - Text transfer learning(see screenshot below)In general the predicted values should not change if you decide to pass a batch instead of a single item.P.S. you can get nicely formatted code in the forum by surrounding the code snippet with 3 backticks “`”Screen Shot 2021-02-12 at 5.19.16 pm1562×792 124 KB
2021-02-12T16:47:15Z
@lewtunI see, thank you so much for checking and sharing that. Also, I will keep in mind about the 3 backticks, thanks! I will wait for@morganto check it out.
2021-02-23T23:46:53Z
Hello,Is there any update on this?
2021-03-09T13:58:27Z
Hey@rashub, sorry just seeing this!FastHugs is quite out of date at this point, I would recommend using the Blurr library for HuggingFace + fastaiGitHub - ohmeow/blurr: A library that integrates huggingface transformers with version 2 of the fastai frameworkThe fastai docs also have GPT2 + fastai example too in case you hadn’t seen:docs.fast.aiTutorial - Transformers | fastaiAn example of how to incorporate the transfomers library from HuggingFace with fastai
2021-03-09T17:46:42Z
Hi@morgan, no problem. Thanks for the suggestions. I will try it out!