Token positions when using the Inference API
Hi,I am looking to run some NER models via the Inference API, but I am running into some issues. My problem is that the Inference API does not seem to return token positions. Consider this request:curl -X POST https://api-inference.huggingface.co/models/dslim/bert-base-NER \
-H "Authorization: Bearer <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d "Hello Sarah Jessia Parker who lives in New York."It returns:[
  {
    "entity_group": "PER",
    "score": 0.9959956109523773,
    "word": "Sarah Jessia Parker"
  },
  {
    "entity_group": "LOC",
    "score": 0.9994343519210815,
    "word": "New York"
  }
]So it finds the right tokens (and, nicely, returns the identified tokens grouped correctly.) However, there is no indication of where the tokens start in the input text.Confusingly, the model page (https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessia+Parker+who+lives+in+New+York.)highlights the right tokens, which would suggest youcanget token positions. (unless it’s doing something terribly hacky like looking for the first occurence of a particular token).Is there an option I’m missing?
2020-11-24T16:26:17Z
Indeed, the page seems to just be highlighting the first occurence of a token. Note how it (probably) picks up the wrong “Jessica” in this example:https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessica+Parker+who+Jessica+lives+in+New+York.Is there a way to extract the token position from the Inference API?
2020-11-25T16:34:29Z
Hi@fdb,Thanks for the note. You are perfectly right in your assumptions.In earlier versions of tokenizers/transformers the offsets of tokens were not necessarily available (because theencodeoperation was destructive). Because offsets were not available, the API did not send this information.However thanks totokenizers@anthonyand in the near future 4.0 version of transformers, theencodeoperation won’t be destructive anymore, sooffsetswillbe available.There is a PR for transformers for this pipelinehttps://github.com/huggingface/transformers/pull/8781And the API inference is already running the fix:curl -X POST -d 'Hello Sarah Jessica Parker who Jessica lives in New York' https://api-inference.huggingface.co/models/dslim/bert-base-NER[
{"entity_group":"PER","score":0.9960219860076904,"word":"Sarah Jessica Parker","start":6,"end":26},
{"entity_group":"PER","score":0.9771094918251038,"word":"Jessica","start":31,"end":38},
{"entity_group":"LOC","score":0.9994266927242279,"word":"New York","start":48,"end":56}
]https://huggingface.co/dslim/bert-base-NER?text=Hello+Sarah+Jessica+Parker+who+Jessica+lives+in+New+York. should soon follow.
2020-11-25T17:50:07Z
Hi@Narsil,Indeed, I now see start/end positions in my responses from the inference API. Thanks for adding those fields!
2020-11-25T17:53:04Z
The hard work was not done by me