BERT-100k_REGEX / special_tokens_map.json
yzimmermann's picture
Upload tokenizer
6bf5c82 verified
raw
history blame contribute delete
109 Bytes
{
"bos_token": "^",
"eos_token": "&",
"mask_token": "[MASK]",
"pad_token": " ",
"unk_token": "?"
}