Where is the source to benchmark's dataset entries on the model's website
If I go tohttps://huggingface.co/Helsinki-NLP/opus-mt-ru-en, I see a bunch of dataset entries in the benchmark table. How do I know what they are so that if I do my own training I can compare apples to apples?For example this particular model has a list:[...]
|newstest2015-enru.ru.en |30.4 |0.568|
|newstest2016-enru.ru.en |30.1 |0.565|
[...]
newstest2019-ruen.ru.en 	31.4 	0.576
Tatoeba.ru.en 	61.1 	0.736After some research I have derived that most likely these are WMT datasets (e.g.https://www.statmt.org/wmt16/), but I could be wrong. And even if I got it right, I can’t tell whethernewstest2015is actually wmt16 or wmt15? This is because wmt16 doesn’t include any data from 2016. It’s made from News Crawl articles up to and including 2015, according tohttp://www.statmt.org/wmt16/translation-task.html. So I can’t tell whether the year innewstest2016-enru.ru.enrefers to the name of the dataset or the last included year of the News Crawl dump.Any suggestions to how I could find which entry is the right one if I finetune on wmt16?edit: Since there is wmt19 out there and their scorecard contains the “newstest2019” as the most recent entry, most likely the year listed in the entry is of the WMT release and not of the News Crawl data. So if I train on wmt16 I’d compare withnewstest2016-enru.ru.en.That’s said, perhaps, it’d make it easier for the users if the contributed model’s webpage identifed which datasets it has in its benchmark table, with a link to a source or at least an official name so the former can be found. Also since there is a high link rot - perhaps, a backup link towaybackmachine.Thank you.p.s. now that I have investigated this model, the helpful links from the benchmark entry would have been:all but last entry:http://opus.nlpl.eu/WMT-News.phpand maybe the originalhttp://www.statmt.org/wmt19/last entry:http://opus.nlpl.eu/Tatoeba.phpand maybe the originalhttps://tatoeba.org/eng/
2020-08-10T07:10:39Z
I guess the only way is to open a issue on their github repo or contact them directly to ask.
2020-08-10T17:09:00Z
Oh, I see. By looking at the other models, a model card is just a text entry. For some reason I thought there was a specific API/form for each field.Then, yes, it’s just a matter of putting the notes in place - I will contact them via their github - thank you for this suggestion,@RichardWang!Filed an issue therehttps://github.com/Helsinki-NLP/OPUS-MT-train/issues/17