{"action":"closed","issue":{"url":"https://api.github.com/repos/glample/tagger/issues/31","repository_url":"https://api.github.com/repos/glample/tagger","labels_url":"https://api.github.com/repos/glample/tagger/issues/31/labels{/name}","comments_url":"https://api.github.com/repos/glample/tagger/issues/31/comments","events_url":"https://api.github.com/repos/glample/tagger/issues/31/events","html_url":"https://github.com/glample/tagger/issues/31","id":203263864,"number":31,"title":"Reproducing the result for the LSTM-CRF model","user":{"login":"ronxin","id":3887115,"avatar_url":"https://avatars2.githubusercontent.com/u/3887115?v=4","gravatar_id":"","url":"https://api.github.com/users/ronxin","html_url":"https://github.com/ronxin","followers_url":"https://api.github.com/users/ronxin/followers","following_url":"https://api.github.com/users/ronxin/following{/other_user}","gists_url":"https://api.github.com/users/ronxin/gists{/gist_id}","starred_url":"https://api.github.com/users/ronxin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ronxin/subscriptions","organizations_url":"https://api.github.com/users/ronxin/orgs","repos_url":"https://api.github.com/users/ronxin/repos","events_url":"https://api.github.com/users/ronxin/events{/privacy}","received_events_url":"https://api.github.com/users/ronxin/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2017-01-26T00:34:46Z","updated_at":"2018-01-01T13:52:47Z","closed_at":"2018-01-01T13:52:47Z","author_association":"NONE","body":"I am trying to reproduce the reported result for the LSTM-CRF model on the English dataset included in the repository, but haven't yet succeeded in getting the reported F1 score (90.94).\r\n\r\nI used all the default parameter values, which seem to match what is in the paper, and I used eng.train for training, eng.testa for validation, and eng.testb for testing. After the program stopped (100 epochs on the training set), I got the best dev score to be 89.74, and the corresponding test score is 83.55. Am I missing something?\r\n\r\nAlso, in `train.py`, why is \"**New best score on test**\" reported? Shouldn't one only report the test score of the model that does the best on validation?\r\n\r\nI will keep checking if I have anything misconfigured, but at the meantime, any help or insight would be appreciated! Thanks again for the awesome paper + code!"}}
{"actor":{"display_login":"glample"}}