{"action":"created","issue":{"url":"https://api.github.com/repos/cdli-gh/mtaac_work/issues/1","repository_url":"https://api.github.com/repos/cdli-gh/mtaac_work","labels_url":"https://api.github.com/repos/cdli-gh/mtaac_work/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/cdli-gh/mtaac_work/issues/1/comments","events_url":"https://api.github.com/repos/cdli-gh/mtaac_work/issues/1/events","html_url":"https://github.com/cdli-gh/mtaac_work/issues/1","id":247167075,"number":1,"title":"Test tools for automated lemmatization","user":{"login":"khoidt","id":7487683,"avatar_url":"https://avatars0.githubusercontent.com/u/7487683?v=4","gravatar_id":"","url":"https://api.github.com/users/khoidt","html_url":"https://github.com/khoidt","followers_url":"https://api.github.com/users/khoidt/followers","following_url":"https://api.github.com/users/khoidt/following{/other_user}","gists_url":"https://api.github.com/users/khoidt/gists{/gist_id}","starred_url":"https://api.github.com/users/khoidt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/khoidt/subscriptions","organizations_url":"https://api.github.com/users/khoidt/orgs","repos_url":"https://api.github.com/users/khoidt/repos","events_url":"https://api.github.com/users/khoidt/events{/privacy}","received_events_url":"https://api.github.com/users/khoidt/received_events","type":"User","site_admin":false},"labels":[{"id":702295053,"url":"https://api.github.com/repos/cdli-gh/mtaac_work/labels/in%20progress","name":"in progress","color":"ededed","default":false}],"state":"open","locked":false,"assignee":{"login":"jayanthjaiswal","id":2018825,"avatar_url":"https://avatars0.githubusercontent.com/u/2018825?v=4","gravatar_id":"","url":"https://api.github.com/users/jayanthjaiswal","html_url":"https://github.com/jayanthjaiswal","followers_url":"https://api.github.com/users/jayanthjaiswal/followers","following_url":"https://api.github.com/users/jayanthjaiswal/following{/other_user}","gists_url":"https://api.github.com/users/jayanthjaiswal/gists{/gist_id}","starred_url":"https://api.github.com/users/jayanthjaiswal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jayanthjaiswal/subscriptions","organizations_url":"https://api.github.com/users/jayanthjaiswal/orgs","repos_url":"https://api.github.com/users/jayanthjaiswal/repos","events_url":"https://api.github.com/users/jayanthjaiswal/events{/privacy}","received_events_url":"https://api.github.com/users/jayanthjaiswal/received_events","type":"User","site_admin":false},"assignees":[{"login":"jayanthjaiswal","id":2018825,"avatar_url":"https://avatars0.githubusercontent.com/u/2018825?v=4","gravatar_id":"","url":"https://api.github.com/users/jayanthjaiswal","html_url":"https://github.com/jayanthjaiswal","followers_url":"https://api.github.com/users/jayanthjaiswal/followers","following_url":"https://api.github.com/users/jayanthjaiswal/following{/other_user}","gists_url":"https://api.github.com/users/jayanthjaiswal/gists{/gist_id}","starred_url":"https://api.github.com/users/jayanthjaiswal/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jayanthjaiswal/subscriptions","organizations_url":"https://api.github.com/users/jayanthjaiswal/orgs","repos_url":"https://api.github.com/users/jayanthjaiswal/repos","events_url":"https://api.github.com/users/jayanthjaiswal/events{/privacy}","received_events_url":"https://api.github.com/users/jayanthjaiswal/received_events","type":"User","site_admin":false},{"login":"khoidt","id":7487683,"avatar_url":"https://avatars0.githubusercontent.com/u/7487683?v=4","gravatar_id":"","url":"https://api.github.com/users/khoidt","html_url":"https://github.com/khoidt","followers_url":"https://api.github.com/users/khoidt/followers","following_url":"https://api.github.com/users/khoidt/following{/other_user}","gists_url":"https://api.github.com/users/khoidt/gists{/gist_id}","starred_url":"https://api.github.com/users/khoidt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/khoidt/subscriptions","organizations_url":"https://api.github.com/users/khoidt/orgs","repos_url":"https://api.github.com/users/khoidt/repos","events_url":"https://api.github.com/users/khoidt/events{/privacy}","received_events_url":"https://api.github.com/users/khoidt/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/cdli-gh/mtaac_work/milestones/1","html_url":"https://github.com/cdli-gh/mtaac_work/milestone/1","labels_url":"https://api.github.com/repos/cdli-gh/mtaac_work/milestones/1/labels","id":2831447,"number":1,"title":"M 1/4","description":"This is the first of four milestones of the project. Documentation and first research report should be completed for this milestone.","creator":{"login":"epageperron","id":14226547,"avatar_url":"https://avatars0.githubusercontent.com/u/14226547?v=4","gravatar_id":"","url":"https://api.github.com/users/epageperron","html_url":"https://github.com/epageperron","followers_url":"https://api.github.com/users/epageperron/followers","following_url":"https://api.github.com/users/epageperron/following{/other_user}","gists_url":"https://api.github.com/users/epageperron/gists{/gist_id}","starred_url":"https://api.github.com/users/epageperron/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/epageperron/subscriptions","organizations_url":"https://api.github.com/users/epageperron/orgs","repos_url":"https://api.github.com/users/epageperron/repos","events_url":"https://api.github.com/users/epageperron/events{/privacy}","received_events_url":"https://api.github.com/users/epageperron/received_events","type":"User","site_admin":false},"open_issues":4,"closed_issues":0,"state":"open","created_at":"2017-10-11T20:04:25Z","updated_at":"2017-10-13T12:05:31Z","due_on":"2017-12-22T08:00:00Z","closed_at":null},"comments":8,"created_at":"2017-08-01T18:54:52Z","updated_at":"2017-11-02T11:50:15Z","closed_at":null,"author_association":"COLLABORATOR","body":"## Summary\r\nFor the project, we need to test tools for automated lemmatization, stemming and morphology induction. Please try to get them running on and evaluate against ETCSL CoNLL data. If you run into difficulties, you can ask Kathrin for help (she's also preferring Python), if that doesn't help, have a try with the next tool on the list. Note that our top priority is to eliminate morphological variation, not so much to produce linguistically plausible lemmas.\r\n\r\n## Tasks\r\n- [x] CSTlemmatizer  (use the \"deprecated\" mode to train flex rules directly from it, https://github.com/kuhumcst/cstlemma)\r\n- [ ] Affixtrain + Bratmac (https://github.com/kuhumcst/affixtrain/tree/master/example)\r\n- [x] Affixtrain + CSTlemmatizer\r\n- [x] Morfessor (unsupervised morphology induction, train on word forms, only; https://pypi.python.org/pypi/Morfessor)\r\n- [ ] Clark 2003 (unsupervised POS tagging, train on word forms, only; http://www.cs.rhul.ac.uk/home/alexc/pos2.tar.gz)\r\n- [x] Marmot/Lemming (POS and morph: http://cistern.cis.lmu.de/marmot/, http://cistern.cis.lmu.de/lemming/)\r\n- [ ] the Oracc lemmatizer\r\n- [ ] Search for related (newer) tools from scientific literature, e.g. using ACL anthology or google scholar.\r\n\r\n## Notes\r\nCSTlemmatizer, Affixtrain and Marmot are supervised, train on WORD + LEMMA (+ POS + MORPH).\r\n\r\n\r\n## Roadmap Data\r\n\r\n#### 🗓 Start Date: 08-01-2017\r\n#### 🗓 Expected Date: 11-15-2017\r\n#### 💪 Label: wp\r\n#### 📈 Progress (0-1): 0.4\r\n\r\nSee Gantt: http://cdli-dev.org/gantt/mtaac_work/\r\n"},"comment":{"url":"https://api.github.com/repos/cdli-gh/mtaac_work/issues/comments/341397844","html_url":"https://github.com/cdli-gh/mtaac_work/issues/1#issuecomment-341397844","issue_url":"https://api.github.com/repos/cdli-gh/mtaac_work/issues/1","id":341397844,"user":{"login":"khoidt","id":7487683,"avatar_url":"https://avatars0.githubusercontent.com/u/7487683?v=4","gravatar_id":"","url":"https://api.github.com/users/khoidt","html_url":"https://github.com/khoidt","followers_url":"https://api.github.com/users/khoidt/followers","following_url":"https://api.github.com/users/khoidt/following{/other_user}","gists_url":"https://api.github.com/users/khoidt/gists{/gist_id}","starred_url":"https://api.github.com/users/khoidt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/khoidt/subscriptions","organizations_url":"https://api.github.com/users/khoidt/orgs","repos_url":"https://api.github.com/users/khoidt/repos","events_url":"https://api.github.com/users/khoidt/events{/privacy}","received_events_url":"https://api.github.com/users/khoidt/received_events","type":"User","site_admin":false},"created_at":"2017-11-02T11:50:15Z","updated_at":"2017-11-02T11:50:15Z","author_association":"COLLABORATOR","body":"@jayanthjaiswal Great job! As for the evaluation, the following is crucial:\r\n\r\n**Stem granularity**\r\n(Total number of unique golden (control set) lemmata / Total number of unique predicted (test result) lemmata) -1\r\n\r\n**Correct token forms**\r\nTotal number of correctly predicted lemmata (i.e. corresponding with golden): quantity and percent\r\n\r\n**Incorrect token forms**\r\nTotal number of incorrectly predicted lemmata (i.e. not corresponding with golden): quantity and percent\r\n\r\n**Correct stem forms**\r\nTotal number of correctly predicted lemmata in stem test (i.e. lemma given as form): quantity and percent\r\n\r\n**Incorrect stem forms**\r\nTotal number of incorrectly predicted lemmata in stem test (i.e. lemma given as form): quantity and percent\r\n\r\nIf you have any questions on the evaluation, write me a PM on Slack. I'm available most of the time."}}
{"actor":{"display_login":"khoidt"}}