{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/nmt/issues/112","repository_url":"https://api.github.com/repos/tensorflow/nmt","labels_url":"https://api.github.com/repos/tensorflow/nmt/issues/112/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/nmt/issues/112/comments","events_url":"https://api.github.com/repos/tensorflow/nmt/issues/112/events","html_url":"https://github.com/tensorflow/nmt/issues/112","id":258393113,"number":112,"title":"finetuning the model with new data samples","user":{"login":"kmario23","id":5196297,"avatar_url":"https://avatars2.githubusercontent.com/u/5196297?v=4","gravatar_id":"","url":"https://api.github.com/users/kmario23","html_url":"https://github.com/kmario23","followers_url":"https://api.github.com/users/kmario23/followers","following_url":"https://api.github.com/users/kmario23/following{/other_user}","gists_url":"https://api.github.com/users/kmario23/gists{/gist_id}","starred_url":"https://api.github.com/users/kmario23/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmario23/subscriptions","organizations_url":"https://api.github.com/users/kmario23/orgs","repos_url":"https://api.github.com/users/kmario23/repos","events_url":"https://api.github.com/users/kmario23/events{/privacy}","received_events_url":"https://api.github.com/users/kmario23/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-09-18T07:45:33Z","updated_at":"2017-09-20T07:16:23Z","closed_at":null,"author_association":"NONE","body":"I have trained a seq2seq NMT model (EN-DE) with 1M samples and saved the latest checkpoint. Now, I have some domain-specific data of 50K sentence pairs which has **not** been seen in previous training data. How can I adapt the current model to this new data?\r\n\r\nSpecifically, I'd like to `finetune` the model to the new domain and as I increase the number of samples in that domain, the model should output reasonably well translation for test sentences in that domain. The `finetuning` is commonly done in computer vision but I'm not sure how to achieve this in `seq2seq` architecture. \r\n\r\nI'm aware of the fact that the `vocabulary` files for both languages has to be updated according to the new sentence pairs. But, to achieve this, do we have to again start training from scratch? Isn't there a smarter way to _continue training_ from the current checkpoint after dynamically updating the necessary components?\r\n\r\nAny ideas or relevant papers which address this issue?"},"comment":{"url":"https://api.github.com/repos/tensorflow/nmt/issues/comments/330766230","html_url":"https://github.com/tensorflow/nmt/issues/112#issuecomment-330766230","issue_url":"https://api.github.com/repos/tensorflow/nmt/issues/112","id":330766230,"user":{"login":"lmthang","id":396613,"avatar_url":"https://avatars3.githubusercontent.com/u/396613?v=4","gravatar_id":"","url":"https://api.github.com/users/lmthang","html_url":"https://github.com/lmthang","followers_url":"https://api.github.com/users/lmthang/followers","following_url":"https://api.github.com/users/lmthang/following{/other_user}","gists_url":"https://api.github.com/users/lmthang/gists{/gist_id}","starred_url":"https://api.github.com/users/lmthang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lmthang/subscriptions","organizations_url":"https://api.github.com/users/lmthang/orgs","repos_url":"https://api.github.com/users/lmthang/repos","events_url":"https://api.github.com/users/lmthang/events{/privacy}","received_events_url":"https://api.github.com/users/lmthang/received_events","type":"User","site_admin":false},"created_at":"2017-09-20T07:16:23Z","updated_at":"2017-09-20T07:16:23Z","author_association":"CONTRIBUTOR","body":"You can change the train_prefix but keep the out_dir argument. Technically, it should work (or would require minimal changes to get things to work). I would recommend using subword units (BPE) if you haven't to present unseen words. Also, make sure you back up the checkpoints before trying :)\r\n\r\nSee section 3 here for NMT adaptation https://nlp.stanford.edu/~lmthang/data/papers/iwslt15.pdf.\r\n"}}
{"actor":{"display_login":"lmthang"}}