{"action":"created","issue":{"url":"https://api.github.com/repos/spotify/luigi/issues/2276","repository_url":"https://api.github.com/repos/spotify/luigi","labels_url":"https://api.github.com/repos/spotify/luigi/issues/2276/labels{/name}","comments_url":"https://api.github.com/repos/spotify/luigi/issues/2276/comments","events_url":"https://api.github.com/repos/spotify/luigi/issues/2276/events","html_url":"https://github.com/spotify/luigi/pull/2276","id":270843744,"number":2276,"title":"Added JupyterNotebookTask to run Jupyter notebooks as luigi tasks","user":{"login":"mattiaciollaro","id":14982397,"avatar_url":"https://avatars3.githubusercontent.com/u/14982397?v=4","gravatar_id":"","url":"https://api.github.com/users/mattiaciollaro","html_url":"https://github.com/mattiaciollaro","followers_url":"https://api.github.com/users/mattiaciollaro/followers","following_url":"https://api.github.com/users/mattiaciollaro/following{/other_user}","gists_url":"https://api.github.com/users/mattiaciollaro/gists{/gist_id}","starred_url":"https://api.github.com/users/mattiaciollaro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattiaciollaro/subscriptions","organizations_url":"https://api.github.com/users/mattiaciollaro/orgs","repos_url":"https://api.github.com/users/mattiaciollaro/repos","events_url":"https://api.github.com/users/mattiaciollaro/events{/privacy}","received_events_url":"https://api.github.com/users/mattiaciollaro/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-11-02T23:50:50Z","updated_at":"2017-11-03T01:57:49Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/spotify/luigi/pulls/2276","html_url":"https://github.com/spotify/luigi/pull/2276","diff_url":"https://github.com/spotify/luigi/pull/2276.diff","patch_url":"https://github.com/spotify/luigi/pull/2276.patch"},"body":"<!--- This template is optional. Please use it as a starting point to help guide PRs -->\r\n\r\n<!--- Provide a general summary of your changes in the Title above -->\r\n\r\n## Description\r\n<!--- Describe your changes -->\r\nA task designed to run Jupyter notebooks as luigi tasks that also allows to pass parameters to the notebook from outside of the notebook.\r\n\r\nJupyterNotebookTask requires the `nbconvert` and the `nbformat` modules.\r\n\r\n## Motivation and Context\r\n<!--- Why is this change required? What problem does it solve? -->\r\n<!--- If it fixes an open issue, please link to the issue here. -->\r\nJupyterNotebookTask allows to easily integrate in a luigi pipeline code that has been written in Jupyter Notebooks.\r\n\r\nFrequently, users write code in Jupyter notebooks and then want to stitch them together in luigi pipelines. Pipelines with notebooks are harder to write and execute because it is difficult to pass parameters from outside of the notebooks to the notebooks themselves.\r\n\r\nJupyterNotebookTask passes parameters such as `self.input()[...].path` and `self.output().path` to the notebook, along with any additional user-defined parameters. The notebook is executed via nbconvert by the task's run method.\r\n\r\nThe run method of JupyterNotebookTask allows the user to specify a `pars` dictionary with the parameters to be passed to the notebook. Then, during the execution of the run method, the task\r\nwrites this dictionary to a temporary JSON file in the same location of the notebook. The temporary file is always named `notebook_title.ipynbpars`, where `notebook_title` is the title of the Jupyter notebook. The notebook can access the `pars` dictionary by reading the contents of the JSON file. Once the run method is executed (successfully or not), the temporary JSON file is deleted.\r\n\r\n## Have you tested this? If so, how?\r\n<!--- Valid responses are \"I have included unit tests.\" or --> \r\n<!--- \"I ran my jobs with this code and it works for me.\" -->\r\n\r\nThis has been successfully tested both in Python 2.7 and Python 3.6 with R and Python notebooks.\r\n\r\n<!---\r\nfor more information on how to submit valuable contributions,\r\nsee https://opensource.guide/how-to-contribute/#how-to-submit-a-contribution\r\n-->"},"comment":{"url":"https://api.github.com/repos/spotify/luigi/issues/comments/341606804","html_url":"https://github.com/spotify/luigi/pull/2276#issuecomment-341606804","issue_url":"https://api.github.com/repos/spotify/luigi/issues/2276","id":341606804,"user":{"login":"erikbern","id":1027979,"avatar_url":"https://avatars2.githubusercontent.com/u/1027979?v=4","gravatar_id":"","url":"https://api.github.com/users/erikbern","html_url":"https://github.com/erikbern","followers_url":"https://api.github.com/users/erikbern/followers","following_url":"https://api.github.com/users/erikbern/following{/other_user}","gists_url":"https://api.github.com/users/erikbern/gists{/gist_id}","starred_url":"https://api.github.com/users/erikbern/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erikbern/subscriptions","organizations_url":"https://api.github.com/users/erikbern/orgs","repos_url":"https://api.github.com/users/erikbern/repos","events_url":"https://api.github.com/users/erikbern/events{/privacy}","received_events_url":"https://api.github.com/users/erikbern/received_events","type":"User","site_admin":false},"created_at":"2017-11-03T01:57:49Z","updated_at":"2017-11-03T01:57:49Z","author_association":"COLLABORATOR","body":"cool!"}}
{"actor":{"display_login":"erikbern"}}