{"action":"created","issue":{"url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/8","repository_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat","labels_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/8/comments","events_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/8/events","html_url":"https://github.com/epogrebnyak/data-team-ru-stat/issues/8","id":236734044,"number":8,"title":"Data pipeline ","user":{"login":"epogrebnyak","id":9265326,"avatar_url":"https://avatars0.githubusercontent.com/u/9265326?v=3","gravatar_id":"","url":"https://api.github.com/users/epogrebnyak","html_url":"https://github.com/epogrebnyak","followers_url":"https://api.github.com/users/epogrebnyak/followers","following_url":"https://api.github.com/users/epogrebnyak/following{/other_user}","gists_url":"https://api.github.com/users/epogrebnyak/gists{/gist_id}","starred_url":"https://api.github.com/users/epogrebnyak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/epogrebnyak/subscriptions","organizations_url":"https://api.github.com/users/epogrebnyak/orgs","repos_url":"https://api.github.com/users/epogrebnyak/repos","events_url":"https://api.github.com/users/epogrebnyak/events{/privacy}","received_events_url":"https://api.github.com/users/epogrebnyak/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"neotheicebird","id":4669552,"avatar_url":"https://avatars0.githubusercontent.com/u/4669552?v=3","gravatar_id":"","url":"https://api.github.com/users/neotheicebird","html_url":"https://github.com/neotheicebird","followers_url":"https://api.github.com/users/neotheicebird/followers","following_url":"https://api.github.com/users/neotheicebird/following{/other_user}","gists_url":"https://api.github.com/users/neotheicebird/gists{/gist_id}","starred_url":"https://api.github.com/users/neotheicebird/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/neotheicebird/subscriptions","organizations_url":"https://api.github.com/users/neotheicebird/orgs","repos_url":"https://api.github.com/users/neotheicebird/repos","events_url":"https://api.github.com/users/neotheicebird/events{/privacy}","received_events_url":"https://api.github.com/users/neotheicebird/received_events","type":"User","site_admin":false},"assignees":[{"login":"neotheicebird","id":4669552,"avatar_url":"https://avatars0.githubusercontent.com/u/4669552?v=3","gravatar_id":"","url":"https://api.github.com/users/neotheicebird","html_url":"https://github.com/neotheicebird","followers_url":"https://api.github.com/users/neotheicebird/followers","following_url":"https://api.github.com/users/neotheicebird/following{/other_user}","gists_url":"https://api.github.com/users/neotheicebird/gists{/gist_id}","starred_url":"https://api.github.com/users/neotheicebird/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/neotheicebird/subscriptions","organizations_url":"https://api.github.com/users/neotheicebird/orgs","repos_url":"https://api.github.com/users/neotheicebird/repos","events_url":"https://api.github.com/users/neotheicebird/events{/privacy}","received_events_url":"https://api.github.com/users/neotheicebird/received_events","type":"User","site_admin":false},{"login":"Rotzke","id":9944454,"avatar_url":"https://avatars1.githubusercontent.com/u/9944454?v=3","gravatar_id":"","url":"https://api.github.com/users/Rotzke","html_url":"https://github.com/Rotzke","followers_url":"https://api.github.com/users/Rotzke/followers","following_url":"https://api.github.com/users/Rotzke/following{/other_user}","gists_url":"https://api.github.com/users/Rotzke/gists{/gist_id}","starred_url":"https://api.github.com/users/Rotzke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Rotzke/subscriptions","organizations_url":"https://api.github.com/users/Rotzke/orgs","repos_url":"https://api.github.com/users/Rotzke/repos","events_url":"https://api.github.com/users/Rotzke/events{/privacy}","received_events_url":"https://api.github.com/users/Rotzke/received_events","type":"User","site_admin":false},{"login":"Akrishna91","id":10672772,"avatar_url":"https://avatars1.githubusercontent.com/u/10672772?v=3","gravatar_id":"","url":"https://api.github.com/users/Akrishna91","html_url":"https://github.com/Akrishna91","followers_url":"https://api.github.com/users/Akrishna91/followers","following_url":"https://api.github.com/users/Akrishna91/following{/other_user}","gists_url":"https://api.github.com/users/Akrishna91/gists{/gist_id}","starred_url":"https://api.github.com/users/Akrishna91/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Akrishna91/subscriptions","organizations_url":"https://api.github.com/users/Akrishna91/orgs","repos_url":"https://api.github.com/users/Akrishna91/repos","events_url":"https://api.github.com/users/Akrishna91/events{/privacy}","received_events_url":"https://api.github.com/users/Akrishna91/received_events","type":"User","site_admin":false},{"login":"MrBorusLee","id":13832204,"avatar_url":"https://avatars0.githubusercontent.com/u/13832204?v=3","gravatar_id":"","url":"https://api.github.com/users/MrBorusLee","html_url":"https://github.com/MrBorusLee","followers_url":"https://api.github.com/users/MrBorusLee/followers","following_url":"https://api.github.com/users/MrBorusLee/following{/other_user}","gists_url":"https://api.github.com/users/MrBorusLee/gists{/gist_id}","starred_url":"https://api.github.com/users/MrBorusLee/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MrBorusLee/subscriptions","organizations_url":"https://api.github.com/users/MrBorusLee/orgs","repos_url":"https://api.github.com/users/MrBorusLee/repos","events_url":"https://api.github.com/users/MrBorusLee/events{/privacy}","received_events_url":"https://api.github.com/users/MrBorusLee/received_events","type":"User","site_admin":false},{"login":"rohlin17","id":13986866,"avatar_url":"https://avatars2.githubusercontent.com/u/13986866?v=3","gravatar_id":"","url":"https://api.github.com/users/rohlin17","html_url":"https://github.com/rohlin17","followers_url":"https://api.github.com/users/rohlin17/followers","following_url":"https://api.github.com/users/rohlin17/following{/other_user}","gists_url":"https://api.github.com/users/rohlin17/gists{/gist_id}","starred_url":"https://api.github.com/users/rohlin17/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rohlin17/subscriptions","organizations_url":"https://api.github.com/users/rohlin17/orgs","repos_url":"https://api.github.com/users/rohlin17/repos","events_url":"https://api.github.com/users/rohlin17/events{/privacy}","received_events_url":"https://api.github.com/users/rohlin17/received_events","type":"User","site_admin":false}],"milestone":null,"comments":2,"created_at":"2017-06-18T17:23:14Z","updated_at":"2017-06-18T17:54:08Z","closed_at":null,"body":"Our project is about aggregating data from individual parsers under common namespace and releasing the data through final API (correct if something missing):\r\n\r\n1. Raw data source - a file or a web page at Rosstat or other agency.\r\n2. Parser reads source and provides resulting CSVs in its ```data/processed``` folder.\r\n3. Some automated and human eye validation is done during/after parsing.\r\n4. 'daemon' collects ```data/processed``` CSVs from several parsers in common database.\r\n5. The user queries database through API to get aggregated data from parsers\r\n(the simplest call returns aggregate CSV with variables from diffrernt parsers).\r\n6. The user reads data from from URL or stores data as a local file. \r\n7. The user writes code in Jupiter notebook with R or pandas using the data.\r\n\r\n\r\nComments welcome."},"comment":{"url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/comments/309292352","html_url":"https://github.com/epogrebnyak/data-team-ru-stat/issues/8#issuecomment-309292352","issue_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/8","id":309292352,"user":{"login":"Rotzke","id":9944454,"avatar_url":"https://avatars1.githubusercontent.com/u/9944454?v=3","gravatar_id":"","url":"https://api.github.com/users/Rotzke","html_url":"https://github.com/Rotzke","followers_url":"https://api.github.com/users/Rotzke/followers","following_url":"https://api.github.com/users/Rotzke/following{/other_user}","gists_url":"https://api.github.com/users/Rotzke/gists{/gist_id}","starred_url":"https://api.github.com/users/Rotzke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Rotzke/subscriptions","organizations_url":"https://api.github.com/users/Rotzke/orgs","repos_url":"https://api.github.com/users/Rotzke/repos","events_url":"https://api.github.com/users/Rotzke/events{/privacy}","received_events_url":"https://api.github.com/users/Rotzke/received_events","type":"User","site_admin":false},"created_at":"2017-06-18T17:54:07Z","updated_at":"2017-06-18T17:54:07Z","body":"@neotheicebird Yeah, we discussed it at the meeting - using AWS API."}}
{"actor":{"display_login":"Rotzke"}}