{"action":"created","issue":{"url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/5","repository_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat","labels_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/5/labels{/name}","comments_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/5/comments","events_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/5/events","html_url":"https://github.com/epogrebnyak/data-team-ru-stat/issues/5","id":236392804,"number":5,"title":"Comments are open to Intro - part 1. Case for machine-readable data in economic analysis.","user":{"login":"epogrebnyak","id":9265326,"avatar_url":"https://avatars0.githubusercontent.com/u/9265326?v=3","gravatar_id":"","url":"https://api.github.com/users/epogrebnyak","html_url":"https://github.com/epogrebnyak","followers_url":"https://api.github.com/users/epogrebnyak/followers","following_url":"https://api.github.com/users/epogrebnyak/following{/other_user}","gists_url":"https://api.github.com/users/epogrebnyak/gists{/gist_id}","starred_url":"https://api.github.com/users/epogrebnyak/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/epogrebnyak/subscriptions","organizations_url":"https://api.github.com/users/epogrebnyak/orgs","repos_url":"https://api.github.com/users/epogrebnyak/repos","events_url":"https://api.github.com/users/epogrebnyak/events{/privacy}","received_events_url":"https://api.github.com/users/epogrebnyak/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-06-16T06:34:42Z","updated_at":"2017-06-16T07:37:37Z","closed_at":null,"body":"\r\nMy profession is economic analysis which is a range of tasks from \r\nsimple spreadsheet models (eg discounted cash flow to valuate a company)\r\nto econometric models (eg exchange rate forecasting). \r\n\r\nExcel is deeply rooted into economic analysis, but this is changing. \r\nIn Excel it is hard to make your graphs or models truely reproducible\r\nand transparent. In my mind economic analysis, say in a bank, will soon \r\nbe 'robotised', as many other jobs. A new generation analyst will \r\ncommand a collection of apps, not Excel spreadsheets. \r\n \r\nFor non-Excel workflow you need the following \r\n(see also [here](https://github.com/epogrebnyak/data-team-ru-stat/issues/4#issuecomment-308704058)):\r\n- *datasource*: a machine-readable datasource like FRED or other open data API\r\n- *data analysis tools*: something to dig data with and make a bunch of plots, \r\n  eg Pandas/R + possibly Jupiter notebook (former IPyhton)\r\n- *modelling tools*: a model to fit data and do forecasting, it is R/Gretl likely for econometrics, \r\n  something else for machine learning (not too familiar myself). Data analysis and modelling tool \r\n  can be the same package, eg R. \r\n- *presentation*: some way to produce reports and presentation which avoids or complements \r\nWord or Powerpoint: a Jupiter notebook, Gitpitch, or other\r\n\r\nBuilding a new workflow (important disclaimer: still not achieved for me) can go \r\ntop-down (new tools, old data) or bottom up (new data source first). Somehow due to \r\nquailty of machine readable data sources I went the 'start from the bottom' way, to be covered in \r\nsecond part of introduction.\r\n\r\nAny questions here:\r\n- Does \"datasource - analysis - model - presentation\" structure hold for projects you worked on? \r\n- What other tools/sources you find useful of each stage\r\n- Other comments/ideas?\r\n\r\nMay write in Russian, will summarise later."},"comment":{"url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/comments/308956258","html_url":"https://github.com/epogrebnyak/data-team-ru-stat/issues/5#issuecomment-308956258","issue_url":"https://api.github.com/repos/epogrebnyak/data-team-ru-stat/issues/5","id":308956258,"user":{"login":"Rotzke","id":9944454,"avatar_url":"https://avatars1.githubusercontent.com/u/9944454?v=3","gravatar_id":"","url":"https://api.github.com/users/Rotzke","html_url":"https://github.com/Rotzke","followers_url":"https://api.github.com/users/Rotzke/followers","following_url":"https://api.github.com/users/Rotzke/following{/other_user}","gists_url":"https://api.github.com/users/Rotzke/gists{/gist_id}","starred_url":"https://api.github.com/users/Rotzke/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Rotzke/subscriptions","organizations_url":"https://api.github.com/users/Rotzke/orgs","repos_url":"https://api.github.com/users/Rotzke/repos","events_url":"https://api.github.com/users/Rotzke/events{/privacy}","received_events_url":"https://api.github.com/users/Rotzke/received_events","type":"User","site_admin":false},"created_at":"2017-06-16T07:37:37Z","updated_at":"2017-06-16T07:37:37Z","body":"Projects in which I was involved were much sophisticated, with two main flows:\r\n\r\n- **ETL** (Extract/Load/Transform) part for data engineers, or sometimes data architects or database administrators (DBA).\r\n- **DAD** (Discover/Access/Distill) for data scientists.\r\n\r\nI was thinking about the structure of project according to README and structure of _Cookiecutter_ system, this tasks allocation model fits fine.\r\n\r\nYou didn't clearly tell what is the final desired form of the project @epogrebnyak but I assume that web application would be a great choice. \r\nWe could make a backend part - raw data gathering and storing with our own scraping framework/array plus MongoDB for keeping an archive so that scientists like you could easily get access to it, plus frontend on, let's say, Django, which @MrBorusLee is proficient with, and Redux NoSQL for quick access to current and actual data. Also, REST API would be great for both team and clients convenience.\r\n\r\nAs a concrete foundation for this, I would propose Amazon AWS, as mentioned before - it has a whole year of a free tier to let us perform tests and adjustments before full product deployment.\r\n\r\nAlso, would be cool to place this issue on next online meeting agenda, for it needs concurrent discussion, I think.\r\n\r\nThanks for attention! "}}
{"actor":{"display_login":"Rotzke"}}