{"action":"created","issue":{"url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/issues/36","repository_url":"https://api.github.com/repos/datasciencebr/serenata-toolbox","labels_url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/issues/36/labels{/name}","comments_url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/issues/36/comments","events_url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/issues/36/events","html_url":"https://github.com/datasciencebr/serenata-toolbox/pull/36","id":219463369,"number":36,"title":"Changing reimbursement data collection to conform new version","user":{"login":"luizcavalcanti","id":948562,"avatar_url":"https://avatars1.githubusercontent.com/u/948562?v=3","gravatar_id":"","url":"https://api.github.com/users/luizcavalcanti","html_url":"https://github.com/luizcavalcanti","followers_url":"https://api.github.com/users/luizcavalcanti/followers","following_url":"https://api.github.com/users/luizcavalcanti/following{/other_user}","gists_url":"https://api.github.com/users/luizcavalcanti/gists{/gist_id}","starred_url":"https://api.github.com/users/luizcavalcanti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/luizcavalcanti/subscriptions","organizations_url":"https://api.github.com/users/luizcavalcanti/orgs","repos_url":"https://api.github.com/users/luizcavalcanti/repos","events_url":"https://api.github.com/users/luizcavalcanti/events{/privacy}","received_events_url":"https://api.github.com/users/luizcavalcanti/received_events","type":"User","site_admin":false},"labels":[{"id":535024207,"url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/labels/work%20in%20progress","name":"work in progress","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":8,"created_at":"2017-04-05T04:10:55Z","updated_at":"2017-04-23T16:37:13Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/pulls/36","html_url":"https://github.com/datasciencebr/serenata-toolbox/pull/36","diff_url":"https://github.com/datasciencebr/serenata-toolbox/pull/36.diff","patch_url":"https://github.com/datasciencebr/serenata-toolbox/pull/36.patch"},"body":"The brazilian congress made available a new version of the reimbursement\r\ndataset, which now can be downloaded already in csv format and separated\r\nby year. This somewhat simplifies the current fetching routine, but it\r\nalso demands a lot of small changes in the way the dataset is aquired,\r\ntranslated, loaded and merged.\r\n\r\nI this rather large commit, the unit tests were also modified to conform\r\nto the new way of fetching, and the \"convert to csv\" test was removed,\r\nsince this operation is not needed anymore. The convert_to_csv() mehtod\r\nfrom the CEAPDataset class was kept, though, so we don't break rosie\r\nuntil it removes the call to this method.\r\n\r\nAnother quite drastic change was made in the test cases' names. A number\r\nwas added to them, so we can 'make sure' a proper order is followed\r\nduring integration tests. I strongly believe there is a better way of\r\ndoing this. I can work on it in a near-future commit.\r\n\r\nSigned-off-by: Luiz Carlos Cavalcanti <cavalcanti.luiz@gmail.com>"},"comment":{"url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/issues/comments/296455535","html_url":"https://github.com/datasciencebr/serenata-toolbox/pull/36#issuecomment-296455535","issue_url":"https://api.github.com/repos/datasciencebr/serenata-toolbox/issues/36","id":296455535,"user":{"login":"cuducos","id":4732915,"avatar_url":"https://avatars1.githubusercontent.com/u/4732915?v=3","gravatar_id":"","url":"https://api.github.com/users/cuducos","html_url":"https://github.com/cuducos","followers_url":"https://api.github.com/users/cuducos/followers","following_url":"https://api.github.com/users/cuducos/following{/other_user}","gists_url":"https://api.github.com/users/cuducos/gists{/gist_id}","starred_url":"https://api.github.com/users/cuducos/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cuducos/subscriptions","organizations_url":"https://api.github.com/users/cuducos/orgs","repos_url":"https://api.github.com/users/cuducos/repos","events_url":"https://api.github.com/users/cuducos/events{/privacy}","received_events_url":"https://api.github.com/users/cuducos/received_events","type":"User","site_admin":false},"created_at":"2017-04-23T16:37:13Z","updated_at":"2017-04-23T16:37:13Z","body":"> I did no notebook on this, only compared with interactive python and some bash utils, if that count as analysis. In short, I found out that the data is the same between the API versions.\r\n\r\nI encourage you to do either share this analysis here (as a comment) or, even better: once this is merged, add a notebook to `serenata-de-amor` repo with the comparison between the two latest versions of `reimbursements.xz` (yours and current latest).\r\n\r\n> About testing, what kind of tests do you foresee in this case? Is it worth to validate the translation itself? Or even make tests fail if there is new subquota categories?\r\n\r\nI would use `unittest.mock` to avoid I/O basically. I don't need to actually run, for example, `urlretrieve` — I need to be sure it was called with the proper arguments. This means we don't depend on file system (mocking read/write methods) and network (mocking requests), i.e. faster tests, and tests restricted to what actually need to be tested.\r\n\r\nThese mocks are not trivial, so I don't push everyone to write them — unless anyone is up for pairing on that one of these days ; )"}}
{"actor":{"display_login":"cuducos"}}