{"action":"created","issue":{"url":"https://api.github.com/repos/metabase/metabase/issues/6590","repository_url":"https://api.github.com/repos/metabase/metabase","labels_url":"https://api.github.com/repos/metabase/metabase/issues/6590/labels{/name}","comments_url":"https://api.github.com/repos/metabase/metabase/issues/6590/comments","events_url":"https://api.github.com/repos/metabase/metabase/issues/6590/events","html_url":"https://github.com/metabase/metabase/pull/6590","id":283044512,"number":6590,"title":"Improve `/database/:id/fields` response time","user":{"login":"senior","id":215910,"avatar_url":"https://avatars3.githubusercontent.com/u/215910?v=4","gravatar_id":"","url":"https://api.github.com/users/senior","html_url":"https://github.com/senior","followers_url":"https://api.github.com/users/senior/followers","following_url":"https://api.github.com/users/senior/following{/other_user}","gists_url":"https://api.github.com/users/senior/gists{/gist_id}","starred_url":"https://api.github.com/users/senior/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/senior/subscriptions","organizations_url":"https://api.github.com/users/senior/orgs","repos_url":"https://api.github.com/users/senior/repos","events_url":"https://api.github.com/users/senior/events{/privacy}","received_events_url":"https://api.github.com/users/senior/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2017-12-18T22:37:41Z","updated_at":"2017-12-19T20:18:05Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/metabase/metabase/pulls/6590","html_url":"https://github.com/metabase/metabase/pull/6590","diff_url":"https://github.com/metabase/metabase/pull/6590.diff","patch_url":"https://github.com/metabase/metabase/pull/6590.patch"},"body":"This PR makes two small changes to the `/database/:id/fields` API call. The first is using the `generate-stream` function from Cheshire, rather than using `generate-string`. This avoids holding the entire string representation of the fields response in memory before returning it. Very rough benchmarking looks like it consumes about half the memory as the previous solution. This would eventually get GC'd anyway, but that takes CPU cycles to, so if you create less garbage, you clean less garbage, it's faster.\r\n\r\nThe second part switches to a `java.io.PipedInputStream` and `java.io.PipedOutputStream`. This will allow streaming JSON responses to the client. Although the total response time is roughly the same, we can return the response initially and the client can parse the response as it is received. With the test schema I have locally, I went from roughly a 70s wait for the initial response (4s to receive the data) to 28s waiting for the response (46s receiving)."},"comment":{"url":"https://api.github.com/repos/metabase/metabase/issues/comments/352874411","html_url":"https://github.com/metabase/metabase/pull/6590#issuecomment-352874411","issue_url":"https://api.github.com/repos/metabase/metabase/issues/6590","id":352874411,"user":{"login":"salsakran","id":107439,"avatar_url":"https://avatars0.githubusercontent.com/u/107439?v=4","gravatar_id":"","url":"https://api.github.com/users/salsakran","html_url":"https://github.com/salsakran","followers_url":"https://api.github.com/users/salsakran/followers","following_url":"https://api.github.com/users/salsakran/following{/other_user}","gists_url":"https://api.github.com/users/salsakran/gists{/gist_id}","starred_url":"https://api.github.com/users/salsakran/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/salsakran/subscriptions","organizations_url":"https://api.github.com/users/salsakran/orgs","repos_url":"https://api.github.com/users/salsakran/repos","events_url":"https://api.github.com/users/salsakran/events{/privacy}","received_events_url":"https://api.github.com/users/salsakran/received_events","type":"User","site_admin":false},"created_at":"2017-12-19T20:18:05Z","updated_at":"2017-12-19T20:18:05Z","author_association":"CONTRIBUTOR","body":"Just for posterity, this would be even more awesome if we could switch the frontend to do streaming json parsing and add streaming to toucan. \r\n\r\nMerging for the immediate memory improvements."}}
{"actor":{"display_login":"salsakran"}}