{"action":"created","issue":{"url":"https://api.github.com/repos/druid-io/druid/issues/3674","repository_url":"https://api.github.com/repos/druid-io/druid","labels_url":"https://api.github.com/repos/druid-io/druid/issues/3674/labels{/name}","comments_url":"https://api.github.com/repos/druid-io/druid/issues/3674/comments","events_url":"https://api.github.com/repos/druid-io/druid/issues/3674/events","html_url":"https://github.com/druid-io/druid/pull/3674","id":188379431,"number":3674,"title":"Use timestamp in millis as Map key instead of DateTime object","user":{"login":"praveev","id":297278,"avatar_url":"https://avatars3.githubusercontent.com/u/297278?v=3","gravatar_id":"","url":"https://api.github.com/users/praveev","html_url":"https://github.com/praveev","followers_url":"https://api.github.com/users/praveev/followers","following_url":"https://api.github.com/users/praveev/following{/other_user}","gists_url":"https://api.github.com/users/praveev/gists{/gist_id}","starred_url":"https://api.github.com/users/praveev/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/praveev/subscriptions","organizations_url":"https://api.github.com/users/praveev/orgs","repos_url":"https://api.github.com/users/praveev/repos","events_url":"https://api.github.com/users/praveev/events{/privacy}","received_events_url":"https://api.github.com/users/praveev/received_events","type":"User","site_admin":false},"labels":[{"id":321808195,"url":"https://api.github.com/repos/druid-io/druid/labels/Improvement","name":"Improvement","color":"0052cc","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/druid-io/druid/milestones/21","html_url":"https://github.com/druid-io/druid/milestone/21","labels_url":"https://api.github.com/repos/druid-io/druid/milestones/21/labels","id":1831000,"number":21,"title":"0.10.0","description":"","creator":{"login":"fjy","id":428325,"avatar_url":"https://avatars3.githubusercontent.com/u/428325?v=3","gravatar_id":"","url":"https://api.github.com/users/fjy","html_url":"https://github.com/fjy","followers_url":"https://api.github.com/users/fjy/followers","following_url":"https://api.github.com/users/fjy/following{/other_user}","gists_url":"https://api.github.com/users/fjy/gists{/gist_id}","starred_url":"https://api.github.com/users/fjy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fjy/subscriptions","organizations_url":"https://api.github.com/users/fjy/orgs","repos_url":"https://api.github.com/users/fjy/repos","events_url":"https://api.github.com/users/fjy/events{/privacy}","received_events_url":"https://api.github.com/users/fjy/received_events","type":"User","site_admin":false},"open_issues":0,"closed_issues":309,"state":"open","created_at":"2016-06-15T22:35:58Z","updated_at":"2017-04-18T20:34:34Z","due_on":null,"closed_at":null},"comments":7,"created_at":"2016-11-09T23:25:49Z","updated_at":"2017-05-08T23:22:46Z","closed_at":"2016-11-11T18:20:20Z","pull_request":{"url":"https://api.github.com/repos/druid-io/druid/pulls/3674","html_url":"https://github.com/druid-io/druid/pull/3674","diff_url":"https://github.com/druid-io/druid/pull/3674.diff","patch_url":"https://github.com/druid-io/druid/pull/3674.patch"},"body":"This PR addresses 3 issues:\r\n1. Replace DateTime to Long if it used as a key in a Map b/c of some issues described below.\r\n2. Document caveats surrounding timezoned segments\r\n3. Fix an issue where `hashed` partitions doesn't respect timzone when creating the partitions.json file.\r\n\r\n**Replacing DateTime with Long as map key**\r\nWhen you use DateTime as key for the map, you'll see unexpected behavior when DateTime has timezone associated.\r\n\r\nFor example: Here's a piece of code that fails when compiled with `-Duser.timezone=UTC` setting. \r\n```\r\nDateTime odt = DateTime.now(DateTimeUtils.getZone(DateTimeZone.forID(\"America/Los_Angeles\")));\r\nHashMap<DateTime, String> map = new HashMap<>();\r\nmap.put(odt, \"abc\");\r\nDateTime dt = new DateTime(odt.getMillis());\r\nSystem.out.println(map.get(dt));  // will print null\r\n```\r\nWith the above, simple DateTime lookup fails. DateTime sometimes uses YYYY-MM-DDT07:00:00Z vs YYYY-MM-DDT:00:00:000-07:00Z as the key although both point to the same instant in time.\r\n\r\nSo replacing the key to be a Long, is more predictable and addresses this issue. I've tried to replace all the places in the code where DateTime is used as key.\r\n\r\nWhen we try to apply timezones to the intervals to produce timezoned segments, it breaks because of this bug.\r\n\r\n**Caveats around timezoned segments**\r\nSince the timezone is respected only for batch ingestion with explicitly specified intervals, I've updated the docs to call out that timezoned segments are not supported for realtime kafka ingestion and if intervals are not specified for batch ingestion.\r\n\r\n**partitions.json doesn't respect timezone with hashed partition**\r\nAlso, DetermineHashedPartitionsJob creates the partitions.json file under the directory that isn't timezone specific. This is because SegmentGranularity's getBucket() doesn't respect timezone whereas getBucketInterval() from the SegmentGranularitySpec does. So I ended up using that instead. I believe the intervals have 1-1 correspondence with the segments being created.\r\n"},"comment":{"url":"https://api.github.com/repos/druid-io/druid/issues/comments/300018078","html_url":"https://github.com/druid-io/druid/pull/3674#issuecomment-300018078","issue_url":"https://api.github.com/repos/druid-io/druid/issues/3674","id":300018078,"user":{"login":"leventov","id":609240,"avatar_url":"https://avatars3.githubusercontent.com/u/609240?v=3","gravatar_id":"","url":"https://api.github.com/users/leventov","html_url":"https://github.com/leventov","followers_url":"https://api.github.com/users/leventov/followers","following_url":"https://api.github.com/users/leventov/following{/other_user}","gists_url":"https://api.github.com/users/leventov/gists{/gist_id}","starred_url":"https://api.github.com/users/leventov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leventov/subscriptions","organizations_url":"https://api.github.com/users/leventov/orgs","repos_url":"https://api.github.com/users/leventov/repos","events_url":"https://api.github.com/users/leventov/events{/privacy}","received_events_url":"https://api.github.com/users/leventov/received_events","type":"User","site_admin":false},"created_at":"2017-05-08T23:22:46Z","updated_at":"2017-05-08T23:22:46Z","body":"@praveev https://github.com/druid-io/druid/pull/4252/files#diff-632d3638579cd432dce6307939579389R82"}}
{"actor":{"display_login":"leventov"}}