{"action":"created","comment":{"url":"https://api.github.com/repos/seznam/euphoria/pulls/comments/106893952","pull_request_review_id":27835939,"id":106893952,"diff_hunk":"@@ -0,0 +1,68 @@\n+/**\n+ * Copyright 2016 Seznam.cz, a.s.\n+ *\n+ * Licensed under the Apache License, Version 2.0 (the \"License\");\n+ * you may not use this file except in compliance with the License.\n+ * You may obtain a copy of the License at\n+ *\n+ *     http://www.apache.org/licenses/LICENSE-2.0\n+ *\n+ * Unless required by applicable law or agreed to in writing, software\n+ * distributed under the License is distributed on an \"AS IS\" BASIS,\n+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n+ * See the License for the specific language governing permissions and\n+ * limitations under the License.\n+ */\n+package cz.seznam.euphoria.flink.streaming.windowing;\n+\n+import cz.seznam.euphoria.core.client.dataset.windowing.Batch;\n+import cz.seznam.euphoria.core.client.dataset.windowing.Window;\n+import cz.seznam.euphoria.core.client.dataset.windowing.WindowedElement;\n+import cz.seznam.euphoria.core.client.dataset.windowing.Windowing;\n+import cz.seznam.euphoria.core.client.triggers.Trigger;\n+import cz.seznam.euphoria.core.client.triggers.TriggerContext;\n+\n+import java.util.Collections;\n+import java.util.Set;\n+\n+public class AttachedWindowing<T, WID extends Window> implements Windowing<T, WID> {\n+\n+  @Override\n+  @SuppressWarnings(\"unchecked\")\n+  public Set<WID> assignWindowsToElement(WindowedElement<?, T> el) {\n+    return Collections.singleton((WID) el.getWindow());\n+  }\n+\n+  @Override\n+  public Trigger<WID> getTrigger() {\n+    return new AttachedWindowTrigger<>();\n+  }\n+\n+  private static class AttachedWindowTrigger<WID extends Window> implements Trigger<WID> {\n+\n+    @Override\n+    public TriggerResult onElement(long time, WID window, TriggerContext ctx) {\n+      // FIXME batch window shouldn't be used in stream flow in the future\n+      // issue #38 on GitHub\n+      if (window instanceof Batch.BatchWindow) return TriggerResult.NOOP;","path":"euphoria-flink/src/main/java/cz/seznam/euphoria/flink/streaming/windowing/AttachedWindowing.java","position":47,"original_position":47,"commit_id":"c887c8e8c505c3aeb98d7da7d80398ed09f79107","original_commit_id":"c887c8e8c505c3aeb98d7da7d80398ed09f79107","user":{"login":"xitep","id":8274125,"avatar_url":"https://avatars0.githubusercontent.com/u/8274125?v=3","gravatar_id":"","url":"https://api.github.com/users/xitep","html_url":"https://github.com/xitep","followers_url":"https://api.github.com/users/xitep/followers","following_url":"https://api.github.com/users/xitep/following{/other_user}","gists_url":"https://api.github.com/users/xitep/gists{/gist_id}","starred_url":"https://api.github.com/users/xitep/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xitep/subscriptions","organizations_url":"https://api.github.com/users/xitep/orgs","repos_url":"https://api.github.com/users/xitep/repos","events_url":"https://api.github.com/users/xitep/events{/privacy}","received_events_url":"https://api.github.com/users/xitep/received_events","type":"User","site_admin":false},"body":"what about throwing an exception here?","created_at":"2017-03-20T12:59:59Z","updated_at":"2017-03-20T13:03:32Z","html_url":"https://github.com/seznam/euphoria/pull/50#discussion_r106893952","pull_request_url":"https://api.github.com/repos/seznam/euphoria/pulls/50","_links":{"self":{"href":"https://api.github.com/repos/seznam/euphoria/pulls/comments/106893952"},"html":{"href":"https://github.com/seznam/euphoria/pull/50#discussion_r106893952"},"pull_request":{"href":"https://api.github.com/repos/seznam/euphoria/pulls/50"}}},"pull_request":{"url":"https://api.github.com/repos/seznam/euphoria/pulls/50","id":111446177,"html_url":"https://github.com/seznam/euphoria/pull/50","diff_url":"https://github.com/seznam/euphoria/pull/50.diff","patch_url":"https://github.com/seznam/euphoria/pull/50.patch","issue_url":"https://api.github.com/repos/seznam/euphoria/issues/50","number":50,"state":"open","locked":false,"title":"#27 [euphoria-flink] Rewrite windowing to native implementation of StreamOperator","user":{"login":"vanekjar","id":6866668,"avatar_url":"https://avatars0.githubusercontent.com/u/6866668?v=3","gravatar_id":"","url":"https://api.github.com/users/vanekjar","html_url":"https://github.com/vanekjar","followers_url":"https://api.github.com/users/vanekjar/followers","following_url":"https://api.github.com/users/vanekjar/following{/other_user}","gists_url":"https://api.github.com/users/vanekjar/gists{/gist_id}","starred_url":"https://api.github.com/users/vanekjar/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vanekjar/subscriptions","organizations_url":"https://api.github.com/users/vanekjar/orgs","repos_url":"https://api.github.com/users/vanekjar/repos","events_url":"https://api.github.com/users/vanekjar/events{/privacy}","received_events_url":"https://api.github.com/users/vanekjar/received_events","type":"User","site_admin":false},"body":"Finally, my PR is ready.\r\n\r\nThe code for handling windows is still over-complicated, but I deleted more code than actually added which is always a good sign.\r\n\r\nInstead of using `ProcessFunction` I directly implemented custom `StreamOperator`. It turned out during implementation that `ProcessFunction` API is not so powerful for our use case - for example it's not possible to access the current `key` in callbacks.\r\n\r\nThere are a few pitfalls in the current PR that I am not very satisfied with:\r\n\r\n * Rewriting windowing haven't brought any performance benefit. Runtime is more or less the same as with previous implementation. On the other hand it now opens new opportunities to optimize - for example removing duplicate `timestamp` from `WindowedElement` flowing through the pipeline and use the built-in Flink timestamp instead. This will come in another PR soon.\r\n\r\n * I had to remove the functionality of flushing remaining windows in case of `bounded stream` end. So far all remaining windows were flushed to output in case of EOS. It proved that tracking of all existing windows had a huge performance drawback considering the registered window set must be persisted for fault-tolerance. I left that functionality only for unit testing (works for `TestFlinkExecutor` in local mode). Now it opens a discussion if we need this functionality of `bounded stream` in production.  \r\n\r\nThanks for the review.\r\n\r\n* Resolves: #33 \r\n* Closes: #27 ","created_at":"2017-03-19T11:21:26Z","updated_at":"2017-03-20T13:03:32Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3a16f60ede4e0e040377144b218e4d4357d80fbb","assignee":{"login":"xitep","id":8274125,"avatar_url":"https://avatars0.githubusercontent.com/u/8274125?v=3","gravatar_id":"","url":"https://api.github.com/users/xitep","html_url":"https://github.com/xitep","followers_url":"https://api.github.com/users/xitep/followers","following_url":"https://api.github.com/users/xitep/following{/other_user}","gists_url":"https://api.github.com/users/xitep/gists{/gist_id}","starred_url":"https://api.github.com/users/xitep/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xitep/subscriptions","organizations_url":"https://api.github.com/users/xitep/orgs","repos_url":"https://api.github.com/users/xitep/repos","events_url":"https://api.github.com/users/xitep/events{/privacy}","received_events_url":"https://api.github.com/users/xitep/received_events","type":"User","site_admin":false},"assignees":[{"login":"xitep","id":8274125,"avatar_url":"https://avatars0.githubusercontent.com/u/8274125?v=3","gravatar_id":"","url":"https://api.github.com/users/xitep","html_url":"https://github.com/xitep","followers_url":"https://api.github.com/users/xitep/followers","following_url":"https://api.github.com/users/xitep/following{/other_user}","gists_url":"https://api.github.com/users/xitep/gists{/gist_id}","starred_url":"https://api.github.com/users/xitep/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xitep/subscriptions","organizations_url":"https://api.github.com/users/xitep/orgs","repos_url":"https://api.github.com/users/xitep/repos","events_url":"https://api.github.com/users/xitep/events{/privacy}","received_events_url":"https://api.github.com/users/xitep/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/seznam/euphoria/pulls/50/commits","review_comments_url":"https://api.github.com/repos/seznam/euphoria/pulls/50/comments","review_comment_url":"https://api.github.com/repos/seznam/euphoria/pulls/comments{/number}","comments_url":"https://api.github.com/repos/seznam/euphoria/issues/50/comments","statuses_url":"https://api.github.com/repos/seznam/euphoria/statuses/c887c8e8c505c3aeb98d7da7d80398ed09f79107","head":{"label":"seznam:27/ProcessFunction","ref":"27/ProcessFunction","sha":"c887c8e8c505c3aeb98d7da7d80398ed09f79107","user":{"login":"seznam","id":192214,"avatar_url":"https://avatars0.githubusercontent.com/u/192214?v=3","gravatar_id":"","url":"https://api.github.com/users/seznam","html_url":"https://github.com/seznam","followers_url":"https://api.github.com/users/seznam/followers","following_url":"https://api.github.com/users/seznam/following{/other_user}","gists_url":"https://api.github.com/users/seznam/gists{/gist_id}","starred_url":"https://api.github.com/users/seznam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seznam/subscriptions","organizations_url":"https://api.github.com/users/seznam/orgs","repos_url":"https://api.github.com/users/seznam/repos","events_url":"https://api.github.com/users/seznam/events{/privacy}","received_events_url":"https://api.github.com/users/seznam/received_events","type":"Organization","site_admin":false},"repo":{"id":79946335,"name":"euphoria","full_name":"seznam/euphoria","owner":{"login":"seznam","id":192214,"avatar_url":"https://avatars0.githubusercontent.com/u/192214?v=3","gravatar_id":"","url":"https://api.github.com/users/seznam","html_url":"https://github.com/seznam","followers_url":"https://api.github.com/users/seznam/followers","following_url":"https://api.github.com/users/seznam/following{/other_user}","gists_url":"https://api.github.com/users/seznam/gists{/gist_id}","starred_url":"https://api.github.com/users/seznam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seznam/subscriptions","organizations_url":"https://api.github.com/users/seznam/orgs","repos_url":"https://api.github.com/users/seznam/repos","events_url":"https://api.github.com/users/seznam/events{/privacy}","received_events_url":"https://api.github.com/users/seznam/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/seznam/euphoria","description":"Euphoria is an open source Java API for creating unified big-data processing flows. It provides an engine independent programming model which can express both batch and stream transformations.","fork":false,"url":"https://api.github.com/repos/seznam/euphoria","forks_url":"https://api.github.com/repos/seznam/euphoria/forks","keys_url":"https://api.github.com/repos/seznam/euphoria/keys{/key_id}","collaborators_url":"https://api.github.com/repos/seznam/euphoria/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/seznam/euphoria/teams","hooks_url":"https://api.github.com/repos/seznam/euphoria/hooks","issue_events_url":"https://api.github.com/repos/seznam/euphoria/issues/events{/number}","events_url":"https://api.github.com/repos/seznam/euphoria/events","assignees_url":"https://api.github.com/repos/seznam/euphoria/assignees{/user}","branches_url":"https://api.github.com/repos/seznam/euphoria/branches{/branch}","tags_url":"https://api.github.com/repos/seznam/euphoria/tags","blobs_url":"https://api.github.com/repos/seznam/euphoria/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/seznam/euphoria/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/seznam/euphoria/git/refs{/sha}","trees_url":"https://api.github.com/repos/seznam/euphoria/git/trees{/sha}","statuses_url":"https://api.github.com/repos/seznam/euphoria/statuses/{sha}","languages_url":"https://api.github.com/repos/seznam/euphoria/languages","stargazers_url":"https://api.github.com/repos/seznam/euphoria/stargazers","contributors_url":"https://api.github.com/repos/seznam/euphoria/contributors","subscribers_url":"https://api.github.com/repos/seznam/euphoria/subscribers","subscription_url":"https://api.github.com/repos/seznam/euphoria/subscription","commits_url":"https://api.github.com/repos/seznam/euphoria/commits{/sha}","git_commits_url":"https://api.github.com/repos/seznam/euphoria/git/commits{/sha}","comments_url":"https://api.github.com/repos/seznam/euphoria/comments{/number}","issue_comment_url":"https://api.github.com/repos/seznam/euphoria/issues/comments{/number}","contents_url":"https://api.github.com/repos/seznam/euphoria/contents/{+path}","compare_url":"https://api.github.com/repos/seznam/euphoria/compare/{base}...{head}","merges_url":"https://api.github.com/repos/seznam/euphoria/merges","archive_url":"https://api.github.com/repos/seznam/euphoria/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/seznam/euphoria/downloads","issues_url":"https://api.github.com/repos/seznam/euphoria/issues{/number}","pulls_url":"https://api.github.com/repos/seznam/euphoria/pulls{/number}","milestones_url":"https://api.github.com/repos/seznam/euphoria/milestones{/number}","notifications_url":"https://api.github.com/repos/seznam/euphoria/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/seznam/euphoria/labels{/name}","releases_url":"https://api.github.com/repos/seznam/euphoria/releases{/id}","deployments_url":"https://api.github.com/repos/seznam/euphoria/deployments","created_at":"2017-01-24T19:32:26Z","updated_at":"2017-03-20T06:45:07Z","pushed_at":"2017-03-20T11:50:36Z","git_url":"git://github.com/seznam/euphoria.git","ssh_url":"git@github.com:seznam/euphoria.git","clone_url":"https://github.com/seznam/euphoria.git","svn_url":"https://github.com/seznam/euphoria","homepage":"","size":1877,"stargazers_count":15,"watchers_count":15,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":27,"forks":0,"open_issues":27,"watchers":15,"default_branch":"master"}},"base":{"label":"seznam:master","ref":"master","sha":"743389b2991982d12acfbb630588764c48f978fa","user":{"login":"seznam","id":192214,"avatar_url":"https://avatars0.githubusercontent.com/u/192214?v=3","gravatar_id":"","url":"https://api.github.com/users/seznam","html_url":"https://github.com/seznam","followers_url":"https://api.github.com/users/seznam/followers","following_url":"https://api.github.com/users/seznam/following{/other_user}","gists_url":"https://api.github.com/users/seznam/gists{/gist_id}","starred_url":"https://api.github.com/users/seznam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seznam/subscriptions","organizations_url":"https://api.github.com/users/seznam/orgs","repos_url":"https://api.github.com/users/seznam/repos","events_url":"https://api.github.com/users/seznam/events{/privacy}","received_events_url":"https://api.github.com/users/seznam/received_events","type":"Organization","site_admin":false},"repo":{"id":79946335,"name":"euphoria","full_name":"seznam/euphoria","owner":{"login":"seznam","id":192214,"avatar_url":"https://avatars0.githubusercontent.com/u/192214?v=3","gravatar_id":"","url":"https://api.github.com/users/seznam","html_url":"https://github.com/seznam","followers_url":"https://api.github.com/users/seznam/followers","following_url":"https://api.github.com/users/seznam/following{/other_user}","gists_url":"https://api.github.com/users/seznam/gists{/gist_id}","starred_url":"https://api.github.com/users/seznam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seznam/subscriptions","organizations_url":"https://api.github.com/users/seznam/orgs","repos_url":"https://api.github.com/users/seznam/repos","events_url":"https://api.github.com/users/seznam/events{/privacy}","received_events_url":"https://api.github.com/users/seznam/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/seznam/euphoria","description":"Euphoria is an open source Java API for creating unified big-data processing flows. It provides an engine independent programming model which can express both batch and stream transformations.","fork":false,"url":"https://api.github.com/repos/seznam/euphoria","forks_url":"https://api.github.com/repos/seznam/euphoria/forks","keys_url":"https://api.github.com/repos/seznam/euphoria/keys{/key_id}","collaborators_url":"https://api.github.com/repos/seznam/euphoria/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/seznam/euphoria/teams","hooks_url":"https://api.github.com/repos/seznam/euphoria/hooks","issue_events_url":"https://api.github.com/repos/seznam/euphoria/issues/events{/number}","events_url":"https://api.github.com/repos/seznam/euphoria/events","assignees_url":"https://api.github.com/repos/seznam/euphoria/assignees{/user}","branches_url":"https://api.github.com/repos/seznam/euphoria/branches{/branch}","tags_url":"https://api.github.com/repos/seznam/euphoria/tags","blobs_url":"https://api.github.com/repos/seznam/euphoria/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/seznam/euphoria/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/seznam/euphoria/git/refs{/sha}","trees_url":"https://api.github.com/repos/seznam/euphoria/git/trees{/sha}","statuses_url":"https://api.github.com/repos/seznam/euphoria/statuses/{sha}","languages_url":"https://api.github.com/repos/seznam/euphoria/languages","stargazers_url":"https://api.github.com/repos/seznam/euphoria/stargazers","contributors_url":"https://api.github.com/repos/seznam/euphoria/contributors","subscribers_url":"https://api.github.com/repos/seznam/euphoria/subscribers","subscription_url":"https://api.github.com/repos/seznam/euphoria/subscription","commits_url":"https://api.github.com/repos/seznam/euphoria/commits{/sha}","git_commits_url":"https://api.github.com/repos/seznam/euphoria/git/commits{/sha}","comments_url":"https://api.github.com/repos/seznam/euphoria/comments{/number}","issue_comment_url":"https://api.github.com/repos/seznam/euphoria/issues/comments{/number}","contents_url":"https://api.github.com/repos/seznam/euphoria/contents/{+path}","compare_url":"https://api.github.com/repos/seznam/euphoria/compare/{base}...{head}","merges_url":"https://api.github.com/repos/seznam/euphoria/merges","archive_url":"https://api.github.com/repos/seznam/euphoria/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/seznam/euphoria/downloads","issues_url":"https://api.github.com/repos/seznam/euphoria/issues{/number}","pulls_url":"https://api.github.com/repos/seznam/euphoria/pulls{/number}","milestones_url":"https://api.github.com/repos/seznam/euphoria/milestones{/number}","notifications_url":"https://api.github.com/repos/seznam/euphoria/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/seznam/euphoria/labels{/name}","releases_url":"https://api.github.com/repos/seznam/euphoria/releases{/id}","deployments_url":"https://api.github.com/repos/seznam/euphoria/deployments","created_at":"2017-01-24T19:32:26Z","updated_at":"2017-03-20T06:45:07Z","pushed_at":"2017-03-20T11:50:36Z","git_url":"git://github.com/seznam/euphoria.git","ssh_url":"git@github.com:seznam/euphoria.git","clone_url":"https://github.com/seznam/euphoria.git","svn_url":"https://github.com/seznam/euphoria","homepage":"","size":1877,"stargazers_count":15,"watchers_count":15,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":27,"forks":0,"open_issues":27,"watchers":15,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/seznam/euphoria/pulls/50"},"html":{"href":"https://github.com/seznam/euphoria/pull/50"},"issue":{"href":"https://api.github.com/repos/seznam/euphoria/issues/50"},"comments":{"href":"https://api.github.com/repos/seznam/euphoria/issues/50/comments"},"review_comments":{"href":"https://api.github.com/repos/seznam/euphoria/pulls/50/comments"},"review_comment":{"href":"https://api.github.com/repos/seznam/euphoria/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/seznam/euphoria/pulls/50/commits"},"statuses":{"href":"https://api.github.com/repos/seznam/euphoria/statuses/c887c8e8c505c3aeb98d7da7d80398ed09f79107"}}}}
{"actor":{"display_login":"xitep"}}