{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/19495","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/19495/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/19495/comments","events_url":"https://api.github.com/repos/apache/spark/issues/19495/events","html_url":"https://github.com/apache/spark/pull/19495","id":265449006,"number":19495,"title":"[SPARK-22278][SS] Expose current event time watermark and current processing time in GroupState","user":{"login":"tdas","id":663212,"avatar_url":"https://avatars1.githubusercontent.com/u/663212?v=4","gravatar_id":"","url":"https://api.github.com/users/tdas","html_url":"https://github.com/tdas","followers_url":"https://api.github.com/users/tdas/followers","following_url":"https://api.github.com/users/tdas/following{/other_user}","gists_url":"https://api.github.com/users/tdas/gists{/gist_id}","starred_url":"https://api.github.com/users/tdas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tdas/subscriptions","organizations_url":"https://api.github.com/users/tdas/orgs","repos_url":"https://api.github.com/users/tdas/repos","events_url":"https://api.github.com/users/tdas/events{/privacy}","received_events_url":"https://api.github.com/users/tdas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2017-10-14T00:18:54Z","updated_at":"2017-10-17T21:58:23Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/19495","html_url":"https://github.com/apache/spark/pull/19495","diff_url":"https://github.com/apache/spark/pull/19495.diff","patch_url":"https://github.com/apache/spark/pull/19495.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nComplex state-updating and/or timeout-handling logic in mapGroupsWithState functions may require taking decisions based on the current event-time watermark and/or processing time. Currently, you can use the SQL function `current_timestamp` to get the current processing time, but it needs to be passed inserted in every row with a select, and then passed through the encoder, which isn't efficient. Furthermore, there is no way to get the current watermark.\r\n\r\nThis PR exposes both of them through the GroupState API. \r\nAdditionally, it also cleans up some of the GroupState docs. \r\n\r\n## How was this patch tested?\r\n\r\nNew unit tests"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/337385690","html_url":"https://github.com/apache/spark/pull/19495#issuecomment-337385690","issue_url":"https://api.github.com/repos/apache/spark/issues/19495","id":337385690,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2017-10-17T21:58:23Z","updated_at":"2017-10-17T21:58:23Z","author_association":"NONE","body":"**[Test build #82859 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82859/testReport)** for PR 19495 at commit [`037e036`](https://github.com/apache/spark/commit/037e03658b2102cb1025019b2a22535a5ecb3f50)."}}
{"actor":{"display_login":"SparkQA"}}