{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/17540","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/17540/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/17540/comments","events_url":"https://api.github.com/repos/apache/spark/issues/17540/events","html_url":"https://github.com/apache/spark/pull/17540","id":219659267,"number":17540,"title":"[SPARK-20213][SQL][UI] Fix DataFrameWriter operations in SQL UI tab.","user":{"login":"rdblue","id":87915,"avatar_url":"https://avatars2.githubusercontent.com/u/87915?v=3","gravatar_id":"","url":"https://api.github.com/users/rdblue","html_url":"https://github.com/rdblue","followers_url":"https://api.github.com/users/rdblue/followers","following_url":"https://api.github.com/users/rdblue/following{/other_user}","gists_url":"https://api.github.com/users/rdblue/gists{/gist_id}","starred_url":"https://api.github.com/users/rdblue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rdblue/subscriptions","organizations_url":"https://api.github.com/users/rdblue/orgs","repos_url":"https://api.github.com/users/rdblue/repos","events_url":"https://api.github.com/users/rdblue/events{/privacy}","received_events_url":"https://api.github.com/users/rdblue/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":28,"created_at":"2017-04-05T17:33:00Z","updated_at":"2017-04-18T00:51:33Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/17540","html_url":"https://github.com/apache/spark/pull/17540","diff_url":"https://github.com/apache/spark/pull/17540.diff","patch_url":"https://github.com/apache/spark/pull/17540.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nWraps `DataFrameWriter` operations in `SQLExecution.withNewExecutionId` so that `SparkListenerSQLExecutionStart` and `SparkListenerSQLExecutionEnd` are sent and the query shows up in the SQL tab of the UI.\r\n\r\n## How was this patch tested?\r\n\r\nTested by hand that `insertInto` results in queries in the SQL tab.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/294639152","html_url":"https://github.com/apache/spark/pull/17540#issuecomment-294639152","issue_url":"https://api.github.com/repos/apache/spark/issues/17540","id":294639152,"user":{"login":"cloud-fan","id":3182036,"avatar_url":"https://avatars0.githubusercontent.com/u/3182036?v=3","gravatar_id":"","url":"https://api.github.com/users/cloud-fan","html_url":"https://github.com/cloud-fan","followers_url":"https://api.github.com/users/cloud-fan/followers","following_url":"https://api.github.com/users/cloud-fan/following{/other_user}","gists_url":"https://api.github.com/users/cloud-fan/gists{/gist_id}","starred_url":"https://api.github.com/users/cloud-fan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cloud-fan/subscriptions","organizations_url":"https://api.github.com/users/cloud-fan/orgs","repos_url":"https://api.github.com/users/cloud-fan/repos","events_url":"https://api.github.com/users/cloud-fan/events{/privacy}","received_events_url":"https://api.github.com/users/cloud-fan/received_events","type":"User","site_admin":false},"created_at":"2017-04-18T00:51:33Z","updated_at":"2017-04-18T00:51:33Z","body":"@rdblue You need to click the \"consle output\" tag on the left to see the pyspark failures. Here it is\r\n```\r\nERROR: test_stream_exception (pyspark.sql.tests.SQLTests)\r\n----------------------------------------------------------------------\r\nTraceback (most recent call last):\r\n  File \"/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/tests.py\", line 1438, in test_stream_exception\r\n    sq.processAllAvailable()\r\n  File \"/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/streaming.py\", line 149, in processAllAvailable\r\n    return self._jsq.processAllAvailable()\r\n  File \"/home/jenkins/workspace/SparkPullRequestBuilder/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py\", line 1133, in __call__\r\n    answer, self.gateway_client, self.target_id, self.name)\r\n  File \"/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/sql/utils.py\", line 75, in deco\r\n    raise StreamingQueryException(s.split(': ', 1)[1], stackTrace)\r\nStreamingQueryException: u'spark.sql.execution.id is already set\\n=== Streaming Query ===\\nIdentifier: query_explain [id = 568c91fc-0d6d-4fc8-a7d7-db94b6603c8a, runId = e646eee0-22c5-4620-9987-8a0d1c682476]\\nCurrent Committed Offsets: {}\\nCurrent Available Offsets: {FileStreamSource[file:/home/jenkins/workspace/SparkPullRequestBuilder/python/test_support/sql/streaming]: {\"logOffset\":0}}\\n\\nCurrent State: ACTIVE\\nThread State: RUNNABLE\\n\\nLogical Plan:\\nFileStreamSource[file:/home/jenkins/workspace/SparkPullRequestBuilder/python/test_support/sql/streaming]'\r\n```"}}
{"actor":{"display_login":"cloud-fan"}}