{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/16579","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/16579/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/16579/comments","events_url":"https://api.github.com/repos/apache/spark/issues/16579/events","html_url":"https://github.com/apache/spark/pull/16579","id":200705754,"number":16579,"title":"[SPARK-19218][SQL] Fix SET command to show a result correctly and in a sorted order","user":{"login":"dongjoon-hyun","id":9700541,"avatar_url":"https://avatars.githubusercontent.com/u/9700541?v=3","gravatar_id":"","url":"https://api.github.com/users/dongjoon-hyun","html_url":"https://github.com/dongjoon-hyun","followers_url":"https://api.github.com/users/dongjoon-hyun/followers","following_url":"https://api.github.com/users/dongjoon-hyun/following{/other_user}","gists_url":"https://api.github.com/users/dongjoon-hyun/gists{/gist_id}","starred_url":"https://api.github.com/users/dongjoon-hyun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dongjoon-hyun/subscriptions","organizations_url":"https://api.github.com/users/dongjoon-hyun/orgs","repos_url":"https://api.github.com/users/dongjoon-hyun/repos","events_url":"https://api.github.com/users/dongjoon-hyun/events{/privacy}","received_events_url":"https://api.github.com/users/dongjoon-hyun/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":24,"created_at":"2017-01-13T19:06:34Z","updated_at":"2017-01-22T01:09:50Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/16579","html_url":"https://github.com/apache/spark/pull/16579","diff_url":"https://github.com/apache/spark/pull/16579.diff","patch_url":"https://github.com/apache/spark/pull/16579.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR aims to fix the following two things.\r\n\r\n1. `sql(\"SET -v\").collect()` or `sql(\"SET -v\").show()` raises the following exceptions for String configuration with default value, `null`. For the test, please see [Jenkins result](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/71539/testReport/) and https://github.com/apache/spark/commit/60953bf1f1ba144e709fdae3903a390ff9479fd0 in #16624 .\r\n\r\n```\r\nsbt.ForkMain$ForkError: java.lang.RuntimeException: Error while decoding: java.lang.NullPointerException\r\ncreateexternalrow(input[0, string, false].toString, input[1, string, false].toString, input[2, string, false].toString, StructField(key,StringType,false), StructField(value,StringType,false), StructField(meaning,StringType,false))\r\n:- input[0, string, false].toString\r\n:  +- input[0, string, false]\r\n:- input[1, string, false].toString\r\n:  +- input[1, string, false]\r\n+- input[2, string, false].toString\r\n   +- input[2, string, false]\r\n```\r\n\r\n2. `SET` and `SET -v` commands show unsorted result. We had better show a sorted result for UX.\r\n   Also, this is compatible with Hive.\r\n\r\n**BEFORE**\r\n```\r\nscala> sql(\"set\").show(false)\r\n...\r\n|spark.driver.host              |10.22.16.140                                                                                                                                 |\r\n|spark.driver.port              |63893                                                                                                                                        |\r\n|spark.repl.class.uri           |spark://10.22.16.140:63893/classes                                                                                                           |\r\n...\r\n|spark.app.name                 |Spark shell                                                                                                                                  |\r\n|spark.driver.memory            |4G                                                                                                                                           |\r\n|spark.executor.id              |driver                                                                                                                                       |\r\n|spark.submit.deployMode        |client                                                                                                                                       |\r\n|spark.master                   |local[*]                                                                                                                                     |\r\n|spark.home                     |/Users/dhyun/spark                                                                                                                           |\r\n|spark.sql.catalogImplementation|hive                                                                                                                                         |\r\n|spark.app.id                   |local-1484333618945                                                                                                                          |\r\n```\r\n\r\n**AFTER**\r\n\r\n```\r\nscala> sql(\"set\").show(false)\r\n...\r\n|spark.app.id                   |local-1484333925649                                                                                                                          |\r\n|spark.app.name                 |Spark shell                                                                                                                                  |\r\n|spark.driver.host              |10.22.16.140                                                                                                                                 |\r\n|spark.driver.memory            |4G                                                                                                                                           |\r\n|spark.driver.port              |64994                                                                                                                                        |\r\n|spark.executor.id              |driver                                                                                                                                       |\r\n|spark.jars                     |                                                                                                                                             |\r\n|spark.master                   |local[*]                                                                                                                                     |\r\n|spark.repl.class.uri           |spark://10.22.16.140:64994/classes                                                                                                           |\r\n|spark.sql.catalogImplementation|hive                                                                                                                                         |\r\n|spark.submit.deployMode        |client                                                                                                                                       |\r\n```\r\n\r\n## How was this patch tested?\r\n\r\nJenkins with a new test case."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/274299835","html_url":"https://github.com/apache/spark/pull/16579#issuecomment-274299835","issue_url":"https://api.github.com/repos/apache/spark/issues/16579","id":274299835,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2017-01-22T01:09:50Z","updated_at":"2017-01-22T01:09:50Z","body":"Test PASSed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71777/\nTest PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}