{"action":"closed","issue":{"url":"https://api.github.com/repos/tidyverse/dplyr/issues/2727","repository_url":"https://api.github.com/repos/tidyverse/dplyr","labels_url":"https://api.github.com/repos/tidyverse/dplyr/issues/2727/labels{/name}","comments_url":"https://api.github.com/repos/tidyverse/dplyr/issues/2727/comments","events_url":"https://api.github.com/repos/tidyverse/dplyr/issues/2727/events","html_url":"https://github.com/tidyverse/dplyr/issues/2727","id":225696876,"number":2727,"title":"Type mismatch error for filter function with dplyr over a spark 2.0.2 data frame","user":{"login":"paramiyer","id":13696044,"avatar_url":"https://avatars0.githubusercontent.com/u/13696044?v=3","gravatar_id":"","url":"https://api.github.com/users/paramiyer","html_url":"https://github.com/paramiyer","followers_url":"https://api.github.com/users/paramiyer/followers","following_url":"https://api.github.com/users/paramiyer/following{/other_user}","gists_url":"https://api.github.com/users/paramiyer/gists{/gist_id}","starred_url":"https://api.github.com/users/paramiyer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paramiyer/subscriptions","organizations_url":"https://api.github.com/users/paramiyer/orgs","repos_url":"https://api.github.com/users/paramiyer/repos","events_url":"https://api.github.com/users/paramiyer/events{/privacy}","received_events_url":"https://api.github.com/users/paramiyer/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-05-02T13:35:42Z","updated_at":"2017-05-02T13:53:04Z","closed_at":"2017-05-02T13:53:04Z","body":"I am currently working on Rstudio over a rhel cluster. I use spark 2.0.2 over a yarn client & have installed the following versions of sparklyr & dplyr\r\n\r\nsparklyr_0.5.4 ; dplyr_0.5.0\r\n\r\nA simple test on the following lines results in error\r\n\r\n```\r\ndata = copy_to(sc, iris)\r\nfilter(data , Sepal_Length >5)\r\n\r\nError in filter(data, Sepal_Length > 5) : \r\n(list) object cannot be coerced to type 'double'\r\n```\r\nI checked with the read & all looks fine\r\n\r\n```\r\nhead(data)\r\nSource:   query [6 x 5]\r\nDatabase: spark connection master=yarn-client app=sparklyr local=FALSE\r\n\r\nSepal_Length Sepal_Width Petal_Length Petal_Width Species\r\n     <dbl>       <dbl>        <dbl>       <dbl>   <chr>\r\n1    5.1         3.5          1.4         0.2    setosa\r\n2    4.9         3.0          1.4         0.2    setosa\r\n3    4.7         3.2          1.3         0.2    setosa\r\n4    4.6         3.1          1.5         0.2    setosa\r\n5    5.0         3.6          1.4         0.2    setosa\r\n6    5.4         3.9          1.7         0.4    setosa\r\n```\r\nis this a known bug & are there known fixes for this?\r\nShould I be reverting to an older spark version?"}}
{"actor":{"display_login":"hadley"}}