{"action":"created","issue":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/16807","repository_url":"https://api.github.com/repos/pandas-dev/pandas","labels_url":"https://api.github.com/repos/pandas-dev/pandas/issues/16807/labels{/name}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/16807/comments","events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/16807/events","html_url":"https://github.com/pandas-dev/pandas/issues/16807","id":239880187,"number":16807,"title":"BUG: SparseDataFrame constructor issues","user":{"login":"kbattocchi","id":3801192,"avatar_url":"https://avatars0.githubusercontent.com/u/3801192?v=4","gravatar_id":"","url":"https://api.github.com/users/kbattocchi","html_url":"https://github.com/kbattocchi","followers_url":"https://api.github.com/users/kbattocchi/followers","following_url":"https://api.github.com/users/kbattocchi/following{/other_user}","gists_url":"https://api.github.com/users/kbattocchi/gists{/gist_id}","starred_url":"https://api.github.com/users/kbattocchi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kbattocchi/subscriptions","organizations_url":"https://api.github.com/users/kbattocchi/orgs","repos_url":"https://api.github.com/users/kbattocchi/repos","events_url":"https://api.github.com/users/kbattocchi/events{/privacy}","received_events_url":"https://api.github.com/users/kbattocchi/received_events","type":"User","site_admin":false},"labels":[{"id":76811,"url":"https://api.github.com/repos/pandas-dev/pandas/labels/Bug","name":"Bug","color":"e10c02","default":false},{"id":195647922,"url":"https://api.github.com/repos/pandas-dev/pandas/labels/Difficulty%20Intermediate","name":"Difficulty Intermediate","color":"fbca04","default":false},{"id":195648017,"url":"https://api.github.com/repos/pandas-dev/pandas/labels/Effort%20Medium","name":"Effort Medium","color":"006b75","default":false},{"id":2822342,"url":"https://api.github.com/repos/pandas-dev/pandas/labels/Missing-data","name":"Missing-data","color":"d7e102","default":false},{"id":49182326,"url":"https://api.github.com/repos/pandas-dev/pandas/labels/Sparse","name":"Sparse","color":"009800","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/pandas-dev/pandas/milestones/32","html_url":"https://github.com/pandas-dev/pandas/milestone/32","labels_url":"https://api.github.com/repos/pandas-dev/pandas/milestones/32/labels","id":933188,"number":32,"title":"Next Major Release","description":"","creator":{"login":"jreback","id":953992,"avatar_url":"https://avatars2.githubusercontent.com/u/953992?v=4","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":1301,"closed_issues":237,"state":"open","created_at":"2015-01-13T10:53:19Z","updated_at":"2017-12-10T15:49:00Z","due_on":"2018-08-31T07:00:00Z","closed_at":null},"comments":2,"created_at":"2017-06-30T20:23:51Z","updated_at":"2017-12-10T16:01:59Z","closed_at":null,"author_association":"NONE","body":"I've noticed a bunch of issues with SparseDataFrame's constructor, particularly around support for non-`float64` datatypes:\r\n\r\n- [ ]  when `data` is not provided, the default_fill_value is not used to fill the data (noted in passing #5470):\r\n```python\r\n>>> pd.SparseDataFrame(columns=list(\"ab\"), index=range(4), default_fill_value=0.0)\r\n    a   b\r\n0 NaN NaN\r\n1 NaN NaN\r\n2 NaN NaN\r\n3 NaN NaN\r\n```\r\n~~- [ ] when `data` is not provided and the `dtype` is set to `int64`, the constructor call fails~~\r\n```python\r\n>>> pd.SparseDataFrame(columns=list(\"ab\"), index=range(4), dtype=np.int64)\r\nValueError: cannot convert float NaN to integer\r\n```\r\n\r\n- [ ] when `data` is an empty sparse matrix with `int64` entries and no `dtype` is specified, the type of the SparseDataFrame is still `float64`. Worse, even when `dtype` is specified, it still doesn't work.\r\n```python\r\n>>> pd.SparseDataFrame(coo_matrix((4,3), dtype=np.int64))\r\n    0   1   2\r\n0 NaN NaN NaN\r\n1 NaN NaN NaN\r\n2 NaN NaN NaN\r\n3 NaN NaN NaN\r\n>>> pd.SparseDataFrame(coo_matrix((4,3), dtype=np.int64), dtype=np.int64)\r\n    0   1   2\r\n0 NaN NaN NaN\r\n1 NaN NaN NaN\r\n2 NaN NaN NaN\r\n3 NaN NaN NaN\r\n```\r\n- [ ] probably due to the same issue, given a sparse matrix with underlying elements of type `int64`, if any of the values in a column is `0` then that column will be treated as `float64`s:\r\n```python\r\n>>> pd.SparseDataFrame(coo_matrix(np.arange(12).reshape(4,3), dtype=np.int64))\r\n     0   1   2\r\n0  NaN   1   2\r\n1  3.0   4   5\r\n2  6.0   7   8\r\n3  9.0  10  11\r\n```\r\n(as long as at least one value is non-zero, then this can be worked around by setting `default_fill_value`; however, if all of the values in one column are zero then this no longer works).\r\n\r\n#### Output of ``pd.show_versions()``\r\n\r\n<details>\r\n\r\nINSTALLED VERSIONS\r\ncommit: None\r\npython: 3.5.2.final.0\r\npython-bits: 64\r\nOS: Windows\r\nOS-release: 10\r\nmachine: AMD64\r\nprocessor: Intel64 Family 6 Model 63 Stepping 2, GenuineIntel\r\nbyteorder: little\r\nLC_ALL: None\r\nLANG: None\r\nLOCALE: None.None\r\n\r\npandas: 0.20.2\r\npytest: 2.9.2\r\npip: 8.1.2\r\nsetuptools: 27.2.0\r\nCython: 0.25.2\r\nnumpy: 1.13.0\r\nscipy: 0.18.1\r\nxarray: None\r\nIPython: 5.1.0\r\nsphinx: 1.4.6\r\npatsy: 0.4.1\r\ndateutil: 2.5.3\r\npytz: 2016.6.1\r\nblosc: None\r\nbottleneck: 1.1.0\r\ntables: 3.2.2\r\nnumexpr: 2.6.1\r\nfeather: None\r\nmatplotlib: 2.0.2\r\nopenpyxl: 2.3.2\r\nxlrd: 1.0.0\r\nxlwt: 1.1.2\r\nxlsxwriter: 0.9.3\r\nlxml: 3.6.4\r\nbs4: 4.5.1\r\nhtml5lib: None\r\nsqlalchemy: 1.0.13\r\npymysql: None\r\npsycopg2: None\r\njinja2: 2.8\r\ns3fs: None\r\npandas_gbq: None\r\npandas_datareader: 0.4.0\r\n\r\n</details>\r\n"},"comment":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/comments/350558269","html_url":"https://github.com/pandas-dev/pandas/issues/16807#issuecomment-350558269","issue_url":"https://api.github.com/repos/pandas-dev/pandas/issues/16807","id":350558269,"user":{"login":"jreback","id":953992,"avatar_url":"https://avatars2.githubusercontent.com/u/953992?v=4","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"created_at":"2017-12-10T16:01:59Z","updated_at":"2017-12-10T16:01:59Z","author_association":"CONTRIBUTOR","body":"yes,\r\n\r\nthough like to have some tests that assert the fail for 2-4 as well."}}
{"actor":{"display_login":"jreback"}}