{"action":"created","issue":{"url":"https://api.github.com/repos/airbnb/superset/issues/2008","repository_url":"https://api.github.com/repos/airbnb/superset","labels_url":"https://api.github.com/repos/airbnb/superset/issues/2008/labels{/name}","comments_url":"https://api.github.com/repos/airbnb/superset/issues/2008/comments","events_url":"https://api.github.com/repos/airbnb/superset/issues/2008/events","html_url":"https://github.com/airbnb/superset/issues/2008","id":202092408,"number":2008,"title":"the column char case make key error when creating  pivot_table","user":{"login":"jiamo","id":732751,"avatar_url":"https://avatars.githubusercontent.com/u/732751?v=3","gravatar_id":"","url":"https://api.github.com/users/jiamo","html_url":"https://github.com/jiamo","followers_url":"https://api.github.com/users/jiamo/followers","following_url":"https://api.github.com/users/jiamo/following{/other_user}","gists_url":"https://api.github.com/users/jiamo/gists{/gist_id}","starred_url":"https://api.github.com/users/jiamo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jiamo/subscriptions","organizations_url":"https://api.github.com/users/jiamo/orgs","repos_url":"https://api.github.com/users/jiamo/repos","events_url":"https://api.github.com/users/jiamo/events{/privacy}","received_events_url":"https://api.github.com/users/jiamo/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-01-20T09:28:12Z","updated_at":"2017-02-13T14:59:54Z","closed_at":"2017-02-08T15:07:57Z","body":"### Superset version\r\nmaster branch\r\n\r\n### error\r\nthe origin datafame get from sql:\r\n```\r\n2017-01-20 00:54:40,148:WARNING:Kylin:Transactional rollback is not supported\r\n    AFFILIATE_ID  SUM__REVENUE_POINTS\r\n0            1804                116578\r\n1            1411                1114083\r\n\r\n```\r\nthe traceback:\r\n```\r\n[224 rows x 2 columns]\r\n2017-01-20 00:56:08,701:ERROR:root:\"['sum__revenue_points'] not in index\"\r\nTraceback (most recent call last):\r\n  File \"/home/ubuntu/yxy_1/superset/superset/views.py\", line 1406, in explore_json\r\n    payload = viz_obj.get_payload()\r\n  File \"/home/ubuntu/yxy_1/superset/superset/viz.py\", line 360, in get_payload\r\n    data = self.get_data()\r\n  File \"/home/ubuntu/yxy_1/superset/superset/viz.py\", line 593, in get_data\r\n    margins=True,\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/tools/pivot.py\", line 158, in pivot_table\r\n    margins_name=margins_name)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/tools/pivot.py\", line 183, in _add_margins\r\n    grand_margin = _compute_grand_margin(data, values, aggfunc, margins_name)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/tools/pivot.py\", line 244, in _compute_grand_margin\r\n    for k, v in data[values].iteritems():\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/frame.py\", line 1991, in __getitem__\r\n    return self._getitem_array(key)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/frame.py\", line 2035, in _getitem_array\r\n    indexer = self.ix._convert_to_indexer(key, axis=1)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/indexing.py\", line 1214, in _convert_to_indexer\r\n    raise KeyError('%s not in index' % objarr[mask])\r\nKeyError: \"['sum__revenue_points'] not in index\"\r\n \r\n```\r\n\r\nSo, I need change column to `SUM__REVENUE_POINTS` to make it works.  \r\nMaybe we should use low-case characters in our backend server when handing dataframe.\r\n\r\nI also find another place have this issure too: In time seirse view\r\n\r\nthe origin dataframe is \r\n\r\n```\r\n  __TIMESTAMP  SUM_REVENUE_POINTS  SUM_REVENUE_POINTS\r\n0  2017-01-13             9535297              955297\r\n```\r\n\r\nbut there is some place to using this lower space:\r\n\r\n```\r\nTraceback (most recent call last):\r\n  File \"/home/ubuntu/yxy_1/superset/superset/views.py\", line 1406, in explore_json\r\n    payload = viz_obj.get_payload()\r\n  File \"/home/ubuntu/yxy_1/superset/superset/viz.py\", line 360, in get_payload\r\n    data = self.get_data()\r\n  File \"/home/ubuntu/yxy_1/superset/superset/viz.py\", line 1387, in get_data\r\n    df = self.get_df()\r\n  File \"/home/ubuntu/yxy_1/superset/superset/viz.py\", line 1329, in get_df\r\n    values=metrics)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/tools/pivot.py\", line 120, in pivot_table\r\n    grouped = data.groupby(keys)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/generic.py\", line 3778, in groupby\r\n    **kwargs)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/groupby.py\", line 1427, in groupby\r\n    return klass(obj, by, **kwds)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/groupby.py\", line 354, in __init__\r\n    mutated=self.mutated)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/groupby.py\", line 2383, in _get_grouper\r\n    in_axis, name, gpr = True, gpr, obj[gpr]\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/frame.py\", line 1997, in __getitem__\r\n    return self._getitem_column(key)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/frame.py\", line 2009, in _getitem_column\r\n    result = result[key]\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/frame.py\", line 1997, in __getitem__\r\n    return self._getitem_column(key)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/frame.py\", line 2004, in _getitem_column\r\n    return self._get_item_cache(key)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/generic.py\", line 1350, in _get_item_cache\r\n    values = self._data.get(item)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/core/internals.py\", line 3290, in get\r\n    loc = self.items.get_loc(item)\r\n  File \"/home/ubuntu/.pyenv/versions/superset/lib/python3.5/site-packages/pandas/indexes/base.py\", line 1947, in get_loc\r\n    return self._engine.get_loc(self._maybe_cast_indexer(key))\r\n  File \"pandas/index.pyx\", line 137, in pandas.index.IndexEngine.get_loc (pandas/index.c:4154)\r\n  File \"pandas/index.pyx\", line 159, in pandas.index.IndexEngine.get_loc (pandas/index.c:4018)\r\n  File \"pandas/hashtable.pyx\", line 675, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12368)\r\n  File \"pandas/hashtable.pyx\", line 683, in pandas.hashtable.PyObjectHashTable.get_item (pandas/hashtable.c:12322)\r\nKeyError: '__timestamp'\r\n\r\n```\r\nHowever, this time I don't have place to change '__timestamp' to `__TIMESTAMP`.\r\nI also find `DTTM_ALIAS = '__timestamp'`. So I can't get understand why the  dataframe created by superset have big case column `__TIMESTAMP`."},"comment":{"url":"https://api.github.com/repos/airbnb/superset/issues/comments/279415442","html_url":"https://github.com/airbnb/superset/issues/2008#issuecomment-279415442","issue_url":"https://api.github.com/repos/airbnb/superset/issues/2008","id":279415442,"user":{"login":"jiamo","id":732751,"avatar_url":"https://avatars.githubusercontent.com/u/732751?v=3","gravatar_id":"","url":"https://api.github.com/users/jiamo","html_url":"https://github.com/jiamo","followers_url":"https://api.github.com/users/jiamo/followers","following_url":"https://api.github.com/users/jiamo/following{/other_user}","gists_url":"https://api.github.com/users/jiamo/gists{/gist_id}","starred_url":"https://api.github.com/users/jiamo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jiamo/subscriptions","organizations_url":"https://api.github.com/users/jiamo/orgs","repos_url":"https://api.github.com/users/jiamo/repos","events_url":"https://api.github.com/users/jiamo/events{/privacy}","received_events_url":"https://api.github.com/users/jiamo/received_events","type":"User","site_admin":false},"created_at":"2017-02-13T14:59:54Z","updated_at":"2017-02-13T14:59:54Z","body":"May be it is not fit for you:  here is\r\n\r\n```\r\n     def to_series(self, df, classed='', title_suffix=''):\r\n@@ -1230,8 +1248,11 @@ class NVD3TimeSeriesViz(NVD3Viz):\r\n             else:\r\n                 cols.append(col)\r\n         df.columns = cols\r\n+        # add df\r\n+        df.index = pd.to_datetime(df.index, utc=False)\r\n         series = df.to_dict('series')\r\n\r\n```\r\n\r\nand somewhere:\r\n\r\n```\r\n class BaseViz(object):\r\n\r\n     \"\"\"All visualizations derive this base class\"\"\"\r\n@@ -223,6 +233,7 @@ class BaseViz(object):\r\n                     df[DTTM_ALIAS] += timedelta(hours=self.datasource.offset)\r\n             df.replace([np.inf, -np.inf], np.nan)\r\n             df = df.fillna(0)\r\n+        df.columns = [x.lower() for x in df.columns]\r\n\r\n```\r\n\r\n"}}
{"actor":{"display_login":"jiamo"}}