{"action":"created","issue":{"url":"https://api.github.com/repos/statsmodels/statsmodels/issues/2449","repository_url":"https://api.github.com/repos/statsmodels/statsmodels","labels_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/2449/labels{/name}","comments_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/2449/comments","events_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/2449/events","html_url":"https://github.com/statsmodels/statsmodels/issues/2449","id":88573455,"number":2449,"title":"[Bug/Doc] lowess returns nan and does not warn if there are too few neighbors","user":{"login":"carlshan","id":1524687,"avatar_url":"https://avatars0.githubusercontent.com/u/1524687?v=4","gravatar_id":"","url":"https://api.github.com/users/carlshan","html_url":"https://github.com/carlshan","followers_url":"https://api.github.com/users/carlshan/followers","following_url":"https://api.github.com/users/carlshan/following{/other_user}","gists_url":"https://api.github.com/users/carlshan/gists{/gist_id}","starred_url":"https://api.github.com/users/carlshan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/carlshan/subscriptions","organizations_url":"https://api.github.com/users/carlshan/orgs","repos_url":"https://api.github.com/users/carlshan/repos","events_url":"https://api.github.com/users/carlshan/events{/privacy}","received_events_url":"https://api.github.com/users/carlshan/received_events","type":"User","site_admin":false},"labels":[{"id":4344793,"url":"https://api.github.com/repos/statsmodels/statsmodels/labels/comp-nonparametric","name":"comp-nonparametric","color":"ededed","default":false},{"id":142960,"url":"https://api.github.com/repos/statsmodels/statsmodels/labels/type-bug","name":"type-bug","color":"ff9999","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2015-06-16T00:11:56Z","updated_at":"2017-12-07T12:58:59Z","closed_at":null,"author_association":"NONE","body":"**Statsmodels version**: 0.6.1\n**Issue**: when the # of unique values in X is less than the number of unique neighbors needed to compute LOWESS, nans are returned but no error or warning is given.\n**Expected behavior**: a warning should be raised that LOWESS cannot be computed due to lack of sufficient unique neighbors.\n\n(Workaround: you could jitter all the x values by some tiny epsilon to create more unique values, and then rerun lowess(y, x).)\n\n``` python\nimport statsmodels as sm\nprint sm.__version__ # '0.6.1'\nfrom sm.nonparametric.smoothers_lowess import lowess\nimport numpy as np\n\n# initializing data\nx = np.random.choice(range(0, 3), 100) # only 3 unique values: {0, 1, 2}\ny = np.random.choice(np.arange(0, 1, 0.1), 100)\n\npreds = lowess(y, x)[:, 1] # Slicing to get the predicted y values\n# no warning or exception raised, despite the fact that all predictions are nan\nprint preds\n>>> array([ nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,  nan,\n        nan])\n\n# Trying again, with a few more unique x values\nx = np.random.choice(range(0, 5), 100) # now, with 5 unique values\ny = np.random.choice(np.arange(0, 1, 0.1), 100)\nprint preds\n>>> array([ 0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,\n        0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,\n        0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,\n        0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,  0.4558193 ,\n        0.4558193 ,  0.4558193 ,  0.42940657,  0.42940657,  0.42940657,\n        0.42940657,  0.42940657,  0.42940657,  0.42940657,  0.42940657,\n        0.42940657,  0.42940657,  0.42940657,  0.42940657,  0.42940657,\n        0.42940657,  0.42940657,  0.42940657,  0.42940657,  0.42940657,\n        0.42940657,  0.42940657,  0.42940657,  0.40486366,  0.40486366,\n        0.40486366,  0.40486366,  0.40486366,  0.40486366,  0.40486366,\n        0.40486366,  0.40486366,  0.40486366,  0.40486366,  0.40486366,\n        0.40486366,  0.40486366,  0.40486366,  0.40486366,  0.40486366,\n        0.40486366,  0.40486366,  0.40486366,  0.41932243,  0.41932243,\n        0.41932243,  0.41932243,  0.41932243,  0.41932243,  0.41932243,\n        0.41932243,  0.41932243,  0.41932243,  0.41932243,  0.41932243,\n        0.41932243,  0.41932243,  0.41932243,  0.41932243,  0.41932243,\n        0.41932243,  0.41932243,  0.41932243,  0.48931377,  0.48931377,\n        0.48931377,  0.48931377,  0.48931377,  0.48931377,  0.48931377,\n        0.48931377,  0.48931377,  0.48931377,  0.48931377,  0.48931377,\n        0.48931377,  0.48931377,  0.48931377,  0.48931377,  0.48931377])\n```\n"},"comment":{"url":"https://api.github.com/repos/statsmodels/statsmodels/issues/comments/349959856","html_url":"https://github.com/statsmodels/statsmodels/issues/2449#issuecomment-349959856","issue_url":"https://api.github.com/repos/statsmodels/statsmodels/issues/2449","id":349959856,"user":{"login":"josef-pkt","id":440735,"avatar_url":"https://avatars1.githubusercontent.com/u/440735?v=4","gravatar_id":"","url":"https://api.github.com/users/josef-pkt","html_url":"https://github.com/josef-pkt","followers_url":"https://api.github.com/users/josef-pkt/followers","following_url":"https://api.github.com/users/josef-pkt/following{/other_user}","gists_url":"https://api.github.com/users/josef-pkt/gists{/gist_id}","starred_url":"https://api.github.com/users/josef-pkt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/josef-pkt/subscriptions","organizations_url":"https://api.github.com/users/josef-pkt/orgs","repos_url":"https://api.github.com/users/josef-pkt/repos","events_url":"https://api.github.com/users/josef-pkt/events{/privacy}","received_events_url":"https://api.github.com/users/josef-pkt/received_events","type":"User","site_admin":false},"created_at":"2017-12-07T12:58:59Z","updated_at":"2017-12-07T12:58:59Z","author_association":"OWNER","body":"returning the nan array is useful if we still want to plot, because matplotlib just doesn't show nan values. That is we should get the plot but without the lowess curve.\r\n\r\nGiven that we need to sort by `x` we could make a check for unique counts with `np.max(np.nonzero(np.diff(x))) > something` that shouldn't be very expensive.  I'm not sure what the threshold `something` should be. Based on my old comments it is not so clear when the results become strange or nan.\r\nWe could add an option `exog_ties` or x_ties with \"none\" (ignore), \"raise\" and \"warn\".  "}}
{"actor":{"display_login":"josef-pkt"}}