{"action":"closed","issue":{"url":"https://api.github.com/repos/joshday/SparseRegression.jl/issues/13","repository_url":"https://api.github.com/repos/joshday/SparseRegression.jl","labels_url":"https://api.github.com/repos/joshday/SparseRegression.jl/issues/13/labels{/name}","comments_url":"https://api.github.com/repos/joshday/SparseRegression.jl/issues/13/comments","events_url":"https://api.github.com/repos/joshday/SparseRegression.jl/issues/13/events","html_url":"https://github.com/joshday/SparseRegression.jl/issues/13","id":262136272,"number":13,"title":"error with all betas positive","user":{"login":"crsl4","id":8419800,"avatar_url":"https://avatars3.githubusercontent.com/u/8419800?v=4","gravatar_id":"","url":"https://api.github.com/users/crsl4","html_url":"https://github.com/crsl4","followers_url":"https://api.github.com/users/crsl4/followers","following_url":"https://api.github.com/users/crsl4/following{/other_user}","gists_url":"https://api.github.com/users/crsl4/gists{/gist_id}","starred_url":"https://api.github.com/users/crsl4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/crsl4/subscriptions","organizations_url":"https://api.github.com/users/crsl4/orgs","repos_url":"https://api.github.com/users/crsl4/repos","events_url":"https://api.github.com/users/crsl4/events{/privacy}","received_events_url":"https://api.github.com/users/crsl4/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2017-10-02T16:01:54Z","updated_at":"2017-10-02T19:01:32Z","closed_at":"2017-10-02T19:01:31Z","author_association":"NONE","body":"I was trying to code cross validation, when I realized that no matter which subset of the data I selected, I would always get all betas positive.\r\nThis could be a bug in the optimization?\r\nI don't get the same results when I run with `GLMNet`.\r\n\r\n```julia\r\nusing DataFrames, SparseRegression\r\ndat = readtable(\"test.txt\", header=true)\r\ny = convert(Array{Float64,1},dat[:,1]) ##1st column: response\r\nX = convert(Array{Float64,2},dat[:,2:end]) ##10,000 SNPs\r\nobs = Obs(X, y)\r\ns = SModel(obs, LogisticRegression(), L1Penalty())\r\n@time learn!(s, ProxGrad(obs,0.001), MaxIter(10000), Converged(coef))\r\n#23.880103 seconds (106.87 k allocations: 765.675 MiB, 0.69% gc time)\r\n#SparseRegression.SModel{LossFunctions.LogitMarginLoss,PenaltyFunctions.L1Penalty}\r\n #> β        : [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0  …  0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]\r\n #> λ factor : [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1  …  0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]\r\n #> Loss     : LogitMarginLoss\r\n #> Penalty  : L1Penalty\r\nfind(x -> x != 0,s.β) ##103 SNPs for λ=0.1\r\nmean(s.β[find(x -> x != 0,s.β)])\r\nfind(x -> x < 0,s.β) ## 0\r\n```\r\n\r\nUsing  `GLMNet`:\r\n\r\n```julia\r\nusing GLMNet, Distributions\r\ny1 = Bool.(y)\r\ny2 = .!y1\r\nyy = [y1 y2]\r\npath = @time glmnet(X, convert(Matrix{Float64}, yy), Binomial())\r\n```\r\nIn `GLMNet`, the biggest lambda used is 0.089, and with this lambda, all betas are equal to 0 (no covariates in the model). `SparseRegression` is using lambda=0.1 (even bigger lambda), so I would expect all betas to be close to zero (but the mean is equal to 0.013, not that close to zero).\r\nCould there by any rounding error somewhere, or something, making these estimated coefficients further from zero?\r\n\r\nWhen trying with a different lambda (the best lambda by CV in `GLMNet` which yields 19 covariates in the model), we still get all positive betas for 170 covariates:\r\n\r\n```julia\r\ncv = @time glmnetcv(X, yy, Binomial())\r\n#11.709583 seconds (299.73 k allocations: 623.351 MiB, 1.03% gc time)\r\n#Logistic GLMNet Cross Validation\r\n#100 models for 10000 predictors in 10 folds\r\n#Best λ 0.062 (mean loss 1.380, std 0.008)\r\nobs = Obs(X, y)\r\ns = SModel(obs, LogisticRegression(), L1Penalty(), fill(0.062,size(X,2)))\r\n@time learn!(s, ProxGrad(obs,0.001), MaxIter(10000), Converged(coef))\r\nfind(x -> x != 0,s.β) ##170 SNPs for λ=0.1\r\nmean(s.β[find(x -> x != 0,s.β)])\r\nfind(x -> x < 0,s.β) ## 0\r\n```\r\nI attach a test data file to reproduce the error.\r\n[test.txt](https://github.com/joshday/SparseRegression.jl/files/1349500/test.txt)\r\n"}}
{"actor":{"display_login":"crsl4"}}