{"action":"created","issue":{"url":"https://api.github.com/repos/tomncooper/CSC8101-Documentation/issues/3","repository_url":"https://api.github.com/repos/tomncooper/CSC8101-Documentation","labels_url":"https://api.github.com/repos/tomncooper/CSC8101-Documentation/issues/3/labels{/name}","comments_url":"https://api.github.com/repos/tomncooper/CSC8101-Documentation/issues/3/comments","events_url":"https://api.github.com/repos/tomncooper/CSC8101-Documentation/issues/3/events","html_url":"https://github.com/tomncooper/CSC8101-Documentation/issues/3","id":207515978,"number":3,"title":"Memory problems and time to run ALS.train","user":{"login":"Tom-Ryder","id":23359943,"avatar_url":"https://avatars.githubusercontent.com/u/23359943?v=3","gravatar_id":"","url":"https://api.github.com/users/Tom-Ryder","html_url":"https://github.com/Tom-Ryder","followers_url":"https://api.github.com/users/Tom-Ryder/followers","following_url":"https://api.github.com/users/Tom-Ryder/following{/other_user}","gists_url":"https://api.github.com/users/Tom-Ryder/gists{/gist_id}","starred_url":"https://api.github.com/users/Tom-Ryder/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Tom-Ryder/subscriptions","organizations_url":"https://api.github.com/users/Tom-Ryder/orgs","repos_url":"https://api.github.com/users/Tom-Ryder/repos","events_url":"https://api.github.com/users/Tom-Ryder/events{/privacy}","received_events_url":"https://api.github.com/users/Tom-Ryder/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2017-02-14T13:31:54Z","updated_at":"2017-02-15T14:21:01Z","closed_at":null,"body":"I am attempting to run ALS.train w/ parameters as suggested on an 80/20 split of the data. I have not as of yet been able to obtain a result, with the computation taking an extremely long time. I have, as suggested in a previous issue, increased my memory size - but to no avail.\r\n\r\nI have received a number of system warnings from my vm's terminal, the latest being: \r\n\r\n\"[Stage 20:>                                                        (0 + 2) / 37]17/02/14 13:28:31 WARN LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK\r\n'\r\n\r\nThis was previously:\r\n\r\n\"17/02/14 13:10:59 WARN MemoryStore: Failed to reserve initial memory threshold of 1024.0 KB for computing block rdd_27_67 in memory.\"\r\n\r\nbefore increasing the memory size. "},"comment":{"url":"https://api.github.com/repos/tomncooper/CSC8101-Documentation/issues/comments/280022836","html_url":"https://github.com/tomncooper/CSC8101-Documentation/issues/3#issuecomment-280022836","issue_url":"https://api.github.com/repos/tomncooper/CSC8101-Documentation/issues/3","id":280022836,"user":{"login":"a8256303","id":23357901,"avatar_url":"https://avatars.githubusercontent.com/u/23357901?v=3","gravatar_id":"","url":"https://api.github.com/users/a8256303","html_url":"https://github.com/a8256303","followers_url":"https://api.github.com/users/a8256303/followers","following_url":"https://api.github.com/users/a8256303/following{/other_user}","gists_url":"https://api.github.com/users/a8256303/gists{/gist_id}","starred_url":"https://api.github.com/users/a8256303/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/a8256303/subscriptions","organizations_url":"https://api.github.com/users/a8256303/orgs","repos_url":"https://api.github.com/users/a8256303/repos","events_url":"https://api.github.com/users/a8256303/events{/privacy}","received_events_url":"https://api.github.com/users/a8256303/received_events","type":"User","site_admin":false},"created_at":"2017-02-15T14:21:01Z","updated_at":"2017-02-15T14:21:01Z","body":"I don't think the splitting of the data is the problem. I can split using the inbuilt functions and look at the new data quite quickly. \r\n`ratingsSubset=ratings.sample(0.00001,1)`\r\n`trainData,testData = ratingsSubset.randomSplit([0.8, 0.2],seed=1)`"}}
{"actor":{"display_login":"a8256303"}}