{"action":"created","issue":{"url":"https://api.github.com/repos/deepchem/deepchem/issues/568","repository_url":"https://api.github.com/repos/deepchem/deepchem","labels_url":"https://api.github.com/repos/deepchem/deepchem/issues/568/labels{/name}","comments_url":"https://api.github.com/repos/deepchem/deepchem/issues/568/comments","events_url":"https://api.github.com/repos/deepchem/deepchem/issues/568/events","html_url":"https://github.com/deepchem/deepchem/pull/568","id":230445219,"number":568,"title":"DeepChem learns to play tictactoe... Badly","user":{"login":"lilleswing","id":1225605,"avatar_url":"https://avatars1.githubusercontent.com/u/1225605?v=3","gravatar_id":"","url":"https://api.github.com/users/lilleswing","html_url":"https://github.com/lilleswing","followers_url":"https://api.github.com/users/lilleswing/followers","following_url":"https://api.github.com/users/lilleswing/following{/other_user}","gists_url":"https://api.github.com/users/lilleswing/gists{/gist_id}","starred_url":"https://api.github.com/users/lilleswing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lilleswing/subscriptions","organizations_url":"https://api.github.com/users/lilleswing/orgs","repos_url":"https://api.github.com/users/lilleswing/repos","events_url":"https://api.github.com/users/lilleswing/events{/privacy}","received_events_url":"https://api.github.com/users/lilleswing/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2017-05-22T16:08:24Z","updated_at":"2017-05-22T22:44:13Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/deepchem/deepchem/pulls/568","html_url":"https://github.com/deepchem/deepchem/pull/568","diff_url":"https://github.com/deepchem/deepchem/pull/568.diff","patch_url":"https://github.com/deepchem/deepchem/pull/568.patch"},"body":"I played around with a3c and RL in Deepchem seeing if I could train a tictactoe agent.\r\n\r\nThe agent isn't good yet and is still having issues.\r\n\r\n1) after training for a long time the Dense networks output NaN instead of float32 values\r\n2) The agent just makes illegal moves constantly.\r\n\r\nI still wanted to put this  up for CR because I found some small things in A3C that might trip up others so I wanted to fix them.\r\n\r\n1) Deep Copy the environment's state in the worker thread.  Environments will most likely mutate their environment on step commands, without this all states would just be the final state for the rollout.\r\n\r\n2) Ininitalize the global variables after the worker threads are created.  This might be wrong @peastman .  This was needed because workers were creating their own variables on worker thread creation.  How did you get it to work otherwise?  Do we want the layers shared between the workers or each worker to have their own network?"},"comment":{"url":"https://api.github.com/repos/deepchem/deepchem/issues/comments/303240243","html_url":"https://github.com/deepchem/deepchem/pull/568#issuecomment-303240243","issue_url":"https://api.github.com/repos/deepchem/deepchem/issues/568","id":303240243,"user":{"login":"rbharath","id":1057676,"avatar_url":"https://avatars0.githubusercontent.com/u/1057676?v=3","gravatar_id":"","url":"https://api.github.com/users/rbharath","html_url":"https://github.com/rbharath","followers_url":"https://api.github.com/users/rbharath/followers","following_url":"https://api.github.com/users/rbharath/following{/other_user}","gists_url":"https://api.github.com/users/rbharath/gists{/gist_id}","starred_url":"https://api.github.com/users/rbharath/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rbharath/subscriptions","organizations_url":"https://api.github.com/users/rbharath/orgs","repos_url":"https://api.github.com/users/rbharath/repos","events_url":"https://api.github.com/users/rbharath/events{/privacy}","received_events_url":"https://api.github.com/users/rbharath/received_events","type":"User","site_admin":false},"created_at":"2017-05-22T22:44:13Z","updated_at":"2017-05-22T22:44:13Z","body":"I really like the idea of adding more environments to DeepChem. New environments are the only real way of benchmarking our RL suite, and it would be great to build up our capabilities. In fact, I have a couple extra suggestions:\r\n\r\n- Can you make a new submodule `dc.rl.envs` that hosts environments and move the Tic-tac-toe environment there?\r\n- Can you add a section to the benchmarks on the `README.md` that documents best performance on the Tic-Tac-Toe environment? I think we should start keeping running numbers on the RL environments like we do for MoleculeNet (betting RL is going to be increasingly important moving forward)."}}
{"actor":{"display_login":"rbharath"}}