{"action":"created","issue":{"url":"https://api.github.com/repos/apache/storm/issues/2400","repository_url":"https://api.github.com/repos/apache/storm","labels_url":"https://api.github.com/repos/apache/storm/issues/2400/labels{/name}","comments_url":"https://api.github.com/repos/apache/storm/issues/2400/comments","events_url":"https://api.github.com/repos/apache/storm/issues/2400/events","html_url":"https://github.com/apache/storm/pull/2400","id":271050682,"number":2400,"title":" STORM-2792: Remove RAS EvictionPolicy and cleanup ","user":{"login":"revans2","id":3441321,"avatar_url":"https://avatars0.githubusercontent.com/u/3441321?v=4","gravatar_id":"","url":"https://api.github.com/users/revans2","html_url":"https://github.com/revans2","followers_url":"https://api.github.com/users/revans2/followers","following_url":"https://api.github.com/users/revans2/following{/other_user}","gists_url":"https://api.github.com/users/revans2/gists{/gist_id}","starred_url":"https://api.github.com/users/revans2/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/revans2/subscriptions","organizations_url":"https://api.github.com/users/revans2/orgs","repos_url":"https://api.github.com/users/revans2/repos","events_url":"https://api.github.com/users/revans2/events{/privacy}","received_events_url":"https://api.github.com/users/revans2/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2017-11-03T17:05:53Z","updated_at":"2017-11-06T21:20:39Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/storm/pulls/2400","html_url":"https://github.com/apache/storm/pull/2400","diff_url":"https://github.com/apache/storm/pull/2400.diff","patch_url":"https://github.com/apache/storm/pull/2400.patch"},"body":""},"comment":{"url":"https://api.github.com/repos/apache/storm/issues/comments/342290628","html_url":"https://github.com/apache/storm/pull/2400#issuecomment-342290628","issue_url":"https://api.github.com/repos/apache/storm/issues/2400","id":342290628,"user":{"login":"revans2","id":3441321,"avatar_url":"https://avatars0.githubusercontent.com/u/3441321?v=4","gravatar_id":"","url":"https://api.github.com/users/revans2","html_url":"https://github.com/revans2","followers_url":"https://api.github.com/users/revans2/followers","following_url":"https://api.github.com/users/revans2/following{/other_user}","gists_url":"https://api.github.com/users/revans2/gists{/gist_id}","starred_url":"https://api.github.com/users/revans2/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/revans2/subscriptions","organizations_url":"https://api.github.com/users/revans2/orgs","repos_url":"https://api.github.com/users/revans2/repos","events_url":"https://api.github.com/users/revans2/events{/privacy}","received_events_url":"https://api.github.com/users/revans2/received_events","type":"User","site_admin":false},"created_at":"2017-11-06T21:20:39Z","updated_at":"2017-11-06T21:20:39Z","author_association":"CONTRIBUTOR","body":"@jerrypeng,\r\n\r\nI am not sure your intuition is right, but this is all still theoretical until we roll it out and see what happens in real life.  We have run some simulations that at least for our simulated load it does not appear to be any worse and in cases where there are GPU like resources it is a lot better. \r\n\r\nSolving fragmentation should mostly be around matching the ratio of resources in a request to the resources left on a node, which is a lot of what your initial RAS paper was about.  The problem is that when sorting the nodes we prioritize proximity to other parts of the same topology over fixing fragmentation.  So fragmentation really only matters when scheduling the first executor on a node.  Because of this I don't think it is the size of the topology that matters as much as it is the size of the individual components.  To really solve this we need a way to balance the desire to co-locate components with how well does this request fill what is left on the node.  I am hopeful that when we finish https://issues.apache.org/jira/browse/STORM-2684 the scheduler will group parts of a topology that give it the biggest win within a single \"super component\" and then if we need to we can look at having a config that controls when to switch from sorting by co-locating to sorting to reduce fragmentation.  i.e. when do we move on to the next node in the rack even if this one is not full because we are starting to run low on resource X.\r\n\r\nIf you have suggestions or want to collaborate on it that would be great, but you know how hard it is to for us to get legal approval to share too much more than just code. So for now we want to try and get this feature rolled out and then monitor it to see how it goes and if we need to adjust anything. \r\n\r\n@govind-menon,\r\n\r\nDo you have some of the simulation results in a human readable format we can share?  Also if you have the code we used to run the simulation putting an Apache license on it posting it would be great so that others can reproduce what we have done."}}
{"actor":{"display_login":"revans2"}}