{"action":"created","issue":{"url":"https://api.github.com/repos/hpcugent/easybuild-framework/issues/2136","repository_url":"https://api.github.com/repos/hpcugent/easybuild-framework","labels_url":"https://api.github.com/repos/hpcugent/easybuild-framework/issues/2136/labels{/name}","comments_url":"https://api.github.com/repos/hpcugent/easybuild-framework/issues/2136/comments","events_url":"https://api.github.com/repos/hpcugent/easybuild-framework/issues/2136/events","html_url":"https://github.com/hpcugent/easybuild-framework/pull/2136","id":210878061,"number":2136,"title":"(WIP) Automagic update dependencies","user":{"login":"damianam","id":10760206,"avatar_url":"https://avatars2.githubusercontent.com/u/10760206?v=3","gravatar_id":"","url":"https://api.github.com/users/damianam","html_url":"https://github.com/damianam","followers_url":"https://api.github.com/users/damianam/followers","following_url":"https://api.github.com/users/damianam/following{/other_user}","gists_url":"https://api.github.com/users/damianam/gists{/gist_id}","starred_url":"https://api.github.com/users/damianam/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/damianam/subscriptions","organizations_url":"https://api.github.com/users/damianam/orgs","repos_url":"https://api.github.com/users/damianam/repos","events_url":"https://api.github.com/users/damianam/events{/privacy}","received_events_url":"https://api.github.com/users/damianam/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2017-02-28T19:10:43Z","updated_at":"2017-03-01T07:48:39Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/hpcugent/easybuild-framework/pulls/2136","html_url":"https://github.com/hpcugent/easybuild-framework/pull/2136","diff_url":"https://github.com/hpcugent/easybuild-framework/pull/2136.diff","patch_url":"https://github.com/hpcugent/easybuild-framework/pull/2136.patch"},"body":"This PR is to implement the automatic update of dependencies as discussed in the User Meeting.\r\n\r\nMy intended path forward is similar to how `--minimal-toolchains` was implemented:\r\n\r\n* In `easyconfig.py` expand the parsing of easyconfigs, to account for `--try-dependencies-version={AVAILABLE,NEWER,software:version}`\r\n* That means replacing the dependencies version as they are read.\r\n* For that I'll need to implement a robot-like function, that finds the best candidate\r\n    * Finding the best candidate is not trivial. On one side there is `robot_find_minimal_toolchain_of_dependency` which should be used to find the toolchain. On the opposite side that function (or possibly `robot_find_easyconfig` or `select_or_generate_ec`, I'd have to take a deeper look at them) would have to be extended to find non-fixed dependency versions to accommodate `AVAILABLE` and `NEWER`, and non-existing dependency versions when using `software:version`.\r\n* With the best candidate selected at parsing time we can:\r\n  * tweak and dump the dependency, if a new one is generated (`software:version` use case)\r\n  * tweak and dump the easyconfig, with the updated list of dependencies.\r\n  * This will happen before the dependency resolution, as opposed to all the other `--try-X` options. For those, `resolve_dependencies` gets called twice, one within `tweak`, and then the real deal.\r\n* With the appropriate easyconfigs generated we can proceed as normal\r\n\r\nI started looking into implementing this in the robot as other `--try-X` options, but I think there it would be way more complicated. The main difference lies in the other `--try-X` options having clear candidates to be tweaked and they are selected with `resolve_dependencies`. Here it is way more open and I was struggling to find a proper structure to implement this there. So maybe this is the easiest way forward.\r\n\r\nAlso, to untangle things, I think it makes sense to pull out some find-related functions to a separate file. `easyconfig.py` is already big enough, and certainly a function called `robot_*` and being called from `robot.py` is a good candidate to be moved out.\r\n\r\nI also find confusing the double call to `resolve_dependencies` for `--try-X` options. Is there any reason for that? Wouldn't it make sense to get a fixed dependency tree and then tweak and dump the whole tree, instead of generating a bunch of tweaked files and then resolve the tree and maybe select them? This has nothing to do with this PR, but it took me a while to figure out how things are working.\r\n\r\nThoughts? @victorusu, @boegel, @ocaisa "},"comment":{"url":"https://api.github.com/repos/hpcugent/easybuild-framework/issues/comments/283269215","html_url":"https://github.com/hpcugent/easybuild-framework/pull/2136#issuecomment-283269215","issue_url":"https://api.github.com/repos/hpcugent/easybuild-framework/issues/2136","id":283269215,"user":{"login":"ocaisa","id":3840437,"avatar_url":"https://avatars0.githubusercontent.com/u/3840437?v=3","gravatar_id":"","url":"https://api.github.com/users/ocaisa","html_url":"https://github.com/ocaisa","followers_url":"https://api.github.com/users/ocaisa/followers","following_url":"https://api.github.com/users/ocaisa/following{/other_user}","gists_url":"https://api.github.com/users/ocaisa/gists{/gist_id}","starred_url":"https://api.github.com/users/ocaisa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ocaisa/subscriptions","organizations_url":"https://api.github.com/users/ocaisa/orgs","repos_url":"https://api.github.com/users/ocaisa/repos","events_url":"https://api.github.com/users/ocaisa/events{/privacy}","received_events_url":"https://api.github.com/users/ocaisa/received_events","type":"User","site_admin":false},"created_at":"2017-03-01T07:48:39Z","updated_at":"2017-03-01T07:48:39Z","body":"There is a potential issue in that kind of approach. If an easyconfig for an updated dep already exists, this should be preferred to a tweaked version...but if such an easyconfig had a different set of dependencies to one that was resolved by the robot you would have to bomb out because you'd know your robot result is no longer correct.  It is most likely to be a corner case though since I expect that the auto-update of deps will be restricted to minor version updates."}}
{"actor":{"display_login":"ocaisa"}}