{"action":"created","issue":{"url":"https://api.github.com/repos/benchflow/benchflow/issues/337","repository_url":"https://api.github.com/repos/benchflow/benchflow","labels_url":"https://api.github.com/repos/benchflow/benchflow/issues/337/labels{/name}","comments_url":"https://api.github.com/repos/benchflow/benchflow/issues/337/comments","events_url":"https://api.github.com/repos/benchflow/benchflow/issues/337/events","html_url":"https://github.com/benchflow/benchflow/issues/337","id":228470568,"number":337,"title":"Test Termination Criteria - Max Time","user":{"login":"jeppe-style","id":8646217,"avatar_url":"https://avatars1.githubusercontent.com/u/8646217?v=3","gravatar_id":"","url":"https://api.github.com/users/jeppe-style","html_url":"https://github.com/jeppe-style","followers_url":"https://api.github.com/users/jeppe-style/followers","following_url":"https://api.github.com/users/jeppe-style/following{/other_user}","gists_url":"https://api.github.com/users/jeppe-style/gists{/gist_id}","starred_url":"https://api.github.com/users/jeppe-style/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeppe-style/subscriptions","organizations_url":"https://api.github.com/users/jeppe-style/orgs","repos_url":"https://api.github.com/users/jeppe-style/repos","events_url":"https://api.github.com/users/jeppe-style/events{/privacy}","received_events_url":"https://api.github.com/users/jeppe-style/received_events","type":"User","site_admin":false},"labels":[{"id":565702938,"url":"https://api.github.com/repos/benchflow/benchflow/labels/benchflow-test-manager","name":"benchflow-test-manager","color":"ffcc00","default":false},{"id":285822989,"url":"https://api.github.com/repos/benchflow/benchflow/labels/feature","name":"feature","color":"eb6420","default":false}],"state":"open","locked":false,"assignee":{"login":"jeppe-style","id":8646217,"avatar_url":"https://avatars1.githubusercontent.com/u/8646217?v=3","gravatar_id":"","url":"https://api.github.com/users/jeppe-style","html_url":"https://github.com/jeppe-style","followers_url":"https://api.github.com/users/jeppe-style/followers","following_url":"https://api.github.com/users/jeppe-style/following{/other_user}","gists_url":"https://api.github.com/users/jeppe-style/gists{/gist_id}","starred_url":"https://api.github.com/users/jeppe-style/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeppe-style/subscriptions","organizations_url":"https://api.github.com/users/jeppe-style/orgs","repos_url":"https://api.github.com/users/jeppe-style/repos","events_url":"https://api.github.com/users/jeppe-style/events{/privacy}","received_events_url":"https://api.github.com/users/jeppe-style/received_events","type":"User","site_admin":false},"assignees":[{"login":"jeppe-style","id":8646217,"avatar_url":"https://avatars1.githubusercontent.com/u/8646217?v=3","gravatar_id":"","url":"https://api.github.com/users/jeppe-style","html_url":"https://github.com/jeppe-style","followers_url":"https://api.github.com/users/jeppe-style/followers","following_url":"https://api.github.com/users/jeppe-style/following{/other_user}","gists_url":"https://api.github.com/users/jeppe-style/gists{/gist_id}","starred_url":"https://api.github.com/users/jeppe-style/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jeppe-style/subscriptions","organizations_url":"https://api.github.com/users/jeppe-style/orgs","repos_url":"https://api.github.com/users/jeppe-style/repos","events_url":"https://api.github.com/users/jeppe-style/events{/privacy}","received_events_url":"https://api.github.com/users/jeppe-style/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/benchflow/benchflow/milestones/1","html_url":"https://github.com/benchflow/benchflow/milestone/1","labels_url":"https://api.github.com/repos/benchflow/benchflow/milestones/1/labels","id":1408478,"number":1,"title":"Priorities","description":"Stuff we must do soon. We never change or rename this milestone.","creator":{"login":"VincenzoFerme","id":7163641,"avatar_url":"https://avatars0.githubusercontent.com/u/7163641?v=3","gravatar_id":"","url":"https://api.github.com/users/VincenzoFerme","html_url":"https://github.com/VincenzoFerme","followers_url":"https://api.github.com/users/VincenzoFerme/followers","following_url":"https://api.github.com/users/VincenzoFerme/following{/other_user}","gists_url":"https://api.github.com/users/VincenzoFerme/gists{/gist_id}","starred_url":"https://api.github.com/users/VincenzoFerme/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VincenzoFerme/subscriptions","organizations_url":"https://api.github.com/users/VincenzoFerme/orgs","repos_url":"https://api.github.com/users/VincenzoFerme/repos","events_url":"https://api.github.com/users/VincenzoFerme/events{/privacy}","received_events_url":"https://api.github.com/users/VincenzoFerme/received_events","type":"User","site_admin":false},"open_issues":84,"closed_issues":167,"state":"open","created_at":"2015-11-12T14:33:55Z","updated_at":"2017-06-21T17:25:03Z","due_on":null,"closed_at":null},"comments":2,"created_at":"2017-05-13T12:35:02Z","updated_at":"2017-06-22T12:41:59Z","closed_at":null,"body":"```\r\nconfiguration:\r\n  termination_criteria:\r\n    test:\r\n      max_running_time: { String : <amount of time + unit> }\r\n```\r\nWhen specified in the test definition there should be a timeout triggered after the specified time has passed so that the test can be stopped (any running experiments must also be stopped) and set to state `Terminated` `Partially Complete`.\r\n\r\n@VincenzoFerme when do we start to count this time? When the test has ben submitted by the user and become part the lifecycle? Another option is to only count the time it is actually running (state=running), e.g., we do not count the waiting time. => Count only the running time\r\n\r\nWe can use a [ScheduledExecutorService](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ScheduledExecutorService.html) to schedule a `Runnable` (`TestTimeout`) that notifies the Scheduler that the termination criteria has been reached after a given delay. \r\n\r\nWhen submitting the `Runnable` we get a `Future` that can be stored in a `ConcurrentMap<String Future>` (key=testID). In this way we can remove the `TestTimeout` if the test completes before the max time.\r\n\r\nThe `ScheduledExecutorService` can be instantiated in the application and fetched in the same way as the `TaskExecutorService`."},"comment":{"url":"https://api.github.com/repos/benchflow/benchflow/issues/comments/310368457","html_url":"https://github.com/benchflow/benchflow/issues/337#issuecomment-310368457","issue_url":"https://api.github.com/repos/benchflow/benchflow/issues/337","id":310368457,"user":{"login":"VincenzoFerme","id":7163641,"avatar_url":"https://avatars0.githubusercontent.com/u/7163641?v=3","gravatar_id":"","url":"https://api.github.com/users/VincenzoFerme","html_url":"https://github.com/VincenzoFerme","followers_url":"https://api.github.com/users/VincenzoFerme/followers","following_url":"https://api.github.com/users/VincenzoFerme/following{/other_user}","gists_url":"https://api.github.com/users/VincenzoFerme/gists{/gist_id}","starred_url":"https://api.github.com/users/VincenzoFerme/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/VincenzoFerme/subscriptions","organizations_url":"https://api.github.com/users/VincenzoFerme/orgs","repos_url":"https://api.github.com/users/VincenzoFerme/repos","events_url":"https://api.github.com/users/VincenzoFerme/events{/privacy}","received_events_url":"https://api.github.com/users/VincenzoFerme/received_events","type":"User","site_admin":false},"created_at":"2017-06-22T12:41:59Z","updated_at":"2017-06-22T12:41:59Z","body":"So if I got it right we are going to have tests waiting in a queue to go to the running state, then according to the type of task, you are going to schedule the thread in one of the two queues you handle, right?"}}
{"actor":{"display_login":"VincenzoFerme"}}