{"action":"created","comment":{"url":"https://api.github.com/repos/broadinstitute/picard/pulls/comments/145230905","pull_request_review_id":70001001,"id":145230905,"diff_hunk":"@@ -21,21 +26,63 @@\n public class TestDataProviders {\n \n     @Test\n-    public void testAllDataProviders() throws IllegalAccessException, InstantiationException, InvocationTargetException {\n-        int i = 0;\n-        final ClassFinder classFinder = new ClassFinder();\n-        classFinder.find(\"picard\", Object.class);\n-        for (Class testClass : classFinder.getClasses()) {\n-            if (Modifier.isAbstract(testClass.getModifiers())) continue;\n-            for (final Method method : testClass.getMethods()) {\n-                if (method.isAnnotationPresent(DataProvider.class)) {\n-                    System.err.println(\"Method: \" + testClass.getName() + \":\" + method.getName());\n-                    method.invoke(testClass.newInstance());\n-                    i++;\n+    public void independentTestOfDataProviderTest() throws Exception {\n+        final Iterator<Object[]> data = TestNGUtil.getDataProviders(\"picard\");\n+\n+        Assert.assertTrue(data.hasNext(), \"Found no data from testAllDataProvidersdata. Something is wrong.\");\n+\n+        Assert.assertEquals( StreamSupport.stream(Spliterators.spliteratorUnknownSize(data, 0), false)\n+                        .filter(c -> ((Method) c[0]).getName().equals(\"testAllDataProvidersdata\")).count(), 1,\n+                \"getDataProviders didn't find testAllDataProvidersdata, which is in this class. Something is wrong.\");\n+    }\n+\n+    @DataProvider(name = \"DataprovidersThatDontTestThemselves\")\n+    public Iterator<Object[]> testAllDataProvidersdata() throws Exception {\n+        return TestNGUtil.getDataProviders(\"picard\");\n+    }\n+\n+    // runs all the @DataProviders it gets from DataprovidersThatDontTestThemselves.\n+    // runs the BeforeSuite and BeforeClass methods of the same class before it runs the provider itself.\n+\n+    // @NoInjection annotations required according to this test:\n+    // https://github.com/cbeust/testng/blob/master/src/test/java/test/inject/NoInjectionTest.java\n+    @Test(dataProvider = \"DataprovidersThatDontTestThemselves\")\n+    public void testDataProviderswithDP(@NoInjection final Method method, final Class clazz) throws\n+            IllegalAccessException, InstantiationException {\n+\n+        Object instance = clazz.newInstance();\n+\n+        Set<Method> methodSet = Sets.newHashSet();\n+        methodSet.addAll(Arrays.asList(clazz.getDeclaredMethods()));\n+        methodSet.addAll(Arrays.asList(clazz.getMethods()));\n+\n+        // Some tests assume that the @BeforeSuite methods will be called before the @DataProviders","path":"src/test/java/picard/TestDataProviders.java","position":65,"original_position":65,"commit_id":"2f6926070da01a4e7f38bcab1a5db5c8d009a12f","original_commit_id":"2f6926070da01a4e7f38bcab1a5db5c8d009a12f","user":{"login":"yfarjoun","id":2745554,"avatar_url":"https://avatars3.githubusercontent.com/u/2745554?v=4","gravatar_id":"","url":"https://api.github.com/users/yfarjoun","html_url":"https://github.com/yfarjoun","followers_url":"https://api.github.com/users/yfarjoun/followers","following_url":"https://api.github.com/users/yfarjoun/following{/other_user}","gists_url":"https://api.github.com/users/yfarjoun/gists{/gist_id}","starred_url":"https://api.github.com/users/yfarjoun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yfarjoun/subscriptions","organizations_url":"https://api.github.com/users/yfarjoun/orgs","repos_url":"https://api.github.com/users/yfarjoun/repos","events_url":"https://api.github.com/users/yfarjoun/events{/privacy}","received_events_url":"https://api.github.com/users/yfarjoun/received_events","type":"User","site_admin":false},"body":"OK. I'll see if I can leave out classes that have a @BeforeSuite and not call it.","created_at":"2017-10-17T19:26:46Z","updated_at":"2017-10-17T19:26:46Z","html_url":"https://github.com/broadinstitute/picard/pull/941#discussion_r145230905","pull_request_url":"https://api.github.com/repos/broadinstitute/picard/pulls/941","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/comments/145230905"},"html":{"href":"https://github.com/broadinstitute/picard/pull/941#discussion_r145230905"},"pull_request":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/941"}},"in_reply_to_id":144580834},"pull_request":{"url":"https://api.github.com/repos/broadinstitute/picard/pulls/941","id":143724203,"html_url":"https://github.com/broadinstitute/picard/pull/941","diff_url":"https://github.com/broadinstitute/picard/pull/941.diff","patch_url":"https://github.com/broadinstitute/picard/pull/941.patch","issue_url":"https://api.github.com/repos/broadinstitute/picard/issues/941","number":941,"state":"open","locked":false,"title":"- Converted the tests of @DataProviders to using....a @DataProvider!","user":{"login":"yfarjoun","id":2745554,"avatar_url":"https://avatars3.githubusercontent.com/u/2745554?v=4","gravatar_id":"","url":"https://api.github.com/users/yfarjoun","html_url":"https://github.com/yfarjoun","followers_url":"https://api.github.com/users/yfarjoun/followers","following_url":"https://api.github.com/users/yfarjoun/following{/other_user}","gists_url":"https://api.github.com/users/yfarjoun/gists{/gist_id}","starred_url":"https://api.github.com/users/yfarjoun/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yfarjoun/subscriptions","organizations_url":"https://api.github.com/users/yfarjoun/orgs","repos_url":"https://api.github.com/users/yfarjoun/repos","events_url":"https://api.github.com/users/yfarjoun/events{/privacy}","received_events_url":"https://api.github.com/users/yfarjoun/received_events","type":"User","site_admin":false},"body":"### Description\r\nImproves the Test of DataProviders to use a @DataProvider which will show all the tests separately so that in case of failures it will not abort after the first failure.\r\n\r\n----\r\n\r\n### Checklist (never delete this)\r\n\r\nNever delete this, it is our record that procedure was followed. If you find that for whatever reason one of the checklist points doesn't apply to your PR, you can leave it unchecked but please add an explanation below.\r\n\r\n#### Content\r\n- [ ] Added or modified tests to cover changes and any new functionality\r\n- [ ] Edited the README / documentation (if applicable)\r\n- [ ] All tests passing on Travis\r\n\r\n#### Review\r\n- [ ] Final thumbs-up from reviewer\r\n- [ ] Rebase, squash and reword as applicable\r\n\r\nFor more detailed guidelines, see https://github.com/broadinstitute/picard/wiki/Guidelines-for-pull-requests\r\n\r\n","created_at":"2017-09-28T18:38:30Z","updated_at":"2017-10-17T19:26:46Z","closed_at":null,"merged_at":null,"merge_commit_sha":"b9898632c2a00c10dd1843c2e0e8f8737a3d740b","assignee":{"login":"pshapiro4broad","id":1799360,"avatar_url":"https://avatars2.githubusercontent.com/u/1799360?v=4","gravatar_id":"","url":"https://api.github.com/users/pshapiro4broad","html_url":"https://github.com/pshapiro4broad","followers_url":"https://api.github.com/users/pshapiro4broad/followers","following_url":"https://api.github.com/users/pshapiro4broad/following{/other_user}","gists_url":"https://api.github.com/users/pshapiro4broad/gists{/gist_id}","starred_url":"https://api.github.com/users/pshapiro4broad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pshapiro4broad/subscriptions","organizations_url":"https://api.github.com/users/pshapiro4broad/orgs","repos_url":"https://api.github.com/users/pshapiro4broad/repos","events_url":"https://api.github.com/users/pshapiro4broad/events{/privacy}","received_events_url":"https://api.github.com/users/pshapiro4broad/received_events","type":"User","site_admin":false},"assignees":[{"login":"pshapiro4broad","id":1799360,"avatar_url":"https://avatars2.githubusercontent.com/u/1799360?v=4","gravatar_id":"","url":"https://api.github.com/users/pshapiro4broad","html_url":"https://github.com/pshapiro4broad","followers_url":"https://api.github.com/users/pshapiro4broad/followers","following_url":"https://api.github.com/users/pshapiro4broad/following{/other_user}","gists_url":"https://api.github.com/users/pshapiro4broad/gists{/gist_id}","starred_url":"https://api.github.com/users/pshapiro4broad/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pshapiro4broad/subscriptions","organizations_url":"https://api.github.com/users/pshapiro4broad/orgs","repos_url":"https://api.github.com/users/pshapiro4broad/repos","events_url":"https://api.github.com/users/pshapiro4broad/events{/privacy}","received_events_url":"https://api.github.com/users/pshapiro4broad/received_events","type":"User","site_admin":false}],"requested_reviewers":[],"milestone":null,"commits_url":"https://api.github.com/repos/broadinstitute/picard/pulls/941/commits","review_comments_url":"https://api.github.com/repos/broadinstitute/picard/pulls/941/comments","review_comment_url":"https://api.github.com/repos/broadinstitute/picard/pulls/comments{/number}","comments_url":"https://api.github.com/repos/broadinstitute/picard/issues/941/comments","statuses_url":"https://api.github.com/repos/broadinstitute/picard/statuses/2f6926070da01a4e7f38bcab1a5db5c8d009a12f","head":{"label":"broadinstitute:yf_dataProviders_for_the_win","ref":"yf_dataProviders_for_the_win","sha":"2f6926070da01a4e7f38bcab1a5db5c8d009a12f","user":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"repo":{"id":18225913,"name":"picard","full_name":"broadinstitute/picard","owner":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/broadinstitute/picard","description":"A set of command line tools (in Java) for manipulating high-throughput sequencing (HTS) data and formats such as SAM/BAM/CRAM and VCF.","fork":false,"url":"https://api.github.com/repos/broadinstitute/picard","forks_url":"https://api.github.com/repos/broadinstitute/picard/forks","keys_url":"https://api.github.com/repos/broadinstitute/picard/keys{/key_id}","collaborators_url":"https://api.github.com/repos/broadinstitute/picard/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/broadinstitute/picard/teams","hooks_url":"https://api.github.com/repos/broadinstitute/picard/hooks","issue_events_url":"https://api.github.com/repos/broadinstitute/picard/issues/events{/number}","events_url":"https://api.github.com/repos/broadinstitute/picard/events","assignees_url":"https://api.github.com/repos/broadinstitute/picard/assignees{/user}","branches_url":"https://api.github.com/repos/broadinstitute/picard/branches{/branch}","tags_url":"https://api.github.com/repos/broadinstitute/picard/tags","blobs_url":"https://api.github.com/repos/broadinstitute/picard/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/broadinstitute/picard/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/broadinstitute/picard/git/refs{/sha}","trees_url":"https://api.github.com/repos/broadinstitute/picard/git/trees{/sha}","statuses_url":"https://api.github.com/repos/broadinstitute/picard/statuses/{sha}","languages_url":"https://api.github.com/repos/broadinstitute/picard/languages","stargazers_url":"https://api.github.com/repos/broadinstitute/picard/stargazers","contributors_url":"https://api.github.com/repos/broadinstitute/picard/contributors","subscribers_url":"https://api.github.com/repos/broadinstitute/picard/subscribers","subscription_url":"https://api.github.com/repos/broadinstitute/picard/subscription","commits_url":"https://api.github.com/repos/broadinstitute/picard/commits{/sha}","git_commits_url":"https://api.github.com/repos/broadinstitute/picard/git/commits{/sha}","comments_url":"https://api.github.com/repos/broadinstitute/picard/comments{/number}","issue_comment_url":"https://api.github.com/repos/broadinstitute/picard/issues/comments{/number}","contents_url":"https://api.github.com/repos/broadinstitute/picard/contents/{+path}","compare_url":"https://api.github.com/repos/broadinstitute/picard/compare/{base}...{head}","merges_url":"https://api.github.com/repos/broadinstitute/picard/merges","archive_url":"https://api.github.com/repos/broadinstitute/picard/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/broadinstitute/picard/downloads","issues_url":"https://api.github.com/repos/broadinstitute/picard/issues{/number}","pulls_url":"https://api.github.com/repos/broadinstitute/picard/pulls{/number}","milestones_url":"https://api.github.com/repos/broadinstitute/picard/milestones{/number}","notifications_url":"https://api.github.com/repos/broadinstitute/picard/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/broadinstitute/picard/labels{/name}","releases_url":"https://api.github.com/repos/broadinstitute/picard/releases{/id}","deployments_url":"https://api.github.com/repos/broadinstitute/picard/deployments","created_at":"2014-03-28T20:43:35Z","updated_at":"2017-10-16T15:26:56Z","pushed_at":"2017-10-16T19:15:00Z","git_url":"git://github.com/broadinstitute/picard.git","ssh_url":"git@github.com:broadinstitute/picard.git","clone_url":"https://github.com/broadinstitute/picard.git","svn_url":"https://github.com/broadinstitute/picard","homepage":"http://broadinstitute.github.io/picard/","size":113474,"stargazers_count":315,"watchers_count":315,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":214,"mirror_url":null,"open_issues_count":104,"forks":214,"open_issues":104,"watchers":315,"default_branch":"master"}},"base":{"label":"broadinstitute:master","ref":"master","sha":"d4b04a99a64f66658b9bfbff2217f4586772adb4","user":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"repo":{"id":18225913,"name":"picard","full_name":"broadinstitute/picard","owner":{"login":"broadinstitute","id":393552,"avatar_url":"https://avatars0.githubusercontent.com/u/393552?v=4","gravatar_id":"","url":"https://api.github.com/users/broadinstitute","html_url":"https://github.com/broadinstitute","followers_url":"https://api.github.com/users/broadinstitute/followers","following_url":"https://api.github.com/users/broadinstitute/following{/other_user}","gists_url":"https://api.github.com/users/broadinstitute/gists{/gist_id}","starred_url":"https://api.github.com/users/broadinstitute/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/broadinstitute/subscriptions","organizations_url":"https://api.github.com/users/broadinstitute/orgs","repos_url":"https://api.github.com/users/broadinstitute/repos","events_url":"https://api.github.com/users/broadinstitute/events{/privacy}","received_events_url":"https://api.github.com/users/broadinstitute/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/broadinstitute/picard","description":"A set of command line tools (in Java) for manipulating high-throughput sequencing (HTS) data and formats such as SAM/BAM/CRAM and VCF.","fork":false,"url":"https://api.github.com/repos/broadinstitute/picard","forks_url":"https://api.github.com/repos/broadinstitute/picard/forks","keys_url":"https://api.github.com/repos/broadinstitute/picard/keys{/key_id}","collaborators_url":"https://api.github.com/repos/broadinstitute/picard/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/broadinstitute/picard/teams","hooks_url":"https://api.github.com/repos/broadinstitute/picard/hooks","issue_events_url":"https://api.github.com/repos/broadinstitute/picard/issues/events{/number}","events_url":"https://api.github.com/repos/broadinstitute/picard/events","assignees_url":"https://api.github.com/repos/broadinstitute/picard/assignees{/user}","branches_url":"https://api.github.com/repos/broadinstitute/picard/branches{/branch}","tags_url":"https://api.github.com/repos/broadinstitute/picard/tags","blobs_url":"https://api.github.com/repos/broadinstitute/picard/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/broadinstitute/picard/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/broadinstitute/picard/git/refs{/sha}","trees_url":"https://api.github.com/repos/broadinstitute/picard/git/trees{/sha}","statuses_url":"https://api.github.com/repos/broadinstitute/picard/statuses/{sha}","languages_url":"https://api.github.com/repos/broadinstitute/picard/languages","stargazers_url":"https://api.github.com/repos/broadinstitute/picard/stargazers","contributors_url":"https://api.github.com/repos/broadinstitute/picard/contributors","subscribers_url":"https://api.github.com/repos/broadinstitute/picard/subscribers","subscription_url":"https://api.github.com/repos/broadinstitute/picard/subscription","commits_url":"https://api.github.com/repos/broadinstitute/picard/commits{/sha}","git_commits_url":"https://api.github.com/repos/broadinstitute/picard/git/commits{/sha}","comments_url":"https://api.github.com/repos/broadinstitute/picard/comments{/number}","issue_comment_url":"https://api.github.com/repos/broadinstitute/picard/issues/comments{/number}","contents_url":"https://api.github.com/repos/broadinstitute/picard/contents/{+path}","compare_url":"https://api.github.com/repos/broadinstitute/picard/compare/{base}...{head}","merges_url":"https://api.github.com/repos/broadinstitute/picard/merges","archive_url":"https://api.github.com/repos/broadinstitute/picard/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/broadinstitute/picard/downloads","issues_url":"https://api.github.com/repos/broadinstitute/picard/issues{/number}","pulls_url":"https://api.github.com/repos/broadinstitute/picard/pulls{/number}","milestones_url":"https://api.github.com/repos/broadinstitute/picard/milestones{/number}","notifications_url":"https://api.github.com/repos/broadinstitute/picard/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/broadinstitute/picard/labels{/name}","releases_url":"https://api.github.com/repos/broadinstitute/picard/releases{/id}","deployments_url":"https://api.github.com/repos/broadinstitute/picard/deployments","created_at":"2014-03-28T20:43:35Z","updated_at":"2017-10-16T15:26:56Z","pushed_at":"2017-10-16T19:15:00Z","git_url":"git://github.com/broadinstitute/picard.git","ssh_url":"git@github.com:broadinstitute/picard.git","clone_url":"https://github.com/broadinstitute/picard.git","svn_url":"https://github.com/broadinstitute/picard","homepage":"http://broadinstitute.github.io/picard/","size":113474,"stargazers_count":315,"watchers_count":315,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":214,"mirror_url":null,"open_issues_count":104,"forks":214,"open_issues":104,"watchers":315,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/941"},"html":{"href":"https://github.com/broadinstitute/picard/pull/941"},"issue":{"href":"https://api.github.com/repos/broadinstitute/picard/issues/941"},"comments":{"href":"https://api.github.com/repos/broadinstitute/picard/issues/941/comments"},"review_comments":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/941/comments"},"review_comment":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/broadinstitute/picard/pulls/941/commits"},"statuses":{"href":"https://api.github.com/repos/broadinstitute/picard/statuses/2f6926070da01a4e7f38bcab1a5db5c8d009a12f"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"yfarjoun"}}