{'fraction_non_alphanumeric': 0.026672798344447, 'fraction_numerical': 0.019314785008047827, 'mean_word_length': 5.387665198237886, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The American Board of Vascular Medicine Endovascular Board Examination – Raising the bar by testing the test

Over the past several decades, efforts to measure, publicly report, and reward physician performance have gained increasing importance. However, currently available metrics to assess physician quality and clinical performance are far from ideal.1 Board certification was designed as one such measure, to provide an overall assessment of physician competence.2 Certification by a medical specialty board is meant to indicate that a physician has the knowledge, experience, and skills for providing quality health care within a given specialty. Yet, data supporting the association between board certification status and provision of superior quality of care are limited and somewhat controversial.3–6 Therefore, it is important to critically evaluate the content of board certification exams on an ongoing basis to ensure that it is not only current, but also directly relevant to the care to be provided by the group of physicians who will be taking the exam. In 2002, the Society of Vascular Medicine (SVM) conducted a survey of its members to determine how interested they would be in obtaining certification in vascular medicine.7 Among those who responded, 95% voted in favor of being recognized as a vascular medicine specialist through board certification.7 This led to the inception of the American Board of Vascular Medicine (ABVM) in 2003 as a collaborative effort of the SVM, American College of Cardiology, and Society for Cardiovascular Angiography and Interventions. Between 2003 and 2005, volunteer subject matter experts (SMEs) in the fields of cardiology, vascular medicine, radiology, and vascular surgery convened to develop an outline for the ABVM examinations.7 A comprehensive test blueprint was created by the SMEs from which test items were developed and reviewed, and ultimately 175 and 100 items were selected for the general examination and endovascular examination, respectively. The first ABVM certification exam was offered in 2005. In 2014, the number of examination questions was changed to 150 items on the general examination and 135 items (110 scored and 25 experimental items) on the endovascular examination.7 The ABVM examination items and the overall exam itself have consistently demonstrated satisfactory performance characteristics (as reflected by the p-value, point-biserial correlation for each item, and Cronbach’s alpha reliability coefficient, Livingston decision consistency index for the overall exam).8 Notwithstanding the above positive attributes, the proportion of items based on multidisciplinary guideline recommendations and/or evidence-based data has not been evaluated. To address this question, in this issue of Vascular Medicine, Slovut and colleagues9 examined the Class of Recommendation (COR) and Level of Evidence (LOE) for each of the 110 scored items in the 2015 ABVM endovascular examination in an effort to establish a new evidencebased metric for improving the quality of certification examinations. In addition to the traditional COR I–III and LOE A–C, items that were not eligible for assignment using the traditional evidence-based metrics were categorized as COR X and LOE X.10 The proportion of items assigned to each COR category was: COR I=15%, COR II=40%, COR III=3%, COR X=42%. Similarly, the proportion of items assigned to each LOE was: Level A=12%, Level B=34%, Level C=32%, LOE X=22%.9 The authors concluded that ‘more than half of the 2015 ABVM endovascular board examination items were supported by strong scientific evidence or fact-based knowledge’, and that ‘the use of alternate classification schema, such as ones based on evidence-based metrics (COR and LOE), may be powerful tools for improving certification exams in healthcare’.9 Although the article by Slovut and colleagues9 represents an important contribution to the literature, there are certain limitations that need mention. First, the use of a pooled consensus approach to categorize items can certainly be a source of significant bias. A more sound approach would be to have two or more experts categorize the questions and then measure inter-rater agreement using Cohen’s kappa The American Board of Vascular Medicine Endovascular Board Examination – Raising the bar by testing the test