{'fraction_non_alphanumeric': 0.020587377933245884, 'fraction_numerical': 0.006230870135548754, 'mean_word_length': 5.488179669030733, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Exploring the impact of the United States-based accreditation council for pharmacy education certification on pharmacy education outside the United States

Highlights • ACPE certification appears to have positively impacted certified international pharmacy programs.• ACPE certification was particularily helpful in promoting assessment, preceptor quality and development.• ACPE certification aided in improving teaching and learning methods in both didactic and experiential curricula, and in developing an inclusive feedback system to receive input from various stakeholders.


Introduction
International Services Program (ISP) at the Accreditation Council for Pharmacy Education (ACPE) was established in 2011 to evaluate and certify pharmacy degree programs that are based outside the United States (US) and its territories (ACPE, 2011). Since its inception, the value of ACPE certification has become increasingly recognized, not only by the academic and support staff of the certified programs but also by their national educational systems (Alkhateeb et al., 2018). To prepare more competitive graduates in the global market, many higher education institutions seek to internationalize their educational systems (Chan, 2013). For example, as of July 2020, 15 colleges and schools of pharmacy (CS Pharmacy) located outside the US have attained the US-based ACPE certification (ACPE, 2011;Vlasses et al., 2020). A list of these CS Pharmacy is presented in Table 1.
In addition to certification, the ACPE offers precertification, for a limited period of time, to newly instituted international CS Pharmacy prior to graduating its first class to ensure that the new pro-gram is progressing well towards full certification. The ACPE Board generates, adopts, and adjusts criteria, policies, and procedures that apply to the certification and precertification processes (ACPE, 2011). Similar to the ACPE accreditation process for US based CS Pharmacy programs, the ACPE certification process for international pharmacy programs provides quality assurance that the programs meet the 26 quality criteria, which are comparable to the 2016 ACPE 25 standards utilized for US pharmacy programs (ACPE, 2011). The 26 quality criteria are the same for international Bachelor of Science as well as Doctor of Pharmacy (PharmD) programs. These quality criteria are published by the ACPE and have been generated by gathering input from international experts in pharmacy education as well as from a series of quality assurance guidelines from the global community (ACPE Certification Crietria, 2016).
International pharmacy programs can apply for ACPE certification if they meet the eligibility criteria that are published by the ISP (ACPE, 2011;Alkhateeb et al., 2018). The ISP oversees a peerreview process, which includes a series of reviews by a team of faculty experts, including the International Commission (IC) and the ACPE Board, and schedules a site-evaluation visit to further examine the applicant's institution. The goal of this visit is to verify that the written self-study report and the program's characteristics meet the certification criteria. The self-study report and corresponding documents are reviewed by the IC that is comprised of faculty experts from US and international pharmacy programs. A final decision regarding the certification (or precertification) status is determined by the ACPE Board during one of their biannual meetings based on comprehensive reviews, focused on-site evaluations, and relevant communications with representatives from the international CS Pharmacy (Vlasses et al., 2020). With more international pharmacy programs striving to enhance the quality of their programs, questions arise concerning the impact of the ISP self-study process and its outcomes on pharmacy education, as well as the impact of certification on the programmatic operation, structure, finances, and outcomes.
To our knowledge, there has not been any previous research to record and assess the feedback from certified international pharmacy programs on the outcomes of ACPE certification. Investigating the reported impact of certification on certified programs will provide more guidance to international pharmacy programs interested in achieving the ACPE certification in the future. In addition, this investigation provides information to the global pharmacy community about how the certification process has assisted the certified programs to enhance their overall quality. Therefore, this study aimed to investigate the reported impact, and explore the perceived outcomes, of the ACPE certification of international pharmacy programs on pharmacy education, programmatic operation, finances, and student learning outcomes among pharmacy programs that have attained the ACPE certification as of July 2020. Considering the growing number of pharmacy programs outside the US seeking ACPE-certification, we hypothesized that the ACPE certification has a positive impact on the financial resources, operation, and structure of the certified programs.

Design
This research was based on structured, standardized interviews (Merriam & Tisdell, 2009). Data collection relied on a crosssectional questionnaire, with a series of predetermined questions that included both quantitative and qualitative items. The questionnaire items assessed the perceived impact of ACPE certification on certified CS Pharmacy. We sought assistance from one of the investigators' Office of Educational Research and Statistics to ensure that the questions were generated based on sound research study and to collect reliable information for the intended objectives.

Participants and procedures
Based on data from the ACPE, a total of 15 CS Pharmacy outside the US and its territories offered ACPE-certified program or programs as of July 2020. We received the list of contact administrators for the certified CS Pharmacy from the ISP office at the ACPE. An administrator from each of these fully certified CS Pharmacy was invited by email to participate in the study. Three weeks later, a reminder was sent to individuals who did not respond to the first email invitation. Individuals who expressed willingness to participate signed an informed consent form. Subsequently, online live interviews were scheduled, and a copy of the questionnaire was shared with the participants prior to the scheduled interview. The interviews were conducted via Zoom Ò . In addition to responding to the questionnaire items, interviewees were given the opportunity to share any thoughts and concerns regarding the impact of ACPE certification on their programs.
The Office of Research Integrity at one of the researchers' academic institutions reviewed the research and determined it to be exempt. Interviews were conducted from May to July 2020 and lasted for approximately 45 min each. Each online interview included at least 2 of the researchers, where one was primarily conversing with the interviewee while the other was taking notes and recording responses.

Instrument
The questionnaire included three major sections (the questionnaire is available from the authors upon request). The first section included 11 items and assessed the impact of ACPE certification on financial resources. This information was collected because CS Pharmacy programs may have to allocate more funds that bring certain units or processes to meet ISP criteria. Participants were presented with a series of areas/units that could have been affected by the ACPE certification. The impact of ACPE certification on financial resources was shown by presenting the frequency of affirmative responses on each item in this section. One open-ended question assessed whether the increased funds impacted other areas in their programs that were not included in the addressed items. The second section consisted of 30 items and assessed the impact of the ACPE certification on the operation and structure of the certified program. These items addressed several programmatic and curricular indicators, including promotion and tenure guidelines, scholarship output, collaboration with professional organizations, faculty retention and evaluation, engagement with advisory boards and other stakeholders, and the impact of assessment. Each of these items was assessed with a 5-point Likert scale that ranged from ''Strongly Agree' to 'Strongly Disagree." Subsequently, participants were offered to comment on the items for which they indicated that the ACPE certification did not improve the operation and/or structure of their program.
The last section addressed the characteristics of the certified programs, including the interviewees' role in their programs, their national accreditation in their countries, the type of offered programs in the schools, as well as the numbers of faculty, staff, and students. The basic characteristics of the participating CS Pharmacy were described by presenting the frequency of responses to specific demographic questions. Finally, interviewees' open responses from the first two sections were transcribed and reviewed. The comments were grouped into themes, depending on the content of the open responses.

Statistical analysis
All data, including qualitative and quantitative responses, were transcribed into electronic files to facilitate data analysis. Quantitative data was described by presenting frequencies, means, and standard deviations for items measured on a Likert scale, as well as frequencies for items measured on a dichotomous scale (Yes, No). On the other hand, qualitative data was reviewed for the purpose of categorizing different thoughts and comments into separate codes, each code representing an independent idea or construct. Then, the codes were grouped into themes based on recurring ideas or identified patterns. Then, the themes were presented in a cohesive manner that describes participants' overall thoughts on the impact of ACPE certification on their programs. Finally, atypical responses were identified to point out any deviation from the common themes.

Sample characteristics
Administrators from 13 ACPE-certified CS Pharmacy participated in this study (87% respondent rate). Five CS Pharmacy were based in public universities, whereas 8 CS Pharmacy were based in private universities. The 13 participants included 5 Deans and 8 other administrators. All participants were selected based on the active engagement and leadership roles they played during their certification self-study process. All ACPE-certified programs from the 13 CS Pharmacy were accredited by national accrediting agencies in their origin countries. Of the CS Pharmacy programs that were included in the study, six CS Pharmacy offered certified Bachelor's Degree programs, four CS Pharmacy offered PharmD degree programs, and three CS Pharmacy offered both degree programs. Additionally, some of these programs offered M.S., PhD, PGY1, PGY2, and other types of postgraduate training programs. The CS Pharmacy varied widely in age (median = 29 years), as well as in faculty, staff, and student total numbers (medians were 48, 28, 622 for faculty, staff, and students in all program years, respectively). Table 2 presents the number of participants indicating increased operational funding for several units or projects as a result of the gained ACPE certification. Despite resulting in increased availability of funds for educational resources in classrooms, technology resources, creating a simulation laboratory, and library resources for many of the certified programs, the ACPE certification reportedly had no or limited impact on faculty salaries, funds for faculty to attend academic and pharmacy conferences, extramural funding, as well as creating a model community pharmacy.

The impact of ACPE certification on financial resources
In terms of open-response items, 2 themes were identified: fixed salaries and already well-developed and funded items. The interviewees reported that oftentimes allocation of funds is set according to government-mandated guidelines and regulations. In fact, some participants mentioned that budgets, salaries, and faculty promotion guidelines are strictly decided and distributed at a governmental level. One of the participants stated that ''There is no financial independence. Budgets and salaries are decided at a governmental level. For example, salaries depend on rank and citizenship. Two individuals with the same citizenship and rank will have the same exact salary." In addition, some participants thought the limited impact of ACPE certification on operational funding was attributed to already developed and well-funded units (e.g., a model community pharmacy, a simulation lab, or a computer lab) prior to the ACPE certification. Nonetheless, few participants acknowledged the increased funding to improve these already created units in their CS Pharmacy.

The impact of ACPE certification on operation and structure
Frequencies, standards deviations, as well as average scores for individual items in this section are presented in Table 3. There was a great variability among participant responses on the impact of ACPE certification on the operation and structure of their programs. For example, participants reported high agreement on the impact of ACPE certification on increasing awareness of how assessment aids quality improvement, evidenced by this item having the highest agreement average score (4.69 on a scale from 1 to 5) and the lowest standard deviation (SD = 0.48) as shown in Table 3. However, some participants mentioned that faculty's workload increased due to faculty members' active involvement in preparing the self-study and contributing to ACPE certification process while continuing to manage their normal academic workload (i.e., meeting their regular teaching, research, and service expectations). Finally, the impact of ACPE certification was limited in areas where schools have little or no ability to change, including what is decided at the local or governmental levels, such as teaching load and faculty promotion and tenure guidelines.
Open responses from this section fell into four primary themes. First, despite being optimistic about the long-term impact of the ACPE certification on the competency of their graduates, the majority of participants were unsure yet on the benefits of the certification on their national licensure examinations pass rates. For example, a participant stated how it is difficult to understand the impact of the ACPE certification due to multiple changes taking place around the same time: There was a new management around the same time the ACPE certification was obtained. Therefore, it is not clear whether the positive changes were precisely caused by the ACPE certification. Also, some of these questions are too early to answer. We had certification for only 2 years, which is not long enough these great long-term benefits. We expect to achieve these long-term benefits in the future.
Some participants believed that their national licensure exams were typically easy to pass, and therefore, they did not think that the ACPE certification could significantly influence the pass rates. Other participants were unsure because their national licensure exam was recently introduced (they did not have a licensure exam in their country in the past), or because of the short period of time since attaining the certification as a reason for not realizing the impact of certification on this area yet.
The second theme was related to collaboration and partnership with different units within the same CS Pharmacy or the university. Participants mentioned that the ACPE certification assisted them in having stronger inter-departmental collaboration within the certified program, better collaboration with other health care colleges and schools within the same university to better design and organize interprofessional education (IPE) activities, and more diverse partnerships with pharmaceutical companies and local institutions, which facilitated placing students into unique experiential learning opportunities. For instance, a participant said: ''The ACPE certification helped improve collaboration and establish stronger relationship between different departments in the school (practice, basic sciences, and experiential)." The third theme was related to assessment as participants spoke highly on the impact of ACPE certification on assessment processes within their CS Pharmacy. One participant mentioned that ''the certification also emphasized and established a culture of assessment through creating a comprehensive assessment system." The fourth theme was related to teaching and scholarship load. Some participants mentioned that teaching load, as well as promotion and tenure guidelines, are decided at a governmental level and that the certification had little or no impact on these issues. Some participants mentioned that their CS Pharmacy programs were already doing well in certain aspects, such as research and publication output. Therefore, the high scholarship productivity within their programs was a result of their own program effort that predated the ACPE certification. It is noteworthy that participants reported highly variable views on the impact of ACPE certification on scholarship output, evidenced by this item having the highest standard deviation (SD = 1.05) as shown in Table 3.

Discussion
This study aimed to explore the impact of ACPE certification on certified pharmacy programs outside the US. Generally, participants shared their perceptions and reported highly positive impact, little or no impact, as well as some negative impact of ACPE certification on certain areas or units within their programs. In terms of the positive impact, many programs mentioned the certification-associated benefits related to managing assessment processes and data collection practices. Many of the pharmacy programs seem to struggle with this area before certification, and therefore, needed to make significant changes to how data is collected, measured, stored, and retrieved within their programs and universities. A few areas have been highly affected by the ACPE certification as indicated by an average score of 4.5 or more as shown in Table 3. These areas included assessing preceptor quality and development, improving teaching and learning methods in the didactic and experiential curricula, and developing an inclusive feedback system to receive input from various stakeholders. Since the impact on the above areas was highest among the factors mentioned in Table 3, one would predict international CS Pharmacy need to pay close attention to these areas during the certification's self-study process.
In terms of areas with little or no change as a result of certification, financial resources were the most prominent examples. The collected data revealed that ACPE certification had a limited impact on financial resources among the certified programs. Reportedly, governmental policies or local regulations confined what a program can change, and therefore, the ACPE certification was of little value in this context. This observation provides important information for the ACPE-ISP that if areas listed in Table 2 are critical for quality assurance, they need to be included in the quality criteria when a future revision of quality criteria is warranted. Other examples included budgetary issues as well as tenure and promotion guidelines, which again could not be decided at the CS Pharmacy level. Nevertheless, most participants thought that the ACPE certification helped them function in a more organized manner, where data collection, quality control, and assessment processes are more valued and realized. In addition, it made them aware of all the updates and best practices that take place in the world of pharmacy education around the globe.
Some participants mentioned that seeking the ACPE certification was encouraged by their national pharmacy accrediting agency, which is part of their government body. Indeed, many of the national accreditation criteria and standards match the ISP quality criteria (Sales et al., 2019). This reflects the ACPE-ISP's successful efforts that have been accomplished by working with the local governmental accreditation institutions. In a study conducted by Jorgenson et al, it was demonstrated that delivery of cultural competency activities improved knowledge, attitudes and confidence related to cultural competency among pharmacy students who studied at Qatar University in Qatar and the University of Saskatchewan in Canada (Jorgenson et al., 2016). The results indicate how important it is to integrate cultural competency in pharmacy curricula. Many of the ACPE certified CS Pharmacy admit international students with culturally and ethnically different backgrounds and, as a result, they have a large diverse student body at their programs. There is a lack of addressing and assessing cultural competency in the ACPE certification quality criteria. While we did not generate any questions in our study to assess this area, we believe it is important to include a criterion related to cultural competency or cultural sensitivity in the future revision of the quality criteria.
The negative thoughts on ACPE certification were related to two topics. First, preparing and applying for certification was a timeconsuming process, and it resulted in an increased workload for the CS Pharmacy employees. Some participants believed that achieving the ACPE certification was a time-consuming process, and therefore, it caused an increased workload for everyone at their programs. To help mitigate the challenges, participants mentioned the importance of starting data collection early in the certification process, seeking guidance from programs that have already been through the process to learn more on the best practices, or hiring consultants who oversee the entire certification process.
Pharmacy programs considering applying for ACPE certification are encouraged to prepare well in advance and allocate enough resources, especially administrators, faculty, and staff time towards achieving this goal. The second topic that was negatively perceived by participants was certification as compared to accreditation. Based on the participants' comments, they were willing to do the extra work and take the additional steps that allow their programs to be accredited and not just certified. It is worth mentioning that the term 'certification' is only used by the ACPE-ISP because many comparable international accreditation agencies use 'accreditation' or 'international accreditation' terminology.
The study findings should be interpreted in light of the fact that only certification administrators from certified programs were interviewed in this study. Therefore, certified CS Pharmacy faculty may have different perceptions that were not captured in, and were not within the scope of, this study. Additionally, the study results provided a snapshot in time on the processes that were implemented as a result of ACPE certification. Repeated studies are required to survey newly certified programs in the future, and to evaluate the sustainability and evolution of the ACPE certified CS Pharmacy, particularly in terms of long-term impact to those factors mentioned in Tables 2 and 3. Globally, the pharmacist role in providing direct patient care is expanding beyond the basic product-oriented roles of dispensing prescriptions, particularly with an increased interest in clinical pharmacy (Alsharif, 2017). Many international CS Pharmacy are witnessing the imperative roles that US pharmacists play in the Triple Aim of healthcare by reducing the healthcare costs, improving the health of populations, and creating a positive experience for patients (Berwick et al., 2008). As the world continues to fight global pandemics, including the devastating impacts from the COVID-19 pandemic on public health, it becomes more evident that pharmacy education and pharmacists can play a critical role in the primary health care strategies (Pinto et al., 2021). Because of pharmacists' unique expertise and accessibility, enhancing the quality of international pharmacy programs will have a lasting impact on global health care (Wilby et al., 2019). It has been suggested that there are no globally agreed standards for pharmacy education, and that identifying consistent global standards can support and provide consistent patient care globally (Micallef and Kayyali, 2020). The engagement of ACPE-ISP with the international CS Pharmacy accreditation agencies can be instrumental in achieving overarching and consistent expectations for pharmacy education and training globally.
In conclusion, participants had a somewhat positive overall view regarding the ACPE certification despite the difference in settings, locations, and needs of certified programs. Some of the CS Pharmacy programs were confined to limited changes in certain financial aspects and faculty promotion due to governmental policies and regulatory guidelines. Otherwise, the certification has assisted pharmacy programs in increasing the awareness of what assessment brings to the program for quality improvement, assessing preceptor quality and development, improving teaching and learning methods in both didactic and experiential curricula, and developing an inclusive feedback system to receive input from various stakeholders. In addition, updating the review process and criteria in order to allow for international pharmacy program accreditation, and not just certification, may encourage more international CS Pharmacy programs to approach the ACPE for guidance and help to enhance the quality of their programs, especially if it can coordinate its criteria with national accrediting agencies in foreign countries.

Funding/Support
This research has not been supported by any grants/funds.

Contribution to the literature
Because there are no globally accepted standards for pharmacy education, and because identifying shared global standards can support and provide a consistent patient care process worldwide, this study aimed to shed light on how the ACPE certification criteria impact the operation and structure of international pharmacy programs. All in all, the findings of this study provide guidance to international pharmacy programs that are interested in applying for the ACPE certification in the future. Additionally, the results can be of interest to the ACPE's International Services Program (ISP) to adjust certification's criteria that aim to enhance the quality of the evaluation process. To our knowledge, this will be the first research study to explore and report the impact of ACPE certification on international colleges and schools of pharmacy since the ACPE's ISP was established in 2011. The engagement of ACPE-ISP with the international Colleges and Schools of Pharmacy accreditation agencies can be instrumental to achieve overarching and consistent expectations for pharmacy education and training globally.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Reza Karimi serves on ACPE Board of Directors and the presented information does not represent ACPE or the boards' opinions or views.