{'fraction_non_alphanumeric': 0.033894259458169235, 'fraction_numerical': 0.015843179377013964, 'mean_word_length': 5.367211246200608, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 8, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Specifying who delivers behaviour change interventions: development of an Intervention Source Ontology

Background: Identifying how behaviour change interventions are delivered, including by whom, is key to understanding intervention effectiveness. However, information about who delivers interventions is reported inconsistently in intervention evaluations, limiting communication and knowledge accumulation. This paper reports a method for consistent reporting: The Intervention Source Ontology. This forms one part of the Behaviour Change Intervention Ontology, which aims to cover all aspects of behaviour change interventions . Methods: The Intervention Source Ontology was developed following methods for ontology development and maintenance used in the Human Behaviour-Change Project, with seven key steps: 1) define the scope of the ontology, 2) identify key entities and develop their preliminary definitions by reviewing existing classification systems (top-down) and reviewing 100 behaviour change intervention reports (bottom-up), 3) refine the ontology by piloting the preliminary ontology on 100 reports, 4) stakeholder review by 34 behavioural science and public health experts, 5) inter-rater reliability testing of annotating intervention reports using the ontology, 6) specify ontological relationships between entities and 7) disseminate and maintain the Intervention Source Ontology. Results: The Intervention Source Ontology consists of 140 entities. Key areas of the ontology include Occupational Role of Source, Relatedness between Person Source and the Target Population, Sociodemographic attributes and Expertise. Inter-rater reliability was found to be 0.60 for those familiar with the ontology and 0.59 for those unfamiliar with it, levels of agreement considered ‘acceptable’. Conclusions: Information about who delivers behaviour change interventions can be reliably specified using the Intervention Source Ontology. For human-delivered interventions, the ontology can be used to classify source characteristics in existing behaviour change reports and enable clearer specification of intervention sources in reporting.


Introduction
Behaviour change interventions vary greatly in the manner they are delivered. Understanding and reporting the delivery of an intervention is essential to understanding its effectiveness, the mechanisms of change and reasons for variation in effectiveness (Dombrowski et al., 2016). To enable implementation of effective behaviour change interventions and replication of interventions in scientific enquiry, it is essential to provide clear and well-specified reporting of all aspects of interventions, including delivery Michie et al., 2020). The delivery of an intervention includes three core components: i) mode of delivery, i.e. how the intervention content is provided to participants such as via a website or face-to-face counselling (Dombrowski et al., 2016;Marques et al., 2020), ii) schedule, i.e. how intervention content is provided to participants over the course of an intervention and iii) source, i.e. who (individually or collectively as a group or organisation) delivers the intervention content (Michie et al., 2021). In this paper, we consider intervention source.
Pre-existing relationships between an intervention's source and participants may have an impact on intervention effectiveness. For example, peer-based interventions for health behaviours in adults may lead to greater effects than staff-delivered interventions (Cabassa et al., 2017;Webel et al., 2010). Interventions are often developed to be delivered by a specific source, such as delivery of smoking cessation interventions to patients by nurses (Rice et al., 2017), and their effectiveness may not generalise to delivery by other sources (Glasgow et al., 2003). Those delivering the intervention may require certain levels of knowledge and skills, or to have undergone intervention-specific training.
Guidelines aim to improve the quality of research reporting in terms of consistency, specificity and comprehensiveness. However, the most widely used of these, the CONsolidated Standards of Reporting Trials statement (CONSORT;Schulz et al., 2010) and its extension for social and psychology interventions (CONSORT-SPI; Montgomery et al., 2018) do not specify reporting who delivers interventions. The Template for Intervention Description and Replication checklist (TIDieR; Hoffmann et al., 2014) includes one item (Item 5: "Who provided -For each category of intervention provider (e.g. psychologist, nursing assistant), describe their expertise, background and any specific training given") but this is not further elaborated. The reporting of source characteristics within intervention reports using these brief guidelines is often poor. For example, specification of intervention providers was found to be one of four areas requiring improvement in obesity interventions (Rauh et al., 2021). We currently lack a method for specifying characteristics of an intervention's source within behaviour change interventions.
An appropriate method for specifying and classifying interventions is an ontology: a data structure that enables precise specification of knowledge in a given domain (Arp et al., 2015) (see glossary of italicised terms in Table 1). Ontologies provide a set of: i) unique and unambiguous identifiers representing types of entity (objects, attributes and/or processes), ii) labels and definitions corresponding to these identifiers, and ii) specified relationships between the entities (Arp et al., 2015;Larsen et al., 2017;Norris et al., 2019). These labels, definitions and relationships provide formal specification and a 'controlled vocabulary' for the given domain. Ontologies are dynamic representations that are maintained and updated according to new evidence in the given field (Arp et al., 2015;, in fields such as molecular biology (Ashburner et al., 2000). Ontologies are also machine-readable, hence suitable for annotation, evidence synthesis and other computational applications (Hastings, 2017).
No ontology currently exists to describe the full detail of behaviour change intervention sources (Norris et al., 2019). A comprehensive Behaviour Change Intervention Ontology (BCIO) is being developed as part of the Human Behaviour-Change Project Michie et al., 2020). The BCIO consists of an upper level with 42 entities, one of which is Behaviour change intervention source, specified as part of the Delivery in a given behaviour change intervention (BCI) scenario (Michie et al., 2021). The current study used the ontology development methodology specified for the Human-Behaviour Change Project (Wright et al., 2020) to develop a reliable ontology for specifying and classifying who delivers interventions. This paper reports the development and final version of the Intervention Source Ontology.

Methods
The Intervention Source Ontology was developed in an iterative process of seven steps (Wright et al., 2020).
Step 1 -Development of the scope and definition of the Intervention Source Ontology A definition and overall topic for the ontology was set by reviewing dictionaries and the TIDieR reporting guidelines (Hoffmann et al., 2014)

(March 2018).
Step 2 -Identifying key entities and developing the preliminary Intervention Source Ontology An initial prototype version of the ontology was developed using both a bottom-up and top-down approach. In the bottom-up approach, 100 published reports of behaviour change interventions were reviewed to develop an initial list of source characteristics.This number of reports was chosen to generate a good initial range of entities for inclusion in the ontology (Wright et al., 2020). These reports were part of a large dataset of behaviour change intervention reports partially coded for behaviour change techniques, mechanisms of action, and modes of delivery, covering a range of health behaviours (Carey et al., 2019;Michie et al., 2015).
In the top-down approach, existing classification systems related to intervention source characteristics were reviewed, including: i) occupational classification systems (e.g. International Standard Classification of Occupations (ISCO-08; Ganzeboom & Treiman, 1996;International Labour Office, 2012), ii) published ontologies containing terms related to behaviour

Source Annotation
Process of coding selected parts of documents or other resources to identify the presence of ontology entities  Annotation guidance manual Written guidance on how to identify and tag pieces of text from intervention evaluation reports with specific codes relating to entities in the ontology, using EPPI-Reviewer software.

Basic Formal Ontology (BFO) An upper level ontology consisting of continuants and occurrents
developed to support integration, especially of data obtained through scientific research. Arp et al., 2015.

Continuant
Entities within an ontology that continue to exist over time, for example, objects and spatial regions. Arp et al., 2015.

Entity
Anything that exists, that can be a continuant or an occurrent as defined in the Basic Formal Ontology. Arp et al., 2015.

EPPI-Reviewer
A web-based software program for managing and analysing data in all types of systematic review (meta-analysis, framework synthesis, thematic synthesis etc. It manages references, stores PDF files and facilitates qualitative and quantitative analyses such as metaanalysis and thematic synthesis. It also has a facilitate to annotate published papers. http://www.obofoundry.org/principles/fp-000-summary.html

Occurrent
Entities within an ontology that extend over time, for example, processes. Arp et al., 2015.

Source
Ontology A standardised representational framework providing a set of terms for the consistent description (or "annotation" or "tagging") of data and information across disciplinary and research community boundaries. Arp et al., 2015.

Parent class
A class within an ontology that is hierarchically related to one or more child (subsumed) classes such that all members of the child class are also members of the parent class and all properties of the parent class are also properties of the child class. Arp et al., 2015.

ROBOT
An automated command line tool for ontology workflows. Step 3 -Refining the ontology through literature annotation, discussion and revision The preliminary ontology was revised by the research team based on annotations of published intervention evaluation reports. Using EPPI-Reviewer 4 software (Thomas & Brunton, 2010), two researchers independently annotated 120 behaviour change intervention evaluation reports using the revised source ontology, with 80 reports on smoking cessation and 40 on physical activity interventions. This number of reports was used as no more additional entities were apparent for inclusion in the ontology (Wright et al., 2020). An open alternative to this software used for annotation is PDFAnno (Shindo et al., 2018). Information on how and when to annotate each source characteristic was provided in an annotation guidance manual to promote standardisation of the procedure. Discrepancies were discussed and the ontology structure, definitions and annotation guidance were revised accordingly. All reports were randomised controlled trials from one of three datasets: Cochrane reviews, papers annotated for behaviour change techniques and papers already annotated for Behaviour Change Techniques within the ICSmoke systematic review project (Black et al., 2020;de Bruin et al., 2016)(List of papers used in development of ontology: https://osf.io/6djfk/) (June -October 2018).
Step 4 -Expert stakeholder review A total of 123 members of an existing international panel of behavioural scientists and public health expert stakeholders at UCL's Centre for Behaviour Change (Wright et al., 2020) were invited to give feedback on the Intervention Source Ontology resulting from Step 3. These invited experts comprised i) 90 behavioural scientists who had provided feedback on previous projects at the Centre for Behaviour Change, ii) 20 experts from under-represented countries (i.e. not USA, Australis, Canada, UK and Netherlands: the five most commonly represented countries) identified through the BCTTv1 database, and iii) 13 stakeholders who expressed interest in being involved in Human Behaviour-Change Project stakeholder initiatives in response to invitations on Twitter and the project newsletter.
Invitations to participate were sent via email, with feedback collected through an online questionnaire using Qualtrics TM software (full survey is provided as extended data (West et al., 2020)). The task was designed to take no longer than 45 minutes to complete. The task asked experts to: 1. identify the characteristics of an intervention's source that were of interest to them when trying to understand variation in the effectiveness of behaviour change interventions (open-ended question). Experts were advised to consider a specific behaviour when answering this question e.g. 'physical activity'.
2. provide feedback on the completeness and comprehensiveness of the Intervention Source Ontology.
Experts were asked to indicate: i) if there were any entities missing (if yes, which should be added), ii) if there were any entities of definitions that should be changed (if yes, what changes should be considered), and iii) if there were any entities that should be placed in a different location in the Intervention Source Ontology.
A thematic analysis of the qualitative responses was done by two researchers (EN & AW), with the larger study team providing feedback on the analysis of the results. Themes were derived directly from responses. Feedback from this expert consultation was discussed by the research team and the Intervention Source Ontology and annotation guidance were revised. Experts did not provide further feedback on these revisions (November 2018 -March 2019).
Step 5 -Inter-rater reliability of annotations using the Intervention Source Ontology Assessment of inter-rater reliability of the annotations by two researchers leading the development of the ontology was conducted using 50 papers from Cochrane reviews (25 for smoking cessation and 25 for physical activity). This number of papers was selected as 50 papers would give a 10-15% margin of error around the estimated percentage agreement between coders (Gwet, 2014; Wright et al., 2020). Inter-rater reliability was also assessed for annotations by two behaviour change experts unfamiliar with the ontology but with experience in annotating behaviour change intervention reports. Inter-rater reliability with experts unfamiliar with the ontology was performed in two stages: i) a random sample of 50 randomised controlled trials from a database of papers coded by Behaviour Change Techniques, with no restrictions on the outcome behaviour, with revisions to the ontology made accordingly, ii) another random sample of 30 randomised controlled trials with no restrictions on the outcome behaviour from the same database. Inter-rater reliability was assessed using Krippendorff's Alpha (Hayes & Krippendorrff, 2007)  Step 6 -Specifying relationships between Intervention Source Ontology entities The research team established relationships between ontology entities to formalise the knowledge present in the ontology. This process was conducted in line with Basic Formal Ontology principles which have been used extensively in biomedical ontologies (Arp et al., 2015). The suitability of common relationships from Basic Formal Ontology (Arp et al., 2015), the Relation Ontology (Smith et al., 2005) and the Information Artifact Ontology (Ceusters, 2012) were assessed, including the basic hierarchical relationship 'is_a' which holds between classes where one class is a subclass of another and 'is_about' which holds between a class representing an informationcontaining entity and the class that represents what the information is about (January 2021).
Step 7 -Disseminating and maintaining the Intervention Source Ontology The Intervention Source Ontology was initially developed as a table of entities, with separate rows for each entity annotated with a primary label, definition, synonyms, examples and relationships. When the Intervention Source Ontology was at a stable level of development for initial release, it was converted into Web Ontology Language (OWL) (Antoniou & Van Harmelen, 2004) format, enabling it to be viewed and visualised using ontology software such as Protégé and to be compatible with other ontologies. The conversion to OWL used the ROBOT ontology toolkit library (Jackson et al., 2019), which provides a facility to create well-formatted ontologies from templates. A ROBOT template is a comma-separated values (CSV) file that can be prepared easily in common spreadsheet software, annotated with instructions for translation from spreadsheet columns to OWL language and metadata attributes.

Results
Step 1 -Defining the scope of the Intervention Source Ontology An intervention's source was defined as 'A role played by a person, population or organisation that provides a behaviour change intervention'. A 'role' in Basic Formal Ontology (BFO) is a type of attribute that is "externally grounded", that is, it depends on the context or situation of an entity.
Step 2 -Identifying key entities and developing the preliminary Intervention Source Ontology The International Standard Classification of Occupations (-08 version) (ISCO-08; International Labour Office, 2012) was identified as a relevant classification system for describing sources' occupational roles. ISCO-08 is a four-level hierarchically structured classification that allows all occupations internationally to be classified into 436-unit groups. The classification gives job content (tasks and duties) priority over national education and training requirements. Therefore, occupations that involve broadly the same sets of tasks and duties are grouped in the same category in ISCO-08, even where national skill level requirements, measured in terms of formal education, are different.
In assessing the suitability of ISCO to classify occupational roles within the ontology, some minor modifications were made to comply with ontology development guidance regarding specifying entities and reducing redundancy of terms (Grimm & Wissman, 2011 Step 3 -Refinement of the Intervention Source Ontology Annotations of published intervention evaluations resulted in changes being made to the Source ontology.

Entities removed from version 0.1 to version 0.2 of ontology:
Lower-levels related to occupational roles not identified as being candidate intervention sources were removed to reduce the size of the ontology, such as Travel Attendants, Conductors and Guides, Street & Market Salespersons and Other Sales Workers.
Entities added from version 0.1 to version 0.2 of ontology: Firstly, Person source was expanded to include Student or trainee and lower-levels related to this, reflecting the Source may be an individual currently studying for a qualification. Second, entities to describe Sociodemographics of Source were added to reflect descriptive attributes of the Source reported, such as age, gender and ethnicity. Such characteristics may be reported to characterise the Source, or may reflect selection criterion for Source, e.g recruiting sources of the same gender or from the same ethnic group as the target population. Thirdly, Relatedness between Source and the target population was added, with sub-levels such as Parent or guardian, to reflect where the Source is recruited by virtue of having a pre-existing relationship to the target population. Fourth, entities to describe Payment model of Source were added, with lower-levels reflecting whether the Source was rewarded in a monetary or non-monetary manner for delivering the intervention, or whether the intervention was delivered on a voluntary basis. Last, Organisation Source as a general higher-level term was added to reflect interventions that are delivered by an organisation rather than by a Person Source (individual). For example, a Stop Smoking mass media campaign in England, such as "NHS Smokefree", is delivered by the National Health Service (NHS) nationally (Organisation Source), whilst components are also delivered by individual staff members at local levels (Person Source).
Version 0.2 of the Intervention Source Ontology as a result of these refinements had a 6-level hierarchical structure, containing 186 unique entities (https://osf.io/7zved/).
Step 4 -Expert stakeholder review Of the 123 experts contacted, 103 were from 'more-represented' countries and 20 from 'less-represented' countries. Of the 34 experts that responded, 85.3% (29/34) completed the survey, with 27 from 'well-represented' and 7 from 'less-represented' countries. Experts' responses and how these were addressed within the ontology development are reported at https://osf.io/ 58qkt/ (West et al., 2020).
Entities added from version 0.2 to version 0.3 of ontology following expert stakeholder review: First, entities to capture physical and health characteristics of Source were added, including 'Source Health status' e.g. body mass index (BMI), body shape, appearance and 'Source Target Behaviour' e.g. Source's smoking status or physical activity level. Second, experts requested elaboration of definition for the ISCO entity 'Psychologist' to include a broader range of psychology areas including counselling, forensic, health and neuropsychology. Also, an entity 'Source involved in co-production of intervention' was added to reflect where those delivering the intervention were involved in its development.
Version 0.3 of the Intervention Source Ontology as a result of these expert stakeholder recommendations had a 6-level hierarchical structure, containing 196 unique entities (https://osf. io/zfn25/).

Entities removed from version 0.3 to version 1 of ontology following inter-rater reliability testing:
Annotators unfamiliar with the ontology suggested that the v0.3 196-entity ontology could be shortened to improve usability. First, ISCO areas less relevant to describing Intervention Source were capped at their top-level. For example, the lower-levels of 'Managers' (ISCO: 1) were removed, such as 'Chief Executives, Senior Officials and Legislators' (ISCO: 11) and lower-levels of 'Clerical Support Worker' (ISCO: 4) were removed, such as 'General and Keyboard Clerk' (ISCO: 41), as not being involved in intervention delivery.
Annotators unfamiliar with the ontology annotated another random sample of 30 randomised controlled trials from the same database with the following target behaviours: physical activity (k=15), sexual behaviours (k=5), alcohol (k=4) and other behaviours such as diet, smoking and tooth brushing (k=6). The inter-rater reliability for these annotations was 'acceptable' (a=0.57; https://osf.io/sg45y/). Version 1 of the Intervention Source Ontology as a result of these annotator recommendations contains 140 unique entities (Table 2).
Step 6 -Specifying the relationships between Intervention Source Ontology entities Relationships from the Relation Ontology (Smith et al., 2005) were used to connect classes, namely the basic hierarchical relationship 'is_a' which holds between classes where one class is a subclass of another class (e.g Medical doctor is_a Health professional), 'has_role' which connects a role bearer to a role it holds (e.g Person has_role Person source role), and 'has_participant' where one class is involved in the process of another class (e.g Supervision of person source has_participant Person source). The relationship 'is_about' from the Information Artifact Ontology (Ceusters, 2012) was also used to represent one class presenting information about another (e.g Total number of people able to deliver intervention is_about Source).
Step 7 -Making the Intervention Source Ontology machine-readable and available online A downloadable version of the final Intervention Source Ontology is available from GitHub (Norris et al., 2021). The hierarchical structure, URIs, labels and definitions for all entities are described in Table 2. The ontology is accompanied by an annotation guidance manual that provides guidance on how to annotate for these entities in BCI reports (available at https://osf.io/e6dzm/).

Discussion
This study developed the Intervention Source Ontology (Hastings et al., 2021) to specify the characteristics of who delivers behaviour change interventions, as part of the Behaviour Change Intervention Ontology Michie et al., 2020). The ontology consists of 140 entities across key areas of Occupational Role of Source, Relatedness between Person Source and the Target Population, Sociodemographic attributes and Expertise. Inter-rater reliability was found to be acceptable for those familiar and unfamiliar with the ontology, as assessed by Krippendorff's alpha (a=0.60 and 0.59 legal, social and cultural professional A legal, social and cultural professional that communicates ideas, impressions and facts in a wide range of media to achieve particular effects, interprets a composition such as a musical score or a script to perform or direct the performance, and hosts the presentation of such performance and other media events. relatedness between person source and the target population A relatedness between person source and the target population that is an individual who is related to another person as they are descended from a common progenitor, related by marriage or other legal tie, or by a feeling of closeness.

parent or guardian [BCIO:010096]
family member A family member that is a mother, father or legal carer of a child. respectively). This suggests that the Intervention Source Ontology and associated annotating guidance can be applied with acceptable consistency, and provides confidence in the methods developed for creating the Behaviour Change Intervention Ontology Wright et al., 2020; https://www.humanbehaviourchange.org/).
Ontologies should be maintained and updated according to new evidence about entities and relationships (Arp et al., 2015;He et al., 2018). The Intervention Source Ontology and all other ontologies within the Human Behaviour-Change Project will be updated according to advances in behavioural science and by online ontology user feedback via GitHub. When the full Behaviour Change Intervention Ontology has been released it will be submitted to the OBO Foundry (Smith et al., 2007) Strengths and limitations A strength of this work is the integration of international expert stakeholder feedback in appraising and revising the ontology, a practice which is uncommon in ontology development (Norris et al., 2019). Involving a variety of domain experts provides a range of relevant knowledge and perspectives to test the ontology, a process used successfully in developing other taxonomies and frameworks (e.g. BCTTv1, Michie et al., 2015; Linking BCTs and Mechanisms of Action, Carey et al., 2019;MAGI framework, Borek et al., 2019;TIPPME, Hollands et al., 2017).
A limitation of this work is that there was a preponderance of intervention evaluation reports from high income countries, potentially limiting the ontology's applicability to interventions in low-and middle-income countries. However, incorporating ISCO-08, an occupational classification system designed to have worldwide relevance (International Labour Office, 2012) should enhance the Source ontology's global applicability. A second limitation is that intervention reports annotated within the ontology development addressed only two health-related behaviours: smoking cessation and physical activity. This was due to the ontology being developed within the Human Behaviour-Change Project, which is using interventions in these behavioural domains as initial use cases . However, external inter-rater reliability was performed across diverse behaviours and found to be acceptable. Future application of the ontology to a wider collection of behaviours and contexts will allow it to be extended and improved.
In addition to contributing to the larger Behaviour Change Intervention Ontology, the Intervention Source Ontology provides a stand-alone classification system for describing and reporting source characteristics. It can be used to describe who, individually or organisationally, will deliver or has delivered an intervention and to synthesise evidence across studies.

Conclusion
The Intervention Source Ontology provides a classification system that can be used reliably to specify the characteristics of who delivers interventions. It will contribute to improved research reporting and replication, simplifying the process of evidence synthesis across diverse studies. The ontology can be used in conjunction with machine-readable tools, such as the Artificial Intelligence algorithms contributing to the Knowledge System being developed within the Human Behaviour-Change Project . The Intervention Source Ontology is intended to act as a basis to be elaborated on in future research, as an ongoing and collaborative process. The ontology will increase understanding of what intervention sources are most effective for given intervention scenarios, varying in target population, behavioural domain, setting and a large number of intervention characteristics.  (Hastings et al., 2021) This project contains the following extended data: -An archived version 1 of the Intervention Source Ontology Data are available under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0).

Software availability
Source code used to calculate alpha for IRR available from: https://github.com/HumanBehaviourChangeProject/Automation-InterRater-Reliability. Thank you for the opportunity to review this manuscript which sought to develop an ontology for specifying who delivers interventions. This will be an important contribution for the implementation science field as it will support trialists in their reporting of interventions which is particularly important as the field is advancing in respect to adaptation and optimisation. The authors have presented a strong rationale for the need for the ontology, used and described robust methods for this study and presented the results clearly. I really enjoyed reviewing this paper and have minimal feedback for the authors, just a few questions that they may wish to consider. Congratulations.
When I read the title of paper I immediately thought of Justin Presseau et al work (i.e. AACTT) Framework which had built off TACT Framework. I wondered why this wasn't made mention of in your paper and how your ontology potentially links or extends this work? In Step 2 the authors note that 100 published reports of behaviour change interventions were reviewed. It may not matter but I was wondering about the characteristics of these reports. It may be useful to just give a brief description of the characteristics of these studies.
The response rate in Step 4 was quite low. Do the authors think that this could have had any impact on the findings for the thematic analysis?
This may not be universal, but one issue that we often face in our work in community settings i.e. schools, childcare services is the employment status of the individuals (e.g. casual, temporary/permanent, part-time/fulltime) who are delivering the intervention. This has a big impact on implementation interventions that are impacted by staff turnover. I was left wondering if there was somewhere in the ontology where trialists could be encouraged to capture and report this.

If applicable, is the statistical analysis and its interpretation appropriate? Yes
Are all the source data underlying the results available to ensure full reproducibility? Yes For "target behaviour of person source", I am not sure I understand what this is -is it the extent to which the interventionist personally exhibits the target behaviour they are working on with participants? Please clarify. Idem for 'psychological influence on intervention delivery of person source.

7.
For 'knowledge or skill', please specify if pre-existing or acquired in preparation for the study.
For 'supervision of person source', please provide an example of supervision (e.g., supervised through weekly meetings etc.).

10.
For 'payment of person source", please provide an example. 11.
For 'source involved in co-production', please clarify if co-production can only occur with patients/community members or can it also occur with HCP's?

12.
For 'organization", please provide examples. Idem for organization source role and organization source.

13.
Minor comment: There is a typo on page 6 in the paragraph describing step 4 -should read "Australia". 14.

If applicable, is the statistical analysis and its interpretation appropriate? Not applicable
Are all the source data underlying the results available to ensure full reproducibility? Yes

Are the conclusions drawn adequately supported by the results? Yes
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: Health psychology, behavioural medicine, behaviour change, behavioural intervention development methods, behavioural trial methods, chronic disease prevention I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.