{'fraction_non_alphanumeric': 0.025288806911950297, 'fraction_numerical': 0.010193185127657508, 'mean_word_length': 5.275662503807493, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Concurrent Validity and Interrater Reliability of the “Clinical Schedules for Primary Care Psychiatry”

1Primary Care Psychiatry Program, Tele Medicine Centre, Department of Psychiatry, National Institute of Mental Health and Neurosciences (NIMHANS), Bengaluru, Karnataka, India 2District Mental Health Programme–Ramanagara District, Government of Karnataka, Karnataka, India 3National Health Mission, Department of Health and Family Welfare Services, Government of Karnataka, Bengaluru, Karnataka, India 4Department of Biostatistics, National Institute of Mental Health and Neurosciences (NIMHANS), Bengaluru, Karnataka, India Address for correspondence Narayana Manjunatha, MD, Department of Psychiatry, Tele Medicine Centre, National Institute of Mental Health and Neurosciences, Bengaluru, Karnataka, India (e-mail: manjunatha.adc@gmail.com).


Introduction
Psychiatric illness is a major contributor to the global burden of disease. It is an oft-recorded observation that the access to specialized mental health care in low-and middle-income countries including India is significantly restricted by numerous factors, such as limited numbers of trained professionals and their inequitable distribution, stigma and prejudice, lack of awareness, and limited resources and lacunae in policy making. 1 While the list of challenges is lengthy, the treatment gap for mental disorders is also wide, particularly for common mental disorders (CMDs) and substance use disorders (SUDs). As per the findings of the National Mental Health Survey of India 2016, the prevalence of psychiatric illnesses in India is 10.6% with a treatment gap ranging from 60% for CMDs to Validation of Clinical Schedules for Primary Care Psychiatry Kulkarni et al. 90% for SUDs. [2][3][4] The World Health Organization (WHO) has recommended a strategy to tackle this through the integration of psychiatric care with primary health care. [5][6][7] Under the National Mental Health Program of India, it was proposed to provide psychiatric services by training primary care doctors (PCDs) in the early identification and treatment of psychiatric disorders at primary care level. 5 There are many attempts to continue in favor of implementing this strategy. 8 Earlier studies have found that CMDs including anxiety, depression, and somatization present in 17 to 46% of patients attending primary health centers. 9 However, PCDs may not have the requisite training to diagnose psychiatric disorders in these patients. The past studies have found that PCDs may fail to correctly diagnose and treat nearly 50 to 75% of patients initially presenting with psychiatric disorders. 8,[10][11][12] Ideally, screening for psychiatric disorders by PCDs needs to be achieved in a time-limited manner with reasonable accuracy, as against the psychiatric clinical interview performed by psychiatrists which is considered as a gold standard. 13 Inadequate knowledge about diagnostic criteria of CMDs, lack of awareness about the appropriate interview questions that need to be asked to identify these disorders, and limitations of time in a busy clinic lead to underdiagnosis by PCDs. 8 The WHO 14 has developed the Mental Health Gap Action Program (mhGAP) and evolved a tool called the mhGAP Intervention Guide for use in nonspecialized settings for the first-and second-line health-care professionals. It is very lengthy, complex, and not specific for PCDs. It includes nine priority-based conditions (mental, neurological, and SUDs) such as depression, psychosis, bipolar disorders, epilepsy, developmental and behavioural disorders in children and adolescents, dementia, alcohol and drug use disorders, selfharm/suicide and other significant emotional or medically unexplained complaints. These are rather high-prevalent psychiatric conditions at primary care with higher treatment gap. It is not on PCD friendly manual, especially for developing countries, like India, considering heavy patients load at primary care. Guidelines of child mental health conditions and dementia in mhGAp manual are impossible to implement at Indian primary health centers by PCDs. Despite higher prevalence at primary care, mhGAP manual grossly missed anxiety and somatization disorders, and management of tobacco addiction. Despite very low prevalence of psychosis, especially bipolar disorder at primary care, mhGAP laid heavy focus on management of these conditions. This manual does not allow safer, wider range of pharmacological agents for highly prevalent psychiatric conditions, for example, escitalopram is commonly prescribed antidepressants in India. Emphasis heavily laid on psychosocial interventions for these conditions, which PCDs unlikely to practice in India. It also requires special training for its application and requires support for its interventions. It also grossly lacks integrated, yet comprehensive screener (WHO, 2016). There are many disease-specific screening tools, such as Patient Health Questionnaire-9 (PHQ-9), 15 Primary Care Screening Questionnaire for Depression (PSD), 16 and Global Mental Health Assessment Tool for Primary Care. 17 However, there are limitations, such as the absence of a single screening tool for most commonly encountered conditions in primary care, lack of cultural adaptation for Indian settings, and considerations for use in primary care settings. [15][16][17] Hence, there is a need of integrated all-in-one tool for PCDs with incorporation of principle of general clinical practice.
A sole aim of designing "Clinical Schedules for Primary Care Psychiatry" (CSP) is an all-in-one integrated tool for PCDs to provide the first-line safe and effective pharmacotherapy to highly prevalent (may not be priority based) psychiatric disorders at primary care with rapid screening using culturally appropriate questionnaire. 18 CSP Version 2.1 ( Department of Psychiatry, National Institute of Mental Health and Neurosciences, Bengaluru, India) consists of screener, classification of psychiatric disorders adapted for use in primary care settings, diagnostic criteria, referral points, and management guidelines of only six high-prevalent psychiatric disorders at primary care where highly effective and safe prescription medications are available. These six psychiatric disorders are tobacco addiction, alcohol (harmful and addiction), psychotic, depressive, anxiety (panic and generalized anxiety disorders), and somatization disorders (acronyms as TAP DAS). The CSP screener consists of a questionnaire containing 21 culturally appropriate questions to screen patients of these six highly prevalent psychiatric disorders. In simple, CSP use three cluster-based transdiagnostic classification of psychiatric disorders adopted for the use of PCDs; CMDs cluster subdivided as predominantly depressive, anxiety, somatization, or mixed symptoms; psychotic disorders cluster is subdivided as acute (cover acute psychosis and mania), chronic (schizophrenia and schizoaffective disorders), or episodic (focus on bipolar disorder); and alcohol disorders simplified as harmful (frequent and infrequent type) and addiction with simplified diagnostic criteria for primary care use. Discussion about designing curriculum and screener questionnaire of this CSP is beyond the scope of this article and is discussed elsewhere. CSP is available on request for readers. The screener includes a hint or anchor column which leads to a broad diagnosis. This has been extensively used in digitally driven primary care psychiatry program from Telemedicine Centre of National Institute of Mental Health and Neurosciences (NIMHANS), Bengaluru, for the purpose of training PCDs to identify and manage psychiatric disorders. 19 CSP requires a very brief training about how to ask the screening questions and making a diagnosis using the hint/anchor column. Further, the diagnostic categories lead to management guidelines including medication, brief counseling advice, and follow-up advice along with strategies for referral. Thus, PCDs must be sensitized to detect "red flags" or warning signs of psychiatric illness particularly for identifying CMDs which have a treatment gap of 50 to 60%. 2,6 Those patients who are likely to have problems after ruling out those due to medical illness and SUDs need to be picked up. The questions must be suited to reflect the pragmatics of the reallife scenario of primary care practice in India. CSP screener has this inbuilt pragmatism of real-world scenario.
The aim of this study was to validate the CSP and establish its psychometric properties with the hypothesis being that the CSP would be able to help PCDs to establish psychiatric caseness (meaning to achieve a broader diagnosis) after brief training in its use. This would establish its practical applicability to enable PCDs to identify psychiatric diagnoses in primary care settings.

Study Center Location
The study was conducted at the outpatient services of three primary health care centers (PHCs) under the District Mental Health Program (DMHP) at Ramanagara district in Karnataka, India, after administrative approval from the National Health Mission, Government of Karnataka. The study was also approved by the Institute Ethics Committee at the NIMHANS at Bengaluru in Karnataka, India. This study was performed from October 2017 to June 2018.

Participants
Written informed consent from PCDs working at the above-mentioned PHCs was obtained to participate in this study. After providing consent, the PCDs were briefly trained about the CSP, particularly how to ask questions contained in the screener, how to use the hint/anchor column to lead to the diagnosis with the investigator explaining briefly about the diagnostic categories and management guidelines of the CSP. This entire discussion would take around 15 to 20 minutes/doctor.

Psychometric Properties
Validity and reliability for the CSP needed to be established. To establish an agreement, the diagnoses made by the PCDs using the CSP were compared with those made by a qualified psychiatrist using a semistructured psychiatric interview to make a diagnosis, confirmed by the International Statistical Classification of Diseases and Related Health Problems 10th edition (ICD-10) checklist which was used for concurrent validity. 20,21 This ICD-10 symptoms checklist is a semistructured diagnostic instrument used for clinicians' assessment of F0 to F6 categories in the ICD-10 classification of mental and behavioral disorders and tested in the field trials of the ICD-10 diagnostic criteria for research. 22 To establish interrater reliability, a total of 20 patients were simultaneously and independently interviewed by two PCDs and diagnoses were compared.

Sample Size
A total sample of 210 patients were calculated (using the principle of 10 patients per item of the screening questionnaire). 23 However, a total of 180 patients (n = 180) who provided written informed consent included in the study. Those under the age of 18 years were excluded. Eleven patients declined to participate due to logistical reasons, such as greater waiting time for a second interview by the psychiatrist.

Sampling
As a part of DMHP Ramanagara, mental health outreach camps and "Super-Tuesday Manochaitanya" clinics would be conducted at PHCs. Subsequent to the camps, the psychiatrist would visit OPD of the nearby PHCs, where consecutive adult patients presenting to the outpatient clinic (around 15-20 patients at the time that the investigator would visit) of the PHCs who provided written informed consent were included for the study.

Procedure
The three PCDs who consented were trained independently about method to use CSP its screener, classification system and diagnostic criteria, and to make psychiatric diagnosis. Patients who consented for the study were administered CSP in addition to regular examination at the PHC. The PCDs would initially examine the patients and first note if the patients had symptoms which could be explained by medical illness alone. If yes, then they would not proceed with the CSP and continue treatment as usual. If, however, these symptoms were medically unexplained, they would need to check if they had lasted greater than a fortnight. If not, then the patient could be reassured and monitored at follow-up. If yes, then the PCDs would administer the questions contained in the CSP to determine if the patient likely had a psychiatric illness (psychiatric caseness). The screener was verbally translated into the vernacular language, that is, Kannada by PCDs at the time of administering it. Appropriate diagnosis was marked in the classification provided in CSP below the screener after checking the diagnostic criteria.
Soon after PCDs consultation, a psychiatrist (K.K.) performed a semistructured psychiatric interview for the same patients and the diagnosis was confirmed with the ICD-10 checklist. This interview involved history-taking and mental-state examination during which clinical characteristics of patients were noted for making the diagnosis. The psychiatrist was blind to the diagnosis made by PCDs. The patients who were diagnosed by psychiatrist to have CMD/SMD/ SUD were advised appropriate investigations, advised for follow-up, and started with appropriate treatment.

Statistical Analysis
Descriptive statistics were used for sociodemographic and relevant clinical variables. Sensitivity, specificity, and positive predictive value were calculated for the use of CSP to detect and diagnose psychiatric illness. Concurrent validity consisting of comparing the diagnoses made by PCDs using the CSP, and the diagnoses made by the investigator conducting a psychiatric interview to make a diagnosis confirmed by the ICD-10 checklist used as a gold standard. Cohen's kappa (κ) was calculated for interrater reliability. The Statistical Package for the Social Sciences (SPSS version 16; SPSS Inc., Chicago, United States) was used for the purpose of the analysis.

Sociodemographic Characteristics
A total of 180 patients were included in the study. Of these, 116 (64.4%) were female and 64 (35.6%) were male. The mean age was 44.5 ± 15.5 years. All of them are residents of three PHCs.
Validation of Clinical Schedules for Primary Care Psychiatry Kulkarni et al.

Agreement and Concurrent Validity
There was agreement between a diagnosis of psychiatric illness made by the PCDs and the psychiatrist for 142 (78.9%) patients (►Table 1). The Cohen's κ for the concurrent validity was 0.57, suggestive of moderate agreement. 24

Sensitivity, Specificity, and Positive Predictive Value
The sensitivity of the CSP for making diagnoses comparable to those made by a psychiatrist was 91.1% (►Fig. 1). The specificity of the CSP for making diagnoses comparable to those made by a psychiatrist was 68.3%. The positive predictive value of using the screener was 69.2% (►Table 2).

Interrater Reliability
The interrater reliability for 20 patients showed κ = 0.7, suggestive of good agreement.

Discussion
The CSP demonstrates a reasonably high sensitivity of 91% to detect psychiatric illness with a specificity of 68% and a positive predictive value of 69%. Thus, the CSP has high sensitivity with a lower specificity and positive-predictive value. It is important to consider the implications of the same. This would imply that a higher number of patients are likely to be diagnosed as having psychiatric illness and treated for the same, resulting in false positives. It may also lead to   underinvestigation and underdiagnosis of a possible medical condition. One must also consider the implications in terms of stigma which is attached to mental illness in India, while labeling a patient as having a psychiatric illness. Patients may be treated with psychotropic drugs, albeit these are relatively safe. It is important to remain open-minded about the diagnosis of those patients who do not appear to have a good response with these drugs. In the real world, the specificity of the CSP can be improved with skill-based clinical training of the PCDs. Therefore, the pragmatic aspect of striking a balance between its sensitivity and specificity needs to be considered as equally important. Furthermore, it is important to consider here the sensitivity and specificity of previously mentioned questionnaires which have been used in comparable settings. The PSD has a relatively high sensitivity and specificity of 96 and 87% at a cut-off of two or more questions out of a total of four. The authors found that relatively high specificity and positive-predictive value could be obtained by determining a threshold of two or more questions as cut-off. Changing this threshold to a higher number would lower sensitivity and lowering it would bring down the specificity. 16 The aforementioned PHQ-9 (depression), however, has a sensitivity of 91% and a specificity of 76% which is comparable to the CSP itself. The authors of the PHQ-9 have similarly discussed that the most favorable combination of sensitivity and specificity could be obtained using a cut-off score of ≥9, wherein a higher score would increase specificity at the cost of reducing sensitivity. 15 It must, however, be discussed again that the above-mentioned questionnaires are designed to screen for a particular condition, such as depression, whereas the CSP is designed for rapid screening for highly prevalent psychiatric conditions in a primary care setting. Therefore, a tool with higher sensitivity may be preferable in these settings. The problem of high-false positives in psychiatric illnesses has often been examined by earlier studies. In fact, a study which particularly looked at the problem of higher false-positive rates in screening for psychiatric illnesses in primary care declared that even those who were falsely positive on screening warranted greater clinical attention because these patients had higher functional impairment and higher rates of service uses than those who were true negatives. 25 Other studies have also recommended taking a closer look at the patients who screen falsely positive suggesting that other forms of psychopathology or a protean medical condition may be responsible. 26 Yet another study which looked at psychiatric diagnoses itself stated that the thresholds for making a diagnosis for psychiatric illness in screening should always be set, so that the benefits outweigh the risks and that patients who screen as false positives must be kept in follow-up and treatment as they might need further investigation and care. 27

Limitations
However, it is important to acknowledge the limitations of the study, most importantly, the fact the CSP was administered by a small number of motivated and interested PCDs. Comorbid psychiatric diagnoses are not made using the screener. Furthermore, it was verbally translated into Kannada at the time of administration to the patients, which may lead to some variation in how the questions were asked. CSP should be seen as a knowledge enhancer rather than a skill enhancer in clinical settings. Hence, the results of this article should also be seen as being a translation of the knowledge-enhancing effect of CSP in clinical settings rather than the skill-enhancing effect of a clinical training program (especially other programs, such as the direct skills transfer effect of an innovative on consultation training of PCDs which is now being implemented at our center).

Conclusion
In conclusion, the CSP is designed for use by PCDs and demonstrates a high sensitivity and fairly high specificity for making psychiatric diagnoses at primary care. It is important to keep in mind that it requires a pragmatic clinical training which has higher translational quotient for its administration as well as needs further refinement in its validation using a larger sample size. Pragmatically, its clinical effectiveness and acceptability remain to be examined.

Future Directions
The next step in its validation would be to acquire a larger sample size so that most disorders, including SUDs and SMDs, have an adequate representation. It may also be necessary to revise some of the questionnaires of the CSP to help improve its specificity and positive predictive value. Specificity of CSP can be improved by clinical skill-based training of PCDs (one such example is the on-consultation training is being conducted for PCDs at this center) which shall be tested later. 19 It would also be necessary to study the clinical usefulness and effectiveness of CSP with its acceptability by PCDs working in real-life clinical scenario with patients.

Funding
None.