{'fraction_non_alphanumeric': 0.03174667042208184, 'fraction_numerical': 0.014686355771939, 'mean_word_length': 5.247863247863248, 'pattern_counts': {'":': 0, '<': 7, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 1, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Research Paper: Psychometric Properties of COVID-19 Dot-probe Task in Iranian Adults

* Corresponding Author: Saeed Nasiry, MSc. Address: Department of Clinical Psychology, School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran. Tel: +98 (939) 8399387 E-mail: snasiry@sbmu.ac.ir Objective: After the COVID-19 outbreak, corona anxiety has become prevalent all over the world. To understand and treat this type of anxiety, researchers have examined its relationship with attentional bias, a phenomenon closely associated with other types of anxiety. The dot-probe task is a common instrument used for the evaluation of attentional bias. However, the psychometric properties of this instrument, when used for the assessment of attentional bias towards coronarelated stimuli, are unknown. This study aimed to evaluate the psychometric properties of the COVID-19 dot-probe task to see whether its application in COVID-19 studies is justified.


Introduction
ased on the World Health Organization statement in March 2020, the outbreak of coronavirus is a pandemic and it is not only endangering physical health, but also deteriorating personal, familial, and public mental health (Jakovljevic, Bjedov, Jaksic, & Jakovljevic, 2020). CO-VID-19 has changed how health care systems operate and how people behave in private and social situations, so the general population's way of living and their priorities have altered (Fiorillo & Gorwood, 2020). The outbreak has decreased public mental health through the sudden rise of mortality rates, joblessness, stigmatization, seclusion, and lack of close contact with friends and loved ones (Marčinko, Jakovljević, Jakšić, Bjedov, & Mindoljević Drakulić, 2020). Recent studies suggest that people with a mental health condition are more vulnerable to the deteriorating psychological effects of the COVID-19 outbreak. These people are especially prone to tension and stress, and the current situation can exacerbate their already psychological problems (Yao, Chen, & Xu, 2020).
While the psychological issues caused by the coronavirus pandemic are wide-ranging, it has specifically affected the general population's anxiety (Wang et al., 2020). Sorokowski et al. (2020) have shown that people's anxiety levels can significantly increase just by being ex-posed to the information related to coronavirus pandemic. Also, the media's way of handling this situation has made it virtually impossible to keep away from this information in daily life. Ojiaku, Iorfa, Mefoh, Ezeuzo and Odinko (2020) have also found that anxiety caused by coronavirus outbreak can negatively predict psychological health. As the deteriorating effects of the coronavirus pandemic on the general population's mental wellbeing increase, the number of individuals requiring psychotherapeutic interventions also increases (Marčinko et al., 2020). Studies suggest that anxiety disorders are among the most common consequences of this circumstance (Huang & Zhao, 2020).
In response to these issues, researchers in psychology are trying to understand how corona anxiety occurs in people and how it can be treated. The theoretical background suggests that attentional bias (a cognitive deficiency in selective information processing) heightens attention towards signs of threat. So, it has a causal role in anxiety disorders, and extensive empirical evidence supports this notion (Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & Van Ijzendoorn, 2007;Van Bockstaele et al., 2014). Accordingly, attentional bias towards corona-related information is being investigated in recent studies as an essential factor in the increased anxiety during the outbreak of COVID-19. Also, the findings support the causal role of corona-related attentional bias

Plain Language Summary
With the worldwide outbreak of coronavirus, people have increasingly become anxious about the threats of this pandemic. Since individuals who suffer from anxiety tend to pay more attention towards threatful stimuli (a phenomenon which is called attentional bias), researchers in Psychology have begun to investigate the relationship between corona anxiety and attentional bias. To do this, they use a common method of measuring attentional bias, which is called the dot-probe task. Using this tool, they show threatful and neutral pictures to the participants and see how their cognitive reactions is affected. However, the validity and reliability of this method for measuring attentional bias related to coronavirus have not been investigated. The study objective was to evaluate this validity and reliability. The results showed that the standard version of the COVID-19 dot-probe task lacks the acceptable levels of psychometric properties. In contrast, the response-based version of the instrument increases all of these psychometric properties to an acceptable level. Thus, the COVID-19 dot-probe task is a valid and reliable instrument for measuring corona-related attentional bias if the response-based method of computation is used for calculating the measures of attentional bias. in corona anxiety (Hagerty & Williams, 2020;Loreta et al., 2020;Nasiry & Ameli, 2021;Schudy et al., 2020).
However, the reliability of the results of such studies depends on the reliability of the instruments used in the assessment of study variables. The most commonly utilized tool for the measurement of threat-related attentional bias is the dot-probe task (Schmukle, 2005) which has also been used for the measurement of corona-related attentional bias (Loreta et al., 2020;Nasiry & Ameli, 2021). The dot-probe task consists of trials in which pairs of threatful and neutral pictures are displayed on a computer screen for a short amount of time. Then a single dot (or any other probe) is shown where one of the pictures was placed previously. The participant's task is to use the computer's keyboard to indicate on which side of the screen the probe has appeared. The time spent for each trial (i.e., reaction time [RT]) is recorded and used for the calculation of the measures of attentional bias. The lower RTs for probes that appear in place of threatful pictures and higher RTs for probes that appear in place of neutral pictures indicate higher attentional bias.
Other instruments similar to the dot-probe task with less common use have been developed and utilized to measure attentional bias. The emotional flanker task, which was designed by Nikolaou, Field, and Duka (2013), measures the participants' attentional bias using their reaction times to specific trials. These trials consisted of one central arrow and multiple flanking arrows that are pointed either in the same direction as the central arrow (congruent trials) or in the opposite direction (incongruent trials). The participants are required to react to the direction of the central arrow while ignoring the emotional or neutral pictures displayed behind the arrows. Another instrument for measuring attentional bias is the relevant feature visual search task, developed by Ohman, Flykt, and Esteves (2001), with a variant called irrelevant feature visual search task proposed by Dodd, Vogt, Turkileri, and Notebaert (2017). These instruments comprised trials in which the participants were asked to search and locate a target picture amidst a series of distracting pictures. Variation in the emotionality or the neutrality of the target and distractor stimuli in different trials leads to different RTs which are recorded and utilized to calculate the measures of attentional bias.
Poor psychometric properties of the instruments measuring attentional bias have caused significant concerns among the researchers in this field (Evans, Walukevich, Seager, & Britton, 2018;McNally, 2019;Rodebaugh et al., 2016). Empirical evidence demonstrates that the emotional flanker task lacks reliability and validity in the context of anxiety research (Van Bockstaele et al., 2020). However, the relevant feature and the irrelevant feature variants of the visual search task show a medium level of reliability (Van Bockstaele, Salemink, Bögels, & Wiers, 2017) and lack any indications of validity (Van Bockstaele et al., 2020). Although the dot-probe task has been most commonly used in the studies investigating anxiety disorders, including corona anxiety (Loreta et al., 2020;Nasiry & Ameli, 2021), the standard version of this instrument shows poor psychometric properties and, therefore, might lead to inconsistent results across different studies (Cristea, Kok, & Cuijpers, 2015;Schmukle, 2005). To resolve this issue, a new version of this instrument with a response-based computational approach has been developed recently, which shows high levels of reliability and validity . But neither the standard version nor the response-based version of the dot-probe task has been psychometrically evaluated when used with stimuli related to COVID-19. This condition can challenge the reliability of the results of the studies which use this instrument to investigate attentional bias and anxiety related to COVID-19.
Because the studies investigating attentional bias related to COVID-19 are important and the psychometric properties of the dot-probe task with corona-related stimuli have not been examined, we aimed to determine the internal consistency, test-retest reliability, and criterion validity of the standard and response-based versions of COVID-19 dot-probe task. Considering the preliminary results of previous research, we expected the psychometric properties of the response-based dot-probe task to be superior to the standard version of this instrument.

Study participants
The study participants were recruited using online forums and social media platforms from the target population of all non-clinical adults in Iran. The inclusion criteria comprised 1) having at least 18 years of age, 2) being able to work efficiently with computers, and 3) delivering an informed willingness to contribute to the study. The exclusion criteria were 1) suffering from serious physical diseases that required immediate medical treatment, 2) having the diagnosis of psychological disorders, and 3) not completing the study. Overall, 362 Iranian adults (62% female) aged 18 to 42 years (28.3±4.9 years) were enrolled in the study. They did not report any history of receiving psychological diagnoses from an expert in clinical psychology or psychiatry and were not under medical treatment for any physical illnesses.

Standard dot-probe task
The dot-probe task used in this study was developed based on the original work of MacLeod, Mathews, and Tata (1986). The task comprised several trials; each began with the appearance of a cross in the middle of the computer screen. The participants were asked to stare at this cross. The fixation cross would remain on the screen for 500 ms, and then a pair of pictures were displayed alongside each other. This part of the trial would also last for 500 ms, and then the images were replaced with a single probe (which was either < or >) that was shown in place of one of the pictures with a random order. The participant's task was to use the arrow keys on their computer's keyboard to quickly and accurately indicate the place of each probe. Each trial included either one corona-related stimulus (e.g., picture of a mask, glove, virus, etc.) and one neutral stimulus (e.g., image of a building, door, window, etc.), or two neutral stimuli. To select the stimuli used in the task, a sample of 73 unbiased individuals rated a pool of potential pictures (scores ranged from 0 to 100) based on their relatedness to coronavirus. The task included 120 trials divided into 40 congruent trials, 40 incongruent trials, and 40 neutral trials. In each congruent trial, a pair of threatful and neutral pictures were shown, and the probe replaced the threatful picture. Whereas, in the incongruent trials, the probes replaced the neutral pictures and not the threatful ones. In the neutral trials, both images were neutral, and the probe was placed in each direction equally.
In the older and more basic versions of the dot-probe task, only the Reaction Times (RTs) to congruent and incongruent trials were recorded and compared with each other to calculate the measure of attentional bias. If the total RTs of incongruent trials minus the total RTs of congruent trials was a positive score, it showed vigilant attentional bias towards threatful stimuli. In contrast, if the score was negative, it indicated avoidant attentional bias away from threatful stimuli. In cases where there was no attentional bias, the score would have been close to zero. Koster, Crombez, Verschuere, and De Houwer (2004) developed a method of calculating separate measures for the orientation and the disengagement of attention by comparing the mean RTs of congruent and incongruent trials with the mean RT of neutral trials. Subtracting the mean RT of congruent trials from the mean RT of neutral trials results in either a positive score indicating a faster orientation of attention to threatful stimuli or a negative score showing a slower orientation of attention. Similarly, subtracting the mean RT of neutral trials from the mean RT of incongruent trials produces either a positive score showing slower disengagement of attention from the threatful stimuli or a negative score indicating faster disengagement of attention.

Response-based dot-probe task
The procedure of the response-based version of the dot-probe task is the same as the standard version. Still, it has a different method of utilizing trial reaction times to compute the measures of attentional bias . In this version, the RT of each trial response (hence the name response-based) is individually compared with a mean RT that functions as a reference. The calculation of the measures of attentional bias is done by subtracting the RT of each congruent trial from the mean RT of incongruent trials and producing two sets of results. Positive results are produced from congruent trial responses with RTs lower than the reference RT and indicate attentional vigilance towards threatful stimuli. Therefore, the mean of this set of results can be used as a measure of vigilant attentional bias. Conversely, negative results are produced from congruent trial responses with RTs higher than the reference RT and indicate attentional avoidance from threatful stimuli. Thus the mean of this set of results can be used as a measure of avoidant attentional bias. By combining this computational method with the Koster et al. (2004) method, separate measures of orientation and disengagement of attention can be computed using the mean RT of neutral trials as the reference to be compared with individual RTs of congruent and incongruent trials. With this method, four different mean scores are produced for slow and fast orientation to and disengagement from threatful stimuli.

Corona Disease Anxiety Scale (CDAS)
Alipour, Ghadami, Alipour, and Abdollahzadeh (2020) have developed a self-report questionnaire called the Corona Disease Anxiety Scale (CDAS) to assess the anxiety about the coronavirus pandemic and have evaluated its psychometric properties in an Iranian sample. This questionnaire includes 18 items rated on a 4-point Likert scale, with the total score ranging from 0 to 54. Exploratory factor analysis has shown that CDAS comprised two subscales: one evaluating the psychological symptoms and the other assessing physical symptoms, and each containing 9 items. Confirmatory factor analysis has confirmed this factor structure and approved the questionnaire's structural validity. Internal consistency of this questionnaire was also confirmed with the Cronbach α coefficients of 0.91, 0.87, and 0.86 for the whole measure and its subscales (subscale 1: psychological symptoms and subscale 2: physical symptoms), respectively. Also, the scores of CDAS are significantly correlated with the scores of the General Health Questionnaire-28 (GHQ-28), which confirms its criterion validity.

Study procedure
To observe the social distancing, the study procedures were all carried out online. The participant was asked to complete the dot-probe task in a private space with no distractions while keeping an ideal distance of 60 cm from their computer's screen and then fill out the CDAS. After two weeks, 146 participants repeated this procedure to provide retest data. There were no significant differences in terms of age, gender, and corona anxiety scores between the participants who repeated the dot-probe task and those who did not (P>0.05). Utilizing SPSS v26, internal consistency was analyzed using split-half reliability and the Cronbach α; test-retest reliability was determined using Intraclass Correlation Coefficient (ICC), and criterion validity was evaluated using correlation coefficient between dot-probe task and CDAS questionnaire.
Before the study, all participants were informed about the objectives, procedure, and implications of the study, and written consent was obtained from them. During the investigation, all ethical standards, including the guidelines defined in the Declaration of Helsinki in 1975, were considered. The participants were aware that their information would remain confidential and that they could leave the study at any point. After completing the study, its results and their meanings were simply explained to the participants, and they were thanked for making this research possible.

Results
The results indicated that both split-half reliability and Cronbach α are unacceptable for measures of attentional bias, attentional orientation, and attentional disengagement in the standard version of the dot-probe task (Table  1). For the response-based version of the dot-probe task, when mean RT of incongruent trials was used as a reference to divide attentional bias into vigilant and avoidant scores, split-half reliability ranged from acceptable to good, and Cronbach α ranged from good to excellent. When mean RT of neutral trials was used to divide orientation and disengagement of attention into fast and slow scores, split-half reliability ranged from acceptable to good, and the Cronbach α was good. Details of the internal consistency of the two versions of the dot-probe task are presented in Table 1.
No indication of test-retest reliability was evidenced for the standard version of dot probe task. Insignificant ICC between two measurements of attentional bias (r=0.11, P=0.21), orientation of attention (r=-0.13, P=0.39) and disengagement of attention (r=0.09, P=0.12), showed that these measures are not stable over time.
The response-based version of the dot-probe task showed varying degrees of stability over time, with some of its measures reaching acceptable levels of ICC between two measurements. Specifically, when mean RT of incongruent trials was used as a reference to attain separate measures of attentional bias, both vigilant (r=0.77, P<0.01) and avoidant (r=0.72, P<0.01) attentional biases showed acceptable levels of test-retest reliability. When mean RT of neutral trials were used as a reference, fast orientation of attention (r=0.71, P<0.01) showed acceptable level test-retest reliability, while the level of test-retest reliability for slow orientation of attention (r=0.49, P<0.01) was unacceptable. Moreover, fast disengagement (r=0.21, P=0.03) and slow disengagement (r=0.36, P<0.01) of attention, showed significant ICC between two measurements, but did not reach an acceptable level of test-retest reliability.
None of the measures of attentional bias in the standard version of the dot-probe task were significantly associated with corona anxiety (all P>0.05). In the responsebased version of the task, vigilant attentional bias, fast orientation of attention, and slow disengagement of attention were significantly associated with corona anxiety. However, there were no significant correlations between corona anxiety and measures of avoidant attentional bias, slow orientation of attention, and fast disengagement of attention (Table 2).

Discussion
In this study, psychometric properties of the COVID-19 dot-probe task were investigated, using the standard and response-based methods of computation for measures of attentional bias related to COVID-19 stimuli. While standard measures did not show acceptable internal consistency, test-retest reliability, or criterion validity (which was consistent with previous studies) (Brown et al., 2014;Price et al., 2015;Schmukle, 2005), responsebased measures showed acceptable to excellent internal consistency, variable test-retest reliability and partial associations with corona anxiety.
Based on these results, the standard version of dotprobe task in clinical studies of corona-related attentional bias and corona anxiety is questionable. Since this instrument has poor psychometric properties, it might not detect associations of corona-related attentional bias with associated variables or evaluate the effects of treatments targeting this type of attentional bias. But the response-based version of dot-probe task showed improvements in internal consistency and stability over time. However, some of its measures showed levels of test-retest reliability that are not regarded as acceptable levels for application in clinical research (Polit, 2014).
Nevertheless, since the internal consistency of responsebased measures was very high, their limited test-retest reliability did not result from an overall deficiency of psychometric properties. Furthermore, the temporal instability of these measures cannot be attributed to changes in corona anxiety since it was stable (r=0.83, P<0.01) across multiple measurements. Therefore, low test-retest reliability of response-based measures may indicate that corona-related attentional bias has innate variability over time. Despite this variability, some of the response-based measures (i.e., vigilant and avoidant attentional biases and fast orientation of attention) had acceptable test-retest reliability. Also, studies show that attention measures that are reliable over time can detect changes in attentional bias after treatments like attentional bias modification (Heeren, Philippot, & Koster, 2015). Criterion validity of the response-based version of dot-probe task was also limited, since only vigilant attentional bias, the fast orientation of attention, and slow disengagement of attention were significantly associated with corona anxiety. This specificity of associations demonstrates that in a non-clinical sample, different patterns of attentional bias towards corona-related stimuli go hand in hand with the severity of corona anxiety. In contrast, patterns of the attentional bias away from corona-related stimuli vary independent from the severity of corona anxiety, and this indicates that future research in this area should be focused more on the understanding and manipulation of attentional bias towards corona-related stimuli.
There were some limitations in this study that should be mentioned. First, the study participants were all nonclinical adults, which limits the generalizability of the results since previous research has shown that associations between attentional bias and anxiety might only be observable in a clinical sample (Briggs-Gowan et al., 2015). Moreover, there are other methods of computation for the measures of attentional bias in dot-probe task, for example, the ones used by Zvielli, Bernstein, and Koster (2014). Although they have more limited usage than response-based method, they may have different psychometric properties, and therefore could be examined in future studies.

Conclusion
The COVID-19 dot-probe task is a psychometrically sound instrument for evaluating corona-related attentional bias if the response-based method of computation is used for calculating the measures of attentional bias. This method improves the internal consistency, test-retest reliability, and criterion validity of the instrument. This version of the dot-probe task can help researchers investigate the mechanisms of corona anxiety and its relationship with corona-related attentional bias and develop therapeutic methods for ameliorating corona anxiety through modification of this type of attentional bias.

Compliance with ethical guidelines
All ethical standards, including the guidelines defined in the Declaration of Helsinki in 1975, were considered in conducting the present study. The participants were assured that their information would remain confidential and that they could leave the study at any time. All participants were informed about the objectives, procedure, and implications of the study, and written consent was obtained from all of them.

Funding
This research did not receive any grant from funding agencies in the public, commercial, or non-profit sectors.