{'fraction_non_alphanumeric': 0.030469277515192437, 'fraction_numerical': 0.019285955435516543, 'mean_word_length': 4.961509433962264, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
No effect of auditory–visual spatial disparity on temporal recalibration

It is known that the brain adaptively recalibrates itself to small (∼100 ms) auditory–visual (AV) temporal asynchronies so as to maintain intersensory temporal coherence. Here we explored whether spatial disparity between a sound and light affects AV temporal recalibration. Participants were exposed to a train of asynchronous AV stimulus pairs (sound-first or light-first) with sounds and lights emanating from either the same or a different location. Following a short exposure phase, participants were tested on an AV temporal order judgement (TOJ) task. Temporal recalibration manifested itself as a shift of subjective simultaneity in the direction of the adapted audiovisual lag. The shift was equally big when exposure and test stimuli were presented from the same or different locations. These results provide strong evidence for the idea that spatial co-localisation is not a necessary constraint for intersensory pairing to occur.


Introduction
In many circumstances people experience external events by a number of diVerent sensory modalities. For example, when someone is talking, there is auditory and visual information that is initially processed by specialized neural pathways. Ultimately, though, the diVerent sensory signals are integrated into a coherent multimodal percept of the speaker. Many behavioural and neurophysiological studies have emphasized the importance of spatial co-localisation and temporal synchrony for intersensory pairing to occur (e.g., Welch and Warren 1980;Bedford 1989;Stein and Meredith 1993;Radeau 1994;Bertelson 1999;Welch 1999). However, there is accumulating evidence that some intersensory phenomena may not require spatial alignment (Welch et al. 1986;Scheier et al. 1999;Morein-Zamir et al. 2003;Murray et al. 2004;Teder-Salejarvi et al. 2005;Vroomen and Keetels 2006;Keetels et al. 2007). In the present study, we explored the importance of spatial alignment for audio-visual (AV) temporal recalibration.
Temporal recalibration refers to the phenomenon that the brain adapts itself to (small) temporal asynchronies. In a multi-modal percept, it usually appears that information from diVerent senses arrive at the same time. This occurs, despite the fact that there are natural asynchronies between the senses caused by diVerences in signal transduction time through air and diVerences in neural transmission time. At least two options are available to handle these asynchronies: one is concerned with immediate corrections, the other is important for adaptation on a longer time scale. As concerns the immediate eVect, several studies have shown that the brain corrects for small AV temporal asynchronies by shifting one or both modalities on the time scale so that the temporal discordance is reduced. For example, when a sound and a light are presented at slightly diVerent onset times (usually in the order of »100 ms), the temporal asynchrony is reduced by a capturing eVect of the light by the sound; a phenomenon called temporal ventriloquism (Scheier et al. 1999;Fendrich and Corballis 2001;Morein-Zamir et al. 2003;Vroomen and de Gelder 2004;Stekelenburg and Vroomen 2005;Vroomen and Keetels 2006). Temporal ventriloquism can, for example, be demonstrated by the use of a visual temporal order judgment (TOJ) task in which participants are presented two lights at various stimulus onset asynchronies (SOAs) and judge which light came Wrst. By presenting a sound before the Wrst and after the second light, the just noticeable diVerence (JND) improves (i.e. participants become more sensitive), presumably because the two sounds attract the temporal occurrence of the two lights, and thus eVectively pull the lights further apart in time (Scheier et al. 1999;Morein-Zamir et al. 2003;Vroomen and Keetels 2006).
There are also long-term eVects reXecting an adaptive change to AV asynchrony, a phenomenon called temporal recalibration (Fujisaki et al. 2004;Vroomen et al. 2004). For example, Vroomen et al. studied temporal recalibration by exposing participants to 3 min of sound and light Xashes with a constant time lag, after which an AV TOJ or AV simultaneity task was performed. Following exposure, observers were given AV test stimuli and judged whether the sound or the light came Wrst, or whether the sound and light were simultaneous or successive. The results showed that the point of subjective simultaneity (PSS), the point of perceived temporal alignment between the sound and the light, was shifted in the direction of the exposure lag. So, following exposure to a train of sound-Wrst stimulus pairs, participants perceived sound-Wrst trials as more simultaneous than after light-Wrst exposure. Fujisaki et al. (2004) demonstrated similar Wndings and also provided somewhat mixed evidence that temporal recalibration may generalize to diVerent test stimuli than the ones presented during exposure. The authors adapted participants to asynchronous tone-Xash stimulus pairs and later tested them on the "bounce" illusion (Sekuler et al. 1997). In this illusion, two visual targets that move across each other can be perceived either to bounce oV or to stream through each other. A brief sound presented at the moment that the visual targets coincide generally biases visual perception in favour of a bouncing motion, while without sound observers tend to report a streaming percept. Following exposure to asynchronous sound-light pairs, the optimal delay for obtaining the bounce illusion was shifted in the same direction, but in other conditions, the magnitude of the after-eVect was smaller for some of the cross-adaptation conditions. Temporal recalibration may also occur between other modalities than AV. For example, Navarra et al. (2006) demonstrated audio-tactile temporal recalibration by exposing participants to streams of brief auditory and tactile stimuli presented in synchrony, or else with the auditory stimulus leading by 75 ms. Rather than a shift in the PSS, they observed that the JND to resolve audio-tactile temporal order was larger after exposure to the desynchronized streams than after exposure to the synchronous streams. The authors argued that the temporal window for integration was widened due to audio-tactile asynchrony.
The goal of the present study was to explore whether spatial disparity between a sound and light aVects temporal recalibration. According to the "common notion" of intersensory pairing, intersensory eVects should be bigger when the individual components of a multisensory stimulus come from the same location (e.g. Welch and Warren 1980;Bedford 1989;Stein and Meredith 1993;Radeau 1994;Bertelson 1999;Welch 1999). However, Vroomen and Keetels (2006) demonstrated that, at least for temporal ventriloquism, spatial correspondence between sound and light is not important. In their study, a visual TOJ task was used with a sound presented before the Wrst and after the second light. Temporal ventriloquism manifested itself as an improvement in the JNDs but, crucially, the improvement was unaVected by whether the sounds came from the same or a diVerent position as the lights, whether the sounds were static or moved, or whether the sounds and lights came from the same or opposite sides of Wxation. Keetels et al. (2007) further examined how principles of auditory grouping (Bregman 1990) relate to intersensory pairing. They embedded two sounds that normally enhance sensitivity on the visual temporal order judgement task in a sequence of Xanker sounds, which either had the same or diVerent frequency, rhythm, or location. In all experiments, temporal ventriloquism only occurred when the two capture sounds diVered from the Xankers, thus demonstrating that intramodal grouping of the sounds in the auditory stream took priority over intersensory pairing. By combining principles of auditory grouping with intersensory pairing, they also demonstrated that the capture sounds could, counter-intuitively, be more eVective when their locations diVered from that of the lights rather than when they came from the same position, thus demonstrating that sound location mattered for auditory grouping, but not intersensory pairing.
Here we examined whether, like in temporal ventriloquism, spatial disparity is ignored when temporal recalibration is at stake. Participants were exposed for 3 min to a train of asynchronous sounds and lights that came either from the same or a diVerent location. Following exposure, participants performed an AV TOJ task with sounds and lights from either the same or diVerent location. This design allowed us to address two questions. First, we could test whether temporal recalibration is aVected by spatial disparity between the sounds and lights. Recalibration is usually considered to be a low-level perceptual learning phenomenon necessary for re-alignment of the senses . Observing an after-eVect following exposure to spatially disparate sound-light pairs would provide strong evidence that spatial co-occurrence is, even at this early stage, not necessary for intersensory pairing to occur. Secondly, the use of an exposure-test design allowed us to introduce a change between the exposure and test stimulus so that we could test whether after-eVects generalize to diVerent test stimuli. Here we tested whether spatial similarity between the exposure and test sound aVects after-eVects. If spatial co-location plays no role in intersensory pairing, one would expect stimulus generalization across space to be complete.

Participants
Thirty students from Tilburg University received course credits for their participation. All reported normal hearing and normal or corrected-to-normal vision. They were tested individually and were unaware of the purpose of the experiment. The study was carried out along the principles laid down in the Helsinki Declaration and informed consent from the participants was obtained.

Stimuli
Participants sat at a table in a dimly lit and sound-proof booth. Head movements were precluded by a chin-rest. Visual stimuli were presented by a green LED, positioned at central location, at 70 cm from the subject's eyes (diameter of 0.5 cm, luminance of 40 cd/m 2 ). Auditory stimuli were 88 dB sound bursts presented by one of two loudspeakers; one directly behind the green LED and the other placed laterally at 70 cm distance on either the far left or the far right of the subject (i.e., 90 degrees of spatial separation between the sound and light). See Fig. 1 for a schematic view of the experimental set-up. The sounds and lights each had a duration of 10 ms. A small red LED, placed 2 cm below the green LED, was constantly lit during the experiment and served as Wxation point.

Design
Three within-subjects factors were used: exposure lag during the exposure phase (¡100 and +100 ms, with negative values indicating that the sound was presented Wrst), location of the sound during exposure (exposure-sound central or lateral) and SOA between the sound and light of the test stimuli (¡240, ¡120, ¡90, ¡60, ¡30, 0, +30, +60, +90, +120, and +240 ms, with negative values indicating that the sound came Wrst). The location of the test sound (central or lateral) was a between-subjects variable. Half of the participants were tested with central test sounds, the other with lateral test sounds. These factors yielded 44 equi-probable conditions for each location of the test sound (2 £ 2 £ 11), each presented 12 times for a total of 528 trials. Trials were presented in eight blocks of 66 trials each. The exposure lag and the location of the exposure sound were constant within a block, while the SOA between sound and light varied randomly. The order of the blocks was counterbalanced across participants. In half of the blocks with a lateral exposure sound, the sound came from the left, in the other half from the right. The lateral test sounds were presented from the same side as during exposure.

Procedure
Each block started with an exposure phase consisting of 240 repetitions (»3 min) of a sound-light stimulus pair (ISI = 750 ms) with a constant lag (¡100 or +100 ms) between the sound and the light. After a 2,500 ms delay, the Wrst test Fig. 1 Schematic illustration of the experimental conditions. In the exposure phase, the subject was exposed to a sound-light pair with 100 ms temporal oVset (either sound-Wrst or light Wrst). During exposure, sounds were either presented from central (a, c) or lateral location (b, d). In the test phase, sound-light pairs were presented with a particular SOA ranging between ¡240 and 240 ms, with negative values indicating that the sound was presented Wrst. Sounds of the teststimulus pair either came from central (a, b) or lateral location (c, d ) trial then started. To ensure that participants were Wxating the light during exposure, they had to detect the occasional occurrence of the oVset (150 ms) of the Wxation light (i.e., a catch trial). Participants then pushed a special button.
The test phase consisted of two parts: a short AV reexposure phase followed by three AV test trials of which the temporal order of the sound and light had to be judged. The re-exposure phase consisted of a train of ten soundlight pairs with the same lag, ISI, and sound location as used during the immediately preceding exposure phase. After 1 s, the three AV test trials were presented with a variable SOA between the sounds and lights. The participant's task was to judge whether the sound or the light of the test stimulus was presented Wrst. An unspeeded response was made by pressing one of two designated keys on a response box. The next test stimulus was presented 500 ms after a response, and the re-exposure phase of the next trial started 1,000 ms after the response on the third test stimulus.
To acquaint participants with the TOJ task, experimental blocks were preceded by four practice blocks in which no exposure preceded the test trials. The Wrst two practice blocks were to acquaint participants with the response buttons, and consisted of 16 trials in which only the largest SOAs were presented ( §240 and §120). During this part, participants received verbal feedback ("correct" or "wrong") about whether they gave the correct response or not. The next two practice blocks consisted of 66 trials in which all SOAs were presented 6 times randomly without verbal feedback. Total testing lasted approximately 2.5 h.

Results
Trials of the practice session were excluded from analyses. The proportion of "light-Wrst" responses was for each participant calculated for each combination of exposure lag (¡100, +100 ms), location of the exposure sound (central, or lateral), location of the test sound (central, or lateral) and SOA (ranging from ¡240 to +240 ms). Performance on catch trials was Xawless, indicating that participants were indeed looking at the Wxation light during exposure. For each combination of exposure lag, location of the exposure sound and location of the test sound, an individually determined psychometric function was calculated over the SOAs by Wtting a cumulative normal distribution using maximum likelihood estimation. The mean of the resulting distribution (the interpolated 50% crossover point) is the point of subjective simultaneity (henceforth the PSS), and the slope is a measure of the sharpness with which stimuli are distinguished from one another. The slope is inversely related to the just noticeable diVerence (JND) and represents the interval (absolute SOA) at which 25 and 75% visual-Wrst responses were given.
The PSS and the JND data are shown in Fig. 2 and Table 1. Temporal recalibration was expected to manifest itself as a shift of the PSS in the direction of the exposure lag. The temporal recalibration eVect (TRE) was therefore computed by subtracting the PSS following auditory-Wrst exposure from visual-Wrst exposure.
An overall 2 £ 2 £ 2 ANOVA with as within-subjects factors exposure lag, location of the exposure sound and as between-subjects factor location of the test sound was run on the JNDs. None of the eVects was signiWcant (all P > 0.08), except for a second-order interaction between exposure lag, exposure location and test location, F(1,28) = 4.6, P = 0.041. Inspection of Table 1 shows that the diVerences between the JNDs (on average 38.7 ms) were rather small and unsystematic.
The ANOVA on the TREs only showed a signiWcant eVect of exposure lag, F(1,28) = 23.0, P < 0.001, demonstrating, as predicted, that the exposure phase shifted the PSS such that there were more visual-Wrst responses after sound-Wrst exposure than after light-Wrst exposure (i.e. the TRE). The average TRE was 12.9 ms or 6.5% of the exposure lag. The overall size of this eVect corresponds well with previous reports (Fujisaki et al. obtained an average TRE of 12.5%; Vroomen et al. an average TRE of 6.7%). There were, furthermore, no main eVects of location of the exposure and test sound, and the crucial interaction between the location of the exposure and test sound was non-signiWcant (all F < 1). Temporal recalibration thus manifested itself no matter whether exposure sounds came from central or lateral location, and whether the location of the exposure and test sounds was changed or not.

Discussion
The goal of the present study was to address whether spatially co-located AV asynchronous stimulus pairs induce temporal recalibration as much as spatially dislocated stimuli do, and whether spatial correspondence between the exposure and test sound aVects the size of this eVect. Results showed that in all cases, there were clear temporal recalibration eVects as subjective simultaneity was shifted in the direction of the adapted audiovisual lag. The shift was equally big for spatially separated and spatially colocated exposure stimuli. Apparently, spatial separation between the sound and light did not hinder temporal realignment. Stimulus generalization across space was also complete, as the shift in temporal alignment was equally big for when the exposure and test sound came from the same or diVerent positions. The results therefore support the notion that spatial alignment between the senses is unimportant for AV pairing in the temporal domain. The results are also in line with previous reports on temporal ventriloquism (Keetels et al. 2007;Vroomen and Keetels 2006) where it was shown that spatial separation does not aVect the capturing eVect of a light by a sound. Taken together, these Wndings provide strong evidence that spatial co-occurrence is, even at early perceptual stages, not a necessary constraint for intersensory pairing.
One might object, though, that spatial ventriloquism has diminished the potential eVects of spatial discordance. It is well-known that the apparent location of a sound can be shifted towards a visual stimulus that is presented at approximately the same time (Howard and Templeton 1966;Radeau and Bertelson 1978;Welch 1978;Bertelson and Radeau 1981;Bertelson 1994Bertelson , 1999Radeau 1994). Could it be, then, that the AV spatial discordance in our setup was diminished, if not became unnoticeable due to spatial ventriloquism? If so, one may not observe an eVect of spatial separation on temporal recalibration. This argument, though, seems highly unlikely because it is known that spatial ventriloquism dramatically declines whenever spatial separation exceeds approximately 15 degrees (Slutsky and Recanzone 2001;Godfroy et al. 2003). Given that we maximized the spatial separation between the sound and light (i.e., at 90 degrees azimuth), and that informal testing indeed conWrmed that spatial separation was clearly noticeable, it seems safe to assume that spatial ventriloquism did not diminish the eVect of spatial discordance.
One might also ask whether the visual task as used during the exposure phase (i.e., detection of the oVset of visual Wxation) resulted in an attentional shift towards the visual modality. According to the "law of prior entry" (Titchener 1908), attending to one sensory modality speeds up the perception of stimuli in that modality, resulting in a change in the PSS (see also Shore et al. 2001Shore et al. , 2005Spence et al. 2001;Schneider and Bavelier 2003;   2005b). Our visual task might thus result in a shift of the PSS towards more "visual-Wrst" responses. However, this shift should be uniform for all conditions, and given that temporal recalibration is expressed as a diVerence in the PSS between exposure lags, the possible role of attention will be subtracted out. A remarkable aspect of the data is that previous studies have demonstrated that AV temporal order judgements become more sensitive (i.e. smaller JND) when the sound and light of the test stimuli are spatially separated (see also Bertelson and Aschersleben 2003;Spence et al. 2003;Zampini et al. 2003aZampini et al. , b, 2005aKeetels and Vroomen 2005).
Here, there was a small trend in this direction (average JND of 39.1 vs. 38.2 ms, for spatially co-located vs. separated test stimuli, respectively), but the eVect was non-signiWcant. Possibly, we might have picked up this diVerence if the eVect were measured as a within-subjects factor. For the current purpose, though, this was considered to be unpractical because it would have doubled individual testing time. Despite that we did not observe an eVect of AV spatial separation on the JNDs, the data speak on the interpretation on this eVect. At least two explanations have been brought up for the improved temporal sensitivity when the locations of test sound and light diVer. One is that there is more intersensory integration with as a consequence that the temporal discordance is fused; the other is that there are extra spatial cues that help TOJ performance (Spence et al. 2003). Given that our results show that intersensory pairing occurs independent of a spatial mismatch (see also Vroomen and Keetels 2006;Keetels et al. 2007), it seems more likely that the previously observed eVects of spatial separation on temporal sensitivity were induced by the availability of redundant spatial cues rather than fusion per se.
To conclude, our results provide strong evidence for the claim that commonality in space between a sound and light is not relevant for AV pairing in the temporal domain. This may, at Wrst sight, seem unlikely, because after all, most natural multisensory events are spatially and temporally aligned. However, a critical assumption that underlies the idea of spatial correspondence for cross-modal pairing is that space has the same function in vision and audition. This notion, though is arguable, as it has been proposed that the role of space in hearing is to steer vision (HeVner and HeVner 1992), while in vision it is an indispensable attribute (Kubovy and Van Valkenburg 2001). If one accepts that auditory spatial perception evolved for steering vision, but not for deciding whether sound and light belong together, there is no reason why cross-modal interactions would require spatial co-localization. Our results therefore have also important implications for designing multimodal devices or creating virtual reality environments, as they show that the brain can ignore cross-modal discordance in space. Table 1 Mean points of subjective simultaneity (PSSs) in ms, and mean just noticeable diVerences (JND) in parentheses Exposure stimulus pairs were presented with an auditory-visual Lag (AV-lag) of ¡100 and +100 ms with sounds either central or lateral; the location of the test stimulus sound was either central or lateral. The temporal recalibration eVect (TRE) reXects the diVerence in PSSs between the ¡100 and +100 ms audio-visual lags