{'fraction_non_alphanumeric': 0.028641518366292956, 'fraction_numerical': 0.013858105054765337, 'mean_word_length': 4.96483121550507, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Systematic Review of Extended Reality (XR) for Understanding and Augmenting Vision Loss

Over the past decade, extended reality (XR) has emerged as an assistive technology not only to augment residual vision of people losing their sight but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis. To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. Here we present a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology to further visual accessibility. In contrast to other reviews, we sample studies from multiple scientific disciplines, focus on augmentation of a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users. We summarize prominent findings from different XR research areas, show how the landscape has changed over the last decade, and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the suitability and usability of different XR-based accessibility aids. By broadening end-user participation to early stages of the design process and shifting the focus from behavioral performance to qualitative assessments of usability, future research has the potential to develop XR technologies that may not only allow for studying vision loss, but also enable novel visual accessibility aids with the potential to impact the lives of millions of people living with vision loss.


Introduction
In recent years, rapid technological advances have led to an increase in the number of assistive technology and electronic mobility aids for people with blindness or low vision (BLV) 1,2,3,4 .
These assistive devices use various sensors (e.g., cameras, depth and ultrasonic sensors) to capture the environment and often apply computer vision and signal processing techniques to detect, recognize, or enhance text, people, and obstacles. Some systems translate vision into alternative modalities, such as converting a visual object into auditory or haptic signals 2 .
Although vision substitution techniques would be essential for people who are completely blind, the majority of people with visual impairment have useful residual vision and prefer to use it to observe the environment 4 . Additionally, some of those who have lost their vision are eligible to receive a visual neuroprosthesis, which is a device that electronically stimulates neurons in the visual pathway to restore a rudimentary form of vision.
State-of-the-art devices facilitate the implementation of complex computer vision algorithms to drastically improve the functionality of residual vision. Specifically, extended reality (XR) is a rapidly advancing technology with direct contributions to our understanding of low vision and an inherent ability to augment residual vision in a useful way. XR is a universal term that encompasses virtual reality (VR), augmented reality (AR), and other immersive mixed reality (MR) environments 5 . While there is some overlap among these terms, experiments with VR technology generally allow researchers to monitor participants in a controlled 3D environment, whereas AR devices integrate and enhance real-world surroundings.
To make the best use of these emerging technologies, it is valuable and timely to understand the state of this research and identify any shortcomings that are present. This includes identifying visual enhancements and information processing techniques that have already been successful at improving visual function and quality of life for their end users. It is also valuable to take into account human factor considerations, such as the individual preferences and accessibility needs of people with different levels of residual vision and underlying conditions. Accommodating these individual differences may determine the success and usability of XRbased visual aids in the near future.
In serve of this, we make three contributions: • We provide a systematic literature review of 227 publications from 106 different venues assessing the potential of XR technology development to further visual accessibility. In contrast to other recent reviews 2,6,4 , we sample studies from multiple scientific disciplines, focus on accessibility aids that visually augment a person's residual vision, and require studies to feature a quantitative evaluation with appropriate end users.
• We perform a meta-analysis that gives a holistic view of the prevalent types of XR technologies, approaches, and tasks used in BLV research, and shows how the landscape has changed over the last decade.
• We summarize prominent findings from different XR research areas and identify scientific gaps in the literature. Specifically, we highlight the need for real-world validation, the broadening of end-user participation, and a more nuanced understanding of the suitability and usability of different XR-based accessibility aids.

Related Work
A number of review articles have reported on the potential and limitations of XR technologies for people with visual impairment. These reviews highlighted a multitude of sensor-based technologies, ranging from smartphones 2 to VR head-mounted displays (HMDs) 4,7 , which could be used to recognize commercial products 6 , detect obstacles and reduce navigation time 8,4 , or support social interactions 9 .
However, these articles also pointed to several gaps in the literature and suggested potential avenues for future research. On the technology side, some studies suggested to use smart clothing 8 for nearby obstacle detection and to integrate devices with existing Internet of Things (IoT) infrastructure 6 . On the behavioral side, Kelly and Smith 10 lamented that most studies in their review lacked methodological rigor. More recently, Brule et al. 3 highlighted the need for adequate quantitative empirical evaluation by involving BLV end users in the design process. This sentiment was shared by Butler et al. 1 , who further highlighted the need to broaden application areas and ask for more in situ evaluation. These works demonstrate that systematic reviews can highlight key issues in accessibility research and provoke the field to reflect and improve on their practice. However, all of the above reviews focused mostly on technology that offered nonvisual feedback to the user (e.g., via text-to-speech software or vibrotactile devices), such as electronic travel aids, electronic orientation aids, position locator devices, and sensory substitution devices. Although such devices can form the basis of practical accessibility aids, no systematic review has yet focused on XR technology that uses vision as the primary feedback mechanism; for instance, by visually augmenting the scene in an AR display to enhance the end user's residual vision.
Given the rapid rate at which new research is being conducted in the use of these new assistive technologies, we consider it timely to review the field in order to better understand the nature and potential of this research.

Scope
The goal of this review is thus to summarize recent research in XR applications for people with BLV and identify trends that can inform the development of future assistive technologies. This includes quantifying the number of studies, summarizing the major findings, identifying gaps in current practices, and making a number of specific recommendations for future research.
Specifically, the goal was to answer a number of questions regarding the use of XR in BLV research: • What are the main types of XR technologies used in BLV research?
• What experimental tasks are studied, and how?
• What are key challenges or scientific gaps that researchers should focus on in the future?
To define "reality" in extended reality (XR), we required that all studies had their end users interact with an immersive environment. Being able to move one's body has been identified as a key factor for immersion 11 ; hence we excluded studies that did not update the environment based on the participant's eye, head, or body movements. Studies with monitors usually do not allow for head or body movements, but can still be used to represent a virtual space if complemented with head or eye tracking. Smart devices (and their applications) were only included if they were used to augment the environment in some way (as opposed to, for instance, augmenting a website). For example, a text magnifier that could be pointed at text in real life would be included, because the first-person view would update based on what the magnifier is pointing at; but an on-screen text magnifying app would not.
While much has been written about the theoretical and technical aspects of accessibility technology, we specifically wanted to focus on studies that incorporated appropriate quantitative empirical evaluation, as suggested by 3,1 . Articles were excluded if not an original work (e.g., review papers), if they solely proposed new technology or methods, or if they were focused on a survey about basic device use (i.e., "how often do you use your smart device to read text"). Survey studies were included if they focused on participants' perceived experience while using a specific technology.

Systematic Review Process
In contrast to a traditional review, systematic reviews can provide a more complete and less biased picture of the type of work being undertaking in the field and point to key challenges moving forward 12 . To help reduce bias and encourage a holistic review, we followed the PRISMA protocol 13 , which is a method for systematically searching databases with a list of keywords and documenting every step ( Figure 1). This includes reporting the number of papers excluded from further analysis along with the reasons for exclusion.
To cover a large body of research independent of their publication venue, we searched three databases (Google Scholar, IEEE Xplore, and PubMed) on the 17th of January, 2022. Each search included different keyword pairs (  Figure 1. PRISMA flow diagram. The results from three databases (Google Scholar, IEEE Xplore, and PubMed) were searched to identify work that combined XR technology with low vision research. After removing duplicates, improperly dated studies, and studies that did not involve human subjects research, we ended up with 227 articles to be included in the review.
The remaining 4, 807 articles were reviewed by the research team and assessed for eligibility.
A total of 4, 580 papers were manually removed. The majority of these (n = 3, 522) were deemed outside the scope of the review as they presented a visual accessibility prototype that (even though it may operate on vision as an input modality) offered only nonvisual feedback to the user (see Section 3.1). In addition, we had to exclude a large number of theoretical studies that did not evaluate their proposed design on appropriate end users (n = 952). Furthermore, we removed 73 papers not available in English and 33 papers that could not be found online (most of these turned out to be manually entered citations on Google Scholar).
The remaining 227 studies were included in the review.

Interactive Collection
The identified articles are available to the reader both as a BIB file (see Supplementary Materials) and as an interactive collection that can be accessed at https://app.litmaps.com/ shared/map/CE0C5D29-8F18-4F2D-9866-0BE1EA4AF288. Created with the free online platform "Litmaps", visitors are able to inspect individual articles and see how they are connected to other articles in the collection.
An example visualization is shown in Fig. ??, where each paper is represented by a circle whose size is proportional to the number of citations the paper received to date. The publication date increases moving left to right, and papers are spread over the y-axis according to how similar their titles are. To calculate title similarity, Litmaps uses Allen AI's SPECTER model 14 , which projects the title of each paper into a 600-dimensional space before it is reduced to one dimension using UMAP. This view can be customized at the above URL, allowing visitors to cluster by keyword, title similarity, or citation count.
venues also included top-tier conferences in mixed reality (e.g., IEEE VR, ISMAR, and UIST),

Publications by Research Area
To get a better understanding of the research areas and applications covered by the corpus of identified papers, we inspected all 227 articles and thematically sorted them into one of four nested categories (Fig. 2): • XR studies for people with low vision (n = 166), defined as having some residual light perception, further divided into: • "perception" studies where XR was used as a tool to study visual perception and behavior of people with low vision (n = 90), often involving lenses, prisms, and HMDs to simulate a specific eye condition; and • "augmentation" studies that primarily focused on the development of novel XRbased assistive devices and augmentation strategies (n = 76). This included techniques like depth encoding, obstacle highlighting, and magnifications.
• XR studies for people who are totally blind (n = 61), defined as having no residual light perception, further divided into: • "perception" studies where XR was used as a tool to study the visual perception and behavior of blind people whose vision was restored with a visual prosthesis (n = 27), often studying the perceptual or behavioral consequences of a specific prosthesis design; and • "augmentation" studies that used XR to augment the artificial vision provided by a visual prosthesis (n = 34), often used to evaluate the effectiveness of specific visual augmentation strategies.
As is evident from Fig. 2, 73% of studies focused on low vision as opposed to blindness; however, within these two broad categories there was a roughly equal focus on augmentation and perception. Interestingly, low vision augmentation studies extensively involved low vision participants (87% of studies), whereas all three other categories predominantly relied on computer simulations of the visual condition under study that would be presented to sighted participants. Roughly 7% of low vision studies included both sighted participants (e.g., to evaluate a prototype using simulated low vision) and people with low vision (PLV) (e.g., to validate their system on appropriate end users). This is in stark contrast to the prosthetic  Figure 2. The 227 articles included in this review were manually assessed and categorized by 1) whether the end users were people with low vision (defined as having some residual light perception) or people who were totally blind (no light perception), 2) whether the article used XR technology to study visual perception and behavior or proposed a new XR augmentation technology, and 3) whether the article involved BLV end users, simulations of the relevant impairment condition, or both.
vision studies, none of which involved both sighted and blind participants.
To get a better understanding of the main types of XR technologies and experimental tasks used in BLV research (see research questions in Section 3.1), we screened every article in the collection to identify which XR display type was used, which experimental task was studied (and how), and whether BLV end users were involved (Table 4).  Table 4. Experimental tasks studied, extent of BLV end-user involvement, and XR display type used. Note that publications involve end users in multiple ways. If more than one task was studied, or more than one display type used, the more rigorous metric was used. A: augmentation, P: perception.
We were also interested in knowing whether these studies were conducted with input or feedback from BLV end users by reporting at least one of the following: • conduction of a pre-study qualitative assessment (e.g., surveys, questionnaires, or interviews) with BLV participants by the study authors, which was used to inform the design of a device/application; • evaluation of perceptual or behavioral performance of the proposed simulation, device, or application with BLV end users; • conduction of a post-study qualitative assessment (e.g., surveys or interviews) with BLV participants by the study authors, which was used to report about the usability of a device/application.
These numbers are summarized in Table 4. While 76 studies (33%) used BLV end users to evaluate performance, most prosthetic vision studies (95%) did not. Of the six studies that recruited bionic eye users, none consulted with BLV users about their information needs, and only two studies based their work on previous findings about the information needs of prosthesis users 27,28 . Additionally, while many studies used BLV participants, very few (n = 31) conducted post-study qualitative assessments.
While all of these device types have been used in low vision research, prosthetic vision studies have so far been restricted to monitors and VR/AR wearables.

Publications by Year
Additionally, we wanted to know how the field has progressed over the last decade. Fig. 5 summarizes the number of studies for each of our four major groups. We found that the overall number of papers in the corpus has been steadily increasing over the last decade, largely driven by increasing interest in XR for low vision (augmentation: 500.0% increase, perception: 141.7% increase since 2010).

Research Areas
XR is a rapidly advancing technology that may not only enable near-future visual accessibility aids but may also serve as a tool to study visual perception and behavior of people with blindness or low vision. Activity in this multidisciplinary space is not just confined to human-computer interaction, but also includes research in cognitive psychology and visual neuroscience as well as clinical applications.
In this section, we summarize the main research activities and findings across the four nested categories introduced in the previous section. We also aim to identify trends that can inform the development of future assistive technologies.

XR for Studying Perception & Behavior of People with Low Vision
XR technology is increasingly being used to study visual perception and behavior of people with low vision (PLV) (39.6% of papers in our collection). Visual impairments such as age-related macular degeneration (AMD), glaucoma, and retinitis pigmentosa (RP) produce scotomas; that is, area(s) of the retina where the functioning of retinal cells is altered or diminished 29,30 .
Scotomas can lead to changes in visual function such as visual field loss (VFL), which may affect perceptual or behavioral performance 29,30 . Most of the studies in this category attempted to measure visual function either by recruiting PLV for testing a specific task (e.g., 31,32 ) or by using low-vision simulations with sighted participants (e.g., 29,33  view based on where the user is looking ("gaze-contingent display"), which can be used to simulate specific eye conditions in real time (e.g., 39,33,29 ; also known as "altered reality" 40 ).
While VR and AR headsets allow for similar experimental designs, researchers have direct control of the environment when using VR. The primary advantage of this approach is the ability to flexibly remove, add, or modify many different features of visual input. Lin and colleagues 54 undertook the VR study with the largest sample size by recruiting 21 participants to perform a reading task while wearing a VR HMD integrated with closed circuit television (CCTV) magnification software. The CCTV software allowed participants to control the size, brightness, and contrast of the screen. The study showed that the HMD VR-based system led to better reading acuity compared to using a regular screen 54 . These studies highlight how XR can be a tool for training and rehabilitation of skills for PLV.

Common Limitations
An open question is to which extent low-vision simulations match the visual experience of real  39 ). In addition, sighted participants recruited for SLV typically ranged between 20 and 30 years of age, which is much younger than most people with central vision loss due to AMD 55 . Perceptual and behavioral differences between SLV participants and PLV may therefore be partially due to age difference 56 . Future work could focus on a more direct comparison between the behavioral performance of sighted participants viewing SLV with real PLV.
A related limitation is the relative lack of PLV involvement in this line of research. While 75 of the 90 studies in this category referenced at least one previous study involving PLV (Table 4), we found only one study that grounded their simulation directly in clinical data (e.g., 29 ). In addition, only a few studies aimed to assess the quality of their simulation by comparing performance to PLV. Future studies could thus work more directly with PLV and/or rehabilitation specialists, which may allow for a deeper understanding of how most simulations differ from the daily challenges that PLV have to deal with.

XR for Augmenting the Residual Vision of People with Low Vision
Another 33.5% of papers in our collection focused on the use of XR technology to augment and enhance the residual vision of PLV. This can range from handheld or wearables magnifying devices, to applications for smartphones and tablets, to wearable devices like HMDs and smartglasses. Whereas VR allows for PLV to experience otherwise unsafe tasks in a controlled virtual environment, AR is better suited as a real-life visual accessibility aid 57 , as it is allows for real-time interaction with an overlay of the real and digital world (similar to a hearing aid).
Augmentation studies in this category focused on a variety of tasks, ranging from reading to face recognition (e.g., 58,59,60 ) and obstacle avoidance (e.g., 43,61,62 ). Similar to the previous section, most of the studies in this category evaluated their augmentation prototype either directly on PLV (e.g., 60,63 or indirectly by using low-vision simulations with sighted participants (e.g., 64,65,66 ). Some studies also used both (e.g., 67 ).

XR for Augmenting Simulated Low Vision
A small number of studies (n = 17) used SLV not just to study perception and behavior of PLV, but to find ways to improve it. For instance, low-level image manipulations such as increased text magnification and contrast were found to lead to faster reading speeds 68 , and enhancing the contours of faces and objects in a visual search task led to faster search times for older participants 69 . Interestingly, magnification was more beneficial for simulated blurry vision compared to a simulated scotoma 68 , whereas contrast enhancement affected reading speed equally across SLV conditions. Similarly, temporal subsampling of an image ("image jitter") was shown to improve peripheral acuity, word recognition, and facial emotion discrimination 70,71 .
Many of these studies aimed to understand how smart glasses could be used to help PLV (e.g., 64,43,67 (Fig. ??A). The enhancement being tested was the Laplacian edge detection method, where a positive method enhanced edges while a negative method enhanced the surrounding of edges Contrast sensitivity thresholds had improved with the enhancement method 64 . Huang and colleagues 43 tested 24 normally sighted participants on a navigation task with a voice based sign reading application for the Hololens. All participants wore goggles modified with occlusion foils during the task to simulate reduced acuity. Results indicated that participants walked more slowly and took more time with the sign-reading application. Overall, participant walked on more direct paths and were more confident with the application.

XR for Studying Augmentations for Low Vision Participants
Although results from SLV studies are notable, the ultimate goal of an XR accessibility aid should be to probably improve the residual vision of real PLV. The majority (n = 64) of studies in this category thus evaluated their prototypes on appropriate end users. search for an object, or shoot a moving target while wearing a HTC VIVE, eleven participants completed the tasks much faster and more accurately with SeeingVR than without the application 76 . "ForeSee" 72,77 is an AR application that uses a combination of low-level image enhancement methods (e.g., magnification, contrast enhancement, edge enhancement) for reading text in near-and far-distance viewing conditions ( Fig. ??B). "CueSee" 73 is an AR application to enhance recognition of products on a grocery shelf with the help of five different visual cues including magnification, color enhancement, flashing bounding boxes, and rotation ( Fig. ??C). Participants identified items on a grocery shelf significantly faster using CueSee than without it.
Other groups focused on gaze-contingent image manipulations. For instance, Calabrese and colleagues 60 were able to improve face recognition for people with AMD via a magnification effect that could highlight and enhance the appearance of a face based on where the user was looking.
Enhancements for mobility, safety, and navigation have been studied using PLV wearing modified glasses or HMD systems. Houston and colleagues 63 tested the ability of people with VFL to navigate a virtual mall while wearing specially designed glasses that could expand the binocular visual field by up to 40 degrees (peripheral prisms). Twenty-four participants were asked to report obstacles and pedestrians while navigating the virtual mall. Interestingly, the detection of hazards on the same side of the visual field defect improved significantly for most participants, even without training 63 . In another study, participants with various diagnosed forms of visual impairment were able to safely complete a stair navigation task with the help of an AR HMD designed to highlight stair edges 77 .
Whereas most studies focused on enhancing a user's residual vision, others built custom HMDs to simplify the visual scene. A notable example is the work by van Rheede and colleagues 65 , which built an HMD integrated with an infrared depth camera to create a depth map, which was relayed to the user as a grayscale map: the closer the obstacle, the brighter the representation on the display (Fig. ??D). Participants were then instructed to avoid foam obstacles while navigating a hallway; six of the eleven participants completed the obstacle course without any collisions.
A number of studies also assessed the usability of the proposed XR technologies (n = 19).
One study reported that most PLV preferred a compact device similar to a regular pair of glasses with buttons for inconspicuous interactions 79 . Another study pointed to the portability of a head-mounted system paired with a smartphone as a camera as the preferred form factor for a reading aid 80 . The ForeSee work 72 highlighted the need to give users the option to choose from several enhancement modes. Another AR study found that alphanumeric representation of information may be better for those with relatively higher acuity, whereas symbolic representation may be better suited for those with worse acuity 81 . Audio feedback was generally liked by participants as well 75 .

Common Limitations
Although most works evaluated their XR prototype on PLV as a proof of concept, relatively few studies (n = 20) recorded participant feedback after the study was conducted PLV (e.g., 76,75,82,83 ). However, this may be an important step towards designing more usable accessibility aids that are sensitive to the information needs of PLV 4 . For instance, Williams et al. 84 compared sighted and blind navigation and found that both groups understand navigation differently, leading sighted people to struggle in guiding blind companions. In addition, people with BLV use a combination of devices and technology to complement their existing orientation and mobility skills 84 , which may lead to a wide variety of navigation styles 85,4 .
The future XR devices being used to help PLV with daily tasks sounds promising. Further collaboration between researchers and end users could benefit device design by augmenting the visual environment based on user-specific needs.
Despite demonstrating an improvement in task performance, many studies reported an in-crease in trial completion time (e.g., 65,67

XR for Studying Perception & Behavior of People with Prosthetic
Vision XR technology has not only been used to augment the vision of PLV but also to study the rudimentary vision restored to blind people by a visual neuroprosthesis ("bionic eye"; 15.0% of papers in our collection) 86 . Similar to conventional AR HMDs, visual prostheses typically contain an external camera mounted on a pair of glasses that is used to relay the visual scene to the user. However, in contrast to conventional AR HMDs, visual prostheses also consist of an implantable microstimulator (implanted in the eye or the visual cortex) which decodes the visual information and electrically stimulates neurons in the visual pathway to evoke visual percepts ("phosphenes"). Existing bionic eyes generally provide an improved ability to localize high-contrast objects and perform basic orientation & mobility tasks (e.g., 87 ). While this could be considered a rudimentary form of AR on its own, a good number of studies used VR to simulate the perception produced by these devices.

XR for Simulating Prosthetic Vision
To investigate functional recovery and experiment with different implant designs, researchers have been developing XR prototypes that rely on simulated prosthetic vision (SPV) (n = 26).
The classical method relies on sighted subjects wearing a VR headset, who are then deprived of natural viewing and only perceive phosphenes displayed in the HMD. This viewing mode has been termed "transformative reality" 88,89 (as opposed to "altered reality", which is typically used to describe SLV approaches 40 ). This allows sighted participants to "see" through the eyes of the bionic eye user, taking into account their head and/or eye movements as they explore a virtual environment 90 . . Surprisingly, although the majority of the tasks used HMDs, none of the studies allowed for a fully immersive environment that would allow the subject to walk around and interact with it. Studies in this category primarily used SPV to study basic behavior in these tasks, but some studies also used these tasks to focus on another behavior. One example of this is the work by Sanchez-Garcia et al. 101 . In this work, participants (n = 24) were tasked to find and recognize objects in a scene with different field of views (FOVs) (20 • , 40 • , or 60 • ) and number of phosphenes (200 or 500). The authors showed counter-intuitive results, with a higher FOV resulting in significantly worse performance and longer recognition times. However, they argued that phosphene density may be more important for object recognition than FOV, which is consistent with earlier findings 102 .
Another study 103 relied on AR smart glasses to simulate the artificial vision provided by the PRIMA subretinal implant 104 . This device was developed for people with geographic atrophy as commonly experienced with AMD, where vision is first lost in the macula. To simulate this, the authors needed to combine SPV in the macula and natural vision in the periphery. The authors accomplished this by using AR smartglasses with black tape occluding the central FOV so only the LED overlay was visible in this area. With this setup, they were able to make testable predictions about the visual acuity to be expected from PRIMA 104 , which is currently in clinical trials.
Lastly, an increasing number of studies are focusing on spatial cognition tasks, such as obstacle avoidance 105,106,107 and wayfinding 108 . By design, these tasks require a more immersive setup that allows for the incorporation of head and eye movements as well as locomotion 109 .
The majority of studies identified (n = 5) incorporated a fully immersive design for their task, while the remaining studies (n = 2) used VR HMDs but required their subjects to sit/stand in place and use a keyboard/controller to move. The studies also used relatively low subject counts ranging from 5 to 17 subjects (µ = 10.57 ± 3.82). The majority of tasks were simply "proof of concept" experiments showing that users were able to navigate effectively with SPV.

XR for Studying Prosthesis Users
Rachitskaya et al. 28 was the only XR study to recruit real bionic eye users and to mention consultation with the BLV community during development, having utilized an interdisciplinary team that incorporated ophthalmologists and rehabilitation specialists. Since there currently is no standardized procedure for vision rehabilitation across different Argus II implantation centers, Rachitskaya and colleagues 28 developed a Computer-Assisted Rehabilitation Environment (CAREN), which consists of a motion capture system, control software with a 180°curved projection screen, a motion platform, and a treadmill. Participants (n = 8) donned a harness, had access to handrails on the treadmill, and were accompanied by a physical therapist. Af-ter using CAREN twice weekly for 4 weeks, participants showed significant improvements in walking speed and object localization, demonstrating that immersive technology may provide a solution for the standardization of effective rehabilitation approaches to augment bionic eye performance.

Common Limitations
An open question is to which extent prosthetic vision simulations match the visual experience of real prosthesis users. Similar to SLV research, many SPV studies base their simulations on crude approximations of prosthetic vision, assuming that each electrode acts as a small independent light source that produces a distinct focal spot of light 111 . However, a growing body of evidence suggests that the vision generated by current visual prostheses is "fundamen- This requires users to scan the environment with strategic head movements while trying to piece together the information 112 . It is therefore unclear how the findings of most SPV studies would translate to real bionic eye users.
Additionally, only a single study in our collection worked with real bionic eye users or rehabilitation specialists 28 . XR may offer a unique method for safe training and rehabilitation, but is severely underutilized in comparison to research on XR for low vision.

XR for Augmenting Prosthetic Vision
Another 15% of papers in our collection focused on the use of XR technology to augment and enhance prosthetic vision, either through simulations (n = 29) or the use of peripherals and extra sensors to extract visual scene information (n = 5).

XR for Augmenting the Visual Scene Using Simulated Prosthetic Vision
A popular trend for simulated prosthetic vision (SPV) is utilizing novel augmentation strategies to aid scene understanding. One approach is using computer vision to enhance certain image features or regions of interest, at the expense of discarding less important or distracting information. Various studies have explored strategies based on visual saliency (e.g., 119 ), background subtraction and scene retargeting (e.g., 120 ), and depth mapping to highlight nearby obstacles(e.g., 121,122,123 ). For instance, McCarthy and colleagues 122 used an RGB-D camera mounted on a pair of AR glasses to augment the visual scene with depth information (Fig. ??B). The study used a custom augmented reality setup utilizing an HMD with an attached stereo camera. The images from the camera were sent to a laptop on the participant's back, and were processed into a simplified model of bionic vision using a pixel display with twenty phosphenes. Among their tested image processing strategies, augmented depth proved the most effective at highlighting hazards in the path; this mode modified the depth information from the stereo cameras to detect objects while simultaneously removing the ground from the scene. They found a significantly reduced rate of collisions, even in the presence of low-contrast trip hazards. These findings were later evaluated with real bionic eye users 124 .
The majority (n = 18) of SPV studies in this category used monitors (n = 3), VR headsets (n = 14), and AR glasses (n = 1) to improve performance on recognition tasks, such as identifying faces 97,125,126 , text 127,128 , and objects 120,129 . ??A). They found that using a region-of-interest view improved acuity while a wide FOV was better for mobility, highlighting the use of testing multiple forms of enhancement with various tasks 102 . In a similar manner to the spatial behavioral SPV studies, the majority (n = 6) of augmented SPV studies also used VR HMDs with two studies using AR smartglasses 135,122,119 .Most of the studies were fully immersive (n = 6), but two used VR HMDs without positional tracking 134,108 . Out of these 8 studies, only 1 study used eye-tracking 102 or presented monocular stimuli 119 . These studies also suffered from relatively low subject counts ranging from 4 to 19 subjects (µ = 10.75 ± 4.62).
The remaining 3 studies identified simulation studies focused on low-level visual functioning 136,137,138 . Two of these studies used AR for enhancing an SPV scene with one using infrared (IR) overlays for counting people/actions in a scene, 137 , and the other using RGB-D cameras for depth overlays in a target localization task 136 . The third study used a VR HMD with eye tracking to study the effects of gaze in a simulated retinal prosthesis 138 . All three studies performed basic low-level visual function testing but suggested the augmentations could be valuable for more advanced scene understanding.

XR for Studying Augmentations for Prosthetic Vision
While a bionic eye is technically itself a technology that augments vision, several studies focused on augmentation strategies outside the basic stimulation patterns of the device. This includes thermal imaging 139,140 , audio-visual cross-modal mapping 141 , and depth detection with object segmentation 123 . For example, Sadeghi et al. 27 tested the ability of bionic eye users to perform a series of practical tasks (e.g., identifying hot objects, estimating the distance to a nearby person) while using a thermal camera. The study highlighted improved performance across all tested tasks, including tasks where thermal integration would be considered an obvious benefit (e.g., identifying the closer side of a hot cup, identifying a missing bowl that was heated), but also tasks such as determining whether an escalator was moving towards or away from them. In another study by Kartha et al. 123 , Argus II users completed various tasks with a distance-filtered input. In this study, the removal of distant clutter was able to improve participant performance across a variety of tasks including size, depth, and walking direction discrimination. While these augmentations were relatively simple, they were still able to improve performance across a variety of tasks. These results are promising, and present the possibility for more advanced augmentation methods to be useful in the future.

Common Limitations
Similar to the previous section, SPV studies used to augment prosthetic vision relied on the same crude approximations of the visual experience of real bionic eye users. Although there is no shortage of publications that demonstrate a proof-of-concept augmentation strategy, only a few studies discussed the usability aspects of their proposed technology (e.g., 27,123 ).
Additionally, very few SPV studies allowed participants to move around in an immersive way (n = 6 out of 29), and all those that did were focused on spatial cognition. Typical real-life scenarios cannot be mastered while stationary, and future studies may benefit from allowing participants to move around their environment. Many studies used SPV to assess the benefit of their proposed technology, but only two used models based on neuroscience, considered gaze (n = 6), or presented monocular stimuli (n = 5).
Because the involvement of real bionic eye users remains limited (500 implantees worldwide) and challenging (e.g., constant assistance, increased setup time, travel cost), it is not surprising that most behavioral results were reported on a relatively small sample size limited to one to five participants. While XR technology in combination with SPV may provide a more cost-effective alternative to prototyping novel augmentation strategies 90 , future studies should consider a more direct comparison between their theoretical predictions and the visual experience reported by real bionic eye users 112 .

The Main Types of XR Technologies Used in BLV Research
VR and AR HMDs were the most common (n = 58 and n = 55, respectively). AR HMDs have a slight advantage over VR given that they can be used in real-life situations, rather than simulations. AR can also be used as an accessibility tool, such as enhancing text (e.g., 43,142 ; similar to VR), or, more notably, highlighting obstacles while navigating a real environment (e.g., 74,65 ).
VR wearables were by far the most popular device type among the studies in our corpus, prevalent in both low vision and prosthetic vision research (Table 4). Interest in these devices has been more or less constant over the last decade, but has seen a recent increase in 2020-21 ( Desktop monitors were another trusted device type with constant interest over the years (Table 6). Monitors can be particularly useful if they are used as a gaze-contingent display to study changes in eye movements (Table 4). Nonelectronic wearables (glasses, goggles, etc.) are the inexpensive, but third most common option in our dataset (e.g., 146,49,147 ).

The Experimental Tasks Studied with XR
For both low and prosthetic vision, visual search and recognition tasks were the most common (n = 114), followed by spatial cognition tasks related to orientation & mobility (n = 78) and low-level visual function testing (n = 35). While acuity is known as the standard for assessing visual function, the degree to which it associates to other tasks still remains to be explored, especially for changes in acuity with SLV and SPV. Future work would benefit from comparing performance across tasks not limited to low-level visual function. Future development of AR HMDs could potentially improve BLV function for a range of tasks rather than one.
With the exception of low vision augmentation studies, it is worth noting that most work involved sighted participants viewing SLV (n = 95 out of 166) or SPV (n = 55 out of 61).
While this may reflect difficulty recruiting PLV, simulations have so far proved a valuable tool to enable large-scale behavioral studies and the quick prototyping of novel augmentation strategies.

Key Challenges and Scientific Gaps
XR technologies have seen major improvements in functionality and costs over the past decade, and the interest to use these devices in BLV research has risen accordingly.
However, there are a number of challenges and limitations that were common across the different research areas: • It is unclear to what extent simulations of low vision and prosthetic vision match the visual experience of people with BLV. Only a few studies thoroughly grounded their simulations in real patient data (e.g., 29,117 , and studies using simulations had much younger participants than the target BLV group (µ = 29.2 and µ = 59.72 years, respectively). Many studies used crude approximations of the underlying eye condition and ignored the immersiveness of their simulation. For example, only 40% of studies used a gaze-contingent display, meaning that sighted participants could artificially increase their FOV through eye movements. Ignoring this aspect could result in simulated participants performing much better on tasks than those with the condition being studied.
Addressing this could lead to more insightful simulations.
• While many studies (n = 76) used BLV participants to test the performance of a new system, very few studies consulted with the BLV community (n = 46) during the early phases of their study. Instead, the majority of studies focused on technical developments such as exploring different computer vision and enhancement techniques (e.g., 148,65,122 ) and reporting quantitative measures such as mobility efficiency and errors, obstacle detection rates, and clinical visual measurements 63,74,149 . Less emphasis has been placed on understanding the usability and suitability of these aids in people with different levels of residual vision and underlying conditions, or whether or not these accessibility aids address the information needs of BLV users.
• Even fewer studies (n = 31) collected BLV participants' opinions after the study. While the proposed systems may have improved performance in specific tasks, the systems must also be user-friendly and avoid steep learning curves. Struggling to adapt to new technologies may limit device use or prevent end users from acquiring the necessary skill set to fully utilize a new accessibility aid. Surveying user preferences at the end of a performance evaluation could lead to insights that may increase usability and adoption rates.
Unfortunately, to the best of our knowledge, none of the reviewed devices and applications have found widespread adoption. While the underlying reasons may be nuanced and manifold, many studies in our collection seem poised for success in the near future (e.g. 64,122,76 ). We are hopeful that addressing the highlighted gaps in the existing literature will lead to increased usability of different XR-based accessibility aids.

Conclusion
In conclusion, our systematic review has highlighted the benefits of XR technology for BLV research, but challenges still remain. By broadening end-user participation to early stages of the design process and shifting the focus from behavioral performance to qualitative assessments of usability, future research has the potential to develop XR technologies that may not only allow for studying vision loss, but also enable novel visual accessibility aids with the potential to impact the lives of millions of people living with vision loss.