{'fraction_non_alphanumeric': 0.024634995253808255, 'fraction_numerical': 0.00987659901460019, 'mean_word_length': 4.9720610068835205, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020

This paper presents a systematic literature review of 292 publications from 97 unique venues on touch-based graphics for people who are blind or have low vision, from 2010 to mid-2020. It is the first review of its kind on touch-based accessible graphics. It is timely because it allows us to assess the impact of new technologies such as commodity 3D printing and low-cost electronics on the production and presentation of accessible graphics. As expected our review shows an increase in publications from 2014 that we can attribute to these developments. It also reveals the need to: broaden application areas, especially to the workplace; broaden end-user participation throughout the full design process; and conduct more in situ evaluation. This work is linked to an online living resource to be shared with the wider community.

are blind or have low vision (BLV) [13]. Tactile graphics, along with image descriptions, are now the most common method for proving accessible graphics to BLV people. They find particular use in schools to provide BLV students with access to educational graphics such as STEM diagrams, charts and maps, and are frequently used in Orientation & Mobility (O&M) training. Tactile graphics, however, suffer several drawbacks. They require specialist equipment to produce and they are usually designed by an experienced transcriber, meaning that production is slow and costly. Thus tactiles are not well suited to, for instance, on-the-fly creation of graphics in the classroom, and because of their static, two-dimensional nature they are restricted in the types of graphical information they can directly convey. The use of tactile graphics also requires explicit training and braille literacy, limiting the number of BLV people able to effectively use them.
Consequently, a major focus of assistive technology development has been to develop alternatives to tactile graphics that overcome some of their limitations. For instance, interactive audio labels do away with the need to understand braille and refreshable tactile displays remove the need for printing the tactile graphic. And over the past decade researchers have leveraged emerging technologies including 'maker' technologies such as 3D printing. Another focus of research has been to automate or at least semi-automate the production of tactile graphics or other accessible formats. But despite this activity there has been no broad critical review of the research into these new 'touch-based' technologies for presenting accessible graphics to BLV people. This paper presents a systematic review and critique of research involving new technologies for touch-based graphics for BLV people over the period of 2010 to mid-2020. Systematic reviews are vital as they allow a research field to reflect and improve on best practice [6]. Our review focuses on research investigating new production or presentation methods related to touch-based graphics. By this we mean presentation methods that, like tactile graphics, rely on the reader using the position of fingers or hands to understand the spatial layout of the graphic. While other senses such as hearing can also be used, spatial understanding needs to be driven primarily by touch.
Primary research venues for assistive and inclusive technologies were reviewed, as well as keyword searches for literature outside of these venues. In total, 352 papers were identified based on titles and abstracts. After an initial review, 59 were excluded because arXiv:2102.01273v1 [cs.HC] 2 Feb 2021 they did not meet the selection criteria, leaving 292 papers in the review corpus. These were analysed on the following criteria: primary research contributions; graphic content; application domain; presentation technologies; methodology, including stakeholder involvement; and availability of outcomes.
Contributions: This systematic review contributes to the field of accessible graphics in the following ways: • It is the first systematic review of research relating to touchbased accessible graphics. It critically reviews the work over the past decade, highlighting how the field has evolved. • It summarises the results of studies comparing different touch-based presentation techniques. • It identifies key omissions and concerns, revealing a number of recommendations for the assistive technologies research community. These include the need to: broaden application areas, especially to the workplace; strengthen comparative studies; conduct more authentic evaluation by increasing in situ and longitudinal studies; and broaden end-user participation in the initial design process. • It provides an accessible living resource named AGRep (Accessible Graphics Repository), capturing what has been done in the field of accessible graphics technology, to be shared with the wider community and be continually updated. This repository contains all papers from this systematic review.

BACKGROUND 2.1 Provision of Touch-Based Accessible Graphics for those who are Blind or have Low Vision
Accessibility guidelines recommend the use of tactile graphics for presenting graphics, such as maps or charts, in which spatial relationships are important [8,37]. Tactile graphics, however, require specialist equipment to produce and, as specialist transcribers are typically required, are expensive and time consuming to produce. Furthermore, the reader needs sufficient tactile reading skills, including the ability to read braille, to best understand the graphic content.
The widespread use of touch screen technologies, the arrival of low-cost maker technologies, and developments in computer vision and machine learning that can facilitate automated transcription, have opened up opportunities to take the provision of touch-based graphics into a new era. As such, new techniques for the creation of accessible graphics is a significant area of research. Key areas of research include new approaches to 3D-printed tactile models [7,18,25,46,51], haptic feedback [11,14,38,39,50], robotic systems [22], touch-triggered auditory feedback [10,42,43], and automated tactile generation [9,33,48,54], such as utilising online mapping systems [17,49,55].
Given the rapid rate at which new research is being conducted in the use of these new technologies for touch-based graphics, it is timely to review the field in order to better understand the nature of this research, and to identify best practice to better inform research in this context.

The Value of Systematic Literature Reviews
Systematic Literature Reviews (SLRs) are conducted across many different research fields. They are undertaken by researchers in order to examine a research field more holistically by understanding its recent areas of focus, the gaps, and looking to the future of the field. When conducted in a rigorous manner they provide a vehicle by which a research field can reflect upon and improve its best practice.
While most research is underpinned by a review of related work, it is often very targeted and typically self-serving to the research being presented. SLRs seek to be broader in scope, synthesising "existing work in a manner that is fair and seen to be fair" [29], as well as remove bias [36]. Not only can they provide a more complete picture of the type of work being undertaken, they can also examine a breadth of settings, method and contexts [3,29]. Thus they allow us to understand the current state of the field in a more holistic manner.
Further to this, SLRs capture a point in time and reveal what has led the research field to this point. This is important in technologyrelated fields, such as assistive technologies, where advances in technology and techniques have a large influence on the direction that a research field takes.
Finally, SLRs allow us to identify the key challenges moving forward, regarding both topic and method. They not only identify the current state of the art, but also the gaps that are yet to be addressed and provide data for rational decision making [36]. They can be influential in guiding the research agenda for the years to follow. As such, "The aim of an SLR is not just to aggregate all existing evidence on a research question; it is also intended to support the development of evidence-based guidelines for practitioners." [28]. For example, the work of Barkhuus and Rode [2] presents a systematically derived set of accepted evaluation methods in the CHI community and demonstrates how they are important to specific influential research communities.
There is evidence of the benefit of systematic reviews within the domain of assistive technologies. Kelly and Smith [27], in their multi-decade review found that, while of benefit to the professional community, most work lacked methodological rigour. Wabiński and Moscicka [53] analysed works relating to automatic tactile map production and found that very few of the working solutions reported ever made it to widespread adoption. More recently, Brulé et. al. [6] sought to better understand the nature of empirical evaluation of assistive technologies for BLV users, and found that meaningful participation by BLV end users in evaluation was lacking, as well as a disproportionately low involvement of low vision participants relative to their proportion in the BLV community. Other reviews focus on a subset of work, such as the development of interactive maps [12,55], and highlight research questions specific to that context. These works demonstrate that systematic reviews can shine a light on key issues and provoke the assistive technologies research community to reflect and improve on their practice.

METHODOLOGY
This SLR has adopted a process consistent with that recommended by Kitchenham [29]. The process is also consistent with that reported in [6], a systematic review in a similar domain relating to BLV technology.

Scope
This work focused on a number of questions regarding research publications in touch-based graphics over the past decade. These are consistent with other SLRs, whereby it is the intention to capture a snapshot of current practice, identify gaps, and foreshadow the future of research in the area. Key questions are: • What are the main types of graphics and application domains studied in research exploring touch-based graphics? • What are the primary presentation technologies and how have they changed over the past decade? • What methodological approaches are used and how inclusive are they of the end users? • What are key gaps or challenges that researchers should consider moving forward?
This SLR aims to cover work relating to technological developments in touch-based graphics for people who are blind or have low vision (BLV). On this basis, the research team documented a number of key inclusion/exclusion examples that informed the selection of works. Notable inclusion criteria required that: • The work must relate primarily to novel touch-based graphics representations. In this context novelty refers to focus on a new production or presentation approach. A broad interpretation of 'new' was adopted when considering work for inclusion; • Auditory work could be considered if a touch interaction strategy was the basis; • Traditional touch-based graphics, such as tactile graphics, are only included if part of a comparative study with new technologies or if exploring new production techniques. Papers solely describing the use of tactile graphics were not regarded as 'novel'.
Key exclusion examples were: • Blind or low vision users are not an identified focus of the work; • Primarily focusing on braille text; • Exclusive focus on user interfaces; • Audio interfaces with no touch interaction. A ten-year period was chosen for the review, rather than a longer period. This was for two primary reasons. Firstly, a large volume of relevant papers was identified in this period (over 350). It was also felt the last decade covered the emergence of topical new technologies such as 3D printing.
In order to identify relevant work, multiple search approaches were used to reduce publication bias [29]. A systematic review of 'gold standard' venues was combined with a keyword search. Gold standard venues were those that fell into one of the following categories: . These include those recognised as top-tier venues by [6]. All works in these venues were reviewed, using titles and abstracts as the primary guide. As per [29], a liberal and inclusive approach was taken, knowing that works could be removed at a later stage of the process. All works in the venues were considered, including full papers, short papers, late-breaking works, etc.
In order to identify work published in other venues, a keyword search was also undertaken. This could capture work presented outside of venues specifically associated with assistive technologies. A Google Scholar advanced search was conducted to identify articles between 2010 and 2020, using the following keywords: • At least one of: disability; accessibility; blind; low vision; • Plus at least one of: tactile; touch; haptic; • Plus at least one of: graphic; diagram; map. Rather than search specific databases e.g. ACM/IEEE, Google Scholar was used as it includes these databases. We did not use exact phrases in the search, thus the search results included synonyms. We reviewed the first 100 search results, whereby no new relevant papers were found on the next results page. After conducting the search using these two methods, 352 papers were included in the initial corpus.

Systematic Review Process
Based on the identified aims of the SLR, the research team created a set of analysis criteria by which each paper would be reviewed. Each criteria was discussed by the entire team to ensure agreement of its meaning was reached. Criteria were presented in a Google Form, with one form submission made for each paper.
A small cross-section of papers were reviewed by the entire team to assess the appropriateness of the analysis criteria. The research team then met to discuss the outcomes of the preliminary review, during which a revised and expanded set of analysis criteria was developed. Using a revised Google Form to capture data, all papers were reviewed against the new criteria. Results were captured for analysis in an accompanying Google Sheet. The following criteria were analysed: contributions; graphics and their application domain; presentation technology; production methods; research methodology; evaluation methods and participation; and distribution of technology outcomes.
All 352 papers were fully reviewed. During this process each paper was also re-assessed for relevance [29] and could be excluded from the corpus if it was deemed to be outside the scope of the literature review, as based on the initial inclusion criteria. If a reviewer believed a paper should be excluded, another reviewer would also analyse the paper. If both believed the paper should be excluded then it was formally removed from the data set. If there was disagreement, a third reviewer would make the final decision. This ensured reliability of the inclusions and exclusions [29]. Examples of papers that were excluded at this stage include those where only widely adopted production and presentation methods with tactile diagrams were used, work where the context of application was not identified as being for BLV end users, and papers where the work only related to accessibility of user interfaces and not graphical information.

Tools for Analysis and Sharing
To support the authors as well as the wider touch-based graphics research community, the corpus of identified documents has been catalogued in a publicly accessible web format, based on the interactive sentiment visualisation tool SentimentViz [31]. Like SentimentViz, our tool -AGRep (Accessible Graphics Repository) -is designed to help users to "discover interesting patterns and facilitate data exploration" [31].
Using AGRep ( Figure 1) it is possible to explore and perform custom searches on the corpus of resources based on our categorisation markup, access persistent identifiers or URLs for each resource, and view tabulated summary data. Additionally, as AGRep is envisioned as a living resource capturing what has been done in the field of accessible graphics technology, users can update the corpus with newly published resources in the future.

Accessibility Modifications.
AGRep has been customised to improve its level of accessibility. During early stages of development the researchers held an informal session with a BLV participant to understand how the accessibility of the tool could be improved. This involved a walk-through of a test deployment of SentimentViz on the participant's personal technology.
Based on this informal session, the following modifications were made to the source code of SentimentViz to increase its accessibility for the purpose of AGRep: • A help dialogue page added, outlining how to best use and navigate the tool; • Alternate text added to all images, icons and glyph symbols; • The tabbing order of page elements optimised to improve screen reader performance; • A change from grid-based layout of resources to row-based layout; • Elements that update per user input marked up so that screen readers can interpret dynamically changing content.
The AGRep tool contains all papers from the corpus of this SLR, as well as more recent 2020 works that meet the inclusion criteria. The process for external parties to add new works for inclusion is given on the repository website. The AGRep tool is located at https://agrepository.github.io

ANALYSIS AND FINDINGS 4.1 Paper Counts, Venues and Authors
On completion of the full review, the final corpus contained 292 papers from 97 unique venues. Following are the basic characteristics of the corpus. Note that the largest body of work came from ICCHP. As this conference runs every second year, data is aggregated in two-year blocks, so as not to misrepresent alternate years when ICCHP was not held. Table 1 shows the distribution of papers over the decade. The number of papers can be seen to have significantly risen in 2014. A possible reason for this will be discussed when we examine paper content later in this review.
The corpus was drawn from 97 unique venues. As can be seen in Table 2, the vast majority of venues from which papers were chosen (70.41%) were represented by only one paper in the corpus. Only 4 venues had on average one or more paper per year. These were: ICCHP (61 papers); ASSETS (44 papers); CHI (35 papers); and TACCESS (10 papers).
The majority of papers collected were classified as 'Full Papers' by their venue (Table 3). There were also a number that were considered as 'Short Papers' or were an accompanying paper to a poster or demonstration. Short papers typically do not have the same expectation of rigour regarding evaluation, and include such things as preliminary studies or late-breaking work. The majority of short, poster and demo papers were in the top three venues (ICCHP, AS-SETS and CHI) with a little under half being considered 'Full Papers' in those specific venues.
Across the 292 papers, there were 641 unique authors. All authors were counted in the metrics regardless of the author order and weightings were not applied to authorship. The large number of authors is consistent with the finding that there are a very high number of venues with only one paper ( Table 2). Table 4 shows that 488 authors (76.13%) contributed to only one paper in the corpus, i.e. have only one work relating to touch-based graphics in the last 10 years. This suggests that a significant amount of research is a 'one-off' by authors: they explore a single idea regarding an innovation with touch-based graphics and do not further develop or evaluate it. In accord with this, very few authors have multiple publications. Only 32 authors (5%) have 5 or more papers published relating to touch-based graphics over the past decade.
The long tail in both venues and authors shows little evidence of 'deep dives' into a particular research direction by researchers in the field. While this is somewhat speculative, it raises the concern that most authors do not have a deep knowledge of what others have done in the field and are possibly 'reinventing the wheel'. It also raises the question of whether one-off research will lead to meaningful change in BLV community practice. On the other hand, it may be that to effect change in this area, many different approaches need to be tried to 'see what sticks'. Indeed, this was illustrated clearly in the Transforming Braille Project, initiated in 2011 to develop a low-cost refreshable braille display and bring it to market, whereby over 60 projects were examined before leading to a commercialised product [45]. This does not, however, stop us considering the question of how work in this domain can be better shared and built upon to effect meaningful change more quickly.      Three categories dominated the types of graphics represented in the corpus: STEM related (115); Maps and Plans (110); and General or Unspecified (104). STEM related graphics encapsulated a number of other sub-types, including those relating to math, science and technology. "General or Unspecified" was a catch-all for graphics that were not explicitly defined or could be applied across domains. These included those represented by work exploring general presentation systems. Graphics captured by "Other" include: Puzzles and Games (4); Screen Layout (2); and Patterns (1) and Handwriting (1).
The focus on STEM related graphics and maps and plans is to be expected as it reflects that the main application areas for tactile graphics are in education and Orientation and Mobility (O&M) training. Maps and plans also lend themselves to technology driven innovations. For example, they are a strong use case for 3D printing, audio labelling, and in particular automated creation, where open data is easily available.
"Arts and Culture" is the other main category of graphic found in the corpus. This is somewhat surprising as they are not a typical use case for the provision of tactile graphics. It is exciting to see that researchers are exploring innovation outside of the more 'obvious' application areas of tactile graphics.
We also examined if the type of graphic had varied over the past decade. We found there was no significant change in focus. The only exception being a small increase in graphics relating to Arts and Culture in the latter years.
The intended application domain of the research publications ( Figure 6) is consistent with the types of graphics they consider. Education and O&M dominate, which is reflective of the the primary application areas for tactile diagrams. The category of "Other" only captured one other use case, being for research.  Graphics that are not related to daily living, but rather quality of life across the whole life span, are much less of a focus. An exception are those relating to arts and culture. We believe an important area of future research is to consider other graphics, such as for instance, sports graphics, which are common in popular culture but currently unavailable to the BLV community.
An even more glaring omission is that graphics for the workplace were only considered in 5 of the papers in the corpus. This is very low and suggests significant inequities that may exist in the provision of accessible graphics in the workplace. This should be recognised as not just an opportunity for future research but one that must be addressed to support BLV people in the workplace, not just in schools or mobility training.  Next we considered the area in which a paper made its primary contribution (Table 7). A paper could make more than one contribution.

Primary Contributions
The majority of works collected were those with a new application of existing technologies or techniques. That is not unexpected, and reflects that many works came from publication venues outside those related specifically to assistive technologies. Those works often explored the provision of touch-based graphics in a new context, for example the arts, cartography, etc.
Behind this, three contributions shared the majority of focus of other papers. These were new presentation methods, new production methods, and new interaction methods. It is significant and positive to see that researchers are not simply focusing on new presentation methods for touch-based graphics but are also devoting equal attention to their creation and the ways in which the reader can interact with them.
Within new production techniques, the dominant topic is the automation of accessible graphic creation, especially within the context of maps and plans. This is very important, as the timely and cost effective production of accessible graphics is a significant barrier to their more widespread use, especially in contexts outside of education and O&M. As such it is heartening to see the strong interest by researchers but it remains an area that would benefit from increased attention.
Other contributions that were captured by papers in the corpus included: reviews of technology (6); development or evaluation of guidelines or standards (5); exploration of design spaces and approaches (5); feasibility or usability studies (3); and understanding of tactile perception (2).
We also evaluated whether a paper was an improvement of an existing production technique or presentation technology, i.e. if it was heavily based on a preceding piece of research. While this is a more subjective measure than our other measures we believe it is still suggestive. Within the corpus, 38 studies were considered to be focused on improving existing work. This relatively low number is consistent with the 'long tails' of both authors and venues as presented in section 4.1, and again should be an area of reflection for researchers.

Differing Contribution Focus of Venue:
We discovered that different venues appear to implicitly have a slightly different focus regarding contribution. Table 8 shows the contributions of the papers in each of the 4 primary publications venues. ICCHP was more heavily skewed toward new production methods and case study areas, while ASSETS and CHI explored presentation and interaction methods more strongly. This may be consistent with the nature of ICCHP having (in general) greater involvement by practitioners. Notably, ASSETS and ICCHP had a larger number of papers that were improvement of existing techniques and technologies than the other venues.

Comparative Studies:
Given our focus on new presentation technologies, we were interested to summarise the findings from user studies comparing different presentation technologies. We extracted these papers from the corpus and a summary of the studies and results can be seen in Table 14 in the Appendix. Most studies focused on maps but a few used simple charts or STEM materials as stimuli. The tasks were quite varied. Almost all were laboratory studies, with only 3 in situ evaluations [24,30,40].
Overall, comparisons with more than a single study support the following conclusions: • 3D printed model with/without audio labels vs tactile graphic [4, 15, 19-21, 25, 30, 41]: 3D model is preferred, better short-term recall using 3D model (but not long-term recall). Some indication that the representation of third dimension is easier to understand on a 3D model. Symbols are easier to discriminate and iconic symbols more understandable with a 3D model. • Touchscreen with overlay and audio labels vs tactile graphics [5,35]: Touchscreen with overlay and audio labels preferred and faster. • Touchscreen with audio labels with/without vibration vs tactile graphic [16,23,52,55]: Tactile graphic is preferred and is explored more quickly. Some indication that it is difficult to understand detailed geometry with touchscreen. • Force feedback with audio labels with/without vibration vs tactile graphic [44,56]: Faster with tactile graphic. Conflicting results on accuracy.
However, this examination revealed that more controlled studies are required to better understand the comparative advantages of these different presentation methods. These studies should consider a wider range of graphics and more systematically investigate how low-level tasks are supported as well as the use of these methods in situ and longitudinally. Table 9 identifies the presentation technologies used in the corpus. These technologies can be combined. The 'base' presentation technologies which provide information about the spatial layout of the graphic are, in descending order of occurrence: tactile graphics; 3D models; touch screens; refreshable tactile displays; and force feedback devices. These base technologies were frequently combined with the other presentation technologies. In the case of 3D models, audio labels were common while touch screens also provided audio labels and sometimes tactile overlays and/or vibratory feedback.

Presentation Technologies
It may be surprising that tactile graphics, a well established technology, features so prominently. This reflects that they are currently the 'gold standard' for accessible graphics provision and so are a frequent benchmark for comparing new presentation technologies or are the focus of new production methods. Figure 2 shows the changes in presentation technologies over time. The most striking change is the sharp increase in the use of 3D printed models from 2014-2015. This is not unexpected, given that commodity printers arrived at this time, and over the past 5 years they have both dropped in price and been more widely adopted by researchers and practitioners in assistive graphics provision. There has also been an associated increase in the use of audio labels, as the two technologies are frequently combined.

Emerging Technologies:
We believe that the primary reason for the overall increase in publications about touch-based graphics noted earlier is the arrival of low cost 'maker' technologies such as 3D printing in 2014. This led to lower research and development costs for prototyping new presentation technologies as well as interest in how these technologies could be directly used to create accessible graphics.

Technologies Out Of Favour:
We also see that there has been diminishing attention to force feedback devices and sonification. The early part of the decade had a focus on the use of force feedback devices, such as the 'Phantom Omni,' but their use dropped off  Improvement   ASSETS  15  17  12  17  3  0  10  CHI  20  13  7  12  2  1  4  ICCHP  23  15  23  10  3  0  9  TACCESS  8  3  0  3 1 0 2   almost completely by decade's end. We conjecture this is because of their expense and limited uptake by the BLV community. The reduction in sonification research is curious. However, it may be that our requirement that sonification is employed in touchbased interfaces has resulted in a downward trend that is not representative of the broader research agenda. Even though this may be the case, with the availability of new ways to trigger audio labelling (such as with a moble phone, or with capacative touch sensors on 3D models), this seems a presentation technology worth pursuing.

Underrepresented Technologies:
One of the most uncommon presentation technologies is low-vision visuals, though it has increased over time. This suggests research in this area may be skewed toward people who are blind as opposed to having low-vision despite the fact that there are many more people with low vision than blindness. This was consistent with the findings of [6]. It may also reflect that the majority of people with low-vision do not wish to use touch-based graphics, however this would need to be further investigated.

Technologies by Research Contribution:
The technologies represented in the corpus can also be considered in light of the research contributions of each work. Figure 3 shows the four main research contribution types, with the associated number of papers for each technology. Note that there can be more than one technology associated to the contribution (e.g. a case study may utilise both tactile graphics and 3D printed models). Similarly, a paper can be categorised with more than one contribution.
Examining contribution against technology reveals a number of interesting differences. When looking at case studies, 3D printing is the most common. This seems remarkable given that commodity 3D printing only emerged midway through the review period. However, it is likely that the rapid high levels of interest in 3D printing by both researchers and practitioners in accessible graphics and early positive evaluations fuelled interest in a wide variety of applications.
New production methods were dominated by tactile graphics. Within the corpus, new production methods focused heavily on automatically converting data or existing graphics into a traditional tactile graphic. As such, this is unsurprising. However, this highlights a significant research opportunity: How can similar automation processes be used in the production of 3D printed models? This is especially important given the amount of work exploring the use of 3D printing for presenting touch-based graphics, and that the design of 3D models is often a barrier to their wider adoption.
We were surprised that new presentation methods were dominated by audio labels. However, this is likely due to the fact they can be used in conjunction with both touchscreens and 3D printed models. It is encouraging to see a commodity technology such as touch screens being used in the exploration of new presentation methods, as well as the more recently emerging refreshable tactile displays. 3D printed models are somewhat underrepresented in this contribution category. This is likely due to their more recent emergence; they are first being evaluated for their efficacy in different application areas before being fully explored as the basis for new presentation methods.

End-User Participation
We were particularly interested to understand the level of involvement of end users in the research. While BLV people are the primary end users, end users also include sector professionals such as transcribers or O&M trainers. Table 10 shows the involvement of participants in each research study, across the design, prototype and evaluation stages of each study. The participants are divided into three categories: BLV, Professional and Sighted. Note that participants might take part across multiple stages within a single study. Similarly, members from multiple participant groups may be used within a single stage within a study (e.g. both blind and low vision participants within an evaluation).
Importantly, BLV participants are by far the most represented group, with the strongest representation by blind participants in the evaluation stage. One thing that is noticeable is that BLV participants, however, are much less likely to be involved in the design or prototyping stages than the final evaluation. This is potentially a serious problem as it means end user feedback is only considered late in the development. The following section explores BLV representation further.

BLV and Professional Involvement:
Participation by BLV people across the corpus was in 62.80% of the studies, which may be considered low for work that is intended to be for the benefit of a particular community. This increases marginally if sector professionals are included, to 66.89% ( Figure 11).
However, this must be considered in light of the paper type ( Figure 12). Short papers, posters and demonstrations had lower representation. This is primarily a constraint of either the space afforded the work, or that it is often the initial stages of new work. Full papers present a better insight into end user involvement, and as such are the focus of discussion. When only considering full papers, the involvement of BLV people in the research increases to 73.26%. While this is strong, there is still considerable scope for improvement.
It could be expected that the primary venues associated with assistive technologies would have a higher involvement by the BLV community. When considering full papers only, this is mostly the case, with ASSETS, CHI and TACCESS all showing levels of engagement by BLV pople in over 85% of papers. Indeed, when sector professionals community are considered as well, all three venues have 100% involvement. ICCHP, however, is an exception. BLV involvement is still only in the order of 69% and there is no involvement with professionals within the corpus. Involvement by end users in the other venues, including venues not normally associated with assistive technologies or accessibility, is lower than the top 3 venues. Of note is that only 1 publication out of 143 papers in other venues had the involvement of a sector professional.
One concern that emerges is that studies are skewed toward blind participants as opposed to participants with low vision. This may reflect the lack of focus on low-vision visuals that we previously identified. It is, however, consistent with the findings of [6,47] and may reflect that researchers believe touch-based graphics are of greater importance to those who are blind. It may also be that new kinds of touch-based graphics such as touchscreens with audio labels or 3D printed models will be of benefit to people with low    vision, however a focus on blind participants more stringently evaluates the technology.

Participant Numbers:
The numbers of participants in each study is also variable. In studies presented in full papers, there was a median of 10 BLV participants. While this may appear to be a relatively low number of participants, it is understandable given the low incidence of blindness and the difficulty that can arise with recruiting participants. With small sample sizes it is difficult to find statistical significance, so this low number may account for the greater number of qualitative studies (161) that were conducted in

Lack of In-Situ and Longitudinal Studies:
One noticeable limitation of many of the studies undertaken was that they were undertaken in controlled environments, such as a laboratory (148 studies) rather than in the field or in situ (47 studies). While research that involves prototype bespoke technologies may be constrained in their ability to be conducted outside of a controlled environment, it does suggest that little of the work being evaluated is at the stage where it can used or distributed in its end user environment. Moreover, in almost all studies the user learned the system and was tested on its use in the same session. While short studies can provide immediate feedback for further development, they do not account for the time it can take to develop new tactile literacies, become comfortable with a new presentation method, or transcend learning curves that a new technology may have. As such, more longitudinal studies should be considered to truly evaluate new developments in touch-based graphics and to obtain greater buy in and adoption.

Restriction to functional evaluation:
The evaluation undertaken in most studies was also strongly focused on function, especially when exploring new presentation methods. Rarely did the studies encapsulate other important factors such as engagement, pleasure or stimulation. Given that an important part of the development of visual graphics is in their aesthetic pleasure, it is suggested that research in this area starts to consider more holistic evaluation criteria.

Availability of Outcomes
As previously stated, commercial work was not considered as part of the literature review. However, in order to gauge the potential ongoing impact of the research in the corpus, the availability of its outcomes was also evaluated during analysis. This was based upon statements in the paper and so probably understates the actual availability of the research. Table 13 shows the availability of the outcomes of each research paper in the corpus. Of the work where the nature of availability was clear, almost half the outcomes of the research were considered to be unavailable to the broader research community or end user, due to the research being undertaken with a custom prototype. Close to 40% of the outcomes are available in some form, whether it be through the use of commodity technologies or with  It is encouraging that 81 papers did use commodity technologies. However, it is also acknowledged that there is significant prototyping being undertaken utilising low cost technologies (such as 3D printing, simple electronics, or simple vision systems). This may signal a shift toward research that will not require significant cost to become more widely adopted. As such, continued work should strive to balance innovation with widely adopted commodity technologies as well as with low cost development technologies in order to lead to impact in the BLV community.

CONCLUSION AND RECOMMENDATIONS
This paper presents a systematic review of literature relating to touch-based graphics over the period of 2010 to mid-2020. Analysis of the work has provided insights into the changing nature of this research, as well as highlighting 'calls to action' for researchers in the field. The following are key areas for researchers to consider as well as opportunities for future work: • Broadening Application Areas: The areas of application for touch-based graphics should be broadened to include graphics outside the education and O&M context. While those contexts are fundamental to daily living for the BLV community, it is important for researchers to consider other domain areas. An application area in considerable need of attention is graphics for the workplace. This is not just an opportunity for future research but is also vital for increased support for BLV people in the workplace. • Strengthening Comparative Studies: More controlled studies are required in order to better understand the comparative advantages of new presentation methods. Comparative studies can then provide important evidence that can inform decision making regarding adoption of new technologies by the wider BLV community and its stakeholders. These studies should also consider a wider range of graphics. • Authentic Evaluation: Laboratory studies dominate the research surveyed. While this is an important first step in research innovation, it is also critical that new developments in touch-based graphics are evaluated in the context in which they will be primarily used and evaluated over longer periods of time. This should be a priority for researchers, as it will also support research becoming more widely adopted in the BLV community. • BLV Involvement: While BLV involvement in comparative studies and evaluations is relatively high, there is much less involvement in initial design or prototype evaluation. We suggest that the design of new technologies for touch-based graphics, like other areas of assistive technology design, would greatly benefit from using a more participatory design processes, in particular during the creation of accessible graphics. It is also important that the low vision community is equitably represented in research studies.
It is hoped that this systematic review of work can support the assistive technologies research community by highlighting gaps in current practice as well as opportunities for work in the future. It forms the basis for a web-based living resource that we hope will be a valuable tool for accessibility researchers for many years to come, and lead to more meaningful impact of this research for the BLV community.  18 BLV adults More accurate with 3D print than embossed graphic; faster with 3D print than embossed and swell tactile graphic [5] Tactile graphic (swell paper); touchscreen + overlay + audio labels

Map
Memorise map then answer questions about route, landmark position, layout; tested both short-term and long-term recall 24 BLV adults Learning time less with audiolabels; slight preference for audiolabels; no effect on short or longterm recall; [15]