{'fraction_non_alphanumeric': 0.02691947565543071, 'fraction_numerical': 0.0025749063670411983, 'mean_word_length': 4.951253481894151, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Adaptive Assistance : Dynamically tailoring assistive technologies for interactive computer users

Almost one in five Americans has a disability, according to the U.S. Census. Additionally, the disabled are the only minority group that any person might end up joining at any time. This is especially likely among the elderly, a growing group in today’s societies. People with disabilities are far less likely to use the Internet [11], and one reason for this is difficulty with inaccessible technology. Assistive technologies such as soft keyboards, magnifiers, and so on can help with this. However, current state-of-the-art assistive technologies know little or nothing about the user’s context (such as what applications are open, what the user is interacting with, common tasks, and so on). Today’s assistive technologies interacting with computers are static technologies. This limits their efficiency. We believe that adaptive assistance, which dynamically adapts to the context of use, can help to increase the accessibility of desktop computing interactions. We are proposing to create assistive technologies that will work at the operating-system level, with any application the user is running, but will dynamically customize themselves based on information about what the user is doing. Some important things that customization can help with are (1) making specific targets, such as buttons on the screen, very easy to acquire (2) replacing portions of applications with more accessible versions (such as replacing a complex menu with a dialog box) (3) reducing the number of steps necessary to complete a task based on common approaches (4) increasing the efficiency of assistive interventions (such as tailoring the shortcut keys on a scanning keyboard to the application at hand). Our plan is to develop a suite of tools supporting progress on these adaptations in an application-independent fashion. In other words, our tools will work with any and all applications, and will not require that applications be written in a specific language or with a specific toolkit. Once a tool is installed, it will simply work with whatever applications are in use. Our tool will operate on syntactic, lexical and semantic information about applications. For example, by leveraging existing accessibility hooks and OS hooks, we can learn about semantic information such as whether the user is interacting with a common application such as web browser or email tool. We can also learn lexical information such as the relative locations of different windows. We will combine this information with information gathered by applying computer vision techniques to extract additional information from the desktop. For example, syntactic information about the boundaries of buttons may be identified in this fashion. As an example, consider the email interface shown in Figure 1. The user is dragging a mail message into a folder. The techniques described above can tell us that the user is working inside an email tool; that it contains a hierarchical interactor on its left side consisting of many small targets and the text names of those targets, and that to the left of that is a separate application with many potential targets (the application dock). Computer vision can be used to identify the location of specific targets within the hierarchical interactor and dock, as well as the names of visible IMAP folders (taken from text on the screen). By tailoring adaptive technologies to this situation, large efficiency gains are possible. Suppose that the user selects a message and starts to drag it. A simple lexically-based heuristic suggests that they are likely to be heading for the nearest set of targets (the hierarchy), but that if they cross them they are heading for the Application Dock. Figure 1(a,b,c,d,e) show mock-ups of a set of possible adaptive tools. Fig. 1(a) shows a shortcut tool that divides the screen into likely target areas and likely targets, and annotates each with keyboard key names that can be pressed to jump the mouse to the appropriate part of the screen. Fig. 1(b, c, d) show three ways of making targets easier to acquire with a mouse (magnification; a pie menu; and gravity wells). Fig. 1(e) shows an intelligent magnifier that shows