{'fraction_non_alphanumeric': 0.02247696111485727, 'fraction_numerical': 0.0028096201393571587, 'mean_word_length': 4.996293800539084, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Mobility and orientation aid for blind persons using artificial vision

Blind or vision-impaired persons are limited in their normal life activities. Mobility and orientation of blind persons is an ever-present research subject because no total solution has yet been reached for these activities that pose certain risks for the affected persons. The current work presents the design and development of a device conceived on capturing environment information through stereoscopic vision. The images captured by a couple of video cameras are transferred and processed by configurable and sequential FPGA and DSP devices that issue action signals to a tactile feedback system. Optimal processing algorithms are implemented to perform this feedback in real time. The components selected permit portability; that is, to readily get used to wearing the device.


Introduction
The human vision system processes information collected by a very large number of receptors -the rod and cones cells of the eyes. Visual acuity is a measuring means for evaluating the vision of any given individual by comparing it to normal values for the average population. [1]. A decreased level of visual acuity is currently ameliorated by different approaches based on methods and devices that enhance the diminished vision or, else, the option is to substitute for the diminished or lost vision by using other senses. Thus, when vision loss is total (blindness), a "sensorial substitution" is performed. This solution entails using an alternative human sense to collect information that is normally captured by other sense. Two main problems that visually impaired persons have to face are the impossibility of reading standard printed material and the restrictions on readily moving and be oriented through places. Thus, if a visually impaired person fails in reading a word, he will experience a frustrating loss of information. Likewise, an error in displacement orientation may put him at risk for an accident, maybe involving other persons. This risk chance is increased when moving and heading through different environments, which normally are non-structured and non-predictable. Visually impaired persons typically use displacement aids, ranking from blind-sticks to more sophisticated electronic devices, the so called Electronic Travel Aids-ETA [2]. The ETAs use various physical principles and, at present, there is not any standardized or totally complete vision aid available in the market. The various approaches are based on physical principles of ultrasound, laser and vision [11] [12][13] [14]. All devices used in vision substitution need three basic components: an interface that captures data on environment variables, a system to process these data and, finally, an element that renders this information in a useful way to the person.
The current work presents the design and development of an ETA that uses artificial vision to gather environment information. Multiple vision systems, in general, and stereoscopic vision system in particular, show an advantage in being capable of re-construing 3D information using 2D images. From this fact, research is made on finding optimal algorithms that help reducing the processing times while allowing for implementing them in portable (wearable) devices for efficient personal-use performance.

Stereoscopic Vision
Stereoscopic vision entails capturing environment information with two cameras, processing the images and rendering data in a useful way. The procedure is mathematically founded on epipolar geometry, which simplifies the calculus on the grounds of knowing the position of the cameras. From the computational viewpoint, a stereoscopic vision system has to solve two kinds of problems [3] [4]. The first one, known as correspondence, consists on determining which element, say, on the left-side eye (left-side image) correspond to a respective element on the right-side eye (right-side image). The second problem, called re-construction, is explained as follows. The 3D vision perception from the environment is acquired by a normally-seeing person thanks to the interpretation elaborated in his brain on the evaluated difference of the position of a given image fragment on the retina, called disparity, among the "corresponding" image fragments or elements. Therefore, using the pair of images of the stereoscopic vision and previously knowing the position (disparity) of a single object on both images, it is possible to calculate the distance (Z) to the object using the following equation: where T is the distance between cameras, f is the focal distance, and x l , x r are the object positions on both images. In order to obtain the correspondence, various methods were first analyzed [5] and, considering their efficacy and efficiency levels, the SAD (sum of absolute differences) approach was chosen. Knowing the disparity value for each pixel (x,y) of the scene, a new image is formed called Disparity Map [6].

Conceptual View of the Work
The purpose of this work is to design a portable (wearable) device to help in the rehabilitation phase of visually impaired persons, so as to help them move and correctly orientate themselves using tactile feedback. The stereoscopic vision approach opted for in this work renders real-time information to the actuator system. For an efficient practical use, small-sized, low-power and high-performance hardware tools are needed in the design. The device is implemented with two digital cameras to capture stereoscopic images that are transmitted to a digital system processor (DSP). This element generates the necessary information to be transmitted to the user through a 27-actuator system (small vibrating motors). The various components are communicated via a programmable logic (FPGA) onto which a 'glue logic' is implemented to link the various elements among themselves. Figure 1 shows a general description of the hardware components of the design.

Hardware and Software Components
The cameras -that acquire the images simultaneously-are built with CMOS technology. They use the YCbCr Standards, employing only gray-level information (Y) and the corresponding synchronization signals. In order to synchronize the cameras between themselves and to control their data transference rates, a shared CLK signal, external to both cameras, was generated. The FPGA uses the synchronization signals from the cameras, in order to store the simultaneous cameras images orderly in an external memory. The Spartan3 FPGA from Xilinx, mounted on a Diligent Spartan3 Kit allows transmitting the images in serial form to the DSP. The DSP Blackfin ADSP-BF535 from Analog Devices, mounted o the Development Kit ADSP-BF535 EZ-KIT Lite, was used as the image processing core. The chosen algorithm processes both images simultaneously and collects distance information via a disparity map. Then, after a step for reducing the information, the driving signals for the actuator system are created.
The actuator system (tactile display) is composed by 3 rows of 9 small (1cm diameter) vibrating motors each, located around the waist. The array is stimulated through pulse-width modulation (PWM). The separating distance between the actuators takes into account the resolution capability for stimuli on this body area [11]. In order to immobilize the display and to improve the perception of vibrating stimuli, an elastic band was used, located over the display. Figure 2 shows the internal arrangement of the vibrating motors.  Figure 2. Internal arrangement of a vibrating motor.

Development of the Prototype
The algorithms implemented on the hardware are: a method to correct via software the mechanical alignment of the cameras; a correlation technique to obtain the relationship between images; the development of the disparity map and the application of the data reduction means to activate the actuators.
To develop the algorithms, design constraints and simplifications are established in order to optimize them [7]. Since the operation is done in real time, a minimum computation time of one second is needed, which is equal to the estimated duration of a step during the walking action with a moderate pace. Besides, having adequate mechanical alignment and no distortion lenses allows assuming that the images are rectified; that is, the epipolar conjugated lines are co-linear and are in parallel to one of the image axes. Alignment is a process intended for making the optical axes of the cameras be parallel and co-linear between each other, between the co-plane sensors, as well as to avoid any relative rotation of any one camera respecting the other one. The work incorporates first a coarse, pattern-based manual alignment to modify the mounting position of the cameras. This is corrected via software for a fine alignment.
In the software-based alignment correction method, two images were chosen that focus towards a far place, likely to be considered the infinitum reference. On such a stage, this region must be located at the same position on both images. The alignment process uses the SAD (Sum of Absolute Differences) correlation method. The last stage of the alignment process involves obtaining a pair of images, smaller than the original ones, whose infinite referential pattern is located on the same point on both images (See Figure 3).  Once a pair of aligned images is attained, the disparity map is built as a 2D array on the same referential framework of one of the images whose values represent the disparity of each point respecting the complementary image. Using these values, the distance between objects can be computed, using the SAD method as well.
The algorithm to obtain the disparity map renders the position where a same pattern lies on the rightside image. When comparing this value with that of the position of the central pixel of the left-side window, the disparity value is obtained for this position. Upon finishing this operation for every left-side image pixel, a disparity map will be formed and sent to the actuator system via an algorithm to reduce the information.
In order to meet the requirements of processing times (i.e., real-time processing), some adjustments are made to decrease the number of calculations. In order to have better reliability in SAD computations, a selection process was included to choose the adequate values of correlation minima that present the lowest correspondence error possible. This allows determining with the lowest error possible the values of disparity and distance [7]. The reduction algorithm consists in using the disparity map to generate the information matrix that is feed-back to the user. This 3x9 matrix is sent to the actuators via the FPGA.
Because of the size of the above-chosen actuator array, the disparity map should be partitioned into 27 regions corresponding to the proposed arrangement (see Figure 4). Once this matrix has been generated, it is send to the FPGA through a communication algorithm, using 5 input/output lines in direct connection with the DSP.

Preliminary evaluations
Several experimental tests helped to determine the optimal values for the different variables that affect both the efficacy and efficiency of the alignment process, and to make the disparity map. These tests were based on determining the optimal value of window size, image size and the set up of the algorithm that choose the minimum correlation values. As a preliminary test, the performance of the stereoscopic vision system was evaluated with persons of normal vision. To this aim, a LED array that simulates a tactile display was used. turned out more meaningful by using a mask that limits the visual field of the person wearing it, in order to see only the LED array ( Figure 5).

Figure 5.
Simulation of the vision-aid device modified with a LED array.

Evaluation of the prototype of the Vision-Aid Device
On a second evaluative stage, tests were made using the developed tactile display as a means to feedback the processed information. These tests were made with normal vision person, whose vision were physically suppressed as described above, and with blind persons (See Figure 6). However, since this latter case is more relevant, the experiences made with blind persons will be detailed below. Each experience was divided into a series of stages. The first one, the learning stage, consists of sending a series of fixed vibrating stimuli to the user, which correspond to various patterns that may occur under normal conditions. These patterns were loaded directly on the DSP, and they correspond to such obstacles as a tree or a pillar, persons, objects at a far distance, aisles, stairways, and other features. As a second stage, or identification stage of dynamic environments, moving persons at various distances were asked to traverse the stage, so as to have the device user learn to discern moving objects. Then, with the aim at having the user learn how to know his own location when wearing the device, he was put in a static surround, and he was asked to detect objects located around him. This way, he was asked to approach the various objects until touching them with his hands or with the cane. Finally, the user was allowed to use by himself all the knowledge gained through experiences, in order to improve orientation and displacement through non structured environments. He was let walk freely, with just the device and cane, and moving through the various objects found on his way.  Figure 6. Evaluation of the final prototype worn by a blind person.

Results
The results obtained through preliminary experiences allowed determining the importance of the timeintegration of the information, thanks to the processing speed attained. It was also possible to verify the proper performance of the algorithms developed in the work.
Considering the experiences made with the final device, the results attained in the tests made with normal-vision persons were successful, and they rendered important information for adjusting the device.
The various stages carried out during the experiences with blind persons produced several data and results. The first stage allowed the user be familiarized with the device's operation, and to solve probable adjusting problems to the various device components. On the second stage, the users were able to identify the motion of objects through his surrounds, and to get familiarized with the response speed of the device, as well as to learn to distinguish the various space depth levels.
As a result of the last experience, the users commented on the device as a useful means for personal displacement, as well as their capability for discriminating both temporally and spatially, the various objects or persons located in their surrounds. On a first learning stage, the device helped them fundamentally to avoid colliding with fixed and moving objects. This is a very important capability for them, because their habit is to walk and avoid objects by means of physical contact. On the other hand, the device allows them keep walking along a line without the need of using a cane touching an adjacent wall or a similar feature. This was visibly verified, because the blind persons were able to move through aisles, corridors or walkways without losing orientation, and without using the cane for such a purpose.

Discussion
As it can be noted from the results, the device attained acceptable responses in open spaces. In addition, it was possible to determine that the device performance improves considerably when the environment lighting is uniform. Work is being made in order to detect small level changes in the floor, which are not easily detected with the stereoscopic vision system yet. This latter point does not pose a significant hindrance when the blind person uses a cane altogether with the device, a compounding aid that allows him/her gain greater confidence for safer walking. Through the various experiences made, a notable point was the processing speed attained (3 disparity maps per second) which allowed the user the time-integration of the information. Although the actuator system has a low resolution, this condition is compensated by dynamic feedback, which eases the identification of moving objects and allows a fast orientation of the user through the surrounding areas.
An important factor for blind persons is the aesthetics and wearability of the device. These aspects were not regarded in this development stage, though the components were selected to achieve an industrial prototype that fits to these requirements.

Conclusions
This work has developed an environment-detection system for the portable vision-aid device developed here has demonstrated being an effective assisting means for blind persons to walk through non structured environments. The device allows tactile feeding back the relevant environment information that is processed in real time by a stereoscopic vision system. The device is made with market-available elements which have large computing power. The prototype renders additional information to blind persons without altering any of the usual surround perception mechanisms. This is due to the fact that the vibrating feedback operates on the abdominal areas, which are not used for moving or walking. Even though this prototype has a significant size and weight, the market prototype shall be conceived with optimal conditions for aesthetics and comfortable wearability. The results attained so far are encouraging, because the device allows the users avoid colliding obstacles and to orient themselves quickly in the near surroundings. Figures  Each figure should have a brief caption describing it and, if necessary, a key to interpret the various lines and symbols on the figure.