{'fraction_non_alphanumeric': 0.024757804090419805, 'fraction_numerical': 0.00598762109795479, 'mean_word_length': 4.605203619909502, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Multi-sensor perceptual system for mobile robot and sensor fusion-based localization

This paper presents an Extended Kalman Filter (EKF) approach to localize a mobile robot with two quadrature encoders, a compass sensor, a laser range finder (LRF) and an omni-directional camera. The prediction step is performed by employing the kinematic model of the robot as well as estimating the input noise covariance matrix as being proportional to the wheel's angular speed. At the correction step, the measurements from all sensors including incremental pulses of the encoders, line segments of the LRF, robot orientation of the compass and deflection angular of the omni-directional camera are fused. Experiments in an indoor structured environment were implemented and the good localization results prove the effectiveness and applicability of the algorithm.


INTRODUCTION
A mobile autonomous robot requires accurate positioning for its navigational action. From current readings of the sensors, robot must determine exactly its position and orientation in the environment. There are some kind of sensors which can be used for robot. Each sensor often only measure one or two environmental parameters with a limited accuracy. Naturally, more used sensor, more accuracy for determine position of robot. That is reason the sensor fusion method has been carried out in modern robots in order to increase the accuracy of measurements. Almost implementation of this method are based on probabilistic inferences. The Extended Kalman Filter (EKF) is the most efficient probabilistic solution to simultaneously estimate the robot position and orientation based on some interoceptive and exteroceptive sensor information.
Mobile robots often use optical encoders as interoceptive sensor for determining of position following a method named as the odometry. However, due to accumulated error, the uncertainty of estimated position by odometric system is increased time by time during robot's moving. In order to overcome this disadvantage, by using a Kalman filter with other measurements from one or some exteroceptive sensor combined with odometric measures, estimated position value becomes more accurate. That means the estimated trajectory is nearer to the true trajectory.
Several works have been reported to cope with the problem of sensor fusion. Ying Zhang has developed a Bayesian technique to estimate the state evolution, which prevents from dealing with Dirac delta function [1]. Al-Dhaher and Makesy proposed a generic architecture that employed an standard Kalman filter and fuzzy logic techniques to adaptively incorporate sensor measurements in a high accurate manner [2]. In [3], a Kalman filter update equation was developed to obtain the correspondence of a line segment to a model, and this correspondence was then used to correct position estimation. In [4], an extended Kalman filter was conducted to fuse the DGPS and odometry measurements for outdoor-robot navigation.
In our work, one novel platform of mobile robot with some modern sensor was designed and installed. A positioning software was developed which include the Extended Kalman Filter (EKF). The input information is an odometric system, a compass sensor, a LRF and an omni-directional camera. The output information is the robot's pose including the position and orientation. The experiment shows that the estimated output values is nearest to the true trajectories when measurements from all sensors are fused together.
The paper is arranged as follows. Details of the sensor system are described in Section II. The algorithm for sensor fusion using EKF is explained in Section III. Section IV introduces experimental results. The paper concludes with an evaluation of the system.

II. SENSOR SYSTEM DESIGN
The sensor system consists of a compass sensor, a LRF, an Omni-directional camera and three quadrature encoders. Fig.1 describes the sensors in relation with communication channels in a mobile robot. The optical quadrature encoders are used as measurement for positioning and feedback for a closed-loop motor speed controller. An optical encoder is basically a mechanical light chopper that produces a certain number of sine or square wave pulses for each shaft revolution. As the diameter of wheel and the gear coefficient are known, the angular position and speed of wheel can be calculated. In the quadrature encoder, a second light chopper is placed 90 degrees shifted with respect to the original resulting in twin square waves as shown in fig.2. Observed phase relationship between waves is employed to determine the direction of the rotation.   From the center of the omni-directional image (x c ,y c ), the coordinate relation between a pixel in the omni-directional image (x 0 , y 0 ) and its projection in the cylinder plane (x p ,y p ) is as follows: Images captured by the omni-directional camera contains rich information about the position and orientation of objects. The main advantages of the camera include the continuous appearance of landmarks in the field view and the conservation of line feature over image transformations [8].
The quality of images however much depends on the lighting condition and may appear random noises. In addition, the resolution of the image is not uniform due to nonlinear characteristic of the hyperbolical mirror. Consequently, care should be maintain when performing image processing algorithms.
In this work, the omni-directional camera is used as an absolute orientation measurement and is fused with other sensor to create a complete perceptual system for the robot. The method is based on the detection of a red vertical landmarks located at a fixed position (x m , y m ). The conservation of line feature ensures that the shape of the landmark is unchanged in both omni-directional and panorama images. The robot's orientation determination then becomes the problem of calculating the orientation x p /2 of the landmark ( fig.5). The algorithm can be summarized as follows: From the capture image, a digital filter is applied to eliminate random noises. The red area is then detected and the image is transformed from the RGB color space to the grey scale. Applying Hough transform, the vertical line is extracted and the value x p /2 corresponding to the robot's orientation is obtained. Figure 5.
Line detection using Houh transform A 2D laser range finder (LRF) with a range from 0.04m to 80m is developed for the system. Its operation is based on the time-of-flight measurement principle. A single laser pulse is emitted out and reflected by an object surface within the range of the sensor. The lapsed time between emission and reception of the laser pulse is used to calculate the distance between the object and the sensor. By an integrated rotating mirror, the laser pulses sweep a radial range in its front so that a 2D field/area is defined [10]. In each environment scan, the LRF gives a set of distance data to the obstacles at the angles. Fig.6 shows the view field of one LRF scan. Based on this data, environmental features can be found [9].   The proposed sensor platform equips the robot with the ability to perceive many parameters of the environment. Their combination, however, presents even more useful information. In this work, the raw data of three different types of sensors including the compass sensor, the quadrature encoder and the LRF is syndicated inside an EKF. The aim is to determine the robot position during operation as accurately as possible.
We start with the kinematic model of the mobile robot. The two wheeled, differential-drive mobile robot with nonslipping and pure rolling is considered. Fig.9 shows the coordinate systems and notations for the robot, where (X G , Y G ) is the global coordinate, (X R , Y R ) is the local coordinate relative to the robot chassis. R denotes the radius of driven wheels, and L denotes the distance between the wheels.
During one sampling period Δt, the rotational speed of the left and right wheels ω L and ω R create corresponding increment distances Δs L and Δs R traveled by the left and right wheels respectively: In practice, (3) is not really accurate due to unavoidable errors appeared in the system. Errors can be both systematic such as the imperfectness of robot model and nonsystematic such as the slip of wheels. These errors have accumulative characteristic so that they can break the stability of the system if appropriate compensation is not considered. In our system, the compensation is carried out by the EKF.

[
] T x yθ = x be the state vector. This state can be observed by some absolute measurements, z. These measurements are described by a nonlinear function, h, of the robot coordinates and an independent Gaussian noise process, v. Denoting the function (3) is f, with an input vector u, the robot can be described by: where the random variables w k and v k represent the process and measurement noise respectively. They are assumed to be independent to each other, white, and with normal probability distributions: The steps to calculate the EKF are then realized as below: 1. Prediction step with time update equations: where ˆn k − ∈ ℜ x is the priori state estimate at step k given knowledge of the process prior to step k, ˆk − P denotes the covariance matrix of the state-prediction error, A k is the Jacobian matrix of partial derivates of f to x: W is the Jacobian matrix of partial derivates of f to w: sin( / 2) / 2 sin( / 2) / 2 | ; os( / 2) / 2 os( / 2) / 2 (8) and 1 k − Q is the input-noise covariance matrix which depends on the standard deviations of noise of the right-wheel rotational speed and the left-wheel rotational speed. They are modeled as being proportional to the rotational speed ω R,k and ω L,k of these wheels at step k. This results in the variances equal to The input-noise covariance matrix Q k is defined as: 2. Correction step with measurement update equations: where ˆn k ∈ ℜ x is the posteriori state estimate at step k given measurement k z , K k is the Kalman gain, H is the Jacobian matrix of partial derivates of h to x: At the first scan of LRF, a global map of environment is constructed composed of a set of line segments described by parameters β j and ρ j . The line equation in normal form is: When the robot moves, a new scan of LRF is performed and a new map of environment, namely local map, is constructed which also consists of a set of line segments described by the equation: x R cosψ i + y R sinψ i = r i (15) where ψ i and r i are the parameters of lines ( fig.10). The line segments of the global and local map are then matched using weighted line fitting algorithm [11]. The matching line parameters i r and i ψ from the local map are collected in the vector z k , which is used as the input for the correction step of the EKF to update the robot's state: From (16) and (19), the data of the compass sensor is utilized to correct the robot orientation. At step k, this data is employed as the absolute measurement for the element θ k of z k . The noise of this measurement is achieved from the manufactory specification. The accuracy of 0.1 0 corresponds to the noise variance of 0.01. This collected into the covariance matrix R i,k (19) gives R k for the correction step of EKF.

IV. EXPERIMENTS
The setup of the sensor system has been described in Section II. In this section, experiments are conducted to evaluate the efficiency of the fusion algorithm.

A. Experimental Setup
A rectangular shaped flat-wall environment constructed from several wooden plates surrounded by a cement wall is setup. The robot is a two wheeled, differential-drive mobile robot. Its wheel diameter is 10 cm and the distance between two drive wheels is 60cm.
The speed stability of the motor during each sampling time step is an important factor to maintain the efficiency of the EKF. For that reason, motors are controlled by microprocessorbased electronic circuits which carries out a PID algorithm. The stability checked by a measuring program written in LABVIEW is ±5%.
The compass sensor has the accuracy of 0.1 0 . The LRF has the accuracy of 30mm in distance and 0.25 0 in deflect angle. The sampling time Δt of the EKF is 100ms. The proportional factor δ of the input-noise covariance matrix Q k is experimentally estimated as follows. The deviations between the true robot position and the position estimated by the kinematic model when driving the robot straight forwards several times (from the minimum to the maximum tangential speed of the robot) is determined. The deviations between the true robot orientation and the orientation estimated by the kinematic model when rotating the robot around its own axis several times (from the minimum to the maximum angular speed of the robot) is also determined. Based on the error in position and orientation, the parameter δ is calculated. In our system, the δ is estimated to be the value 0.01. Fig.11 shows the robot and environment structure in experiments.

B. Sensor Fusion Algorithm Evaluation
In order to evaluate the efficiency of the fusion algorithm, different configurations of the EKF were implemented. Fig.12 describes the trajectories of a robot movement estimated by five methods: the odometry, the EKF with compass sensor, the EKF with LRF, the EKF with omni-directional camera and the EKF with the combination of all sensor. The deviations between each trajectory and the real one are represented in fig.13.  In another experiment, the robot is programmed to follow predefined paths under two different scenarios: with and without the EKF. Fig.14a represents the trajectories of the robot moving along a rounded rectangular path in which the one with dots corresponds to the non-existence of EKF and the one with pluses corresponds to the existence of EKF in the implementation. The trajectories in case the robot follows a rounded triangular path is shown in fig.14b.
It is concluded that the EKF algorithm improves the robot localization and the combination of all available sensors gives the optimal result. It is necessary to develop a perceptual system for the mobile robot. The system is required to not only be wellworking but also critically support various levels of perception. In this work, many types of sensors including position speed encoders, integrated sonar ranging sensors, compass and laser finder sensors, the global positioning system (GPS) and the visual system have been implemented in a real mobile robot platform. An EKF has been designed to fuse the raw data of compass sensor and LRF. Experiments show that this novel combination significantly improves the accuracy of robot localization and is sufficient to ensure the success of robot navigation. Further investigation will be continued with more sensor combination to better support the localization in outdoor environments.