{'fraction_non_alphanumeric': 0.032600511971088694, 'fraction_numerical': 0.012874567083270592, 'mean_word_length': 4.262480190174326, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 5, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Indoor Mobile Robot Navigation by Center Following based on Monocular Vision

research trends on computer vision, especially on application of robotics, and on advanced approachs for computer vision (such as omnidirectional vision). Among them, research on RFID technology integrating stereo vision to localize an indoor mobile robot is included in this book. Besides, this book includes many research on omnidirectional vision, and the combination of omnidirectional vision with robotics. This book features representative work on the computer vision, and it puts more focus on robotics vision and omnidirectioal vision. The intended audience is anyone who wishes to become familiar with the latest research work on computer vision, especially its applications on robots. The contents of this book allow the reader to know more technical aspects and applications of computer vision. Researchers and instructors will benefit from this book.

There are various moving methods where the indoor mobile robot moves by using the landmark while estimating the self-localization using the camera image (Tomono & Yuta, 2004;Rous et al., 2005;Doki et al., 2008). The method of setting up the artificial landmark in the wall and ceiling, and using the natural landmark, such as on the corner of the door are proposed. However, it is necessary to give environmental information to use the landmark beforehand, and it is difficult to move in the unknown environment. There is other method that the route is generated from the map prepared beforehand, and the robot follows the generated route (Tomono & Yuta, 2004). But this method is also need the environmental information in advance. In addition, the methods for detecting the wall and door that is an indoors common object are proposed (Moradi et al., 2006;Liu et al., 2006;Murillo et al., 2008). For the robot navigation, it is a mainstream method to follow the robot generated the route with the prior environmental information. When the environmental information is unknown, the wall following and the center following methods are proposed (Joochim & Chamnongthai, 2002;Vassallo et al., 2000;Ebner & Zell, 2000). The purpose of this research is development of the indoor mobile robot that can move even in unknown environment. Then, the center following type mobile robot is targeted the use of neither the landmark nor map information. Furthermore, the collision avoidance of the obstacle and wall is a big problem in the automatic moving in an unknown environment. Then, we develop the robot which moves at the center of the corridor when the obstacle does not exist. When the obstacle exists forward, the avoidance or stop movement is worked according to the size and position of the obstacle.

Approach
We develop the powered wheelchair based mobile robot. Our mobile robot is consisted of a general USB camera and a laptop, as shown in Fig. 1. The dimension of our robot is L = 116 cm, W = 56 cm. The camera is mounted in front of the mobile robot. The position C L of the camera is from the center of rear axle forward to 87 cm, its ground height is C H = 40 cm, and the elevation downward angle is C φ = 15 degrees. The acquired image size from the camera is 320 times 240 pixels. Two obtained images are shown in Fig. 2. The bottom of the image is about 60 cm forward of the robot, and top of the image is about 20 m forward. The moving speed of the robot is 0.823 km/h.  The developed mobile robot in this research has the following characteristics.

•
The prior landmark or map information of the moving environment is not needed.

•
A special device is not needed, and the moving environment is recognized in real time with only one general camera.

•
The robot moves at the center of the corridor when the obstacle does not exist. The avoidance or stop movement is worked according to the size and position of the obstacle when the obstacle exists forward. The process flow of our mobile robot is as follows roughly. The frontal view information is obtained by using the color imaging camera which is mounted in front of the robot. Then two boundary lines between the wall and corridor are detected. To detect these lines, we apply not a normal Hough transform but a piece wise linear Hough transform to reduce the processing time. As the space between two boundary lines, namely, the detected corridor region, we apply the proposed appearance based obstacle detection method which is improved the method proposed by Ulrich and Nourbakhsh (Ulrich & Nourbakhsh, 2000). When an obstacle exists, the size and place of obstacle is computed, and then, the robot works the avoidance or stop movement according to the information of obstacle. Otherwise, it moves toward the center of the corridor automatically. Figure 3 shows the process flow of the above mentioned. Here, in this research, the mobile robot is assumed to be moved on the indoor corridor, and human or the plant is targeted to the obstacle.

Appearance based moving environment recognition
Ulrich and Nourbakhsh proposed the appearance based obstacle region detection method (called Ulrich's method). They apply their method to the whole image. In this research, because our robot is the indoor mobile robot, not only the corridor region but also the wall and doors are observed as shown in Fig. 2. Moreover, in general, it is a straight line though the corridor in a university, a hospital, and a general building, has a little concavity and convexity by the pillar. Then, two boundary lines between the wall and corridor are detected first, different from the Ulrich's method, and the corridor region is extracted. The reduction of the processing cost is designed by applying the obstacle detection method only to the detected corridor region. The moving direction of the robot is decided by using the detected boundary line.

Line detection method
The left and right boundary lines of the corridor are almost considered to be a straight line. To detect a straight line, we use a well-known Hough transform. In general Hough transform, one point of the image space (x -y space) corresponds to one sine curve of the θspace. This relational expressed as = x cos θ + y sin θ. When some points on one straight line of the x -y space shown in Fig. 4(a) are transformed onto the θ -space, sine curves on the θ -space intersects by one point Q 1 as shown in Fig. 4(b). The Hough transform is an effective method to detect straight line, however, it has the problem with a lot of calculation costs. Then, in this research, to reduce the processing time, we apply not a normal Hough transform but a piece wise linear Hough transform (PLHT) (Koshimizu & Numada, 1989). This method was proposed by Koshimizu and Numada, and this method achieves the high speed processing by representing the Hough curve as piece wise linear approximation in Hough plane. PLHT is divided into m of θ axis (0 <= θ < ) of Hough plane, and divided points denote θ k , k = 1, 2, ..., m as shown in Fig. 4(c). This method considers m line segments which connect a section (θ k-1 , k-1 ) -(θ k , k ), and a piece wise line is called as PLH segment. This segment is obtained from following equation.

Boundary pixel detection
To apply PLHT, we first detect two boundary lines from original image. In general, there is a baseboard which color is darker than that of the wall and corridor, at the bottom side of wall, namely, the boundary between the wall and corridor as shown in Fig. 2. Thus, we detect the boundary pixel using the characteristics of this baseboard. There is edge detection method for detecting the boundary pixel. Figure 5(a) shows the applied result of well-known method of Sobel vertical edge detector to Fig. 2(a). The height of the baseboard is about 7.5 cm, and two edges, one is the edge between the wall and baseboard, and the other is the edge between the baseboard and corridor, are detected. As for the general corridor, because wax is coating, reflectivity of the corridor is higher than wall, and the baseboard, the door, and the lighting reflect to the corridor region. Especially, a part of the corridor near the boundary is unclear because the baseboard reflects. On the other hand, the edge between the wall and baseboard is clear. Then, this paper detects this edge as the boundary pixel. The binary vertical edge pixel (called the temporary edge pixel) is detected by applying Sobel edge detector from the original image. Only when the density value of the temporary edge pixel is lower than the density value of the upside pixel, it is assumed that is an edge pixel. This uses the characteristic whose density value of the wall is higher than the density value of the baseboard. Figure 5(b) shows the applied result of our boundary pixel detection method to Fig. 2(a). Only the edge in the upper part of the baseboard has been detected. In addition, the detected edge pixel is fewer, and it causes the reduction of the processing cost to apply PLHT described in the next section.
Here, the boundary pixel detection method using Sobel edge detector is called edge 1, the proposed method is called edge 2.

Boundary detection by PLHT
Right and left two straight lines are detected as a boundary line by applying PLHT based on the boundary pixel detected by the previous section. Here, the boundary line may occlude by the obstacle. In this case, the obtained boundary pixel decreases, and a wrong boundary line is detected. To avoid this problem, the information of the past boundary line is used.
The computed two parameters at frame f of PLHT denote θ f and f . If either |θ f -θ f-1 | > 3 or | ff-1 | > 10 is satisfied, we consider that the wrong boundary line is detected, and we set Figure 6 shows the applied result of our method to

Ulrich's method
The corridor region is between two boundary lines detected in the previous section, and the obstacle detection method is applied in this region. Ulrich and Nourbakhsh proposed the appearance based obstacle detection method using the histogram (Ulrich & Nourbakhsh, 2000). Their method generates the histogram of the reference region set in the image. In their method, a trapezoid region (called the reference region) is set at the bottom side of the image as shown at the left of Fig. 7. This method assumes that the color of obstacle differs from this reference region. Then, any pixel that differs in appearance from this region is classified as an obstacle. Concretely, the bin value of histogram which each pixel belongs is computed, and when this value is lower than the threshold value, it considers that this color differs from the reference region, and this pixel is classified as the obstacle region, otherwise, it is classified as the corridor region. As a result, the pixel whose color not included in the reference region is detected as the obstacle. This method is based on three assumptions that are reasonable for a variety of indoor environments: 1. obstacles differ in appearance from the corridor, 2. the corridor is relatively flat, 3. there are no overhanging obstacles.
In this research, we first applied their method. The original image is converted from RGB (red-green-blue) color space into HLS (hue-lightness-saturation) color space (Foley et al. 1993). Then, the histogram of L value is generated with the reference region. The bin value Hist(L(x, y)) of generated histogram and threshold value T L are compared, where L(x, y) is the L value at pixel (x, y). When Hist(L(x, y)) > T L then the pixel (x, y) is classified into the corridor region, when Hist(L(x, y)) <= T L then it classified into the obstacle region. The applied result of their method is shown in the second row of Fig. 7. The wrong region, such as the lighting by fluorescent or sunlight, the shadow of the person or other obstacle, are detected, though such regions does not existed in the reference region.

Proposed method
The false detection of the obstacle is caused by the influence of the lighting by Ulrich's method. Then, we proposed the improved detection method to solve above problems. To remove reflected region by lighting, the high luminance color is removed. In particular, if L(x, y) > 0.7 then the pixel is classified into corridor region even if its color does not include in the reference region. The result is shown in the third row of Fig. 7. Observing Fig. 7(a)(b), it can be confirmed that the lighting region has been removed compared with Ulrich's method of the second row. The pixel which luminance has intermediate value remains as an obstacle only by removing high luminance region as shown in Fig. 7(b)-(d). Then, two characteristics are used to solve this problem. The pixel of the intermediate value as shown in Fig. 7(b)-(d), remains as obstacle region only by removing high luminance. Then, to solve this problem, we use two characteristics. One is that the edge appears between the obstacle and corridor. Because there is roundness in a part of the edge, the object whose surface is almost flat, such as shoes and box, put on the corridor is detected as edge. On the other hand, as long as the corridor is not all-reflective material such as mirror, the reflected edge is blurred. Then, we use binary edge image described in 3.1.2. The other is the false detection is occurred easily to the dark color obstacle, which is not removed with edge information. Then, we pay attention not the edge but the color of object, and even if the edge is not detected, the darker pixel is classified into the obstacle region. The process flow of proposed method is as follows. At first, Ulrich's method is applied to detect temporary obstacle. Next, when the pixel detected as the temporary obstacle is not an edge or a dark pixel, it classifies into the corridor region. This classification process is scanned from the bottom side of the image in the temporary obstacle to the upper side. The target line changes to the next line without scanning when the obstacle is detected. The processing time is shortened though it is a little because all temporary obstacle pixels are not scanned. The applied result of proposed method is shown in the fourth row of Fig. 7. It can be confirmed that the obstacle is detected accurately compared with the result of the second and the third rows of Fig. 7.

Target obstacle region detection
The obstacle region is detected to the corridor region by the proposed method as shown in the fourth row of Fig. 7

Correspondence of image space and real space
In this research, the search range [D n , D f ] is given to detect the obstacle region described in the previous section. Moreover, in order to work the center following movement, stop movement, and avoidance movement, the distance from the mobile robot to the obstacle and the width of the corridor are needed. Here, the position of the camera mounted in the robot is fixed. At this time, the position (X, Y) of a real space at the arbitrary position (x, y) in the image can be estimated. Then, to obtain the correspondence of image space and the real space, we develop two expressions for the conversion with the real space and image space of the horizontal axis and vertical axis. The vertical position Iy [pixel] in the image space is corresponding to the distance Ry [cm] from the robot in a real space. We put several markers on the corridor. We measure the position of the image space at each marker, and calculate the conversion function. Figure  8(a) shows the measurement point of the marker and the conversion function curve. The conversion function is the fourth polynomials derived by the least square method as follow.
The conversion between the horizontal position Ix [pixel] in the image space and distance Rx [cm] in the real space is calculated as well as the above mentioned. Here, the marker was put from the robot to three kinds of distances of 1 m, 2 m, and 3 m. Figure 8

Center following movement
When the vanishing point obtained as an intersection of two boundary lines is assumed to be a moving direction, a location about 20 m or more away from the mobile robot should be assumed to be the destination. In this case, when the mobile robot is located at wall side, it www.intechopen.com does not immediately return to the central position. It returns gradually spending long time in the position of the center. Then, a central position of the corridor of 1 m forward is assumed to be the destination. The right of Fig. 7(a)-(c) shows the detected moving direction of the center following movement. The blue line is the center line of the corridor, the red point in the terminal of yellow line is the target position. In Fig. 7(c), though the obstacle is detected forward, the center following movement is worked because it is in the distance.

Stopping movement and avoidance movement
About the stopping movement and avoidance movement, we present these movements by using Fig. 9. In our system, the collision avoidance movement avoids the obstacle so as not to collide with the obstacle according to the size and the position of the obstacle. It moves again at the center of the corridor after it avoids. In this case, the strategy to avoid the obstacle by a smooth route is considered. However, the main point of this research is to move the mobile robot in real time frontal environment recognition of the robot using monocular vision. Then, the robot is moved in a polygonal line route with a few parameters as shown in Fig. 9. The avoidance angle θ and avoidance distance d 1 toward from point P to point Q are computed, and the robot rotates θ in point P, and goes straight d 1 and moves to point Q. The robot rotates -θ in point Q and it stands in parallel direction to the wall. Afterwards, the robot goes straight d 2 and moves to point R. Next, the robot rotates θ in point R, and goes straight d 1 and moves to point S. The robot returns to the center of the corridor again by rotating -θ at the end. Afterwards, the center following movement is restarted again. In a word, it returns to the center of the corridor by four turn movement. Here, the target obstacle is a person or a plant described in 2, and we set d 2 = 1 m. Point Q is a central position of the open space in corridor region. QR is parallel to PS. When the robot reaches point P, and the obstacle is detected forward d 0 , two widths of two free spaces w R and w L on right and left both sides of the obstacle are measured. The stopping movement or avoidance movement is worked according to following condition (1)-(3).
Where, W and L are the total width and total length of the mobile robot. a is a margin distance in which the robot avoid colliding with the wall and obstacle. When the condition (1) is satisfied, the mobile robot works the avoidance movement in the direction of the left side as shown in Fig. 9. Oppositely, when the condition (2) is satisfied, the avoidance movement is worked in the direction of the right side. When the condition (3) is satisfied, there is no free space where the mobile robot can move to right and left both sides. The stopping movement keeps until the obstacle goes some place. Fig. 7(d) shows the moving direction that is judged the avoidance movement. The light blue line means the bottom of the detected obstacle region. A red point which is the tip of yellow line is a target direction. www.intechopen.com

Boundary detection estimation
The boundary detection was applied to four kinds of method that combined Hough transform, PLHT where the number of division is m = 9, and two boundary pixel detection methods (edge 1 and edge 2) for ten scenes taken beforehand. To evaluate the detection accuracy, the angle error of the detected boundary is computed with visual observation. The error angle and processing time per a frame are shown in Table 1. The number of average frame of ten scenes is 557 frames. We used a laptop (CPU: Core2 Duo T7300 2.00GHz, main memory: 2GB), and an USB camera Logicool Pro 3000. The detection accuracy of PLHT is lower than that of the Hough transform in the both boundary pixel methods edge 1 and edge 2 from Table 1. However, it is greatly improved the processing time, the processing time is shorten 1/4 and 1/2 with edge 1 and edge 2, respectively. In addition, by applying proposed boundary pixel detection method edge 2, the error angle of 1.52 degrees, and the processing time of 26.6 ms/frame was obtained. Thus, we were able to prove the improvement of both the detection accuracy and processing time.

Stopping experiment
The proposed method was implemented on the mobile robot, and the recognition system of the moving environment in real time was constructed. Six obstacles shown in Fig. 10 at the position of 3 m forward of the mobile robot were put respectively, and the stopping movement was carried out five times per each obstacle on the corridor which width is 2 m. Table 2 shows the distance d 0 ' where the mobile robot actually stopped, the distance d 0 where the robot detected the obstacle. Here, we set the threshold area T O = 1800 pixel to work the stopping movement. We set the search range D n = 1 m and D f = 2 m in www.intechopen.com consideration of the moving speed of the robot. It stopped surely within the search range that the distance from the robot to the obstacle had given with D n and D f though it differed depending on the size of the obstacle. As a result, it can be confirmed that the proposed method can correctly detect the obstacle. Moreover, the difference between d 0 and d 0 ' was occured, and alway d 0 > d 0 ' is satisfied is the braking distance of the mobile robot. Though, the proposed mobile robot is worked either the stopping movement or avoidance movement according to the size and position of the obstacle, this experiment was to verify the accuracy of the stopping movement, and whenever the obstacle was detected, it was stopped. Stopped distance d 0 ' = 155 cm is short though the width 57 cm of the chair is the largest of six obstacles. Since, the open space under the chair was existed due to the legs and casters, S O of the chair was small. As for this case, the comparable result was seen in the person on the front and the side. Moreover, the processing time was 9.7 fps using the same camera and a laptop of previous experiment. It was confirmed to be able to recognize a running environment forward in real time.

Moving experiment
The moving experiment of the mobile robot in five scenes that changed the location of the obstacle was carried out. The target route and result route of the mobile robot at an axle center, and the location of the obstacle is shown in Fig. 11. The start point of the robot was (0, 0), and the goal point G was (1000, 0). In Fig.11(a)-(d), the obstacle was the potted plant which diameter was 45 cm, and the center location of each scene was, (500, 0) at (a), (500, 45) at (b), (500, -45) at (c), and (500, 105) at (d). In Fig. 11(e), we put the potted plant at (500, 105) and two persons were stood at location of (500, 0) and (500, -105), respectively. Fig. 11. The resulting route of the moving experiment.
www.intechopen.com Table 3 shows the computed distance d 0 , and avoidance angle θ, and four error values, Δx S , Δy S , Δθ S , these are in point S, and Δy G in point G. When there was avoidance space like Fig.   11(a)-(c) even when the obstacle exists forward, the robot avoided correctly and returned the center of the corridor again. Moreover, the robot moved straight without working the avoidance movement when not colliding by driving straight ahead even if the obstacle existed forward like Fig. 11(d), and it stopped when there was no avoidance space like Fig.  11(e). The maximum error distance in point S was 22.5 cm and 30.8 cm of x and y axes, respectively. The maximum error distance in point G was 17.0 cm. It is thought that the width of the corridor is narrow in the distance, and it caused the error easily in a positive direction.  Table 3. Result of traveling experiment.

Face-to-face experiment
The previous experiment was the moving experiment with the stationary obstacle. In this experiment, we use two mobile robots with the same function as a moving obstacle. Two center following robots R 1 and R 2 were made to coexist in the same environment, and faceto-face moving experiment was carried out. R 1 is a robot that has used by the previous experiment, and R 2 is a newly developed robot whose total length is 107 cm and the width is 60 cm. The experiment place is on the corridor which width is 340 cm, and this width is enough to work the avoidance movement. Two robots were put face-to-face on both sides of 10 m away. Figure 12 and Fig. 13 show the experiment scenes taken by video camera and camera mounted on R 1 , respectively. In these figures, R 1 moved from the far side to the near side, and R 2 moved from the near side to the far side. In Fig. 12(a) and Fig. 13(a), these are the initial scenes. In Fig. 12(b) and Fig. 13(b), R 1 was stopped to detect R 2 as the obstacle, at point P, and R 2 was also stopped to detect R 1 . In Fig. 12(c) and Fig. 13(c), R 1 was rotated θ to work avoidance movement. In Fig. 12(d) and Fig. 13(d), R 1 was at point Q after second rotation. In this time, R 2 moved again at the center of the corridor when R 1 moved outside the view of the camera of R 2 . In Fig. 12(e) and Fig. 13(e), R 1 was stopped at point S, and Fig.  12(f) and Fig. 13(f) are scenes when R 1 was worked four rotation. After that, R 1 moved again at the center of the corridor after having correctly worked the avoidance movement. It was confirmed that two robots were able to coexist under the same environment.

Conclusions and future works
This paper proposed the appearance based method for detecting two boundary lines between the wall and corridor and the obstacle region through the image processing based on monocular vision. Moreover, the proposed method was implemented in the wheelchair based indoor mobile robot. The developed robot moved at the center of the corridor, and it worked the stopping or avoidance movement according to the size and position of the obstacle even in the moving environmental information was unknown.
There is a problem only that our robot is possible to go straight though it can move at the center of the corridor while avoiding the obstacle. Then the future work is the corner www.intechopen.com detection to turn the corridor. Furthermore, our robot is only allowed to move automatically with center following. Thus, we give the destination of the robot, and it moves to the destination automatically.