{'fraction_non_alphanumeric': 0.022560724578015643, 'fraction_numerical': 0.012268423219431866, 'mean_word_length': 4.430583501006036, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
SMART HUMAN FOLLOWING BABY STROLLER USING COMPUTER VISION

A baby stroller is a vehicle that is used by people to carry their toddlers making their life easier or carrying out other activities. However, such a vehicle still requires human labor to carry from one place to another place. Moreover, carrying such a vehicle can be troublesome for elderly people, disabled persons, or pregnant women. We live in an era where every aspect of human life is becoming simpler with the use of an autonomous machine. The recent development in the field of Artificial Intelligence (AI) and Machine Learning (ML) has allowed us to achieve autonomous surveillance robots to autonomous self-driving cars. This advancement of AI and ML can also be implemented in such a way that can help a person carry their babies for them which reduces their labor. This paper proposes a human following baby carrier car that can follow a human without any kind of help from the person. This reduces the labor work for a person and provides much freedom to carry out their activities. Such a carrier can be operated by elderly people, disabled people, pregnant women, or by any other without the need for much labor. The proposed system relies on the computer vision analysis that uses the camera feed to detect the person to be followed and sends a command to a microcontroller that operates the Carrier.


Introduction
A baby stroller is a transportation module mostly used by the parent to carry their toddler or baby from one place to another. The stroller has become very popular among parents that nowadays it can be seen in many places. The stroller benefits the parents in many ways, such as relieving them from carrying the baby in their hands, providing a comfortable bed for the baby, and easy transportation. The stroller also ensures the safety of the baby from the surrounding place. However, the parents are still required to carry or move the stroller by themselves which can be tedious. Also, the stroller requires force to move which can be problematic if the carrier is a pregnant mother or an old person. To address these issues, modern technologies can use to make the process easier and safer for both the carrier and the baby. An automatic human following stroller relieves the carrier from applying their force to move the carrier. Therefore, the stroller can be controlled by any person. This can benefit old persons who can't apply more force or a pregnant mother who should not apply more force. Moreover, the parents or the carrier can do much more work freely as they don't have to carry the stroller by themselves. Such a stroller can follow the human on its own by detecting the human and analyzing the environment using computer vision.

9
The trace of the early development of the baby stroller goes as far as 18 century. William Kent is considered the inventor of the baby stroller (Sehat & Nirmal, 2017). In 1733 William Kent first invented the baby carriage which latter was commercialized by Benjamin Potter Cardle in 1830 (Sehat & Nirmal, 2017). Since then, the baby stroller became very popular and throughout history, many people have designed and developed a better version of the baby stroller. In 1980, Kenzo Kassai invented a collapsible stroller that increases the size length forward (Sehat & Nirmal, 2017). In 1990, Ramon J. Cabagnero improved the baby stroller by making a foldable frame and a slide-out handle (Cabagnero, 1990). In 2000, Joan Rura developed a baby stroller that carries a batteryoperated cooling mechanism on top of the baby basket (Sehat & Nirmal, 2017). In 2010, Richard E. Cone II designed an easily foldable baby stroller that can be folded and collapsed for easy transportation (Cone, 2010). In 2013, Shun-Min Chen and Xiao-Hong Xiao designed a baby stoller which can bes used with a car seat (Chen & Xiao, 2013). In 2014, Rachelle Davis Hatfield and Nicole Tena designed a baby stroller with lighting system that helps the carrier to see front in a dark area (Hatfield & Tena, 2014). These development and improvement of the baby stroller ensures the well-being, comftness and safety of the baby. Our proposed baby stroller with human following capability help the parents to easy transport the stroller. It also reduces the workload on humans and allow them easily carry out their daily activities.
To ensure the safety of the baby inside the baby stroller is also very important otherwise many unexpected incidents can happen. Its been found that during the span of 20 years from 1990 to 2010 an average of 17187 children were injured in the United States (Fowler et al., 2016). From 2012 to 2015, approximately 248 pram-related and stroller-related injuries happened in Singapore (Tripathi et al., 2016). The study also shows that around 79% of the injuries (178 cases) are due to the lack of supervision of the adult. Also, product design defects and the wrong use of the stroller's poor performance of the stroller are some of the reasons behind the unexpected accident. (Frisbee & Hennes, 2000) (Powell et al., 2002). An automatic human following a baby stroller can work without the supervision of an adult. Hence even if an adult forgets about the baby carrier in a shopping mall or in populationheavy places, the stroller can follow the parent.
The development of humans following cars can be found in many research areas. To help medical staff and workers in their daily work Kharaga B. K. et al. designed a human following car (Kharka et al., n.d.). To help humans carry their shopping items in the cart Simarjit S. M. et al. designed a human following cart (Sharma,Kartike, n.d.). Muhammad S. H. et al. designed a robot that moves toward a human target by tracking a handmade tag that is placed on the human body ((PDF) Design and Development of Human Following Robot, n.d.). Such development inspired us to use computer vision to track the human body and analyzes the video to follow the human.
Motivating from all these works, in this paper we have designed and developed a human following baby stroller. The stroller can automatically track a human and follows the person as the person moves in the environment. The paper has been divided into several sections each of the topics has been discussed in great detail. In the methodology section, we discussed the process of our baby stroller. In the result section, we have presented the performance of the actual human following a baby stroller. In the discussion section we have discussed the findings of the result and presented our work from the perspective of everyday life and finally presented some of the improvements that can be made to the baby stroller in the future to make an even more sophisticated system.

Materials and Methods
In this work, we have developed an automated human following baby stroller with the help of computer vision analysis that can follow a human on its own. The development process involved implementing the design of the baby stroller, the design of the electrical circuit, and developing the software to run it. The working process of the whole system is given in Figure 1. In this smart baby stroller, a camera attached in front of a baby stroller sends a video feed to a small-scale computer. The computer processes the feed and detects humans. Further, the computer extracts human pose data and calculates the instruction in which the stroller should be moved to keep the human in its view and follow them. These instructions are sent to the microcontroller that controls the motor peripherals. The whole process has been divided into several parts which are discussed in the following sub-sections.

Camera Stream Process Frame Detect Human
Extract Pose Landmark Detect Changes Command Stroller to follow

Hardware Implementation
The making of a baby stroller involves making a soft, comfortable, ergonomic design that will keep the baby in a comfortable position. In our case, we have only designed a demo car to demonstrate our proposed human following mechanism. Hence the design does not portray an actual baby stroller but rather a demo car. The images of our demo car are shown in Figure 2. However, from the images, it can easily be seen that, in an actual baby stroller, we only have to include a camera, computer, and battery. The rest of the design can be any type of baby stroller.
We have made the demo car using an aluminum frame and plastic material, The body of the aluminum frame is 20 by 16 inches. The aluminum frame provides the structure of the whole car, while the plastic makes the surface. This surface will hold all the electronic circuits and the battery. Since an actual baby stroller will be tall enough, the camera can be attached at a higher position to keep track of the human. To mimic this, we have made a tall structure using the aluminum frame in which we have attached the camera. This camera will be used as our primary source to view the surroundings.
The use of aluminum has reduced some of the weight since the car stand on a regular tire and it is moved using a 6-volt dc motor. Therefore, during the whole process, we have considered the weight. The current weight of the full car is about 2.3 kg. The long stand that has been placed in the middle holds another small aluminum piece where a smartphone phone can be placed. Around the long aluminum four wires have been attached from the top of the phone holder to the bottom of the car. These wires will absorb any kind of vibration and provide more support to the structure. We have used a dc gear motor to move the car. These dc gear motors are 3-6 volt and can reach a Revolution Per Minute (RPM) of 100-240. The gear motor provides around 800gf cm min at 3-volt. We have attached the motors to the four corners of the aluminum chassis. As a wheel, we have used plastic tires. The size of the wheel is 65 by 26 mm.

Circuit Diagram
To control the motors and communicate with other peripherals we have designed an electrical circuit. In every electrical system, the circuit provides a connection between various electrical components. In this section, we have provided the circuit diagram of our demo baby stroller and discussed the function of each component in great detail. For our baby stroller, the circuit mainly involves a microcontroller, a motor driver, and batteries. As the microcontroller, we have used the ESP32 and as the motor driver, we have used the L298NN. Below we have discussed each of the circuit components and later we have provided the full schematic of the circuit diagram. ESP32 Microcontroller: ESP32 is a System on Chip (SoC) microcontroller that has a built-in integrated Wireless Fidelity WIFI 802.11 b/g/n and Bluetooth version 4.2. This chip has been developed by the Espressif Systems company. The chip has two cores clocked up to 240 MHz and has several GPIO from 17 to 36. In Figure 3, we can see there are two buttons on the ESP32, one is the boot button another is the reset button. This microcontroller is suitable for our baby stroller project since it has built-in WIFI capability. The microcontroller also has 520KB of onchip SRAM and 4 MB of external SPI flash. The microcontroller we used in this project has 32 pins where 8 are PWM pins, 4 are SPI, 2 I2C, 18 ADC, and 2 DAC pins. As the microcontroller has both WIFI and Bluetooth it is a good Internet of Things (IoT) device.

L298N Motor Driver:
The L298N motor driver is a motor driver module that consists of an L298N motor driver IC. A figure of the L298N motor driver is shown in Figure 4. The module can control up to 4 DC motors or 2 DC motors with directional and speed control. The module has a total of 13 pins. IN1, IN2, and ENA pins use to control the motor direction and speed of motor A which is connected to pins OUT1 and OUT2. IN3, IN4, ENB, OUT3, and OUT4 do the same thing for motor B. This module has a built-in 78M05 voltage regulator for switching the logic circuitry inside L298N IC. The driver is safe to be supplied with a 12-volt power supply while the jumper wire is connected. But up to 35-volt can also be supplied if the jumper wire is removed. In that case, an extra 5-volt must be supplied to the driver to power up the motor driver. Since we will power the motor driver with an 11.1-volt battery, we kept the jumper.  The circuit diagram is shown in Figure 5. According to the diagram, we have connected the four motors with the motor driver to easily control the motor spin direction and the speed. The motor driver has been powered by an external 11.1v battery. To send a command to the motor driver, ESP32 has been used. Further, the WIFI capability of the ESP32 has been used to communicate between the computer and the ESP32. The ESP32 also has been powered via a 5v power supply.

Software Implementation
The process from tracking humans to commanding the motors to move the baby stroller heavily relies on the software design. To design such software, we have adopted two languages C++ to program the ESP32 and python to track humans and send commands to ESP32. Below we have discussed first discussed the ESP32. Latter we have discussed the framework that has been used to track human's latter the process of sending commands to ESP32 has been discussed. Finally, we have discussed the workings of the whole system.
We have used the WIFI of ESP32 to communicate between the microcontroller ad the computer which provided wireless communication between them. The WIFI of the ESP32 can be used in three modes, which are, WIFI_STA, WIFI_AP, and WIFI_STA_AP. In WIFI_AP mode the microcontroller configures itself as the access point to which other electronic devices connect to the ESP32. Whereas in ESP_STA mode the ESP32 connects to another network such as a router. Finally, in WIFI_STA_AP the ESP32 both acts as an access point and also can connect to another network or a router. For our uses, we have set the ESP32 in WIFI_STA mode and connected the local network. It is necessary that the computer also connects to this same local network since there will be communication between them. Among several communication methods, WebSocket is a communication protocol that provides a persistent connection between a client and a server having a full-duplex communication channel over a single TCP connection. That data can be both sent to a server from the client and vice versa. In WebSocket, the client establishes a connection to the server through a process known as a WebSocket handshake that starts with HTTP requests.
The ESP32 microcontroller can be programmed in both C++ and Micropython. We have programmed the microcontroller using C++ and created a basic WebSocket that takes car input from any computer that sends data to it. The demo baby stroller takes five different inputs which are, Move Forward, Move Backword, Rotate Left, Rotate Right, and Speed of the stroller. The microcontroller takes the input from the computer and sends a command to the motor driver to drive the baby stroller accordingly. The part of tracking humans has been implemented using an open-source framework called MediaPipe developed by Camillo L. et al. (Lugaresi et al., 2019). MediaPipe is a framework that is implemented for building pipelines to perform inference over the sensory data (Lugaresi et al., 2019). The framework offers several ML solutions such as Face Detection, Face Mesh, Pose, Hollistic, Object Detection, etc. for live and streaming media (Home, n.d.). We have implemented the Mediapipe to get holistic of the human body and further processed it to track the human body. The MediaPipe Pose is an ML solution that provides 33 3D landmarks and a background segmentation mask (Pose, n.d.). The pose landmark models provide the pose landmark (x, y, z) of the nose, left shoulder, right shoulder, and 30 other points of the human body. Below we have provided all pose landmarks in Figure 6. The x and y provide the position of the landmark in 2D space while the z provides the information about the landmark's distance from the camera providing a 3D perception.  We have used an android phone to stream video from the baby stroller to the computer via a free android app called droid cam. The video frame was converted to 640 by 480 pixels which were given to the MediaPipe pose model to get the human body landmarks. To track the human, we have taken the left shoulder (11) and right shoulder (12) and extracted their landmark value (x, y) We have taken the shoulder as our tracking object as their landmark value was much more stable than any other landmark. These values are then converted into pixels taking the video stream frame size into account which was 640 by 480. Then the midpoint of the two points is calculated to get one single point that gives the location of the tracked human in a 640 by 480 frame. This midpoint (x, y) value has been compared with a custom threshold value continuously frame by frame to acknowledge any changes. If the changes in the midpoint surpass the provided threshold value the baby stroller will move accordingly to again keep the midpoint within the provided threshold value. This process is illustrated in the following Figure 7. In the figure, the midpoint is shown using a blue circle and the threshold border is shown using a green rectangle. If the midpoint is in position "A" then the computer will command the stroller to move backward and then rotate to the right to bring the blue circle to inside the provided threshold value or in this case the green rectangle. The threshold value is a parameter that can be customized as needed. In Algorithm 1 all the possible cases are taken and computer decisions are illustrated as pseudocode.

If midpoint value (y) is greater than
Move the stroller forward.

If midpoint value (y) is less than
Move the stroller backward.

If the midpoint value (x) is less than and midpoint value (y) is between and
Rotate the stroller to left.
If the midpoint value (x) is greater than and midpoint value (y) is between and Rotate the stroller to right.

Results
In this section, the result of our work is presented. Firstly, we uploaded the program to the ESP32, that successfully connected to the local WIFI network and created a WebSocket communication to take input from the computer. The smartphone has also been connected to the same network and configured to stream the video feed via an IP address. Finally, connecting the computer to the same network we run the main program. The program established a connection to the ESP32 WebSocket and also receives the video stream from the smartphone.
As the person shown in Figure 8 moves from the green rectangle. The computer detects the changes and commands the car to move forward. As the car moves forward then it rotates to take the blue circle to the center or inside the green rectangle. The demo baby stroller car most often accurately follows the human. The system also works when the person moves backward, in this case, the pose model gets the landmark from the human's backside. Since there is a speed limit of the motor and the camera view angle, if the person moves too fast, the car may fail to move that quickly. The demo car always maintained a distance of around three feet. Therefore, there is no chance for the car to run on the follower. We have tested the human following demo car in several places, and the performance of detecting and tracking the human was good. In most cases, it was accurately detecting the human body landmarks. The test was also conducted under several light conditions, in this case, the performance was also satisfactory. However, there was a slight delay in the reaction of the car, as the whole system was relying on wireless connectivity.

Discussion
As we have seen in the result section the demo car followed a human quite flawlessly however it also has some limitations. The current design of the baby stroller relies on a smartphone camera. Hence there is a delay to stream video from the smartphone to the computer and then from the computer to the microcontroller. Also, there was some unwanted movement on the long stand due to the speed of the car. However, such a delay and unstable stand didn't seem to cause many issues with the performance of the human following feature.
This baby stroller can help any person whether he is disabled or healthy to carry their baby. The stroller also reduces the workload on women who can be seen mostly carrying a baby. From the perspective of medical science, the baby stroller reduces the workload on parents and may reduce some of their stresses. It can also reduce the chance of losing the baby if the parents ever forget about their baby's stroller and move away from it. In this scenario, the baby follower will follow the parents. Many of the parents have multiple babies of different ages in which case it becomes very difficult to take care of all of them. In that case, the mother can take full care of her other child without worrying about the other baby. Again today mother works in the office and many have a fulltime job. In those cases, they can use the baby stroller to follow them to wherever they go without leaving their baby in their office. In today's world, it can be seen many parents live alone only with their baby, in which case the baby stroller can be very useful for her/him. It can be seen that in many tourist place parent carry their babies in their hands, which can be harmful if carries for a longer time. In those cases, the baby stroller can be used and the parents can also relieve themselves from carrying their babies in hand. Regardless of the use, the human following the baby stroller provides better security than just leaving the baby in a regular baby stroller.
The current design of the human following baby stroller leaves huge room for improvement and further advancement. As the current design relies on WIFI since each device (smartphone, computer, microcontroller) is separate. In the future, one single powerful computer can be used to both directly get the camera feed and analysis the feed onboard and then finally send commands to the motors. The quality, quantity, and range of viewing angles are other factors that can be improved to advance the baby stroller. Multiple sensors can be attached to get much more information about the surroundings to react to the baby stroller in a more advanced and sophisticated way. A baby monitor system can also be installed on the baby carrier to monitor the baby's health in real-time which can also be used as data to diagnose the baby's growth and mental health. Also, such a device always carries a risk of malfunction, hence it has to be taken extremely seriously to implement a more sophisticated decision-making system and more sensory data to prevent any unwanted catastrophe or harm to a baby. Also, in many cases, the human following the baby stroller may perform lower than a sophisticated human such as while crossing the road the baby stroller may not realize the traffic sign and may try to cross the road, which puts the baby's life in danger. But with the further advancement of technology, such an issue can be addressed and solved. Moreover, such a stroller can also reduce the rate of some of the injuries discussed previously if used correctly.

Conclusion
A smart baby stroller can already be seen in our everyday life that helps the parents to carry their baby more safely and to carry out their everyday tasks more easily. But a human following a baby stroller brings extra security and a more hassle-free stroller to the parents. As the stroller can automatically follow the parents even when they unintentionally forget about their baby stroller due to stress, pressure, or inattentiveness in a shopping mall or any other place. Also, it reduces the workload on the parents as they can carry out their daily work without much interference with the baby stroller. Our presented system shows it can follow the human without much of a failure which provides more stability to the system. In future work, we can implement a baby monitoring system to monitor the health condition of the baby. Also implement an emergency system to alarm the parents if the stroller faces any unwanted accident such as if it fails to detect the parents, or has any malfunction.