Autonomous Person-Following Telepresence Robot Using Monocular Camera and Deep Learning YOLO
Keywords:
Person following, Service robot, Telepresence robot, You Only Look Once (YOLO).Abstract
Telepresence robots (TRs) are increasingly important for remote communication and collaboration, particularly in situations where physical presence is not possible. One key feature of TRs is person-following, which relies on the detection and distance estimation of individuals. This study proposes an autonomous person-following TR using a monocular camera and deep-learning YOLO for person detection and distance estimation. To compensate for the monocular camera's inability to provide depth information, a novel distance estimation algorithm based on focal length and person width is introduced. The estimated width information of the detected person is extracted from the bounding box generated by YOLO. A pre-trained model using the MS COCO dataset is employed with YOLO for the person detection task. For robot movement control, a region-based controller is proposed to enable the robot to move based on the detected person's location in the image captured by the camera. Finally, integration and deployment of the proposed method in the TR is carried out using the Robot Operating System (ROS). Experimental results demonstrate that the TR can successfully follow a person using the proposed algorithm, thus highlighting its effectiveness for person-following tasks.References
A. U. Batmaz, J. Maiero, E. Kruijff, B. E. Riecke, C. Neustaedter and W. Stuerzlinger, How automatic speed control based on distance affects user behaviours in telepresence robot navigation within dense conference-like environments, PLoS One, 15(11), 2020, e0242078.
S. D. Sierra Marín et al., Expectations and perceptions of healthcare professionals for robot deployment in hospital environments during the COVID-19 pandemic, Frontiers in Robotics and AI, 8, 2021, 102.
L. Riano, C. Burbridge and T. M. McGinnity, A study of enhanced robot autonomy in telepresence, Proceedings of Artificial Intelligence and Cognitive Systems, 2011, 271–283.
L. Yang, B. Jones, C. Neustaedter and S. Singhal, Shopping over distance through a telepresence robot, Proceedings of the ACM on Human-Computer Interaction, 2 , 2018, 1-18.
A. Di Nuovo et al., The multi-modal interface of Robot-Era multi-robot services tailored for the elderly, Intelligent Service Robotics, 11(1), 2018, 109-126.
A. Cosgun, D. A. Florencio and H. I. Christensen, Autonomous person following for telepresence robots, IEEE International Conference on Robotics and Automation, Macau, China, 2013, 4335-4342.
A. Leigh, J. Pineau, N. Olmedo and H. Zhang, Person tracking and following with 2D laser scanners, IEEE International Conference on Robotics and Automation (ICRA), Seattle, USA, 2015, 726-733.
M. Cen, Y. Huang, X. Zhong, X. Peng and C. Zou, Real-time obstacle avoidance and person following based on adaptive window approach, IEEE International Conference on Mechatronics and Automation (ICMA), Tianjin, China, 2019, 64-69.
J. Cai and T. Matsumaru, Human detecting and following mobile robot using a laser range sensor, Journal of Robotics and Mechatronics, 26(6), 2014, 718-734.
A. H. Adiwahono et al., Human tracking and following in dynamic environment for service robots, IEEE Region 10 Conference, Penang, Malaysia, 2017, 3068-3073.
K. Koide and J. Miura, Identification of a specific person using color, height, and gait features for a person following robot, Robotics and Autonomous Systems, 84, 2016, 76-87.
G. Doisy, A. Jevtić and S. Bodiroža, Spatially unconstrained, gesture-based human-robot interaction, 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Tokyo, Japan, 2013, 117-118.
F. Basso, M. Munaro, S. Michieletto, E. Pagello and E. Menegatti, Fast and robust multi-people tracking from RGB-D data for a mobile robot, Intelligent Autonomous Systems, 12, 2013, 265-276.
X. Cheng, Y. Jia, J. Su and Y. Wu, Person-following for telepresence robots using web cameras, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 2019, 2096-2101.
K. Koide, J. Miura and E. Menegatti, Monocular person tracking and identification with on-line deep feature selection for person following robots, Robotics and Autonomous Systems, 124, 2020, 103348.
M. Bajracharya, B. Moghaddam, A. Howard, S. Brennan and L. H. Matthies, A fast stereo-based system for detecting and tracking pedestrians from a moving vehicle, The International Journal of Robotics Research, 28(11-12), 2009, 1466-1485.
B. X. Chen, R. Sahdev and J. K. Tsotsos, Integrating stereo vision with a CNN tracker for a person-following robot, International Conference on Computer Vision Systems (Lecture Notes in Computer Science), 2017, 300-313.
J. Satake and J. Miura, Robust stereo-based person detection and tracking for a person following robot, ICRA Workshop on People Detection and Tracking, Kobe, Japan, 2009, 1-10.
M. J. Islam, J. Hong and J. Sattar, Person-following by autonomous robots: A categorical overview, The International Journal of Robotics Research, 38(14), 2019, 1581-1618.
B. Bhavya Sree, V. Yashwanth Bharadwaj and N. Neelima, An inter-comparative survey on state-of-the-art detectors-R-CNN, YOLO, and SSD, Intelligent Manufacturing and Energy Sustainability, 2021, 475-483.
J. Redmon, S. Divvala, R. Girshick and A. Farhadi, You only look once: Unified, real-time object detection, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, 779-788.
P. Bharati and A. Pramanik, Deep Learning techniques R-CNN to mask R-CNN: A survey, Computational Intelligence in Pattern Recognition, 2020, 657-668.
H. Deshpande, A. Singh and H. Herunde, Comparative analysis on YOLO object detection with OpenCV, International Journal of Research in Industrial Engineering, 9(1), 2020, 46-64.
M. Li, Z. Zhang, L. Lei, X. Wang and X. Guo, Agricultural greenhouses detection in high-resolution satellite images based on convolutional neural networks: comparison of faster R-CNN, YOLO v3 and SSD, Sensors, 20(17), 2020, 4938.
Downloads
Published
How to Cite
Issue
Section
License
Authors who publish with this journal agree to the following terms:Authors hold and retain copyright, and grant the journal right of first publication, with the work after publication simultaneously licensed under a Creative Commons Attribution 4.0 License CC BY that permits any use, reproduction and distribution of the work and article without further permission provided that the original work is properly cited.
Authors are permitted and encouraged to post their work online in institutional repositories, website and other social media before and after publication, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
