Text Detection and Recognition in Natural Scenes by Mobile Robot

Abstract

Detecting and identifying signboards on their route is crucial for all autonomous and semi-autonomous vehicles, such as delivery robots, UAVs, UGVs, etc. If autonomous systems interact more with their environments, they have the ability to improve their operational aspects. Extracting and comprehending textual information embedded in urban areas has recently grown in importance and popularity, especially for autonomous vehicles. Text detection and recognition in urban areas (e.g., store names and street nameplates, signs) is challenging due to the natural environment factors such as lighting, obstructions, weather conditions, and shooting angles, as well as large variability in scene characteristics in terms of text size, color, and background type. In this study, we proposed three stages text detection and recognition approach for outdoor applications of autonomous and semi-autonomous mobile robots. The first step of the proposed approach is to detect the text in urban areas using the "Efficient And Accurate Scene Text Detector (EAST)" algorithm. Easy, Tesseract, and Keras Optical Character Recognition (OCR) algorithms were applied to the detected text to perform a comparative analysis of character recognition methods. As the last step, we used the Sequence Matcher to the recognized text values to improve the method's impact on OCR algorithms in urban areas. Experiments were held on the university campus by an 8-wheeled mobile robot, and a video stream process was carried out through the camera mounted on the top of the mobile robot. The results demonstrate that the Efficient And Accurate Scene Text Detector (EAST) text detection algorithm combined with Keras OCR outperforms other algorithms and reaches an accuracy of 91.6%