Back to search

Universidad de la Laguna

Desarrollo de un sistema de lectura de textos sobre imágenes de video

Abstract

dc:description.abstract

Since the early days of computer science researchers sought to devise a machine which could automatically read text to help people with visual impairments. The problem of extracting and recognising text on document images has been largely resolved, but reading text from images of natural scenes remains a challenge. Scene text can present uneven lighting, complex backgrounds or perspective and lens distortion; it usually appears as short sentences or isolated words and shows a very diverse set of typefaces. However, video sequences of natural scenes provide a temporal redundancy that can be exploited to compensate for some of these deficiencies. Here we present a complete end-to-end, real-time scene text reading system on video images based on perspective aware text tracking. The main contribution of this work is a system that automatically detects, recognises and tracks text in videos of natural scenes in real-time. The focus of our method is on large text found in outdoor environments, such as shop signs, street names and billboards. We introduce novel efficient techniques for text detection, text aggregation and text perspective estimation. Furthermore, we propose using a set of Unscented Kalman Filters (UKF) to maintain each text region¿s identity and to continuously track the homography transformation of the text into a fronto-parallel view, thereby being resilient to erratic camera motion and wide baseline changes in orientation. The orientation of each text line is estimated using a method that relies on the geometry of the characters themselves to estimate a rectifying homography. This is done irrespective of the view of the text over a large range of orientations. We also demonstrate a wearable head-mounted device for text reading that encases a camera for image acquisition and a pair of headphones for synthesized speech output. Our system is designed for continuous and unsupervised operation over long periods of time. It is completely automatic and features quick failure recovery and interactive text reading. It is also highly parallelised in order to maximize the usage of available processing power and to achieve real-time operation. We show comparative results that improve the current state-of-the-art when correcting perspective deformation of scene text. The end-to-end system performance is demonstrated on sequences recorded in outdoor scenarios. Finally, we also release a dataset of text tracking videos along with the annotated ground-truth of text regions.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Merino Gracia, Carlos
Advisors dc:contributor.advisor
  • Sigut Saavedra, José Francisco
  • González Mora, José Luis

Rights

dc:rights.accessrights
Statement dc:rights.accessrights
  • info:eu-repo/semantics/openAccess
  • Attribution-NonCommercial-NoDerivatives 4.0 Internacional

Identifiers

dc:identifier.*
Repository record dc:identifier.uri
http://riull.ull.es/xmlui/handle/915/26489

Chain of custody

source
Harvested from
Universidad de la Laguna
Base URL
riull.ull.es/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Merino Gracia, Carlos. Desarrollo de un sistema de lectura de textos sobre imágenes de video. 2015. http://riull.ull.es/xmlui/handle/915/26489