Talking with signs: a simple method to detect nouns and numbers in a non annotated signs language corpus
Abstract
People with deafness or hearing disabilities who aim to use computer based systems rely on state-of-art video classification and human action recognition techniques that combine traditional movement pat-tern recognition and deep learning techniques. In this work we present a pipeline for semi-automatic video annotation applied to a non-annotated Peru-vian Signs Language (PSL) corpus along with a novel method for a progressive detection of PSL elements (nSDm). We produced a set of video annotations in-dicating signs appearances for a small set of nouns and numbers along with a labeled PSL dataset (PSL dataset). A model obtained after ensemble a 2D CNN trained with movement patterns extracted from the PSL dataset using Lucas Kanade Opticalflow, and a RNN with LSTM cells trained with raw RGB frames extracted from the PSL dataset reporting state-of-art results over the PSL dataset on signs classification tasks in terms of AUC, Precision and Recall.
Temas
Redes neuronales (Computación)
Algoritmos computacionales
Reconocimiento óptico de patrones
Algoritmos computacionales
Reconocimiento óptico de patrones
Para optar el título de
Maestro en Informática
Collections
The following license files are associated with this item: