Publications - transformers

Multilingual Sign Language Translation with Unified Datasets and Pose-Based TransformersWorkshop paper

Pedro Dal Bianco, Oscar Stanchi, Facundo Quiroga, Franco Ronchetti

Workshop on Sign Language Processing (WSLP), IJCNLP-AACL 2025

One pose-based translation model trained jointly on German, Greek, Argentinian and Indian sign languages through a standardized data layer. Multilingual pre-training followed by a short language-specific fine-tuning beats the monolingual baselines on three of the four datasets.

sign-language-translation multilingual pose-based transformers

SignAttention: On the Interpretability of Transformer Models for Sign Language TranslationWorkshop paper

Pedro Dal Bianco*, Oscar Stanchi*, Facundo Quiroga, Franco Ronchetti, Enzo Ferrante

Interpretable AI Workshop (IAI) and LatinX in AI Workshop (oral), NeurIPS 2024

The first comprehensive interpretability analysis of a Transformer sign language translation model, on Greek Sign Language. The model attends to clusters of frames aligned diagonally with the glosses, and shifts from the video to its previous predictions as the translation progresses.

sign-language-translation interpretability transformers

A Study on pose-based deep learning models for gloss-free Sign Language TranslationJournal article

Pedro Dal Bianco, Gastón Ríos, Waldo Hasperué, Oscar Stanchi, Facundo Quiroga, Franco Ronchetti

Journal of Computer Science and Technology (JCS&T) 24(2), e09, 2024

A gloss-free, pose-based Transformer for sign language translation. Using pose keypoints instead of video greatly reduces the size of the data and of the model. Trained and evaluated on RWTH-Phoenix-2014T and the Greek Sign Language dataset (GSL).

sign-language-translation gloss-free pose-based transformers