Machine Translation for Sign Languages

2026-09-30

AI Summary

This study addresses the challenges of sign language machine translation, including dataset scarcity, inadequate evaluation metrics, and the difficulty of modeling three-dimensional multimodal structures. By integrating computer vision with sign language linguistics, this work leverages pose estimation and Transformer architectures to systematically trace the technological evolution from isolated sign recognition to end-to-end translation. It delineates the unique characteristics of sign language computation and incorporates perspectives from the Deaf community to establish an interdisciplinary collaboration framework that balances technical innovation with ethical data governance. The core contribution lies in demonstrating that sustained progress in sign language translation depends not solely on algorithmic advancements but fundamentally on deep, sustained collaboration among stakeholders and the Deaf community.

Abstract

Sign language machine translation has progressed substantially over the past decade, evolving from isolated sign recognition to end-to-end translation systems. Advances in pose estimation, transformer architectures, and large-scale dataset collection have driven progress, yet challenges remain. Datasets are limited compared to spoken-language resources; evaluation metrics inadequately capture the linguistic quality of output; and models must capture the simultaneous, multi-layered, and three-dimensional structure of sign languages. This manuscript provides a comprehensive review that seeks to balance technical challenges with stakeholder considerations. We examine the linguistic properties that make sign languages computationally unique, trace the evolution of recognition, translation, and production systems, and analyze ongoing technical challenges. Crucially, we address ethical considerations around data governance, community involvement, and appropriate use. Drawing on interdisciplinary perspectives spanning computer vision, sign language linguistics, and deaf studies, our analysis emphasizes that continued progress requires sustained collaboration across these fields and with deaf communities.

Problem

• Sign Language Machine Translation
• Dataset Scarcity
• Evaluation Metrics
• Linguistic Complexity
• Ethical Considerations

Innovation

• Sign Language Machine Translation
• End-to-End Translation
• Pose Estimation
• Transformer Architectures
• Ethical Considerations