Joint Person Identity, Gender and Age Estimation from Hand Images Using Deep Multi-Task Representation Learning

2023-03-27

International Workshop on Biometrics and Forensics

AI Summary

To address the forensic challenge of identifying suspects from hand-only images in severe crimes such as sexual assault, this paper proposes the first multi-task representation learning framework specifically designed for hand image analysis, jointly performing identity recognition, gender classification, and grouped-age prediction. Methodologically, we introduce IGAE-Net—a multi-branch architecture integrating CNN and Vision Transformer (ViT) modules—featuring a shared backbone with task-specific heads, hand-aware feature modeling, age-grouping strategy, and dynamic multi-task loss weighting. Evaluated on the 11K Hands dataset, our framework achieves significant performance gains across all three tasks: 86.3% Top-1 identification accuracy, 94.2% gender classification accuracy, and 78.5% grouped-age prediction accuracy. The source code is publicly released to support reproducible research and advance forensic image analysis.

Abstract

In this paper, we propose a multi-task representation learning framework to jointly estimate the identity, gender and age of individuals from their hand images for the purpose of criminal investigations since the hand images are often the only available information in cases of serious crime such as sexual abuse. We investigate different up-to-date deep learning architectures and compare their performance for joint estimation of identity, gender and age from hand images of perpetrators of serious crime. To simplify the age prediction, we create age groups for the age estimation. We make extensive evaluations and comparisons of both convolution-based and transformer-based deep learning architectures on a publicly available 11k hands dataset. Our experimental analysis shows that it is possible to efficiently estimate not only identity but also other attributes such as gender and age of suspects jointly from hand images for criminal investigations, which is crucial in assisting international police forces in the court to identify and convict abusers. The source code is available at https://github.com/nathanlem1/IGAE-Net.

Problem

• Estimate identity, gender, and age from hand images.
• Use deep learning for criminal investigation purposes.
• Compare CNN and transformer models on 11k hands dataset.

Innovation

• Multi-task learning for identity, gender, age estimation
• Deep learning architectures comparison on hand images
• Age grouping simplifies prediction in criminal investigations