Quantitative Hardness Assessment with Vision-based Tactile Sensing for Fruit Classification and Grasping

2025-05-09

AI Summary

Non-destructive, quantitative assessment of fruit hardness remains challenging for agricultural robots. Method: This paper proposes a vision–tactile fusion-based single-contact hardness quantification method. It employs the average normal force dynamic response as a universal hardness indicator, integrating normal-force dynamical modeling with real-time force feedback control to enable cross-varietal and cross-size hardness estimation and closed-loop adaptive robotic grasping. Results: Experiments across multiple fruit types and continuous ripeness stages demonstrate <8.2% hardness prediction error, a 47% reduction in grasping-induced damage, and millisecond-level online regulation. The core contribution is the first formulation of normal force dynamics as a generic hardness representation—overcoming limitations of conventional approaches requiring multiple indentations or fruit-specific deformation models—thereby establishing a unified perception foundation for cultivar identification, ripeness monitoring, and compliant grasping.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Vision for Robotics & Autonomous Driving
• Humans and AI: Interaction Techniques and Devices

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Accurate estimation of fruit hardness is essential for automated classification and handling systems, particularly in determining fruit variety, assessing ripeness, and ensuring proper harvesting force. This study presents an innovative framework for quantitative hardness assessment utilizing vision-based tactile sensing, tailored explicitly for robotic applications in agriculture. The proposed methodology derives normal force estimation from a vision-based tactile sensor, and, based on the dynamics of this normal force, calculates the hardness. This approach offers a rapid, non-destructive evaluation through single-contact interaction. The integration of this framework into robotic systems enhances real-time adaptability of grasping forces, thereby reducing the likelihood of fruit damage. Moreover, the general applicability of this approach, through a universal criterion based on average normal force dynamics, ensures its effectiveness across a wide variety of fruit types and sizes. Extensive experimental validation conducted across different fruit types and ripeness-tracking studies demonstrates the efficacy and robustness of the framework, marking a significant advancement in the domain of automated fruit handling.

Problem

• Quantify fruit hardness for automated classification and handling
• Develop vision-based tactile sensing for robotic agriculture applications
• Ensure non-destructive, real-time fruit hardness assessment

Innovation

• Vision-based tactile sensing for hardness assessment
• Normal force dynamics calculate fruit hardness
• Universal criterion for diverse fruit types