Automated Marine Biofouling Assessment: Benchmarking Computer Vision and Multimodal LLMs on the Level of Fouling Scale

2026-01-28

AI Summary

This study addresses the safety risks and scalability limitations of manual diver-based inspection for hull biofouling by proposing an automated assessment framework that integrates computer vision with a zero-shot multimodal large language model (LLM). The approach employs convolutional neural networks and Transformer-based segmentation models to quantify biofouling coverage, while leveraging structured prompting and retrieval-augmented generation with the LLM for fouling severity classification. The work presents the first systematic comparison between specialized vision models and zero-shot LLMs in this domain, revealing that the former achieve higher accuracy at extreme severity levels but are constrained by data imbalance, whereas the latter offer competitive, interpretable judgments without task-specific training. The proposed hybrid method balances scalability and interpretability, demonstrating the potential of multimodal collaboration for marine biofouling assessment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Marine biofouling on vessel hulls poses major ecological, economic, and biosecurity risks. Traditional survey methods rely on diver inspections, which are hazardous and limited in scalability. This work investigates automated classification of biofouling severity on the Level of Fouling (LoF) scale using both custom computer vision models and large multimodal language models (LLMs). Convolutional neural networks, transformer-based segmentation, and zero-shot LLMs were evaluated on an expert-labelled dataset from the New Zealand Ministry for Primary Industries. Computer vision models showed high accuracy at extreme LoF categories but struggled with intermediate levels due to dataset imbalance and image framing. LLMs, guided by structured prompts and retrieval, achieved competitive performance without training and provided interpretable outputs. The results demonstrate complementary strengths across approaches and suggest that hybrid methods integrating segmentation coverage with LLM reasoning offer a promising pathway toward scalable and interpretable biofouling assessment.

Problem

• marine biofouling
• Level of Fouling
• automated assessment
• computer vision
• multimodal LLMs

Innovation

• multimodal LLMs
• biofouling assessment
• computer vision
• zero-shot learning
• hybrid reasoning