Autoregressive Drillhole Modelling Under Distribution Shift

2026-10-01

AI Summary

This study addresses the challenge of limited cross-regional generalization in drilling formation prediction caused by distribution shifts. To this end, it introduces the DrillBench benchmark alongside a novel framework. Methodologically, we identify a "transfer boundary" phenomenon and demonstrate that autoregressive modeling exhibits greater robustness to distribution shifts than spatial interpolation. Furthermore, we pioneer a backbone-agnostic strategy that synergizes large-scale pretraining with dynamic spatial retrieval. Experimental results indicate that the proposed approach significantly improves local prediction accuracy while substantially enhancing generalization under severe spatial and cross-provincial distribution shifts, achieving state-of-the-art performance on the most distant test sets.

Abstract

Autoregressive modelling has achieved remarkable success in language and sequence tasks by learning to predict future states from previous observation. Mineral-exploration drillholes provide a natural but largely unexplored setting for this paradigm: as drilling proceeds, lithology is revealed sequentially from shallow to deep, making prediction of deeper strata inherently autoregressive. Existing drillhole modelling, however, is dominated by spatial interpolation and reconstruction, or largely rely on masked modelling, leaving strictly autoregressive prediction largely underexplored. We introduce DrillBench, a benchmark of 49,671 Western Australian drillholes for next-layer prediction and autoregressive stratigraphic generation across a graded transfer spectrum, from local prediction through spatial shift to cross geological province transfer. Benchmarking classical, geostatistical, and neural models reveals a clear \emph{transfer boundary}: spatial and geochemical conditioning provides large local gains but deteriorates sharply under stronger shift, whereas lithology-sequence autoregressive models transfer more robustly. Guided by this finding, we develop a backbone-agnostic recipe combining large-scale pretraining on historical drillholes with spatial retrieval of neighbouring lithology. Retrieval is most effective in weathered cover, when local spatial continuity remains informative, whereas pretraining contributes more strongly in bedrock and under broader geological shift. Together, they retain strong local performance while improving generalisation under spatial and cross-province shift, most markedly on the most distant splits. The benchmark and code are available at https://github.com/yihaoding/drillbench.

Problem

• autoregressive modelling
• drillhole prediction
• distribution shift
• lithology sequence
• transfer generalization

Innovation

• Autoregressive Modelling
• Distribution Shift
• Drillhole Prediction
• Transfer Learning
• Spatial Retrieval