Decoupling Perception from Description: Computation-Grounded Representation Alignment between Multivariate Time Series and Language

2026-08-05

AI Summary

This work addresses the self-supervision pitfalls in aligning multivariate time series with natural language, where large language models (LLMs) simultaneously perform perception and description, limiting label quality due to their immature perceptual capabilities. Moreover, existing datasets are predominantly univariate, hindering the modeling of cross-channel dependencies. To overcome these challenges, the authors propose a computationally anchored representation alignment framework that decouples perception—handled by a deterministic module extracting multivariate statistical features—from linguistic expression—performed by a 4B-parameter LLM. This approach is the first to break the trilemma among reliability, fidelity, and scalability. The resulting CGTime system substantially outperforms larger general-purpose models on multivariate understanding tasks, achieving the highest factual accuracy score (0.283) on held-out test sets, significantly surpassing GPT-4o-mini (0.173) and GPT-5.4-nano (0.203), with more accurate descriptions and broader statistical coverage.

Abstract

Training multimodal models to align time series with language runs into a self-supervision trap. The usual recipe asks an LLM to read a series and write a description, so label quality is capped by the perceptual skill the model is supposed to learn. The data can never teach more than the labeler already knows. A second gap makes this worse: most datasets use a single variable, but the patterns that matter (cross-channel correlation, lead-lag structure, co-occurring anomalies) appear only with several variables, right where the labeling LLM's limits are most exposed. These two problems create a trilemma: existing methods are reliable, realistic, or scalable, but none achieves all three. We resolve this by decoupling perception from description. Deterministic code computes a set of statistics from real, open-source multivariate series; the LLM verbalizes those precomputed facts. Perception, which LLMs do poorly, is handled by computation, while the LLM handles expression. This produces CGTime, our 4B-parameter computation-grounded time-series-language model. CGTime outperforms far larger general-purpose models on multivariate understanding tasks: it attains the best multivariate fact score on our held-out benchmark (0.283 vs. 0.173 for GPT-4o-mini and 0.203 for GPT-5.4-nano), a gap that survives Holm-corrected paired significance tests against every baseline. It also states verifiable numerical facts in generated captions more accurately and covers a broader range of statistical properties.

Problem

• multivariate time series
• language alignment
• self-supervision trap
• perception-decoupling
• label quality

Innovation

• computation-grounded representation
• multivariate time series
• perception-description decoupling
• statistical alignment
• time-series-language model