Representing LLMs in Prompt Semantic Task Space

2025-09-26

AI Summary

Selecting the most suitable pre-trained large language model (LLM) for a given task remains challenging due to the vast number of available models and the lack of efficient, interpretable, and training-free selection mechanisms. Method: This paper proposes a fine-tuning-free LLM representation framework that models an LLM as a linear operator in a prompt-semantic task space—constructed from semantic embeddings of prompts—and characterizes its task transformation behavior via closed-form geometric computations. Contribution/Results: The method requires zero training, enables real-time inference, and provides cross-task interpretability. Experiments demonstrate state-of-the-art performance in both model selection and performance prediction, particularly excelling in zero-shot generalization to unseen tasks and previously unencountered models—highlighting strong extrapolation capability and practical utility.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) achieve impressive results over various tasks, and ever-expanding public repositories contain an abundance of pre-trained models. Therefore, identifying the best-performing LLM for a given task is a significant challenge. Previous works have suggested learning LLM representations to address this. However, these approaches present limited scalability and require costly retraining to encompass additional models and datasets. Moreover, the produced representation utilizes distinct spaces that cannot be easily interpreted. This work presents an efficient, training-free approach to representing LLMs as linear operators within the prompts' semantic task space, thus providing a highly interpretable representation of the models' application. Our method utilizes closed-form computation of geometrical properties and ensures exceptional scalability and real-time adaptability to dynamically expanding repositories. We demonstrate our approach on success prediction and model selection tasks, achieving competitive or state-of-the-art results with notable performance in out-of-sample scenarios.

Problem

• Identifying optimal LLMs for specific tasks from abundant repositories
• Overcoming limited scalability and costly retraining in model representation
• Creating interpretable LLM representations without requiring model training

Innovation

• Training-free LLM representation as linear operators
• Utilizes prompts' semantic task space for interpretability
• Closed-form computation ensures scalability and real-time adaptability