Personality as Relational Infrastructure: User Perceptions of Personality-Trait-Infused LLM Messaging

2026-02-06

AI Summary

This study investigates whether the effectiveness of large language models (LLMs) in delivering personality-tailored messages within just-in-time adaptive interventions (JITAIs)—by incorporating the Big Five personality traits—stems from trial-level optimization of individual messages or from user-level perceptions shaped by sustained exposure. We conducted a controlled experiment in a physical activity intervention context, comparing four LLM prompting strategies: baseline prompting, few-shot prompting, fine-tuning, and retrieval-augmented generation. Using ordinal multilevel modeling to analyze user perceptions, we found no evidence of trial-level effects; however, participants who received a higher proportion of personality-tailored messages reported significantly greater perceived personalization and appropriateness, along with lower negative affect. This work is the first to disentangle and empirically validate these dual mechanisms of personality-based personalization, proposing personality as a relational infrastructure in human–AI interaction.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Intelligent User Interfaces
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Digital behaviour change systems increasingly rely on repeated, system-initiated messages to support users in everyday contexts. LLMs enable these messages to be personalised consistently across interactions, yet it remains unclear whether such personalisation improves individual messages or instead shapes users'perceptions through patterns of exposure. We explore this question in the context of LLM-generated JITAIs, which are short, context-aware messages delivered at moments deemed appropriate to support behaviour change, using physical activity as an application domain. In a controlled retrospective study, 90 participants evaluated messages generated using four LLM strategies: baseline prompting, few-shot prompting, fine-tuned models, and retrieval augmented generation, each implemented with and without Big Five Personality Traits to produce personality-aligned communication across multiple scenarios. Using ordinal multilevel models with within-between decomposition, we distinguish trial-level effects, whether personality information improves evaluations of individual messages, from person-level exposure effects, whether participants receiving higher proportions of personality-informed messages exhibit systematically different overall perceptions. Results showed no trial-level associations, but participants who received higher proportions of BFPT-informed messages rated the messages as more personalised, appropriate, and reported less negative affect. We use Communication Accommodation Theory for post-hoc analysis. These results suggest that personality-based personalisation in behaviour change systems may operate primarily through aggregate exposure rather than per-message optimisation, with implications for how adaptive systems are designed and evaluated in sustained human-AI interaction. In-situ longitudinal studies are needed to validate these findings in real-world contexts.

Problem

• personality
• LLM
• behaviour change
• personalisation
• user perception

Innovation

• personality-infused LLM
• just-in-time adaptive interventions (JITAIs)
• cumulative exposure effect
• communication accommodation theory
• behavior change systems