Personalities at Play: Probing Alignment in AI Teammates

2026-02-27

AI Summary

This study addresses how to enable AI teammates to exhibit predictable and interactionally meaningful personality traits in collaborative settings. The authors propose a multidimensional personality alignment evaluation framework that integrates self-report measures (BFI-44), linguistic behavior analysis (LIWC-22), and long-term memory representations to systematically assess the personality consistency of large language models—such as GPT-4o and Claude-3.7 Sonnet—under varied role prompts. Results indicate that LLMs can generate significantly distinct profiles across the Big Five personality dimensions, with long-term memory proving more effective than dialogue behavior in conveying neuroticism, conscientiousness, and agreeableness, while openness remains challenging to elicit consistently. The work underscores the critical role of system design in shaping AI personality expression and offers a novel pathway toward developing collaborative AI agents with coherent and stable personality traits.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Humans and AI: Learning Human Values and PreferencesNatural Language Processing: (Large) Language Models

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsSocial Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Collaborative problem solving and learning are shaped by who or what is on the team. As large language models (LLMs) increasingly function as collaborators rather than tools, a key question is whether AI teammates can be aligned to express personality in predictable ways that matter for interaction and learning. We investigate AI personality alignment through a three-lens evaluation framework spanning self-perception (standardized self-report), behavioral expression (team dialogue), and reflective expression (memory construction). We first administered the Big Five Inventory (BFI-44) to LLM-based teammates across four providers (GPT-4o, Claude-3.7 Sonnet, Gemini-2.5 Pro, Grok-3), 32 high/low trait configurations, and multiple prompting strategies. LLMs produced sharply differentiated Big Five profiles, but prompt semantic richness added little beyond simple trait assignment, while provider differences and baseline "default" personalities were substantial. Role framing also mattered: several models refused the assessment without context, yet complied when framed as a collaborative teammate. We then simulated AI participation in authentic team transcripts using high-trait personas and analyzed both generated utterances and structured long-term memories with LIWC-22. Personality signals in conversation were generally subtle and most detectable for Extraversion, whereas memory representations amplified trait-specific signals, especially for Neuroticism, Conscientiousness, and Agreeableness; Openness remained difficult to elicit robustly. Together, results suggest that AI personality is measurable but multi-layered and context-dependent, and that evaluating personality-aligned AI teammates requires attention to memory and system-level design, not conversation-only behavior.

Problem

• AI personality alignment
• large language models
• collaborative problem solving
• Big Five personality
• human-AI interaction

Innovation

• personality alignment
• large language models
• multi-lens evaluation
• AI teammates
• memory representation