FOCUS: Decoupling Expert Personas in LLMs to Enhance Domain Expert Capabilities

2026-08-06

AI Summary

This work addresses the instability of expert roles in current large language models when deployed in high-stakes domains such as healthcare and finance, where models often exhibit either excessive aggressiveness or undue conservatism due to cross-domain coupling. To resolve this, the authors propose FOCUS, a novel method that enables decoupled control of expert roles by automatically extracting role-specific vectors and applying orthogonal decomposition to isolate domain-specific representations. This is complemented by a context-aware expert gating mechanism that adaptively activates the most suitable role, along with a two-stage fine-tuning strategy and gating selection regularization. Experiments demonstrate that FOCUS significantly outperforms existing role-control techniques across financial, legal, medical, and cross-domain benchmarks, markedly improving both role alignment and task accuracy in single- and multi-domain settings.

Abstract

Large Language Models (LLMs) can exhibit diverse personas, and activating expert personas has been shown to improve domain expertise and task accuracy. However, existing persona control methods often suffer from cross-domain coupling, which may lead to overly aggressive behavior in high-caution domains such as healthcare, or excessive conservatism in risk-sensitive domains such as financial trading. To address this issue, we propose FOCUS (\textbf{\underline{F}}ine-tuning with \textbf{\underline{O}}rthogonal \textbf{\underline{C}}ontrol for \textbf{\underline{U}}ncoupled persona\textbf{\underline{S}}). FOCUS first automatically extracts expert persona vectors from LLMs, then applies orthogonal decomposition to decouple domain-specific expert personas, and finally introduces an expert gating module to adaptively control persona activation according to task contexts. With a two-stage training strategy and a gated selection regularizer, the model learns to activate appropriate personas for both single-domain and cross-domain tasks. Experiments on financial, legal, medical, and cross-domain benchmarks show that FOCUS improves task accuracy and outperforms existing persona control methods. Our code is available at \href{https://anonymous.4open.science/r/openpersona-48F4}{this url}.

Problem

• persona decoupling
• domain expertise
• cross-domain coupling
• expert personas
• large language models

Innovation

• persona decoupling
• orthogonal decomposition
• expert gating
• domain-specific personas
• LLM fine-tuning