Predicting Steering Vectors and Adapter Weights for Few-Shot Author-Style Transfer

2026-10-02

AI Summary

This study addresses the challenge of few-shot author style transfer in large language models, where the formal conventions of scientific writing cause severe entanglement between style and content, hindering pure style signal extraction. To tackle few-shot style-conditioned summarization, it proposes three disentanglement strategies: activation steering via author-level contrastive learning, steering vector prediction, and hypernetwork-generated LoRA adapters. The findings reveal that manually curated and predicted steering vectors are nearly orthogonal yet yield comparable performance, confirming the existence of multiple independent directions for style conditioning. Furthermore, hypernetworks achieve the optimal trade-off between style imitation and generation quality across both seen and unseen authors. Contrastive steering also outperforms predefined style-bank approaches by effectively preserving fluency while precisely capturing stylistic signals.

Abstract

Adapting large language models to an individual author's style from a few examples is challenging, and scientific writing sharpens the difficulty: formal conventions leave little surface variation, and authors write about their own topics, so extracted ``style'' easily entangles with content. We study style-conditioned abstract generation from a few example abstracts per author and propose three methods: (1) contrastive activation steering, (2) a network that predicts steering vectors, and (3) a hypernetwork that predicts LoRA adapters. We find a consistent trade-off between style imitation and output quality: fine-tuning buys most of the available style signal but forfeits fluency, while the hypernetwork achieves the best trade-off on both seen and unseen authors. Our steering operates at author level, contrasting an author's abstracts against style-neutral generations for the same content. This holds topic fixed, removes the need for a predefined style inventory, and outperforms inventory-based steering. % [EDIT 1a] softened "no single optimal axis" claim Moreover, our analyses demonstrate that manually extracted and predicted steering vectors are near-orthogonal yet score comparably, indicating that style conditioning here can admit at least two unrelated directions rather than requiring one particular axis.

Problem

• few-shot author-style transfer
• large language models
• scientific writing
• style-content disentanglement
• style-conditioned generation

Innovation

• Few-Shot Author-Style Transfer
• Contrastive Activation Steering
• Hypernetwork
• LoRA Adapters
• Steering Vectors