BridgeAlign: Bridging Preference Alignment for Humanities and Social Sciences

2026-07-29

AI Summary

Existing preference alignment methods for large language models in the humanities and social sciences (HSS) are hindered by the absence of objective ground truths and reliance on fine-grained quality judgments. This work proposes the first three-stage preference alignment framework tailored for HSS: it begins by curating high-quality seed documents, then generates preference triplets through role-based instruction inversion, and finally introduces a fine-grained optimization mechanism grounded in quality rubrics. This mechanism integrates heuristic and LLM-based filtering, question-answering consistency verification, and controlled degradation strategies to construct near-boundary preference pairs. Evaluated across 17 benchmarks, the approach outperforms 11 strong baselines; notably, Qwen3-8B achieves state-of-the-art performance on both human preference and knowledge capability metrics, demonstrating for the first time a synergistic improvement in both dimensions without requiring trade-offs.

Abstract

While data synthesis for large language models (LLMs) is prevalent, it primarily targets domains with verifiable answers, overlooking open-ended humanities and social sciences (HSS), where nuanced quality judgments matter more than objective correctness. This makes preference alignment a natural paradigm for broad HSS tasks. Yet existing methods are either costly or not tailored to broad HSS disciplines. We thus propose BridgeAlign, among the first preference-alignment pipelines for broad HSS disciplines, with three phases: i) Seed Curation: curating HSS seed documents from web corpora via heuristic/LLM-based filtering and text refinement; ii) Preference Data Synthesis: generating preference triplets via persona-based instruction inversion with Q&A consistency checks; iii) Preference Optimization: moving beyond naive human-vs-model heuristics by first grounding preferences in HSS quality rubric, then generating transitional responses via controlled quality degradation to form near-boundary preference pairs for finer-grained quality discrimination. Aligning over 210k synthetic preference samples, BridgeAlign enables Qwen3-8B to achieve the best average across 17 benchmarks against 11 strong baselines; importantly, leading on both human-preference and knowledge-based capabilities at once, with no trade-off between them, as supported by extensive experiments and contextualized by existing theories.

Problem

• preference alignment
• humanities and social sciences
• data synthesis
• quality judgment
• large language models

Innovation

• preference alignment
• humanities and social sciences
• preference data synthesis
• quality rubric
• controlled quality degradation