reflexive design

Current large language model (LLM) evaluations suffer from methodological homogeneity and detachment from real-world societal needs. Method: This paper reframes LLM evaluation around the core objective of *socio-technical gap*—systematically measuring an LLM’s capacity to fulfill diverse, authentic application requirements. We introduce the first interdisciplinary evaluation paradigm integrating human–computer interaction (HCI), explainable AI (XAI), and natural language generation (NLG), structured along three dimensions: context-driven scenario design, stakeholder需求 mapping, and feasibility-aware trade-off analysis. Contribution/Results: We propose the first conceptual framework for the socio-technical gap, identifying critical deficiencies in existing benchmarks; establish a principled, real-world–oriented evaluation pathway; and articulate foundational open research questions. By shifting focus from technical metrics to human-centered outcomes, this paradigm advances LLM assessment toward equitable, socially grounded deployment and actively supports narrowing the socio-technical gap.

To address inherent viewpoint biases in large language models (LLMs), this paper proposes a pluralistic perspective collaborative reasoning framework grounded in deliberative democracy theory. Methodologically, it constructs configurable identity agents—incorporating nationally representative demographic modeling—orchestrates structured multi-round deliberation, implements dynamic information sharing, and introduces a moderator agent to regulate deliberative quality. Its key contribution lies in the systematic integration of political philosophy’s deliberative democracy paradigm into LLM-based multi-agent system design, achieving theory-driven architectural innovation without compromising engineering scalability. Empirical evaluation demonstrates: (1) six case studies confirm theoretical fidelity; (2) three randomized experiments show that simulated focus group outputs outperform zero-shot baselines on 75% of tasks and exhibit strong alignment with real audience responses (Pearson’s *r* > 0.82).

Contemporary information retrieval (IR) research predominantly adopts a reactive, risk-avoidance stance toward societal harms, neglecting proactive critical reflection on and constructive articulation of the sociotechnical imaginaries embedded in IR systems. Method: This paper advances a paradigm shift—centering explicit, pluralistic sociotechnical imaginaries—and systematically integrates democratic theory, critical theory, and social justice praxis to formulate a new agenda for socially just information access. Drawing on interdisciplinary perspectives from science and technology studies (STS), human–computer interaction, media studies, and critical information studies, it moves beyond algorithm-centric approaches toward institutional critique and collaborative action. Contribution/Results: The paper introduces, for the first time in IR, an operational “theory of change” framework—a systematic guide enabling the field to reorient its research agenda, technical visions, and modes of collaboration—from harm mitigation toward values-driven design practice.

This paper identifies three overlooked technical latent elements—heuristic models, critical assumptions, and parameter specifications—in interdisciplinary social computing research. Often lacking rigorous computational theoretical foundations, these elements implicitly encode normative design intentions, leading to accountability displacement and failures in socio-technical scrutiny. Method: Drawing on conceptual analysis, critical technical practice, and socio-technical systems theory, the study systematically defines and deconstructs these elements, identifying six interrelated risk dimensions. Contribution/Results: The paper introduces the first methodology-oriented warning framework explicitly targeting modeling-process transparency and cross-disciplinary accountability. Designed to support algorithmic governance, AI ethics, and human-AI collaboration research, the framework provides an actionable, deep socio-technical audit pathway that foregrounds epistemic responsibility in computational social science practice.

Contemporary NLP models excel at formal language tasks but exhibit systemic deficiencies in bias mitigation, robustness evaluation, and understanding societal impact—stemming from a lack of deep perception of social context, user diversity, and real-world consequences, termed “social awareness.” This paper introduces, for the first time, a conceptual framework for social awareness, formally defining its three core dimensions: contextual sensitivity, value embedding, and impact traceability—and positions it as a foundational capability paradigm for next-generation NLP. Methodologically, the work integrates interdisciplinary approaches including computational social science, participatory AI design, counterfactual social reasoning, multidimensional fairness metrics, and socially grounded interpretability auditing. It yields the first comprehensive roadmap for socially aware NLP development, identifying critical technical challenges and benchmarking gaps. The framework has catalyzed empirical research across more than ten international teams, advancing both theory and practice in responsible NLP.

This study addresses the limitations of prevailing culturally sensitive approaches in natural language processing (NLP), which often remain confined to surface-level representations and overlook deeper sociocultural contexts and power structures. The paper proposes a sociotechnical framework grounded in pluralistic epistemologies that moves beyond monolithic cultural adaptation paradigms by integrating local knowledge systems into NLP design. Through a five-layer model of technical activity, the authors systematically examine how culture is operationalized in NLP systems and expose critical gaps in current methods concerning governance, power dynamics, and sociocultural embeddedness. This approach reframes culture from a static object of representation to a dynamic, actionable construct, thereby advancing more reflexive and inclusive forms of cultural alignment in NLP.

This study critiques the prevailing paradigm that equates design with problem-solving, which often reduces complex socio-political issues to tractable technical challenges—a tendency emblematic of “technological solutionism” that overlooks the inherently political and value-laden nature of problem formulation. To address this, the project introduces the WPR (What’s the Problem Represented to be?) analytical framework from policy studies into design and technology research for the first time. Integrating insights from philosophy of technology and design theory, it offers a dual discursive-material critique of technological artifacts. This approach systematically uncovers the problem representations and ideological presuppositions embedded within technical objects, providing a practical reflective tool that not only challenges technological solutionism but also opens new interdisciplinary pathways between design theory and the philosophy of technology.

This study addresses a critical gap in human-computer interaction (HCI) research: the frequent neglect of the normative implications and ethical consequences inherent in conceptual design. To confront this issue, the work introduces philosophical thought experiments as a systematic method for the first time in HCI, constructing hypothetical scenarios and applying logical reasoning to critically interrogate stakeholder assumptions within value-sensitive design and to refine theories such as contextual integrity. This approach effectively uncovers the ethical dimensions and potential technological harms embedded in foundational HCI concepts, challenging prevailing frameworks while catalyzing the development of new theoretical insights. By doing so, it offers the field a forward-looking, systematic tool for ethical analysis that enhances both conceptual rigor and practical responsibility in interactive system design.

This study investigates the mechanisms underlying the emergence of social bias in artificial intelligence systems, practitioners’ understandings of these issues, and potential mitigation strategies. Employing a qualitative multiple-case design grounded in an interpretivist paradigm, the research integrates intersectionality theory and cognitive science, drawing on semi-structured interviews, document analysis, and triangulation to examine AI practitioners’ experiences across design, development, and governance. Findings reveal that algorithmic bias is deeply rooted in historical inequities, exclusionary assumptions, and organizational pressures for efficiency, underscoring the insufficiency of purely technical fixes. The study proposes an innovative approach that embeds ethical considerations early in the development lifecycle, strengthens structural accountability, fosters diverse stakeholder participation and cognitive awareness, and actively reshapes organizational culture to cultivate AI systems that are transparent, accountable, and aligned with community values.

This study addresses the limitation of prevailing generative AI evaluations that treat models as isolated predictors, thereby neglecting their sociotechnical nature as dynamically co-constructed within diverse cultural contexts. The work proposes reconceptualizing generative AI as a machine–society–human (MaSH) recursively co-constituted system, shifting evaluative focus from static outputs to value-laden interactions. Methodologically, evaluation is reframed as an embodied, recursive process through a distributive benchmark grounded in the World Values Survey, incorporating structured prompt sets and anchor-aware scoring, alongside a participatory realist methodology. Empirical analysis reveals value drift in early GPT-3 versions and demonstrates, in a real estate scenario, that the proposed framework effectively captures the dynamic societal impacts of generative AI, surpassing the constraints of conventional static benchmarks.