text classification

Taxonomy construction for classifying unstructured text (e.g., personal goal statements) is time-intensive, prone to researcher bias, and suffers from poor reproducibility. Method: This paper proposes a human–AI collaborative, iterative text analysis paradigm that integrates top-down and bottom-up strategies to enable dynamic taxonomy generation, evaluation, refinement, and validation. Leveraging prompt engineering, it facilitates multi-turn collaboration between domain researchers and large language models (LLMs), with human feedback driving iterative taxonomy optimization. Intercoder reliability is quantified using Cohen’s κ within a structured coding framework. Results: Empirical evaluation in a life-domain dataset achieves κ > 0.85, significantly enhancing analytical efficiency, reliability, and reproducibility. This work pioneers deep integration of LLMs into the qualitative analysis closed loop, offering a novel methodology for low-bias, high-fidelity open-text classification.

Existing LLM-generated text detection methods suffer from poor generalizability, vulnerability to paraphrasing, limited interpretability, and reliance on access to target models or labeled data. Method: This paper proposes an unsupervised, cross-domain dual-track stylistic analysis framework that jointly models discrete stylistic features (e.g., syntactic and lexical patterns) and continuous semantic embeddings. It employs multi-domain style alignment and an end-to-end unsupervised classifier for unified representation learning—requiring neither model internals nor human annotations. Contribution/Results: The method achieves state-of-the-art in-domain accuracy across five diverse domains; improves cross-domain detection performance by up to 36.30% over baselines; and demonstrates strong robustness against human-AI hybrid texts and adversarial paraphrasing. These results empirically validate stylistic signals as reliable, generalizable indicators for trustworthy detection.

Large language models (LLMs) face dual challenges in scientific literature understanding: insufficient domain-specific knowledge and poor task alignment. To address these, we propose a “knowledge injection–task alignment” collaborative adaptation framework. Our method introduces a novel scientific text quality enhancement pipeline and constructs SciLitIns—the first high-quality instruction dataset tailored to niche scientific domains—generated via an LLM-driven synthetic instruction approach. The pipeline integrates robust PDF parsing, multi-stage quality filtering, continued pretraining (CPT), and supervised fine-tuning (SFT). The resulting model, SciLitLLM, achieves significant performance gains over general-purpose baselines across multiple scientific literature understanding benchmarks, empirically validating the efficacy of synergistic knowledge enhancement and task-specific refinement. Moreover, the framework demonstrates cross-domain transferability, establishing a systematic paradigm for adapting foundation models to specialized scientific domains.

The proliferation of highly realistic text generated by large language models has intensified risks related to misinformation and academic misconduct, underscoring the urgent need for reliable detection methods. This work proposes an online classifier that distinguishes human- from model-generated text without relying on watermarks or prior knowledge of the generative model, operating efficiently on CPU alone. By integrating statistical learning with computationally efficient feature modeling, the method introduces, for the first time, controllable statistical inference guarantees that rigorously bound Type I error while achieving high statistical power, superior classification accuracy, and strong computational efficiency. Empirical evaluations demonstrate that the proposed detector significantly outperforms existing approaches across multiple benchmarks.

The increasing prevalence of AI-generated text in educational and academic settings blurs authorship attribution and exacerbates authenticity crises. Method: We propose the first fine-grained detection framework distinguishing four categories: human-written, machine-generated, machine-generated text subsequently edited by humans, and human-written text subsequently polished by machines—extending beyond conventional binary classification by explicitly modeling ambiguous “human-AI hybrid” behaviors (e.g., obfuscating edits vs. pedagogically compliant polishing). Our approach integrates multi-scale linguistic feature extraction, contrastive representation learning, and an ensemble classifier that jointly leverages statistical, neural, and stylistic cues, enabling zero-shot transfer and cross-model generalization. Contribution/Results: Evaluated on texts generated by GPT-4, Claude, Llama, and others, our framework achieves a mean accuracy of 92.3%, significantly outperforming state-of-the-art binary detectors. The code and models are open-sourced and have undergone preliminary deployment and validation in educational institutions.

This study addresses the challenge of organizing scientific knowledge amid the exponential growth of scholarly literature by proposing an automatic hierarchical classification method based on large language models (LLMs). Leveraging in-context learning (ICL) and prompt chaining, the approach performs three-level categorization—domain, discipline, and topic—within the Open Research Knowledge Graph (ORKG) taxonomy. The first systematic evaluation demonstrates that prompt chaining significantly outperforms conventional ICL, surpassing existing state-of-the-art models particularly at the domain and discipline levels. Although accuracy at the finest-grained topic level remains moderate (approximately 50%), this work validates that off-the-shelf LLMs, without fine-tuning, can effectively support hierarchical semantic organization of scientific texts through carefully engineered prompting strategies.

This study systematically investigates stylistic differences between human- and large language model (LLM)-generated texts across genres, models, and decoding strategies to inform responsible LLM deployment. Leveraging Biber’s multidimensional framework of register variation, the authors conduct a large-scale comparative analysis of texts produced by 11 LLMs across eight genres and four decoding strategies. The findings reveal that model type and genre exert substantially stronger influences on textual style than prompting or decoding choices. Notably, chat-oriented models exhibit pronounced clustering in stylistic space, and key linguistic features of LLM-generated text demonstrate robustness across generation conditions, with genre effects consistently outweighing those of text origin.

Large language models exhibit inconsistent performance in social science text classification. This study systematically investigates the impact of three key prompt engineering components—label descriptions, instructional guidance, and few-shot examples—on classification accuracy. Through controlled experiments across multiple mainstream large language models, the authors find that moderately enriching prompt context significantly improves performance, whereas excessive augmentation can degrade it. Moreover, the optimal prompt configuration is highly dependent on the specific model, task, and data batch. The findings underscore the necessity of task- and model-specific validation and reveal a “less-is-more” principle in prompt design, offering practical guidance for achieving efficient and stable text classification in social science applications.

This study addresses the current lack of interdisciplinary understanding regarding the integration pathways, efficacy boundaries, and systemic risks of large language models (LLMs) across natural sciences, social sciences, and humanities. Through a systematic literature review and illustrative case analyses, it critically evaluates the deployment of LLMs throughout the research lifecycle—including hypothesis generation, literature synthesis, data analysis, and scholarly writing. The work identifies ten previously underappreciated systemic risks, such as diminished researcher autonomy, AI-induced confirmation bias, ambiguous authorship, and inequitable access to technology. It further demonstrates how LLMs, while enhancing efficiency, simultaneously introduce challenges like hallucination, irreproducibility, data bias, and model opacity. To guide responsible adoption, the study proposes an interdisciplinary governance framework and a roadmap for explainable AI research in scholarly contexts.

This study addresses key challenges faced by social science researchers when using large language models (LLMs) for text annotation—namely, poor reproducibility, annotation errors that compromise statistical inference, and high technical barriers. To overcome these issues, the authors propose the first end-to-end LLM-based text annotation framework tailored specifically for the social sciences and humanities (SSH). The framework integrates structured prompt engineering, open-source LLM API integration, cross-validation, and error propagation modeling, with an explicit emphasis on avoiding prompt overfitting and quantifying annotation uncertainty. Implemented in both Python and R, this approach establishes a transparent, reproducible, and scalable workflow that substantially enhances the reliability, efficiency, and methodological rigor of automated text annotation in SSH research.