UCSC-NLP at SemEval-2026 Task 13: Multi-View Generalization and Diagnostic Analysis of Machine-Generated Code Detection

๐Ÿ“… 2026-04-28
๐Ÿ“ˆ Citations: 0
โœจ Influential: 0
๐Ÿ“„ PDF
๐Ÿค– AI Summary
This study addresses the challenges posed by AI-generated code in academic integrity, hiring assessments, and software security by proposing a multi-perspective invariant representation learning framework for robust code provenance detection across multiple programming languages and generative models. The approach integrates structural prefixes, lexical normalization, symmetric KL divergence consistency loss, token dropout, and hybrid content augmentation, along with a class-weighting strategy to mitigate performance degradation caused by extreme class imbalance in multi-class settings. Fine-tuned from UniXcoder-base, the model achieves a macro F1 score of 0.845 on binary classification tasks and significantly improves the multi-class macro F1 from 0.086 to 0.345โ€”a relative gain of 301%โ€”demonstrating its effectiveness and strong generalization capability.
๐Ÿ“ Abstract
With the rapid growth of large language models for code generation, distinguishing between human-written and AI-generated code has become increasingly critical for academic integrity, hiring evaluations, and software security. We present our system for SemEval-2026 Task 13: Multilingual Machine-Generated Code Detection, participating in Subtask A (binary detection) and Subtask B (multi-class attribution across 10 LLM families). For Subtask A, we fine-tune UniXcoder-base with a multi-view training framework that promotes generator-invariant representations. The framework combines domain-specific structural prefixes, delexicalization with symmetric KL consistency loss, token dropout, and mixed-content augmentation. Our system achieves 0.993 macro F1 on validation and 0.845 macro F1 on the test set, which spans unseen languages and domains. For Subtask B, we show that severe class imbalance (88.4% human code, 221:1 majority-to-minority ratio) causes catastrophic minority-class failure under standard fine-tuning, with macro F1 collapsing to 0.086 despite 88.4% accuracy. A class-weighted extension trained for 3 epochs recovers macro F1 to 0.345 (+301% relative), confirming that multi-class attribution requires imbalance-aware training strategies.
Problem

Research questions and friction points this paper is trying to address.

machine-generated code detection
code attribution
class imbalance
multilingual code analysis
AI-generated code
Innovation

Methods, ideas, or system contributions that make the work stand out.

multi-view training
generator-invariant representation
class imbalance
delexicalization
mixed-content augmentation
K
Kargi Chauhan
University of California, Santa Cruz
S
Sadiba Nusrat Nur
University of California, Santa Cruz