METIS: Mentoring Engine for Thoughtful Inquiry&Solutions

2026-01-19

AI Summary

This work addresses the challenge undergraduate students face in transforming research ideas into complete papers due to limited access to expert guidance. To this end, we propose METIS—a phased, tool-augmented AI research mentor that supports the entire workflow from topic selection to manuscript completion. METIS employs a stage-aware tutoring architecture integrating literature retrieval, method validation, writing guidance, and a memory mechanism, enhanced by task routing and document grounding strategies. In evaluations involving 90 single-turn prompts, LLM judges preferred METIS over Claude Sonnet 4.5 and GPT-5 with preference rates of 71% and 54%, respectively. Student assessments further indicated that METIS significantly outperformed baselines in clarity and actionability, yielding final outputs of marginally higher quality than those produced by GPT-5.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Many students lack access to expert research mentorship. We ask whether an AI mentor can move undergraduates from an idea to a paper. We build METIS, a tool-augmented, stage-aware assistant with literature search, curated guidelines, methodology checks, and memory. We evaluate METIS against GPT-5 and Claude Sonnet 4.5 across six writing stages using LLM-as-a-judge pairwise preferences, student-persona rubrics, short multi-turn tutoring, and evidence/compliance checks. On 90 single-turn prompts, LLM judges preferred METIS to Claude Sonnet 4.5 in 71% and to GPT-5 in 54%. Student scores (clarity/actionability/constraint-fit; 90 prompts x 3 judges) are higher across stages. In multi-turn sessions (five scenarios/agent), METIS yields slightly higher final quality than GPT-5. Gains concentrate in document-grounded stages (D-F), consistent with stage-aware routing and groundings failure modes include premature tool routing, shallow grounding, and occasional stage misclassification.

Problem

• research mentorship
• undergraduate research
• AI mentoring
• scientific writing
• idea-to-paper

Innovation

• stage-aware AI mentor
• tool-augmented reasoning
• document-grounded assistance
• research mentorship automation
• LLM-as-a-judge evaluation