Riddle Quest : The Enigma of Words

2026-01-27

AI Summary

This work proposes riddles as lightweight probes to systematically evaluate the breadth of reasoning in large language models when handling linguistic ambiguity and polysemy. To this end, the authors develop a riddle generation and evaluation pipeline comprising triplet construction, semantic mapping, stylized generation, and answer validation, integrating structured fact extraction, semantic attribute selection, and prompt-engineered text generation, along with an answer-space verification mechanism. Experimental results reveal that while models frequently identify the intended target answer, they consistently overlook other plausible interpretations, exposing limitations in their capacity for comprehensive semantic reasoning. This study represents the first systematic use of analogical riddles to probe models’ ability to encompass multiple valid answers, thereby offering a novel diagnostic framework for assessing interpretive flexibility in language understanding.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Cognitive Modeling & Cognitive Systems: Analogy
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Riddles are concise linguistic puzzles that describe an object or idea through indirect, figurative, or playful clues. They are a longstanding form of creative expression, requiring the solver to interpret hints, recognize patterns, and draw inferences to identify the answers. In this work, we introduce a simple pipeline for creating and evaluating analogy-based riddles. The system includes a triples creator that builds structured facts about a concept, a semantic mapper that selects attributes useful for analogy, a stylized generator that turns them into riddle clues, and a validator that collects all possible answers the riddle could point to. We use this validator to study whether large language models can recover the full answer set for different riddle types. Our case study shows that while models often guess the main intended answer, they frequently miss other valid interpretations. This highlights the value of riddles as a lightweight tool for examining reasoning coverage and ambiguity handling in language models.

Problem

• riddles
• analogy-based reasoning
• large language models
• ambiguity handling
• reasoning coverage

Innovation

• analogy-based riddles
• reasoning coverage
• ambiguity handling
• structured fact generation
• large language model evaluation