StructSim: Measuring Idea Similarity at Scale Through Structural Representation

2026-09-28

AI Summary

This study addresses the limitation of text embeddings in capturing structural similarities and varying levels of abstraction among creative ideas. To overcome this, we propose a structured decomposition-based evaluation framework that deconstructs ideas into core components—such as purpose and mechanism—and constructs multi-layered concept graphs. By introducing shared structural representations and set-level mechanism coverage metrics, the framework enables component-wise overlap analysis to precisely quantify distinctions between core mechanisms and implementation details. Experimental results demonstrate that our approach improves alignment with expert judgments by 31%, significantly enhancing the evaluation of both similarity and diversity in large-scale ideation tasks.

Abstract

Measuring idea similarity is fundamental to creativity evaluation, especially as LLMs enable idea generation at increasing scale. However, text embeddings collapse an idea into a single vector, making it difficult to capture structural similarity, including partial overlap across core and supporting components and differences across levels of abstraction. We introduce a shared structural representation that decomposes ideas into purpose, mechanism, and implementation components and organizes related components in a multi-layer concept graph. From this representation, we define measures of pairwise similarity and set-level mechanism coverage for assessing idea diversity. We evaluate our approach using controlled idea triples and assessments from 12 experts, focusing on differences in core mechanisms, implementations, and supporting components. Our method improves alignment with expert judgments of structural similarity by 31% over the embedding baseline and better reflects expert assessments of idea set coverage, supporting scalable evaluation of idea similarity and diversity.

Problem

• idea similarity
• creativity evaluation
• structural representation
• text embeddings
• idea diversity

Innovation

• Structural Representation
• Idea Similarity
• Concept Graph
• Creativity Evaluation
• Mechanism Coverage