🤖 AI Summary
This paper addresses three key challenges in entity relation mining from unstructured text: (1) difficulty in extracting meaningful entity associations, (2) weak support for dynamic knowledge graph construction, and (3) poor cross-domain adaptability. To this end, we propose the first domain-agnostic, modular, and plug-and-play end-to-end framework for entity relation mining. Methodologically, it integrates multi-source, configurable entity extraction—including DBpedia Spotlight, spaCy NER, and dictionary- or rule-based phrase extraction—with co-occurrence graph modeling, frequency-based statistics, and a novel quantitative relation scoring mechanism for ranked association inference. The framework further supports document filtering, dynamic knowledge graph augmentation, and rapid adaptation to diverse business scenarios. Empirical evaluation on two financial tasks—brand-product discovery and supplier risk monitoring—demonstrates its effectiveness, significantly reducing development redundancy while improving prototyping efficiency and system reusability.
📝 Abstract
Extracting useful signals or pattern to support important business decisions for example analyzing investment product traction and discovering customer preference, risk monitoring etc. from unstructured text is a challenging task. Capturing interaction of entities or concepts and association mining is a crucial component in text mining, enabling information extraction and reasoning over and knowledge discovery from text. Furthermore, it can be used to enrich or filter knowledge graphs to guide exploration processes, descriptive analytics and uncover hidden stories in the text. In this paper, we introduce a domain independent pipeline i.e., generalized framework to enable document filtering, entity extraction using various sources (or techniques) as plug-ins and association mining to build any text mining business use-case and quantitatively define a scoring metric for ranking purpose. The proposed framework has three major components a) Document filtering: filtering documents/text of interest from massive amount of texts b) Configurable entity extraction pipeline: include entity extraction techniques i.e., i) DBpedia Spotlight, ii) Spacy NER, iii) Custom Entity Matcher, iv) Phrase extraction (or dictionary) based c) Association Relationship Mining: To generates co-occurrence graph to analyse potential relationships among entities, concepts. Further, co-occurrence count based frequency statistics provide a holistic window to observe association trends or buzz rate in specific business context. The paper demonstrates the usage of framework as fundamental building box in two financial use-cases namely brand product discovery and vendor risk monitoring. We aim that such framework will remove duplicated effort, minimize the development effort, and encourage reusability and rapid prototyping in association mining business applications for institutions.