Towards Structured Knowledge: Advancing Triple Extraction from Regional Trade Agreements using Large Language Models

2025-09-29

AI Summary

This study addresses structured knowledge extraction from Regional Trade Agreement (RTA) texts—a challenging domain characterized by legal complexity and scarce high-quality annotations. Method: We propose a large language model (LLM)-based subject-predicate-object (SPO) triplet extraction method. To mitigate data scarcity, we design a few-shot prompting framework integrating both positive and negative examples, supporting zero-shot, one-shot, and multi-strategy few-shot settings without supervision. Leveraging Llama 3.1 as the base model, we conduct rigorous quantitative evaluation (precision, recall, F1-score) and qualitative analysis on real-world RTA corpora. Contribution/Results: Our approach significantly improves extraction accuracy and interpretability over unsupervised baselines. It enables construction of a lightweight, economics-oriented trade knowledge graph. Notably, this work constitutes the first systematic investigation into LLMs’ generalization capability and knowledge structuring potential for complex international economic and trade legal documents.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Information ExtractionData Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

This study investigates the effectiveness of Large Language Models (LLMs) for the extraction of structured knowledge in the form of Subject-Predicate-Object triples. We apply the setup for the domain of Economics application. The findings can be applied to a wide range of scenarios, including the creation of economic trade knowledge graphs from natural language legal trade agreement texts. As a use case, we apply the model to regional trade agreement texts to extract trade-related information triples. In particular, we explore the zero-shot, one-shot and few-shot prompting techniques, incorporating positive and negative examples, and evaluate their performance based on quantitative and qualitative metrics. Specifically, we used Llama 3.1 model to process the unstructured regional trade agreement texts and extract triples. We discuss key insights, challenges, and potential future directions, emphasizing the significance of language models in economic applications.

Problem

• Extracting structured triples from trade agreements
• Evaluating LLM prompting techniques for knowledge extraction
• Building economic knowledge graphs from legal texts

Innovation

• Using LLMs for triple extraction from trade agreements
• Applying zero-shot one-shot few-shot prompting techniques
• Employing Llama 3.1 model to process unstructured texts