🤖 AI Summary
This study presents the first systematic evaluation of large language models (LLMs) for news summarization in supply chain risk analysis. We address three key challenges: difficulty in identifying compliance and operational risks from heterogeneous news sources, low information density, and high redundancy in generated summaries. To this end, we propose an automated summarization framework integrating few-shot prompting, multi-source aggregation, and structured generation. We further introduce a three-dimensional evaluation metric encompassing readability, redundancy suppression, and risk identification accuracy. Experimental results show that Few-Shot GPT-4o mini significantly outperforms mainstream LLMs—achieving +23.6% higher risk identification accuracy, +18.4% improved readability, and −31.2% reduced redundancy. A user study confirms that summaries generated by our method enhance enterprise risk response efficiency by 40%, validating its practical effectiveness in real-world business scenarios.
📝 Abstract
The automation of news analysis and summarization presents a promising solution to the challenge of processing and analyzing vast amounts of information prevalent in today's information society. Large Language Models (LLMs) have demonstrated the capability to transform vast amounts of textual data into concise and easily comprehensible summaries, offering an effective solution to the problem of information overload and providing users with a quick overview of relevant information. A particularly significant application of this technology lies in supply chain risk analysis. Companies must monitor the news about their suppliers and respond to incidents for several critical reasons, including compliance with laws and regulations, risk management, and maintaining supply chain resilience. This paper develops an automated news summarization system for supply chain risk analysis using LLMs. The proposed solution aggregates news from various sources, summarizes them using LLMs, and presents the condensed information to users in a clear and concise format. This approach enables companies to optimize their information processing and make informed decisions. Our study addresses two main research questions: (1) Are LLMs effective in automating news summarization, particularly in the context of supply chain risk analysis? (2) How effective are various LLMs in terms of readability, duplicate detection, and risk identification in their summarization quality? In this paper, we conducted an offline study using a range of publicly available LLMs at the time and complemented it with a user study focused on the top performing systems of the offline experiments to evaluate their effectiveness further. Our results demonstrate that LLMs, particularly Few-Shot GPT-4o mini, offer significant improvements in summary quality and risk identification.