Tracking GPTs Third Party Service: Automation, Analysis, and Insights

2025-06-18

AI Summary

This study addresses the critical challenge of insufficient privacy configuration disclosure and the difficulty of systematically assessing data privacy impacts in GPTs’ integrations with third-party services. To this end, we propose GPTs-ThirdSpy, the first fully automated framework for reverse-engineering third-party privacy configurations within GPTs, performing dynamic API traffic analysis, metadata pattern matching, and rule-driven policy parsing—culminating in a standardized privacy metadata model. Empirically evaluated across over 100,000 GPT instances, GPTs-ThirdSpy precisely identifies 327 distinct third-party services and their respective privacy disclosure statuses. Our work fills a fundamental gap in auditability research for third-party integrations in the GPT ecosystem and provides a reproducible, scalable empirical foundation and methodological framework for GDPR compliance assessment and regulatory oversight of large language models.

Technology Category

• Machine Learning: Privacy
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

ChatGPT has quickly advanced from simple natural language processing to tackling more sophisticated and specialized tasks. Drawing inspiration from the success of mobile app ecosystems, OpenAI allows developers to create applications that interact with third-party services, known as GPTs. GPTs can choose to leverage third-party services to integrate with specialized APIs for domain-specific applications. However, the way these disclose privacy setting information limits accessibility and analysis, making it challenging to systematically evaluate the data privacy implications of third-party integrate to GPTs. In order to support academic research on the integration of third-party services in GPTs, we introduce GPTs-ThirdSpy, an automated framework designed to extract privacy settings of GPTs. GPTs-ThirdSpy provides academic researchers with real-time, reliable metadata on third-party services used by GPTs, enabling in-depth analysis of their integration, compliance, and potential security risks. By systematically collecting and structuring this data, GPTs-ThirdSpy facilitates large-scale research on the transparency and regulatory challenges associated with the GPT app ecosystem.

Problem

• Analyzing third-party service integration in GPTs for privacy risks
• Automating extraction of GPTs' privacy settings for research
• Evaluating transparency and compliance challenges in GPT ecosystems

Innovation

• Automated framework for GPTs privacy extraction
• Real-time metadata on third-party services
• Large-scale analysis of GPT ecosystem transparency