Prompt-based Personality Profiling: Reinforcement Learning for Relevance Filtering

2024-09-06

arXiv.org

AI Summary

To address the challenges of Transformer context-length limitations and high computational costs—stemming from processing lengthy user-generated texts (e.g., large-scale Twitter feeds)—this paper proposes a two-stage Big Five personality inference framework tailored for author profiling. First, a reinforcement learning–driven relevance filter automatically selects salient tweets; second, zero-shot prompting is applied to predict personality traits. Crucially, the filter introduces a novel reward design that requires no human annotation, synergistically integrating LLMs’ zero-shot capabilities with domain-specific characteristics of Twitter text—thus eliminating reliance on supervised signals. Experiments on real-world imbalanced data show that our method achieves comparable performance to full-tweet input using only 30% of the context. On balanced datasets, it significantly improves prediction accuracy, demonstrating a favorable trade-off between efficiency and precision.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Author profiling is the task of inferring characteristics about individuals by analyzing content they share. Supervised machine learning still dominates automatic systems that perform this task, despite the popularity of prompting large language models to address natural language understanding tasks. One reason is that the classification instances consist of large amounts of posts, potentially a whole user profile, which may exceed the input length of Transformers. Even if a model can use a large context window, the entirety of posts makes the application of API-accessed black box systems costly and slow, next to issues which come with such"needle-in-the-haystack"tasks. To mitigate this limitation, we propose a new method for author profiling which aims at distinguishing relevant from irrelevant content first, followed by the actual user profiling only with relevant data. To circumvent the need for relevance-annotated data, we optimize this relevance filter via reinforcement learning with a reward function that utilizes the zero-shot capabilities of large language models. We evaluate our method for Big Five personality trait prediction on two Twitter corpora. On publicly available real-world data with a skewed label distribution, our method shows similar efficacy to using all posts in a user profile, but with a substantially shorter context. An evaluation on a version of these data balanced with artificial posts shows that the filtering to relevant posts leads to a significantly improved accuracy of the predictions.

Problem

• Filtering relevant content for efficient author profiling
• Reducing input length for Transformer-based models
• Improving accuracy with reinforcement learning relevance filters

Innovation

• Reinforcement learning optimizes relevance filtering
• Zero-shot LLMs guide reward function design
• Filtered content improves profiling accuracy efficiently