Mutagenesis screen to map the functions of parameters of Large Language Models

2024-08-21

AI Summary

This work investigates the mapping between model parameters and functional outputs in large language models (LLMs). To address the black-box nature of this mapping, we propose a parameter-level “extreme mutation” method inspired by biological mutagenesis screening: systematically setting individual elements of weight matrices in LLaMA2-7b and Zephyr to ±∞, followed by phenotype classification and spatial pattern analysis. Our approach reveals three key findings: (1) axial clustering of parameter sensitivity across layers; (2) a two-dimensional asymmetric structure in the Gate matrix; and (3) cross-matrix “author-type” mutations—where mutated neurons group jointly by output initial token and row coordinate. We identify multi-level functional regions governing high-level behaviors, enabling fine-grained, style-controllable editing (e.g., poetry vs. dialogue generation). These results establish the interpretability and editability of parameter-level functional maps, offering a principled framework for mechanistic understanding of LLM behavior.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) have significantly advanced artificial intelligence, excelling in numerous tasks. Although the functionality of a model is inherently tied to its parameters, a systematic method for exploring the connections between the parameters and the functionality are lacking. Models sharing similar structure and parameter counts exhibit significant performance disparities across various tasks, prompting investigations into the varying patterns that govern their performance. We adopted a mutagenesis screen approach inspired by the methods used in biological studies, to investigate Llama2-7b and Zephyr. This technique involved mutating elements within the models' matrices to their maximum or minimum values to examine the relationship between model parameters and their functionalities. Our research uncovered multiple levels of fine structures within both models. Many matrices showed a mixture of maximum and minimum mutations following mutagenesis, but others were predominantly sensitive to one type. Notably, mutations that produced phenotypes, especially those with severe outcomes, tended to cluster along axes. Additionally, the location of maximum and minimum mutations often displayed a complementary pattern on matrix in both models, with the Gate matrix showing a unique two-dimensional asymmetry after rearrangement. In Zephyr, certain mutations consistently resulted in poetic or conversational rather than descriptive outputs. These"writer"mutations grouped according to the high-frequency initial word of the output, with a marked tendency to share the row coordinate even when they are in different matrices. Our findings affirm that the mutagenesis screen is an effective tool for deciphering the complexities of large language models and identifying unexpected ways to expand their potential, providing deeper insights into the foundational aspects of AI systems.

Problem

• Large Language Models
• Parameter Influence
• Task Performance Variation

Innovation

• Parameter Extreme Tuning
• LLM Functional Analysis
• Matrix Property Exploration