CircuitKIT : Circuit Discovery, Evaluation, and Application Toolkit for Mechanistic Interpretability

2026-07-21

AI Summary

Existing circuit analysis methods are fragmented and lack a unified framework to support the end-to-end pipeline from discovery and evaluation to downstream interventions, often relying on manually crafted contrastive prompts that hinder reproducibility and scalability. This work proposes the first end-to-end circuit analysis toolkit, built upon a typed, serializable circuit representation that integrates multiple discovery algorithms, declarative task mapping, diagnostic utilities, and intervention modules—including pruning, editing, and steering—to enable fully interpretable mechanistic analysis throughout the pipeline. The framework facilitates automated contrastive prompt generation, cross-task transfer, and algorithmic comparison, substantially lowering barriers to both research and practical application. The complete library, along with examples and documentation, has been open-sourced to provide the community with standardized, reusable infrastructure.

Abstract

Circuit analysis can support not only model explanation but also downstream interventions such as pruning, editing, steering, and selective fine-tuning. However, conducting such analyses currently requires stitching together separate implementations for discovery, evaluation, and intervention, as well as hand-authoring the contrastive prompts required by many discovery methods. This fragmentation makes methods difficult to compare and limits their application beyond canonical tasks. We introduce CircuitKIT, a source-available library that connects the circuit-analysis workflow through a typed, serializable representation. CircuitKIT provides a suite of discovery algorithms, declarative interfaces for mapping structured data into discovery tasks, complementary circuit diagnostics, and downstream application modules. Together, these components provide common infrastructure for conducting and comparing circuit analyses. The library, examples, notebooks, and documentation are released at https://github.com/Lexsi-Labs/CircuitKIT .

Problem

• circuit analysis
• mechanistic interpretability
• fragmentation
• contrastive prompts
• downstream interventions

Innovation

• Circuit Analysis
• Mechanistic Interpretability
• Declarative Interface
• Serializable Representation
• Downstream Intervention