ablation studies

Systematically removing or varying components of a system to attribute effects and measure component-level contributions; used to decide prompt elements, adapter placement, or which model submodule drives observed improvements.

ablationstudies

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

Hardness, Structural Knowledge, and Opportunity: An Analytical Framework for Modular Performance Modeling

Sep 13, 2025
OG
Omid Gheibi
🏛️ Carnegie Mellon University | University of South Carolina

Configuration space explosion complicates performance impact modeling, while gray-box approaches rely on structural knowledge (e.g., module execution graphs) to improve model accuracy—yet the mechanisms by which structural features (e.g., number of modules or configuration options) and structural knowledge influence modeling difficulty and optimization potential remain unclear. Method: We formally define “modeling hardness” and “improvement opportunity,” establishing an analytical framework and matrix to quantify the interplay among system structural complexity, structural knowledge level, and modeling benefit. Controlled experiments on synthetic systems integrate module execution graph analysis with gray-box modeling. Contribution/Results: We identify module count and configuration option count as dominant determinants of modeling hardness. Under high hardness, strong structural knowledge significantly increases improvement opportunity. Structural knowledge primarily enhances ranking accuracy, whereas hardness predominantly degrades prediction accuracy. Our findings provide theoretical foundations and strategic guidance for allocating structural knowledge investment according to specific modeling objectives.

Analyzing the varying impact of knowledge and hardness on different performance metricsInvestigating how structural aspects and knowledge affect modular performance modeling opportunitiesQuantifying modeling hardness driven by module and option counts

How fair are we? From conceptualization to automated assessment of fairness definitions

Apr 15, 2024
GD
Giordano d'Aloisio
🏛️ University of L'Aquila

Existing automated fairness-checking approaches for software rely on predefined rules and lack support for user-defined fairness criteria—particularly challenging in emerging domains such as software engineering (SE) recommendation systems and Arduino hardware component recommendation, where bias mitigation demands domain-specific adaptability. Method: We propose MODNESS, a model-driven framework built upon Model-Driven Engineering (MDE) and a Domain-Specific Modeling Language (DSML), enabling users to flexibly specify fairness concepts, compose multi-dimensional fairness metrics, and automatically generate executable assessment code within a dedicated modeling environment. Contribution/Results: MODNESS is the first framework to enable user-driven fairness modeling and code generation; it pioneers fairness evaluation in SE and Arduino component recommendation—two previously unaddressed application domains; and it overcomes expressiveness and domain-adaptation limitations of prior tools. Empirical evaluation demonstrates end-to-end fairness assessment capability, significantly improving flexibility and cross-domain applicability.

Bias in RecommendationsCustomizable StandardsSoftware Fairness

This study addresses the challenge of quantifying the complexity and cost induced by external requirement changes when detailed knowledge of a system’s internal logic is unavailable. To this end, the authors propose a black-box assessment method based on a directed graph of component coupling. By analyzing component interfaces and integrating multi-view modeling—graphical, algebraic, and tabular—the approach uniquely links interface characteristics to cost factors, enabling computable bounded estimates of change-induced complexity and associated costs. The method was validated through a large-scale integration case in a retail banking platform, demonstrating its effectiveness and providing architects and operations teams with actionable, quantitative insights for system design and maintenance.

component interfacecost estimationexternal requirements

This work addresses the lack of quantitative feedback on structural elements that impede robotic disassembly in current product design, which hinders disassembly optimization. The authors propose a CAD-based method that constructs a contact–connection–constraint graph to analyze robotic disassembly sequences and quantify the influence of individual components. For the first time, this influence is mapped onto the geometric model to generate a 3D heatmap, enabling automatic identification and recommendation of key fasteners that can be eliminated without compromising structural integrity. Experiments on seven household appliances demonstrate that the approach successfully identifies redundant fasteners, removes 8–132 structural constraints, reduces tool changes, and shortens robotic travel distance by 165–1675 mm within allowable structural limits.

design for disassemblyfastener reductionremanufacturing

Quality in model-driven engineering: a tertiary study

Jun 23, 2016
MG
M. Goulão
🏛️ Universidade Nova de Lisboa | University of Maribor

Empirical evidence on the impact of Model-Driven Engineering (MDE) on software quality is fragmented and lacks systematic integration. Method: This paper conducts the first tertiary study dedicated to MDE quality research, systematically analyzing 22 published systematic literature reviews and mapping studies. It establishes a three-tier analytical framework to characterize research distribution, evidential strength, and methodological maturity in the MDE–quality domain. Results: Maintainability is the most studied quality attribute; however, among 83 identified research questions, 80 focus solely on conceptual or syntactic model-to-code mappings, with few conducting empirical comparisons. Crucially, MDE’s actual impact on quality in industrial development contexts remains markedly under-investigated. The study exposes a structural bias toward “re-modeling over validation” in current research and identifies critical gaps requiring urgent attention: rigorous experimental design, industry-based empirical validation, and multi-attribute quality assessment frameworks.

Aggregating consolidated findings on quality impact in model-driven engineeringAnalyzing software quality attributes most affected by MDE approachesIdentifying under-explored research areas needing further empirical validation

Latest Papers

What's happening recently
View more

This work addresses the inefficiency and lack of guidance faced by front-end developers when manually selecting plausible and natural attribute values for instantiating reusable UI components within a vast design space. To tackle this challenge, the paper introduces the concept of “discriminative variants,” which uniquely integrates symbolic reasoning with large language models (LLMs). Symbolic reasoning identifies visually salient attributes, while the LLM leverages real-world knowledge to generate component instances that balance fidelity to exemplars with meaningful differentiation. This approach shifts the paradigm from ad hoc manual configuration to structured exploration of the design space. A user study (n=12) demonstrates that the generated variants effectively aid developers in comprehending the design space, significantly improving both instantiation efficiency and user experience, while maintaining strong domain relevance.

component instantiationdesign spacefront-end development

This study addresses the lack of systematic, large-scale analyses of structural properties in software feature models, which has hindered the understanding and evolution of variability models. For the first time, it systematically applies large-scale network analysis to 5,709 variability models drawn from 20 repositories. By constructing graphs capturing transitive dependencies and conflicts among features, and integrating graph modeling with network-theoretic and statistical analyses, the work uncovers cross-domain structural commonalities—such as dependency dominance, high centralization, and characteristic degree distributions—as well as domain-specific deviations. These findings provide novel empirical insights and a foundation for identifying pivotal features, guiding modular decomposition, and assessing structural fragility in variability-intensive systems.

feature modelsnetwork analysissoftware configuration

This study addresses the challenges of co-evolution in two-layer modeling (2LM), where fragmented knowledge between metamodels and models hinders consistent evolution. To tackle this, the authors propose the first reproducible empirical framework that applies identical evolutionary changes—via preregistered mutation experiments—to semantically equivalent multi-level modeling (MLM) and 2LM scenarios, automatically detecting inconsistencies and quantifying maintenance effort. By integrating automated consistency checking, a blind mapping protocol, and hypothesis testing, the approach operationalizes co-evolution cost into two measurable variables. Results demonstrate that MLM significantly reduces both inconsistency occurrences and the scope of required modifications due to its structural unification, thereby confirming its advantage in mitigating cascading maintenance costs and establishing a benchmark protocol for evaluating the impact of modeling paradigms.

cascading maintenance burdenco-evolutionModel-Driven Engineering

Current Software Bill of Materials (SBOM) tools struggle to comprehensively identify security vulnerabilities due to the absence of a unified standard for component identification, thereby jeopardizing software supply chain security. This work introduces the Component Introduction Mechanism (CIM) analysis framework—the first of its kind—to systematically evaluate the component detection capabilities of cdxgen, syft, trivy, ORT, and Microsoft’s sbom-tool across real-world projects in six programming languages: Python, Java, Go, PHP, Rust, and C. The study reveals that existing tools commonly suffer from incomplete CIM coverage, ambiguous component definitions, and shared blind spots, leading to significant ambiguities and omissions in generated SBOMs. These findings underscore the urgent need for community-wide consensus on component identification and provide an empirical foundation and strategic guidance for developing more reliable SBOM technologies.

component identificationcomponent inclusionSBOM ambiguity

Existing interpretability methods struggle to distinguish whether model components genuinely encode a target capability or merely propagate upstream signals. This work proposes Weight Patching, a source-directed intervention in weight space that operates on isomorphic models exhibiting varying behavioral strengths. By substituting specific module weights and anchoring behavioral interfaces via vector alignment, the method precisely localizes source-level mechanisms within large language models. The framework enables, for the first time, tracing the pathway of capability transmission from shallow source carriers to downstream execution circuits, thereby supporting mechanism-aware model merging. Experiments on instruction-following tasks successfully identify critical mechanistic components, significantly improving selective fusion of expert models, with findings further validated externally.

behavioral capabilityLLMsmechanistic interpretability

Hot Scholars

NN

Nassir Navab

Professor of Computer Science, Technische Universität München
AK

Axel Krieger

Associate Professor, Johns Hopkins University
Medical DevicesRobotics
GP

Gernot Plank

Medical University of Graz
computational cardiologycardiac modellingcardiac mechanics
YC

Yash Chitalia

University of Louisville
RoboticsControl TheoryMachine Learning
WA

Wolfgang A. Wall

Professor of Computational Mechanics, Technical University of Munich (TUM)
Computational MechanicsComputational Methods in Applied Science and Engineering