Profiling Lightweight Large Language Models

2026-07-22

AI Summary

This work addresses the limitations of existing proxy metrics—such as parameter count and FLOPs—in evaluating lightweight large language models, which often fail to capture the true trade-offs between accuracy and resource consumption in real-world deployments. To overcome this, we propose PTME, a novel framework that enables precision-aware, multidimensional resource profiling through direct hardware measurements on local edge devices. PTME jointly evaluates model performance across code generation, mathematical reasoning, and multitask comprehension in terms of accuracy, execution time, peak memory usage, and energy consumption. Our experiments reveal that static proxy metrics poorly predict actual accuracy, that tightening resource constraints primarily increases latency rather than energy use, and that no single model dominates across all dimensions. By identifying Pareto-optimal solutions, PTME provides a data-driven foundation for deployment optimization.

Abstract

Lightweight large language models (LLMs) are increasingly being deployed locally on personal computers and are expected to play a growing role in resource-constrained edge and mobile environments. In such settings, energy consumption, execution time, and memory usage directly affect practical usability, yet existing evaluations of LLM efficiency largely rely on proxy descriptors such as parameter count or FLOPs, often decoupled from task precision. This paper introduces a PTME-based experimental framework for the precision-aware profiling of lightweight LLM inference, jointly measuring Precision, execution Time, peak Memory usage, and Energy consumption through direct hardware-level measurements. The methodology is applied to a representative set of lightweight LLMs executed locally under edge-class resource envelopes on a controlled desktop platform, using benchmarks spanning code generation, mathematical reasoning, and multi-task understanding. We find that static proxy descriptors approximate inference cost well but fail to predict precision. Tightening the resource envelope increases cost without affecting precision, amplifying execution time more strongly than energy and penalizing larger models the most. Moreover, no single model dominates across all PTME dimensions, and a Pareto analysis reveals non-dominated configurations that would be hidden by accuracy-only or efficiency-only assessments, providing practical guidance for selecting models under different resource envelopes. These results show that selecting lightweight LLMs by size, FLOPs, latency, or accuracy alone can select the wrong deployment candidate; PTME profiling exposes configurations that preserve useful accuracy at lower physical cost.

Problem

• lightweight LLMs
• efficiency evaluation
• precision-aware profiling
• resource-constrained deployment
• proxy metrics

Innovation

• PTME profiling
• lightweight LLMs
• precision-aware evaluation
• hardware-level measurement
• Pareto analysis