client-side llm summarization

Resource-constrained edge devices face challenges in accurately understanding user intent from UI interaction traces, while simultaneously ensuring privacy preservation and real-time responsiveness. Method: This paper proposes a two-stage decomposed architecture: (1) generating structured sequential summaries of interaction behaviors, followed by (2) lightweight intent inference based on these summaries. The approach integrates context-aggregated enhancement and task-adaptive fine-tuning to strengthen semantic modeling capabilities of small models. Contribution/Results: Experimental results demonstrate that, under identical privacy guarantees and low-latency constraints, the proposed method achieves higher intent recognition accuracy than state-of-the-art large multimodal language models. It establishes an efficient, privacy-aware, and real-time interaction understanding paradigm for on-device intelligent agents.

Cloud-hosted proprietary large language models (LLMs) pose significant challenges in user intent parsing—including privacy leakage, lack of user autonomy, and limited scalability. Method: This paper systematically evaluates the feasibility of open-weight LLMs as core components of a localized, intent-driven operating system. We design a lightweight intent parsing framework leveraging multiple open-source models (e.g., Llama-3, Qwen, Phi-3), integrating natural language understanding, workflow generation, and on-device inference. Empirical evaluation is conducted across multi-application collaborative tasks, with GPT-4 as a performance benchmark. Results: Several open-weight models achieve intent recognition accuracy comparable to closed-source counterparts under local deployment—while ensuring data privacy, enabling offline operation, and preserving user control. To our knowledge, this is the first study to empirically validate open LLMs as enablers of decentralized, language-first interaction paradigms. It establishes an open architectural foundation and empirical evidence for trustworthy, scalable intent-oriented operating systems.

This study systematically evaluates the performance of 19 small language models (SLMs) on news summarization under resource-constrained settings, benchmarking against 70B-parameter large language models (LLMs) across summary quality, coherence, factual consistency, and length efficiency. Method: Using a 2,000-sample news corpus, we employ a multi-dimensional evaluation framework combining human assessment with automated metrics—ROUGE, BERTScore, and fact-checking—and conduct controlled prompt experiments, instruction-tuning ablations, and computational complexity analysis. Contribution/Results: We empirically demonstrate that top-performing SLMs (e.g., Phi3-Mini) match 70B LLMs in summary quality while reducing output length by 35% on average. Simpler prompts outperform complex instructions, and instruction tuning yields no significant gains for news summarization. This work establishes, for the first time, the practical efficacy boundary and efficient deployment pathway for SLMs in news summarization.

Deploying large language models (LLMs) on edge devices faces fundamental challenges including constrained computational resources, limited memory capacity, and hardware heterogeneity. To address these, this paper systematically surveys the full lifecycle of edge LLMs and introduces a comprehensive, stack-wide technical taxonomy—spanning lightweight model design (e.g., pruning, quantization, distillation), runtime optimization (e.g., memory-aware inference, device-adaptive scheduling), and on-device deployment (e.g., cloud-edge collaborative frameworks). It proposes a novel cross-platform co-deployment paradigm that unifies pre-deployment model compression with dynamic execution optimization. Based on a rigorous synthesis of over 120 state-of-the-art studies, the work identifies five persistent technical bottlenecks and six key future research directions. The resulting methodology provides both a reusable conceptual framework and practical guidelines for deploying AI at the edge.

This work investigates whether pretrained language models (PLMs) implicitly encode user preferences and collaborative signals in their representation spaces. To this end, we propose TextCF—a purely text-driven collaborative filtering framework that eliminates the need for item ID embeddings. TextCF employs lightweight linear projections to map item title representations extracted from PLMs (e.g., LLaMA, BERT) into a recommendation-aware latent space. Theoretically, we establish a homomorphic structure between linguistic and collaborative representation spaces. Empirically, TextCF significantly outperforms state-of-the-art ID-based CF methods across multiple public benchmarks—demonstrating, for the first time, that title text alone suffices to achieve superior recommendation performance. Moreover, TextCF exhibits zero-shot recommendation capability and inherent potential for user intent awareness. Collectively, it introduces a new paradigm for recommender systems that is initialization-friendly, generalizable, and inherently interpretable.

Existing collaborative systems predominantly focus on individual tasks, limiting their capacity to support authentic joint activities and shared experiences. To address this, we propose “semantic interaction”—a novel paradigm that integrates natural language processing (NLP) techniques to enable group co-discussion within co-located spaces. We instantiate this paradigm through CollEagle, an interactive tabletop system that supports low-effort information creation, collective externalization, dynamic reorganization, and structured management of shared artifacts. Crucially, CollEagle embeds deep semantic understanding directly into the collaborative interface—marking a departure from traditional task-centric design approaches. This integration significantly enhances interaction quality and shared cognition among participants. Preliminary empirical evaluation demonstrates that semantic interaction effectively modulates collaborative behaviors, offering both theoretical grounding and practical design guidelines for collaboration interfaces targeting joint activity.

This work addresses the challenges of deploying automatic summarization for multi-party dialogues in industrial settings, where dynamic shifts in user requirements, highly subjective evaluation criteria, and scarce annotated data render conventional static dataset approaches inadequate. To bridge this gap, the authors propose a practical, lifecycle-oriented framework for adaptive summarization systems, leveraging an agent-based architecture to decompose tasks, integrating large language model prompt engineering with component-level optimization, and introducing a robust evaluation mechanism tailored to handle requirement volatility and subjectivity. Moving beyond static research paradigms, the project delivers a reusable industrial development guideline and uncovers critical practical insights—such as the impact of data quality and risks of vendor lock-in—thereby significantly enhancing system reliability and adaptability in real-world applications.

This work addresses the high computational overhead and scalability challenges inherent in training and inference of large language models (LLMs) by proposing the first cloud-native distributed systems framework specifically designed for LLMs. The framework integrates key technologies including microservices, auto-scaling, cloud-edge collaboration, serverless inference, and federated learning, while also exploring a forward-looking pathway toward quantum computing integration. By systematically optimizing data management and resource scheduling, the proposed architecture significantly enhances deployment efficiency and elasticity. It provides robust technical foundations for the efficient operation and continuous evolution of LLMs, thereby fostering synergies across industry, academia, and research communities and advancing standardization efforts in the field.

To address the prohibitively high computational cost and inability of large language models (LLMs) to meet stringent low-latency and high-throughput requirements in LinkedIn’s semantic job search, this paper proposes a co-optimization framework for small language models (SLMs) with pure-text decoders. Our method jointly optimizes structured pruning and semantic-aware context compression: structured pruning reduces model parameters by 40%, while context-aware input sequence compression achieves an average 10× reduction in context length. We further integrate GPU kernel optimization and a lightweight serving architecture. Evaluated in LinkedIn’s production environment, the system achieves a 10× throughput improvement—reaching one million queries per second—with P99 latency consistently below 50 ms, while preserving retrieval quality (Recall@10 degradation < 0.3%). This work establishes an efficient, scalable deployment paradigm for SLMs in large-scale semantic search.

This study investigates whether attention heads and input embeddings in large language models (LLMs) genuinely encode human-interpretable semantic information. Addressing concerns that prevailing interpretability methods—such as those based on attention weights or embedding analyses—may be confounded by data artifacts or methodological biases, the authors employ token-level relational structural probes and map human-interpretable attributes onto the embedding space to systematically evaluate the validity of these approaches across multiple Transformer layers. Their findings reveal that both widely adopted explanation techniques fail to reliably reflect the model’s true semantic capabilities, thereby challenging the foundational assumptions underlying current claims about LLMs’ “understanding.” This work carries significant implications for deploying LLMs in edge and distributed computing environments, where interpretability and reliability are critical.