ArXiv

SVD-Surgeon: Optimal Singular-Value Surgery for Large Language Model Compression

2026-06-22 · 16:33 UTC ·Mahmoud Safari, Frank Hutter ·1 min read
Authors
Mahmoud Safari, Frank Hutter
Categories
cs.LG, cs.CL
arXiv
https://arxiv.org/abs/2606.23568v1
PDF
https://arxiv.org/pdf/2606.23568v1

Brief

SVD-Surgeon addresses LLM deployment costs by bringing OBS-style, second-order pruning to singular values: it treats singular values as parameters, computes a closed-form correction for retained values after truncation, and provides a saliency metric for pruning. The training-free method plugs into existing SVD compressors (tested with SVD-LLM) and reportedly improves the perplexity–compression trade-off on OPT models and LLaMA 2-7B without retraining.

Why it matters

SVD-Surgeon adapts the Optimal Brain Surgeon (OBS) framework to the singular-value basis, treating each singular value as a parameter and deriving a closed-form, second-order update that compensates for singular values removed by truncation.

Key details

  • The method is training-free, operates directly on SVD factorizations (so it can be layered on top of existing SVD compressors such as SVD-LLM), and—per the authors—improves the perplexity–compression trade-off on the OPT family and LLaMA 2-7B without any retraining.
  • Paper by Mahmoud Safari and Frank Hutter, arXiv:2606.23568v1 (submitted 2026-06-22); 8 pages with 3 figures and 5 tables (PDF available).
Source evidence

Abstract

Large language models (LLMs) achieve remarkable performance across a wide range of tasks, but their deployment is constrained by substantial memory and compute requirements. Low-rank compression via singular value decomposition (SVD) is an effective remedy, but existing methods focus on how to factorize and which components to keep. We introduce SVD-Surgeon, a training-free method that brings the Optimal Brain Surgeon (OBS) framework to the singular-value basis. Treating each singular value as a parameter, it computes a closed-form update of the retained singular values that compensates, to second order in the model loss, for those removed by truncation. The same analysis yields a saliency for choosing which values to prune. As it operates directly on the singular-value factorization, SVD-Surgeon can be layered on top of existing SVD compressors. Applied to SVD-LLM, a leading SVD-based method, it improves the perplexity-compression trade-off on the OPT family and LLaMA 2-7B without any retraining.

Comment: 8 pages, 3 figures, 5 tables; appendix