ArXiv

MixFrag: Fragility-Guided Mixed-Precision Post-Training Quantization for Vision Transformers

2026-07-30 · 17:43 UTC ·Md. Mehrab Hossain Opi, Robiul Islam Ryad, Md. Umar Faruk ·1 min read
Authors
Md. Mehrab Hossain Opi, Robiul Islam Ryad, Md. Umar Faruk
Categories
cs.CV, cs.LG
arXiv
https://arxiv.org/abs/2607.28589v1
PDF
https://arxiv.org/pdf/2607.28589v1

Brief

MixFrag addresses heterogeneous quantization sensitivity in Vision Transformers by computing per-component fragility with KL divergence between full-precision and isolated quantized outputs on a small calibration set, then solving bit allocation as a Multiple-Choice Knapsack Problem. Experiments (ImageNet-1K, COCO) show competitive classification and up to a 9.6 AP gain on COCO under MP3/MP3; analyses validate the fragility metric. Full text not available; summary based on the abstract.

Why it matters

MixFrag measures component-level quantization fragility via the Kullback–Leibler (KL) divergence between full-precision and isolated quantized output distributions using a small calibration set, then formulates layer-wise bit assignment as a Multiple-Choice Knapsack Problem (MCKP) to meet a target bit budget.

Key details

  • On ImageNet-1K across multiple Vision Transformer architectures MixFrag yields competitive classification under practical mixed-precision settings, and on COCO object detection/instance segmentation it improves the previous best mixed-precision PTQ method by up to 9.6 AP in the challenging MP3/MP3 setting (arXiv 2026-07-30).
  • Authors Md. Mehrab Hossain Opi, Robiul Islam Ryad, and Md. Umar Faruk report additional analyses showing the proposed fragility metric strongly correlates with the learned bit allocations (arXiv:2607.28589v1, 2026-07-30).
Source evidence

Abstract

Post-training quantization (PTQ) has emerged as an effective solution for deploying Vision Transformers (ViTs) on resource-constrained devices. However, existing PTQ methods typically employ uniform bit-widths across transformer components, overlooking their heterogeneous sensitivity to quantization and leading to inefficient precision allocation. In this paper, we propose {MixFrag, a fragility-guided mixed-precision PTQ framework for Vision Transformers. MixFrag first estimates component-level quantization fragility by measuring the Kullback--Leibler (KL) divergence between full-precision and isolated quantized output distributions using a small calibration set. It then formulates bit allocation as a Multiple-Choice Knapsack Problem (MCKP), enabling adaptive layer-wise precision assignment under a target bit budget. Extensive experiments on ImageNet-1K across multiple Vision Transformer architectures demonstrate that MixFrag achieves competitive classification performance under practical mixed-precision settings. Furthermore, evaluations on COCO object detection and instance segmentation show that MixFrag achieves state-of-the-art performance among existing mixed-precision PTQ methods, improving the previous best method by up to 9.6 AP under the challenging MP3/MP3 setting. Additional analyses validate the proposed fragility metric and demonstrate its strong correlation with the learned bit allocation. These results establish MixFrag as an effective framework for mixed-precision post-training quantization of Vision Transformers.