A Theoretical Analysis of Migration Strength Selection in SmoothQuant

Post-training quantization (PTQ) is essential for efficient inference of large language models, yet remains challenging due to activation outliers with large magnitudes. SmoothQuant mitigates this issue by migrating quantization difficulty from activations to weights using a global migration strength α. However, the choice of α in prior work is empirically motivated and lacks theoretical grounding, while a global α ignores layer heterogeneity. In this paper, we provide a theoretical analysis of SmoothQuant by formulating it as a quantization-noise balancing problem. Under standard quantization noise assumptions, we derive an explicit objective that characterizes the trade-off between activation and weight quantization errors, revealing that the error-minimizing migration strength depends on layer-specific activation and weight statistics. Motivated by this analysis, we propose a layer-wise smoothing strategy in which each layer adopts its own theory-guided α. With minimal modifications to the original method and no additional runtime overhead, the proposed approach consistently reduces layer output error and improves accuracy on LLaMA models under W8A8 quantization, demonstrating the practical effectiveness of the theoretical insights. Our implementation is publicly available.

2026 International Joint Conference on Neural Networks (IJCNN), 1-8. IEEE