[Submitted on 25 May 2026 (v1), last revised 27 Jul 2026 (this version, v5)]

View PDF HTML (experimental)

Abstract:The deployment of Large Language Models (LLMs) and Vision Transformers (ViTs) on edge devices is significantly constrained by memory capacity and the critical timing bottlenecks introduced by dense Multiply--Accumulate (MAC) arrays. In the ultra-low-bit regime, logarithmic Power-of-Two (PoT) quantization provides a hardware-efficient alternative by replacing general multiplications in the dominant dot-product computation with bit-shift operations. However, its non-uniform exponential lattice inherently suffers from a \textbf{Low Angular Resolution Regime}, a structural limitation that becomes particularly pronounced below 4-bit precision and can substantially degrade the representation of high-dimensional feature manifolds.
To address this geometric limitation, we propose Geometric Orthogonal Residual Projection Quantization (GoQuant), an algorithm--hardware co-design framework for multiplier-reduced low-bit inference. By formulating quantization as a dual-basis geometric projection, GoQuant constructs a higher-resolution residual lattice while retaining a shift-and-add inner-product structure. Its analytical solver further avoids computationally intensive gradient-based or iterative search procedures. The data-free Geometric-Only (GEO) mode quantizes LLaMA-2-7B in only 0.47 minutes, while the Activation-Refined (REF) mode completes full-model quantization in approximately \textbf{4.4 minutes}.

Submission history

From: Maoyang Xiang [view email]
[v1] Mon, 25 May 2026 17:52:46 UTC (416 KB)
[v2] Tue, 26 May 2026 18:01:20 UTC (416 KB)
[v3] Thu, 28 May 2026 05:26:15 UTC (416 KB)
[v4] Sun, 31 May 2026 09:04:58 UTC (416 KB)
[v5] Mon, 27 Jul 2026 23:05:22 UTC (394 KB)