[Submitted on 6 Mar 2026 (v1), last revised 23 Jul 2026 (this version, v3)]

View PDF HTML (experimental)

Abstract:Quantizing open-vocabulary object detection (OVOD) models reduces their memory and computational costs, but extremely low-bit quantization severely degrades both cross-modal (region-text) and intra-modal (region-region) alignments. This multimodal degradation is a unique challenge that prior quantization methods for closed-vocabulary detectors fail to resolve. To overcome this, we propose Quantization-Aware Training with Multimodal Alignment (QATMA), the first multimodal-aware and architecture-agnostic QAT framework tailored for OVOD. QATMA integrates two key components: (i) Curriculum QAT, which partitions the detector by functional roles and progressively expands the quantization scope to suppress error accumulation and ensure stable optimization; and (ii) Text-anchored Pairwise Similarity Distillation, which transfers both region-text and region-region alignments from a full-precision teacher model via pairwise cosine similarities in the joint embedding space. Experimental results on LVIS and COCO zero-shot benchmarks demonstrate that QATMA significantly outperforms existing QAT baselines under extremely low-bit settings, achieving gains of up to 4.3 and 7.6 AP, respectively.

Submission history

From: Jibum Kim [view email]
[v1] Fri, 6 Mar 2026 06:57:55 UTC (2,515 KB)
[v2] Mon, 9 Mar 2026 07:42:44 UTC (2,515 KB)
[v3] Thu, 23 Jul 2026 06:43:29 UTC (12,042 KB)