A Systematic Taxonomy of Large Language Model Inference Optimization: Bridging Model-Level and API-Layer Latency Reduction Strategies

Engineering notes

Key topics: autonomous driving, large language model, deployment. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

The deployment of Large Language Models (LLMs) as autonomous agents in next generation infrastructures, such as Agentic AI-Native 6G networks, introduces severe latency, memory, and operational cost bottlenecks.Existing optimization approaches frequently treat model compression and system orchestration in isolation, failing to adequately address the multidimensional constraints of modern production environments.This paper presents a systematic, multi criteria taxonomy of LLM inference optimization, bridging granular model level compression techniques with macro level API-layer orchestration strategies.We conduct a rigorous comparative analysis of post training quantization algorithms specifically highlighting activationaware methods like AWQ alongside GPTQ and GGUF evaluating their trade offs across memory efficiency, inference throughput, and predictive quality.Our findings reveal that deployment efficiency is not solely dictated by numerical precision but is fundamentally a co-design problem requiring optimized hardware kernels and adaptive, importance aware parameter protection.Furthermore, we demonstrate that systemlevel interventions, such as intelligent model routing, prompt caching, and deterministic decoupling, yield immediate latency reductions without altering underlying model weights.Ultimately, this study provides a comprehensive, scenario based decision framework for matching specific quantization algorithms and APIlayer strategies to distinct deployment environments, ranging from high throughput GPU serving to resourceconstrained edge devices and safety critical autonomous systems.By unifying these perspectives, we establish a foundational guide for practitioners aiming to build reliable, high performance inference pipelines that ensure scalable and economically sustainable AI integration across diverse industrial domains.

Links and sources

• PDF from original source

Comments