EQISP: efficient quantized image signal processing with multi-scale pyramid fusion for resource constrained embodied perception

Engineering notes

QCNN employs dynamic fixed-point hybrid quantization, which adjusts parameter ranges according to the linear relationship between threshold standard deviation and fractional length, thereby significantly reducing the computational load.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Introduction Resource-constrained environmental perception requires autonomous robots and embodied intelligent systems to process visual signals efficiently while preserving image fidelity in complex real-world environments. However, converting high dynamic range RAW sensor data into perceptually faithful RGB images remains computationally expensive, thereby limiting the deployment of neural image signal processors on edge platforms with restricted memory, energy, and computational budgets. Methods Consequently, this study proposes the enhanced quantized image signal processor (EQISP), comprising the quantized convolutional neural network (QCNN) and the unified pyramid fusion algorithm (UPFA). QCNN employs dynamic fixed-point hybrid quantization, which adjusts parameter ranges according to the linear relationship between threshold standard deviation and fractional length, thereby significantly reducing the computational load. Meanwhile, UPFA utilizes Gaussian pyramids to capture global illumination and Laplacian pyramids to preserve fine details, enabling multi-scale, multi-exposure fusion and iterative reconstruction to mitigate detail loss induced by quantization. Results Comprehensive comparative experiments demonstrated that EQISP achieved a PSNR of 22.90 dB, an SSIM of 0.9278, and 164.843 GFLOPs. Compared with the PyNET baseline, EQISP improved the PSNR by 1.71 dB while reducing the computational cost by a factor of 4.24. Furthermore, deployment experiments on an NVIDIA Jetson TX2 development board showed that EQISP achieved a model size of 57 MB, an inference latency of 189 ms, an inference speed of 6.1 FPS, and a peak memory usage of 2.2 GB. Discussion These results provide practical evidence that EQISP can serve as an efficient and scalable visual front end for resource-constrained embodied perception systems.

Links and sources

• PDF from original source

Comments