<b>Hybrid Architecture Edge Object Detector Based on Structured Pruning and Attention Reparameterization</b>

Engineering notes

Experiments show that the proposed method achieves 43.0% mAP on COCO dataset, 112.3 FPS on Jetson Xavier, and the model size is only 5.8 MB. In terms of accuracy, speed and model size, it is superior to mainstream methods such as YOLOv8n, MobileDet and EfficientDet-D0, and also shows optimal robustness in edge scenes such as low light and motion blur.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Edge computing scenarios put forward strict requirements on inference delay and computational power consumption of target detectors, and the existing lightweight models are difficult to achieve the best balance between accuracy and speed. In this paper, a hybrid architecture edge object detector based on structured pruning and attention reparameterization is proposed, which is co-optimized from three levels: architecture design, model compression and inference acceleration. Firstly, a CNN-Transformer dual-branch heterogeneous backbone network is constructed, and the basic mAP is increased to 44.2% on the premise of only increasing the number of parameters by 15% by fusing local details and global context features through cross-modal interaction modules. Then, a structured pruning algorithm driven by hierarchical sensitivity is proposed, which combines Hessian trace and batch normalization scaling factor to dynamically determine the redundancy of each layer. The differentiable mask and Group Lasso sparse regularity are introduced to realize training, namely pruning. With the layer-by-layer feature reconstruction compensation mechanism, the accuracy retention rate can reach 97.5% when FLOPs are reduced by 50.4%. The attention reparameterization technique during inference is further proposed, which uses the linear additivity of affine transformations to losslessly fuse the multi-branch attention structure in the training stage into a single convolutional layer, and combines piecewise linear fitting to deal with Softmax nonlinearity, completely eliminating the attention computation cost during inference. Experiments show that the proposed method achieves 43.0% mAP on COCO dataset, 112.3 FPS on Jetson Xavier, and the model size is only 5.8 MB. In terms of accuracy, speed and model size, it is superior to mainstream methods such as YOLOv8n, MobileDet and EfficientDet-D0, and also shows optimal robustness in edge scenes such as low light and motion blur.

Links and sources

• PDF from original source

Comments