Mobile real-time detection transformer for computationally constrained devices

Engineering notes

The recent state-of-the-art model, Real Time Detection Transformer (RT-DETR), has surpassed YOLOv8 in terms of both accuracy and speed. MRTDETR achieves 44.6% mAP with a train batch size of 4 on NVIDIA GeForce RTX 3050 OEM GPU and 43.1% mAP with a train batch size of 64 on NVIDIA H100 PCIe GPUs, hence surpassing existing implementations of FasterRCNN, MobileFormer, and MobileViT.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

The recent state-of-the-art model, Real Time Detection Transformer (RT-DETR), has surpassed YOLOv8 in terms of both accuracy and speed. We present a model based on the RT-DETR architecture, with a lightweight backbone, to achieve compactness and high speed. Our model, Mobile RT-DETR (MRTDETR), utilises a series of MobileNetv3 blocks. Specifically, the layers 7, 13, and 15 of the MobileNetv3 backbone were used as inputs to the encoder. Separable self attention was also explored, but not retained. MRTDETR achieves 44.6% mAP with a train batch size of 4 on NVIDIA GeForce RTX 3050 OEM GPU and 43.1% mAP with a train batch size of 64 on NVIDIA H100 PCIe GPUs, hence surpassing existing implementations of FasterRCNN, MobileFormer, and MobileViT. With 15.23 GFLOPs, MRTDETR comes close to the performance of the state-of-the-art YOLOv8 and YOLOv9 detectors of similar size while having $$46.7\%$$ and $$42.3\%$$ less FLOPs correspondingly. The model has been tested with Raspberry Pi 5, alongside other models (including quantised models), to profile edge related metrics. It has an average inference time of 1,967.4 milliseconds with a standard deviation of 71.9 milliseconds, takes 435.86 MB of VmRSS memory and 1,350.27 MB of VmSize memory, and has an ONNX file size of 43.1 MB. The MRTDETR model outperforms all other models in optimality, with respect to the trade-off between accuracy and FLOPs. By supplementing model complexity metrics and performance metrics, with real-world profiling results, a more relevant evaluation of the proposed MRTDETR model is provided.

Links and sources

• PDF from original source

Comments