RT-YOLACT: Real-Time Instance Segmentation on Zynq SoC for Autonomous Driving

Engineering notes

Key topics: autonomous driving system, autonomous driving, end-to-end, lane detection, instance segmentation, deployment. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Edge autonomous driving systems are need to process visual information in real time under strict limits on computation, memory, and power. Instance segmentation, especially for lane detection, remains challenging to deploy on embedded platforms due to the complexity of modern deep learning models. We present RT-YOLACT (Real-Time YOLACT), an optimized instance segmentation framework tailored for FPGA-SoC deployment. Our approach integrates input resolution reduction, a lightweight MobileNetV4 backbone, INT8 quantization, and streamlined post-processing. The pipeline executes on Ultra96-V2 with DPU B1600, achieving 16.64 FPS end-to-end—a 13.4× speedup over the baseline with only 2.35% accuracy loss. Model size and parameters are reduced by 87%, and post-processing latency drops from 200.88 ms to 11.31 ms. These results demonstrate that real-time instance segmentation can be realized on ultra-low-power FPGA-SoC platforms.

Links and sources

• Official / arXiv page

Comments