Hierarchical token-based communication for robust semantic scene perception under network instability

Engineering notes

Experimental results show that UDSTC achieves higher semantic completion accuracy on masked regions, outperforms the SePaint and StreetUnveiler baselines under the same budget, and exhibits smaller performance degradation under severe latency and high packet loss, demonstrating strong adaptability to the instability of self-organizing networks.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

In self-organizing vehicular networks, vehicles establish transient connections through local communications, while the topology, bandwidth, and latency fluctuate with driving conditions and messages may arrive late or out of order. Under such conditions, the key challenge for high-quality semantic scene completion is that completion requires spatial evidence over a broader range, whereas communication resources are limited and information exhibits heterogeneous timeliness. We propose UDSTC, a utility-driven semantic-token communication method for collaborative semantic occlusion completion. On each vehicle, UDSTC produces local BEV semantic predictions from on-board observations and constructs spatial-block tokens, which are compressed and transmitted using hierarchical codebook-indexed integer messages to accommodate different budget levels. A utility-estimation-driven token selection strategy and multi-hop budget scheduling are introduced to prioritize content that is most critical for the missing regions while respecting token lifetime constraints. At the receiver side, UDSTC builds an asynchronous scene memory that absorbs information from multiple vehicles and heterogeneous arrival times via time compensation and time-decayed writing. For decoding, UDSTC employs mask-conditioned generative completion, jointly leveraging ego features and external context to recover missing regions, and unifies completion loss, utility learning, and budget constraints into a single training objective.We construct an evaluation protocol on the Waymo Open Dataset by synthesizing continuous black occlusions and corresponding BEV missing regions, and validate the method through main-result comparisons, budget–performance trade-off studies, multi-hop and TTL sensitivity analyses, as well as delay and packet-loss perturbations. Experimental results show that UDSTC achieves higher semantic completion accuracy on masked regions, outperforms the SePaint and StreetUnveiler baselines under the same budget, and exhibits smaller performance degradation under severe latency and high packet loss, demonstrating strong adaptability to the instability of self-organizing networks.

Links and sources

• PDF from original source

Comments