In this paper, we propose 4D panoptic LiDAR segmentation to assign a semantic class and a temporally consistent instance ID to a sequence of 3D points. To this end, we present an approach and a point centric evaluation metric. Our approach determines a semantic class for every point while modeling object instances as probability distributions in the 4D spatio-temporal domain. We process multiple point clouds in parallel and resolve point-to-instance associations, effectively alleviating the need for explicit temporal data association. Inspired by recent advances in benchmarking of multi-object tracking, we propose to adopt a new evaluation metric that separates the semantic and point-to-instance association aspects of the task. With this work, we aim at paving the road for future developments of temporal LiDAR panoptic perception.

Our paper introduce two main ideas for tackling 4D Lidar Panpoptic Segmentation.

i) Forming 4D volumes using consecutive lidar frames using ego-motion and obtain instance and semantic segmentation predictions on both space and time.


ii) A point-based encoder-decoder network which works on 4D volumes and outputs necessary ingredients to cluster points via gaussian distribution function, as well as semantic interpretation for all points.