English | 简体中文

Ye Li¹    Lingdong Kong²    Hanjiang Hu³    Xiaohao Xu¹    Xiaonan Huang^1
1University of Michigan, Ann Arbor    ²National University of Singapore    ³Carnegie Mellon University

Place3D is a full-cycle pipeline that encompasses LiDAR placement optimization, data generation, and downstream evaluations.

• To identify the most effective configurations for multi-LiDAR systems, we introduce the Surrogate Metric of the Semantic Occupancy Grids (M-SOG) to evaluate LiDAR placement quality.
• Leveraging the M-SOG metric, we propose a novel optimization strategy to refine multi-LiDAR placements.
• Centered around the theme of multi-condition multi-LiDAR perception, we collect a 364,000-frame dataset from both clean and adverse conditions.

We showcase exceptional robustness in both 3D object detection and LiDAR semantic segmentation tasks, under diverse adverse weather and sensor failure conditions.

Waymo	Motional	Cruise	Pony.ai
Zoox	Toyota	Momenta	Ford

Visit our project page to explore more examples. 🚙

• [2024.03] - Our paper is available on arXiv. The code has been made publicly accessible. 🚀

• Installation
• Data Preparation
• Sensor Placement
• Place3D Pipeline
• Getting Started
• SOG Generation
• Model Zoo
• Place3D Benchmark
• TODO List
• Citation
• License
• Acknowledgements

For details related to installation and environment setups, kindly refer to INSTALL.md.

The Place3D dataset consists of a total of eleven LiDAR placements, in which seven baselines are inspired by existing self-driving configurations from autonomous vehicle companies and four LiDAR placements are obtained by sensor optimization.

Each LiDAR placement contains four LiDAR sensors. For each LiDAR configuration, the sub-dataset consists of 13,600 frames of samples, comprising 11,200 samples for training and 2,400 samples for validation, following the split ratio used in nuScenes.

Town 1	Town 3	Town 4	Town 6

We choose four maps (Towns 1, 3, 4, and 6) in CARLA v0.9.10 to collect point cloud data and generate ground truth information. For each map, we manually set 6 ego-vehicle routes to cover all roads with no roads overlapped. The frequency of the simulation is set to 20 Hz.

Our datasets are hosted by OpenDataLab.

OpenDataLab is a pioneering open data platform for the large AI model era, making datasets accessible. By using OpenDataLab, researchers can obtain free formatted datasets in various fields.

Kindly refer to DATA_PREPARE.md for the details to prepare the Place3D dataset.

Center	Line	Pyramid	Square
Trapezoid	Line-Roll	Ours (Det)	Ours (Seg)

We first generate Probabilistic SOG (b) by merging multiple frames of data synthesized in CARLA into dense point cloud (a) and obtain voxels covered by LiDAR rays to compute the Surrogate Metric of Semantic Occupancy Grids (M-SOG) based on the joint probability distribution of covered voxels (c). We propose a CMA-ES-based optimization strategy to maximize M-SOG, finding optimal LiDAR placement (d). To verify the effectiveness of our optimization strategy, we contribute a multi-condition multi-LiDAR dataset (e). We train several state-of-the-art models and evaluate the performance of baseline and optimized placements on both clean and corruption data (f). Our optimized placement generally outperforms all baseline placements in experiments.

To learn more usage about this codebase, kindly refer to GET_STARTED.md.

We generate dense point clouds (b) by merging multiple frames of raw point clouds (a). Then, we voxelize the dense point clouds into Semantic Occupancy Grids (SOG) (c), and traverse all frames of dense point clouds to compute Probabilistic Semantic Occupancy Grids (P-SOG) (d).

• Initial release. 🚀
• Add LiDAR placement benchmarks.
• Add LiDAR placement datasets.
• Add acknowledgments.
• Add citations.
• Add more 3D perception models.

If you find this work helpful for your research, please kindly consider citing our papers:

@article{li2024place3d,
    author = {Ye Li and Lingdong Kong and Hanjiang Hu and Xiaohao Xu and Xiaonan Huang},
    title = {Optimizing LiDAR Placements for Robust Driving Perception in Adverse Conditions},
    journal = {arXiv preprint arXiv:2403.17009},
    year = {2024},
}

@misc{mmdet3d,
    title = {MMDetection3D: OpenMMLab Next-Generation Platform for General 3D Object Detection},
    author = {MMDetection3D Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmdetection3d}},
    year = {2020}
}

This work is under the Apache License Version 2.0, while some specific implementations in this codebase might be with other licenses. Kindly refer to LICENSE.md for a more careful check, if you are using our code for commercial matters.

This work is developed based on the MMDetection3D codebase.

MMDetection3D is an open-source toolbox based on PyTorch, towards the next-generation platform for general 3D perception. It is a part of the OpenMMLab project developed by MMLab.

We acknowledge the use of the following public resources, during the course of this work: ¹CARLA, ²nuScenes-devkit, ³SemanticKITTI-API, ⁴MinkowskiEngine, ⁵TorchSparse, ⁶Open3D-ML, and ⁷Robo3D.

We thank the exceptional contributions from the above open-source repositories! ❤️