Stereo R-CNN focuses on accurate 3D object detection and estimation using image-only data in autonomous driving scenarios. It features simultaneous object detection and association for stereo images, 3D box estimation using 2D information, accurate dense alignment for 3D box refinement.

This implementation is tested with Anaconda Python 3.6 and Pytorch 1.7.0

1.1 Create Conda Environment:

    conda create -n env_stereo python=3.6
    conda activate env_stereo

1.2 Install PyTorch 1.7.0

Go to https://pytorch.org/ and choose your cuda version and run the command generated from the website

1.3 Clone Repo, Install Requirements, and Build:

    git clone https://github.com/Mohmed-Khaled/LT-Stereo-RCNN.git
    cd LT-Stereo-RCNN
    pip install -r requirements.txt
    ./build.sh

2.1 Create the folder for placing the model

    mkdir models_stereo

2.2 Download pretrained model One Drive/Google Drive and put it into models_stereo/

2.3 Run Demo

    python demo.py

If everything goes well, you will see the detection result on the left, right and bird's eye view image respectively.

Some sample results:

3.1 Download the left image, right image, calibration, labels and point clouds (optional for visualization) from KITTI Object Benchmark. Make sure the structure looks like:

yourfolder/
        training
        image_2
        image_3
        label_2
        calib
        velodyne

3.2 Create symlinks:

    cd data/kitti
    ln -s yourfolder object
    cd ../..

Download the Res-101 pretrained weight One Drive/Google Drive, and put it into data/pretrained_model

Set corresponding CUDA_VISIBLE_DEVICES in train.sh, and simply run

    ./train.sh

It consumes ~11G GPU memery for training.

The trained model and training log are saved in /models_stereo by default.

You can evaluate the 3D detection performance using either our provided model or your trained model. Set corresponding CUDA_VISIBLE_DEVICES in test.sh, and run

    ./test.sh

The results are saved in models_stereo/result/data by default.

To evalute average precision for your result run this script

    ./eval_results.sh

The evaluation code is based on the object development kit provided by KITTI Object Benchmark

This project is based on Stereo R-CNN by Peiliang Li, Xiaozhi Chen and Shaojie Shen from the HKUST Aerial Robotics Group, and DJI.

The source code is released under MIT license.