masked_orb_slam
说明： masked_orb_slam，“掩蔽orb-SLAM3：使用掩蔽的自动驾驶场景的动态元素排除”的中心项目回购...
(The central project repo of "Masked ORB-SLAM3: Dynamic Element Exclusion for Autonomous Driving Scenarios Using Masked R-CNN for Increased Localization Accuracy". This was the major design project for EECS568.)
(The central project repo of "Masked ORB-SLAM3: Dynamic Element Exclusion for Autonomous Driving Scenarios Using Masked R-CNN for Increased Localization Accuracy". This was the major design project for EECS568.)
文件列表:
.vscode (0, 2022-12-20)
.vscode\c_cpp_properties.json (423, 2022-12-20)
CMakeLists.txt (5288, 2022-12-20)
Changelog.md (1632, 2022-12-20)
Dependencies.md (1965, 2022-12-20)
Examples (0, 2022-12-20)
Examples\FullFrameTrajectory.txt (41426, 2022-12-20)
Examples\Monocular-Inertial (0, 2022-12-20)
Examples\Monocular-Inertial\EuRoC.yaml (2470, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU (0, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH01.txt (5174011, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH02.txt (4271931, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH03.txt (3788423, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH04.txt (2850986, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH05.txt (3192548, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V101.txt (4084379, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V102.txt (2388754, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V103.txt (3002519, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V201.txt (3194928, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V202.txt (3281662, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V203.txt (3260558, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps (0, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH01.txt (73640, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH02.txt (60800, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH03.txt (54000, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH04.txt (40660, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH05.txt (45460, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V101.txt (58240, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V102.txt (34200, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V103.txt (42980, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V201.txt (45600, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V202.txt (46960, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V203.txt (38440, 2022-12-20)
Examples\Monocular-Inertial\TUM_512.yaml (3424, 2022-12-20)
Examples\Monocular-Inertial\TUM_512_far.yaml (3590, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU (0, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU\dataset-corridor1_512.txt (6034922, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU\dataset-corridor2_1024.txt (6807522, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU\dataset-corridor2_512.txt (6807522, 2022-12-20)
... ...
.vscode\c_cpp_properties.json (423, 2022-12-20)
CMakeLists.txt (5288, 2022-12-20)
Changelog.md (1632, 2022-12-20)
Dependencies.md (1965, 2022-12-20)
Examples (0, 2022-12-20)
Examples\FullFrameTrajectory.txt (41426, 2022-12-20)
Examples\Monocular-Inertial (0, 2022-12-20)
Examples\Monocular-Inertial\EuRoC.yaml (2470, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU (0, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH01.txt (5174011, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH02.txt (4271931, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH03.txt (3788423, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH04.txt (2850986, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\MH05.txt (3192548, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V101.txt (4084379, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V102.txt (2388754, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V103.txt (3002519, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V201.txt (3194928, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V202.txt (3281662, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_IMU\V203.txt (3260558, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps (0, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH01.txt (73640, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH02.txt (60800, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH03.txt (54000, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH04.txt (40660, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\MH05.txt (45460, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V101.txt (58240, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V102.txt (34200, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V103.txt (42980, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V201.txt (45600, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V202.txt (46960, 2022-12-20)
Examples\Monocular-Inertial\EuRoC_TimeStamps\V203.txt (38440, 2022-12-20)
Examples\Monocular-Inertial\TUM_512.yaml (3424, 2022-12-20)
Examples\Monocular-Inertial\TUM_512_far.yaml (3590, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU (0, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU\dataset-corridor1_512.txt (6034922, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU\dataset-corridor2_1024.txt (6807522, 2022-12-20)
Examples\Monocular-Inertial\TUM_IMU\dataset-corridor2_512.txt (6807522, 2022-12-20)
... ...
# Masked ORB-SLAM3
## Project Overview
This is Team 6's final project git repository for EECS 568: Mobile Robotics course at University of Michigan. The title of our project is Masked ORB-SLAM3: Dynamic Element Exclusion for Autonomous Driving Scenarios Using Masked R-CNN for an Increased Localization Accuracy.
The masked ORB-SLAM3 dynamic object localization pipeline-architecture involved using Mask generated from masked R-CNN for a chosen dynamic on the road driving as opposed to the typical usage of semantic masks in ORB-SLAM for localization.
## Getting Started First we recommend you read through our paper uploaded on this repository/docs. Next, read the 2 directly related works: [ORB-SLAM3](https://github.com/UZ-SLAMLab/ORB_SLAM3?msclkid=d3a88f5eb81f11ec968f0535c7080186) and [DynaSLAM](https://github.com/BertaBescos/DynaSLAM?msclkid=fe9695b7b81f11ec9d82fc4c92af772c).
## Masked ORB-SLAM3 Architecture Thus, based off the obvious inaccuracy in semantic masks for the ORB-SLAM, we propose following architecture for our "Masked ORB-SLAM3 pipeline":
The visualization of the masks we used for our framework can be seen below:
## Installation and Running To observe and remove the impact of dynamic objects in ORB-SLAM3 we selected one of the most dynamic datasets from the KITTI datasets. We observed that the ORB-SLAM 3 performs better if the dynamic content from the image frames is removed. We remove dynamic objects from the images by performing instance segmentation and then passing binary masks into the ORB-SLAM3 pipeline. We then use these masks to remove tracking points that overlap with our masks.
--- ## Acknowledgments We express our sincere thanks to our EECS 568 instructor Prof. Maani Ghaffari Jadidi for his guidance, as well as the GSIs Tzu-Yuan (Justin) Lin and Jingyu (JY) Song for all the support they provided this semester. The link to our course website can be found [here](http://robots.engin.umich.edu/mobilerobotics/). We would like to also acknowledge our profound gratitude towards Carlos Campos, Richard Elvira, Juan J. Gomez Rodriguez, Jose M. M. Montiel, Juan D. Tardos., for their seminal work in ORB-SLAM3, as well as Berta Bescos, Jose M. Facil, Javier Civera and Jose Neira for their work in DynaSLAM. --- ## References The papers that we have referred for ideation, pipeline architecture and bench-marking of results against already existing localization results can be found here.
## Getting Started First we recommend you read through our paper uploaded on this repository/docs. Next, read the 2 directly related works: [ORB-SLAM3](https://github.com/UZ-SLAMLab/ORB_SLAM3?msclkid=d3a88f5eb81f11ec968f0535c7080186) and [DynaSLAM](https://github.com/BertaBescos/DynaSLAM?msclkid=fe9695b7b81f11ec9d82fc4c92af772c).
## Masked ORB-SLAM3 Architecture Thus, based off the obvious inaccuracy in semantic masks for the ORB-SLAM, we propose following architecture for our "Masked ORB-SLAM3 pipeline":
The visualization of the masks we used for our framework can be seen below:
## Installation and Running To observe and remove the impact of dynamic objects in ORB-SLAM3 we selected one of the most dynamic datasets from the KITTI datasets. We observed that the ORB-SLAM 3 performs better if the dynamic content from the image frames is removed. We remove dynamic objects from the images by performing instance segmentation and then passing binary masks into the ORB-SLAM3 pipeline. We then use these masks to remove tracking points that overlap with our masks.
Running the Docker Image
This docker is based on ros melodic ubuntu 18. There are two versions available: - CPU based (Xorg Nouveau display) - Nvidia Cuda based. To check if you are running the nvidia driver, simply run `nvidia-smi` and see if get anything. Based on which graphic driver you are running, you should choose the proper docker. For cuda version, you need to have [nvidia-docker setup](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html) on your machine.Compilation and Running
Steps to compile the Orbslam3 on the sample dataset: - `./download_dataset_sample.sh` - `build_container_cpu.sh` or `build_container_cuda.sh` depending on your machine. Now you should see ORB_SLAM3 is compiling. To run a test example: - `docker exec -it orbslam3 bash` - `cd /ORB_SLAM3/Examples&&./euroc_examples.sh` You can use vscode remote development (recommended) or sublime to change codes. - `docker exec -it orbslam3 bash` - `subl /ORB_SLAM3`Running Our Implementation
Once the container can be run the sample data from the instructions, you may add your own data (for example from the KITTI dataset) into the Datasets folder with the same hierarchy. To use our masked implementation, clone our repository and run the `build.sh` command. Then run `mono_kitti` (looking at `euroc_examples.sh` can show examples of how the original implementation is run) with an additional command line argument appended at the end containing the directory of the segmentation masks. If you are using WSL make sure you are running an x-server so you can see the GUI. --- ## Project Team and Workload Distribution Our team members are: - [**Aditya Om**](https://www.linkedin.com/in/adityaom/) - [**Aman Kushwaha**](https://www.linkedin.com/in/raghav-varshney/) - [**Kyle Liebler**](https://www.linkedin.com/in/lieblius/) - [**Ping-Hua Lin**](https://www.linkedin.com/in/michaelphlin/) - [**Zhuowen Shen**](https://www.linkedin.com/in/zhuowenshen7558/) We had divided the work responsibility amongst ourselves in the following manner: | S. No. | Work Item | Done By | |--------|---------------------------|-------------------| | 1. | KITTI Run in ORB-SLAM | All | | 2. | Choosing right data (KITTI / semantic data set) | All | | 3. | GT Semantic + Mask GT data + ORB-SLAM | Kyle, Ping-Hua, Zhuowen | | 4. | Mask-RCNN+ Mask RCNN data + ORB-SLAM | Kyle, Aditya, Aman | | 5. | Bench-marking research & Comparison | Aditya, Ping-Hua | | 6. | Documentation | All | | 7. | Project website and Git | Kyle, Aditya |--- ## Acknowledgments We express our sincere thanks to our EECS 568 instructor Prof. Maani Ghaffari Jadidi for his guidance, as well as the GSIs Tzu-Yuan (Justin) Lin and Jingyu (JY) Song for all the support they provided this semester. The link to our course website can be found [here](http://robots.engin.umich.edu/mobilerobotics/). We would like to also acknowledge our profound gratitude towards Carlos Campos, Richard Elvira, Juan J. Gomez Rodriguez, Jose M. M. Montiel, Juan D. Tardos., for their seminal work in ORB-SLAM3, as well as Berta Bescos, Jose M. Facil, Javier Civera and Jose Neira for their work in DynaSLAM. --- ## References The papers that we have referred for ideation, pipeline architecture and bench-marking of results against already existing localization results can be found here.
近期下载者：
相关文件：
收藏者：