dasha_f
6e1a22ba8b
assets/conveyors (274 МБ) - ленты и угловая секция NVIDIA, на которые ссылается сцена относительным путём. Раньше исключались как перекачиваемые, но без них сцена не композится из коробки. cv/ - код стереодвижков, которые вызывает control_test, без весов: * defom-stereo - рабочий бейзлайн (DEFOM vitl, вход 480, iters 24) * crestereo - второй движок, точнее по габаритам (MAE 23.5 против 32.8 мм) * fast-foundationstereo - проверялся, в бейзлайн не вошёл * circular_section.py - показатель кругового сечения, перенесён в measure_plane.py: выравнивает облако по СОБСТВЕННЫМ главным осям и режет на пяти высотах вдоль каждой. Три самодельные версии (мировые оси, одно сечение) давали хуже; результаты проверки на эталонной геометрии - в circular_section_results.json Веса по-прежнему не в репозитории - источники в MODELS.md. Наборы кадров прежних прогонов (cv/flow_*, 1.26 ГБ) исключены: это выход, а не исходники. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
47 lines
3.2 KiB
Markdown
47 lines
3.2 KiB
Markdown
# CREStereo-Pytorch
|
|
Non-official Pytorch implementation of the CREStereo (CVPR 2022 Oral) model converted from the original MegEngine implementation.
|
|
|
|
|
|
|
|
**update 2023/01/03**:
|
|
- enable DistributedDataParallel (DDP) training, training time is much faster than before.
|
|
|
|
```shell
|
|
# train DDP
|
|
# change 'dist' to True in /cfgs/train.yaml file
|
|
python -m torch.distributed.launch --nproc_per_node=8 train.py
|
|
# train DP
|
|
# change 'dist' to False in /cfgs/train.yaml file
|
|
python train.py
|
|
```
|
|
|
|
# Important
|
|
- This is just an effort to try to implement the CREStereo model into Pytorch from MegEngine due to the issues of the framework to convert to other formats (https://github.com/megvii-research/CREStereo/issues/3).
|
|
- I am not the author of the paper, and I am don't fully understand what the model is doing. Therefore, there might be small differences with the original model that might impact the performance.
|
|
- I have not added any license, since the repository uses code from different repositories. Check the License section below for more detail.
|
|
|
|
# Pretrained model
|
|
- Download the model from [here](https://drive.google.com/file/d/1D2s1v4VhJlNz98FQpFxf_kBAKQVN_7xo/view?usp=sharing) and save it into the **[models](https://github.com/ibaiGorordo/CREStereo-Pytorch/tree/main/models)** folder.
|
|
- The model was converted from the original **[MegEngine weights](https://drive.google.com/file/d/1Wx_-zDQh7BUFBmN9im_26DFpnf3AkXj4/view)** using the `convert_weights.py` script. Place the MegEngine weights (crestereo_eth3d.mge) file into the **[models](https://github.com/ibaiGorordo/CREStereo-Pytorch/tree/main/models)** folder before the conversion.
|
|
|
|
# ONNX Conversion
|
|
- After either downloading the pretrained weights or training your own model, you will have a `models/crestereo_eth3d.pth` file. If you want to run your model with ONNX, you need to run the convert_to_onnx.py script. The script has two parts:
|
|
1. Convert the model to an ONNX model that takes in left, right images as well as an initial flow estimate (takes a few seconds)
|
|
2. Convert the model to an ONNX model that takes in left, right images and NO initial flow estimate (takes several minutes and requires pytorch >= 1.12)
|
|
(afaik) You will need both models to get the same results as you do from test_model.py.
|
|
- Run the test_onnx_model.py script to verify your models work as expected!
|
|
- NOTE: although the test_model.py script works with any size images as input, once you have converted your
|
|
Pytorch model into ONNX models, you must provide them with the image sizes used at conversion time or it will not work.
|
|
|
|
# Licences:
|
|
- CREStereo (Apache License 2.0): https://github.com/megvii-research/CREStereo/blob/master/LICENSE
|
|
- RAFT (BSD 3-Clause):https://github.com/princeton-vl/RAFT/blob/master/LICENSE
|
|
- LoFTR (Apache License 2.0):https://github.com/zju3dv/LoFTR/blob/master/LICENSE
|
|
|
|
# References:
|
|
- CREStereo: https://github.com/megvii-research/CREStereo
|
|
- RAFT: https://github.com/princeton-vl/RAFT
|
|
- LoFTR: https://github.com/zju3dv/LoFTR
|
|
- Grid sample replacement: https://zenn.dev/pinto0309/scraps/7d4032067d0160
|
|
- torch2mge: https://github.com/MegEngine/torch2mge
|