[논문리뷰] LiDAR2Map: LiDAR-based distillation scheme - LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation (CVPR 2023)

💡 본 문서는 'LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillation' 논문을 정리해놓은 글입니다.
해당 논문은 CLIP 같은 멀티모달 모델의 language embedding을 NeRF 안에 집어넣어 NeRF를 Multi Modal로 확장 가능성을 보여준 논문이니 참고하시기 바랍니다.
- Paper: https://openaccess.thecvf.com/content/CVPR2023/papers/Wang_LiDAR2Map_In_Defense_of_LiDAR-Based_Semantic_Map_Construction_Using_Online_CVPR_2023_paper.pdf
- Github: https://github.com/songw-zju/LiDAR2Map
- Youtube: https://www.youtube.com/watch?v=nr25xFZbx8U 

Contribution

• BEV Feature Pyramid Decoder (BEV-FPD)
• LiDAR-based network: an online Camera-to-LiDAR distillation scheme. 
  • mainly use LiDAR data and only extract image features as auxiliary network during training.
  • Feature Distill + Logit Distill

LiDAR2Map Framework

1. BEV Feature Pyramid Decoder (BEV-FPD)

2. Position-Guided Feature Fusion Module (PGF2M)

we take advantage of the multi-scale BEV features {F˜BEV i } N i=1 from BEV-FPD for the feature-level distillation.

+ feature fusion module

• knowledge distilation
• 카메라 이미지에서 얻은 풍부한 의미 정보를 활용하여 LiDAR 모델의 성능을 향상시키는 데 사용
• 실제 test에는 Lidar 시퀀스만 실행되니 속도적으로도 이득