# Efficient 3D Semantic Segmentation with Superpoint Transformer

**Authors:** Damien Robert, Hugo Raguet, Loic Landrieu  
**arXiv:** [2306.08045](https://arxiv.org/abs/2306.08045)  
**Date:** 2023-06-13  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.08045v1/

## Key points

- The paper introduces a novel superpoint-based transformer architecture for efficient semantic segmentation of large-scale 3D scenes.
- The method incorporates a fast algorithm to partition point clouds into a hierarchical superpoint structure, making preprocessing seven times faster than existing superpoint-based approaches.
- The model leverages a self-attention mechanism to capture relationships between superpoints at multiple scales, leading to state-of-the-art performance on three benchmark datasets.
- The approach is up to 200 times more compact than other state-of-the-art models while maintaining similar performance with only 212k parameters.
- The model can be trained on a single GPU in three hours for a fold of the S3DIS dataset, which is significantly fewer GPU-hours than the best performing methods.
- In an ablation study, the authors evaluate several design choices and find that handcrafted features have a positive impact on performance and characterizing relative position and relationship between superpoints is crucial for leveraging context.
- Modeling long relationships and using hierarchical superpoints are also important improvements.
- Overall, this paper presents an efficient method for semantic segmentation of large scale 3D scenes with state-of-the art performance on benchmark datasets while being significantly more compact than other models and requiring fewer GPU hours for training.

## Summary

The paper introduces a novel superpoint-based transformer architecture for efficient semantic segmentation of large-scale 3D scenes. The method incorporates a fast algorithm to partition point clouds into a hierarchical superpoint structure, which makes the preprocessing seven times faster than existing superpoint-based approaches. Additionally, the model leverages a self-attention mechanism to capture the relationships between superpoints at multiple scales, leading to state-of-the-art performance on three challenging benchmark datasets: S3DIS (76.0% mIoU 6-fold validation), KITTI-360 (63.5% on Val), and DALES (79.6%). The authors report that their approach is up to 200 times more compact than other state-of-the-art models while maintaining similar performance with only 212k parameters. Furthermore, their model can be trained on a single GPU in three hours for a fold of the S3DIS dataset, which is seven to seventy times fewer GPU-hours than the best performing methods. In an ablation study, the authors evaluate the impact of several design choices and report their observations. They find that handcrafted features have a positive impact on performance and that characterizing relative position and relationship between superpoints is crucial for leveraging context. They also highlight the importance of modeling long relationships and assess several improvements made possible by using hierarchical superpoints. Overall, this paper presents an efficient method for semantic segmentation of large scale 3D scenes with state-of-the art performance on benchmark datasets while being significantly more compact than other models and requiring fewer GPU hours for training. The code and models are available at github.com/drprojects/superpoint_transformer.