Benchmark Results - ScanNet Benchmark

This table lists the benchmark results for the 3D semantic label scenario.

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

OccuSeg+Semantic	0.764 1	0.758 16	0.796 4	0.839 2	0.746 1	0.907 1	0.562 1	0.850 3	0.680 1	0.672 1	0.978 1	0.610 1	0.335 1	0.777 1	0.819 7	0.847 1	0.830 1	0.691 1	0.972 1	0.885 1	0.727 1

MinkowskiNet	0.736 2	0.859 3	0.818 2	0.832 3	0.709 4	0.840 3	0.521 4	0.853 2	0.660 2	0.643 2	0.951 6	0.544 3	0.286 9	0.731 3	0.893 1	0.675 9	0.772 6	0.683 2	0.874 14	0.852 3	0.727 1
C. Choy, J. Gwak, S. Savarese: 4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks. CVPR 2019
SparseConvNet	0.725 3	0.647 25	0.821 1	0.846 1	0.721 3	0.869 2	0.533 2	0.754 10	0.603 6	0.614 3	0.955 2	0.572 2	0.325 2	0.710 4	0.870 2	0.724 4	0.823 2	0.628 5	0.934 3	0.865 2	0.683 5

JSENet	0.699 5	0.881 1	0.762 9	0.821 4	0.667 6	0.800 11	0.522 3	0.792 6	0.613 4	0.607 4	0.935 23	0.492 7	0.205 20	0.576 12	0.853 3	0.691 6	0.758 8	0.652 4	0.872 16	0.828 6	0.649 7

MatchingNet	0.724 4	0.812 10	0.812 3	0.810 5	0.735 2	0.834 4	0.495 7	0.860 1	0.572 12	0.602 5	0.954 3	0.512 4	0.280 10	0.757 2	0.845 5	0.725 3	0.780 5	0.606 8	0.937 2	0.851 4	0.700 4

KP-FCNN	0.684 7	0.847 5	0.758 12	0.784 9	0.647 8	0.814 8	0.473 9	0.772 8	0.605 5	0.594 6	0.935 23	0.450 13	0.181 27	0.587 9	0.805 8	0.690 7	0.785 4	0.614 6	0.882 10	0.819 7	0.632 10
H. Thomas, C. Qi, J. Deschaud, B. Marcotegui, F. Goulette, L. Guibas.: KPConv: Flexible and Deformable Convolution for Point Clouds. ICCV 2019
CU-Hybrid Net	0.693 6	0.596 29	0.789 5	0.803 7	0.677 5	0.800 11	0.469 11	0.846 4	0.554 16	0.591 7	0.948 12	0.500 5	0.316 4	0.609 7	0.847 4	0.732 2	0.808 3	0.593 9	0.894 8	0.839 5	0.652 6

joint point-based	0.634 14	0.614 27	0.778 7	0.667 22	0.633 10	0.825 6	0.420 18	0.804 5	0.467 28	0.561 8	0.951 6	0.494 6	0.291 6	0.566 13	0.458 24	0.579 24	0.764 7	0.559 18	0.838 20	0.814 8	0.598 17
Hung-Yueh Chiang, Yen-Liang Lin, Yueh-Cheng Liu, Winston H. Hsu: A Unified Point-Based Framework for 3D Segmentation. 3DV 2019
MVPNet	0.641 11	0.831 7	0.715 17	0.671 20	0.590 15	0.781 17	0.394 22	0.679 21	0.642 3	0.553 9	0.937 21	0.462 10	0.256 12	0.649 5	0.406 29	0.626 17	0.691 17	0.666 3	0.877 11	0.792 17	0.608 14
Maximilian Jaritz, Jiayuan Gu, Hao Su: Multi-view PointNet for 3D Scene Understanding. GMDL Workshop, ICCV 2019
PointASNL	0.666 8	0.703 21	0.781 6	0.751 15	0.655 7	0.830 5	0.471 10	0.769 9	0.474 26	0.537 10	0.951 6	0.475 8	0.279 11	0.635 6	0.698 13	0.675 9	0.751 10	0.553 19	0.816 24	0.806 10	0.703 3
Xu Yan, Chaoda Zheng, Zhen Li, Sheng Wang, Shuguang Cui: PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Sampling. CVPR 2020
SConv	0.636 13	0.830 8	0.697 21	0.752 14	0.572 19	0.780 18	0.445 15	0.716 15	0.529 17	0.530 11	0.951 6	0.446 15	0.170 28	0.507 20	0.666 15	0.636 16	0.682 19	0.541 23	0.886 9	0.799 13	0.594 18

FusionAwareConv	0.630 17	0.604 28	0.741 14	0.766 12	0.590 15	0.747 24	0.501 6	0.734 11	0.503 21	0.527 12	0.919 31	0.454 12	0.323 3	0.550 14	0.420 28	0.678 8	0.688 18	0.544 21	0.896 7	0.795 15	0.627 11
Jiazhao Zhang, Chenyang Zhu, Lintao Zheng, Kai Xu: Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation. CVPR 2020
HPEIN	0.618 20	0.729 18	0.668 26	0.647 23	0.597 14	0.766 21	0.414 19	0.680 20	0.520 18	0.525 13	0.946 14	0.432 17	0.215 18	0.493 25	0.599 19	0.638 15	0.617 29	0.570 12	0.897 6	0.806 10	0.605 15
Li Jiang, Hengshuang Zhao, Shu Liu, Xiaoyong Shen, Chi-Wing Fu, Jiaya Jia: Hierarchical Point-Edge Interaction Network for Point Cloud Semantic Segmentation. ICCV 2019
FPConv	0.639 12	0.785 11	0.760 10	0.713 16	0.603 13	0.798 13	0.392 23	0.534 31	0.603 6	0.524 14	0.948 12	0.457 11	0.250 13	0.538 15	0.723 11	0.598 20	0.696 16	0.614 6	0.872 16	0.799 13	0.567 22
Yiqun Lin, Zizheng Yan, Haibin Huang, Dong Du, Ligang Liu, Shuguang Cui, Xiaoguang Han: FPConv: Learning Local Flattening for Point Convolution. CVPR 2020
DCM-Net	0.658 10	0.778 13	0.702 18	0.806 6	0.619 11	0.813 9	0.468 12	0.693 18	0.494 23	0.524 14	0.941 17	0.449 14	0.298 5	0.510 18	0.821 6	0.675 9	0.727 11	0.568 13	0.826 21	0.803 12	0.637 9
Jonas Schult, Francis Engelmann, Theodora Kontogianni, Bastian Leibe: DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes. CVPR 2020 [Oral]
SIConv	0.625 18	0.830 8	0.694 23	0.757 13	0.563 20	0.772 20	0.448 13	0.647 25	0.520 18	0.509 16	0.949 10	0.431 18	0.191 25	0.496 23	0.614 18	0.647 13	0.672 21	0.535 24	0.876 12	0.783 18	0.571 20

SPH3D-GCN	0.610 21	0.858 4	0.772 8	0.489 35	0.532 22	0.792 14	0.404 21	0.643 26	0.570 13	0.507 17	0.935 23	0.414 21	0.046 41	0.510 18	0.702 12	0.602 19	0.705 13	0.549 20	0.859 19	0.773 21	0.534 25
Huan Lei, Naveed Akhtar, and Ajmal Mian: Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds. TPAMI 2020
PointConv	0.666 8	0.781 12	0.759 11	0.699 17	0.644 9	0.822 7	0.475 8	0.779 7	0.564 15	0.504 18	0.953 4	0.428 19	0.203 22	0.586 10	0.754 9	0.661 12	0.753 9	0.588 10	0.902 4	0.813 9	0.642 8
Wenxuan Wu, Zhongang Qi, Li Fuxin: PointConv: Deep Convolutional Networks on 3D Point Clouds. CVPR 2019
MCCNN	0.633 15	0.866 2	0.731 15	0.771 10	0.576 18	0.809 10	0.410 20	0.684 19	0.497 22	0.491 19	0.949 10	0.466 9	0.105 36	0.581 11	0.646 16	0.620 18	0.680 20	0.542 22	0.817 23	0.795 15	0.618 12
P. Hermosilla, T. Ritschel, P.P. Vazquez, A. Vinacua, T. Ropinski: Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds. SIGGRAPH Asia 2018
3DSM_DMMF	0.631 16	0.626 26	0.745 13	0.801 8	0.607 12	0.751 23	0.506 5	0.729 13	0.565 14	0.491 19	0.866 38	0.434 16	0.197 24	0.595 8	0.630 17	0.709 5	0.705 13	0.560 17	0.875 13	0.740 28	0.491 31

SegGCN	0.589 24	0.833 6	0.731 15	0.539 33	0.514 23	0.789 15	0.448 13	0.467 32	0.573 11	0.484 21	0.936 22	0.396 22	0.061 40	0.501 21	0.507 23	0.594 21	0.700 15	0.563 16	0.874 14	0.771 22	0.493 30
Huan Lei, Naveed Akhtar, and Ajmal Mian: SegGCN: Efficient 3D Point Cloud Segmentation with Fuzzy Spherical Kernel. CVPR 2020
CCRFNet	0.589 24	0.766 14	0.659 29	0.683 19	0.470 28	0.740 26	0.387 26	0.620 27	0.490 24	0.476 22	0.922 29	0.355 29	0.245 14	0.511 17	0.511 21	0.571 25	0.643 25	0.493 28	0.872 16	0.762 23	0.600 16

DVVNet	0.562 27	0.648 24	0.700 19	0.770 11	0.586 17	0.687 31	0.333 29	0.650 23	0.514 20	0.475 23	0.906 35	0.359 27	0.223 17	0.340 32	0.442 27	0.422 35	0.668 22	0.501 26	0.708 33	0.779 20	0.534 25

FRPointConv	0.619 19	0.762 15	0.697 21	0.602 28	0.547 21	0.782 16	0.432 17	0.650 23	0.581 10	0.458 24	0.952 5	0.428 19	0.130 33	0.496 23	0.747 10	0.641 14	0.710 12	0.568 13	0.902 4	0.782 19	0.511 28

3DMV, FTSDF	0.501 30	0.558 33	0.608 35	0.424 40	0.478 27	0.690 30	0.246 36	0.586 28	0.468 27	0.450 25	0.911 33	0.394 23	0.160 30	0.438 26	0.212 37	0.432 34	0.541 35	0.475 29	0.742 31	0.727 29	0.477 32

LAP-D	0.594 22	0.720 19	0.692 24	0.637 25	0.456 29	0.773 19	0.391 25	0.730 12	0.587 8	0.445 26	0.940 19	0.381 24	0.288 7	0.434 27	0.453 25	0.591 22	0.649 23	0.581 11	0.777 28	0.749 27	0.610 13

DPC	0.592 23	0.720 19	0.700 19	0.602 28	0.480 26	0.762 22	0.380 27	0.713 16	0.585 9	0.437 27	0.940 19	0.369 26	0.288 7	0.434 27	0.509 22	0.590 23	0.639 27	0.567 15	0.772 29	0.755 25	0.592 19
Francis Engelmann, Theodora Kontogianni, Bastian Leibe: Dilated Point Convolutions: On the Receptive Field Size of Point Convolutions on 3D Point Clouds. ICRA 2020
TextureNet	0.566 26	0.672 23	0.664 27	0.671 20	0.494 24	0.719 27	0.445 15	0.678 22	0.411 33	0.396 28	0.935 23	0.356 28	0.225 16	0.412 29	0.535 20	0.565 26	0.636 28	0.464 30	0.794 27	0.680 33	0.568 21
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkerhouser, Matthias Niessner, Leonidas Guibas: TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes. CVPR
subcloud_weak	0.411 38	0.479 38	0.650 30	0.475 36	0.285 41	0.519 42	0.087 43	0.725 14	0.396 35	0.386 29	0.621 43	0.250 37	0.117 34	0.338 33	0.443 26	0.188 43	0.594 30	0.369 40	0.377 43	0.616 38	0.306 38

3DMV	0.484 32	0.484 37	0.538 38	0.643 24	0.424 30	0.606 39	0.310 30	0.574 29	0.433 32	0.378 30	0.796 40	0.301 32	0.214 19	0.537 16	0.208 38	0.472 33	0.507 39	0.413 37	0.693 34	0.602 39	0.539 24
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
Pointnet++ & Feature	0.557 28	0.735 17	0.661 28	0.686 18	0.491 25	0.744 25	0.392 23	0.539 30	0.451 29	0.375 31	0.946 14	0.376 25	0.205 20	0.403 30	0.356 31	0.553 27	0.643 25	0.497 27	0.824 22	0.756 24	0.515 27

PCNN	0.498 31	0.559 32	0.644 32	0.560 32	0.420 31	0.711 29	0.229 38	0.414 33	0.436 30	0.352 32	0.941 17	0.324 31	0.155 31	0.238 37	0.387 30	0.493 29	0.529 36	0.509 25	0.813 25	0.751 26	0.504 29

PointCNN with RGB	0.458 33	0.577 31	0.611 34	0.356 42	0.321 38	0.715 28	0.299 32	0.376 36	0.328 39	0.319 33	0.944 16	0.285 34	0.164 29	0.216 40	0.229 36	0.484 31	0.545 34	0.456 32	0.755 30	0.709 30	0.475 33
Yangyan Li, Rui Bu, Mingchao Sun, Baoquan Chen: PointCNN. NeurIPS 2018
PNET2	0.442 35	0.548 34	0.548 37	0.597 30	0.363 35	0.628 37	0.300 31	0.292 37	0.374 36	0.307 34	0.881 37	0.268 35	0.186 26	0.238 37	0.204 39	0.407 36	0.506 40	0.449 33	0.667 35	0.620 37	0.462 34

PanopticFusion-label	0.529 29	0.491 36	0.688 25	0.604 27	0.386 32	0.632 36	0.225 40	0.705 17	0.434 31	0.293 35	0.815 39	0.348 30	0.241 15	0.499 22	0.669 14	0.507 28	0.649 23	0.442 34	0.796 26	0.602 39	0.561 23
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
Tangent Convolutions	0.438 37	0.437 40	0.646 31	0.474 37	0.369 34	0.645 35	0.353 28	0.258 39	0.282 41	0.279 36	0.918 32	0.298 33	0.147 32	0.283 34	0.294 33	0.487 30	0.562 32	0.427 36	0.619 37	0.633 36	0.352 37
Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, Qian-Yi Zhou: Tangent convolutions for dense prediction in 3d. CVPR 2018
SurfaceConvPF	0.442 35	0.505 35	0.622 33	0.380 41	0.342 37	0.654 34	0.227 39	0.397 35	0.367 37	0.276 37	0.924 28	0.240 38	0.198 23	0.359 31	0.262 34	0.366 37	0.581 31	0.435 35	0.640 36	0.668 34	0.398 35
Hao Pan, Shilin Liu, Yang Liu, Xin Tong: Convolutional Neural Networks on 3D Surfaces Using Parallel Frames.
ScanNet+FTSDF	0.383 40	0.297 42	0.491 40	0.432 39	0.358 36	0.612 38	0.274 34	0.116 41	0.411 33	0.265 38	0.904 36	0.229 39	0.079 38	0.250 35	0.185 40	0.320 40	0.510 37	0.385 38	0.548 39	0.597 41	0.394 36

PointNet++	0.339 41	0.584 30	0.478 41	0.458 38	0.256 42	0.360 43	0.250 35	0.247 40	0.278 42	0.261 39	0.677 42	0.183 41	0.117 34	0.212 41	0.145 42	0.364 38	0.346 43	0.232 43	0.548 39	0.523 42	0.252 41
Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas: pointnet++: deep hierarchical feature learning on point sets in a metric space.
FCPN	0.447 34	0.679 22	0.604 36	0.578 31	0.380 33	0.682 32	0.291 33	0.106 42	0.483 25	0.258 40	0.920 30	0.258 36	0.025 42	0.231 39	0.325 32	0.480 32	0.560 33	0.463 31	0.725 32	0.666 35	0.231 42
Dario Rethage, Johanna Wald, Jürgen Sturm, Nassir Navab, Federico Tombari: Fully-Convolutional Point Networks for Large-Scale Point Clouds. ECCV 2018
SPLAT Net	0.393 39	0.472 39	0.511 39	0.606 26	0.311 39	0.656 33	0.245 37	0.405 34	0.328 39	0.197 41	0.927 27	0.227 40	0.000 44	0.001 44	0.249 35	0.271 42	0.510 37	0.383 39	0.593 38	0.699 31	0.267 40
Hang Su, Varun Jampani, Deqing Sun, Subhransu Maji, Evangelos Kalogerakis, Ming-Hsuan Yang, Jan Kautz: SPLATNet: Sparse Lattice Networks for Point Cloud Processing. CVPR 2018
ScanNet	0.306 43	0.203 43	0.366 42	0.501 34	0.311 39	0.524 41	0.211 41	0.002 44	0.342 38	0.189 42	0.786 41	0.145 43	0.102 37	0.245 36	0.152 41	0.318 41	0.348 42	0.300 42	0.460 42	0.437 43	0.182 43
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17
SSC-UNet	0.308 42	0.353 41	0.290 43	0.278 43	0.166 43	0.553 40	0.169 42	0.286 38	0.147 43	0.148 43	0.908 34	0.182 42	0.064 39	0.023 43	0.018 44	0.354 39	0.363 41	0.345 41	0.546 41	0.685 32	0.278 39

ERROR	0.054 44	0.000 44	0.041 44	0.172 44	0.030 44	0.062 44	0.001 44	0.035 43	0.004 44	0.051 44	0.143 44	0.019 44	0.003 43	0.041 42	0.050 43	0.003 44	0.054 44	0.018 44	0.005 44	0.264 44	0.082 44

This table lists the benchmark results for the 3D semantic instance scenario.

Method	avg ap 50%	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

OccuSeg+instance	0.672 1	1.000 1	0.758 7	0.682 3	0.576 1	0.842 1	0.477 1	0.504 11	0.524 2	0.567 1	0.585 1	0.451 2	0.557 6	1.000 1	0.751 1	0.797 2	0.563 5	1.000 1	0.467 4

PCJC	0.578 5	1.000 1	0.810 3	0.583 8	0.449 8	0.813 2	0.042 11	0.603 6	0.341 7	0.490 2	0.465 4	0.410 4	0.650 2	0.835 10	0.264 14	0.694 6	0.561 6	0.889 13	0.504 2

SSEN	0.575 6	1.000 1	0.761 6	0.473 10	0.477 6	0.795 4	0.066 9	0.529 9	0.658 1	0.460 3	0.461 5	0.380 5	0.331 13	0.859 5	0.401 10	0.692 7	0.653 1	1.000 1	0.348 10

GICN	0.638 2	1.000 1	0.895 1	0.800 1	0.480 5	0.676 7	0.144 3	0.737 2	0.354 6	0.447 4	0.400 10	0.365 6	0.700 1	1.000 1	0.569 3	0.836 1	0.599 2	1.000 1	0.473 3

PointGroup	0.636 3	1.000 1	0.765 5	0.624 4	0.505 4	0.797 3	0.116 6	0.696 3	0.384 5	0.441 5	0.559 2	0.476 1	0.596 4	1.000 1	0.666 2	0.756 4	0.556 8	0.997 6	0.513 1
Li Jiang, Hengshuang Zhao, Shaoshuai Shi, Shu Liu, Chi-Wing Fu, Jiaya Jia: PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation. CVPR 2020 [oral]
3D-MPA	0.611 4	1.000 1	0.833 2	0.765 2	0.526 2	0.756 6	0.136 5	0.588 8	0.470 3	0.438 6	0.432 8	0.358 7	0.650 2	0.857 6	0.429 7	0.765 3	0.557 7	1.000 1	0.430 6
Francis Engelmann, Martin Bokeloh, Alireza Fathi, Bastian Leibe, Matthias Nießner: 3D-MPA: Multi Proposal Aggregation for 3D Semantic Instance Segmentation. CVPR 2020
MTML	0.549 7	1.000 1	0.807 4	0.588 7	0.327 11	0.647 8	0.004 17	0.815 1	0.180 11	0.418 7	0.364 12	0.182 10	0.445 8	1.000 1	0.442 6	0.688 8	0.571 4	1.000 1	0.396 7
Jean Lahoud, Bernard Ghanem, Marc Pollefeys, Martin R. Oswald: 3D Instance Segmentation via Multi-task Metric Learning. ICCV 2019 [oral]
ResNet-backbone	0.459 12	1.000 1	0.737 8	0.159 19	0.259 13	0.587 13	0.138 4	0.475 12	0.217 10	0.416 8	0.408 9	0.128 13	0.315 14	0.714 12	0.411 8	0.536 15	0.590 3	0.873 16	0.304 12

Sparse R-CNN	0.515 8	1.000 1	0.538 15	0.282 14	0.468 7	0.790 5	0.173 2	0.345 13	0.429 4	0.413 9	0.484 3	0.176 11	0.595 5	0.591 13	0.522 4	0.668 9	0.476 11	0.986 7	0.327 11

MASC	0.447 13	0.528 17	0.555 14	0.381 11	0.382 9	0.633 9	0.002 18	0.509 10	0.260 9	0.361 10	0.432 7	0.327 8	0.451 7	0.571 14	0.367 12	0.639 10	0.386 12	0.980 8	0.276 13
Chen Liu, Yasutaka Furukawa: MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation.
Occipital-SCS	0.512 9	1.000 1	0.716 9	0.509 9	0.506 3	0.611 10	0.092 8	0.602 7	0.177 12	0.346 11	0.383 11	0.165 12	0.442 9	0.850 9	0.386 11	0.618 11	0.543 9	0.889 13	0.389 8

3D-BoNet	0.488 10	1.000 1	0.672 12	0.590 6	0.301 12	0.484 17	0.098 7	0.620 4	0.306 8	0.341 12	0.259 14	0.125 14	0.434 10	0.796 11	0.402 9	0.499 16	0.513 10	0.909 12	0.439 5
Bo Yang, Jianan Wang, Ronald Clark, Qingyong Hu, Sen Wang, Andrew Markham, Niki Trigoni: Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds. NeurIPS 2019 Spotlight
3D-SIS	0.382 14	1.000 1	0.432 17	0.245 15	0.190 15	0.577 14	0.013 15	0.263 15	0.033 19	0.320 13	0.240 15	0.075 16	0.422 11	0.857 6	0.117 17	0.699 5	0.271 17	0.883 15	0.235 15
Ji Hou, Angela Dai, Matthias Niessner: 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans. CVPR 2019
R-PointNet	0.306 16	0.500 19	0.405 18	0.311 12	0.348 10	0.589 12	0.054 10	0.068 19	0.126 14	0.283 14	0.290 13	0.028 18	0.219 15	0.214 19	0.331 13	0.396 18	0.275 16	0.821 18	0.245 14

PanopticFusion-inst	0.478 11	0.667 13	0.712 11	0.595 5	0.259 14	0.550 15	0.000 20	0.613 5	0.175 13	0.250 15	0.434 6	0.437 3	0.411 12	0.857 6	0.485 5	0.591 14	0.267 18	0.944 9	0.359 9
Gaku Narita, Takashi Seno, Tomoya Ishikawa, Yohsuke Kaji: PanopticFusion: Online Volumetric Semantic Mapping at the Level of Stuff and Things. IROS 2019 (to appear)
UNet-backbone	0.319 15	0.667 13	0.715 10	0.233 16	0.189 16	0.479 18	0.008 16	0.218 16	0.067 18	0.201 16	0.173 16	0.107 15	0.123 17	0.438 16	0.150 15	0.615 12	0.355 13	0.916 11	0.093 20

PE	0.188 20	0.528 17	0.046 21	0.017 21	0.062 20	0.600 11	0.000 20	0.003 22	0.030 20	0.177 17	0.032 20	0.013 21	0.000 22	0.487 15	0.090 20	0.093 22	0.261 19	0.916 10	0.029 21

Region	0.248 17	0.667 13	0.437 16	0.188 17	0.153 17	0.491 16	0.000 20	0.208 17	0.094 16	0.153 18	0.099 18	0.057 17	0.217 16	0.119 20	0.039 21	0.466 17	0.302 15	0.640 19	0.140 17

Seg-Cluster	0.215 19	0.370 20	0.337 20	0.285 13	0.105 18	0.325 20	0.025 14	0.282 14	0.085 17	0.105 19	0.107 17	0.007 22	0.079 19	0.317 18	0.114 18	0.309 20	0.304 14	0.587 20	0.123 19

3D-BEVIS	0.248 17	0.667 13	0.566 13	0.076 20	0.035 22	0.394 19	0.027 13	0.035 20	0.098 15	0.099 20	0.030 21	0.025 19	0.098 18	0.375 17	0.126 16	0.604 13	0.181 20	0.854 17	0.171 16
Cathrin Elich, Francis Engelmann, Jonas Schult, Theodora Kontogianni, Bastian Leibe: 3D-BEVIS: Birds-Eye-View Instance Segmentation.
Sgpn_scannet	0.143 21	0.208 22	0.390 19	0.169 18	0.065 19	0.275 21	0.029 12	0.069 18	0.000 21	0.087 21	0.043 19	0.014 20	0.027 21	0.000 21	0.112 19	0.351 19	0.168 21	0.438 21	0.138 18

MaskRCNN 2d->3d Proj	0.058 22	0.333 21	0.002 22	0.000 22	0.053 21	0.002 22	0.002 19	0.021 21	0.000 21	0.045 22	0.024 22	0.238 9	0.065 20	0.000 21	0.014 22	0.107 21	0.020 22	0.110 22	0.006 22

Method	avg iou	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	floor	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	wall	window

CU-Hybrid-2D Net	0.636 1	0.825 1	0.820 1	0.179 13	0.648 1	0.463 1	0.549 1	0.742 1	0.676 1	0.628 1	0.961 1	0.420 1	0.379 3	0.684 1	0.381 7	0.732 1	0.723 1	0.599 1	0.827 5	0.851 1	0.634 1

DMMF_3d	0.605 2	0.651 4	0.744 5	0.782 1	0.637 2	0.387 2	0.536 2	0.732 2	0.590 3	0.540 2	0.856 9	0.359 4	0.306 8	0.596 4	0.539 1	0.627 8	0.706 2	0.497 4	0.785 9	0.757 8	0.476 9

RFBNet	0.592 5	0.616 5	0.758 2	0.659 4	0.581 5	0.330 7	0.469 5	0.655 8	0.543 7	0.524 3	0.924 2	0.355 5	0.336 6	0.572 6	0.479 5	0.671 5	0.648 4	0.480 5	0.814 7	0.814 2	0.614 3

DCRedNet	0.583 6	0.682 3	0.723 6	0.542 7	0.510 8	0.310 9	0.451 6	0.668 6	0.549 6	0.520 4	0.920 3	0.375 2	0.446 1	0.528 8	0.417 6	0.670 6	0.577 10	0.478 6	0.862 2	0.806 3	0.628 2

SSMA	0.577 7	0.695 2	0.716 8	0.439 9	0.563 6	0.314 8	0.444 7	0.719 3	0.551 5	0.503 5	0.887 8	0.346 7	0.348 5	0.603 2	0.353 9	0.709 2	0.600 8	0.457 8	0.901 1	0.786 4	0.599 4
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
MCA-Net	0.595 4	0.533 9	0.756 3	0.746 3	0.590 3	0.334 6	0.506 3	0.670 5	0.587 4	0.500 6	0.905 6	0.366 3	0.352 4	0.601 3	0.506 4	0.669 7	0.648 4	0.501 3	0.839 4	0.769 7	0.516 8

FuseNet	0.535 8	0.570 7	0.681 9	0.182 12	0.512 7	0.290 10	0.431 8	0.659 7	0.504 9	0.495 7	0.903 7	0.308 8	0.428 2	0.523 9	0.365 8	0.676 3	0.621 7	0.470 7	0.762 10	0.779 5	0.541 6
Caner Hazirbas, Lingni Ma, Csaba Domokos, Daniel Cremers: FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture. ACCV 2016
DMMF	0.597 3	0.543 8	0.755 4	0.749 2	0.585 4	0.338 4	0.494 4	0.704 4	0.598 2	0.494 8	0.911 4	0.347 6	0.327 7	0.593 5	0.527 2	0.675 4	0.646 6	0.513 2	0.842 3	0.774 6	0.527 7

ILC-PSPNet	0.475 11	0.490 10	0.581 11	0.289 11	0.507 9	0.067 13	0.379 10	0.610 10	0.417 12	0.435 9	0.822 12	0.278 9	0.267 9	0.503 10	0.228 10	0.616 10	0.533 11	0.375 11	0.820 6	0.729 9	0.560 5

3DMV (2d proj)	0.498 10	0.481 11	0.612 10	0.579 6	0.456 10	0.343 3	0.384 9	0.623 9	0.525 8	0.381 10	0.845 10	0.254 10	0.264 10	0.557 7	0.182 11	0.581 11	0.598 9	0.429 10	0.760 11	0.661 12	0.446 11
Angela Dai, Matthias Niessner: 3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation. ECCV'18
AdapNet++	0.503 9	0.613 6	0.722 7	0.418 10	0.358 13	0.337 5	0.370 11	0.479 11	0.443 10	0.368 11	0.907 5	0.207 11	0.213 12	0.464 11	0.525 3	0.618 9	0.657 3	0.450 9	0.788 8	0.721 10	0.408 12
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. International Journal of Computer Vision, 2019
Enet (reimpl)	0.376 12	0.264 13	0.452 13	0.452 8	0.365 11	0.181 11	0.143 13	0.456 12	0.409 13	0.346 12	0.769 13	0.164 12	0.218 11	0.359 12	0.123 13	0.403 13	0.381 13	0.313 13	0.571 12	0.685 11	0.472 10
Re-implementation of Adam Paszke, Abhishek Chaurasia, Sangpil Kim, Eugenio Culurciello: ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation.
ScanNet (2d proj)	0.330 13	0.293 12	0.521 12	0.657 5	0.361 12	0.161 12	0.250 12	0.004 13	0.440 11	0.183 13	0.836 11	0.125 13	0.060 13	0.319 13	0.132 12	0.417 12	0.412 12	0.344 12	0.541 13	0.427 13	0.109 13
Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, Matthias Nießner: ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes. CVPR'17

Method	avg recall	apartment	bathroom	bedroom / hotel	bookstore / library	conference room	copy/mail room	hallway	kitchen	laundry room	living room / lounge	misc	office	storage / basement / garage

multi-task learner	0.700 1	0.500 1	1.000 1	0.882 2	0.500 1	1.000 1	1.000 1	0.500 1	1.000 1	1.000 1	0.778 1	0.000 2	0.938 1	0.000 1
Shengyu Huang, Mikhail Usvyatsov, Konrad Schindler: Indoor Scene Recognition in 3D.
SE-ResNeXt-SSMA	0.498 2	0.000 3	0.812 2	0.941 1	0.500 1	0.500 2	0.500 2	0.500 1	0.429 3	0.500 2	0.667 2	0.500 1	0.625 2	0.000 1
Abhinav Valada, Rohit Mohan, Wolfram Burgard: Self-Supervised Model Adaptation for Multimodal Semantic Segmentation. arXiv
resnet50_scannet	0.353 3	0.250 2	0.812 2	0.529 3	0.500 1	0.500 2	0.000 3	0.500 1	0.571 2	0.000 3	0.556 3	0.000 2	0.375 3	0.000 1

3D Semantic label benchmark

3D Semantic instance benchmark

2D Semantic label benchmark

2D Semantic instance benchmark

Scene type classification benchmark

Method	avg ap	bathtub	bed	bookshelf	cabinet	chair	counter	curtain	desk	door	otherfurniture	picture	refrigerator	shower curtain	sink	sofa	table	toilet	window

MaskRCNN_ScanNet	0.119 1	0.129 1	0.212 1	0.002 1	0.112 1	0.148 1	0.014 1	0.205 1	0.044 1	0.066 1	0.078 1	0.095 1	0.142 1	0.030 1	0.128 1	0.139 1	0.080 1	0.459 1	0.057 1
Re-implementation of Kaiming He, Georgia Gkioxari, Piotr Dollár, Ross Girshick: Mask R-CNN. ICCV'17