Submitted by Rohit Mohan.

Submission data

Full nameSelf-Supervised Model Adaptation for Multimodal Semantic Segmentation
DescriptionBenchmarking the SSMA mutimodal semantic segmentation framework that dynamically adapts the fusion of modality-specific features while being sensitive to the object category, spatial location and scene context in a self-supervised manner. The model was trained with the visual RGB image and the HHA encoded depth image as input to the network. The architecture consists of two modality-specific encoder streams that fuse intermediate encoder representations into a single decoder using the SSMA fusion mechanism which optimally combines complementary features.
Publication titleSelf-Supervised Model Adaptation for Multimodal Semantic Segmentation
Publication authorsAbhinav Valada, Rohit Mohan, Wolfram Burgard
Publication venueInternational Journal of Computer Vision, 2019
Publication URLhttps://arxiv.org/abs/1808.03833
Input Data TypesUses Color,Uses Geometry        Uses 2D
Programming language(s)Python, Tensorflow
HardwareIntel Xeon E5 CPU, NVIDIA TITAN X (Pascal)
Websitehttp://deepscene.cs.uni-freiburg.de
Source code or download URLhttps://github.com/DeepSceneSeg/SSMA
Submission creation date29 Dec, 2018
Last edited18 Jul, 2019

2D semantic label results

Infoavg ioubathtubbedbookshelfcabinetchaircountercurtaindeskdoorfloorotherfurniturepicturerefrigeratorshower curtainsinksofatabletoiletwallwindow
copyleft0.5770.6950.7160.4390.5630.3140.4440.7190.5510.5030.8870.3460.3480.6030.3530.7090.6000.4570.9010.7860.599