The 370 sequences of the DeepDeform dataset release (train and validation set) may be used for learning the parameters of the algorithms. The test data should be used strictly for reporting the final results -- this benchmark is not meant for iterative testing sessions or parameter tweaking.
Parameter tuning is only allowed on the training data. Evaluating on the test data via this evaluation server must only be done once for the final system. It is not permitted to use it to train systems, for example by trying out different parameter values and choosing the best. Only one version must be evaluated (which performed best on the training/validation data). This is to avoid overfitting on the test data. Results of different parameter settings of an algorithm can therefore only be reported on the training set. To help enforcing this policy, we block updates to the test set results of a method for two weeks after a test set submission.
It is not permitted to register on this webpage with multiple e-mail addresses. We will ban users or domains if required.
Download: If you would like to download the DeepDeform data, please fill out this google form and, once accepted, we will send you the link to download the data.
Tasks Data Requirements: For all tasks, both color and depth images can be used as input.
Evaluation Scripts: Additionally, we provide some helper scripts for loading, storing and visualizing flow images. The evaluation scripts used for each task are also provided.
RGB-D Data: 3D data is provided as RGB-D video sequences, where color and depth images are already aligned. Color images are provided as 8-bit RGB
Camera Parameters: A 4x4 intrinsic matrix is given for every sequence (because different cameras were used for data capture, every sequence can have different intrinsic matrix). Since the color and depth images are aligned, no extrinsic transformation is necessary.
Optical Flow Data: Dense optical flow data is provided as custom binary image of resolution 640x480 with extension
Scene Flow Data: Dense scene flow data is provided as custom binary image of resolution 640x480 with extension
Object Mask Data: A few frames per sequences also include foreground dynamic object annotation. The mask image is given as 16-bit
Sparse Match Annotations: We provide manual sparse match annotations for a few frame pairs for every sequence. They are stored in
Sparse Occlusion Annotations: We provide manual sparse occlusion annotations for a few frame pairs for every sequence. They are stored in
We provide a dataset split of 340 train, 30 validation and 30 test sequences. We made sure that there is no instance overlap between train, validation and test sequences. Labels for each subset are given as:
Manually annotated sparse matches.
Densely aligned optical and scene flow images with the use of sparse matches as a guidance.
Densely aligned optical and scene flow images using self-supervision (DynamicFusion pipeline) for a few sequences.
Dynamic object annotations for a few frames per sequence.
Results for a method must be uploaded as a single .zip or .7z file (7z is preferred due to smaller file sizes), which when unzipped must contain in the root the prediction files. There must not be any additional files or folders in the archive except those specified below.
Format for 2D Optical Flow Prediction
For the 2D optical flow prediction task, results must be provided in a custom binary format
A submission must contain an optical flow prediction for each test frame pair, e.g.:
Format for Non-rigid RGB-D Reconstruction
For non-rigid RGB-D reconstruction task, results must be provided as a subset of
For instance, a submission should look like:
|-- seq000_100_000000.ply [-- CONSISTENT POINTS 0 --]
|-- seq000_100_000050.ply [-- CONSISTENT POINTS 0 --]
|-- seq000_100_000100.ply [-- CONSISTENT POINTS 0 --]
|-- seq000_200_000000.ply [-- CONSISTENT POINTS 1 --]
|-- seq000_200_000050.ply [-- CONSISTENT POINTS 1 --]
|-- seq000_200_000100.ply [-- CONSISTENT POINTS 1 --]
|-- seq000_200_000150.ply [-- CONSISTENT POINTS 1 --]
|-- seq000_200_000200.ply [-- CONSISTENT POINTS 1 --]