In reconstructing soft objects under different deformation states with RGB-D sensors, the results usually suffer from incomplete geometries and textures due to self-occlusion, such as dynamic wrinkles on a garment. A priori template is usually used for addressing this issue, but it requires complex scanning and an elaborate setup. This paper proposes a new framework to reconstruct a deformable soft object with complete geometry and consistent texture by introducing an incremental-completion self-generated template (SGT). By building a non-rigid registration that combines geometry and optical flow features, the SGT is dynamically updated and completed by supplementing the information from each initial state model. Then the updated SGT is reversely deformed to each state to obtain a sequence of dynamic reconstructed results with consistent geometry. Furthermore, a consistent Markov random field is also proposed to constrain mesh models in different states to generate consistent texture and guide non-rigid deformation. Experimental results show that our method achieves multi-state highquality reconstruction effects, which provides a new solution for dynamically reconstructing colored soft objects.

Fig 1. Our Pipeline.