References & Citations
Computer Science > Computer Vision and Pattern Recognition
Title: UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets
(Submitted on 8 Mar 2024 (v1), last revised 17 May 2024 (this version, v3))
Abstract: Generalizable neural implicit surface reconstruction aims to obtain an accurate underlying geometry given a limited number of multi-view images from unseen scenes. However, existing methods select only informative and relevant views using predefined scores for training and testing phases. This constraint renders the model impractical in real-world scenarios, where the availability of favorable combinations cannot always be ensured. We introduce and validate a view-combination score to indicate the effectiveness of the input view combination. We observe that previous methods output degenerate solutions under arbitrary and unfavorable sets. Building upon this finding, we propose UFORecon, a robust view-combination generalizable surface reconstruction framework. To achieve this, we apply cross-view matching transformers to model interactions between source images and build correlation frustums to capture global correlations. Additionally, we explicitly encode pairwise feature similarities as view-consistent priors. Our proposed framework significantly outperforms previous methods in terms of view-combination generalizability and also in the conventional generalizable protocol trained with favorable view-combinations. The code is available at this https URL
Submission history
From: Youngju Na [view email][v1] Fri, 8 Mar 2024 06:27:13 GMT (8070kb,D)
[v2] Mon, 11 Mar 2024 05:03:56 GMT (8069kb,D)
[v3] Fri, 17 May 2024 13:17:50 GMT (8069kb,D)
Link back to: arXiv, form interface, contact.