Neural Implicit Representations for Physical Parameter Inference from a Single Video
Hofherr F, Koestler L, Bernard F, Cremers D (2023)
Publication Type: Conference contribution
Publication year: 2023
Publisher: Institute of Electrical and Electronics Engineers Inc.
Pages Range: 2092-2102
Conference Proceedings Title: Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023
Event location: Waikoloa, HI, USA
ISBN: 9781665493468
DOI: 10.1109/WACV56688.2023.00213
Abstract
Neural networks have recently been used to analyze diverse physical systems and to identify the underlying dynamics. While existing methods achieve impressive results, they are limited by their strong demand for training data and their weak generalization abilities to out-of-distribution data. To overcome these limitations, we propose to combine neural implicit representations for appearance modeling with neural ordinary differential equations (ODEs) for modelling planar physical phenomena to obtain a dynamic scene representation that can be identified directly from visual observations. Our proposed model combines several unique advantages: (i) Contrary to existing approaches that require large training datasets, we are able to identify physical parameters from only a single video. (ii) The use of neural implicit representations enables the processing of high-resolution videos and the synthesis of photo-realistic images. (iii) The embedded neural ODE has a known parametric form that allows for the identification of interpretable physical parameters, and (iv) long-term prediction in state space. (v) Furthermore, the photo-realistic rendering of novel scenes with modified physical parameters becomes possible.
Involved external institutions
Technische Universität München (TUM)
Germany (DE)
Rheinische Friedrich-Wilhelms-Universität Bonn
Germany (DE)
Germany (DE)
Rheinische Friedrich-Wilhelms-Universität Bonn
Germany (DE)
How to cite
APA:
Hofherr, F., Koestler, L., Bernard, F., & Cremers, D. (2023). Neural Implicit Representations for Physical Parameter Inference from a Single Video. In Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023 (pp. 2092-2102). Waikoloa, HI, USA: Institute of Electrical and Electronics Engineers Inc..
MLA:
Hofherr, Florian, et al. "Neural Implicit Representations for Physical Parameter Inference from a Single Video." Proceedings of the 23rd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2023, Waikoloa, HI, USA Institute of Electrical and Electronics Engineers Inc., 2023. 2092-2102.
BibTeX: Download