Photometric Depth Super-Resolution

Haefner B, Peng S, Verma A, Queau Y, Cremers D (2020)

Publication Type: Journal article

Publication year: 2020

Journal

IEEE Transactions on Pattern Analysis and Machine Intelligence Institute of Electrical and Electronics Engineers (IEEE)

Book Volume: 42

Pages Range: 2453-2464

Article Number: 8738841

Journal Issue: 10

DOI: 10.1109/TPAMI.2019.2923621

Abstract

This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and real-world scenarios.

Involved external institutions

Technische Universität München (TUM) DE Germany (DE) University of Illinois at Urbana-Champaign US United States (USA) (US) Groupe de Recherche en Informatique, Image, Automatique et Instrumentation de Caen (GREYC) FR France (FR)

How to cite

APA:

Haefner, B., Peng, S., Verma, A., Queau, Y., & Cremers, D. (2020). Photometric Depth Super-Resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(10), 2453-2464. https://doi.org/10.1109/TPAMI.2019.2923621

MLA:

Haefner, Bjoern, et al. "Photometric Depth Super-Resolution." IEEE Transactions on Pattern Analysis and Machine Intelligence 42.10 (2020): 2453-2464.

BibTeX: Download