Unsupervised learning of disparity maps from stereo images
-
1
Max Planck Institute for Biological Cybernetics, Germany
-
2
University of California, Redwood Center for Theoretical Neuroscience, United States
The visual perception of depth is a striking ability of the human visual system and an active part of research in fields like neurobiology, psychology, robotics, or computer vision. In real world scenarios, many different cues, such as shading, occlusion, or disparity are combined to perceive depth. As can be shown using random dot stereograms, however, disparity alone is sufficient for the generation of depth perception [1]. To compute the disparity map of an image, matching image regions in both images have to be found, i.e. the correspondence problem has to be solved. After this, it is possible to infer the depth of the scene. Specifically, we address the correspondence problem by inferring the transformations between image patches of the left and the right image. The transformations are modeled as Lie groups which can be learned efficiently [3]. First, we start from the assumption that horizontal disparity is caused by a horizontal shift only. In that case, the transformation matrix can be constructed analytically according to the Fourier shift theorem. The correspondence problem is then solved locally by finding the best matching shift for a complete image patch. The infinitesimal generators of a Lie group allow us to determine shifts smoothly down to subpixel resolution. In a second step, we use the general Lie group framework to allow for more general transformations. In this way, we infer a number of transform coefficients per image patch. We finally obtain the disparity map by combining the coefficients of (overlapping) image patches to a global disparity map. The stereo images were created using our 3D natural stereo image rendering system [2]. The advantage of these images is that we have ground truth information of the depth maps and full control over the camera parameters for the given scene. Finally, we explore how the obtained disparity maps can be used to compute accurate depth maps.
References
1. Bela Julesz. Binocular depth perception of computer-generated images. The Bell System Technical Journal, 39(5):1125-1163, 1960.
2. Jörn-Philipp Lies and Matthias Bethge. Image library for unsupervised learning of depth from stereo. In Frontiers in Computational Neuroscience. Conference Abstract: Bernstein Symposium 2008, 2008.
3. Jimmy Wang, Jascha Sohl-Dickstein, and Bruno Olshausen. Unsupervised learning of lie group operators from image sequences. In Frontiers in Systems Neuroscience. Conference Abstract: Computational and systems neuroscience, 2009.
Conference:
Bernstein Conference on Computational Neuroscience, Frankfurt am Main, Germany, 30 Sep - 2 Oct, 2009.
Presentation Type:
Poster Presentation
Topic:
Abstracts
Citation:
Lies
J,
Wang
J,
Sohl-Dickstein
J,
Olshausen
BA and
Bethge
M
(2009). Unsupervised learning of disparity maps from stereo images.
Front. Comput. Neurosci.
Conference Abstract:
Bernstein Conference on Computational Neuroscience.
doi: 10.3389/conf.neuro.10.2009.14.126
Copyright:
The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers.
They are made available through the Frontiers publishing platform as a service to conference organizers and presenters.
The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated.
Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed.
For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions.
Received:
27 Aug 2009;
Published Online:
27 Aug 2009.
*
Correspondence:
Jörn-Philipp Lies, Max Planck Institute for Biological Cybernetics, Tubingen, Germany, philipp.lies@tuebingen.mpg.de