The University of Kentucky Center for Visualization & Virtual Environments

Real-time Stereo 3D Reconstruction

Volumetric Reconstruction for Scenes with Textureless Regions and Specular Highlights

Investigators:
Ruigang Yang
Sifang Li

Collaborators:
Greg Welch (University of North Carolina),
Marc Pollefeys (University of North Carolina)

Sponsors:
UK Office of Research, National Science Foundation


Papers:
Yang, Ruigang, Marc Pollefeys and Greg Welch. “Dealing with Textureless Regions and Specular Highlights – A Progressive Space Carving Scheme Using a Novel Photo-consistency Measure,” 9th IEEE International Conference on Computer Vision (ICCV 2003), Nice, France (October 13-16, 2003).


Project Description:
There has been a considerable amount of work on volumetric scene reconstruction from multiple views. Most of this work can be considered variations of the Space Carving framework by Kutulakos and Seitz. Under this framework, an initial bounding volume is divided into a regular 3D voxel grid, then inconsistent voxels are removed until the remaining voxels are photo-consistent with a set of input images. That is, rendered images of the resulting voxels from each input viewpoint should reproduce the actual image as closely as possible. Because of the flexibility of the volumetric representation and the elegant treatment of visibility, space carving approaches have been used to achieve strong results on a variety of both natural and artificial scenes. However such approaches typically run into difficulty when applied to scenes with textureless or specular surfaces.

We present two extensions to the Space Carving framework. The first is a progressive scheme to better reconstruct surfaces lacking sufficient textures. The second is a novel photo-consistency measure that is valid for both specular and diffuse surfaces, under unknown lighting conditions.

Our techniques are particularly effective for scenes in which textureless regions and specular highlights are the norm instead of the exception, for example, in a surgical environment. Such a scene poses significant challenges for traditional reconstruction techniques. However, results from our techniques are encouraging.

Currently we are investigating better representation schemes for large-scale reconstruction and more general photo-consistency measures that can be applied to other types of materials such as translucent or reflective ones.

NOTICE: Some websites to which these materials provide links for the convenience of users are not managed by the University of Kentucky. The University does not review, control, or take responsibility for the contents of those sites. Equal Opportunity/Affirmative Action Employer.