David Nister’s Research Page

My research interests are in computer vision, video processing and graphics in general, including reconstruction, recognition, comprehension, search and interaction based on visual input, advanced rendering and visualization. An incomplete list of my interests is:

 

• Structure from Motion
• Multiple View Geometry
• Stereo Vision
• 3D Reconstruction
• Vision-based Localisation and Mapping
• Visual Navigation and Obstacle Detection
• Vehicle Guidance
• Automotive Safety
• Aerial Imaging
• Video Mosaicing
• Statistical methods for Computer Vision and Sensor Integration
• Feature Detection and Tracking
• Recognition
• Visual searching
• Machine Learning
• Artificial Intelligence
• Tracking
• Security and Surveillance
• Matchmoving
• Augmented Reality
• Virtual Reality
• View-based Rendering
• Hardware for Vision and Image Processing
• Image and Video Compression
• Light Fields

 

One of my interests is reconstruction of a 3D environment from image or video data. I built a system that completely automatically builds a textured 3D-model based solely on an uncalibrated video sequence of a scene. Great care was taken to obtain robust results even with imperfect input data. Representative input and output is shown in Figure 1. To see a video, click the lower left image.

 

     

      

 

Figure 1. The top row shows some frames of the original video sequence. A textured triangulation of the scene is derived completely automatically from the video. The triangulation is shown without texture on the second row and textured on the third and fourth row.

 

The result is obtained in several steps. First, the unknown camera motion and calibration are recovered. This is done on the basis of feature correspondences that are found across images using appearance matching. The camera motion and calibration are found through robust geometric estimation and self-calibration. Once the geometry of the original image views is found, dense reconstruction of the scene is performed. This is achieved by fusing many multi-view stereo estimates together into a coherent description of the environment.

 

I have also created software that can estimate the motion of a single camera or stereo-head in real-time, with low delay, based solely on visual input. A sparse representation of points in the scene is also obtained. An example is shown in Figure 2. To see a video, click the lower left image.

 

 

Figure 2. Example results from the real-time system. The top row shows some frames from a turntable sequence. Features are tracked continuously throughout the sequence (shown in the middle) and the camera motion relative to the scene along with a sparse representation of the scene is estimated on the fly. The final reconstruction is shown at the bottom. No a priori knowledge of the motion nor the fact that it closes on itself was used in the estimation. The estimation was done with low latency and not in batch mode. The estimated circular camera trajectory is a good verification of the accuracy of the result.

 

This type of system can be used in numerous applications, for example vehicle guidance and obstacle avoidance, automotive safety, 3D modeling, virtual and augmented reality, aerial imaging, cartography, photogrammetry and camera calibration.