  |
Pixel Router Even with the recent rapid advancement in hardware, the demand from high-end graphics applications (including video games) seems to always outpace the capability that a single GPU can offer. As we migrate from a single GPU to multiple GPUs or eventually GPU clusters, how to effectively assemble the final image from these distributed rendering nodes becomes an important issue. Here we propose to develop a flexible pixel compositor to solve this problem.
|
 |
Spatial-Depth Supre Resolution for Range Images We present a new post-processing step to enhance the resolution of range images. Using one or two registered and potentially high-resolution color images as reference, we iteratively refine the input low-resolution range image, in terms of both its spatial resolution and depth precision. Evaluation using the Middlebury benchmark shows across-the-board improvement for sub-pixel accuracy. We also demonstrated its effectiveness for spatial resolution enhancement up to 100X with a single reference image.
|
  |
Light Fall-off Stereo LFS-a new method for rcomputing depth from scenes beyond lambertian reflectance and texture. Compared to previous reconstruction methods for non-lamebrain scenes, LFS needs as few as two images, does not require calibrated camera or light sources, or reference objects in the scene.
|
 |
3D Urban Reconstruction from Video
This project aims at developing a fully automated system for the accurate and rapid 3D reconstruction of urban environments from video streams. The system collects multiple video streams, as well as GPS and INS measurements in order to place the reconstructed models in geo-registered coordinates. Besides high quality in terms of both geometry and appearance, we aim at real-time performance on a combination of CPUs and GPUs. |
 
|
BRDF Invariant Stereo using Light Transport Constancy
Nearly all existing methods for stereo reconstruction assume that scene reflectance is Lambertian and make use of brightness constancy as a matching invariant. We introduce a new invariant for stereo reconstruction called Light Transport Constancy, which allows completely arbitrary scene reflectance (BRDFs).
|
  
|
Towards Space-time Light Field Rendering
In this paper we propose a novel framework, space-time light field rendering, which allows continuous exploration of a dynamic scene in both spatial and temporal domain with unsynchronized input video sequences.
|
|
|
 |
Projector-Whiteboard-Camera System for Remote Collaboration
In a typical remote collaboration setup, two or more projector-camera pairs are "cross-wired" to form a full-duplex system for two-way communication. A whiteboard can be used as the projector screen, and in that case, the whiteboard server as an output device as well as an input device. Users can write on the whiteboard to comment on what is projected or to add new thoughts in the discussion.
|
|
Wide-area Rapid Iris Image Capture with Pan-tilt-zoom Cameras
In response to the DHS interest in fast biometric measurement, this project will develop a system to rapidly capture iris images of moving human subjects at long range. Working in concert with a currently available commercial iris identification software package, our system will provide fast, accurate, and automated biometric identification first for homeland security and also for other fields requiring identification or authentication.
|
|
High Quality and Real-time Stereo Algorithms
We have been working on designing algorithms for dense two-frame stereo matching problem aiming at both high reconstruction quality and real-time performance. Evaluation using the benchmark Middlebury stereo database shows that our algorithms are among the best in terms of both quality and speed. |
| |
| Dr. Yang's Previous and Current Work: |
| 3D Reconstruction and View Synthesis (from May 2001) |
|
3D Physically-based 2D View Synthesis
As part of Dr. Yang's thesis work, he is working on a new statistical approach for view synthesis. It is particular effective for texture-less regions and specular highlights, two major problems that most existing reconstruction techniques would have difficulty with. We are preparing to report our work to ICCV 2003. Some initial results are presented on the left, the top row shows several input images while the bottom row shows the reconstructed point cloud. |
|
Real-time Stereo
A multi-resolution stereo algorithm that can be implemented on commodity graphics hardware. A paper and a live demo appeared in CVPR 2003. |
|
Real-time View Synthesis on Graphics Hardware
We present a novel use of commodity graphics hardware that effectively combines a plane-sweeping algorithm with view synthesis for real-time, on-line 3D scene acquisition and view synthesis. The heart of our method is to use programmable Pixel Shader technology to square intensity differences between reference image pixels, and then to choose final colors that correspond to the minimum difference, i.e. the most consistent color. We filed an invention disclosure with UNC.
|
Internship at Microsoft Research
(Mentor: Zhengyou Zhang), Summer 2001 |
|
Dr. Yang's internship at Microsoft Research (MSR) during summer 2001 has focused on maintaining eye-contact for desktop video teleconferencing. They took a model-based approach that incorporates a detailed individualized three-dimensional head model with stereoscopic analysis. This approach is very effective; they probably achieved the most realistic results in published literature for eye gaze correction. In the process, they can also get very accurate 3D tracking results of the head pose. The images show the face model projected on the tracked head. MSR has filed two patent applications for our algorithms and systems.
|
| Large Format Display (2000-2001) |
|
PixelFlex: A Reconfigurable Multi-Projector Display System
The PixelFlex system is composed of ceiling-mounted projectors, each with computer-controlled pan, tilt, zoom and focus; and a camera for closed-loop calibration. Working collectively, these controllable projectors function as a single logical display capable of being easily modified into a variety of spatial formats. The left image shows a stacked configuration that can be used for stereo display. |
|
Automatic Projector Display Surface Estimation Using Every-Day Imagery
We introduce a new method for continuous display surface auto-calibration. Using a camera that observes the display surface, we match image features in whatever imagery is being projected, with the corresponding features that appear on the display surface, to continually refine an estimate for the display surface geometry. In effect we enjoy the high signal-to-noise ratio of "structured" light (without getting to choose the structure) and the unobtrusive nature of passive correlation-based methods. |
| Tele-Immersion (1998-current) |
|
We want to design a system that facilitate many-to-many teleconferencing. Instead of providing a perceptively correct view for every single user, we strive to provide the best approximating view for the entire group as a whole. We demonstrate two real-time acquisition-through-rendering algorithms: one is based on view dependent texture mapping with automatically acquired approximating geometry, and the other uses an array of cameras to perform Light Field style rendering. |
|
The goal of Tele-Immersion is to enable users at geographically distributed sites to collaborate in real time in a shared, simulated environment as if they were in the same physical room. While the entire project was a interdisciplinary, multi-site collaboration, Dr. Yang was mainly invovled in in real-time data capture and distribution. |
|
2D Immersive Teleconferencing
We worked on improving the field of view and resolution for 2D video teleconferencing. The result is a simple, yet effective technique for producing geometrically correct imagery for teleconferencing environments. The necessary image transformations are derived by finding a direct one-to-one mapping between a capture device and a display device for a fixed viewer location, thus completely avoiding the need for any intermediate, complex representations of screen geometry, capture and display distortions, and viewer location. Using this technique, we can easily build an immersive teleconferencing system using multiple projectors and cameras.
|
|
|
|
|
