profile image

Liang Wang,  Ph.D.

Researcher

Microsoft Applied Sciences Group


          

 

Short Bio

Liang Wang is a Researcher at Microsoft Corp's Applied Sciences Group, where he has been working on the invention and development of novel human-computer interfaces using computer vision technologies since 2010. His research interests include a variety of topics in computer vision, computer graphics, and image processing, especially on stereo matching, structure from motion, 3D reconstruction, image-based modeling and rendering, segmentation and matting for live video, and projector-camera display systems. Liang Wang received his Bachelor’s degree in Computer Science and Engineering from Beihang University, Beijing, China in 2004. He then received his Doctor of Science degree in Computer Science from University of Kentucky in 2012. More information about Liang and his research can be found in his CV and Google scholar profile.


Journal Articles

Real-Time Stereo Using Approximated Joint Bilateral Filtering and Dynamic Programming

Liang Wang, Ruigang Yang, Minglun Gong, and Miao Liao 

Journal of Real-Time Image Processing (JRTIP), Accepted August 2012

 

Automatic Real-Time Video Matting Using Time-of-Flight Camera and Multichannel Poisson Equations (pdf)

Liang Wang, Minglun Gong, Chenxi Zhang, Ruigang Yang, Cha Zhang, and Yee-Hong Yang 

International Journal of Computer Vision (IJCV), Vol. 97, No. 1,  Page 104-121, March 2012

[supplementary material – 10MB]

 

Reliability Fusion of Time-of-Flight Depth and Stereo Geometry for High Quality Depth Maps (pdf)

Jiejie Zhu, Liang Wang, Ruigang Yang, James Davis, and Zhigeng Pan

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 33, No. 7, Page 1400-1414, May 2011

 

Spatial-Temporal Fusion for High Accuracy Depth Maps using Dynamic MRFs (pdf)

Jiejie Zhu, Liang Wang, Jizhou Gao, and Ruigang Yang

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 32, No. 5, Page 899-909, May 2010

 

Stereo Matching with Color-weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling (pdf)

Qingxiong Yang, Liang Wang, Ruigang Yang, Henrik Stewenius, and David Nister

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 31, No. 3, Page 492-504, March 2009

 

Detailed Real-Time Urban 3D Reconstruction From Video (pdf)

M. Pollefeys, D. Nister, J.-M. Frahm, A. Akbarzadeh, P. Mordohai, B. Clipp, C. Engels, D. Gallup, S.-J. Kim, P. Merrell, C. Salmi, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stewenius, R. Yang, G. Welch, and H. Towles

International Journal of Computer Vision (IJCV), Vol. 78, Issue 2, Pages 143 - 167,  July 2008

   

BRDF Invariant Stereo using Light Transport Constancy (pdf)

Liang Wang, Ruigang Yang, and James Davis

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 29, No. 9, Page 1616-1626, September 2007

 

A Performance Study on Different Cost Aggregation Approaches used in Real-time Stereo Matching (pdf)

Minglun Gong, Ruigang Yang, Liang Wang, and Mingwei Gong

International Journal of Computer Vision (IJCV), Vol. 75, No. 2, Page 283-296, November 2007

  

* Note: IEEE-TPAMI and IJCV are considered to be the top two computer vision journals and both have very high impact factors that ranked among top 3 in all computer science journals.

 


Selected Conference & Workshop Publications

Global Stereo Matching Leveraged by Sparse Ground Control Points (pdf)  

Liang Wang and  Ruigang Yang

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2011

 

Semantic Segmentation of Urban Scenes Using Dense Depth Maps (pdf)

Chenxi Zhang, Liang Wang, and Ruigang Yang

European Conference on Computer Vision (ECCV), 2010

[supplementary material – 12.3 MB]

 

A Constant-Space Belief Propagation Algorithm for Stereo Matching (pdf)

Qingxiong Yang, Liang Wang, and Narendra Ahuja

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2010

 

TofCut: Towards Robust Real-Time Foreground Extraction Using a Time-of-Flight Camera (pdf)

Liang Wang, Chenxi Zhang, Ruigang Yang, and Cha Zhang

Fifth International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), 2010 (Oral)

[supplementary material – 11MB], [binary segmentation datasets]

 

Search Space Reduction for MRF Stereo (pdf)

Liang Wang, Hailin Jin, and Ruigang Yang

European Conference on Computer Vision (ECCV), 2008

 

Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images (pdf)

Liang Wang, Hailin Jin, Ruigang Yang, and Minglun Gong

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2008

 

Fusion of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps (pdf)

Jiejie Zhu, Liang Wang, Ruigang Yang, and James Davis

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2008 (Oral)

 

Multi-Projector Display with Continuous Self-Calibration (pdf)

Jin Zhou, Liang Wang, Amir Akbarzadeh, and Ruigang Yang

Fifth ACM/IEEE International Workshop on Projector-Camera Systems (PROCAMS), in conjunction with SIGGRAPH 2008 (Best Paper Award)

[supplementary material – 4 MB]

  

Real-Time Visibility-Based Fusion of Depth Maps (pdf)

P. Merrell, A. Akbarzadeh, L. Wang, P. Mordohai, J-M. Frahm, R. Yang, D. Nister, and M. Pollefeys

IEEE International Conference on Computer Vision (ICCV), 2007 (Oral)

 

Light Fall-off Stereo (pdf)

Miao Liao, Liang Wang, Ruigang Yang, and Minglun Gong

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2007

  

High-Quality Real-Time Stereo using Adaptive Cost Aggregation and Dynamic Programming (pdf)

Liang Wang, Miao Liao, Minglun Gong, Ruigang Yang, and David Nister

Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), 2006 (Oral)

[live demo video – 8.6MB]

 

How Far Can We Go with Local Optimization in Real-Time Stereo Matching (pdf)

Liang Wang, Mingwei Gong, Minglun Gong, and Ruigang Yang

Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), 2006

 

Towards Urban 3D Reconstruction from Video (pdf)

A. Akbarzadeh, J.-M. Frahm, P. Mordohai, B. Clipp, C. Engels, D. Gallup, P. Merrell, M. Phelps, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stewenius, R. Yang, G. Welch, H. Towles, D. Nister, and M. Pollefeys

Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), 2006 (Invited Paper)

 

Real-Time Global Stereo Matching Using Hierarchical Belief Propagation (pdf)

Qingxiong Yang, Liang Wang, Ruigang Yang, Shengnan Wang, Miao Liao, and David Nister

The British Machine Vision Conference (BMVC), 2006

 

Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling (pdf)

Qingxiong Yang, Liang Wang, Ruigang Yang, Henrik Stewenius, and David Nister

IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2006

 

BRDF Invariant Stereo using Light Transport Constancy (pdf)

James Davis, Ruigang Yang, and Liang Wang

IEEE International Conference on Computer Vision (ICCV), 2005 (Oral)

 

* Note: ICCV, CVPR, and ECCV are considered to be the top 3 computer vision conferences and are highly competitive with low acceptance rate <30%.

 


Selected Research Projects

Multi-projector display with continuous self-calibration

Most existing calibration techniques for multi-projector display system require that the display configuration remain fixed during the display process. We present a new approach to continuously re-calibrate the projection system to automatically adapt to the display configuration changes, while the multi-projector system is being used without interruption. By rigidly attaching a camera to each projector, we argument the projector with sensing capability and use the camera to provide online close-loop control. In contrast to previous auto or continuous projector calibration solutions, our approach can be used on surfaces of arbitrary geometry and can handle both projector and display surface movement, yielding more flexible system configuration and better scalability. related paper PROCAMS08, video clips planar (2.27MB),  corner (1.66MB),  curved (3MB)
 

Search space reduction for MRF stereo

     

We present an algorithm to reduce per-pixel search ranges for Markov Random Fields-based stereo algorithms. Our algorithm is based on the intuitions that reliably matched pixels need less regularization in the energy minimization and neighboring pixels should have similar disparity search ranges if their pixel values are similar. We propose a novel bi-labeling process to classify reliable and unreliable pixels that incorporate left-right consistency checks. We then propagate the reliable disparities into unreliable regions to form a complete disparity map and construct per-pixel search ranges based on the difference between the disparity map after propagation and the one computed from a winner-take-all method. Experimental results evaluated on the Middlebury stereo benchmark show our proposed algorithm is able to achieve 77% average reduction rate while preserving satisfactory accuracy. related paper ECCV08
 

Stereoscopic inpainting  

We present a novel algorithm for simultaneous color and depth inpainting. The algorithm takes stereo images and estimated disparity maps as input and fills in missing color and depth information introduced by occlusions or object removal. We first complete the disparities for the occlusion regions using a segmentation-based approach. The completed disparities can be used to facilitate the user in labeling objects to be removed. Since part of the removed regions in one image is visible in the other, we mutually complete the two images through 3D warping. Finally, we complete the remaining unknown regions using a depth-assisted texture synthesis technique, which simultaneously fills in both color and depth. We demonstrate the effectiveness of the proposed algorithm on several challenging data sets. related paper CVPR08

 

Light fall-off stereo

 

We present light fall-off stereo–LFS–a new method for computing depth from scenes beyond Lambertian reflectance and texture. LFS takes a number of images from a stationary camera as the illumination source moves away from the scene. Based on the inverse square law for light intensity, the ratio images are directly related to scene depth from the perspective of the light source. Using this as the invariant, we developed both local and global methods for depth recovery. Compared to previous reconstruction methods for non-lamebrain scenes, LFS needs as few as two images, does not require calibrated camera or light sources, or reference objects in the scene. related papers CVPR07, ICIP08

 

3D urban reconstruction

   

The goal of this project is to build a data collection system and a processing pipeline for automatic geo-registered 3D reconstruction of urban scenes from video. The system collects multiple video streams, as well as GPS and INS measurements in order to place the reconstructed models in geo-registered coordinates. Besides high quality in terms of both geometry and appearance, we aim at real-time performance. Even though our processing pipeline is currently far from being real-time, we select techniques and we design processing modules that can achieve fast performance on multiple CPUs and GPUs aiming at real-time performance in the near future. (find more) related papers 3DPVT06, 3D-ARCH07, ICCV07, IJCV08

 

High quality and real-time stereo algorithms

 

I have been working on designing algorithms for dense two-frame stereo matching problem aiming at both high reconstruction quality and real-time performance. Evaluation using the benchmark Middlebury stereo database shows that our algorithms are among the best in terms of both quality and speed. The real-time performance mainly comes from the parallelism of today’s commodity graphics hardware. related papers 3DPVT06-oral, 3DPVT06, BMVC06, CVPR06, IJCV07, TPAMI08

 

BRDF invariant stereo using light transport constancy

Nearly all existing methods for stereo reconstruction assume that scene reflectance is Lambertian and make use of brightness constancy as a matching invariant. We introduce a new invariant for stereo reconstruction called Light Transport Constancy, which allows completely arbitrary scene reflectance (BRDFs). This invariant can be used to formulate a rank constraint on multi-view stereo matching when the scene is observed by several lighting configurations, in which only the lighting intensity varies. In addition, we show that this multi-view constraint can be used with as few as two cameras and two lighting configurations. This new constraint can be used to provide BRDF invariance to any existing stereo method, whenever appropriate lighting variation is available. related papers ICCV05, TPAMI07