Fusion of Passive Stereo and Time-of-Flight (Active)
Time-of-flight
range sensors have error characteristics which are complementary to passive
stereo. They provide real time depth estimates in conditions where passive
stereo does not work well, such as on white walls. In contrast, these sensors
are noisy and often perform poorly on the textured scenes for which stereo
excels. We introduce a method for combining the results from both methods that
performs better than either alone. A depth probability distribution function
from each method is calculated and then merged. In addition, stereo methods
have long used global methods such as belief propagation and graph cuts to
improve results, and we apply these methods to this sensor. Since time-of-flight
devices have primarily been used as individual sensors, they are typically
poorly calibrated. We introduce a method that substantially improves upon the
manufacturer’s calibration. We show that these techniques lead to improved
accuracy and robustness.
Our results show by successfully fusion them, the improved quality of depth from static scenes is increased 50%. For dynamic scenes, we added a temporal smoothness term in the original MRF model, where the cost aggregation is performed in spatial-temporal domain to successfully reduce flicker in a video sequence.
People:
Ruigang Yang, Jiejie Zhu, Liang Wang, Miao Liao, Jizhou Gao, Zhigeng Pan
Related Publications:
Jiejie
Zhu,
Liang Wang, Ruigang Yang, James Davis and Zhigeng Pan. Reliability Fusion
of Time-of-Flight Depth and Stereo for High Accuracy Depth Maps. IEEE Transactions on
Pattern Analysis and Machine Intelligence (TPAMI). 2009 (major
revision)
Jiejie
Zhu,
Liang Wang, Jizhou Gao and Ruigang Yang. Spatial-Temporal Fusion for
High Accuracy Depth Maps using Dynamic MRFs. IEEE Transactions on Pattern
Analysis and Machine Intelligence (TPAMI). 2009 (accepted)
Jiejie
Zhu, Miao
Liao, Ruigang Yang and Zhigeng Pan. Joint Depth and Alpha Matte Optimization via
Fusion of Time-of-Flight sensor and Stereo. IEEE Computer Society Conference on Computer Vision and Pattern
Recognition (CVPR). 2009
Jiejie Zhu, Liang Wang , Ruigang
Yang and James Davis. Fusion of
Time-of-Flight Depth and Stereo for High Accuracy Depth Maps. IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (CVPR). 2008 (Oral presentation. acceptance <4%)
Sponsors:
the
the
the NSF of China
(No.60533080), 863 project of China (2006AA01Z335) and Open Project of State
Key Lab of CAD&CG, Zhejiang University (No.A0812)