2D-to-3D Conversion
In spite of the rapid development of 3D displays, there is still little 3D content to be displayed on these 3D devices. 3D content shortage has become a severe bottleneck to the growth of the 3D industry, and there is huge demand to develop an effective and efficient 3D content generation method. On the other hand, there is an enormous amount of high-quality 2D video content available, and it's alluring to think that this body of artistic and entertainment data might be converted into 3D formats of similar high quality.We introduce various monocular depth cues (e.g. perspective, occlusion, focus/defocus) to infer the depth information and optimize the depth information over the entire video sequence.
The introduct ion of 2D-to-3D conversion technique is published in IEEE Multimedia Magazine and was selected as the “Highlight of the Month” paper in December 2011 and appears on the Front Page of the IEEE Computer Now website. The 2D-to-3D conversion algorithm is integrated into the commercial 3D editing and conversion software “Roxio Creator” by the U.S. Company SONIC Solutions; the product is now on sale in the international market and shares about 62% of the North America market (data of the year 2010).
 |  |
Pipeline
 |
| Fig: Converting 2D Video to 3D |
Products
 |  |
| Fig: TV set-top box | Fig: Roxio Creator |
 |
| Fig: Software "Cachalot" |
Publications
Wu C, Er G, Xie X, Li T, Cao, X, Dai Q. "A novel method for semi-automatic 2D to 3D video conversion". IEEE 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video, pp. 1-4, 2008
Cao X, Bovik A, Wang Y, Dai Q. "Converting 2D video to 3D: An efficient path to a 3D experience". IEEE MultiMedia, 18 (4): 12-17, 2011
Cao X, Li Z, Dai Q. "Semi-automatic 2D-to-3D conversion using disparity propagation". IEEE Transactions on Broadcasting, 57 (2): 491-499, 2011
Li Z, Cao X, Dai Q. "A novel method for 2D-to-3D video conversion using bi-directional motion estimation". IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1429-1432, 2012
Hua X, Xu J, Ji X, Yan F, Ma Z, Cao X. "Adaptive 3D content generation via depth video based rendering". IEEE International Conference on 3D Imaging (IC3D), pp. 1-6, 2014
Xu J, Yan F, Cao X. "Stereoacuity-guided depth image based rendering", IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2014
Huang W, Cao X, Lu K, Dai Q, Bovik A. "Towards Naturalistic 2D-to-3D Conversion". IEEE Transactions on Image Processing, 24 (2): 724-733, 2015