Nowadays 3Dstereo leaves nobody indifferent – it has got a lot of delighted supporters and many critical opponents. 3Dadvocates admit that stereo video has some shortcomings but nevertheless believe in a successful future of stereoscopic content, because it’s just natural for a human being to have a three dimensional vision. The opponents sentence 3Dto death because of its technical problems and stereoscopic errors, causing headache and eye-strain. However, many problems of any kind can be solved, if to try to understand their reasons.
Graphics & Media Lab (further: the Lab) of Lomonosov Moscow State University managed to do it in its VQMT3D(Video Quality Measurement Tool 3D) Project. The Project’s purpose was to analyze the stereo films and to investigate the potential reasons of the irritating annoyance, causing headache and eye-strain by the viewers. The Lab also planned to find inexpensive ways to automatically detect the stereo errors that occur in S3Dfilms during the conversion process; these methods enable a better quality control, which is crucial for delivering the best experience for 3Dviewers.
Image #1: D visualization of the cardboard effect (Image source)
Video Quality Assessment Project
The Lab has already issued the first and second parts of the Project in the spring 2013. We hope that many industry professionals had a chance to get acquainted with these reports and our Project could be of practical use for post-production houses and 2D-to-3Dconversion studios. Now we’d like to present you the 3Dreport of VQMT3DProject that is dedicated to quality analysis of S3Dfilms converted from 2D footage during postproduction. Let us remind you that the first two reports were focused on films captured with stereoscopic camera systems.
In the 3Dreport we’ve investigated and analyzed the artifacts that are the most typical for the converted films: edge sharpness mismatch and the cardboard effect (image #1).
But we were surprised to discover that some errors, obviously specific to captured S3Dvideo, such as color mismatch and vertical parallax, were also present in converted S3Dfilms.
The cardboard effect (Image #2) is at present more frequent to find not in classical stereoscopic movies created using two cameras, but in 3Dmovies converted from 2D format by means of computer-graphics techniques. The programmers did not simulate actual shooting with a stereo camera, they calculated the image pairs from the original 2D shots, transforming each original shot in accordance with a so-called depth map. These maps are based on the image analysis and represent the producer’s understanding of the spatial distribution of the objects in the scene.
Image #2: Example of the cardboard effect (Image source)
Andrew Enyart (Stereographer, VFX Supervisor): ‘Cardboard’ effects are a direct result of not paying attention to an object’s expected geometry on the basis of proximity to the viewer and background cues. The error is heightened by appearance of depth in the more distant background. We use this background depth as a gauge for scene scale, making the relatively flat foreground seem wrong or ‘cardboard.’
The term edge sharpness mismatch (Image #3) describes defective stereo pairs with specific asymmetric impairment. Under the viewing conditions in a real environment, such situations rarely occur. During the 2D-3Dconversion workflow, edge sharpness mismatch can be caused by:
• imperfections in the depth map along the object edges (which usually correspond to strong depth discontinuities) • imperfections in the occlusion-filling process
The report consists of three main parts:
Description of the potential causes of visual discomfort, specific to converted S3Dvideo. Brief information on how to understand the graphics and visualizations. Detailed analysis of five 3Dfilms containing: • per-frame analysis and charts showing metric values for each frame • visualizations of artifacts in frames that earned a poor rating according to our metrics Overall film comparison with the charts, depicting the average metric values and distribution of these values in each film
In the Appendix we compare 2D and S3Dversions of the films. This comparison had several goals:
- Analysis and classification of different 2D-3Dconversion techniques
- Assessment of 2D-3Dconversion quality from a slightly different perspective
- Development of 2D-3D conversion quality-control tools
We discovered many techniques that were applied to simplify the conversion process: foreground-object enlargement, background modification and hair removal (Images #4, #5).
Film List:
• Alice in Wonderland (2010) • Clash of the Titans (2010) • Conan the Barbarian (2011) • Green Lantern (2011) • Abraham Lincoln: Vampire Hunter (2012)
Acknowledgements
We are grateful for valuable comments and opinions on this report to the following stereographers and industry professionals (in alphabetical order):
• Robert Black (3DR&D, 3DConsultant / Stereographer) • Ross Copeland (Online Editor / Stereographer & Colourist, Post Production Consultant) • Andrew Enyart (Stereographer, VFX Supervisor) • Srboljub Hetlerovic (Stereographer, VFX Supervisor) • Jon Karafin (Senior Scientist and Director of Production Technology at RealD) • Greg Passmore (Director PassmoreLab) • Daniele Pugni (3DMotion Graphic Designer)
Plans For the Future
In the next reports we plan to analyze a stereo-window violation effect, which may be especially annoying in the most visible parts of the image. This error appears in both captured and converted S3Dfilms. The 4th part will be published on VQMT3Dproject homepage this autumn.
How Could We Help
As already mentioned, our Project’s aim is to improve stereoscopic films. The Lab intends to provide a reliable quality-assessment tool and therefore to help filmmakers and conversion studios to produce high-quality 3Dvideo. We are open for cooperation and are ready to evaluate the S3Dfull-length films or just the samples. Please send you requests and questions to 3dmovietest@graphics.cs.msu.ru.
Regarding the article “Stereoscopic Errors in Converted Films: How and Where They Arise”, a caption for a frame capture of an archer is labeled “cardboarding”. This is not cardboarding. This is an example of poor background replacement or infillling adjacent to horizontal displacement of vertical occlusions. This is corrected by several methods, depending on the available image information. These include cleanplating, inpainting, segmentation replacement, lateral shifting, pixel remapping, and pixel stretching. All of these take time, and some depth artists take shortcuts, especially on fast action. The indicated frame appears to be an attempt to use motion blur to fill in the “hole”.
Cardboarding is a very different impression, that of different characters or objects appearing to be flat “cut-outs” placed at different distances, somewhat like in a “pop-up” book. It can appear in conversions, but most often appears in poor live action phototgraphy. There are several causes, a common one being using too long a focal length for the lenses for too narrow a stereo base. The stereo base and focal length must be kept proportional in order to keep the scene tautomorphic (proportions equivalent along all three axes).