3DMedia recognized early on the need to create 3D-enabled consumer products that allow users to create and enjoy their own personal 3D content. For this reason, 3DMedia developed several proprietary technologies for licensing to consumer product manufacturers. These technologies cover various aspects of the 3D spectrum including capture, creation, adjustment, correction, distribution, and modeling. Patents have been granted for several of those technologies and others are at different stages of the patenting process.
Most of these technologies have been encapsulated within the ViewFusion technology which comes into various flavors:
· ViewFusion 1C: This is a capture technology that is applicable to systems with a single camera. ViewFusion 1C is software that can be embedded into smartphones and cameras to assist users in taking better quality 3D photos
· ViewFusion 1.5C: This is a capture technology that utilizes a main high-quality camera and a smaller auxiliary camera with lower quality. ViewFusion 1.5C is software that can be embedded in smartphones and cameras with high-performance CPUs and requires changes to the system design to integrate the second sensor. In cameras with low-performance CPUs, ViewFusion may require hardware acceleration to run in real-time.
· ViewFusion 2C: This is a creation technology that is applicable to systems with two identical cameras. ViewFusion 2C is software that can be embedded in smartphones and cameras to improve the quality of the captured 3D photos or 3D video.
· ViewFusion Creation: This is a creation technology that creates a 3D image using at least two photos captured by 1C, 1.5C, or 2C systems. ViewFusion Creation is a family of various technologies and can be licensed separately. Some of them can be embedded in devices in the form of software while others require hardware acceleration.
· ColorFusion: This is a processing step that can be applied before or after the Creation process to compensate for color differences between the two images that will be used to form a 3D pair. This is particularly useful in 1.5C and 2C systems where the different sensors can have different color characteristics. It can also be applied in 1C systems when it is not possible to lock the exposure and other capture parameters between the two shots.
· ViewFusion Correction: This technology measures the disparity of a 3D image and adjusts the image depending on the screen size, viewing distance, and user preferences, to ensure that the disparity is within comfortable limits and to provide an optimal 3D viewing experience. This is useful regardless of how the 3D image was created. It can even minimize viewing issues caused by moving objects in 3D images created using ViewFusion 1C.
The ViewFusion capture and creation technologies can work independently of each other. The Capture part typically runs in devices such as cameras, smartphones, and tablets. The Creation part can run on these devices as well as on PCs, HDTVs, and in the Cloud.
ViewFusion 1C Capture is a software-based solution that utilizes a single camera to create 3D photos of static scenes. ViewFusion 1C requires a user to take two shots of the subject side-by-side using a single camera.
To create fully immersive 3D photos/video and maximize the user experience without causing eye stress, it is necessary to determine the proper depth of a scene and capture two views of the scene at proper positions. One of the main principles of 3D photography is the parallax budget which determines the total depth on a scene. Even humans cannot see 3D for objects that are too close or too far because our two eyes are at fixed locations. The ViewFusion 1C capture technology allows users to vary the distance of the two cameras to optimize the 3D experience for a particular scene.
The following figure shows the benefits of ViewFusion 1C capture technology compared to dual-camera fixed 3D systems. In dual-camera systems the comfortable 3D depth of the view will be limited and bounded by the distance of the two cameras. However, in ViewFusion 1C the distance of the two photos and therefore the depth perception can be optimized for each scene.
The steps involved with the ViewFusion 1C capture technology are:
1. Read image data from first photo
2. Analyze first captured image
3. Analyze liveview image while camera is moving
4. Estimate the distance of the current position of the camera with respect to the first position
5. Determine the ideal distance of the second camera position with respect to the first one that will result in the optimum distance for 3D creation
6. Instruct the camera to automatically take the second photo when the camera is at the right position
The ViewFusion 1C capture technology has been patented and can be found in Methods, systems, and computer-readable storage media for selecting image capture positions to generate three-dimensional (3D) images (US 8,436,893 B2) and Methods, systems, and computer-readable storage media for identifying a rough depth map in a scene and for determining a stereo-base distance for three-dimensional (3d) content creation (20120162374) patent application.
The benefits of ViewFusion 1C capture technology can be summarized as:
· It is a software solution that can run on existing camera/smartphone/tablet platforms without hardware changes
· It can identify the optimal distance between the two capture positions for a given scene, thus creating a 3D photo with optimal depth perception
· It makes the 3D capture process very easy by automatically triggering the capture of the second photo when the camera is at the ideal position
ViewFusion 1.5C is one of 3DMedia’s most important developments and it will revolutionize the implementation of 3D in consumer products. Using this disruptive technology consumer product manufacturers can use the main camera and very small /inexpensive auxiliary camera to create 3D videos and photos instead of the classic dual-camera approach. This results into significant cost reduction, and will enable the manufacturing of high-quality 3D products at prices consumers can affrod.
Examples of products that can be manufactured using the ViewFusion 1.5C technology are shown below:
In all Digital Single Lens Reflex (i.e., DSLR), mirrorless, as well as high- and mid-end point-and-shoot cameras with zoom lenses, it is not possible to utilize dual sensors/lenses due to both size and cost. ViewFusion 1.5C is the only viable solution for those market segments. Also, in smartphones and tablets the addition a second sensor of the same quality as the main one significantly increases the system cost. ViewFusion 1.5C can offer the same capability at a fraction of the cost of dual-camera systems.
Another important element of ViewFusion 1.5C is that it can enhance 2D photography as well:
1. The auxiliary sensor can be used to capture a photo at a different exposure than the main sensor to generate a High-Dynamic-Range (HDR) photo
2. The auxiliary sensor can be used to implement motion de-blurring when capturing a 2D photo
3. The auxiliary sensor can assist in implementing better motion stabilization
The benefits of ViewFusion 1.5C capture technology can be summarized as:
1. Can be used to capture 3D photos as well as 3D video with a only a small increase on the Bill Of Materials
2. Can augment the camera functionality with other 2D functions, thus enabling the creation of a featured camera/smartphone rather than a 3D one
The ViewFusion 1.5C capture technology has been patented and can be found in Primary and auxiliary image capture devices for image processing and related methods (US 8,441,520 B2) and US 8,274,552
ViewFusion Creation technology covers a generalized set of tools and methods that can be applied to two images captured by 1C, 1.5C, or 2C systems to generate a 3D image. This technology is applicable to both still 3D photos as well as 3D video. It performs a variety of image processing steps to correct various stereoscopic parameters and optimize the generated 3D content:
· Performs corrections for vertical shift, as well as rotation and tilt between two captured photos in all dimensions. This capability is very important for 1C systems since users cannot perfectly align the camera when taking two shots side-by-side. The same operation is also applicable to both 1.5 and 2C systems since sensors and camera modules cannot be aligned perfectly during manufacturing and is very expensive to discard cameras whose alignment is not within the required range. ViewFusion Creation will correct alignment errors on the fly and will also compensate for any misalignment that may result from a product drop.
· It performs a series of post-processing steps to adjust various stereoscopic parameters to increase the comfort level and generate a high-quality and immersive 3D photo
· In cases of color mismatch between the two sensors, ColorFusion can compensate for color differences between the captured images
· In cases where the stereoscopic parameters exceed the limits of human comfort (e.g., large disparities, and object movement), ViewFusion Correction can identify and correct such areas, thus creating 3D content that is easy to enjoy
The ViewFusion Creation technology has been patented and can be found in Methods, systems, and computer-readable storage media for creating three-dimensional (3D) images of a scene (US 8,508,580 B2) and Methods, systems, and computer-readable storage media for generating three-dimensional (3d) images of a scene (20110255775)
3DMedia has also been developing other technologies in the 3D space as well as other related areas. Here is a quick summary:
Modular and Open Cameras
This technology describes new concepts in designing consumer imaging products and covers areas from hardware to software. More information on this technology can be found in the Modular and open platform image capture devices and related methods (20130111464) patent application.
Single-Shot ViewFusion 1C
ViewFusion 1C delivers very good quality 3D photos, but it requires two shots of the same scene taken at two different times. This approach has problems with moving objects are present. Using the Single-Shot 1C technology, a camera can scan the entire scene, calculate the depth based on focus information, and then create the second view utilizing pixels from the captured photo and the depth information generated by the scanning process. More information can be found in the Methods, systems, and computer-readable storage media for generating stereoscopic content via depth map creation (20110025830) patent application.
ViewFusion 1C Video
Although the current 1C technology covers only still photos, it is highly desirable to create 3D videos using a single camera. This technology involves a significant amount of processing and is not suitable for incorporating into a consumer product today. More information about this technology can be found in the Methods, systems, and computer program products for creating three-dimensional video sequences (20120105602) patent application.