Integral 3D TV
Masahiro Kawakita1, Jun Arai1, Hisayuki Sasaki1, Hitoshi Hiura1,
Masato Miura1, Makoto Okui1, and Fumio Okano2
1Japan Broadcasting Corporation (NHK), Science & Technology Research Laboratories,
2NHK Engineering Services, Inc.,
NHK STRL is researching into three-dimensional (3D) broadcast to achieve the ultimate in future TV broadcasting. Recently, 3D cinemas have attracted interest in many countries and display companies are planning sales of stereoscopic 3D displays using special glasses. Stereoscopic 3D display can provide wide and high-resolution 3D video images. However, they are hard to use for full-scale TV broadcasts because there still remain various unsolved issues related to visual fatigue. A 3D TV based on a spatial-imaging method has the potential to display ideal natural-looking 3D images without viewers needing special glasses. We are studying integral 3D TV based on the integral photography method, which is a spatial-imaging method.
The 3D display methods can be roughly classified into four types: binocular, multi-view, volumetric, and spatial-imaging . The binocular, also called stereoscopic, method uses two images: one for the left eye and one for the right eye. This method can display large, high-quality 3D images that are suitable for digital cinema and amusement parks. This method mainly uses binocular disparity and convergence. On the other hand, multi-view offers motion parallax. Binocular and multi-view methods cannot reconstruct 3D optical images in space, and the points of convergence and accommodation do not match, so visual fatigue does not disappear. The volumetric method generally multiplexes the images for each depth position by using mechanical or electrical active devices. In this method, the points of convergence and accommodation match, but it is difficult to avoid phantom effects and occlusion. The spatial-imaging method reproduces light rays from the objects to reconstruct an optical image in space. Integral photography and holography are classified as using this method. This method has the potential to solve the problems of visual fatigue because we observe light rays similar to that of a 3D image in space. On the other hand, a lot of image data is needed to capture, transmit, and display images in this method. Therefore, we used an extremely high-resolution video for the spatial-imaging 3D display method.
The integral photography method is a spatial-imaging technique proposed by G. Lippmann in 1908 . As shown in Fig.1, this method uses a lens array both to record and reconstruct 3D images. The numerous small elemental images of objects were incident on the recording medium through the lens array. The reconstructed light rays from the elemental images on the recording medium behind the lens array converged at a point to reconstruct optical 3D images. One advantage of this method is that the 3D images were recorded and replayed under natural light without coherent laser light. This method can display ideal 3D images that have parallax in both horizontal and vertical directions. In addition, special glasses were not required to see 3D images, which is important for TV broadcasting [3-5].
Fig. 1 Basic principle of integral photography
Fig. 2 Real-time integral 3D TV system
Figure 2 shows the basic scheme of a real-time integral 3D TV system using a high-resolution video camera and projector. The elemental images that were captured (Fig. 3) by a high-resolution camera using a lens array were transmitted to the display side and were reconstructed as 3D images by the display lens array. The total number of pixels in the elemental images was expressed by multiplying the number of elemental lenses by the number of pixels in each elemental image. To display a 3D image as a TV image with enough resolution, extremely high-resolution was required for both the camera and display system. Therefore, we are studying how to improve the resolution and viewing angle of 3D images by using an extremely high-resolution video system [6, 7].
Fig. 3 Elemental images
Super Hi-Vision (SHV) TV has a resolution of 7680(H) × 4320(V) pixels, which is 16 times that of high definition (HD) TV [8, 9]. Developments of elemental technology of SHV, such as camera, display, audio, compression coding, and transmission techniques, are advancing with the aim of improving the next generation of HDTV. To enhance the quality of integral 3D images, we used the SHV video technology for capturing and displaying the elemental images. We have developed the integral 3D TV system that has a dual-green diagonal-pixel-offset method SHV camera and projectors [10, 11]. Figure 4 shows motion parallax of the reconstructed 3D image. The size of the 3D image is 21.6 inches, and the number of elemental lenses in the lens array is 182(H) × 140(V). The measured maximum viewing angle was also expanded to 24.5 degrees, which is nearly double that obtained with our previous prototype system.
Fig. 4 Motion parallax of produced 3D images
In future work, we will study how to enhance the resolution and quality of the 3D image by improving the number of elemental lenses and the resolution of the elemental image.
We thank JVC KENWOOD Holdings, Inc. for developing the SHV projector. Part of this research has been supported by the Japanese National Institute of Information and Communications Technology (program period; 2006-2010).
 T. Okoshi, “Three-Dimensional Imaging Techniques,” Academic, New York, 1971.
 M. G. Lippmann, “Epreuves reversibles donnant la sensation du relief,” J. Phys. (Paris) (4th series) 7, 821-825, 1908.
 F. Okano, H. Hoshino, J. Arai, and I. Yuyama, “Real-time pickup method for a three-dimensional image based on integral photography,” Appl. Opt., 36(7), 1598-1603, 1997.
 H. Hoshino, F. Okano, H. Isono, and I. Yuyama, “Analysis of resolution limitation of integral photography,” J. Opt. Soc. Am. A, 15, 2059-2065, 1998.
 J. Arai, F. Okano, H. Hoshino, and I. Yuyama, “Gradient-index lens array method based on real-time integral photography for three-dimensional images,” Appl. Opt., 37(11), 2034-2045, 1998.
 J. Arai, M. Okui, T. Yamashita, and F. Okano “Integral three-dimensional television using a 2000-scanning video system,” Appl. Opt., 45(8), 1704-1712, 2006.
 F. Okano, M. Kawakita, J. Arai, H. Sasaki, T. Yamashita, M. Sato, K. Suehiro, and Y. Haino, “Three-dimensional integral television using extremely high-resolution video system with 4,000 scanning lines,” Proceedings of SPIE, 6778-23, 677805.1-677805.13, 2007.
 M. Kanazawa, Y. Kusakabe, Y. Nojiri, Y. Haino, M. Sato, K. Doi, “Super Hi-Vision projection display for next generation TV,” Journal of the Society for Information Display, 15, 837-843, 2007.
 T. Yamashita, S. Huang, R. Funatsu, B. Mansoorian, K. Mitani, Y. Nojiri, “Experimental color video capturing equipment with three 33-megapixel CMOS image sensors,” SPIE, Bellingham, 7249-18, 72490H.1-72490H.10, 2009.
 K. Suehiro, M. Yoshimura, Y. Haino, M. Sato, J. Arai, M. Kawakita, and F. Okano, “Integral 3D TV using ultrahigh-definition D-ILA device” IS&T / SPIE’s 20th Annual Symposium, Electronic Imaging 2008, 6803-43-1 - 6803-43-12, 2008.
 M. Kawakita, H. Sasaki, J. Arai, F. Okano, K. Suehiro, Y. Haino, M. Yoshimura, and M. Sato, “Improvements of 3-D image quality in integral display by reducing distortion errors,” Proceedings of SPIE, pp. 6803-41-1 - 6803-41-11, 2008.
Masahiro Kawakita received his B.S. and M.S. degrees in physics from Kyushu University, Ph.D. degree in electronic engineering from Tokyo University, Japan, in 1988, 1990 and 2005, respectively. In 1990, he joined Japan Broadcasting Corporation (NHK), Tokyo. Since 1993, he has been at Science & Technology Research Laboratories of NHK, where he has been engaged in research on applications of liquid crystal devices and optically addressed spatial modulators, three-dimensional TV camera and display system.
Jun Arai received the B.S., M.S., and Ph.D. degrees in applied physics from Waseda University, Tokyo, Japan, in 1993, 1995, and 2005, respectively. In 1995, he joined the Science and Technology Research Laboratories, Japan Broadcasting Corporation (NHK), Tokyo. Since then, he has been working on the three-dimensional imaging system.
Hisayuki Sasaki received his B.S. degree in engineering systems and M.S. degree in engineering mechanics from University of Tsukuba, Ibaraki, Japan, in 1999 and 2001, respectively. He joined the Japan Broadcasting Corporation (NHK) in 2001 and worked at the Akita Broadcasting Station. Since 2006, he has been engaged in research on 3-D television systems including integral photography and extremely high resolution imagery at the NHK Science & Technology Research Laboratories. His research interests are areas of 3-D imaging systems, 3-D image processing and virtual reality.
Hitoshi Hiura received his B.S. and M.S. degrees from Toyohashi University of Technology, Ph.D. degree from Tokushima University, Japan, in 2005, 2007 and 2010, respectively. In 2007, he joined the Science and Technology Research Laboratories, Japan Broadcasting Corporation (NHK), Tokyo. Since then, he has been working on the three-dimensional imaging system.
Masato Miura received the B.S., M.S. and Ph.D. degrees in Computer and Systems Engineering from Kobe University, Hyogo, Japan, in 2004, 2005, and 2008, respectively. Since 2008, he has been with the Science & Technology Research Laboratories, Japan Broadcasting Corporation (NHK), Tokyo, Japan. His research interests are the areas of three-dimensional (3-D) television systems and 3-D image processing.
Makoto Okui received the B.S., M.E., and D.E degrees from Tokyo Institute of Technology, Tokyo, Japan, in 1978, 1980, and 2006, respectively. Since 1983, he has been with NHK Science and Technology Research Laboratories, Tokyo, Japan. From 1988 to 1995, he was involved in the development of various enhanced television systems, including high-definition television (HDTV) systems for terrestrial broadcasting. During 2006-2009, he worked for National Institute of Information and Communications Technology (NICT) of Japan as a Group Leader of 3-D Spatial Imaging and Sound Group. He is currently a Senior Research Engineer of the Advanced Television Systems Research Division at NHK Science and Technology Research Laboratories. His current research interests include 3-D display and 3-D television.
Fumio Okano (Senior Member, IEEE) received the B.S., M.S., and Ph.D. degrees in electrical engineering from Tohoku University, Sendai, Japan, in 1976, 1978, and 1996, respectively. He joined Japan Broadcasting Corporation (NHK), Tokyo, in 1978. Since 1981, he has been engaged in research on high-definition television (HDTV) cameras, HDTV systems, 3-D television, and extremely high resolution imagery at NHK Science and Technology Research Laboratories. Dr. Okano is a member of the Optical Society of America (OSA), and the Society of Photo-Optical Instrumentation Engineers (SPIE).
Back to E-magazine List