Image processing device, and image processing method, and program
| Title: | Image processing device, and image processing method, and program |
|---|---|
| Patent Number: | 9,532,028 |
| Publication Date: | December 27, 2016 |
| Appl. No: | 14/233195 |
| Application Filed: | June 21, 2012 |
| Abstract: | A plurality of viewpoint images are acquired and images for three-dimensional image display are generated without causing a reduction in polarization performance. Provided are: a first polarization means which has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough; a second polarization means which is made up of a third polarization region that allows only the transmitted light of either one of the first polarization region or the second polarization region to be transmitted therethrough, and a total transmission region that allows the transmitted light of the first polarization region and the second polarization region to be transmitted therethrough; an imaging element into which the transmitted light of the second polarization means is input; and an image processing unit which executes signal processing for an output signal of the imaging element. The image processing unit generates parallax information in which subject distance is reflected by applying a first viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means, and a second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means, and generates a left-eye-purpose image and a right-eye-purpose image for three-dimensional image display by means of image conversion processing of the first viewpoint image for which the parallax information is applied. |
| Inventors: | Kobayashi, Seiji (Tokyo, JP); Ito, Atsushi (Kanagawa, JP) |
| Assignees: | SONY CORPORATION (Tokyo, JP) |
| Claim: | 1. An image processing device comprising: a first polarization means which has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough; a second polarization means into which the transmitted light of the first polarization means enters, and which is made up of a third polarization region that allows only the transmitted light of either one of the first polarization region or the second polarization region to be transmitted therethrough, and a total transmission region that allows the transmitted light of the first polarization region and the second polarization region to be transmitted therethrough; an imaging element into which the transmitted light of the second polarization means is input; and an image processing unit which executes signal processing for an output signal of the imaging element, wherein the image processing unit generates parallax information in which subject distance is reflected by applying a first viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means, and a second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means, and generates a left-eye-purpose image and a right-eye-purpose image for three-dimensional image display by means of image conversion processing of the first viewpoint image for which the parallax information is applied. |
| Claim: | 2. The image processing device according to claim 1 , wherein the first viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means is a normal image in which the center location of the first polarization means serves as a viewpoint, the second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means is a right-eye-purpose image or a left-eye-purpose image in which either one of the gravity center locations of the first polarization region, or the second polarization region of the first polarization means serves as the viewpoint, and The image processing unit: applies the normal image that is the first viewpoint image and a viewpoint image of either the right-eye-purpose image or the left-eye-purpose image that is the second viewpoint image to generate a left-eye-purpose image or a right-eye-purpose image as a third viewpoint image that is different from the second viewpoint image, and applies the second viewpoint image and the third viewpoint image to generate the parallax information. |
| Claim: | 3. The image processing device according to claim 2 , wherein the image processing unit generates the left-eye-purpose image or the right-eye-purpose image as the third viewpoint image that is different from the second viewpoint image, by means of processing in which a pixel value of the right-eye-purpose image or the left-eye-purpose image that is the second viewpoint image is subtracted from a pixel value of the normal image that is the first viewpoint image. |
| Claim: | 4. The image processing device according to claim 1 , wherein the parallax information generated by the image processing unit is a depth map in which subject distance information of an image is reflected, and the image processing unit generates the left-eye-purpose image and the right-eye-purpose image for the three-dimensional image display by means of the image conversion processing of the first viewpoint image in which the depth map is applied. |
| Claim: | 5. The image processing device according to claim 1 , wherein the third polarization region and the total transmission region of the second polarization means are configurations that are repeatedly set in multiple-pixel units in a first direction that is one direction of an imaging element plane. |
| Claim: | 6. The image processing device according to claim 5 , wherein the third polarization region of the second polarization means is configured of a wire grid polarizer, and is a configuration having wires that extend in a second direction perpendicular to the first direction of the imaging element. |
| Claim: | 7. The image processing device according to claim 1 , wherein the image processing unit executes brightness adjustment with which the brightness level of a viewpoint image of either the right-eye-purpose image or the left-eye-purpose image that is the second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means is made to match the brightness level of the first viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means, and applies the second viewpoint image subsequent to the brightness adjustment to generate the parallax information. |
| Claim: | 8. The image processing device according to claim 1 , wherein the image processing unit includes an interpolation processing unit which interpolates pixel values of pixel locations corresponding to the third polarization region that are not included in an image that is based on imaging element output from the pixel locations of the imaging element corresponding to the total transmission region of the second polarization means, and a demosaic processing unit which sets the pixel values for colors to the pixel locations by means of demosaic processing of an interpolated image generated by the interpolation processing unit and generates the first viewpoint image, and the image processing unit executes the image conversion processing in which the parallax information is applied, with respect to the first viewpoint image generated by the demosaic processing unit, and generates the left-eye-purpose image and the right-eye-purpose image for the three-dimensional image display. |
| Claim: | 9. The image processing device according to claim 1 , wherein the image conversion processing executed by the image processing unit is 2D/3D conversion processing with which the left-eye-purpose image and the right-eye-purpose image for the three-dimensional image display are generated by the means of image conversion processing in which the parallax information is applied with respect to the first viewpoint image, which is a two-dimensional (2D) image. |
| Claim: | 10. The image processing device according to claim 1 , wherein the image conversion processing which is executed by the image processing unit and in which the parallax information is applied is 2D/3D conversion processing with which the left-eye-purpose image and the right-eye-purpose image for the three-dimensional image display are generated by carrying out image shift processing corresponding to the subject distance with respect to the first viewpoint image. |
| Claim: | 11. An image processing device which executes image processing for an image captured by an imaging means, wherein the imaging means is a configuration which captures an image that is based on the transmitted light of a first polarization means that has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough, the image processing device inputs a first viewpoint image made up of all of the transmitted light of the first polarization region and the second polarization region, and a second viewpoint image made up of only the transmitted light of either one of the first polarization region or the second polarization region, generates parallax information in which subject distance is reflected by applying the first viewpoint image and the second viewpoint image, and generates a left-eye-purpose image and a right-eye-purpose image for three-dimensional image display by means of image conversion processing of the first viewpoint image for which the parallax information is applied. |
| Claim: | 12. An image processing device comprising: a first polarization means which has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough; a second polarization means into which the transmitted light of the first polarization means enters, and which is made up of a third polarization region that allows only the transmitted light of the first polarization region to be transmitted therethrough, a fourth polarization region that allows only the transmitted light of the second polarization region to be transmitted therethrough, and a total transmission region that allows the transmitted light of the first polarization region and the second polarization region to be transmitted therethrough; an imaging element into which the transmitted light of the second polarization means is input; and an image processing unit which executes signal processing for an output signal of the imaging element, wherein the image processing unit applies a viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means, and a viewpoint image that is based on the transmitted light of the fourth polarization region to generate parallax information in which subject distance is reflected, and applies the parallax information to generate a left-eye-purpose image and a right-eye-purpose image for three-dimensional image display by means of image conversion processing of a viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means. |
| Claim: | 13. An image processing method for executing, in an image processing device, image processing for an image captured by an imaging means, wherein the imaging means is a configuration which captures an image that is based on transmitted light of a first polarization means that has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough, and in the image processing method, in an image processing unit, a first viewpoint image made up of all of the transmitted light of the first polarization region and the second polarization region, and a second viewpoint image made up of only the transmitted light of either one of the first polarization region or the second polarization region are input, parallax information in which subject distance is reflected by applying the first viewpoint image and the second viewpoint image is generated, and a left-eye-purpose image and a right-eye-purpose image for three-dimensional image display are generated by means of image conversion processing of the first viewpoint image for which the parallax information is applied. |
| Claim: | 14. A non-transitory computer-readable medium having instructions stored therein, a set of computer-executable instructions for causing a computer to perform steps comprising capturing an image that is based on the transmitted light of a first polarization means that has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough, wherein a first viewpoint image made up of all of the transmitted light of the first polarization region and the second polarization region, and a second viewpoint image made up of only the transmitted light of either one of the first polarization region or the second polarization region to be input, parallax information in which subject distance is reflected by applying the first viewpoint image and the second viewpoint image to be generated, and a left-eye-purpose image and a right-eye-purpose image for three-dimensional image display to be generated by means of image conversion processing of the first viewpoint image for which the parallax information is applied. |
| Patent References Cited: | 2005/0089212 April 2005 Mashitani 2010/0321777 December 2010 Martinez 06-054991 July 1994 2004-228629 August 2004 2004-309868 November 2004 2011-035853 February 2011 WO 2009/147814 December 2009 |
| Primary Examiner: | Sun, Yulin |
| Attorney, Agent or Firm: | Chip Law Group |
| Accession Number: | edspgr.09532028 |
| Database: | USPTO Patent Grants |
| Language: | English |
|---|