Compound eye camera

Description: TECHNICAL FIELD [Detailed description of the invention]

【0001】

[Technical field to which the invention belongs]

The present invention relates to a camera capable of capturing and displaying a stereoscopic image and a two-dimensional image.

【0002】

[Conventional technology]

When shooting and displaying a stereoscopic image, a system considered so far includes, for example, a stereoscopic television device disclosed in Japanese Patent Application Laid-Open No. 62-21396. In such a stereoscopic image shooting / display system, basically, a set of images having parallax is obtained from a plurality of cameras, and the stereoscopic image is provided to the photographer by a stereoscopic image display device dedicated to the system. Is. Typical stereoscopic image display methods, including those mentioned in this example, are as follows.

【0003】

(1) As a method of a stereoscopic image display device, there is a method in which the left and right images are separated by using polarized glasses with different polarization states for the right eye image and the left eye image. A liquid crystal shutter is provided on the display side to make the polarization state different, the polarization state is switched by synchronizing with the field signal of the display image, and the observer wearing polarized glasses separates the left and right images one eye at a time. This is a method that enables stereoscopic viewing.

【0004】

In addition, instead of the method of having a liquid crystal shutter on the display side, there is also a liquid crystal shutter glasses method in which the glasses are provided with a liquid crystal shutter and the glasses are turned on / off in synchronization with the display of the left and right images.

【0005】

(2) Further, in a stereoscopic image display device that does not use polarized glasses, there is a method in which a lenticular lens is provided on the front surface of the display to spatially separate images that enter the left and right eyes. FIG. 6A is an explanatory view of a conventional example of the method using a lenticular lens, and shows a cross-sectional view from above the observer's head. In FIG. 6, 60 indicates a display pixel of a liquid crystal display, and a glass substrate, a color filter, an electrode, a polarizing plate, a backlight, and the like are omitted. The display pixel unit 60 includes an opening 61 in which a color filter forming a pixel is arranged and a black matrix 62 that separates the pixels. The openings are arranged as shown in FIG. 6 (b). On the surface of the liquid crystal display, a lenticular lens 63 having a semicircular cross section as shown and each extending in a direction perpendicular to the paper surface is provided, and the display pixel portion 60 of the liquid crystal display is located on the focal plane thereof. It is designed to do. As shown in the figure, the image for the right eye (R) and the image for the left eye (L) are alternately arranged in a striped pair on the display pixel unit 60 corresponding to one pitch of the lenticular lens 63. The lenticular lens 63 optically separates and forms an image on the observer's right eye Er and left eye El, enabling stereoscopic vision. In the figure, the spatial area where each of the images for the right eye and the left eye can be observed by the lenticular lens in the center of the display is shown, and the spatial areas separated to the left and right are similarly separated for each of the other lenticular lenses. The left and right images overlap at the positions of the left and right eyes of the observer, and the left and right images are uniformly separated and observed over the entire screen. In this method, the striped images arranged alternately from the two parallax images must be combined and displayed, so that the horizontal resolution of the image display device is halved.

【0006】

(3) Further, there is a rear barrier wrench method as a method of (IAF1139) that the present inventors have already applied for. As shown in FIG. 5, in the display element 330 capable of three-dimensional display, 10 is a display pixel portion composed of a liquid crystal layer or the like, and is formed between the glass substrates 11. 5 is a backlight that serves as an illumination light source. In front of it, a mask substrate 6 having a mask pattern having a checkered opening through which light is transmitted is arranged. The mask pattern is made of a metal vapor deposition film such as chromium or a light absorbing material, and is manufactured by patterning on a mask substrate made of glass or resin. Two lenticular lenses 7 and 8 made of transparent resin or glass that are orthogonal to each other as microlenses are arranged between the mask substrate 6 and the liquid crystal display 10 for image display. Further, a polymer-dispersed liquid crystal 9 is arranged between the lenticular lens and the image display liquid crystal display 10.

【0007】

The left and right images are alternately arranged in a horizontal stripe in the vertical direction on the image display liquid crystal display 10.

【0008】

The light from the backlight 5 passes through each aperture of the mask substrate 6, passes through the lenticular lenses 7 and 8, and then passes through the polymer-dispersed liquid crystal 9 to illuminate the image liquid crystal display 10 and is used by the photographer. The previous image is separately observed as a left-right differential image in both eyes.

【0009】

This makes it possible for the photographer to observe a three-dimensional image. At this time, an electric field is applied to the polymer-dispersed liquid crystal 9, and the light directed by the mask 6 and the lenticular lenses 7 and 8 keeps its directivity, that is, the image is separated into both eyes of the observer. The liquid crystal display 10 for images is illuminated so as to be observed.

【0010】

[Problems to be Solved by the Invention]

Although the above three stereoscopic image display devices have been described above, any of these devices is provided with a drive circuit corresponding to any of the devices when connecting to the imaging unit to form an integrated compound eye camera. There was no compound-eye camera with a built-in display that produced prompt output to the display device of.

【0011】

An object of the present invention is to solve the above-mentioned problems in a stereoscopic image capturing display device, to provide a stereoscopic image display drive circuit capable of responding to any stereoscopic display device connected, and to provide a stereoscopic image captured on this display unit. It is a system that makes it possible to display. As a result, the user can always observe the stereoscopic image during shooting using the stereoscopic display desired by the user, and the stereoscopic effect can be adjusted while shooting. It is to supply a stereoscopic image system that enables reproduction. Further, this system enables recording and reproduction of not only a stereoscopic image system but also other modes such as a panoramic image.

【0012】

[Means for solving problems]

The camera of the present invention includes a plurality of imaging means and a stereoscopic display drive circuit corresponding to a plurality of stereoscopic displays. Further, the camera of the present invention includes a plurality of imaging means, a stereoscopic display drive circuit corresponding to the plurality of stereoscopic displays, and means for determining the type of the connected stereoscopic display.

【0013】

[Usage] According to the first and second inventions according to the present application, it is possible to take a stereoscopic image and observe a stereoscopic image on various stereoscopic displays, and to take a picture while confirming a stereoscopic effect.

【0014】

BEST MODE FOR CARRYING OUT THE INVENTION

[First Example] Hereinafter, examples of the present invention will be described with reference to the drawings. FIG. 2 is a block diagram showing the configuration inside the camera when capturing a stereoscopic image. Taking the case where a rear barrier wrench type object is connected as a stereoscopic display as an example, the signal flow and the processing flow will be described with reference to FIG.

【0015】

In FIG. 2, 321, 322 are CCDs, 323 are CCD vertical drivers, 324 and 325 are CDS / AGC circuits, 342 and 343 are clamp circuits, 326 and 327 are A / D converters, 320 are timing generators, 328, 329 is a color processing circuit, 334 is a signal processing circuit, 332 is VRAM, and 333 is a memory. The 335 is a compression / decompression circuit that performs, for example, JPEG compression. Reference numeral 341 is a digital interface, such as USB. 340 is a storage medium. Here, for example, a flash memory is used. 338 is an MPU, 339 is a working memory, 337 is a camera control unit, and 360 is a display management unit.

【0016】

First, when the operator inputs an operation such as video recording / playback to the camera control unit 337, a signal for this human power is sent from the camera control unit 337 to the MPU 338, and the MPU 338 controls each unit. Here, it is assumed that the stereoscopic image shooting mode is selected. Images taken by two imaging optical systems (not shown) are imaged on the image sensors of CCDs 321, 322. The video is photoelectrically converted by the CCDs 321,322, and both images are converted into digital signals by the A / D converters 326 and 327 via the CDS / AGC circuits 324 and 325 and the clamp circuits 342 and 343 in the next stage. At this time, since the left and right image signals are driven and processed synchronously under the control of the CCD vertical driver 323 and the timing generator 320, the images captured at the same time on the left and right are processed. The CCDs 321,322 have both a frame storage mode and a field storage mode. Here, the frame storage mode is taken as an example, and the image on the CCD in which the frames are stored is taken as an example of a progressive scan (a method of reading out one line at a time). explain. The left and right images converted into digital signals by the A / D converters 326 and 327 are sent to the respective color processing circuits 328 and 329. In the color processing circuits 328 and 329, the digital signal is subjected to color conversion processing and the like.

【0017】

The left and right digital signals that have undergone color conversion are input to the signal processing circuit 334, and when the rear barrier type stereoscopic display 330 is connected, the display management unit 360 transmits the information to the signal processing circuit, and the rear barrier type stereoscopic display 330 is connected. It is converted to the pixel size suitable for the display, and the left and right images are alternately combined one line at the top and one line at the top and transferred to VRAM332.

【0018】

At the same time, the image data is also stored in the process memory 333. The signal processing circuit 334 performs bidirectional control in this way.

【0019】

At this point, the signals captured by the CCD are held as images in the process memory 333 and in the VRAM 332, respectively.

【0020】

The contents of VRAM332 are used to generate a stereoscopic image signal on the liquid crystal display 330 in the compound eye camera. This VRAM332 is a memory for display and has a capacity larger than the capacity of the image to be displayed on the liquid crystal display 330. ..

【0021】

Since the number of pixels of the image held in the process memory 333 and the number of pixels of the display image of the liquid crystal display 330 are not always the same, the signal processing circuit 334 is provided with a function of thinning out or interpolating the same.

【0022】

The right and left images written in the VRAM 332 are alternately displayed on the liquid crystal display 330 via the liquid crystal display control circuit 331 and the display management unit 360 for each scanning line. This allows the observer to observe the stereoscopic image.

【0023】

Next, a means for determining which type of connected display is used will be described with reference to FIGS. 1 and 2. A part of Fig. 2 corresponds to Fig. 1. In FIG. 1, 360 is a display management unit, 330 is a rear burr display, 370 is a wrench display, 380 is a display device for liquid crystal shutter glasses, 381 is a liquid crystal shutter glasses, 331 is a rear barrier LCD control circuit, and 332 is a wrench display. Output circuit, 383 is the output circuit for the liquid crystal shutter glasses system, 332 is the VRAM, 334 is the signal processing circuit, 333 is the memory, 384 is the connector, and Fig. 2 is the rear-variable display 330 shown in Fig. 1. It shows the time.

【0024】

The user connects the display he / she wants to use with the imaging block and the connector 384. After being connected, the display management unit 360 determines which type of display is connected and sends the information to the signal processing circuit 334. When the display connector 384 is connected, the switch inside the connector that turns on when the connector is connected works to notify the display management unit 360 of the means for determining the display type. This determines the display.

【0025】

Further, when the connector 384 is connected, the display management unit 360 and the display may communicate with each other, and the display may be discriminated by sending and receiving a signal indicating the type of the display. For example, the information of the display is encoded, and the packet of the type signal is transmitted from the display according to the packet request from the display management unit 360.

【0026】

In addition, the user may directly select the connected display by the UI connected to the camera control unit 337.

【0027】

The signal processing circuit 334, which knows what kind it is, generates a desired image and transmits it to the VRAM 332. The transmitted image is sent to the display management unit 360 by an output circuit according to the type, here, 331, 382, or 383. The display management unit 360 sends this image to the display device. Needless to say, in the case of the liquid crystal shutter glasses system, the synchronization signal for driving the glasses is also output. In this way, it is possible to output a desired output to the display device used by the user.

【0028】

Next, Fig. 4 schematically shows the generation of a composite image when the connected display uses the rear barrier wrench method.

【0029】

In FIG. 4, 40 and 41 are images captured by a CCD, 42 and 43 are images compressed in half lengthwise and widthwise, and 44 are interlaced composite images. The number of effective pixels of the CCD is 640 * 240 (1 field), and the number of display pixels of the LCD is 320 * 240.

【0030】

The left and right images imaged on the CCD by the imaging optical system become digital signals as described above, are color-converted, and are 640 * 240 as effective pixels as shown in 40 and 41, respectively. (L0, L1, ... L239, R0, R1, R239 for each line) This signal is held in the process memory 333 as it is through the signal processing circuit 334, and the signal processing circuit 334. First, the left and right images 40 and 41 are converted into 320 * 240 size images 42 and 43 according to the LCD size. (L0', L1'... L239', R0', R1'... R239'for each line) This conversion may be simple decimation or interpolation.

【0031】

Next, the left and right images 42 and 43 converted to 320 * 240 are combined alternately for each line as shown in 44, such as L0', R0', L2', R2'... R238'. To. The combined image is written to VRAM332. Further, when the operator selects the stereoscopic image shooting mode via the camera control unit 337, the mode is transmitted to the rear barrier type LCD control circuit 331, and an electric field is applied to the polymer dispersed liquid crystal 9.

【0032】

That is, the rear barrier type LCD control circuit 331 outputs two signals, a video signal to be displayed and a signal for controlling the polymer-dispersed liquid crystal 9. As described above, it is possible to observe three-dimensionally by this.

【0033】

Next, image recording will be described. A magnetic tape, a magnetic disk, an optical disk, a semiconductor memory, or the like can be used as the type of recording medium, but here, a flash memory will be described as an example.

【0034】

The interface to the recording medium saves the stereoscopic video signal as a file in the free area of the recording medium 340 in the digital format, and also registers it in the file management area. This starts when the photographer manually operates the desired operation to the camera control unit 337 to start recording, and when the desired shooting is identified by the MPU 338, the contents of the process memory 333 are transmitted via the signal processing circuit 334. The information sent to the compression circuit 335 is compressed, and the compressed data is held in the work memory 339. Here, JPEG is used as an example of compression.

【0035】

The compressed data is stored in the work memory 339, and for example, a file name such as s001L.jpg or s001R.jpg is given to the compressed data, and the left and right compressed images are recorded as a pair for file management. At this time, the identification information for identifying the pair is also recorded in the file management area at the same time. Further, a thumbnail image is recorded together with the above-mentioned main image.

【0036】

Here, the thumbnail image refers to an image reduced with respect to the above image, for example, an image having a size of 80 * 60. This thumbnail image, like the compressed image creation of this image, reduces the contents of the process memory 333 to a size of 80 * 60 once via the signal processing circuit 334, and then sends it to the compression circuit 335 to compress the information. And the compressed data is held in the work memory 339. Here, too, JPEG is used as an example of compression.

【0037】

The compressed data is stored in the work memory 339, and for example, a file name such as ss001L.jpg or ss001R.jpg is given to the compressed data, and the left and right thumbnail compressed images are recorded as a pair for file management. At this time, the identification information for identifying the pair is also recorded in the file management area at the same time as in this image.

【0038】

The above is the flow of stereoscopic image recording, and the camera user can observe the stereoscopic image on the display and perform the recording operation only when desired. As a result, the degree of freedom during shooting is large, and the stereoscopic effect can be confirmed even when the camera is moved while shooting.

【0039】

Next, reproduction of the stereoscopic image recorded on the recording medium 340 will be described. Since a plurality of file stereoscopic images are recorded in the medium, the management area of the recording medium 340 is checked and the image file registration data is sent to the MPU 338.

【0040】

In this case, the MPU338 selects an image file that can be played back as a stereo, arranges the corresponding image file name data in an arbitrary display format, reads the image file equivalent to the thumbnail image from the recording medium 340, and works memory. Hold at 339. Since the thumbnail images held in the work memory 339 are JPEG-compressed, nine thumbnail images are selected, sent as data to the signal processing circuit 334, and displayed on the stereoscopic display as shown in FIG. At this time, the stereoscopic display is in the two-dimensional display mode, and the thumbnail image simultaneously displays the flag information (S in the figure) which means the stereoscopic image.

【0041】

In FIG. 3, 100 is a flag indicating a thumbnail image, and S is a flag indicating a stereoscopic image. The operator selects the image file to be played back from the displayed thumbnail image and manually inputs it to the camera control unit 337. The input signal is sent from the camera control unit 337 to the MPU 338, the data of the selected file is read from the recording medium 340, and transferred to the work memory 339. After that, the information in the work memory is decompressed via the compression / decompression circuit 335 and sent to the process memory 333. After that, as described above, the size is converted to VRAM332, interlaced and composited, and displayed on the display as a stereoscopic image.

【0042】

In this way, the captured stereoscopic image can be easily reproduced. Further, by arranging a microphone (not shown) together with each imaging optical system, a more three-dimensional effect can be obtained not only for video but also for audio.

【0043】

[Second Example] Next, a case where the display device for liquid crystal shutter glasses is connected instead of the rear barrier wrench type display 330 will be described. Since the place where the image signal is obtained from the imaging system is the same as described above, only the part that generates the stereoscopic image will be described.

【0044】

The left and right digital signals that have undergone color conversion are input to the signal processing circuit 334, and when the display device 380 for liquid crystal shutter glasses is connected, the display management unit 360 transmits the information to the signal processing circuit 334, and the liquid crystal shutter. It is converted to the pixel size suitable for a glasses-type three-dimensional display, and the left and right images are alternately combined for each field and transferred to VRAM332.

【0045】

Since the number of pixels of the display image of the liquid crystal shutter glasses type stereoscopic display 380 is not always the same, the signal processing circuit 334 is provided with a function of thinning out or interpolating the number of pixels. The right image and the left image written in the VRAM 332 for each field are alternately displayed for each field on the stereoscopic display 380 via the liquid crystal shutter glasses type output circuit 383 and further via the display management unit 360. At this time, a synchronization signal for opening and closing the shutter of the glasses is also output. This allows the observer to observe the stereoscopic image.

【0046】

[Third Example] Next, a case where a rear barrier wrench type display 330 and a display device for liquid crystal shutter glasses are connected at the same time will be described.

【0047】

Each display method is described in the first embodiment and the second embodiment described above, but in the case of the rear barrier wrench method, the left and right images come to each scanning line in the image stored in VRAM332. On the other hand, in the case of the liquid crystal shutter glasses method, the left and right images come for each field. Therefore, in order to support both modes, the capacity of VRAM332 is prepared in advance to cover both modes. A video signal is sent from the VRAM 332 corresponding to both methods to the respective wrench type display output device 382 and the liquid crystal shutter glasses type output circuit 383, and this video signal is further transmitted to each display via the display management unit 360. .. This makes it possible to handle when both types are connected.

【0048】

[Fourth Example] In the third embodiment, an example in which VRAM332 is sufficiently prepared and corresponding is described, but VRAM332 may not always be sufficiently obtained due to the system configuration. For this reason, when two displays are selected, the display management unit may decide which display to output according to the set priority and output the two displays.

【0049】

For example, when the output for the liquid crystal shutter is output, the output of the rear barrier wrench method is stopped. And so on. This has another advantage of reducing power consumption. This priority setting may be fixed or user setting.

【0050】

[Effect of the invention]

As described above, according to the first and second inventions according to the present application, a stereoscopic image display drive circuit capable of supporting any stereoscopic display device connected in the stereoscopic image capturing display device is provided, and the display unit is provided with a stereoscopic image display drive circuit. , It is possible to make a system that can display the captured stereoscopic image, so that the stereoscopic image can always be observed during shooting using the stereoscopic display desired by the user, and the stereoscopic effect can be adjusted while shooting. It is possible to supply a stereoscopic image system that enables the image to be reproduced by a desired display device at the time of reproduction even after shooting.

【0051】

Further, this system enables recording and reproduction of not only a stereoscopic image system but also other modes such as a panoramic image. In addition, after recording, a thumbnail image is displayed on a stereoscopic display to enable an overview of the recorded image.

[Simple explanation of drawings]

[Figure 1]

Overview diagram showing the function of the display management unit of the present invention [Figure 2]

The figure which shows the structure of the compound eye camera in this Example [Fig. 3]

Diagram when viewing the image list in this embodiment [Fig. 4]

Diagram showing the concept of 3D image generation by the rear barrier wrench method [Fig. 5]

Explanatory drawing of the rear barrier wrench type display [Fig. 6]

Block diagram showing the prior art [Explanation of symbols]

5 Backlight that serves as a lighting source 6 Checkered open lomask substrate 7, 8 Wrench Killer Lens 9 Polymer-dispersed liquid crystal 10 Display pixel part consisting of liquid crystal layer, etc. 11 Glass substrate Images captured by 40, 41 CCD 42, 43 Image compressed to 1/2 in height and width 44 Interlaced composite image 60 LCD display pixels 61 opening 62 Black matrix that separates pixels 63 Wrench Killer Lens 320 Timing Generator 321, 322 CCD 323 CCD vertical driver 324, 325 CDS AGC circuit 326, 327 A / D converter 328, 329 color processing circuit 330 Rear burr display 331 Rear barrier LCD control circuit 332 VRAM 333 memory 334 Signal processing circuit 335 compression / intermediate circuit 337 Camera control unit 338 MPU 339 working memory 340 Storage medium 341 Digital interface 342, 343 Clamp circuit 360 Display Management Department 370 wrench display 380 Display device for LCD shutter glasses 381 LCD shutter glasses 382 Wrench display output circuit 383 Output circuit for LCD shutter glasses method 384 connector Er Observer's right eye El left eye