Image pickup apparatus

The present invention relates to an image pickup device to which a solid-state image pickup element such as a digital color camera or a video movie camera is applied.

[0002] In a digital color camera, in response to pressing a release button, a field image is exposed to a solid-state image sensor such as a CCD or CMOS sensor for a desired time, and one obtained from this. An image signal representing a still image of a screen is converted into a digital signal and subjected to a predetermined process such as a YC process to obtain an image signal of a predetermined format. A digital image signal representing the captured image is recorded in the semiconductor memory for each image. The recorded image signal is read out at any time, reproduced as a displayable or printable signal, and output to a monitor or the like for display.

[0003] Conventionally, in a digital color camera, focus detection is performed by using an image pickup device. Here, the focus detection of the contrast detection method is used. The focus adjustment of the contrast detection method is to obtain the sharpness of the object image formed by the imaging optical system by evaluating the output of the solid-state image sensor with a predetermined function, and to take the extreme value of the function value of the imaging optical system. It adjusts the position on the optical axis. Evaluation functions include those that add the absolute value of the difference between adjacent luminance signals within the focal detection region, those that add the square of the difference between adjacent luminance signals within the focal detection region, or R, G, B. There is one that processes the difference between adjacent signals for each pixel signal of the above in the same manner.

[0004] In general, in the focus detection of such a contrast detection method, in order to obtain the evaluation function value while slightly moving the position on the optical axis of the imaging optical system, it is considerably necessary to adjust the focus until focusing. There is a problem that it takes time.

[0005] Further, as disclosed in USP4410804, a pair or two pairs of light receiving parts are provided for each microlens array in which a pair or two pairs of light receiving parts are arranged two-dimensionally, and the light receiving parts are projected onto the pupil of the imaging optical system by the microlens. There is also an imaging device incorporating a so-called phase difference detection type focus detection device that divides the pupil. What is the phase difference detection method? An object image is formed using two luminous fluxes that have passed through different parts of the pupil of the imaging optical system, and the positional phase difference between the two object images is detected based on the output of the image sensor. This is converted into the defocus amount of the image pickup optical system.

[0006] Since the defocus amount can be obtained in the focus detection of the phase difference detection method, there is an advantage that the time required for focusing can be significantly shortened as compared with the contrast detection method.

[0007] However, in the solid-state image sensor having the structure disclosed in USP4410804, a pair or two pairs formed by a light flux passing through a part of the pupil of the image pickup optical system. Since the images are output, when these are evaluated from the viewpoint of the image pickup performance of the object image, they are said to be extremely low quality images. This is because an unnatural image blur occurs due to the eccentric luminous flux, especially in an out-of-focus background or foreground.

[0008] Therefore, an object of the present invention is to provide an imaging device capable of performing adjustments for imaging at high speed and obtaining high-quality image output.

[Means for Solving the Problems] In order to solve the above problems, as one means of the present invention, each is a first photoelectric conversion unit that photoelectrically converts a light flux from a first region of an ejection pupil. A second photoelectric conversion unit that photoelectrically converts a light flux from the second region of the ejection pupil, and an amplification means commonly provided in the first photoelectric conversion unit and the second photoelectric conversion unit. A plurality of pixels including a first transfer means for transferring the signal of the first photoelectric conversion unit to the amplification means and a second transfer means for transferring the signal of the second photoelectric conversion unit to the amplification means. The first operation of mixing the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit at the input unit of the amplification means and outputting the mixed signal from the amplification means, and the first operation. An imaging apparatus comprising: a driving means for controlling a second operation of selectively outputting a signal of a photoelectric conversion unit 1 and a signal of the second photoelectric conversion unit from the amplification means. provide.

[Embodiment of the Invention] (First Embodiment) First, the first embodiment will be described.

[0017] FIG. 1 is a configuration diagram of an image pickup optical system according to the present embodiment, and is a zoom optical system of an image pickup apparatus using an image pickup element. The left hand in the figure is the object side, and the right hand is the image plane side. In the figure, the imaging optical system 24 includes a negative first group (grp1) consisting of a negative lens, a positive lens, and a positive lens, a negative lens, a negative second group (grp2) consisting of a negative and positive junction lens, and an aperture. It is composed of ST, a positive third group consisting of positive lenses (grp3), and a fourth group consisting of negative and positive junction lenses (grp4). F1 is an infrared (1R) cut filter, LPF is an optical low-pass filter, and L1 is the optical axis of the imaging optical system 24.

As shown by the arrows in the figure, as the focal length moves from the wide angle to the telephoto by zooming, the negative second group grp2 moves toward the image plane side and the positive fourth group grp4 moves toward the subject side at the same time.

The image pickup optical system 24 has a lens drive mechanism (not shown), and uses a motor and a gear train to move the negative second group grp2 in the optical axis direction, so that the subject image is focused on the image pickup element 100. Adjust the focus so that.

FIG. 27 is a perspective view of the image pickup apparatus. In the figure, 201 is a front lens group that collectively shows the first group (grp1) and the second group (grp2) on the object side of the aperture ST of the imaging optical system 24, and 202 is the imaging optical system 24. Of these, the rear lens group shows the third group (grp3), the fourth group (grp4), and the optical low-pass filter LPF, which are on the image plane side of the aperture ST, and depends on the light beam passing through the aperture of the aperture ST. A subject image is formed on the image pickup element 100. The aperture ST rotates around the axis L2 and selectively takes four positions depending on the driving force of a motor (not shown). The aperture ST is provided with the five openings shown by 204 to 208, the openings 204, 205, 206 are openings for imaging, and the openings 207 and 208 are large defocus detection openings.

Next, the image pickup device 100 used in the present embodiment will be described.

[0022] The image pickup device 100 is a CMOS process compatible sensor (hereinafter abbreviated as CMOS sensor) which is one of the amplification type image pickup devices. This type of sensor has been published in the literature such as IEEE TRANSACTIONS ON ELECTRON DEVICE, VOL41, PP452 ~ 453,1994.

[0023] One of the features of the CMOS sensor is that the MOS transistor of the light receiving part and the MOS transistor of the peripheral circuit can be formed in the same process, so that the number of masks and the process process can be significantly reduced as compared with the CCD. Be done.

[0024] FIG. 2 is a circuit configuration diagram of an area sensor unit in the image sensor 100. The figure shows a two-dimensional area sensor with 2 columns x 2 rows of pixels, but in reality, as shown in Fig. 4, the number of pixels is increased to 1920 columns x 1080 rows, etc. to obtain a practical resolution. .. Note that 61 to 67 shown in FIG. 4 are focus detection regions described later. The focus detection area is arranged vertically to make it easier to capture the brightness distribution in a vertically long object such as a human.

In FIG. 2, 1 and 51 are first and second photoelectric conversion units composed of a MOS transistor gate and a depletion layer under the gate, 2 and 52 are photogates, 3 and 53 are transfer switch MOS transistors, and 4 is. Reset MOS transistor, 5 source follower amplifier MOS transistor, 6 vertical selection switch MOS transistor, 7 source follower load MOS transistor, 8 dark output transfer MOS transistor, 9 bright output transfer MOS transistor, 10 dark output Storage capacity C_TN, 11 is the bright output storage capacity C_TS, 12 and 54 are vertical transfer MOS transistors, 13 and 55 are vertical output line reset MOS transistors, 14 is a differential output amplifier, 15 is a vertical scanning unit, and 16 is a horizontal scanning unit.

[0026] The feature of the area sensor unit of the present embodiment is that two photoelectric conversion units are configured in one pixel, and a floating diffusion region (hereinafter, FD region) and a source follower amplifier, which are conventionally provided for each photoelectric conversion unit, are provided. Only one is formed in the two photoelectric conversion units, and the two photoelectric conversion regions are connected to the FD region via the MOS transistor switch.

[0027] Therefore, the charges of the two photoelectric conversion units can be transferred to the floating diffusion unit simultaneously or separately, and the signal charges of the two photoelectric conversion units can be added only by the timing of the transfer MOS transistor connected to the FD region. Non-addition can be easily performed. Using this structure, the first output mode that performs the photoelectric conversion output by the luminous flux from the entire exit pupil of the imaging optical system and the photoelectric conversion output by the luminous flux from a part of the exit pupil of the imaging lens are powdered second. It controls the output mode of. In the first output mode in which signals are added at the pixel level, a signal with less noise can be obtained as compared with a method in which signals are added after being read.

[0028] Fig. 3 shows a cross-sectional view of the light receiving portion (for example, 30-11). The light receiving units 30-21, 30-12, 30-22, etc. have the same structure.

[0029] In the figure, 17 is a p-type well, 18 and 58 are gate oxide films, 19, 59 are first-layer poly Si, 20, 50 are second-layer poly Si, and 21 is an n + floating diffusion region.

[0030] The FD region 21 is connected to the first photoelectric conversion unit 1 and the second photoelectric conversion unit 51 via the transfer MOS transistors 3 and 53. In the figure, the first photoelectric conversion unit 1 and the second photoelectric conversion unit 51 are drawn separately, but the boundary portion is actually extremely small, and in practice, the first hand portion 1 and the second photoelectric conversion unit 1 and the second photoelectric conversion unit are drawn separately. 51 can be considered to be in contact. Hereinafter, the adjacent first photoelectric conversion unit and the second photoelectric conversion unit will be collectively referred to as a light receiving unit.

Reference numeral 22 denotes a color filter that transmits light in a specific wavelength range, and 23 is a microlens for efficiently guiding the light flux from the imaging optical system 24 to the first and second photoelectric conversion units.

FIG. 5 is a plan view showing the arrangement of pixels and color filters. Here, only 4 columns x 4 rows are extracted and shown. Each pixel including the light receiving part and the MOS transistor is laid out in a substantially square shape, and is arranged adjacent to each other in a grid pattern. The light receiving units 30-11, 30-21, 30-12, and 30-22 described above with reference to FIG. 2 are located in the pixels 70-11, 70-21, 70-12, and 70-22, and here. It is represented as 72-11, 72-21, 72-12, 72-22.

[0033] Further, in this area sensor unit, R (red) G (green) B (blue) color filters are alternately arranged on each pixel to form a so-called Bayer array in which four pixels are a set. There is. In the Bayer array, the overall image performance is improved by arranging more G pixels than R and B pixels, which are easy for the observer to feel strongly when looking at the image. Generally, in this type of image sensor, the luminance signal is mainly generated from G, and the color signal is generated from R, G, and B.

[0034] As described above, each pixel has two photoelectric conversion units. In the figure, R, G, and B are photoelectric conversion units equipped with red, green, and blue color filters, and 1 or 2 following RGB is the first photoelectric conversion unit or the second photoelectric conversion unit. Represents the distinction between. For example, R1 means a first photoelectric conversion unit provided with a red color filter, and G2 means a second photoelectric conversion unit provided with a green color filter.

Further, the ratio of the light receiving portion to each pixel is about several tens of percent, and in order to advantageously utilize the light flux emitted from the imaging optical system 24, a condensing lens is provided for each light receiving portion. It is necessary to provide the light to be deflected to the light receiving portion so as to reach other than the light receiving portion.

[0036] FIG. 6 is a plan view showing the positional relationship between the microlens provided on the front surface of the image pickup device and the light receiving portion for this purpose. The microlenses 71-11 to 71-44 are axially symmetric spherical lenses or aspherical lenses in which the center of the light receiving portion and the optical axis are approximately aligned with each other, each having a rectangular effective portion and having a convex shape on the light incident side. They are densely arranged in a grid pattern. FIG. 7 is a diagram showing the surface state of this microlens when viewed from an angle.

Next, the operation of this microlens will be described in detail.

[0038] FIG. 8 is a cross-sectional view of the area sensor unit. The imaging optical system 24 is located on the left side of the figure, and the luminous flux emitted from the imaging optical system 24 passes through the 1R cut filter F1 and the optical low-pass filter LPF, and first microlenses 71-11, 71-21, 71-31, It is incident on 71-41. A color filter is placed behind each microlens, where only the desired wavelength range is selected to reach each light receiver from 72-11 to 72-41. The color filters form a Bayer array as explained with reference to Fig. 5, and there are three types of RGB. In addition, since it is a Bayer array, two of them appear in the cross section, 22G is a green transmission color filter and 22R is a red transmission color filter.

[0039] The power of each microlens is set so as to project each light receiving portion of the image pickup device onto the exit pupil of the image pickup optical system 24. At this time, the projection magnification is set so that the projected image of each light receiving unit is larger than the exit pupil when the aperture of the imaging optical system is opened, and the amount of light incident on the light receiving unit and the opening degree of the aperture ST of the imaging optical system are set. Make the relationship approximately linear. In this way, when the subject brightness and the sensitivity of the image sensor are given, the aperture value and the shutter speed can be calculated by the same method as that of a film camera. That is, the amount of incident light becomes proportional to the aperture aperture area, and the APEX method calculation is established. As with a film camera, the amount of exposure can be calculated using a general light meter, and the shooting operation is extremely easy.

[0040] Further, since the luminous flux for focus detection changes according to the opening degree of the aperture ST, the imaged luminous flux and the focus detection light always match, and the focusing detection result is the actual best of the imaging optical system at the aberration level. It has the advantage that it matches the resolution position well. When the object image formed by the imaging optical system 24 is located on the microlens, the image obtained from the imaging element 100 is the sharpest.

[0041] It is desirable that the imaging optical system 24 is a telecentric system so that the incident angle of the main light ray on the image sensor is 0 degrees from the viewpoint of the pupil projection brightness by the microlens, but the miniaturization and the zoom ratio Due to the demand for high magnification, it may not be a complete telecentric system. In this case, the microlens and the light receiving portion may be slightly eccentric, and the amount of eccentricity may be a function of the distance from the optical axis of the imaging optical system to the light receiving portion. In general, if this amount of eccentricity is monotonically increased according to the distance, the light receiving portion around the screen can be correctly projected onto the exit pupil of the imaging optical system 24.

In order to facilitate understanding, FIGS. 9 (a) and 9 (b) show the light flux incident on the first photoelectric conversion unit and the second photoelectric conversion unit for the light receiving unit 72-11 shown in FIG. It is the figure which showed each of the incident luminous flux separately. In FIG. 9 (a) showing the luminous flux incident on the first photoelectric conversion unit, FIG. 9 (b) shows the luminous flux incident on the first photoelectric conversion unit G1 and the light flux incident on the second photoelectric conversion unit. In), it can be seen that the luminous flux from the upper part of the figure is incident on the second photoelectric conversion unit G2.

[0043] Therefore, the luminous flux incident on the second photoelectric conversion unit in the entire image sensor is as shown in FIG. The luminous flux incident on any position of the area sensor unit is the luminous flux passing through the upper half of the aperture ST. On the other hand, the luminous flux incident on the first photoelectric conversion unit of the entire image sensor may be considered as being vertically inverted with the optical axis L1 of the image sensor as the axis of symmetry. That is, the state of division of the exit pupil is as shown in FIG. In the figure, 210 is an exit pupil of the imaging optical system 24 when the aperture ST is in the open state, and is a virtual image of the aperture 204 of the aperture ST viewed through the rear lens group 202. 211 is the first region on the exit pupil through which the light flux incident on the first photoelectric conversion part of the image sensor 100 passes, and 211 is the first region on the exit pupil through which the light flux incident on the second photoelectric conversion part of the image sensor 100 passes. There are two areas.

[0044] Reference numerals 215 and 216 are exit pupils when the aperture ST is stopped down, and are virtual images of the apertures 205 and 206 of the aperture ST viewed through the rear lens group 202. When the aperture ST is stopped down, each of the first area 211 and the second area 212 becomes smaller depending on the state of the aperture, but from the image signal obtained from the first photoelectric conversion unit and the second photoelectric conversion unit. Each of the obtained image signals is formed from a half-luminous flux obtained by dividing the exit pupil of the imaging optical system 24 into approximately two parts.

[0045] In the above optical system, for example, when an object image is formed in front of the image sensor 100, the semiluminous flux passing above the exit pupil shifts downward on the image sensor 100. , The semi-luminous flux passing under the exit pupil shifts upward. That is, the pair of image signals formed by the luminous flux passing through each half of the pupil of the imaging optical system 24 are phase-shifted in the vertical directions of FIGS. 5, 6 and 8 according to the imaging state of the object image. .. As described above, in the present embodiment, as a dividing means for dividing the image from the subject into two images, one microlens is provided in the two photoelectric conversion units, but other configurations may be used. Any structure may be used as long as the image from the subject can be divided into two images.

[0046] When image information is captured by using an image sensor, due to the sampling of the discrete image, a low frequency "foldback" different from that frequency is applied to the high frequency component of the spatial brightness distribution on the object. A noise image called "noise" is generated. This is what is called "color moiré" in digital color cameras. Furthermore, when performing focus detection by the phase difference detection method using the output of the image sensor, the phase information of the object image is not correctly reflected in the image output if there is this "folding noise", so the amount of defocus is detected. Deteriorates accuracy.

Assuming that the pixel pitch of the image sensor 100 used here is P, as shown in FIG. 12, the pixel pitches of the pixels provided with the green transmission color filter (G) are 2P in the vertical and horizontal directions and diagonally. 2P, and the pitch of the pixel equipped with the blue transmission filter (B) and the pitch of the pixel equipped with the red transmission filter (R) are 2P in the vertical and horizontal directions, respectively, and 22P in the diagonal direction. Further, the pixel pitches between Gs, Bs, and Rs in the vertical direction used for focus detection are all 2P.

[0048] Assuming that the pitch P = 0.0025 mm and considering the vertical and horizontal directions, the Nyquist frequency νn is νn = 1 / (2 × 2P) = 100. Therefore, the response characteristics of the image sensor 100 are as shown in FIG. 11 for all of R, G, and B. In the figure, the horizontal axis is the spatial frequency, the vertical axis is the response, and the lift on the high frequency side after the response becomes zero is the foldback noise.

[0049] In order to prevent such a defect, it is necessary to prevent the high frequency component of the object from reaching the image sensor at the optical image level. In order to suppress the generation of "folding noise", an optical low-pass filter LPF made of crystal is placed between the imaging optical system 24 and the imaging element 100, and the MTF (modulation transfer function) is 0 at the Nyquist frequency νn. This purpose can be almost achieved by setting the characteristics of the optical low-pass filter LPF so as to approach.

[0050] However, in the optical low-pass filter LPF utilizing the birefringence characteristic of quartz, the MTF does not always become 0 at the Nyquist frequency νn, so that a harmful high frequency component may be taken in for focus detection. However, since the sampling of the images of R and B is half-pitch-shifted, and for G, there is a series that is half-pitch-shifted like the relationship between R and B, so there are two series of R, B, and G. Sufficient focus detection accuracy can be obtained by obtaining the focus detection output for each and averaging them.

Further, when the signals are in focus, the phase error is evenly applied to the pair of signals, so that even if this phenomenon occurs, the focus detection error does not occur. This means that even if only one focus detection output is obtained and cannot be averaged, only the final focus can be guaranteed.

[0052] Further, as a characteristic of such an optical low-pass filter, one light ray incident on the optical low-pass filter LPF, which is an average flat plate, is emitted as two separate parallel light rays. If the incident light and the emitted light are parallel, the projection of each light receiving portion by the microlens onto the exit pupil of the imaging optical system 24 causes only a blur of about 2P, which is not a problem in practice. An optical low-pass filter is also known in which a single light beam incident on the optical low-pass filter is diffused and emitted. However, since the incident light is not parallel in this type, the imaging optics of each light receiving portion by a microlens is used. The projection of the system 24 onto the exit pupil is greatly disrupted, and the focus detection system does not work.

FIG. 13 is a block diagram showing an internal configuration including peripheral circuits of the image sensor 100. In the image pickup element 100, a timing generation unit 101, an area sensor unit 102, a vertical scanning unit 103 and a horizontal scanning unit 104 for selecting pixel output, an analog signal processing unit 105, and an A / D conversion unit for analog / digital conversion are included. 106, a digital signal processing unit 107 that converts a digitized signal into an output signal, and an interface unit 108 that outputs a digital image signal to the outside and receives command data from the outside are provided.

[0054] The area sensor unit 102 is the CMOS sensor described above.

The timing generation unit 101 generates a timing signal for reading the image signal photoelectrically converted by each pixel based on the master clock which is a reference frequency from the outside, and the vertical and horizontal scanning units 103 and 104 generate the timing signal. , The required scanning control is performed according to this timing signal, and the photoelectrically converted charge is read out in each pixel. The vertical synchronization signal and the horizontal synchronization signal are output from the timing generation unit 101 to the outside, and the synchronization signal is supplied for a system that requires a timing signal outside the image sensor.

[0056] The analog signal processing unit 105 is for outputting the image signal read from the area sensor unit 102 to the A / D conversion unit 106 after performing noise reduction processing, amplification processing, gamma processing, and clamping processing. The A / D conversion unit 106 converts this image signal into a digital signal and outputs it, and the digital signal processing unit 107 outputs the image signal digitally converted by the A / D conversion unit 106 to the interface unit 108.

[0057] The interface unit 108 outputs the digital image signal output from the digital signal processing unit 107 to the outside of the image sensor 100.

[0058] Further, the image sensor 100 can control the mode, output signal form, signal output timing, etc. of the image sensor 100 by responding to commands from the outside, and when a required command is given to the interface unit 108 from the outside, the interface unit 108 Each component is controlled so as to control the response to the received command.

[0059] The image sensor 100 has a first output mode in which a photoelectric conversion output is performed by a light flux from the entire exit pupil of the image pickup lens, and a second output in which a photoelectric conversion output is performed by a light flux from a part of the exit pupil of the image pickup lens. The mode can be controlled.

Next, an image output for focus detection, which is an adjustment for imaging, will be described. As described with reference to FIGS. 8, 9, and 10, the focus detection image is obtained by photoelectrically converting a pair of object images formed by the second output mode at substantially the same timing, and performing the first photoelectric conversion. The image signal from the unit and the image signal from the second photoelectric conversion unit are separately output.

[0061] When the subject image projected on the focus detection region is defocused, the phases of these image signals are shifted in the pupil division direction. Therefore, if the focus detection region is a rectangle with the division direction of the pupil of the imaging optical system in the longitudinal direction, the phase shift amount is detected using a long and continuous image signal, and the shift amount detection is more accurate. Can be increased.

[0062] In order to easily obtain an image for focus detection, an output position designation command is prepared as one of the commands of the digital signal processing unit 107 described above. The signal processing 107e, ..., N shown in FIG. 14 are the output position designation commands, and among the imaging regions of the area sensor unit 102, any of the focus detection regions 61 to 67 shown in FIG. Alternatively, specify a combination of these. Further, instead of selecting from a predetermined focus detection area, by preparing a pointing device such as a trackball, some may be arbitrarily specified from several hundreds of focus detection areas.

[0063] The line including the designated specific region is configured to output an image in which the charge accumulation level is optimized for focus detection. In order to obtain an appropriate signal level within the focus detection area, the electronic shutter is set for each focus detection area. The CCD type image sensor has the same charge accumulation time for all pixels, but the image sensor 100 has a structure that reads out in pixel units, line units, or block units by taking advantage of the features of the CMOS sensor. In addition, the start and end of the accumulation time can be different for each unit. Here, the charge accumulation time is changed in units of vertical lines so that the image in the focus detection region can effectively use the A / D conversion range.

[0064] The electronic shutter function will be described with reference to FIG. When reading out pixels in a vertical line sequence on the screen as shown in Fig. 15 (a), first reset the invalid charge stored in each pixel as shown in Fig. 15 (b), and perform the effective storage operation. Start. After a certain period of time has elapsed, the pixels are read out and an image signal is given to the signal processing unit of the next stage. The timing of the effective storage operation differs for each vertical line, but the storage time is substantially the same.

[0065] When shortening the charge accumulation time for the entire screen, the effective accumulation operation time is shortened by shifting the pixel reset timing for resetting the invalid charge to the rear as shown in FIG. 15 (c). Yes, the electronic shutter operation is performed. This operation may be performed by switching the mode of the timing generation unit 101 by the electronic shutter command received by the interface unit 108.

[0066] Further, when the effective storage operation time is set in units of vertical lines, it is as shown in FIG. 15 (d). FIG. 15 (d) shows an example in which the focal point detection region is assumed to be in the second row and the effective accumulation time is different from that of the other lines in order to make the signal level in the second row appropriate.

[0067] The focus detection image thus obtained is a pair of images with half luminous fluxes in each of the second output modes as described above, and the accumulation time is different from that of the other lines. However, by adding these in signal processing and applying a gain that cancels out the difference in storage time, it is possible to easily create an upper part under the same conditions as other lines in signal processing. Therefore, when the captured image is displayed on the monitor, it is possible to obtain the information of the entire image while performing the focus detection by adding such a process.

Next, returning to FIGS. 2 and 3, details of the charge storage operation of the image sensor will be described.

[0069] First, the control pulse φPG for expanding the depletion layer under the photo gates 2 and 52.₀₀, ΦPGe₀ Apply a positive voltage to. FD unit 21 is accumulating, control pulse φR to prevent blooming₀ Set to high and power supply V_DDFix it to. When the photon hν is irradiated and carriers are generated under the photogates 2 and 52, electrons are accumulated in the depletion layer under the photogates 2 and 52, and the holes are discharged through the P-type well 17.

[0070] An energy barrier by the transfer MOS transistor 3 is formed between the photoelectric conversion unit 1 and the FD unit 21, and an energy barrier by the transfer MOS transistor 53 is formed between the photoelectric conversion unit 51 and the FD unit 21. Therefore, it exists under the electron photogates 2 and 52 during the photocharge accumulation. After that, if the horizontal scanning unit is scanned and the electric storage operation is performed in the same manner, charges are accumulated in all the photoelectric conversion units.

[0071] In the read state, the barrier under the transfer MOS transistor 3 or 53 is removed, and the control pulse φPG is used so that the electrons under the photo gates 2 and 52 are completely transferred to the FD unit 21.₀₀, ΦPGe₀ , Control pulse φTX₀₀, ΦTXe₀ To set.

The reading operation of the image pickup device will be described with reference to the timing chart of FIG. This timing chart is for the second output mode in which the two photoelectric conversion units are output independently, and is used for reading the focus detection image.

[0073] First, the horizontal output line is reset by setting the control pulse φL to high by the timing output from the horizontal scanning unit 16. Also, the control pulse φR₀ , φPG₀₀, φPGe₀ Is set to high, the reset MOS transistor 4 is turned on, and the first layer poly Si19 of the photogate 2 is set to high. Time T₀ In, control pulse φS₀ Is set to high, the selection switch MOS transistor 6 is turned on, and the light receiving unit 30-11 is selected. Next, control pulse φR₀ Is set to low, the reset of the FD area 21 is stopped, the FD area 21 is placed in a floating state, the gate and source of the source follower amplifier MOS transistor 5 are passed through, and then the time T₁ In the control pulse φT_N Is set to high, and the dark voltage in the FD area 21 is stored in the storage capacity C by the source follower operation._TNOutput to 10.

Next, in order to perform the photoelectric conversion output of the first photoelectric conversion unit, the control pulse φTX of the first photoelectric conversion unit is performed.₀₀After conducting the transfer switch MOS transistor 3 with high, time T₂ In control pulse φPG₀₀As low. At this time, it is preferable to raise the potential well extending under the photo gate 2 so that the light generating carrier is completely transferred to the FD region 21.

[0075] Time T₂ By transferring the electric charge from the photoelectric conversion unit 1 of the photodiode to the FD region 21, the potential of the FD region 21 changes according to the light. At this time, since the source follower amplifier MOS transistor 5 is in a floating state, the potential of the FD region 21 is set to the time T.₃ In the control pulse φT_S As high as storage capacity C_TSOutput to 11. At this point, the dark output and optical output of the first photoelectric conversion unit have a storage capacity of C, respectively._TN10 and C_TSAccumulated at 11 and time T₄ The vertical output line is reset by temporarily setting the control pulse φHC to high, and the vertical output line is reset by conducting the MOS transistors 13 and 55. Output the output. Storage capacity C_TN10 and C_TSDifferential output V by the working amplifier 14 of 11_OUT If this is taken, a signal with good S / N with pixel random noise and fixed pattern noise removed can be obtained.

[0076] Note that the light charge of the light receiving unit 30-12 is stored at the same time as the light receiving unit 30-11._TNAnd C_TSThe timing pulse from the vertical scanning unit 15 is delayed by one light receiving portion and read out to the vertical output line, and is output from the differential amplifier 14. Since it is the difference between the timing pulses of one light receiving part, the accumulation times of both can be regarded as substantially the same.

Next, the storage capacity C_TSAfter outputting bright output to 11, control pulse φR₀ Is set to high, the reset MOS transistor 4 is conducted, and the FD region 21 is reset to the power supply VDD.

[0078] After the vertical transfer of the first photoelectric conversion unit is completed, the reading of the second photoelectric conversion unit is performed. The reading of the second photoelectric conversion unit is the control pulse φR.₀ Is set to low, the reset of the FD area 21 is stopped, the FD area 21 is placed in a floating state, the gate and source of the source follower amplifier MOS transistor 5 are passed through, and then the control pulse φT is set at time T5._N Is set to high, and the dark voltage in the FD area 21 is stored in the storage capacity C by the source follower operation._TNOutput to 10.

[0079] In order to perform the photoelectric conversion output of the second photoelectric conversion unit, the control pulse φTXe of the second photoelectric conversion unit is performed.₀ After conducting the transfer switch MOS transistor 53 as high, the control pulse φPGe0 is lowered as low at time T6.

[0080] At time T6, the electric charge from the photoelectric conversion unit 2 of the photodiode is transferred to the FD region 21, so that the potential of the FD region 21 changes according to the light. At this time, since the source follower amplifier MOS transistor 5 is in a floating state, the potential of the FD region 21 is controlled at time T7 by the control pulse φT._S As high as storage capacity C_TSOutput to 11. At this point, the dark output and optical output of the second photoelectric conversion unit have a storage capacity of C, respectively._TN10 and C_TSThe vertical output line reset MOS transistors 13 and 55 are conducted with the control pulse φHC at time T8 as a temporary high to reset the vertical output line, and the scanning timing signal of the vertical scanning unit 15 is used during the vertical transfer period. Make the vertical output line output the dark output and light output of the pixel. At this time, the storage capacity C_TN10 and C_TSDifferential output V by 11 differential amplifiers 14_OUT To get.

[0081] By the above driving, the reading of the first and second photoelectric conversion units can be performed independently.

[0082] After that, if the horizontal scanning unit is scanned and the reading operation is performed in the same manner, the independent output of the all photoelectric conversion unit can be obtained. That is, in the case of the next row, first the control pulse φS₁ Is high, then φR₁ Is set to low, and then the control pulse φT_N , ΦTX₀₁Is set to high, and the control pulse φPG₀₁Low, control pulse φT_S Is high, and the control pulse φHC is temporarily set to high, and the signals of the first photoelectric conversion section of the light receiving sections 30-21 and 30-22 are read out. Then, the control pulse φTXe₁ , ΦPGe₁ And the control pulse is applied in the same manner as described above, and the signals of the second photoelectric conversion units of the light receiving units 30-21 and 30-22 are read out.

[0083] By providing one source follower in each of the two photoelectric conversion units without providing a source follower in each of the two photoelectric conversion units of one pixel, the source follower amplifier MOS transistor 5, the selection switch MOS transistor 6, and the reset MOS are provided. Transistor 4 can be halved from the conventional one. As a result, the aperture ratio of the photoelectric conversion portion of the pixel is improved, miniaturization can be realized by integrating the pixels, and it becomes extremely easy to use for focus detection. Further, by sharing the FD region 21 between the two photoelectric conversion units, it is not necessary to increase the capacitance of the gate portion of the source follower amplifier MOS transistor 5, so that it is possible to prevent a decrease in sensitivity.

Next, a first output mode for outputting a signal based on the luminous flux from the entire pupil of the objective lens by adding the signals of the first and second photoelectric conversion units in the FD region 21 will be described. This operation mode corresponds to image output by a normal image sensor.

[0085] FIG. 17 shows a timing chart when the signals of the first and second photoelectric conversion units are added. In Fig. 16 of the non-addition mode, the control pulse φTX₀₀And control pulse φTXe₀ , Control pulse φPG₀₀And control pulse φPGe₀ The timing of was shifted, but in the case of addition, it is the same timing. That is, in order to read from the first photoelectric conversion unit and the second photoelectric conversion unit of the light receiving unit 30-11 at the same time, first, the control pulse φT_N Is set to high and the noise component is read from the horizontal output line, and the control pulse φTX₀₀And control pulse φTXe₀ , And control pulse φPG₀₀And control pulse φPGe₀ Are transferred to the FD area 21 at the same time as high and low, respectively. This makes it possible to add the signals of the two upper and lower photoelectric conversion units at the same time in the FD region 21. Since the signal is added at the image level, it is not affected by the amplifier noise, and the image has a high S / N that cannot be obtained by the addition after reading the signal.

As described above, according to the timing of FIG. 16, the focus detection image for performing the focus detection of the photographing lens is output, and according to the timing of FIG. 17, a normal image using the total luminous flux is output. be able to. That is, the first region of the ejection pupil is controlled by the control means including the transfer switch MOS transistors 3 and 53, the source follower amplifier MOS transistor 5, and the vertical scanning unit 15 that controls the pulse timing to the transfer switch MOS transistors 5 and 53. The signals from the first photoelectric conversion unit that photoelectrically converts the light beam from the second region and the second photoelectric conversion unit that photoelectrically converts the light beam from the second region are added and read out without addition. In other words, it controls the first output mode in which the photoelectric conversion output is performed by the luminous flux from the entire exit pupil of the imaging lens and the second output mode in which the photoelectric conversion output is performed by the luminous flux from a part of the exit pupil of the imaging lens. It is possible.

[0087] According to the first output mode, no unnatural image blurring occurs, and an image obtained by an image sensor originally designed to obtain an image using the total luminous flux of the pupil of the imaging optical system. It is possible to obtain a high-quality image with the same level of S / N as the above.

[0088] Now, signal processing for focus detection will be described. FIG. 18 is an enlarged view of the focus detection region 61. Other focus detection areas 62, 63, 64, 65, 66, 67 have similar structures. 19 to 26 are diagrams showing digital signals of images output from the interface unit 108.

As shown in the figure, the focus detection region 61 includes two sets of pixel sequences composed of 12 light receiving units. The pixel row 82 is composed of the light receiving parts 80-1, 80-2, ..., 80-12, and the pixel row 83 is composed of the light receiving parts 81-1, 81-2, ..., 81-12. Has been done. Since the color filters of the area sensor unit 102 have a Bayer arrangement, two types of color filters are alternately arranged in each pixel sequence. Therefore, for focus detection, each pixel sequence is classified according to the type of color filter, and a pair of image signals consisting of a signal from the first photoelectric conversion unit and a signal from the second photoelectric conversion unit are further classified from each. To generate. Therefore, a total of four pairs of image signals can be generated from the focus detection region 61. As described above, one focal point detection region has a substantially uniform accumulation time.

19 to 22 show these four pairs of image signals.

[0091] Fig. 19 shows the image signals from the light receiving units 80-1, 80-3, ..., 80-11 provided with the green color filter in the pixel sequence 82, and 84 is the first photoelectric conversion shown by G1. The signal of the part, 85, is the signal of the second photoelectric conversion part shown by G2.

[0092] FIG. 20 shows image signals from light receiving units 81-2, 81-4, ..., 81-12 having a green color filter in the pixel sequence 83, and 86 is the first photoelectric conversion shown by G1. The signal of the part, 87, is the signal of the second photoelectric conversion part shown by G2.

[0093] Fig. 21 shows the image signals from the light receiving units 80-2, 80-4, ..., 80-12 provided with the red color filter in the pixel row 82, and 88 is the first photoelectric conversion shown by R1. The signal of the unit, 89, is the signal of the second photoelectric conversion unit indicated by R2.

FIG. 22 shows the image signals from the light receiving units 81-1, 81-3, ..., 81-11 provided with the blue color filter in the pixel sequence 83, and 90 is the first photoelectric conversion shown by B1. The signal of the unit, 91, is the signal of the second photoelectric conversion unit shown by B2.

[0095] These are examples in which the subject image formed on the focus detection region 61 by the imaging optical system 24 has shades of orange and yellow, and the contrast of green shown in FIGS. 19 and 20 is high. The red color shown in FIG. 21 has a low contrast but is strong in intensity, and the blue color shown in FIG. 22 has a low contrast and intensity. The figure shows a state in which the object image is defocused, and it can be seen that the phases of the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit are shifted as shown by arrows A and B.

[0096] Further, FIGS. 23 to 26 are signals in a state where the subject image is in focus, and the signals shown in FIGS. 19 to 22 change due to the movement of the second group grp2 of the imaging optical system 24, respectively. It shows how to do it.

[0097] Fig. 23 shows the image signals from the light receiving units 80-1, 80-3, ..., 80-11 provided with the green color filter in the pixel row 82, and 184 is the first photoelectric conversion shown by G1. The signal of the unit, 185, is the signal of the second photoelectric conversion unit shown by G2.

[0098] Fig. 24 shows the image signals from the light receiving units 81-2, 81-4, ..., 81-12 provided with the green color filter in the pixel sequence 83, and 186 is the first photoelectric conversion shown by G1. The signal of the part, 187, is the signal of the second photoelectric conversion part shown by G2.

[0099] FIG. 25 shows image signals from light receiving units 80-2, 80-4, ..., 80-12 having a red color filter in the pixel row 82, and 188 is the first photoelectric conversion shown by R1. The signal of the unit, 189, is the signal of the second photoelectric conversion unit indicated by R2.

[0100] Fig. 26 shows the image signals from the light receiving units 81-1, 81-3, ..., 81-11 provided with the blue color filter in the pixel sequence 83, and 190 is the first photoelectric conversion shown by B1. The signal of the part, 191 is the signal of the second photoelectric conversion part shown by B2.

[0101] When the object is in focus, the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit have the same phase. Therefore, focusing can be detected by determining the identity of the pair of signals. Furthermore, the defocus amount can be obtained by detecting the phase shift amount using a known method using a correlation calculation. If the obtained defocus amount is converted into the amount to drive the second group grp2 of the imaging optical system 24, automatic focus adjustment is possible. Since the amount of drive of the lens can be known in advance, it is usually necessary to drive the lens to the in-focus position almost once, and extremely high-speed focus adjustment can be realized.

[0102] Further, although the color-separated signals are used here, if the colors are not separated, the contrast tends to be low because it corresponds to obtaining a signal obtained by adding them together, and as a result, an undetectable state is likely to occur. .. On the other hand, if a color-separated signal is used, high contrast does not always appear in all RGB signals as shown here, but conversely, a high contrast signal can be obtained in any of RGB. Focus detection is possible in most cases.

Although an object image that has passed through the optical low-pass filter LPF is incident on the image sensor 100, a high frequency component exceeding the Nyquist frequency νn = 1 (2 × 2P) of the focus detection system due to the characteristics of the optical system. It does not mean that it does not take in at all. Therefore, depending on the pattern of the object, the phase of the object image is not reflected in the phase of the signal, and the focus detection result may include some errors.

[0104] When there is no phase difference between the images, the phase error is evenly applied to the pair of signals, so that even if this phenomenon occurs, the focus detection error does not occur. That is, the signals shown in FIGS. 23 and 24 do not cause an error in the focusing determination, but the signals shown in FIGS. 19 and 20 cause an error in the defocus amount detection.

As can be seen from FIG. 18, since the sampling positions of the subject images of the signals 86 and 87 are shifted by half a pitch with respect to the signals 84 and 85, the focus detection results calculated from the signals 84 and 85 and the signals By averaging the focus detection results calculated from 86 and 87 to obtain the final focus detection result, the focus detection error can be reduced and the above-mentioned problems can be solved.

Similarly, since the sampling positions of the subjects of the signals 90 and 91 are shifted by half a pitch with respect to the signals 88 and 89, the focus detection results calculated from the signals 88 and 89 and the signals 90 and 91 are calculated. Focus detection error can be reduced by averaging the focus detection results to obtain the final focus detection result. In this case, the focus detection results focusing on red and blue are averaged, and as a result, the chromatic aberration of the imaging optical system 24 is also averaged, which is preferable.

[0107] Further, the focus detection result calculated from the signals 84 and 85, the focus detection result calculated from the signals 86 and 87, the focus detection result calculated from the signals 88 and 89, and the focus calculated from the signals 90 and 91. Higher end point detection accuracy can be obtained by selecting and averaging only the detection results with high reliability. For example, in the case of a low-contrast signal as in the case of FIGS. 21 and 22, the focus detection result may not be used for focus adjustment.

[0108] The above description has been made on the premise that the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit have a relationship in which only the phase is shifted. This premise holds when the amount of defocus is relatively small. Next, the correspondence to the large defocus will be described.

[0109] In the image pickup apparatus shown in FIG. 27, the aperture ST is provided with the five openings shown by 204 to 208, the openings 204, 205, 206 are openings for imaging, and the openings 207 and 208 are openings for large defocus detection. is there. At the time of imaging, one of the openings 204, 205, 206 is automatically selected according to the brightness of the object to be photographed. Alternatively, the user may arbitrarily select one of the openings 204,205,206. As the size of the aperture is reduced, that is, when the aperture 205 is selected over the aperture 204 and the aperture 206 is selected over the aperture 205, the range of focus on the field side becomes deeper and the charge accumulation time by the electronic shutter becomes deeper. Becomes longer.

[0110] The microlens provided on the image sensor projects each light receiving portion of the image sensor onto the exit pupil of the image pickup optical system 24, and the amount of light incident on each light receiving portion of the image sensor 100 and the opening degree of the aperture ST In order to make the relationship linear, the power is set so that the projected image of each light receiving unit is larger than the exit pupil when the aperture of the image sensor is open. That is, when the projected image of the light receiving portion and the aperture are compared on the aperture ST, the projected image of the light receiving portion is even larger than the largest aperture 204. In this way, when the amount of light incident on the image sensor is approximately proportional to the aperture area of the aperture and the subject brightness and sensitivity of the image sensor are given, the aperture value and shutter speed can be calculated in the same way as with a film camera. Can be done. That is, the amount of incident light is compared with the aperture area of the diaphragm, and the APEX method calculation is established.

[0111] As described above with reference to FIG. 28, regardless of which of the openings 204, 205, and 206 is used for imaging, the formed image is due to the semiluminous flux obtained by dividing the circle into two. Generally, an object image is given by a combo of a point image and an object's luminance pattern, but when the amount of defocus of the object image becomes large, the shape of an exit pupil appears in the point image. As a result, the image is overlaid with a blur in the form of a circle divided into two.

[0112] If a pair of focal detection images is formed via a pair of exit pupils having overlapping shapes due to translation, the signal of the first photoelectric conversion unit may be signal regardless of the individual pupil shapes. The relationship between the signal and the signal of the second photoelectric conversion unit is that only the phase is shifted. However, in the case of this imaging device, the shapes of the first region 211 and the second region 212 on the exit pupil are in an inverted relationship with each other and do not overlap due to translation. Therefore, the bokeh shape superimposed on the image also has an inverted relationship, and the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit are shifted in phase while having different shapes. At the time of large defocus, the phase difference detection of the image does not go well, and the defocus amount detection error is large.

[0113] Further, as another factor that increases the defocus amount detection error at the time of large defocus, there is a manufacturing error of the microlens. As described above, the microlens projects the light receiving portion onto the exit pupil of the imaging optical system. If the projection positions are different for each pixel, the phase shift amount at the time of defocus will be different for each pixel. This effect becomes more serious as the amount of defocus increases. However, since the microlens is very fine, it has to tolerate some manufacturing variation in practice.

Therefore, at the time of large defocus detection, the openings 207 and 208 of the aperture ST for detecting large defocus are used. In this case, the state of division of the exit pupil is as shown in FIG. In the figure, 210 and 214 are virtual images of the aperture openings 207 and 208 viewed through the rear lens group 202, and are the third and fourth regions on the exit pupil formed by the apertures 207 and 208.

[0115] The third region 213 is contained in the first region 211 and the fourth region 214 is contained in the second region 212, so that the openings 207 and 208 determine the shape of the exit pupil. The luminous flux that has passed through the aperture 207 is incident on the first photoelectric conversion unit, and the luminous flux that has passed through the aperture 208 is incident on the second photoelectric conversion unit.

[0116] The first region 211 and the second region 212 are affected by the pupil projection accuracy by the microlens, but the third region 213 and the fourth region 214 are not affected by the pupil projection accuracy by the microlens. .. Therefore, even if the positions of the first region 211 and the second region 212 fluctuate for each pixel due to a manufacturing error of the microlens, if the openings 207 and 208 are used, the determined third region 213 and th are Four regions 214 can be obtained.

[0117] If the openings 207 and 208 are, for example, ellipses or circles, the relationship between the signal of the first photoelectric conversion unit and the signal of the second photoelectric conversion unit is perfect because they have the same shape that overlaps in parallel movement. Only the phase is shifted. Moreover, it is not affected by manufacturing errors of microlenses. Therefore, even if the defocus is large, the detection error of the defocus amount can be suppressed to an extremely small size. In addition, if the incident height of the center of gravity of the apertures 207 and 208 is selected to be about 0.7 times the open aperture diameter, the amount of defocus to the best resolution for an imaging optical system having general spherical aberration characteristics can be accurately determined at the aberration level. Can be detected.

[0118] When the second group grp2 and the fourth group grp4 of the imaging optical system 24 move in the optical axis direction and a zoom operation is performed, the open F number generally changes, but the opening 207 with respect to the open aperture Since the relationship between the positions of the center of gravity of the 208 maintains a constant ratio, it is convenient that the incident height of the center of gravity of the apertures 207 and 208 is always about 0.7 times the open aperture diameter regardless of the zoom position.

[0119] At the time of the first focus adjustment operation in which the object is unlikely to be in focus, first, the apertures 207 and 208 are used for focus adjustment and object observation that can cope with even a large defocus. It is advisable to perform the monitor display, perform the remaining focus adjustment and monitor display using any of the apertures 204, 205, and 206 in the subsequent focus adjustment operation, and configure the sequence of the imaging device to prepare for the subsequent imaging.

[0120] Since this imaging device does not require a dedicated focus detecting device for detecting the defocus amount of the imaging optical system, a mirror or prism for optical path division is not required. Therefore, the imaging system can be miniaturized.

(Second Embodiment) The second embodiment will be described.

[0122] FIG. 30 is a circuit configuration diagram of an area sensor unit according to the present embodiment. The present embodiment is characterized in that a pn photodiode is used instead of a photogate for the photoelectric conversion unit, and other points are the same as those of the first embodiment.

[0123] The charges of the two photoelectric conversion units can be transferred to the floating diffusion unit simultaneously or separately, and the signal charges of the two photoelectric conversion units can be added or not added only by the timing of the transfer MOS transistor connected to the FD region. Can be done easily. Using this structure, it is possible to switch between the first light receiving mode that receives the luminous flux from the entire exit pupil of the imaging lens and the second light receiving mode that receives the luminous flux from a part of the exit pupil of the imaging lens. There is.

[0124] In FIG. 30, 301 and 351 are first and second photoelectric conversion units composed of pn photodiodes, 303 and 353 are transfer switch MOS transistors, 304 are reset MOS transistors, and 305 are source follower amplifier MOS transistors, 306. Is a vertical selection switch MOS transistor, 307 is a source follower load MOS transistor, 308 is a dark output transfer MOS transistor, 309 is a bright output transfer MOS transistor, and 310 is a dark output storage capacity C._TN, 311 is bright output storage capacity C_TS, 312 and 354 are vertical transfer MOS transistors, 313 and 355 are vertical output line reset MOS transistors, 314 is a differential output amplifier, 315 is a vertical scanning unit, and 316 is a horizontal scanning unit.

FIG. 31 shows a cross-sectional view of a light receiving unit (for example, 330-11). The light receiving units 330-21, 330-12, 330-22, etc. have the same structure.

[0126] In the figure, 317 is a P-type well, 318 and 358 are gate oxide films, 320 and 350 are poly Si, and 321 is an n + floating diffusion region.

[0127] 340 and 390 are n layers and are concentrations that can be completely depleted. The FD region 321 is connected to the first photoelectric conversion unit 301 and the second photoelectric conversion unit 351 via the transfer MOS transistors 303 and 353. The electric charge generated by the control pulse φTX is completely transferred to the FD unit 321, and the signal can be added or not added by the control pulse φTX. In the figure, the first photoelectric conversion unit 301 and the second photoelectric conversion unit 351 are drawn separately, but the boundary portion is actually extremely small, and the first photoelectric conversion unit 301 and the second photoelectric conversion unit are practically used. Part 351 can be considered to be in contact.

[0128] 322 is a color filter that transmits light of a specific wavelength, and 323 is a microlens for efficiently guiding the light flux from the imaging optical system 324 to the first and second photoelectric conversion units.

[0129] The charge storage operation of the image sensor will be described. First, the FD unit 321 is accumulating, and the control pulse φR is used to prevent blooming.₀ Set to high and power supply V_DDFix it to. When the photon hν is irradiated, electrons are accumulated in the pn photodiodes 301 and 351 and the holes are discharged through the P-type well 317.

[0130] An energy barrier by the transfer MOS transistor 303 is formed between the photoelectric conversion unit 301 and the FD unit 321, and an energy barrier by the transfer MOS transistor 53 is formed between the photoelectric conversion unit 51 and the FD unit 21. Therefore, it exists in the electron pn photodiodes 301 and 351 during the photocharge accumulation. After that, if the horizontal scanning unit is scanned and the charge accumulation operation is performed in the same manner, charges are accumulated in all the photoelectric conversion units.

[0131] When the read state is reached, the barrier under the transfer MOS transistor 303 or 353 is removed, and the control pulse φTX is used so that the electrons of the pn photodiodes 301 and 351 are completely transferred to the FD unit 321.₀₀, ΦTXe₀ To set.

[0132] From the timing chart of FIG. 16, φPG₀₀, ΦPGe₀ By using the image excluding the above, it is possible to read out the image for focus detection by the second light receiving mode in which the two photoelectric conversion units are output independently.

[0133] First, the control pulse φR₀ FD section 321 is powered by V_DDReset to, control pulse φS₀ Is set to high and the dark output is stored in the storage capacity 310, and then the control pulse φTX₀₀The optical charge stored in the pn photodiode 301 is transferred to the storage capacitance 311 via the source follower MOS transistor 305 and the selection switch MOS transistor 306, and the noise component is canceled by the differential amplifier 314. Image signal V of photoelectric conversion unit_OUT Is output.

[0134] Control pulse φR₀ FD section 321 is powered by V_DDReset to, then control pulse φTXe₀ Is set to high, the optical charge stored in the pn photodiode 351 is transferred to the storage capacity 311 via the source follower MOS transistor 305 and the selection switch MOS transistor 306, the noise component is canceled by the differential amplifier 314, and the second photoelectric is used. Converter image signal V_OUT Is output.

[0135] Further, from the timing chart according to FIG. 17, φPG_00,φPGe₀ By supplying the control pulse excluding the above, the image signal can be read out in the first light receiving mode in which the charges of the two pn photodiodes 301 and 351 are added.

(Third Embodiment) The third embodiment will be described.

[0137] In the first and second embodiments, one source follower amplifier MOS transistor is provided in the two photoelectric conversion units, and the charge is added in the floating diffusion region. However, in the present embodiment, the figure is shown. As shown in 32, one source follower amplifier MOS transistor is provided in one photoelectric conversion unit, and addition is performed outside the pixel.

[0138] By providing two capacitances 401 and 402 for each row and simultaneously outputting pulses to the transistors 403 and 404 by the vertical shift register, signals from the capacitances 401 and 402 are simultaneously output to the output line 405. Add. Then, the pulses are sequentially output to the transistor 403 and the transistor 404 to output one pixel at a time.

[0139] Further, in FIG. 33, the dotted line portion in FIG. 32 has another circuit configuration. As shown in FIG. 32, a transistor 407 may be provided on the output line 406, and addition or non-addition may be performed by turning the transistor ON or OFF.

(Fourth Embodiment) The fourth embodiment will be described.

[0141] Based on FIG. 34, a case where the image pickup device of the first or second embodiment described above is applied to an image pickup device such as a still camera will be described in detail.

[0142] In FIG. 33, 501 is a barrier that also serves as a lens protector and a main switch, 502 is a lens that forms an optical image of a subject on an image sensor 100, and 503 is an aperture for varying the amount of light passing through the lens 502. 100 is an image sensor for capturing the subject imaged by the lens 502 as an image signal, 508 is an image sensor 4, a timing generator that outputs various timing signals to the signal processing unit 507, and 509 is various calculations and the entire image sensor. The overall control / calculation unit that controls the image, 510 is a memory unit for temporarily storing image data, 511 is an interface unit for recording or reading on a recording medium, and 512 is for recording or reading image data. Detachable recording medium such as a semiconductor memory, 513 is an interface unit for communicating with an external computer or the like.

Next, the operation of the image pickup apparatus at the time of imaging in the above-described configuration will be described.

[0144] When the barrier 501 is opened, the main power is turned on, and then the control system is turned on. Then, in order to control the exposure amount, the overall control / calculation unit 509 opens the aperture 503, and the signal output from the image sensor 504 is input to the signal processing unit 7. Based on the data, the exposure calculation is performed by the overall control / calculation unit 509. The brightness is determined based on the result of this photometry, and the overall control / calculation unit 509 controls the aperture according to the result.

Next, based on the non-additive signals described in the first and second embodiments output from the image sensor 100, the high frequency component is extracted and the calculation of the distance to the subject is performed as a whole control / calculation unit 9. Do it with. After that, the lens is driven to determine whether or not it is in focus, and when it is determined that the lens is not in focus, the lens is driven again to measure the distance.

[0146] Then, the main exposure starts after the focusing is confirmed. When the exposure is completed, the addition signals output from the image sensor 100 and described in the first and second embodiments are color-corrected through the signal processing unit 507 and written to the memory unit by the overall control / calculation unit 509. .. After that, the data stored in the memory unit 510 passes through the recording medium control I / F unit under the control of the overall control / calculation unit 9, and is recorded in the removable recording medium 512 such as a semiconductor memory. Further, the image may be processed by directly inputting to a computer or the like through the external I / F unit 513.

[0147] In the first to third embodiments described above, the signal addition from the photoelectric conversion unit is added in the image sensor. For example, the signal processing unit 507 shown in FIG. 34 and the like are shown. The configuration may be such that it is performed outside the image sensor of.

[Effects of the Invention] According to the present invention, adjustments for imaging can be performed at high speed, and high-quality image output can be obtained.

[0149] The image pickup apparatus according to the first and second embodiments described above has the following effects.

(1) Focus detection by the phase difference detection method by the image sensor becomes possible, and high-speed focus adjustment based on the detected defocus amount can be realized.

(2) At the same time, a high-quality image having the same level of S / N as an image obtained by an image sensor originally designed to obtain an image using the total luminous flux of the pupil of the imaging optical system is obtained. I was able to.

[0152] Moreover, at this time, unnatural image blurring does not occur.

(4) Since a dedicated focus detection device for detecting the defocus amount of the imaging optical system is not required, a mirror or prism for optical path division is not required. Therefore, the imaging system can be miniaturized.

BRIEF DESCRIPTION OF THE DRAWINGS [Fig. 1] Fig. 1 is a configuration diagram of an imaging optical system.

FIG. 2 is a circuit configuration diagram of an area sensor unit in the image sensor 100.

FIG. 3 is a cross-sectional view of a light receiving portion.

FIG. 4 is an explanatory diagram of an imaging region and a focus detection region.

FIG. 5 is a plan view showing the arrangement of pixels and color filters.

FIG. 6 is a plan view showing a positional relationship between a microlens and a light receiving portion.

FIG. 7 is a diagram showing a surface state of a microlens as viewed from an angle.

FIG. 8 is a cross-sectional view of an area sensor unit.

9 (a) and 9 (b) are views showing a light flux incident on the first photoelectric conversion unit and a light flux incident on the second photoelectric conversion unit, respectively.

FIG. 10 is a diagram showing a luminous flux incident on a second photoelectric conversion unit.

FIG. 11 is a diagram showing the response characteristics of the image sensor 100.

FIG. 12 is a diagram for explaining the pitch of pixels.

FIG. 13 is a block diagram showing an internal configuration including peripheral circuits of the image sensor 100.

FIG. 14 is a diagram for explaining an output position designation command.

15 (a), (b), (c), and (b) are diagrams for explaining the electronic shutter function, (a) is an explanatory diagram for reading pixels in a vertical line sequence, and (b) is an accumulation diagram. The timing chart when the time is substantially the same, (c) is the timing chart when the charge accumulation time is shortened for the entire screen, and (d) is the signal level of the second column different from the other lines. It is a timing chart of the case.

FIG. 16 is a timing chart when the signals of the first and second photoelectric conversion units are read out independently.

FIG. 17 is a timing chart when the signals of the first and second photoelectric conversion units are added.

FIG. 18 is an enlarged view of a focus detection region 61.

FIG. 19 is a diagram showing image signals from light receiving units 80-1, 80-3, ..., 80-11 provided with a green color filter in the pixel row 82.

FIG. 20 is a diagram showing image signals from light receiving units 81-2, 81-4, ..., 81-12 provided with a green color filter in the pixel sequence 83.

FIG. 21 is a diagram showing image signals from light receiving units 80-2, 80-4, ..., 80-12 provided with a red color filter in the pixel row 82.

FIG. 22 is a diagram showing image signals from light receiving units 81-1, 81-3, ..., 81-11 provided with a blue color filter in the pixel sequence 83.

FIG. 23 is a diagram showing image signals from light receiving units 80-1, 80-3, ..., 80-11 provided with a green color filter in the pixel row 82.

FIG. 24 is a diagram showing image signals from light receiving units 81-2, 81-4, ..., 81-12 provided with a green color filter in the pixel sequence 83.

FIG. 25 is a diagram showing image signals from light receiving units 80-2, 80-4, ..., 80-12 having a red color filter in the pixel row 82.

FIG. 26 is a diagram showing image signals from light receiving units 81-1, 81-3, ..., 81-11 provided with a blue color filter in the pixel sequence 83.

FIG. 27 is a perspective view of an imaging device.

FIG. 28 is an explanatory diagram showing a state of division of exit pupils.

FIG. 29 is an explanatory diagram showing a state of division of an exit pupil when a large defocus is detected.

FIG. 30 is a circuit configuration diagram of an area sensor unit in the image sensor 100.

FIG. 31 is a cross-sectional view of a light receiving portion.

FIG. 32 is a circuit configuration diagram of an area sensor unit in the image sensor 100.

FIG. 33 is a circuit configuration diagram of an area sensor unit in the image sensor 100.

FIG. 34 is a diagram showing an image pickup apparatus using an image pickup device.

[Description of Code] 1,51 MOS Transistor 1st and 2nd photoelectric conversion unit 2, 52 Photogate 3, 53 Transfer switch MOS transistor 23 Microlens 24 Imaging optical system 100 Image sensor 207 , 208 Large defocus detection aperture 210, 215, 216 Exit pupil 211, 212 First and second regions on exit pupil 213, 214 Third and fourth regions on exit pupil ST Aperture