Imaging apparatus and method

The present invention relates to an imaging device and an imaging method, and more particularly to an imaging device and an imaging method capable of detecting a face region from image data.

In recent years, DSCs that detect a person's face and perform all kinds of control based on that information have been disclosed. For example, Patent Document 1 describes a digital camera imaging device that automatically presses a release button by detecting a face image so that a person's face can be reliably photographed. Further, Patent Document 2 describes an imaging device that releases a shutter when a preset number of eyes are detected during self-timer shooting.<patcit num="1"><text>Japanese Unexamined Patent Publication No. 2003-92700</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 2004-336265</text></patcit>

<p> For example, in a typical wedding situation, when you want to shoot the moment when a couple kisses, you often cannot capture the moment well. On the other hand, recent advances in face detection technology have made it possible to instantly detect the faces of multiple people at once.</p><p> The present invention has been made in view of such circumstances, and in the above-mentioned situation, an image capable of capturing a moment when a couple kisses or a moment when a parent and a child are close to each other using the face detection function. The purpose is to provide the device.</p>

<p> In order to achieve the above object, the imaging apparatus according to the present invention includes an imaging means that captures a subject image and outputs the captured image data, a face detecting means that detects a person's face in the image data, and the face. The first is a predetermined face distance calculation means for calculating the distance between each face of a plurality of faces detected by the detection means, and a predetermined distance between the faces calculated by the face distance calculation means after receiving a shooting instruction. It is characterized in that it is provided with a control means for controlling the image pickup means so as to start photographing when the value becomes equal to or less than the threshold value of.</p><p> As a result, it is possible to capture the moment when the distance between the faces approaches within a predetermined threshold value.</p><p> In order to achieve the above object, the imaging apparatus according to the present invention includes an imaging means that captures a subject image and outputs the captured image data, a face detecting means that detects a person's face in the image data, and the face. The face distance calculation means for calculating the distance between each face of a plurality of faces detected by the detection means, and the distance between the faces calculated by the face distance calculation means after a shooting instruction is given with respect to the time change. It is characterized in that it is provided with a control means for controlling the imaging means so that shooting is started when the value reaches the minimum value.</p><p> This makes it possible to capture the moment when the faces are closest to each other.</p><p> A face size comparing means for comparing the sizes of a plurality of faces detected by the face detecting means is provided, and the face distance calculating means determines that the difference in size is equal to or less than a predetermined second threshold value. It is preferable to calculate the distance between the faces for the plurality of faces.</p><p> As a result, since the distance between the faces is calculated for the faces that are close in the depth direction from the imaging device, it is possible to prevent erroneous shooting when different faces overlap in the depth direction.</p><p> A face direction comparing means for comparing the orientations of a plurality of faces detected by the face detecting means is provided, and in the face distance calculating means, the face direction comparing means faces each other's faces or face each other. The distance between the faces for calculating the distance between the faces may be calculated for a plurality of faces determined to be either side faces or a combination of the front face and the side face facing the front face.</p><p> As a result, the distance between the faces is calculated with respect to the faces in a predetermined orientation, so that it is possible to shoot a more appropriate scene.</p><p> In order to achieve the above object, the imaging method according to the present invention includes an imaging step of capturing a subject image and outputting the captured image data, a face detection step of detecting a person's face in the image data, and the face. The first is a predetermined face distance calculation step for calculating the distance between each face of a plurality of faces detected by the detection step, and a predetermined distance between the faces calculated by the face distance calculation step after receiving a shooting instruction. It is characterized by including a control step for controlling the imaging step so that shooting is started when the value becomes equal to or less than the threshold value of.</p><p> As a result, it is possible to capture the moment when the distance between the faces approaches within a predetermined threshold value.</p><p> In order to achieve the above object, the imaging method according to the present invention includes an imaging step of capturing a subject image and outputting the captured image data, a face detection step of detecting a person's face in the image data, and the face. The face distance calculation step for calculating the distance between each face of a plurality of faces detected by the detection step and the distance between the faces calculated by the face distance calculation step after receiving a shooting instruction are subject to time change. It is characterized by including a control step for controlling the imaging step so that shooting is started when the value reaches the minimum value.</p><p> This makes it possible to capture the moment when the faces are closest to each other.</p>

<p> According to the present invention, the distance between the faces is calculated and the moment when the distance between the faces approaches within a predetermined threshold value is photographed. Therefore, the moment when the couple's face approaches and the moment of kissing is photographed in a situation such as a wedding ceremony. be able to. Further, since the distance between the faces is calculated for the faces having a short distance in the depth direction by comparing the sizes of the faces, it is possible to prevent erroneous shooting when the faces having different distances in the depth direction overlap. Further, by comparing the orientations of the faces, the distance between the faces is calculated for the combination in which the orientations of the faces are within a predetermined value, so that it is possible to shoot a more appropriate scene.</p>

Hereinafter, preferred embodiments of the imaging apparatus according to the present invention will be described with reference to the accompanying drawings.

&lt;First embodiment&gt; FIG. 1 is a block diagram showing an example of the internal configuration of the digital camera 1 according to the first embodiment of the present invention.

In the figure, the CPU 11 controls each circuit in the digital camera 1 based on the input of various operation switches, and executes processing according to the camera control program.

The control of each circuit of the CPU 11 is performed via the BUS 12. Further, the CPU 11 exchanges necessary data with and from the memory 13. The inside of the memory 13 is divided into a ROM area and a RAM area. In this ROM area, a GUI such as a camera control program, an opening image at startup, an ending image at stop, and a menu image used for operating the digital camera 1 is used. Image for screen saver, image for progress display during processing (image of hourglass whose scale changes, etc.), key operation sound (shutter sound, etc.), warning sound, voice data indicating error sound, etc. It has been recorded.

First, when the power is turned on, the CPU 11 detects this, turns on the power in the camera, displays the opening image stored in the ROM area of the memory 13 for a certain period of time, and then puts it in the shooting standby state in the shooting mode. In this shooting standby state, the CPU 11 causes the image display device 14 to display a moving image (through image).

The user (photographer) framing while looking at the through image displayed on the image display device 14, confirming the subject to be photographed, confirming the image after shooting, and setting the shooting conditions.

Further, in this shooting standby state, the face detection circuit 22 detects the face in the subject image and displays the detected face frame on the image display device 14. In the couple shooting mode described later, the face distance calculation circuit 23 calculates the distance between each face when there are a plurality of faces detected by the face detection circuit 22. The details of the face detection circuit 22 and the face distance calculation circuit 23 will be described later.

When the release button 15 is pressed during the shooting standby state, the CPU 11 performs focus control, metering, and exposure control based on the detection result of the AE / AF detection circuit 16, and zoom motor driver 17 and focus motor driver 18. The zoom lens 19 and the focus lens 20 are driven via these lenses, and a subject image is formed on the light receiving surface of the CCD 21 via these lenses. The CCD21 converts the subject image formed on the light receiving surface into a signal charge of an amount corresponding to the amount of light. This signal charge is read out to the shift register by the read gate pulse applied from the timing generator 25, and is sequentially read out as a voltage signal corresponding to the signal charge by the register transfer pulse.

In the voltage signal output from the CCD21, the analog R, G, and B signals are converted into digital R, G, and B signals by the A / D converter 26, respectively, and the RAM area of the memory 13 is temporarily converted by the image input controller 27. Stored in.

The image signal processing circuit 28 reads the raw data of R, G, and B stored in the RAM area of the memory 13 and applies digital gain to these data according to the light source type to adjust the white balance and gamma (gamma). R, G, B signals are generated by performing gradation conversion) processing, sharpness processing, etc. Further, YC signal processing is performed to generate a luminance signal Y, a chroma signal Cr, and a Cb (YC signal), which are stored in the RAM area of the memory 13 again.

The YC signal stored in the RAM area of the memory 13 as described above is compressed into a predetermined format by the compression processing circuit 29, and then recorded on the recording medium 31 detachable from the digital camera 1 via the media controller 30. Will be done.

When the playback mode is selected by operating the mode button 24, the image file of the final frame recorded on the recording medium 31 is read out via the media controller 30. The compressed data of the read image file is decompressed into an uncompressed YC signal via the compression processing circuit 29.

The stretched YC signal is converted into a signal format for display by the display circuit 32 and output to the image display device 14. As a result, the image display device 14 displays the image of the final frame recorded on the recording medium 31.

Next, the face detection circuit 22 and the face distance calculation circuit 23 will be described with reference to FIGS. 2 and 3.

FIG. 2 is a conceptual diagram of the face detection circuit 22. When the face detection circuit 22 inputs the image data of the subject image formed on the light receiving surface of the CCD 21 or the image data recorded on the recording medium 31, the face detection circuit 22 analyzes these image data to detect the face of a person. The number of detected faces and the position, size and orientation of each face are output.

FIG. 3 is a conceptual diagram of the face distance calculation circuit 23. When the position and size of each face in the image data are input, the face distance calculation circuit 23 calculates the distance between each face, and the calculated distance is 1% or less of the target face size. When there is a certain face combination, the exposure start signal is output.

Next, the operation of the digital camera 1 in the couple shooting mode will be described with reference to FIG. FIG. 4 is a flowchart showing the operation of the digital camera 1 in the couple shooting mode. When the digital camera 1 of the first embodiment according to the present invention is set to the couple shooting mode, after pressing the release button 15, in a combination of faces of the same size, shooting is performed when the distance between the faces is close. Will be done.

First, it is determined whether or not the release button 15 has been pressed (step S41). In the shooting standby state until the release button 15 is pressed, as described above, a through image is displayed on the image display device 14, the face detection circuit 22 detects the face in the subject image, and the detected face is frame-displayed. I do. FIG. 5 is a diagram showing an image display device 14 in which the detected face is displayed in a frame.

When the release button 15 is pressed, it is determined whether or not the couple shooting mode is set (step S42). If it is not in the couple shooting mode, normal shooting is performed (step S46), and the shooting data is recorded on the recording medium 31 (step S47).

When the couple shooting mode is set, it is determined whether or not the face detection circuit 22 has detected a plurality of faces (step S43). If the face detection circuit 22 does not detect a face, or if only one face is detected, face detection is continued until a plurality of faces are detected.

When multiple faces are detected, it is next determined whether or not there is a combination of face sizes within 100 ± 20% (step S44). As described above, the face detection circuit 22 can detect the face in the image data and at the same time calculate the size of the detected face, and the CPU 11 has a face size ratio of 100 from each face combination. Extract pairs within ± 20%.

In the case of FIG. 5, the face detection circuit 22 calculates the sizes A and B of each of the two detected faces, and the CPU 11 obtains the ratio A / B × 100 of the two face sizes, which is 100 ± 20. Determine if it is within%.

If there is no combination in which the face size ratio is within 100 ± 20%, the process returns to step S43, and face detection is performed again.

If there is a combination with a face size ratio of 100 ± 20% or less, it is determined whether the distance of the combination is smaller than 1% of the face size (step S45). The CPU 11 inputs the position information and size information of each face to the face distance calculation circuit 23 for the combination of faces whose face size ratio is within 100 ± 20%. The face distance calculation circuit 23 calculates the distance between the faces of the input face combination and compares it with the size of the face. If the calculated distance between faces is greater than 1% of the face size, the process returns to step S43, and face detection is performed again. When the calculated distance between faces is 1% or less of the face size, an exposure start signal is output.

In the case of FIG. 5, the face distance calculation circuit calculates the distance C between two faces and compares C with 1% of A. Here, 1% of B may be used, or the average value of A and B may be calculated and be 1% of this average value.

When the face distance calculation circuit 23 outputs an exposure start signal, the CPU 11 shoots based on this start signal (step S46), and the shooting data is recorded on the recording medium 31 (step S47).

In this couple shooting mode, if multiple faces are not detected, if there is no combination with a face size ratio of 100 ± 20% or less, and if the calculated distance between faces is greater than 1% of the face size, , Even if you press the release button 15, the shooting will not be performed and the same routine will be repeated, but this may be canceled by pressing the release button 15 again, or a time limit is set and the time limit is set. This shooting may be canceled after a lapse of time. Alternatively, normal shooting may be performed by pressing the release button 15 again.

In the present embodiment, the face size ratio is 100% ± 20% and the face distance is 1% or less of the face size as the conditions for shooting, but the present invention is not limited to these values. It may be decided as appropriate.

As for the face distance, an absolute value may be used regardless of the detected face size (for example, A and B in FIG. 5). For example, shooting may be performed when the distance between the faces is within 10 pixels.

In this way, by detecting that the face is approaching and then taking a picture, it is possible to take a picture without missing a photo opportunity such as a couple's kiss scene. Further, by calculating the distance for faces of the same size, it is possible to prevent erroneous shooting when a face separated in the depth direction approaches in the lateral direction.

&lt;Second embodiment&gt; The operation of the digital camera 1 according to the second embodiment of the present invention in the couple shooting mode will be described with reference to FIG. FIG. 6 is a flowchart showing the operation of the digital camera 1 in the couple shooting mode. In the couple shooting mode of the digital camera 1 according to the second embodiment of the present invention, after pressing the release button 15, the distance between the faces becomes the minimum value with respect to the time change in the combination of faces of the same size. Shooting is done at that time.

First, it is determined whether or not the release button 15 has been pressed (step S61). In the shooting standby state until the release button 15 is pressed, a through image is displayed on the image display device 14 and the face detection circuit 22 detects and detects a face in the subject image, as in the first embodiment. The frame of the face is displayed. FIG. 7 is a diagram showing the display of the image display device 14 at this time, and the detected face is displayed in a frame.

When the release button 15 is pressed, it is determined whether or not the couple shooting mode is set (step S62). If it is not in the couple shooting mode, normal shooting is performed (step S66), and the shooting data is recorded on the recording medium 31 (step S67).

When the couple shooting mode is set, it is determined whether or not the face detection circuit 22 has detected a plurality of faces (step S63). If the face detection circuit 22 does not detect a face, or if only one face is detected, face detection is continued until a plurality of faces are detected.

When multiple faces are detected, it is next determined whether or not there is a combination of face sizes within 100 ± 20% (step S64). Similar to the first embodiment, the CPU 11 detects a combination in which the face size ratio is within 100 ± 20% from the face size calculated by the face detection circuit 22.

If there is no combination in which the face size ratio is within 100 ± 20%, the process returns to step S63, and face detection is performed again.

If there is a combination in which the face size ratio is within 100 ± 20%, it is determined whether or not the distance L of the combination is the minimum value (step S65). The CPU 11 inputs the combination of faces whose face size ratio is within 100 ± 20% to the face distance calculation circuit 23. The face distance calculation circuit 23 of the second embodiment calculates the distance between the faces of the input face combination, and outputs an exposure start signal when the value becomes the minimum value with respect to the time change.

Regarding this minimum value, the distance between the faces is calculated at a predetermined time interval, and the value immediately before the increase in the distance is determined to be the minimum value. The method of determining the minimum value will be described with reference to FIG.

FIG. 7 is a diagram showing the display of the image display device 14 when the distance between the faces is calculated at a predetermined time interval during the couple shooting mode shooting. The display of the image display device 14 at time Ta is shown in FIG. 7 (a), the display of the image display device 14 at time Tb is shown in FIG. 7 (b), the display of the image display device 14 at time Tc is shown in FIG. 7 (c), and the time Td is displayed. FIG. 7 (d) shows the display of the image display device 14 in the above, showing how Ta Tb Tc Td have elapsed at predetermined time T intervals. In FIG. 7 (a), the distance between the faces is L1. In FIG. 7 (b) after the lapse of a predetermined time T from FIG. 7 (a), the distance between the faces is L2, which is a decrease from L1. In FIG. 7 (c) after the lapse of a predetermined time T from FIG. 7 (b), the distance between the faces is L3 (L3 &lt;0), which is further reduced from L2. In FIG. 7 (d) after the lapse of a predetermined time T from FIG. 7 (c), the distance between the faces is L4 (L4 &lt;0), which is larger than that of L3. Therefore, the face distance calculation circuit 23 determines that L3 is the minimum value of the distance, and outputs an exposure start signal.

When the face distance calculation circuit 23 outputs an exposure start signal, the CPU 11 shoots based on this output signal (step S66), and the shooting data is recorded on the recording medium 31 (step S67).

In this way, by detecting that the face is closest to the face and then taking a picture, it is possible to take a picture without missing a photo opportunity such as a couple's kiss scene. Further, by calculating the distance for faces of the same size, it is possible to prevent erroneous shooting when a face separated in the depth direction approaches in the lateral direction.

Further, the shorter the time interval T for calculating the distance between the faces, the more appropriate the timing can be taken, but the amount of calculation increases, so that the time interval T may be appropriately determined.

&lt;Third embodiment&gt; The operation of the digital camera 1 according to the third embodiment of the present invention in the couple shooting mode will be described with reference to FIG. FIG. 8 is a flowchart showing the operation of the digital camera 1 in the couple shooting mode. In the couple shooting mode of the digital camera 1 according to the third embodiment of the present invention, after pressing the release button 15, the faces of the same size and facing each other are face-to-face or face-to-face. Alternatively, in a combination determined to be either a front face or a profile facing the front face, the actual shooting is performed when the distance between the faces is short.

First, it is determined whether or not the release button 15 has been pressed (step S81). In shooting standby state until the release button 15 is depressed, as in the first embodiment, the through image is displayed on the image display device 14, or was the face detecting circuit 22 detects faces in an object image, The frame of the detected face is displayed.

When the release button 15 is pressed, it is determined whether or not the couple shooting mode is set (step S82). If it is not in the couple shooting mode, normal shooting is performed, but the description is omitted here.

When the couple shooting mode is set, it is determined whether or not the face detection circuit 22 has detected a plurality of faces (step S83). If the face detection circuit 22 does not detect a face, or if only one face is detected, face detection is continued until a plurality of faces are detected.

When multiple faces are detected, it is next determined whether or not there is a combination of face sizes within 100 ± 20% (step S84). If there is no combination in which the face size ratio is within 100 ± 20%, the process returns to step S83 and face detection is performed again.

If there are combinations with a face size ratio of 100 ± 20% or less, then the orientation of each face is determined for that combination (step S85). As described above, the face detection circuit 22 can detect the face in the image data and at the same time calculate the orientation of the detected face, and the CPU 11 can calculate the orientation of the detected face from the combinations of faces of the same size. Extract combinations with a face orientation angle of 90 degrees or less.

The angle of the orientation of the face will be described with reference to FIG. 9 (a), (b), (c), and (d) are diagrams showing the display of the image display device 14 during shooting in the couple shooting mode, and FIGS. 9 (a') and 9 (b'). , (C') and (d') are bird's-eye views for showing the result of analyzing the detected face orientation in the display of FIGS. 9 (a), (b), (c) and (d), respectively. is there. In the couple shooting mode of this digital camera 1, in the combination of faces of the same size, the faces of each face are facing each other, or the faces facing each other, or the front face and the front face. The next step, the distance between the faces, is calculated for any combination of the side faces facing the face, but whether or not each face is from the front to the direction of the other party is determined. Judgment is made based on whether or not the face is facing within 90 degrees. 9 (a'), (b'), (c'), and (d') show the angles within 90 degrees from the front to the direction of the other party with diagonal lines.

In the case of FIG. 9 (a), as shown in FIG. 9 (a'), since each other faces the front, it is determined that the faces are both within the diagonal line, and the process proceeds to the next step. In the case of FIG. 9 (b), as shown in FIG. 9 (b'), one faces the front and the other faces the vacant direction. Proceed to the step of. In FIG. 9 (c), as shown in FIG. 9 (c'), since each other faces the other's direction, it is determined that the direction of the face is within the diagonal line, and the process proceeds to the next step. In FIG. 9 (d), as shown in FIG. 9 (d'), the person on the right side is facing the front, but the person on the left side is not facing the other person, so that the person on the left side is facing the face. Is judged to be outside the diagonal line and does not proceed to the next step.

Here, the direction of the face is set within 90 degrees from the front to the direction of the other party, but the direction of this face is not limited to this angle, and even if they are not facing the other party, they are in the same direction. You may take a picture when you are facing.

Judgment is made regarding the orientation of the face as described above. If there is no combination in which the orientation of each face is within 90 degrees as shown in FIG. 9D, the process returns to step S83 and face detection is performed again.

If there is a combination of faces within 90 degrees as shown in Fig. 9 (a), (b), and (c), is the distance between the faces smaller than 10 pixels in the combination of faces? Is determined (step S86).

The CPU 11 inputs to the face distance calculation circuit 23 for a combination of faces whose face size ratio is within 100 ± 20% and whose face orientation is within 90 degrees. The face distance calculation circuit 23 calculates the distance between the faces of the input face combination. If the calculated distance between the faces is larger than 10 pixels, the process returns to step S83, and face detection is performed again. When the calculated distance between the faces is 10 pixels or less, the exposure start signal is output.

When the face distance calculation circuit 23 outputs an exposure start signal, the CPU 11 shoots based on this start signal (step S87), and the shooting data is recorded on the recording medium 31 (step S88).

When the recording of the shooting data is completed, the face distance calculation circuit 23 again calculates the distance between the faces of the same face combination and determines whether or not it is smaller than 10 pixels (step S89). Here, if the distance between the faces is still 10 pixels or less, the exposure start signal is output again and shooting is performed (step S87). When the distance of this face combination becomes larger than 10 pixels, the shooting ends.

In this way, by calculating the distance for a combination of the same size and each face orientation within a predetermined value, it is possible to shoot for a more optimal shutter chance. In addition, by continuing shooting while the face is approaching, it is possible to reliably shoot.

<figref num="1">FIG. 1 is a block diagram showing an example of the internal configuration of the digital camera 1 according to the first embodiment of the present invention.</figref><figref num="2">FIG. 2 is a conceptual diagram of the face detection circuit 22.</figref><figref num="3">FIG. 3 is a conceptual diagram of the face distance calculation circuit 23.</figref><figref num="4">FIG. 4 is a flowchart showing the operation of the digital camera 1 according to the first embodiment of the present invention in the couple shooting mode.</figref><figref num="5">FIG. 5 is a diagram showing a display of the image display device 14 in the couple shooting mode.</figref><figref num="6">FIG. 6 is a flowchart showing the operation of the digital camera 1 according to the second embodiment of the present invention in the couple shooting mode.</figref><figref num="7">FIG. 7 is a diagram showing a display of the image display device 14.</figref><figref num="8">FIG. 8 is a flowchart showing the operation of the digital camera 1 in the couple shooting mode.</figref><figref num="9">FIG. 9 is a diagram showing the display of the image display device 14 during shooting in the couple shooting mode, and a bird's-eye view for showing the result of analyzing the detected face orientation in each display.</figref>

Code description

1 ... Digital camera, 11 ... CPU, 12 ... BUS, 13 ... Memory, 14 ... Image display, 15 ... Shutter button, 21 ... CCD, 22 ... Face detection circuit, 23 ... face distance calculation circuit, 24 ... mode button, 25 ... timing generator, 26 ... A / D converter, 30 ... media controller, 31 ... recording media , 32 ... Display circuit