Image processing apparatus, image processing method, and program

The present invention relates to an image processing apparatus and an image processing method suitable for use in, for example, a digital camera.

Conventionally, as one type of image processing performed in digital cameras and the like, soft focus processing is known to create a soft atmosphere by adding blur to the original image similar to that caused by chromatic aberration and spherical aberration of a dedicated lens. Has been done. As a technique related to such soft focus processing, for example, in Patent Document 1 below, a person's face is detected from the original image so as not to be uniform regardless of the content of the original image, and the processing intensity is determined according to the size. The technology to control is disclosed.

<p num="0003"><patcit num="1"><text>Japanese Unexamined Patent Publication No. 2000-69277</text></patcit></p>

<p num="0004"> However, even if the processing intensity is controlled according to the size of the person's face as described above, the person's face and other parts cannot be blurred with an appropriate processing intensity. The same applies to subjects other than people and image processing other than soft focus, and there is a problem that the processing intensity of image processing for the main subject and other parts cannot be appropriately controlled.</p><p num="0005"> The present invention has been made in view of such conventional problems, and provides an image processing apparatus, an image processing method, and a program capable of appropriately controlling the processing intensity of an image in which a main subject is captured during image processing. The purpose is to do.</p>

<p num="0006"> In order to solve the above problems, in the image processing apparatus of the present invention, an acquisition means for acquiring the position and size of a noteworthy subject in the original image acquired by photographing, and the attention acquired by the acquisition means. A mask information generating means that generates mask information indicating a relationship between a position to be subjected to a predetermined image processing in the original image and its intensity based on the position and size of a subject to be created, and a mask generated by the mask information generating means. It is characterized by comprising an image generation means for generating a new image in which information is used to perform predetermined image processing of different intensities for each position in the original image.</p>

<p num="0007"> According to the present invention, it is possible to appropriately control the processing intensity of an image in which a main subject is captured during image processing.</p>

<figref num="1">It is a block diagram of a digital camera common to each embodiment of this invention.</figref><figref num="2">It is a flowchart which shows the shooting process in 1st Embodiment.</figref><figref num="3">(a) is a diagram showing a first mask image, and (b) is a diagram showing a second mask image.</figref><figref num="4">It is a figure which shows the example of the image generated by the photographing process in 1st Embodiment.</figref><figref num="5">It is a figure which shows the generation method of the 3rd mask image.</figref><figref num="6">It is a figure which shows the other form of the 2nd mask image.</figref><figref num="7">It is a flowchart which shows the shooting process in 2nd Embodiment.</figref>

Hereinafter, embodiments of the present invention will be described.

(Embodiment 1) First, the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing an electrical configuration of a digital camera 1 as an embodiment of an image processing device according to this embodiment and a second embodiment described later.

As shown in FIG. 1, the digital camera 1 includes a control unit 2, a lens unit 3, an imaging unit 4, a display unit 5, an image storage unit 6, a program storage unit 7, a face detection unit 8, an operation unit 9, and a power supply. Each part of part 10 is provided.

The control unit 2 includes a CPU (Central Processing Unit) and its peripheral circuits, an internal memory for work such as RAM (Random Access memory), and an image processing circuit, and controls each part of the digital camera 1 and will be described later. Various image processing is also performed.

The lens unit 3 includes a lens group including lenses for focus adjustment and zoom, a motor for driving the lens group, an actuator for opening and closing the diaphragm and an actuator for adjusting the opening degree, and the like.

The image sensor 4 performs predetermined analog processing on a CCD (Charge Coupled Device) or CMOS (Complementary Meta10xide Semiconductor) type image sensor and an image sensor output from the image sensor, and then converts the image sensor into a digital signal. It consists of AFE (Analog Front End). While the shooting mode is set in the digital camera 1, the imaging unit 4 sequentially images the subject through the lens group of the lens unit 3 at a predetermined frame rate, and controls the image data (imaging data) of the subject 2 Supply to.

The image data supplied to the control unit 2 is supplied to the display unit 5 and displayed as a live view image on the display unit 5.

The image data supplied from the imaging unit 4 to the control unit 2 at the time of shooting, which is executed triggered by the shooting execution instruction by the operation unit 9, which will be described later, is gamma correction, white balance adjustment, and R, G, B color components for each pixel. After being subjected to development processing consisting of data generation, YUV conversion, etc., it is compressed by the JPEG (Joint Photographic Expert Group) method. Various attribute information is added to the compressed image data, and the compressed image data is stored in the image storage unit 6 as a still image file conforming to the Exif (Exchangeable Image File Format) standard.

The image storage unit 6 is composed of, for example, a flash memory built in the digital camera 1, various memory cards that can be attached to and detached from the digital camera 1, and a card interface that enables data input / output to / from the memory card.

The image data stored as a still image file in the image storage unit 6 is read out by the control unit 2 as necessary while the playback mode is set in the digital camera 1, expanded, and then displayed in the display unit 5. It is supplied and displayed on the screen.

The display unit 5 includes a color liquid crystal display panel and a display drive circuit that drives the color liquid crystal display panel according to image data or the like supplied from the control unit 2. As described above, the display unit 5 displays the subject image in a live view in the shooting mode, and displays an existing shot image composed of image data stored as a still image file in the image storage unit 6 in the playback mode.

The program storage unit 7 is composed of, for example, a ROM (Read Only Memory) or a non-volatile memory such as a flash memory in which stored data can be rewritten at any time. The program storage unit 7 stores in advance a program for causing the control unit 2 to perform a process described later.

Further, the program storage unit 7 is used for a program for causing the control unit 2 to perform AE (Auto Exposure) control, AF (Auto Focus) control, AWB (Auto White Balance) control, and AE control. Various data such as data constituting a program diagram showing a combination of shutter speed, ISO sensitivity, and aperture value are also stored. The AF control by the control unit 2 is a control by a so-called contrast detection method in which the image sensor of the image pickup unit 4 is used as an AF sensor.

Further, the program storage unit 7 stores a program for causing the control unit 2 to perform a photographing process described later, and mask data 101 used at that time. The mask data 101 is image data representing the first mask image M1 shown in FIG. 3A, and is grayscale image data.

Further, the first mask image M1 is an image corresponding to the second mask image M2 shown in FIG. 3B divided into four in the circumferential direction, and the second mask image M2 is a solid black image. In the background, the image has an elliptical gradation region in which the brightness of each pixel is maximized at the center and gradually decreases toward the periphery.

Here, in the present embodiment, the second mask image M2 corresponds to the basic information of the present invention used for generating the third mask image M3 described later, and the shape (elliptical shape) of the gradation region is the present. It is a basic shape of a processing area to which a predetermined image processing is performed in the present invention. Further, it is shown that the change in the brightness of each pixel in the gradation region is the change in the intensity of the predetermined image processing in the present invention, and specifically, the processing intensity gradually increases from the center side to the peripheral side. The content.

The face detection unit 8 has an image processing circuit for performing face detection processing for detecting the face portion of an arbitrary person from the image captured by the imaging unit 4, and a memory for storing various data used for the face detection processing. It is composed of a storage memory for work and the like, and supplies area information indicating an area corresponding to the face portion to the control unit 2.

The face detection process in the face detection unit 8 searches for eyes, nose, and mouth that are in a predetermined positional relationship using image recognition techniques such as binarization, contour extraction, and pattern matching, and a rectangular region in which they exist (hereinafter,). , A face region), and this is a known general-purpose process for detecting that region as a face portion.

The operation unit 9 has a plurality of operation switches such as a power button, a shutter button, a mode changeover switch used for switching between a shooting mode and a playback mode, a zoom lever, and a control button used for selecting and setting various functions. Consists of. The operation status of the operation switch in the operation unit 9 is monitored by the control unit 2 at any time.

The power supply unit 10 is composed of a rechargeable battery, a DC / DC converter, and the like, and supplies power required for each operation to each unit of the digital camera 1.

On the other hand, the digital camera 1 of the present embodiment is provided with a soft focus function as a function that can be used in the shooting mode. The soft focus function is a function that automatically performs a predetermined process including a soft focus process similar to the conventional one on an image obtained at the time of imaging, and records the processed image as a final captured image.

Then, in the digital camera 1, in the shooting mode when the use of the soft focus function is set by the user, the control unit 2 shows the flowchart in FIG. 2 according to the program stored in the program storage unit 7. Execute the shooting process.

That is, the control unit 2 sets the shooting mode, captures the subject image by the imaging unit 4, starts the live view display on the display unit 5, and enters the shooting standby state. As shown in FIG. 2, the user instructs the user to execute the shooting. Until it is done (step SA1: NO), the shooting standby state is continued. Although not shown, the control unit 2 performs AF control in the shooting standby state.

After that, when the user instructs to execute the shooting (step SA1: YES), the control unit 2 applies standard development processing for recording the image to the data of the captured image obtained by the imaging unit 4 to perform standard development. Generate an image (step SA2).

Subsequently, the control unit 2 performs soft focus processing on the generated standard developed image, and generates a soft focus image, that is, a processed image with the same blur added due to chromatic aberration, spherical aberration, etc. of the dedicated lens. (Step SA3). The soft focus process is a process using a known image processing technique consisting of one or a plurality of steps of the process.

Next, the control unit 2 causes the face detection unit 8 to perform face detection processing on the standard developed image as a processing target (step SA4).

Here, if the face detection unit 8 cannot detect the face of an arbitrary person from the standard developed image (step SA5: NO), the control unit 2 uses the above-mentioned soft focus image as a captured image, that is, finalizes the soft focus image. After performing compression processing and adding attribute information as a typical recorded image, the image is saved as a still image file in the image storage unit 6 (step SA6), and the shooting process is completed.

On the other hand, when the face of an arbitrary person is detected in the above standard developed image (step SA5: YES), for example, the standard developed image G1 shown in FIG. 4 is generated by the process of step SA2, and the process of step SA3. When the soft focus image G2 shown in FIG. 4 is generated, the control unit 2 executes the processes after step SA7.

Here, the outline will be described prior to the specific processing contents after step SA7. That is, the control unit 2 generates the second mask image M2 of FIG. 3 (b) using the first mask image M1 of FIG. 3 (a) described above, and uses the second mask image M2 of FIG. 3 (a). The third mask image M3 shown in FIG. 4 is generated. The third mask image M3 is a grayscale image like the first mask image M1 and has the same image size (number of pixels) as the standard developed image G1.

After that, the control unit 2 transfers the transparency of each pixel of the standard developed image G1 according to the brightness of each pixel of the third mask image M3, that is, the background side when the standard developed image G1 is superimposed on the background image. The composite image G3 shown in FIG. 4 is generated by setting the degree to which the image of 1 is transparent and synthesizing the standard developed image G1 with the soft focus image G2 as the background. That is, the standard developed image G1 is synthesized with the background soft focus image G2 by a known α blend to generate the composite image G3. The ellipse X shown by the broken line in the composite image G3 in FIG. 4 conveniently shows the region corresponding to the gradation region (effect reduction region described later) in the third mask image M3 described above.

Hereinafter, the processing after step SA7 will be specifically described. First, the control unit 2 acquires the center position and the size of the face region detected by the face detection unit 8 in the standard developed image G1 (step SA7). FIG. 5A is a diagram showing a standard developed image G1, a face region W, and a center O thereof. In the process of step SA7, the control unit 2 acquires the vertical size A of the face area W as the size of the face area W, and acquires the coordinate position in the image as the position of the center O of the face area W.

Next, the control unit 2 determines the size of the second mask image M2 when generating the third mask image M3 shown in FIG. 4 based on the acquired data (coordinate positions of the size A and the center O). Calculate the center position (step SA8). The size of the second mask image M2 calculated here is the vertical size C of the portion corresponding to the second mask image M2 in the third mask image M3 generated later, and the center position is the first. It is the coordinate position in the third mask image M3 of the center P of the mask image M2 of 2.

In the process of step SA8, the control unit 2 calculates the vertical size C by multiplying the vertical size A of the face area W by a predetermined multiple. The example shown in FIG. 5 (b) is an example in which the above-mentioned predetermined multiple is doubled.

Further, in the process of step SA8, the control unit 2 sets the coordinate position of the center O of the face region W in the standard developed image G1 as the coordinate position of the center P of the second mask image M2 in the third mask image M3. The coordinate position moved upward by a predetermined ratio (n%) of the vertical size A is calculated. For example, if the coordinate position of the center of the face area W is (x0, y0), the movement amount B in the y-axis direction is A × n%, and the coordinate position of the center P of the second mask image M2 is (x0, y0). -B) is acquired. The example shown in FIG. 5 (b) is an example when the above predetermined ratio is set to 40%.

Here, the predetermined multiple and the predetermined ratio described above take into consideration the difference between the face region W specified by the face detection process in step SA4 and the face portion of the actual person excluding the hair portion in the standard developed image G1. It is a determined value, specifically, a value determined based on an empirical rule.

Subsequently, the control unit 2 processes the first mask image M1 shown in FIG. 3A based on the above-mentioned calculated data (vertical size C of the second mask image M2 and the coordinate position of the center P). , Generate the third mask image M3 shown in FIG. 4, which is a mask image for image processing (step SA9).

Specifically, the control unit 2 first flips the first mask image M1 vertically and horizontally to generate three new mask images, and connects them to the first mask image M1. Generates the second mask image M2 shown in FIG. 3 (b). Next, the control unit 2 enlarges the generated second mask image M2 to a size in which the vertical size becomes the above-mentioned vertical size C, smoothes the enlarged second mask image M2, and performs each. Ensure a state in which the brightness value of the pixel changes smoothly.

After that, the control unit 2 uses the smoothed second mask image M2 as a background with a grayscale black color image having the same pixel size as the standard developed image G1, and the center P is the previously calculated coordinates. A third mask image M3 is generated by overwriting and synthesizing the image while it is placed at the position. At that time, the surplus portion protruding from the black image is trimmed on the enlarged and smoothed second mask image M2.

Then, after generating the third mask image M3, the control unit 2 generates the composite image G3 by synthesizing the standard developed image G1 with the soft focus image G2 by α blend using the third mask image M3. (Step SA10).

At that time, the control unit 2 sets the transmittance corresponding to the brightness of each pixel of the third mask image M3 to each pixel of the standard developed image G1 as described above. More specifically, each pixel of the standard developed image G1 has a transmittance of 0% to 100% according to the brightness value of the corresponding pixel of the third mask image M3, that is, a pixel having a higher brightness of the corresponding pixel. Set a low transmittance. That is, the minimum transmittance (0%: completely opaque) is set for the pixel at the center P corresponding to the second mask image M2, and the transmittance that gradually increases toward the periphery of the gradation region is set for the other pixels. Then, the maximum transmittance (100%: completely transparent) is set in the other areas of the third mask image M3 excluding the gradation area.

When the standard developed image G1 is α-blended with the soft focus image G2, the RGB value of each pixel is calculated using the α value (1 to 0) corresponding to the above-mentioned transmittance.

As a result, as shown in FIG. 4, while ensuring the effect of soft focus on the entire image, the bokeh effect of the soft focus process on the face of the person is reduced compared to other parts, and the intensity of the bokeh effect is the main focus. A composite image G3 is obtained in which the degree of reduction of the bokeh effect gradually decreases from the side toward the periphery, in other words, the degree of reduction of the blur effect gradually decreases toward the periphery.

After that, the control unit 2 compresses the generated composite image G3 as a captured image, that is, a final recorded image, adds attribute information, and then saves the generated composite image G3 as a still image file in the image storage unit 6 (step SA11). ), End the shooting process.

Therefore, in the present embodiment, when the main subject at the time of shooting using the soft focus function, that is, the subject to be noted in the shot image is a person, the processing on the face portion of the person is performed regardless of the processing intensity of the soft focus. The intensity, that is, the bokeh effect can be appropriately controlled, and as a result, a good captured image can be obtained as a recorded image.

Moreover, in the final captured image (composite image G3) obtained as a recorded image, the degree of reduction of the blur effect by the soft focus processing gradually decreases from almost the center of the person's face toward the periphery, and the final image is obtained. In a typical photographed image, since there is no boundary of the degree of blur between the face portion of the person and another area, it is possible to secure a natural effect of the photographed image.

Further, by setting the position of the face area W obtained by the general-purpose face detection process as the position of the face portion of the person in the standard developed image G1, the processing load required for acquiring the position can be lightened.

Further, when generating the third mask image M3 described above, the center P of the gradation region in the third mask image M3 is located at a position different from the center O of the face region W as described above (upper side in the present embodiment). ), So even if you use the technique of detecting the face of a known person in the area, the position of the gradation area, that is, the position of the area that reduces the blur effect due to the soft focus processing (hereinafter referred to as the effect reduction area). Can be matched with the face part of the actual person. As a result, it is possible to obtain a better photographed image as the final photographed image while reducing the processing load required for acquiring the position of the face portion of the person.

Further, when the third mask image M3 is generated, the second mask image M2 is enlarged according to the size of the face area W, so that the size of the effect reduction area (gradation area) is set to the size of the human face. It can be sized to reliably cover the portion. This also makes it possible to obtain a better photographed image as the final photographed image.

Further, when the main subject is an object other than a person, for example, a landscape, it is possible to obtain a captured image (soft focus image) in which the effect of soft focus is ensured on the entire screen as in the conventional case.

Further, since the third mask image M3 is generated only when the main subject is a person, when the main subject is an object other than the person, the useless processing required for generating the third mask image M3 is performed. Can be reduced.

On the other hand, a third mask image M3 is generated based on the first mask image M1 stored in advance. In other words, since the basic shape, which is the basic information of the effect reduction region described above, and the change contents of the reduction degree of the processing effect are stored in advance as the first mask image M1, it is necessary to generate the third mask image M3. Processing can be reduced.

Further, since the first mask image M1 is an image corresponding to the second mask image M2 divided into four, the basic information of the third mask image M3 can be efficiently stored.

Here, in the present embodiment, the processing assuming that the standard developed image G1 has one face of a person has been described, but when the standard developed image G1 has a plurality of faces, the process has been described. , When generating the third mask image M3, it is sufficient to secure a plurality of effect reduction areas according to the position and size of each face. Further, in that case, when an overlapping region in which a plurality of effect reducing regions overlap occurs, the degree of reduction of the processing effect in the overlapping region may be set as the degree of reduction of the effect reducing region on the larger side.

Further, in the present embodiment, the position and size of the effect reduction area secured in the third mask image M3 are adjusted according to the position and size of the face of the person (the size of the face area W), but the effect is further improved. The shape of the reduction region may be adjusted according to the shape of the face of the person. Specifically, the ratio of the vertical size to the horizontal size of the face region W, that is, the flatness of the effect reduction region (elliptical region) may be adjusted between the thin face and the thick face. In that case, an accurate effect reduction region can be secured by the third mask image M3. It should be noted that the difference in the shape of the face of a person includes not only the literal shape (thick or thin) due to the individual difference of the person, but also the one caused by the difference in the shooting direction.

Further, in the present embodiment, it is assumed that the orientation of the person's face and the orientation of the image match, but for example, when a person in a lying state is photographed, the person's face is in the sideways orientation. The vertical direction may not match the orientation of the image. Therefore, in carrying out the present invention, for example, in the face detection process described above, the orientation of the person's face, that is, the rotation direction and the amount of rotation in the image is acquired, and in the generation of the third mask image M3, the person's face is generated. It is desirable to rotate the second mask image M2 according to the orientation of the face. This also makes it possible to secure an accurate effect reduction area by the third mask image M3.

Further, as described above, when adjusting the shape and orientation of the effect reduction area secured in the third mask image M3 according to the ratio of the vertical size and the horizontal size of the face area W and the orientation of the person's face, If a plurality of first mask images M1 having different shapes and orientations of the effect reduction region are stored in advance and selectively used to generate the third mask image M3, the processing required for the generation is performed. Can be reduced.

Further, in the present embodiment, the second mask image M2 shown in FIG. 3B is used for the generation of the third mask image M3, but the present invention is not limited to this, and the generation of the third mask image M3 is not limited to this. Can also use, for example, another mask image M21 having a gradation region in which the maximum brightness pixel position Q is eccentric within an elliptical region as shown in FIG.

Even in that case, by gradually reducing the change in the degree of reduction from the center side to the peripheral side, as in the present embodiment, the final captured image is between the face portion of the person and other regions. Since there is no boundary between the degree of blurring, a natural effect can be ensured by the captured image.

In that case, when the third mask image M3 is generated, the pixel position Q is arranged so as to be the position of the center P of the second mask image M2 shown in FIG. 5B in the present embodiment. Then, the effect reduction area can be matched with the face part of the actual person. Further, if the first mask image M1 is an image obtained by dividing the other mask image M21 into left and right in the figure, the basic information of the third mask image M3 can be efficiently stored.

Further, although the case where the shape of the effect reduction region secured in the third mask image M3 is elliptical has been described, the shape of the effect reduction region can be changed as appropriate.

That is, when the target of the effect reduction area is other than the face of a person, for example, an animal such as a cat is horizontally long even if it is oval, a flower is a perfect circle, and a building is a rectangle. The shape of the effect reduction region may be appropriately changed according to the subject of interest or the shooting mode in which shooting is performed according to the subject of interest.

(Embodiment 2) Next, a second embodiment of the present invention will be described. Unlike the first embodiment, the subject that is in focus by AF control at the time of shooting is the main subject, that is, the subject that should be noted. That is, in the present embodiment, in the digital camera 1 having the configuration shown in FIG. 1, the following data and programs are stored in the program storage unit 7 in addition to the various data described above.

That is, in the present embodiment, the program storage unit 7 stores a distance acquisition table showing the relationship between the adjustment position of the lens group in the lens unit 3 and the focus distance (distance to the subject in focus). In addition, the program storage unit 7 has a program for causing the control unit 2 to perform AF control by so-called continuous AF that continuously focuses on a subject that is relatively moving in the image, and a program for causing the control unit 2 to perform AF control by so-called continuous AF. A program for causing the control unit 2 to perform the shooting process shown in FIG. 7 in the shooting mode when the use of the soft focus function is set by the user is stored.

Here, the difference between the shooting process in the present embodiment and the shooting process in the first embodiment will be described first. In the present embodiment, the control unit 2 focuses on AF control without performing face detection at the time of shooting. A third mask image corresponding to the distance to the subject (hereinafter referred to as the focus distance) and the in-focus position is generated, and the final captured image (recorded image) is generated using the third mask image.

Next, the processing of the control unit 2 in the present embodiment will be specifically described with reference to the flowchart of FIG. As is clear from the figure, the processing of steps SB1 to SB3 is the same as that of the first embodiment.

That is, when the user instructs the user to execute shooting (step SB1: YES) in the shooting standby state accompanied by the live view display and AF control, the control unit 2 causes the imaging unit 4 to capture an image for recording, and the captured image is captured. The standard development process is performed on the data of (step SB2) to generate a standard development image, and the entire generated standard development image is subjected to soft focus processing to generate a soft focus image (step SB3).

Next, the control unit 2 acquires the focus distance at the time when the shooting execution is instructed and the in-focus position, that is, the coordinates in focus in the image (step SB4). The focus distance is acquired from the above-mentioned distance acquisition table based on the adjustment position of the lens group at the time when the shooting execution is instructed.

Next, the control unit 2 acquires the size and center position of the second mask image M2 (see FIG. 5), which is the reference mask image, based on the above acquired data (focus distance and focus position) (see FIG. 5). Step SB5).

The size of the second mask image M2 acquired here is the vertical size of the portion corresponding to the second mask image M2 in the third mask image (not shown) generated later, as in the first embodiment. Is. Further, the vertical size acquired at this time changes according to the focal length and the focal length of the lens unit 3, that is, the zoom magnification. It becomes smaller as the focus distance becomes longer or as the focal length of the lens unit 3 becomes longer. The control unit 2 uses the focus distance and the focal length of the lens unit 3 as parameters, and as the focus distance becomes longer, the focus distance becomes smaller. Alternatively, the vertical size is acquired by calculation using a predetermined function in which the vertical size becomes smaller as the focal length of the lens unit 3 becomes longer. That is, the control unit 2 adjusts the size of the portion corresponding to the second mask image M2 in the third mask image so that it becomes smaller as the focal length becomes longer or as the focal length of the lens unit 3 becomes longer. To do.

Subsequently, the control unit 2 processes the first mask image M1 shown in FIG. 3A based on the above acquired data (vertical size of the second mask image M2 and the focusing position), and performs image processing. Generate a third mask image, which is a mask image for (step SB6).

That is, the control unit 2 generates the second mask image M2 from the first mask image M1 as in the first embodiment, and then expands the size to a size in which the vertical size becomes the previously acquired vertical size. Then, the enlarged second mask image M2 is smoothed. After that, the control unit 2 puts the smoothed second mask image M2 into a grayscale black solid color image having the same pixel size as the standard developed image, and puts its center P at the previously calculated focusing position. A third mask image is generated by overwriting and compositing in the arranged state. In this embodiment as well, as illustrated in FIG. 5B, if the enlarged image has a surplus portion that extends beyond the black image, the surplus portion is trimmed.

Then, the control unit 2 uses the generated third mask image M3 to synthesize the soft focus image generated in step SB3 with the standard developed image by α-blending to generate a composite image (step SB7). The specific processing is the same as that of the first embodiment.

As a result, it is possible to obtain an image in which the subject portion that was in focus at the time of shooting is not blurred while ensuring the effect of soft focus on the entire image as a composite image.

After that, the control unit 2 compresses the generated composite image as a captured image (recorded image), adds attribute information, saves it as a still image file in the image storage unit 6 (step SB8), and captures the image. To finish.

Therefore, also in the present embodiment, when shooting using the soft focus function, if the main subject that the photographer is paying attention to is a person, the face of the person is secured while ensuring the effect of soft focus on the entire image. It is possible to obtain a photographed image in which the portion is not blurred. Further, when the main subject is other than a person, it is possible to obtain a captured image in which the subject portion is not blurred. That is, also in the present embodiment, the processing intensity for the main subject can be appropriately controlled regardless of the intensity of the soft focus, and as a result, a good captured image can be obtained.

Further, also in the present embodiment, the second mask image M2 is an image having an elliptical gradation region, and the effect reduction region secured in the third mask image M3 is substantially the center of the focused subject portion. It gradually decreases from the to the periphery. Therefore, in the final captured image, there is no boundary of the degree of blur between the focused subject portion and the other region, so that a natural effect can be ensured by the captured image.

Moreover, in the present embodiment, when the third mask image M3 is generated, the size of the second mask image M2 is adjusted according to the focal length or the focal length of the lens unit 3. Therefore, when the main subject (notable subject) is assumed to have an actual size such as a person's face or flowers, the third mask image M3 does not require complicated subject recognition processing. The size of the effect reduction area to be secured can be set to an appropriate size.

In the shooting process described in the present embodiment, a composite image obtained by synthesizing a soft focus image with a standard developed image by α-blending is always recorded as a shot image. However, for example, in the digital camera 1, as a lower operation mode of the shooting mode, the shooting conditions and the contents of the development processing described above are automatically set to the contents suitable for shooting a specific object such as the portrait shooting mode. When the configuration has a specific shooting mode, the control unit 2 may perform the following shooting processing.

That is, at the time of shooting using the soft focus function, the control unit 2 is made to confirm whether or not the shooting mode is a specific shooting mode after generating the soft focus image by the process of step SB3 described above. Then, when the shooting mode is a specific shooting mode, a composite image obtained by synthesizing a standard developed image with a soft focus image by α blending is recorded as a shooting image, and conversely, when the shooting mode is not a specific shooting mode, the generated soft focus image is recorded. The control unit 2 may be allowed to perform the process of recording the captured image as it is.

Then, only when the shooting mode is a specific shooting mode, when recording a composite image obtained by combining a soft focus image with a standard developed image by α blending as a shooting image, the actual size of the main subject is determined from the shooting mode. Can be assumed in advance. Therefore, in such a case, it is necessary to ensure that the size of the effect reduction area to be secured in the third mask image M3 is an appropriate size corresponding to the actual size of the main subject as described above. it can.

Further, also in the second embodiment, the case where the shape of the effect reduction region secured in the third mask image M3 is elliptical has been described, but as in the first embodiment, the shape of the effect reduction region is appropriately formed. Can be changed.

Further, for example, when a shooting mode suitable for shooting a specific object is provided as a lower operation mode of the shooting mode as described above, the shape of the effect reduction region is set to the type of those shooting modes. The shape may be adjusted accordingly. In that case, as the first mask image M1 representing the basic shape of the effect reduction region, a plurality of first mask images representing different basic shapes corresponding to the shooting modes may be prepared in advance.

Here, in the first and second embodiments described above, the first mask image M1 is the program storage unit 7 as mask information representing the basic shape and the like of the effect reduction region to be secured in the third mask image M3. The configuration for generating the second mask image M2 from the first mask image M1 was described. However, the second mask image M2 may be stored in the program storage unit 7 as mask information indicating the basic shape of the effect reduction region.

Further, in the first and second embodiments, when the standard developed image acquired at the time of shooting and the soft focus image generated from the standard developed image are combined by α blend, the soft focus image is used as the background. In carrying out the invention, a standard developed image may be used as a background. In that case, for example, an image in which black and white are inverted may be generated as the third mask image M3.

In either case, the standard developed image and the soft focus image are combined using the third mask image M3, that is, the mask information representing the effect reduction region for reducing the blur effect due to the soft focus processing, and finally. Since the captured image is generated, for example, compared to the case where the effect is changed for each pixel from the standard developed image and the final captured image is directly generated, even if the image processing is relatively simple, the desired final image is obtained. The captured image can be easily generated. In particular, since the standard developed image and the soft focus image are combined by α blending, the desired final captured image can be generated by extremely simple image processing.

Further, in the first and second embodiments, a case where the predetermined image processing performed on the image obtained at the time of imaging is a soft focus processing for obtaining a blurring effect of adding blur to create a soft atmosphere will be described. did. However, the present invention can also be applied when the predetermined image processing is an image processing other than the soft focus processing.

The image processing may be, for example, a process for adding a specific hue to the original image or adjusting the brightness, or a process for converting the original image into a painting-like image. Further, the effect of the predetermined image processing does not have to extend to the entire original image, and may be an effect secured in a specific part of the original image such as a cross screen effect.

Further, in the first and second embodiments, a case where a new composite image is generated using the standard developed image acquired at the time of shooting as the original image has been described. However, the original image in the present invention may be a photographed image already recorded in the image storage unit 6 as long as it is acquired by photographing.

Even in that case, the composite image G3 as shown in FIG. 4 can be generated by the same processing as in the first embodiment. Further, if the focus position, focus distance, and the like at the time of shooting are stored in the still image file as additional information of the shot image, the subject to be noted can be specified by the same method as in the second embodiment.

Further, the present invention can be implemented not only in the digital camera 1 but also in other devices. Other devices include, for example, digital photo frames, general-purpose personal computers, and smartphones whose main purpose is to display captured images taken by an arbitrary digital camera for viewing.

Further, as described above, the original image in the present invention may be an image obtained by photographing, regardless of, for example, when the present invention is carried out with a digital camera or when it is carried out with another device. The original image may be a captured image that has already undergone arbitrary image processing.

Although some embodiments of the present invention and modified examples thereof have been described above, these can be appropriately modified as long as the effects of the present invention can be obtained, and the modified embodiments are also claimed for patent. The invention is included in the scope of the invention described in the above, and the invention equivalent to the invention. The inventions described in the claims of the present application are added below. [Claim 1] An acquisition means for acquiring the position and size of a noteworthy subject in the original image acquired by shooting, and Based on the position and size of the notable subject acquired by the acquisition means, the mask information generation means for generating mask information indicating the relationship between the position where a predetermined image processing is performed on the original image and the intensity thereof, and the mask information generation means. An image generation means that uses the mask information generated by the mask information generation means to generate a new image in which predetermined image processing of different intensities is applied to each position in the original image. An image processing device comprising. [Claim 2] Further provided with a detection means for detecting the notable subject as a region in the original image, the present invention is further provided. The acquisition means acquires the position and size of the notable subject area detected by the detection means in the original image. The image processing apparatus according to claim 1. [Claim 3] The mask information generation means generates mask information representing an effect reduction region indicating a relationship between a position where the processing effect of image processing in the original image should be reduced and the degree of reduction thereof. The image generation means uses the mask information generated by the mask information generation means to ensure the processing effect of the predetermined image processing on the original image and to the portion corresponding to the notable subject. Generate a new image with reduced processing effect 2. The image processing apparatus according to claim 2. [Claim 4] The image processing apparatus according to claim 2 or 3, wherein the mask information generation means generates mask information for performing image processing at a center position whose center position is different from that of the subject area acquired by the acquisition means. [Claim 5] The image processing apparatus according to claim 3 or 4, wherein the mask information generating means generates the mask information representing the effect reducing region having a different size depending on the size of the subject region. [Claim 6] The acquisition means further acquires the orientation of the subject of interest in the subject area detected by the detection means. The mask information generation means generates the mask information indicating the relationship between the position where a predetermined image processing is performed on the original image and the intensity thereof according to the orientation of the subject acquired by the acquisition means. The image processing apparatus according to any one of claims 2 to 5, wherein the image processing apparatus is characterized in that. [Claim 7] The mask information generating means generates the mask information on condition that the subject of interest can be detected by the detecting means. The image generation means generates a new image on condition that a notable subject can be detected by the detection means. The image processing apparatus according to any one of claims 2 to 6, wherein the image processing apparatus is characterized by the above. [Claim 8] The seventh aspect of claim 7, wherein the image generation means generates a processed image obtained by subjecting the original image to the predetermined image processing as a new image when the subject of interest cannot be detected by the detection means. Image processing device. [Claim 9] Further provided with setting means for presetting the shape of the processing area to which the predetermined image processing is performed according to the type of the subject to be noticed to be detected by the detection means. The mask information generating means specifies a processing area corresponding to the position and size of the notable subject area detected by the detecting means and having a shape preset by the setting means, and also specifies the processing area. Generates mask information indicating the relationship between each position in the processed processing area and the intensity of a predetermined image processing. The image processing apparatus according to any one of claims 2 to 8. [Claim 10] The setting means is a ratio of the shape of the processing area to which the predetermined image processing is performed and the distance from the predetermined reference point in the processing area according to the type of the subject to be noticed to be detected by the detection means. Further set the intensity of image processing according to The mask information generating means is based on the intensity of image processing according to the ratio of the size of the notable subject area detected by the detecting means to the distance from a predetermined reference point preset by the setting means. , Generates mask information indicating the relationship between each position in the specified processing area and the intensity of a predetermined image processing. 9. The image processing apparatus according to claim 9. [Claim 11] The setting means is a ratio of the shape of the processing area to which the predetermined image processing is performed and the distance from the predetermined reference point in the processing area according to the type of the subject to be noticed to be detected by the detection means. Predetermined intensity of image processing according to The mask information generating means performs predetermined image processing according to the ratio of the size of the notable subject area detected by the detecting means to the direction and distance from a predetermined reference point preset by the setting means. Based on the intensity, mask information indicating the relationship between each position in the specified processing area and the intensity of a predetermined image processing is generated. 10. The image processing apparatus according to claim 10. [Claim 12] Specific means for identifying the subject that was the target of focus adjustment in the original image acquired by shooting as a subject of interest, and An acquisition means for acquiring the focus information of the subject specified by the specific means, and Based on the focus information acquired by the acquisition means, the mask information generation means that generates mask information indicating the relationship between the position where a predetermined image processing is performed on the original image and the intensity thereof, and the mask information generation means. An image generation means that uses the mask information generated by the mask information generation means to generate a new image in which predetermined image processing of different intensities is applied to each position in the original image. An image processing device comprising. [Claim 13] The acquisition means further acquires the focus distance and / or the focal length of the lens as the focus information of the subject, in addition to the position of the subject in the original image when the focus adjustment is performed. The mask information generating means shows the relationship between the position and the focus distance of the subject acquired by the acquisition means and / or the position of performing a predetermined image processing on the original image according to the focal length of the lens and the intensity thereof. Generate the mask information 12. The image processing apparatus according to claim 12. [Claim 14] Further provided with a storage means for storing the basic information of the mask information, The mask information generation means generates the mask information using the basic information stored in the storage means. The image processing apparatus according to any one of claims 1 to 13. [Claim 15] The image processing apparatus according to claim 14, wherein the basic information stored in the storage means includes a basic shape of a processing area to be subjected to the predetermined image processing. [Claim 16] The basic information stored in the storage means is the content of change in the intensity of the predetermined image processing in the processing area to which the predetermined image processing is performed, and the content of the change gradually increasing from the center side to the peripheral side. 15. The image processing apparatus according to claim 15, wherein the image processing apparatus includes. [Claim 17] The image generating means synthesizes the original image and a processed image obtained by subjecting the original image to the predetermined image processing by using the mask information generated by the mask information generating means to create the new image. The image processing apparatus according to any one of claims 1 to 16, wherein the image processing apparatus is generated. [Claim 18] The mask information generation means generates the mask information indicating the transmittance for each pixel, and generates the mask information. The image generating means transmits and synthesizes one of the processed image or the original image to the processed image or the other of the original image by setting the transmittance indicated by the mask information for each pixel. Generate an image 17. The image processing apparatus according to claim 17. [Claim 19] Further equipped with imaging means, The acquisition means acquires the position and size of a noteworthy subject in the original image acquired by shooting with the imaging means. The image processing apparatus according to any one of claims 1 to 18. [Claim 20] Acquisition processing to acquire the position and size of a noteworthy subject in the original image acquired by shooting, Based on the position and size of the notable subject acquired by the acquisition process, the mask information generation process that generates mask information indicating the relationship between the position where the predetermined image processing is performed in the original image and the intensity thereof, and the mask information generation process. An image generation process that uses the mask information generated by the mask information generation process to generate a new image in which predetermined image processing of different intensities is applied to each position in the original image. An image processing method comprising. [Claim 21] On the computer An acquisition function that acquires the position and size of a noteworthy subject in the original image acquired by shooting, Based on the position and size of the notable subject acquired by the acquisition function, a mask information generation function that generates mask information indicating a relationship between a position where a predetermined image processing is performed on the original image and its intensity, and a mask information generation function. An image generation function that uses the mask information generated by the mask information generation function to generate a new image in which predetermined image processing of different intensities is applied to each position in the original image. A program characterized by realizing. [Claim 22] Specific processing that identifies the subject that was the target of focus adjustment in the original image acquired by shooting as a noteworthy subject, and The acquisition process for acquiring the focus information of the subject specified by the specific process, and the acquisition process. Based on the focus information acquired by the acquisition process, a mask information generation process that generates mask information indicating a relationship between a position where a predetermined image process is performed on the original image and its intensity, and a mask information generation process. An image generation process that uses the mask information generated by the mask information generation process to generate a new image in which predetermined image processing of different intensities is applied to each position in the original image. An image processing apparatus comprising. [Claim 23] On the computer A specific function that identifies the subject that was the target of focus adjustment in the original image acquired by shooting as a noteworthy subject, and An acquisition function for acquiring the focus information of the subject specified by the specific function, and Based on the focus information acquired by the acquisition function, a mask information generation function that generates mask information indicating a relationship between a position where a predetermined image processing is performed on the original image and its intensity, and a mask information generation function. An image generation function that uses the mask information generated by the mask information generation function to generate a new image in which predetermined image processing of different intensities is applied to each position in the original image. A program characterized by realizing.

1 Digital camera 2 Control unit 3 Lens part 4 Imaging unit 5 Display 6 Image storage 7 Program storage 8 Face detector 9 Operation unit 10 Power supply 101 mask data M1 first mask image M2 second mask image M3 3rd mask image M11 mask image G1 standard developed image G2 soft focus image G3 composite image A Vertical size of face area B Movement amount C Vertical size of the second mask image after enlargement W face area O Center of face area P Center of the second mask image Q Center of the second mask image in the second embodiment