Image processing system and image processing program for defective pixel correction

The present invention relates to an image processing system and an image processing program for correcting defective pixels.

Conventionally, an image sensor has been widely used as a means for photoelectric conversion in an image pickup device such as an electronic camera. In such an image pickup device, defective pixels may occur due to defects in the light receiving element or non-uniformity of the light receiving surface. In such a defective pixel, the output of the pixel becomes abnormal, and it is reproduced as a bright spot or a black spot. In order to correct such defective pixels, techniques such as replacement are used (see, for example, Patent Document 1).<patcit num="1"><text>Japanese Unexamined Patent Publication No. 5-41868</text></patcit>

<p> However, in the invention of Patent Document 1 described above, it is difficult to perform sufficient correction depending on the state of the subject and the conditions at the time of shooting, and there are cases where a satisfactory correction effect cannot be obtained. An object of the present invention is to provide an image processing system and an image processing program capable of realizing more accurate and high-quality correction of defective pixels.</p>

<p> The image processing system of the present invention includes an image sensor, an acquisition unit that acquires information indicating the position of defective pixels generated in the image sensor, and an acquisition unit.<u style="single">Based on the information indicating the position of the defective pixel</u>Through the image sensor<u style="single">Corresponds to the defective pixel generated in the image sensor in the generated image.</u>A creation unit that creates identification information indicating the position of a defective pixel, an interpolation unit that performs interpolation processing on the image, and a neighboring pixel for each defective pixel when the interpolation unit performs the interpolation processing. Detect correlation<u style="single">At the same time, based on the identification information, it is determined whether or not the pixel adjacent to the defective pixel in the image is a defective pixel.</u>When the detection unit and the interpolation unit perform the interpolation processing, the detection by the detection unit<u style="single">And judgment</u>Depending on the result<u style="single">The direction in which three or more defective pixels including the defective pixel in the image are continuously present in an arbitrary direction and the defective pixels are continuous is the same as the direction having the highest correlation. If, in the image, based on the pixel values of the nearest pixels of the same color.</u>It includes an estimation unit that estimates the pixel value for each defective pixel.</p><p> Preferably, the image processing system includes an image pickup device and a computer that records an image processing program, and the image pickup device includes the image pickup device, the acquisition unit, and the creation unit, and the image processing. The program may cause the computer to function as the detection unit and the estimation unit. Further, preferably, the creating unit of the imaging device may create the identification information as the header information of the image, and the computer may acquire the header information together with the image.</p><p><u style="single"> Ma</u>In addition, preferably, the creating unit sets the pixel value of the defective pixel to at least one of the maximum value and the minimum value that the pixel value can take, in order to create identification information indicating the position of the defective pixel in the image. You may replace it.</p><p> Further, preferably, an image processing unit that performs image processing including at least one of white balance adjustment, color processing, and gradation conversion processing on the image before the estimation unit estimates the pixel value. The image processing unit may perform the image processing without modifying the identification information. Further, preferably, the creating unit may create the identification information in the state where the gradation information is the largest in the image pickup system by the image pickup device.<u style="single">Another image processing system of the present invention indicates an image pickup element that generates an image composed of a plurality of color components, an acquisition unit that acquires information indicating the position of a defect pixel generated in the image pickup element, and a position of the defect pixel. Based on the information, in the image generated via the image pickup element, a creation unit that creates identification information indicating the position of the defect pixel corresponding to the defect pixel generated in the image pickup element, and an interpolation process for the image. When the interpolation unit performs the interpolation process, the interpolating unit detects the correlation with the neighboring pixels for each defective pixel, and is adjacent to the defective pixel in the image based on the identification information. A detection unit that determines whether or not the pixel to be used is a defective pixel, and when the interpolation unit performs the interpolation process, the pixel value for each defective pixel in the image is determined according to the detection result by the detection unit. The detection unit includes an estimation unit for estimating, and the detection unit performs interpolation on the defective pixels of the first color component based on nearby pixels of a second color component different from the first color component. The pixel value of the second color component in the defective pixel of the first color component is calculated, and based on the calculated pixel value of the second color component, the defective pixel of the first color component and the defective pixel of the first color component It is characterized in that the correlation with the pixel value of a pixel in the vicinity of the second color component is detected.</u></p><p> The image processing program of the present invention is an image processing program that realizes image processing on an image having identification information indicating that it is a defective pixel at a position of a defective pixel by a computer, and indicates that the defective pixel is the defective pixel. The acquisition step of acquiring the image having the identification information and<u style="single">Before</u>Interpolation step to perform interpolation processing on the recorded image<u style="single">When the interpolation process is performed in the interpolation unit step, the correlation with the neighboring pixels is detected for each of the defective pixels, and the pixels adjacent to the defective pixels in the image are based on the identification information. Three or more including the defective pixel in the image according to the detection step for determining whether or not the pixel is defective and the result of the detection and determination in the detection step when the interpolation process is performed in the interpolation step. If the defective pixels of are continuously present in an arbitrary direction, and the direction in which the defective pixels are continuous is the same as the direction having the highest correlation, the nearest same color is used. It has an estimation step of estimating the pixel value for each defective pixel in the image based on the pixel value of the pixel.</u>It is characterized by that.</p>

<p> According to the image processing system and the image processing program of the present invention, more accurate and high-quality correction of defective pixels can be realized.</p>

<figref num="1">It is a figure which shows the structure of the electronic camera 1 of 1st Embodiment, and the flow of processing.</figref><figref num="2">It is a flowchart which shows the operation of the interpolation and defect pixel correction part 7.</figref><figref num="3">It is a figure explaining the correction of a defective pixel.</figref><figref num="4">It is another figure explaining the correction of a defective pixel.</figref><figref num="5">It is another figure explaining the correction of a defective pixel.</figref><figref num="6">It is another flowchart which shows the operation of the interpolation and the defect pixel correction part 7.</figref><figref num="7">It is a figure which shows the structure of the image processing system 100 of 2nd Embodiment, and the flow of processing.</figref><figref num="8">It is another figure explaining the correction of a defective pixel.</figref>

&lt;&lt; First Embodiment &gt;&gt; Hereinafter, the first embodiment of the present invention will be described with reference to the drawings. In the first embodiment, the electronic camera 1 provided with the image processing system of the present invention will be used for description. As shown in FIG. 1, the electronic camera 1 includes an image sensor 2 that captures a subject image, an A / D conversion unit 3 that performs A / D conversion, and a defective pixel identification information creation that creates identification information that is a feature of the present invention. Part 4, white balance adjustment part 5 for white balance adjustment, gamma processing part 6 for gamma processing, interpolation and defective pixel correction part 7 for interpolation processing and defective pixel correction, which is a feature of the present invention, color reproduction processing and color It includes a color processing unit 8 that performs spatial conversion and the like, a contour enhancement unit 9 that performs contour enhancement processing, and a compression unit 10 that compresses an image.

The signal output from the image sensor 2 is converted from an analog signal to a digital signal by the A / D converter 3. In the conventional technique, after this A / D conversion, the address information of the defective pixel is detected, and the pixel value of the pixel is replaced with a nearby value based on the detected address information. Further, in the present embodiment, the image sensor 2 has a bayer arrangement. In the present embodiment, after the A / D conversion, the defective pixel identification information creation unit 4 creates identification information indicating that the pixel is a defective pixel. For example, when the A / D converter 3 has an accuracy of 12 bits, the pixel value of the defective pixel is replaced with 4095. 4095 is the maximum value that the pixel value can take, and 4095 is a mark indicating that it is a defective pixel as it is. It should be noted that the defective pixel may be automatically detected from the average value of nearby pixels instead of the address information of the defective pixel.

When the pixel value in the defective pixel is replaced, the white balance adjusting unit 5 adjusts the white balance, and the gamma processing unit 6 performs gamma processing. Since the specific methods of white balance adjustment and gamma treatment are the same as those of known techniques, the description thereof will be omitted. However, when a mark (pixel value: 4095) indicating that the pixel is defective is detected in each process, the white balance adjusting unit 5 performs image processing without modifying the identification information by performing the following processing.

That is, if the data before white balance adjustment is Db and the data after white balance adjustment is Da, if Db = 4095 then Da = 4095 if Db 4095 then if Da 4095 then Da = 4094 Clip processing is performed. The gamma processing unit 6 is also subjected to the same processing to perform image processing without modifying the identification information.

Next, the interpolation and defect pixel correction unit 7 performs correction processing for defective pixels at the same time as interpolation. FIG. 2 is a flowchart showing the operation of the interpolation and defect pixel correction unit 7. In step S1, the interpolation and defect pixel correction unit 7 checks the pixels included in the gamma-processed image in a predetermined order, and whether or not a mark (pixel value: 4095) indicating that the pixel is defective is detected. To judge. In this embodiment, the process proceeds from the upper left pixel to the right, and when the first line ends, the process proceeds to the leftmost pixel of the second line. Then, when the interpolation and defect pixel correction unit 7 determines that the mark has been detected, the process proceeds to step S2. On the other hand, if no mark is detected even after checking all the pixels included in the image after gamma processing, step S described later will be performed.<u style="single">9</u>Proceed to.

In step S2, the interpolation and defect pixel correction unit 7 determines whether or not the mark is detected in the pixel adjacent to the pixel in which the mark is detected (hereinafter, referred to as focused pixel). For example, when the pixel p0 showing "" in FIG. 3A is the pixel of interest, the interpolation and defect pixel correction unit 7 determines whether or not a mark exists in the four pixels shown in the frame E0, and pays attention to it. If it is determined that the mark is detected in the pixel adjacent to the pixel, the process proceeds to step S3, and if it is determined that the mark is not detected in the pixel adjacent to the pixel of interest, the process proceeds to step S7 described later.

In step S3, the interpolation and defect pixel correction unit 7 determines whether or not a mark is detected in an adjacent pixel. For example, in step S2, when a mark is detected in the pixel p1 showing "" in FIG. 3A, the interpolation and defect pixel correction unit 7 determines whether or not the mark exists in the pixel shown in the frame E1. To do. Further, in step S2, when a mark is detected in the pixels p2, 3 and 4 shown by "" in FIG. Determine if the mark is present. Then, if it is determined that the mark is detected in the adjacent pixels, the process proceeds to step S4, and if it is determined that the mark is not detected in the adjacent pixels, the process proceeds to step S7, which will be described later. In step S3, the case of "Yes" is a case where three marks are continuously present in any of the vertical, horizontal, and diagonal directions (defective pixels are present).

In step S4, the interpolation and defect pixel correction unit 7 determines whether or not all the marks detected in steps S1 to S3 are those of G pixels. Since the image sensor 2 of the present embodiment has a bayer array, all of them are G pixels, that is, as shown in L1 or L2 shown in FIG. 3B, marks are detected in diagonally continuous G pixels. That is to say. When the interpolation and defect pixel correction unit 7 determines that all of them are G pixels, the process proceeds to step S6 described later. On the other hand, if it is determined that all of them are not G pixels (including at least two of G pixels, R pixels, and B pixels), the process proceeds to step S5.

In step S5, the interpolation and defect pixel correction unit 7 determines whether or not the continuous direction and the correlation direction of the defective pixels are the same. The interpolation and defect pixel correction unit 7 first detects the correlation between the pixel of interest and a pixel in the vicinity thereof. For example, as shown in FIG. 4, the interpolation and defect pixel correction unit 7 detects the correlation in four directions of vertical, horizontal, and diagonal (a to d) centering on the pixel of interest p0. Then, it is determined whether or not the correlation direction, which is the direction having the highest correlation, and the continuous direction of the defective pixels are the same. When the interpolation and defect pixel correction unit 7 determines that the continuous direction and the correlation direction of the defective pixels are the same, the process proceeds to step S6. On the other hand, if it is determined that the continuous direction and the correlation direction of the defective pixels are not the same, the process proceeds to step S7 described later.

In step S6, the interpolation and defect pixel correction unit 7 estimates the pixel value of the pixel of interest based on the pixel value of the nearest pixel of the same color. Then, the interpolation and defect pixel correction unit 7 proceeds to step S9, which will be described later. In step S7, the interpolation and defect pixel correction unit 7 detects the correlation between the pixel of interest and a pixel in the vicinity thereof. The specific method of detection will be described later. The cases where the processing of step S7 is performed are the case where it is determined by the determination of step S2 and step S3 that three or more defective pixels are not continuous, and the case where the determination of step S5 determines the continuous direction of the defective pixels. This is the case when it is determined that the correlation direction is not the same as that of. In this case, the interpolation and defect pixel correction unit 7 corrects the defect pixel described later.

In step S8, the interpolation and defect pixel correction unit 7 estimates the pixel value of the pixel of interest based on the correlation detected in step S7. The specific method of estimation will be described later. In step S9, the interpolation and defect pixel correction unit 7 performs interpolation processing similar to that of the known technique. In the interpolation process, it is preferable to perform the interpolation process by using the information related to the correlation detected in each step described above. Even when the mark (pixel value: 4095) indicating that the pixel is defective is not detected in step S1, the same interpolation processing as in the known technique is performed.

In step S10, the interpolation and defect pixel correction unit 7 determines whether or not the processing has been performed on all the pixels included in the image after the gamma processing, and if it is determined that the processing has been performed on all the pixels, a series of processing is performed. finish. On the other hand, if all the pixels have not been processed, the process returns to step S1 and the subsequent pixels are processed from step S1. When the interpolation and defect pixel correction unit 7 completes a series of processes described in the flowchart of FIG. 2, the color processing unit 8 performs color processing on the image after interpolation and defect pixel correction, and the contour enhancement unit 9 performs color processing. The outline is emphasized, the image is compressed by the compression unit 10, and the image is output to a recording unit (not shown). Specific methods such as color processing, contour enhancement, and compression are the same as those of known techniques, and thus description thereof will be omitted.

Next, the details of the correlation detection and the estimation of the pixel value of the pixel of interest described in steps S7 and S8 of the flowchart of FIG. 2 will be described. &lt;First method&gt; The interpolation and defect pixel correction unit 7 detects the correlation between the pixel of interest and a pixel in the vicinity thereof. The interpolation and defect pixel correction unit 7 detects the correlation in the four directions of vertical, horizontal, and diagonal (a to d) shown in FIG. For example, when the pixel of interest is the R pixel in the center of FIG. 5A, the vertical direction (direction of arrow a in FIG. 4) having the highest correlation among the detected correlations in each direction is determined as the correction direction. Then, as shown in FIG. 5B, the pixel value of the pixel of interest is estimated by obtaining the average value of the upper and lower pixels in the vertical direction and performing replacement.

&lt;Second method&gt; FIG. 6 is a flowchart showing the operation of the interpolation and defect pixel correction unit 7 when the second method is performed. In step S21, the interpolation and defect pixel correction unit 7 determines whether or not the pixel of interest is a B pixel. When the interpolation and defect pixel correction unit 7 determines that the pixel of interest is the B pixel, the process proceeds to step S22. On the other hand, if it is determined that the pixel of interest is not a B pixel (G pixel or R pixel), the process proceeds to step S25 described later.

In step S22, the interpolation and defect pixel correction unit 7 interpolates the G pixel component and the R pixel component of the attention pixel based on the pixel values of the G pixel and the R pixel adjacent to the B pixel which is the attention pixel. The G pixel component generated by interpolation is referred to as "G'pixel", and the R pixel component is referred to as "R'pixel". In step S23, the interpolation and defect pixel correction unit 7 detects the correlation with the neighboring G pixels and R pixels by using the G'pixels and R'pixels generated in step S22.

In step S24, the interpolation and defect pixel correction unit 7 estimates the pixel value of the B pixel, which is the pixel of interest, based on the correlation detected in step S23. The interpolation and defect pixel correction unit 7 determines the correction direction based on the correlation in each direction detected in step S23, obtains the average value of nearby pixels in the correction direction, and performs replacement. Estimate the pixel value of a pixel.

If it is determined in step S21 that the pixel of interest is not a B pixel (G pixel or R pixel), in step S25, the interpolation and defect pixel correction unit 7 determines whether or not the pixel of interest is a G pixel. When the interpolation and defect pixel correction unit 7 determines that the pixel of interest is a G pixel, the process proceeds to step S26. On the other hand, if it is determined that the pixel of interest is not a G pixel (R pixel), the process proceeds to step S29 described later.

In steps S26 to S28, the interpolation and defect pixel correction unit 7 performs processing following steps S22 to S24 described above. That is, in step S26, the B pixel component (B'pixel) and the R pixel component (R'pixel) of the attention pixel are based on the pixel values of the B pixel and the R pixel adjacent to the G pixel which is the attention pixel. ) Is interpolated. Then, in step S27, the B'pixel and the R'pixel are used to detect the correlation with the neighboring B pixel and R pixel. Then, in step S28, the pixel value of the G pixel, which is the pixel of interest, is estimated based on the correlation detected in step S27.

Further, in steps S29 to S31, the interpolation and defect pixel correction unit 7 performs processing following steps S22 to S24 described above. That is, in step S29, the B pixel component (B'pixel) and the G pixel component (G'pixel) of the attention pixel are based on the pixel values of the B pixel and the G pixel adjacent to the R pixel which is the attention pixel. ) Is interpolated. Then, in step S30, the B'pixel and the G'pixel are used to detect the correlation with the neighboring B pixel and the G pixel. Then, in step S31, the pixel value of the R pixel, which is the pixel of interest, is estimated based on the correlation detected in step S30.

As described above, according to the first embodiment, the information indicating the position of the defective pixel generated in the image sensor is acquired, and the identification information indicating the position of the defective pixel in the image generated via the image sensor is created. .. Then, the correlation with the neighboring pixels is detected for each defective pixel, and the pixel value for each defective pixel is estimated according to the detection result. Therefore, more accurate and high-quality correction of defective pixels can be realized.

Further, according to the first embodiment, when the image is subjected to the interpolation processing, the correlation is detected and the pixel value is estimated. Therefore, the correlation information calculated in the interpolation process can be used to efficiently detect the correlation and estimate the pixel value. Further, according to the first embodiment, it is determined whether or not the pixel adjacent to the defective pixel is a defective pixel based on the identification information. Therefore, it is possible to realize the correction according to the state of the pixel adjacent to the defective pixel of interest.

Further, according to the first embodiment, three or more defective pixels including the defective pixel of interest are continuously present in an arbitrary direction, and the defective pixels are the highest in the continuous direction. When the directions having the correlation are the same, the pixel value of the defective pixel of interest is estimated based on the pixel value of the nearest pixel of the same color. Therefore, deterioration of the image can be minimized for defective pixels for which the defective pixel correction of the present invention is not appropriate.

Further, according to the first embodiment, as the creation of the identification information indicating the position of the defective pixel in the image, the pixel value of the defective pixel is replaced with at least one of the maximum value and the minimum value that the pixel value can take. Therefore, the identification information can be embedded in the image itself. Further, according to the first embodiment, before estimating the pixel value, the image is subjected to image processing including at least one of white balance adjustment, color processing, and gradation conversion processing. , Perform image processing without modifying the identification information. Therefore, the identification information can be kept until the pixel value is estimated, and the pixel value can be estimated accurately.

In the first embodiment, an example in which a mark (pixel value: 4095) indicating that the pixel is defective is used as the identification information, but the defect pixel identification information creation unit 4 creates the address information of the defective pixel as header information. You may try to do it. &lt;&lt; Second Embodiment &gt;&gt; Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. In the second embodiment, as shown in FIG. 7, an image processing system 100 including an electronic camera and a computer will be used for explanation.

The electronic camera of the second embodiment has the same configuration as the electronic camera of the first embodiment. Therefore, the following description will be made using the same reference numerals as those of the electronic camera 1 of the first embodiment. However, in the second embodiment, the electronic camera 1 outputs the image data after the identification information is created by the defective pixel identification information creation unit 4 to the computer 20, as in the first embodiment. When the identification information is a specific pixel value (4095 in the first embodiment), only the image is output, and when the identification information is the header information (including the address information of the defective pixel), it is together with the image. Output header information. Further, in each part of the computer 20, the same configuration as that of the electronic camera 1 of the first embodiment will be described using the same reference numerals as those of the first embodiment.

When an image is acquired from the electronic camera 1, the white balance adjusting unit 5 adjusts the white balance in the same manner as in the first embodiment. However, when the identification information indicating that the pixel is defective is detected in each process, the white balance adjusting unit 5 performs image processing without modifying the identification information, as in the first embodiment. Then, the interpolation and defect pixel correction unit 7 performs correction processing for defective pixels at the same time as interpolation for the image after white balance adjustment. In the first embodiment, defective pixels are corrected at the same time as interpolation for the image after gamma processing by the gamma processing unit 6. In comparison with this, in the second embodiment, defect pixel correction processing is performed at the same time as interpolation for the image before gamma processing. That is, by performing the defect pixel correction process in a state where there is a large amount of gradation information, more accurate and high-quality defect pixel correction can be realized in the computer. The interpolation and the defect pixel correction are performed in the same manner as in the first embodiment.

Then, the color reproduction processing unit 21 performs color reproduction processing on the image after defect pixel correction, the gamma processing unit 6 performs gamma processing, and the color space conversion unit 22 performs color space conversion from RGB to YCbCr. I do. Since the specific methods of color reproduction processing and color space conversion are the same as those of known techniques, the description thereof will be omitted. Note that, unlike the first embodiment, the gamma processing unit 6 performs gamma processing on the image after the defect pixel correction, so that it is not necessary to pay attention to the modification of the identification information as in the first embodiment.

Further, the contour enhancement unit 9 performs contour enhancement, the noise reduction unit 23 executes noise reduction, and the compression unit 10 compresses the image and outputs it to a recording unit (not shown) or the like. Specific methods such as contour enhancement, noise reduction, and compression are the same as those of known techniques, and thus description thereof will be omitted. As described above, according to the second embodiment, the image pickup device is composed of an image pickup device and a computer that records an image processing program, and the image pickup device acquires information indicating the positions of defective pixels generated in the image pickup device and sets the image pickup device. Identification information indicating the position of the defective pixel in the image generated through the screen is created. Then, the computer detects the correlation with the neighboring pixels for each defective pixel, and estimates the pixel value for each defective pixel according to the detection result. Therefore, more accurate and high-quality correction of defective pixels can be realized. In addition, it is possible to reduce the processing in the image pickup apparatus and to realize high-quality correction of defective pixels in the computer, which meets the user's request.

Further, according to the second embodiment, the imaging device creates the identification information as the header information of the image, and the computer acquires the header information together with the image. Therefore, the identification information can be used without replacing the pixel values. Further, even if the compression process is performed before the image pickup device is supplied to the computer, the identification information can be retained without being modified.

Further, according to the second embodiment, the identification information is created in the state where the gradation information is the largest in the image pickup system using the image pickup device. Therefore, more accurate and high-quality correction of defective pixels can be realized. Further, according to the second embodiment, when the computer realizes image processing for an image having identification information indicating that it is a defective pixel at the position of the defective pixel, it is before the interpolation processing is completed. The pixel value of the defective pixel is replaced based on the nearby pixel value. Therefore, it is possible to more accurately correct defective pixels.

When the image is supplied from the image pickup apparatus to the computer as in the second embodiment, it is preferable not to perform the image compression process. This is for the reliable retention of the identification information. Alternatively, as described in the first embodiment, instead of using the maximum value that the pixel value can take as a mark indicating that it is a defective pixel, the minimum value may be used, or both may be used. Is also good. That is, when the output value is larger than the intermediate value, the maximum value is used, and when the output value is smaller than the intermediate value, the minimum value is used. By using the two types of values in this way, the identification information can be retained without being modified even if the compression process is performed before being supplied from the image pickup apparatus to the computer.

Further, in the case of supplying an image from the imaging device to the computer as in the second embodiment, particularly when outputting image data such as RAW data, the program of the computer on the receiving side is included in the RAW data or the like. In some cases, the "mark indicating that the pixel is defective" (identification information) cannot be recognized. That is, even if the minimum value or the maximum value is used as the "mark indicating that it is a defective pixel", the image processing program of the computer that cannot recognize this cannot appropriately correct the defective pixel, so that it is a bright spot. And black spots.

Therefore, the final digit of the digital value of the image data such as RAW data may be used as a "mark indicating that the pixel is defective". For example, for defective pixels, all the last digits are set to "0", and for pixels that are not defective pixels, all the last digits are set to "1". That is, among the defective pixels, those whose last digit is "1" are all changed to "0". On the other hand, among the pixels that are not defective pixels, those whose final digit is "0" are all changed to "1". In this way, changing the last digit of the digital value has almost no effect on the actual pixel value. Therefore, by using the last digit of the digital value as a mark, even if the computer program on the receiving side cannot recognize the "mark indicating that it is a defective pixel" contained in the data such as RAW, the deterioration of the image is minimized. Can be suppressed to.

When creating the above-mentioned "mark indicating that the pixel is defective", the last digit of the digital value of the defective pixel itself in the image data output by the A / D converter 3 may be used as the mark. The image data output by the A / D conversion unit 3 may be corrected by, for example, replacement, as in a known technique, and the final digit of the digital value of the defective pixel in the corrected image data may be used as a mark.

Further, in the second embodiment, an example of acquiring an image from an electronic camera is shown, but if the image to be acquired has the identification information of the present invention, it should be acquired from another external device, a recording medium, or the like. You may. Further, such a program may be installed on the Internet to correct defective pixels on the Internet. Further, in each of the above embodiments, although the description has been made using the image sensor of the Bayer array, the present invention may be applied to the image sensor of another array.

Further, the defective pixels in each of the above embodiments include pixels that do not output correctly according to the amount of received light (no reaction even though they are receiving light, reaction even though they are not receiving light), fixed pattern noise, and the like. To do. Further, the order of each process in each of the above embodiments is not limited to this example. The order may be changed or a part may be omitted depending on the configuration and the desired image quality. For example, interpolation and correction of defective pixels may be performed separately.

For example, as shown in FIG. 8, when defective pixels (pixels marked with x) are complicatedly continuous, it is difficult to deal with them by the above-mentioned method. The pixel values of the defective pixels may be estimated, and after the pixel values of all the pixels are aligned, the whole may be subjected to interpolation processing. The defective pixel in good condition means that the number of defective pixels in the adjacent pixel is smaller. In the example of FIG. 8, first, the defective pixel shown in (1) is corrected (pixel value is estimated). As a result, the defective pixel shown in (2), which has improved conditions, is corrected. Further, by correcting the defective pixel shown in (3), the defective pixel shown in (4) can also be corrected under favorable conditions.