Image processing apparatus and method, and recording medium therefor

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an image processing apparatus and a method and a recording medium therefor. More particularly, the image processing apparatus and method and a recording medium of the present invention are ideally adapted for correcting the gray level of an image according to the dynamic range of an image output apparatus, such as a display or a printer.

[0003] 2. Description of the Related Art

[0004] Recent developments in image signal acquiring equipment incorporating a solid-state image sensing device or a charge-coupled device (CCD), such as a digital camera or a scanner, have led to the improved performance of the solid-state image sensing device, permitting multi-stage exposures. This has made it possible to acquire image signals in a wider gray level dynamic range (hereinafter referred to as “the wide dynamic range image signals”).

[0005] Meanwhile, the recording formats of media for recording image signals, the formats of signals output to displays or printers, broadcast signal formats, and the like are still limited to the conventional dynamic ranges of 8-bit width or 10-bit width (hereinafter referred to as “the narrow dynamic range”.

[0006] Accordingly, to output image signals from electronic equipment capable of acquiring image signals of a wide dynamic range to other electronic equipment of the conventional narrow dynamic range, it is necessary to correct the gray levels of the image signals of the wide dynamic range, that is, to narrow the dynamic range.

[0007] Japanese Unexamined Patent Application Publication No. 9-331539 discloses a technique for correcting the dynamic range of image signals by individually correcting the gray levels of the signals of the three primary colors, Red, Green, and Blue (R, G, and B,) in a broadcasting video camera. FIG. 1 illustrates an example of the configuration of a gray level correction processor in such a broadcasting video camera. In the gray level correction processor, the three primary color signals R, G, and B are supplied to a correction parameter arithmetic circuit 1, and also to their corresponding gray level correction circuits <b>2</b>R, <b>2</b>G, and <b>2</b>B, respectively.

[0008] The correction parameter arithmetic circuit <b>1</b> uses the received three primary color signals R, G, and B to generate luminance signals, and computes correction parameters on the basis of the luminance signals, then outputs the computed correction parameters to the gray level correction circuits <b>2</b>R, <b>2</b>G, and <b>2</b>B.

[0009] The gray level correction circuit <b>2</b>R includes a lookup table (hereinafter referred to as “LUT”) for correcting the gray level of a red signal R. The circuit <b>2</b>R checks the correction parameter and the red signal R, which have been received from the correction parameter arithmetic circuit <b>1</b>, against the built-in LUT, and outputs the associated value to a gamma correction circuit <b>3</b> as a correction value. The gray level correction circuit <b>2</b>G includes a LUT for correcting the gray level of a green signal G. The circuit <b>2</b>G checks the correction parameter and the green signal G, which have been received from the correction parameter arithmetic circuit <b>1</b>, against the built-in LUT, and outputs the associated value to the gamma correction circuit <b>3</b> as a correction value. The gray level correction circuit <b>2</b>B includes a LUT for correcting the gray level of a blue signal B. The circuit <b>2</b>B checks the correction parameter and the blue signal B, which have been received from the correction parameter arithmetic circuit <b>1</b>, against the built-in LUT, and outputs the associated value to the gamma correction circuit <b>3</b> as a correction value.

[0010] A gamma correction circuit <b>3</b>R carries out a gamma correction on the corrected red signal R received from the gray level correction circuit <b>2</b>R, and outputs the result to a matrix circuit <b>4</b>. A gamma correction circuit <b>3</b>G carries out a gamma correction on the corrected green signal G received from the gray level correction circuit <b>2</b>G, and outputs the result to the matrix circuit <b>4</b>. A gamma correction circuit <b>3</b>B carries out a gamma correction on the corrected blue signal B received from the gray level correction circuit <b>2</b>B, and outputs the result to the matrix circuit <b>4</b>. The matrix circuit <b>4</b> coverts the gamma-corrected three primary color signals R, G, and B into a luminance signal Y₀and color-difference signals Cr₀and Cb₀.

[0011] The gray level correction processor shown in FIG. 1 performs the gray level correction on the three primary color signals R, G, and B, respectively, at the same ratio. Hence, it is possible to compress the dynamic range of image signals to a desired width without causing a change in hue.

[0012] The gray level correction processor, however, requires the gray level correction circuits <b>2</b> and the gamma correction circuits <b>3</b> for the three primary color signals R, G, and B, respectively. This leads to a problem in that using the gray level correction processor in a consumer appliance, such as a digital camera, a video camera, or a television receiver, inevitably results in higher cost and a greater circuit scale.

[0013] Japanese Unexamined Patent Application Publication No. 11-55598 has disclosed a technique for correcting the dynamic range of image signals by performing gray level corrections only on the luminance signal Y among the image signals in a television receiver. FIG. 2 illustrates an example of the configuration of a gray level correction processor for such a television receiver.

[0014] In the gray level correction processor, a received luminance signal Y is supplied to a correction parameter arithmetic circuit <b>11</b>, a luminance correction circuit <b>12</b>, and a color-difference correction circuit <b>13</b>, while received color-difference signals Cr and Cb are supplied to the color-difference correction circuit <b>13</b>.

[0015] Based on a received luminance signal Y, the correction parameter arithmetic circuit <b>11</b> computes a correction parameter with which an optimum gray level correction will be made in the luminance correction circuit <b>12</b>, and outputs the computed correction parameter to the luminance correction circuit <b>12</b>. The luminance correction circuit <b>12</b>, which includes a LUT for correcting the gray level of the luminance signal Y, checks the correction parameter and the luminance signal Y received from the correction parameter arithmetic circuit <b>11</b> against the built-in LUT, and outputs an associated value as a correction value Y₀. The correction value Y₀is also supplied to the color-difference correction circuit <b>13</b>.

[0016] The color-difference correction circuit <b>13</b> performs normalization by multiplying the color-difference signals Cr and Cb by Y₀/Y, which is a ratio of the luminance signal Y before gray level correction to the luminance signal Y₀after the correction, and generates color-difference signals Cr₀and Cb₀so as to maintain the ratio of the color-difference signals Cr and Cb to the input luminance signal Y.

[0017] The gray level correction processor shown in FIG. 2 requires fewer constituent parts since it is adapted to carry out gray level corrections only on the luminance signal Y, thus controlling an increase in cost and circuit scale. The gray level correction processor of FIG. 2, however, presents the problems described below.

[0018] First, an error occurs in correcting the luminance of a pixel having a larger color-difference signal value. The relationship between three primary color signals R, G and B, and the luminance signal Y or the color-difference signals Cr and Cb is represented by expressions (1) and (2) shown below:

<i>Y=</i>0.30<i>R+</i>0.59<i>G+</i>0.11<i>B </i>  (1)

<i>Cr=R-Y</i>

<i>Cb=B-Y </i>  (2)

[0019] In correcting the gray level of the luminance signal Y, the information regarding the color-difference signals Cr and Cb is not taken into account. The information indicating the balance among the three primary color signals R, G, and B is missing in the luminance signal Y. For this reason, the gray level correction of the luminance signal Y is not properly made on a pixel in which the three primary color signals R, G, and B are unbalanced, that is, a pixel having a greater color difference.

[0020] For instance, if an input red signal R has a maximum value (assumed to be 1), while input green signal G and blue signal B have minimum values (assumed to be 0), then the input luminance signal Y will be 0.3.

[0021] Such a pixel should be converted into a pixel in which the red signal R has the maximum value (assumed to be 1), while the green signal G and the blue signal B have the minimum value (assumed to be 0) after a gray level correction is made. After this conversion, the luminance signal Y of the pixel will be 0.3.

[0022] However, if, for example, the luminance correction circuit <b>12</b> is provided with the luminance correction LUT as shown in FIG. 3, then the luminance signal Y =0.3 will be converted into the luminance signal Y₀=0.6 in accordance with the luminance correction LUT, generating an error, as compared with a proper correction value (0.3 in this case). This problem may arise with all pixels other than achromatic pixels.

[0023] Second, a change in hue takes place. To be more specific, if, for example, the input red signal R has a maximum value, while the input green signal G and blue signal B have minimum values, then the color-difference signal Cr will have the maximum value. At this time, if the luminance signal Y =0.3 is converted into Y₀=0.6 in accordance with the LUT shown in FIG. 3, then the color-difference correction circuit <b>13</b> will multiply the color-difference signal Cr by 2 (=0.6/0.3).

[0024] The color-difference signal Cr, however, already has the maximum value, so that clipping will cause no changes, whereas the color-difference signal Cb, which does not take the maximum value, will be multiplied by 2. This poses a problem in that the value of Cr/Y cannot be maintained at a constant level and a hue change undesirably takes place.

[0025] Even if the value of Cr/Y is maintained constant by avoiding clipping on the color-difference signal Cr, the clipping will be eventually effected on the red signal R at the time of conversion into the three primary color signals R, G, and B, thus again causing an undesirable hue change to take place.

SUMMARY OF THE INVENTION

[0026] The present invention has been made with a view toward solving the above problems, and it is an object of the present invention to control the occurrence of a change in hue by adjusting the correction amount for a color-difference signal according to the gray level correction characteristic of a luminance signal when gray level correction is made only on a luminance signal Y of a pixel having a great color-difference signal value.

[0027] To this end, according to one aspect of the present invention, there is provided an image processing apparatus including a first generating device for generating a first conversion rule by using a first element of an input image signal, a second generating device for generating a second conversion rule on the basis of the first conversion rule, a first correcting device for correcting the first element of the input image signal by using the first conversion rule, an acquiring device for acquiring a correction parameter associated with the first element of the input image signal by using the second conversion rule, and a second correcting device for correcting a second element of the input image signal by using the correction parameter.

[0028] The first generating device may include a preparing device for preparing a histogram of the first element of the input image signal, an accumulating device for accumulating histograms to prepare a cumulative histogram, and an approximating device for approximating the cumulative histogram to a predetermined logarithmic curve thereby generating the first conversion rule.

[0029] The second correcting device may be adapted to correct the second element of the input image signal by multiplying the second element by the correction parameter.

[0030] The image processing apparatus according to the present invention may further include a converting device for converting an optical signal of a subject into a color signal, and a calculating device for calculating the first and second elements of the image signal on the basis of the color signal.

[0031] The converting device may be adapted to convert the optical signal of the subject into a red signal, a green signal, or a blue signal.

[0032] The converting device may be adapted to convert the optical signal of the subject into a yellow signal, a cyan signal, or a green signal.

[0033] The converting device may be adapted to convert the optical signal of the subject into a yellow signal, a cyan signal, a magenta signal, or a green signal.

[0034] According to another aspect of the present invention, there is provided an image processing method including a first generating step for generating a first conversion rule by using a first element of an input image signal, a second generating step for generating a second conversion rule on the basis of the first conversion rule, a first correcting step for correcting the first element of the input image signal by using the first conversion rule, an acquiring step for acquiring a correction parameter associated with the first element of the input image signal by using the second conversion rule, and a second correcting step for correcting a second element of the input image signal by using the correction parameter.

[0035] According to yet another aspect of the present invention, there is provided a recording medium in which a program has been recorded, the program including a first generating step for generating a first conversion rule by using a first element of an input image signal, a second generating step for generating a second conversion rule on the basis of the first conversion rule, a first correcting step for correcting the first element of the input image signal by using the first conversion rule, an acquiring step for acquiring a correction parameter associated with the first element of the input image signal by using the second conversion rule, and a second correcting step for correcting a second element of the input image signal by using the correction parameter.

[0036] According to the image processing apparatus and method and the program recorded in the recording medium therefor in accordance with the present invention, a first conversion rule is generated by using a first element of an input image signal, and a second conversion rule is generated on the basis of the first conversion rule. Furthermore, the first conversion rule is used to correct the first element of the input image signal, a correction parameter associated with the first element of the input image signal is acquired by using the second conversion rule, and a second element of the input image signal is corrected by using the correction parameter.

[0037] Additional features and advantages of the present invention are described in, and will be apparent from, the following Detailed Description of the Invention and the figures.

BRIEF DESCRIPTION OF THE FIGURES

[0038]FIG. 1 is a block diagram illustrating a configuration example of a conventional gray level correction processor for individually correcting the gray levels of the three primary color signals R, G, and B;

[0039]FIG. 2 is a block diagram illustrating a configuration example of a conventional gray level correction processor for performing gray level correction only on a luminance signal Y in an image signal;

[0040]FIG. 3 is a diagram illustrating the gray level correction characteristic of the luminance signal;

[0041]FIG. 4 is a diagram illustrating the principle of the gray level correction of a digital camera that is an embodiment of the present invention;

[0042]FIG. 5 is another diagram illustrating the principle of the gray level correction of the digital camera that is the embodiment of the present invention;

[0043]FIG. 6 is a block diagram showing a first configuration example of the digital camera that is an embodiment of the present invention;

[0044]FIG. 7 is a diagram showing an example of a color filter <b>36</b>;

[0045]FIG. 8 is a block diagram showing a first configuration example of an image processor <b>41</b> of a digital camera <b>30</b>;

[0046]FIG. 9 is a block diagram showing example configurations of a luminance correction circuit <b>59</b> and a correction parameter arithmetic circuit <b>60</b>;

[0047]FIG. 10 is a flowchart illustrating the gray level correction processing carried out by the image processor <b>41</b>;

[0048]FIG. 11 is a block diagram showing a second configuration example of the image processor <b>41</b> of the digital camera <b>30</b>;

[0049]FIG. 12 is a block diagram showing a third configuration example of the image processor <b>41</b> of the digital camera <b>30</b>;

[0050]FIG. 13 is a block diagram showing a fourth configuration example of the image processor <b>41</b> of the digital camera <b>30</b>;

[0051]FIG. 14 is a block diagram showing a second configuration example of the digital camera, which is an embodiment of the present invention; and

[0052]FIG. 15 is a block diagram showing a configuration example of an image processor <b>112</b> of a digital camera <b>110</b>.

DETAILED DESCRIPTION OF THE INVENTION

[0053] The following will describe a digital camera that is an embodiment of the present invention. First, the principle of operation for correcting the gray levels of image signals in the digital camera will be explained with reference to FIGS. 4 and 5.

[0054]FIG. 4 illustrates an example of the LUT for correcting the gray level of luminance signal Y. The axis of abscissa of the chart shown in FIG. 4 indicates an input luminance level Y, while the axis of ordinates indicates an output luminance level Y<b>0</b>. The processing for generating the LUT for correcting the gray levels of the luminance signals Y will now be described.

[0055] First, a histogram of the luminance signals Y of an image, which indicates the distribution of the frequency of the luminance signals Y at each signal level, is generated. Then, the frequency at each signal level in the histogram is accumulated to generate a cumulative histogram (curve A in the chart), and the dynamic ranges of output luminance signals Y₀are applied to the axis of ordinates. Subsequently, considering Weber-Fechner law regarding human visual characteristics (the sensory amount of humans is proportional to the logarithm of stimulation intensity), the curve A is subjected to logarithmic approximation to acquire a curve B. The curve B is obtained by expression (3) given below:

<i>Y</i>₀<i>=a·log</i>(<i>Y</i>)+<i>b </i>  (3)

[0056] Expression (3) is employed to obtain expression (4) given below:

<i>Y</i>₀<i>=E</i>₂(<i>E</i>₁(<i>Y</i>))=<i>log</i>(<i>E</i>₁(<i>Y</i>))+<i>b=log</i>(<i>Y</i>^a)+<i>b </i>  (4)

[0057] where a function E₁(x) denotes the processing for compressing the dynamic range of the luminance signal Y by using an exponential function. A function E₂(x) denotes the processing for changing the input/output characteristics of luminance according to visual characteristics, while maintaining the dynamic ranges.

[0058] To prevent the color-difference signals Cr and Cb from being clipped, no color correction will be made on the function E₁(x), and the color balance will be maintained for the function E₂(x).

[0059] In other words, expression (5) shown below will hold:

<i>Cr</i>₀<i>/Y</i>₀<i>=Cra/Ya </i>

<i>Cb</i>₀<i>/Y</i>₀<i>=Cba/Ya </i>  (5)

[0060] where Cr and Cb denote input color-difference signals, Cr₀and Cb₀denote output color-difference signals, the gray levels of which have been corrected, and “a” denotes a predetermined parameter, which will be discussed hereinafter.

[0061] Empirically approximating expression (3) leads to expression (6) shown below: <maths id="MATH-US-00001" num="1"><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><msub><mi>Cr</mi><mn>0</mn></msub><mo>≈</mo><mrow><mfrac><msup><mrow><mo>(</mo><msub><mi>Y</mi><mn>0</mn></msub><mo>)</mo></mrow><mfrac><mn>1</mn><mi>a</mi></mfrac></msup><mi>Y</mi></mfrac><mo></mo><mi>Cr</mi></mrow></mrow><mo></mo><mstyle>&lt;mtext /&gt;</mstyle><mo></mo><mrow><msub><mi>Cb</mi><mn>0</mn></msub><mo>≈</mo><mrow><mfrac><msup><mrow><mo>(</mo><msub><mi>Y</mi><mn>0</mn></msub><mo>)</mo></mrow><mfrac><mn>1</mn><mi>a</mi></mfrac></msup><mi>Y</mi></mfrac><mo></mo><mi>Cb</mi></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>6</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img file="US20020171663A1-20021121-M00001.TIF" id="EMI-M00001" he="46.0971" wi="216.027" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00001" attachment-type="nb" file="US20020171663A1-20021121-M00001.NB" /></attachments></maths>

[0062] Expression (6) represents the gray level correction characteristics for the color-difference signals Cr and Cb.

[0063] Thus, expression (3) is applied to make gray level corrections on the luminance signals Y, and expression (6) is applied to make gray level corrections on the color-difference signals Cr and Cb. This makes it possible to narrow the dynamic range of image signals, while controlling the occurrence of hue changes.

[0064] Referring now to FIG. 5, the descriptions will be given of a method for calculating the parameter “a” used for correcting the gray levels of the color-difference signals Cr and Cb by using only the input luminance signals Y. The chart shown in FIG. 5 is obtained by logarithmically converting the axis of abscissa of the chart shown in FIG. 4. A curve A′ corresponds to the curve A shown in FIG. 4. A straight line B′ is obtained by linearly approximating the curve A′. Hence, the straight line B′ can be considered to correspond to a curve B shown in FIG. 4.

[0065] Accordingly, the gray level correction functions of the color-difference signals Cr and Cb as represented by expression (7) below can be obtained by determining a gradient “α” of the straight line B′, then by substituting the value of “α” for “α” of expression (6). <maths id="MATH-US-00002" num="2"><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><msub><mi>Cr</mi><mn>0</mn></msub><mo>=</mo><mrow><mi>kCr</mi><mo>=</mo><mrow><mfrac><msup><mrow><mo>(</mo><msub><mi>Y</mi><mn>0</mn></msub><mo>)</mo></mrow><mfrac><mn>1</mn><mi>α</mi></mfrac></msup><mi>Y</mi></mfrac><mo></mo><mi>Cr</mi></mrow></mrow></mrow><mo></mo><mstyle>&lt;mtext /&gt;</mstyle><mo></mo><mrow><msub><mi>Cb</mi><mn>0</mn></msub><mo>=</mo><mrow><mi>kCb</mi><mo>=</mo><mrow><mfrac><msup><mrow><mo>(</mo><msub><mi>Y</mi><mn>0</mn></msub><mo>)</mo></mrow><mfrac><mn>1</mn><mi>α</mi></mfrac></msup><mi>Y</mi></mfrac><mo></mo><mi>Cb</mi></mrow></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>7</mn><mo>)</mo></mrow></mtd></mtr></mtable></math><img file="US20020171663A1-20021121-M00002.TIF" id="EMI-M00002" he="46.0971" wi="216.027" img-format="tif" img-content="mf" /><attachments><attachment idref="MATHEMATICA-00002" attachment-type="nb" file="US20020171663A1-20021121-M00002.NB" /></attachments></maths>

[0066] where “k” in expression (7) denotes a correction value parameter of the color-difference signals.

[0067] Referring now to FIG. 6, a first configuration example of the digital camera, which is the embodiment of the present invention, will be described A digital camera <b>30</b> using a single-chip CCD acquires the optical images of a subject in the form of image signals of a wide dynamic range (e.g., 16-bit width) and converts them into image signals of a narrow dynamic range (e.g., 8-bit width), as necessary, then outputs the image signals of the narrow dynamic range.

[0068] The optical system of the digital camera <b>30</b> includes a lens <b>31</b> for condensing the optical images of a subject, an aperture <b>32</b> for adjusting the quantity of light of optical images, a shutter <b>33</b> for adjusting the incident time of optical images, an infrared blocking filter (IR filter) <b>34</b> for removing light rays in an infrared range in which a CCD <b>37</b> is extremely sensitive, an optical low-pass filter (LPF) <b>35</b> for removing high-frequency components from light to restrain turnovers that occur when the CCD <b>37</b> carries out discrete sampling, and a color filter <b>36</b> having the primary colors (red (R), green (G), and blue (B)) arranged as shown in, for example, FIG. 7A, for extracting the three primary colors, R, G, and B, from optical signals.

[0069] The lens <b>31</b> may be provided with a coating for shielding against infrared ranges so as to obviate the need for providing the infrared blocking filter <b>34</b>.

[0070]FIG. 7 shows the color filter <b>36</b> covering the ranges for the four pixels of the CCD <b>37</b>. For the color filter <b>36</b>, the filter of the complementary colors (yellow (Ye), cyan (Cy), and magenta (Mg)) and green (G) arranged as shown in, for example, FIG. 7B or FIG. 7C may be used.

[0071] The digital camera <b>30</b> is constructed with the CCD <b>37</b> which photoelectrically converts the optical images received through the intermediary of the optical system and outputs image signals of a wide dynamic range to a correlated double sampling (CDS) circuit <b>38</b>. The CDS circuit <b>38</b> samples the image signals supplied from the CCD <b>37</b>, thereby reducing noise. An auto gain controller (AGC) <b>39</b> is provided for electrically amplifying the amplitudes of image signals according to the brightness of a subject and an A/D converter <b>40</b> digitizes the analog image signals. An image processor <b>41</b> formed primarily of a digital signal processor (DSP) converts the digital image signals of a wide dynamic range into digital image signals of a narrow dynamic range. Meanwhile, a timing generator <b>42</b> is provided for generating horizontal scanning timings and vertical scanning timings of the CCD <b>37</b>. A control unit <b>43</b> controls a drive <b>45</b> to read a control program stored in a magnetic disk <b>46</b>, an optical disk <b>47</b>, a magneto-optical disk <b>48</b>, or a semiconductor memory <b>49</b>, and controls the entire digital camera <b>30</b> mainly according to the read control program and a command from a user entered through an operation unit <b>44</b>.

[0072] In addition to changing the dynamic range of input digital image signals, the image processor <b>41</b> compresses and encodes input digital image signals into still images according to the joint photographic experts group (JPEG) method or the like and records the still images in a predetermined recording medium, or compresses and encodes input digital image signals into animation images according to the moving picture experts group (MPEG) method or the like and records the animation images in a predetermined recording medium, or reads encoded image signals from a predetermined recording medium and decodes them.

[0073] In the digital camera <b>30</b>, an optical image entered through the intermediary of the optical system which includes all of the components from the lens <b>31</b> through the color filter <b>36</b>, is converted into an image signal of a wide dynamic range by the CCD <b>37</b>. The image wsignal is then subjected to noise reduction processing by the CDS circuit <b>38</b> and amplitude amplification by the AGC <b>39</b>. The signal is then digitized by the A/D converter <b>40</b>, and supplied to the image processor <b>41</b>, where the digital image signal of the wide dynamic range supplied to the image processor <b>41</b> is converted into a digital image signal of a narrow dynamic range.

[0074]FIG. 8 shows in detail a first configuration of the image processor <b>41</b>. One of the three primary color signals R, G, and B, which have been digitized, is supplied for one pixel to the image processor <b>41</b>.

[0075] A white balance circuit (WB) <b>51</b> adjusts the primary color signal level of each pixel to accomplish color balance in an image, and outputs the image to a gamma (γ) correction circuit <b>52</b>. The gamma correction circuit <b>52</b> carries out gamma correction on the primary color signal of each pixel, and supplies the result to a delay circuit (DL) <b>53</b>. The delay circuit <b>53</b> synchronizes the phases of the signals of color channels that have different sampling phases. The signals will be used for the processing in which an RGB interpolation and synchronization circuit <b>54</b> restores the three primary color signals R, G, and B of each pixel by using a predetermined number of adjacent pixels. The signals are also used for the processing in which an aperture correction signal generating circuit <b>55</b> generates edge highlighting signals by employing a predetermined number of adjacent pixels. The delay circuit <b>53</b> outputs the signals with synchronized phases to the RGB interpolation and synchronization circuit <b>54</b> and the aperture correction signal generating circuit <b>55</b>.

[0076] In order to restore the three primary color signals R, G, and B for each of all pixels, the RGB interpolation and synchronization circuit <b>54</b> interpolates missing color signals by using the red signals R, the green signals G, or the blue signals B of pixels adjacent to target pixels, and synchronizes the phases of the three primary color signals R, G, and B for the pixels, then simultaneously outputs them to an adder <b>56</b>. The aperture correction signal generating circuit <b>55</b> generates the edge highlighting signal for each of all pixels by using the red signal R, the green signal G, or the blue signal B of a pixel adjacent to a target pixel, and outputs the generated signals to the adder <b>56</b>.

[0077] The adder <b>56</b> adds the edge highlighting signals received from the aperture correction signal generating circuit <b>55</b> to the three primary color signals R, G, and B received from the RGB interpolation and synchronization circuit <b>54</b> to perform an f characteristic correction, and outputs the results to a luminance signal generating matrix circuit <b>57</b> and a band limiting filter <b>58</b>.

[0078] The luminance signal generating matrix circuit <b>57</b> substitutes the three primary color signals R, G, and B received from the adder <b>56</b> into expression (1) to generate the luminance signal Y, and outputs the luminance signal Y to a luminance correction circuit <b>59</b> and a correction parameter arithmetic circuit <b>60</b>.

[0079] The band limiting filter <b>58</b> restricts the bands of the three primary color signals R, G, and B from the adder <b>56</b> in order to adjust them to the bands of the color difference signals Cr and Cb, then outputs the three primary color signals to a color-difference signal generating matrix circuit <b>61</b>. The color-difference signal generating matrix circuit <b>61</b> substitutes the three primary color signals R, G, and B with the restricted bands into expressions (1) and (2) thereby to generate the color-difference signals Cr and Cb, and outputs the color-difference signals Cr and Cb to a color-difference correction circuit <b>62</b>.

[0080] The luminance correction circuit <b>59</b> corrects the luminance signal Y received from the luminance signal generating matrix circuit <b>57</b> into a luminance signal Y₀, and outputs the corrected luminance signal Y₀to the correction parameter arithmetic circuit <b>60</b>. The correction parameter arithmetic circuit <b>60</b> generates a correction parameter k corresponding to the luminance signal Y received from the luminance signal generating matrix circuit <b>57</b>, and outputs the correction parameter k to the color-difference correction circuit <b>62</b>. The color-difference correction circuit <b>62</b> multiplies the color-difference signals Cr and Cb received from the color-difference signal generating matrix circuit <b>61</b> by the correction parameter k received from the correction parameter arithmetic circuit <b>60</b> so as to generate color-difference signals Cr₀and Cb₀.

[0081]FIG. 9 shows the detailed configuration examples of the luminance correction circuit <b>59</b> and the correction parameter arithmetic circuit <b>60</b> in the image processor <b>41</b>.

[0082] A histogram generating circuit <b>71</b> of the luminance correction circuit <b>59</b> generates, in units of images, the histograms of the luminance signals Y sequentially supplied from the luminance signal generating matrix circuit <b>57</b>, and outputs the histograms to a luminance signal correction LUT generating circuit <b>72</b>. The luminance signal correction LUT generating circuit <b>72</b> accumulates the histograms of the luminance signals Y supplied from the histogram generating circuit <b>71</b> so as to create a cumulative histogram, as indicated by a curve A in FIG. 4, and performs logarithmic approximation on the cumulative histogram to create the luminance signal correction LUT, as indicated by a curve B in FIG. 4, then outputs the luminance signal correction LUT to a LUT memory <b>73</b>. The LUT memory <b>73</b> stores the luminance signal correction LUTs supplied from the luminance signal correction LUT generating circuit <b>72</b>, and applies the luminance signals Y received from the luminance signal generating matrix circuit <b>57</b> to the luminance signal correction LUTs to obtain corrected luminance signals Y₀.

[0083] The color-difference signal correction parameter LUT generating circuit <b>81</b> refers to a luminance signal correction LUT retained in the LUT memory <b>73</b> to generate a color-difference signal correction parameter LUT indicating the correspondence relationship between the luminance signal Y and the correction parameter k (expression (7)), and outputs the generated color-difference signal correction parameter LUT to a LUT memory <b>82</b>. The LUT memory <b>82</b> stores the color-difference signal correction parameter LUTs received from the color-difference signal correction parameter LUT generating circuit <b>81</b>, and also applies the luminance signals Y received from the luminance signal generating matrix circuit <b>57</b> to the color-difference signal correction parameter LUTs to obtain the correction parameter k, then outputs the correction parameter k to the color-difference correction circuit <b>62</b>.

[0084] Referring now to the flowchart of FIG. 10, the gray level correction processing carried out by the image processor <b>41</b> will be described. The gray level correction processing is performed on the luminance signals Y output from the luminance signal generating matrix circuit <b>57</b> and the color-difference signals Cr and Cb output from the color-difference signal generating matrix circuit <b>61</b>.

[0085] In step S<b>1</b>, the histogram generating circuit <b>71</b> of the luminance correction circuit <b>59</b> generates, in units of images, the histograms of the luminance signals Y received from the luminance signal generating matrix circuit <b>57</b>, and outputs the histograms to the luminance signal correction LUT generating circuit <b>72</b>. In step S<b>2</b>, the luminance signal correction LUT generating circuit <b>72</b> accumulates the histograms of the luminance signals Y received from the histogram generating circuit <b>71</b> to generate a cumulative histogram, and subjects the cumulative histogram to logarithmic approximation thereby to generate a luminance signal correction LUT. The generated luminance signal correction LUT is stored in the LUT memory <b>73</b>.

[0086] In step S<b>3</b>, the color-difference signal correction parameter LUT generating circuit <b>81</b> refers to a luminance signal correction LUT retained in the LUT memory <b>73</b> to generate a color-difference signal correction parameter LUT, and stores the generated color-difference signal correction parameter LUT in the LUT memory <b>82</b>.

[0087] In step S<b>4</b>, the LUT memory <b>73</b> applies the luminance signal Y received from the luminance signal generating matrix circuit <b>57</b> to the luminance signal correction LUT to acquire a corrected luminance signal Y₀.

[0088] In step S<b>5</b>, the LUT memory <b>82</b> applies the luminance signal Y received from the luminance signal generating matrix circuit <b>57</b> to the color-difference signal correction parameter LUT, and outputs the obtained correction parameter k to the color-difference correction circuit <b>62</b>.

[0089] In step S<b>6</b>, the color-difference correction circuit <b>62</b> multiplies the color-difference signals Cr and Cb received from the color-difference signal generating matrix circuit <b>61</b> by the correction parameter k from the LUT memory <b>73</b> so as to obtain corrected color-difference signals Cr₀and Cb₀.

[0090] As explained above, according to the gray level correction processing carried out by the image processor <b>41</b>, the correction parameter k acquired from the color-difference signal correction parameter LUT generated on the basis of the luminance signal Y gray level correction LUT is used as a multiplier to correct the color-difference signals Cr and Cb. Hence, the corrected color-difference signals Cr₀and Cb₀do not overflow, thus restraining changes in hue.

[0091]FIG. 11 shows the details of a second configuration example of the image processor <b>41</b>. The second configuration example is different from the first configuration example shown in FIG. 8 in that the gamma correction circuit <b>52</b> has been replaced by a gamma correction circuit <b>71</b> provided between the RGB interpolation &amp; synchronization circuit <b>54</b> and the adder <b>56</b>. In the second configuration example shown in FIG. 11, the same circuits as those in the first configuration example shown in FIG. 8 will be assigned the same reference numerals, and the descriptions thereof will not be repeated.

[0092]FIG. 12 shows in detail a third configuration example of the image processor <b>41</b>. The third configuration example is different from the first configuration example shown in FIG. 8 in that the gamma correction circuit <b>52</b> and the adder <b>56</b> have been removed. Instead, a gamma correction circuit <b>81</b>, an adder <b>82</b>, and a gamma correction circuit <b>83</b> are provided in the third configuration example.

[0093] The gamma correction circuit <b>81</b> carries out gamma correction on the three primary color signals R, G, and B from the RGB interpolation &amp; synchronization circuit <b>54</b>, and outputs the results to the band limiting filter <b>58</b>. The adder <b>82</b> adds an edge highlighting signal from the aperture correction signal generating circuit <b>55</b> to the luminance signal Y from the luminance signal generating matrix circuit <b>57</b> thereby to make an f characteristic correction, and outputs the result to the gamma correction circuit <b>83</b>. The gamma correction circuit <b>83</b> carries out gamma correction on the luminance signal Y with the corrected f characteristic received from the adder <b>82</b>, and outputs the result to the luminance correction circuit <b>59</b> and the correction parameter arithmetic circuit <b>60</b>. In the third configuration example shown in FIG. 12, the same circuits as those in the first configuration example shown in FIG. 8 are assigned the same reference numerals, and the descriptions thereof will not be repeated.

[0094]FIG. 13 shows a detailed block diagram of a fourth configuration example of the image processor <b>41</b>. In the fourth configuration example, the color filter <b>36</b> of the digital camera <b>30</b> shown in FIG. 6 uses the filter of the complementary colors arranged as shown in FIG. 7B or FIG. 7C.

[0095] If the color filter <b>36</b> uses the filter of the complementary colors arranged as shown in FIG. 7B, then digitized complementary color signals Ye, G, and Cy will be supplied to the image processor <b>41</b> shown in FIG. 13. If the color filter <b>36</b> uses the filter of the complementary colors arranged as shown in FIG. 7C, then digitized complementary color signals Ye, Cy, G, and Mg will be supplied to the image processor <b>41</b> shown in FIG. 13. The following will describe a case where the color filter <b>36</b> uses the filter of the complementary colors arranged as shown in FIG. 7B, and the image processor <b>41</b> shown in FIG. 13 receives digitized complementary color signals Ye, G, and Cy.

[0096] A delay circuit <b>91</b> synchronizes the phases of the signals of the complementary color channels having different sampling phases that have been received from an upper stage. The delay circuit <b>91</b> supplies the signals with the synchronized phases to a luminance signal generating circuit <b>92</b>, an aperture correction signal generating circuit <b>93</b>, and an interpolation and band limiting filter <b>94</b>. The luminance signal generating circuit <b>92</b> generates the luminance signals Y for all pixels by using the complementary color signals Ye, G, and Cy of the pixels in the neighborhood, and outputs the generated luminance signals Y to an adder <b>95</b>. The aperture correction signal generating circuit <b>93</b> generates edge highlighting signals respectively corresponding to all the pixels by using the complementary color signals Ye, G, and Cy of the pixels in the neighborhood, and outputs the generated edge highlighting signals to the adder <b>95</b>.

[0097] In order to restore the complementary color signals Ye, G, and Cy for each of all pixels, the interpolation and band limiting filter <b>94</b> interpolates missing complementary color signals by using the complementary color signals Ye, G, and Cy of pixels adjacent to target pixels, and synchronizes the phases of the complementary color signals Ye, G, and Cy for the pixels, then outputs them to a primary color separating matrix circuit <b>97</b>.

[0098] The adder <b>95</b> adds the edge highlighting signals received from the aperture correction signal generating circuit <b>93</b> to the luminance signals Y received from the luminance signal generating circuit <b>92</b> thereby to effect an f characteristic correction, and outputs the results to a gamma correction circuit <b>96</b>. The gamma correction circuit <b>96</b> carries out gamma correction on the luminance signals Y, which has been subjected to the f characteristic correction, from the adder <b>95</b>, and outputs the results to a luminance correction circuit <b>59</b> and the correction parameter arithmetic circuit <b>60</b>.

[0099] The primary color separating matrix circuit <b>97</b> generates the three primary color signals R, G, and B by using the complementary color signals Ye, G, and Cy for each pixel supplied from the interpolation and band limiting filter <b>94</b>, and outputs the generated signals to a white balance circuit <b>98</b>.

[0100] The white balance circuit <b>98</b> adjusts the levels of the three primary color signals R, G, and B of each pixel to accomplish color balance in an image, and outputs the image to a gamma correction circuit <b>99</b>. The gamma correction circuit <b>99</b> carries out gamma correction on the three primary color signals R, G, and B of each pixel, and supplies the result to a band limiting filter <b>58</b>.

[0101] In the fourth configuration example shown in FIG. 13, the same circuits as those of the first configuration example shown in FIG. 8 are assigned the same reference numerals, and the descriptions thereof will not be repeated.

[0102]FIG. 14 shows a second configuration example of the digital camera according to the embodiment of the present invention. As in the case of the digital camera <b>30</b> shown in FIG. 6, a digital camera <b>110</b> acquires the optical images of a subject in the form of image signals of a wide dynamic range (e.g., 16-bit width) and converts them into image signals of a narrow dynamic range (e.g., 8-bit width), as necessary, then outputs the image signals of the narrow dynamic range.

[0103] The digital camera <b>110</b> uses a three-chip CCD (a CCD <b>37</b>R through <b>37</b>B), whereas the digital camera <b>30</b> shown in FIG. 6 uses the single-chip CCD (the CCD <b>37</b>). This means that the digital camera <b>110</b> obviates the need for the color filter <b>36</b> used in the digital camera <b>30</b>. In place of the color filter <b>36</b>, a color separation prism <b>111</b> is provided for separating optical signals input through the intermediary of a lens <b>31</b> through a low pass filter <b>35</b> into the three primary color signals R, G, and B. Furthermore, the digital camera <b>110</b> is provided with the CCD <b>37</b>R through an A/D converter <b>40</b>R, a CCD <b>37</b>G through an A/D converter <b>40</b>G, and the CCD <b>37</b>B through an A/D converter <b>40</b>B for the three primary color signals R, G, and B, respectively, in place of the CCD <b>37</b> through the A/D converter <b>40</b> in the digital camera <b>30</b>. An image processor <b>112</b> of the digital camera <b>110</b> receives the three primary color signals R, G, and B of a wide dynamic range for each pixel.

[0104]FIG. 15 detailedly shows a configuration example of the image processor <b>112</b>. The image processor <b>112</b> receives all the three primary color signals R, G, and B for each pixel.

[0105] A white balance circuit <b>121</b>R adjusts the level of an input red signal R and outputs the result to a gamma correction circuit <b>122</b>R and a delay circuit <b>123</b>. A white balance <b>121</b>G adjusts the level of an input green signal G and outputs the result to a gamma correction circuit <b>122</b>G and the delay circuit <b>123</b>. The white balance circuit <b>121</b>B adjusts the level of an input blue signal B and outputs the result to a gamma correction circuit <b>122</b>B and the delay circuit <b>123</b>.

[0106] The gamma correction circuit <b>122</b>R carries out gamma correction on the red signal R from the white balance circuit <b>121</b>R, and outputs the result to a luminance signal generating matrix circuit <b>125</b> and a band limiting filter <b>127</b>. The gamma correction circuit <b>122</b>G carries out gamma correction on the green signal G from the white balance circuit <b>121</b>G, and outputs the result to the luminance signal generating matrix circuit <b>125</b> and a band limiting filter <b>127</b>. The gamma correction circuit <b>122</b>B carries out gamma correction on the blue signal B from the white balance circuit <b>121</b>B, and outputs the result to the luminance signal generating matrix circuit <b>125</b> and the band limiting filter <b>127</b>.

[0107] The delay circuit <b>123</b> simultaneously supplies the three primary color signals R, G, and B of a few pixels in the vicinity of a target pixel to an aperture correction signal generating circuit <b>124</b>. The signals are used for generating the edge highlighting signal for each pixel at the aperture correction signal generating circuit <b>124</b>. The aperture correction signal generating circuit <b>124</b> generates edge highlighting signals for all pixels by using the three primary color signals R, G, and B of the pixels in the neighborhood, and supplies the generated edge highlighting signals to an adder <b>126</b>.

[0108] The luminance signal generating matrix circuit <b>125</b> substitutes the three primary color signals R, G, and B received from the gamma correction circuit <b>122</b>R through <b>122</b>B into expression (1) to generate the luminance signal Y, and outputs the generated luminance signal Y to the adder <b>126</b>. The adder <b>126</b> adds the edge highlighting signal received from the aperture correction signal generating circuit <b>124</b> to the luminance signal Y received from the luminance signal generating matrix circuit <b>125</b> thereby to perform the f characteristic correction, and outputs the result to a luminance correction circuit <b>59</b> and a correction parameter arithmetic circuit <b>60</b>.

[0109] The band limiting filter <b>127</b> restricts the bands of the three primary color signals R, G, and B received from the gamma correction circuits <b>122</b>R through <b>122</b>B so as to adjust the bands of the three primary color signals to the bands of the color-difference signals Cr and Cb, and outputs the results to a color-difference signal generating matrix circuit <b>61</b>.

[0110] In the configuration example of the image processor <b>112</b> shown in FIG. 15, the same circuits as those in the first configuration example of the image processor <b>41</b> shown in FIG. 8 are assigned the same reference numerals, so that the descriptions will not be repeated.

[0111] The present invention can be applied also to a case where gray levels are changed without changing the dynamic ranges of images.

[0112] Moreover, the present invention can be applied to electronic equipment adapted to process image signals, including, for example, scanners, facsimiles, and copier, in addition to the digital camera in the embodiment.

[0113] It is possible to carry out a series of the processing steps described above by hardware and also by software. To carry out the series of processing steps by software, a computer in which a program constituting the software has been installed in dedicated hardware is used, or the program constituting the software is installed from a program storing medium into, for example, a general-purpose personal computer or the like that is capable of implementing various functions by installing various programs.

[0114] As shown in FIG. 6, the recording medium includes the magnetic disk <b>46</b> (including a floppy disk), the optical disk <b>47</b> (including a compact disc-read only memory (CD-ROM) and a digital versatile disc (DVD)), the magneto-optical disc <b>48</b> (including a mini-disc (MD)), or a package medium constituted by the semiconductor memory <b>49</b> or the like in which the program has been recorded and which is distributed separately from a computer to a user to provide the program. The recording medium also includes a ROM or a hard disk or the like in which the program has been recorded and provided preinstalled in a computer.

[0115] In the specification, the steps for describing the program recorded in the recording medium include the steps of processing carried out in time series in the described sequence and also the steps of processing carried out in parallel or individually rather than in time series.

[0116] Thus, according to the image processing apparatus and method, and the program in a recording medium, a first conversion rule is generated by using a first element of an input image signal, and a second conversion rule is generated on the basis of the first conversion rule. By using the second conversion rule, a correction parameter for the first element of the input image signal is acquired, and a second element of the input image signal is corrected by using the correction parameter. With this arrangement, the occurrence of hue changes can be restrained.

[0117] It should be understood that various changes and modifications to the presently preferred embodiments described herein will be apparent to those skilled in the art. Such changes and modifications can be made without departing from the spirit and scope of the present invention and without diminishing its intended advantages. It is therefore intended that such changes and modifications be covered by the appended claims.