Real-time image generator

p-0002The present application claims priority under 35 U.S.C. 119 to Korean Patent Application No. 10-2008-0138068 (filed on Dec. 31, 2008), which is hereby incorporated by reference in its entirety.

BACKGROUND

p-0003Wide dynamic range (WDR) image generators in the related art may be categorized into TRC types and TMO (Tone Mapping Operation) types. A TRC type calculates a proper tone curve of images, and performs tone-mapping based on the proper tone curve. A TMO type performs tone-mapping by using an operator calculating a luminance component proper to each pixel based on pre-calculated statistics of images.

p-0004The TRC type has a fast calculation time and a fast task completion time, with deteriorating performance. In contrast, the TMO type has better performance, but with a relatively long calculation time and a relatively slow task completion time.

p-0005A structure of a WDR image generator embodying such the TMO type tone mapping is shown in <figref idrefs="DRAWINGS">FIG. 1</figref>. <figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram illustrating a related WDR image generator. A color space transfer to RGB block <b>1</b> shown in <figref idrefs="DRAWINGS">FIG. 1</figref> transfers RGB (Red, Green and Blue) information of an input image into YCbCr color space having a brightness component (Y) and a chrominance component (C). Here, a mathematical formula of the luminance gain component (Y) is as follows:

p-0006<br /><i>L</i>_w=0.2654<i>*R</i>_W+0.6704<i>*G</i>_w+0.0642<i>*B</i>_w  FORMULA 1

p-0007A scene luminance calculation block <b>2</b> outputs only the luminance component L_w(i, j and t_k)) in the YCbCr color space transferred from the color space transfer to RGB <b>1</b>. A log-average luminance calculation block <b>5</b> calculates the luminance sum of the image by using an output cumulative value of the scene luminance calculation <b>2</b> and a natural log. This calculation is as follows:

p-0008<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>L</mi><mi>avg</mi></msub><mo>=</mo><mrow><mi>exp</mi><mo>(</mo><mfrac><mrow><mo>∑</mo><mrow><mi>log</mi><mo></mo><mrow><mo>(</mo><mrow><mi>δ</mi><mo>+</mo><msub><mi>L</mi><mi>w</mi></msub></mrow><mo>)</mo></mrow></mrow></mrow><mi>N</mi></mfrac><mo>)</mo></mrow></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>2</mn></mrow></mtd></mtr></mtable></math></maths>

p-0009The log-average luminance calculation <b>5</b> calculates an average value (L_avg(t_k−1)) of the luminance components of the images by using above Formula 1 and Formula 2 and it induces L_sof a next image by using a linear scaling parameter ‘α’ as follows:

p-0010<maths id="MATH-US-00002" num="00002"><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><msub><mi>L</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi></mi><mo>,</mo><mi>j</mi><mo>,</mo><msub><mi>t</mi><mi>k</mi></msub></mrow><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mfrac><mi>α</mi><mrow><msub><mi>L</mi><mi>avg</mi></msub><mo></mo><mrow><mo>(</mo><msub><mi>t</mi><mrow><mi>k</mi><mo>-</mo><mn>1</mn></mrow></msub><mo>)</mo></mrow></mrow></mfrac><mo>*</mo><mrow><msub><mi>L</mi><mi>w</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi></mi><mo>,</mo><mi>j</mi><mo>,</mo><msub><mi>t</mi><mi>k</mi></msub></mrow><mo>)</mo></mrow></mrow></mrow></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>3</mn></mrow></mtd></mtr></mtable></math></maths>

p-0011L_dthat is an operator applicable to the image is induced as shown in following Formula 4 by using L_sinduced from Formula 3.

p-0012<maths id="MATH-US-00003" num="00003"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>L</mi><mi>d</mi></msub><mo>=</mo><mfrac><mrow><mrow><msub><mi>L</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi></mi><mo>,</mo><mi>j</mi><mo>,</mo><msub><mi>t</mi><mi>k</mi></msub></mrow><mo>)</mo></mrow></mrow><mo>*</mo><mrow><mo>(</mo><mrow><mn>1.0</mn><mo>+</mo><mfrac><mrow><msub><mi>L</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi></mi><mo>,</mo><mi>j</mi><mo>,</mo><msub><mi>t</mi><mi>k</mi></msub></mrow><mo>)</mo></mrow></mrow><mrow><msubsup><mi>L</mi><mi>white</mi><mn>2</mn></msubsup><mo></mo><mrow><mo>(</mo><msub><mi>t</mi><mrow><mi>k</mi><mo>-</mo><mn>1</mn></mrow></msub><mo>)</mo></mrow></mrow></mfrac></mrow><mo>)</mo></mrow></mrow><mrow><mn>1.0</mn><mo>+</mo><mrow><msub><mi>L</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi></mi><mo>,</mo><mi>j</mi><mo>,</mo><msub><mi>t</mi><mi>k</mi></msub></mrow><mo>)</mo></mrow></mrow></mrow></mfrac></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>4</mn></mrow></mtd></mtr></mtable></math></maths>

p-0013The operator L_dgained from Formula 4 is applied to a color image C_winput from following Formula 5 and a final output image C_dis gained. Here, C_dis referenced to as R_d, G_dand B_d.

p-0014<maths id="MATH-US-00004" num="00004"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>C</mi><mi>d</mi></msub><mo>=</mo><mrow><msub><mi>L</mi><mi>d</mi></msub><mo>*</mo><msup><mrow><mo>(</mo><mfrac><msub><mi>C</mi><mi>w</mi></msub><msub><mi>L</mi><mi>w</mi></msub></mfrac><mo>)</mo></mrow><mi>γ</mi></msup></mrow></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>5</mn></mrow></mtd></mtr></mtable></math></maths>

p-0015In this way, the related WDR image generator performs a log operation with respect to the natural log and the exponential function. Because of that, the related WDR image generator has a relatively large load for performing real-time both of processing another image, the natural log operation, and the exponential function. In addition, WDR image generator is difficult to synchronize with the other blocks.

p-0016Furthermore, the related WDR image generator has to perform above Formula 3 to Formula 5 whenever each of the pixels is inputted in the blocks. Because of that, the related WDR image generator has to perform 4 division log operations that require the largest space and much time. As a result, the related WDR image generator is not appropriate for a high resolution image higher than 8 mega pixel level that requires a high speed clock.

p-0017Still further, the related WDR image generator uses the color space transfer calculation for performing the real-time image process as shown in Formula 1 and it would cause color distortion of the image because of correlation between the color component and the luminance component in the actual image. To prevent such color distortion, operators making proper computational adjustments may be required to correct color distortion, such as Formula 3 to Formula 5. In addition, flexibility of the image processing algorithm would deteriorate because the tuning point (y) of Formula 5 is added.

SUMMARY

p-0018Embodiments relate to image processing technology, more particularly, to a real-time wide dynamic range (WDR) image generator. Embodiments relate to a real-time image generator for real-time high speed image processing used in processing a high resolution image. Embodiments relate to a real-time image generator enabling real-time image processing even when other functions for processing images are performed.

p-0019Embodiments relate to a real-time image generator may include a first block extracting only a luminance component having a saturation, hue, and value domain from red, green and blue values of an image. A second block outputs a log summation value and pixel count value with respect to a luminance component of an overall image by using the extracted luminance component and a natural log value. A third block calculates a luminance average value of the image by using the natural log summation value and the pixel count value outputted in the second block, the third block generating a tone mapping look up table including a tone mapping operator (L_d) for each luminance range to obtain a final output image using the calculated luminance average value. The third block outputs a tone mapped red, green and blue value by multiplying a corresponding tone mapping operator (L_d) of the tone mapping look up table by a red, green and blue value of the input image.

DRAWINGS

p-0020<figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram illustrating a structure of a related WDR image generator.

p-0021<figref idrefs="DRAWINGS">FIG. 2</figref> is a block diagram illustrating an image pipeline for image processing.

p-0022<figref idrefs="DRAWINGS">FIG. 3</figref> is a block diagram illustrating a global tone mapping block of a WDR processing block.

p-0023<figref idrefs="DRAWINGS">FIG. 4</figref> is a diagram illustrating an embodiment of a look-up table of log (LUT of LOG) according to embodiments.

p-0024<figref idrefs="DRAWINGS">FIG. 5</figref> is a diagram illustrating a method of a natural log for an input image in the look-up table according to embodiments.

p-0025<figref idrefs="DRAWINGS">FIG. 6</figref> is a diagram illustrating an image processing timing in a firmware block.

p-0026<figref idrefs="DRAWINGS">FIG. 7</figref> is a block diagram illustrating a local tone mapping block.

p-0027<figref idrefs="DRAWINGS">FIG. 8</figref> is a diagram illustrating a triangle-shaped function for reducing logic computation applied to the local tone mapping.

DESCRIPTION

p-0028A real-time image generator according to embodiments will be described below with reference to the accompanying drawings. <figref idrefs="DRAWINGS">FIG. 2</figref> is a block diagram illustrating an image pipeline for image processing. A WDR image processing function of the image pipeline <b>2</b> shown in <figref idrefs="DRAWINGS">FIG. 2</figref> may include a process of calculating an operator of each pixel in a TMO type tone mapping function. A natural log and an exponential function may be simplified to be processed real-time without effect to the other functions in the image processing.

p-0029As shown in <figref idrefs="DRAWINGS">FIG. 2</figref>, a WDR processing block <b>14</b> may be provided in a RGB processing block <b>13</b>. An RGB to YUV processing block <b>15</b> and WDR processing may be performed before color space transfer from RGB to YUV. RGB data may be inputted in the WDR processing block <b>14</b> and RGB data is outputted.

p-0030As shown in <figref idrefs="DRAWINGS">FIG. 2</figref>, the image pipeline <b>10</b> may be configured to output an input image in real-time. Blocks of the image pipeline <b>10</b> according to embodiments may have an auxiliary structure of a firmware block <b>20</b>. Here, the firmware block may use, for example, a high performance Arm9 series core.

p-0031A sensor interface block <b>11</b>, a Bayer processing block, Y processing block <b>16</b> and C processing block <b>17</b> shown in <figref idrefs="DRAWINGS">FIG. 2</figref> are a known configuration of the image pipeline <b>10</b> and detailed description thereof will be omitted accordingly.

p-0032The WDR processing block <b>14</b> provided between the RGB processing block <b>13</b> and the RGB to YUV processing block <b>15</b> may be in communication with the firmware block <b>20</b>. The WDR processing block <b>14</b> shown in <figref idrefs="DRAWINGS">FIG. 2</figref> may include a global tone mapping block and a local tone mapping block.

p-0033<figref idrefs="DRAWINGS">FIG. 3</figref> is a block diagram illustrating the global tone mapping block included in the WDR processing block. As shown in <figref idrefs="DRAWINGS">FIG. 3</figref>, the global tone mapping block may include a WDR photometry block <b>50</b> and a WDR global tone mapping block <b>60</b>.

p-0034First of all, according to embodiments, only a luminance component (V) of HSV domain from a RGB value of an image may be used. Because of that, only the luminance component (V) may be extracted from the RGB value of the image in a RGB to V block <b>40</b>, except H and S components that are color components. The output of the RGB to V block <b>40</b> may be inputted in the RDR photometry block <b>50</b>.

p-0035The WDR photometry block <b>50</b> may calculate a natural log average value of the image, to perform a global tone mapping of a next image frame, and it outputs the natural log average value to the WDR global tone mapping block <b>60</b>. That is, if a current image frame is the n^thimage frame, the natural log average value gained in the RDR photometry block <b>50</b> may be used to generate a TMO look up table (LUT of TMO) <b>62</b> of the WDR global tone mapping block <b>60</b> which will be used in (n+1)^thimage frame. Specifically, a calculation log luminance summation block <b>51</b> calculates a log sum of the luminance component outputted in the RGB to V <b>40</b> by using the natural log in the log look up table (LUT of LOG) <b>52</b>.

p-0036As shown in <figref idrefs="DRAWINGS">FIG. 3</figref>, the output of the calculation log luminance summation block <b>51</b> may be written in a register <b>53</b>, together with a pixel count value of the processed image. Here, the output of the calculation log luminance summation <b>51</b> and the pixel count value may be written in the register <b>53</b> in the last pulse of a single period of a reference pulse (Href).

p-0037Two registers <b>53</b> of the WDR photometry block <b>50</b> hold 32-bit values. A value of the register <b>53</b>, which may be updated at the moment when the single period final pulse of the reference pulse (Href) finishes, is used to generate the LUT of TMO <b>62</b> for each pixel count value in the actual WDR global tone mapping block <b>60</b>. Here, the LUT of TMO <b>62</b> may correspond to SRAM in hardware. That is, the value of the register <b>53</b> updated in the LUT of TMO <b>62</b> of the WDR global tone mapping block <b>60</b> may be written in a Vsync invalid period and applied to the next image frame.

p-0038A process of performing computation with respect to the luminance component outputted in the RGB to V block <b>40</b> by using the natural log of the LUT of LOG <b>52</b> will be described below. In the related art, a natural log operation may be performed to obtain a natural log average value of a corresponding frame. However, according to embodiments, the LUT of LOG <b>52</b> includes 32 steps and it predicts and reads a natural log value of corresponding luminance. 13*32-bit (=416 bit) SRAM may be used in the look up table of LOG <b>52</b> used at this time.

p-0039<figref idrefs="DRAWINGS">FIG. 4</figref> is a diagram illustrating a configuration of the look up table of LOG (LUT of LOG) according to embodiments. If a look up table of a natural log with respect to all luminance values 0˜1023 of an input image, 13*1024 bit=13299 bit=12.987 byte, will be required, this may be difficult to use.

p-0040According to embodiments, samples may be processed in 32 steps, as shown in <figref idrefs="DRAWINGS">FIG. 4</figref>, to compose the log look up table <b>52</b>. A period having the input image included therein is searched and predicted. This embodiment is shown in <figref idrefs="DRAWINGS">FIG. 5</figref>. <figref idrefs="DRAWINGS">FIG. 5</figref> is a diagram illustrating a method of applying a natural log with respect to the input image in the LUT of LOG according to embodiments.

p-0041If natural log values (Y_n, Y_cur, Y_n+1) for the luminance values (V_n, V_cur, and V_n+1) inputted in the WDR photometry block <b>50</b> are given, the relevant Y_curmay be obtained from following Formula 6:

p-0042<maths id="MATH-US-00005" num="00005"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>Y</mi><mi>cur</mi></msub><mo>=</mo><mfrac><mrow><mrow><mrow><mo>(</mo><mrow><msub><mi>V</mi><mrow><mi>n</mi><mo>+</mo><mn>1</mn></mrow></msub><mo>-</mo><msub><mi>V</mi><mi>cur</mi></msub></mrow><mo>)</mo></mrow><mo>*</mo><msub><mi>Y</mi><mi>n</mi></msub></mrow><mo>+</mo><mrow><mrow><mo>(</mo><mrow><msub><mi>V</mi><mi>cur</mi></msub><mo>-</mo><msub><mi>V</mi><mi>n</mi></msub></mrow><mo>)</mo></mrow><mo>*</mo><msub><mi>Y</mi><mrow><mi>n</mi><mo>+</mo><mn>1</mn></mrow></msub></mrow></mrow><mrow><mrow><mo>(</mo><mrow><msub><mi>V</mi><mrow><mi>n</mi><mo>+</mo><mn>1</mn></mrow></msub><mo>-</mo><msub><mi>V</mi><mi>cur</mi></msub></mrow><mo>)</mo></mrow><mo>+</mo><mrow><mo>(</mo><mrow><msub><mi>V</mi><mi>cur</mi></msub><mo>-</mo><msub><mi>V</mi><mi>n</mi></msub></mrow><mo>)</mo></mrow></mrow></mfrac></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>6</mn></mrow></mtd></mtr></mtable></math></maths>

p-0043Since embodiments use the look up table in the natural log operation, logical computation may be reduced, and thus image processing speed may be improved. That is, the natural log operation for calculating the natural log value of the image in the real-time image generator is configured of the 32 step look up table and thus the logical operation used for the natural log operation is reduced, with the improved speed. Also, real-time operation is possible with not effects to the other function blocks during the image processing.

p-0044A process of generating the LUT of TMO <b>62</b> of the WDR global tone mapping block <b>60</b> by using both the output of the calculation log luminance summation block <b>51</b> and the pixel count value will be described below. After the natural log value for the input image is obtained by using the above LUT of LOG <b>52</b> and Formula 6, sum of log luminance of pixels in a region of interest (ROI) and the pixel count value are updated in the register <b>53</b>. This update is performed at the moment when the last pulse of the reference pulse (H_ref) with respect to the region of interest (ROI) finishes.

p-0045An average value of input image is calculated by using two values of the registers <b>53</b> in the firmware block <b>20</b> (<b>70</b> in <figref idrefs="DRAWINGS">FIG. 3</figref>) from the moment when the last pulse finishes before the Vsync invalid period finishes as shown in following Formula 7.

p-0046<maths id="MATH-US-00006" num="00006"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>L</mi><mi>avg</mi></msub><mo>=</mo><mrow><mi>exp</mi><mo></mo><mrow><mo>(</mo><mfrac><mrow><mi>sum</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mi>of</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mi>log</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mi>luminance</mi></mrow><mrow><mi>pixel</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mi>count</mi></mrow></mfrac><mo>)</mo></mrow></mrow></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>7</mn></mrow></mtd></mtr></mtable></math></maths>

p-0047After the average value (L_avg) of the image is gained, L_d(TMO) is calculated for each luminance range as shown in Formula 8 only to generate the LUT of TMO <b>62</b>.

p-0048<maths id="MATH-US-00007" num="00007"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>L</mi><mrow><mi>d</mi><mo></mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo></mo><mn>1</mn></mrow></msub><mo>=</mo><mfrac><mrow><mo>(</mo><mrow><msub><mi>L</mi><mi>s</mi></msub><mo>*</mo><mrow><mo>(</mo><mrow><mn>1</mn><mo>+</mo><mfrac><msub><mi>L</mi><mi>s</mi></msub><msubsup><mi>L</mi><mi>max</mi><mn>2</mn></msubsup></mfrac></mrow><mo>)</mo></mrow></mrow><mo>)</mo></mrow><mrow><mo>(</mo><mrow><mn>1</mn><mo>+</mo><msub><mi>L</mi><mi>s</mi></msub></mrow><mo>)</mo></mrow></mfrac></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>8</mn></mrow></mtd></mtr><mtr><mtd><mrow><msub><mi>L</mi><mi>s</mi></msub><mo>=</mo><mfrac><mrow><mi>α</mi><mo>*</mo><msub><mi>V</mi><mi>cur</mi></msub></mrow><msub><mi>L</mi><mi>avg</mi></msub></mfrac></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>9</mn></mrow></mtd></mtr></mtable></math></maths>

p-0049To calculate L_d1, which is an operator, L_sis calculated via α, which is a linear scaling parameter as shown in Formula 9. L_d1, which is an operator applicable to the image, is calculated by using L_scalculated from Formula 9.

p-0050After the operator L_d1is generated by using above Formula 8 and Formula 9, the generated L_d1is scaled corresponding to a bit level of the input image, for example, 8 bit or 10 bit. After that, a final operator L_dis calculated as shown in following Formula 10:

p-0051<maths id="MATH-US-00008" num="00008"><math overflow="scroll"><mtable><mtr><mtd><mrow><msub><mi>L</mi><mi>d</mi></msub><mo>=</mo><mfrac><mrow><mi>max</mi><mo></mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo></mo><mi>LuVal</mi><mo>*</mo><msub><mi>L</mi><mrow><mi>d</mi><mo></mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo></mo><mn>1</mn></mrow></msub></mrow><mrow><msub><mi>V</mi><mi>cur</mi></msub><mo>*</mo><mi>max</mi><mo></mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle><mo></mo><mi>Ld</mi></mrow></mfrac></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>10</mn></mrow></mtd></mtr></mtable></math></maths>

p-0052The value ‘maxLuVal’ of Formula 10 is a maximum one of all luminance values 0˜255 or 0˜1023 of the input image. In case of an 8 bit input image, the maximum value is 255 and in case of 10 bit input image, it is 1023. The value ‘maxLd’ is a maximum operator which is calculated, when ‘V_cur=maxLuVal’ is inputted in Formula 8 and Formula 9. The calculations of Formula 8 to Formula 10 may be performed with respect to a luminance range, for example, 0˜255 in case of a 8 bit input image and 0˜1023 in case of a 10 bit input image such that the LUT of TMO <b>62</b> is completed. The firmware block may use, for example, a high performance Arm9 series core.

p-0053The LUT of TMO <b>62</b> according to embodiments may be calculated in the firmware block <b>20</b> (<b>70</b> in <figref idrefs="DRAWINGS">FIG. 3</figref>). The firmware block may use, for example, a high performance Arm9 series core. The LUT of TMO <b>62</b> may be inputted in the register corresponding to the corresponding LUT of TMO <b>62</b> in the synchronization invalid (Vsync Invalid) period before a next image frame starts. As a result, addition of hardware logic may not be required at all and the number of the logical operations may not be increased even when division calculation is used in the formulas.

p-0054<figref idrefs="DRAWINGS">FIG. 6</figref> illustrates a timing of application of the register corresponding to the LUT of TMO after the processes of above Formula 8 to Formula 10 are performed in the firmware block <b>10</b> (<b>70</b> in <figref idrefs="DRAWINGS">FIG. 3</figref>). The Arm9 series firmware block <b>20</b> (<b>70</b> in <figref idrefs="DRAWINGS">FIG. 3</figref>), with the relatively high core capacity, may operate at a high speed, for example, of 200 Mhz and thus the operation performance timing of the firmware block may be as shown in the timing of <figref idrefs="DRAWINGS">FIG. 6</figref>.

p-0055The global tone mapping operation will be described as follows.

p-0056If a current image frame is the n^thimage frame, the LUT of TMO <b>62</b> shown in <figref idrefs="DRAWINGS">FIG. 3</figref> is a look up table with respect to the tone mapping operation (TMO) performed in the (n+1)^thimage frame. Multiplication operation of the operator L_dby R, G and B values of the input image as shown in <figref idrefs="DRAWINGS">FIG. 3</figref> is performed by using the (n+1)^thobtained TMO look up table <b>62</b> in the n^thimage frame. An output RGB value (R_o, G_oand B_o) is outputted as the result of the corresponding global tone mapping.

p-0057As follows, the local tone mapping block will be described. If the above global tone mapping is performed, a bright region maintains a current value and pixels of a dark region are brighter corresponding to a current camera image such that an overall contrast may be improved. However, such the global tone mapping improves the overall contrast of the image, with deteriorating a local contrast, for example, an overall hazy output image.

p-0058To improve such the local contrast, the local tone mapping for improving the local contrast should be performed after the global tone mapping according to embodiments.

p-0059<figref idrefs="DRAWINGS">FIG. 7</figref> is a block diagram illustrating the local tone mapping block.

p-0060R, G and B values inputted in the RGB to V block <b>80</b> are R_o, G_oand B_ovalues gained based on the result of the global tone mapping. Currently inputted R_i, G_iand B_ivalues are re-transferred into luminance components in the RGB to V block <b>80</b> and the transferred luminance components are inputted in a line memory <b>90</b>. hence, operation shown in following Formula 11 is performed in a local tone mapping operation block <b>91</b>:

p-0061<maths id="MATH-US-00009" num="00009"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>localAvg</mi><mo>=</mo><mfrac><mrow><mo>(</mo><mrow><mrow><mo></mo><mtable><mtr><mtd><msub><mi>V</mi><mn>11</mn></msub></mtd><mtd><msub><mi>V</mi><mn>12</mn></msub></mtd><mtd><msub><mi>V</mi><mn>13</mn></msub></mtd></mtr><mtr><mtd><msub><mi>V</mi><mn>21</mn></msub></mtd><mtd><msub><mi>V</mi><mn>22</mn></msub></mtd><mtd><msub><mi>V</mi><mn>23</mn></msub></mtd></mtr><mtr><mtd><msub><mi>V</mi><mn>31</mn></msub></mtd><mtd><msub><mi>V</mi><mn>32</mn></msub></mtd><mtd><msub><mi>V</mi><mn>33</mn></msub></mtd></mtr></mtable><mo></mo></mrow><mo>·</mo><mrow><mo></mo><mtable><mtr><mtd><mn>104</mn></mtd><mtd><mn>118</mn></mtd><mtd><mn>104</mn></mtd></mtr><mtr><mtd><mn>118</mn></mtd><mtd><mn>134</mn></mtd><mtd><mn>118</mn></mtd></mtr><mtr><mtd><mn>104</mn></mtd><mtd><mn>118</mn></mtd><mtd><mn>104</mn></mtd></mtr></mtable><mo></mo></mrow></mrow><mo>)</mo></mrow><mn>1024</mn></mfrac></mrow></mtd><mtd><mrow><mi>FORMULA</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><mn>11</mn></mrow></mtd></mtr></mtable></math></maths>

p-0062A dot shown in <figref idrefs="DRAWINGS">FIG. 11</figref> is a reference to a convolution operation. After calculating a local average in a 3×3 kernel of <figref idrefs="DRAWINGS">FIG. 11</figref>, a difference value (diffVal) between a V22 that is a center value in Row 2 Column 2 of Formula 11 and the local average may be calculated.

p-0063<br />diffVal=<i>V</i>₂₂−localAvg  FORMULA 12

p-0064To improve a local contrast of a pixel center of the currently inputted image, the difference value (dffVal) calculated from Formula 12 is added to the current pixel center value (V22) as shown in following Formula 13:

p-0065<br /><i>V</i>_out<i>=V</i>₂₂+diffVal  FORMULA 13

p-0066That is, if the local average value is smaller than the center pixel value, an absolute value of the difference values (diffVal) is added to the center value (V22) to increase the local average value correspondingly. If the local average value is larger than the center pixel value, the absolute value of the difference values (diffVal) is subtracted from the center pixel value (V22) to heighten contrast in the local region.

p-0067Such the local tone mapping makes up for the characteristic of overall image contrast deterioration caused by the global tone mapping.

p-0068The result of the local tone mapping may be gained from above Formula 13 and here the local contrast is emphasized too much in case the difference value (diffVal) is used as it is, such that the image may look awkward. Because of that, above Formula 13 may be converted like following Formula 14 to use:

p-0069<br /><i>V</i>_out<i>=V</i>₂₂+weight(<i>V</i>₂₂)*diffVal  FORMULA 14

p-0070The ‘Weight’ of Formula 14 may be calculated by using a triangle-shaped function simply shown in <figref idrefs="DRAWINGS">FIG. 8</figref> to reduce logical operations. <figref idrefs="DRAWINGS">FIG. 8</figref> is a diagram illustrating the triangle-shaped function applicable to the local tone mapping to reduce logical operations.

p-0071The weight look up table <b>93</b> using the triangle-shaped function shown in <figref idrefs="DRAWINGS">FIG. 8</figref> is configured to perform an operation of multiplying a corresponding weight value to the difference value (diffVal) outputted from the local tone mapping operation block <b>91</b>. For that multiplication operation, a multiplier <b>92</b> may be provided and a center value (Vc), for example, V22 may be added to the output of the multiplier <b>92</b>. Here, for this addition, an adder <b>94</b> may be further provided.

p-0072After calculating a rate of the input to the output, input images (R_i, G_iand G_j) having passed a delayer <b>97</b> may be multiplied by the calculated rate to calculate the final outputs (R_o, G_oand B_o) as shown in Formula 15. The input for calculating the rate is V_out(=V_o) calculated from Formula 14 and the output is the center value (V_c).

p-0073<br /><i>R</i>_o<i>=R</i>_i*ratio

p-0074<br /><i>G</i>_o<i>=G</i>_i*ratio

p-0075<br /><i>B</i>_o<i>=B</i>_i*ratio  FORMULA 15

p-0076Here, the ratio is V_o/V_cthat is the center value divided by the output (V_o) of the adder <b>94</b>.

p-0077According to embodiments, there are following effects.

p-0078In embodiments, the log look up table may be configured instead of using a natural log operation. As a result, logical operations may be reduced and the image processing speed may be improved accordingly. That is, the 32 stepped log look up table according to embodiments will replace the natural log operation for calculating the log average value according to the related real-time image generator, such that the logical operations used to represent the natural log operation may be reduced. As a result, the image processing speed may be improved enough to enable real-time image processing without any influence on the other function blocks.

p-0079Furthermore, complexity of the related TMO type logical operation may be improved and thus the image processing speed may be improved. That is, in the WDR algorithm based on the related art TMO, the tone mapping operator for every pixel has to be calculated. Because of that, the usage of the division operation requiring many logical operations would be increased and the other operation logic may be required for every pixel. As a result, the TPM type logical operation in the related art has a disadvantage of decreased overall operational speed. Since the clock rate is not high in an image having the low resolution, for example, under SVGA level that is less 800×600, there is no problem. However, in a digital still image camera having higher than a 8 mega pixel resolution, a high clock rate is required. The WDR algorithm based on the related tone mapping operations is not proper to this case. However, according to embodiments, the tone mapping operation requiring many operations may be transferred to a look-up table, and the look up table of the required tone mapping operation may be generated one time in a single frame.

p-0080Still further, only the HSV domain luminance components are used from the RGB values of the image according to embodiments. As a result, color distortion may not occur. That is, the color components may be also used in the luminance component calculation as shown in Formula 1 according to the related art image generator. Because of the interference between the luminance components and the color components, color distortion would occur. However, according to embodiments, only the HSV domain luminance components are used in embodiments. A completely orthogonal relation is formed between the SV plane that is the color related component and the V plane that is the luminance component. As a result, required operations may be transferred simply as shown in Formula 7 to Formula 10.

p-0081Still further, the input V_iof the luminance related component and the output Vo of the tone mapping operation may be calculated. The ratio of the output to the input (ratio=V_o/V_i) may be multiplied by the input R, G and B values simply, such that the output may be identical to the output calculated by transferring the tone mapping operation in HSV domain into HSV and RGB. As a result, without using all of the three components (H, S and V) by transferring RGB into HSV, embodiments only use the transferred luminance component. In addition, the logical operations used to transfer RGB into HSV and vice versa may be reduced and, therefore the memory used may be reduced.

p-0082It will be obvious and apparent to those skilled in the art that various modifications and variations can be made in the embodiments disclosed. Thus, it is intended that the disclosed embodiments cover the obvious and apparent modifications and variations, provided that they are within the scope of the appended claims and their equivalents.