Display of high dynamic range images on a global dimming display using bit depth reduction mapping metadata

CROSS-REFERENCE TO RELATED APPLICATIONS

0001This application claims priority to <patcit id="pcit0001" dnum="US61416728" dnum-type="L"><text>United States Provisional Patent Application No. 61/416,728 filed 23 November 2010</text></patcit>.

TECHNICAL FIELD

0002This disclosure addresses reduction of undesirable visually perceptible artifacts when an image encoded with a bit depth <i>m</i> is displayed on a display having a bit depth <i>n</i>, where <i>m&lt;n.</i> Metadata distributed with the image characterizes the image's luminance range.

BACKGROUND

0003A display's bit depth corresponds to the number of levels of brightness that each of the display's pixels can reproduce or display. Higher bit depth displays can reproduce more discrete levels of brightness. For example, a display having a bit depth of 1 may represent either one of 2¹=2 levels of brightness in each pixel (e.g. each pixel can be ON or OFF). By contrast, in a display having a bit depth of 10, the display may be able to control each pixel to have one of 2¹⁰= 1024 distinct levels of brightness.

0004A color display may provide separate sub-pixels for each of a number of primary colors. For example, a display may provide pixels comprising sub-pixels that are red, green and blue. The luminance and color of a pixel can be controlled by varying the brightness of each sub-pixel. Greater bit-depth in the control of sub-pixels permits more discrete levels of luminance to be represented and also permits a color display to display a larger numbers of different colors. With a bit-depth of one for each sub-pixel one sub-pixel can be black or red, another sub-pixel can be black or green, and the third sub-pixel can be black or blue. In this case, the pixel may represent any of 2^1×2^1×2¹=2³=8 colors.

0005A color display having a bit depth of 8 is capable of representing any one of 2⁸=256 levels of brightness in each displayed sub-pixel. For example, a value of 0 may represent a value at the bottom of the sub-pixel's luminance range (typically black) and a value of 255 may represent a value at the top of the sub-pixel's luminance range. Such a display can theoretically display any one of 2⁸·2⁸·2⁸=2²⁴=16,777,216 colors in each pixel.

0006To facilitate their display, images are encoded using various coding schemes. Bit depth is an attribute of such schemes. If an image is encoded using a bit depth of 8 then each one of the encoded image's sub-pixels (or pixels in the case of a monochrome image) may represent any one of 2⁸=256 levels of brightness. If the same image is encoded using a bit depth of 10 then each one of the encoded image's sub-pixels (or pixels in the case of a monochrome image) may represent any one of 2¹⁰=1,024 levels of brightness. Thus, a higher bit depth provides a finer granularity within the luminance range of the pixels.

0007Some encoding schemes for color images do not directly specify brightness of individual sub-pixels. For example, the LUV scheme specifies overall luminance (L) for a pixel and specifies color for the pixel using two chroma coordinate values U and V. Again, a greater bit depth can be used to increase the number of distinct luminance steps and/or colors that can be represented by the image data.

0008It is generally desirable that the luminance steps in a displayed image be small enough that a luminance difference of one step is not readily perceptible to the human visual system (HVS). Steps larger than this can result in visible artefacts such as banding, particularly in image regions where the luminance is slowly varying. Since higher bit depths make possible finer steps, higher bit depths are desirable for displaying images having higher luminance ranges. However, images encoded with higher bit depths are larger (i.e. consume more computer memory or storage space and more bandwidth on communication links) than images encoded with lower bit depths, and accordingly require increased computational processing time and resources before they can be displayed. Consequently, images are often encoded for distribution at lower than optimum bit depths, notwithstanding the availability of displays having higher bit depth capabilities and the fact that images are often initially acquired at higher bit depths.

0009<patcit id="pcit0002" dnum="WO9937096A1"><text>WO 99/37096 A1</text></patcit> discloses a method and apparatus for preserving the dynamic range of a high dynamic range information stream. The high dynamic range information stream is segmented, whereby each segment is remapped to a low dynamic range information stream. An auxiliary information stream includes segment and associated remapping information such that the initial high dynamic range information stream may be recovered in a decoding process.

0010<patcit id="pcit0003" dnum="US6829301B1"><text>US 6 829 301 B1</text></patcit> discloses a method and concomitant apparatus for compressing, multiplexing and encrypting, transporting, decrypting, decompressing and presenting high quality video information in a manner that substantially preserves the fidelity of the video information in a system utilizing standard quality circuits to implement high quality compression, transport and decompression.

0011<nplcit id="ncit0001" npl-type="s"><text>Kondo, T. et al.: "Adaptive Dynamic Range Coding Scheme for future Consumer Digital VTR", Video, Audio and Data Recording, International Conference, London, GB, vol. 79, 1 Jan 1988, pages 219-226, XP000472806</text></nplcit>, discloses an adaptive dynamic range coding scheme by dividing a picture into small blocks each having a reduced dynamic range. Image quality increases the more the image spatially correlates.

0012<patcit id="pcit0004" dnum="US2008018506A1"><text>US 2008/018506 A1</text></patcit> discloses a device and method for processing multimedia data to generate enhanced quality multimedia data at a receiver based on encoder assisted post-processing. The method includes identifying an indicator of a post-processing technique, encoding first multimedia data to form first encoded data, processing the first encoded data to form second multimedia data, the processing comprising decoding the first encoded data and applying the post-processing technique identified by the indicator, comparing the second multimedia data to the first multimedia data to determine difference information indicative of differences between the second multimedia data and the first multimedia data, and generating second encoded data based on the difference information.

0013<nplcit id="ncit0002" npl-type="s"><text>Chen, Min et al.: "JPEG Compatible Coding of High Dynamic Range Imagery using Tone Mapping and Vector Quantization", 25th Picture Coding Symposium, 24 April 2006, XP030080209</text></nplcit>, discloses a method for compressing high dynamic range images by tone mapping the high dynamic range images to low dynamic range images. Detail information is stored in metadata to provide an invertible tone mapping.

0014<patcit id="pcit0005" dnum="EP2144444A1"><text>EP 2 144 444 A1</text></patcit> discloses a method of compressing a stream of video frame data wherein tone mapping functions are determined for video frames of said stream, said tone mapping functions being different from one another for frames relating to different scenes and wherein it is suggested that the tone mapping functions will be altered for frames of the stream relating to the same scene.

0015<patcit id="pcit0006" dnum="US2010085374A1"><text>US 2010/085374 A1</text></patcit> discloses a liquid crystal display (LCD) device with an improved contrast ratio and a reduced electric power consumption and a corresponding driving method. The LCD device and the driving method thereof generate a data modulation control signal using single frame image data to be displayed on a liquid crystal panel, and then generating a plurality of modulated local dimming control signals for a plurality of divisional regions of the divided single frame image data. A modulated data, in which the single frame image data is compensated with the data modulation control signal is generated and applied to the liquid crystal panel. A plurality of driver signals each corresponding to the modulated local dimming control signals are applied to the blocks of a backlight unit. The divisional regions are opposite to the blocks, respectively.

0016There is a need for practical and cost effective methods and apparatus for distributing and reproducing images (both still and video images) having a desired image quality.

SUMMARY OF THE INVENTION

0017The invention is defined by the independent claims. The dependent claims concern optional features of the invention. This invention provides methods and apparatus that may be applied in the distribution, processing and display of image data. Aspects of the invention provide: methods for displaying image data that have been encoded in a lower bit depth format for distribution; display apparatus for displaying image data that have been encoded in a lower bit depth format for distribution; and an image processing system, comprising the display apparatus and an image encoding apparatus for encoding image data in a lower bit depth format for distribution. The invention may be applied, for example in distribution and display of movies, television programming and other video content. The invention may, for example, be embodied in televisions, video monitors, computer monitors, computer systems, special purpose displays and the like.

0018A first aspect of the invention provides a method for displaying images. The method comprises obtaining image data in a first format having a first bit depth and corresponding metadata. Based on the metadata the method generates a tone mapping for mapping the image data to a second format having a second bit depth greater than the first bit depth. The method then applies the tone mapping to map the image data to the second format and displays the second format data.

0019A second aspect of the invention provides a display apparatus for performing the method for displaying images according to the first aspect of the invention.

0020A third aspect of the invention provides an image processing system comprising the display apparatus according to the second aspect of the invention and an image encoding apparatus comprising an image analyzer configured to determine a range of values in image data for an image in a first format and to generate a mapping for the image from the first format to a second format having a lower bit depth than the first format. The image encoding apparatus comprises a mapping unit configured to map the image from the first format to the second format according to the mapping and an encoding unit configured to encode the second format image data and metadata representing the mapping into a distribution format.

0021Further aspects of the invention and features of example embodiments of the invention are described below and illustrated in the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

0022The appended drawings illustrate example non-limiting embodiments of the invention. <ul id="ul0001" list-style="none" compact="compact"><li><figref idref="f0001">Figure 1</figref> is a simplified block diagram depiction of a prior art imaging system in which n+-bit image data is encoded with a bit depth of m-bits and distributed to a display having a bit depth <i>n,</i> where <i>m&lt;n.</i></li><li><figref idref="f0001">Figure 2</figref> is a simplified schematic depiction of an imaging system in which n+-bit depth image data is distributed over distribution paths having a reduced bit depth. The example distribution paths include cable TV, satellite TV, disc (e.g. a DVD or a Blu-ray^™ disc) and internet paths.</li><li><figref idref="f0002">Figure 3</figref> schematically depicts display of bright and dark images respectively on a dimmable display, in accordance with the prior art.</li><li><figref idref="f0003">Figure 4</figref> schematically illustrates re-mapping a luminance range of image data.</li><li><figref idref="f0003">Figure 5</figref> is a flow chart illustrating a method of generating images used by the invention;</li><li><figref idref="f0004">Figure 6</figref> is a simplified flowchart illustrating a method of encoding, distributing and displaying images.</li><li><figref idref="f0005">Figure 7</figref> graphically depicts shifting and rescaling luminance steps while encoding image data to obtain improved image quality from image data presented in a format having limited bit-depth. In the illustrated example embodiment the bit depth is 5 bits.</li><li><figref idref="f0006">Figure 8</figref> schematically depicts distributing image data by way of a <i>m</i>-bit depth encoded image for display on a display capable of a bit depth of <i>n</i>-bits.</li><li><figref idref="f0007">Figure 9</figref> schematically depicts a system for encoding, distributing and displaying images.</li></ul>

DETAILED DESCRIPTION

0023Throughout the following description specific details are set forth in order to provide a more thorough understanding to persons skilled in the art. However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. Accordingly, the description and drawings are to be regarded in an illustrative, rather than a restrictive, sense.

BACKGROUND

0024<figref idref="f0001">Figure 1</figref> illustrates an example case in which image data <b>10</b> having a bit depth of at least n-bits (n+-bits) is converted to a lower bit depth of m-bits and encoded for distribution by an image processor <b>14.</b><i>m</i>-bit signal <b>15</b> is delivered to an image processor <b>16</b> (which may, for example, be internal to a display or external to the display). Image processor <b>16</b> renders the signal in a bit depth of n-bits for display on display <b>12.</b>

0025The bit depth of the image which is to be displayed can in practice constrain a display to an effective bit depth equal to that of the image, notwithstanding the display's potentially higher bit depth capability. For example, one approach to displaying an image encoded with a bit depth of 8 bits on a display having a bit depth of 10 bits is to use the 8-bit image data as the most-significant 8 bits of a 10-bit drive signal for the display. This has the advantage of using almost the full luminance range of the display. However, the resulting image may have banding or other undesirable visually perceptible artifacts because the full bit depth of the display is not utilized and so steps between adjacent luminance levels may be visible. Such artifacts can be particularly noticeable where the luminance range is large as, for example, in the case where the display is a high-brightness, high dynamic range display. Using the 8-bit depth image data as the least significant bits of the 10-bit drive signal for the display is also undesirable as the luminance range capability of the display is not fully or adequately utilized.

0026<figref idref="f0001">Figure 2</figref> depicts a high definition television (HDTV) <b>30</b> and a high definition computer monitor <b>32</b> connected to a computer <b>31.</b> Computer <b>31</b> and HDTV <b>30</b> are both connected to receive image data over various distribution media paths. HDTV <b>30</b> and computer monitor <b>32</b> are each capable of displaying image data having a bit depth of <i>n</i> bits. In each case, even if the image data originally had a larger bit depth, the image data is distributed in a lower bit-depth format (e.g. <i>m</i>-bits). In this example, image data <b>20</b> initially has a bit depth of n-bits or greater (<i>n</i>+-bits) and each distribution path includes at least one section in which the image data is carried in a format having a bit depth of <i>m</i> bits with <i>m&lt;n. m</i> is not necessarily the same for all distribution paths. Illustrated in <figref idref="f0001">Figure 2</figref> are an internet distribution path <b>21,</b> a disc (e.g. a DVD or a Blu-ray^™ disc) distribution path <b>22,</b> a cable TV distribution path <b>23,</b> and a satellite TV distribution path <b>24.</b>

0027Image data <b>20</b> is obtained from a suitable source such as a camera <b>25.</b> Internet distribution path <b>21</b> carries image data <b>20</b> in an m-bit depth format across the internet. Disc distribution path <b>22</b> records image data <b>20</b> in an m-bit depth format on a disc <b>26</b> that is played back on a disc player <b>27.</b> Cable television distribution path <b>23</b> carries image data <b>20</b> in an m-bit depth format across a cable television system network <b>28</b> to a cable TV set-top box <b>29</b> which distributes lower bit depth (i.e. m-bit, where <i>m</i>&lt;<i>n</i>) encoded versions of image data <b>20</b> to an n-bit backlit high dynamic range image display such as high definition television (HDTV) <b>30,</b> or high definition computer monitor <b>32.</b> Satellite TV distribution path <b>24</b> includes a satellite transmitter <b>33</b> which distributes an m-bit encoded versions of image data <b>20</b> to a satellite receiver <b>34.</b> It will be understood that imaging system variants having fewer or more image sources, fewer or more image processors, and fewer or more displays are possible.

0028Some backlit displays have a "global dimming" capability whereby all of the lighting elements in the display's backlight can be simultaneously dimmed or turned off. Global dimming can improve a display's luminance range in comparison to an equivalent display having no dimming capability.

0029Some other backlit displays have a "local dimming" capability whereby individual lighting elements or groups of lighting elements in the display's backlight can be selectively dimmed or turned off. Local dimming can significantly improve a display's dynamic range in comparison to an equivalent display having no dimming capability, or improve a display's local contrast and simultaneous range in comparison to an equivalent display having a global dimming capability.

0030<figref idref="f0002">Figure 3</figref> schematically depicts display of bright and dark images respectively on a dimmable display, using a log(<i>L</i>) graphical representation, where L represents luminance in nits. High dynamic range image data may, for example, specify luminance values ranging from 0.001 to 10,000 nits, corresponding to bar <b>41</b> in <figref idref="f0002">Figure 3</figref>. A display may be capable of displaying images having luminance values ranging from 0.1 to 600 nits, for example, corresponding to bar <b>42</b> indicated in <figref idref="f0002">Figure 3</figref>. If the display has a dimming capability then the same display may also be capable of displaying dark images having luminance values ranging from 0.01 to 60 nits, when the display's dimming capability is fully utilized-corresponding to bar <b>43</b> indicated in <figref idref="f0002">Figure 3</figref>.

0031<figref idref="f0002">Figure 3</figref> includes a histogram for one scene "Scene A" that may be displayed on the display. Scene A corresponds to a high dynamic range image of a bright scene characterized by luminance values ranging from 0.005 to 10,000 nits. The display's dimming capability is not used in order to minimize reduction of the image's brighter characteristics when the Scene A image is displayed. Pixels of Scene A for which the specified brightness exceeds 600 nits may be clipped to 600 nits or otherwise processed (e.g., scaled, tone-mapped, etc.) for display within the range indicated by bar <b>42.</b> Similarly, pixels of Scene A for which the specified brightness is less than 0.1 nits may be clipped to 0.1 nits or otherwise processed (e.g., scaled, tone-mapped, etc.) for display within the range indicated by bar <b>42.</b>

0032<figref idref="f0002">Figure 3</figref> also includes a histogram for another scene "Scene B" that corresponds to a high dynamic range image of a dark scene characterized by luminance values ranging from 0.001 to 200 nits. The display's dimming capability may be utilized to improve retention of the image's luminance characteristics within an intermediate luminance range indicated by bar <b>44</b> when the Scene B image is displayed. Any pixels of Scene B for which the specified brightness exceeds 200 nits may be clipped to 200 nits or otherwise processed (e.g., scaled, tone-mapped, etc.) for display within the range indicated by bar <b>44.</b> Similarly, pixels of Scene B for which the specified brightness is less than about 0.03 nits may be clipped to 0.03 nits or otherwise processed (e.g., scaled, tone-mapped, etc.) for display within the range indicated by bar <b>44.</b>

CONTENT METADATA IMAGE ENHANCEMENT

0033Embodiments of this invention redefine the mapping between image data values and corresponding luminance values. The redefinition may, for example scale and/or apply offsets to this mapping. This approach may be applied to obtain better quality images when image data is distributed in a lower bit-depth format. The redefined mapping is specified by metadata that is associated with the image data. The metadata may be encoded in the image data, delivered with the image data or delivered separately from the image data. The redefined mapping is implemented at a display in part through control of global dimming

0034<figref idref="f0003">Figure 4</figref> provides an exaggerated illustration of how remapping of image data values to luminance levels can be used to improve image quality. Specifically, the set of lines <b>49</b> illustrates a possible mapping of 128 image data values (a bit-depth of 7) to luminance values over a 500 nit luminance range. Lines <b>50</b> in <figref idref="f0003">Figure 4</figref> graphically depict a possible mapping of 32 image data values (a bit-depth of 5) to luminance values over the same 500 nit luminance range. In each case a gamma of 2.4 has been used to distribute the steps, as is common. The image data values for lines <b>49</b> may, for example, be provided by a 7-bit binary number (bit depth of 7). The image data values for lines <b>50</b> may, for example, be provided by a 5-bit binary number (bit depth of 5). It can be seen that the steps between adjacent luminance values in lines <b>50</b> are significantly larger than the steps between adjacent luminance values in lines <b>49.</b> Such large steps may be perceptible to the HVS and may result in visible artifacts, such as banding, in an image.

0035Encoding image data originally in a higher-bit depth format (as exemplified by lines <b>49</b>) into a lower bit-depth format (as exemplified by lines <b>50</b>) results in multiple discrete luminance values from the higher-bit depth image data being encoded as the same luminance value in the lower bit-depth image data and in the steps between adjacent, distinctly representable, luminance values being increased.

0036In many images, image data values are concentrated toward higher- or lower luminance and are not spread uniformly over the entire range of possible image data values. For example, a dark image may have no or few image values above some threshold value and a bright image may have no or few image values below some threshold value. For such images, altering the mapping between image data values and luminance values can allow smaller steps between adjacent luminance values without increasing the bit depth used to represent the image data values.

0037Consider the case of a dark image represented in higher-bit depth image data <b>49</b> in which no pixels have luminance values of over 100 nits. Mapping that image data to lower bit-depth image data <b>50</b> could result in significant loss of detail since lower bit-depth image data <b>50</b> can represent only 1/4 as many discrete brightness levels in the range of 0 to 100 nits as higher bit-depth image data <b>49.</b>

0038In <figref idref="f0003">Figure 4</figref>, the set of lines <b>52</b> illustrate luminance values in a case where the same lower bit-depth image data values corresponding to lines <b>50</b> have been re-mapped to correspond to luminance values in the range of 0 to 100 nits. In this case, the remapping comprises scaling by a factor that is less than 1 (a factor of 0.2 in the illustrated example). It can be seen that the steps between adjacent luminance values are reduced as compared to the steps between lines <b>50.</b> Mapping higher-bit depth image data <b>49</b> in the range of 0 to 100 nits to lower bit-depth data using the mapping exemplified by lines <b>52</b> can preserve detail that would have been lost if one used a conventional mapping like that of lines <b>50.</b>

0039In <figref idref="f0003">Figure 4</figref>, the set of lines <b>54</b> illustrate luminance values in a case where the same image data values corresponding to lines <b>50</b> have been re-mapped to luminance values in the range of 250 to 500 nits. In this case, the remapping includes both scaling and translation by an offset <b>56.</b> Again, it can be seen that the steps between adjacent luminance values are reduced as compared to the steps between lines <b>50.</b> Where higher bit-depth image data <b>49</b> depicts an image consisting essentially of luminance values in the range of 250 to 500 nits then a mapping as exemplified by lines <b>54</b> may be used to preserve detail when encoding that higher bit-depth image data in a lower bit-depth form.

0040Remapping of image data values to luminance values (or equivalent) may be performed for an entire image or scene (group of images) or for local regions within an image or scene. If remapping is done for local regions then the local regions may be regions specified by a predetermined grid, regions identified by analysis of image data or the like.

0041Where an image or part of an image has been encoded into a lower bit-depth representation using a special mapping then information specifying what mapping was used may be preserved and used subsequently to convert the image data back into a higher bit-depth representation for display. The information characterizing the mapping may, for example, be associated with the lower bit-depth image data as metadata.

0042<figref idref="f0003">Figure 5</figref> illustrates a method <b>55</b> that applies remapping as described above. Method <b>55</b> begins by analyzing an image or local region within an image in block <b>56.</b> The analysis determines whether the luminance values in the image or local region fall (or mostly fall) within a limited range. If so, then block <b>57</b> generates a mapping that can be used to map luminance values to image data values in a reduced bit-depth representation of the image and vice versa. For example, if the luminance values are all in the range of 250 nits to 500 nits then the mapping may correspond to lines <b>54</b> in <figref idref="f0003">Figure 4</figref>. In some embodiments, block <b>57</b> may perform clipping, compression, tone-mapping or like operations on image data to cause the luminance values to fall within a limited range. In some embodiments block <b>57</b> selects a mapping most suited to an image or local region within an image from among a number of predetermined mappings. In some embodiments, block <b>57</b> generates a custom mapping based on characteristics of the image or local region within the image.

0043Block <b>57</b> may generate a mapping based upon analysis of statistical information regarding the image or local region within the image. For example, block <b>57</b> may generate a mapping based upon a histogram of the image or local region within the image. The histogram may, for example, provide, for different luminance values or equivalent or different ranges of luminance values or equivalent the number of pixels corresponding to the luminance value or range or luminance values. From a histogram, one can determine whether: the image is primarily bright, primarily dark, or includes both bright and dark regions, whether the image includes a broad range of luminance values, includes a narrow range of luminance values, or the like. Such evaluation may be applied to select an appropriate mapping.

0044In block <b>58</b> the mapping of block <b>57</b> is applied to map luminance values (or equivalents) from image data to image data values that are encoded at a reduced bit depth. In block <b>59</b> the encoded image data is transmitted over a transmission path. In block <b>60</b> metadata that characterizes the mapping is also transmitted. Block <b>60</b> may comprise, for example, encoding the metadata in the image data itself, encoding the metadata as part of a data package that includes the image data, transmitting the metadata together with the image data or transmitting the metadata on a separate transmission path from the image data.

0045The metadata may specify a predetermined mapping and/or may provide parameters for a mapping (the parameters may, for example directly or indirectly specify an offset and/or a scaling factor and/or parameters of a linear or non-linear mapping function). In some embodiments, different mapping functions may be selected for images having different characteristics. For example the mapping used for images made up primarily of very bright highlights may have a different functional form than the mapping function used for images made up primarily of dark shadows.

0046In block <b>62</b> the metadata is applied to recreate the mapping and in block <b>64</b> the recreated mapping is applied to reformat the image data to a greater bit depth. In block <b>66</b> the reformatted image data is displayed. In method <b>55,</b> blocks <b>56</b> to <b>58</b> may be performed, for example, at a source of image data, blocks <b>62</b> and <b>64</b> may be performed at a display or at a device on a transmission path (such as a set top box or a transcoder, for example) upstream from a display.

0047It can be appreciated that application of method <b>55</b> to a series of images or local regions within an image can result in the same luminance values being encoded as different image data values for different ones of the images and/or for different local regions in the reduced bit-depth encoded image data. Application of method <b>55</b> to a series of images or local regions within an image may result in different luminance values being encoded as the same image data values for different ones of the images and/or for different local regions in the reduced bit-depth encoded image data.

0048<figref idref="f0004">Figure 6</figref> illustrates another example method <b>70</b> which can exploit dimming displays. In method <b>70,</b> metadata characterizing the luminance range of a high dynamic range image <b>71</b> can be distributed, with an m-bit encoded version of the image, to a high dynamic range display <b>72</b> which may be a non-dimming display, a global dimming display or a local dimming display. Display <b>72</b> has an n-bit depth capability, where <i>m&lt;n.</i> Image <b>71</b> may be obtained from any one of a variety of image sources, and display <b>72</b> may be any one of a variety of display types.

0049The luminance characteristic of image <b>71</b> is analyzed (<figref idref="f0004">Figure 6</figref>, block <b>74)</b> by a processor external to display <b>72.</b> The processor may comprise any one of a variety of image processors. If the block <b>74</b> analysis determines that image <b>71</b> is relatively bright (block <b>74</b> "bright" branch) then image <b>71</b> is m-bit encoded (block <b>76</b>) utilizing a mapping in which the brightness of each pixel is mapped to one of 2<i>^m</i> image data levels with a mapping such that the luminance values corresponding to the image data levels span image <b>71's</b> luminance range (which, because image <b>71</b> has no or only a few dark pixels, is smaller than the maximum luminance range that image <b>71</b> could have).

0050If the block <b>74</b> analysis determines that image <b>71</b> is relatively dark (block <b>74</b> "dark" branch) then image <b>71</b> is m-bit encoded (block <b>78</b>) utilizing a mapping in which the brightness of each pixel is mapped to one of 2<i>^m</i> image data levels with a mapping such that the luminance values corresponding to the image data levels span image <b>71's</b> relatively dark luminance range (which, because image <b>71</b> has no or only a few bright pixels, is also smaller than the maximum luminance range that image <b>71</b> could have). In general, image <b>71</b> is m-bit encoded utilizing a mapping in which the luminance or equivalent for each pixel is mapped to one of 2<i>^m</i> image data values such that the luminance values corresponding to the image data values span image <b>71's</b> luminance range.

0051In some embodiments the luminance range of image <b>71</b> (or a region within image <b>71</b>) excludes luminance values for outlying pixels. For example, the luminance range on which the mapping is based may not encompass luminance values below the X^th percentile for brightness and above the Y^th percentile for brightness. In some embodiments, the image analysis of block <b>74</b> may be configured such that a standard mapping is applied in the event that block <b>74</b> determines that the image is neither bright nor dark.

0052Metadata characterizing the mapping of block <b>76</b> (or from which the mapping can be inferred) is generated in block <b>76A.</b> Metadata characterizing the mapping of block <b>78</b> (or from which the mapping can be inferred) is generated in block <b>78A.</b> In either case the metadata may, for example, characterize the luminance range of the image data being encoded in the corresponding block <b>76</b> or <b>78.</b>

0053In block <b>80</b> the m-bit encoded image data is distributed. In block <b>79</b> the metadata is distributed. In some embodiments block <b>79</b> comprises embedding the metadata into the encoded image data so that distributing the encoded image data also distributes the metadata. Alternatively, the metadata may be distributed in another manner. In some embodiments the encoded image data and metadata are distributed via one or more distribution channels like those described above in relation to <figref idref="f0001">Figure 2</figref>.

0054In some embodiments, image data <b>71</b> has previously been analyzed (upstream from blocks <b>76</b> and <b>78</b>) and includes or is associated with metadata which directly or indirectly characterizes: the luminance ranges of images or regions within images represented in image data <b>71</b> or mappings to be applied to those images or regions within images. In such embodiments, blocks <b>76</b> and <b>78</b> may comprise extracting or accessing the previously-determined metadata and encoding the image data according to mappings based on that metadata. In some embodiments, block <b>74</b> characterizes the luminance characteristics of a current image or region within an image and the functions of blocks <b>76</b> and <b>78</b> are combined in a block which encodes the image data for the current image or region using a suitable mapping based upon the determination made in block <b>74.</b>

0055Upon receipt of the m-bit encoded version of image <b>71</b> at a display or other device, method <b>70</b> proceeds differently depending upon whether the device is a legacy device which is not configured to process the metadata (block <b>82</b> "YES" branch) or a display that can process the metadata (block <b>82</b> "NO" branch).

0056For a legacy device, the <i>m</i>-bit encoded version of image <b>71</b> is decoded but the metadata is ignored ( block <b>84</b>). The decoded image is accordingly displayed without regard to the metadata, thus maintaining compatibility with legacy displays. If the device is equipped to process the metadata ( block <b>82</b> "NO" branch) then the metadata is used to improve display of images in the encoded image data. The improved image quality is obtained by using metadata to control a dimming capability of display <b>72.</b> Metadata associated with the <i>m</i>-bit encoded version of image <b>71</b> is applied to recreate the mapping utilized to <i>m</i>-bit encode image <b>71</b> and the recreated mapping is applied to decode the image data to <i>n</i>-bit depth.

0057In some embodiments, method 70 includes a step (block <b>86</b>) of distinguishing between a global dimming display and a local dimming display. In method <b>70</b> where the display <b>72</b> is a global dimming display (block <b>82</b> "YES" branch and block <b>86</b> "GLOBAL" branch) then the m-bit encoded version of image <b>71</b> is decoded and the associated metadata (which may, for example, be derived or extracted from the encoded image data) is processed to determine the image's luminance characteristic. Display <b>72's</b> backlight is then adjusted (block <b>88</b>) in accordance with the metadata to best achieve the image's luminance characteristic, so that the m-bit encoded version of image <b>71</b> is mapped correctly to output light levels based on the metadata.

0058If display <b>72</b> is a local dimming display ( block <b>82</b> "NO" branch and block <b>86</b> "LOCAL" branch) then the m-bit encoded version of image <b>71</b> is decoded and the metadata associated with image <b>71</b> is processed to determine the image's local luminance characteristics. Display <b>72's</b> individual backlight elements are then selectively adjusted (block <b>90</b>) in accordance with the metadata to best achieve the image's luminance characteristic.

0059Decoding of m-bit encoded image <b>71</b> and backlight adjustment are interdependent. <i>m</i>-bit encoded image data <b>71</b> are decoded and the display's backlight are adjusted such that the combined re-mapping and backlight adjustment recreates or approximates the mapping utilized to <i>m</i>-bit encode image data <b>71</b> in step <b>76</b> or <b>78.</b> For instance, where a mapping utilized in step <b>76</b> or step <b>78</b> comprises a scaling and a translation by an offset, <i>m</i>-bit encoded image <b>71</b> may be decoded to apply the scaling and the brightness of the display's backlight may be adjusted to apply the offset.

0060In method <b>70,</b> for global dimming displays and local dimming displays, image quality is improved because the initial mapping to the m-bit encoded image data is altered to take into account global or local luminance characteristics such that steps between adjacent luminance levels are reduced, thereby reducing or eliminating perceptible artefacts resulting from larger steps between luminance levels.

0061<figref idref="f0005">Figure 7</figref> is similar to <figref idref="f0003">Figure 4</figref> but less exaggerated. <figref idref="f0005">Figure 7</figref> shows how levels of m-bit encoded data may be mapped to different luminance levels for a bright image, a dim image, and an image that includes a full range of luminance values.

0062In embodiments where different mappings are applied to local regions within an image additional processing may be performed to avoid perceptible boundaries between different local regions (whether defined according to a predetermined grid or otherwise). The additional processing may comprise, for example, one or more of: <ul id="ul0002" list-style="bullet" compact="compact"><li>spatial filtering;</li><li>dithering in the vicinity of region boundaries (e.g. in a vicinity of a region boundary randomly or quasi-randomly applying mappings from different adjacent regions to pixels);</li><li>blending between adjacent mappings (e.g. in the vicinity of a region boundary computing pixel values for a plurality of mappings associated with adjacent regions and interpolating between the mappings so as to provide a smooth transition from one mapping to another across the boundary region). In some embodiments the blending may be performed according to spline curves. The spline curves may be predefined or defined by metadata.</li></ul>

0063<figref idref="f0006">Figure 8</figref> schematically depicts another example embodiment of the invention. Original image data <b>100</b> is represented in <figref idref="f0006">Figure 8</figref> by a histogram. Original image data <b>100</b> is in a format providing a higher bit depth. For example, image data <b>100</b> may comprise data in a 16-bit integer or 16-bit float data format. Such formats permit a pixel to be assigned any one of a very large number of discrete luminance values within a full luminance range. Thus, steps between adjacent luminance values in image data <b>100</b> can be small. The maximum and minimum luminance values of original image data <b>100</b> span the maximum range <b>100A</b> of luminance values that can be represented by the data format.

0064Original image data <b>100</b> is windowed as indicated by <b>101</b> to provide windowed image data <b>102.</b> Windowed image data <b>102</b> comprises luminance values within a reduced luminance range <b>102A</b> that is smaller than the full luminance range <b>100A</b> that can be represented by the format of image data <b>100.</b> Windowing <b>101</b> may comprise clipping luminance values to endpoints of reduced luminance range <b>102A,</b> compressing luminance values that are outside of reduced luminance range <b>102A</b> into reduced luminance range <b>102A,</b> tone-mapping luminance values that are outside of reduced luminance range <b>102A</b> into reduced luminance range <b>102A,</b> a combination of these, or the like. In some embodiments, windowing <b>101</b> comprises tone compression, such that within a particular luminance range, windowed image data <b>102</b> comprises fewer different luminance values than original image data <b>100.</b> Reduced luminance range <b>102A</b> and the windowing operations by which windowed image data <b>102</b> is obtained from image data <b>100</b> may be selected to preserve details in original image data <b>100.</b> Windowed image data <b>102</b> may still be represented in a format having a relatively high bit depth (which may be the same format as that of original image data <b>100</b>).

0065As indicated by <b>103</b> windowed image data <b>102</b> is mapped onto values in a reduced bit-depth format to yield reduced bit-depth image data <b>104.</b> The mapping applied by <b>103</b> is such that the reduced luminance range <b>102A</b> of windowed image data <b>102</b> is mapped to a greater proportion of the range <b>104A</b> that can be represented by the reduced bit-depth format (i.e., the maximum range of the lower bit-depth representation) than would be the case if the full luminance range <b>100A</b> of image data <b>100</b> had been mapped to range <b>104A.</b> In some embodiments the mapping of reduced luminance range <b>102A</b> is to the full extent of range <b>104A</b> (or almost the full extent of range <b>104A,</b> for example, more than 85% or 90% or 95% or 98% of range <b>104A</b>). Thus all or almost all of the distinct values that can be represented by the reduced bit-depth format are used to represent values between the lower and upper ends of reduced luminance range <b>102A</b> of windowed luminance data <b>102.</b> Mapping <b>103</b> may comprise mapping <i>n</i>+-bit depth image data to <i>m</i>-bit image data with <i>m&lt;n.</i>

0066Metadata <b>107</b> characterizes the mapping used in mapping <b>103,</b> and, optionally, the windowing performed at <b>101,</b> such that a downstream process can use metadata <b>107</b> to determine how to reverse the mapping performed at <b>103,</b> and, optionally, part or all of the windowing performed at <b>101.</b>

0067Reduced bit-depth data <b>104</b> may be distributed by any suitable transport mechanism. The lower precision of reduced bit-depth data <b>104</b> results in reduced bit-depth data <b>104</b> being smaller and more compressible than original image data <b>100.</b> Metadata <b>107</b> is distributed with reduced bit-depth data <b>104.</b> Reduced bit depth data <b>104</b> is distributed to a display or an intermediate device at which it is desired to convert reduced bit depth data <b>104</b> back into a higher bit-depth format for display.

0068At the display or other device, reduced bit depth data is re-mapped as indicated by <b>105</b> to provide reconstructed data <b>106.</b> Reconstructed data <b>106</b> has a greater bit depth than reduced bit-depth data <b>104.</b> Metadata <b>107</b> is used by remapping step <b>105</b> to determine how the remapping ought to be carried out to most closely recreate windowed data <b>102.</b> Reconstructed data <b>106</b> can then be displayed.

0069In some embodiments, mapping <b>103</b> comprises mapping the top end of windowed image data <b>102</b> to the top end of reduced bit-depth image data <b>104</b> and the bottom end of windowed image data <b>102</b> to the lower end of reduced bit-depth data <b>104</b> and mapping intermediate values of windowed image data <b>102</b> to corresponding intermediate values of reduced bit-depth image data <b>104</b> according to a mapping that is uniform linearly or logarithmically. In other embodiments intermediate values are mapped according to a non-uniform mapping such that discrete levels of the reduced bit-depth image data <b>104</b> are separated by smaller luminance steps in luminance ranges having a lot of tone detail and separated by correspondingly larger luminance steps in luminance ranges in which there is not a lot of tone detail as compared to a uniform mapping. The amount of tone detail present in any particular luminance sub-range within luminance range <b>102A</b> of windowed image data <b>102</b> may, for example, be estimated from an image histogram as an increasing function of the number of pixels in the luminance sub-range and the number of discrete luminance values those pixels have in the luminance sub-range. Thus, portions of the luminance range of windowed image data <b>102</b> in which there are few pixels and few distinct luminance values could be mapped to reduced bit-depth data <b>104</b> such that there are relatively large steps between adjacent luminance values and portions of the luminance range of windowed image data <b>102</b> in which there are many pixels and many distinct luminance values could be mapped to reduced bit-depth data <b>104</b> such that there are relatively small steps between adjacent luminance values.

0070For example, the correspondence between image data values and luminances in original image data <b>100</b> and windowed image data <b>102</b> may be a linear or logarithmic relationship. By contrast, the correspondence between image data values and luminances in reduced bit-depth image data <b>104</b> may follow a nonlinear function such as a double s curve or a curve approximated by a polynomial.

0071As an example, a high bit depth high dynamic range image may be provided as a 10-bit log-encoded signal having an expected response range of 0.001 to 600 nits. The display has a global dimmable backlight, and may have an 8-bit liquid crystal display (LCD) panel having a 700:1 contrast ratio and a peak brightness capability of 500 nits. The luminance metadata characterizing one or more selected luminance characteristics of the image is derived as explained above. For example, the distribution of luminance values in the image can be used to derive an optimal tone curve corresponding to an 8-bit mapped version of the image. The optimal tone curve may take into account the capabilities of the display so as to maintain the most detail in the image. The metadata, e.g. the tone curve and black level are then sent to the display along with the image data. Knowing the image's luminance range from the metadata the display can map the image data into the appropriate luminance range at the higher native bit-depth of the display.

0072In some embodiments for at least some images, the lower bit-depth encoded data received by a display (or other intermediate image processing device) is directly mapped to bits of the higher bit-depth drive for the display. For example, in some cases, 8-bit lower-bit depth data may be mapped directly to the most significant bits of 10-bit image data for driving the display.

0073In contrast, a conventional prior art imaging system would typically use a predefined, invariant, mapping scheme to map the 10-bit log-encoded signal to produce an 8-bit power/gamma encoded version of the image, which would then be displayed using fixed brightness and response. As all of the mappings are fixed, dark scenes that are clear and detailed in the original image data input would be limited to fewer bits and more brightness on the bottom end.

0074<figref idref="f0007">Figure 9</figref> schematically illustrates apparatus <b>110</b> according to an example embodiment of the invention. Apparatus <b>110</b> comprises an image data preparation part <b>111A</b> and an image data restoration part <b>111B.</b> Image data preparation part <b>111A</b> does not necessarily operate in real time (i.e. fast enough to process video frames at the display rate of the video frames) although it may do so. Image data preparation part <b>111A</b> prepares the image data for distribution and image data restoration part <b>111B</b> restores image data for display.

0075Image data preparation part comprises a data store <b>112</b> containing original image data <b>114.</b> Original image data <b>114</b> may comprise still or video images. In the following discussion it is assumed that original image data <b>114</b> comprises video data and the video data comprises a series of video frames. Apparatus <b>110</b> comprises an image analysis module <b>116.</b> Image analysis module <b>116</b> determines a luminance range of a frame (or, in some embodiments, of a series of frames, or of a local region within a frame or series of frames). Optionally image analysis module <b>116</b> also evaluates whether conditions exist such that the frame may be windowed to have a smaller luminance range without undesirable degradation in image quality and, if so, the degree and/or type of windowing to be applied. In some embodiments image analysis module <b>116</b> generates or obtains a histogram for the frame and performs analysis on the histogram. The histogram may, for example, plot the number of pixels in an image having each pixel value.

0076In the illustrated embodiment, apparatus <b>110</b> comprises a windowing module <b>118</b> that performs windowing on each frame in response to a control signal from image analysis module <b>116.</b> A mapping unit <b>119</b> maps the windowed frames from windowing module <b>118</b> to a lower bit depth using a mapping based upon the luminance range of each windowed frame. The mapping can vary from frame-to-frame. In some embodiments mapping unit <b>119</b> provides a plurality of predetermined mappings. For example, mapping unit <b>119</b> may comprise a plurality of lookup tables that accept a luminance value (or equivalent) from the windowed frame as input and output a corresponding mapped value. Each lookup table may correspond to a different mapping. As another example, different mappings may be specified by different parameter values used by a processor (a plurality of sets of predetermined processor values may optionally be provided). As another example, different mappings may be provided by different logic pathways and/or by different software modules.

0077An encoder <b>120</b> encodes the lower bit depth image data together with metadata in a distribution format <b>122.</b> Metadata may be encoded into standard metadata paths such as supplemental enhancement information (SEI) messages, broadcast teletext, or the like. The image data in distribution format <b>122</b> may then be distributed, for example, by writing it to a disc or other distribution medium, broadcasting on a cable television or other broadcast medium, file transfer, streaming data transfer or the like. The distribution channel is indicated by <b>125</b> in <figref idref="f0007">Figure 9. Figure 9</figref> illustrates two architectures. In one case, the distributed image data is processed by a processor <b>130A</b> internal to a display <b>132A.</b> In another case, the distributed image data is processed by an external processor <b>130B</b> external to a display <b>132B.</b>

0078Displays <b>132A</b> and <b>132B</b> each have a bit depth greater than that of the distributed image data. Processors <b>130A</b> and <b>130B</b> each comprise a decoder <b>134</b> that decodes the distributed image data and a metadata extractor <b>136</b> that extracts the metadata from the distributed image data. A mapper <b>140</b> maps the distributed image data to the bit depth of the display according to a mapping set according to the metadata. The mapper may, for example, comprise a set of lookup tables or logic paths that implement different predetermined mappings, a logic path that implements mappings according to supplied parameter values or control inputs, a processor that implements a mapping algorithm that performs mappings according to supplied parameter values or control inputs, a processor that executes one of a plurality of predetermined mapping algorithms, or the like.

CONCLUSION

0079Systems and modules described herein may comprise software, firmware, hardware, or any combination(s) of software, firmware, or hardware suitable for the purposes described herein. Furthermore, aspects of the system can be embodied in a special purpose computer or data processor that is specifically programmed, configured, or constructed to perform one or more of the computer-executable instructions explained in detail herein.

0080Image processing and processing steps as described above may be performed in hardware, software or suitable combinations of hardware and software. For example, such image processing may be performed by a data processor (such as one or more microprocessors, graphics processors, digital signal processors or the like) executing software and/or firmware instructions which cause the data processor to implement methods as described herein. Such methods may also be performed by logic circuits which may be hard configured or configurable (such as, for example logic circuits provided by a field-programmable gate array "FPGA").

0081Certain implementations of the invention comprise computer processors which execute software instructions which cause the processors to perform a method of the invention. For example, one or more processors in a video workstation, set top box, display, transcoder or the like may implement methods as described herein by executing software instructions in a program memory accessible to the processors.

0082The methods of the invention may also be provided in the form of a program product. The program product may comprise any non-transitory medium which carries a set of computer-readable signals comprising instructions which, when executed by a data processor, cause the data processor to execute a method of the invention. Program products according to the invention may be in any of a wide variety of forms. The program product may comprise, for example, physical media such as magnetic data storage media including floppy diskettes, hard disk drives, optical data storage media including CD ROMs, DVDs, electronic data storage media including ROMs, flash RAM, hardwired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, or the like. The computer-readable signals on the program product may optionally be compressed or encrypted. Computer instructions, data structures, and other data used in the practice of the technology may be distributed over the Internet or over other networks (including wireless networks), on a propagated signal on a propagation medium (e.g., an electromagnetic wave(s), a sound wave, etc.) over a period of time, or they may be provided on any analog or digital network (packet switched, circuit switched, or other scheme).

0083Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise," "comprising," and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to." As used herein, the terms "connected," "coupled," or any variant thereof, means any connection or coupling, either direct or indirect, between two or more elements; the coupling of connection between the elements can be physical, logical, or a combination thereof. Additionally, the words "herein," "above," "below," and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. The word "or," in reference to a list of two or more items, covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.

0084As will be apparent to those skilled in the art in the light of the foregoing disclosure, many alterations, modifications, additions and permutations are possible in the practice of this invention without departing from the scope thereof. The embodiments described herein are only examples. Other example embodiments may be obtained, without limitation, by combining features of the disclosed embodiments.