Brightness characteristic generation method
3 claims: 3 independent, 0 dependent
- 1動画像を構成する複数のフレームのそれぞれについて、当該フレームを構成する複数の画素のうち第1輝度以下の画素の数を、当該フレームを構成する全画素の数で除すことで得られた値を、第1輝度特性として決定する第1決定ステップと、 前記第1決定ステップにおいて決定された前記第1輝度特性を出力する出力ステップと、 前記出力ステップにおいて出力された前記第1輝度特性を、前記動画像の時系列における輝度特性を示す特性情報として、前記動画像と共に映像データとして記録する記録ステップと、 を含む 輝度特性生成方法。
- 2動画像を構成する複数のフレームのそれぞれについて、当該フレームを構成する複数の画素のうち第1輝度以下の画素の数を、当該フレームを構成する全画素の数で除すことで得られた値を、第1輝度特性として決定する第1決定ステップと、 前記第1決定ステップにおいて決定された前記第1輝度特性を出力する出力ステップと、 前記動画像を前記複数のフレームのそれぞれについて、当該フレームにおける全画素を輝度が小さい画素から並べたとき、全画素の99.99%の数における輝度である最大輝度を特定し、特定した前記最大輝度を第2輝度特性として決定する第2決定ステップ と、 を含み、 前記出力ステップでは、前記第1輝度特性とともに、前記第2決定ステップで決定された前記第2輝度特性を出力する 輝 度特性生成方法。
- 3さらに、 前記出力ステップにおいて出力された前記第1輝度特性および前記第2輝度特性を、前記動画像の時系列における輝度特性を示す特性情報として、前記動画像とともに映像データとして記録する記録ステップを含む 請求項 2 に記載の輝度特性生成方法。
Independent claims3
210 paragraphs, as filed
The present disclosure relates to a luminance characteristic generation method for generating a luminance characteristic of an image.
Patent Document 1 describes a display device that updates a display method of an HDR signal based on dynamic HDR metadata in HDR (High Dynamic Range) video.
<p><patcit num="1"><text>JP-A-2017-184249</text></patcit></p>
<p><nplcit num="1"><text>White Paper Blu-ray Disc Read-Only Format (Ultra HD Blu-ray), Audio Visual Application Format Specifications for BD-ROM Version 3.1, August 2016 (http://www.blu-raydisc.com/Assets/Downloadablefile/BD) -ROM_Part3_V3.1_WhitePaper_160729_clean.pdf)</text></nplcit></p>
<p> The present disclosure provides a method for generating a luminance characteristic that can improve the quality of an image displayed on an image display device.</p>
<p> In the luminance characteristic generation method according to one aspect of the present disclosure, for each of the plurality of frames constituting the moving image, the number of pixels having the first luminance or less among the plurality of pixels constituting the frame constitutes the frame. A first determination step in which a value obtained by dividing by the number of all pixels is determined as a first luminance characteristic, an output step for outputting the first luminance characteristic determined in the first determination step, and an output step.<u style="single">A recording step in which the first luminance characteristic output in the output step is recorded as video data together with the moving image as characteristic information indicating the luminance characteristic in the time series of the moving image.</u>including.<u style="single"> In the luminance characteristic generation method according to another aspect of the present disclosure, for each of the plurality of frames constituting the moving image, the number of pixels having the first luminance or less among the plurality of pixels constituting the frame is determined. A first determination step in which a value obtained by dividing by the number of all constituent pixels is determined as a first luminance characteristic, and an output step for outputting the first luminance characteristic determined in the first luminance characteristic. When all the pixels in the frame are arranged from the pixel having the lowest brightness for each of the plurality of frames, the maximum brightness, which is the brightness in 99.99% of all the pixels, is specified, and the specified maximum brightness is specified. Is included as a second luminance characteristic, and in the output step, the second luminance characteristic determined in the second luminance characteristic is output together with the first luminance characteristic.</u></p>
<p> The present disclosure can provide a method for generating a luminance characteristic that can improve the quality of an image displayed on an image display device.</p>
<figref num="1">FIG. 1 is a diagram for explaining the evolution of video technology.</figref><figref num="2">FIG. 2 is a diagram for explaining the relationship between video production, distribution method, and display device when introducing a new video expression into the content.</figref><figref num="3A">FIG. 3A is a diagram showing an example of a tone map.</figref><figref num="3B">FIG. 3B is a diagram showing an example of a tone map.</figref><figref num="4A">FIG. 4A is a diagram showing an example of a static tone map.</figref><figref num="4B">FIG. 4B is a diagram showing an example of a dynamic tone map.</figref><figref num="5A">FIG. 5A is a diagram showing an example of EOTF (Electro-Optical Transfer Function) corresponding to HDR and SDR respectively.</figref><figref num="5B">FIG. 5B is a diagram showing an example of inverse EOTF corresponding to HDR and SDR respectively.</figref><figref num="6">FIG. 6 is a diagram showing the relationship between the brightness of the input image and the brightness output to the actual display.</figref><figref num="7">FIG. 7 is a diagram showing an example of dynamic metadata.</figref><figref num="8">FIG. 8 is a diagram for explaining the 99Y and DY100 calculation methods.</figref><figref num="9">FIG. 9 is a diagram for explaining a calculation method of 18G.</figref><figref num="10">FIG. 10 is a block diagram showing an example of the configuration of the video display device of the embodiment.</figref><figref num="11">FIG. 11 is a block diagram showing an example of the configuration of the HDR signal converter of the embodiment.</figref><figref num="12">FIG. 12 is a flowchart showing the operation of the video display device according to the present embodiment.</figref><figref num="13">FIG. 13 is a diagram for explaining a method of generating a conversion curve generated when the threshold value TH_A <luminance compression rate <1.</figref><figref num="14">FIG. 14 is a diagram for explaining a method of generating a conversion curve generated when DY100 <threshold value TH.</figref><figref num="15">FIG. 15 is a flowchart for explaining the tone map processing of the first example.</figref><figref num="16">FIG. 16 is a diagram for explaining a method of generating a conversion curve in the second example.</figref><figref num="17">FIG. 17 is a diagram for explaining a method of generating a conversion curve in the second example.</figref><figref num="18">FIG. 18 is a diagram for explaining a conversion curve generated in the tone map processing of the third example.</figref><figref num="19">FIG. 19 is a diagram showing an example of a conversion curve generated when both maxRGB Percentile [90] and maxRGB Percentile [98] have values closer to 18G than 99Y.</figref><figref num="20">FIG. 20 is a diagram showing an example of a conversion curve generated when both maxRGB Percentile [90] and maxRGB Percentile [98] have values closer to 99Y than 18G.</figref><figref num="21">FIG. 21 is a flowchart for explaining the tone map processing of the fifth example.</figref><figref num="22">FIG. 22 is a diagram for explaining a conversion curve generated in the tone map processing of the fifth example.</figref><figref num="23">FIG. 23 is a block diagram showing an example of the configuration of the generator of the embodiment.</figref><figref num="24">FIG. 24 is a block diagram showing an example of the configuration of the generation unit of the embodiment.</figref><figref num="25">FIG. 25 is a flowchart showing an example of the generation method.</figref><figref num="26">FIG. 26 is a flowchart showing a process of determining the luminance characteristics in the generation method.</figref>
[1-1. Background]
First, the transition of video technology will be described with reference to FIG. FIG. 1 is a diagram for explaining the evolution of video technology.
Until now, the focus has been on increasing the number of display pixels in order to improve the image quality of images, and from 720 x 480 pixels of Standard Definition (SD) to 1920 x 1080 pixels of High Definition (HD) have become widespread. There is.
In recent years, the introduction of so-called 4K video with Ultra High Definition (UHD) 3840 x 1920 pixels or 4K 4096 x 2048 pixels has been started with the aim of further improving image quality.
With the introduction of 4K, expansion of dynamic range, expansion of color gamut, addition or improvement of frame rate, etc. are being considered.
Among them, regarding the dynamic range, HDR (High) is used as a method for expressing bright light such as specular reflected light, which cannot be expressed by the current TV signal, with a brightness closer to reality while maintaining the dark gradation. Dynamic Range) is attracting attention. Specifically, the conventional television signal is called SDR (Standard Dynamic Range), and the maximum brightness is 100 nits. On the other hand, HDR is expected to expand the maximum brightness to 1000 nits or more. For HDR, standardization of mastering display standards is underway in SMPTE (Society of Motion Picture & Television Engineers) and ITU-R (International Telecommunications Union Radiocommunication Sector).
As with HD and UHD, HDR is expected to be applied to broadcasting, packaged media (Blu-ray (registered trademark) Disc, etc.), and Internet distribution.
[1-2. Relationship between master generation, distribution method, and display device]
FIG. 2 is a diagram for explaining the relationship between video production, distribution method, and display device when introducing a new video expression into the content.
When introducing a new video expression (increasing the number of pixels, etc.) to improve the image quality, it is necessary to (1) change the master for Home Entertainment on the video production side, as shown in Fig. 2. .. Accordingly, it is necessary to update (2) distribution methods such as broadcasting, communication, and package media, and (3) display devices such as televisions or projectors that display the images.
[1-3. Tone map]
Tone mapping is based on the relationship between the brightness of the HDR image and the maximum brightness of the image display device (Display Peak Luminance: DPL). When the maximum brightness of the image (Maximum Content Luminance Level: Max CLL) exceeds the DPL, This is a process that converts the brightness of the image and keeps the brightness of the image within the DPL. By this processing, the image can be displayed without losing the information near the maximum brightness of the image. Since this conversion depends on the characteristics of the video display device and the way of thinking of how to display it, a different conversion curve is used for each video display device.
3A and 3B are diagrams showing an example of a tone map. FIG. 3A shows the case where the DPL is 500 nits, and FIG. 3B shows the case where the DPL is 1000 nits. Further, FIGS. 3A and 3B are examples of a tone map when MaxCLL displays an image of 1000 nits and a tone map when MaxCLL displays an image of 4000 nits.
As shown in Fig. 3A, when the DPL is 500 nits, the brightness is converted so that both images can display up to MaxCLL at 500 nits or less, but the degree of conversion is greater for images with higher MaxCLL.
As shown in FIG. 3B, when the DPL is 1000 nits, the tone map is not performed in the video with the Max CLL of 1000 nits. For video with MaxCLL 4000 nits, a tone map is performed and the brightness of 4000 nits is converted to 1000 nits and displayed.
[1-4. Dynamic Metadata and Dynamic Tone Map]
FIG. 4A is a diagram showing an example of a tone map using static metadata. FIG. 4B is a diagram showing an example of a dynamic tone map using dynamic metadata.
As shown in Figure 4A, when static metadata (MaxCLL) is used, MaxCLL shows the highest brightness in a series of images, so the video display device only has a fixed tone map for the series of images. I can't. On the other hand, as shown in (a) of FIG. 4B, the video display device uses metadata (referred to as Dynamic Max CLL here) that matches the time-varying brightness, so that the tone is low when the brightness is low. It is possible to realize an optimum tone map according to the time-varying brightness, such as performing a tone map when the brightness is high ((c) in Fig. 4B) without performing a map ((b) in Fig. 4B). Dynamic metadata is a dynamic luminance characteristic that indicates a time-series change in the luminance characteristic of an image. The luminance characteristics of the image used for the dynamic metadata are, for example, the maximum luminance and the average luminance in a predetermined section of the image. In the present disclosure, the maximum luminance of an image will be described as an example of the luminance characteristic of the image. The predetermined section of the image is, for example, a scene, a chapter, a frame, or the like.
[1-5. About EOTF]
Here, EOTF will be described with reference to FIGS. 5A and 5B.
FIG. 5A is a diagram showing an example of EOTF (Electro-Optical Transfer Function) corresponding to HDR and SDR respectively.
EOTF is generally called a gamma curve, which shows the correspondence between a code value and a luminance value, and converts the code value into a luminance value. That is, the EOTF is the relational information indicating the correspondence between the plurality of code values and the luminance values.
In addition, FIG. 5B is a diagram showing an example of reverse EOTF corresponding to HDR and SDR, respectively.
The inverse EOTF indicates the correspondence between the luminance value and the code value, and contrary to the EOTF, the luminance value is quantized and converted into a code value. That is, the inverse EOTF is the relational information indicating the correspondence between the luminance value and the plurality of code values. For example, when the brightness value of an image corresponding to HDR is expressed by a code value of 10-bit gradation, the brightness value in the brightness range of HDR up to 10000 nit is quantized and 1024 sets from 0 to 1023 are arranged. Mapped to a number. That is, by quantization based on the inverse EOTF, the luminance value in the luminance range of 0 to 10000 nit (the luminance value of the image corresponding to HDR) is converted into the HDR signal which is a 10-bit code value. In HDR-compatible EOTF (hereinafter referred to as "HDR EOTF") or HDR-compatible reverse EOTF (hereinafter referred to as "HDR reverse EOTF"), SDR-compatible EOTF (hereinafter "SDR EOTF") It is possible to express a higher brightness value than the inverse EOTF corresponding to SDR (hereinafter referred to as "inverse EOTF of SDR"). For example, in FIGS. 5A and 5B, the maximum luminance (peak luminance) is 10000 nits. That is, the luminance range of HDR includes the entire luminance range of SDR, and the peak luminance of HDR is larger than the peak luminance of SDR. The brightness range of HDR is a brightness range in which the maximum value of 100 nits is expanded to 10000 nits as compared with the brightness range of SDR.
For example, HDR's EOTF and HDR's reverse EOTF are, for example, SMPTE 2084 standardized by the Society of Motion Picture and Television Engineers (SMPTE).
[1-6. Conventional technology]
In the prior art, metadata showing one maximum luminance information is shown for the entire content, and display processing is performed for one content using a tone map of one setting. Therefore, for example, in a dark scene where there is no high-luminance information in the scene, even if there is no high-luminance information in the scene, the conventional video display device can display up to high-luminance, and the maximum content indicated by the maximum-luminance information is maximum. Perform a tone map that matches the brightness to the maximum display brightness of the video display device.
However, this problem can be solved by providing the video display device with dynamic metadata (dynamic metadata) indicating brightness information or the like for each scene. That is, the video display device can perform the optimum tone map for each scene using the dynamic metadata, and can improve the brightness and the gradation.
[1-6-1. Issue 1]
As described above, the problems of the prior art can be solved by using the dynamic range of the brightness for each scene.
However, just by using the maximum luminance information as shown in Fig. 6, it is not possible to know how to specifically draw the conversion curve used for the tone map, and depending on the content, the luminance and gradation may be impaired. There is.
[1-6-2. Solution to Problem 1]
Therefore, in the present disclosure, dynamic metadata for optimizing the conversion curve used for the tone map is defined, and an algorithm for generating the conversion curve used for the tone map is shown based on the dynamic metadata. In this way, by optimizing the tone map using the dynamic metadata or the feature data of the video corresponding to the dynamic metadata obtained by analyzing the main video, the video display device can display the video. By weighting each brightness, it is possible to display an image that realizes an optimum tone map.
Details will be shown later as examples.
Note that FIG. 6 is a diagram showing the relationship between the brightness (Scene Luminance) of the input image and the brightness (Display Luminance) output to the actual display. That is, FIG. 6 shows the conversion curve used for the tone map.
[1-7. Dynamic metadata]
FIG. 7 is a diagram showing an example of dynamic metadata.
The dynamic metadata contains the information shown in FIG.
Specifically, the dynamic metadata is 99Y, 18G, maxRGB Percentile (1%), maxRGB Percentile (25%), maxRGB Percentile (50%) maxRGB Percentile (75%) maxRGB Percentile (90%), maxRGB Percentile ( Includes 95%), (99.98%), verrage maxRGB, knee_point_x, knee_point_y, Bezier_anchor (0-9), DY100. In particular, the dynamic metadata may include 99Y and DY100.
These dynamic metadata may be information indicating the brightness characteristics of each of the plurality of frames constituting the main video, or may be information indicating the brightness characteristics of each scene in a plurality of frames. When the information indicates the brightness characteristics of each scene, the dynamic metadata may be the maximum value or the average value of the brightness characteristics of a plurality of frames constituting each scene.
In the following, a case where the dynamic metadata is information indicating the brightness characteristics for each scene will be described as an example. Here, FIG. 8 is a diagram for explaining the 99Y and DY100 calculation methods. FIG. 9 is a diagram for explaining a calculation method of 18G. 8 and 9 are diagrams showing the luminance distribution of the pixels in one frame.
99Y<sub>F</sub>As shown in FIG. 8, for all the pixels in one frame, when the luminance histogram showing the relationship between the luminance and the number of pixels of the luminance is accumulated from the low luminance side, the maximum luminance value in the range not exceeding 99.99% is obtained. Shown. 99Y is 99Y of multiple frames that make up the scene.<sub>F</sub>Is set to the maximum value of. Note that 99Y is an integer from 0 to 4095 obtained by normalizing the luminance value (0 nit to 10000 nit) to 12 bits (0 to 4095).
18G is required under the following conditions.
For each of the multiple frames that make up one scene, for the brightness histogram of the frame, find the cumulative value histogram (50 nit cumulative histogram) with a width of 50 nits starting from the target brightness. In the 50 nit integrated histogram Find the point where the cumulative value is maximum (maximum cumulative value) and its brightness value. Increase the brightness from the above brightness value, and set the brightness value below 10% of the maximum cumulative value to 18G for the first time. From 0nit to 18G 18G is a valid value when the brightness distribution of is more than 80% of the total, and 18G is an invalid value when it is not.
That is, 18G is derived as the brightness value in Distribution_Cuttoff shown in FIG. 18G is the average of 18G of multiple frames that make up the scene. If 18G is equal to 0, it must not be included in the 18G calculation.
Distribution_Cuttoff is a threshold value obtained by multiplying Cutoff_Threshold and Distribution_Peak. Distribution_Peak is the number of pixels at the peak of the brightness distribution of the pixels in the frame. Cutoff_Threshold is, for example, 0.10. That is, Distribution_Cutoff is 10% of the number of pixels of Distribution_Peak.
maxRGB Percentile [k] is calculated by the following calculation. In addition, k is any one of 1, 25, 50, 75, 90, 95, and 99.98.
Normalize the value obtained by EOTF conversion of each pixel RGB of the screen Calculate the histogram of the maximum value (maxRGB) of the normalized RGB value for each pixel Accumulate and count from 0 Let maxRGB Percentile [k] be the value when the pixel area is k%.
averrage maxRGB is a value obtained by averaging the maxRGB value for each pixel of one frame on the entire screen.
knee_point_x and knee_point_y represent the range of the straight part of the conversion curve used in the tone map. That is, the conversion curve is a linear expression from (0,0) to (knee_point_x, knee_point_y).
Bezier_anchor (0-9) shows the Bezier coefficient for determining the conversion curve used for the tone map from the curve derived by the Bezier coefficient in the region above knee_point_x, y.
DY100 (Distribution Y100nit)<sub>F</sub>Indicates the ratio of the number of pixels of 100 nits or less to the total number of pixels in the brightness histogram of one frame. DY100<sub>F</sub>Is the number of pixels accumulated in the range of 0 to 100.23 [nit] of pixels in one frame in order to know the distribution of low-brightness pixels more accurately, instead of estimating from the "Distribution MaxRGB Percentile" value. is there. In addition, DY100<sub>F</sub>Is derived as a percentile of pixels of 100.23 nit (10bit / [0: 1023] and Y = 520) or less in the frame. DY100 is a DY100 with multiple frames that make up the scene.<sub>F</sub>Is set to the average value of. DY100 is an integer from 0 to 100.
The pixel brightness value (unit: nit or cd / m) for deriving both 99Y and DY100.<sup>2</sup>) Must be converted from Y by the method defined in SMPTE ST.2084. Also, Y'[0: 1], normalized Y, must be converted from the R'G'B' pixel value by the method defined in ITU-T BT.2020, as shown in Equation 1 below. Must be.
Y'= 0.2627R'+ 0.6780G' + 0.0593B'(Equation 1)
The dynamic metadata is included in the video data as content information, and is added to the main video for each scene and sent to the video display device from the video playback device that is playing back the video. However, the luminance characteristics of the main video corresponding to the dynamic metadata can also be obtained by analyzing the main video of the content. That is, the present disclosure also includes a tone map performed based on the luminance characteristics corresponding to the dynamic metadata obtained by analyzing the main image in the image display device. In the present disclosure, as the dynamic luminance characteristic indicating the time-series change of the luminance characteristic of the moving image, the luminance obtained by analyzing the dynamic metadata and the main image in frame units and in scene units composed of a plurality of frames. Examples such as characteristics.
(18G generation method) When determining the Knee Point of the tone map, information on the brightness distribution is required. In particular, since the kneepoint is set by obtaining the degree of concentration in the luminance histogram of the entire frame, an accurate value is required.
The percentile value is an integrated value from the small brightness of the pixels in one frame, and is discrete. Therefore, the percentile value indicates a determined area distribution of brightness. Therefore, the percentile value does not sufficiently represent the degree of concentration of brightness. For example, when trying to determine the brightness concentration using maxRGB Percentile [75], for example, up to 74% of the brightness is concentrated, and there is 75% of the brightness away from 74% of the brightness, and up to 75%. Cannot be determined from the case where the brightness is concentrated.
Therefore, by using 18G, it is possible to detect the continuity of the luminance distribution. In order to represent the luminance concentration, 18G has the luminance value (Distribution_Peak_count_Luminance), which is the maximum count, and its maximum. Detect the count number (Distribution_Peak). In addition, the brightness value is increased from Distribution_Peak_count_Luminance, the brightness with a count number less than 10% of Distribution_Peak is specified, and the specified brightness is set to 18G_mesure.
As the first problem of 18G generation, in the calculation from a simple histogram, the count number becomes small momentarily, and there is a risk that an incorrect value will be taken. On the other hand, in the 18G generation method, by constructing a luminance histogram with an integral value of 50 nit width, the sensitivity of the instantaneous change is reduced and the optimum value can be obtained.
In addition, as the second issue of 18G generation, even if the value obtained from the luminance histogram with the integrated value of 50 nit width is used, the peaks are highly concentrated, but the concentration is high in the case of an average luminance distribution for the entire frame. May be small. In this way, in order to exclude the low concentration, the condition that the integrated value (percentile value) of the luminance histogram up to 18G exceeds 80% is added to the obtained result. As a result, if the integrated value is less than 80%, it will not be 18G, so it is possible to exclude the case where the concentration ratio is small.
[1-8. Configuration]
Next, the configuration of the video display device 100 according to the present embodiment will be described.
FIG. 10 is a block diagram showing an example of the configuration of the video display device of the embodiment. FIG. 11 is a block diagram showing an example of the configuration of the HDR signal converter of the embodiment.
The video display device 100 includes a video receiving unit 110, a tone map processing unit 120, and a display unit 130.
The video receiving unit 110 receives video data including a main video which is a moving image and a dynamic luminance characteristic. That is, the video receiving unit 110 functions as an acquisition unit for acquiring video data. The video receiving unit 110 outputs the received video data to the tone map processing unit 120.
The tone map processing unit 120 is a processing unit that performs a tone map using a predetermined conversion curve. The tone map processing unit 120 includes an HDR signal converter 121 and a tone map generator 122.
The HDR signal converter 121 optimizes the brightness information of the main video, which is the HDR video, to the brightness of the display unit 130 and outputs the information.
Further, as shown in FIG. 11, the HDR signal converter 121 includes an input signal-luminance conversion circuit 123 that converts a code value indicating the brightness of the main image into brightness, and a code value that converts the brightness into the brightness of the display unit 130. It has a luminance-output level conversion circuit 124 to convert to. The brightness-output level conversion circuit 124 performs tone map processing using the conversion curve generated by the tone map generator 122, and optimally displays the image obtained by performing the tone map processing on the display unit 130. It can be displayed.
The tone map generator 122 optimizes the conversion curve used in the HDR signal converter 121 according to the brightness of the display unit 130. The tone map generation device 122 acquires the dynamic luminance characteristic of the video data from the video receiving unit 110, and performs an optimum conversion curve generation process for use in the tone map according to the dynamic luminance characteristic. A specific method of the conversion curve generation process will be described separately in Examples 1 to 5.
The display unit 130 displays the image after the tone map processing.
[1-9. Operation]
FIG. 12 is a flowchart showing the operation of the video display device according to the present embodiment.
In the video display device 100, first, the video receiving unit 110 acquires video data including a main video that is a moving image and a dynamic luminance characteristic that indicates a time-series change in the luminance characteristics of the type video (S1).
Next, the tone map processing unit 120 uses the dynamic brightness characteristic and the conversion curve optimal for the dynamic brightness characteristic and the maximum display brightness according to the maximum brightness of the display device, which is the maximum brightness of the display device, to use the main image. Perform tone map processing (S2).
Then, the display unit 130 displays the image after the tone map processing (S3).
[1-10. Tone map processing]
Next, an example of tone map processing will be described.
[1-10-1. First example of tone map processing]
The tone map processing in the first example will be described.
In the first example, an example of generating a tone map using DY100 and 99Y as dynamic metadata will be described.
First, the tone map generator 122 specifies the luminance dynamic range of the main image using 99Y, and determines whether (1) the luminance compression is performed or (2) the luminance compression is not performed for the specified luminance dynamic range. .. The luminance compression is a process of reducing the luminance dynamic range of the image displayed on the display unit 130 by reducing the luminance of the main image. Luminance compression means that, for example, when the maximum brightness of the main image exceeds the maximum display brightness, the maximum brightness of the main image cannot be displayed on the display unit 130, so that the maximum brightness of the main image becomes the maximum display brightness. This is a process of reducing the luminance dynamic range. Further, when the tone map generator 122 performs luminance compression, the luminance dynamic range of the main image is further subjected to (1-1) luminance compression in the high luminance region or (1-2) luminance compression in the entire range. You may decide.
Further, the tone map generator 122 controls the brightness compression ratio of the dark part according to DY100 in order to maintain the gradation and visibility of the dark part when compressing the entire range (1-2) described above. ..
For example, the tone map generator 122 calculates the brightness compressibility from the dynamic metadata 99Y and the display brightness (DPB: Display Peak Brightness) indicating the maximum display brightness by using the following equation 2.
Luminance compression rate = DPB / 99Y (Equation 2)
The tone map generator 122 generates different conversion curves according to the calculated luminance compression ratio. The tone map generator 122 performs a process of generating different conversion curves according to the luminance compression rate, as shown in the following (1) to (3), for example.
(1) When the luminance compression ratio is 1 or more In this case, the tone map generator 122 does not generate a conversion curve because tone map processing is not required. That is, in this case, the tone map generator 122 determines that the luminance compression is not performed.
Therefore, the HDR signal converter 121 outputs the luminance in the range up to 99Y of the moving image without converting it as it is. Therefore, Scene Luminance and Display Luminance are equal.
(2) When the threshold value TH_A <luminance compression rate <1 The threshold value TH_A is a value obtained by multiplying DPB by a predetermined coefficient (for example, a number greater than 0.5 and less than 1). As the predetermined coefficient, a value optimized based on experience is used. In this case, since the luminance compression ratio is close to 1 and the luminance compression is small, a conversion curve for luminance compression is generated in the high luminance region. That is, in this case, the tone map generator 122 determines that the luminance is compressed in the high luminance region.
Specifically, as shown in "3" of FIG. 13, the tone map generator 122 inputs the output luminance (that is, the display luminance) to the input luminance (that is, the scene luminance) in the low luminance region (i) below the knee point. Generates a conversion curve that has a PQ curve that outputs with the same brightness as (ii) and has a curve that compresses the brightness of 99Y in accordance with DPB in a high brightness region larger than (ii) kneepoint.
(3) Luminance compression rate <threshold TH_A In this case, since the luminance compression rate becomes large, the tone map generator 122 generates a conversion curve that maintains the balance of the entire image by performing luminance compression over the entire range. That is, in this case, the tone map generator 122 determines that the entire range is luminance-compressed.
Here, we pay particular attention to the low-luminance region of 100 nits or less when compressing the entire range. In the PQ method expressed by absolute brightness, if brightness compression is performed even in a low brightness region, the compression rate becomes large in many cases. Therefore, most of the low-luminance region is output with a small value, which may impair the details of the low-luminance region. To avoid this, the tone map generator 122 uses the parameters of the DY100 to generate a conversion curve for performing individual processing on signals of 100 nits or less. DY100 represents the screen occupancy of pixels included in the low-luminance region of 100 nits or less. Therefore, the tone map generator 122 generates a conversion curve that controls the compression rate for the luminance of 100 nits or less when the luminance is compressed in the entire range.
That is, the tone map generator 122 specifically generates different conversion curves depending on whether or not DY100 exceeds the threshold value TH. For the threshold value TH, a threshold value optimized based on experience is used.
(3-1) When DY100> Threshold TH In this case, the tone map generator 122 has important details below 100 nit because the ratio of pixels having a brightness of 100 nit or less to the total number of pixels is larger than the threshold TH. Is determined, and a conversion curve that does not compress the brightness is generated in the brightness region of 100 nits or less. That is, when the tone map generator 122 has a luminance region below the first luminance (for example, KneePoint) as a low-luminance region and a luminance region above the KneePoint as a high-luminance region, among (i) dynamic luminance characteristics, When DY100 as the first brightness characteristic, which indicates the number of pixels of 100 nit or less as the second brightness among the pixels included in the low brightness region in one frame of the moving image, exceeds the threshold TH, the brightness in the range of 100 nit or less is changed. Does not generate the first transformation curve. The tone map generator 122 generates, for example, the conversion curve shown in FIG. 17 (c), which will be described later.
(3-2) When DY100 <threshold TH In this case, the tone map generator 122 considers that the influence of luminance compression is small because the ratio of the pixels having the luminance of 100 nit or less to the total number of pixels is the threshold TH or less. Judgment is made, and a conversion curve that compresses the brightness even in the low brightness region is generated. That is, when the DY100 is equal to or less than the threshold value TH, the tone map generator 122 generates a second conversion curve that makes the luminance in the range of 100 nit or less smaller than the luminance. The tone map generator 122 generates, for example, the conversion curve shown in FIG. 17 (b), which will be described later.
In this case, the luminance compression rate may be changed according to the value of DY100, and the maximum compression rate (for example, 0.8) may be specified and protected. That is, the tone map generator 122 may generate a conversion curve having a slope of less than 1 in the range of 100 nits or less as the second conversion curve. Further, the tone map generator 122 may generate a conversion curve as the second conversion curve, in which the smaller the value indicated by DY100, the smaller the ratio of the brightness in the range of 100 nits or less to the brightness.
FIG. 14 is a diagram for explaining a method of generating a conversion curve generated when DY100 <threshold value TH. FIG. 14 shows a graph showing the relationship between the display brightness (output brightness) and the scene brightness (input brightness), that is, an example of the second conversion curve.
As shown in FIG. 14, the second conversion curve is a curve in which the luminance is compressed at a compression rate of 0.8 to 1.0 of the PQ curve in the range up to 100 nits in the low luminance region. When DY100 <threshold TH, the compression rate is less than 1.0. KneePoint is 100 nit. Then, the second conversion curve is an inverse gamma-based curve A × Input up to 99Y in the high brightness region.<sup>(1 / 2.2) * B</sup>It becomes the curve of. Finally, the second conversion curve is a curve obtained by smoothly connecting the curve in the low-luminance region and the curve in the high-luminance region described above by a curve having a continuous slope.
Next, the tone map processing of the first example will be described using a flowchart.
FIG. 15 is a flowchart for explaining the tone map processing of the first example.
The tone map processing is performed by the tone map processing unit 120.
In the first example, when the tone map processing of step S2 in the flowchart of FIG. 12 described above is started, the tone map generator 122 acquires the dynamic metadata of the video data (S11).
Then, the tone map generator 122 calculates the luminance compression rate using 99Y of the dynamic metadata and the DPB of the video display device 100, and determines whether or not the calculated luminance compression rate exceeds 1. Judge (S12).
When the tone map generator 122 determines that the luminance compression rate exceeds 1 (Yes in S12), it determines that there is no luminance compression, and outputs a conversion curve without luminance compression to the HDR signal converter 121. Then, the HDR signal converter 121 performs tone map processing A using a conversion curve that does not perform luminance compression, and outputs the video signal obtained by tone map processing A to the display unit 130 (S13).
When the tone map generator 122 determines that the luminance compression rate is 1 or less (No in S12), the tone map generator 122 determines whether or not the luminance compression rate exceeds the threshold value TH_A (S14).
When the tone map generator 122 determines that the luminance compression rate exceeds the threshold value TH_A (Yes in S14), it determines that the luminance compression is performed in the high luminance region, and the conversion curve shown in FIG. 13 is converted into the HDR signal converter. Output to 121. Then, the HDR signal converter 121 performs tone map processing B using the conversion curve shown in FIG. 13, and outputs the video signal obtained by tone map processing B to the display unit 130 (S15).
When the tone map generator 122 determines that the luminance compression rate is equal to or less than the threshold value TH_A (No in S14), the tone map generator 122 determines whether or not DY100 of the acquired dynamic metadata exceeds the threshold value TH (S16). ..
When the tone map generator 122 determines that the DY100 exceeds the threshold value TH (Yes in S16), the tone map generator 122 outputs a conversion curve that does not compress the luminance in the luminance region of 100 nit or less to the HDR signal converter 121. Then, the HDR signal converter 121 performs tone map processing C using the output conversion curve, and outputs the video signal obtained by tone map processing C to the display unit 130 (S17).
When the tone map generator 122 determines that the DY100 is equal to or less than the threshold value TH (No in S16), the HDR signal converter 121 sets a conversion curve that makes the brightness in the range of 100 nit or less smaller than the brightness shown in FIG. Output to. Then, the HDR signal converter 121 performs tone map processing D using the output conversion curve, and outputs the video signal obtained by tone map processing D to the display unit 130 (S18).
[1-10-2. Second example of tone map processing]
The tone map processing in the second example will be described. The second example describes how to generate a transformation curve different from the first example using DY100 and 99Y.
In this example, the tone map generator 122 finds the maximum value of the input signal by 99Y and determines the slope at 100 nit points by DY100.
FIG. 16 is a diagram for explaining a method of generating a conversion curve in the second example.
In the second example, the tone map generator 122 generates a conversion curve composed of the origin coordinates, 100nit coordinates (Ks), and maximum value coordinates (99Y, DPB: Display Peak Brightness) shown in FIG. The tone map generator 122 uses DY100 to calculate the output luminance (F1s (100)) of the input luminance at 100 nits, and determines the slope of the conversion curve at 100 nits or less and the slope of the conversion curve at 100 nit coordinates Ks, for example. , 1.0, etc.
Next, the tone map generator 122 uses 99Y to determine the slope of the conversion curve in 99Y, and generates a tone map by connecting the three points of the origin coordinate, the 100nit coordinate Ks, and the maximum value coordinate with a spline curve.
Specifically, as shown in FIG. 17 (b), the tone map generator 122 tilts up to 100 nit coordinates by reducing the value of F1s (100) when DY100 is equal to or less than a predetermined threshold value. A smaller conversion curve may be generated. Further, in this case, the tone map generator 122 generates a conversion curve in which the slope in the range of 100 nit or more is larger than the slope in the range of less than 100 nit by increasing the slope in the 100 nit coordinate Ks. As a result, it is possible to secure the tonality in the range of 100 nits or more.
Further, the tone map generator 122 may generate a conversion curve in which the slope is weighted using the percentile information at that time. The tone map generator 122 has, for example, a gradation in a high luminance region (near 99Y) when maxRGB Percentile [75] becomes a value close to 99Y (for example, a value within a predetermined luminance range based on 99Y). It can be determined that there are many components. Therefore, the tone map generator 122 may generate a conversion curve that increases the slope of 99Y from maxRGB Percentile [75]. This makes it possible to improve the gradation between maxRGB Percentile [75] and 99Y.
In FIG. 16, Ks (100, F1 (100)) indicates the knee point of the scene determined by evaluating DY100. In addition, F1s (Vi) indicates a linear function for a low-level signal (low-luminance region) of a scene, such as a brightness range of 0 to 100 nit of a video input signal. F2s (Vi) indicates a spline curve function for medium and high level signals (medium and high brightness region) in the scene, such as the 100nit to 99Y brightness range of the video input signal.
[1-10-3. Third example of tone map processing]
The tone map processing in the third example will be described. In the third example, the tone map processing in the high luminance region will be described.
FIG. 18 is a diagram for explaining a conversion curve generated in the tone map processing of the third example.
In this case, the tone map generator 122 uses the dynamic metadata 18G and 99Y to generate a conversion curve that expresses the luminance range from 18G to 99Y from 18G to the maximum luminance (Max_Luminance) of the video display device 100. .. Therefore, the tone map generator 122 uses at least one of the values of maxRGB Percentile [90] and maxRGB Percentile [98] to obtain the gradation in the high brightness region as shown in FIG. 18 (b). Generate a conversion curve with increased weight. That is, the tone map generator 122 generates a conversion curve with a large inclination in the high-luminance region.
FIG. 19 is a diagram showing an example of a conversion curve generated when both maxRGB Percentile [90] and maxRGB Percentile [98] have values closer to 18G than 99Y. FIG. 20 is a diagram showing an example of a conversion curve generated when both maxRGB Percentile [90] and maxRGB Percentile [98] have values closer to 99Y than 18G.
For example, the tone map generator 122 sets the upper limit of the kneePoint setting range, for example, maxRGB Percentile [90] when both maxRGB Percentile [90] and maxRGB Percentile [98] are closer to 18G than 99Y. 98]. As a result, the tone map generator 122 increased the tonality weight from 18G to maxRGB Percentile [98], that is, the slope is close to 1 (for example, the slope is greater than 0.8), as shown in FIG. Generate a conversion curve.
On the contrary, the tone map generator 122 sets knee_high_point to the vicinity of Max_Luminance when maxRGB Percentile [90] and maxRGB Percentile [98] are closer to 99Y than 18G. As a result, the tone map generator 122 increases the tonality weight from knee_high_point to 99Y, that is, generates a conversion curve whose slope is close to 1 (for example, the slope is greater than 0.8), as shown in FIG. To do.
In addition to the examples of FIGS. 19 and 20, it is possible to generate a tone map weighted by these values. The tone map generator 122 may generate a transformation curve using dynamic metadata such as knee_point_x and knee_point_y, bezier_anchor.
The tone map processing of the third example is performed when (2) threshold value TH_A <luminance compression rate <1 in the first example, or (3) brightness compression rate <threshold value TH_A (3-1) DY100>. This may be done when the threshold value is TH.
As described above, in the tone map processing B or tone map processing C in FIG. 15, the tone map generator 122 has (i) the cumulative value from 0 of 90% of all pixels in the maxRGB histogram of each pixel in one frame. The third brightness (that is, at least one of maxRGB Percentile [90] and maxRGB Percentile [98]) when it becomes at least one of 90% and 98% as the first ratio is the maximum brightness in the one frame. When the value is closer to 18G as the second luminance than (that is, 99Y), a conversion curve in which the slope from the second luminance to the third luminance is larger than the slope in the range exceeding the third luminance is generated.
Further, in the tone map processing B or tone map processing C in FIG. 15, the tone map generator 122 has (ii) a third brightness to a third brightness when the third brightness is closer to the maximum brightness than the second brightness. Generates a conversion curve in which the slope to brightness is smaller than the slope in the range above the third brightness.
As a result, when at least one of maxRGB Percentile [90] and maxRGB Percentile [98] has a value closer to 18G than 99Y, the tone map processing unit 120 concentrates at least 90% of all pixels on the brightness close to 18G. It can be said that it is doing. Therefore, by performing tone map processing using a conversion curve in which the slope from 18G to maxRGB Percentile [98] is larger than the slope exceeding maxRGB Percentile [98], the gradation from 18G to maxRGB Percentile [98] is performed. You can increase the weight of sex. Therefore, it is possible to improve the gradation of the pixels having the brightness concentrated in one frame, and it is possible to improve the quality of the image.
Further, in the tone map processing unit 120, when at least one of maxRGB Percentile [90] and maxRGB Percentile [98] has a value closer to 99Y than 18G, the remaining 10% or 2% of pixels are as high as 99Y. It can be said that it is concentrated in the brightness region. Therefore, by performing tone map processing using a conversion curve in which the slope from maxRGB Percentile [90] to 99Y is larger than the slope from 18G to maxRGB Percentile [90], from maxRGB Percentile [90] to 99Y. The weight of gradation can be increased. Therefore, it is possible to improve the gradation of the pixels having the brightness concentrated in one frame, and it is possible to improve the quality of the image.
In the third example, 18G is exemplified as the second luminance, but the second luminance may be 100 nits.
[1-10-4. Fourth example of tone map processing]
Put the fourth example Tone map processing that will be described. In the fourth example, the tone map processing in the low-luminance region will be described.
In the first example, when the luminance compression is performed, when the DY100 is equal to or less than the threshold value TH_A, the luminance compression is performed in the low luminance region, but the present invention is not limited to this. For example, it may be determined whether or not to perform luminance compression in the low luminance region according to the value of maxRGB Percentile [1].
Specifically, the tone map generator 122 has a large luminance distribution in the low luminance region (dark area) depending on whether the maxRGB Percentile [1] of the dynamic metadata has a luminance higher than a predetermined luminance. Determine if it is low. Then, when the tone map generator 122 determines that the luminance distribution in the dark portion is small, for example, as shown in FIG. 17 (b) described above, the tone map generator 122 compresses the luminance dynamic range by tone-mapping the dark portion side, and compresses the entire luminance dynamic range. You may generate a transformation curve that extends the dynamic range of. For example, if maxRGB Percentile [1] is 200 nits, the area occupied by pixels with a brightness of 200 nits or less is 1% or less of the total area of the screen. Therefore, even if the luminance information of 200 nits or less is compressed, the effect as a whole is small. At that time, by setting the inclination of the region of 200 nits or more to 1, it is possible to output a video signal that maintains the gradation of the input signal of 200 nits or more.
[1-10-5. Fifth example of tone map processing]
The tone map processing in the fifth example will be described. In the fifth example, a method of determining the Knee Point will be described.
The tone map generator 122 uses 99Y and 18G to determine the KneePoint of the conversion curve (the point at which content brightness begins to be compressed). Thereby, the contrast in the specific luminance region can be appropriately improved.
For example, when 99Y is higher than the DPB of the video display device 100, the tone map generator 122 uses 18G to determine the KneePoint in the conversion curve with the convergence point as 99Y. The tone map generator 122 sets the following values for 99Y.
The tone map generator 122 sets the upper limit value at which the conversion curve can visually recognize all gradations as kneepoint_max. Further, the tone map generator 122 sets a value at which the conversion curve can be visually recognized evenly in all gradations as kneepoint_min. Here, kneepoint_max is the upper limit value of the luminance range for determining KneePoint, and kneepoint_min is the lower limit value of the luminance range for determining KneePoint.
Further, in the tone map generator 122, 80% or more of the luminance distribution is concentrated below the value indicated by 18G, so the kneepoint is determined using 18G.
FIG. 21 is a flowchart for explaining the tone map processing of the fifth example. FIG. 22 is a diagram for explaining a conversion curve generated in the tone map processing of the fifth example.
In the fifth example, when the tone map processing of step S2 in the flowchart of FIG. 12 described above is started, the tone map generator 122 acquires the dynamic metadata of the video data (S21).
Then, the tone map generator 122 determines whether or not 99Y of the dynamic metadata is equal to or less than the DPB of the video display device 100 (S22).
When the tone map generator 122 determines that 99Y is less than or equal to DPB (Yes in S22), the tone map generator 122 sets Knee_end, which is the convergence point on the high-luminance side of the conversion curve, to DPB (S23).
When the tone map generator 122 determines that 99Y exceeds DPB (No in S22), Knee_end is set to 99Y (S24).
The tone map generator 122 then determines whether 18G is less than KneePoint_max (S25).
When the tone map generator 122 determines that 18G is equal to or greater than KneePoint_max (No in S25), KneePoint is set to KneePoint_max (S26).
When the tone map generator 122 determines that 18G is less than KneePoint_max (Yes in S25), it determines whether 18G is less than KneePoint_min (S27).
When the tone map generator 122 determines that 18G is less than KneePoint_min (Yes in S27), it sets KneePoint to KneePoint_min (S28).
When the tone map generator 122 determines that 18G is equal to or higher than KneePoint (No in S27), KneePoint is set to 18G.
When steps S23, S26, S28, and S29 are completed, the tone map generator 122 ends the processing here.
The tone map generator 122 may generate the conversion curves of the first to fourth examples above by using the KneePoint set in this process.
[2. How to generate dynamic metadata]
Next, a method of generating dynamic metadata will be described.
In the following, we will disclose a method for generating metadata necessary to solve conventional problems by using a dynamic tone map.
[2-1. Configuration of generator]
The configuration of the generator that generates dynamic metadata will be described.
FIG. 23 is a block diagram showing an example of the configuration of the generator of the embodiment. FIG. 24 is a block diagram showing an example of the configuration of the generation unit of the embodiment.
The generation device 200 includes a video receiving unit 210, a generating unit 220, and a memory 230.
The video receiving unit 210 receives the main video which is a moving image. That is, the video receiving unit 210 functions as an acquisition unit for acquiring the main video. The video receiving unit 210 outputs the received main video to the generating unit 220.
The generation unit 220 analyzes the main video output by the video reception unit 210 to generate dynamic metadata showing the brightness characteristics for each scene. Specifically, the generation unit 220 generates dynamic metadata for each frame, and temporarily stores the dynamic metadata for one scene composed of a plurality of frames in the memory 230. Then, the generation unit 220 generates the dynamic metadata for each scene by taking the average or the maximum value using the dynamic metadata for one scene. The generation unit 220 may output the metadata generated for each frame.
Further, the generation unit 220 includes a video information luminance conversion unit 221, a luminance histogram generation unit 222, and a determination unit 223.
The video information luminance conversion unit 221 converts the video signal input as an RGB value into a luminance signal.
The luminance histogram generation unit 222 generates a luminance histogram from the signal information obtained by the luminance conversion in the video information luminance conversion unit 221.
The determination unit 223 determines the dynamic metadata for each frame using the luminance histogram generated by the luminance histogram generation unit 222. Further, the determination unit 223 performs a process of merging similar video information in time using the dynamic data of a plurality of frames for one scene temporarily stored in the memory 230. Here, merging is a process of taking the maximum value in the scene (similar frame) in 99Y, and a process of taking the average value in the scene (similar frame) in 100DY.
The memory 230 temporarily stores the dynamic metadata for each frame generated by the generation unit 220.
[2-2. Operation of generator]
Next, a method of generating dynamic metadata by the generator will be described.
FIG. 25 is a flowchart showing an example of the generation method.
First, in the generator 200, the video receiving unit 210 acquires the main video (S31).
Next, the generation unit 220 starts a loop that repeats steps S32 and S33 for each of the plurality of frames constituting the main video acquired by the video reception unit 210.
The generation unit 220 determines the luminance characteristics of the frame to be processed (S32). The details of the luminance characteristic determination process will be described with reference to FIG.
FIG. 26 is a flowchart showing a process of determining the luminance characteristics in the generation method.
The generation unit 220 generates a luminance histogram by analyzing the luminance of all the pixels of the frame to be processed (S41).
Next, the generation unit 220 starts counting the integrated values in order from the lowest brightness in the brightness histogram (S42). Specifically, the generation unit 220 counts the pixels having the set brightness while increasing the brightness histogram by 1 nit in order from 0 nit.
Then, the generation unit 220 determines whether or not the luminance value to be counted is 100 nits (S43).
When the generation unit 220 determines that the brightness to be counted is 100 nits (Yes in S43), the generation unit 220 determines DY100 as the value obtained by dividing the integrated value counted up to now by the total number of pixels (S44). That is, for each of the plurality of frames constituting the moving image, the generation unit 220 sets the number of pixels of 100 nits or less as the predetermined brightness among the plurality of pixels constituting the frame to the number of all pixels constituting the frame. The value obtained by dividing by is determined as DY100 as the first luminance characteristic.
After step S44, or when the generator 220 determines that the brightness to be counted is not 100 nits (No in S43), whether or not the value obtained by dividing the current integrated value by the total number of pixels exceeds 99.99%. Is determined (S45).
When the generation unit 220 determines that the value obtained by dividing the current integrated value by the total number of pixels exceeds 99.99% (Yes in S45), the generation unit 220 determines the brightness of the current count target to 99Y (S46). That is, here, the generation unit 2220 determines the maximum brightness, which is the brightness of 99.99% of all the pixels when all the pixels in the frame are arranged from the pixel having the lowest brightness for each of the plurality of frames of the moving image. It is specified, and the specified maximum brightness is determined as 99Y as the second brightness characteristic.
Whether or not 100DY and 99Y have been determined after step S46 or when the generation unit 220 determines that the value obtained by dividing the current integrated value by the total number of pixels is 99.99% or less (No in S45). Is determined (S47).
When it is determined that 100DY and 99Y have been determined (Yes in S47), the generation unit 220 ends the process of determining the luminance characteristics.
When the generation unit 220 determines that 100DY or 99Y has not been determined (No in S47), the generation unit 220 increases the brightness to be counted by 1 nit and returns to step S43.
Returning to FIG. 25, when the generation unit 220 finishes the process of determining the luminance characteristic, the generator 220 outputs the determined luminance characteristic to the memory 230 (S33). Then, when the luminance characteristics for one scene are accumulated in the memory 230, the generation unit 220 performs merging using the plurality of luminance characteristics for one scene, outputs the dynamic metadata after merging, and performs processing. finish.
The generator 200 records the output dynamic metadata in the SEI information of the content. That is, the generation device 200 may record the dynamic metadata together with the moving image on a recording medium such as an HDD, SSD, or BD.
As the dynamic metadata generated by the generation device 200, an example of generating DY100 and 99Y among the dynamic metadata is described, but other dynamic metadata may be generated in the same manner.
In this way, the generation device 200 can generate dynamic metadata by analyzing the moving image. Therefore, since the video display device can acquire dynamic metadata indicating the dynamic luminance characteristics of the moving image in addition to the moving image, the tone map processing is executed according to the luminance characteristics of the moving image indicated by the dynamic metadata. be able to. That is, the video display device can execute the dynamic tone map without analyzing the moving image, and the processing load can be reduced. Further, since the video display device can reduce the processing time related to the analysis of the moving image, the dynamic tone map can be effectively executed on the moving image.
[3. Modification example]
The main image is, for example, an HDR image. The HDR video may be, for example, a Blu-ray disc, a DVD, a video distribution site on the Internet, a broadcast, or a video in an HDD (Hard Disk Drive).
The video playback device may be a device such as a disc player, a disc recorder, a set-top box, a television, a personal computer, or a smartphone that decodes a compressed video signal from a recording medium, broadcast, or the Internet and sends it to a video display device. Further, a part or all of the functions of the video reproduction device may be included in the video display device 100.
The video signal transmission means for transmitting the video signal from the video playback device to the video display device may be a video signal such as HDMI (registered trademark), DVI, or DP that is transmitted in an uncompressed state, or may be transmitted through a network. It may be a means for transmitting a video signal in a compressed format.
The maximum brightness information or tone map information of the video display device may be set in the video playback device by the user inputting the maximum brightness information or the tone map information to the video playback device using a remote controller or the like, or the user may play back the video. It may be performed by inputting using the operating device provided in the device. Alternatively, the user acquires such information using the Internet or other means, stores the acquired information in a portable storage medium, and sends the information to a video playback device via the portable storage medium. May be good. Further, the video playback device may be directly connected to the Internet, and the video playback device may acquire such information from the database of the server. Further, the video reproduction device may display a test pattern on the video display device and acquire and store such information while confirming the characteristics of the video display device using the displayed test pattern.
Although the video display method and the luminance characteristic generation method according to the embodiment of the present disclosure have been described above, the present disclosure is not limited to this embodiment.
Further, each processing unit included in the video display device and the generation device according to the above embodiment is typically realized as an LSI which is an integrated circuit. These may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them.
Further, the integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of circuit cells inside the LSI may be used.
Further, in each of the above-described embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.
Further, the present disclosure may be realized as various methods executed by a video display device and a generation device.
Further, the division of the functional block in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be transferred to other functional blocks. You may. Further, the functions of a plurality of functional blocks having similar functions may be processed by a single hardware or software in parallel or in a time division manner.
Further, the order in which each step in the flowchart is executed is for exemplifying the present disclosure in detail, and may be an order other than the above. Further, a part of the above steps may be executed at the same time (parallel) as other steps.
The video display device and the generation device according to one or more aspects have been described above based on the embodiment, but the present disclosure is not limited to this embodiment. As long as the gist of the present disclosure is not deviated, various modifications that can be conceived by those skilled in the art are applied to the present embodiment, and a form constructed by combining components in different embodiments is also within the scope of one or more embodiments. May be included within.
The present disclosure can be applied to a method for generating luminance characteristics.
100 Video display device 110 Video receiver 120 Tone map processing unit 121 HDR signal converter 122 Tone map generator 123 Input signal-luminance conversion circuit 124 Brightness-output level conversion circuit 130 Display unit 200 Generator 210 Video receiver 220 Generation unit 221 Video information Luminance conversion unit 222 Luminance histogram generation unit 223 Determination unit 230 Memory
29 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US11455708B2 | Cited by | United States of America | Search report |
| JP2000035774A | Cites | Japan | – |
| WO2016027423A1 | Cites | World Intellectual Property Organization (WIPO) | – |
| US20160360171A1 | Cites | United States of America | – |
| WO2017089146A1 | Cites | World Intellectual Property Organization (WIPO) | – |
| JP2003339057A | Cites | Japan | – |
19 members in 4 offices
Priority claims14
| Document | Office | Kind | Date |
|---|---|---|---|
| 201762558092 | United States of America | P | |
| 201762558092 | United States of America | P | |
| 62558092 | United States of America | – | |
| 201762580918 | United States of America | P | |
| 201762580918 | United States of America | P | |
| 62580918 | United States of America | – | |
| 2018006865 | Japan | W | |
| 2018006865 | Japan | W | |
| 62558092 | – | – | – |
| 62580918 | – | – | – |
| JP2018006865 | – | – | – |
| US201762558092P | – | – | – |
| US201762580918P | – | – | – |
| WO2018JP06865 | – | – | – |
Members19
| Document | Office | Kind | |
|---|---|---|---|
| WO2019053916A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2019053917A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2019335149A1 | United States of America | A1 | |
| JPWO2019053916A1 | Japan | A1 | |
| JPWO2019053917A1 | Japan | A1 | |
| US2019342603A1 | United States of America | A1 | |
| US10659745B2 | United States of America | B2 | |
| JP6697755B2 | Japan | B2 | |
| JP6709986B2This record | Japan | B2 | |
| EP3684061A1 | European Patent Office (EPO) | A1 | |
| EP3684061A4 | European Patent Office (EPO) | A4 | |
| EP3684063A1 | European Patent Office (EPO) | A1 | |
| EP3684063A4 | European Patent Office (EPO) | A4 | |
| US2020236334A1 | United States of America | A1 | |
| US10742945B2 | United States of America | B2 | |
| EP3684061B1 | European Patent Office (EPO) | B1 | |
| US11228747B2 | United States of America | B2 | |
| EP3684063B1 | European Patent Office (EPO) | B1 | |
| EP4220541A1 | European Patent Office (EPO) | A1 |
6 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Written notification of patent or utility model registrationJAPANESE INTERMEDIATE CODE: R151R151 | R151 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 6709986
- Publication, DOCDB
- 6709986
- Publication, EPODOC
- JP6709986B
- Application
- 2019510373
- Application, DOCDB
- 2019510373
- Application, EPODOC
- JP20190510373
Titles2
- Japanese
- 輝度特性生成方法
- English
- Luminance characteristic generation method
Classification
- CPC, 9
- H04N9/8205
- H04N9/73
- H04N5/20
- G11B27/00
- G06T2207/10016
- G06T5/92
- H04N5/66
- H04N5/92
- H04N21/431
- IPC, 4
- H04N5 91
- H04N5 92
- H04N5 926
- G11B27 00
