Coder
Summary by NHIP
Context-based arithmetic coder
The coder converts multivalued input data into a binary sequence and calculates prediction code amounts using grouped contexts. A selector divides the sequence into groups, and multiple circuits compute amounts based on at least a section range in arithmetic coding before an adder sums them.
Claim Score by NHIP
Abstract
A coder has a binarizing circuit (130) for converting multivalued data into a binary symbol sequence, the multivalued data being generated from an input signal and having a plurality of contexts, an arithmetic code amount approximating circuit (200) for calculating a prediction code amount in the predetermined coding unit from the binary symbol sequence, and a coding circuit (102) for coding the input signal arithmetically on the basis of the prediction code amount. The arithmetic code amount approximating circuit (200) includes a selector (230) for dividing the binary symbol sequence into a plurality of groups based on the contexts, a plurality of code amount approximating circuits (211-214) for calculating, from the binary symbol sequence divided into a plurality of groups, the prediction code amount of the group based on at least the section range in arithmetic coding, and an adder (231) for adding the prediction code amounts from all code amount approximating circuits and outputting the prediction code amount in the specified coding unit.

Term
1.3 yearsleft in the term
Expires 10 January 2028, including 91 days of term adjustment.
- Priority
- Filed
- Granted
- Today
- Expires
10 claims: 1 independent, 9 dependent
- 1Broadest claimClaim Score 44, average(NHIP)A coder comprising:a blocking circuit that generates, from an input signal, multivalued data, in specified coding unit, having a plurality of contexts;a binarizing circuit that converts the multivalued data generated by the blocking circuit into a binary symbol sequence;an arithmetic code amount approximating circuit that calculates a prediction code amount in the coding unit from the binary symbol sequence;and a coding circuit that codes the input signal arithmetically on the basis of the prediction code amount, wherein the arithmetic code amount approximating circuit includes: a selector that divides the binary symbol sequence contained in the specified coding unit into a plurality of groups based on the contexts;a plurality of code amount approximating circuits, each of which is provided in each group and calculates, from the binary symbol sequence divided into a plurality of groups, the prediction code amount of the group, based on at least a section range in arithmetic coding;and an adder that adds the prediction code amounts from all code amount approximating circuits, and outputs the prediction code amount in the specified coding unit.
139 paragraphs in 7 sections, as filed
TECHNICAL FIELD
The present invention relates to a coder and coding method, and more particularly to a Coder and coding method applying arithmetic coding.
BACKGROUND ART
Arithmetic coding is a technology capable of reversibly compressing the quantity of information up to theoretical limits depending on the occurrence probability of information source symbols. Arithmetic coding is applied in the image coding fields in JPEG2000 standard (ISO/IEC15444), H.264/MPEG4-AVC standard (see, for example, non-patent document 1, hereinafter called H.264 standard), and others. For example, in H.264, Context Adaptive Binary Arithmetic Coding (CABAC) is applied, and coding of high efficiency is realized depending on the probability characteristic of syntax.
Context Adaptive Binary Arithmetic Coding in H.264 is explained by referring to <figref idrefs="DRAWINGS">FIG. 22</figref>. <figref idrefs="DRAWINGS">FIG. 22</figref> is a diagram explaining a context adaptive binary arithmetic coding circuit.
A context adaptive binary arithmetic coding circuit <b>145</b> in <figref idrefs="DRAWINGS">FIG. 22</figref> includes a binarizing circuit <b>130</b>, a context calculating circuit <b>520</b>, and an arithmetic coding circuit <b>140</b>. The operation of the circuits is explained below.
The binarizing circuit <b>130</b> binarizes multivalued input data composed of conversion coefficient data, flags and other coding information. Binarization is performed by judging the syntax element showing the type of the input data from the control information, and using a method such as unary binarization and fixed-length binarization depending on the data probability characteristic as shown in Table 1.
<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="4"><colspec colname="offset" colwidth="14pt" align="left" /><colspec colname="1" colwidth="56pt" align="left" /><colspec colname="2" colwidth="140pt" align="center" /><colspec colname="3" colwidth="7pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="3" rowsep="1">TABLE 1</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row><row><entry /><entry>Value of syntax</entry><entry /><entry /></row><row><entry /><entry>element</entry><entry>Bin string</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="8"><colspec colname="offset" colwidth="14pt" align="left" /><colspec colname="1" colwidth="56pt" align="left" /><colspec colname="2" colwidth="14pt" align="center" /><colspec colname="3" colwidth="35pt" align="center" /><colspec colname="4" colwidth="14pt" align="center" /><colspec colname="5" colwidth="35pt" align="center" /><colspec colname="6" colwidth="14pt" align="center" /><colspec colname="7" colwidth="35pt" align="center" /><tbody valign="top"><row><entry /><entry>0 (I N × N)</entry><entry>0</entry><entry /><entry /><entry /><entry /><entry /></row><row><entry /><entry>1</entry><entry>1</entry><entry>0</entry></row><row><entry /><entry>2</entry><entry>1</entry><entry>1</entry><entry>0</entry></row><row><entry /><entry>3</entry><entry>1</entry><entry>1</entry><entry>1</entry><entry>0</entry></row><row><entry /><entry>4</entry><entry>1</entry><entry>1</entry><entry>1</entry><entry>1</entry><entry>0</entry></row><row><entry /><entry>5</entry><entry>1</entry><entry>1</entry><entry>1</entry><entry>1</entry><entry>1</entry><entry>0</entry></row><row><entry /><entry>. . .</entry></row><row><entry /><entry>binIdx</entry><entry>0</entry><entry>1</entry><entry>2</entry><entry>3</entry><entry>4</entry><entry>5</entry></row><row><entry /><entry namest="offset" nameend="7" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
In the unary binarization shown in Table 1, binarization is performed by converting multivalued input data 0, 1, 2, 3, 4, 5, . . . into binary symbols (Bin string) 0, 10, 110, 1110, 11110, 111110, . . . . Binarized binary symbol sequences are input into the (binary) arithmetic coding circuit <b>140</b>. Unary binarization is used for binarizing values of equal to or smaller than 14 in coeff_abs_level_minus1 showing the level of conversion coefficient.
The context calculating circuit <b>520</b> determines the value of context index (ctxIdx) used in coding of present 1 bit in the binary symbol sequence on the basis of the control information showing the syntax element according to Table 2 specified in the H.264 standard.
<tables id="TABLE-US-00002" num="00002"><table frame="none" colsep="0" rowsep="0" pgwide="1"><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="offset" colwidth="182pt" align="left" /><colspec colname="1" colwidth="112pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="1" rowsep="1">TABLE 2</entry></row></thead><tbody valign="top"><row><entry /><entry namest="offset" nameend="1" align="center" rowsep="1" /></row><row><entry /><entry>Slice type</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="6"><colspec colname="offset" colwidth="77pt" align="left" /><colspec colname="1" colwidth="105pt" align="left" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="28pt" align="center" /><colspec colname="4" colwidth="28pt" align="center" /><colspec colname="5" colwidth="28pt" align="center" /><tbody valign="top"><row><entry /><entry>Syntax element</entry><entry>S1</entry><entry>I</entry><entry>P, SP</entry><entry>B</entry></row><row><entry /><entry namest="offset" nameend="5" align="center" rowsep="1" /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="6"><colspec colname="1" colwidth="77pt" align="left" /><colspec colname="2" colwidth="105pt" align="left" /><colspec colname="3" colwidth="28pt" align="center" /><colspec colname="4" colwidth="28pt" align="center" /><colspec colname="5" colwidth="28pt" align="center" /><colspec colname="6" colwidth="28pt" align="center" /><tbody valign="top"><row><entry>slice_data( )</entry><entry>mb_skip_flag</entry><entry /><entry /><entry>11-13</entry><entry>24-26</entry></row><row><entry /><entry>mb_field_decoding_flag</entry><entry>70-72</entry><entry>70-72</entry><entry>70-72</entry><entry>70-72</entry></row><row><entry>macroblock_layer( )</entry><entry>mb_type</entry><entry> 0-10</entry><entry> 3-10</entry><entry>14-20</entry><entry>27-35</entry></row><row><entry /><entry>transform_size_8×8_flag</entry><entry>na</entry><entry>399-401</entry><entry>399-401</entry><entry>399-401</entry></row><row><entry /><entry>coded_block_pattern (luma)</entry><entry>73-76</entry><entry>73-76</entry><entry>73-76</entry><entry>73-76</entry></row><row><entry /><entry>coded_block_pattern (chroma)</entry><entry>77-84</entry><entry>77-84</entry><entry>77-84</entry><entry>77-84</entry></row><row><entry /><entry>mb_qp_delta</entry><entry>60-63</entry><entry>60-63</entry><entry>60-63</entry><entry>60-63</entry></row><row><entry>mb_pred( )</entry><entry>prev_intra4×4_pred_mode_flag</entry><entry>68</entry><entry>68</entry><entry>68</entry><entry>68</entry></row><row><entry /><entry>rem_intra4×4_pred_mode</entry><entry>69</entry><entry>69</entry><entry>69</entry><entry>69</entry></row><row><entry /><entry>prev_intra8×8_pred_mode_flag</entry><entry>na</entry><entry>68</entry><entry>68</entry><entry>68</entry></row><row><entry /><entry>rem_intra8×8_pred_mode</entry><entry>na</entry><entry>69</entry><entry>69</entry><entry>69</entry></row><row><entry /><entry>intra_chroma_pred_mode</entry><entry>64-67</entry><entry>64-67</entry><entry>64-67</entry><entry>64-67</entry></row><row><entry>mb_pred( ) and</entry><entry>ref_idx_I0</entry><entry /><entry /><entry>54-59</entry><entry>54-59</entry></row><row><entry>sub_mb_pred( )</entry><entry>ref_idx_I1</entry><entry /><entry /><entry /><entry>54-59</entry></row><row><entry /><entry>mvd_I0[ ][ ][0]</entry><entry /><entry /><entry>40-46</entry><entry>40-46</entry></row><row><entry /><entry>mvd_I1[ ][ ][0]</entry><entry /><entry /><entry /><entry>40-46</entry></row><row><entry /><entry>mvd_I0[ ][ ][1]</entry><entry /><entry /><entry>47-53</entry><entry>47-53</entry></row><row><entry /><entry>mvd_I1[ ][ ][1]</entry><entry /><entry /><entry /><entry>47-53</entry></row><row><entry>sub_mb_pred( )</entry><entry>sub_mb_type</entry><entry /><entry /><entry>21-23</entry><entry>36-39</entry></row><row><entry>residual_block_cabac( )</entry><entry>coded_block_flag</entry><entry> 85-104</entry><entry> 85-104</entry><entry> 85-104</entry><entry> 85-104</entry></row><row><entry /><entry>significant_coeff_flag[ ]</entry><entry>105-165</entry><entry>105-165</entry><entry>105-165</entry><entry>105-165</entry></row><row><entry /><entry /><entry>277-337</entry><entry>277-337</entry><entry>277-337</entry><entry>277-337</entry></row><row><entry /><entry /><entry /><entry>402-416</entry><entry>402-416</entry><entry>402-416</entry></row><row><entry /><entry /><entry /><entry>436-450</entry><entry>436-450</entry><entry>436-450</entry></row><row><entry /><entry>last_significant_coeff_flag[ ]</entry><entry>166-226</entry><entry>166-226</entry><entry>166-226</entry><entry>166-226</entry></row><row><entry /><entry /><entry>338-398</entry><entry>338-398</entry><entry>338-398</entry><entry>338-398</entry></row><row><entry /><entry /><entry /><entry>417-425</entry><entry>417-425</entry><entry>417-425</entry></row><row><entry /><entry /><entry /><entry>451-459</entry><entry>451-459</entry><entry>451-459</entry></row><row><entry /><entry>coeff_abs_level_minus1[ ]</entry><entry>227-275</entry><entry>227-275</entry><entry>227-275</entry><entry>227-275</entry></row><row><entry /><entry /><entry /><entry>426-435</entry><entry>426-435</entry><entry>426-435</entry></row><row><entry namest="1" nameend="6" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
As known from Table 2, ctxIdx can receive 460 values from 0 to 459. In one syntax element, selection of ctxIdx varies depending on the slice type. For example, as ctxIdx of mb_field_decoding flag is 70-72 in slice type I, when determining the value of ctxIdx for the syntax element having a plurality of values of ctxIdx, it is determined by the operation of ctxIdx=ctxIdxOffset+ctxIdxInc, on the basis of the values of offset value ctxIdxOffset and increment value ctxIdxInc. ctxIdxOffset is an intrinsic value of syntax element shown in Table 3 (partly excerpted from the table of H.264 standard), and ctxIdxInc is uniformly determined from a plurality of values according to the rule determined in every syntax element (see the H.264 standard) In the case of mb_field_decoding_flag, ctxIdxOffset is 70, and any value from 0 to 2 is selected as ctxIdxInc depending on the condition.
The context calculating circuit <b>520</b> initializes and stores the occurrence probability information by calculation shown in <figref idrefs="DRAWINGS">FIG. 23</figref> in every 460 values of ctxIdx at the time of start of slice. The occurrence probability information includes a set of MPS (valMPS) showing the symbol of higher occurrence probability of binary symbols 0 and 1, and pState (pStateIdx) showing the occurrence probability. The occurrence probability information corresponding to the determined value of ctxIdx is called the “context information”.
The context calculating circuit <b>520</b> generates the context information, and inputs into the arithmetic coding circuit <b>140</b>. Thus, depending on the syntax element for arithmetic coding, the occurrence probability of the symbol to be entered in the arithmetic coding circuit <b>140</b>, that is, the context information is changed over adaptively. As a result, in the binary symbol sequence dynamically changing in the occurrence probability of binary symbols, an optimum arithmetic coding is realized. Herein, the total bit length of output codes occurring at the time of process of multivalued input data in a specific section is called the “code amount.”
Concept of operation of the arithmetic coding circuit <b>140</b> is explained by referring to <figref idrefs="DRAWINGS">FIGS. 24A and 24B</figref>. Suppose binary sequences of {0, 0.0, 1} are supplied in the condition in which ctxIdx is constant at a certain value, and the occurrence probability obtained from the context information is 0.75 (0.11 in binary notation) for symbol <b>0</b>, and 0.25 for symbol <b>1</b>. In this case, as shown in <figref idrefs="DRAWINGS">FIG. 24</figref> A, the present context information is expressed by MPS=0 showing the symbol of higher occurrence probability, and occurrence probability pState=0.11. Indeed, the actual pStateIdx is expressed by an index value in a range from 0 to 63, but a binary probability value is used herein for the sake of simplicity. The context information is updated by this binary arithmetic coding, and is returned to the context calculating circuit <b>520</b>. In the context calculating circuit <b>520</b>, the occurrence probability information of the corresponding ctxIdx is updated. This value is utilized again in the next coding process of the same ctxIdx.
Referring now to <figref idrefs="DRAWINGS">FIG. 24B</figref>, when a first input binary value “0” is entered, the section of [0, 1] is narrowed to the 0 side divided by probability 0.11, and becomes [0, 0.11]. When a second input “0” is entered, the section of [0, 0.11] is narrowed to [0, 0.1001]. Herein, 0.1001=0.11×0.11. Similarly, by a third input “0”, the section becomes [0, 0.011011]. When a fourth input “1” is entered, the section is narrowed to the 1 side, and the final section is narrowed to [0.01010001, 0.011011]. Of the values contained in this final section, the one having the shortest word length is the code word. That is, since 0.011 is contained in this section, the value “011” below the decimal point is an output binary sequence, and the four-bit input value is compressed to 3 bits.
In actual operation of the arithmetic coding circuit <b>140</b>, since this section is handled by the finite word length, the output bit is established depending on the values of codILow and codIRange in which the left end value of the section is codILow (0.01010001, etc.), the section range is codIRange (0.00011011, etc.), and codIRange and codILow are shifted by 1 bit to the left, and the resolution of the section is enhanced, which is called renormalization. The actual 1-bit arithmetic coding process is executed according to the flowcharts shown in <figref idrefs="DRAWINGS">FIG. 25</figref> and <figref idrefs="DRAWINGS">FIG. 26</figref>.
<figref idrefs="DRAWINGS">FIG. 25</figref> shows the arithmetic coding process (<b>820</b>) of one binary (Bin) input, consisting of initialization of a variable (<b>821</b>), judging whether Bin is MPS or not (<b>822</b>), updating of codILow and codIRange if not MPS (<b>823</b>), judging the value of pStateIdx (<b>824</b>), inverting of MPS (<b>825</b>), updating of occurrence probability pStateIdx (<b>826</b>, <b>827</b>), and renormalization (<b>830</b>). <figref idrefs="DRAWINGS">FIG. 25</figref> shows the process of renormalization <b>830</b>.
In <figref idrefs="DRAWINGS">FIG. 25</figref> and <figref idrefs="DRAWINGS">FIG. 26</figref>, codIRange and codILow are internal state variables, and are initialized only at the beginning of a slice. Right after coding of the 4-bit input value shown in <figref idrefs="DRAWINGS">FIG. 24B</figref>, when coding an input value different in ctxIdx, only the context information is changed, and the values right after 4-bit coding are used for codIRange and codILow. In the case of arithmetic coding of conversion coefficient data, multivalued input data different in ctxIdx are continuously coded, such as significant_coeff_flag of luminance <b>64</b> coefficient, coeff_abs_level_minus1 of luminance <b>64</b> coefficient, significant_coeff_flag of color difference DC component, and coeff_abs_level_minus1 of color difference AC component, and the internal state is initialized only at the beginning of a slice; therefore, all syntax elements including them must be coded sequentially only within the slice, and cannot be processed in parallel.
On the other hand, in moving image coding, since input images having various characteristics are compressed at a specified bit rate, it is required to control the code amount by properly controlling the quantization parameter when coding each macro block. To enhance the precision of code amount control, it is general to code repeatedly by changing the parameters in the unit such as slice, frame, or GOP (group of pictures). The repeated coding includes two types of coding, provisional coding mainly for predicting the code amount, and final coding.
Conventional provisional coding methods are roughly classified into actual entropy coding same as in final coding, and substitute operation without performing entropy coding. The former method in the conventional coder generally uses the variable length coding for entropy coding. Recently, arithmetic coding is proposed for entropy coding. As mentioned above, arithmetic coding is coding process in bit unit, and it is sequential coding in a slice, and if arithmetic coding is applied in provisional coding, a tremendous code amount is required. Hence, in a circuit in which the real-time processing is required, it is quite difficult to apply arithmetic coding in provisional coding.
Accordingly, as provisional coding method when arithmetic coding is applied in final coding method, the latter method is proposed (substitute operation for entropy coding). For example, as disclosed in patent document 1, without performing arithmetic coding, the input binary symbol amount is used as the prediction value of output code amount, and the processing amount when selecting the optimum coding mode can be saved. Patent document 2 proposes a method of prediction by using functions.
Patent document 1: JP-A-2005-318296
Patent document 2: JP-A-2005-203905
Non-patent document 1: ISO/IEC14496-10 Advanced video coding for generic audiovisual services
DISCLOSURE OF INVENTION
Problems to be Solved by the Invention
In the coder using arithmetic coding, however, the code amount cannot be predicted precisely in the conventional system performing provisional coding without arithmetic coding.
That is, in the system proposed in patent document 1, the function operates somewhat satisfactorily as relative comparison between a plurality of coding modes, but since arithmetic coding is not performed, the prediction value of code amount cannot be calculated precisely. In the system proposed in patent document 2, since it is completely different from the operation of arithmetic coding, the code amount cannot be predicted accurately.
The invention is conceived in the light of such problems, and it is hence a primary object thereof to present a coder using arithmetic coding, and more particularly a code amount predicting method of high precision substantially shortened in the processing time in the circuit while approximate operation is performed in the arithmetic coding.
Means for Solving the Problems
In the invention, in provisional coding, the binary symbol sequence is divided into a plurality of groups based on the context, and the predicted code amount is determined by approximating the arithmetic code amount in parallel on the binary symbol sequence classified into groups.
More specifically, the coder of the invention includes a blocking circuit for generating multivalued data in specified coding unit (for example, macro block), having a plurality of contexts, from an input signal, a binarizing circuit for converting the multivalued data generated by the blocking circuit into a binary symbol sequence, an arithmetic code amount approximating circuit for calculating the prediction code amount in the coding unit from the binary symbol sequence, and a coding circuit for coding the input signal arithmetically on the basis of the prediction code amount, in which the arithmetic code amount approximating circuit includes a selector for dividing the binary symbol sequence contained in the specified coding unit into a plurality of groups based on the contexts, a plurality of code amount approximating circuits, each of which is a circuit provided in each group, for calculating, from the binary symbol sequence divided into a plurality of groups, the prediction code amount of the group based on at least the section range in arithmetic coding, and an adder for adding the prediction code amounts from all code amount approximating circuits, and outputting the prediction code amount in the specified coding unit.
Effects of the Invention
According to the invention, in the coder using arithmetic coding, approximate calculation of arithmetic coding of a plurality of groups divided according to the contexts can be executed in parallel, and by operation similar to arithmetic coding, the processing time can be saved substantially as compared with arithmetic coding. As a result, by using a circuit suppressed in operation frequency, a coder based on coding value prediction at high precision is realized. The invention also presents a video coder of high image quality.
BRIEF DESCRIPTION OF DRAWINGS
<figref idrefs="DRAWINGS">FIG. 1</figref> is a configuration diagram of a video coder.
<figref idrefs="DRAWINGS">FIG. 2</figref> is a configuration diagram of an arithmetic code amount approximating circuit.
<figref idrefs="DRAWINGS">FIG. 3</figref> is an explanatory diagram of parallel processing in the arithmetic code amount approximating circuit.
<figref idrefs="DRAWINGS">FIG. 4</figref> is a diagram showing syntax elements to be processed in the arithmetic code amount approximating circuit.
<figref idrefs="DRAWINGS">FIG. 5</figref> is a flowchart of code amount approximating process of 1 unit in sequential process.
<figref idrefs="DRAWINGS">FIG. 6</figref> is a flowchart of unit initializing process.
<figref idrefs="DRAWINGS">FIG. 7</figref> is a flowchart of code amount approximating process of 1 Bin.
<figref idrefs="DRAWINGS">FIG. 8</figref> is a flowchart of code amount approximating process of 1 Bin.
<figref idrefs="DRAWINGS">FIG. 9</figref> is a flowchart of micro code amount approximating process of 1 unit.
<figref idrefs="DRAWINGS">FIG. 10A</figref> and <figref idrefs="DRAWINGS">FIG. 10B</figref> are explanatory diagrams of fraction code amount table.
<figref idrefs="DRAWINGS">FIG. 11</figref> is a flowchart of code amount approximating process of 1 unit in unary binalization.
<figref idrefs="DRAWINGS">FIG. 12</figref> is a flowchart of code amount approximating process on all Bin in a unit.
<figref idrefs="DRAWINGS">FIG. 13</figref> is an explanatory diagram of an integer code amount table.
<figref idrefs="DRAWINGS">FIG. 14</figref> is an explanatory diagram of initialization of an integer code amount table.
<figref idrefs="DRAWINGS">FIG. 15</figref> is a flowchart of code amount approximating process of 1 unit in bypass.
<figref idrefs="DRAWINGS">FIG. 16</figref> is a diagram of prediction precision of an arithmetic code amount approximating circuit.
<figref idrefs="DRAWINGS">FIG. 17A</figref> is a diagram of one frame composed of 1920×1080 pixels, and <figref idrefs="DRAWINGS">FIG. 17B</figref> is a diagram of one frame composed of 120×68 macro blocks.
<figref idrefs="DRAWINGS">FIG. 18A</figref> and <figref idrefs="DRAWINGS">FIG. 18</figref> B are diagrams of selection example of provisional quantization parameter.
<figref idrefs="DRAWINGS">FIG. 19A</figref>, <figref idrefs="DRAWINGS">FIG. 19B</figref>, and <figref idrefs="DRAWINGS">FIG. 19C</figref> are diagrams of selection example of provisional quantization parameter.
<figref idrefs="DRAWINGS">FIG. 20A</figref> and <figref idrefs="DRAWINGS">FIG. 20B</figref> are diagrams of an example of data accumulated in a code amount integrating memory.
<figref idrefs="DRAWINGS">FIG. 21</figref> is an explanatory diagram of calculating method of optimum quantization parameter.
<figref idrefs="DRAWINGS">FIG. 22</figref> is an explanatory diagram of a conventional context adaptive arithmetic coding circuit.
<figref idrefs="DRAWINGS">FIG. 23</figref> is an explanatory diagram of initialization of occurrence probability information.
<figref idrefs="DRAWINGS">FIG. 24A</figref> and <figref idrefs="DRAWINGS">FIG. 24B</figref> explanatory diagrams of concept of arithmetic coding.
<figref idrefs="DRAWINGS">FIG. 25</figref> is a flowchart of arithmetic coding process on input of 1 Bin.
<figref idrefs="DRAWINGS">FIG. 26</figref> is a flowchart of renormalization process.
DESCRIPTION OF THE REFERENCE NUMERALS
<ul><li id="ul0001-0001" num="0056"><b>100</b> Video coder</li><li id="ul0001-0002" num="0057"><b>101</b> First coding circuit</li><li id="ul0001-0003" num="0058"><b>102</b> Second coding circuit</li><li id="ul0001-0004" num="0059"><b>111</b> Quantization parameter generating circuit</li><li id="ul0001-0005" num="0060"><b>112</b> Quantization parameter calculating circuit</li><li id="ul0001-0006" num="0061"><b>121</b> First quantizing circuit</li><li id="ul0001-0007" num="0062"><b>122</b> Second quantizing circuit</li><li id="ul0001-0008" num="0063"><b>124</b> Inverse quantizing circuit</li><li id="ul0001-0009" num="0064"><b>131</b> First binarizing circuit</li><li id="ul0001-0010" num="0065"><b>132</b> Second binarizing circuit</li><li id="ul0001-0011" num="0066"><b>142</b> Arithmetic coding circuit</li><li id="ul0001-0012" num="0067"><b>151</b> Frame memory</li><li id="ul0001-0013" num="0068"><b>152</b> Prediction mode memory</li><li id="ul0001-0014" num="0069"><b>153</b> Code amount integration memory</li><li id="ul0001-0015" num="0070"><b>161</b> First blocking circuit</li><li id="ul0001-0016" num="0071"><b>162</b> Second blocking circuit</li><li id="ul0001-0017" num="0072"><b>171</b> First intra-prediction generating circuit</li><li id="ul0001-0018" num="0073"><b>172</b> Second intra-prediction generating circuit</li><li id="ul0001-0019" num="0074"><b>173</b> Prediction mode detecting circuit</li><li id="ul0001-0020" num="0075"><b>181</b> First DCT circuit</li><li id="ul0001-0021" num="0076"><b>182</b> Second DCT circuit</li><li id="ul0001-0022" num="0077"><b>184</b> Inverse DCT circuit</li><li id="ul0001-0023" num="0078"><b>191</b> Video signal input</li><li id="ul0001-0024" num="0079"><b>192</b> Output bit stream/E</li><li id="ul0001-0025" num="0080"><b>200</b> Arithmetic code amount approximating circuit</li><li id="ul0001-0026" num="0081"><b>211</b> to <b>214</b> Arithmetic code amount approximating circuit</li><li id="ul0001-0027" num="0082"><b>230</b> Selector</li><li id="ul0001-0028" num="0083"><b>231</b> Adder</li></ul>
BEST MODE FOR CARRYING OUT THE INVENTION
An embodiment of the invention is described below while referring to the accompanying drawings,
1. Configuration of Video Coder
This embodiment relates to a video coder for coding intra-frames that can be decoded within a frame, and controlling the code amount based on the code amount prediction of high precision by using a circuit in which operation frequency is suppressed low.
<figref idrefs="DRAWINGS">FIG. 1</figref> shows a configuration of a video coder in an embodiment of the invention. In <figref idrefs="DRAWINGS">FIG. 1</figref>, a video coder <b>100</b> includes a first coding circuit <b>101</b> for coding provisionally, a second coding circuit <b>102</b> for coding finally, a frame memory <b>151</b>, a prediction mode memory <b>152</b>, and a code amount integration memory <b>153</b>.
The first coding circuit <b>101</b> includes a first blocking circuit <b>161</b> for dividing an input signal into blocks, a prediction mode detecting circuit <b>173</b>, a first intra-prediction generating circuit <b>171</b>, a first DCT circuit <b>181</b> as orthogonal converting circuit, a quantization parameter generating circuit <b>111</b>, a first quantizing circuit <b>121</b>, a first binarizing circuit <b>131</b>, and an arithmetic code amount approximating circuit <b>200</b>.
The second coding circuit <b>102</b> includes a second blocking circuit <b>162</b>, a second intra-prediction generating means <b>172</b>, a second DCT circuit <b>182</b>, a quantization parameter calculating circuit <b>112</b>, a second quantizing circuit <b>122</b>, a second binarizing circuit <b>132</b>, an arithmetic coding circuit <b>142</b>, an inverse quantizing circuit <b>124</b>, and an inverse DCT circuit <b>184</b>.
In the video coder having the configuration as explained above, its operation is explained.
2. Operation of Video Coder
The video coder receives and processes video signals <b>191</b> in frame unit. The following explanation refers to a flow of process of video signals of one frame for the sake of simplicity.
When digital video signals <b>191</b> of one frame are entered in the video coder <b>100</b>, the first coding circuit <b>101</b> provisionally codes the entered digital video signals of one frame. In this provisional coding, the prediction mode value detected by prediction mode detecting circuit <b>173</b> is stored in the prediction mode memory <b>152</b>, and the code amount output from the arithmetic code amount approximating circuit <b>200</b> is stored in the code amount integration memory <b>153</b>.
The digital video signals <b>191</b> entered in the video coder <b>100</b> are stored in the frame memory <b>151</b>, and delayed by a specified time in one frame, and are sent into the second coding circuit <b>102</b>. The second coding circuit <b>102</b> codes the digital video signals of one frame, and outputs an obtained stream <b>192</b>. In coding by the second coding circuit <b>102</b>, the prediction mode value stored in the prediction mode memory <b>152</b>, and the code amount stored in the code amount integration memory <b>153</b> are entered, and the same frame is coded. The specific process in the first and second coding circuits <b>101</b>, <b>102</b> is described below.
2.1 Provisional Coding by First Coding Circuit
The provisional coding process of the first coding circuit <b>101</b> is explained.
The blocking circuit <b>161</b> receives digital video signals <b>191</b>, and divides the video signal of one frame into a plurality of macro blocks composed of 16×16 pixels. The macro block includes a plurality of blocks as units of DCT or intra-prediction as described below.
The intra-prediction generating circuit <b>171</b> predicts the values of pixels of the macro block in various prediction modes (intra-prediction), from the pixels of entered digital video signals, and pixels adjacent to pixels of entered digital video signals <b>191</b> in every 16×16 pixel macro block. The intra-prediction may be executed in the unit of 8×8 pixel block or 4×4 pixel block, instead of macro block units.
The prediction mode detecting circuit <b>173</b> detects the prediction mode value from the entered digital video signals <b>191</b>. The prediction mode value includes field macro block decoding flag (mb_field_decoding_flag), luminance 4×4 intra-prediction mode (Intra4×4PredMode), luminance 8×8 intra-prediction mode (Intra8×8PredMode), and color difference intra-prediction mode (intra_chroma_pred_mode). The detected prediction mode values are accumulated in the prediction mode memory <b>152</b>.
In the entered digital video signal <b>191</b>, the difference of each pixel value of macro block to be coded, and the pixel prediction value of the macro block to be coded generated in the intra-prediction generating circuit <b>171</b> is calculated, and a block of differential value of 16×16 pixels is generated, and output into the DCT circuit <b>181</b>. The intra-prediction may be executed in the unit of 8×8 pixel block or 4×4 pixel block, instead of macro block units.
The DCT circuit <b>181</b> processes the block of differential value by orthogonal transformation such as DCT (discrete cosine transform). This processing is carried out usually in the block unit of 4×4 pixels or 8×8 pixels, and the coefficient data of frequency component is output. The coefficient data is put into the quantizing circuit <b>121</b>, and is quantized according to the quantization parameter given from the quantization parameter generating circuit <b>111</b>. The quantization parameter is selected and determined from the plurality of parameter candidates by the quantization parameter generating circuit <b>111</b>.
The coefficient data quantized by the quantizing circuit <b>121</b> is binarized in the binarizing circuit <b>131</b>. The arithmetic code amount approximating circuit <b>200</b> determines and outputs the prediction code amount from the binarized coefficient data. The prediction code amount output from the arithmetic code amount approximating circuit <b>200</b> is recorded in the code amount integration memory <b>153</b>.
2.1.1 Operation of Arithmetic Code Amount Approximating Circuit
The arithmetic code amount approximating circuit <b>200</b> operates similarly to arithmetic coding, and determines a prediction code amount in every macro block, that is, an approximate value of code amount of arithmetic coding. <figref idrefs="DRAWINGS">FIG. 2</figref> shows a configuration of the arithmetic code amount approximating circuit <b>200</b>. The arithmetic code amount approximating circuit <b>200</b> includes a context calculating circuit <b>521</b>, a selector <b>230</b>, first to fourth code amount approximating circuits <b>211</b> to <b>214</b>, and an adder <b>231</b>. The first to fourth code amount approximating circuits <b>211</b> to <b>214</b> are selected depending on the value of context index (ctxIdx).
In the arithmetic code amount approximating circuit <b>200</b> having such configuration, the operation is described below.
The multivalued input data <b>501</b> is converted into binary symbol (binary) sequence <b>503</b> in the binarizing circuit <b>130</b>. The binarizing circuit <b>131</b> is basically a same circuit as the binarizing circuit <b>130</b> included in the context adaptive arithmetic coding circuit <b>145</b> in <figref idrefs="DRAWINGS">FIG. 22</figref>, but the binary symbol sequence <b>503</b> may not be always a serial output, and may be a parallel output. The context calculating circuit <b>521</b> outputs the present context index (ctxIdx) <b>505</b> to the selector <b>230</b> on the basis of the control information <b>502</b> entered simultaneously with the multivalued input data <b>501</b>. The selector <b>230</b> distributes the present binary symbol <b>503</b> to any one of the first to fourth code amount approximating circuits <b>211</b> to <b>214</b> on the basis of the context index (ctxIdx) <b>505</b>. The binary symbols are distributed so that the first and second code amount approximating circuits <b>211</b>, <b>212</b> may process the data about the luminance, and that the third to fourth code amount approximating circuits <b>213</b>, <b>214</b> may process the data relating to the color difference.
In the arithmetic code amount approximating circuit <b>200</b>, to execute the code amount approximating process simultaneously in the first to fourth code amount approximating circuits <b>211</b> to <b>214</b>, parallel processing is realized as shown in <figref idrefs="DRAWINGS">FIG. 3</figref>, and the processing time for approximating the code amount can be shortened.
Distribution of binary symbols <b>503</b> among the code amount approximating circuits <b>211</b> to <b>214</b> on the basis of the context index (ctxIdx) is explained more specifically by referring to <figref idrefs="DRAWINGS">FIG. 4</figref>.
<figref idrefs="DRAWINGS">FIG. 4</figref> is a diagram explaining the correspondence between ctxIdx and the first to fourth code amount approximating circuits <b>211</b> to <b>214</b> to be assigned, relating to the syntax element belonging to residual_block_cabac( ) in Table 1. As known from the diagram, for example, if ctxIdx=402, it belongs to significant_coeff_flag of luminance, and the first code amount approximating circuit <b>211</b> is selected. Although coeff_sign_flag is not shown in Table 1 because it is bypassed in arithmetic coding, it is handled as bypass in code amount approximating circuits, and is used in approximation of code amount.
The code amounts thus calculated in the code amount approximating circuits <b>211</b> to <b>214</b> are added in the adder <b>231</b>, and output as prediction code amount <b>290</b> in every specific section.
Processing executed in the code amount approximating circuits <b>211</b> to <b>214</b> consists of three processes as shown in flowcharts in <figref idrefs="DRAWINGS">FIG. 5</figref>, <figref idrefs="DRAWINGS">FIG. 11</figref>, and <figref idrefs="DRAWINGS">FIG. 15</figref>. The process shown in <figref idrefs="DRAWINGS">FIG. 11</figref> is applied in syntax element using unary binarization in binary processing such as coeff_abs_level_minus1. The process shown in <figref idrefs="DRAWINGS">FIG. 5</figref> shows the sequential process applied to an ordinary syntax element not applied in the process of <figref idrefs="DRAWINGS">FIG. 11</figref>. The process shown in <figref idrefs="DRAWINGS">FIG. 15</figref> is the process applied in the binary symbol (hereinafter, referred to as “Bin”) handled as bypass as mentioned above. The processes in <figref idrefs="DRAWINGS">FIG. 5</figref>, <figref idrefs="DRAWINGS">FIG. 11</figref>, and <figref idrefs="DRAWINGS">FIG. 15</figref> are executed in every unit. The unit is a set of binary symbols (Bin), and is determined in a set of Bin in which the values of ordinary ctxIdx are continuously the same.
(a) Arithmetic Code Amount Approximating Process by Sequential Process
Referring now to <figref idrefs="DRAWINGS">FIG. 5</figref> to <figref idrefs="DRAWINGS">FIG. 9</figref>, the code amount approximating process by sequential process is explained. <figref idrefs="DRAWINGS">FIG. 5</figref> shows a flowchart of code amount approximating process of 1 unit by sequential process. In this code amount approximating process, first (upon start of the unit), unit initializing process (S<b>11</b>) is executed, and code amount approximating process of 1 Bin (S<b>12</b>) is repeated until the unit is terminated (S<b>13</b>), finally (at the end of the unit) micro code amount approximating process of 1 unit is executed (S<b>14</b>), and the process is terminated.
Referring to <figref idrefs="DRAWINGS">FIG. 6</figref>, the detail of unit initializing process at step S<b>11</b> is explained. In the unit initializing process, the width of a section (codIRange), the integer portion of unit code amount (or integer code amount) (codeLengthInt), and the fractional portion of unit code amount (or fraction coding unit) (codeLengthFrac) are individually initialized (S<b>21</b>). The unit code amount is divided thus into the integer portion and the fractional portion, and the code amount is approximated at the precision of decimal fraction. That is, codeLengthFrac expresses the portion below the decimal point at 9-bit precision, and ranges in values from 0 to 511.
Referring to <figref idrefs="DRAWINGS">FIG. 7</figref>, the detail of code amount approximating process of 1 Bin at step S<b>12</b> is explained. In the code amount approximating process of 1 Bin, first, the variable is initialized (S<b>31</b>). Next, it is judged if Bin is MPS or not (S<b>32</b>). If Bin is not MPS, codIRange is updated (S<b>33</b>), and the value of pStateIdx is judged (S<b>34</b>). If the value of pStateIdx is 0, the MPS is inverted (S<b>35</b>), and the occurrence probability pStateIdx is updated (S<b>36</b>). If Bin is MPS at step S<b>32</b>, only updating of pStateIdx is executed (S<b>37</b>). Finally, the process is terminated by executing the code amount adding process of 1 Bin (S<b>38</b>). The flowchart in <figref idrefs="DRAWINGS">FIG. 7</figref> is similar to the flowchart of arithmetic coding process shown in <figref idrefs="DRAWINGS">FIG. 25</figref>, except that codILow is not calculated at step S<b>33</b>, and that the code amount adding process of 1 Bin (S<b>38</b>) is executed instead of step (S<b>830</b>) of renormalization in <figref idrefs="DRAWINGS">FIG. 25</figref>.
Referring to <figref idrefs="DRAWINGS">FIG. 8</figref>, the detail of code amount adding process of 1 Bin at step S<b>38</b> is explained. In the code amount adding process of 1 Bin, the process of adding 1 to the integer portion codeLengthInt of unit code amount is repeated by shifting codIRange by 1 bit to the left until codIRange becomes 0x100 or more (S<b>41</b>, S<b>42</b>). The flowchart in <figref idrefs="DRAWINGS">FIG. 8</figref> is similar to the flowchart of renormalization in <figref idrefs="DRAWINGS">FIG. 26</figref>, except that only the portion of operating codIRange is extracted, and that the number of times of left shift of codIRange is added to the code amount.
Referring to <figref idrefs="DRAWINGS">FIG. 9</figref>, the operation of micro code amount approximating process of 1 unit at step S<b>14</b> is explained. In the micro code amount approximating process of 1 unit shown in <figref idrefs="DRAWINGS">FIG. 9</figref>, the fractional portion codeLengthFrac of the unit code amount is determined by the table reference function readTableCdFrac( ) (S<b>51</b>). <figref idrefs="DRAWINGS">FIGS. 10A and 10B</figref> show the table reference function readTableCdFrac( ). The fraction code amount table mTbl is composed of 256 constants (see <figref idrefs="DRAWINGS">FIG. 10A</figref>). The fraction code amount table mTbl is initialized by the function initTableCdFrac( ) (see <figref idrefs="DRAWINGS">FIG. 10B</figref>). The fraction code amount table mTbl is a logarithmic table in which mTbl [0]=511, and mTbl [255]=1. At the time of codIRange=256, mTbl [0] is 511, and at the time of codIRange=511, mTbl [255] is 1. Conceptually, the closer is the codIRange to 512, the more frequently appears the MPS relatively, and the fraction code amount is shorter. The closer is the codIRange to 256, the more frequently appears the LPS relatively, and the fraction code amount is longer. This relation is approximated at high precision by the logarithmic curve shown in the formula of initTableCdFrac( ).
(b) Code Amount Approximating Process on Syntax Element Using Unary Binarization
Referring to <figref idrefs="DRAWINGS">FIG. 11</figref>, the following is the explanation about code amount approximating process on syntax element using unary binarization.
In the code amount approximating process using unary binarization shown in <figref idrefs="DRAWINGS">FIG. 11</figref>, first, the unit initializing process is executed (S<b>61</b>), and the code amount approximating process on all Bin in the unit (S<b>62</b>) is executed, finally the micro code amount approximating process of 1 unit (S<b>63</b>) is executed, and the process is terminated. The code amount approximating process of 1 unit shown in <figref idrefs="DRAWINGS">FIG. 11</figref> is similar to the code amount approximating process of 1 unit of sequential process in <figref idrefs="DRAWINGS">FIG. 5</figref>, except that the repeated process of the step of code amount approximating process of 1 Bin (S<b>12</b>) is replaced by the code amount approximating process on all Bin in the unit (S<b>62</b>).
This is an explanation of syntax element applying the code amount approximating process of unary binarization shown in <figref idrefs="DRAWINGS">FIG. 11</figref> as an example of coeff_abs_level_minus1. This coeff_abs_level_minus1 is different in ctxIdx depending on the position binIdx of Bin shown in Table 1. Accordingly, except for binIdx=0, the Bin in a range of 1≦binIdx≦13 is handled as 1 unit. That is, the subject is that the column of “Value of syntax element” in Table 1 is 1 or more to 14 or less. The maximum value is 14 because of the truncated type in which the maximum value binIdx=13 terminates with code “1”. When the number of codes “1” after binIdx=1 is expressed by variable run_length, the value of run_length is 0 or more to 13 or less. Since it is the bypass if binIdx=14 or more, in this example, the code amount approximating process of unary binarization is applied to all coefficients of which value of coeff_abs_level_minus1 is 1 or more, except for the bypass portion.
Referring to <figref idrefs="DRAWINGS">FIG. 12</figref>, the detail of code amount approximating process on all Bin in the unit at step S<b>62</b> is explained. First, the code amount is added to all Bin (S<b>71</b>). In this process, the integer portion codeLengthInt of the unit code amount is added by using table reference function readTableCdInt( ), and the codIRange is updated by using table reference function readTableRng( ). The context information of all Bin is updated (S<b>72</b>). In this process, valMPS is updated by using table reference function readTableMps( ), and pStateIdx is updated by using table reference function readTableState( ).
Referring to <figref idrefs="DRAWINGS">FIG. 13</figref>, each table reference function is explained. As shown in <figref idrefs="DRAWINGS">FIG. 13</figref>, since the maximum value of the bit is 13, it is defined that MAX_RUN=13. The table reference functions readTableCdInt( ), readTableRng( ), readTableMps( ), and readTableState( ) returns the values by referring respectively to integer code amount table (mCdlIntTbl), range table (mRngTbl), MPS table (mMpsTbl), and pState table (mStateTbl). These tables receive all coding patterns and all occurrence probability patterns of binary symbol sequences.
Referring to <figref idrefs="DRAWINGS">FIG. 14</figref>, the detail of initializing process of each table is explained. All of integer code amount table (mCdlIntTbl), range table (mRngTbl), MPS table (mMpsTbl), and pState table (mStateTbl) are initialized by executing the initialization function initTableCdInt( ) on every run_length, valMPS, and pStateIdx. This initializing process conforms to the operation of code amount approximating process of 1 unit by sequential process shown in <figref idrefs="DRAWINGS">FIG. 5</figref>. In <figref idrefs="DRAWINGS">FIG. 14</figref>, Proc1Bin (binVal, &valMPS, &pStateIdx) corresponds to the code amount approximating process (S<b>12</b>) of 1 Bin in <figref idrefs="DRAWINGS">FIG. 5</figref>, and binVal, valMPS, pStateIdx, and codIRange are processed equivalently to the process in <figref idrefs="DRAWINGS">FIG. 7</figref>. However, the code amount adding process of 1 Bin (S<b>38</b>) in <figref idrefs="DRAWINGS">FIG. 7</figref> is executed by “while” process at two positions in <figref idrefs="DRAWINGS">FIG. 14</figref>. Hence, the tables are initialized so that the integer portions of the code amount on all Bin can be calculated in batch.
(c) Arithmetic Code Amount Approximating Process on Bin Handled as Bypass
Referring to <figref idrefs="DRAWINGS">FIG. 15</figref>, the detail of code amount approximating process of 1 unit on Bin handled as bypass is explained. In this code amount approximating process, after the unit initializing process (S<b>81</b>), the integer portion codeLengthInt of the unit code amount is added 1 by 1 until the end of the unit (S<b>82</b>, S<b>83</b>). That is, in this code amount approximating process, the code amount of 1 Bin is counted as merely the code amount of 1 bit.
The adder <b>231</b> shown in <figref idrefs="DRAWINGS">FIG. 2</figref> calculates the prediction code amount as follows from the approximate value of the unit code amount obtained by three code amount approximating processes shown in <figref idrefs="DRAWINGS">FIG. 5</figref>, <figref idrefs="DRAWINGS">FIG. 11</figref>, and <figref idrefs="DRAWINGS">FIG. 15</figref>.
In all units included in the macro block, the integer portion (codeLengthInt) of unit code amount and fractional portion (codeLengthFrac) of unit code amount output from all code amount approximating circuits <b>211</b> to <b>214</b> are added respectively, and the integer portion of macro block prediction code amount (MBcodeLengthInt) and fractional portion of macro block prediction code amount (MBcodeLengthFrac) are calculated. The prediction code amount MBcodeLength of final macro block is determined in the following formula. <br /><i>MB</i>codeLength=<i>MB</i>codeLengthInt+(<i>MB</i>codeLengthFrac>>9) (1)
The code amount approximating processes of 1 unit described above (see <figref idrefs="DRAWINGS">FIG. 5</figref>, <figref idrefs="DRAWINGS">FIG. 11</figref>, <figref idrefs="DRAWINGS">FIG. 15</figref>) are can be processed in parallel in every unit, and the reason is explained below.
As mentioned above, two internal states codIRange and codILow were hindrance to parallel processing of context adaptive arithmetic coding process, but codIRange is initialized to a fixed value in every unit by unit initializing process shown in <figref idrefs="DRAWINGS">FIG. 6</figref>, and codILow does not exist in the entire process shown in <figref idrefs="DRAWINGS">FIG. 5</figref> to <figref idrefs="DRAWINGS">FIG. 15</figref>. Hence, the code amount approximating process of each unit can be executed in parallel in time on the circuit as shown in <figref idrefs="DRAWINGS">FIG. 3</figref>.
As explained in <figref idrefs="DRAWINGS">FIG. 11</figref>, meanwhile, in the code amount approximating process of 1 unit (unary binarization), a plurality of Bin can be processed in batch. This is possible by preparing a code group of unary binarization of a specified range, and a table based on combination of all valMPS and all pStateIdx.
The condition for approximating the code amount of the macro block at high precision by the arithmetic code amount approximating circuit <b>200</b> is explained.
The condition is that the rate of the generated code amount of the generated code amount of the entire macro block by ctxIdx covered by the entire code amount approximating circuits (total of #<b>1</b> to #<b>4</b>) is close to 100% as shown in <figref idrefs="DRAWINGS">FIG. 4</figref>, when the code amount approximating processes of 1 unit shown in <figref idrefs="DRAWINGS">FIG. 5</figref>, <figref idrefs="DRAWINGS">FIG. 11</figref> and <figref idrefs="DRAWINGS">FIG. 15</figref> are applied to the first to fourth code amount approximating circuits <b>211</b> to <b>214</b>. In the example shown in <figref idrefs="DRAWINGS">FIG. 4</figref>, the syntax element used in code amount approximating process is selected in the preliminary condition that the generated code amount of the syntax element belonging to residual_block_cabac( ) is close to 100% of the generated code amount of the entire macro block. If the generated code amount occupied by other syntax elements cannot be ignored, such syntax elements must be also the object of code amount approximating process.
<figref idrefs="DRAWINGS">FIG. 16</figref> shows the precision of prediction of prediction code amount <b>290</b> by the arithmetic code amount approximating circuit <b>200</b> according to the embodiment. In <figref idrefs="DRAWINGS">FIG. 16</figref>, the rate of error of prediction code amount of macro block unit and the generated code amount by arithmetic coding is plotted over 1 frame (8160 macro blocks). The error exists in four types of frames different in statistic characteristics (sequence #<b>1</b> to #<b>4</b>), in which the average is 0.47% and the standard deviation is 1.55%, and a high precision is obtained.
2.2 Final Coding by Second Coding Circuit
The operation of coding by the second coding circuit <b>102</b> in <figref idrefs="DRAWINGS">FIG. 1</figref> is explained. When provisional coding of one frame is finished by the first coding circuit <b>101</b>, the video signals of one frame accumulated in the frame memory <b>151</b> are put into the blocking circuit <b>162</b>. The pixel values of the macro block to be coded are compared with the pixel prediction values generated by the intra-prediction generating circuit <b>172</b> on the basis of the prediction mode values read out from the prediction mode memory <b>152</b>, and the differential value is calculated. The block of the differential value is processed by DCT in the DCT circuit <b>182</b>, and the coefficient data of frequency component is output. The coefficient data is input into the quantizing circuit <b>122</b>, and is quantized. At this time, the quantization parameter is calculated in the quantization parameter calculating circuit <b>112</b>. The quantized coefficient data is converted into a binary symbol in the binarizing circuit <b>132</b>, and is coded in the arithmetic coding circuit <b>142</b>, and is output as a bit stream <b>192</b>.
2.3 Setting of Quantization Parameter in Provisional Coding in First Coding Circuit <b>101</b>
A specific example of provisional coding in the first coding circuit <b>101</b> is explained in <figref idrefs="DRAWINGS">FIG. 17A</figref>, in which an optimum quantization parameter is calculated in the slice unit in coding of 1 frame of 1920×1080 pixels.
As shown in <figref idrefs="DRAWINGS">FIG. 17B</figref>, the frame of 1920×1080 pixels is composed of 8160 (=120×68) macro blocks of 16×16 pixels. The slice for dividing the frame may be defined as a set of arbitrary continuous macro blocks, and suppose, for example, one slice contains 2040 macro blocks, while one frame is composed of 4 slices in this case, it may be set as N=Nmb=2040 (=8160/4).
Before start of provisional coding, provisional quantization parameter QP<sub>1</sub>(n) to be used in the quantizing circuit <b>121</b> of the first coding circuit <b>101</b> is determined. The provisional quantization parameter is determined by selecting one from a plurality of candidates in every macro block. That is, the provisional quantization parameter QP<sub>1</sub>(n) for one macro block can be determined by selecting one from arbitrary different Q types of representative quantization parameters qp<sub>1</sub>(x) (x=0, 1, . . . , Q-1), and setting it to the provisional quantization parameter QP<sub>1</sub>(n) for the macro block.
For example, suppose to give two types [0 and 20] of representative parameters qp<sub>1</sub>(x)=0, 20. Herein, the notation of F(x)={A<sub>0</sub>, A<sub>1</sub>, . . . , A<sub>Q-1</sub>} (A<sub>n </sub>being an integer) means F(0)=A<sub>0</sub>, F(1)=A<sub>1</sub>, . . . , F(Q−1)=A<sub>Q-1</sub>. The provisional quantization parameter of each macro block is selected from two types of representative quantization parameters so as to equalize the occurrence frequency of each provisional quantization parameter. In this example, the quantization parameter <b>0</b> is given to a half of macro blocks, and the quantization parameter <b>20</b> is given to other half of macro blocks. To prevent deviation of provisional quantization parameters within the slice, the provisional parameters are disposed alternately or randomly in the slice. For example, in progressive scanning, as shown in <figref idrefs="DRAWINGS">FIG. 18A</figref>, provisional quantization parameters are given alternately in the ordinary raster scan order, and in interlace scanning, provisional quantization parameters are given alternately in every pair of macro blocks as shown in <figref idrefs="DRAWINGS">FIG. 18B</figref>.
Referring now to <figref idrefs="DRAWINGS">FIGS. 19A-19C</figref>, this is to explain the method of determining the type Q of representative quantization parameter qp<sub>1</sub>(x) in provisional coding, and its value qp<sub>1</sub>(x) (x=0, 1, . . . , Q-1).
The type Q of representative quantization parameter used in the quantizing circuit <b>121</b> is defined as a divisor of 2040 (number of macro blocks contained in one slice), or when macro blocks are paired for coding for interlace scanning, it may be defined as a divisor of its half or 1020. When the type Q of representative quantization parameter is determined in this manner, the number of times of occurrence of an arbitrary representative quantization parameter within a slice can be set constant. For example, in the case of progressive scanning, Q can be selected from 5, 10, 20 as a divisor of 2040. In the case of each Q, the number of times of occurrence of each representative parameter in one slice is 408 times, 204 times, and 102 times, respectively.
As the representative quantization parameter qp<sub>1</sub>(x), Q different values are selected from the range of quantization parameters that can be selected by the encoder. In this case, for the ease of code amount prediction for optimum coding, it is preferred to distribute the values appropriately. For example, when the number of macro blocks contained in one slice is 2040, and the quantization parameters that can be selected by the encoder are in a range of 0 to 51, as shown in <figref idrefs="DRAWINGS">FIG. 19A</figref>, Q=10, pq<sub>1</sub>(x)={0, 4, 8, 12, 16, 22, 28, 34, 42, 51} may be selected as the representative quantization parameter qp<sub>1</sub>. At this time, in the case of progressive scanning, the provisional quantization parameter as shown in <figref idrefs="DRAWINGS">FIG. 19B</figref> is applied, or in the case of interlace scanning, the provisional quantization parameter as shown in <figref idrefs="DRAWINGS">FIG. 19C</figref> is applied. The representative quantization parameter preferably contains the maximum value and minimum value of the quantization parameter.
The following is the explanation about the data accumulated in the code amount integration memory <b>153</b> as the result of provisional coding in the first coding circuit <b>101</b>. In provisional coding, in one slice including N=2040 macro blocks, in every macro block using the same representative quantization parameter qp<sub>1</sub>(x), the sum of prediction code amounts R<sub>1</sub>(n) output from the arithmetic code amount approximating circuit <b>200</b> is calculated. This sum is accumulated in the code amount integration memory <b>153</b> as prediction code amount r<sub>1</sub>(x) of every QP. In the example shown in <figref idrefs="DRAWINGS">FIGS. 20A and 20B</figref>, in each one of qp<sub>1</sub>(x)={0, 4, 8, 12, 16, 22, 28, 34, 42, 51} (x=0, 1, . . . , 9), the sum of prediction code amounts R<sub>1</sub>(n) in every representative quantization parameter, that is, the prediction code amount of every QP, r<sub>1</sub>(x)={95626, 82252, 64451, 48648, 31785, 19098, 10373, 5439, 2259, 1261} is accumulated in the code amount integration memory <b>153</b>.
2.4 Calculation of Quantization Parameter in Final Coding in Second Coding Circuit
<figref idrefs="DRAWINGS">FIG. 21</figref> shows a specific example of calculating the optimum quantization parameter QP<sub>2</sub>(n) in final coding in the quantization parameter calculating circuit <b>112</b> of the second coding circuit <b>102</b>. Herein, suppose target code amount T=300,000 bits of slice assigned by using information such as bit rate, frame rate, frame complicatedness, and slice coding type.
From the representative quantization parameter qp<sub>1</sub>(x) and every-QP prediction code amount r<sub>1</sub>(x), the relation of quantization parameter QP and slice prediction code amount may be plotted on the axes of coordinates as shown in <figref idrefs="DRAWINGS">FIG. 21</figref>. That is, r<sub>1</sub>(x) is the sum of prediction code amounts of every 204 macro blocks in each one of x=QP, and hence the slice prediction code amount of each QP is determined by multiplying r<sub>1</sub>(x) by 10 times. Further, by linear interpolation of plotted slice prediction code amount, a graph of slice prediction code amount on all QP values (0≦QP≦51) can be compiled. By determining the intersection of the slice target code amount T=300,000 and this graph, the optimum quantization parameter QP<sub>opt </sub>for generating the target code amount can be determined. The value of QP<sub>opt </sub>can be determined in formula (2) by linear interpolation. <br /><i>QP</i><sub>opt</sub><i>=q+{QP</i><sub>1</sub>(<i>q+</i>1)−<i>QP</i><sub>1</sub>(<i>q</i>)}×{<i>Q*r</i><sub>1</sub>(<i>q</i>)−<i>T}/{Q×r</i><sub>1</sub>(<i>q</i>)−<i>Q×r</i><sub>1</sub>(<i>q+</i>1)} (2)<br /> where q is an integer in a range of 0≦q≦Q−1, satisfying the relation of Q×r<sub>1</sub>(q+1) 5 T≦Q×r<sub>1</sub>(q). Herein, one optimum quantization parameter is used throughout one slice. If T>Q×r<sub>1</sub>(q) in all q, q is 0, and if T≦Q×r<sub>1</sub>(q) in all q, q is Q−1.
Since the target code amount T of the slice is 300,000 bits, according to formula (2), QP<sub>opt </sub>is 16.84. <br /><i>QP</i><sub>opt</sub>=16+(22−16)×(317850−300,000)/(317850−190980)≈16.84
Since the quantization parameter is an integer, the optimum quantization parameter QP<sub>opt </sub>is 17. Depending on the amount below the decimal point, the smaller side value of 16 may be selected.
Thus, the optimum quantization parameter QP<sub>2</sub>(n)=QP<sub>opt </sub>can be calculated. Besides, by using QP<sub>opt </sub>as initial value for coding of N macro blocks, the subsequent quantization parameters can be changed dynamically.
As explained herein, by using the arithmetic code amount approximating circuit <b>200</b> of the embodiment, the output stream <b>192</b> in the video coder can be obtained.
In the example explained above, the arithmetic code amount approximating circuit <b>200</b> is composed by assuming the syntax element contained in residual_block_cabac( ) in Table 2, but all other syntax elements may be assumed such as mb_pred( ) or macroblock_layer( ). In this case, basically, the range of the intended ctxIdx may be extended as shown in <figref idrefs="DRAWINGS">FIG. 4</figref>.
3. Modified Examples
The foregoing embodiment relates to the configuration of the arithmetic code amount approximating circuit <b>200</b> for coding the intra-frame only, but it is also applicable when coding the P-frame or B-frame of prediction between frames. In the P-frame or B-frame, the code amount necessary for motion vector is increased, and syntax elements contained in mb_pred( ) or sub_mb_pred( ) in Table 2 may be added as the object of processing.
In the embodiment, as shown in <figref idrefs="DRAWINGS">FIG. 2</figref>, four code amount approximating circuits are used, but the number of code amount approximating circuits is not particularly specified. In the example of unary binarization, batch processing without using sequential process of Bin is explained, but evidently the concept of the embodiment can be applied also to the binary coding method other than unary binarization. The table of integer portion and the table of fractional portion are explained separately, but the tables can be integrated on the basis of the codIRange value.
In the embodiment, N is the number of macro blocks contained in the slice unit. But by coding provisionally or coding in a smaller unit, the prediction code amount R<sub>2</sub>(n) may be obtained from the arithmetic code amount approximating circuit <b>200</b>, and the result may be reflected in selection of provisional quantization parameter, and in such a case, the value of N may be set at a smaller value conforming to the coding unit. The value of N is fixed in the explanation, but may be changed dynamically in the frame unit or other unit. The graph of slice prediction code amount is determined by linear interpolation, but may be also determined by spline interpolation or more advanced numerical interpolation.
In the embodiment, provisional coding is processed only once, but may be executed plural times if real-time coding is not required, or more processes can be executed in a specific delay by pipeline or parallel processing of provisional coding.
The bit stream coded by using the embodiment can be recorded in various recording media such as tape, optical disk, magnetic disk, and semiconductor memory, and stored in a re-distributable form.
4. Summary
As explained herein, according to the embodiment, by parallel operation of a plurality of approximating circuits provided corresponding to the context of the binary symbol sequences, the approximating operation of arithmetic coding of a plurality of groups divided on the basis of the context can be executed in parallel. That is, the approximate value of code amount of arithmetic coding of binary symbol sequences can be determined. As a result, in spite of the operation similar to arithmetic coding, as compared with the normal arithmetic coding, the processing time is saved substantially, and a coder is realized on the basis of code amount prediction of high precision by using a circuit suppressed low in the operation frequency. By using such coder, a video coder of high image quality is presented.
INDUSTRIAL APPLICABILITY
The invention realizes a coder of high image quality on the basis of code amount prediction of high precision by using a circuit suppressed low in the operation frequency, and is hence particularly effective in a recording apparatus such as camera recorder and image recording device in which real-time operation is demanded in a small circuit scale.
Although the present invention has been described in connection with specified embodiments thereof, many other modifications, corrections and applications are apparent to those skilled in the art. Therefore, the present invention is not limited by the disclosure provided herein but limited only to the scope of the appended claims. The present disclosure relates to subject matter contained in Japanese Patent Application No. 2006-323384, filed on Nov. 30, 2006, which is expressly incorporated herein by reference in its entirety.
Contents7
25 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25
Every citation, both waysCites: the store holds 29 of 30
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US2011158465A1 | Cited by | United States of America | Pre-grant |
| US9113231B2 | Cited by | United States of America | Applicant |
| US9105091B2 | Cited by | United States of America | Applicant |
| US2011142419A1 | Cited by | United States of America | Pre-grant |
| US2011135143A1 | Cited by | United States of America | Pre-grant |
| US8594366B2 | Cited by | United States of America | Applicant |
| US2015071363A1 | Cited by | United States of America | Pre-grant |
| US8824727B2 | Cited by | United States of America | Applicant |
| US9042455B2 | Cited by | United States of America | Applicant |
| US8948443B2 | Cited by | United States of America | Applicant |
| US9066103B2 | Cited by | United States of America | Applicant |
| US2010238056A1 | Cited by | United States of America | Pre-grant |
| US10045051B2 | Cited by | United States of America | Search report |
| US2002114527A1 | Cites | United States of America | Applicant |
| US2002114529A1 | Cites | United States of America | Applicant |
| JP2003018593A | Cites | Japan | Applicant |
| WO2004028165A2 | Cites | World Intellectual Property Organization (WIPO) | Applicant |
| JP2004135251A | Cites | Japan | Applicant |
| JP2004135252A | Cites | Japan | Applicant |
| US2005129122A1 | Cites | United States of America | Search report |
| US2005129320A1 | Cites | United States of America | Applicant |
| JP2005151391A | Cites | Japan | Applicant |
| US2005156762A1 | Cites | United States of America | Applicant |
| US2005180505A1 | Cites | United States of America | Applicant |
| JP2005184232A | Cites | Japan | Applicant |
| JP2005203905A | Cites | Japan | Applicant |
| US2005243930A1 | Cites | United States of America | Applicant |
| US2005249289A1 | Cites | United States of America | Search report |
| JP2005252374A | Cites | Japan | Applicant |
| JP2005318296A | Cites | Japan | Applicant |
| US2007033019A1 | Cites | United States of America | Applicant |
| US2007100613A1 | Cites | United States of America | Applicant |
| US2007255558A1 | Cites | United States of America | Applicant |
| US2008275698A1 | Cites | United States of America | Applicant |
| US2009012781A1 | Cites | United States of America | Applicant |
| US5410352A | Cites | United States of America | Applicant |
| US5870145A | Cites | United States of America | Applicant |
| US6411231B1 | Cites | United States of America | Search report |
| US6850175B1 | Cites | United States of America | Applicant |
| WO9628937A1 | Cites | World Intellectual Property Organization (WIPO) | Applicant |
| JPH05252403A | Cites | Japan | Applicant |
| JPH1155528A | Cites | Japan | Search report |
| English language abstract of JP 2005-184232 A, Jul. 7, 2005. | Non-patent | – | Applicant |
| English language abstract of JP 2004-135252 A, Apr. 30, 2004. | Non-patent | – | Applicant |
| English language abstract of JP 2004-135251 A, Apr. 30, 2004. | Non-patent | – | Applicant |
| English language abstract of JP 2005-151391 A, Jun. 9, 2005. | Non-patent | – | Applicant |
| English language abstract of JP 2003-18593 A, Jan. 17, 2003. | Non-patent | – | Applicant |
| English language abstract of JP 2005-318296 A, Nov. 10, 2005. | Non-patent | – | Applicant |
| English language abstract of JP 2005-203905 A, Jul. 28, 2005. | Non-patent | – | Applicant |
| Yamakage et al., "HD DVD ni Mochiru Dogazo Fugoka Gijutsu", Toshiba Review, vol. 60, No. 1, Jan. 1, 2005, pp. 17-20. | Non-patent | – | Applicant |
| ITU-T Recommendation H.264, "Series H: Audiovisual and multimedia Systems, Infrastructure of audiovisual services-Coding of moving video, Advanced video coding for generic audiovisual services," Nov. 2007. | Non-patent | – | Applicant |
| English language abstract of JP 2005-252374 A, Sep. 15, 2005. | Non-patent | – | Applicant |
| English language abstract of JP 5-252403 A, Sep. 28, 1993. | Non-patent | – | Applicant |
| U.S. Appl. No. 12/439,021 to Tanaka, filed Feb. 26, 2009. | Non-patent | – | Applicant |
10 members in 5 offices
Priority claims8
| Document | Office | Kind | Date |
|---|---|---|---|
| 2006323384 | Japan | A | |
| 2006323384 | Japan | A | |
| 2007069824 | Japan | W | |
| 2007069824 | Japan | W | |
| 2006323384 | – | – | – |
| JP20060323384 | – | – | – |
| PCTJP2007069824 | – | – | – |
| WO2007JP69824 | – | – | – |
Members10
| Document | Office | Kind | |
|---|---|---|---|
| WO2008065814A1 | World Intellectual Property Organization (WIPO) | A1 | |
| CN101502123A | China | A | |
| EP2091257A1 | European Patent Office (EPO) | A1 | |
| JP4354520B2 | Japan | B2 | |
| US2010007532A1 | United States of America | A1 | |
| JPWO2008065814A1 | Japan | A1 | |
| US7839312B2This record | United States of America | B2 | |
| EP2091257A4 | European Patent Office (EPO) | A4 | |
| CN101502123B | China | B | |
| EP2091257B1 | European Patent Office (EPO) | B1 |
37 transactions on the USPTO file
Allowed without a rejection on record.
- Non-final rejections
- 0
- Final rejections
- 0
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Payment of Maintenance Fee, 12th Year, Large EntityM1553 | M1553 | |
| Payment of Maintenance Fee, 8th Year, Large EntityM1552 | M1552 | |
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Patent Issue Date Used in PTA CalculationAllowedPTAC | PTAC | |
| Email NotificationEML_NTR | EML_NTR | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Response to Reasons for AllowanceREAS | REAS | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Email NotificationEML_NTR | EML_NTR | |
| PG-Pub Issue NotificationPG-ISSUE | PG-ISSUE | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Email NotificationEML_NTR | EML_NTR | |
| Email NotificationEML_NTR | EML_NTR | |
| Filing ReceiptFLRCPT.O | FLRCPT.O | |
| Notice of DO/EO Acceptance MailedM903 | M903 | |
| Sent to Classification ContractorPGPC | PGPC | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Reference capture on IDSRCAP | RCAP | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Request for Foreign Priority (Priority Papers May Be Included)RQPR | RQPR | |
| 371 Completion Date371COMP | 371COMP | |
| Cleared by OIPE CSRL194 | L194 | |
| IFW Scan & PACR Auto Security ReviewSCAN | SCAN | |
| Initial Exam Team nnIEXX | IEXX |
6 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Maintenance fee paymentMAFP | MAFP | |
| Maintenance fee paymentMAFP | MAFP | |
| Fee paymentFPAY | FPAY | |
| Fee payment procedurePAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Information on status: patent grantGrantedPATENTED CASESTCF | STCF | |
| AssignmentAS | AS |
Numbers
- Publication
- 07839312
- Publication, DOCDB
- 7839312
- Publication, EPODOC
- US7839312
- Application
- 12439004
- Application, DOCDB
- 43900407
- Application, EPODOC
- US20070439004
Titles
- English
- Coder
Patent term adjustment
- A delay
- +91 daysthe office missed an examination deadline
- Net adjustment
- 91 days
Classification
- CPC, 9
- H03M7/4006
- H04N19/70
- H04N19/46
- H04N19/13
- H04N19/149
- H04N19/15
- H04N19/126
- H04N19/194
- H04N19/436
- IPC, 16
- H03M7 00
- H04N1 41
- H03M7 40
- H04N19 00
- H04N19 13
- H04N19 134
- H04N19 136
- H04N19 146
- H04N19 189
- H04N19 423
- H04N19 593
- H04N19 60
- H04N19 61
- H04N19 625
- H04N19 85
- H04N19 91
- USPC, 3
- 341107000
- 341050000
- 341051000