Apparatus, method and program for coding moving pictures
24 claims: 13 independent, 11 dependent
- 1画像をブロック毎に符号化することでビットストリームを生成する画像符号化方法であって、 符号化対象ブロックに空間的または時間的に隣接するブロックの符号化に用いられた予測方向、動きベクトルおよび参照ピクチャインデックスに基づいて、前記符号化対象ブロックの符号化に用いられる予測方向、動きベクトルおよび参照ピクチャインデックスの候補であるマージ候補を第1マージ候補として導出する第1導出ステップと、 予め定められたベクトルを動きベクトルとして持つマージ候補を第2マージ候補として導出する第2導出ステップと、 導出された前記第1マージ候補および前記第2マージ候補の中から、前記符号化対象ブロックの符号化に用いられるマージ候補を選択する選択ステップと、 選択された前記マージ候補を特定するためのインデックスを前記ビットストリームに付加する符号化ステップとを含み、 前記第2導出ステップでは、参照可能な参照ピクチャ毎に前記第2マージ候補を導出する 画像符号化方法。
- 2前記予め定められたベクトルは、零ベクトルである 請求項1に記載の画像符号化方法。
- 3前記画像符号化方法は、さらに、 マージ候補の最大数を決定する決定ステップと、 導出された前記第1マージ候補の数が前記最大数より小さいか否かを判定する判定ステップとを含み、 前記第2導出ステップでは、前記第1マージ候補の数が前記最大数より小さいと判定された場合に、前記第2マージ候補を導出する 請求項1または2に記載の画像符号化方法。
- 4前記符号化ステップでは、決定された前記最大数を用いて前記インデックスを符号化し、符号化された前記インデックスを前記ビットストリームに付加する 請求項3に記載の画像符号化方法。
- 5前記符号化ステップでは、さらに、決定された前記最大数を示す情報を前記ビットストリームに付加する 請求項3または4に記載の画像符号化方法。
- 6前記第1導出ステップでは、予測方向、動きベクトルおよび参照ピクチャインデックスの組合せが既に導出された第1マージ候補と重複しないマージ候補を前記第1マージ候補として導出する 請求項1~5のいずれか1項に記載の画像符号化方法。
- 7前記画像符号化方法は、さらに、 第1規格に準拠する第1符号化処理、または第2規格に準拠する第2符号化処理に、符号化処理を切り替える切り替えステップと、 切り替えられた前記符号化処理が準拠する前記第1規格または前記第2規格を示す識別情報を前記ビットストリームに付加する付加ステップとを含み、 前記符号化処理が前記第1符号化処理に切り替えられた場合に、前記第1符号化処理として、前記第1導出ステップと、前記第2導出ステップと、前記選択ステップと、前記符号化ステップとが行われる 請求項1に記載の画像符号化方法。
- 8画像をブロック毎に符号化することでビットストリームを生成する画像符号化装置であって、 符号化対象ブロックに空間的または時間的に隣接するブロックの符号化に用いられた予測方向、動きベクトルおよび参照ピクチャインデックスに基づいて、前記符号化対象ブロックの符号化に用いられる予測方向、動きベクトルおよび参照ピクチャインデックスの候補であるマージ候補を第1マージ候補として導出する第1導出部と、 予め定められたベクトルを動きベクトルとして持つマージ候補を第2マージ候補として導出する第2導出部と、 導出された前記第1マージ候補および前記第2マージ候補の中から、前記符号化対象ブロックの符号化に用いられるマージ候補を選択する予測制御部と、 選択された前記マージ候補を特定するためのインデックスを前記ビットストリームに付加する符号化部とを備え、 前記第2導出部は、参照可能な参照ピクチャ毎に前記第2マージ候補を導出する 画像符号化装置。
- 9ビットストリームに含まれる符号化画像をブロック毎に復号する画像復号方法であって、 復号対象ブロックに空間的または時間的に隣接するブロックの復号に用いられた予測方向、動きベクトルおよび参照ピクチャインデックスに基づいて、前記復号対象ブロックの復号に用いられる予測方向、動きベクトルおよび参照ピクチャインデックスの候補であるマージ候補を第1マージ候補として導出する第1導出ステップと、 予め定められたベクトルを動きベクトルとして持つマージ候補を第2マージ候補として導出する第2導出ステップと、 前記ビットストリームから、マージ候補を特定するためのインデックスを取得する取得ステップと、 取得された前記インデックスに基づいて、前記第1マージ候補および前記第2マージ候補の中から、前記復号対象ブロックの復号に用いられるマージ候補を選択する選択ステップとを含み、 前記第2導出ステップでは、参照可能な参照ピクチャ毎に前記第2マージ候補を導出する 画像復号方法。
- 10前記予め定められたベクトルは、零ベクトルである 請求項9に記載の画像復号方法。
- 11前記画像復号方法は、さらに、 マージ候補の最大数を決定する決定ステップと、 導出された前記第1マージ候補の数が前記最大数より小さいか否かを判定する判定ステップとを含み、 前記第2導出ステップでは、導出された前記第1マージ候補の数が前記最大数より小さいと判定された場合に、前記第2マージ候補を導出する 請求項9または10に記載の画像復号方法。
- 12前記取得ステップでは、前記ビットストリームに付加された符号化された前記インデックスを、決定された前記最大数を用いて復号することにより、前記インデックスを取得する 請求項11に記載の画像復号方法。
- 13前記決定ステップでは、前記ビットストリームに付加された最大数を示す情報に基づいて、前記最大数を決定する 請求項11または12に記載の画像復号方法。
- 14前記第1導出ステップでは、予測方向、動きベクトルおよび参照ピクチャインデックスの組合せが既に導出された第1マージ候補と重複しないマージ候補を前記第1マージ候補として導出する 請求項9~13のいずれか1項に記載の画像復号方法。
- 15前記画像復号方法は、さらに、 前記ビットストリームに付加された第1規格または第2規格を示す識別情報に応じて、前記第1規格に準拠する第1復号処理、または前記第2規格に準拠する第2復号処理に、復号処理を切り替える切り替えステップを含み、 前記復号処理が第1復号処理に切り替えられた場合に、前記第1復号処理として、前記第1導出ステップと、前記第2導出ステップと、前記取得ステップと、前記選択ステップとが行われる 請求項9に記載の画像復号方法。
- 16ビットストリームに含まれる符号化画像をブロック毎に復号する画像復号装置であって、 復号対象ブロックに空間的または時間的に隣接するブロックの復号に用いられた予測方向、動きベクトルおよび参照ピクチャインデックスに基づいて、前記復号対象ブロックの復号に用いられる予測方向、動きベクトルおよび参照ピクチャインデックスの候補であるマージ候補を第1マージ候補として導出する第1導出部と、 予め定められたベクトルを動きベクトルとして持つマージ候補を第2マージ候補として導出する第2導出部と、 前記ビットストリームから、マージ候補を特定するためのインデックスを取得する復号部と、 取得された前記インデックスに基づいて、前記第1マージ候補および前記第2マージ候補の中から、前記復号対象ブロックの復号に用いられるマージ候補を選択する予測制御部とを備え、 前記第2導出部は、参照可能な参照ピクチャ毎に前記第2マージ候補を導出する 画像復号装置。
- 17画像をブロック毎に符号化することでビットストリームを生成する画像符号化装置と、 請求項16に記載の画像復号装置とを備え、 前記画像符号化装置は、 符号化対象ブロックに空間的または時間的に隣接するブロックの符号化に用いられた予測方向、動きベクトルおよび参照ピクチャインデックスに基づいて、前記符号化対象ブロックの符号化に用いられる予測方向、動きベクトルおよび参照ピクチャインデックスの候補であるマージ候補を第1マージ候補として導出する第1導出部と、 予め定められたベクトルを動きベクトルとして持つマージ候補を第2マージ候補として導出する第2導出部と、 導出された前記第1マージ候補および前記第2マージ候補の中から、前記符号化対象ブロックの符号化に用いられるマージ候補を選択する予測制御部と、 選択された前記マージ候補を特定するためのインデックスを前記ビットストリームに付加する符号化部とを備え、 前記画像符号化装置の前記第2導出部は、参照可能な参照ピクチャ毎に前記第2マージ候補を導出する 画像符号化復号装置。
- 20符号化対象ブロックを符号化する画像符号化方法であって、 第1ブロックの符号化に用いられた第1動きベクトルを有する第1候補を導出し、 第1零ベクトルである第2動きベクトルと、前記第2動きベクトルに対応する第1参照ピクチャを特定する第1参照ピクチャインデックス値とを有する第2候補を導出し、 第2零ベクトルである第3動きベクトルと、前記第3動きベクトルに対応する第2参照ピクチャを特定する第2参照ピクチャインデックス値であって前記第1参照ピクチャインデックス値と異なる第2参照ピクチャインデックス値とを有する第3候補を導出し、 動きベクトルを有する候補に対応するインデックスを符号化し、 前記動きベクトルは前記符号化対象ブロックの符号化に用いられ、 前記候補は、前記第1候補、前記第2候補および前記第3候補を含む複数の候補のうちの1つである 画像符号化方法。
- 21復号対象ブロックを復号する画像復号方法であって、 第1ブロックの復号に用いられた第1動きベクトルを有する第1候補を導出し、 第1零ベクトルである第2動きベクトルと、前記第2動きベクトルに対応する第1参照ピクチャを特定する第1参照ピクチャインデックス値とを有する第2候補を導出し、 第2零ベクトルである第3動きベクトルと、前記第3動きベクトルに対応する第2参照ピクチャを特定する第2参照ピクチャインデックス値であって前記第1参照ピクチャインデックス値と異なる第2参照ピクチャインデックス値とを有する第3候補を導出し、 動きベクトルを有する候補に対応する符号化されたインデックスを復号し、 前記動きベクトルは前記復号対象ブロックの復号に用いられ、 前記候補は、前記第1候補、前記第2候補および前記第3候補を含む複数の候補のうちの1つである 画像復号方法。
- 22第1ブロックの復号に用いられた第1動きベクトルを有する第1候補を導出し、 第1零ベクトルである第2動きベクトルと、前記第2動きベクトルに対応する第1参照ピクチャを特定する第1参照ピクチャインデックス値とを有する第2候補を導出し、 第2零ベクトルである第3動きベクトルと、前記第3動きベクトルに対応する第2参照ピクチャを特定する第2参照ピクチャインデックス値であって前記第1参照ピクチャインデックス値と異なる第2参照ピクチャインデックス値とを有する第3候補を導出し、 復号対象ブロックの復号に用いられる動きベクトルを導出し、 前記動きベクトルを有する候補は、前記第1候補、前記第2候補および前記第3候補を含む複数の候補のうちの1つである 動きベクトル導出方法。
- 23符号化対象ブロックを符号化する画像符号化装置であって、 第1ブロックの符号化に用いられた第1動きベクトルを有する第1候補を導出する第1導出器と、 第1零ベクトルである第2動きベクトルと、前記第2動きベクトルに対応する第1参照ピクチャを特定する第1参照ピクチャインデックス値とを有する第2候補を導出する第2導出器と、 第2零ベクトルである第3動きベクトルと、前記第3動きベクトルに対応する第2参照ピクチャを特定する第2参照ピクチャインデックス値であって前記第1参照ピクチャインデックス値と異なる第2参照ピクチャインデックス値とを有する第3候補を導出する第3導出器と、 動きベクトルを有する候補に対応するインデックスを符号化する符号化器とを備え、 前記動きベクトルは前記符号化対象ブロックの符号化に用いられ、 前記候補は、前記第1候補、前記第2候補および前記第3候補を含む複数の候補のうちの1つである 画像符号化装置。
- 24復号対象ブロックを復号する画像復号装置であって、 第1ブロックの復号に用いられた第1動きベクトルを有する第1候補を導出する第1導出器と、 第1零ベクトルである第2動きベクトルと、前記第2動きベクトルに対応する第1参照ピクチャを特定する第1参照ピクチャインデックス値とを有する第2候補を導出する第2導出器と、 第2零ベクトルである第3動きベクトルと、前記第3動きベクトルに対応する第2参照ピクチャを特定する第2参照ピクチャインデックス値であって前記第1参照ピクチャインデックス値と異なる第2参照ピクチャインデックス値とを有する第3候補を導出する第3導出器と、 動きベクトルを有する候補に対応する符号化されたインデックスを復号する復号器とを備え、 前記動きベクトルは前記復号対象ブロックの復号に用いられ、 前記候補は、前記第1候補、前記第2候補および前記第3候補を含む複数の候補のうちの1つである 画像復号装置。
- 25第1ブロックの復号に用いられた第1動きベクトルを有する第1候補を導出する第1導出器と、 第1零ベクトルである第2動きベクトルと、前記第2動きベクトルに対応する第1参照ピクチャを特定する第1参照ピクチャインデックス値とを有する第2候補を導出する第2導出器と、 第2零ベクトルである第3動きベクトルと、前記第3動きベクトルに対応する第2参照ピクチャを特定する第2参照ピクチャインデックス値であって前記第1参照ピクチャインデックス値と異なる第2参照ピクチャインデックス値とを有する第3候補を導出する第3導出器と、 復号対象ブロックの復号に用いられる動きベクトルを導出する第4導出器とを備え、 前記動きベクトルを有する候補は、前記第1候補、前記第2候補および前記第3候補を含む複数の候補のうちの1つである 動きベクトル導出装置。
- 26第1ブロックの復号に用いられた第1動きベクトルを有する第1候補を導出する第1導出器と、 第1零ベクトルである第2動きベクトルと、前記第2動きベクトルに対応する第1参照ピクチャを特定する第1参照ピクチャインデックス値とを有する第2候補を導出する第2導出器と、 第2零ベクトルである第3動きベクトルと、前記第3動きベクトルに対応する第2参照ピクチャを特定する第2参照ピクチャインデックス値であって前記第1参照ピクチャインデックス値と異なる第2参照ピクチャインデックス値とを有する第3候補を導出する第3導出器と、 動きベクトルを有する候補に対応する符号化されたインデックスを復号する復号器とを備え、 前記動きベクトルは復号対象ブロックの復号に用いられ、 前記候補は、前記第1候補、前記第2候補および前記第3候補を含む複数の候補のうちの1つである 集積回路。
Independent claims24
438 paragraphs, as filed
The present invention relates to an image coding method and an image decoding method.
In the moving image coding process, the amount of information is generally compressed by utilizing the spatial and temporal redundancy of the moving image. Here, in general, as a method of utilizing the redundancy in the spatial direction, conversion to the frequency domain is used. Further, as a method of utilizing the redundancy in the time direction, an inter-picture prediction (hereinafter referred to as "inter-prediction") coding process is used. In the inter-prediction coding process, when encoding a certain picture, the encoded picture that is in front of or behind the coded picture in the order of display time is used as a reference picture. Then, the motion vector is derived by detecting the motion of the coded target picture with respect to the reference picture. Then, by calculating the difference between the predicted image data obtained by performing motion compensation based on the derived motion vector and the image data of the picture to be encoded, the redundancy in the time direction is removed (for example, non-redundancy). See Patent Document 1). Here, in motion detection, the difference value between the coded block in the coded picture and the block in the reference picture is calculated, and the block in the reference picture having the smallest difference value is determined as the reference block. Then, the motion vector is detected by using the coded block and the reference block.
<p num="0003"><nplcit num="1"><text>ITU-T Recommendation H.264 "Advanced video coding for generic audiovisual services", March 2010</text></nplcit><nplcit num="2"><text>JCT-VC, WD3: Working Draft 3 of High-Efficiency Video Coding, JCTVC-E603, March 2011.</text></nplcit></p>
<p num="0004"> However, in the above-mentioned conventional technique, it is desired to improve the coding efficiency in image coding and decoding using inter-prediction.</p><p num="0005"> Therefore, an object of the present invention is to provide an image coding method and an image decoding method capable of improving the coding efficiency in image coding and decoding using inter-prediction.</p>
<p num="0006"> The image coding method according to one aspect of the present invention is an image coding method that generates a bit stream by coding an image block by block, and is a block that is spatially or temporally adjacent to a block to be coded. Based on the prediction direction, motion vector, and reference picture index used for encoding, the merge candidates that are candidates for the prediction direction, motion vector, and reference picture index used for encoding the coded block are first merged. A first derivation step for deriving as a candidate, a second derivation step for deriving a merge candidate having a predetermined vector as a motion vector as a second merge candidate, and the derived first merge candidate and the second merge candidate. It includes a selection step of selecting a merge candidate used for encoding the coded block, and a coding step of adding an index for identifying the selected merge candidate to the bit stream.</p><p num="0007"> It should be noted that these general or specific aspects may be realized by a recording medium such as a system, a method, an integrated circuit, a computer program, or a computer-readable CD-ROM (Compact Disc Read Only Memory). It may be realized by any combination of methods, integrated circuits, computer programs and recording media.</p>
<p num="0008"> According to one aspect of the present invention, it is possible to improve the coding efficiency in image coding and decoding using inter-prediction.</p>
<figref num="1A">FIG. 1A is a diagram for explaining an example of a reference picture list in the B picture.</figref><figref num="1B">FIG. 1B is a diagram showing an example of a reference picture list in the prediction direction 0 in the B picture.</figref><figref num="1C">FIG. 1C is a diagram showing an example of a reference picture list in the prediction direction 1 in the B picture.</figref><figref num="2">FIG. 2 is a diagram for explaining a motion vector in the time prediction motion vector mode.</figref><figref num="3">FIG. 3 is a diagram showing an example of motion vectors of adjacent blocks used in the merge mode.</figref><figref num="4">FIG. 4 is a diagram for explaining an example of the merge block candidate list.</figref><figref num="5">FIG. 5 is a diagram showing the relationship between the merge block candidate size and the bit string assigned to the merge block index.</figref><figref num="6">FIG. 6 is a flowchart showing an example of the coding process when the merge mode is used.</figref><figref num="7">FIG. 7 is a flowchart showing the decoding process when the merge mode is used.</figref><figref num="8">FIG. 8 is a diagram showing the syntax for attaching a merge block index to a bitstream.</figref><figref num="9">FIG. 9 is a block diagram showing a configuration of the image coding apparatus according to the first embodiment.</figref><figref num="10">FIG. 10 is a flowchart showing a processing operation of the image coding apparatus according to the first embodiment.</figref><figref num="11">FIG. 11 is a diagram showing an example of a merge block candidate list according to the first embodiment.</figref><figref num="12">FIG. 12 is a flowchart showing the calculation process of the merge block candidate and the merge block candidate list size in the first embodiment.</figref><figref num="13">FIG. 13 is a flowchart showing a process of determining whether the merge block candidate in the first embodiment is a mergeable candidate and updating the number of mergeable candidates.</figref><figref num="14">FIG. 14 is a flowchart showing the additional processing of the zero merge block candidate in the first embodiment.</figref><figref num="15">FIG. 15 is a flowchart showing a determination process for determining whether or not a zero merge block candidate exists in the first embodiment.</figref><figref num="16">FIG. 16 is a diagram showing an example of the zero merge block in the first embodiment.</figref><figref num="17">FIG. 17 is a flowchart showing a process related to selection of merge block candidates in the first embodiment.</figref><figref num="18">FIG. 18 is a block diagram showing a configuration of the image coding apparatus according to the second embodiment.</figref><figref num="19">FIG. 19 is a flowchart showing a processing operation of the image coding apparatus according to the second embodiment.</figref><figref num="20">FIG. 20 is a block diagram showing a configuration of the image decoding device according to the third embodiment.</figref><figref num="21">FIG. 21 is a flowchart showing a processing operation of the image decoding apparatus according to the third embodiment.</figref><figref num="22">FIG. 22 is a block diagram showing a configuration of the image decoding device according to the fourth embodiment.</figref><figref num="23">FIG. 23 is a flowchart showing the processing operation of the image decoding apparatus according to the fourth embodiment.</figref><figref num="24">FIG. 24 is a block diagram showing a configuration of the image coding apparatus according to the fifth embodiment.</figref><figref num="25">FIG. 25 is a flowchart showing the processing operation of the image coding apparatus according to the fifth embodiment.</figref><figref num="26">FIG. 26 is a diagram showing an example of the merge block candidate list according to the fifth embodiment.</figref><figref num="27">FIG. 27 is a flowchart showing the calculation process of the merge block candidate and the merge block candidate list size in the fifth embodiment.</figref><figref num="28">FIG. 28 is a flowchart showing the update process of the number of mergeable candidates in the fifth embodiment.</figref><figref num="29">FIG. 29 is a flowchart showing the process of adding a new candidate in the fifth embodiment.</figref><figref num="30">FIG. 30 is a block diagram showing the configuration of the image coding apparatus according to the sixth embodiment.</figref><figref num="31">FIG. 31 is a flowchart showing a processing operation of the image coding apparatus according to the sixth embodiment.</figref><figref num="32">FIG. 32 is a block diagram showing the configuration of the image decoding device according to the seventh embodiment.</figref><figref num="33">FIG. 33 is a flowchart showing the processing operation of the image decoding apparatus according to the seventh embodiment.</figref><figref num="34">FIG. 34 is a flowchart showing the merge block candidate list size setting process according to the seventh embodiment.</figref><figref num="35">FIG. 35 is a flowchart showing the calculation process of the merge block candidate in the seventh embodiment.</figref><figref num="36">FIG. 36 is a diagram showing an example of syntax when adding a merge block index to a bit stream.</figref><figref num="37">FIG. 37 is a diagram showing an example of syntax when the merge block candidate list size is fixed to the maximum number of merge block candidates.</figref><figref num="38">FIG. 38 is a block diagram showing a configuration of the image decoding device according to the eighth embodiment.</figref><figref num="39">FIG. 39 is a flowchart showing the processing operation of the image decoding apparatus according to the eighth embodiment.</figref><figref num="40">FIG. 40 is an overall configuration diagram of a content supply system that realizes a content distribution service.</figref><figref num="41">FIG. 41 is an overall configuration diagram of the digital broadcasting system.</figref><figref num="42">FIG. 42 is a block diagram showing a configuration example of a television.</figref><figref num="43">FIG. 43 is a block diagram showing a configuration example of an information reproduction / recording unit that reads / writes information to / from a recording medium such as an optical disc.</figref><figref num="44">FIG. 44 is a diagram showing a structural example of a recording medium which is an optical disc.</figref><figref num="45A">FIG. 45A is a diagram showing an example of a mobile phone.</figref><figref num="45B">FIG. 45B is a block diagram showing a configuration example of a mobile phone.</figref><figref num="46">FIG. 46 is a diagram showing the structure of the multiplexed data.</figref><figref num="47">FIG. 47 is a diagram schematically showing how each stream is multiplexed in the multiplexed data.</figref><figref num="48">FIG. 48 is a more detailed diagram of how the video stream is stored in the PES packet sequence.</figref><figref num="49">FIG. 49 is a diagram showing the structure of TS packets and source packets in the multiplexed data.</figref><figref num="50">FIG. 50 is a diagram showing a data structure of PMT.</figref><figref num="51">FIG. 51 is a diagram showing an internal configuration of multiplexed data information.</figref><figref num="52">FIG. 52 is a diagram showing an internal configuration of stream attribute information.</figref><figref num="53">FIG. 53 is a diagram showing steps for identifying video data.</figref><figref num="54">FIG. 54 is a block diagram showing a configuration example of an integrated circuit that realizes the moving image coding method and the moving image decoding method of each embodiment.</figref><figref num="55">FIG. 55 is a diagram showing a configuration for switching the drive frequency.</figref><figref num="56">FIG. 56 is a diagram showing steps for identifying video data and switching the drive frequency.</figref><figref num="57">FIG. 57 is a diagram showing an example of a look-up table in which the video data standard and the drive frequency are associated with each other.</figref><figref num="58A">FIG. 58A is a diagram showing an example of a configuration in which the module of the signal processing unit is shared.</figref><figref num="58B">FIG. 58B is a diagram showing another example of the configuration in which the module of the signal processing unit is shared.</figref>
(Knowledge that became the basis of the present invention) In the moving image coding method called H.264, which has already been standardized, three types of picture types, I picture, P picture, and B picture, are used to compress the amount of information.
The I-picture is not encoded by the inter-predictive coding process. That is, the I picture is encoded by the intra-picture prediction (hereinafter referred to as "intra-prediction") coding process. The P-pictures are inter-predictively coded with reference to one already encoded picture in front of or behind the coded picture in display time order. The B-pictures are inter-predictively coded with reference to two already encoded pictures in front of or behind the coded picture in display time order.
In inter-predictive coding, a reference picture list is generated to identify the reference picture. The reference picture list is a list in which the reference picture index is assigned to the encoded reference picture referenced by the inter-prediction. For example, in the B picture, since the encoding can be performed by referring to the two pictures, two reference picture lists (L0 and L1) are generated.
FIG. 1A is a diagram for explaining an example of a reference picture list in the B picture. FIG. 1B shows an example of the reference picture list 0 (L0) in the prediction direction 0 in the bidirectional prediction. Here, in the reference picture list 0, the value 0 of the reference picture index 0 is assigned to the reference picture 0 in the display order 2. Further, the value 1 of the reference picture index 0 is assigned to the reference picture 1 in the display order 1. Further, the value 2 of the reference picture index 0 is assigned the reference picture 2 of the display order 0. That is, the reference picture having a smaller value is assigned to the reference picture that is closer in time to the coded picture in the display order.
On the other hand, FIG. 1C shows an example of the reference picture list 1 (L1) in the prediction direction 1 in the bidirectional prediction. Here, in the reference picture list 1, the value 0 of the reference picture index 1 is assigned to the reference picture 1 in the display order 1. Further, the value 1 of the reference picture index 1 is assigned to the reference picture 0 of the display order 2. Further, the value 2 of the reference picture index 2 is assigned to the reference picture 2 in the display order 0.
In this way, it is possible to assign a different reference picture index value to each reference picture for each prediction direction (reference pictures 0 and 1 in FIG. 1A), or to assign the same reference picture index value (the same reference picture index value). Reference picture 2) in Figure 1A.
Further, in the moving image coding method (Non-Patent Document 1) called H.264, a motion vector detection mode is used as a coding mode for inter-prediction of each coded block in the B picture. In the motion vector detection mode, the difference value between the predicted image data and the image data of the coded target block and the motion vector used for generating the predicted image data are encoded. Further, in the motion vector detection mode, bidirectional prediction and one-way prediction can be selected as the prediction direction. In bidirectional prediction, a prediction image is generated by referring to two already encoded pictures in front of or behind the picture to be encoded. In one-way prediction, a prediction image is generated by referring to one already encoded picture in front or behind.
Further, in the moving image coding method called H.264, a coding mode called a time prediction motion vector mode can be selected when deriving a motion vector in B picture coding. The inter-prediction coding method in the time-predicted motion vector mode will be described with reference to FIG. FIG. 2 is a diagram for explaining a motion vector in the time prediction motion vector mode. Specifically, FIG. 2 shows a case where the block a of the picture B2 is encoded in the time prediction motion vector mode.
Here, the motion vector vb used for encoding the block b (hereinafter referred to as co-located block) at the same position as the block a in the picture P3 which is the reference picture behind the picture B2 is It's being used. The motion vector vb is the motion vector used when block b was encoded with reference to picture P1.
Using the motion vector parallel to the motion vector vb, two reference blocks for block a are obtained from the forward reference picture, picture P1, and the backward reference picture, picture P3. Then, the block a is encoded by performing a two-way prediction based on the two acquired reference blocks. That is, the motion vector used when encoding the block a is the motion vector va1 for the picture P1 and the motion vector va2 for the picture P3.
Further, a merge mode is being studied as an inter-prediction mode for each coded block in a B picture or a P picture (Non-Patent Document 2). In the merge mode, the coded block is coded by copying the prediction direction, motion vector, and reference picture index used to code the adjacent blocks of the coded block. At this time, the index of the adjacent block used for copying is attached to the bit stream. This allows the decoding side to select the motion direction, motion vector, and reference picture index used for coding. A specific example will be described with reference to FIG.
FIG. 3 is a diagram showing an example of motion vectors of adjacent blocks used in the merge mode. In FIG. 3, the adjacent block A is a coded block to the left of the coded block. The adjacent block B is a coded block that is adjacent to the coded block. The adjacent block C is a coded block adjacent to the upper right of the coded block. The adjacent block D is a coded block adjacent to the lower left of the coded block.
Further, the adjacent block A is a block encoded by one-way prediction in the prediction direction 0. The adjacent block A has a motion vector MvL0_A in the prediction direction 0 as a motion vector with respect to the reference picture indicated by the reference picture index RefL0_A in the prediction direction 0. Here, MvL0 indicates a motion vector that refers to the reference picture specified by the reference picture list 0 (L0). Further, MvL1 indicates a motion vector that refers to the reference picture specified by the reference picture list 1 (L1).
Further, the adjacent block B is a block encoded by the one-way prediction in the prediction direction 1. The adjacent block B has a motion vector MvL1_B in the prediction direction 1 as a motion vector with respect to the reference picture indicated by the reference picture index RefL1_B in the prediction direction 1.
The adjacent block C is a block encoded by the intra prediction.
Further, the adjacent block D is a block encoded by one-way prediction in the prediction direction 0. The adjacent block D has a motion vector MvL0_D in the prediction direction 0 as a motion vector with respect to the reference picture indicated by the reference picture index RefL0_D in the prediction direction 0.
In such a case, for example, the prediction direction, motion vector and reference picture index of adjacent blocks A to D, and the prediction direction, motion vector and reference picture index by the time prediction motion vector mode obtained by using the co-located block. From among these, the one with the highest coding efficiency is selected as the prediction direction, motion vector, and reference picture index of the block to be coded. Then, a merge block index representing a block of the selected prediction direction, motion vector, and reference picture index is attached to the bitstream.
For example, when the adjacent block A is selected, the block to be encoded is encoded using the motion vector MvL0_A in the prediction direction 0 and the reference picture index RefL0_A. Then, only the value 0 of the merge block index indicating that the adjacent block A is used as shown in FIG. 4 is attached to the bit stream. As a result, the amount of information in the prediction direction, the motion vector, and the reference picture index can be reduced.
Further, as shown in FIG. 4, in the merge mode, candidates that cannot be used for coding (hereinafter referred to as unmergeable candidates), or combinations of prediction direction, motion vector, and reference picture index are used. Candidates that match each other (hereinafter referred to as "duplicate candidates") are deleted from the merge block candidates.
By reducing the number of merge block candidates in this way, the amount of code allocated to the merge block index is reduced. Here, the fact that merging is not possible means that the merge block candidate is (1) a block encoded by intra-prediction, and (2) a slice containing a block to be encoded or a block outside the picture boundary. It indicates that there is, or (3) it is a block that has not been encoded yet.
In the example of FIG. 4, the adjacent block C is encoded by the intra prediction. Therefore, the merge block candidate with the merge block index 3 is a non-mergeable candidate and is deleted from the merge block candidate list. Further, the adjacent block D has the same prediction direction, motion vector, and reference picture index as the adjacent block A. Their order of merging block candidate merging block index 4 is deleted from the merge block candidate list. As a result, the number of merge block candidates is finally set to 3, and the list size of the merge block candidate list is set to 3.
The merge block index is assigned a bit string and is variable-length encoded, as shown in FIG. 5, according to the size of the merge block candidate list size. In this way, in the merge mode, the code amount is reduced by changing the bit string assigned to the merge mode index according to the size of the merge block candidate list size.
FIG. 6 is a flowchart showing an example of the coding process when the merge mode is used. In step S1001, the motion vector, reference picture index, and prediction direction of the merge block candidate are obtained from the adjacent block and the co-located block. In step S1002, duplicate and non-mergeable candidates are removed from the merge block candidates. In step S1003, the number of merge block candidates after the deletion process is set to the merge block candidate list size. In step S1004, the merge block index used for coding the coded block is determined. In step S1005, the determined merge block index is variable length encoded using the bit string determined by the merge block candidate list size.
FIG. 7 is a flowchart showing an example of the decoding process when the merge mode is used. In step S2001, the motion vector, the reference picture index, and the prediction direction of the merge block candidate are obtained from the adjacent block and the co-located block. In step S2002, duplicate and non-mergeable candidates are removed from the merge block candidates. In step S2003, the number of merge block candidates after the deletion process is set to the merge block candidate list size. In step S2004, the merge block index used to decrypt the block to be decrypted is decoded from the bitstream using the merge block candidate list size. In step S2005, a predicted image is generated and a decoding process is performed using the merge block candidates indicated by the decrypted merge block index.
Figure 8 shows the syntax for adding a merge block index to a bitstream. In Figure 8, merge_idx represents the merge block index. merge_flag represents the merge flag. NumMergeCand represents the merge block candidate list size. In this NumMergeCand, the number of merge block candidates after removing unmergeable candidates and duplicate candidates from the merge block candidates is set.
As described above, the image is encoded or decoded using the merge mode.
However, in the above merge mode, the motion vector when encoding the coded target block is calculated from the merge block candidates and the like adjacent to the coded target block. Therefore, for example, when the adjacent block is the animal body region and the coded block is the stationary region, the motion vector that can be used in the merge mode is affected by the animal body region, so that the prediction accuracy of the merge mode May not improve and the coding efficiency may decrease.
Therefore, the image coding method according to one aspect of the present invention is an image coding method that generates a bit stream by coding an image block by block, and is spatially or temporally adjacent to a block to be coded. Based on the prediction direction, motion vector, and reference picture index used to encode the block to be encoded, a merge candidate that is a candidate for the prediction direction, motion vector, and reference picture index used to encode the coded block is selected. 1 A first derivation step for deriving as a merge candidate, a second derivation step for deriving a merge candidate having a predetermined vector as a motion vector as a second merge candidate, and the derived first merge candidate and the second merge candidate. From the merge candidates, a selection step of selecting a merge candidate to be used for encoding the coded block and a coding step of adding an index for identifying the selected merge candidate to the bit stream are performed. Including.
According to this, a merge candidate having a predetermined vector as a motion vector can be derived as a second merge candidate. Therefore, for example, a merge candidate having a motion vector for a stationary region can be derived as a second merge candidate. That is, it is possible to efficiently encode the coded target block having a predetermined motion, and it is possible to improve the coding efficiency.
For example, in the second derivation step, the second merge candidate may be derived for each reference picture that can be referred to.
According to this, the second merge candidate can be derived for each reference picture. Therefore, the types of merge candidates can be increased, and the coding efficiency can be further improved.
For example, the predetermined vector may be a zero vector.
According to this, since the predetermined vector is a zero vector, it is possible to derive a merge candidate having a motion vector for the rest region. Therefore, when the coded block is a rest region, the coding efficiency can be improved.
For example, the image coding method further includes a determination step of determining the maximum number of merge candidates and a determination step of determining whether the number of derived first merge candidates is smaller than the maximum number. In the second derivation step, the second merge candidate may be derived when it is determined that the number of the first merge candidates is smaller than the maximum number.
According to this, when it is determined that the first merge candidate is smaller than the maximum number, the second merge candidate can be derived. Therefore, the number of merge candidates can be increased within a range not exceeding the maximum number, and the coding efficiency can be improved.
For example, in the coding step, the index may be encoded using the determined maximum number and the encoded index may be added to the bitstream.
According to this, the index for identifying the merge candidate can be encoded using the determined maximum number. That is, the index can be encoded independently of the number of merge candidates that are actually derived. Therefore, even if the information necessary for deriving the merge candidate (for example, information such as the co-located block) is lost, the index can be decoded on the decoding side, and the error tolerance can be improved. Further, on the decoding side, the index can be decoded independently of the number of merge candidates actually derived. That is, on the decoding side, the index decoding process can be performed without waiting for the merge candidate derivation process. That is, it is possible to generate a bit stream capable of performing the merge candidate derivation process and the index decoding process in parallel.
For example, in the coding step, information indicating the determined maximum number may be further added to the bitstream.
According to this, information indicating the determined maximum number can be added to the bit stream. Therefore, the maximum number can be switched in an appropriate unit, and the coding efficiency can be improved.
For example, in the first derivation step, a merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index does not overlap with the first merge candidate already derived may be derived as the first merge candidate.
According to this, the merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index overlaps with the first merge candidate already derived can be excluded from the first merge candidate. As a result, the number of second merge candidates can be increased, and the types of combinations of prediction direction, motion vector, and reference picture index that can be selected as merge candidates can be increased. Therefore, it is possible to further improve the coding efficiency.
For example, the image coding method further includes a switching step of switching the coding process to the first coding process conforming to the first standard or the second coding process conforming to the second standard, and the switching step. When the coding process is switched to the first coding process, including an additional step of adding identification information indicating the first standard or the second standard to which the coding process conforms to the bit stream. As the first coding process, the first derivation step, the second derivation step, the selection step, and the coding step may be performed.
According to this, it is possible to switch between the first coding process conforming to the first standard and the second coding process conforming to the second standard.
Further, the image decoding method according to one aspect of the present invention is an image decoding method for decoding a coded image included in a bit stream for each block, and decoding a block spatially or temporally adjacent to a block to be decoded. Based on the prediction direction, motion vector, and reference picture index used in the above, the merge candidate that is a candidate for the prediction direction, motion vector, and reference picture index used for decoding the decoding target block is derived as the first merge candidate. 1 Derivation step, a second derivation step of deriving a merge candidate having a predetermined vector as a motion vector as a second merge candidate, and an acquisition step of acquiring an index for identifying the merge candidate from the bit stream. , A selection step of selecting a merge candidate to be used for decoding the decoding target block from the first merge candidate and the second merge candidate based on the acquired index is included.
According to this, a merge candidate having a predetermined vector as a motion vector can be derived as a second merge candidate. Therefore, for example, a merge candidate having a motion vector for a stationary region can be derived as a second merge candidate. That is, a block having a predetermined motion can appropriately decode a bit stream encoded efficiently, and a bit stream with improved coding efficiency can be appropriately decoded.
For example, in the second derivation step, the second merge candidate may be derived for each reference picture that can be referred to.
According to this, the second merge candidate can be derived for each reference picture. Therefore, the types of merge candidates can be increased, and a bit stream with further improved coding efficiency can be appropriately decoded.
For example, the predetermined vector may be a zero vector.
According to this, since the predetermined vector is a zero vector, it is possible to derive a merge candidate having a motion vector for the rest region. Therefore, it is possible to appropriately decode the bit stream with improved coding efficiency.
For example, the image decoding method further includes a determination step of determining the maximum number of merge candidates and a determination step of determining whether or not the number of the derived first merge candidates is smaller than the maximum number. In the second derivation step, the second merge candidate may be derived when it is determined that the number of the derived first merge candidates is smaller than the maximum number.
According to this, when it is determined that the number of the first merge candidates is smaller than the maximum number, the second merge candidates can be derived. Therefore, the number of merge candidates can be increased within a range not exceeding the maximum number, and a bit stream with improved coding efficiency can be appropriately decoded.
For example, in the acquisition step, the index may be acquired by decoding the encoded index added to the bitstream using the determined maximum number.
According to this, the index for identifying the merge candidate can be decoded using the determined maximum number. That is, the index can be decrypted independently of the number of merge candidates actually derived. Therefore, even if the information necessary for deriving the merge candidate (for example, information such as a co-located block) is lost, the index can be decoded and the error tolerance can be improved. Further, the index decoding process can be performed without waiting for the merge candidate derivation process, and the merge candidate derivation process and the index decoding process can be performed in parallel.
For example, in the determination step, the maximum number may be determined based on the information indicating the maximum number added to the bit stream.
According to this, the maximum number can be determined based on the information added to the bit stream. Therefore, it is possible to decode the encoded image by switching the maximum number in an appropriate unit.
For example, in the first derivation step, a merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index does not overlap with the first merge candidate already derived may be derived as the first merge candidate.
According to this, the merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index overlaps with the first merge candidate already derived can be excluded from the first merge candidate. As a result, the number of second merge candidates can be increased, and the types of combinations of prediction direction, motion vector, and reference picture index that can be selected as merge candidates can be increased. Therefore, it is possible to appropriately decode the bit stream with further improved coding efficiency.
For example, the image decoding method further applies to the first decoding process conforming to the first standard, or the second standard, depending on the identification information indicating the first standard or the second standard added to the bit stream. The compliant second decoding process includes a switching step for switching the decoding process, and when the decoding process is switched to the first decoding process, the first decoding process includes the first derivation step and the second derivation process. The step, the acquisition step, and the selection step may be performed.
According to this, it is possible to switch between the first decoding process conforming to the first standard and the second decoding process conforming to the second standard.
It should be noted that these general or specific embodiments may be realized in a recording medium such as a system, method, integrated circuit, computer program or computer readable CD-ROM, system, method, integrated circuit, computer program. Alternatively, it may be realized by any combination of recording media.
Hereinafter, the image coding device and the image decoding device according to one aspect of the present invention will be specifically described with reference to the drawings.
It should be noted that all of the embodiments described below show a specific example of the present invention. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, step order, and the like shown in the following embodiments are examples, and are not intended to limit the present invention. Further, among the components in the following embodiments, the components not described in the independent claims indicating the highest level concept are described as arbitrary components.
(Embodiment 1) FIG. 9 is a block diagram showing a configuration of the image coding apparatus according to the first embodiment. The image coding device 100 generates a bit stream by encoding the image block by block.
As shown in FIG. 9, the image coding apparatus 100 includes a subtraction unit 101, an orthogonal conversion unit 102, a quantization unit 103, an inverse quantization unit 104, an inverse orthogonal conversion unit 105, and an addition unit 106. Block memory 107, frame memory 108, intra-prediction unit 109, inter-prediction unit 110, inter-prediction control unit 111, picture type determination unit 112, switch 113, merge block candidate calculation unit 114, and colPic memory. It includes 115 and a variable length coding unit 116.
The subtraction unit 101 generates prediction error data by subtracting the prediction image data from the input image data included in the input image string for each block.
The orthogonal conversion unit 102 converts the generated prediction error data from the image region to the frequency domain.
The quantization unit 103 performs a quantization process on the prediction error data converted into the frequency domain.
The inverse quantization unit 104 performs an inverse quantization process on the prediction error data quantized by the quantization unit 103.
The inverse orthogonal conversion unit 105 converts the frequency domain to the image domain of the inverse quantization processed prediction error data.
The addition unit 106 generates reconstructed image data by adding the prediction image data and the prediction error data that has been inversely quantized by the inverse orthogonal conversion unit 105 for each block.
Reconstructed image data is stored in the block memory 107 in block units.
Reconstructed image data is stored in the frame memory 108 in frame units.
The picture type determination unit 112 determines whether to encode the input image data with a picture type of I picture, B picture, or P picture. Then, the picture type determination unit 112 generates picture type information indicating the determined picture type.
The intra prediction unit 109 generates the intra prediction image data of the coded block by performing the intra prediction using the reconstructed image data of each block stored in the block memory 107.
The inter-prediction unit 110 performs inter-prediction using the reconstructed image data for each frame stored in the frame memory 108 and the motion vector derived by motion detection or the like, thereby performing inter-prediction image of the coded block. Generate data.
The switch 113 outputs the intra prediction image data generated by the intra prediction unit 109 to the subtraction unit 101 and the addition unit 106 as the prediction image data of the code target block when the coded block is intra-predicted and encoded. To do. On the other hand, when the coded block is inter-predicted and encoded, the switch 113 uses the inter-predicted image data generated by the inter-predicted unit 110 as the predicted image data of the coded block in the subtracting unit 101 and the adding unit 106. Output to.
The merge block candidate calculation unit 114 uses the motion vector of the adjacent block of the coded block and the motion vector of the co-located block stored in the colPic memory 115 (colPic information) to merge the merge mode. Derive block candidates. Further, the merge block candidate calculation unit 114 adds the derived merge block candidates to the merge block candidate list.
Further, the merge block candidate calculation unit 114 derives a merge block candidate (hereinafter, referred to as "zero merge block candidate") having a prediction direction, a motion vector, and a reference picture index for a stationary region as a new candidate by a method described later. To do. Then, the merge block candidate calculation unit 114 adds the derived zero merge block candidate to the merge block candidate list as a new merge block candidate. Further, the merge block candidate calculation unit 114 calculates the number of merge block candidates.
Further, the merge block candidate calculation unit 114 assigns the value of the merge block index to each of the derived merge block candidates. Then, the merge block candidate calculation unit 114 transmits the merge block candidate and the merge block index to the inter-prediction control unit 111. Further, the merge block candidate calculation unit 114 transmits the calculated number of merge block candidates to the variable length coding unit 116.
The inter-prediction control unit 111 has the smallest prediction mode among the prediction mode (motion detection mode) using the motion vector derived by motion detection and the prediction mode (merge mode) using the motion vector derived from the merge block candidate. Select the prediction mode that gives the error. Further, the inter-prediction control unit 111 transmits a merge flag indicating whether the prediction mode is the merge mode to the variable-length coding unit 116. Further, the inter-prediction control unit 111 transmits the merge block index corresponding to the determined merge block candidate to the variable length coding unit 116 when the merge mode is selected as the prediction mode. Further, the inter-prediction control unit 111 transfers colPic information including the motion vector of the coded block to the colPic memory 115.
The variable-length coding unit 116 generates a bit stream by performing variable-length coding processing on the quantized prediction error data, the merge flag, and the picture type information. Further, the variable length coding unit 116 sets the number of merge block candidates to the merge block candidate list size. Then, the variable-length coding unit 116 assigns a bit string corresponding to the merge block candidate list size to the merge block index used for coding, and performs variable-length coding on the assigned bit string.
FIG. 10 is a flowchart showing a processing operation of the image coding apparatus 100 according to the first embodiment.
In step S101, the merge block candidate calculation unit 114 derives the merge block candidate from the adjacent block and the co-located block of the coded block. Further, the merge block candidate calculation unit 114 calculates the merge block candidate list size by a method described later.
For example, in the case shown in FIG. 3, the merge block candidate calculation unit 114 selects adjacent blocks A to D as merge block candidates. Further, the merge block candidate calculation unit 114 calculates a co-located merge block having a motion vector, a reference picture index, and a prediction direction calculated from the motion vector of the co-located block in the time prediction mode as merge block candidates.
The merge block candidate calculation unit 114 assigns a merge block index to each merge block candidate as shown in FIG. 11 (a). Then, the merge block candidate calculation unit 114 deletes unmergeable candidates and duplicate candidates, and adds new zero merge block candidates by a method described later, thereby performing a merge block candidate list as shown in FIG. 11 (b). , And the merge block candidate list size is calculated.
The smaller the value of the merge block index, the shorter the code is assigned. That is, when the value of the merge block index is small, the amount of information required for the merge block index is small.
On the other hand, as the value of the merge block index increases, the amount of information required for the merge block index increases. Therefore, if a merge block index with a smaller value is assigned to a merge block candidate that is likely to have a more accurate motion vector and reference picture index, the coding efficiency will be higher.
Therefore, the merge block candidate calculation unit 114 may, for example, measure the number of times selected as a merge block for each merge block candidate, and assign a merge block index having a small value to a block having a large number of times. Specifically, it is conceivable to specify the merge block selected in the adjacent block and reduce the value of the merge block index for the specified merge block when coding the target block.
If the merge block candidate does not have information such as a motion vector (a block encoded by intra prediction, a block located outside the boundary of a picture or slice, or is still encoded). It cannot be used for encoding if it is a block that has not been used.
In the present embodiment, the merge block candidate that cannot be used for encoding is referred to as an unmergeable candidate. Further, a merge block candidate that can be used for encoding is called a mergeable candidate. Further, among a plurality of merge block candidates, a candidate whose motion vector, reference picture index, and prediction direction all match with any other merge block candidate is called a duplicate candidate.
In the case of FIG. 3, the adjacent block C is a block encoded by the intra prediction, and is therefore a candidate that cannot be merged. Further, the adjacent block D is a duplicate candidate because the motion vector, the reference picture index, and the prediction direction all match the adjacent block A.
In step S102, the inter-prediction control unit 111 determines the prediction error of the predicted image generated by using the motion vector derived by the motion detection and the prediction error of the predicted image generated by using the motion vector obtained from the merge block candidate. And are compared by the method described later, and the prediction mode is selected. Here, if the selected prediction mode is the merge mode, the inter-prediction control unit 111 sets the merge flag to 1, and if not, sets the merge flag to 0.
In step S103, it is determined whether the merge flag is 1 (that is, whether the prediction mode is the merge mode).
Here, if the determination result in step S103 is true (Yes in S103), the variable length encoding unit 116 adds the merge flag to the bit stream in step S104. Further, in step S105, the variable-length coding unit 116 allocates a bit string corresponding to the merge block candidate list size as shown in FIG. 5 to the merge block index of the merge block candidates used for coding. Then, the variable-length coding unit 116 performs variable-length coding on the assigned bit string.
On the other hand, if the determination result in step S103 is false (No in S103), in step S106, the variable length coding unit 116 adds the merge flag and the motion detection vector mode information to the bit stream.
In the present embodiment, as shown in FIG. 11A, "0" is assigned as the value of the merge block index corresponding to the adjacent block A. In addition, "1" is assigned as the value of the merge block index corresponding to the adjacent block B. Also, "2" is assigned as the value of the merge block index corresponding to the co-located merge block. In addition, "3" is assigned as the value of the merge block index corresponding to the adjacent block C. In addition, "4" is assigned as the value of the merge block index corresponding to the adjacent block D.
Note that the method of assigning the value of the merge block index is not necessarily limited to this example. For example, the variable-length encoding unit 116 assigns a small value to the original merge block candidate and a large value to the new zero merge block candidate when a new zero merge block candidate is added by using the method described later. You may assign a value. That is, the variable-length encoding unit 116 may assign a merge block index with a small value in preference to the original merge block candidate.
Also, the merge block candidates are not necessarily limited to the positions of adjacent blocks A to D. For example, an adjacent block located above the lower left adjacent block D may be used as a merge block candidate. Also, not all adjacent blocks need to be used as merge block candidates. For example, only adjacent blocks A and B may be used as merge block candidates.
Further, in the present embodiment, in step S105 of FIG. 10, the variable length coding unit 116 adds the merge block index to the bit stream, but it is not always necessary to add the merge block index to the bit stream. For example, the variable-length encoding unit 116 does not have to add the merge block index to the bitstream when the merge block candidate list size is 1. As a result, the amount of information in the merge block index can be reduced.
FIG. 12 is a flowchart showing the detailed processing of step S101 of FIG. Specifically, FIG. 12 shows a method of calculating the merge block candidate and the merge block candidate list size. Hereinafter, FIG. 12 will be described.
In step S111, the merge block candidate calculation unit 114 determines whether or not the merge block candidate [N] is a mergeable candidate by a method described later.
Here, N is an index value for representing each merge block candidate. In this embodiment, N takes a value from 0 to 4. Specifically, the adjacent block A in FIG. 3 is assigned to the merge block candidate [0]. In addition, the adjacent block B in FIG. 3 is assigned to the merge block candidate [1]. In addition, a co-located merge block is assigned to the merge block candidate [2]. In addition, the adjacent block C in FIG. 3 is assigned to the merge block candidate [3]. In addition, the adjacent block D in FIG. 3 is assigned to the merge block candidate [4].
In step S112, the merge block candidate calculation unit 114 acquires the motion vector, the reference picture index, and the prediction direction of the merge block candidate [N] and adds them to the merge block candidate list.
In step S113, the merge block candidate calculation unit 114 searches for unmergeable candidates and duplicate candidates from the merge block candidate list and deletes them, as shown in FIG.
In step S114, the merge block candidate calculation unit 114 adds a new zero merge block candidate to the merge block candidate list by a method described later. Here, when adding a new zero merge block candidate, the value of the merge block index may be reassigned so that the merge block index with a smaller value is assigned in preference to the original merge block candidate. That is, the merge block candidate calculation unit 114 may reassign the value of the merge block index so that a new zero merge block candidate is assigned a merge block index having a large value. As a result, the amount of code in the merge block index can be reduced.
In step S115, the merge block candidate calculation unit 114 sets the number of merge block candidates after adding zero merge blocks to the merge block candidate list size. In the example of FIG. 11, the number of merge block candidates is calculated as "5" and the merge block candidate list size is set to "5" by the method described later.
The new zero merge block candidate in step S114 is a candidate newly added to the merge block candidate when the number of merge block candidates does not reach the maximum number of merge block candidates by the method described later. As described above, when the number of merge block candidates does not reach the maximum number of merge block candidates, the image coding apparatus 100 can improve the coding efficiency by adding a new zero merge block candidate.
FIG. 13 is a flowchart showing the detailed processing of step S111 of FIG. Specifically, FIG. 13 shows a method of determining whether the merge block candidate [N] is a mergeable candidate and updating the number of mergeable candidates. Hereinafter, FIG. 13 will be described.
In step S121, the merge block candidate calculation unit 114 positions the merge block candidate [N] outside the slice or picture boundary containing (1) the block encoded by the intra prediction or (2) the block to be encoded. Determine if it is a block to be used, or (3) a block that has not yet been encoded.
Here, if the determination result in step S121 is true (Yes in S121), the merge block candidate calculation unit 114 sets the merge block candidate [N] as a non-mergeable candidate in step S122. On the other hand, if the determination result in step S121 is false (No in S121), the merge block candidate calculation unit 114 sets the merge block candidate [N] as a mergeable candidate in step S123.
FIG. 14 is a flowchart showing the detailed processing of step S114 of FIG. Specifically, FIG. 14 shows how to add zero merge block candidates. Hereinafter, FIG. 14 will be described.
In step S131, the merge block candidate calculation unit 114 determines whether or not the number of merge block candidates is smaller than the maximum number of merge block candidates. That is, the merge block candidate calculation unit 114 determines whether or not the number of merge block candidates has reached the maximum number of merge block candidates.
Here, if the determination result in step S131 is true (Yes in S131), in step S132, the merge block candidate calculation unit 114 has a new zero merge block candidate that can be added to the merge block candidate list as a merge block candidate. Whether or not it is determined by the method described later. Here, if the determination result in step S132 is true (Yes in S132), in step S133, the merge block candidate calculation unit 114 assigns the value of the merge block index to the new zero merge block candidate, and new to the merge block candidate list. zero Add merge block candidates. Further, in step S134, the merge block candidate calculation unit 114 adds 1 to the number of merge block candidates.
On the other hand, if the determination result in step S131 or step S132 is false (No in S131 or S132), the new zero merge block candidate addition process is terminated. That is, if the number of merge block candidates has reached the maximum number of merge block candidates, or if there are no new zero merge block candidates, the new zero merge block candidate addition process ends.
FIG. 15 is a flowchart showing the detailed processing of step S132 of FIG. Specifically, FIG. 15 shows a method of determining whether or not a zero merge block candidate exists. Hereinafter, FIG. 15 will be described.
In step S141, the merge block candidate calculation unit 114 updates the value of the reference picture index refIdxL0 in the prediction direction 0 and the value of the reference picture index refIdxL1 in the prediction direction 1 used when generating the zero merge block candidate. The reference picture indexes refIdxL0 and refIdxL1 are set to "-1" as initial values. Then, "+1" is added to the reference picture indexes refIdxL0 and refIdxL1 each time the process of step S141 is performed. That is, the merge block candidate calculation unit 114 first selects a zero merge block candidate having a motion vector (zero vector) with a value of 0 and a reference picture index with a value of 0 as a merge block candidate as a zero merge block candidate for a stationary region. Add to list. Next, the merge block candidate calculation unit 114 adds a zero merge block candidate having a motion vector having a value of 0 and a reference picture index having a value of 1 to the merge block candidate list.
In step S142, the merge block candidate calculation unit 114 determines that the value of the updated reference picture index refIdxL0 in the prediction direction 0 is smaller than the maximum number of references in the reference picture list 0 in the prediction direction 0, and the updated reference in the prediction direction 1 Determines whether the value of the picture index refIdxL1 is smaller than the maximum number of references in the reference picture list 1 in the prediction direction 1.
Here, if the determination result in step S142 is true (Yes in S142), in step S143, the merge block candidate calculation unit 114 sets the motion vector (0,0) and the reference picture index refIdxL0 having the value 0 to zero merge block. Assign to the prediction direction 0 of. Further, in step S144, the merge block candidate calculation unit 114 assigns the motion vector (0,0) and the reference picture index refIdxL1 having the value 0 to the prediction direction 1 of the zero merge block.
Through the processing of steps S143 and S144, the merge block candidate calculation unit 114 calculates the zero merge block for bidirectional prediction. FIG. 16 shows an example of the calculated zero merge block.
In step S145, the merge block candidate calculation unit 114 determines whether or not a merge block candidate having the same motion vector, reference picture index, and prediction direction as the calculated zero merge block candidate already exists in the merge block candidate list. judge. That is, the merge block candidate calculation unit 114 determines whether or not the calculated zero merge block candidate is a duplicate candidate.
Here, if the determination result in step S145 is false (No in S145), the merge block candidate calculation unit 114 sets that there are zero merge block candidates in step S146.
On the other hand, if the determination result in step S142 is false (No in S142) or the determination result in step S145 is true (Yes in S145), in step S147, the merge block candidate calculation unit 114 indicates that there are no zero merge block candidates. judge.
In this way, the merge block candidate calculation unit 114 calculates a zero merge block candidate having a motion vector having a value of 0 for each reference picture that can be referred to. Then, the merge block candidate calculation unit 114 newly adds the calculated zero merge block candidate to the merge block candidate list. As a result, the image coding apparatus 100 can improve the coding efficiency of the merge mode, especially when the coded block is a stationary region.
FIG. 17 is a flowchart showing the detailed processing of step S102 of FIG. Specifically, FIG. 17 shows a process related to selection of merge block candidates. Hereinafter, FIG. 17 will be described.
In step S151, the inter-prediction control unit 111 sets the merge block candidate index to 0, sets the minimum prediction error to the prediction error (cost) of the motion vector detection mode, and sets the merge flag to 0. Here, the cost is calculated by, for example, the following formula of the RD optimization model.
(Equation 1) Cost = D + λR
In Equation 1, D represents the coding distortion. For example, the sum of the difference absolute values between the pixel value obtained by encoding and decoding the coded block using the predicted image generated by a certain motion vector and the original pixel value of the coded block is set as D. Used. Further, R represents the generated code amount. The amount of code required to encode the motion vector used to generate the predicted image is used as R. Λ is an undetermined multiplier of Lagrange.
In step S152, the inter-prediction control unit 111 determines whether the value of the merge block candidate index is smaller than the number of merge block candidates of the coded block. That is, the inter-prediction control unit 111 determines whether or not there is a merge block candidate for which the following steps S153 to S155 have not yet been processed.
Here, if the determination result in step S152 is true (Yes in S152), in step S153, the inter-prediction control unit 111 calculates the cost of the merge block candidate to which the merge block candidate index is assigned. Then, in step S154, the inter-prediction control unit 111 determines whether or not the calculated merge block candidate cost is smaller than the minimum prediction error.
Here, if the determination result in step S154 is true (Yes in S154), in step S155, the inter-prediction control unit 111 updates the values of the minimum prediction error, the merge block index, and the merge flag. On the other hand, if the determination result in step S154 is false (No in S154), the inter-prediction control unit 111 does not update the values of the minimum prediction error, the merge block index, and the merge flag.
In step S156, the inter-prediction control unit 111 adds 1 to the value of the merge block candidate index, and repeats steps S152 to S156.
On the other hand, if the determination result in step S152 is false (No in S152), that is, if there are no unprocessed merge block candidates, in step S157, the inter-prediction control unit 111 sets the merge flag and the merge flag. Determine the value of the merge block index.
As described above, according to the image coding apparatus 100 according to the present embodiment, a new merge block candidate having a motion vector for a stationary region and a reference picture index is added to the merge block candidate list as the merge block candidate. This makes it possible to improve the coding efficiency. More specifically, the image encoding device 100 calculates a merge block candidate having a motion vector with a value of 0 for each reference picture that can be referred to, and newly adds the calculated merge block candidate to the merge block candidate list. By doing so, it is possible to improve the coding efficiency of the merge mode, especially when the coded block is a stationary region.
In the present embodiment, an example of calculating a merge block candidate having a motion vector having a value of 0 as a motion vector for a stationary region is shown, but this is not always the case. For example, the merge block candidate calculation unit 114 may use a predetermined vector (for example, (0,)) having a value slightly larger than or slightly smaller than the value 0 in order to consider minute camera shake during video shooting. A merge block candidate having (1) etc.) as a motion vector may be calculated as a new candidate instead of the zero merge block candidate. In this case, the variable-length coding unit 116 may add offset parameters (OffsetX, OffsetY) or the like to the header or the like of the sequence, the picture, or the slice. In this case, the merge block candidate calculation unit 114 may calculate the merge block candidate having the motion vector (OffsetX, OffsetY) as a new candidate.
In the present embodiment, an example in which the merge flag is always added to the bit stream in the merge mode is shown, but the present invention is not necessarily limited to this. For example, the merge mode may be forcibly selected according to the block shape or the like used for inter-prediction of the coded block. In that case, the amount of information may be reduced by not adding the merge flag to the bitstream.
In the present embodiment, an example using a merge mode in which the prediction direction, the motion vector, and the reference picture index are copied from the adjacent blocks of the coded target block to encode the coded target block is shown. , Not necessarily limited to this. For example, the skip merge mode may be used. In the skip merge mode, the prediction direction, motion vector, and reference picture index are copied from the adjacent blocks of the coded block in the same way as in the merge mode, using the merge block candidate list created as shown in (b) of Fig. 11. Then, the coded block is encoded. As a result, if all the prediction error data of the coded block is 0, the skip flag is set to 1 and the skip flag and the merge block index are added to the bitstream. If the prediction error data is not 0, the skip flag is set to 0 to add the skip flag, merge flag, merge block index, and prediction error data to the bitstream.
In the present embodiment, an example using a merge mode in which the prediction direction, the motion vector, and the reference picture index are copied from the adjacent blocks of the coded block to encode the coded block is shown. Not necessarily limited to this. For example, the motion vector in the motion vector detection mode may be encoded by using the merge block candidate list created as shown in FIG. 11 (b). That is, the difference is obtained by subtracting the motion vector of the merge block candidate specified by the merge block index from the motion vector in the motion vector detection mode. Then, the obtained difference and the merge block index may be attached to the bitstream.
In addition, the motion vector MV_Merge of the merge block candidate is scaled by using the reference picture index RefIdx_ME of the motion detection mode and the reference picture index RefIdx_Merge of the merge block candidate, and the merge block candidate after scaling is scaled from the motion vector of the motion detection mode. The difference may be obtained by reducing the motion vector scaledMV_Merge. Then, the obtained difference and the merge block index may be added to the bitstream. An example of the scaling formula is shown below.
(Equation 2) scaledMV_Merge = MV_Merge × (POC (RefIdx_ME) -curPOC) / (POC (RefIdx_Merge) -curPOC)
Here, POC (RefIdx_ME) indicates the display order of the reference pictures indicated by the reference picture index RefIdx_ME. POC (RefIdx_Merge) indicates the display order of the reference pictures indicated by the reference picture index RefIdx_Merge. curPOC indicates the display order of the pictures to be encoded.
In the present embodiment, an example of generating zero merge block candidates for bidirectional prediction from a motion vector having a value of 0, a reference picture index in the prediction direction 0, and a reference picture index in the prediction direction 1 is shown, but it is not always the case. Not limited to this. For example, the merge block candidate calculation unit 114 may generate a zero merge block candidate with a prediction direction of 0 and add it to the merge block candidate list by using a motion vector with a value of 0 and a reference picture index with a prediction direction of 0. Absent. Similarly, the merge block candidate calculation unit 114 generates zero merge block candidates in the prediction direction 1 by using the motion vector with the value 0 and the reference picture index in the prediction direction 1, and adds them to the merge block candidate list. It doesn't matter.
In the present embodiment, an example of generating zero merge block candidates while adding "+1" to the reference picture index in order from the value 0 is shown, but the present invention is not necessarily limited to this. For example, the merge block candidate calculation unit 114 may generate zero merge block candidates in order from the reference picture index assigned to the reference picture having the closest distance in the display order from the coded target picture.
(Embodiment 2) In the first embodiment, the merge block candidate calculation unit 114 determines in step S145 of FIG. 15 whether or not the zero merge block candidate is a duplicate candidate, but it is not always necessary to determine in this way. For example, the determination in step S145 may be omitted. As a result, the image coding apparatus 100 can reduce the amount of processing for deriving the merge block candidates.
Further, in the first embodiment, the zero merge block candidates are added to the merge block candidate list until the number of merge block candidates reaches the maximum number of merge block candidates, but the present invention is not necessarily limited to this. For example, in step S131 of FIG. 14, the merge block candidate calculation unit 114 determines whether or not the number of merge block candidates is a threshold value smaller than the maximum number of merge block candidates and reaches a preset threshold value. It doesn't matter. As a result, the image coding apparatus 100 can reduce the amount of processing for deriving the merge block candidate list.
Further, in the first embodiment, when the number of merge block candidates reaches the maximum number of merge block candidates, the addition of zero merge block candidates is completed, but this is not necessarily the case. For example, in step S131 of FIG. 14, the merge block candidate calculation unit 114 does not determine whether the number of merge block candidates has reached the maximum number of merge block candidates, and merge blocks until there are no new zero merge block candidates. You may add zero merge block candidates to the candidate list. As a result, the image coding apparatus 100 can widen the range of selection of merge block candidates and can improve the coding efficiency.
A modified example of the image coding device according to the first embodiment as described above will be specifically described below as the image coding device according to the second embodiment.
FIG. 18 is a block diagram showing a configuration of the image coding device 200 according to the second embodiment. The image coding device 200 generates a bit stream by encoding an image block by block. The image coding device 200 includes a merge candidate derivation unit 210, a prediction control unit 220, and a coding unit 230.
The merge candidate derivation unit 210 corresponds to the merge block candidate calculation unit 114 in the first embodiment. The merge candidate derivation unit 210 derives the merge candidate. Then, the merge candidate derivation unit 210 generates, for example, a merge candidate list in which each derived merge candidate is associated with an index for identifying the merge candidate (hereinafter, referred to as a merge index).
The merge candidate is a candidate for the prediction direction, motion vector, and reference picture index used for coding the coded block. That is, a merge candidate contains at least one set of prediction direction, motion vector, and reference picture index.
The merge candidate corresponds to the merge block candidate of the first embodiment. The merge candidate list is the same as the merge block candidate list.
As shown in FIG. 18, the merge candidate derivation unit 210 includes a first derivation unit 211 and a second derivation unit 212.
The first derivation unit 211 derives the first merge candidate based on the prediction direction, motion vector, and reference picture index used for coding the block spatially or temporally adjacent to the block to be coded. Then, the first derivation unit 211, for example, associates the first merge candidate derived in this way with the merge index and registers it in the merge candidate list.
The spatially adjacent block is a block in the picture including the coded block, and is a block adjacent to the coded block. Specifically, the spatially adjacent blocks are, for example, adjacent blocks A to D shown in FIG.
The temporally adjacent blocks are blocks included in a picture different from the picture including the coded block, and are blocks corresponding to the coded block. Specifically, the blocks adjacent in time are, for example, co-located blocks.
The blocks that are adjacent in time do not necessarily have to be blocks at the same positions as the blocks to be encoded (co-located blocks). For example, the blocks adjacent in time may be blocks adjacent to the co-located block.
Note that the first derivation unit 211 obtains, for example, the prediction direction, motion vector, and reference picture index used for coding the blocks spatially adjacent to the coded target block, excluding the unmergeable block. 1 It may be derived as a merge candidate. An unmergeable block is a block encoded by intra-prediction, a block located outside the slice or picture boundary containing the block to be encoded, or a block that has not yet been encoded. As a result, the first derivation unit 211 can derive the first merge candidate from an appropriate block in order to obtain the merge candidate.
The second derivation unit 212 derives a merge candidate having a predetermined vector as a motion vector as a second merge candidate. Specifically, the second derivation unit 212 derives a second merge candidate for each reference picture that can be referred to, for example. As a result, the image coding apparatus 200 can increase the types of merge candidates and further improve the coding efficiency.
The second derivation unit 212 does not necessarily have to derive the second merge candidate for each reference picture that can be referred to. For example, the second derivation unit 212 may derive a second merge candidate for a predetermined number of reference pictures.
The predetermined vector may be a zero vector, for example, as in the first embodiment. As a result, the second derivation unit 212 can derive a merge candidate having a motion vector for the stationary region. Therefore, the image coding apparatus 200 can improve the coding efficiency when the coded block is a stationary region. The predetermined vector does not necessarily have to be a zero vector.
Further, the second derivation unit 212 registers the second merge candidate derived in this way in the merge candidate list in association with the merge index, for example. At this time, the second derivation unit 212 registers the second merge candidate in the merge candidate list so that the merge index having a value smaller than that of the second merge candidate is assigned to the first merge candidate, as in the first embodiment. You may. As a result, the image coding apparatus 200 can reduce the amount of coding and improve the coding efficiency when the first merge candidate is more likely to be selected as the merge candidate used for coding than the second merge candidate. Can be made to.
The prediction control unit 220 selects a merge candidate used for coding the coded block from the derived first merge candidates and second merge candidates. That is, the prediction control unit 220 selects the merge candidate used for coding the coded block from the merge candidate list.
The encoding unit 230 adds an index (merge index) for identifying the selected merge candidate to the bitstream. For example, the encoding unit 230 encodes the merge index using the sum of the number of derived first merge candidates and the number of second merge candidates (the number of merge candidates), and bitstreams the encoded merge index. To add to.
Next, various operations of the image coding apparatus 200 configured as described above will be described.
FIG. 19 is a flowchart showing a processing operation of the image coding apparatus 200 according to the second embodiment.
First, the first derivation unit 211 derives the first merge candidate (S201). Subsequently, the second derivation unit 212 derives the second merge candidate (S202).
Then, the prediction control unit 220 selects the merge candidate used for coding the coded block from the first merge candidate and the second merge candidate (S203). For example, the prediction control unit 220 selects the merge candidate having the minimum cost shown in Equation 1 from the merge candidate list, as in the first embodiment.
Finally, the encoding unit 230 adds an index to the bitstream to identify the selected merge candidate (S204).
As described above, according to the image coding apparatus 200 according to the present embodiment, a merge candidate having a predetermined vector as a motion vector can be derived as a second merge candidate. Therefore, the image coding apparatus 200 can derive a merge candidate having, for example, a motion vector for a stationary region, as a second merge candidate. That is, the image coding device 200 can efficiently encode the coded target block having a predetermined motion, and can improve the coding efficiency.
(Embodiment 3) FIG. 20 is a block diagram showing a configuration of the image decoding device 300 according to the third embodiment. The image decoding device 300 is a device corresponding to the image coding device 100 according to the first embodiment. The image decoding device 300, for example, decodes the coded image included in the bit stream generated by the image coding device 100 according to the first embodiment block by block.
As shown in FIG. 20, the image decoding device 300 includes a variable length decoding unit 301, an inverse quantization unit 302, an inverse orthogonal conversion unit 303, an addition unit 304, a block memory 305, a frame memory 306, and an intra. It includes a prediction unit 307, an inter prediction unit 308, an inter prediction control unit 309, a switch 310, a merge block candidate calculation unit 311 and a colPic memory 312.
The variable-length decoding unit 301 performs a variable-length decoding process on the input bit stream to generate picture type information, a merge flag, and a quantization coefficient. Further, the variable length decoding unit 301 performs a variable length decoding process of the merge block index using the number of merge block candidates calculated by the merge block candidate calculation unit 311.
The inverse quantization unit 302 performs an inverse quantization process on the quantization coefficient obtained by the variable length decoding process.
The inverse orthogonal conversion unit 303 generates prediction error data by converting the orthogonal conversion coefficient obtained by the inverse quantization process from the frequency domain to the image domain.
In the block memory 305, the decoded image data generated by adding the prediction error data and the predicted image data is stored in block units.
The decoded image data is stored in the frame memory 306 in frame units.
The intra prediction unit 307 generates predicted image data of a block to be decoded by performing intra prediction using the decoded image data of each block stored in the block memory 305.
The inter-prediction unit 308 generates predicted image data of a block to be decoded by inter-predicting using the decoded image data of each frame stored in the frame memory 306.
When the decoding target block is intra-predictively decoded, the switch 310 outputs the intra-prediction image data generated by the intra-prediction unit 307 to the addition unit 304 as the prediction image data of the decoding target block. On the other hand, when the decoding target block is inter-predicted and decoded, the switch 310 outputs the inter-predicted image data generated by the inter-predicted unit 308 to the adding unit 304 as the predicted image data of the decoding target block.
The merge block candidate calculation unit 311 derives the merge block candidate by using the motion vector of the adjacent block of the block to be decoded and the motion vector of the co-located block stored in the colPic memory 312 (colPic information). To do. Further, the merge block candidate calculation unit 311 adds the derived merge block candidate to the merge block candidate list.
Further, the merge block candidate calculation unit 311 derives a merge block candidate having a motion vector for a stationary region and a reference picture index as a zero merge block candidate by a method described later. Then, the merge block candidate calculation unit 311 adds the derived zero merge block candidate to the merge block candidate list as a new merge block candidate. Further, the merge block candidate calculation unit 311 calculates the number of merge block candidates.
In addition, the merge block candidate calculation unit 311 assigns the value of the merge block index to each of the derived merge block candidates. Then, the merge block candidate calculation unit 311 transmits the merge block candidate to which the value of the merge block index is assigned to the inter prediction control unit 309. Further, the merge block candidate calculation unit 311 transmits the calculated number of merge block candidates to the variable length decoding unit 301.
If the decoded merge flag is "0", the inter-prediction control unit 309 causes the inter-prediction unit 308 to generate an inter-prediction image by using the information of the motion vector detection mode. On the other hand, if the merge flag is "1", the inter-prediction control unit 309 determines the motion vector, the reference picture index, and the prediction direction to be used for the inter-prediction from the plurality of merge block candidates based on the decoded merge block index. To do. Then, the inter-prediction control unit 309 causes the inter-prediction unit 308 to generate an inter-prediction image using the determined motion vector, the reference picture index, and the prediction direction. Further, the inter-prediction control unit 309 transfers the colPic information including the motion vector of the decoding target block to the colPic memory 312.
Finally, the addition unit 304 generates the decoded image data by adding the prediction image data and the prediction error data.
FIG. 21 is a flowchart showing a processing operation of the image decoding apparatus 300 according to the third embodiment.
In step S301, the variable length decoding unit 301 decodes the merge flag.
If the merge flag is "1" in step S302 (Yes in S302), in step S303, the merge block candidate calculation unit 311 generates merge block candidates in the same manner as in step S101 of FIG. Further, the merge block candidate calculation unit 311 calculates the number of merge block candidates as the merge block candidate list size.
In step S304, the variable length decoding unit 301 uses the merge block candidate list size to perform variable length decoding of the merge block index in the bitstream.
In step S305, the inter-prediction control unit 309 causes the inter-prediction unit 308 to generate an inter-prediction image using the motion vector of the merge block candidate indicated by the decoded merge block index, the reference picture index, and the prediction direction.
If the merge flag is "0" in step S302 (No in S302), in step S306, the inter-prediction unit 308 uses the motion vector detection mode information decoded by the variable-length decoding unit 301 to generate the inter-prediction image. To generate.
If the merge block candidate list size calculated in step S303 is "1", the merge block index may be estimated to be "0" without being decoded.
As described above, according to the image decoding apparatus 300 according to the third embodiment, it is possible to add a new merge block candidate having a motion vector for the stationary region and a reference picture index to the merge block candidate list as the merge block candidate. it can. As a result, the image decoding device 300 can appropriately decode the bit stream with improved coding efficiency. More specifically, by calculating a merge block candidate having a motion vector with a value of 0 for each reference picture that can be referred to and newly adding it to the merge block candidate list, the coded block is particularly in the stationary region. In some cases, it becomes possible to appropriately decode the bitstream with improved coding efficiency in the merge mode.
(Embodiment 4) The image decoding apparatus according to the third embodiment includes the components as shown in FIG. 20, but does not necessarily include all the components. Hereinafter, the image decoding apparatus according to the fourth embodiment will be specifically described as a modification of the image decoding apparatus according to the third embodiment.
FIG. 22 is a block diagram showing a configuration of the image decoding device 400 according to the fourth embodiment. The image decoding device 400 is a device corresponding to the image coding device 200 according to the second embodiment. The image decoding device 400, for example, decodes the coded image included in the bit stream generated by the image coding device 200 according to the second embodiment block by block.
As shown in FIG. 22, the image decoding device 400 includes a merge candidate derivation unit 410, a decoding unit 420, and a prediction control unit 430.
The merge candidate derivation unit 410 corresponds to the merge block candidate calculation unit 311 in the third embodiment. The merge candidate derivation unit 410 derives the merge candidate. Then, the merge candidate derivation unit 410 generates, for example, a merge candidate list in which each derived merge candidate is associated with an index (merge index) for specifying the merge candidate.
As shown in FIG. 22, the merge candidate derivation unit 410 includes a first derivation unit 411 and a second derivation unit 412.
The first derivation unit 411 derives the first merge candidate in the first derivation unit 411 in the same manner as the first derivation unit 211 of the second embodiment. Specifically, the first derivation unit 411 derives the first merge candidate based on the prediction direction, motion vector, and reference picture index used to decode the block spatially or temporally adjacent to the block to be decoded. .. Then, for example, the first derivation unit 411 associates the first merge candidate derived in this way with the merge index and registers it in the merge candidate list.
The second derivation unit 412 derives a merge candidate having a predetermined vector as a motion vector as a second merge candidate. Specifically, the second derivation unit 412 derives the second merge candidate in the same manner as the second derivation unit 212 of the second embodiment. Then, the second derivation unit 412, for example, associates the second merge candidate derived in this way with the merge index and registers it in the merge candidate list.
More specifically, the second derivation unit 412 derives a second merge candidate for each reference picture that can be referred to, for example. As a result, the types of merge candidates can be increased, and a bit stream with further improved coding efficiency can be appropriately decoded.
The predetermined vector may be a zero vector, for example, as in the first embodiment. As a result, the second derivation unit 412 can derive a merge candidate having a motion vector for the stationary region. Therefore, the image decoding device 400 can appropriately decode the bit stream with improved coding efficiency.
The decoding unit 420 acquires an index for identifying a merge candidate from the bit stream. For example, the decoding unit 420 decodes the encoded merge index added to the bitstream by using the sum of the number of derived first merge candidates and the number of second merge candidates (the number of merge candidates). By doing so, the merge index is obtained.
The prediction control unit 430 selects a merge candidate to be used for decoding the decryption target block from the derived first merge candidate and second merge candidate based on the acquired index. That is, the prediction control unit 430 selects a merge candidate used for decoding the decryption target block from the merge candidate list.
Next, various operations of the image decoding apparatus 400 configured as described above will be described.
FIG. 23 is a flowchart showing the processing operation of the image decoding apparatus 400 according to the fourth embodiment.
First, the first derivation unit 411 derives the first merge candidate (S401). Subsequently, the second derivation unit 412 derives the second merge candidate (S402). Then, the decoding unit 420 acquires the merge index from the bit stream (S403).
Finally, the predictive control unit 220 selects a merge candidate used for decoding the decryption target block from the first merge candidate and the second merge candidate based on the acquired index (S404).
As described above, according to the image decoding apparatus 400 according to the present embodiment, a merge candidate having a predetermined vector as a motion vector can be derived as a second merge candidate. Therefore, the image decoding device 400 can derive, for example, a merge candidate having a motion vector for a stationary region as a second merge candidate. That is, the image decoding device 400 can appropriately decode an image in which a block having a predetermined motion is efficiently encoded, and can appropriately decode a bit stream having improved coding efficiency. It will be possible.
(Embodiment 5) In the fifth embodiment, the method of deriving the merge block candidate list size is different from that of the first embodiment. The method of deriving the merge block candidate list size in the present embodiment will be described in detail.
In the merge mode of the first embodiment, the number of merge block candidates is set in the merge block candidate list size used when encoding or decoding the merge block index. This number of merge block candidates is obtained after deleting unmergeable candidates or duplicate candidates using reference picture information including co-located blocks and the like.
Therefore, when the number of merge block candidates does not match between the image coding device and the image decoding device, the bit string assigned to the merge block index does not match between the image coding device and the image decoding device. As a result, the image decoding device may not be able to correctly decode the bitstream.
For example, when the information of the reference picture referenced as the co-located block is lost due to packet loss or the like generated in the transmission line or the like, the motion vector or the reference picture index of the co-located block becomes unknown. Therefore, the information of the merge block candidate generated from the co-located block becomes unknown. In such a case, it becomes impossible to correctly delete the unmergeable candidate or the duplicate candidate from the merge block candidates at the time of decryption. As a result, the image decoding device cannot correctly obtain the merge block candidate list size, and cannot normally decode the merge block index.
Therefore, the image coding apparatus according to the present embodiment calculates the merge block candidate list size used when encoding or decoding the merge block index by a method that does not depend on the reference picture information including the co-located block and the like. .. Therefore, the image coding device can improve the error tolerance.
FIG. 24 is a block diagram showing a configuration of the image coding device 500 according to the fifth embodiment. In FIG. 24, the same components as those in FIG. 9 are designated by the same reference numerals, and the description thereof will be omitted.
As shown in FIG. 24, the image coding apparatus 500 includes a subtraction unit 101, an orthogonal conversion unit 102, a quantization unit 103, an inverse quantization unit 104, an inverse orthogonal conversion unit 105, a block memory 107, and the like. Frame memory 108, intra prediction unit 109, inter prediction unit 110, inter prediction control unit 111, picture type determination unit 112, switch 113, merge block candidate calculation unit 514, colPic memory 115, and variable length. A coding unit 516 is provided.
The merge block candidate calculation unit 514 uses the motion vector of the adjacent block of the coded block and the motion vector of the co-located block stored in the colPic memory 115 (colPic information) to merge the merge mode. Derive block candidates. Then, the merge block candidate calculation unit 514 calculates the number of mergeable candidates by the method described later.
Further, the merge block candidate calculation unit 514 assigns the value of the merge block index to the derived merge block candidate. Then, the merge block candidate calculation unit 514 transmits the merge block candidate and the merge block index to the inter-prediction control unit 111. Further, the merge block candidate calculation unit 514 transmits the calculated number of mergeable candidates to the variable length coding unit 116.
The variable-length coding unit 516 generates a bit stream by performing variable-length coding processing on the quantized prediction error data, the merge flag, and the picture type information. Further, the variable length encoding unit 516 sets the number of mergeable candidates to the merge block candidate list size. Then, the variable-length coding unit 516 assigns a bit string corresponding to the merge block candidate list size to the merge block index used for coding to perform variable-length coding.
FIG. 25 is a flowchart showing the processing operation of the image coding apparatus 500 according to the fifth embodiment. In FIG. 25, the same steps as in FIG. 10 are designated by the same reference numerals, and the description thereof will be omitted as appropriate.
In step S501, the merge block candidate calculation unit 514 derives the merge block candidate from the adjacent block and the co-located block of the coded block. Further, the merge block candidate calculation unit 514 calculates the merge block candidate list size by a method described later.
For example, in the case shown in FIG. 3, the merge block candidate calculation unit 514 selects adjacent blocks A to D as merge block candidates. Further, the merge block candidate calculation unit 514 calculates the co-located merge block including the motion vector calculated by the time prediction mode from the motion vector of the co-located block as the merge block candidate.
The merge block candidate calculation unit 514 assigns a merge block index to each merge block candidate as shown in FIG. 26 (a). Then, the merge block candidate calculation unit 514 deletes the unmergeable candidate and the duplicate candidate and adds a new candidate by the method described later, thereby performing the merge block candidate list as shown in FIG. 26 (b) and the merge block candidate list. Calculate the merge block candidate list size.
The smaller the value of the merge block index, the shorter the code is assigned. That is, when the value of the merge block index is small, the amount of information required for the merge block index is small.
On the other hand, as the value of the merge block index increases, the amount of information required for the merge block index increases. Therefore, if a merge block index with a smaller value is assigned to a merge block candidate that is likely to have a more accurate motion vector and reference picture index, the coding efficiency will be higher.
Therefore, the merge block candidate calculation unit 514 may, for example, measure the number of times selected as a merge block for each merge block candidate, and assign a merge block index having a small value to a block having a large number of times. Specifically, it is conceivable to specify the merge block selected in the adjacent block and reduce the value of the merge block index for the specified merge block when coding the target block.
If the merge block candidate does not have information such as motion vectors (a block encoded by intra-prediction, a block located outside the boundary of a picture or slice, or is still encoded). The merge block candidate is not available for encoding, for example, if it is not a block.
In the present embodiment, the merge block candidate that cannot be used for encoding is referred to as an unmergeable candidate. Further, a merge block candidate that can be used for encoding is called a mergeable candidate. Further, among a plurality of merge block candidates, a candidate whose motion vector, reference picture index, and prediction direction all match with any other merge block candidate is called a duplicate candidate.
In the case of FIG. 3, the adjacent block C is a block encoded by the intra prediction, and is therefore a candidate that cannot be merged. Further, the adjacent block D is a duplicate candidate because the motion vector, the reference picture index, and the prediction direction all match the adjacent block A.
In step S102, the inter-prediction control unit 111 determines the prediction error of the predicted image generated by using the motion vector derived by the motion detection and the prediction error of the predicted image generated by using the motion vector obtained from the merge block candidate. Compare with and select the prediction mode. Here, if the selected prediction mode is the merge mode, the inter-prediction control unit 111 sets the merge flag to 1, and if not, sets the merge flag to 0.
In step S103, it is determined whether the merge flag is 1 (that is, whether the prediction mode is the merge mode).
Here, if the determination result in step S103 is true (Yes in S103), the variable length coding unit 516 adds a merge flag to the bit stream in step S104. Further, in step S505, the variable-length coding unit 516 allocates a bit string corresponding to the merge block candidate list size as shown in FIG. 5 to the merge block index of the merge block candidates used for coding. Then, the variable-length coding unit 516 performs variable-length coding on the assigned bit string.
On the other hand, if the determination result in step S103 is false (No in S103), in step S106, the variable length coding unit 516 adds the merge flag and the motion detection vector mode information to the bit stream.
In the present embodiment, as shown in FIG. 26A, the value of the merge block index is assigned 0 as the value of the merge block index corresponding to the adjacent block A. In addition, "1" is assigned as the value of the merge block index corresponding to the adjacent block B. Also, "2" is assigned as the value of the merge block index corresponding to the co-located merge block. In addition, "3" is assigned as the value of the merge block index corresponding to the adjacent block C. In addition, "4" is assigned as the value of the merge block index corresponding to the adjacent block D.
Note that the method of assigning the value of the merge block index is not necessarily limited to this example. For example, the variable-length coding unit 516 is an original merge block when a new candidate is added by using the method described in the first embodiment, or when a new candidate is added by using a method described later. Candidates may be assigned a small value and new candidates may be assigned a large value. That is, the variable-length encoding unit 516 may assign a merge block block index having a small value in preference to the original merge block candidate.
Also, the merge block candidates are not necessarily limited to the positions of adjacent blocks A to D. For example, an adjacent block located above the lower left adjacent block D may be used as a merge block candidate. Also, not all adjacent blocks need to be used as merge block candidates. For example, only adjacent blocks A and B may be used as merge block candidates.
Further, in the present embodiment, in step S505 of FIG. 25, the variable length coding unit 516 adds the merge block index to the bit stream, but it is not always necessary to add the merge block index to the bit stream. For example, the variable-length encoding unit 116 does not have to add the merge block index to the bitstream when the merge block candidate list size is 1. As a result, the amount of information in the merge block index can be reduced.
FIG. 27 is a flowchart showing the detailed processing of step S501 of FIG. 25. Specifically, FIG. 27 shows the merge block candidate and the method of calculating the merge block candidate list size. Hereinafter, FIG. 27 will be described.
In step S511, the merge block candidate calculation unit 514 determines whether or not the merge block candidate [N] is a mergeable candidate by a method described later. Then, the merge block candidate calculation unit 514 updates the number of mergeable candidates according to the determination result.
Here, N is an index value for representing each merge block candidate. In this embodiment, N takes a value from 0 to 4. Specifically, the adjacent block A in FIG. 3 is assigned to the merge block candidate [0]. In addition, the adjacent block B in FIG. 3 is assigned to the merge block candidate [1]. In addition, a co-located merge block is assigned to the merge block candidate [2]. In addition, the adjacent block C in FIG. 3 is assigned to the merge block candidate [3]. In addition, the adjacent block D in FIG. 5 is assigned to the merge block candidate [4].
In step S512, the merge block candidate calculation unit 514 acquires the motion vector, the reference picture index, and the prediction direction of the merge block candidate [N] and adds them to the merge block candidate list.
In step S513, the merge block candidate calculation unit 514 searches for unmergeable candidates and duplicate candidates from the merge block candidate list and deletes them, as shown in FIG. 26.
In step S514, the merge block candidate calculation unit 514 adds a new candidate to the merge block candidate list by the method described in the first embodiment or the method described later. Here, when adding a new candidate, the merge block candidate calculation unit 514 reassigns the value of the merge block index so that the merge block index with a smaller value is assigned in preference to the original merge block candidate. You may go. That is, the merge block candidate calculation unit 514 may reassign the value of the merge block index so that the merge block index having a large value is assigned to the new candidate. As a result, the amount of code in the merge block index can be reduced.
In step S515, the merge block candidate calculation unit 514 sets the number of mergeable candidates calculated in step S511 to the merge block candidate list size. In the example of FIG. 26, the number of mergeable candidates is calculated as 4 and the merge block candidate list size is set to 4 by the method described later.
The new candidate in step S514 is newly added to the merge block candidate when the number of merge block candidates has not reached the number of mergeable candidates by the method described in the first embodiment or the method described later. Candidates. For example, a new candidate is a merge block candidate having a predetermined vector (for example, a zero vector) as a motion vector. Further, for example, the new candidate may be an adjacent block located above the lower left adjacent block D in FIG. Further, the new candidate may be, for example, a block corresponding to adjacent blocks A to D of the co-located block. Further, the new candidate may be, for example, a block having a motion vector, a reference picture index, a statistical value in a prediction direction, and the like in the entire screen of the reference picture or in a certain area. In this way, when the number of merge block candidates has not reached the number of mergeable candidates, the merge block candidate calculation unit 514 adds a new candidate having a new motion vector, a reference picture index, and a prediction direction. The coding efficiency can be improved.
FIG. 28 is a flowchart showing the detailed processing of step S511 of FIG. 27. Specifically, FIG. 28 shows a method of determining whether the merge block candidate [N] is a mergeable candidate and updating the number of mergeable candidates. Hereinafter, FIG. 28 will be described.
In step S521, the merge block candidate calculation unit 514 determines that the merge block candidate [N] is located outside the slice or picture boundary containing (1) the block encoded by the intra prediction or (2) the block to be encoded. Determine if it is a block to be used, or (3) a block that has not yet been encoded.
Here, if the determination result in step S521 is true (Yes in S521), the merge block candidate calculation unit 514 sets the merge block candidate [N] as a non-mergeable candidate in step S522. On the other hand, if the determination result in step S521 is false (No in S521), the merge block candidate calculation unit 514 sets the merge block candidate [N] as a mergeable candidate in step S523.
In step S524, the merge block candidate calculation unit 514 determines whether the merge block candidate [N] is a mergeable candidate or a co-located merge block candidate. Here, if the determination result in step S524 is true (Yes in S524), in step S525, the merge block candidate calculation unit 514 adds 1 to the number of merge block candidates and updates the number of merge block candidates. On the other hand, if the determination result in step S524 is false (No in S524), the merge block candidate calculation unit 514 does not update the number of mergeable candidates.
In this way, when the merge block candidate is a co-located merge block, the merge block candidate calculation unit 514 determines that the number of mergeable candidates is 1 regardless of whether the co-located block is a mergeable candidate or a mergeable candidate. Is added. As a result, even if the information of the co-located merge block is lost due to packet loss or the like, the number of mergeable candidates does not match between the image encoding device and the image decoding device.
This number of mergeable candidates is set to the merge block candidate list size in step S515 of FIG. Further, in step S505 of FIG. 25, the merge block candidate list size is used for variable length coding of the merge block index. As a result, even if the reference picture information including the co-located block or the like is lost, the image coding apparatus 500 can generate a bit stream capable of normally decoding the merge block index.
FIG. 29 is a flowchart showing the detailed processing of step S514 of FIG. 27. Specifically, FIG. 29 shows a method of adding a new candidate. Hereinafter, FIG. 29 will be described.
In step S531, the merge block candidate calculation unit 514 determines whether or not the number of merge block candidates is smaller than the number of mergeable candidates. That is, the merge block candidate calculation unit 514 determines whether or not the number of merge block candidates has reached the number of mergeable candidates.
Here, if the determination result in step S531 is true (Yes in S531), in step S532, the merge block candidate calculation unit 514 determines whether there is a new candidate that can be added to the merge block candidate list as a merge block candidate. judge. Here, if step S532 is true (Yes in S532), the merge block candidate calculation unit 514 assigns the value of the merge block index to the new candidate in step S533, and adds the new candidate to the merge block candidate list. Further, in step S534, 1 is added to the number of merge block candidates.
On the other hand, if the determination result in step S101 or step S532 is false (No in S531 or S532), the new candidate addition process is terminated. That is, if the number of merge block candidates has reached the number of mergeable candidates, or if there are no new candidates, the new candidate addition process is terminated.
As described above, according to the image coding apparatus 500 according to the present embodiment, the merge block candidate list size used when encoding or decoding the merge block index depends on the reference picture information including the co-located block and the like. It can be calculated by a method that does not. This makes it possible for the image coding device 500 to improve error immunity.
More specifically, the image coding apparatus 500 according to the present embodiment can always merge if the merge block candidate is a co-located merge block, regardless of whether the co-located merge block is a mergeable candidate. Add 1 to the number of candidates. Then, the image coding apparatus 500 determines the bit string to be assigned to the merge block index by using the number of mergeable candidates calculated in this way. As a result, the image encoding device 500 can generate a bit stream that can normally decode the merge block index even if the reference picture information including the co-located block is lost.
Further, the image coding apparatus 500 according to the present embodiment merges new candidates having a new motion vector, a reference picture index, and a prediction direction when the number of merge block candidates does not reach the number of mergeable candidates. By adding it as a block candidate, the coding efficiency can be improved.
In the present embodiment, an example in which the merge flag is always added to the bit stream in the merge mode is shown, but the present invention is not necessarily limited to this. For example, the merge mode may be forcibly selected according to the block shape or the like used for inter-prediction of the coded block. In that case, the amount of information may be reduced by not adding the merge flag to the bitstream.
In the present embodiment, an example using a merge mode in which the prediction direction, the motion vector, and the reference picture index are copied from the adjacent blocks of the coded block to encode the coded block is shown. Not necessarily limited to this. For example, the skip merge mode may be used. In the skip merge mode, the prediction direction, motion vector, and reference picture index are copied from the adjacent blocks of the coded block in the same way as in the merge mode, using the merge block candidate list created as shown in (b) of Fig. 26. The block to be encoded is encoded. As a result, if all the prediction error data of the coded block is 0, the skip flag is set to 1 and the skip flag and the merge block index are added to the bitstream. If the prediction error data is not 0, the skip flag is set to 0 to add the skip flag, merge flag, merge block index, and prediction error data to the bitstream.
In this embodiment, an example using a merge mode in which the prediction direction, the motion vector, and the reference picture index are copied from the adjacent blocks of the coded block to encode the coded block is shown. Not necessarily limited to this. For example, the motion vector in the motion vector detection mode may be encoded by using the merge block candidate list created as shown in FIG. 26 (b). That is, the difference is obtained by subtracting the motion vector of the merge block candidate specified by the merge block index from the motion vector in the motion vector detection mode. Then, the obtained difference and the merge block index may be added to the bitstream.
Also, using the reference picture index RefIdx_ME of the motion detection mode and the reference picture index RefIdx_Merge of the merge block candidate, the motion vector MV_Merge of the merge block candidate is scaled as in Equation 2, and after scaling from the motion vector of the motion detection mode. The difference may be obtained by reducing the motion vector scaledMV_Merge of the merge block candidate. Then, the obtained difference and the merge block index may be added to the bitstream.
(Embodiment 6) In the fifth embodiment, the image encoding device calculates by always adding 1 if the merge block candidate is a co-located merge block, regardless of whether the co-located merge block is a mergeable candidate. The number of mergeable candidates was used to determine the bit string to be assigned to the merge block index. However, the image coding apparatus uses, for example, the number of mergeable candidates calculated by always adding 1 to merge block candidates other than the co-located merge block in step S524 of FIG. 28. , You may decide which bit string to allocate to the merge block index. That is, the image encoding device may allocate a bit string to the merge block index using the merge block candidate list size fixed to the maximum value N of the number of merge block candidates. That is, the image encoding device may consider all merge block candidates as mergeable candidates, fix the merge block candidate list size to the maximum value N of the number of merge block candidates, and encode the merge block index. ..
For example, in the fifth embodiment, since the maximum value N of the number of merge block candidates is 5 (adjacent block A, adjacent block B, co-located merge block, adjacent block C, adjacent block D), the image encoding device. May always set the merge block candidate list size to 5 to encode the merge block index. Further, for example, when the maximum value N of the number of merge block candidates is 4 (adjacent block A, adjacent block B, adjacent block C, adjacent block D), the image encoder always sets the merge block candidate list size to 4. May be set to encode the merge block index.
In this way, the image encoding device may determine the merge block candidate list size according to the maximum number of merge block candidates. This makes it possible for the variable-length decoding unit of the image decoding device to generate a bitstream in which the merge block index in the bitstream can be decoded without referring to the information of the adjacent block or the co-located block. , The processing amount of the variable length decoding unit can be reduced.
A modified example of the image coding apparatus according to the fifth embodiment as described above will be specifically described below as the image coding apparatus according to the sixth embodiment.
FIG. 30 is a block diagram showing a configuration of the image coding device 600 according to the sixth embodiment. The image coding device 600 generates a bit stream by encoding an image block by block. The image coding device 600 includes a merge candidate derivation unit 610, a prediction control unit 620, and a coding unit 630.
The merge candidate derivation unit 610 corresponds to the merge block candidate calculation unit 514 in the fifth embodiment. The merge candidate derivation unit 610 derives the merge candidate. Then, the merge candidate derivation unit 610 generates, for example, a merge candidate list in which each derived merge candidate is associated with an index for identifying the merge candidate.
As shown in FIG. 30, the merge candidate derivation unit 610 includes a determination unit 611, a first derivation unit 612, a specific unit 613, a determination unit 614, and a second derivation unit 615.
The determination unit 611 determines the maximum number of merge candidates. That is, the determination unit 611 determines the maximum value N of the number of merge block candidates.
For example, the determination unit 611 determines the maximum number of merge candidates based on the characteristics of the input image sequence (sequence, picture, slice, block, etc.). Further, for example, the determination unit 611 may determine a predetermined number as the maximum number of merge candidates.
The first derivation unit 612 derives the first merge candidate based on the prediction direction, motion vector, and reference picture index used for coding the block spatially or temporally adjacent to the block to be coded. Here, the first derivation unit 612 derives the first merge candidate so that the number of the first merge candidates does not exceed the maximum number. Then, for example, the first derivation unit 612 associates the first merge candidate derived in this way with the merge index and registers it in the merge candidate list.
The first derivation unit 612 obtains, for example, the prediction direction, the motion vector, and the reference picture index used for coding the blocks spatially adjacent to the coded block, excluding the unmergeable block. 1 It may be derived as a merge candidate. An unmergeable block is a block encoded by intra-prediction, a block located outside the slice or picture boundary containing the block to be encoded, or a block that has not yet been encoded. As a result, the first derivation unit 612 can derive the first merge candidate from an appropriate block in order to obtain the merge candidate.
The specific unit 613 identifies the first merge candidate (duplicate candidate) whose prediction direction, motion vector, and reference picture index overlap with other first merge candidates when a plurality of first merge candidates are derived. Then, the specific unit 613 deletes the specified duplicate candidate from the merge candidate list.
The determination unit 614 determines whether or not the number of first merge candidates is smaller than the determined maximum number. Here, the determination unit 614 determines whether or not the number of first merge candidates excluding the specified duplicate first merge candidates is smaller than the determined maximum number.
The second derivation unit 615 derives a merge candidate having a predetermined vector as a motion vector as a second merge candidate when it is determined that the number of first merge candidates is smaller than the determined maximum number. Specifically, the second derivation unit 615 derives the second merge candidate so that the sum of the number of the first merge candidates and the number of the second merge candidates does not exceed the maximum number. Here, the second derivation unit 615 derives the second merge candidate so that the sum of the number of the first merge candidates excluding the duplicate candidates and the number of the second merge candidates does not exceed the maximum number.
The predetermined vector may be a zero vector, for example, as in the fifth embodiment. The predetermined vector does not necessarily have to be a zero vector.
Then, the second derivation unit 615, for example, associates the second merge candidate derived in this way with the merge index and registers it in the merge candidate list. At this time, the second derivation unit 615 may register the second merge candidate in the merge candidate list so that the merge index having a value smaller than that of the second merge candidate is assigned to the first merge candidate. As a result, the image coding apparatus 600 can reduce the amount of coding and improve the coding efficiency when the first merge candidate is more likely to be selected as the merge candidate used for coding than the second merge candidate. Can be made to.
The second derivation unit 615 does not necessarily have to derive the second merge candidate so that the sum of the number of the first merge candidates and the number of the second merge candidates matches the determined maximum number. If the sum of the number of first merge candidates and the number of second merge candidates is less than the determined maximum number, for example, there may be a value of the merge index to which the merge candidates are not associated.
The prediction control unit 620 selects a merge candidate used for coding the coded block from the first merge candidate and the second merge candidate. That is, the prediction control unit 620 selects the merge candidate used for coding the coded block from the merge candidate list.
The coding unit 630 encodes an index (merge index) for identifying the selected merge candidate using the determined maximum number. Specifically, as shown in FIG. 5, the coding unit 630 encodes the bit string assigned to the index value of the selected merge candidate with variable length coding. Further, the coding unit 630 adds the coded index to the bit stream.
Here, the encoding unit 630 may further add information indicating the maximum number determined by the determination unit 611 to the bit stream. Specifically, the encoding unit 630 may write information indicating the maximum number in, for example, a slice header. As a result, the maximum number can be switched in an appropriate unit, and the coding efficiency can be improved.
Note that the coding unit 630 does not necessarily have to add information indicating the maximum number to the bit stream. For example, when the maximum number is predetermined by the standard, or when the maximum number is the same as the default value, the encoding unit 630 does not need to add the information indicating the maximum number to the bit stream.
Next, various operations of the image coding apparatus 600 configured as described above will be described.
FIG. 31 is a flowchart showing the processing operation of the image coding apparatus 600 according to the sixth embodiment.
First, the determination unit 611 determines the maximum number of merge candidates (S601). The first derivation unit 612 derives the first merge candidate (S602). The identification unit 613 identifies the first merge candidate (duplicate candidate) whose prediction direction, motion vector, and reference picture index overlap with other first merge candidates when a plurality of first merge candidates are derived (S603). ).
The determination unit 614 determines whether or not the number of first merge candidates excluding duplicate candidates is smaller than the determined maximum number (S604). Here, when it is determined that the number of first merge candidates excluding duplicate candidates is smaller than the determined maximum number (Yes in S604), the second derivation unit 615 has a predetermined vector as a motion vector. The merge candidate is derived as the second merge candidate (S605). On the other hand, if it is not determined that the number of first merge candidates excluding duplicate candidates is smaller than the determined maximum number (No in S604), the second derivation unit 615 does not derive the second merge candidate. These steps S604 and S605 correspond to step S514 in the fifth embodiment.
The prediction control unit 620 selects the merge candidate used for coding the coded block from the first merge candidate and the second merge candidate (S606). For example, the prediction control unit 620 selects the merge candidate having the minimum cost shown in Equation 1 from the merge candidate list, as in the first embodiment.
The encoding unit 630 encodes the index for identifying the selected merge candidate using the determined maximum number (S607). Further, the coding unit 630 adds the coded index to the bit stream.
As described above, according to the image coding apparatus 600 according to the present embodiment, a merge candidate having a predetermined vector as a motion vector can be derived as a second merge candidate. Therefore, the image coding apparatus 600 can derive a merge candidate having, for example, a motion vector for a stationary region, as a second merge candidate. That is, the image coding apparatus 600 can efficiently encode the coded target block having a predetermined motion, and can improve the coding efficiency.
Further, according to the image coding apparatus 600 according to the present embodiment, the index for identifying the merge candidate can be encoded by using the determined maximum number. That is, the index can be encoded independently of the number of merge candidates that are actually derived. Therefore, even if the information necessary for deriving the merge candidate (for example, information such as the co-located block) is lost, the index can be decoded on the decoding side, and the error tolerance can be improved. Further, on the decoding side, the index can be decoded independently of the number of merge candidates actually derived. That is, on the decoding side, the index decoding process can be performed without waiting for the merge candidate derivation process. That is, it is possible to generate a bit stream capable of performing the merge candidate derivation process and the index decoding process in parallel.
Further, according to the image coding apparatus 600 according to the present embodiment, when it is determined that the number of the first merge candidates is smaller than the maximum number, the second merge candidates can be derived. Therefore, the number of merge candidates can be increased within a range not exceeding the maximum number, and the coding efficiency can be improved.
Further, according to the image coding apparatus 600 according to the present embodiment, the second merge candidate can be derived according to the number of the first merge candidates excluding the duplicate first merge candidate. As a result, the number of second merge candidates can be increased, and the types of combinations of prediction direction, motion vector, and reference picture index that can be selected as merge candidates can be increased. Therefore, it is possible to further improve the coding efficiency.
In the present embodiment, the image coding device 600 includes the specific unit 613, but it is not always necessary to include the specific unit 613. That is, the flowchart shown in FIG. 31 does not necessarily have to include step S603. Even in such a case, the image coding apparatus 600 can encode the index for identifying the merge candidate using the determined maximum number, so that the error tolerance can be improved. It becomes.
Further, in the present embodiment, as shown in FIG. 31, after the first derivation unit 612 derives the first merge candidate, the specific unit 613 identifies the duplicate candidate, but the duplication candidates are not necessarily processed in this order. There is no need to. For example, the first derivation unit 612 identifies duplicate candidates in the process of deriving the first merge candidate, and derives the first merge candidate so that the identified duplicate candidate is not included in the first merge candidate. May be good. That is, the first derivation unit 612 may derive a merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index does not overlap with the already derived first merge candidate as the first merge candidate. More specifically, for example, when a merge candidate based on the left adjacent block has already been derived as the first merge candidate, the merge candidate based on the upper adjacent block does not overlap with the merge candidate based on the left adjacent block. , The first derivation unit 612 may derive the merge candidate based on the upper adjacent block as the first merge candidate. As a result, the first derivation unit 612 can exclude the merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index overlaps with the first merge candidate already derived from the first merge candidate. As a result, the image coding apparatus 600 can increase the number of second merge candidates and increase the types of combinations of prediction direction, motion vector, and reference picture index that can be selected as merge candidates. Therefore, the first derivation unit 612 can further improve the coding efficiency.
Further, in the present embodiment, after the first merge candidate is derived, it is determined whether or not the first merge candidate is smaller than the maximum number, and the second merge candidate is derived, but the processing is not necessarily performed in this order. It doesn't have to be done. For example, the image coding apparatus 600 may first derive a second merge candidate and register the derived second merge candidate in the merge candidate list. After that, the image coding apparatus 600 may derive the first merge candidate and overwrite the second merge candidate registered in the merge candidate list with the derived first merge candidate.
(Embodiment 7) In the seventh embodiment, the method of deriving the merge block candidate list size is different from that of the third embodiment. The method of deriving the merge block candidate list size in the present embodiment will be described in detail.
FIG. 32 is a block diagram showing a configuration of the image decoding device 700 according to the seventh embodiment. In FIG. 32, the same components as those in FIG. 20 are designated by the same reference numerals, and the description thereof will be omitted.
The image decoding device 700 is a device corresponding to the image coding device 500 according to the fifth embodiment. The image decoding device 700, for example, decodes the coded image included in the bit stream generated by the image coding device 500 according to the fifth embodiment block by block.
As shown in FIG. 32, the image decoding device 700 includes a variable length decoding unit 701, an inverse quantization unit 302, an inverse orthogonal conversion unit 303, an addition unit 304, a block memory 305, a frame memory 306, and an intra. It includes a prediction unit 307, an inter prediction unit 308, an inter prediction control unit 309, a switch 310, a merge block candidate calculation unit 711, and a colPic memory 312.
The variable-length decoding unit 701 performs variable-length decoding processing on the input bit stream to generate picture type information, a merge flag, and a quantization coefficient. Further, the variable-length decoding unit 701 performs a variable-length decoding process of the merge block index by using the number of mergeable candidates described later.
The merge block candidate calculation unit 711 uses the motion vector of the adjacent block of the block to be decoded and the motion vector of the co-located block stored in the colPic memory 312 (colPic information) to merge the block in the merge mode. Candidates are derived by the method described later. Further, the merge block candidate calculation unit 711 assigns the value of the merge block index to each of the derived merge block candidates. Then, the merge block candidate calculation unit 711 transmits the merge block candidate and the merge block index to the inter-prediction control unit 309.
FIG. 33 is a flowchart showing the processing operation of the image decoding apparatus according to the seventh embodiment.
In step S701, the variable length decoding unit 701 decodes the merge flag.
If the merge flag is "1" in step S702 (Yes in S702), in step S703, the merge block candidate calculation unit 711 calculates the number of mergeable candidates by the method described later. Then, the merge block candidate calculation unit 711 sets the calculated number of mergeable candidates in the merge block candidate list size.
Subsequently, in step S704, the variable-length decoding unit 701 uses the merge block candidate list size to perform variable-length decoding of the merge block index in the bitstream. In step S705, the merge block candidate calculation unit 711 generates merge block candidates from adjacent blocks and co-located blocks of the blocks to be decoded by the method described in the first or third embodiment or the method described later.
In step S706, the inter-prediction control unit 309 causes the inter-prediction unit 308 to generate an inter-prediction image using the motion vector of the merge block candidate indicated by the decoded merge block index, the reference picture index, and the prediction direction.
If the merge flag is "0" in step S702 (No in step S702), in step S707, the inter-prediction unit 308 uses the motion vector detection mode information decoded by the variable-length decoding unit 701 to perform inter-prediction. Generate an image.
If the merge block candidate list size calculated in step S703 is "1", the merge block index may be estimated to be "0" without being decrypted.
FIG. 34 is a flowchart showing the detailed processing of step S703 of FIG. 33. Specifically, FIG. 34 shows a method of determining whether the merge block candidate [N] is a mergeable candidate and calculating the number of mergeable candidates. Hereinafter, FIG. 34 will be described.
In step S711, the merge block candidate calculation unit 711 determines that the merge block candidate [N] is (1) a block decoded by intra-prediction, or (2) a block located outside the slice or picture boundary containing the block to be decoded. , Or (3) Determine if the block has not yet been decrypted.
Here, if the determination result in step S711 is true (Yes in S711), the merge block candidate calculation unit 711 sets the merge block candidate [N] as a non-mergeable candidate in step S712. On the other hand, if the determination result in step S711 is false (No in S711), the merge block candidate calculation unit 711 sets the merge block candidate [N] as a mergeable candidate in step S713.
In step S714, the merge block candidate calculation unit 711 determines whether the merge block candidate [N] is a mergeable candidate or a co-located merge block candidate. Here, if the determination result in step S714 is true (Yes in S714), the merge block candidate calculation unit 711 updates the number of merge block candidates by adding 1 to the number of merge block candidates in step S715. On the other hand, if the determination result in step S714 is false (No in S714), the merge block candidate calculation unit 711 does not update the number of mergeable candidates.
In this way, when the merge block candidate is a co-located merge block, the merge block candidate calculation unit 711 sets the number of mergeable candidates to 1 regardless of whether the co-located block is a mergeable candidate or a non-mergeable candidate. Is added. As a result, even if the information of the co-located merge block is lost due to packet loss or the like, the number of mergeable candidates does not match between the image encoding device and the image decoding device.
This number of mergeable candidates is set to the merge block candidate list size in step S703 of FIG. Further, in step S704 of FIG. 33, the merge block list size is used for variable length decoding of the merge block index. As a result, even if the reference picture information including the co-located block or the like is lost, the image decoding apparatus 700 can normally decode the merge block index.
FIG. 35 is a flowchart showing the detailed processing of step S705 of FIG. 33. Specifically, FIG. 35 shows a method of calculating merge block candidates. Hereinafter, FIG. 35 will be described.
In step S721, the merge block candidate calculation unit 711 acquires the motion vector, the reference picture index, and the prediction direction of the merge block candidate [N] and adds them to the merge block candidate list.
In step S722, the merge block candidate calculation unit 711 searches for unmergeable candidates and duplicate candidates from the merge block candidate list and deletes them, as shown in FIG. 26.
In step S723, the merge block candidate calculation unit 711 adds a new candidate to the merge block candidate list by the method described in the first or third embodiment or the same method as in FIG. 29.
FIG. 36 shows an example of the syntax for adding a merge block index to a bitstream. In FIG. 36, merge_idx represents the merge block index and merge_flag represents the merge flag. NumMergeCand represents the merge block candidate list size, and in this embodiment, the number of mergeable candidates calculated by the processing flow of FIG. 34 is set.
As described above, according to the image decoding apparatus 700 according to the present embodiment, the merge block candidate list size used when encoding or decoding the merge block index does not depend on the reference picture information including the co-located block and the like. It can be calculated by the method. As a result, the image decoding device 700 can appropriately decode the bit stream with improved error tolerance.
More specifically, the image decoding apparatus 700 according to the present embodiment always has a mergeable candidate if the merge block candidate is a co-located merge block, regardless of whether the co-located merge block is a mergeable candidate. Add 1 to the number. Then, the image decoding device 700 determines the bit string to be assigned to the merge block index by using the number of mergeable candidates calculated in this way. As a result, the image decoding apparatus 700 can normally decode the merge block index even when the reference picture information including the co-located block is lost.
Further, when the number of merge block candidates does not reach the number of mergeable candidates, the image decoding device 700 according to the present embodiment merges blocks new candidates having a new motion vector, a reference picture index, and a prediction direction. By adding it as a candidate, it becomes possible to appropriately decode a bit stream with improved coding efficiency.
(Embodiment 8) In the seventh embodiment, the image decoding device calculates by always adding 1 if the merge block candidate is a co-located merge block, regardless of whether the co-located merge block is a mergeable candidate. The number of mergeable candidates was used to determine the bit string to allocate to the merge block index. However, for example, in step S714 of FIG. 34, the image decoding apparatus uses the number of mergeable candidates calculated by always adding 1 to the merge block candidates other than the co-located merge block. You may decide which bit string to assign to the merge block index. That is, the image decoding device may allocate a bit string to the merge block index using the merge block candidate list size fixed to the maximum value N of the number of merge block candidates. That is, the image decoding device may consider all the merge block candidates as mergeable candidates, fix the merge block candidate list size to the maximum value N of the number of merge block candidates, and decode the merge block index.
For example, in the seventh embodiment, since the maximum value N of the number of merge block candidates is 5 (adjacent block A, adjacent block B, co-located merge block, adjacent block C, adjacent block D), the image decoding apparatus , You may always set the merge block candidate list size to 5 to decrypt the merge block index. As a result, the variable-length decoding unit of the image decoding device can decode the merge block index in the bit stream without referring to the information of the adjacent block or the co-located block. As a result, for example, the processing of step S714 and step S715 in FIG. 34 can be omitted, and the processing amount of the variable length decoding unit can be reduced.
FIG. 37 shows an example of the syntax when the merge block candidate list size is fixed to the maximum number of merge block candidates. If you want to fix the merge block candidate list size to the maximum number of merge block candidates, as shown in Figure 37, you can remove the NumMergeCand from the syntax.
A modified example of the image decoding apparatus according to the seventh embodiment as described above will be specifically described below as the image decoding apparatus according to the eighth embodiment.
FIG. 38 is a block diagram showing a configuration of the image decoding device 800 according to the eighth embodiment. The image decoding device 800 decodes the coded image included in the bit stream block by block. Specifically, the image decoding device 800 decodes, for example, the coded image included in the bit stream generated by the image coding device 600 according to the sixth embodiment block by block. The image decoding device 800 includes a merge candidate derivation unit 810, a decoding unit 820, and a prediction control unit 830.
The merge candidate derivation unit 810 corresponds to the merge block candidate calculation unit 711 in the seventh embodiment. The merge candidate derivation unit 810 derives the merge candidate. Then, the merge candidate derivation unit 810 generates, for example, a merge candidate list in which each derived merge candidate is associated with an index (merge index) for specifying the merge candidate.
As shown in FIG. 38, the merge candidate derivation unit 810 includes a determination unit 811, a first derivation unit 812, a specific unit 813, a determination unit 814, and a second derivation unit 815.
The determination unit 811 determines the maximum number of merge candidates. That is, the determination unit 811 determines the maximum value N of the number of merge block candidates.
For example, the determination unit 811 may determine the maximum number of merge candidates in the same manner as the determination unit 611 of the sixth embodiment. Further, for example, the determination unit 811 may determine the maximum number based on the information indicating the maximum number added to the bit stream. As a result, the image decoding device 800 can decode the encoded image by switching the maximum number in an appropriate unit.
Here, the determination unit 811 is provided in the merge candidate derivation unit 810, but may be provided in the decoding unit 820.
The first derivation unit 812 derives the first merge candidate in the same manner as the first derivation unit 612 of the sixth embodiment. Specifically, the first derivation unit 812 derives the first merge candidate based on the prediction direction, motion vector, and reference picture index used to decode the block spatially or temporally adjacent to the block to be decoded. .. Then, the first derivation unit 812, for example, associates the first merge candidate derived in this way with the merge index and registers it in the merge candidate list.
The first derivation unit 812 first merges, for example, the prediction direction, the motion vector, and the reference picture index used for decoding the blocks spatially adjacent to the decoding target block, excluding the unmergeable blocks. It may be derived as a candidate. As a result, the first derivation unit 812 can derive the first merge candidate from an appropriate block in order to obtain the merge candidate.
The identification unit 813 identifies the first merge candidate (duplicate candidate) whose prediction direction, motion vector, and reference picture index overlap with other first merge candidates when a plurality of first merge candidates are derived. Then, the specific unit 813 deletes the specified duplicate candidate from the merge candidate list.
The determination unit 814 determines whether or not the number of first merge candidates is smaller than the determined maximum number. Here, the determination unit 814 determines whether or not the number of first merge candidates excluding the specified duplicate first merge candidates is smaller than the determined maximum number.
The second derivation unit 815 derives a merge candidate having a predetermined vector as a motion vector as a second merge candidate when it is determined that the number of first merge candidates is smaller than the determined maximum number. Specifically, the second derivation unit 815 derives the second merge candidate so that the sum of the number of the first merge candidates and the number of the second merge candidates does not exceed the maximum number. Here, the second derivation unit 815 derives the second merge candidate so that the sum of the number of the first merge candidates excluding the duplicate candidates and the number of the second merge candidates does not exceed the maximum number.
The predetermined vector may be, for example, a zero vector as in the seventh embodiment. As a result, the second derivation unit 815 can derive a merge candidate having a motion vector for the stationary region. Therefore, the image decoding device 800 can appropriately decode the bit stream with improved coding efficiency. The predetermined vector does not necessarily have to be a zero vector.
Then, the second derivation unit 815 registers, for example, the second merge candidate derived in this way in the merge candidate list in association with the merge index. At this time, the second derivation unit 815 may register the second merge candidate in the merge candidate list so that the merge index having a value smaller than that of the second merge candidate is assigned to the first merge candidate. As a result, the image decoding device 800 can appropriately decode the bit stream with improved coding efficiency.
The second derivation unit 815 does not necessarily have to derive the second merge candidate so that the sum of the number of the first merge candidates and the number of the second merge candidates matches the determined maximum number. If the sum of the number of first merge candidates and the number of second merge candidates is less than the determined maximum number, for example, there may be a value of the merge index to which the merge candidates are not associated.
The decoding unit 820 decodes the encoded index added to the bit stream for identifying the merge candidate using the determined maximum number.
The prediction control unit 830 selects a merge candidate used for decoding the decryption target block from the first merge candidate and the second merge candidate based on the decoded index. That is, the prediction control unit 830 selects the merge candidate used for decoding the decryption target block from the merge candidate list.
Next, various operations of the image decoding apparatus 800 configured as described above will be described.
FIG. 39 is a flowchart showing the processing operation of the image decoding apparatus 800 according to the eighth embodiment.
First, the determination unit 811 determines the maximum number of merge candidates (S801). The first derivation unit 812 derives the first merge candidate (S802). The identification unit 813 identifies the first merge candidate (duplicate candidate) whose prediction direction, motion vector, and reference picture index overlap with other first merge candidates when a plurality of first merge candidates are derived (S803). ).
The determination unit 814 determines whether or not the number of first merge candidates excluding duplicate candidates is smaller than the determined maximum number (S804). Here, when it is determined that the number of first merge candidates excluding duplicate candidates is smaller than the determined maximum number (Yes in S804), the second derivation unit 815 derives the second merge candidate (S805). .. On the other hand, if it is not determined that the number of first merge candidates excluding duplicate candidates is smaller than the determined maximum number (No in S804), the second derivation unit 815 does not derive the second merge candidate.
The decoding unit 820 decodes the encoded index added to the bit stream for identifying the merge candidate using the determined maximum number (S806).
The predictive control unit 830 selects a merge candidate used for decoding the decryption target block from the first merge candidate and the second merge candidate based on the decoded index (S807). For example, the prediction control unit 830 selects the merge candidate having the minimum cost shown in Equation 1 from the merge candidate list, as in the first embodiment.
Here, the index decryption process (S806) was performed after the merge candidates were derived, but it is not always necessary to perform the index decryption process (S806) in such an order. For example, the index decoding process (S806) may be followed by the merge candidate derivation process (S802 to S805). Further, the index decoding process (S806) and the merge candidate derivation process (S802 to S805) may be performed in parallel. Thereby, the processing speed of decoding can be improved.
As described above, according to the image decoding apparatus 800 according to the present embodiment, a merge candidate having a predetermined vector as a motion vector can be derived as a second merge candidate. Therefore, the image decoding device 800 can derive, for example, a merge candidate having a motion vector for a stationary region as a second merge candidate. That is, the image decoding device 800 can appropriately decode a bit stream in which a block having a predetermined motion is efficiently encoded, and appropriately decodes a bit stream having improved coding efficiency. Is possible.
Further, according to the image decoding apparatus 800 according to the present embodiment, the index for identifying the merge candidate can be decoded using the determined maximum number. That is, the index can be decrypted independently of the number of merge candidates actually derived. Therefore, even if the information necessary for deriving the merge candidate (for example, information such as the co-located block) is lost, the image decoding apparatus 800 can decode the index and improve the error tolerance. It becomes. Further, the image decoding apparatus 800 can perform the index decoding process without waiting for the merge candidate derivation process, and can also perform the merge candidate derivation process and the index decoding process in parallel.
Further, according to the image decoding apparatus 800 according to the present embodiment, the second merge candidate can be derived when it is determined that the number of the first merge candidates is smaller than the maximum number. Therefore, the image decoding device 800 can increase the number of merge candidates within a range not exceeding the maximum number, and can appropriately decode the bit stream with improved coding efficiency.
Further, according to the image decoding apparatus 800 according to the present embodiment, the second merge candidate can be derived according to the number of the first merge candidates excluding the duplicate first merge candidate. As a result, the image decoding apparatus 800 can increase the number of second merge candidates and increase the types of combinations of prediction direction, motion vector, and reference picture index that can be selected as merge candidates. Therefore, the image decoding device 800 can appropriately decode the bit stream with further improved coding efficiency.
In the present embodiment, the image decoding device 800 is provided with the specific unit 813, but it is not always necessary to include the specific unit 813 as in the sixth embodiment. That is, the flowchart shown in FIG. 39 does not necessarily have to include step S803. Even in such a case, the image decoding apparatus 800 can decode the index for identifying the merge candidate using the determined maximum number, so that the error tolerance can be improved. ..
Further, in the present embodiment, as shown in FIG. 39, after the first derivation unit 812 derives the first merge candidate, the specific unit 813 identifies the duplicate candidate, but the duplication candidates are not necessarily processed in this order. There is no need to. For example, the first derivation unit 812 may derive a merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index does not overlap with the already derived first merge candidate as the first merge candidate. As a result, the first derivation unit 812 can exclude the merge candidate whose combination of the prediction direction, the motion vector, and the reference picture index overlaps with the already derived first merge candidate from the first merge candidate. As a result, the image decoding apparatus 800 can increase the number of second merge candidates and increase the types of combinations of prediction direction, motion vector, and reference picture index that can be selected as merge candidates. Therefore, the image decoding device 800 can appropriately decode the bit stream with further improved coding efficiency.
Further, in the present embodiment, after the first merge candidate is derived, it is determined whether or not the first merge candidate is smaller than the maximum number, and the second merge candidate is derived, but the processing is not necessarily performed in this order. It doesn't have to be done. For example, the image decoding apparatus 800 may first derive a second merge candidate and register the derived second merge candidate in the merge candidate list. After that, the image decoding apparatus 800 may derive the first merge candidate and overwrite the second merge candidate registered in the merge candidate list with the derived first merge candidate.
Although the image coding device and the image decoding device according to one or more aspects of the present invention have been described above based on the embodiments, the present invention is not limited to the embodiments. As long as it does not deviate from the gist of the present invention, one or more of the present embodiments may be modified by those skilled in the art, or may be constructed by combining components in different embodiments. It may be included within the scope of the embodiment.
In each of the above embodiments, each component may be configured by dedicated hardware or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. Here, the software that realizes the image coding device or the image decoding device of each of the above embodiments is the following program.
That is, this program is an image coding method that generates a bit stream by encoding an image block by block on a computer, and is a prediction direction, a motion vector, and a reference picture used for coding the coded block. Derived with a first derivation step that derives a merge candidate that is an index candidate as a first merge candidate, and a second derivation step that derives a merge candidate that has a predetermined vector as a motion vector as a second merge candidate. A selection step for selecting a merge candidate used for encoding the coded block from the first merge candidate and the second merge candidate, and an index for identifying the selected merge candidate are described above. An image coding method including a coding step to be added to the bitstream is executed.
Alternatively, this program is an image decoding method in which a computer decodes a coded image contained in a bit stream block by block, and is a candidate for a prediction direction, a motion vector, and a reference picture index used for decoding a block to be decoded. A first derivation step of deriving a certain merge candidate as a first merge candidate, a second derivation step of deriving a merge candidate having a predetermined vector as a motion vector as a second merge candidate, and merging from the bit stream. Based on the acquisition step of acquiring the index for identifying the candidate and the acquired index, the merge candidate used for decoding the decryption target block is selected from the first merge candidate and the second merge candidate. Perform an image decoding method that includes a selection step to select.
(Embodiment 9) Each of the above embodiments is performed by recording a program for realizing the configuration of the moving image coding method (image coding method) or the moving image decoding method (image decoding method) shown in each of the above embodiments on a storage medium. It becomes possible to easily carry out the processing shown in the above embodiment in an independent computer system. The storage medium may be a magnetic disk, an optical disk, a magneto-optical disk, an IC card, a semiconductor memory, or the like, as long as it can record a program.
Further, an application example of the moving image coding method (image coding method) and the moving image decoding method (image decoding method) shown in each of the above embodiments and a system using the same will be described here. The system is characterized by having an image coding / decoding device including an image coding device using an image coding method and an image decoding device using an image decoding method. Other configurations in the system can be changed appropriately as the case may be.
FIG. 40 is a diagram showing the overall configuration of the content supply system ex100 that realizes the content distribution service. The communication service provision area is divided into desired sizes, and fixed radio stations ex106, ex107, ex108, ex109, and ex110 are installed in each cell.
This content supply system ex100 is connected to the Internet ex101 via the Internet service provider ex102 and the telephone network ex104, and the base station ex106 to ex110 via the computer ex111, PDA (Personal Digital Assistant) ex112, camera ex113, mobile phone ex114, and game machine ex115. Each device such as is connected.
However, the content supply system ex100 is not limited to the configuration shown in FIG. 40, and any of the elements may be combined and connected. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex106 to ex110, which are fixed radio stations. Further, the devices may be directly connected to each other via short-range radio or the like.
The camera ex113 is a device capable of shooting moving images such as a digital video camera, and the camera ex116 is a device capable of shooting still images and moving images such as a digital camera. In addition, the mobile phone ex114 is a GSM (registered trademark) (Global System for Mobile Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or LTE (Long Term Evolution). The method, HSPA (High Speed Packet Access) mobile phone, PHS (Personal Handyphone System), etc., may be used.
In the content supply system ex100, live distribution and the like are possible by connecting the camera ex113 and the like to the streaming server ex103 through the base station ex109 and the telephone network ex104. In live distribution, content (for example, live music video, etc.) captured by a user using the camera ex113 is encoded as described in each of the above embodiments (that is, in one aspect of the present invention). (Functions as such an image encoding device), and transmits to the streaming server ex103. On the other hand, the streaming server ex103 streams the content data transmitted to the requested client. Clients include a computer ex111, a PDAex112, a camera ex113, a mobile phone ex114, a game machine ex115, and the like, which can decode the coded data. Each device that has received the distributed data decodes and reproduces the received data (that is, functions as an image decoding device according to one aspect of the present invention).
The captured data may be encoded by the camera ex113, the streaming server ex103 that performs the data transmission processing, or may be shared with each other. Similarly, the decryption process of the delivered data may be performed by the client, the streaming server ex103, or shared with each other. Further, not only the camera ex113 but also the still image and / or the moving image data taken by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The coding process in this case may be performed by any of the camera ex116, the computer ex111, and the streaming server ex103, or may be shared with each other.
Further, these coding / decoding processes are generally performed by the computer ex111 or the LSI ex500 of each device. The LSIex500 may be a single chip or a configuration composed of a plurality of chips. In addition, software for video coding / decoding is embedded in some recording medium (CD-ROM, flexible disk, hard disk, etc.) that can be read by a computer ex111 or the like, and the coding / decoding processing is performed using the software. You may. Further, when the mobile phone ex114 is equipped with a camera, the moving image data acquired by the camera may be transmitted. The moving image data at this time is the data encoded by the LSI ex500 of the mobile phone ex114.
Further, the streaming server ex103 may be a plurality of servers or a plurality of computers, and may disperse data for processing, recording, and distribution.
As described above, in the content supply system ex100, the client can receive and reproduce the encoded data. In this way, in the content supply system ex100, the client can receive, decode, and reproduce the information transmitted by the user in real time, and even a user who does not have special rights or equipment can realize personal broadcasting.
Not limited to the example of the content supply system ex100, as shown in FIG. 41, the digital broadcasting system ex200 also includes at least a moving image coding device (image coding device) or moving image decoding according to each of the above embodiments. Any of the devices (image decoding devices) can be incorporated. Specifically, in the broadcasting station ex201, the multiplexed data in which music data or the like is multiplexed with the video data is transmitted to the satellite ex202 or communication via radio waves. This video data is data encoded by the moving image coding method described in each of the above embodiments (that is, data encoded by the image coding apparatus according to one aspect of the present invention). In response to this, the broadcasting satellite ex202 transmits radio waves for broadcasting, and the radio waves are received by the home antenna ex204 capable of receiving satellite broadcasting. A device such as a television (receiver) ex300 or a set-top box (STB) ex217 decodes and reproduces the received multiplexed data (that is, functions as an image decoding device according to one aspect of the present invention).
In addition, the reader / recorder ex218 also reads and decodes the multiplexed data recorded on the recording medium ex215 such as DVD and BD, or encodes the video signal on the recording medium ex215 and, in some cases, multiplexes and writes the music signal. It is possible to implement the moving image decoding device or the moving image coding device shown in each of the above embodiments. In this case, the reproduced video signal is displayed on the monitor ex219, and the video signal can be reproduced in another device or system by the recording medium ex215 in which the multiplexed data is recorded. Further, a moving image decoding device may be mounted in a set-top box ex217 connected to a cable ex203 for cable TV or an antenna ex204 for satellite / terrestrial broadcasting, and this may be displayed on a TV monitor ex219. At this time, the moving image decoding device may be incorporated in the television instead of the set-top box.
FIG. 42 is a diagram showing a television (receiver) ex300 using the moving image decoding method and the moving image coding method described in each of the above embodiments. The TV ex300 acquires or outputs multiplexed data in which audio data is multiplexed on video data via an antenna ex204 or a cable ex203 that receives the above broadcast, and a tuner ex301 that outputs the received multiplexed data. Alternatively, the modulation / demodulation unit ex302 that modulates the multiplexed data to be transmitted to the outside, and the video data and audio data that separate the demodulated multiplexed data into video data and audio data, or are encoded by the signal processing unit ex306. It is provided with a multiplexing / separation unit ex303 that multiplexes the data.
Further, the television ex300 includes an audio signal processing unit ex304 and a video signal processing unit ex305 (an image coding device or an image according to one aspect of the present invention) that decodes each of the audio data and the video data or encodes the respective information. It has a signal processing unit ex306 having a function as a decoding device), a speaker ex307 for outputting a decoded audio signal, and an output unit ex309 having a display unit ex308 such as a display for displaying the decoded video signal. Further, the television ex300 has an interface unit ex317 having an operation input unit ex312 or the like for receiving input of user operation. Further, the television ex300 has a control unit ex310 that controls each unit in an integrated manner, and a power supply circuit unit ex311 that supplies electric power to each unit. In addition to the operation input unit ex312, the interface unit ex317 has a bridge ex313 connected to an external device such as a reader / recorder ex218, a slot unit ex314 for mounting a recording medium ex216 such as an SD card, and an external recording such as a hard disk. It may have a driver ex315 for connecting to media, a modem ex316 for connecting to a telephone network, and the like. The recording medium ex216 is capable of electrically recording information by a non-volatile / volatile semiconductor memory element that is stored. Each part of the TV ex300 is connected to each other via a synchronization bus.
First, a configuration in which the television ex300 decodes and reproduces the multiplexed data acquired from the outside by the antenna ex204 or the like will be described. The television ex300 receives a user operation from the remote controller ex220 or the like, and separates the multiplexed data demodulated by the modulation / demodulation unit ex302 by the multiplexing / separation unit ex303 based on the control of the control unit ex310 having a CPU or the like. Further, the television ex300 decodes the separated audio data by the audio signal processing unit ex304, and decodes the separated video data by the video signal processing unit ex305 using the decoding method described in each of the above embodiments. The decoded audio signal and video signal are output to the outside from the output unit ex309, respectively. When outputting, it is advisable to temporarily store these signals in buffers ex318, ex319, etc. so that the audio signal and the video signal are reproduced in synchronization. Further, the television ex300 may read the multiplexed data from the recording media ex215 and ex216 such as a magnetic / optical disk and an SD card, not from broadcasting or the like. Next, a configuration in which the television ex300 encodes an audio signal or a video signal and transmits it to the outside or writes it to a recording medium or the like will be described. The TV ex300 receives a user operation from a remote controller ex220 or the like, encodes an audio signal by the audio signal processing unit ex304 based on the control of the control unit ex310, and outputs a video signal by the video signal processing unit ex305 according to each of the above embodiments. It is encoded using the coding method described in. The encoded audio signal and video signal are multiplexed by the multiplexing / separating unit ex303 and output to the outside. When multiplexing, it is advisable to temporarily store these signals in buffers ex320, ex321, etc. so that the audio signal and the video signal are synchronized. It should be noted that a plurality of buffers ex318, ex319, ex320, and ex321 may be provided as shown in the figure, or one or more buffers may be shared. Furthermore, in addition to the figures shown, system overflow and underflow can be detected even between the modulation / demodulation section ex302 and the multiplexing / separation section ex303, for example.
In addition to acquiring audio data and video data from broadcasting and recording media, the TV ex300 has a configuration that accepts AV input from a microphone or camera, and performs encoding processing on the data acquired from them. May be good. Although the TV ex300 has been described here as a configuration capable of the above-mentioned coding processing, multiplexing, and external output, these processing cannot be performed, and only the above-mentioned reception, decoding processing, and external output are possible. It may be a configuration.
When reading or writing the multiplexed data from the recording medium with the reader / recorder ex218, the decoding process or the coding process may be performed with either the TV ex300 or the reader / recorder ex218, or the TV ex300. The reader / recorder ex218 may share the work with each other.
As an example, FIG. 43 shows the configuration of the information reproduction / recording unit ex400 when reading or writing data from an optical disc. The information reproduction / recording unit ex400 includes the elements ex401, ex402, ex403, ex404, ex405, ex406, and ex407 described below. The optical head ex401 irradiates the recording surface of the recording medium ex215, which is an optical disk, with a laser spot to write information, detects the reflected light from the recording surface of the recording medium ex215, and reads the information. The modulation recording unit ex402 electrically drives the semiconductor laser built in the optical head ex401 and modulates the laser beam according to the recorded data. The reproduction / demodulation unit ex403 amplifies the reproduction signal by electrically detecting the reflected light from the recording surface by the photodetector built in the optical head ex401, separates and demodulates the signal component recorded on the recording medium ex215, and is necessary. Information is played back. The buffer ex404 temporarily holds the information for recording on the recording medium ex215 and the information reproduced from the recording medium ex215. The disk motor ex405 rotates the recording medium ex215. The servo control unit ex406 moves the optical head ex401 to a predetermined information track while controlling the rotational drive of the disc motor ex405, and performs laser spot tracking processing. The system control unit ex407 controls the entire information reproduction / recording unit ex400. For the above read / write processing, the system control unit ex407 uses various information held in the buffer ex404, and new information is generated / added as necessary, and the modulation recording unit ex402 and the reproduction / demodulation unit are used. This is achieved by recording and reproducing information through the optical head ex401 while coordinating the ex403 and the servo control unit ex406. The system control unit ex407 is composed of, for example, a microprocessor, and executes those processes by executing a read / write program.
In the above, the optical head ex401 has been described as irradiating a laser spot, but it may be configured to perform higher-density recording using near-field light.
FIG. 44 shows a schematic diagram of the recording medium ex215, which is an optical disc. A guide groove (groove) is formed in a spiral shape on the recording surface of the recording medium ex215, and address information indicating an absolute position on the disk is recorded in advance on the information track ex230 by changing the shape of the groove. This address information includes information for specifying the position of the recording block ex231, which is a unit for recording data, and the recording block is specified by playing back the information track ex230 and reading the address information in a device that performs recording or playback. Can be done. Further, the recording medium ex215 includes a data recording area ex233, an inner peripheral area ex232, and an outer peripheral area ex234. The area used for recording user data is the data recording area ex233, and the inner circumference area ex232 and the outer circumference area ex234 arranged on the inner circumference or the outer circumference from the data recording area ex233 are used for specific purposes other than recording user data. Used. The information reproduction / recording unit ex400 reads / writes encoded audio data, video data, or multiplexed data obtained by multiplexing those data with respect to the data recording area ex233 of such a recording medium ex215.
In the above description, an optical disc such as a single-layer DVD or BD has been described as an example, but the present invention is not limited to these, and an optical disc having a multilayer structure and capable of recording other than the surface may be used. In addition, an optical disc with a structure that performs multidimensional recording / playback, such as recording information at the same location on a disc using light of various different wavelength colors and recording different layers of information from various angles. It may be.
Further, in the digital broadcasting system ex200, it is also possible to receive data from the satellite ex202 or the like by the car ex210 having the antenna ex205 and reproduce the moving image on the display device such as the car navigation ex211 of the car ex210. As for the configuration of the car navigation ex211, for example, among the configurations shown in FIG. 42, a configuration including a GPS receiver can be considered, and the same can be considered for a computer ex111, a mobile phone ex114, and the like.
FIG. 45A is a diagram showing a mobile phone ex114 using the moving image decoding method and the moving image coding method described in the above embodiment. The mobile phone ex114 has an antenna ex350 for transmitting and receiving radio waves to and from the base station ex110, a camera unit ex365 capable of taking images and still images, an image captured by the camera unit ex365, an image received by the antenna ex350, etc. The camera is provided with a display unit ex358 such as a liquid crystal display that displays the decoded data. The mobile phone ex114 further includes a main body having an operation key unit ex366, an audio output unit ex357 such as a speaker for outputting audio, an audio input unit ex356 such as a microphone for inputting audio, and a captured video. In the memory unit ex367 that stores encoded data such as still images, recorded audio, received video, still images, mail, etc. or decoded data, or in the interface unit with the recording media that also stores data. It has a certain slot part ex364.
Further, a configuration example of the mobile phone ex114 will be described with reference to FIG. 45B. The mobile phone ex114 has a power supply circuit unit ex361, an operation input control unit ex362, and a video signal processing unit ex355, as opposed to a main control unit ex360 that collectively controls each part of the main body unit having a display unit ex358 and an operation key unit ex366. , Camera interface unit ex363, LCD (Liquid Crystal Display) control unit ex359, modulation / demodulation unit ex352, multiplexing / separation unit ex353, audio signal processing unit ex354, slot unit ex364, memory unit ex367 are connected to each other via bus ex370. ing.
When the call end and the power key are turned on by the user's operation, the power circuit unit ex361 activates the mobile phone ex114 in an operable state by supplying power to each unit from the battery pack.
The mobile phone ex114 converts the voice signal picked up by the voice input unit ex356 in the voice call mode into a digital voice signal by the voice signal processing unit ex354 based on the control of the main control unit ex360 having a CPU, ROM, RAM, etc. , This is subjected to spectrum diffusion processing by the modulation / demodulation unit ex352, digital-analog conversion processing and frequency conversion processing by the transmission / reception unit ex351, and then transmitted via the antenna ex350. In addition, the mobile phone ex114 amplifies the received data received via the antenna ex350 in the voice call mode, performs frequency conversion processing and analog-to-digital conversion processing, and the modulation / demodulation unit ex352 performs spectrum reverse diffusion processing to perform the audio signal processing unit. After converting to an analog audio signal with ex354, this is output from the audio output unit ex357.
Further, when the e-mail is transmitted in the data communication mode, the text data of the e-mail input by the operation of the operation key unit ex366 of the main body unit is sent to the main control unit ex360 via the operation input control unit ex362. The main control unit ex360 performs spread spectrum processing on the modulation / demodulation unit ex352, digital-to-analog conversion processing and frequency conversion processing on the transmission / reception unit ex351, and then transmits the text data to the base station ex110 via the antenna ex350. .. When receiving an e-mail, the received data is processed in almost the reverse manner and output to the display unit ex358.
When transmitting video, still images, or video and audio in the data communication mode, the video signal processing unit ex355 compresses the video signal supplied from the camera unit ex365 by the moving image coding method shown in each of the above embodiments. It is encoded (that is, functions as an image coding device according to one aspect of the present invention), and the encoded video data is transmitted to the multiplexing / separating unit ex353. In addition, the audio signal processing unit ex354 encodes the audio signal picked up by the audio input unit ex356 while the camera unit ex365 is capturing images, still images, etc., and sends the encoded audio data to the multiplexing / separation unit ex353. To do.
The multiplexing / separating unit ex353 multiplexes the encoded video data supplied from the video signal processing unit ex355 and the encoded audio data supplied from the audio signal processing unit ex354 by a predetermined method, and is obtained as a result. The multiplexed data is subjected to spectrum diffusion processing by the modulation / demodulation unit (modulation / demodulation circuit unit) ex352, digital-analog conversion processing and frequency conversion processing by the transmission / reception unit ex351, and then transmitted via the antenna ex350.
Decrypts the multiplexed data received via the antenna ex350 when receiving video file data linked to a homepage, etc. in data communication mode, or when receiving e-mail with video and / or audio attached. In order to do so, the multiplexing / separating unit ex353 separates the multiplexed data into a bit stream of video data and a bit stream of audio data, and processes the video data encoded via the synchronization bus ex370. While supplying to the unit ex355, the encoded audio data is supplied to the audio signal processing unit ex354. The video signal processing unit ex355 decodes the video signal by decoding by the video decoding method corresponding to the video coding method shown in each of the above embodiments (that is, the image according to one aspect of the present invention). (Functions as a decoding device), the video and still images included in the moving image file linked to the homepage are displayed from the display unit ex358 via the LCD control unit ex359. Further, the audio signal processing unit ex354 decodes the audio signal, and the audio output unit ex357 outputs the audio.
Further, the terminals such as the mobile phone ex114 are referred to as transmission / reception terminals having both a encoder and a decoder, as well as transmission terminals having only a encoder and receiving terminals having only a decoder, similar to the television ex300. There are three possible implementation formats. Furthermore, the explanation was given that the digital broadcasting system ex200 receives and transmits multiplexed data in which music data and the like are multiplexed with video data, but data in which character data and the like related to video are multiplexed in addition to audio data. It may be the video data itself instead of the multiplexed data.
As described above, it is possible to use the moving image coding method or moving image decoding method shown in each of the above-described embodiments for any of the above-mentioned devices / systems, and by doing so, in each of the above-described embodiments. The described effect can be obtained.
Further, the present invention is not limited to the above-described embodiment, and various modifications or modifications can be made without departing from the scope of the present invention.
(Embodiment 10) The moving image coding method or apparatus shown in each of the above embodiments is appropriately switched between the moving image coding method or apparatus conforming to different standards such as MPEG-2, MPEG4-AVC, and VC-1 as necessary. By doing so, it is also possible to generate video data.
Here, when a plurality of video data conforming to different standards are generated, it is necessary to select a decoding method corresponding to each standard when decoding. However, since it is not possible to identify which standard the video data to be decoded conforms to, there arises a problem that an appropriate decoding method cannot be selected.
In order to solve this problem, the multiplexed data in which audio data or the like is multiplexed with the video data is configured to include identification information indicating which standard the video data conforms to. The specific configuration of the multiplexed data including the video data generated by the moving image coding method or the apparatus shown in each of the above embodiments will be described below. The multiplexed data is a digital stream in the MPEG-2 transport stream format.
FIG. 46 is a diagram showing the structure of the multiplexed data. As shown in FIG. 46, the multiplexed data is obtained by multiplexing one or more of a video stream, an audio stream, a presentation graphics stream (PG), and an interactive graphics stream. The video stream shows the main and sub-video of the movie, the audio stream (IG) shows the main audio part of the movie and the sub-audio that mixes with the main audio, and the presentation graphics stream shows the subtitles of the movie. Here, the main image indicates a normal image displayed on the screen, and the sub image is an image displayed on a small screen in the main image. In addition, the interactive graphics stream shows an interactive screen created by arranging GUI components on the screen. The video stream is encoded by the moving image coding method or device shown in each of the above embodiments, or a moving image coding method or device conforming to the conventional standards such as MPEG-2, MPEG4-AVC, and VC-1. ing. Audio stream is Dolby AC-3, Dolby Digital It is encoded by a method such as Plus, MLP, DTS, DTS-HD, or linear PCM.
Each stream contained in the multiplexed data is identified by a PID. For example, 0x1011 for video streams used for movie footage, 0x1100 to 0x111F for audio streams, 0x1200 to 0x121F for presentation graphics, and 0x1400 to 0x141F for interactive graphics streams. 0x1B00 to 0x1B1F are assigned to the video stream used for the secondary video, and 0x1A00 to 0x1A1F are assigned to the audio stream used for the secondary audio to be mixed with the main audio.
FIG. 47 is a diagram schematically showing how the multiplexed data is multiplexed. First, the video stream ex235 composed of a plurality of video frames and the audio stream ex238 composed of a plurality of audio frames are converted into PES packet sequences ex236 and ex239, respectively, and converted into TS packets ex237 and ex240, respectively. Similarly, the data of the presentation graphics stream ex241 and the interactive graphics ex244 are converted into PES packet strings ex242 and ex245, respectively, and further converted into TS packets ex243 and ex246. The multiplexing data ex247 is configured by multiplexing these TS packets into a single stream.
Figure 48 shows in more detail how the video stream is stored in the PES packet sequence. The first stage in FIG. 48 shows the video frame sequence of the video stream. The second row shows the PES packet sequence. As shown by the arrows yy1, yy2, yy3, yy4 in FIG. 48, a plurality of Video Presentation Units I picture, B picture, and P picture in the video stream are divided into pictures and stored in the payload of the PES packet. .. Each PES packet has a PES header, and the PES header stores PTS (Presentation Time-Stamp), which is the display time of the picture, and DTS (Decoding Time-Stamp), which is the decoding time of the picture.
FIG. 49 shows the format of the TS packet that is finally written to the multiplexed data. The TS packet is a 188-byte fixed-length packet consisting of a 4-byte TS header containing information such as a PID that identifies the stream and a 184-byte TS payload that stores data. The PES packet is divided and stored in the TS payload. To. In the case of BD-ROM, a 4-byte TP_Extra_Header is added to the TS packet to form a 192-byte source packet, which is written to the multiplexed data. Information such as ATS (Arrival_Time_Stamp) is described in TP_Extra_Header. ATS indicates the transfer start time of the TS packet to the PID filter of the decoder. As shown in the lower part of FIG. 49, source packets are lined up in the multiplexed data, and the number incremented from the beginning of the multiplexed data is called SPN (source packet number).
In addition to each stream such as video, audio, and subtitles, TS packets included in the multiplexed data include PAT (Program Association Table), PMT (Program Map Table), and PCR (Program Clock Reference). The PAT indicates what the PID of the PMT used in the multiplexed data is, and the PID of the PAT itself is registered as 0. The PMT has the PID of each stream such as video, audio, and subtitles included in the multiplexed data and the attribute information of the stream corresponding to each PID, and also has various descriptors related to the multiplexed data. Descriptors include copy control information that instructs whether to allow or disallow copying of multiplexed data. PCR corresponds to ATS in which the PCR packet is transferred to the decoder in order to synchronize ATC (Arrival Time Clock), which is the time axis of ATS, with STC (System Time Clock), which is the time axis of PTS / DTS. Has STC time information.
FIG. 50 is a diagram illustrating the data structure of PMT in detail. At the beginning of the PMT, a PMT header that describes the length of the data included in the PMT is placed. Behind it, a plurality of descriptors related to the multiplexed data are arranged. The copy control information and the like are described as descriptors. After the descriptor, a plurality of stream information about each stream included in the multiplexed data is arranged. The stream information is composed of a stream descriptor in which the stream type, the PID of the stream, and the attribute information of the stream (frame rate, aspect ratio, etc.) are described in order to identify the compression codec of the stream. There are as many stream descriptors as there are streams in the multiplexed data.
When recording on a recording medium or the like, the multiplexed data is recorded together with the multiplexed data information file.
As shown in FIG. 51, the multiplexed data information file is management information of the multiplexed data, has a one-to-one correspondence with the multiplexed data, and is composed of the multiplexed data information, stream attribute information, and an entry map.
As shown in FIG. 51, the multiplexed data information is composed of a system rate, a playback start time, and a playback end time. The system rate indicates the maximum transfer rate of the multiplexed data to the PID filter of the system target decoder described later. The ATS interval included in the multiplexed data is set to be less than or equal to the system rate. The playback start time is the PTS of the first video frame of the multiplexed data, and the playback end time is set by adding the playback interval of one frame to the PTS of the video frame at the end of the multiplexed data.
As the stream attribute information is shown in FIG. 52, the attribute information for each stream included in the multiplexed data is registered for each PID. Attribute information has different information for each video stream, audio stream, presentation graphics stream, and interactive graphics stream. The video stream attribute information includes what kind of compression codec the video stream was compressed with, what the resolution of the individual picture data that makes up the video stream is, what the aspect ratio is, and the frame rate. It has information such as how much it is. The audio stream attribute information includes what compression codec the audio stream was compressed with, how many channels the audio stream contains, what language it supports, what the sampling frequency is, and so on. Has the information of. This information is used for initializing the decoder before the player plays it.
In the present embodiment, the stream type included in the PMT is used among the above-mentioned multiplexed data. When the multiplexed data is recorded on the recording medium, the video stream attribute information included in the multiplexed data information is used. Specifically, in the moving image coding method or apparatus shown in each of the above embodiments, the moving image coding shown in each of the above embodiments is applied to the stream type or video stream attribute information included in the PMT. Provide steps or means to set unique information indicating that the video data is generated by the method or device. With this configuration, it becomes possible to distinguish between the video data generated by the moving image coding method or the apparatus shown in each of the above embodiments and the video data conforming to other standards.
Further, FIG. 53 shows the steps of the moving image decoding method in the present embodiment. In step exS100, the stream type included in the PMT or the video stream attribute information included in the multiplexed data information is acquired from the multiplexed data. Next, in step exS101, it is determined whether or not the stream type or the video stream attribute information indicates that it is the multiplexed data generated by the moving image coding method or apparatus shown in each of the above embodiments. To do. Then, when it is determined that the stream type or the video stream attribute information is generated by the moving image coding method or apparatus shown in each of the above embodiments, in step exS102, each of the above implementations is performed. Decoding is performed by the moving image decoding method shown in the form. If the stream type or video stream attribute information indicates that it conforms to the conventional standards such as MPEG-2, MPEG4-AVC, and VC-1, in step exS103, the conventional method is used. Decoding is performed by a moving image decoding method that conforms to the standard.
In this way, by setting a new eigenvalue in the stream type or the video stream attribute information, it is possible to determine whether or not the video stream decoding method or device shown in each of the above embodiments can be used for decoding. You can judge. Therefore, even when multiplexed data conforming to different standards is input, an appropriate decoding method or device can be selected, so that decoding can be performed without causing an error. Further, the moving image coding method or device shown in the present embodiment, or the moving image decoding method or device can be used for any of the above-mentioned devices and systems.
(Embodiment 11) The moving image coding method and apparatus, moving image decoding method and apparatus shown in each of the above embodiments are typically realized by an LSI which is an integrated circuit. As an example, FIG. 54 shows the configuration of the LSI ex500 integrated into one chip. The LSI ex500 includes the elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509 described below, and each element is connected via the bus ex510. When the power supply is on, the power supply circuit unit ex505 starts up in an operable state by supplying power to each unit.
For example, when performing coding processing, the LSI ex500 is AV based on the control of the control unit ex501 having the CPU ex502, the memory controller ex503, the stream controller ex504, the drive frequency control unit ex512, and the like. AV signal is input from microphone ex117, camera ex113, etc. by I / O ex509. The input AV signal is temporarily stored in an external memory ex511 such as SDRAM. Based on the control of the control unit ex501, the accumulated data is appropriately divided into a plurality of times according to the processing amount and the processing speed and sent to the signal processing unit ex507, and the signal processing unit ex507 encodes the audio signal and / or the video. The signal is coded. Here, the video signal coding process is the coding process described in each of the above embodiments. The signal processing unit ex507 further performs processing such as multiplexing the encoded audio data and the encoded video data in some cases, and outputs the stream I / O ex506 to the outside. This output multiplexed data is transmitted to the base station ex107 or written to the recording medium ex215. It is advisable to temporarily store data in the buffer ex508 so that it will be synchronized when multiplexing.
In the above description, the memory ex511 has been described as an external configuration of the LSI ex500, but it may be a configuration included inside the LSI ex500. The buffer ex508 is not limited to one, and may have multiple buffers. Further, the LSI ex500 may be integrated into one chip or a plurality of chips.
Further, in the above, it is assumed that the control unit ex501 has a CPU ex502, a memory controller ex503, a stream controller ex504, a drive frequency control unit ex512, and the like, but the configuration of the control unit ex501 is not limited to this configuration. For example, the signal processing unit ex507 may be configured to further include a CPU. By providing a CPU inside the signal processing unit ex507, the processing speed can be further improved. Further, as another example, the CPU ex502 may be configured to include a signal processing unit ex507 or, for example, an audio signal processing unit that is a part of the signal processing unit ex507. In such a case, the control unit ex501 is configured to include a signal processing unit ex507 or a CPU ex502 having a part thereof.
Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
Further, the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of circuit cells inside the LSI may be used.
Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. There is a possibility of adaptation of biotechnology.
(Embodiment 12) When decoding the video data generated by the moving image coding method or device shown in each of the above embodiments, the video data conforming to the conventional standards such as MPEG-2, MPEG4-AVC, and VC-1 is decoded. It is conceivable that the processing amount will increase as compared with the case. Therefore, in LSIex500, it is necessary to set the drive frequency higher than the drive frequency of CPUex502 when decoding video data conforming to the conventional standard. However, when the drive frequency is increased, there arises a problem that power consumption increases.
In order to solve this problem, a moving image decoding device such as a television ex300 or LSI ex500 is configured to identify which standard the video data conforms to and switch the drive frequency according to the standard. FIG. 55 shows the configuration ex800 in the present embodiment. The drive frequency switching unit ex803 sets the drive frequency high when the video data is generated by the moving image coding method or device shown in each of the above embodiments. Then, the decoding processing unit ex801 that executes the moving image decoding method shown in each of the above embodiments is instructed to decode the video data. On the other hand, when the video data is video data conforming to the conventional standard, as compared with the case where the video data is generated by the moving image coding method or the apparatus shown in each of the above embodiments. Set the drive frequency low. Then, the decoding processing unit ex802, which conforms to the conventional standard, is instructed to decode the video data.
More specifically, the drive frequency switching unit ex803 is composed of the CPU ex502 and the drive frequency control unit ex512 in FIG. 54. Further, the decoding processing unit ex801 that executes the moving image decoding method shown in each of the above embodiments and the decoding processing unit ex802 that conforms to the conventional standard correspond to the signal processing unit ex507 of FIG. 54. CPUex502 identifies which standard the video data conforms to. Then, the drive frequency control unit ex512 sets the drive frequency based on the signal from the CPU ex502. Further, the signal processing unit ex507 decodes the video data based on the signal from the CPU ex502. Here, for the identification of the video data, for example, it is conceivable to use the identification information described in the tenth embodiment. The identification information is not limited to the information described in the tenth embodiment, and may be any information that can identify which standard the video data conforms to. For example, it is possible to identify which standard the video data conforms to based on an external signal that identifies whether the video data is used for a television or a disc. In some cases, identification may be based on such an external signal. Further, it is conceivable that the drive frequency in the CPUex 502 is selected based on, for example, a lookup table in which the video data standard as shown in FIG. 57 and the drive frequency are associated with each other. The lookup table is stored in the buffer ex508 or the internal memory of the LSI, and the CPU ex502 can select the drive frequency by referring to this lookup table.
FIG. 56 shows the steps to implement the method of this embodiment. First, in step exS200, the signal processing unit ex507 acquires identification information from the multiplexed data. Next, in step exS201, CPUex502 identifies whether or not the video data is generated by the coding method or apparatus shown in each of the above embodiments based on the identification information. When the video data is generated by the coding method or device shown in each of the above embodiments, in step exS202, the CPU ex502 sends a signal for setting the drive frequency high to the drive frequency control unit ex512. Then, the drive frequency control unit ex512 sets a high drive frequency. On the other hand, if it is shown that the video data conforms to the conventional standards such as MPEG-2, MPEG4-AVC, and VC-1, CPUex502 drives the signal to set the drive frequency low in step exS203. Send to frequency control unit ex512. Then, in the drive frequency control unit ex512, the drive frequency is set to be lower than that in the case where the video data is generated by the coding method or the apparatus shown in each of the above embodiments.
Further, the power saving effect can be further enhanced by changing the voltage applied to the LSI ex500 or the device including the LSI ex500 in conjunction with the switching of the drive frequency. For example, when the drive frequency is set low, it is conceivable to set the voltage applied to the LSI ex500 or the device including the LSI ex500 lower than when the drive frequency is set high.
Further, as the driving frequency setting method, when the processing amount at the time of decoding is large, the driving frequency may be set high, and when the processing amount at the time of decoding is small, the driving frequency may be set low. Not limited to the method. For example, when the processing amount for decoding video data conforming to the MPEG4-AVC standard is larger than the processing amount for decoding video data generated by the moving image coding method or device shown in each of the above embodiments. It is conceivable to reverse the setting of the drive frequency as described above.
Further, the method of setting the drive frequency is not limited to the configuration in which the drive frequency is lowered. For example, when the identification information indicates that it is the video data generated by the moving image coding method or the apparatus shown in each of the above embodiments, the voltage applied to the LSI ex500 or the apparatus including the LSI ex500 is set high. However, if it indicates that the video data conforms to the conventional standards such as MPEG-2, MPEG4-AVC, and VC-1, it is also possible to set the voltage applied to the LSI ex500 or the device including the LSI ex500 low. Be done. Further, as another example, when the identification information indicates that it is the video data generated by the moving image coding method or the apparatus shown in each of the above embodiments, the drive of the CPU ex502 is stopped. If it is shown that the video data conforms to the conventional standards such as MPEG-2, MPEG4-AVC, and VC-1, there is a margin in processing, so the drive of CPUex502 should be paused. Is also possible. Even when the identification information indicates that it is the video data generated by the moving image coding method or the apparatus shown in each of the above embodiments, if there is a margin in the processing, the CPU ex502 is temporarily driven. It is also possible to stop it. In this case, it is conceivable to set the stop time shorter than in the case of indicating that the video data conforms to the conventional standards such as MPEG-2, MPEG4-AVC, and VC-1.
In this way, power saving can be achieved by switching the drive frequency according to the standard to which the video data conforms. Further, when the LSI ex500 or the device including the LSI ex500 is driven by using the battery, the life of the battery can be extended along with the power saving.
(Embodiment 13) A plurality of video data conforming to different standards may be input to the above-mentioned devices / systems such as televisions and mobile phones. In this way, in order to enable decoding even when a plurality of video data conforming to different standards are input, the signal processing unit ex507 of LSI ex500 needs to support the plurality of standards. However, if the signal processing unit ex507 corresponding to each standard is used individually, there arises a problem that the circuit scale of the LSI ex500 becomes large and the cost increases.
In order to solve this problem, a decoding processing unit for executing the moving image decoding method shown in each of the above embodiments, and decoding conforming to conventional standards such as MPEG-2, MPEG4-AVC, and VC-1. The configuration is such that a part is shared with the processing unit. An example of this configuration is shown in ex900 of FIG. 58A. For example, the moving image decoding method shown in each of the above embodiments and the moving image decoding method conforming to the MPEG4-AVC standard are processed in processing such as entropy coding, dequantization, deblocking filter, and motion compensation. Some of the contents are common. For common processing content, the decoding processing unit ex902 corresponding to the MPEG4-AVC standard is shared, and for other processing content specific to one aspect of the present invention that does not correspond to the MPEG4-AVC standard, a dedicated decoding processing unit is used. A configuration using ex901 is conceivable. In particular, since one aspect of the present invention is characterized by motion compensation, for example, a dedicated decoding processing unit ex901 is used for motion compensation, and other entropy decoding, deblocking filter, and dequantization are used. It is conceivable to share the decoding processing unit for any or all of the processing. Regarding the sharing of the decoding processing unit, regarding the common processing content, the decoding processing unit for executing the moving image decoding method shown in each of the above embodiments is shared, and the processing content peculiar to the MPEG4-AVC standard is shared. May be configured to use a dedicated decoding processing unit.
In addition, ex1000 in Fig. 58B shows another example of partially sharing the processing. In this example, a dedicated decoding processing unit ex1001 corresponding to the processing content peculiar to one aspect of the present invention, a dedicated decoding processing unit ex1002 corresponding to the processing content peculiar to other conventional standards, and one aspect of the present invention. It is configured to use the common decoding processing unit ex1003 corresponding to the processing contents common to the moving image decoding method according to the above and the moving image decoding method of other conventional standards. Here, the dedicated decoding processing units ex1001 and ex1002 are not necessarily specialized in one aspect of the present invention or processing contents peculiar to other conventional standards, but can execute other general-purpose processing. May be good. It is also possible to implement the configuration of this embodiment with LSI ex500.
As described above, the LSI circuit scale can be reduced by sharing the decoding processing unit for the processing contents common to the moving image decoding method according to one aspect of the present invention and the moving image decoding method of the conventional standard. Moreover, it is possible to reduce the cost.
The image coding method and image decoding method according to one aspect of the present invention are advantageously used in the moving image coding method and decoding method.
100, 200, 500, 600 image encoder 101 subtraction part 102 Orthogonal converter 103 Quantization section 104, 302 Inverse quantization unit 105, 303 Inverse orthogonal converter 106, 304 Adder 107, 305 block memory 108, 306 frame memory 109, 307 Intra Prediction Department 110, 308 Inter Prediction Department 111, 309 Inter-prediction control unit 112 Picture type determination unit 113, 310 switches 114, 311, 514, 711 Merge block candidate calculation unit 115, 312 colPic memory 116, 516 Variable length coder 210, 410, 610, 810 Merge candidate derivation unit 211, 411, 612, 812 1st derivation part 212, 412, 615, 815 2nd out-licensing unit 220, 430, 620, 830 Prediction control unit 230, 630 Encoding section 300, 400, 700, 800 image decoder 301, 701 Variable length decoder 420, 820 Decryptor 611, 811 Decision Department 613, 813 Specific part 614, 814 Judgment unit
62 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34 Sheet 35 Sheet 36 Sheet 37 Sheet 38 Sheet 39 Sheet 40 Sheet 41 Sheet 42 Sheet 43 Sheet 44 Sheet 45 Sheet 46 Sheet 47 Sheet 48 Sheet 49 Sheet 50 Sheet 51 Sheet 52 Sheet 53 Sheet 54 Sheet 55 Sheet 56 Sheet 57 Sheet 58 Sheet 59 Sheet 60 Sheet 61 Sheet 62
Every citation, both waysCites: the store holds 1 of 2
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US10972748B2 | Cited by | United States of America | Applicant |
| US11595684B2 | Cited by | United States of America | Applicant |
| JP2016040966A | Cited by | Japan | Examiner |
| US10623766B2 | Cited by | United States of America | Applicant |
| US10264276B2 | Cited by | United States of America | Applicant |
| WO2006019093A1 | Cites | World Intellectual Property Organization (WIPO) | – |
| Bin Li, Jizheng Xu, Feng Wu, Houqiang Li,On merge candidate construction,Joint Collaborative Team on Video Coding (JCT-VC)o,2011年 3月16日,JCTVC-E146_r3 | Non-patent | – | – |
| Thomas Wiegand, Woo-Jin Han, Benjamin Bross, Jens-Rainer Ohm, Gary J. Sullivan,WD3: Working Draft 3 of High-Efficiency Video Coding,Joint Collaborative Team on Video Coding (JCT-VC),2011年 4月 6日,JCTVC-E603_d2 | Non-patent | – | – |
| Minhua Zhou, Vivienne Sze,A study on HM2.0 bitstream parsing and error resiliency issue,Joint Collaborative Team on Video Coding (JCT-VC),2011年 3月16日,JCTVC-E0118 | Non-patent | – | – |
65 members in 15 offices
Priority claims9
| Document | Office | Kind | Date |
|---|---|---|---|
| 201161490777 | United States of America | P | |
| 201161490777 | United States of America | P | |
| 61490777 | United States of America | – | |
| 2012003386 | Japan | W | |
| 2012003386 | Japan | W | |
| 61490777 | – | – | – |
| JP2012003386 | – | – | – |
| US201161490777P | – | – | – |
| WO2012JP03386 | – | – | – |
Members65
| Document | Office | Kind | |
|---|---|---|---|
| CA2834123A1 | Canada | A1 | |
| CA3062382A1 | Canada | A1 | |
| CA3197949A1 | Canada | A1 | |
| WO2012164880A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2013010869A1 | United States of America | A1 | |
| TW201311008A | Taiwan Province of China | A | |
| MX2013012132A | Mexico | A | |
| CN103518376A | China | A | |
| KR20140026429A | Republic of Korea | A | |
| EP2717575A1 | European Patent Office (EPO) | A1 | |
| EP2717575A4 | European Patent Office (EPO) | A4 | |
| JPWO2012164880A1 | Japan | A1 | |
| JP5865366B2This record | Japan | B2 | |
| JP2016040966A | Japan | A | |
| TWI562612B | Taiwan Province of China | B | |
| BR112013027344A2 | Brazil | A2 | |
| JP6089312B2 | Japan | B2 | |
| US9615107B2 | United States of America | B2 | |
| US2017163987A1 | United States of America | A1 | |
| BR112013027344A8 | Brazil | A8 | |
| CN103518376B | China | B | |
| US9838695B2 | United States of America | B2 | |
| US2018063533A1 | United States of America | A1 | |
| US10034001B2 | United States of America | B2 | |
| KR101896734B1 | Republic of Korea | B1 | |
| EP2717575B1 | European Patent Office (EPO) | B1 | |
| US2018295366A1 | United States of America | A1 | |
| EP3410718A1 | European Patent Office (EPO) | A1 | |
| TR2018019396T4 | Türkiye | T4 | |
| TR201819396T4 | Türkiye | T4 | |
| DK2717575T3 | Denmark | T3 | |
| ES2703799T3 | Spain | T3 | |
| PL2717575T3 | Poland | T3 | |
| US2019124334A1 | United States of America | A1 | |
| US2019124335A1 | United States of America | A1 | |
| EP3410718B1 | European Patent Office (EPO) | B1 | |
| MY173463A | Malaysia | A | |
| CA2834123C | Canada | C | |
| EP3614665A1 | European Patent Office (EPO) | A1 | |
| US10595023B2 | United States of America | B2 | |
| EP3410718B8 | European Patent Office (EPO) | B8 | |
| PL3410718T3 | Poland | T3 | |
| ES2769378T3 | Spain | T3 | |
| US10708598B2 | United States of America | B2 | |
| US10721474B2 | United States of America | B2 | |
| US2020288142A1 | United States of America | A1 | |
| US11115664B2 | United States of America | B2 | |
| US2021360257A1 | United States of America | A1 | |
| EP3614665B1 | European Patent Office (EPO) | B1 | |
| BR112013027344B1 | Brazil | B1 | |
| ES2769378T8 | Spain | T8 | |
| ES2911670T3 | Spain | T3 | |
| EP4007276A1 | European Patent Office (EPO) | A1 | |
| PL3614665T3 | Poland | T3 | |
| US11570444B2 | United States of America | B2 | |
| US2023128151A1 | United States of America | A1 | |
| EP4007276B1 | European Patent Office (EPO) | B1 | |
| EP4213483A1 | European Patent Office (EPO) | A1 | |
| CA3062382C | Canada | C | |
| PL4007276T3 | Poland | T3 | |
| ES2959039T3 | Spain | T3 | |
| US11979582B2 | United States of America | B2 | |
| US2024283941A1 | United States of America | A1 | |
| US12375684B2 | United States of America | B2 | |
| US2025330611A1 | United States of America | A1 |
16 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Written notification of registration of transferJAPANESE INTERMEDIATE CODE: R350R350 | R350 | |
| Request for trust registration of transfer of rightJAPANESE INTERMEDIATE CODE: R313133S131 | S131 | |
| Request for trust registration of transfer of rightJAPANESE INTERMEDIATE CODE: R313133S131 | S131 | |
| Written request for trust registrationJAPANESE INTERMEDIATE CODE: R313Z02SZ02 | SZ02 | |
| Written request for trust registrationJAPANESE INTERMEDIATE CODE: R313Z02SZ02 | SZ02 | |
| Request for trust registration of transfer of rightJAPANESE INTERMEDIATE CODE: R313133S131 | S131 | |
| Written request for trust registrationJAPANESE INTERMEDIATE CODE: R313Z02SZ02 | SZ02 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 5865366
- Publication, DOCDB
- 5865366
- Publication, EPODOC
- JP5865366B
- Application
- 2013517853
- Application, DOCDB
- 2013517853
- Application, EPODOC
- JP20130517853
Titles2
- Japanese
- 画像符号化方法、画像符号化装置、画像復号方法、画像復号装置、および、画像符号化復号装置
- English
- Image coding method, image coding device, image decoding method, image decoding device, and image coding / decoding device
Classification
- CPC, 12
- H04N19/51
- H04N19/102
- H04N19/139
- H04N19/197
- H04N19/105
- H04N19/52
- H04N19/176
- H04N19/70
- H04N19/13
- H04N19/134
- H04N19/577
- H04N19/184
- IPC, 2
- H04N19 52
- H04N19 70
