Decoding device, decoding method, and reception device
Abstract
This record has no abstract on file.
Term
Projected expiry 4 June 2029.
- Priority
- Filed
- Granted
- Today
- Projected expiry
12 claims: 10 independent, 2 dependent
- 1複数の色成分を有する符号化画像データを復号する復号化装置であって、 前記符号化画像データを可変長復号し、可変長復号した結果得られるデータである可変長復号データを生成する可変長復号部と、 前記複数の色成分を所定の数の色成分に分割し、前記可変長復号部が生成した可変長復号データに含まれる前記所定の数の色成分それぞれの復号処理の対象となるデータそれぞれを、前記所定の数の色成分それぞれの復号処理に必要な情報として取得する分割部と、 取得されたそれぞれの前記情報を用いて、前記所定の数の色成分それぞれの復号画像を並行して生成する、前記所定の数の色成分それぞれに対応した前記所定の数の画像再構成部と 、 前記符号化画像データの色差フォーマットに応じて、前記所定の数の色成分への分割方法を決定する制御部とを備え、 前記分割部は、前記制御部が決定した分割方法に従って、前記複数の色成分を前記所定の数の色成分に分割し、前記所定の数の色成分それぞれの復号処理に必要な情報それぞれを取得し、 前記画像再構成部のそれぞれは、前記制御部が決定した分割方法に従って、前記所定の数の色成分それぞれの復号画像を生成する 復 号化装置。
- 2さらに、 前記可変長復号部が生成した可変長復号データを用いて、前記所定の数の色成分それぞれの復号処理のうち少なくとも2つの復号処理の間で共通して必要となる処理である共通処理が行われる場合に、前記共通処理を行い前記共通処理の結果得られるデータである共通処理データを生成するパラメータ計算部を備え、 前記分割部は、前記共通処理データから、前記共通処理が行われる色成分それぞれの復号処理に必要な情報それぞれをさらに取得し、 前記共通処理が行われる色成分に対応したそれぞれの前記画像再構成部は、取得されたそれぞれの前記情報をさらに用いて、前記共通処理が行われる色成分それぞれの復号画像を生成する 請求項1に記載の復号化装置。
- 3前記パラメータ計算部は、前記所定の数の色成分のうちの第一の色成分の復号処理で、前記第一の色成分の復号処理の対象となる第一データから第二データが生成され、かつ、第二の色成分の復号処理で、前記第一データから前記第二データが生成された後に前記第二データから第三データが生成される場合に、前記第二データ及び前記第三データを生成し、 前記分割部は、前記第二データ及び前記第三データを、それぞれ前記第一の色成分及び前記第二の色成分の復号処理に必要な情報として取得し、 前記第一の色成分に対応した前記画像再構成部は、前記第二データを用いて、前記第一の色成分の復号画像を生成し、 前記第二の色成分に対応した前記画像再構成部は、前記第三データを用いて、前記第二の色成分の復号画像を生成する 請求項2に記載の復号化装置。
- 4前記制御部は、前記符号化画像データが有する色成分のうち、第三の色成分の画素数よりも第四の色成分の画素数が多い場合、前記第三の色成分よりも前記第四の色成分を多く分割するように、前記所定の数の色成分への分割方法を決定する 請求項 1~3のいずれか1項 に記載の復号化装置。
- 5前記画像再構成部のそれぞれは、入力される色成分の復号処理に必要な情報が、前記所定の数の色成分のうちのいずれの色成分の復号処理に必要な情報であるかを前記制御部から取得し、当該色成分の復号画像を生成するように指示信号を出力する構成変更部を備える 請求項 1~4のいずれか1項 に記載の復号化装置。
- 6さらに、 前記画像再構成部のそれぞれが生成した前記所定の数の色成分それぞれの復号画像を結合して、前記符号化画像データの復号画像を生成する結果画像結合部を備え、 前記結果画像結合部は、前記制御部が決定した分割方法に基づいて、前記符号化画像データの復号画像を生成する 請求項 1~5 のいずれか1項に記載の復号化装置。
- 7さらに、 前記符号化画像データの領域を複数の領域に分割する領域分割部と、 分割された前記複数の領域それぞれの符号化画像データに対応した複数の前記可変長復号部と、 前記複数の前記可変長復号部のそれぞれに対応した複数の前記分割部と、 前記複数の分割部のそれぞれに対応した複数の前記画像再構成部と、 前記複数の画像再構成部のそれぞれが生成した復号画像を結合して、前記符号化画像データの復号画像を生成する結果画像領域結合部とを備える 請求項1~ 6 のいずれか1項に記載の復号化装置。
- 8さらに、 前記分割部が分割した所定の数の色成分それぞれについて、前記符号化画像データの色成分の領域を複数の領域に分割する領域分割部と、 前記所定の数の色成分それぞれについて、分割された前記複数の領域それぞれに対応した複数の前記画像再構成部とを備える 請求項1~ 6 のいずれか1項に記載の復号化装置。
- 9複数の色成分を有する符号化画像データを復号する復号化方法であって、 前記符号化画像データを可変長復号し、可変長復号した結果得られるデータである可変長復号データを生成する可変長復号ステップと、 前記複数の色成分を所定の数の色成分に分割し、前記可変長復号ステップで生成された可変長復号データに含まれる前記所定の数の色成分それぞれの復号処理の対象となるデータそれぞれを、前記所定の数の色成分それぞれの復号処理に必要な情報として取得する分割ステップと、 取得されたそれぞれの前記情報を用いて、前記所定の数の色成分それぞれの復号画像を並行して生成する画像再構成ステップと 、 前記符号化画像データの色差フォーマットに応じて、前記所定の数の色成分への分割方法を決定する制御ステップとを含み、 前記分割ステップでは、前記制御ステップで決定された分割方法に従って、前記複数の色成分を前記所定の数の色成分に分割し、前記所定の数の色成分それぞれの復号処理に必要な情報それぞれを取得し、 前記画像再構成ステップでは、前記制御ステップで決定された分割方法に従って、前記所定の数の色成分それぞれの復号画像を生成する 復 号化方法。
- 10複数の色成分を有する符号化画像データを復号する復号化装置を制御する集積回路であって、 前記符号化画像データを可変長復号し、可変長復号した結果得られるデータである可変長復号データを生成する可変長復号部と、 前記複数の色成分を所定の数の色成分に分割し、前記可変長復号部が生成した可変長復号データに含まれる前記所定の数の色成分それぞれの復号処理の対象となるデータそれぞれを、前記所定の数の色成分それぞれの復号処理に必要な情報として取得する分割部と、 取得されたそれぞれの前記情報を用いて、前記所定の数の色成分それぞれの復号画像を並行して生成する、前記所定の数の色成分それぞれに対応した前記所定の数の画像再構成部と 、 前記符号化画像データの色差フォーマットに応じて、前記所定の数の色成分への分割方法を決定する制御部とを備え、 前記分割部は、前記制御部が決定した分割方法に従って、前記複数の色成分を前記所定の数の色成分に分割し、前記所定の数の色成分それぞれの復号処理に必要な情報それぞれを取得し、 前記画像再構成部のそれぞれは、前記制御部が決定した分割方法に従って、前記所定の数の色成分それぞれの復号画像を生成する 集 積回路。
- 11複数の色成分を有する符号化画像データを復号するためのプログラムであって、 前記符号化画像データを可変長復号し、可変長復号した結果得られるデータである可変長復号データを生成する可変長復号ステップと、 前記複数の色成分を所定の数の色成分に分割し、前記可変長復号ステップで生成された可変長復号データに含まれる前記所定の数の色成分それぞれの復号処理の対象となるデータそれぞれを、前記所定の数の色成分それぞれの復号処理に必要な情報として取得する分割ステップと、 取得されたそれぞれの前記情報を用いて、前記所定の数の色成分それぞれの復号画像を並行して生成させる復号画像生成ステップと、 生成された前記所定の数の色成分それぞれの復号画像を結合させて、前記符号化画像データの復号画像を生成させる結果画像結合ステップと 、 前記符号化画像データの色差フォーマットに応じて、前記所定の数の色成分への分割方法を決定する制御ステップとをコンピュータに実行させ、 前記分割ステップでは、前記制御ステップで決定された分割方法に従って、前記複数の色成分を前記所定の数の色成分に分割し、前記所定の数の色成分それぞれの復号処理に必要な情報それぞれを取得し、 前記画像再構成ステップでは、前記制御ステップで決定された分割方法に従って、前記所定の数の色成分それぞれの復号画像を生成する プ ログラム。
- 12放送されたストリームを受信する受信装置であって、 前記ストリームから、符号化された複数の色成分を有する動画像データである符号化画像データを分離するデータ分離部と、 前記データ分離部が分離した符号化画像データを復号し、復号画像を生成する請求項 10 に記載の集積回路と、 前記集積回路が生成した復号画像を映像信号として出力する出力制御部と を備える受信装置。
Independent claims12
239 paragraphs, as filed
The present invention relates to a decryption device, a decoding method, and a receiving device for a compressed image, and more particularly to a decoding device, a decoding method, and a receiving device that require high data processing capability.
Conventionally, there is an MPEG (Moving Picture Expert Group) coding technology as a compression coding technology (hereinafter, simply referred to as coding) using a difference between frames. For example, the MPEG2 standard (see Non-Patent Document 1) and the H.264 standard (see Non-Patent Document 2) have been widely put into practical use in the fields of broadcasting and storage in optical disks.
In standards such as digital TV broadcasting and Blu-ray Disc, which are currently being put to practical use in various countries, the color difference format is 4: 2: 0 and the image size is 1920x1080.
On the other hand, discussions are also being held to expand the standard in the future with the aim of further improving the image quality. More specifically, for example, there is a possibility of adopting 4: 2: 2 or 4: 4: 4 in the color difference format and 3840x2160 in the image size.
These standard extensions are accompanied by a dramatic increase in the amount of computation required for decoding, and the development of a single decoder chip is a factor such as processing speed, chip size, and required memory bandwidth at the current technical level. It becomes expensive. Therefore, it is effective to use a plurality of chips having a processing capacity compatible with the current standard in parallel to cope with the expansion of the standard in the future.
In MPEG2, one picture is composed of one or more slices, and one slice is composed of one or more macroblocks. One macroblock is composed of blocks of luminance, color difference Cb, and color difference Cr, and is coded in units of these blocks at the time of coding.
Since the MPEG coded bit stream is hierarchically configured in this way, when decoding using a plurality of chips, various units processed by one chip can be considered. As a conventional method of dividing a coded bit stream for parallel processing, a method of dividing an image information encoded by MPEG into slice units or a method of dividing into a plurality of color components such as luminance information and color difference information. (See, for example, Patent Document 1).
Among these, the method of dividing into a plurality of color components such as luminance information and color difference information has an advantage that data transfer between chips that perform parallel processing is small. In this case, the reference image data for motion compensation used by one chip in the decoding process is only the decoding result of that chip. On the other hand, in the case of spatial division such as slice division and decoding, control for passing the reference image and data transfer between chips are required. Therefore, the method of dividing into the luminance information and the color difference information can be configured and controlled more concisely than the method of dividing by space.
FIG. 1 is a diagram showing a decoding device that divides the conventional luminance information and color difference information described in Patent Document 1 and performs a decoding process. In FIG. 1, the decoding device that divides the conventional luminance information and the color difference information and performs the decoding process includes a luminance decoder 101, a luminance decoder 102, a luminance memory 103, a luminance memory 104, and a combiner 105. ing.
The luminance decoder 101 and the color difference decoder 102 input a coded bit stream of color image information to decode the luminance component and the color difference component of the color image information, respectively.
The luminance memory 103 and the color difference memory 104 store the luminance component and the color difference component of the image information decoded by the luminance decoder 101 and the luminance decoder 102, respectively.
The combiner 105 combines the decoded image of the luminance component output by the luminance decoder 101 with the decoded image of the color difference component output by the color difference decoder 102.
<p num="0013"><patcit num="1"><text>Japanese Unexamined Patent Publication No. 10-164584</text></patcit></p>
<p num="0014"><nplcit num="1"><text>MPEG2 ISO / IEC 13818-2 standard, ITU-T H.262 standard</text></nplcit><nplcit num="2"><text>H.264 ISO / IEC14496-10 standard, ITU-T H.264 standard</text></nplcit></p>
<p num="0015"> However, the conventional decoding apparatus has a problem that the efficiency of the decoding process is lowered when the encoded image data is divided into a plurality of color components and decoded.</p><p num="0016"> In the configuration of the conventional decoding device, the same MPEG coded bit stream is input to the luminance decoder 101 and the color difference decoder 102, and each of the luminance decoder 101 and the color difference decoder 102 performs decoding processing. Since these decoding processes require a large amount of processing, it is important to reduce this amount of processing. Therefore, as a result of diligent research, the inventors of the present application have a problem that the efficiency of parallel processing is lowered because many conventional decoding devices perform overlapping processing by two decoders such as variable length decoding processing. I came to find out that I am.</p><p num="0017"> In particular, since the arithmetic code decoding process, which is a method of variable-length decoding of the H.264 standard, requires a large amount of processing, the effect of the efficiency decrease due to the duplication is large.</p><p num="0018"> The present invention solves the above-mentioned conventional problems, and is a decoding device capable of suppressing a decrease in parallel processing efficiency when decoding encoded image data by dividing it into a plurality of color components. It is an object of the present invention to provide a method and a receiving device.</p>
<p num="0019"> In order to solve the conventional problem, the decoding device according to one aspect of the present invention is a decoding device that decodes coded image data having a plurality of color components, and the coded image data has a variable length. A variable-length decoding unit that generates variable-length decoded data, which is data obtained as a result of decoding and variable-length decoding, and a plurality of color components are divided into a predetermined number of color components, and the variable-length decoding unit is generated. A division unit for acquiring each of the data to be decoded for each of the predetermined number of color components included in the variable-length decoded data as information necessary for the decoding process for each of the predetermined number of color components, and an acquisition unit. Each of the above-mentioned information is used to generate a decoded image of each of the predetermined number of color components in parallel, and includes the predetermined number of image reconstruction units corresponding to each of the predetermined number of color components. ..</p><p num="0020"> According to this, variable-length decoding is performed on the input coded image data, and a decoded image for each color component is generated using the information obtained by the variable-length decoding. That is, since the variable length decoding is performed for a plurality of color components at once, it is not necessary to individually perform the variable length decoding for each color component. Therefore, variable-length decoding is not performed duplicately for each color component, and duplicate processing can be suppressed. Therefore, when the encoded image data is divided into a plurality of color components and decoded, it is possible to suppress a decrease in parallel processing efficiency.</p><p num="0021"> Further, more preferably, using the variable-length decoding data generated by the variable-length decoding unit, it is commonly required between at least two decoding processes of each of the predetermined number of color components. When a common process, which is a process, is performed, a parameter calculation unit that performs the common process and generates common process data that is data obtained as a result of the common process is provided, and the division unit is described from the common process data. Each of the information required for the decoding process of each of the color components to be subjected to the common processing is further acquired, and each of the image reconstructing units corresponding to the color components to which the common processing is performed further obtains each of the acquired information. It is used to generate a decoded image of each color component to which the common processing is performed. Further, more preferably, the parameter calculation unit is the decoding process of the first color component of the predetermined number of color components, and is the first from the first data to be the target of the decoding process of the first color component. (Ii) When the second data is generated and the third data is generated from the second data after the second data is generated from the first data in the decoding process of the second color component, the second data is generated. The data and the third data are generated, and the division unit acquires the second data and the third data as information necessary for decoding the first color component and the second color component, respectively. The image reconstructing unit corresponding to the first color component generates a decoded image of the first color component using the second data, and reconstructs the image corresponding to the second color component. The component unit uses the third data to generate a decoded image of the second color component.</p><p num="0022"> According to this, when a common process is performed between at least two decoding processes, the common process is performed in advance. That is, the common process is performed before the decoding process is performed individually for each color component. Therefore, the common processing is not performed in duplicate for each color component, and the overlapping processing can be suppressed. Therefore, when the encoded image data is divided into a plurality of color components and decoded, it is possible to suppress a decrease in parallel processing efficiency.</p><p num="0023"> Further, preferably, a control unit for determining a method of dividing into the predetermined number of color components according to the color difference format of the coded image data is provided, and the division unit is a division determined by the control unit. According to the method, the plurality of color components are divided into the predetermined number of color components, the information necessary for the decoding process of each of the predetermined number of color components is acquired, and each of the image reconstruction units is described. A decoded image of each of the predetermined number of color components is generated according to the division method determined by the control unit. Further, more preferably, when the control unit has a larger number of pixels of the fourth color component than the number of pixels of the third color component among the color components of the coded image data, the third color The method of dividing into the predetermined number of color components is determined so as to divide the fourth color component more than the components.</p><p num="0024"> According to this, the method of dividing the color components is determined so that the number of pixels in each color component is equalized, and the decoding process is performed. Here, if there is a difference in the processing amount in the decoding processing for each color component, the processing efficiency for the color component having a large processing amount deteriorates. Therefore, by equalizing the processing amount in the decoding process for each color component, it is possible to reduce the factors that deteriorate the processing efficiency and suppress the decrease in the parallel processing efficiency.</p><p num="0025"> Further, preferably, in each of the image reconstruction units, the information required for the decoding process of the input color component is the information required for the decoding process of any of the predetermined number of color components. It is provided with a configuration changing unit that acquires an existence from the control unit and outputs an instruction signal so as to generate a decoded image of the color component. Further, the image combining unit is provided with a result image combining unit that combines the decoded images of the predetermined number of color components generated by each of the image reconstructing units to generate a decoded image of the coded image data. The image combining unit generates a decoded image of the coded image data based on the division method determined by the control unit.</p><p num="0026"> According to this, an instruction signal is output so as to generate a decoded image corresponding to each of the divided color components according to the determined method of dividing the color components. Therefore, no matter what division method the color components are divided into, a decoded image corresponding to each color component can be generated. Further, by combining the decoded images corresponding to each of the divided color components, it is possible to generate a decoded image of the input encoded image data.</p><p num="0027"> Further, a region division unit that divides the region of the coded image data into a plurality of regions, a plurality of variable length decoding units corresponding to the coded image data of each of the divided regions, and the plurality of regions. The plurality of divisions corresponding to each of the variable length decoding units, the plurality of image reconstruction units corresponding to each of the plurality of divisions, and the decoding generated by each of the plurality of image reconstruction units. The images may be combined to provide a result image area combining portion for generating a decoded image of the coded image data.</p><p num="0028"> According to this, the decoding process is performed after the area of the coded image data is divided into a plurality of areas. Therefore, the processing amount of each decoding process can be reduced.</p><p num="0029"> Further, for each of the predetermined number of color components divided by the division unit, for each of the region division unit that divides the region of the color component of the coded image data into a plurality of regions and each of the predetermined number of color components. , The plurality of image reconstruction units corresponding to each of the plurality of divided regions may be provided.</p><p num="0030"> According to this, for each of the divided color components, the decoding process is performed after the region of the coded image data is divided into a plurality of regions. That is, by dividing the color component having a larger number of pixels into a larger area, it is possible to equalize the processing amount of each decoding process. Therefore, by equalizing the processing amount in each decoding process, it is possible to suppress a decrease in parallel processing efficiency.</p><p num="0031"> Further, the present invention can be realized not only as such a decoding device but also as a receiving device including the decoding device. Further, the present invention can be realized as an integrated circuit including each processing unit constituting the decoding device and controlling the decoding device, or can be realized as a method in which the processing of each processing unit is a step. it can. Further, the present invention is realized as a program that causes a computer to execute those steps, is realized as a recording medium such as a computer-readable CD-ROM that records the program, or is used as information, data, or a signal indicating the program. It can also be realized. Then, those programs, information, data and signals may be distributed via a communication network such as the Internet.</p>
<p num="0032"> According to the decoding device of the present invention, when the encoded image data is divided into a plurality of color components and decoded, it is possible to significantly reduce duplicate processing, which requires a huge amount of calculation. The image decoding process can be performed efficiently.</p>
<figref num="1">FIG. 1 is a configuration diagram showing an example of a decoding device that performs decoding processing by dividing conventional luminance information and color difference information.</figref><figref num="2">FIG. 2 is a configuration diagram showing an example of a decoding device according to the first embodiment of the present invention.</figref><figref num="3">FIG. 3 is a configuration diagram showing an example of a stream dividing unit of the decoding device according to the first embodiment of the present invention.</figref><figref num="4">FIG. 4 is a configuration diagram showing an outline of an MPEG2 coded bit stream.</figref><figref num="5">FIG. 5 is a configuration diagram showing an example of a standard MPEG2 decoder.</figref><figref num="6">FIG. 6 is a configuration diagram showing an example of a luminance image reconstruction unit according to the first embodiment of the present invention.</figref><figref num="7">FIG. 7 is a flowchart showing an example of the decoding process performed by the decoding apparatus according to the first embodiment of the present invention.</figref><figref num="8">FIG. 8 is a flowchart showing an example of the luminance color difference partitioning process performed by the partitioning unit of the decoding apparatus according to the first embodiment of the present invention.</figref><figref num="9">FIG. 9 is a configuration diagram showing an example of the decoding device according to the second embodiment of the present invention.</figref><figref num="10">FIG. 10 is a configuration diagram showing an example of a stream dividing unit of the decoding device according to the second embodiment of the present invention.</figref><figref num="11">FIG. 11 is a flowchart showing an example of the decoding process performed by the decoding apparatus according to the second embodiment of the present invention.</figref><figref num="12">FIG. 12 is a flowchart showing an example of the parameter calculation process performed by the parameter calculation unit according to the second embodiment of the present invention.</figref><figref num="13">FIG. 13 is a diagram showing pixel arrangement of the luminance component and the color difference component when the color difference format is 4: 2: 0.</figref><figref num="14">FIG. 14 is a diagram showing pixel arrangement of the luminance component and the color difference component when the color difference format is 4: 2: 2.</figref><figref num="15">FIG. 15 is a diagram showing pixel arrangement of the luminance component and the color difference component when the color difference format is 4: 4: 4.</figref><figref num="16">FIG. 16 is a configuration diagram showing an example of a decoding device according to the third embodiment of the present invention.</figref><figref num="17">FIG. 17 is a configuration diagram showing an example of an image reconstruction unit of the decoding device according to the third embodiment of the present invention.</figref><figref num="18">FIG. 18 is a flowchart showing a process of dividing the input data into two components, a luminance component and a luminance component, in the luminance color difference dividing process of the decoding apparatus according to the third embodiment of the present invention.</figref><figref num="19">FIG. 19 is a flowchart showing a process of dividing the input data into three components, a luminance component, a color difference Cb component, and a color difference Cr component, in the luminance color difference dividing process of the decoding apparatus according to the third embodiment of the present invention.</figref><figref num="20">FIG. 20 is a configuration diagram showing an example of a decoding device according to the fourth embodiment of the present invention.</figref><figref num="21">FIG. 21 is a configuration diagram showing an example of a decoding device according to the fifth embodiment of the present invention.</figref><figref num="22">FIG. 22 is a configuration diagram showing an example of a digital broadcast receiving device.</figref>
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Embodiment 1) FIG. 2 is a configuration diagram showing an example of the decoding device 10 according to the first embodiment of the present invention.
The decoding device 10 is a device that performs MPEG2 decoding processing. As shown in the figure, the decoding device 10 includes a stream dividing unit 201, a luminance image reconstruction unit 202, a color difference image reconstruction unit 203, a luminance memory 204, a luminance memory 205, and a result image combining unit 206. ing. The luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 correspond to the image reconstruction unit described in the claims.
Hereinafter, the operation of the decoding device 10 according to the first embodiment will be described with reference to FIG.
The stream dividing unit 201 divides the input coded image data having a plurality of color components into information necessary for decoding a predetermined number of color components. Specifically, the coded image data has two color components, a luminance component and a color difference component, and a predetermined number of color components are a luminance component and a color difference component. That is, the stream dividing unit 201 divides the input coded bit stream into data necessary for decoding the luminance component and data necessary for decoding the color difference component.
Then, the stream dividing unit 201 outputs the data necessary for decoding the luminance component to the luminance image reconstruction unit 202, and outputs the data necessary for decoding the color difference component to the luminance image reconstruction unit 203. Further, the stream dividing unit 201 outputs data commonly required for both the decoding of the luminance component and the decoding of the color difference component to both the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203.
The luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 generate decoded images of each of a predetermined number of color components in parallel by using the information required for the decoding process of each of a predetermined number of color components. ..
That is, the luminance image reconstruction unit 202 uses the data necessary for decoding the input luminance component and the decoded image of the luminance component stored in the luminance memory 204 to perform inverse quantization, inverse conversion, motion compensation, and the like. Is performed to generate a decoded image of the luminance component. Then, the luminance image reconstruction unit 202 stores the generated decoded image in the luminance memory 204 and outputs it as a result.
Further, the color difference image reconstruction unit 203 uses the input data necessary for decoding the color difference component and the decoded image of the color difference component stored in the color difference memory 205 to perform inverse quantization, inverse conversion, motion compensation, and the like. Is performed to generate a decoded image of the color difference component. Then, the color difference image reconstruction unit 203 stores the generated decoded image in the color difference memory 205 and outputs it as a result.
The luminance memory 204 holds data necessary for the luminance image reconstructing unit 202 to perform the decoding process of the luminance component, such as the decoded image of the luminance component used as the reference image.
The color difference memory 205 holds data necessary for the color difference image reconstructing unit 203, such as a decoded image of the color difference component used as a reference image, to perform decoding processing of the color difference component.
The result image combining unit 206 receives the decoded image of the luminance component output by the luminance image reconstruction unit 202 and the decoded image of the color difference component output by the color difference image reconstruction unit 203 as inputs, and decodes the luminance component and the color difference component. Combined with the image, it is output as a decoding result image.
Next, the operation of the stream dividing unit 201 will be described with reference to the configuration diagram of FIG. FIG. 3 is a configuration diagram showing an example of the stream dividing unit 201 of the decoding device 10 according to the first embodiment of the present invention. The stream dividing unit 201 includes a variable length decoding unit 301 and a dividing unit 302.
The variable-length decoding unit 301 performs variable-length decoding of the input coded image data, and generates variable-length decoded data which is the data obtained as a result of the variable-length decoding. Specifically, the variable-length decoding unit 301 decodes the variable-length code of the input coded bit stream, and outputs the resulting variable-length decoding data to the division unit 302.
The reason why the variable length decoding process is required before the division is performed will be described with reference to the MPEG2 stream configuration schematic diagram of FIG. FIG. 4 is a configuration diagram showing an outline of an MPEG2 coded bit stream.
As shown in the figure, in an MPEG2 coded bitstream, a picture is composed of a start code, a picture header and one or more slices. A slice is composed of a start code, a slice header, and one or more macroblocks. Here, the start code is a fixed-length bit string that indicates a boundary such as a picture or a slice. Further, the macro block is composed of data showing the properties of the macro block, luminance block data, and color difference block data.
These data can be classified into the following three types depending on whether or not they are necessary for decoding the luminance component and the color difference component.
-Data required only for decoding the luminance component -Data required only for decoding color difference components -Data required for both decoding of luminance components and decoding of color difference components For example, the picture header and the slice header are data commonly required for decoding the luminance component and the color difference component.
Further, among the macroblock data, the data indicating the properties of the macroblock includes a macroblock mode and motion information. These data are necessary for both the decoding of the luminance component and the decoding of the color difference component. Further, the luminance block data such as the discrete cosine transform (DCT) coefficient information of the luminance is the data to be the target of the luminance component decoding process, that is, the data necessary for decoding the luminance component. Further, the data of the color difference block such as the DCT coefficient information of the color difference is the data to be the target of the decoding process of the color difference component, that is, the data necessary for decoding the color difference component.
In order to divide the coded bitstream into these three types of data, it is necessary to find the boundaries of each data. Since the picture header and the slice header can be found by detecting the fixed-length start code, they can be divided without performing variable-length decoding.
However, the boundaries of the macroblock, the data inside the macroblock that are commonly required for decoding the luminance component and the decoding of the color difference component, the data that is required for decoding the luminance component, and the boundaries of the data that is required for decoding the color difference component. There is no start code in. In addition, the macroblock data is variable-length coded. Therefore, in a method such as bit parsing, the data required only for decoding the luminance component, the data required only for decoding the luminance component, and the data required for both the decoding of the luminance component and the decoding of the color difference component are separated. This is not possible and it is necessary to decode the variable length code of the encoded bitstream from the beginning.
Returning to FIG. 3, the dividing unit 302 divides a plurality of color components into a predetermined number of color components, and decodes each of the predetermined number of color components included in the variable length decoding data generated by the variable length decoding unit 301. Each of the target data of is acquired as information necessary for decoding processing of each of a predetermined number of color components.
Specifically, the dividing unit 302 divides the color component of the coded image data into two color components, a luminance component and a color difference component. Then, the division unit 302 divides the variable length decoding result input from the variable length decoding unit 301 into data necessary for decoding the luminance component and data necessary for decoding the color difference component, thereby decoding the luminance component. The data to be the target of the above and the data to be the target of the decoding process of the color difference component are acquired.
Then, the division unit 302 outputs the data necessary for decoding the luminance component to the luminance image reconstruction unit 202, and outputs the data necessary for decoding the color difference component to the luminance image reconstruction unit 203. Whether each data is the data required for decoding the luminance component or the data required for decoding the color difference component is determined by the calculation method defined in the standard.
As an example, a picture header, a slice header, a macroblock mode in macroblock data, motion information, and the like are required for both the decoding of the luminance component and the decoding of the color difference component. Further, the DCT coefficient information of the luminance is required for decoding the luminance component, and the DCT coefficient information of the color difference is required for decoding the color difference component. The data required for both the decoding of the luminance component and the decoding of the color difference component are output to both the luminance image reconstructing unit 202 and the color difference image reconstructing unit 203.
Next, the internal configuration of the luminance image reconstruction unit 202 will be described with reference to FIGS. 5 and 6. FIG. 5 is a configuration diagram of a standard MPEG2 decoder, and FIG. 6 is a configuration diagram of the luminance image reconstruction unit 202.
As shown in FIG. 5, a standard MPEG2 decoder 501 includes a variable length decoding unit 502, an inverse quantization unit 503, an inverse conversion unit 504, and a motion compensation unit 505, and outputs a decoding result image to the memory 506. It is used as a reference image for motion compensation of a picture to be decoded later.
The variable-length decoding unit 502 performs a variable-length decoding process on the input coded bit stream, and outputs the result to the inverse quantization unit 503.
The inverse quantization unit 503 performs an inverse quantization process on the data input from the variable length decoding unit 502, and outputs the result to the inverse conversion unit 504.
The inverse transform unit 504 performs inverse discrete cosine transform on the data input from the inverse quantization unit 503, and outputs the result to the motion compensation unit 505.
The motion compensation unit 505 performs motion compensation from the data input from the inverse conversion unit 504 and the reference image stored in the memory 506, and outputs the obtained decoding result image. The inverse quantization unit 503, the inverse conversion unit 504, and the motion compensation unit 505 process the input data of both the luminance component and the color difference component, respectively.
On the other hand, FIG. 6 shows the configuration of the luminance image reconstruction unit 202 in the present embodiment. The luminance image reconstruction unit 202 includes an inverse quantization unit 601, an inverse conversion unit 602, and a motion compensation unit 603, and when the luminance image as a processing result is output to the luminance memory 204 and the subsequent picture is decoded. It is also used as a reference image for motion compensation.
The input signal of the luminance image reconstruction unit 202 is not the MPEG2-encoded bitstream, but the result of variable-length decoding of the MPEG2-encoded bitstream, and only the data necessary for decoding the luminance component is extracted. Therefore, the variable length decoding unit 502, which is necessary for the standard MPEG2 decoder 501, is not required for the luminance image reconstruction unit 202.
Further, the luminance image reconstruction unit 202 includes an inverse quantization unit 601, an inverse conversion unit 602, and a motion compensation unit 603, similarly to the standard MPEG2 decoder 501. The inverse quantization unit 601 performs the same processing as the inverse quantization unit 503, the inverse conversion unit 602 performs the same processing as the inverse conversion unit 504, and the motion compensation unit 603 performs the same processing as the motion compensation unit 505, but the input data is required for decoding the luminance component. Only data is available, and only the processing for the luminance component is performed.
The internal configuration of the color difference image reconstruction unit 203 is similar to the internal configuration of the luminance image reconstruction unit 202, the input is only the data necessary for decoding the color difference component, and the internal processing of the luminance component is performed. However, it is almost the same except that the processing is performed only on the color difference component.
Next, the decoding process performed by the decoding device 10 will be described.
FIG. 7 is a flowchart showing an example of the decoding process performed by the decoding apparatus 10 according to the first embodiment of the present invention.
As shown in the figure, first, a coded bit stream having a plurality of color components is input (step S102).
Then, the variable-length decoding unit 301 decodes the input coded bit stream in a variable-length manner (step S104).
Then, the division unit 302 divides the result of the variable length decoding by the variable length decoding unit 301 into data necessary for decoding the luminance component and data necessary for decoding the color difference component, acquires the data, and obtains the luminance. Output to the image reconstruction unit 202 and the color difference image reconstruction unit 203 (step S106). The details of the luminance color difference dividing process performed by the dividing unit 302 will be described later.
Then, the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 generate the decoded image of the luminance component and the decoded image of the color difference component in parallel (step S108).
Then, the result image combining unit 206 combines the decoded image of the luminance component output by the luminance image reconstruction unit 202 and the decoded image of the color difference component output by the color difference image reconstruction unit 203, and outputs the decoded image as a decoding result image. (Step S110).
Next, the luminance color difference division process (step S106 in FIG. 7) performed by the division unit 302 with respect to the data input from the variable length decoding unit 301 will be described with reference to the flow chart of FIG.
FIG. 8 is a flowchart showing an example of the luminance color difference division processing (step S106 of FIG. 7) performed by the division unit 302 of the decoding device 10 according to the first embodiment of the present invention.
As shown in the figure, first, the division unit 302 determines whether the data input from the variable length decoding unit 301 is necessary for both the decoding of the luminance component and the decoding of the color difference component (step S701).
When the input data is the syntax of the picture header or slice header, or when the macroblock mode or motion information is in the macroblock data, the division unit 302 determines that the data is the decoding process of the luminance component and the color difference component. It is determined that both are necessary (Yes in step S701), and the data is output to the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 (step S702).
When the dividing unit 302 determines that the input data is not required for the decoding process of the luminance component or is not required for the decoding process of the color difference component (No in step S701), the process proceeds to step S703.
Next, the dividing unit 302 determines whether or not the input data is necessary for the decoding process of the luminance component (step S703).
If the input data is the DCT coefficient information of the luminance, the division unit 302 determines that the data is necessary for the decoding process of the luminance component (Yes in step S703), and the data is sent to the luminance image reconstruction unit 202. Is output (step S704).
When the dividing unit 302 determines that the input data is not required for the decoding process of the luminance component (No in step S703), the dividing unit 302 determines that the data is necessary only for the decoding process of the color difference component. The data is output to the color difference image reconstruction unit 203 (step S705).
As a result, the luminance color difference dividing process of the dividing unit 302 (step S106 in FIG. 7) for the input data is completed, and the dividing unit 302 starts the luminance color difference dividing process of the next data.
According to this configuration, in the decoding device 10 that divides the luminance component and the color difference component and performs the decoding process of the MPEG2 coded bit stream in parallel, the same stream is input to the luminance decoder and the color difference decoder. Instead of decoding in parallel while performing duplicate processing in each decoder, the stream division unit 201 is arranged in the previous stage, and only the data necessary for decoding the luminance component is input to the luminance image reconstruction unit 202, and the color difference is obtained. Only the data necessary for decoding the components is input to the color difference image reconstruction unit 203. This makes it possible to reduce overlapping processing between the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203, and makes the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 a standard MPEG2 decoder. It can be constructed more concisely and can reduce computing resources and storage resources.
In the present embodiment, the variable-length decoding unit 301 decodes all the variable-length codes, but the variable-length decoding process is performed on only a part of the coded bit stream that requires the variable-length decoding. May be done. That is, the variable-length decoding unit 301 divides only a part of the coded bit stream that requires variable-length decoding in order for the dividing unit 302 to divide the data required for decoding the brightness component and the data required for decoding the color difference component. On the other hand, the coded bit stream that can be divided by performing the variable length decoding process without performing the variable length decoding process may be output to the brightness image reconstruction unit 202 and the color difference image reconstruction unit 203 as they are.
In this case, the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 require a variable length decoding unit, but generally, the amount of data in the coded bit stream is smaller than that of the data after variable length decoding. Therefore, the stream dividing unit 201 does not decode the entire coded bit stream with variable length, but outputs a part of the data as the coded bit stream as it is, so that the stream dividing unit 201 and the brightness image reconstructing unit 202 In addition, there are advantages such as being able to reduce the capacity of the intermediate buffer for data transfer between the stream dividing unit 201 and the color difference image reconstructing unit 203.
Further, the division unit 302 outputs the data necessary for decoding each of the luminance component and the color difference component to the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 as they are, but after the division, the data is coded again. The image reconstructing unit 202 and the color difference image reconstructing unit 203 may decode the input data and then perform the image decoding process. This also makes it possible to reduce the capacity of the intermediate buffer for data transfer between the stream dividing unit 201 and the luminance image reconstructing unit 202 and the stream dividing unit 201 and the color difference image reconstructing unit 203. The data coding method after division may be the method specified in the image coding standard, or may be any other method.
Further, the luminance memory 204 and the color difference memory 205 may be realized by one device or may be realized by a plurality of devices.
Further, in the present embodiment, the decoding device 10 divides the coded bit stream into two color components, a luminance component and a color difference component, but the dividing method is not limited to this. For example, the decoding device 10 may be divided into three color components of a luminance component, a color difference Cb component, and a color difference Cr component, or may be divided into other components, for example, three color components of R, G, and B. However, a decoder and a memory may be provided for each of the divided color components.
(Embodiment 2) In the decoding device 10 of the first embodiment, the division unit 302 determines whether the input data is data necessary only for decoding the luminance component, data necessary only for decoding the color difference component, or the luminance. Data output control to the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 is performed by determining whether the data is necessary for both the decoding of the component and the decoding of the color difference component.
However, in many image coding standards, as a method of decoding a color difference component, a method of calculating data necessary for decoding a color difference component from data necessary for decoding a luminance component is often defined.
Therefore, the data required for both the decoding of the luminance component and the decoding of the color difference component can be further classified into the following two types.
-Used as common data in decoding the luminance component and decoding the color difference component -The data required for decoding the luminance component is generated from the data contained in the encoded bit stream, and the data required for decoding the color difference component is calculated from the data required for decoding the luminance component. This "used as common data in the decoding of the luminance component and the decoding of the color difference component" does not mean that the data required for decoding the luminance component is calculated from the data required for decoding the luminance component. In addition, it refers to data that can be used as it is for decoding both the luminance component and the color difference component.
Specifically, there are a picture header, a slice header, and the like as examples of data used as common data in the decoding of the luminance component and the decoding of the color difference component. That is, since the picture header or slice header is used in the same manner in the case of decoding the luminance component and the case of decoding the color difference component using the picture header or slice header, the picture header and slice header are common. Data is used in a common sense.
Further, as an example of generating the data required for decoding the brightness component from the data contained in the encoded bit stream and calculating the data required for decoding the color difference component from the data required for decoding the brightness component, MPEG2 There is a motion vector of. The method of calculating the motion vector in MPEG2 will be described below.
As the information about the motion vector, only the difference information of the motion vector is included in the coded bit stream. And this difference information is the data to be the target of the decoding process obtained by the variable length decoding.
On the other hand, using the motion vector of the macroblock already decoded, the predicted value of the motion vector is calculated by the method defined in the standard. The brightness motion vector is calculated from the difference information and the predicted value of the motion vector. Then, the motion vector of the color difference is calculated by the method defined in the standard using the motion vector of the luminance.
This motion vector calculation process is performed in the decoding device 10 according to the first embodiment as follows. First, it is determined that the difference information, which is the result of decoding the coded bit stream by the variable length decoding unit 301, is data commonly required for decoding the luminance component and decoding the color difference component in the dividing unit 302.
As a result, the difference information is output to both the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203.
The luminance image reconstruction unit 202 first calculates a predicted value of the motion vector based on the motion vector of the macroblock that has already been decoded. The luminance image reconstruction unit 202 then calculates the luminance motion vector based on the input difference information and the calculated motion vector predicted value.
Further, the color difference image reconstruction unit 203 first performs the same processing as that performed by the luminance image reconstruction unit 202 to calculate the luminance motion vector. After that, the color difference image reconstruction unit 203 performs a calculation process of the color difference motion vector based on the luminance motion vector.
As described above, the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 duplicate the calculation processing of the luminance motion vector.
When calculating the data required for decoding the color difference component from the data required for decoding the luminance component in addition to the motion vector, the luminance image reconstruction unit 202 and the color difference image reconstruction unit 203 perform the same calculation. Duplicate processing occurs. There is a demand to further improve the efficiency of parallel processing by eliminating this duplication of processing.
Embodiment 2 of the present invention is a decoding device for realizing these requirements.
FIG. 9 is a configuration diagram showing an example of the decoding device 10 according to the second embodiment of the present invention. In the figure, the parts that perform the same operation as the configuration diagram in the first embodiment of FIG. 2 are numbered the same, and the description thereof will be omitted.
As shown in the figure, the decoding device 10 is a device that performs MPEG2 decoding processing, and is a stream dividing unit 801, a luminance image reconstructing unit 202, a color difference image reconstructing unit 802, a luminance memory 204, and a color difference memory. It includes 205 and a result image coupling section 206. The color difference image reconstruction unit 802 corresponds to the "image reconstruction unit" described in the claims.
Hereinafter, the operation of the decoding device 10 according to the second embodiment will be described.
The stream dividing unit 801 performs a variable-length decoding process on the input coded bit stream, and divides the input coded bit stream into data necessary for decoding the luminance component and data necessary for decoding the color difference component. Then, the stream dividing unit 801 outputs the data necessary for decoding the luminance component to the luminance image reconstruction unit 202, and outputs the data necessary for decoding the color difference component to the luminance image reconstruction unit 802.
Further, the stream dividing unit 801 generates data necessary for decoding the luminance component from the data contained in the coded bit stream. Then, for the data required for decoding the color difference component from the data required for decoding the luminance component, the stream dividing unit 801 performs a calculation process of the data required for decoding the color difference component prior to the division.
The color difference image reconstruction unit 802 uses the data necessary for decoding the input color difference component and the decoded image of the color difference component stored in the color difference memory 205 to perform processing such as inverse quantization, inverse conversion, and motion compensation. To generate a decoded image of the color difference component. Then, the color difference image reconstruction unit 802 stores the generated decoded image in the color difference memory 205, and outputs the decoded image of the color difference component to the result image combining unit 206.
The process of calculating the data required for decoding the color difference component from the data required for decoding the luminance component, which is performed by the color difference image reconstruction unit 203 in the first embodiment, is performed by the stream dividing unit 801. Therefore, in the first embodiment, the process of calculating the data required for decoding the color difference component from the data required for decoding the brightness component performed by the color difference image reconstruction unit 203 is performed by the color difference image reconstruction unit 802 in the present embodiment. There is no need to do this, and the color difference image reconstruction unit 802 can be configured more concisely.
Next, the operation of the stream dividing unit 801 will be described in detail with reference to the configuration diagram of FIG.
FIG. 10 is a configuration diagram showing an example of the stream dividing unit 801 of the decoding device 10 according to the second embodiment of the present invention. In the figure, the parts that perform the same operation as the configuration diagram of the stream dividing unit 201 in the first embodiment shown in FIG. 3 are numbered the same, and the description thereof will be omitted.
As shown in the figure, the stream dividing unit 801 includes a variable length decoding unit 301, a parameter calculation unit 901, and a dividing unit 302.
The parameter calculation unit 901 uses the variable-length decoding data generated by the variable-length decoding unit 301 to perform processing that is commonly required between at least two decoding processes of each of a predetermined number of color components. When a certain common process is performed, the common process is performed and the common process data which is the data obtained as a result of the common process is generated. Then, the division unit 302 further acquires each of the information necessary for the decoding process of each color component to which the common process is performed from the common process data. Then, each image reconstruction unit corresponding to the color component to which the common processing is performed further uses the acquired information to generate a decoded image of each color component to which the common processing is performed.
Specifically, the parameter calculation unit 901 is in the decoding process of the first color component of a predetermined number of color components, and the second data from the first data to be the target of the decoding process of the first color component is obtained. The second data and the third data are generated when the second data is generated from the first data and then the third data is generated from the second data in the decoding process of the generated and second color components. To do. Then, the division unit 302 acquires the second data and the third data as information necessary for decoding the first color component and the second color component, respectively. Then, the image reconstruction unit corresponding to the first color component uses the second data to generate a decoded image of the first color component, and the image reconstruction unit corresponding to the second color component is the first. Using the three data, a decoded image of the second color component is generated.
For example, the parameter calculation unit 901 generates data necessary for decoding the brightness component from the data included in the coded bit stream with the data input from the variable length decoding unit 301, and is required for decoding the brightness component. When the data required for decoding the color difference component is generated from the data, the data required for decoding the brightness component and the data required for decoding the color difference component are calculated. That is, the parameter calculation unit 901 calculates the data necessary for decoding the color difference component by using the data necessary for decoding the luminance component. Then, the parameter calculation unit 901 outputs both the data necessary for decoding the calculated color difference component and the data necessary for decoding the luminance component to the division unit 302.
Then, the division unit 302 acquires the data necessary for decoding the luminance component and the data necessary for decoding the color difference component, and outputs each of them to the luminance image reconstruction unit 202 and the color difference image reconstruction unit 802. Then, the luminance image reconstruction unit 202 further uses the data necessary for decoding the luminance component to generate a decoded image of the luminance component, and the color difference image reconstruction unit 802 is the data necessary for decoding the luminance component. Is further used to generate a decoded image of the color difference component.
Next, an example of the decoding process performed by the decoding device 10 in the second embodiment will be described.
FIG. 11 is a flowchart showing an example of the decoding process performed by the decoding apparatus 10 according to the second embodiment of the present invention.
In the figure, a coded bit stream is input (step S202), and the variable length decoding unit 301 decodes the coded bit stream in a variable length (step S204). Since these processes (steps S202 to S204) are the same as the processes (steps S102 to S104) performed by the decoding apparatus 10 in the first embodiment shown in FIG. 7, the description thereof will be omitted.
Then, when the data required for decoding the color difference component is calculated using the data required for decoding the brightness component, the parameter calculation unit 901 needs the data required for decoding the color difference component to decode the brightness component. Calculation is performed using various data (step S205). The details of the parameter calculation process of the parameter calculation unit 901 will be described later.
Then, the division unit 302 divides and acquires the data necessary for decoding, and outputs the data (step S206). Further, the luminance image reconstruction unit 202 and the color difference image reconstruction unit 802 generate a decoded image (step S208), and the result image combining unit 206 combines the decoded images and outputs the decoded image (step S210). ). Since these processes (steps S206 to S210) are the same as the processes (steps S106 to S110) performed by the decoding apparatus 10 in the first embodiment shown in FIG. 7, the description thereof will be omitted.
Next, the parameter calculation process (step S205 in FIG. 11) for the data input from the variable length decoding unit 301 performed by the parameter calculation unit 901 will be described with reference to the flow chart of FIG.
FIG. 12 is a flowchart showing an example of the parameter calculation process (step S205 in FIG. 11) performed by the parameter calculation unit 901 in the second embodiment of the present invention.
As shown in the figure, first, the parameter calculation unit 901 determines whether the data input from the variable length decoding unit 301 is data that is used only for decoding the luminance component or the color difference component ((). Step S1001).
If the input data is the DCT coefficient information of brightness or the DCT coefficient information of color difference, the parameter calculation unit 901 determines that it is used only for decoding the brightness component or the color difference component (Yes in step S1001). , The data is output to the division unit 302 (step S1002), and the parameter calculation process is completed.
When the parameter calculation unit 901 determines that the input data is used for decoding both the luminance component and the color difference component (No in step S1001), the parameter calculation unit 901 proceeds to step S1003.
The parameter calculation unit 901 determines whether or not the input data is used as common data in the decoding of the luminance component and the decoding of the color difference component (step S1003).
For example, when the input data is a picture header, a slice header, or a macro block mode in the macro block data, the parameter calculation unit 901 uses the data as common data for decoding the brightness component and the color difference component. (Yes in step S1003), the data is output to the division unit 302 (step S1002), and the parameter calculation process is terminated.
When data that is not used as common data in decoding the luminance component and decoding the color difference component, such as a motion vector, is input, the parameter calculation unit 901 performs the decoding of the luminance component and the decoding of the color difference component. It is determined that the data is not used as common data (No in step S1003), and the process proceeds to step S1004.
Then, the parameter calculation unit 901 calculates the data required for decoding the color difference component using the data required for decoding the luminance component (step S1004).
The parameter calculation unit 901 is required for the division unit 302 to decode the data required for decoding the luminance component and the color difference component after the calculation process of the data required for decoding the luminance component and the data required for decoding the color difference component is completed. Output the data (step S1002) and end the parameter calculation process.
In this way, out of the data required for both the decoding of the luminance component and the decoding of the color difference component, the data required for decoding the luminance component is generated from the data contained in the encoded bit stream, and the data required for decoding the luminance component is generated for decoding the luminance component. The parameter calculation unit 901 inside the stream division unit 801 performs the calculation process for calculating the data necessary for decoding the luminance component from the necessary data. Then, the parameter calculation unit 901 outputs both the calculated data necessary for decoding the luminance component and the data necessary for decoding the color difference component to the division unit 302. Then, the division unit 302 outputs the data necessary for decoding the luminance component to the luminance image reconstruction unit 202, and outputs the data necessary for decoding the color difference component to the luminance image reconstruction unit 802. As a result, the overlapping processes of the luminance image reconstruction unit 202 and the color difference image reconstruction unit 802 can be collectively performed as preprocessing. Therefore, the processing efficiency can be improved.
The data input to the parameter calculation unit 901 generates the data necessary for decoding the brightness component from the data contained in the encoded bit stream, and the data required for decoding the brightness component is used for decoding the color difference component. When calculating the necessary data, the parameter calculation unit 901 decides to perform a process of calculating the data necessary for decoding the color difference component using the data necessary for decoding the brightness component.
However, in the coding standard of the coded bitstream to be decoded, when the amount of data required for decoding the color difference component, which is calculated using the data required for decoding the brightness component, is enormous, all the calculation processes are parameterized. When the calculation unit 901 performs, a huge storage area is required to hold and transfer all the data necessary for decoding the color difference component in addition to the data required for decoding the brightness component.
Therefore, the data required for decoding the color difference component calculated by the parameter calculation unit 901 is selected, and the parameter calculation unit 901 does not calculate the data required for decoding the color difference component with a large amount of data, and specific data. The division unit 302 outputs the data to the brightness image reconstruction unit 202 and the color difference image reconstruction unit 802, and the color difference image reconstruction unit 802 uses the data necessary for decoding the brightness component to obtain the data required for decoding the color difference component. May be calculated. This has the effect of reducing the capacity of the storage area.
Further, in the parameter calculation unit 901, an example of calculating the data required for decoding the color difference component using the data required for decoding the brightness component was shown, but the data required for decoding the brightness component is calculated according to the coding standard. If a method for calculating from the data required for decoding the color difference component is defined, the parameter calculation unit 901 performs a process of calculating the data required for decoding the brightness component from the data required for decoding the color difference component. May be good.
(Embodiment 3) In the decoding device 10 of the first and second embodiments, the configuration in which the coded bit stream is divided into two components, a luminance component and a color difference component, and decoding processing is performed in parallel has been described. However, in a coding standard such as MPEG2, the color difference is generally composed of the color difference Cb and the color difference Cr.
The relationship between the color difference format and the processing amount will be described using three color difference formats of 4: 2: 0, 4: 2: 2, and 4: 4: 4 as examples.
13 to 15 are diagrams showing pixel arrangements of the luminance component and the color difference component when the color difference formats are 4: 2: 0, 4: 2: 2, and 4: 4: 4, respectively.
When the color difference format is 4: 2: 0, the pixels of the luminance component and the color difference component are arranged as shown in FIG. The number of pixels of the color difference component is half the number of pixels of the luminance component in the horizontal and vertical directions, and the number of pixels of the color difference Cb component and the color difference Cr component is one-fourth of the number of pixels of the luminance component in the entire picture. It becomes. Therefore, the number of pixels of the color difference component is half the number of pixels of the luminance component, and the processing amount is also halved.
When the color difference format is 4: 2: 2, the pixels of the luminance component and the color difference component are arranged as shown in FIG. The number of pixels of the color difference component is half the number of pixels of the luminance component in the horizontal direction, and the number of pixels of the color difference Cb component and the color difference Cr component is half the number of pixels of the luminance component in the entire picture. Therefore, the number of pixels of the color difference component is the same as the number of pixels of the luminance component.
When the color difference format is 4: 4: 4, the pixels of the luminance component and the color difference component are arranged as shown in FIG. In the entire picture, the number of pixels of the color difference Cb component and the color difference Cr component is the same as the number of pixels of the luminance component. Therefore, the number of pixels of the color difference component is twice the number of pixels of the luminance component.
As a result, when the color difference format of the input bitstream is 4: 2: 0 or 4: 2: 2, the luminance component and the color difference component are separated without being divided into the luminance component, the color difference Cb component, and the color difference Cr component. By dividing into two, the processing amount can be sufficiently dispersed.
However, when the color difference format of the input bitstream is 4: 4: 4, the amount of processing of the color difference component is twice that of the luminance component when the input bitstream is divided into the luminance component and the luminance component. In this case, it is better to further divide the color difference component into the color difference Cb component and the color difference Cr component so that the processing amount can be dispersed well.
Therefore, when it is assumed that a coded bit stream having various color difference formats is input to the decoding device 10, depending on the color difference format of the input coded bit stream, there are two components, the brightness component and the color difference component. There is a demand to change the division method so as to divide into three, a brightness component, a color difference Cb component, and a color difference Cr component.
Embodiment 3 of the present invention is a decoding device for meeting these demands.
FIG. 16 is a configuration diagram showing an example of the decoding device 10 according to the third embodiment of the present invention.
The decoding device 10 is a device that performs MPEG2 decoding processing. As shown in the figure, the decoding device 10 includes a stream dividing unit 1401, an image reconstructing unit 1402, 1403, 1404, a memory 1405, 1406, 1407, and a result image combining unit 1408. Hereinafter, the operation of the decoding device 10 according to the third embodiment will be described with reference to the figure.
The stream dividing unit 1401 first performs a variable length decoding process on the input coded bit stream. Then, the stream dividing unit 1401 determines the dividing method of the decoding process according to the color difference format obtained from the variable length decoding result.
If the color difference format of the input encoded bitstream is 4: 2: 0 or 4: 2: 2, the stream divider 1401 is required to decode the data required to decode the brightness component and the color difference component. The data of the variable length decoding result is divided by taking the method of dividing into two with the data.
Then, the stream dividing unit 1401 decodes the brightness image in the subsequent image reconstruction unit 1402, decodes the color difference image in the image reconstruction unit 1403, and decodes the image reconstruction unit 1404. Outputs a signal that controls so that processing is not performed.
Further, the stream dividing unit 1401 outputs a signal for controlling the result image combining unit 1408 so as to combine and output the decoded images of the luminance component and the color difference component.
When the color difference format of the input encoded bit stream is 4: 4: 4, the stream dividing unit 1401 contains the data required for decoding the brightness component, the data required for decoding the color difference Cb component, and the color difference Cr component. The data of the variable length decoding result is divided by the method of dividing into the data necessary for decoding.
Then, the stream dividing unit 1401 causes the image reconstruction unit 1402 in the subsequent stage to decode the brightness image, the image reconstruction unit 1403 to decode the color difference Cb image, and the image reconstruction unit 1404. Outputs a signal that controls to decode the color difference Cr image.
Further, the stream dividing unit 1401 outputs a signal for controlling the result image combining unit 1408 so as to combine and output the decoding result image of the luminance component, the color difference Cb component, and the color difference Cr component.
The image reconstruction units 1402, 1403, and 1404 have the same configuration, and generate decoded images of each of a predetermined number of color components according to the division method determined by the stream division unit 1401. Specifically, the image reconstruction units 1402, 1403, and 1404 decode any of the brightness component, the color difference component, the color difference Cb component, and the color difference Cr component based on the control signal input from the stream dividing unit 1401. The processing is changed so as to process the data necessary for decoding the brightness component, the color difference component, the color difference Cb component, or the color difference Cr component input from the stream dividing unit 1401 to generate a decoded image.
The decoded images generated by each of the image reconstruction units 1402, 1403, and 1404 are output to the result image combination unit 1408. Further, the generated image is stored in the memories 1405, 1406, and 1407, respectively, because it is used as a reference image for motion compensation in the image reconstruction units 1402, 1403, and 1404 when decoding the later picture.
The result image combining unit 1408 combines the decoded images of the predetermined number of color components generated by the image reconstructing units 1402, 1403, and 1404 according to the dividing method determined by the stream dividing unit 1401 to obtain the encoded image data. Generate a decoded image. Specifically, the result image combining unit 1408 synthesizes the decoding results output from the image reconstruction units 1402, 1403, and 1404 based on the control signal input from the stream dividing unit 1401, and obtains the decoding result image. Output.
Next, the operation of the stream dividing unit 1401 will be described in detail. In the drawing, shown in Figure 3 parts having the same operation and configuration diagram of a configuration view and the second embodiment shown in FIG. 10 in the first embodiment are are denoted by the same numerals and description thereof is omitted ..
The stream division unit 1401 includes a variable length decoding unit 301, a parameter calculation unit 901, a control unit 1501, and a division unit 1502.
The control unit 1501 determines a division method for dividing a plurality of color components of the coded image data into a predetermined number of color components according to the color difference format of the input coded image data. That is, when the number of pixels of the fourth color component is larger than the number of pixels of the third color component among the color components of the coded image data, the control unit 1501 is the fourth color component than the third color component. The method of dividing into a predetermined number of color components is determined so as to divide many color components.
Specifically, the control unit 1501 takes the data variable-length decoded by the variable-length decoding unit 301 as input, and determines the division method of the decoding process based on the color difference format included in the input data. Then, the control unit 1501 outputs a signal for controlling the division process performed by the subsequent division unit 1502 according to the determined division method. Further, the control unit 1501 outputs signals to the image reconstruction units 1402, 1403, and 1404 to control which color component is to be decoded.
The division unit 1502 divides a plurality of color components of the input coded image data into a predetermined number of color components according to the division method determined by the control unit 1501, and performs decoding processing for each of the predetermined number of color components. Get each required information. Specifically, the division unit 1502 divides the data input from the parameter calculation unit 901 and acquires the data. Further, the division unit 1502 outputs the data to the image reconstruction units 1402, 1403, and 1404. The method by which the division unit 1502 outputs the data is controlled by the control unit 1501.
Further, the division unit 1502 divides according to the following two modes. The mode of the division unit 1502 is switched by the control signal input from the control unit 1501.
-The input data is divided into two parts, the data required for decoding the brightness component and the data required for decoding the color difference component, and the data required for decoding the brightness component is output to the image reconstruction unit 1402. A mode that outputs the data required for decoding the color difference component to the image reconstruction unit 1403. -The input data is divided into three parts: the data required for decoding the brightness component, the data required for decoding the color difference Cb component, and the data required for decoding the color difference Cr component, and is required for decoding the brightness component. A mode in which data is output to the image reconstruction unit 1402, data required for decoding the color difference Cb component is output to the image reconstruction unit 1403, and data necessary for decoding the color difference Cr component is output to the image reconstruction unit 1404.
Next, the internal configurations of the image reconstruction units 1402, 1403, and 1404 will be described with reference to the configuration diagram of FIG. FIG. 17 is a configuration diagram showing an example of the image reconstruction units 1402, 1403, and 1404 of the decoding device 10 according to the third embodiment of the present invention. Since the image reconstruction units 1402, 1403, and 1404 have the same configuration, only the image reconstruction unit 1402 will be described as a representative.
As shown in the figure, the image reconstruction unit 1402 includes a configuration change unit 1601, an inverse quantization unit 1602, an inverse conversion unit 1603, and a motion compensation unit 1604, and outputs the reconstructed image as a processing result to the memory 1405. Then, it is used as a reference image for motion compensation of a picture to be reconstructed later.
To the image reconstruction unit 1402, one of the brightness component, the color difference component, the color difference Cb component, and the color difference Cr component is sent based on the result of variable length decoding of the MPEG2 coded bit stream instead of the MPEG2 coded bit stream. Only the data necessary for decoding a specific color component is input. Therefore, similarly to the luminance image reconstruction unit 202 in the first embodiment, the function corresponding to the variable length decoding unit 502 existing in the standard MPEG2 decoder 501 becomes unnecessary.
The configuration change unit 1601 acquires from the control unit 1501 which of the predetermined number of color components the information required for the decoding process of the input color component is the information required for the decoding process. , Outputs an instruction signal to generate a decoded image of the color component.
That is, the configuration change unit 1601 inputs the control signal output from the control unit 1501 inside the stream division unit 1401. Then, the configuration change unit 1601 determines whether the input data is the data required for decoding the brightness component, the data required for decoding the color difference component, the data required for decoding the color difference Cb component, or the decoding of the color difference Cr component. It is judged from the input control signal whether the data is necessary for. Then, the configuration change unit 1601 outputs a signal for changing the configuration of the inverse quantization unit 1602, the inverse conversion unit 1603, and the motion compensation unit 1604 to each processing unit so as to process only the input color component. To do.
The inverse quantization unit 1602 is a specific color of any of the brightness component, the color difference component, the color difference Cb component, and the color difference Cr component input from the stream dividing unit 1401 based on the signal from the configuration changing unit 1601. Inverse quantization processing is performed on the data required for decoding the components. Then, the inverse quantization unit 1602 outputs the result to the inverse conversion unit 1603.
The inverse transform unit 1603 identifies one of the brightness component, the color difference component, the color difference Cb component, and the color difference Cr component with respect to the data input from the inverse quantization unit 1602 based on the signal from the configuration change unit 1601. Inverse discrete cosine transform is performed only on the color components of. Then, the inverse conversion unit 1603 outputs the result to the motion compensation unit 1604.
Based on the signal from the configuration change unit 1601, the motion compensation unit 1604 uses the data input from the inverse conversion unit 1603 and the reference image stored in the memory 1405 to obtain a brightness component, a color difference component, a color difference Cb component, and a color difference. Motion compensation is performed for only a specific color component of any of the Cr components, and the obtained decoding result image is output. Further, the motion compensation unit 1604 performs a process of writing back the decoding result image to the memory 1405 in order to use it as a reference image for motion compensation when decoding a later picture.
Next, the decoding process performed by the decoding device 10 according to the third embodiment of the present invention will be described. Here, the decoding process performed by the decoding device 10 according to the third embodiment of the present invention is the same as the decoding process performed by the decoding device 10 according to the second embodiment shown in FIG. However, the details of the luminance color difference division process (step S206 in FIG. 11) in the third embodiment are different from the process in the second embodiment. Therefore, the details of the luminance color difference dividing process (step S206 in FIG. 11) performed by the dividing unit 1502 will be described below with reference to the flow charts of FIGS. 18 and 19.
FIG. 18 is a flowchart showing a process of dividing the input data into two components, a luminance component and a luminance component, in the luminance color difference dividing process of the decoding apparatus 10 according to the third embodiment of the present invention.
FIG. 19 is a flowchart showing a process of dividing the input data into three components, a luminance component, a color difference Cb component, and a color difference Cr component, in the luminance color difference dividing process of the decoding apparatus 10 according to the third embodiment of the present invention.
In the luminance color difference division process performed by the dividing unit 1502, the luminance component and the color difference component are divided and the luminance component, the color difference Cb component, and the color difference Cr component are switched by the control signal input from the control unit 1501. .. Here, the division into the luminance component and the color difference component will be described with reference to FIG. 18, and the division into the luminance component, the color difference Cb component, and the color difference Cr component will be described with reference to FIG.
First, the luminance color difference dividing process for the luminance component and the luminance component will be described with reference to the flow chart of FIG. Regarding the division into the luminance component and the color difference component, the same number is assigned to the same processing as the flow diagram of the luminance color difference division processing in the first embodiment shown in FIG. 8, and the description thereof will be omitted.
When the dividing unit 1502 determines in step S701 that the input data is necessary for both the brightness component decoding process and the color difference component decoding process (Yes in step S701), the image reconstruction process of the brightness component is performed. The data input to both the image reconstruction unit 1402 that performs the image reconstruction unit 1402 and the image reconstruction unit 1403 that performs the image reconstruction processing of the color difference component is output (step S1701).
When the dividing unit 1502 determines in step S701 that the input data is not required for the decoding process of the luminance component or is not required for the decoding process of the color difference component (No in step S701), the input data is input. Determine if the data is needed for the luminance component decoding process (step S703).
Then, when the division unit 1502 determines that the input data is necessary for the decoding process of the luminance component (Yes in step S703), the division unit 1502 outputs the input data to the image reconstruction unit 1402 (step S1702). ..
If the input data is data that is not required for the luminance component decoding process, the dividing unit 1502 determines that the data is required only for the color difference component decoding process (No in step S703). , The input data is output to the image reconstruction unit 1403 (step S1703).
Next, the luminance color difference dividing process for the luminance component, the luminance Cb component, and the luminance Cr component will be described with reference to the flow chart of FIG.
The dividing unit 1502 determines whether or not the data input from the variable length decoding unit 301 is necessary for all of the luminance component decoding process, the color difference Cb component decoding process, and the color difference Cr component decoding process (step S1801).
When the dividing unit 1502 determines that the input data is necessary for all decoding processes of the brightness component, the color difference Cb component, and the color difference Cr component (Yes in step S1801), the image re-decoding process of the brightness component is performed. Data is output to the component unit 1402, the image reconstruction unit 1403 that decodes the color difference Cb component, and the image reconstruction unit 1404 that performs the image reconstruction process of the color difference Cr component (step S1802), and the process ends. To do.
If the dividing unit 1502 determines that the input data is not necessary data for all three color component decodings (No in step S1801), is the input data necessary for decoding the luminance image? Determine if (step S1803).
Then, when the division unit 1502 determines that the input data is necessary for decoding the luminance image (Yes in step S1803), the division unit 1502 outputs the data to the image reconstruction unit 1402 (step S1804), and then goes to step S1805. move on.
If the division unit 1502 determines that the input data is not necessary for decoding the luminance image (No in step S1803), the division unit 1502 proceeds to step S1805 without outputting the data.
Next, the division unit 1502 determines whether or not the input data is necessary for decoding the color difference Cb image (step S1805).
Then, when the division unit 1502 determines that the input data is necessary for decoding the color difference Cb image (Yes in step S1805), the division unit 1502 outputs the data to the image reconstruction unit 1403 (step S1806), and then steps S1807. Proceed to.
If the division unit 1502 determines that the input data is not necessary for decoding the color difference Cb image (No in step S1805), the division unit 1502 proceeds to step S1807 without outputting the data.
Next, the division unit 1502 determines whether or not the input data is necessary for decoding the color difference Cr image (step S1807).
Then, when the division unit 1502 determines that the input data is necessary for decoding the color difference Cr image (Yes in step S1807), the division unit 1502 outputs the data to the image reconstruction unit 1404 (step S1808) and processes the data. To finish.
If the division unit 1502 determines that the input data is not necessary for decoding the color difference Cr image (No in step S1807), the division unit 1502 ends the process as it is.
In this way, when decoding a coded bitstream in the 4: 4: 4 color difference format in which the number of pixels of the color difference component is double that of the brightness component, the color difference component is further divided into the color difference Cb component and the color difference Cr component. By dividing and performing the decoding process in parallel, the processing amounts of the image reconstruction units 1402, 1403, and 1404 can be made substantially the same, and the parallel processing can be performed efficiently.
Further, in a color difference format such as 4: 2: 2 or 4: 2: 0 in which the number of pixels of the color difference component is small, the amount of calculation for decoding the color difference component is smaller than that of the luminance component, so that the color difference Cb component and the color difference Cr By dividing into two components, a luminance component and a color difference component, and decoding instead of dividing them into components, it is possible to reduce the number of arithmetic resources that operate during parallel processing.
Further, the image reconstruction units 1402, 1403 and 1404 can be changed so as to perform decoding processing of the brightness component, the color difference component, the color difference Cb component or the color difference Cr component by the control signal from the control unit 1501 in the stream dividing unit 1401. The configuration enables flexible operation control according to the color difference format of the input stream. Therefore, as compared with a configuration having a decoder specialized only in decoding processing of a specific component, it is possible to effectively utilize computational resources and efficiently perform decoding processing in parallel.
Further, in the present embodiment, the luminance component and the color difference component, or the luminance component, the color difference Cb component, and the color difference Cr component are divided and decoded for any stream, but when the image is small, etc. When the amount of calculation of the coded bit stream to be decoded is small, the luminance component and the color difference component may be decoded by the same image reconstruction unit 1402.
At this time, the parameter calculation unit 901 and the division unit 1502 output the data without changing the data, and the control unit 1501 outputs the control signal so that the division unit 1502 outputs the data without changing the data. In addition, the control unit 1501 also outputs a signal to the image reconstruction unit 1402, which is controlled by the image reconstruction unit 1402 to decode both the luminance component and the color difference component.
As a result, since the effect of parallel processing is small for small images, parallel processing is not performed, and decoding processing is performed by a single image reconstruction unit, so that the computational resources that operate during decoding processing can be reduced, which is efficient. Decoding process is possible.
Further, in the present embodiment, when the luminance component and the color difference component are divided into two and decoded, the image reconstruction unit 1402 performs the luminance component decoding process, and the image reconstruction section 1403 performs the color difference component decoding process. However, since the image reconstruction units 1402, 1403, and 1404 have the same configuration, the allocation is not limited to this. Similarly, in the case of dividing into the luminance component, the color difference Cb component, and the color difference Cr component, the allocation of which color component decoding processing is performed by which image reconstruction unit is not limited to the allocation in the present embodiment. ..
Further, in the present embodiment, the control unit 1501 of the stream dividing unit 1401 outputs a signal to the result image combining unit 1408 so as to combine the decoded images of the luminance component and the color difference component, and the result image combining unit 1408 It was decided to combine the decoded images based on the signal. However, the image reconstruction units 1402, 1403, and 1404 may output the signal, and the resulting image combination unit 1408 may combine the decoded images based on the signal.
(Embodiment 4) In the decoding device 10 of the above-described first to third embodiments, the input coded bit stream is variable-length decoded, divided into a plurality of color components, and the decoding process is performed. However, in the fourth embodiment, the input coded bit stream is region-divided, then variable-length decoding is performed for each region to divide the input coded bit stream into a plurality of color components, and the decoding process is performed.
FIG. 20 is a configuration diagram showing an example of the decoding device 10 according to the fourth embodiment of the present invention.
As shown in the figure, the decoding device 10 according to the fourth embodiment includes a region dividing unit 250, a first region decoding unit 11, a second region decoding unit 12, a third region decoding unit 13, and a result image. It is provided with a region coupling portion 260.
The area division unit 250 divides the area of the coded image data into a plurality of areas. Specifically, the region division unit 250 divides the region of the input coded bit stream into three regions, a first region, a second region, and a third region.
The first region decoding unit 11, the second region decoding unit 12, and the third region decoding unit 13 divide each region divided by the region division unit 250 into a plurality of color components by variable length decoding. Perform decryption processing.
For example, the first region decoding unit 11 performs variable length decoding of the first region, divides the first region into a plurality of color components, and performs decoding processing. Here, since the first region decoding unit 11 has the same configuration as the decoding device 10 in the first embodiment shown in FIG. 2, detailed description thereof will be omitted.
Similarly, the second region decoding unit 12 and the third region decoding unit 13 also perform variable length decoding of the second region and the third region, respectively, and divide the second region and the third region into a plurality of color components to perform decoding processing. Although not shown, the second region decoding unit 12 and the third region decoding unit 13 have the same configuration as the first region decoding unit 11, and therefore detailed description thereof will be omitted.
The result image region combining unit 260 combines the decoded images generated by each of the first region decoding unit 11, the second region decoding unit 12, and the third region decoding unit 13, and the decoded image of the coded image data. To generate.
Specifically, the result image region coupling unit 260 is a decoding result image output by the result image coupling unit 206 of each of the first region decoding unit 11, the second region decoding unit 12, and the third region decoding unit 13. Is combined to generate a decoded image of the input encoded bitstream and output it.
With the above configuration, according to the decoding device 10 in the fourth embodiment, the amount of decoding processing performed by each image reconstruction unit is reduced even when a coded bit stream having a very large capacity is input. can do.
The area divided by the area dividing unit 250 may be two or four or more, instead of three. Then, the processing unit that performs the decoding process is not the first region decoding unit 11, the second region decoding unit 12, and the third region decoding unit 13, but the number of regions divided by the region division unit 250. Depending on the situation, it may be two or four or more.
Further, the result image region coupling unit 260 decodes the first region without going through the result image coupling unit 206 of each of the first region decoding unit 11, the second region decoding unit 12, and the third region decoding unit 13. A decoded image may be generated by combining the decoded images generated by the image reconstruction units of the conversion unit 11, the second region decoding unit 12, and the third region decoding unit 13.
(Embodiment 5) In the decoding device 10 of the fourth embodiment, the input coded bit stream is region-divided, then variable-length decoded in region units, divided into a plurality of color components, and a decoding process is performed. However, in the fifth embodiment, the input coded bit stream is variable-length decoded and divided into a plurality of color components, and then the region is divided and the decoding process is performed in each region.
FIG. 21 is a configuration diagram showing an example of a decoding device according to the fifth embodiment of the present invention.
As shown in the figure, the decoding device 10 according to the fifth embodiment includes a stream dividing unit 201, a first region dividing unit 251, a second region dividing unit 252, a luminance image reconstructing unit 212, 222, and a color difference image reconstructing unit. It includes components 213 and 223, a luminance memory 204, a color difference memory 205, and a result image coupling unit 207.
Here, the stream dividing unit 201, the luminance memory 204, and the color difference memory 205 perform the same processing as the stream dividing unit 201, the luminance memory 204, and the color difference memory 205 in the first embodiment shown in FIG. Therefore, the description thereof will be omitted.
The first region dividing unit 251 and the second region dividing unit 252 divide the region of the color component of the coded image data into a plurality of regions for each of a predetermined number of color components divided by the stream dividing unit 201.
Specifically, the first region division unit 251 divides the region of the luminance component of the input coded bit stream into two regions. Further, the second region dividing unit 252 divides the region of the color difference component of the input coded bit stream into two regions.
The luminance image reconstruction units 212 and 222 and the color difference image reconstruction units 213 and 223 decode each of the predetermined number of color components according to the information required for the decoding process corresponding to each of the plurality of divided regions. Generate an image.
Specifically, the luminance image reconstruction unit 212 and the luminance image reconstruction unit 222 use information necessary for the decoding process corresponding to each of the two regions divided by the first region division unit 251 with respect to the luminance component. Generate a decoded image of each area. Further, the color difference image reconstruction unit 213 and the color difference image reconstruction unit 223 use information necessary for the decoding process corresponding to each of the two regions divided by the second region division unit 252 for the color difference component, respectively. Generate a decoded image of.
Since the luminance image reconstruction unit 212 and the luminance image reconstruction unit 222 have the same configuration as the luminance image reconstruction unit 202 in the first embodiment shown in FIG. 6, detailed description thereof will be omitted. Further, since the color difference image reconstruction unit 213 and the color difference image reconstruction unit 223 have the same configuration as the color difference image reconstruction unit 203 in the first embodiment, detailed description thereof will be omitted.
Further, each of the luminance image reconstruction unit 212, the luminance image reconstruction unit 222, the color difference image reconstruction unit 213, and the color difference image reconstruction unit 223 correspond to the "image reconstruction unit" described in the claims.
The result image combining unit 207 combines the decoded images generated by the luminance image reconstructing units 212 and 222 and the color difference image reconstructing units 213 and 223, and outputs them as the decoded result image of the input coded bit stream.
With the above configuration, according to the decoding device 10 in the fifth embodiment, the color component having a larger number of pixels is divided into a larger number of regions, so that the amount of decoding processing performed by each image reconstruction unit is equalized. Can be transformed into.
For example, when the color difference format is 4: 2: 0, the number of pixels of the luminance component is twice the number of pixels of the color difference component, so that the second region dividing unit 252 does not divide the region of the color difference component, but the first region. The dividing unit 251 divides the area of the luminance component into two. Further, when the color difference format is 4: 4: 4, the number of pixels of the color difference component is twice the number of pixels of the luminance component, so that the first region dividing unit 251 does not divide the region of the luminance component and the second region. The dividing portion 252 divides the area of the color difference component into two. As a result, it is possible to equalize the amount of decoding processing performed by each image reconstruction unit.
The area divided by the first area dividing section 251 and the second area dividing section 252 is not limited to two, and may be three or more. Then, the processing unit that generates the decoded image of the luminance component is not two of the luminance image reconstruction unit 212 and the luminance image reconstruction unit 222, but 3 according to the number of regions divided by the first region division unit 251. It may be one or more. Similarly, the processing unit that generates the decoded image of the color difference component is not the two of the color difference image reconstruction unit 213 and the color difference image reconstruction unit 223, but according to the number of regions divided by the second region division unit 252. There may be three or more.
Further, the result image combining unit 207 includes a luminance component combining unit that combines the decoded images generated by the luminance image reconstruction unit 212 and the luminance image reconstruction unit 222, and a color difference image reconstruction unit 213 and a color difference image reconstruction unit 223. A color difference component coupling portion that combines the generated decoded images may be provided, and the decoding image output by the luminance component coupling portion and the color difference component coupling portion may be combined to output the decoding result image.
In the above-described first to fifth embodiments, the coded bitstream to be decoded is described assuming MPEG2, but the present invention is not limited to MPEG2, and the picture is composed of a plurality of components, for example, MPEG2. Any coding standard may be used as long as it is composed of a luminance component, a color difference Cb component, a color difference Cr component, and the like and is encoded by a variable length code. For example, the encoded bitstream to be decoded may be MPEG4, H.264, VC-1, AVS (Audio Video coding Standard of China).
Further, in the first embodiment, the second embodiment, the fourth embodiment, and the fifth embodiment, the luminance component, the color difference component or the luminance component, the color difference Cb component, and the color difference Cr component are divided into other colors. It may be divided by a component. For example, it may be divided by RGB or HSV.
Further, in the first to fifth embodiments, each functional block constituting the decoding device 10 is typically realized as a program that operates on an information device that requires a CPU or memory, and one of the functions thereof. A part or all of them may be realized as an LSI which is an integrated circuit. These LSIs may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them. Although it is referred to as LSI here, it may be referred to as IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.
Further, the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (FIELD PROGRAMMABLE GATE ARRAY) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of the circuit cells inside the LSI may be used.
Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. There is a possibility of adaptation of biotechnology.
The decoding device 10 according to the present invention can be applied to various devices that decode data encoded by a moving image coding standard such as MPEG2 and H.264. Examples of such devices include digital broadcast receivers, mobile phones, optical disc playback devices such as Blu-ray Discs and DVDs, and personal computers.
FIG. 22 is a configuration diagram showing an example of a case where the decoding device 10 according to the present invention is applied to a receiving device for digital broadcasting.
As shown in the figure, the digital broadcast receiving device 1901 includes a tuner module 1902, a stream decoder 1903, an audio decoder 1904, a ROM 1905, a CPU 1906, a RAM 1907, an output control unit 1908, and a decoding device 1909. In the figure, as the decoding device 1909, the decoding device 10 according to the third embodiment shown in FIG. 16 is shown, but the decoding device 10 according to any one of the first to fifth embodiments. May be good.
The tuner module 1902 outputs digital data (encoded bit stream) of the channel to be viewed from the broadcast wave (RF input).
The stream decoder 1903 separates audio data and video data from the digital data (encoded bit stream) of the channel to be viewed. The stream decoder 1903 corresponds to the "data separator" described in the claims.
The audio decoder 1904 decodes the audio data.
ROM1905 contains programs and data.
The CPU 1906 controls the entire receiving device 1901.
RAM1907 is used as an image memory and a storage area for various data.
The output control unit 1908 synchronizes the decoded video and audio, converts the format, and so on. Further, the output control unit 1908 outputs the decoded image and audio generated by the decoding device 1909 as a video signal and an audio signal.
Here, in this configuration, the stream dividing unit 1401, the image reconstructing unit 1402, 1403, 1404 and the result image combining unit 1408 constituting the decoding device 1909 are individually integrated into one chip, and the memories 1405, 1406, The 1407 has come to be used as an independent device.
However, as for the integration method, each function may be individually integrated into one chip, or may be integrated into one chip so as to include a part or all of them. Furthermore, if there is an integrated circuit technology that can replace the LSI, it is naturally possible to perform integration using that technology.
Further, although the digital broadcast receiving device has been described as an example, the present invention can also be applied to a mobile phone, an optical disc playback device such as a Blu-ray Disc or a DVD, a personal computer, or the like.
Although the decoding device according to the present invention has been described above using the above-described embodiment, the present invention is not limited thereto.
That is, it should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present invention is shown by the claims rather than the above description, and it is intended that all modifications within the meaning and scope equivalent to the claims are included. Further, each component in the above-mentioned plurality of embodiments may be arbitrarily combined as long as the gist of the invention is not deviated.
The decoding device according to the present invention is useful for efficiently decoding moving images in, for example, a digital broadcast receiving device, a mobile phone, an optical disk playback device such as a Blu-ray Disc or a DVD, a personal computer, or the like. ..
10 Decryptor 11 First area decoding unit 12 Second area decoding unit 13 Third area decoding unit 101 Luminance decoder 102 Color difference decoder 103 Brightness memory 104 Color difference memory 105 Coupler 201 Stream split section 202, 212, 222 Luminance image reconstruction section 203, 213, 223 Color difference image reconstruction section 204 Brightness memory 205 Color difference memory 206, 207 Result image joint 250 area division 251 First area division 252 Second area division 260 Result image area joint 301 Variable length decoding unit 302 split 501 standard MPEG2 decoder 502 Variable length decoding unit 503 Inverse quantization unit 504 Inverse converter 505 motion compensation unit 506 memory 601 Inverse quantization unit 602 Inverse converter 603 Motion compensation unit 801 Stream split section 802 Color difference image reconstruction unit 901 Parameter calculation unit 1401 Stream split section 1402, 1403, 1404 Image reconstruction section 1405, 1406, 1407 memory 1408 Result image joint 1501 Control unit 1502 split part 1601 Configuration change part 1602 Inverse quantization unit 1603 Inverse converter 1604 Motion compensation unit 1901 Receiver 1902 Tuner module 1903 stream decoder 1904 Audio decoder 1905 ROM 1906 CPU 1907 RAM 1908 Output control unit 1909 Decryptor
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| JP2003032679A | Cites | Japan | Examiner |
| JP2004056400A | Cites | Japan | Examiner |
| JP2005260639A | Cites | Japan | Search report |
| JP2005260639A | Cites | Japan | Examiner |
| JPH10164584A | Cites | Japan | Search report |
| JPH10164584A | Cites | Japan | Examiner |
| JPH10191392A | Cites | Japan | Search report |
| JPH10191392A | Cites | Japan | Examiner |
| JPH1056641A | Cites | Japan | Examiner |
| JP10056641A | Cites | Japan | – |
| JP2004056400A | Cites | Japan | – |
| JP2003032679A | Cites | Japan | – |
| JP2005260639A | Cites | Japan | – |
| JP10164584A | Cites | Japan | – |
| JP10191392A | Cites | Japan | – |
6 members in 4 offices
Priority claims11
| Document | Office | Kind | Date |
|---|---|---|---|
| 2008151244 | Japan | A | |
| 2008151244 | Japan | A | |
| 2008151244 | Japan | – | |
| 2009002517 | Japan | W | |
| 2009002517 | Japan | W | |
| 2010516740 | Japan | A | |
| 20082008151244 | – | – | – |
| 2009002517 | – | – | – |
| JP20080151244 | – | – | – |
| JP20100516740 | – | – | – |
| WO2009JP02517 | – | – | – |
Members6
| Document | Office | Kind | |
|---|---|---|---|
| WO2009150801A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2010215263A1 | United States of America | A1 | |
| CN102057678A | China | A | |
| JPWO2009150801A1 | Japan | A1 | |
| US8422772B2 | United States of America | B2 | |
| JP5230735B2This record | Japan | B2 |
10 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Cancellation because of no payment of annual feesLAPS | LAPS | |
| Renewal fee payment (event date is renewal date of database)FPAY | FPAY | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 5230735
- Publication, DOCDB
- 5230735
- Publication, EPODOC
- JP5230735B
- Application
- 2010516740
- Application, DOCDB
- 2010516740
- Application, EPODOC
- JP20100516740
Titles2
- Japanese
- 復号化装置、復号化方法及び受信装置
- English
- Decoding device, decoding method and receiving device
Classification
- CPC, 7
- H04N21/42607
- H04N19/61
- H04N19/127
- H04N19/186
- H04N19/44
- H04N19/436
- H04N21/426
- IPC, 16
- H04N7 26
- H04N19 00
- H04N19 102
- H04N19 134
- H04N19 136
- H04N19 139
- H04N19 156
- H04N19 159
- H04N19 423
- H04N19 436
- H04N19 44
- H04N19 46
- H04N19 513
- H04N19 625
- H04N19 70
- H04N19 91