Video Decoding Method Using Adaptive Quantization Matrices

The present invention relates to a moving image coding method for encoding a moving image to generate a stream, a moving image decoding method for decoding a encoded stream, and a stream thereof.

In recent years, we have entered the multimedia era in which audio, images, and other pixel values are handled in an integrated manner, and conventional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones to people, is multimedia. It has come to be taken up as a target. In general, multimedia refers to expressing not only characters but also figures, sounds, especially images, etc. at the same time. However, in order to make the above-mentioned conventional information media a target of multimedia, the information is converted into a digital format. Is an indispensable condition.

However, when estimating the amount of information possessed by each of the above information media as the amount of digital information, the amount of information per character is 1 to 2 bytes in the case of characters, whereas the amount of information per second in the case of voice is 64 Kbits (telephone quality). ) Furthermore, the amount of information required for moving images is 100 Mbits (current TV reception quality) or more per second, and it is not realistic to handle the enormous amount of information as it is in digital format with the above information media. For example, videophones have already been put into practical use by the Integrated Services Digital Network (ISDN), which has a transmission speed of 64Kbit / s to 1.5Mbit / s. That is impossible.

Therefore, information compression technology is needed. For example, in the case of videophones, H.261 and H.263 standard videos recommended by the ITU-T (International Telecommunication Union Telecommunication Standardization Division). Pressure Shrinking technology is used. In addition, according to the MPEG-1 standard information compression technology, it is possible to put image information together with audio information on a normal music CD (compact disc).

Here, MPEG (Moving Picture Experts Group) is an international standard for moving image signal compression standardized by ISO / IEC (International Electrotechnical Commission), and MPEG-1 is an international standard for moving image signals of 1.5 Mbit. It is a standard that compresses TV signal information up to / s, that is, to about 1/100. In addition, since the target quality in the MPEG-1 standard is a medium quality that can be achieved mainly at a transmission speed of about 1.5 Mbit / s, the MPEG-1 standard has been standardized to meet the demand for higher image quality. In 2, the moving image signal is 2 to 15 Mbit / s to achieve TV broadcasting quality. Furthermore, at present, the working group (ISO / IEC) that has been standardizing with MPEG-1 and MPEG-2. JTC1 / SC29 / WG11) achieves a compression ratio higher than MPEG-1 and MPEG-2, and enables encoding, decoding, and manipulation on an object-by-object basis, and MPEG that realizes new functions required in the multimedia era. -4 was standardized. Initially, MPEG-4 was aimed at standardizing low bit rate coding methods, but now it has been extended to more general-purpose coding, including high bit rates including interlaced images. Furthermore, ISO / IEC and ITU-T are currently working together to standardize MPEG-4 AVC and ITU H.264 as next-generation image coding methods with higher compression rates. As of August 2002, a next-generation image coding method called the Committee Draft (CD) has been published.

Generally, in video coding, the amount of information is compressed by reducing redundancy in the temporal and spatial directions. Therefore, in inter-screen prediction coding for the purpose of reducing temporal redundancy, motion is detected and a prediction image is created in block units by referring to the front or rear picture, and the obtained prediction image and coding are performed. Encoding is performed on the difference value from the target picture. Here, a picture is a term representing one screen, and means a frame in a progressive image and a frame or a field in an interlaced image. Here, the interlaced image is an image in which one frame is composed of two fields having different times. In the coding and decoding processing of an interlaced image, one frame can be processed as a frame, it can be processed as two fields, or each block in the frame can be processed as a frame structure or a field structure. it can.

A picture that performs in-screen predictive coding without using a reference image is called an I picture. Further, a picture that refers to only one picture and performs inter-screen prediction coding is called a P picture. Further, a picture capable of performing inter-screen prediction coding by referring to two pictures at the same time is called a B picture. As for the B picture, it is possible to refer to two pictures as any combination of the display time from the front or the back. The reference image (reference picture) can be specified for each block, which is the basic unit of encoding and decoding, but the reference picture described first in the encoded bitstream is the first reference picture. , The one described later is distinguished as the second reference picture. However, as a condition for encoding and decoding these pictures, the referenced picture must already be encoded and decoded.

Motion compensation screen-to-screen predictive coding is used to code the P-picture or B-picture. The motion compensation inter-screen prediction coding is a coding method in which motion compensation is applied to the inter-screen prediction coding. Motion compensation is not simply predicting from the pixel value of the reference picture, but detecting the amount of movement of each part in the picture (hereinafter referred to as the motion vector) and making a prediction considering the amount of movement. This is a method to improve the prediction accuracy and reduce the amount of data. For example, the amount of data is reduced by detecting the motion vector of the picture to be encoded and encoding the predicted residual value between the predicted value obtained by shifting the reference picture by the motion vector and the picture to be coded. In the case of this method, since the motion vector information is required at the time of decoding, the motion vector is also encoded and recorded or transmitted.

The motion vector is detected in macroblock units. Specifically, the macroblock on the side of the picture to be encoded is fixed, and the macroblock on the reference picture side is moved within the search range, which is most similar to the reference block. The motion vector is detected by finding the position of the reference block.

FIG. 1 is an explanatory diagram showing an example of a bitstream data structure. As shown in FIG. 1, the bit stream has the following hierarchical structure. A bitstream is composed of a plurality of Group Of Pictures. By using the group of pictures as the basic unit of coding processing, it is possible to edit moving images and randomly access them. A group of pictures is composed of a plurality of pictures, and each picture has an I picture, a P picture, or a B picture. Each picture is further composed of a plurality of slices. A slice is a strip-shaped area in each picture and is composed of a plurality of macroblocks. Streams, GOPs, pictures, and slices are further composed of a synchronization signal (sync) indicating the delimiter of each unit and a header (header) which is data common to the unit.

Further, when the stream is not a continuous bit stream but is transmitted as a packet or the like which is a unit of fragmented data, the header part and the data part other than the header may be separated and transmitted separately. In that case, the header part and the data part do not become one bit stream as shown in FIG. However, in the case of a packet, even if the transmission order of the header part and the data part is not continuous, the header part corresponding to the corresponding data part is only transmitted in another packet, and it becomes one bit stream. If not, the concept is the same as for the encoded bitstream described in Figure 1.

It is generally said that human visual characteristics are sensitive to low-frequency components in an image, and high-frequency components are not as sensitive as low-frequency components. Further, since the energy of the low frequency component of the image signal is larger than the energy of the high frequency component, the image coding is performed in the order of the low frequency component to the high frequency component. As a result, the number of bits required for coding the high frequency component is larger than the number of bits required for coding the low frequency component.

From the above viewpoint, in the conventional coding method, in the quantization of the conversion coefficient for each frequency obtained by the orthogonal conversion, the quantization step corresponding to the high frequency component is coarser than that of the low frequency component. As a result, a method of significantly improving the compression ratio while slightly deteriorating the subjective image quality has been conventionally adopted.

Since it depends on the image signal how coarse the quantization step of the high frequency component is with respect to the low frequency component, a method of changing the size of the quantization step of each frequency component according to the image is adopted. Has been done. A quantization matrix is used to derive the quantization steps for each frequency component. FIG. 2 is a diagram showing an example of a quantization matrix. In the example of the quantization matrix of FIG. 2, the upper left corresponds to the DC component, the right corresponds to the horizontal high frequency component, and the lower corresponds to the vertical high frequency component. It also shows that the larger the value, the coarser the quantization step. The quantization matrix can usually be changed for each picture, and is described in the header of the picture. Therefore, for example, even if the quantization matrix has the same contents, it is described in the header of each picture and transmitted.

<p> By the way, the current MPEG-4 AVC does not have a quantization matrix like MPEG-2 and MPEG-4. The result is to achieve optimal subjective image quality in current MPEG-4 AVC coding schemes and other schemes that use uniform quantization in all DCT (discrete cosine transform) and DCT-like coefficients. Is getting harder. When introducing such a quantization matrix method into the current MPEG-4 AVC regulations and other standards, it is necessary to make it possible to mount the quantization matrix while emphasizing compatibility with the past.</p><p> Furthermore, as the coding efficiency has improved, it has become possible that MPEG-4 AVC will be used in various application areas. Given its versatility, it will be necessary to use different sets of quantization matrices in different applications, or different sets of quantization matrices in different color channels. The encoder can select different quantization matrices depending on the application and the image to be encoded. Therefore, it is necessary to make it possible to transmit the quantization matrix information more flexibly and effectively by advancing the definition of an efficient quantization matrix and the formulation of the on-board protocol.</p><p> Therefore, the present invention has been made in view of the above circumstances, and provides a moving image coding method and a moving image decoding method capable of reducing the amount of coding and efficiently performing coding and decoding. The purpose is.</p>

<p> In order to achieve the above object, the moving image decoding method according to the present invention is a moving image decoding method for decoding a plurality of coded pictures in block units using a quantization matrix, and is a default quantization method. A holding step of acquiring and holding a second quantization matrix different from the matrix and a matrix ID that identifies the second quantization matrix from the code string, and picture coding data generated by encoding the current picture. Accompanying the above, a step of extracting a matrix ID for specifying the quantization matrix used when encoding the current picture from the code string and a step of using the extracted matrix ID are held in the holding step. A quantization matrix specifying step for specifying a quantization matrix corresponding to the matrix ID from the quantization matrix, and a picture decoding step for decoding the picture-encoded data using the specified quantization matrix. The picture is composed of a brightness component and two color difference components, and in the quantization matrix specifying step, the quantization matrix specified by the matrix ID includes the quantization matrix for the brightness component and the two color differences. If there is a quantization matrix for color difference components corresponding to one of the components, but there is no quantization matrix for color difference components corresponding to the other color difference component of the two color difference components, it exists. By substituting the quantization matrix for the color difference component as the quantization matrix for the non-existing color difference component, the quantization matrix for the color difference component corresponding to the other color difference component of the two color difference components is specified, and the picture is described. In the decoding step, the brightness component of the picture is decoded using the specified quantization matrix for the brightness component, and one of the two color difference components is subjected to decoding. , Decoding is performed using the specified quantization component for color difference component, and for the other color difference component of the two color difference components, the quantization matrix for color difference component specified by substitution is used. Characterized by decryptionTo.</p><p> Further, in order to achieve the above object, the moving image coding method according to the present invention is a moving image coding method for generating a coded stream by encoding a picture constituting a moving image in block units. The picture is converted into a coefficient indicating a spatial frequency component in block units, the converted coefficient is quantized using a quantization matrix, specific information for specifying the quantization matrix used for quantization is generated, and the above-mentioned It is characterized in that specific information is added to the coded stream in predetermined units.</p><p> As a result, it is not necessary to describe the quantization matrix used in a predetermined unit such as a picture, a slice, or a macroblock, the amount of coding can be reduced, and coding can be performed efficiently.</p><p> Further, the quantization matrix may be stored at a position before the position in the coded stream in which the data quantized using the quantization matrix is stored.</p><p> Here, in the storage of the quantization matrix, the information necessary for coding, which is located before the position in the coding stream in which the data quantized using the quantization matrix is stored, is stored. The quantization matrix may be stored in one parameter set or a second parameter set.</p><p> Thereby, the quantization matrix specified by the specific information at the time of decoding can be used.</p><p> Further, the moving image coding method further adds a flag specifying switching between the quantization matrix specified by the specific information and a preset quantization matrix to the coded stream in a predetermined unit. You may.</p><p> Thereby, it is possible to specify the switching between the quantization matrix specified by the specific information and the preset quantization matrix.</p><p> Further, the moving image decoding method according to the present invention is a moving image decoding method for decoding a coded stream encoded by orthogonally converting and quantizing the pictures constituting the moving image in block units. Then, at least one quantization matrix is held, specific information for identifying the quantization matrix used for quantization is extracted from the coded stream in predetermined units, and from at least one held quantization matrix. , The quantization matrix is specified based on the specific information, and the encoded picture is dequantized in block units using the specified quantization matrix, and the dequantized spatial frequency component is shown. It is characterized in that the picture is decoded into the picture by inversely orthogonalizing the coefficients.</p><p> This decodes the encoded stream in which the quantization matrix is pre-transmitted and only the matrix ID that identifies the quantization matrix used in a given unit, such as a picture, slice, or macroblock, is added. Can be done.</p><p> Further, in the moving image decoding method, at least one quantization matrix may be extracted from the coded stream, and the quantization matrix extracted from the coded stream may be held in the holding.</p><p> Here, in the extraction of the quantization matrix, the quantization matrix may be extracted from the first parameter set or the second parameter set in which the information necessary for decoding is stored.</p><p> Thereby, the quantization matrix specified by the specific information can be used.</p><p> Further, the moving image coding method further extracts a flag specifying switching between the quantization matrix specified by the specific information and a preset quantization matrix from the coded stream in a predetermined unit. In specifying the quantization matrix, the quantization matrix specified by the specific information and the preset quantization matrix may be switched.</p><p> As a result, switching between the quantization matrix specified by the specific information and the preset quantization matrix can be performed based on the flag.</p><p> Further, the picture is composed of a brightness component and two color difference components, and when the quantization matrix is specified, if the quantization matrix specified by the specific information does not have the quantization component for the color difference component, the brightness component. It may be specified as a quantization matrix using a quantization matrix.</p><p> Further, the picture is composed of a brightness component and two color difference components, and when the quantization matrix is specified, when the quantization matrix for the color difference component corresponding to the quantization matrix specified by the specific information does not exist. It may be specified as a quantization matrix using the other quantization matrix for color difference components.</p><p> As a result, the coded stream can be decoded even when the color difference quantization matrix is omitted.</p><p> Furthermore, the present invention can be realized not only as such a moving image coding method and a moving image decoding method, but also includes characteristic steps included in such a moving image coding method and a moving image decoding method. It can also be realized as a moving image coding device and a moving image decoding device provided as means. Further, it can be realized as a program for causing a computer to execute those steps, or as a coded stream encoded by the moving image coding method. Needless to say, such programs and coded streams can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.</p>

<p> As is clear from the above description, according to the moving image coding method and the moving image decoding method according to the present invention, the amount of coding can be reduced and coding and decoding can be performed efficiently.</p>

<figref num="1">FIG. 1 is an explanatory diagram showing an example of a bitstream data structure.</figref><figref num="2">FIG. 2 is a diagram showing an example of a quantization matrix.</figref><figref num="3">FIG. 3 is a block diagram showing a configuration of a moving image coding device that realizes the moving image coding method according to the present invention.</figref><figref num="4">FIG. 4 is a diagram showing the correspondence between the sequence parameter set and the picture parameter set and the slice.</figref><figref num="5">FIG. 5 is a diagram showing a part of the configuration of the sequence parameter set.</figref><figref num="6">FIG. 6 is a diagram showing a part of the configuration of the picture parameter set.</figref><figref num="7">FIG. 7 is a diagram showing a description example of the quantization matrix in the parameter set.</figref><figref num="8">FIG. 8 is a flowchart showing an operation when adding a matrix ID.</figref><figref num="9">FIG. 9 is a block diagram showing a configuration of a moving image decoding device that realizes the moving image decoding method according to the present invention.</figref><figref num="10">FIG. 10 is a flowchart showing an operation when specifying the quantization matrix.</figref><figref num="11">FIG. 11 is a flowchart showing an operation when specifying the quantization matrix used for the color difference component.</figref><figref num="12">FIG. 12 is a diagram showing the correspondence between the quantization matrix transmitted as separate data and the quantization matrix used in the sequence.</figref><figref num="13">FIG. 13 is an explanatory diagram of a recording medium for storing a program for realizing the moving image coding method and the moving image decoding method of each embodiment by a computer system, and (a) the recording medium itself. An explanatory diagram showing an example of the physical format of a flexible disk, (b) an explanatory view showing the appearance, cross-sectional structure, and flexible disk of the flexible disk from the front, and (c) recording / playback of the above program on the flexible disk FD. It is explanatory drawing which showed the structure for performing.</figref><figref num="14">FIG. 14 is a block diagram showing an overall configuration of a content supply system that realizes a content distribution service.</figref><figref num="15">FIG. 15 is a diagram showing an example of a mobile phone.</figref><figref num="16">FIG. 16 is a block diagram showing an internal configuration of a mobile phone.</figref><figref num="17">FIG. 17 is a block diagram showing the overall configuration of the digital broadcasting system.</figref>

Embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1) FIG. 3 is a block diagram showing a configuration of a moving image coding device that realizes the moving image coding method according to the present invention.

The moving image coding device 3 is a device that outputs a coded stream Str obtained by compressing and coding the input image signal Vin and converting it into a bit stream such as variable length coding, and is a motion detection unit as shown in FIG. 101, motion compensation unit 102, subtraction unit 103, orthogonal conversion unit 104, quantization unit 105, inverse quantization unit 106, inverse orthogonal conversion unit 107, addition unit 108, picture memory 109, switch 110, variable length coding unit 111 , And a quantization matrix holding unit 112.

The image signal Vin is input to the subtraction unit 103 and the motion detection unit 101. The subtraction unit 103 calculates the difference value between the input image signal Vin and the predicted image, and outputs the difference value to the orthogonal conversion unit 104. The orthogonal conversion unit 104 converts the difference value into a frequency coefficient and outputs the difference value to the quantization unit 105. The quantization unit 105 quantizes the input frequency coefficient using the input quantization matrix WM, and outputs the quantization value Qcoef to the variable length coding unit 111.

The inverse quantization unit 106 inversely quantizes the quantization value Qcoef using the input quantization matrix WM, restores it to a frequency coefficient, and outputs it to the inverse orthogonal conversion unit 107. The inverse orthogonal conversion unit 107 performs inverse frequency conversion from the frequency coefficient to the pixel difference value and outputs it to the addition unit 108. The addition unit 108 adds the pixel difference value and the predicted image output from the motion compensation unit 102 to obtain a decoded image. The switch 110 is turned on when the save of the decoded image is instructed, and the decoded image is saved in the picture memory 109.

On the other hand, the motion detection unit 101 in which the image signal Vin is input in macroblock units searches for the decoded image stored in the picture memory 109, detects the image area closest to the input image signal, and determines the position. Determine the motion vector MV to point to. Motion vector detection is performed in block units that are further divided macroblocks. At this time, since a plurality of pictures can be used as reference pictures, an identification number (reference index Index) for designating the pictures to be referred to is required for each block. The reference index Index makes it possible to specify a reference picture by associating it with the picture number of each picture in the picture memory 109.

The motion compensation unit 102 uses the motion vector MV and the reference index Index detected by the above processing to extract the optimum image area for the predicted image from the decoded image stored in the picture memory 109.

The quantization matrix holding unit 112 holds the quantization matrix WM already transmitted in the parameter set in association with the matrix ID for specifying the quantization matrix WM.

The variable-length coding unit 111 acquires the matrix ID corresponding to the quantization matrix WM used for the quantization from the quantization matrix holding unit 112. Further, the variable-length coding unit 111 performs variable-length coding of the quantization value Qcoef, the matrix ID, the reference index Index, the picture type Ptype, and the motion vector MV to obtain a coded stream Str. At this time, Matori The box ID is added in units of a picture, slice, or macroblock, and specifies the quantization matrix used in the picture, slice, or macroblock.

FIG. 4 is a diagram showing the correspondence between the sequence parameter set and the picture parameter set and the picture. Further, FIG. 5 is a diagram showing a part of the configuration of the sequence parameter set, and FIG. 6 is a diagram showing a part of the configuration of the picture parameter set. A picture is composed of slices, but all slices contained in the same picture have an identifier indicating the same PPS.

In MPEG-4 AVC, there is no concept of header, and common data is placed at the beginning of the stream under the name of parameter set. The parameter set includes a picture parameter set PPS, which is data corresponding to the header of each picture, and a sequence parameter set SPS corresponding to the MPEG-2 GOP or the header of each sequence. The sequence parameter set SPS contains the maximum number of referenceable pictures, image size, etc., and the picture parameter set PPS includes variable length coding types (switching between Huffman coding and arithmetic coding), quantization steps. The initial value of, the number of reference pictures, etc. are included.

An identifier is assigned to the sequence parameter set SPS, and by specifying this identifier in the picture parameter set PPS, it is possible to identify which sequence it belongs to. An identifier is also assigned to the picture parameter set PPS, and by specifying this identifier in the slice, which picture parameter set PPS is used can be identified.

For example, in the example shown in FIG. 4, the identifier (PPS = 1) of the picture parameter set PPS referenced by the slice contained in picture # 1 is included. In addition, the identifier (SPS = 1) of the sequence parameter set SPS to be referenced is included in the picture parameter set PPS # 1.

In addition, the sequence parameter set SPS and the picture parameter set PPS include flags 501 and 601 indicating whether or not the quantization matrix is transmitted, respectively, as shown in FIGS. 5 and 6, respectively. Quantization Matrix 502, 602 is described in.

This quantization matrix includes a 4 × 4 quantization matrix, an 8 × 8 quantization matrix, and the like, depending on the unit for quantization (for example, horizontal 4 × vertical 4 pixels or horizontal 8 × vertical 8 pixels). ..

FIG. 7 is a diagram showing a description example of the quantization matrix in the parameter set.

Since the image signal Vin is composed of a luminance component (luma) and two color difference components (chroma), it is necessary to use different quantization matrices for the luminance component and the two color difference components when performing quantization. Is possible. Further, it is possible to use different quantization matrices in the case of performing in-screen predictive coding (intra) and in the case of performing inter-screen predictive coding (inter).

Therefore, as the quantization matrix, for example, as shown in FIG. 7, a unit for performing quantization, a luminance component and two color difference components, in-screen predictive coding and inter-screen predictive coding, and a quantization matrix corresponding to each are described. be able to.

Next, the operation when the matrix ID is added in the moving image coding apparatus configured as described above will be described. FIG. 8 is a flowchart showing an operation when adding a matrix ID.

The variable-length coding unit 111 acquires the quantization matrix WM used for the quantization (step S101). Next, the variable-length coding unit 111 determines whether or not the acquired quantization matrix WM is held by the quantization matrix holding unit 112 (step S102). Here, when the acquired quantization matrix WM is held in the quantization matrix holding unit 112 (YES in step S102), the variable-length coding unit 111 has a matrix ID corresponding to the acquired quantization matrix WM. Is obtained from the quantization matrix holding unit 112 (step S103). Then, the variable-length coding unit 111 adds the acquired matrix ID in a predetermined unit (for example, a picture, a slice, or a macroblock) (step S104).

On the other hand, when the acquired quantization matrix WM is not held by the quantization matrix holding unit 112 (NO in step S102), the quantization matrix holding unit 112 generates a matrix ID of this quantization matrix WM (NO). Step S105). Then, the quantization matrix holding unit 112 holds the quantization matrix WM in association with the generated matrix ID (step S106). The variable-length coding unit 111 adds the generated matrix ID in a predetermined unit (for example, a picture, a slice, or a macroblock) (step S107). The variable-length coding unit 111 describes the generated matrix ID and the quantization matrix WM in the parameter set (step S108). The parameter set in which the matrix ID and the quantization matrix WM are described is more coded than a predetermined unit to which the matrix ID is added (that is, coded data quantized using the quantization matrix WM). Transmit before in the quantized stream Str.

As described above, the quantization matrix WM is described in the parameter set and transmitted, and only the matrix ID that specifies the quantization matrix WM used is added in a predetermined unit (for example, picture, slice, or macroblock). Therefore, it is not necessary to describe the quantization matrix WM used for each predetermined unit. Therefore, the amount of coding can be reduced and coding can be performed efficiently.

The quantization matrix WM transmitted by the sequence parameter set SPS is updated (Matri). The code ID may be the same) and transmitted using the picture parameter set PPS. In this case, the updated quantization matrix WM is used only when referencing the picture parameter set PPS.

Further, a flag for switching between the default quantization matrix WM and the quantization matrix WM specified by the matrix ID may be added to the coded stream. In this case, the default quantization matrix WM and the quantization matrix WM specified by the matrix ID are switched based on the flag.

FIG. 9 is a block diagram showing a configuration of a moving image decoding device that realizes the moving image decoding method according to the present invention.

The moving image decoding device 2 is a device that decodes the coded stream Str encoded by the moving image coding device 1 as described above, and is a variable length decoding unit 201, a quantization matrix holding unit 202, and a picture. It includes a memory 203, a motion compensation unit 204, an inverse quantization unit 205, an inverse orthogonal conversion unit 206, and an addition unit 207.

The variable length decoding unit 201 decodes the coded stream Str and outputs the quantized value Qcoef, the reference index Index, the picture type Ptype, and the motion vector MV. Further, the variable length decoding unit 201 decodes the coded stream Str, identifies the quantization matrix WM by the extracted matrix ID, and outputs it.

The quantization matrix holding unit 202 holds the quantization matrix WM already transmitted in the parameter set in association with the matrix ID for specifying the quantization matrix WM.

The quantization value Qcoef, the reference index Index, and the motion vector MV are input to the picture memory 203, the motion compensation unit 204, and the inverse quantization unit 205, and the decoding process is performed. The operation is the motion image coding shown in FIG. Same as device 1.

Next, the operation when specifying the quantization matrix in the moving image decoding apparatus configured as described above will be described . FIG. 10 is a flowchart showing an operation when specifying the quantization matrix.

The variable-length decoding unit 201 decodes the coded stream Str and extracts the matrix ID added in a predetermined unit (step S201). Next, the variable-length decoding unit 201 identifies the quantization matrix WM held in the quantization matrix holding unit 202 by the extracted matrix ID (step S202). Then, the variable length decoding unit 201 outputs the specified quantization matrix WM to the inverse quantization unit 205 (step S203).

As described above, the quantization matrix WM is described in the parameter set and transmitted, and in a predetermined unit (for example, a picture, slice, or macroblock), only the matrix ID that identifies the used quantization matrix WM is added. The encoded stream can be decoded.

In the present embodiment, the quantization matrix WM is described in the parameter set and transmitted, but the present invention is not limited to this. For example, it may be transmitted in advance separately from the coded stream.

By the way, as described above, since the image signal Vin is composed of a luminance component (luma) and two color difference components (chroma), the luminance component and the two color difference components are used when quantization is performed. It is possible to use different quantization matrices for each. On the contrary, it is also possible to use the same quantization matrix for the luminance component and the two color difference components without using separate quantization matrices.

Next, the operation when specifying the quantization matrix used for the color difference component will be described. FIG. 11 is a flowchart showing an operation when specifying the quantization matrix used for the color difference component.

The variable length decoding unit 201 determines whether or not there is a corresponding quantization matrix for color difference in the quantization matrix WM specified as described above (step S301). For example, when the quantization target Qcoef to be decoded is the first color difference component, it is determined whether or not there is a first color difference quantization matrix. When the quantization target Qcoef to be decoded is the second color difference component, it is determined whether or not there is a second color difference quantization matrix. Here, if there is a corresponding quantization matrix for color difference (YES in step S301), it is assumed that the corresponding quantization matrix for color difference is used and output to the inverse quantization unit 205 (step S302).

On the other hand, when there is no corresponding quantization matrix for color difference (NO in step S301), it is determined whether or not there is the other quantization matrix for color difference (step S303). For example, when the quantization target Qcoef to be decoded is the first color difference component, it is determined whether or not there is a second color difference quantization matrix. Further, when the quantization value Qcoef to be decoded is the second color difference component, it is determined whether or not there is a first color difference quantization matrix. Here, when there is the other quantization matrix for color difference (YES in step S303), it is assumed that the other quantization matrix for color difference is used and output to the inverse quantization unit 205 (step S304). On the other hand, when there is no other color difference quantization matrix (NO in step S303), the luminance quantization matrix is used and output to the inverse quantization unit 205 (step S305).

As a result, the coded stream can be decoded even when the color difference quantization matrix is omitted.

(Embodiment 2) First, the main points of the present embodiment will be described.

First, if there are multiple sequence-level stream description data structures that can be selected by a portion of the videobitstream, the quantization matrix is described in each data structure separate from any of the sequence header data structures. It shall be.

Second, we define multiple user-customized quantization matrices at the beginning of the sequence video stream. The quantization matrix can be selected for different pictures in different places in the bitstream. MPEG-2 uses the quantization matrix method, but does not use a matrix set that allows one of multiple matrices to be selected. In addition, when the quantization matrix was updated, the quantization matrix had to be newly described.

Therefore, as the third point, the frequency at which the update is performed as a syntax element applied to the quantization update is defined so that the quantization matrix update method suitable for the above case is obtained. In the scheme of this embodiment, a single effective quantization matrix in MPEG-2 and subsequent updates are just one exception.

Next, the outline of the present embodiment will be described.

Some video coding standards may have several segments in the sequence that are encoded using different coding configurations, in which case each segment in the sequence will have a different sequence header or It is necessary to describe the segment header. Since a considerable number of bits are required to transmit the quantization matrix, in this embodiment, all the quantization matrices used in the sequence are placed in a place different from the sequence header or the segment header. For segments that use multiple different sets of quantization matrices in a sequence, you only need to refer to the quantization matrix using something like an identification number, and the matrix is similar to the mechanism used in MPEG-2. It is not necessary to transmit the matrix from the encoder to the decoder every time you use.

All quantization matrices not specified in the video codec standard must be defined and put together. The segments or blocks in the bitstream that describe such a quantization matrix must be placed at the beginning of the sequence bitstream before transmitting the encoded video data. As an option according to each video codec standard, these quantization matrices can be included as part of the video basic stream, or described outside the band, such as a transport stream, packet or file, separate from the body of the video stream. You can also do it.

In many codec standards such as MPEG-2 and MPEG-4, a single sequence segment contains lower level data structures. That is, the sequence segment is a collection of video data into a GOP (group of pictures), pictures, slices, layers, macroblocks, and the like. When a sequence segment header or descriptor refers to one or more sets of quantization matrices, the choice of which set to use depends on the lower level data structure specifications. This point will be described later in this embodiment.

For sequence segments that reference one or more quantized matrix sets, all quantized matrices are transmitted at the beginning of the sequence. The decoder that receives all the quantization matrix at the beginning of the sequence shall hold the quantization matrix in the internal memory in the following way. That is, when the decoder refers to a specific quantization matrix, all of those tables can be used if there is a reference table associated with the quantization matrix. In implementing the syntax provisions, the capabilities of the decoder must be considered and ensured that it fits into the requirements of the application in which the decoder is installed. Therefore, the number of quantization matrices available in a given time must not exceed a certain range.

If a new quantization matrix set is needed but the decoder capabilities do not allow it to store more than one quantization matrix, before the new quantization matrix set is stored and enabled. , The old set must be deleted from the decoder's memory. This is a scenario similar to that used in the MPEG-2 standard.

FIG. 12 is a diagram showing an example of the correspondence between the quantization matrix transmitted as separate data and the quantization matrix used in the sequence.

In the example shown in FIG. 12, it is described that the quantization matrix Q-matrix1 and Q-matrix3 are used in the sequence SEQ1. Further, it is described that the quantization matrix Q-matrix2, Q-matrix4 and Q-matrix5 are used in the sequence SEQ2, and the quantization matrix Q-matrix4 is used in the sequence SEQ3.

Next, the features of the syntax that supports the use of the quantization matrix will be described.

The quantization matrix can be used fixedly for a picture sequence or the entire program.

However, in a more flexible way to achieve higher quality, the quantization scheme and quantization matrix can be changed dynamically. In this case, the question is at what data level and how can the quantization method and matrix be changed. The complexity possible in the application domain limits the number of sets of quantization matrices possible at each data level.

Macroblocks from all stream data structure levels: sequences, segments, pictures, slices (macroblocks are used in almost all codec standards and refer to blocks of 16x16 pixels, but can be claimed for ownership. Table 1 below shows the types of quantization that can be changed from one data to another at the level immediately below in that bitstream, up to (the codec and future codecs may be resized). It has a 6-bit flag, which is a bit of. For example, in MPEG-4 AVC, the level immediately below "Sequence" is "Picture" and the level immediately below "Picture" is "Slice".

<tables num="1"><img file="JP5048826B2_D0001.tif" /></tables>

However, if only bit A is set and bit B is not set, bit C cannot be set. Similarly, if only bit D is set and bit E is not set, then bit F cannot be set.

If both Bit B and Bit C are set, it means that the quantization matrix set can be changed to another set. One quantization matrix set contains one matrix for each block coding mode. The block coding mode includes in-picture prediction in a specific direction, inter-picture prediction block, two-way inter-picture prediction block, and the like.

Bits C and F indicate changes in the quantization scheme, the quantization matrix, or both. At the sequence level of MPEG-4 AVC, if 8x8 non-uniform quantization using a quantization matrix is set, the quantization matrix used can be varied for each "picture" data.

At the highest level of data syntax, such as sequence headers, the default quantization set is specified when the quantization matrix scheme is used.

If bit C or bit F is set to a certain data level, each lower level data header will be flagged to indicate whether to use the default quantization matrix set at that level.

If the flag is on in the low-level data header, is it possible to define a new default quantization set for that data level and use the 6-bit flag at that data level to change the default at a lower data level? Please show me. The same is done for all data levels, up to the lowest level or the lowest level allowed by application requirements.

If bit C or bit F is not set, this flag is not used in lower level data headers and the default is automatically used.

There may be restrictions applied to the recursive signal transmission method for transmitting information about such a quantization method. For example, there is a restriction on the frequency of change of the quantization matrix up to a certain rate.

Next, the default and customizable quantization matrix will be described.

In a video coding standard using the non-uniform quantization matrix method, there may be some pre-defined matrices in the video codec standard. Such default or specified matrices are known to standards-compliant decoders and there is no need to transfer such matrices to the decoder. Similarly, such a quantization matrix can be referenced in the same way as described above. If a defined matrix is available, the decoder shall add the received customized matrix to the quantization matrix stored by the decoder itself. As described above, each quantization matrix is identified by an identification number assigned by the encoder and transmitted to the decoder.

In classifying multiple quantization matrices in bitstream syntax, quantizations of the same size can be grouped together. Information on whether a matrix is used in the inter-picture coding block, in-picture coding, luminance component, or color difference component can also be confirmed by each attribute.

Next, the update of the quantization matrix will be described.

The video codec bitstream syntax allows you to add or update known quantization matrices to your decoder.

If a quantization matrix is associated with a new identification number, this matrix is considered a new quantization matrix and can be referred to by the new identification number. If the identification number is already associated with another quantization matrix, the existing quantization matrix is modified to the new matrix in the decoder. Only a quantization matrix of the same size as the old matrix can be replaced with the old matrix. The state of the quantization matrix in use is grasped on the encoder side. During transmission of the updated quantization matrix, only the quantization matrix that needs to be updated is defined in the network packet.

Next, the description of the quantization matrix in MPEG-4 AVC will be described.

In MPEG-4 AVC, all video data and headers are organized in a bitstream layer called NAL (Network Abstract Layer). NAL is a sequence of many NAL units. Each NAL unit contains a particular type of video data or data header.

MPEG-4 AVC also defines several picture data groups under one data hierarchy. This hierarchy begins with the sequence described by the sequence parameter set. A "sequence" can include multiple pictures with different picture parameter sets. Below the "picture" is a slice, which has a slice header. Slices usually consist of a large number of 16x16 pixel blocks called macroblocks.

When introducing the quantization matrix method to MPEG-4 AVC, the user-defined quantization matrix or the matrix specified by the encoder can be applied to the NAL unit as well. The use of NAL units can be carried out in the following three ways.

(1) Describe all matrix information (including quantization table) associated with each matrix in one NAL unit. (2) Each of the multiple NAL units describes a specific type of quantization matrix and their information. (3) Describe the definition of one quantization matrix for each NAL unit.

In the case of (1) and (2) above, the NAL unit also specifies the total number of quantization matrices. In the case of (3), the total number of user-defined quantization matrices is not clearly shown in the video basic stream. Both the encoder and the decoder must count the total number as the process progresses. As an example of (2), there is a case where a 4 × 4 quantization matrix and an 8 × 8 quantization matrix are classified and each is described in NAL.

In the sequence parameter set, it shall be specified which quantization matrix is used in MPEG-4. A 6-bit flag is defined to indicate which quantization method is used and whether it can be changed at the next level, that is, at the picture level where the header is the picture parameter set.

A sequence parameter set that references a subset of the defined quantization matrix lists all quantization matrix IDs. The ID includes the default ID for the video codec standard and the ID defined by the codec operator specifically for some content. The sequence parameter set can describe some common quantization parameters. The sequence parameter set provides default quantization for each of the 4x4 and 8x8 block inter-picture and intra-picture predictions for the luminance component, and similarly for the inter-picture and intra-picture predictions for the color difference component. You can declare a set of matrices. However, picture parameter sets, slice headers, and macroblock levels can override higher level provisions by declaring their own set of quantization matrices. However, these quantization matrices must also be available in the currently available sequence parameter sets.

If the quantization matrix is applied in NAL units, the quantization matrix can be transmitted at the beginning of the bitstream of the sequence. The position may be after or before the NAL unit that describes the sequence parameter set. After the initial definition, additional customized quantization matrices can be inserted into the bitstream for updates or new matrices. The operation of adding or updating is determined by the quantization matrix ID. Update if you have an ID. If there is no ID, the new matrix will be added to the saved matrix.

(Embodiment 3) Further, by recording the program for realizing the moving image coding method and the moving image decoding method shown in each of the above embodiments on a recording medium such as a flexible disk, the moving image coding method and the moving image decoding method are recorded in each of the above embodiments. The shown processing can be easily carried out in an independent computer system.

FIG. 13 is an explanatory diagram when the moving image coding method and the moving image decoding method of each of the above embodiments are carried out by a computer system using a program recorded on a recording medium such as a flexible disk.

FIG. 13 (b) shows the appearance, cross-sectional structure, and flexible disc of the flexible disc when viewed from the front, and FIG. 13 (a) shows an example of the physical format of the flexible disc, which is the main body of the recording medium. The flexible disk FD is built in the case F, and a plurality of track Trs are concentrically formed on the surface of the disk from the outer circumference toward the inner circumference, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk in which the program is stored, the program is recorded in the area allocated on the flexible disk FD.

Further, FIG. 13 (c) shows a configuration for recording / reproducing the above program on the flexible disk FD. When recording the above program that realizes the moving image coding method and the moving image decoding method on the flexible disk FD, the above program is written from the computer system Cs via the flexible disk drive. Further, when the moving image coding method and the moving image decoding method are constructed in the computer system by the program that realizes the moving image coding method and the moving image decoding method in the flexible disk, the program is executed by the flexible disk drive. Read from the flexible disk and transfer to the computer system.

In the above description, a flexible disk is used as the recording medium, but an optical disk can also be used in the same manner. The recording medium is not limited to this, and any recording medium such as an IC card or ROM cassette that can record a program can be used in the same manner.

(Embodiment 4) Further, here, an application example of the moving image coding method and the moving image decoding method shown in the above embodiment and a system using the same will be described.

FIG. 14 is a block diagram showing the overall configuration of the content supply system ex100 that realizes the content distribution service. The communication service provision area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.

This content supply system ex100 is, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera via the Internet service provider ex102 and the telephone network ex104, and the base stations ex107 to ex110 on the Internet ex101. Each device such as mobile phone ex115 with is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 14, and any combination may be used for connection. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110, which are fixed radio stations.

The camera ex113 is a device capable of shooting moving images such as a digital video camera. In addition, the mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone. Alternatively, it may be PHS (Personal Handyphone System) or the like.

Further, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution based on the coded data transmitted by the user using the camera ex113 becomes possible. The captured data may be encoded by the camera ex113 or by a server or the like that performs data transmission processing. Further, the moving image data taken by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images such as a digital camera. In this case, the moving image data may be encoded by either the camera ex116 or the computer ex111. Further, the coding process is performed by the LSI ex117 of the computer ex111 and the camera ex116. Note that the image coding / decoding software may be incorporated into some storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium that can be read by a computer ex111 or the like. Further, the moving image data may be transmitted by the mobile phone ex115 equipped with a camera. The moving image data at this time is the data encoded by the LSI of the mobile phone ex115.

In this content supply system ex100, the content photographed by the user with the camera ex113, the camera ex116, etc. (for example, a video of a live music) is encoded and transmitted to the streaming server ex103 in the same manner as in the above embodiment. On the other hand, the streaming server ex103 streams the above content data to the requested client. Clients include a computer ex111, a PDAex112, a camera ex113, a mobile phone ex114, and the like, which can decode the encoded data. By doing so, the content supply system ex100 can receive the encoded data at the client and play it back, and further realize personal broadcasting by receiving it in real time at the client, decoding it, and playing it back. It is a system that makes it possible.

For the coding and decoding of each device constituting this system, the moving image coding device or the moving image decoding device shown in each of the above embodiments may be used.

A mobile phone will be described as an example.

FIG. 15 is a diagram showing a mobile phone ex115 using the moving image coding method and the moving image decoding method described in the above embodiment. The mobile phone ex115 is an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, images of a CCD camera, etc., a camera unit ex203 capable of taking still images, an image taken by the camera unit ex203, and an antenna ex201. Display unit ex202 such as liquid crystal display that displays the decoded data of received video, etc., main unit consisting of operation key ex204 group, audio output unit ex208 such as speaker for audio output, audio input To save encoded or decoded data such as audio input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. It has a slot portion ex206 for mounting the recording media ex207 on the recording media ex207 and the mobile phone ex115. The recording medium ex207 is EEPROM (Electrically Erasable), which is a non-volatile memory that can be electrically rewritten or erased in a plastic case such as an SD card. It stores a flash memory element, which is a type of programmable read only memory).

Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 has a power supply circuit unit ex310, an operation input control unit ex304, and image coding for the main control unit ex311 which is designed to collectively control each part of the main body unit provided with the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, multiplex separation unit ex308, recording / playback unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via the synchronization bus ex313. Connected to each other.

The power circuit unit ex310 activates the camera-equipped digital mobile phone ex115 in an operable state by supplying power to each unit from the battery pack when the call ends and the power key is turned on by the user's operation. ..

Based on the control of the main control unit ex311 consisting of CPU, ROM, RAM, etc., the mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305. This is subjected to spectrum diffusion processing by the modulation / demodulation circuit unit ex306, digital-to-analog conversion processing and frequency conversion processing by the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201. In addition, the mobile phone ex115 amplifies the received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, spectrum despread processing by the modulation / demodulation circuit unit ex306, and analog voice by the voice processing unit ex305. After converting to data, this is output via the audio output unit ex208.

Further, when the e-mail is transmitted in the data communication mode, the text data of the e-mail input by the operation of the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing of text data by the modulation / demodulation circuit unit ex306, digital-to-analog conversion processing and frequency conversion processing by the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image coding unit ex312 via the camera interface unit ex303. When the image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

The image coding unit ex312 has a configuration including the moving image coding device described in the present invention, and the image data supplied from the camera unit ex203 is a code used in the moving image coding device shown in the above embodiment. It is converted into coded image data by compression coding according to the coding method, and this is sent to the multiple separation unit ex308. At the same time, the mobile phone ex115 transmits the sound collected by the voice input unit ex205 during imaging by the camera unit ex203 to the multiplex separation unit ex308 as digital voice data via the voice processing unit ex305.

The multiplex separation unit ex308 multiplexes the coded image data supplied from the image coding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit. Spread spectrum processing is performed by ex306, digital-to-analog conversion processing and frequency conversion processing are performed by the transmission / reception circuit unit ex301, and then transmission is performed via the antenna ex201.

When receiving the data of the moving image file linked to the homepage etc. in the data communication mode, the received data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is performed. The data is sent to the multiplex separator ex308.

Further, in order to decode the multiplexed data received via the antenna ex201, the multiplexing separator ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and synchronizes the data. The encoded image data is supplied to the image decoding unit ex309 and the audio data is supplied to the audio processing unit ex305 via the bus ex313.

Next, the image decoding unit ex309 has a configuration including the moving image decoding device described in the present invention, and is a decoding method corresponding to the coding method shown in the above embodiment for a bit stream of image data. The reproduced moving image data is generated by decoding and supplied to the display unit ex202 via the LCD control unit ex302, whereby the moving image data included in the moving image file linked to the homepage, for example, is displayed. At the same time, the audio processing unit ex305 converts the audio data into analog audio data and then supplies the audio data to the audio output unit ex208, whereby, for example, the audio data contained in the moving image file linked to the homepage is reproduced. To.

Not limited to the above system, digital broadcasting by satellite and terrestrial broadcasting has recently become a hot topic, and as shown in FIG. 17, the digital broadcasting system also includes at least a moving image encoding device of the above embodiment. Any of the moving image decoding devices can be incorporated. Specifically, in the broadcasting station ex409, a bit stream of video information is transmitted via radio waves to a communication or a broadcasting satellite ex410. In response to this, the broadcasting satellite ex410 transmits radio waves for broadcasting, receives the radio waves with a home antenna ex406 equipped with satellite broadcasting receiving equipment, and receives the radio waves such as TV (receiver) ex401 or set-top box (STB) ex407. The device decodes the bit stream and plays it back. Further, the moving image decoding device shown in the above embodiment can also be mounted on the playback device ex403 that reads and decodes the bit stream recorded on the storage medium ex402 such as a recording medium such as a CD or DVD. .. In this case, the reproduced video signal is displayed on the monitor ex404. It is also conceivable to mount a moving image decoding device in a set-top box ex407 connected to a cable ex405 for cable TV or an antenna ex406 for satellite / terrestrial broadcasting, and reproduce this on a TV monitor ex408. At this time, the moving image decoding device may be incorporated in the television instead of the set-top box. It is also possible for the car ex412 having the antenna ex411 to receive a signal from the satellite ex410 or the base station ex107 or the like and reproduce the moving image on the display device such as the car navigation ex413 which the car ex412 has.

Further, the image signal can be encoded by the moving image coding device shown in the above embodiment and recorded on a recording medium. Specific examples include a recorder ex420 such as a DVD recorder that records an image signal on a DVD disc ex421 and a disc recorder that records an image signal on a hard disk. It can also be recorded on the SD card ex422. If the recorder ex420 is provided with the moving image decoding device shown in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

The car navigation system ex413 may be configured by excluding the camera unit ex203, the camera interface unit ex303, and the image coding unit ex312 from the configurations shown in FIG. 16, and the same applies to the computer ex111 and the television (receiver). ) Ex401 etc. can also be considered.

In addition, terminals such as the mobile phone ex114 are implemented in three types: a transmitter / receiver terminal having both an encoder and a decoder, a transmitter terminal having only an encoder, and a receiving terminal having only a decoder. Can be considered.

As described above, it is possible to use the moving image coding method or the moving image decoding method shown in the above-described embodiment for any of the above-mentioned devices / systems, and by doing so, the above-described embodiment will be described. The effect can be obtained.

Further, the present invention is not limited to the above-described embodiment, and various modifications or modifications can be made without departing from the scope of the present invention.

Further, each functional block in the block diagram shown in FIGS. 3 and 9 is typically realized as an LSI which is an integrated circuit. This LSI may be integrated into a single chip or a plurality of chips. (For example, functional blocks other than memory may be integrated into a single chip.) Here, LSI is used, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Further, the method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of circuit cells inside the LSI may be used.

Furthermore, if an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology or another technology derived from it, it is naturally possible to integrate functional blocks using that technology. There is a possibility of adaptation of biotechnology.

Further, among the functional blocks, only the means for storing the data to be encoded or decoded may be configured separately without being made into one chip.

As described above, in the moving image coding method and the moving image decoding method according to the present invention, for example, a mobile phone, a DVD device, a personal computer, or the like encodes each picture constituting the moving image to form a coded stream. It is useful as a method for generating and decoding the generated coded stream.