Moving picture encoding method and moving picture decoding method

The present invention relates to a plurality of coded pictures or decoded pictures to encode or decode a moving image signal, a moving image coding method, a moving image coding device, a moving image decoding method, and a moving image decoding device. , And a recording medium containing a program for performing it in software.

In recent years, we have entered the multimedia era in which information such as voice, images, and other pixel values are handled in an integrated manner, and the conventional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones to people. Has come to be taken up as a subject of multimedia. In general, multimedia refers to expressing not only characters but also figures, sounds, especially images, etc. at the same time, but in order to target the above-mentioned conventional information media as multimedia, the information is converted into a digital format. Representation is a prerequisite.

However, when the amount of information possessed by each of the above information media is estimated as the amount of digital information, the amount of information per character is 1 to 2 bytes in the case of characters, whereas the amount of information per second in the case of voice is 64 kbits (telephone quality). ) Furthermore, the amount of information required for moving images is 100 Mbits (current TV reception quality) or more per second, and it is not realistic to handle the enormous amount of information as it is in digital format with the above information media. For example, videophones have already been put into practical use by the Integrated Services Digital Network (ISDN), which has a transmission speed of 64 kbps to 1.5 Mbps, but the information of the video taken by the video camera is sent as it is by ISDN. That is impossible.

Therefore, information compression technology is needed. For example, in the case of videophones, H.261 and H.263 are internationally standardized by the ITU-T (International Telecommunication Union Telecommunication Standardization Division). Standard video compression technology is used. In addition, according to the MPEG-1 standard information compression technology, it is possible to put image information together with audio information on a normal music CD (compact disc).

Here, MPEG (Moving Picture Experts Group) is an international standard for digital compression of moving image signals, and MPEG-1 reduces moving image signals up to 1.5 Mbps, that is, TV signal information to about 1/100. It is a standard that compresses to. In addition, since the transmission speed for the MPEG-1 standard is mainly limited to about 1.5 Mbps, the MPEG-2 standardized to meet the demand for higher image quality has 2 to 2 moving image signals. Compressed to 15Mbps.

Furthermore, at present, the working group (ISO / IEC JTC1 / SC29 / WG11), which has been standardizing MPEG-1 and MPEG-2, has standardized MPEG-4, which has a higher compression ratio. MPEG-4 not only enables highly efficient coding at a low bit rate, but also introduces a powerful error tolerance technology that can reduce subjective image quality deterioration even if a transmission line error occurs. In addition, ITU-T has started standardization activities for H.26L as a next-generation image coding method.

In MPEG-1, MPEG-2, and MPEG-4, the image signal (reference image) of the picture encoded or decoded immediately before is referred to, and the difference value between the reference image and the encoded / decoded image is encoded. Use inter-screen coding (Inter Predictive Coded Picture: hereinafter abbreviated as Inter Picture) to decrypt. As a result, a large compression ratio was made possible (see, for example, Non-Patent Document 1).

Further, the amount of information can be reduced by reducing the redundancy in the temporal direction and the spatial direction. Therefore, in inter-screen predictive coding for the purpose of reducing temporal redundancy, a predictive image is created with reference to a picture (reference image) that has already been encoded / decoded, and the obtained predictive image and coding are performed. Encoding is performed on the difference value from the target picture. Here, a picture is a term representing one screen, and means a frame in a progressive image and a frame or a field in an interlaced image.

In the draft H.26L standard as of September 2001, in order to further improve the compression ratio, not only the picture encoded or decoded immediately before is referred to, but also the coded / decoded image is encoded before the image. It is possible to select an arbitrary picture from a plurality of decoded pictures and use it as a reference image.

FIG. 15 shows an explanatory diagram of the concept of the conventional moving image coding method and moving image decoding method. FIG. 15 shows an example in which an arbitrary picture is selected from the three pictures before the coded / decoded picture and used as a reference image. In FIG. 15, each picture is arranged in the order of display time, and the display time of the picture on the left is earlier. As for the coding order, the picture on the left is coded earlier. Therefore, the order of the bitstream is also picture J₁, Picture J₂, Picture J₃, Picture J₄Is. Therefore, the coded / decoded picture is picture J.₄In the case of picture J₁, Picture J₂, Picture J₃It is possible to select one of them as a reference image, and the coded / decoded picture is picture J.₅In the case of picture J₂, Picture J₃, Picture J₄It is possible to select one of them as a reference image.

FIG. 16 is a block diagram showing a configuration of a conventional moving image coding device. The moving image coding device 4 is a device that outputs an image coding signal Str obtained by compressing and coding the input image signal Vin and converting it into a bit stream such as variable length coding, and is a motion detection unit 401 and a selection unit 402. , Image signal subtraction unit 403, coding unit 404, decoding unit 405, addition unit 406, selection unit 407, and memories 408 to 410.

The motion detection unit 401 reads out the reference images, which are already encoded images, stored in the memories 408 to 410, and compares them with the input image signal Vin to determine the magnitude of the difference value between screens (error energy). ) Is small The reference image Ref and the difference value between the screens are small The motion information MV indicating the pixel position is determined. Usually, the reference image Ref and the pixel position that minimize the error energy are often determined, but recently, the error energy is simply minimized. Instead, a method of determining the motion information MV so that the error energy is small and the compression rate can be large is also used. In addition, the reference image Ref and the pixel position information move together. We will call it information MV. The selection unit 402 outputs a reference image Ref selected from the reference image Ref1, the reference image Ref2, and the reference image Ref3 stored in the memories 408 to 410 based on the reference image instruction signal RefFrm which is a switching instruction signal. .. The subtraction unit 403 calculates the difference image signal Dif between the image signal Vin and the reference image Ref.

The coding unit 404 encodes the difference image signal Dif and the motion information MV, which is information for identifying the reference image. The decoding unit 405 decodes the coded data Coded encoded by the coding unit 404 to obtain the restored difference image signal RecDif. The addition unit 406 adds the reference image Ref and the restored difference image signal RecDif. The selection unit 407 makes it possible to refer to the input decoded image signal Recon as a reference image in the coding of the subsequent picture, so that the decoded image signal Rec1, the decoded image signal Rec2, and the decoded image signal are stored in any of the memories 408 to 410. Output as Rec3.

Next, the operation of the moving image coding device configured as described above will be described. The image signal Vin is input to the image signal subtraction unit 403 and the motion detection unit 401. The motion detection unit 401 reads out the reference image Ref1, the reference image Ref2, and the reference image Ref3, which are already decoded images stored in the memories 408 to 410, and compares them with the input image signal Vin between the screens. The reference image that minimizes the magnitude of the difference value is determined, and the motion information MV, which is information for specifying the reference image and the reference pixel position, is output.

At the same time, the motion detection unit 401 outputs the reference image instruction signal RefFrm, which is a switching instruction signal, so that the selection unit 402 can select the reference image corresponding to the motion information MV and output it as the reference image Ref. In addition, since the correlation between screens is lost due to scene changes, etc., in-screen coding (Intra Coding Picture: hereinafter abbreviated as Intra Picture) that can be restored only by the image coding signal of the coded picture when coded between screens is used. Can also reduce the compression ratio is there. In that case, the motion detection unit 401 indicates that the motion information MV is in-screen coding, and outputs a reference image instruction signal RefFrm for outputting the reference image Ref4 that always outputs a value 0 as the reference image Ref. To do. The value of the reference image Ref4 does not necessarily have to be 0. For example, in the case of a luminance signal or an RGB color signal having a value of 0 to 255, the average value may be 128. I.

Further, in order to prevent error propagation and enable image reproduction from the middle of the image-coded signal, in-screen coding that can be restored only by the image-coded signal of the coded picture is performed for each fixed number of pictures. There is a need. Therefore, the motion detection unit 401 can forcibly switch to the in-screen coding by the instruction of the in-screen coding instruction signal Reset given from the outside.

On the other hand, the subtraction unit 403 to which the image signal Vin is input calculates the difference between the image signal Vin and the reference image Ref selected by the selection unit 402, and outputs the difference image signal Dif to the coding unit 404. Next, the coding unit 404 encodes the difference image signal Dif and the motion information MV output by the motion detection unit 401, and outputs the image coding signal Str and the coded data Coded. Here, the coded data Coded is the data necessary for restoring the image, and the image coded signal Str is the coded data Coded further converted into a bit stream such as variable length coding.

The decoding unit 405 decodes the coded data Coded and outputs the restored difference image signal RecDif to the addition unit 406. The addition unit 406 adds the restored difference image signal RecDif and the reference image Ref selected by the selection unit 402, and outputs the decoded image signal Recon to the selection unit 407. The selection unit 407 outputs the decoded image signal Rec1 as the decoded image signal Rec1, the decoded image signal Rec2, and the decoded image signal Rec3 to any of the memories 408 to 410 so that the decoded image signal Recon can be referred to as a reference image when the subsequent picture is encoded. To do. In this example, the selection unit 407 switches so that the image saved in memory at the oldest time is overwritten by the new decoded image signal Recon.

FIG. 17 is a block diagram showing a configuration of a conventional moving image decoding device. The moving image decoding device 5 is a device that decodes the image coding signal Str encoded by the moving image coding device 4 as described above.

The decoding unit 501 decodes the input image coding signal Str and outputs the restored difference image signal RecDif and the motion information MV. The motion restoration unit 502 decodes the motion information MV and outputs a reference image instruction signal RefFrm. The operations of the selection unit 503, the selection unit 505, and the memories 506 to 508 are the same as those of the selection unit 402, the selection unit 407, and the memories 408 to 410 of the moving image coding device 4 shown in FIG. The addition unit 504 adds the restored difference image signal RecDif and the reference image Ref and outputs the decoded image signal Vout (which corresponds to the decoded image signal Recon in FIG. 16).

The moving image coding device 4 and the moving image decoding device 5 are provided with motion compensation units (not shown) on the output sides of the selection unit 402 and the selection unit 503, respectively, based on the pixel values of the reference images output from the memory. Decimal pixel position, which is a pixel value such as 1/2 pixel position accuracy Motion compensation is performed to interpolate and generate accurate pixel values.<nplcit num="1"><text>ISO / IEC 13818-2 "INTERNATIONAL STANDARD Information technology Generic coding of moving pictures and associated audio information: Video" December 15, 2000, p.7, Intro.4.1.1</text></nplcit>

<p> By the way, in the conventional moving image coding device and the conventional moving image decoding device as described above, the reference image is an in-screen coded image (Intra Picture) or an in-screen coded image. Since then, there is no distinction between screen-encoded images (Inter Picture). example For example, in the explanatory diagram of the concept of the conventional moving image coding method and moving image decoding method of FIG. 15, the picture J₂Is the in-screen coded picture, and the other pictures J₁, Picture J₃, Picture J₄, Picture J₅Is a screen-coded picture, but picture J₄Reference Picture J as an image₁Can also be referenced. If picture J₄Is a picture J as a reference image₁In-screen coded picture J₂It means that the earlier picture is referred to as the reference image.</p><p> However, when the image is reproduced from the middle, for example, the in-screen coded picture J in the middle of the image coding signal J₂Even if you try to decode and play the image from, Picture J₄Decrypted picture J when decoding₁Needs a reference to. Therefore, picture J₄The problem arises that subsequent pictures cannot be decoded correctly.</p><p> Also, for example, a stream error occurs in the middle of the image-encoded signal, and picture J₁Is wrong Therefore, if it cannot be decoded correctly, the in-screen coded picture J₂Can be decrypted correctly But picture J₄Picture J when decoding₁Picture J₄The problem arises that subsequent pictures cannot be decoded correctly.</p><p> Therefore, the present invention has been made in view of the above circumstances, and it is possible to reproduce from an in-screen encoded picture in the middle of an image-encoded signal, and when an error occurs in the stream. However, it is an object of the present invention to provide a moving image decoding method, a moving image decoding device, and the like that can reproduce a picture encoded in the screen and thereafter without any error.</p>

<p> In order to achieve the above object, in the moving image decoding method according to the present invention, in the moving image decoding method, a picture is decoded by either an in-screen decoding method or an inter-screen decoding method, and the screen is described. When decoding a picture by the inter-screen decoding method, the in-screen decoding picture decoded by the in-screen decoding method, the picture located in front of the in-screen decoding picture in the display order, and the in-screen decoding method. A decoding reference picture is selected from at least one of the pictures located after the decoded picture in the display order, and the moving image decoding method decodes the encoded picture to decode the decoded picture. Is obtained, and the decoded picture is an in-screen decoded picture designated as a reference picture as a reference for limiting the decoding reference pictures that can be selected when decoding another encoded picture. Is a decoding step in which the designated information indicating that the decoded picture has been designated as the reference picture is also decoded, and the decoded picture, which is an in-screen decoded picture, is used as the reference picture based on the designated information. The specified step to be specified and the picture decoded before the reference picture are decoded and stored in the memory before the reference picture so as to prevent the picture from being selected as the decoding reference picture. When the decrypted picture, which is an in-screen decoded picture, is designated as the reference picture in the designated step, including the delete step of deleting the picture, the picture is decoded by the inter-screen decoding method after the reference picture. For the picture to be invented, a decoding reference picture is selected from at least one of the decoded picture designated as the reference picture and the picture decoded after the reference picture, and the decoding reference picture is selected from the reference picture. It is characterized by prohibiting a previously decoded picture from being selected as a decoding reference picture.</p><p> Further, the moving image decoding method according to the present invention is a moving image decoding method for decoding a coded picture, and in the moving image decoding method, an in-screen decoding method and an inter-screen decoding method are used. When the picture is decoded by any of the above and the picture is decoded by the inter-screen decoding method, it is displayed from the in-screen decoded picture decoded by the in-screen decoding method and the in-screen decoded picture. The decoding reference picture is selected from at least one of the pictures located in front of the picture in order and the pictures located after in the display order from the decoded picture in the screen. The encoded picture is decoded to obtain the decoded picture, and the decoding target picture limits the coding reference pictures that can be selected when encoding other coding target pictures as the in-screen coded picture. When it is designated as a reference coding reference picture, the decoding step of decoding the specified information together and the decoding that can be selected when decoding the decoding target picture based on the specified information. The decoding reference picture includes a designation step of designating the decryption picture which is an in-screen decoding picture as a reference decoding reference picture when the invention reference picture is restricted, and the decoding picture is used as a decoding reference picture in the designation step. When specified, for a picture to be decoded by the inter-screen decoding method after the decoding reference picture, the decoding picture designated as the decoding reference picture and the decoding reference picture are used. Selecting a decoding reference picture from at least one of the later decoded pictures, and prohibiting a picture decoded before the decoding reference picture from being selected as the decoding reference picture. It is characterized by.</p><p> Further, the moving image decoding method further decodes the picture before the decoding reference picture in order to prohibit the picture decoded before the decoding reference picture from being selected as the decoding reference picture. It may be characterized by including a delete step of deleting the picture stored in the memory.</p><p> Further, in order to achieve the above object, the moving image coding method according to the present invention is a method of encoding a moving image in units of pictures, and is in the screen from a plurality of pictures stored in the memory. A difference between a specific step of specifying one encoded picture after the encoded picture as a reference picture and a reference picture read out from the memory and the encoded picture. A coding step of calculating an image signal and encoding the obtained difference image signal, decoding the encoded difference image signal, adding it to the image signal of the reference picture, and adding the obtained picture to the memory. It is characterized by including a storage step to be stored in.</p><p> Further, the moving image coding method according to the present invention is a method of encoding a moving image in units of pictures, and refers to a plurality of pictures stored in a memory and encoded in the screen as reference pictures. The designated step specified as the reference picture when limiting the number of pictures and the picture encoded after the picture specified in the designated step are only the pictures encoded after the picture specified in the designated step. It is characterized by including a coding step of encoding a signal that is referenced and indicates that the reference picture is a reference picture when limiting the reference picture.</p><p> Further, the moving image coding method according to the present invention is a method of encoding a moving image in units of pictures, and refers to a plurality of pictures stored in a memory and encoded in the screen as reference pictures. A designated step that is specified as a reference picture when limiting, a step that deletes a picture encoded before the picture specified in the specified step, and a step that encodes after the picture specified in the specified step. A signal indicating that the picture to be encoded deletes a picture encoded before the picture specified in the specified step so that only the pictures encoded after the picture specified in the specified step are referred to. It is characterized by including a coding step for encoding.</p><p> Further, the moving image decoding method according to the present invention is a method of decoding a moving image in units of pictures, and includes a decoding step for decoding an input image coding signal and a plurality of methods stored in a memory. A specific step of specifying one picture decoded after the in-screen decoded picture as a reference picture, and an image signal of the read reference picture by reading the specified reference picture from the memory. Is added to the difference image signal of the decoded picture to be decoded, the obtained picture is output to the outside, and a storage step of storing the picture in the memory is included.</p><p> Further, the moving image decoding method according to the present invention is a method of decoding a moving image in units of pictures, and is a reference picture when limiting a reference picture to be referred to when decoding a decoding target picture. The decoding step of decoding the signal indicating that, and the pictures decoded after the reference picture are the reference pictures and decoded after the in-screen decoded picture. It is characterized by including a specific step of identifying only a picture as a reference picture.</p><p> Further, the moving image decoding method according to the present invention is a method of decoding a moving image in units of pictures, and is a reference picture when limiting a reference picture to be referred to when decoding a decoding target picture. A decoding step of decoding the signal indicating that the above is the case, a step of deleting the reference picture which is decoded before the in-screen decoded picture, and a step after the reference picture. The picture decoded in the above is characterized by including a specific step of identifying only the picture decoded after the in-screen decoded picture as the reference picture, which is the reference picture.</p><p> Further, the moving image coding device according to the present invention is a moving image coding device that encodes a moving image in units of pictures, and is encoded in the screen from a plurality of pictures stored in a memory. A specific means for specifying one picture encoded after the picture as a reference picture and a difference image signal which is a difference between the read reference picture and the coded picture are calculated by reading the specified reference picture from the memory. A coding means for encoding the obtained difference image signal, decoding the encoded difference image signal, adding the encoded difference image signal to the image signal of the reference picture, and storing the obtained picture in the memory. It is characterized by having means.</p><p> Further, the moving image decoding device according to the present invention is a moving image decoding device that decodes a moving image in units of pictures, and has a decoding means for decoding an input image coding signal and a memory. A specific means for identifying one picture decoded after the in-screen decoded picture from a plurality of stored pictures as a reference picture, and a reference that reads the specified reference picture from the memory. The image signal of the picture is added to the difference image signal of the decoded picture to be decoded, the obtained picture is output to the outside, and a storage means for storing the picture is provided.</p><p> Further, the present invention is realized as a program for causing a computer to execute the steps in the moving image coding method and the moving image decoding method, and as stream data encoded by the moving image coding method, and is realized as a CD-ROM or communication. It can also be distributed via a recording medium such as a network or a transmission medium.</p><p> For example, the picture is picture J₁, Picture J₂, Picture J₃, Picture J₄, Picture J₅Encoded in the order of, picture J₂Is the in-screen coded picture, and the other pictures J₁, Picture J₃, Picture J₄, Picture J₅Is a screen-encoded picture. That is, the order of the bitstream is picture J₁, Picture J₂, Picture J₃, Picture J₄, Picture J₅Is the order.</p><p> In this case, in the moving image coding method and the moving image decoding method according to the present invention, the picture that can be selected as the reference image when performing interscreen coding and decoding is, for example, picture J.₅Mark When issuing, picture J₂, Picture J₃, Picture J₄Can be referred to. Also, Picture J₄In-screen coded picture J₂Earlier picture J₁Since it is forbidden to refer to Picture J₂And picture J₃Only can be referenced.</p>

<p> As is clear from the above description, the moving image coding method according to the present invention is a method of encoding a moving image in units of pictures, and is an in-screen code from a plurality of pictures stored in a memory. A specific step of specifying one encoded picture after the encoded picture as a reference picture, and a difference image which is a difference between the specified reference picture read from the memory and the read reference picture and the encoded picture. A coding step of calculating a signal and encoding the obtained difference image signal, decoding the encoded difference image signal, adding it to the image signal of the reference picture, and storing the obtained picture in the memory. It is characterized by including a storage step to be stored.</p><p> As a result, since the coding in which the picture before the in-screen coded picture is referred to as the reference image is not performed, the in-screen coded picture in the middle of the image coding signal can be reproduced. Further, even if an error occurs in the stream, an image-encoded signal, which is a stream that can be reproduced without an error after the in-screen encoded picture, can be generated. In this way, it has great advantages in terms of reproduction from the middle, which is important for storage media, and error tolerance during transmission, which is important for wireless / wired transmission, and is excellent in actual applications.</p><p> Further, the moving image decoding method according to the present invention is a method of decoding a moving image in units of pictures, and includes a decoding step for decoding an input image coding signal and a plurality of methods stored in a memory. A specific step of specifying one picture decoded after the in-screen decoded picture as a reference picture, and an image signal of the read reference picture by reading the specified reference picture from the memory. Is added to the difference image signal of the decoded picture to be decoded, the obtained picture is output to the outside, and a storage step of storing the picture in the memory is included.</p><p> As a result, the decoded image signal can be obtained by correctly decoding the encoded image-encoded signal by referring only to the pictures after the in-screen decoded picture as the reference image, and the screen in the middle of the image-encoded signal. Playback can be performed from the internally coded picture. Further, even if an error occurs in the stream, it is possible to reproduce the encoded picture on the screen without any error.</p>

Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 14.

(Embodiment 1) FIG. 1 is a block diagram showing a configuration of an embodiment of a moving image coding device according to the present invention. Devices that operate in the same manner as each unit of the conventional moving image coding device 4 shown in FIG. 16 are designated by the same symbols.

The moving image coding device 1 is a device that outputs an image coding signal Str obtained by compressing and coding the input image signal Vin and converting it into a bit stream such as variable length coding, and outputs the motion detection unit 101 and the selection unit 402. , Image signal subtraction unit 403, coding unit 404, decoding unit 405, addition unit 406, selection unit 407, memories 408 to 410, and counter unit 102.

The counter unit 102 is in the screen by the in-screen coding instruction signal Reset input from the outside. When the coding is instructed, the counting of the number of pictures from the coded pictures in the screen is started, and the result is notified to the motion detection unit 101 as the number of referenceable pictures Num.

The motion detection unit 101 encodes and decodes the reference image Ref1, the reference image Ref2, and the reference image Ref3 stored in the memories 408 to 410 after the in-screen coding based on the number of referenceable pictures Num. Motion information MV showing the reference image Ref and the pixel position where the magnitude of the inter-screen difference value becomes small by comparing each with the image signal Vin. To determine. Normally, the reference image Ref and the pixel position that minimize the error energy are often determined, but the motion information MV is not such that the error energy is minimized, but the error energy is small and the compression ratio can be large. May be determined. To indicate the reference image Ref, the reference image instruction signal RefFrm is output to the selection unit 402.

The selection unit 402 selects from the reference image Ref1, the reference image Ref2, and the reference image Ref3 stored in the memories 408 to 410 based on the reference image instruction signal RefFrm which is the switching instruction signal, and outputs the reference image Ref. ..

The subtraction unit 403 calculates the difference image signal Dif between the image signal Vin and the reference image Ref. The coding unit 404 encodes the difference image signal Dif and the motion information MV which is information for specifying the reference image, and outputs the image coding signal Str and the coding data Coded. Here, the coded data Coded is the data (data obtained by encoding the motion information MV and the difference image signal Dif) for restoring the image, and the image coded signal Str is the coded data Coded with a variable length code. It is converted to a bit stream such as conversion.

The decoding unit 405 decodes the coded data Coded to obtain the restored difference image signal RecDif. The addition unit 406 adds the reference image Ref and the restored difference image signal RecDif. The selection unit 407 makes it possible to refer to the input decoded image signal Recon as a reference image when encoding the subsequent picture, so that the decoded image signal Rec1, the decoded image signal Rec2, and the decoded image signal are stored in any of the memories 408 to 410. Output as Rec3. In the present embodiment, the selection unit 407 switches so that the decoded image signal saved in the memory at the oldest time is overwritten by the new decoded image signal Recon.

Next, the operation of the moving image coding device configured as described above will be described. FIG. 2 is a flow chart showing the operation of the motion detection unit 101.

The image signal Vin is input to the image signal subtraction unit 403 and the motion detection unit 101.

When the image signal Vin is input, the motion detection unit 101 reads out the reference image Ref1 stored in the memory 408 (step S1). Next, the motion detection unit 101 determines whether or not the reference image Ref1 is a picture after the coded picture in the screen. That is, based on the picture number unique to each picture and the referenceable picture number Num notified by the counter unit 102, a judgment is made using, for example, the equation (A) (step S2). Here, the picture number is an identification number attached to the encoded picture and has the following features.

That is, the picture number of the image signal Vin is one higher than the picture number of the most recently saved picture of the pictures stored in the memories 408 to 410 and used as the reference image.

Picture number of the reference image Picture number of image signal Vin-Number of referenceable pictures Num ...... (A)

As a result, when the above equation (A) is satisfied, the reference image Ref1 is a picture after the in-screen encoded picture, so the motion detection unit 101 calculates the difference value between the image signal Vin and the reference image Ref1. (Step S3). On the other hand, if the above equation (A) is not satisfied, the difference is not calculated.

Next, the motion detection unit 101 performs the same processing as the processing performed on the reference image Ref1 described above for the reference image Ref2 stored in the memory 409 and the reference image Ref3 stored in the memory 410 (step). S1 ~ step S3).

Next, the motion detection unit 101 determines the reference image having the smallest inter-screen difference value among the reference images for which the difference has been calculated as described above (step S4). Then, the motion detection unit 101 outputs as motion information MV which is information for specifying the determined reference image, and at the same time, the selected reference image can be selected by the selection unit 402 and output as a reference image Ref. The reference image instruction signal RefFrm, which is a switching instruction signal, is output (step S5).

After calculating the difference value in step S3, a reference image having a small inter-screen difference value is determined among the reference images for which the difference was calculated in step S4, and steps S1, step S2, and step S3. , Step S4 may be repeated in memory 408-410.

On the other hand, the subtraction unit 403 to which the image signal Vin is input calculates the difference between the image signal Vin and the reference image Ref selected by the selection unit 402, and outputs the difference image signal Dif to the coding unit 404. Next, the coding unit 404 encodes the difference image signal Dif and the motion information MV output by the motion detection unit 101, and outputs the image coding signal Str and the encoded data Coded.

The decoding unit 405 decodes the coded data Coded and outputs the restored difference image signal RecDif to the addition unit 406. The addition unit 406 adds the restored difference image signal RecDif and the reference image Ref selected by the selection unit 402, and outputs the decoded image signal Recon to the selection unit 407. The selection unit 407 outputs the decoded image signal Rec1 as the decoded image signal Rec1, the decoded image signal Rec2, and the decoded image signal Rec3 to any of the memories 408 to 410 so that the decoded image signal Recon can be referred to as a reference image when the subsequent picture is encoded. To do.

FIG. 3 is an explanatory diagram showing a picture that can be selected as a reference image at the time of image coding. Picture J as in Figure 15₂Is the in-screen coded picture, and the other pictures J₁, Picture J₃, Picture J₄, Picture J₅Is a screen-encoded picture. The coding order of the bitstream is picture J₁, Picture J₂, Picture J₃, Picture J₄, Picture J₅Is the order.

As shown in FIG. 3, a picture that can be selected as a reference image when performing inter-screen coding by operating the moving image coding device 1 as described above is, for example, picture J.₅To encode Sometimes picture J₂, Picture J₃, Picture J₄Can be referred to. Also, the picture J₄In-screen coded picture J₂Earlier picture J₁Refer to Picture J₂And picture J₃Only can be referenced.

As described above, since the coding is performed by referring only to the pictures encoded after the in-screen coded picture as the reference image, the in-screen coded picture in the middle of the image coding signal is reproduced. It can be carried out. Further, even if an error occurs in the stream, an image-encoded signal Str, which is a stream that can be reproduced without an error after the in-screen encoded picture, can be generated.

Also, in order to play back the image signal recorded on the DVD or hard disk from the middle, this A mechanism that can be restored from the middle is required, and Reset is instructed for the picture that you want to start playing from the middle. The reset instruction may be determined and instructed by the operator when encoding the image, or the reset instruction may be issued every time a certain picture cycle or time elapses.

It should be noted that each picture has information such as whether each picture is in-screen coded or inter-screen coded as auxiliary information of each picture. Therefore, the in-screen coded picture described in the first embodiment is further specified by the in-screen coding instruction signal Reset. The signal indicating that the picture is a picture may be encoded (Fig. 4). Then, the picture is encoded in the screen and is the picture specified by the in-screen coding instruction signal Reset. When the picture is shown to be, the picture stored in the memory before the specified picture may not be referred to.

It also means that the picture stored before the in-screen coded picture is not referred to based on the in-screen coded picture, but the in-screen coded picture causes the in-screen coded picture. Based on the encoded picture, the picture stored before the encoded picture in the screen may be deleted from the memory.

In addition, the in-screen coding instruction signal Reset is stored before the in-screen coded picture. This in-screen coding instruction signal Reset is meant to not refer to the in-screen picture, but to delete the picture stored before the in-screen coded picture from the memory. A control instruction different from that of the control instruction may be encoded.

FIG. 5 is a block diagram showing a configuration of an embodiment of the moving image decoding apparatus according to the present invention. Devices that operate in the same manner as each unit of the conventional moving image decoding device 5 shown in FIG. 17 are designated by the same symbols.

The moving image decoding device 2 is a device that decodes the image coding signal Str encoded by the moving image coding device 1 shown in the first embodiment, and is a decoding unit 201, a motion restoration unit 202, and a selection. It includes a unit 503, an addition unit 504, a selection unit 505, memories 506 to 508, and a counter unit 203.

The decoding unit 201 decodes the input image coding signal Str, and outputs the restored difference image signal RecDif, the motion information MV, and the in-screen coding instruction signal Reset. When the in-screen coding instruction signal Reset is instructed by the in-screen coding instruction signal Reset, the counter unit 203 starts counting the number of pictures from the in-screen coding, and moves the result as the referenceable number of pictures Num to restore the unit 202. Notify to.

The motion restoration unit 202 decodes the motion information MV, determines whether or not the reference image corresponding to the motion information MV is a reference image based on the number of referenceable pictures Num, and then determines whether or not the reference image is a reference image. To determine.

The selection unit 503 selects from the reference image Ref1, the reference image Ref2, and the reference image Ref3 stored in the memories 506 to 508 based on the reference image instruction signal RefFrm which is the switching instruction signal, and outputs the reference image Ref. .. The addition unit 504 adds the restored difference image signal RecDif and the reference image Ref and outputs the decoded image signal Vout1 (which corresponds to the decoded image signal Recon in FIG. 1).

The selection unit 505 makes it possible to refer to the input decoded image signal Vout1 as a reference image when decoding a subsequent picture, so that the decoded image signal Rec1, the decoded image signal Rec2, and the decoded image signal are stored in any of the memories 506 to 508. Output as Rec3. In the present embodiment, the selection unit 505 switches so that the image saved in the memory at the oldest time is overwritten by the new decoded image signal Recon.

Next, the operation of the moving image decoding apparatus configured as described above will be described. FIG. 6 is a flow chart showing the operation of the motion restoration unit 202.

The image coding signal Str is input to the decoding unit 201. The decoding unit 201 decodes the input image coding signal Str and outputs the restored difference image signal RecDif and the motion information MV. Further, when the input image coding signal Str is an in-screen coded picture, the decoding unit 201 outputs an in-screen coding instruction signal Reset.

When the in-screen coding instruction signal Reset is input from the decoding unit 201, the counter unit 203 starts calculating the number of pictures from the in-screen coded pictures, and moves as the referenceable picture number Num to restore the unit 202. Notify to.

The motion restoration unit 202 in which the motion information MV and the number of referenceable pictures Num are input identifies the reference image referenced at the time of coding based on the motion information MV, which is information for identifying the reference image (step S11). .. Next, the motion restoration unit 202 determines whether or not the reference image is a picture after the image decoded in the screen. That is, a determination is made using, for example, the following equation (B) based on the picture number unique to each picture and the referenceable picture number Num notified by the counter unit 203 (step S12).

Picture number of the reference image Picture number of image-coded signal Str-Number of referenceable pictures Num ...... (B)

As a result, when the above equation (B) is satisfied, the reference image is a picture after the image decoded in the screen, so that the motion restoration unit 202 decodes the reference image specified based on the motion information MV. Select it as the reference image to use (step S13).

On the other hand, when the above equation (B) is not satisfied, the reference image is a picture before the in-screen decoded picture. Originally, at the time of coding, it should be encoded by referring only to the pictures after the coded picture in the screen. Therefore, the reference image required for decoding specified based on the motion information MV should be a picture decoded after the in-screen decoded picture obtained from the referenceable picture number Num. However, there is a possibility that the picture decoded before the picture decoded in the screen is referred to due to a transmission error or the like, that is, the above equation (B) is not satisfied. Therefore, the motion restoration unit 202 is selected by one of the methods 1 to 3 described below in which the reference image to be used for decoding is preset when the above equation (B) is not satisfied (step S14).

FIG. 7 is an explanatory diagram illustrating selection by the reference image methods 1 to 3 used for decoding, (a) an explanatory diagram of the reference image used for decoding, and (b) an explanatory diagram showing the positional relationship of the pictures. Is. Here, picture P₄Indicates the time of decoding of picture P₂Is the in-screen decoded picture, and the other pictures P₁, Picture P₃, Picture P₄Is decrypted between screens It is a picture. The order of the picture display time and the order of the picture decoding start time (position in the stream) is picture P.₁, Picture P₂, Picture P₃, Picture P₄The old time is in the order of.

(Method 1) Reference image identified based on motion information MV (picture P₁) Is used as it is for decryption Select as a reference image. In this case, even if the picture P is encoded by referring to the picture before the in-screen coded picture by mistake at the time of coding.₁Is correctly decrypted If it is correct, it can be decrypted correctly.

(Method 2) In-screen decrypted picture (Picture P₂) Is selected as the reference image used for decoding To choose. In this case, since the in-screen decoded picture is the earliest decoded picture among the pictures that can be referred as the reference image, it is before the in-screen decoded picture among the referenceable pictures. It has the strongest correlation with the picture, and there is a high possibility that the image quality will not be impaired if the reference image at the time of encoding is a picture decoded before the in-screen decoded picture.

(Method 3) Immediately decrypted picture (Picture P₃) Is selected as the reference image used for decoding To choose. In general, the shorter the time interval, the higher the correlation between the image signals, and as a result, the probability that the immediately decoded picture becomes the reference image is very high. Therefore, if the reference image identified based on the motion information MV is incorrect, it is highly possible that the picture decoded immediately before the strongest correlation is the original reference image, and the image quality may not be impaired. Highly sex.

It should be noted that any one of these methods 1 to 3 may be used, or a plurality of methods may be used in combination. As an example of the combination, the reference image (picture P) specified based on the motion information MV in the method 1₁) Is selected as the reference image to be used for decoding as it is When it is not possible, for example, the default process of selecting the immediately-decrypted picture shown in method 3 as the reference image to be used for decoding is performed.

Next, the motion restoration unit 202 sequentially reads out the reference image Ref1, the reference image Ref2, and the reference image Ref3 stored in the memories 506 to 508, and determines whether or not the reference image is the reference image selected as described above. Do this (Figure 6, step S15). That is, it specifies where in the memory 506 to 508 the selected reference image is stored. Then, the reference image instruction signal RefFrm, which is a switching instruction signal, is output so that the selection unit 503 can select one of the specified reference image Ref1, reference image Ref2, and reference image Ref3 and output it as the reference image Ref (FIG. 6, step S16).

The addition unit 504 adds the reference image Ref and the restored difference image signal RecDif output by the decoding unit 201, and outputs the decoded image signal Vout1 (which corresponds to the decoded image signal Recon in FIG. 1) to the selection unit 505. To do. The selection unit 505 makes it possible to refer to the input decoded image signal Vout1 as a reference image when decoding a subsequent picture, so that the decoded image signal Rec1, the decoded image signal Rec2, and the decoded image signal are stored in any of the memories 506 to 508. Output as Rec3.

As described above, the decoded image signal Vout1 can be obtained by correctly decoding the encoded image coding signal Str by referring only to the pictures after the decoded picture in the screen as the reference image, and the image coding signal. Playback can be performed from the in-screen encoded picture in the middle of. Further, even if an error occurs in the stream, it is possible to reproduce the encoded picture on the screen without any error.

In the present embodiment, the methods 1 to 3 for selecting the reference image used by the motion restoration unit 202 for decoding are set in advance, but the present invention is not limited to this. For example, if the picture specified by the motion information MV is significantly different from the number of pictures that can be referred to, method 3, if the picture specified by the motion information MV is stored in the memory and can be referred to, method 1, Other than that, you may dynamically switch between the three methods, such as method 2, or two of them, depending on the situation.

In addition, each picture has information such as whether each picture is decoded in the screen or between screens as auxiliary information of each picture. Therefore, the image coding signal Str including the signal indicating that the in-screen coded picture described in the first embodiment is the picture specified by the in-screen coding instruction signal Reset is received, and the screen is displayed. The signal indicating that the picture is the picture specified by the internal coding instruction signal Reset may be decoded. Then, when the picture is encoded in the screen and is indicated to be the picture specified by the in-screen coding instruction signal Reset, it is stored in the memory before the specified picture. It may be possible not to refer to the rendered (decrypted) picture.

It also means that the picture stored before the in-screen decoded picture is not referred to based on the in-screen decoded picture, but the in-screen decoded picture causes the in-screen Based on the decoded picture, the picture stored before the decoded picture in the screen may be deleted from the memory.

Further, in order to delete the picture stored before the in-screen coded picture from the memory, an image coding message including a control command different from the in-screen coding instruction signal Reset is included. The error control may be performed by receiving the No. Str, decoding this control command, determining whether the control command is received together with the signal indicating that the picture is the picture specified by the in-screen coding instruction signal Reset.

(Embodiment 2) Next, the moving image decoding apparatus according to the second embodiment of the present invention will be described.

FIG. 8 is a block diagram showing the configuration of the second embodiment of the moving image decoding apparatus according to the present invention. Devices that operate in the same manner as each unit of the moving image decoding device 2 shown in FIG. 5 are designated by the same symbols, and the description thereof will be omitted.

The moving image decoding device 3 is provided with the selection unit 301 added to the configuration of the moving image decoding device 2 shown in FIG. 5, and the reference image specified by the motion information MV due to a transmission error or the like is decoded in the screen. The operation when the picture is decoded before the picture is different from that of the first embodiment.

The selection unit 301 decodes either the decoded image signal Recon output from the addition unit 504 or the reference image Ref output from the selection unit 503 based on the error notification signal Err notified by the motion restoration unit 302. Output as signal Vout2.

Next, the operation of the moving image decoding apparatus configured as described above will be described, but the description of the same part as that of the first embodiment will be omitted.

FIG. 9 is a flow chart showing the operation of the motion restoration unit 302. The operations of steps S21 to S22 shown in FIG. 9 are the same as the operations of steps S11 to S12 shown in FIG. Next, the motion restoration unit 302 determines whether or not the specified reference image is a picture after the image decoded in the screen, and as a result, when the above equation (B) is not satisfied, that is, the reference image is If the image is decoded before the image decoded in the screen, the error notification signal Err is notified to the selection unit 301 as an error. Further, the motion restoration unit 302 selects the decoded image decoded immediately before as the reference image and the decoded image signal (step S23). That is, through the error notification signal Err If it is known, the selection unit 301 outputs the reference image Ref as the decoded image signal Vout2. Therefore, if the decoded image decoded immediately before is used as the reference image, the decoded image decoded immediately before is used as the decoded image signal. Will be done.

On the other hand, as a result of the above determination, when the above equation (B) is satisfied, the reference image is a picture after the image decoded in the screen, so that the motion restoration unit 302 has the motion information MV as in the second embodiment. The reference image identified based on is selected as the reference image to be used for decoding (step S24).

The operation of steps S25 to S26 shown in FIG. 9 below is the same as the operation of steps S15 to S16 shown in FIG.

Next, the addition unit 504 adds the reference image Ref and the restored difference image signal RecDif output by the decoding unit 201, and outputs the decoded image signal Recon. When the motion restoration unit 302 does not notify the error notification signal Err, the selection unit 301 outputs the decoded image signal Recon output from the addition unit 504 as the decoded image signal Vout2, and notifies the error notification signal Err. If so, the reference image Ref output from the selection unit 503 is output as the decoded image signal Vout2. That is, when the error notification signal Err is notified, the decoded image decoded immediately before is output as it is as the decoded image signal Vout2.

The selection unit 505 makes it possible to refer to the input decoded image signal Vout2 as a reference image when decoding the subsequent picture, so that the decoded image signal Rec1, the decoded image signal Rec2, and the decoded image signal are stored in any of the memories 506 to 508. Output as Rec3.

As described above, when an error of referencing a picture before the image decoded in the screen occurs due to, for example, a transmission error, the reference image Ref, which is the immediately preceding decoded image having the strongest pixel correlation, is decoded. Since it is output as an image signal Vout2, the effect of image deterioration due to errors Can be minimized.

In each of the above embodiments, since the number of memories is 3, a maximum of 3 encoded images can be referred to, but the present invention is not limited to this, and more coding can be performed by increasing the number of memories. It is possible to encode and decode with reference to the image.

Further, in each of the above-described embodiments, the moving image coding device 1 and the moving image decoding devices 2 and 3 are provided with motion compensation units (not shown) on the output sides of the selection unit 402 and the selection unit 503, respectively, and have pixels. Motion compensation is performed to correct the amount of motion between the pictures.

Further, in each of the above embodiments, when performing inter-screen coding, if there is an in-screen coded picture, all of them are restricted so that the picture encoded before that is not referred to as a reference image. However, the present invention is not limited to this. For example, even if the picture is encoded in the screen, the picture encoded before that is not restricted to be referred as the reference image, that is, the picture encoded before that is referred to as in the conventional case. It is also possible to provide an in-screen encoded picture of a type that can be referred to as an image and use it properly as needed. In this case, the information for distinguishing the two types of in-screen coded pictures can be stored in, for example, the header information in the image coding signal Str.

Further, the counts in the counter units 102 and 203 in each of the above embodiments may be in the display order of the pictures (Display order) instead of the coding and decoding order of the pictures (Decoding order).

(Embodiment 3) Further, by recording a program for realizing the configuration of the moving image coding method or the moving image decoding method shown in each of the above embodiments on a recording medium such as a flexible disk, each of the above-mentioned embodiments is carried out. The processing shown in the embodiment can be easily performed in an independent computer system.

FIG. 10 is an explanatory diagram of a case where a computer system is used to store a moving image coding method or a moving image decoding method according to the first and second embodiments.

FIG. 10 (b) shows the appearance, cross-sectional structure, and flexible disc of the flexible disc when viewed from the front, and FIG. 10 (a) shows an example of the physical format of the flexible disc which is the main body of the recording medium. The flexible disk FD is built in the case F, and a plurality of track Trs are concentrically formed on the surface of the disk from the outer circumference toward the inner circumference, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk in which the program is stored, the moving image coding method and the moving image decoding method as the program are recorded in the area allocated on the flexible disk FD.

Further, FIG. 10 (c) shows a configuration for recording / reproducing the above program on the flexible disk FD. When recording the above program on the flexible disk FD, the moving image coding method or the moving image decoding method as the above program is written from the computer system Cs via the flexible disk drive FDD. When the moving image coding method and the moving image decoding method are constructed in the computer system by the program in the flexible disk, the program is read from the flexible disk FD by the flexible disk drive FDD and transferred to the computer system.

In the above description, a flexible disk is used as the recording medium, but an optical disk can also be used in the same manner. The recording medium is not limited to this, and any recording medium such as an IC card or ROM cassette that can record a program can be used in the same manner.

Further, here, an application example of the moving image coding method and the moving image decoding method shown in the above embodiment and a system using the same will be described.

FIG. 11 is a block diagram showing the overall configuration of the content supply system ex100 that realizes the content distribution service. The communication service provision area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.

This content supply system ex100 is, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera via the Internet service provider ex102 and the telephone network ex104, and the base stations ex107 to ex110 on the Internet ex101. Each device such as mobile phone ex115 with is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 11, and any combination may be used for connection. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110, which are fixed radio stations.

The camera ex113 is a device capable of shooting moving images such as a digital video camera. In addition, the mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone. Alternatively, it may be PHS (Personal Handyphone System) or the like.

Further, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution based on the coded data transmitted by the user using the camera ex113 becomes possible. The captured data may be encoded by the camera ex113 or by a server or the like that performs data transmission processing. Further, the moving image data taken by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images such as a digital camera. In this case, the moving image data may be encoded by either the camera ex116 or the computer ex111. Further, the coding process is performed by the LSI ex117 of the computer ex111 and the camera ex116. Note that the image coding / decoding software may be incorporated into some storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium that can be read by a computer ex111 or the like. Further, the moving image data may be transmitted by the mobile phone ex115 equipped with a camera. The moving image data at this time is the data encoded by the LSI of the mobile phone ex115.

In this content supply system ex100, the content photographed by the user with the camera ex113, the camera ex116, etc. (for example, a video of a live music) is encoded and transmitted to the streaming server ex103 in the same manner as in the above embodiment. On the other hand, the streaming server ex103 streams the above content data to the requested client. Clients include a computer ex111, a PDAex112, a camera ex113, a mobile phone ex114, and the like, which can decode the encoded data. By doing so, the content supply system ex100 can receive the encoded data at the client and play it back, and by receiving it in real time at the client, decoding it, and playing it back, it also realizes personal broadcasting. It is a system that makes it possible.

For the coding and decoding of each device constituting this system, the moving image coding device or the moving image decoding device shown in each of the above embodiments may be used.

A mobile phone will be described as an example. FIG. 12 is a diagram showing a mobile phone ex115 using the moving image coding method and the moving image decoding method described in the above embodiment. The mobile phone ex115 is an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, images of a CCD camera, etc., a camera unit ex203 capable of taking still images, an image taken by the camera unit ex203, and an antenna ex201. Display unit ex202 such as liquid crystal display that displays the decoded data of received video, etc., main unit consisting of operation key ex204 group, audio output unit ex208 such as speaker for audio output, audio input To save encoded or decoded data such as audio input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. It has a slot portion ex206 for mounting the recording media ex207 on the recording media ex207 and the mobile phone ex115. The recording medium ex207 is an EEPROM (Electrically Erasable and) which is a non-volatile memory that can be electrically rewritten or erased in a plastic case such as an SD card. It stores a flash memory element, which is a type of Programmable Read Only Memory).

Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 has a power supply circuit unit ex310, an operation input control unit ex304, and image coding for the main control unit ex311 which is designed to collectively control each part of the main body unit provided with the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, multiplex separation unit ex308, recording / playback unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via the synchronization bus ex313. Connected to each other.

The power circuit unit ex310 activates the camera-equipped digital mobile phone ex115 in an operable state by supplying power to each unit from the battery pack when the call ends and the power key is turned on by the user's operation. ..

Based on the control of the main control unit ex311 consisting of CPU, ROM, RAM, etc., the mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305. This is subjected to spectrum diffusion processing by the modulation / demodulation circuit unit ex306, digital-to-analog conversion processing and frequency conversion processing by the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201. Further, the mobile phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog voice by the voice processing unit ex305. After converting to a signal, this is output via the audio output unit ex208.

Further, when the e-mail is transmitted in the data communication mode, the text data of the e-mail input by the operation of the operation key ex204 of the main body unit is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing of text data by the modulation / demodulation circuit unit ex306, digital-to-analog conversion processing and frequency conversion processing by the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image coding unit ex312 via the camera interface unit ex303. When the image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

The image coding unit ex312 has a configuration including the image coding device described in the present invention, and is a coding method using the image data supplied from the camera unit ex203 in the image coding device shown in the above embodiment. It is converted into coded image data by compression coding by, and this is sent to the multiplex separation unit ex308. At the same time, the mobile phone ex115 transmits the sound collected by the voice input unit ex205 during imaging by the camera unit ex203 to the multiplex separation unit ex308 as digital voice data via the voice processing unit ex305.

The multiplex separation unit ex308 multiplexes the coded image data supplied from the image coding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit. Spread spectrum processing is performed by ex306, digital-to-analog conversion processing and frequency conversion processing are performed by the transmission / reception circuit unit ex301, and then transmission is performed via the antenna ex201.

When receiving the data of the moving image file linked to the homepage etc. in the data communication mode, the received signal received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is performed. The data is sent to the multiplex separator ex308.

Further, in order to decode the multiplexed data received via the antenna ex201, the multiplexing separator ex308 separates the multiplexed data into a coded bit stream of image data and a coded bit stream of audio data. The encoded image data is supplied to the image decoding unit ex309 and the audio data is supplied to the audio processing unit ex305 via the synchronization bus ex313.

Next, the image decoding unit ex309 has a configuration including the image decoding device described in the present invention, and is a decoding method corresponding to the coding method shown in the above embodiment for the coded bit stream of the image data. The playback moving image data is generated by decoding with, and this is supplied to the display unit ex202 via the LCD control unit ex302, whereby the moving image data included in the moving image file linked to the homepage is displayed, for example. .. At the same time, the audio processing unit ex305 converts the audio data into an analog audio signal and then supplies the audio data to the audio output unit ex208, whereby, for example, the audio data contained in the moving image file linked to the homepage is reproduced. To.

Not limited to the above system, digital broadcasting by satellite and terrestrial broadcasting has recently become a hot topic, and as shown in FIG. 14, the digital broadcasting system also has at least an image coding device or an image of the above embodiment. Any of the decoding devices can be incorporated. Specifically, at the broadcasting station ex409, a coded bitstream of video information is transmitted via radio waves to a communication or a broadcasting satellite ex410. In response to this, the broadcasting satellite ex410 transmits radio waves for broadcasting, receives the radio waves with a home antenna ex406 equipped with satellite broadcasting receiving equipment, and receives the radio waves such as TV (receiver) ex401 or set-top box (STB) ex407. The device decodes the coded bit stream and reproduces it. Further, the image decoding device shown in the above embodiment can also be mounted on the playback device ex403 that reads and decodes the coded bit stream recorded on the storage medium ex402 such as a recording medium such as a CD or DVD. is there. In this case, the reproduced video signal is displayed on the monitor ex404. It is also conceivable to mount an image decoding device in a set-top box ex407 connected to a cable ex405 for cable TV or an antenna ex406 for satellite / terrestrial broadcasting, and reproduce this on a TV monitor ex408. At this time, the image decoding device may be incorporated in the television instead of the set-top box. It is also possible for the car ex412 having the antenna ex411 to receive a signal from the satellite ex410 or the base station ex107 or the like and reproduce the moving image on the display device such as the car navigation ex413 which the car ex412 has.

Further, the image signal can be encoded by the image coding apparatus shown in the above embodiment and recorded on a recording medium. Specific examples include a DVD recorder that records an image signal on a DVD disc ex421 and a recorder ex420 such as a disc recorder that records on a hard disk. .. It can also be recorded on the SD card ex422. If the recorder ex420 is equipped with the image decoding device shown in the above embodiment, a DVD disc ex421 or an SD card ex42 The image signal recorded in 2 can be played back and displayed on the monitor ex408.

The car navigation system ex413 may be configured by excluding the camera unit ex203, the camera interface unit ex303, and the image coding unit ex312 from the configurations shown in FIG. 13, and the same applies to the computer ex111 and the television (receiver). ) Ex401 etc. can also be considered.

In addition, terminals such as the mobile phone ex114 are implemented in three types: a transmitter / receiver terminal having both an encoder and a decoder, a transmitter terminal having only an encoder, and a receiving terminal having only a decoder. Can be considered.

As described above, it is possible to use the moving image coding method or the moving image decoding method shown in the above-described embodiment for any of the above-mentioned devices / systems, and by doing so, the above-described embodiment will be described. The effect can be obtained.

Further, the moving image coding method and the moving image decoding method shown in the above-described embodiment are effective when the stream data recorded on a storage medium such as a DVD, an SD card, or a memory is played back in the middle.

The moving image decoding method and the moving image decoding apparatus of the present invention can reproduce from an in-screen coded picture in the middle of the image coding signal, and when an error occurs in the stream. However, it has the effect of being able to reproduce the coded picture on the screen without any error, and can be applied to, for example, a digital video camera or a mobile phone.

<figref num="1">It is a block diagram which shows the structure of one Embodiment of the moving image coding apparatus which concerns on this invention.</figref><figref num="2">It is a flow chart which shows the operation of the motion detection unit in the said embodiment.</figref><figref num="3">It is explanatory drawing which shows the picture which can be selected as a reference image at the time of image coding in the said embodiment.</figref><figref num="4">It is a block diagram which shows the structure of the other embodiment of the moving image coding apparatus which concerns on this invention.</figref><figref num="5">It is a block diagram which shows the structure of one Embodiment of the moving image decoding apparatus which concerns on this invention.</figref><figref num="6">It is a flow chart which shows the operation of the motion restoration unit in the said embodiment.</figref><figref num="7">It is explanatory drawing explaining the selection by the method 1 to 3 of the reference image used for decoding in the said embodiment, (a) explanatory drawing of the reference image used for decoding, (b) the positional relationship of a picture. It is explanatory drawing which shows.</figref><figref num="8">It is a block diagram which shows the structure of Embodiment 2 of the moving image decoding apparatus which concerns on this invention.</figref><figref num="9">It is a flow chart which shows the operation of the motion restoration unit in the said embodiment.</figref><figref num="10">It is explanatory drawing about the recording medium for storing the program for realizing the moving image coding method and moving image decoding method of Embodiment 1 and Embodiment 2 by the computer system, and (a) recording medium. An explanatory diagram showing an example of the physical format of the flexible disk, which is the main body, (b) an explanatory view showing the appearance, cross-sectional structure, and flexible disk of the flexible disk from the front, and (c) recording the above program on the flexible disk FD. It is explanatory drawing which showed the structure for performing reproduction.</figref><figref num="11">It is a block diagram which shows the overall structure of the content supply system which realizes a content distribution service.</figref><figref num="12">It is a figure which shows an example of a mobile phone.</figref><figref num="13">It is a block diagram which shows the internal structure of a mobile phone.</figref><figref num="14">It is a block diagram which shows the whole structure of the system for digital broadcasting.</figref><figref num="15">It is explanatory drawing of the concept of the conventional moving image coding method and moving image decoding method.</figref><figref num="16">It is a block diagram which shows the structure of the conventional moving image coding apparatus.</figref><figref num="17">It is a block diagram which shows the structure of the conventional moving image decoding apparatus.</figref>

1,4 video coding device 2, 3, 5 video decoding device 101 Motion detection unit 102, 203 counter unit 201,405 Decryption unit 202, 302 motion restoration unit 301, 402, 407, 503, 505 selection unit 403 Image signal subtraction unit 404 coding unit 406, 504 Addition unit 408 ~ 410, 506 ~ 508 memory Cs computer system FD flexible disk FDD flexible disk drive