Method for coding moving picture

Method for encoding a film

Background of the Invention Field of the Invention

The present invention relates to a film coding system, and more particularly to a film coding system method to increase the efficiency of coding using a long-term reference image.

Description of the relevant technique

It is desirable to detect a change in a scene in a sequence, in order to optimally compress and encode a sequence of moving images. This is because many video applications, such as in news, sports broadcasts, a close-up conversation such as an interview, and video conferences between multiple points, repeatedly contain scene changes. Such a change of scene can take place in an entire film or in a part of the film.

The method of digitally coding the image may change as soon as a change of scene is detected. For example, because there is little similarity between an image in which a scene changes and an image in a previous scene, an image in which a scene changes is encoded by the intram mode in which an image is encoded using prediction of decoded samples within the same image instead of the inter-mode in which an image is encoded by motion compensation of previously decoded reference images.

In more detail, an image in which a change of scene takes place in the entire image, which is encoded in all blocks in the intro mode. Meanwhile, in the case of an image in which a change of scene takes place in any area, all blocks within the areas in which the change of scene takes place are coded in the intro mode. Because such an intro mode generates more bits than the inter-mode, it gives a sequence in which a change of scene

<img file="NL1022331C2_D0001.tif" />

often a fatal problem occurs in an application with a low bit rate.

Generally when a B image is used in a film coding system, the order of coding is different from the order of display.

Figure 1 illustrates a display sequence in which each image is displayed when two B images are used. As shown in Figure 1, an intra-image I is first displayed between the images to be displayed.

Next, two B images B1 and B2 are displayed after the intra-image I. A P image P3 is displayed after the B images are displayed. As described above, the following steps are performed. In other words, the fourth and fifth B images B4 and B5 are displayed after the P image P3 is displayed. A P image P6 is then displayed.

However, the order of encoding a digital image is not the same as the order of display. In other words, the P image is coded for the B image.

Figure 2 illustrates an encoding sequence in which each image is displayed when two B images are used. As shown in Fig. 2, when an intra-image I is coded, the P image P3 is coded before the two B images B1 and B2, which are displayed for the P image P3. Then P6, B4, B5, P9, B7, B8, P12, B10 and B11 are successively coded.

Here, the B images have five modes, such as intro mode, forward mode, backward mode, bi-predictive mode, and direct mode. The bi-predictive mode has two reference images. Two reference images are all placed before or after the B image or one of them is placed before the B image and the other is placed after the B image.

In particular, the direct mode uses time redundancy to maintain motion continuity between two consecutive images. In other words, in the direct mode 35, the forward motion vector and the backward motion vector of the direct mode in the B image are derived from the motion vector of a superimposed block in the next image placed immediately after the B image. A

102233 Such a direct mode does not require overhead bits such as motion information, so that a bit rate can be reduced.

Here, the forward motion vector MVf and the reverse motion vector MVb of the conventional direct mode are obtained by scaling the motion vector MV using time spacing between the images when the superimposed block has a motion vector MV in a subsequent image. In other words, the forward motion vector MVf and the backward motion vector MVb are determined by using the following equations 1 and 2.

Equation 1:

Wf = TRB * MV

TRd

Equation 2:

MVb = (TRb - TRd) * MV

TRd

Where MV is the motion vector of the assembled block in the next image, MVf is the forward motion vector of the direct mode for a B image, MVb is the backward motion vector of the direct mode for the B image, TRd is a distance in time between the next image and a reference image referred to by the motion vector of the assembled block in the next image, and TRb is a time distance between a B image and a reference image which is indicated by the motion vector of the assembled block in the next image.

As a result, the direct mode is an encoding mode to obtain two motion compensated blocks using two motion vectors MVf and MVb, and a predictive block is obtained by averaging or by interpolating calculation of two motion compensated blocks.

1022331 ·

Summary of the invention

The present invention is directed to a coding method for a film that substantially overcomes one or more problems due to the limitations and disadvantages of the particular art.

Accordingly, an object of the present invention is to provide a coding method for a moving image that is capable of increasing coding efficiency by the direct mode using a long-term reference image as a B image.

Another object of the present invention is to provide a moving image coding method capable of reducing the amount of bits when using the inter-mode for the image in which a change of scene occurs.

Additional advantages, objectives and features of the present invention will be explained in part in the description which follows and in part will become apparent to those skilled in the art after studying the following or learning of the practice of the invention. The objects and other advantages of the present invention can be achieved and achieved by the structure which is particularly indicated in the written description and the claims thereof, as well as in the accompanying drawings.

To achieve these objectives and other advantages and in accordance with the object of the present invention, as embodied and described herein in broad terms, a method for determining direct mode motion vectors in a B image includes the step of: when encoding each block in the B image using the direct mode, determining different motion vectors of the direct mode for the B image according to a kind of a reference buffer which is a reference image pointed to by a motion vector of a superimposed block in a save the specified image.

Desirably, the specified image is one of the short-term reference images used in

102233HÉ encoding the B image.

The type of reference image is determined by using a reference image index that has previously been calculated at a merged block in the specified image.

The reference image index is stored in a system buffer.

When a motion vector calculated at a assembled block in the specified image points to a long-term reference image, a direct mode forward motion vector for the B image is a motion vector of the assembled block in the specified image, and a backward motion vector of the direct mode for the B image is set to zero.

The motion vector calculated at the assembled block in the specified image is stored in a system buffer.

When a motion vector calculated at the assembled block in the specified image points to a short-term reference image, direct mode motion vectors 20 for the B image are determined by scaling the motion vector of the assembled block in the specified image by scaling time distance between images.

The motion vector calculated at the assembled block in the specified image is stored in a system buffer.

In another aspect of the present invention, a method for determining motion vectors of the direct mode in a B image includes the step of: when each block in the B image is encoded using the direct mode, differently determining motion vectors of the direct mode for the B image according to a kind of reference buffer that stores a specified image.

The reference buffer comprises a long-term reference buffer and a short-term reference buffer.

Desirably, the specified image is one of a short-term reference image and a long-term reference image.

1022331 "«

When the specified image is in the long-term reference buffer, a direct mode forward motion vector for the B image is a motion vector of the superimposed block in the specified image, and a direct mode backward motion vector for the B image is determined at zero.

The motion vector that is calculated at the merged block in the specified image is stored in a system buffer.

When a motion vector calculated at the combined block in the specified image points to a short-term reference image, direct mode motion vectors for the B image are determined by scaling the motion vectors of the combined block in the specified image by time spacing between images.

The motion vector calculated at the merged block in the specified image is stored in a system buffer.

In another aspect of the present invention, a method for determining direct mode motion vectors in a B image includes the step of: when each block in the B image is encoded using the direct mode, differently determining motion vectors from the direct mode of the B image according to a kind of reference buffer in which a specified image is stored.

The reference buffer contains a long-term reference buffer and a short-term reference buffer.

Desirably, when each block in the B image is encoded using the direct mode, the specified image differently determines motion vectors from the direct mode of the B image according to a kind of reference buffer in which a specified image is stored.

The reference buffer contains a long-term reference buffer and a short-term reference buffer.

It is desirable that the specified image is one of a short-term reference image and a long one

102233Μ term reference image.

When the specified image is in the long-term reference buffer, a direct mode forward motion vector for the B image is a motion vector of the assembled block in the specified image, and a direct mode backward motion vector for the B image is determined at zero.

When the specified image is in the short-term reference buffer, direct mode motion vectors for the B image are determined differently according to the type of reference buffer that stores a reference image referred to by motion vector of the merged block in the specified image.

The type of reference image is determined by the use of a reference image index previously calculated at the combined block in the specified image.

The reference image index is stored in a system buffer.

When a motion vector calculated at the merged block in the specified image points to a long-term reference image, a direct mode forward vector of the B image is a motion vector of the combined block in the specified image, and a backward motion vector of the direct mode of the B image is set to zero.

The motion vector calculated at the merged block in the specified image is stored in a system buffer.

When a motion vector calculated at the combined block in the specified image points to a short-term reference image, direct mode motion vectors for the B image are determined by scaling the motion vector of the combined block in the specified image by time spacing between images.

The motion vector calculated at the merged block in the specified image is stored in a system buffer.

Tomsiw

In another aspect of the present invention, a method for encoding a P image of a moving image in intermode includes the steps of: (a) determining whether a scene change occurs in the P image; and (b) if a change of scene occurs in the P image, encoding the P image with reference to the long-term reference image.

It is desirable that the P image in which a change of scene occurs is one of a scene change and an image of in part a scene change.

If the P image in which a scene change occurs is an image of in part a scene change, the blocks in an area in which a scene change occurs are encoded using a long-term reference image.

A long-term reference buffer storing the long-term reference image is a buffer for storing an image encoded for a predetermined time.

If the P image in which a change in scene occurs is an image of in part a scene change, the blocks in an area in which a scene change does not occur are coded using a short-term reference image.

A short-term reference buffer that stores the short-term reference image is a buffer for storing an image encoded after a predetermined time.

In another aspect of the present invention, a method for encoding a sequence of moving images in a movie encoding system includes the steps of; (a) determining whether a change of scene occurs in a P image; (b) if there is a P image in which a change of scene occurs, coding the P image in inter-mode with reference to a long-term reference image;

(c) during encoding, each block in a P image using direct mode according to coding sequence, determining a kind of a reference buffer that stores a specified image; and (d) calculating motion vectors of the direct mode for the B image according to the type of reference buffer and coding the B image in the direct mode.

The motion vector calculated at the merged block in the specified image is stored in a system buffer.

When the specified image is in the long-term reference buffer in step (d), a direct mode forward motion vector for the B image is a motion vector of the combined block in the specified image, and a direct mode backward motion vector for the B image is set to zero.

When the specified image is in the short-term reference buffer in step (d), direct mode motion vectors for the B image are determined differently according to the type of reference buffer in which a reference image pointed to by the motion vector of the merged block in the specified image.

The type of the reference image is determined using a reference image index previously calculated at a assembled block in the specified image.

The reference image index is stored in a system buffer.

When a motion vector calculated at the combined block in the specified image points to a long-term reference image, a direct mode forward vector for the B image is a motion vector of the combined block in the specified image, and a backward motion vector of the direct mode for the B image is set to zero.

The motion vector calculated at the assembled block in the specified image is stored in a system buffer.

When a motion vector calculated at the merged block in the specified image points to a short-term reference image, direct mode motion vectors for the B image are determined by scaling the motion vector calculated at the merged block in the specified image by means of distance in time between images.

10223311 ·

The motion vector calculated at the merged block in the specified image is stored in a system buffer.

The P image in which a change of scene occurs is one of an image with a scene change and of an image with in part a scene change.

If the P image in which a scene change occurs is an image of in part a scene change, blocks in an area in which a scene change occurs are encoded using a long-term reference image.

A long-term reference buffer in which the long-term reference image is stored is a buffer for storing an image encoded for a predetermined time.

If the P image in which a change of scene occurs is an image of in part a scene change, the blocks in an area in which a scene change does not occur are encoded using a short-term reference image.

A short-term reference buffer that stores the short-term reference image is a buffer for storing an image encoded after a predetermined time.

A short-term reference buffer consists of a first input, first output (FIFO).

The specified image for direct mode coding in the B image is one of the reference images used in coding the B image.

It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and are intended to provide further explanation of the invention as defined in the claims.

Brief description of the drawings

The accompanying drawings, which are attached to provide a further understanding of the present invention and which form part of this application, illustrate embodiments of the present invention and, together with

10223311 · the description to explain the principle of the present invention. In the drawings:

FIG. 1 illustrates an order of display in which each image is displayed when two B images are used;

FIG. 2 illustrates an encoding sequence in which each image is displayed when two B images are used;

Figs. 3A to 3B are flow charts illustrating a method for encoding a series of moving images in a film coding system according to the preferred embodiment of the present invention;

FIG. 4 illustrates a method for encoding a series of moving images in which a change of scene occurs according to a preferred embodiment of the present invention; and

FIG. 5 illustrates a method for coding a B image in the direct mode according to the preferred embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Where possible, the same reference numerals will be used in the drawings to refer to the same or similar parts.

To begin with, before an embodiment of the present invention is described, in films in which a scene change occurs, an image in which the scene change takes place entirely is called an image with scene change and an image in which a scene change takes place in part in the image becomes an image with partly called a scene change.

Figures 3A and 3B are flow charts illustrating a method of coding a series of moving images in a film coding system according to a preferred embodiment of the present invention. With reference to figures 3A

102233 Hf and 3B, images are successively input from a series of moving images (S111).

The type of images were determined (S114). In other words, it is determined whether the input image is a P image or a B image. Here, in this embodiment of the present invention, it is assumed that an encoding with respect to an intra-image is completed in advance.

If the image is the P image, it is determined whether a scene change occurs in the P image or not (S117). Here, the scene change is determined by comparing the P image with an image (P image or B image) that was shown immediately before the P image.

As a result of determining in the step S117, if the scene has been completely changed between the P pictures, the P picture is a scene change scene. Meanwhile, if the P image is determined as an image with a scene change, an encoding is performed with respect to a long-term reference image (S120).

If the P-picture is not a picture with a scene change, it is determined whether the P-picture is the picture with in part a scene change or not (S123).

If the P-picture is an image with in part a scene change, blocks in an area where the scene changes are coded with respect to the long-term reference image by returning to step S120 (S126).

Blocks in an area where the scene does not change are encoded with reference to a short-term reference image (S129, S132).

Here, the long-term reference image is an image stored in a long-term reference buffer, and the short-term reference image is an image stored in a short-term reference buffer.

The short-term reference buffer is provided with a first input, first output (FIFO) in which an image entered first is output first, and with images previously encoded during the relatively short time, stored in the short -term

T0O831tf reference buffer.

The images that have been encoded a relatively long time before are stored in the long-term reference buffer. First images of setting up the scenes in question, ie an intra-image, the scene change image, the image of in part a scene change and the like are stored in the long-term reference buffer.

If there is no scene change image or partial scene change image in the long-term reference10 buffer, the image in which the scene change occurs may be stored there.

Therefore, as shown in Fig. 4, an intra-picture 10 which is the first picture of a scene change of a scene set A1, a first picture of a scene change of P50 of a scene set Bi and a first picture P120 of a partial scene change can be stored in the long-term reference buffer. Here scene collection is a collection of similar images. For example, think of a discussion program in which an announcer appears, a panel A appears, the announcer appears again and the panel A appears again. The scene in which the announcer first appears is scene collection A and the scene in which the panel A appears is then scene collection B. The scene that the announcer reappears is scene collection A, and the scene that panel A reappears is scene collection B.

As described above, when a scene change occurs, the P-picture is coded by inter-mode to be coded with reference to a short-term reference or long-term reference image instead of intram mode. This reduces the amount of bits and improves the efficiency of coding.

Description of steps S117 to S132 will be done with the help of Fig. 4. As shown in Fig. 4, if the P-picture P200 to be encoded now is the scene change picture 35 associated with the scene set B2, the short - do not use long-term reference images stored in the short-term reference buffer. That is because the scene change image P200 is the first image of

102233HÉ the scene set B2, and the scene set of the scene change image P200 is different from the short-term reference images such as P199, P198, P197, etc. that belong to the scene set A2. Therefore, the similarity of the scene change image P200 and the short-term reference images belonging to the scene set A2 is considerably reduced and the accurate coding cannot be obtained from such reference images.

In this case, the P-picture is coded in inter-mode with reference to the other reference pictures P50 and P120 that belong to a scene set Bi and that is the same as a scene set B2.

On the other hand, if the partial change of the scene occurs in the P-picture P250, the coding is performed differently depending on two conditions. In other words, the blocks in the area in which a partial scene collection occurs are coded in inter-mode with reference to the long-term reference images P50 and P120 stored in the long-term reference buffer. The blocks in the area in which the partial scene change does not occur are coded in inter-mode with reference to the short-term reference images P249, P248, P247, etc. stored in the short-term reference buffer.

As described above, after one P-picture is encoded, the next picture is input (S159). If the associated image is a B image, the five predictive modes (intram mode, forward mode, backward mode, bi-predictive mode, and direct mode) are tested and one of them is selected as an optimal coding mode (S135, S138). Mainly direct mode will be described in this specification.

First, one block of the B-picture is read (S141). Of course the other blocks can then be read. Then a kind of reference buffer in which a specified image is stored is examined.

The specified image is determined from the earlier images in the coding order than the B image, independent

<img file="NL1022331C2_D0002.tif" />

02233 of the order of display. In other words, the specified image is one of the reference images used to encode the B image. Therefore, the specified image can be a short-term reference image or a long-term reference image. The short-term reference images may be in the display order before or after the B image and they are stored in the short-term reference buffer. The long-term reference images are stored in the long-term reference buffer. If the specified image is a long-term reference image, the direct mode forward motion vector for the B image is a motion vector of the merged block in the specified image. The direct mode backward motion vector for the B image is set to zero (S150).

However, when the specified image is a short-term reference image, the reference image index and the motion vector calculated at the combined block in the specified image are read (S144). This reference image index and the motion vector have been previously calculated and stored in the system buffer. The reference image index determines whether the motion vector of the combined block in the specified image points to a long-term reference image (S147). As described above, the reference images are stored in the reference 25 buffer containing the short-term reference buffer and the long-term reference buffer.

If the motion vector of the assembled block in the specified image points to the long-term reference image, the B image is coded using the following expressions 3 and 4 (S150).

Expression 3

MVf = MV where MV is a motion vector of the assembled block in the specified image, and MVf is a direct mode forward motion vector for the B image.

TOtH 31w

Expression 4 MVb = 0 where MV is a motion vector of the assembled block in the specified image, and MVb is a direct mode backward 5 vector motion image for the B image.

In other words, if the motion vector of the merged block in the specified image points to the long-term reference image, the direct mode forward vector for the B image is the motion vector of the merged block in the specified image and the backward motion vector is zero.

As shown in Figure 5, in step S150, TRd and TRb are meaningless in the conventional expressions 1 and 2 when the motion vector of the combined block in the specified image P200 points to the long-term reference image P50. In other words, because TRd and TRb is the distance in time including even the other scene set A2 between the specified image P200 that belongs to the scene set B2 and the long-term reference image P50 that belongs to the same scene set B1, the forward motion vector and the backward motion vector of direct mode cannot be calculated using such TRd and TRb.

Referring to Figure 5, a more detailed description will be given. When two B images are inserted into a series of moving images and these are encoded, the P image P200 that is in the encoding order earlier than the B1 and B2 images is first encoded. Because the P-picture P200 is a scene change image in which a scene change occurs, the P-picture P200 is coded in inter-mode from the long-term reference picture P50 stored in the long-term reference buffer. According to the order of coding, the next image to be coded is a Bi image. Because the BI image belongs to a scene set A2, most blocks are encoded in the forward mode from the short-term reference images associated with the scene set A2, or in a bi-predictive mode in which all two reference images belong to the

1022331W scene collection A2. However, intro mode, backward mode, or bi-predictive mode of the P-picture P200 belonging to the other scene set B2, and direct mode to obtain motion vectors from the direct mode of the mixed block in the P-picture P200 are unlikely to be. used as the coding mode for the blocks in the Blbeeld.

On the other hand, since not only the B2 image but also the specified image P200 used for direct mode motion vectors for the B2 image belong to the same scene set B2, the direct mode is chosen as the encoding mode for most blocks in the B2 -statue. In other words, after obtaining the motion vector of each block in the specified image P200 by intermo15 thus from the long-term reference image P50 that belongs to the same scene set B2, the direct mode motion vectors in the B2 image are calculated from the motion vector of the assembled block in the specified image P200. Because the B2 image and the specified image P200 belong to the scene set B2, the long-term reference image P50 also belongs to the scene set B1, and the similarity between the scene set Bi and the scene set B2 is very important. is large, the direct mode can be chosen as the coding mode for most blocks in the B225 image. Therefore, the coding efficiency for the B2 image is improved.

On the other hand, if the motion vector of the assembled block in the specified image points to a short-term reference image, the B-image is encoded using the conventional expressions 1 and 2.

At this time, because the short-term reference image stored in the short-term reference buffer belongs to the same scene set as the B image and another scene set does not exist between the specified image and the short-term reference image, the forward motion vector becomes and the direct mode backward motion vector determined using the conventional expressions 1 and 2 with respect to TRd and TRb that distance in the

102233W represent time.

If one block of the B-picture is coded, the next block of the B-picture is read and then coded (S156). Such a method is performed on all blocks in the B-picture. After the B image is encoded, the next image is input and encoded so that encoding of a movie is achieved (S159).

As described above, according to a coding method for a film of the present invention, the forward motion vector and the reverse motion vector of the direct mode for the B image are determined differently depending on the reference image referred to by the motion vector of the merged block in the specified image. When the B-picture is coded, the direct mode is mainly used as the coding mode to increase the entire coding efficiency.

In accordance with the method of coding the film of the present invention, the P image in which a scene change occurs is coded in inter-mode using motion compensation from a long-term reference, to reduce the amount of bits and increase the coding efficiency.

It will be apparent to those skilled in the art that various changes and changes can be made in the present invention. It is therefore intended that the present invention include the modifications and changes of this invention, provided that they are within the scope of the appended claims and their equivalents.