Method for hierarchically coding video images

The present invention relates to methods and devices for hierarchically encoding and decoding video images in different non-proportional formats. The present invention particularly relates to pictures having a common video portion. For example, the present invention relates to encoding digital television signals into SD and HD formats. Here, SD is an acronym for Standard Definition and HD is an acronym for High Definition.

Video coders with spatial scalability are known. The data flow generated by this video coder has a scalable hierarchy, and the encoded data is hierarchically incorporated into the flow by spatial scalability. The video formats relevant to these coder are video formats in which the high resolution size is twice the low resolution size, allowing binomial decomposition. Therefore, by subsampling and filtering high resolution pictures, a coding method compatible with the QCIF format of size 176x144 and the CIF format of size 352x288, or the CIF format and size 704x576. A coding method compatible with the 4CIF format of is obtained. Here, QCIF is an acronym for Quarter Common Intermediate Format.

With hierarchical coding, it is possible to obtain a base layer related to a low resolution format and an upper layer corresponding to a high resolution format. Additional data related to the upper layer is generally calculated according to the following method. -Encode a low resolution picture and locally decode this picture to get a reconstructed picture. -Scale or zoom the reconstructed low resolution picture, for example by interpolation and filtering, to obtain a picture in high resolution format. -Pixel-by-pixel difference in brightness value between the source picture and the reconstructed picture or the predicted picture based on this reconstructed picture is obtained, and the residuals constituting the data of the upper layer are obtained.

Thus, coding high resolution pictures uses scaled low resolution pictures as predictive pictures.

This coding is only valid if the picture can be overlaid. If the resolution or format of the picture is not proportional, the coding cost of the high resolution picture will not be optimized. Unless the image in the high resolution format is associated with the low resolution format by the binomial transform, some of the pictures in the high resolution format correspond to the video zones outside the zone encoded by the low resolution picture. Therefore, this picture is not adopted as a predictive picture. Therefore, coding the entire high resolution picture can be costly, especially if the high resolution picture has a large video zone that is not in common with the low resolution picture. Moreover, the standard coding mode used cannot optimize the coding cost of high resolution pictures.

An object of the present invention is to overcome the above-mentioned disadvantages. One object of the present invention is to provide a flow of encoded data by a method of hierarchically encoding video images in different non-proportional formats, wherein the first picture (F1). Is format F1, the second picture (F2) is format F2 with a lower resolution than the first format F1, and the video content of images (F1) and (F2) has at least one common part. There is. The method of the invention performs the following steps:-encoding the picture (F2) to provide encoded data for the base layer of the encoded data flow. -Zoom the picture (F2) to get a low resolution zoom picture (Fz) that is the same size as the picture (F1) for the common video part. -Encode a high resolution picture (F1) and use at least one of the following modes to feed the encoded data to the upper layers of the encoded data flow. -Inter-layer predictive coding that finds the predictive block and the motion vector that identifies this block in the preceding low-resolution zoom picture. -Inter-layer predictive coding to find the predictive block and the motion vector that identifies this block in the current low resolution zoom picture.

According to one embodiment, the method is characterized by high resolution achieved by the following steps:-to obtain a residual picture for the predicted picture (Fz), a common picture zone of the video content. Encode. -Decrypt this zone to provide a reconstructed common zone. -At least use the interspatial coding mode to code the non-common parts. In the interspatial coding mode, the reconstructed common mode is used as a prediction picture to obtain a prediction block.

According to one embodiment, the picture (F2) used for zooming is a reconstructed picture obtained by locally decoding the encoded picture (F2).

According to one embodiment, inter-layer predictive coding includes a submode, in which the predictive macroblock is a macroblock that is co-located with the macroblock to be encoded.

According to one embodiment, zooming consists of picture oversampling and filtering.

According to one embodiment, the video image is a subband picture obtained by wavelet time resolution or subband coding of the source picture.

The present invention also relates to a procedure for decoding a flow of digital data structured into at least one base layer consisting of data related to low resolution pictures and a higher layer consisting of data related to high resolution pictures. There is. Here, the low-resolution picture corresponds to at least one video portion of the high-resolution picture. The procedure according to the invention has the following steps:-Extracting data from the base layer associated with the low resolution picture and decoding the low resolution picture. -Zoom the decoded picture to get the zoom picture (Fz). -Extract data from the upper layer, decrypt it, and perform at least one of the following decryption modes: -Inter-layer prediction mode with blocks indicated by motion vectors in preceding low resolution zoom pictures that have been decoded and zoomed. -Inter-layer prediction mode with blocks indicated by motion vectors in the current low resolution picture that has been decoded and zoomed.

According to one embodiment, the upper layer corresponds to the coded data corresponding to the residual picture for the predicted picture (Fz) of the part common to the low resolution picture and the coding corresponding to the edge image related to the non-common part. It is composed of data, and while decoding a high resolution picture, the common part of the low resolution picture is first decoded from the residual picture, and then the decoded common part determined by the motion vector. Zoom pictures and edge images are decoded by performing at least one interspatial mode with blocks.

Thanks to these new modes of encoding high resolution pictures, the compression ratio is improved. Higher image quality can be obtained even if the coding costs of these high-resolution pictures are the same.

Other specific features and benefits will become apparent from the description below. The following description is provided as a non-limiting example and refers to the accompanying drawings. Of these drawings, -Fig. 1 shows a flow chart of the coding method, -Fig. 2 shows the format to be coded, and-Fig. 3 illustrates these coding modes.

The data encoding method is a hierarchical coding method, that is, the flow of encoded data is hierarchically structured, and the data related to the low resolution format is incorporated into the base layer or lower layer, and the high resolution picture. Additional data related to the format of is incorporated in the upper layer. By selecting only the layers that correspond to the required resolution, it is possible to easily select only the data related to the standard or format at the level of the data flow. This involves spatial scalability that is compatible with any time scalability required by standards for resolution formats.

The present invention relates to encoding video content in a non-proportional format having a common video portion and different widths and / or heights. One format has a lower resolution than the other format. This format has a low resolution with a small number of pixels per line or lines defining a common video portion, or a small size if the resolutions are the same.

FIG. 1 shows a flowchart of the coding method according to the present invention.

The first step 1 considers several different formats to be encoded. The coded data flow obtained on the output side of the coder is fed to a decoder compatible with these formats. The choice of format depends on the display device, decoder, or parameters. Here, this parameter is, for example, the transmission speed generated by filtering the data of this coded data flow upstream of the decoder or in the decoder.

In the above example, the first high resolution format F1 and the second low resolution format F2 are used. Each of these formats has a width of L_F1, L_F2That is, the number of pixels on the line and the height H_F1, H_F2That is, it is determined by the number of lines.

The source video supplied to the coder is considered to be in F1 and F2 formats. These source videos correspond to the same video content, but only for some. Only partly means that the video content of these two sources is assumed to be different, that is, mere scaling cannot move one format to the other, in other words. This means that these formats are not proportional.

Geometric parameters that define the video portion common to the two formats are also sent to the coder.

The generation of these source pictures and the calculation of geometric parameters can be performed as follows: From the selected format, the size of the first and second video windows is determined and encoded in each format. These video windows are positioned on top of the original picture to determine the video content of this picture to be. By assumption, these two windows overlap at least partially. These video windows define the video content to be encoded in format F1 and format F2. The size of the first and second windows is selected according to the formats F1 and F2, respectively.

High-resolution and low-resolution source pictures sent to the coder may have the same resolution as the original picture, depending on whether they have been sampled and filtered, or the resolution of the original picture. It may have a different resolution than, or it may be somewhere in between. The term "resolution" here means the number of pixels or lines per line for a given video content. The high resolution picture called (F1) is selected as the reference picture when determining the geometric parameters. Geometric parameters include, for example, the position of the picture (F2) in format F2 within the picture (F1) and the picture (F2) to match the video content of (F2) with the video content (F1) of the intersection. Includes the definition relationship corresponding to the zoom applied to.

Step 2 encodes the video picture into F2 format.

Step 3 decodes this encoded picture and provides a locally decoded picture or a reconstructed picture. In the simplified version, the locally decoded picture may consist of the original unencoded picture, i.e. the source picture (F2). This picture is then scaled or zoomed by the ratio corresponding to the geometric parameter related to the resolution ratio, resulting in a zoomed picture (F2) called (Fz).

In the next step 4, the position of the picture (F2) on the high resolution picture (F1) is determined according to the geometric parameters for this position to match the video content.

The next step 5 is to encode the high resolution picture. This coding takes into account the different coding modes that are the subject of the present invention. These different coding modes will be described later.

Step 6 inserts the coded data related to the picture (F2) into the base layer of the data flow and the coded data specific to the high resolution picture (F1) into the upper layer.

In Figure 2, reference number 11 is the high resolution coding format F1 and reference number 12 is the size L._w, H_wOne window is shown. This window is defined based on geometric parameters and is positioned within a high resolution picture. The video content of this window is size L with reference number 13._F2, H_F2Calculated from the video content in the low resolution coding format F2. A picture in format F2 is encoded and later decoded to give a locally decoded picture. The locally decoded picture is then oversampled to give a picture (Fz) scaled or zoomed to the size of window 12. FIG. 3 shows different modes for encoding high resolution pictures (F1) according to the present invention. The oversampled picture (Fz) is numbered 23, the window positioned within the high resolution picture is numbered 22 and the picture (F1) is numbered 21.

In this example, the window is positioned in the center of the picture (F1), resulting in the right strip 24 and the left strip 25 as uncovered or uncommon zones in the picture (F1). The present invention adds a new coding mode to the already known modes for coding high resolution pictures.

Of the known modes, the first coding mode, called intra-predictive coding, uses one of the previously encoded macroblocks. The current macroblock of a high-definition picture is encoded taking into account one of the adjacent macroblocks of the current macroblock, that is, the macroblock of the current picture to the left or above the current macroblock. This selection is made according to the degree of correlation with the current macroblock to be encoded.

Another similarly known coding, called inter-predictive coding, uses previously encoded high-resolution pictures. The current macroblock of the high resolution picture is encoded from the predicted macroblock, which is the picture block selected from the search window of the preceding high resolution picture. This selection is made according to the degree of correlation with the current macroblock to be encoded, i.e., the selected picture block defined by the motion vector.

A new coding mode proposed for high-resolution pictures is described below: A coding mode called inter-layer predictive coding is a low-resolution picture scaled at the time level to accommodate the high-resolution picture to be encoded. To use. This coding mode utilizes the coding of the information belonging to the base layer for the coding of the information belonging to the upper layer. Predictive blocks are searched for in the picture (Fz), i.e. in the reconstructed picture of the scaled or zoomed format F2. The selection of blocks in the picture (Fz) currently depends on the degree of correlation with the macroblock. Positional information is calculated to identify the selected picture block and transmitted in the form of motion vectors in the data flow.

One variant of the inter-layer predictive coding mode, the other mode uses scaled leading low-resolution pictures. The predictive block is searched for in the preceding picture (Fz), that is, in the scaled preceding reconstruction picture (F2). The selection of blocks in the picture (Fz) currently depends on the degree of correlation with the macroblock. The block is identified based on the motion vector.

A variant of the above mode, the "default" mode, uses the picture (Fz) macroblock found at the same position as the current macroblock to be encoded. In this case, which is only concerned with encoding macroblocks of high resolution pictures in common with low resolution pictures, no motion vector is transmitted. This mode avoids coding the zero motion vector, resulting in low coding costs. A coding mode, known as interspatial mode, applies to the specific structuring of high-resolution picture coding that takes place in two successive phases. The coding of the high-resolution picture is not performed sequentially for each macroblock according to the scanning of the television method, but in the first phase, the zone of the high-resolution picture common to Fz is processed, and in the second phase, the zone of the high-resolution picture is processed. , The processing of the video part that is not common with Fz is performed. Geometric parameters may specify high-resolution picture portions that are common to low-resolution pictures in terms of video content. This portion of the picture is encoded during the first phase by using the reconstructed picture (F2) or the zoomed source picture (F2) as the predictive picture. In this way, a residual picture or a high frequency picture is obtained. The second phase encodes a non-common part or edge image. The present invention proposes a specific coding mode called an interspatial mode from among various coding modes with respect to this coding operation. This coding mode uses the common zone of the picture as the predictive picture and identifies the position of the predictive block in the common zone using a motion vector to code the macroblock of the zone that is not common to the low resolution picture. It consists of becoming. The predicted picture may be reconstructed from a zoom picture or a residual picture, or may be derived from a high-definition source picture by selecting only a part common to this picture.

In step 4 of the coding method, the picture (Fz) is positioned on picture F1. Thus, this method encodes this common picture portion, for example, by calculating the residuals with the picture (Fz) as the predicted picture. The residual picture is then combined with the zoomed reconstruction picture (F2) to give a recombined picture that can be used as a prediction for coding parts that are not in common with the high definition picture.

Referring to FIG. 3, the interspatial coding mode utilizes a common central zone 22 as a predictive picture for edge coding. The common central zone 22 is reconstructed from the zoomed low-resolution picture 23 and the residual picture corresponding to the common zone. The picture blocks in this central zone 22 obtained by the correlation calculation are identified by the motion vector 26 in FIG.

The inter-layer coding mode uses the zoom picture 23 as the predictive picture. The picture block in the zoom picture 23 obtained by the correlation calculation is specified by the motion vector 27 in FIG.

The present invention also relates to decoding a source picture encoded according to the method described above.

Therefore, decoding takes the data of the base layer related to the low resolution picture, decodes this low resolution picture, zooms the decoded picture to get a zoom picture (Fz), and of the following decoding modes: It consists of extracting and decoding the data of the upper layer by performing at least one of.

The decoding modes are an inter-layer prediction mode using a block in a preceding low-resolution picture that has been decoded and zoomed and a prediction block, and a current low resolution that has been decoded and zoomed as a prediction block. Inter-layer prediction mode that uses blocks in the picture.

If the upper layer is composed of a residual picture related to the predicted picture (Fz) related to the part common to the low resolution picture and an edge image related to the part not common, the decoding of the high resolution picture is It takes place in two phases. First, by using a zoomed low-resolution picture as a predictive picture, the intersection of the high-definition picture is calculated from the residual picture, which is the first phase. Next, the picture block is restored from the intersection of the high-definition picture and the block identified by the motion vector, and the restored picture block is predicted for decoding the macroblock of the edge image encoded in the interspatial mode. By performing at least one interspatial decoding mode consisting of using it as a block, uncommon parts or edge images are decoded.

The coding procedure performed for coding the base layer and the upper layer may be a hybrid method using spatial correlation by discrete cosine transform or spatial wavelet transform and time correlation by inter-picture coding. Alternatively, a subband method using subband coding or wavelet coding may be used.

The coding method has been described above for video source pictures. However, it is also possible to encode preprocessed source pictures, such as subband pictures, without departing from the scope of the invention. This method is applicable to both low-frequency and high-frequency subband pictures obtained by spatial filtering or spatiotemporal filtering. Since the same subband decomposition is applied to the low resolution picture and the high resolution picture, the equivalent subbands of low resolution and high resolution can be encoded according to the method described in the present invention. For example, low-resolution pictures and low-frequency subbands of high-resolution pictures can be encoded in this way. This also holds for all other spatial subbands.

<figref num="1">The flowchart of the coding method is shown.</figref><figref num="2">Indicates the format to be encoded.</figref><figref num="3">Indicates the coding mode.</figref>