Method of scalable coding and decoding of video signal

FIELD OF THE INVENTION

The present invention relates generally to methods for scalable video coding and decoding, and more particularly to techniques that encode and decode improving layer quality, yielding information about the movement of the base layer.

BACKGROUND OF THE INVENTION

It is difficult to allocate high bandwidth, such as bandwidth required for television (TV) signals to digital video signals wirelessly via mobile phones or laptops, which are currently widely used or mobile televisions or portable computers that are widely used in the future. Consequently, it is necessary that the standard that must be used in video signal compression scheme for such mobile devices have higher efficiency of video compression.

Further, such mobile devices will inevitably have different inherent characteristics of processing and presentation of the video signals. Consequently, the compressed image must be preset diversely prepared to meet such characteristics, which means that for a single image source must be provided with video data having different image quality with respect to various generalized parameters, such as number of frames per second, resolution and the number of bits per pixel, thus creating a greater load will inevitably to information providers.

For this reason, the information provider prepares compressed video data having a high bit rate (bit rate), for each image source, and when the mobile device requests video data, performs a decoding process of the compressed image and encoding the decoded image in video data, which are suitable for the characteristics of the video signal processing of the mobile device requesting image and then provides the encoded video data. However, such a scheme should be accompanied by a process of re-encoding (decoding scaling + coding +), so that while ensuring image requested by the mobile device, there is a slight time delay. Further, the transcoding process also requires a complicated hardware device and algorithm dependent diversity coding purposes.

To overcome these difficulties was proposed scalable video codec (SVC). SVC is a circuit for coding video signals with high picture quality when encoding a video signal and allows to some extent guarantee the image quality, even if only decode the portion of the complete sequence of images (frames) generated as a result of the encoding (a sequence of frames intermittently selected from the total sequence) .

Scheme (MCTF) the temporal filtering with motion compensation is an example of the encoding scheme proposed for use in scalable video codec. There is a high probability that the MCTF scheme would apply to a transmission medium, such as the environment of mobile transmission which has a limited bandwidth, therefore, MCTF-scheme requires high compression efficiency, i.e. high efficiency coding to reduce the number of bits transmitted per second.

As described above, even if the received and processed only a partial sequence of the sequence of images encoded using the MCTF, which is a scalable scheme, the image quality can be guaranteed to some extent. However, if the bit rate is reduced, image quality deterioration becomes severe. To solve this problem may be provided a separate sub-image sequence for low bitrates, for example smaller screens and / or image sequence, which has a small number of frames per second.

The sequence of the sub-image is called the base layer and the main layer is called a sequence of images to improve the quality. However, because the base layer and improve the quality of encoded receive the same image content with different temporal resolutions and different frame rates, video signals in the presence of an excess of two layers of information (redundancy). Therefore, to improve the encoding efficiency improving layer quality, improve the quality of the video layer, and encoding predicted using motion information and / or texture information of the base layer. This method of encoding method is called inter-layer prediction.

The movements of the base layer used in the method of the interlayer prediction includes information reference index that points to an image (frame) including the reference block, motion vector information which indicates moving to the reference block, information on the partition of the corresponding block ( block, which is placed in the frame of the base layer temporally coincident with a frame layer improving the quality, including the macroblock to be encoded, and has a region covering the macro block when the block is increased according to the relative size of the screen layer quality improvement to the screen size of the base layer) etc.

Figure 1 is a diagram illustrating an embodiment of the traditional method of producing traffic information to improve the quality of the macroblock layer, for example, information about the partition, the reference index information, motion vector information, etc. of the base layer. 1 (a) shows an embodiment, wherein the reference index and a motion vector for a 4x4 sub-block b are extracted from the base layer.

First reference index and a motion vector for each of the four corner pixels of c1 to c4 block to be encoded may be respectively set as a reference index and a motion vector for a block of the base layer corresponding to each pixel.

However, when a block corresponding to each pixel does not exist in the base layer, as is the case when coinciding temporally frame does not exist in a base layer, or when the block corresponding to each corner pixel is encoded in the interlaminar coding mode (intra-mode ), block b may be set as an intra block.

If the block corresponding to the corner pixel, does not use a frame existing in the list of reference images List_0, frame existing in List_0, and a motion vector directed toward the frame in List_0 not specified in the block b. This equally applies to List_1.

The reference index rb (List_x) for the block b is set to the minimum value of the reference indices rci (List_x), defined for the respective corner pixels and vector mvb (List_x) Motion for a block b is set to the average value of motion vectors of the corners of pixels having the specified reference index rb (List_x).

Figure 1 (b) shows an embodiment in which the motion information of 8 × 8 block B is prepared from a 4 × 4 sub-blocks.

In the case where all the four 4 × 4 sub-blocks are inner blocks of 8 × 8 block B is set as an intra-block. In other cases, the reference index information and information on the partition of 8 × 8 block B is determined by the following process.

For relevant 4 × 4 sub-blocks of reference indexes for list List_0 List_1 and reference images are set to the same value. The description is made using List_0, by way of example, and perform the same operation for List_1.

In the case where neither the 4 × 4 sub-block uses frame in List_0, the reference index and a motion vector for List_0 not set for the 8 × 8 block B.

In other cases, the reference index rB (List_0) for 8 × 8 block B is calculated as a minimum value of the reference indices for the four 4 × 4 sub-blocks. Calculates the average motion vector mvmean (List_0) 4 × 4 sub-blocks having the calculated value of the reference index rB (List_0). Further, in a 4 × 4 sub-blocks of reference index and a motion vector for each of i) an intra-block, ii) the block not using List_0 or iii) a block having a reference index rB (List_0), forcibly sets the computed value of the reference index rB (List_0) and the calculated value of the vector mvmean (List_0) movements respectively.

After that, for the 8x8 split-block B is determined as follows. If two motion vectors of neighboring 4 × 4 sub-blocks are equal to each other, the sub-blocks are equal to each other and then combined with each other. Figure 1 (b), if the sub-blocks b1 and b2 are equal to each other, and b3 and b4 are equal to each other, the mode is determined as the partition BLK_8 × 4 mode. At this time, if the sub-blocks b1 and b3 are also equal to each other, split-mode is defined as 8 × BLK_8. Similarly, if the sub-blocks b1 and b3 are equal to each other and the b2 and b4 are equal to each other, split-mode is defined as 8 × BLK_4. In either case, the partition is defined as BLK_4 × 4 mode.

However, when the ratio of the screen size (or resolution) layer to improve the quality of the screen size is not a multiple of the base layer 2 (non-binary case), for example when the screen size of the base layer is 1/3, 2/3, etc. screen size layer quality improvement, it is difficult to receive the traffic information such as information of the reference index information, motion vector or the information on the partition of the base layer, so that the method of the interlayer prediction may not be sufficiently applied to the scalable encoding layer quality improvement .

DISCLOSURE OF INVENTION

Consequently, the present invention takes into consideration the above problems, and an object of the present invention is to provide a method that effectively uses the interlayer prediction method even for non-binary case to improve the coding efficiency.

Another object of the present invention is to provide a method that extracts relevant information about the movement of the base layer, with the result that the ratio of the screen size is set to not a multiple of 2, and the new criteria for this.

According to one aspect of the present invention to accomplish the above-described problems provides a method of encoding a video signal comprising a scalable video coding and generate a bitstream of the first layer, video coding using a predetermined method, and generating a bitstream of the second layer, wherein motion information corresponding to a first layer derived from the second layer, and the reference index for the image block in the first layer is determined using a median criterion based upon the second layer.

According to another aspect of the present invention to accomplish the above tasks provided a method of decoding an encoded bitstream comprising decoding a bitstream of the first layer that has been encoded with scaling and received, and decoding the bit stream of the second layer that has been encoded using a predetermined method and received, wherein the information a movement corresponding to a first layer derived from the second layer, and the reference index for the image block in the first layer is determined using a median criterion based upon the second layer.

According to an embodiment of the present invention, the reference index for the image block may be selected reference index for the block of the second layer corresponding to the maximum number of pixels in the image block, and a reference index for the image block may be selected closer reference index, if the same number of pixels corresponds to more than two blocks in the second layer. If the reference indices for more than two blocks in the second layer corresponding to the same number of pixels are equal to each other, may be selected motion vector having a smaller absolute value.

According to another embodiment of the present invention, as a motion vector for the image block may be selected motion vector for the block of the second layer having said defined reference index among motion vectors of two or more blocks may be selected motion vector having a shorter length if the number of blocks of the second layer having said determined reference index is two or more.

According to a further embodiment of the present invention, the reference index for each of two or more sub-blocks, constituting the image block and having the same size may be determined using a median criterion based upon the second layer and the reference index for the image block may be determined using a median criterion based on the reference index for each of the sub-blocks. If two or more different reference indices nearer reference index may be defined as a reference index for the image block.

According to another embodiment of the present invention, each sub-block constituting the image block can be arbitrarily removed from the block of the second layer corresponding to the sub-block in an intra mode or linear coding - an inter-mode. Reference index and a motion vector for the image block can be obtained from the second layer only on the basis of sub-blocks derived in an inter mode, other than sub-blocks derived in an intra mode.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention, illustrate the preferred embodiments of the invention and together with the description serve to explain the principles of the present invention.

1 is a diagram which shows the embodiment of the traditional method of obtaining information about the movement of the macroblock layer to improve the quality of the base layer;

Figure 2 is a diagram which shows the structure of a video encoding apparatus to which the method applies scalable video coding according to the present invention;

Figure 3 is a diagram which shows an embodiment of a method for producing a reference index and a motion vector for a 4 × 4 sub-blocks of the base layer according to the present invention;

Figure 4 is a diagram which shows an embodiment of a method for producing a reference index and a motion vector for the 8 × 8 block of the base layer according to the present invention;

5 is a diagram which shows an embodiment of a method for producing the mode for an 8 × 8 block of the base layer according to the present invention; and

6 is a diagram which shows the structure of an apparatus for decoding a bitstream encoded with the apparatus of Figure 2.

The features, elements, and aspects of the invention, which have the same numerals in different figures represent the same, equivalent, or similar features, elements, or aspects in accordance with one or more embodiments.

MODES FOR CARRYING OUT THE INVENTION

Here will be described embodiments of the present invention with reference to the accompanying drawings.

2 is a diagram which shows the structure of a video encoding apparatus to which the method applies scalable video coding according to the present invention.

The video encoding device of Figure 2 includes an encoder layer 100 enhance the quality (EL) for scalable encoding the incoming video signal in the macroblock by using, for example, circuitry (MCTF) the temporal filtering with motion compensation and generation of the corresponding control information unit 110 for encoding texture converting data for each encoded macroblock in the compressed bit stream, the control unit 120 coding the motion for encoding the motion vectors of image blocks obtained by the EL-encoder 100 into a compressed bitstream using a predetermined method, the encoder 140 of the base layer (BL) for encoding an incoming video signal, using a predetermined method, such as MPEG 1, 2 or 4, or H.261 or H.264, and generating a sequence of thumbnail screens, for example a sequence of images having a size which is 25% of the original size or 33% of the original size in the non-binary case, and a multiplexer 130 to encapsulate the output of the texture coding unit 110, a sequence of reduced screens encoder 140 of the base layer (BL) and the output coding unit 120 motion multiplexing format data encapsulated in a predetermined transmission format and outputting the data in the transmission format.

-EL encoder 100 performs prediction by subtracting the reference block obtained by motion estimation of an arbitrary macroblock in a video frame (or picture), and may perform an update operation by adding a selective support block difference image between the macroblock and the reference block.

EL-coder 100 may divide the input video sequence into frames which have a difference image, and other images (or frames to which the difference image is added selectively) for example odd-numbered frames and even-numbered frames, and can perform the operation of forecasting and / or an update operation on a number of levels of temporal decomposition, for example up to a level of temporal decomposition on which with respect to one group of pictures (GOP) generating a high-frequency frame (frame generated by the operation prediction, 'H'-frame) and one low-frequency frame (sequence generated by the update operation, 'L'-frame).

EL-coder 100 performs a process of dividing a frame, which is a difference image of the input video frames or the low frequency frames obtained at the previous level of temporal decomposition into macroblocks, each macroblock having a predetermined size, the detection unit having an image most similar to the image of each macroblock, division obtained in the preceding and subsequent frames or into its own frame, generating a predicted image and obtaining motion vectors. EL-100 encoder performs the above process for all macroblocks in the frame, thereby completing a high-frame which is the predicted image for the respective frame.

Alternatively, EL-encoder 100 can detect the reference block for the macroblock in the base layer frame using a prediction method in BL-layer. In this case, the EL-encoder 100 can detect a corresponding block encoded in an intra mode, to coincide in time against the frame of the base layer generated BL-encoder 140 (corresponding to the block is placed in the base layer frame, coinciding temporally with the frame having the macroblock and has a region covering the current macroblock when the block size is increased according to the ratio of the screen layer to improve the quality of the screen size of the base layer), thereby generating the predicted image.

Further, as described above, EL-encoder 100 may not only encode the macroblock layer quality improvement by using the image data (texture) of the base layer using a prediction method in BL-layer, but may also encode a macroblock layer quality improvement based on the motion information, such as a mode, a reference index or motion vector for the corresponding block of the base layer.

Additionally, EL-coder 100 performs a refresh operation for each macroblock in the frame, to which is added to the difference image of the input video frame or the low frequency frames obtained at the previous level temporal decomposition by addition of a macroblock corresponding region in the encoded frame to obtain difference images using prediction operation performed on the basis of a part or the entire region of the macroblock, if necessary. In this case, the motion information of the macroblock to be updated is obtained from the corresponding block of the base layer so that a macroblock may be updated based on the motion information. EL-coder 100 performs the above procedure for all macroblocks of the frame, thus completing a low frequency frame corresponding to the frame.

Details of the method for producing the base layer motion information, such as information about the partition, the reference index information or motion vector information of the macroblock layer layer improving the quality of the base layer during the process of prediction and / or update process according to the present invention.

Embodiment of a method for producing a reference index and a motion vector for each 4 × 4 sub-block of the base layer has been described with reference to Figure 3. To determine the reference index using the median test.

Determine block of the base layer corresponding to each pixel in the 4 × 4 sub-block layer to improve quality. The reference index for the 4 × 4 sub-block layer quality improvement is selected reference index for the block of the base layer corresponding to the maximum number of pixels. In this case, if the same number of pixels corresponds to two blocks of the base layer is chosen nearer reference index. If the reference indices for the two blocks are equal to each other, may be selected motion vector having a smaller absolute value.

As the motion vector selected motion vector for the block of the base layer, which has the selected reference index. If the number of blocks of the base layer that have the selected reference index is two or more, can be selected by a motion vector that has a shorter length among motion vectors of two or more blocks.

Embodiment of a method for producing a reference index and a motion vector for the 8 × 8 block has been described with reference to Figure 4. Similarly, the average used as a criterion for determining a reference index.

First, as shown in Figure 4 (a) to obtain the reference indices and motion vectors for the four 4 × 4 sub-blocks according to the above embodiment of the present invention. The optimal reference index and the optimal motion vector can be obtained from the reference indices and motion vectors for the four sub-blocks.

As another method, the optimal reference index may be selected from the reference index for the blocks of the base layer corresponding to the 8 × 8 block, instead of selecting the possible reference index and a motion vector for the 8 × 8 block of 4 × 4 sub-blocks, for which reference codes and the motion vector is obtained earlier. Similar to the case of 4 × 4 sub-blocks of a reference index which is most frequently used, may be selected from using reference indices.

For example, as shown in Figure 4 (b), for each pixel belonging to the 8 × 8 block, a reference index for the block of the base layer corresponding to the pixel, is associated with a reference index for the corresponding pixel and a reference index associated with the maximum number of pixels, It may be selected as the reference index for the 8 × 8 block.

In this case, if 8 × 8 block to select two or more different reference index as the reference index for the 8 × 8 block is determined closer reference index.

Meanwhile, when the mode for the 8 × 8 block is obtained from the base layer, in the 8 × 8 block coexist intra mode NCHN sub-blocks and the inter-mode NCHN sub-blocks. Therefore, in the present invention, as shown in Figure 5, can also be prepared by treatment of 8 × 8 block consisting of 4 × 4 intra-mode and the 8 × 4 inter mode. In this case, the combination of blocks can be used conventional methods.

If the mode is the 8 × 8 block is obtained and determined, only one reference index is selected based only on sub-blocks of the inter-mode, rather than the 4 × 4 sub-blocks derived in an intra mode, an 8 × 8 block, according to the above method.

The data stream encoded by the above described method, may be transmitted to a decoding apparatus in a wired or wireless manner or can be transferred via the recording medium. The decoding apparatus reconstructs the original video signal in the manner described below.

6 is a diagram of an apparatus for decoding a bitstream encoded with the apparatus of Figure 2. The decoding apparatus of Figure 6 includes a demultiplexer 200 for separating the received bit stream to a compressed stream of the compressed motion vectors and macroblock information stream, a texture decoding unit 210 to restore the compressed data stream into the original macroblocks decompressed stream, motion decoding module 220 to recover the compressed stream motion vector in the original decompressed stream, the decoder 230 layers improve the quality (EL) for performing an inverse transform on the extracted flow information macroblock and the decompressed stream of motion vectors, using, for example, MCTF, and obtain the original video signal, and the decoder 240 of the base layer (BL) decoding the base layer stream by using a predefined method, for example, MCTF, MPEG-4 or H.264.

EL-decoder 230 performs an inverse update operation macroblock in 'L'-frame that has been encoded and decoded or was introduced and generated at a previous temporal level to determine whether the region' H'-frame, which was predicted on the basis of part or and the entire region of the macroblock has been encoded as a difference image, with reference to the reference index and motion vector and subtract the difference image to a certain area of ​​the image value of the macroblock, if necessary. EL-decoder 230 performs the above operation on all macroblocks in the 'L'-frame, thereby completing the' L'-frame on corresponding temporal level.

In this case, information of the reference index and motion vector information may be provided by the decoding unit 220 or the motion vector can be derived from the base layer based on the information or data provided by BL-decoder 240, using the method described with reference to Figures 3 and 4 .

Further, the shape of each sub-block of the macroblock layer, and improve the quality of a motion vector for each sub-block can be prepared based on the information about the partition recorded in the macroblock header. Alternatively, the shape of each sub-block and the macroblock mode and the motion vector for each sub-block can be obtained from the base layer using the method described with reference to Figure 5.

Additionally, EL-decoder 230 performs an inverse operation prediction for each macroblock in the 'H'-frame at an arbitrary temporal level, which has been encoded and input, to define a support area in the' L'-frame at the next temporal level, which has been generated by inverse update operation, with reference to the reference index, information about the partition and motion vector that provides the decoding unit 220 or the motion vector are extracted from the base layer, and to add the value of the reference region image to a difference image macroblock. EL-decoder 230 performs the above operation on all macroblocks in the 'H'-frame, thereby completing the' L'-frame at the next temporal level. Completed 'L'-back frame is updated, so back the updated' L'-frame and concluded 'L'-frame made alternatively.

In the above method coded data stream is reduced in a complete sequence of video frames. In particular, when the operation of the prediction and the update operation is performed N times (N levels of temporal decomposition) in respect of one GOP for the described coding process by using the method MCTF as an example, the image corresponding to the image quality source video signal may be obtained if the inverse update operation and an inverse prediction operation is performed N times during the process of MCTF-decoding. If the inverse update operation and an inverse prediction operation is performed less than N times, the image quality can be slightly degraded, but it can be obtained video sequence, which has a lower bit rate. Consequently, the decoding apparatus is designed to perform inverse update operation and the prediction inverse operation to the extent that is necessary for their implementation.

The above-described decoding apparatus may be installed in the mobile communication terminal or an apparatus for reproducing the recording medium.

Consequently, the present invention can receive information on the partition, the reference index information or motion vector information from the base layer, in particular, may receive traffic information, etc. of the base layer, with the result that the aspect ratio of the screen layer to improve the quality of the screen size of the base layer has a value that is not a multiple of two, thereby improving coding efficiency.

Although a preferred embodiment of the present invention have been disclosed for illustrative purposes, those skilled in the art will appreciate that various modifications, additions and substitutions that may be made without departing from the scope of the invention as defined by the following claims.