Motion prediction method.

(54) Title: MOVEMENT PREDICTION METHOD. (54) Title: MOTION PREDICTION METHOD.

(57) Summary

A motion prediction method is provided. First, a plurality of candidate units are determined that correspond to a current frame. Then a plurality of motion vectors are obtained from the candidate units. A plurality of time scale factors of the candidate units is then calculated according to a plurality of temporal distances between reference frames of the motion vectors and the current frame. Then the motion vectors of the candidate units are scaled according to the time scale factors to obtain a plurality of scaled motion vectors. Finally, a motion vector predictor is selected for motion prediction of the current unit of the candidate units according to the scaled motion vectors.

(57) Abstract

A motion prediction method is provided. First, a plurality of candidate units corresponding to a current trame are determined. A plurality of motion vectors of the candidate units are then obtained. A plurality of temporal scaling factors of the candidate units are then calculated according to a plurality of temporal distances between reference trames of the motion vectors and the current trame. The motion vectors of the candidate units are then scaled according to the temporal scaling factors to obtain a plurality of scaled motion vectors. Finally, a motion vector predictor for motion prediction of the current unit is selected from the candidate units according to the scaled motion vectors.

MOVEMENT PREDICTION METHOD

FIELD OF THE INVENTION

The invention relates to video processing, and more particularly to prediction of video data motion.

BACKGROUND OF THE INVENTION

The new incoming H.264 compression standard can provide good video quality at substantially lower bit rates than previous standards by adopting features such as sub-pixel precision and multiple reference. The video compression process can generally be divided into 5 parts including: interprediction / intra-prediction, transorm / reverse transform, quantize / reverse quantize, loop filter, and entropy encoding. H.264 is used in different applications such as Blue-ray Discs, DVB Broadcast Services, Direct Broadcast Satellite Television Services, Cable TV Services, and Real-Time Video Conferencing.

A torrent of video data comprises a series of frames. Each frame is divided into a plurality of encoding units (eg, macro blocks or extended macro blocks) for video processing. Each encoding unit can be segmented into quad-tree partitions, and each partition is assigned with a motion parameter.

In order to reduce the cost of transmitting huge amounts of motion parameters, a Motion Vector Predictor (MVP) is calculated for each partition by referencing adjacent encoded blocks, then the encoding efficiency can be improved while the movement of the adjacent blocks tends to have a high spatial correlation.

Referring to Figure 1, a schematic diagram of a current unit 100 and a plurality of neighboring units A, B, C, and D is shown. In this example, both current unit 100 and neighboring units A, B, C ,, and D are the same size; however, these units are not necessarily to be the same size. The motion vector predictor (MVP) of the current unit 100 is predicted according to the motion vectors of neighboring units A, B, and C, or A, B, and D if C is not available. When current unit 100 is a 16 χ block and there is a motion vector from neighboring unit C, a median of the motion vectors from neighboring units A, B, and C is determined to be the MVP of the unit current 100. When the current unit 100 is a 16 bloque block and there is no motion vector from neighboring unit C, a median of the motion vectors from neighboring units A, B, and D is determined to be the MVP of the current unit 100. When current unit 100 is an 8 χ 16 partition on a left half of a 16 * 16 block, a motion vector from neighboring unit A is determined to be the MVP of current unit 100. When the current drive 100 is an 8 χ 16 partition in a right half of a 16 χ 16 block, a motion vector from neighboring drive C is determined to be the MVP of the current drive

100. When the current drive 100 is a partition of 16 ^x 8 in an upper half of a block of 16 χ 16, a motion vector of neighboring unit B is determined to be the

MVP of current unit 100. When current unit 100 is a partition of 16 χ 8 in a lower half of a block of χ 16, a motion vector of neighboring unit A is determined to be the MVP of the current unit 100.

When an MVP of a current unit is predicted according to the motion vectors of neighboring units A, B,

C, and D, the motion vectors of neighboring units A,

B, C, and D do not have an appropriate time scale. For example, the reference frame of neighboring units A, B, and C are different, and the motion vectors of neighboring units A, B, and C correspond to the reference frames respectively. The time distances between each of the reference frames and the current frame are different.

The motion vectors of neighboring units A, B, and C must therefore be scaled temporarily according to the temporal distances before predicting the MVP of the current unit 100 according to the motion vectors of neighboring units A, B, and C.

The MVP of the current unit 100 is only predicted according to the motion vectors of neighboring units A, B, C, and D. The prediction accuracy of the MVP can be further improved if more candidate MVPs are considered and selected the best candidate MVP for optimizing distortion rate. For example, we propose the motion vector competition (MVC, Motion

Vector Competition) to select the best MVP from a set of predefined candidates specified at the sequence level. The predefined candidate set may include the H.264 standard predictor (eg, the median MV of neighboring units), MV of the unit placed in the same location in a reference frame as that of the current unit, and the MVs of neighboring units. The recommended number of MVPs in the predefined candidate set is two. The set of predefined candidates, according to the motion vector competition method, is fixed at one level of video sequence.

BRIEF DESCRIPTION OF THE INVENTION

Intention provides a method of predicting movement. First, a plurality of candidate units are determined that correspond to a current unit of a current frame. Then a plurality of motion vectors are obtained from the candidate units. A plurality of time scale factors of the candidate units is then calculated according to a plurality of temporal distances between reference frames of the motion vectors and the current frame. Then the motion vectors of the candidate units are scaled according to the time scale factors to obtain a plurality of scaled motion vectors. Finally, a motion vector predictor is selected for motion prediction of the current unit of the candidate units according to the scaled motion vectors.

The invention also provides a prediction method. First, a plurality of candidate units is determined for predicting movement of a current unit. Then a plurality of encoded units corresponding to the current unit is determined. Then a plurality of motion difference values ​​are calculated between the motion vectors of the candidate units corresponding to the encoded units and the motion vectors of the encoded units. The movement difference values ​​corresponding to the candidate units are then added according to a series of weights to obtain a plurality of weighted sums respectively corresponding to the candidate units. Finally, at least one chosen candidate unit is selected for the prediction of movement of the current unit of the candidate units according to the weighted sums.

A detailed description is given in the following embodiments with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention can be more fully understood by reading the following detailed description and examples with references made to the accompanying drawings, wherein:

Figure 1 is a schematic diagram of a current encoding unit and a plurality of neighboring encoding units.

Figure 2 is a block diagram of a video encoder according to the invention.

Figure 3 is a schematic diagram of the scale of a candidate two-unit motion vector.

Figure 4 is a flow chart of a motion prediction method, time difference adjustment.

Figure 5 is a schematic diagram of a plurality of candidate units for predicting movement of a current unit in accordance with an embodiment of the invention.

Figures 6A and 6B illustrate a flow chart of a motion prediction method with adaptively chosen candidate units according to an embodiment of the invention.

Figure 7 is a schematic diagram of a table recording motion difference values ​​corresponding to different encoded units and candidate units according to an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

The following description is the best contemplated mode for carrying out the invention. This description is made for the purpose of illustrating the general principles of the invention and should not be taken in a limiting sense. The scope of the invention is best determined with reference to the appended claims.

Referring to Figure 2, a block diagram of a video encoder 200 is shown. In one embodiment, video encoder 200 comprises a motion prediction module 202, a subtraction module 204, a transform module 206, a module quantization 208, and an entropy encoding module 210. The video encoder

200 it receives a video input and generates a torrent of bits as an output. Motion prediction module 202 performs motion prediction on the video output to generate the predicted samples and prediction information. After subtraction module 204 subtracts the

<td>samples</td><td>predicted</td><td>of</td><td>the entrance</td><td>of</td><td>video</td><td>for</td><td>obtain</td>

<td>waste,</td><td>reducing</td><td>of</td><td>this way</td><td>the</td><td colspan="2">amount of</td><td>Data of</td>

<td>video of</td><td>those of the</td><td colspan="2">video input</td><td>to</td><td>those of</td><td>the</td><td>waste.</td>

<td>Then the</td><td>waste</td><td>I know</td><td colspan="3">send sequentially</td><td colspan="2">to the module</td>

transform 206 and quantization module 208. Transform module 206 performs a Discrete Cosine Transform (DCT) on the residuals to obtain the transformed residuals. Then the quantization module 208 quantifies the transformed residuals to obtain the quantized residuals. The entropy encoding module 210 then performs the entropy encoding on the quantized residuals and the prediction information to obtain a torrent of bits as a video output.

A motion prediction module 202 predicts a motion vector predictor (MVP) of a current unit in a current frame according to the motion vectors of a plurality of candidate units. In one embodiment, candidate units are neighboring units that are neighboring the current unit. Before the motion prediction module 202 predicts the MVP of the current unit, the time distances between the reference frames of the candidate units and a current frame are calculated, and the motion vectors of the candidate units are scaled according with temporal distances. With reference to the

Figure 3 shows a schematic diagram for scaling a motion vector of two candidate units 310 and 320. A current frame k comprises two candidate units 310 and 320 for predicting motion of a current unit 300. The first candidate unit 310 has a motion vector

MVi with reference to a reference frame i, and a first time distance D is calculated_ik between the reference frame i and the current frame k. The second candidate unit

320 has an MV motion vector₂ with reference to a reference frame 1, and a second time distance Di is calculated_k between reference frame 1 and current frame

k.

Then a target time distance D is calculated_jkbetween the target search frame j and the current frame k. The target search frame j is the selected reference frame. Then a first time scale factor is calculated by dividing the target time distance Dj_k between the first time distance D_ik, and the motion vector

MV_X of the first candidate unit 310 is multiplied by the first time scale factor (Dj_k/ D_ik) to get a scaled motion vector MV_X'corresponding to the first candidate unit 310. Then a second time scale factor is calculated by dividing the target time distance Dj_k between the second time distance D_Xk, and the motion vector MV₂ of the second candidate unit 320 is multiplied by the second time scale factor (Dj_k/Gave_k) to get a scaled motion vector MV₂'corresponding to the second candidate unit 320. Therefore, the scaled motion vectors MV_X'and MV₂both are measured with reference to the target search frame j, and the time distance difference factor is therefore removed from the scaled motion vectors MV_X'and MV₂'. The motion prediction module 202 can then predict the MVP of the current frame 300 according to the scaled motion vectors MV_X'and MV₂'of candidate units 310 and 320.

Referring to Figure 4, a flowchart of a motion prediction method 400 with time difference adjustment is shown. First, a plurality of candidate units is determined for predicting movement of a current unit in a current frame (step 402). The candidate units and the current unit are blocks of the same or different sizes, and each of these units can be an encoding unit, a prediction unit, or a prediction unit partition. In one embodiment, the candidate units comprise a left unit A on the left side of the current unit, a top unit B on the top side of the current unit, a top right unit C in line with the top-right address of the current unit , and a top-left unit D in line with a top-left address of the current unit. Then a plurality of motion vectors of the candidate units is obtained (step 404). A plurality of time scale factors of the candidate units is then calculated according to the time distances between the reference frames of the candidate units and the current frame (step 406). In one embodiment, a plurality of time distances are calculated first between the reference frames of the candidate units and the current frame, a target time distance is also calculated between a target search frame and the current frame, and then the target time distance divided respectively by the time distances corresponding to the candidate units to obtain a plurality of time scale factors corresponding to the candidate units, as shown in Figure 3.

The motion vectors of the candidate units are then scaled according to the time scale factors to obtain a plurality of scaled motion vectors (step 408). In one embodiment, the motion vectors of the candidate units are respectively multiplied by the timescale vectors of the candidate units to obtain the scaled motion vectors of the candidate units, as shown in Figure 3. Then a motion vector predictor of the current unit of the candidate units is selected according to the scaled motion vectors

<td>(step 410).</td><td>In a</td><td>modality, the</td><td>vectors</td><td>of</td><td>movement</td>

<td>scaled it</td><td>order</td><td>, and then:</td><td>Choose</td><td>a</td><td>vector of</td>

<td>movement</td><td>scaled</td><td>middle of the</td><td>vectors</td><td>of</td><td>movement</td>

scaled as the MVP of the current unit.

When the motion prediction module 202 determines an MVP of a current unit according to a motion vector competition method, typically only the motion vectors of two candidate units determined at a sequence level are included in the set of candidates to determine the MVP of the current unit. Furthermore, the pool of candidates is not adaptively determined according to the characteristics of the current unit. The performance of the motion prediction can be improved if the candidate set is determined adaptively according to the characteristics of the current unit.

Referring to Figure 5, a schematic diagram of a plurality of candidate units is shown for motion prediction of a current unit 512 in accordance with one embodiment of the invention. In this embodiment, the current unit 512 and the candidate units are blocks with different sizes, for example, the current unit 512 is a 16 χ 16 block and the candidate units are 4 * blocks. In another modality, the size of the current and candidate units can be the same or different, the size can be 4 χ 4, 8 χ 8, 8 χ 16, 16 χ 8, 16 χ 16, 32 χ 32, or 64 χ 64.

In this embodiment, the motion vectors of the four candidate units A, B, C, and D of the current frame 502 can be taken as candidates to determine the MVP of the current unit 512. Furthermore, the placed unit 514 has the same location in a reference frame 504 as that of the current unit 512 in the current frame 502, and the motion vectors of a plurality of candidate units a ~ j neighboring the placed unit 514 or within the placed unit 514 they can also be taken as candidates to determine the MVP of the current unit 512.

Candidate unit A in current frame 502 is a partition on the left side of current unit 512, Candidate unit B in current frame 502 is a partition on the top side of current unit 512, candidate unit

C in current frame 502 is an online partition with a top-right address of current unit 512, and candidate unit D in current frame 502 is an online partition with a top-left address of current unit 512. The candidate unit in the reference frame

504 is a partition on a left side of the placed unit 514, candidate unit b in the reference frame

504 is a partition on a top side of placed unit 514, candidate unit c in the reference frame

504 it is an inline partition with the top right address of unit 514 placed, and candidate unit d in reference frame 504 is an inline partition with a top-right address of unit 514 placed. Also, candidate unit e in reference frame 504 is a partition within placed unit 514, candidate units f and g in reference frame 504 are partitions on a right hand side of placed unit 514, candidate unit h in Reference frame 504 is an inline partition with a lower-left address of the placed unit 514, candidate unit i in the reference frame

504 it is a partition on a lower side of the placed unit 514, and the candidate unit j in the reference frame 504 is an inline partition with a lower-right address of the placed unit 514. In one embodiment, the set of candidates for determining the MVP of current unit 512 further comprises the calculated motion vectors, eg, a motion vector equal to a median of the motion vectors of candidate units A, B, and C, a motion vector equal to a median of the motion vectors of candidate units A, B, and D, and a scaled MVP derived by a method similar to Figure 4.

After a plurality of motion vectors corresponding to a current unit 512 are determined to be included in the candidate set, at least one motion vector from the candidate set is adaptively selected for motion prediction of the current unit 512. Referring to Figures 6A and 6B, a flow chart of a motion prediction method is shown.

600 with adaptively determining a set of candidates. The pool of candidates for a current unit

512 it is selected from a plurality of motion vectors corresponding to current unit 512 (step 602). Motion vectors may comprise one or a combination of partition / block motion vectors encoded in the same frame, calculated motion vectors, and motion vectors in the reference frame (s). In one modality, the set of candidates corresponding to the current unit 512 shown in the

Figure 5 comprises movement vectors of units A,

B, C, and D in current frame 502 and a motion vector of unit e in reference frame 504. The candidate set can be determined according to one or more of the above statistics, neighbor information, shape of the current unit, and position of the current unit. For example, the plurality of motion vectors corresponding to current unit 512 is ordered according to neighbor information, and the first three motion vectors are selected to be included in the candidate set. A final MVP can be selected from the candidate pool by the motion vector competition method or some other selected method. In some embodiments, the plurality of motion vectors are ordered according to a selection order, and the selection order is determined by the weighted sum of the motion differences. Motion differences are calculated between each of the motion vector predictors and corresponding decoded motion vectors (ie, actual motion vectors) of the candidate units. Weights can be determined by the shape and position of the current unit, or weights can be determined by the shape and position of neighboring blocks.

Referring to Figure 7, a schematic diagram of a table is shown that records the motion difference values ​​corresponding to different encoded units and candidate units according to the invention. For example, unit A is considered to be selected to be the target encoded unit. A motion difference value D is calculated_{A / A} between the movement vectors of unit A and a candidate unit

TO_ñ on a left side of unit A. A motion difference value D is also calculated_B,_TO between the movement vectors of unit A and a candidate unit B_ñ on an upper side of unit A. A motion difference value D is also calculated_AC between the movement vectors of unit A and a candidate unit C_TO in line with the upper-right direction of unit A. A motion difference value D is also calculated_d,_to between the movement vectors of unit A and a candidate unit D_TO in line with a top-left address of unit A.

A movement difference value D is also calculated_to,_TO between the movement vectors of unit A and a unit candidate for_TO on a left side of a placed unit corresponding to unit A. Similarly, the motion difference values ​​D are also calculated_bia, Dj,_ñ corresponding to the coded unit A. Then the calculated motion difference values ​​D_{A; A}, D_{B (A}, D_C,_TO, D_d,_to,

Da, a, D_b,_TO, Dj,_TO corresponding to the target coded unit A are recorded in the table in Figure 7. Then another target B coded unit is selected from the coded units (step 604), and then the motion difference values ​​D are calculated_TO,_B, D_b,_b, D_c,_b, D_d,_b, D_to,_B, D_b,_B, D_{j (B} corresponding to the target encoded unit B and recorded in the table of Figure 7 (step 606). The steps

604 and 606 are repeated until all encoded units

A, B, C, D, and e have been selected to be the target coded unit and the motion difference values ​​corresponding to the A, B, C, D, and coded units have been calculated (step 608).

After all the motion differences corresponding to the encoded units A, B, C, D and e have been calculated, a selection order of the plurality of motion vectors is determined by weighted sum of the motion differences (step 610) . For example, if candidate unit A is selected as the target candidate unit, then the movement difference values ​​D are added together._{A (A}, D_{A (B}, D_{A (C}, D_{A (D}, and D_TO,_and, corresponding to the target candidate unit A according to a series of weights W_TO, W_B, W_c, W_D, and W_and to obtain a weighted sum SA = [(D_{A; A} * W_TO) + (D_TO,_B^x WB) + (Dñ, c ^x Wc) + (D_to,_d x W_D) + (D_TO,_and x W_and)] corresponding to the target candidate unit A, where the weights W_TO, W_B, W_c, W_D, and W_and respectively correspond to one of the encoded units A, B, C, D, and e. The other candidate units B, C, D, e, i, and j are then sequentially selected to be the target candidate unit, and then the weighted sums S are sequentially calculated_B, S_c, S_D, S_and,

..., Si, and Sj corresponding to candidate units B, C, D, e, ..., i, and j (steps 610 and 612).

When all candidate units have been selected to be the target candidate unit and the weighted sums S_TO, S_B, S_c, S_D, S_and, ..., S_iz and Sj corresponding to all candidate units A, B, C, D, e, ..., i, and j have been calculated (step 614), at least one chosen candidate unit is selected for predicting movement of the current unit of candidate A units,

B, C, D, e, i, and j according to the weighted sums S_ñ,

S_B, S_c, S_D, S_and, ..., Si, and Sj corresponding to candidate units A, B, C, D, e, i, and j (step 616). In one modality, the weighted sums S_TO, S_B, S_c, S_D, S_and, ···, Si, and Sj are ordered according to their sizes, and the candidate unit corresponding to the smallest weighted sum is determined to be the chosen candidate unit.

Finally, a motion vector of the current unit 512 is predicted according to the motion vector of the chosen candidate unit.

While the invention has been described by way of example and in terms of the preferred embodiment, it should be understood that the invention is not limited thereto. Rather, it is intended to cover different modifications and similar accommodations (as would be apparent to those skilled in the art). Therefore, the broadest interpretation should be given to the scope of the appended claims to encompass all such modifications and similar accommodations.