Determining method of filtering strength, moving picture coding method, and moving picture decoding method

The present invention is a method for determining the filtering strength for removing coding distortion between blocks, and a moving image coding method for coding or decoding a moving image by filtering according to the determined filtering strength. And the moving image decoding method.

Filters are commonly used in video compression methods to improve image quality and compression ratio. Blocky artifacts typically occur in decrypted pictures of low bitrate video compression due to quantization noise as well as motion compensation. One of the tasks of the filter is to smooth the boundaries of blocks in the decoded picture so that these blocky artifacts are reduced or removed.

For example, video compression methods such as the ISO / IEC 14496-2 Part 10 Commission Draft, which is currently developing standards, use loop filters to improve video compression (eg,). See Non-Patent Document 1). This loop filter is applied to both referenced and non-referenced pictures to improve the quality of the decoded pictures.

FIG. 13 shows the determination algorithm used in the ISO / IEC 14496-2 Part 10 standard committee draft to select the strength of the filter used.

The determination to select the strength of the filter used is performed at the block boundary of two adjacent blocks p and q. It is determined whether or not these blocks p or q are coded in the picture (Intra) (step S102). Here, when one of the block p and the block q is intra-coded (Yes in step S102), it is determined whether or not the block boundary becomes a macroblock boundary (step S103). As a result of this determination, if the block boundary becomes a macroblock boundary, that is, if two blocks are not included in the same macroblock, the strongest filter intensity (Bs = 4) is selected (Yes in step S103). ). On the other hand, if the block boundary does not become a macroblock boundary, that is, if these two blocks are included in the same macroblock, the second intensity (Bs = 3) is selected (No in step S103).

As a result of the above determination (step S102), if both the block p and the block q are not intra-encoded (No in step S102), then either of these two blocks determines the spatial frequency component after orthogonal conversion. It is determined whether or not the indicated coefficient is included (step S104). Here, if either of the two blocks contains a coefficient (Yes in step S104), the strength of the third strongest filter (Bs = 2) is selected. On the other hand, if both of the two blocks do not contain the coefficients, that is, if the coefficients are not encoded in both blocks p and q (No in step S104), then to choose whether filtering is skipped or not. The following determination is made (step S105).

It is determined whether or not the index numbers Ref (p) and Ref (q) of the reference pictures of both the block p and the block q are the same. Also, the vertical (V (p, y) and V (q, y)) and horizontal (V (p, x) and V (q, x)) motion vectors of the two blocks are compared with each other, and more than one pixel. Determine if there is a small difference. As a result of these two determinations, only if the index numbers of the reference pictures of the two blocks are the same and their vertical and horizontal motion vectors are no more than one pixel (No in step S105), these two Boundary filtering between blocks is skipped. In all other cases (Yes in step S105), weak filtering (Bs = 1) is performed at the block boundaries.<nplcit num="1"><text>Joint Video Team (JVT) of ISO / IEC MPEG and ITU-T VCEG Joint Committee Draft 2002-05-10, JVT-C167 9.5 Deblocking Filter</text></nplcit>

<p> Traditional decision algorithms for selecting the strength of a filter do not adequately cover all possible cases for a block in a predictively coded picture that references two pictures. The reason for this is that a macroblock of a predictive coded picture that references two pictures can be predicted using direct, forward, reverse, and modes that refer to the two pictures. These prediction modes are not considered in conventional determination algorithms. Similarly, when one block uses direct mode and the other block uses predictive mode with reference to two pictures, the motion vectors used for comparison are not well described in the prior art.</p><p> Therefore, the present invention has been made in view of the above circumstances, and is a method for determining the filtering intensity, which can determine the optimum filtering intensity even when the predictive coding that refers to two pictures is used. It is an object of the present invention to provide a moving image coding method and a moving image decoding method.</p>

<p> In order to achieve the above object, the method for determining the filtering strength according to the present invention is a method for determining the filtering strength for removing the coding distortion between the blocks constituting the picture, and is already encoded. A parameter acquisition step for acquiring a parameter that is information at the time of encoding of the target block and a coded adjacent block adjacent to the target block, and a picture including the target block and the adjacent block refer to two pictures. In the case of a picture that performs inter-picture predictive coding, a comparison step of comparing the parameters of the target block and the adjacent block and a determination step of determining the filtering intensity based on the comparison result of the comparison step are included. It is characterized by.</p><p> Here, the parameter includes the coding mode of the block, and the comparison step is referred to by the target block and the adjacent block, respectively, based on the coding mode of the target block and the adjacent block, respectively. A picture number determination step for determining whether or not the number of pictures is the same may be included, and in the determination step, different filtering intensities may be determined depending on the determination result in the picture number determination step.</p><p> In addition, the parameter further includes a reference index for uniquely identifying the reference picture, and the comparison step further includes the target block and the target block based on the reference index of each of the target block and the adjacent block. A reference picture determination step for determining whether or not the reference pictures referred to by the adjacent blocks are the same is included, and in the determination step, different filtering intensities are determined according to the determination result in the reference picture determination step. You may.</p><p> Further, the parameter includes a motion vector with respect to the reference picture, and the comparison step further includes a horizontal component of an arbitrary motion vector of the target block based on the motion vector included in the target block and the adjacent block. And at least one of the difference between the horizontal component of any motion vector of the adjacent block or the vertical component of any motion vector of the target block and the vertical component of any motion vector of the adjacent block. A motion vector determination step for determining whether or not the value is equal to or greater than a predetermined value may be included, and in the determination step, different filtering strengths may be determined depending on the determination result in the motion vector determination step.</p><p> This covers all possible cases for a block in a predictive coded picture that references two pictures, and decodes even if predictive coding that references two pictures is used. The filtering strength of the filter that filters the image data to remove the block distortion (coding distortion between blocks), which is high-frequency noise near the boundary between blocks, can be optimally determined. Further, this method of determining the filtering intensity can be applied to both a moving image coding device and a moving image decoding device.</p><p> Further, the method for determining the filtering strength according to the present invention is a method for determining the filtering strength for removing the coding distortion between the blocks constituting the picture, and is a method for determining the filtering strength of the encoded target block and the target. The parameter acquisition step for acquiring the picture type of the picture including the encoded adjacent block adjacent to the block and the inter-picture predictive coding in which the picture type acquired by the parameter acquisition step refers to two pictures are shown. The case is characterized by including a determination step that determines the intensity of filtering more strongly than when showing inter-picture predictive coding that references one picture.</p><p> As a result, even when predictive coding that refers to two pictures is used as described above, the decoded image data is filtered and block distortion (coding between blocks), which is high-frequency noise near the boundary between blocks, is used. The filtering strength of the filter that removes distortion) can be optimally determined. Further, this method of determining the filtering intensity can be applied to both a moving image coding device and a moving image decoding device.</p><p> Further, the moving image coding method according to the present invention is a moving image coding method for encoding each picture constituting a moving image in block units, and the filtering determined by the filtering intensity determination method according to the present invention. It is characterized by including a filtering step for filtering between the target block and the adjacent block depending on the intensity.</p><p> Further, the moving image decoding method according to the present invention is a moving image decoding method for decoding moving image encoded data in which each picture constituting the moving image is encoded in block units, and is related to the present invention. It is characterized by including a filtering step of filtering between the target block and the adjacent block according to the filtering strength determined by the method for determining the filtering strength.</p><p> The present invention can be realized not only as such a method for determining the filtering intensity, a moving image coding method and a moving image decoding method, but also such a method for determining the filtering intensity, a moving image coding method and the like. It can be realized as a filtering strength determination device, a moving image coding device, and a moving image decoding device having characteristic steps included in the moving image decoding method as a means, or as a program that causes a computer to execute those steps. You can also do it. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.</p>

<p> As is clear from the above description, according to the method for determining the filtering intensity according to the present invention, the decoded image data is filtered between blocks even when predictive coding that refers to two pictures is used. The filtering intensity of the filter that removes block distortion, which is high-frequency noise near the boundary, can be optimally determined. Therefore, the moving image can be encoded so as to improve the image quality of the decoded moving image. Further, the method for determining the filtering intensity according to the present invention can be applied to both a moving image coding device and a moving image decoding device, and its practical value is great.</p>

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1) FIG. 1 is a block diagram showing a configuration of a moving image coding apparatus using the method for determining the filtering intensity according to the present invention.

The motion image coding device is a device that compresses and encodes the input motion image and outputs it as a code string, and as shown in FIG. 1, the picture memory 101, the difference calculation unit 102, the prediction residual coding unit 103, and the code. Column generation unit 104, prediction residual decoding unit 105, addition calculation unit 106, motion vector detection unit 107, motion vector storage unit 108, motion compensation coding unit 109, filter processing control unit 110, picture memory 111, switch 112, It includes 113 and an inter-pixel filter 114.

The picture memory 101 stores moving images input in picture units in order of display time. Here, the picture means one unit of coding called a screen including both a frame and a field. The motion vector detection unit 107 uses the encoded decoded image data as a reference picture to detect a motion vector indicating a position predicted to be optimal in the search region in the picture. Further, the motion vector detection unit 107 notifies the motion compensation coding unit 109 and the motion vector storage unit 108 of the detected motion vector.

The motion compensation coding unit 109 determines the coding mode of the block using the motion vector detected by the motion vector detecting unit 107, and generates predicted image data based on this coding mode. This coding mode indicates how to encode the macroblock, and indicates whether it is non-intra-coding (motion compensation coding) or intra-coding. For example, if the correlation between pictures is low and intra-coding is preferable to motion prediction, intra-coding is selected. This coding mode is notified to the filter control unit 110. Further, the motion vector and the coding mode are notified from the motion compensation coding unit 109 to the code string generation unit 104. The motion vector storage unit 108 stores the motion vector detected by the motion vector detection unit 107.

The difference calculation unit 102 calculates the difference between the image data read from the picture memory 101 and the predicted image data input from the motion compensation coding unit 109, and generates the predicted residual image data. The prediction residual coding unit 103 performs coding processing such as orthogonal conversion and quantization on the input predicted residual image data to generate coded data. The code sequence generation unit 104 performs variable length coding or the like on the coded data generated by the prediction residual coding unit 103, and further performs motion vector information and coding input from the motion compensation coding unit 109. A code string is generated by adding mode information and the like.

The predictive residual decoding unit 105 performs decoding processing such as inverse quantization and inverse orthogonal conversion on the input encoded data to generate decoded difference image data. The addition calculation unit 106 adds the decoded difference image data input from the predicted residual decoding unit 105 and the predicted image data input from the motion compensation coding unit 109 to generate the decoded image data. The picture memory 111 stores the filtered decoded image data.

The filter processing control unit 110 performs filtering strength of the inter-pixel filter 114, that is, filter A114a, filter B114b, filter C114c, filter D114d, and no filtering (skip) according to the input motion vector and coding mode. Select which one to use and control the switch 112 and the switch 113. The switch 112 and the switch 113 are switches that selectively connect any of the terminals "1" to "5" under the control of the filter processing control unit 110, respectively. The switch 113 is provided between the output terminal of the addition calculation unit 106 and the input terminal of the inter-pixel filter 114. Further, the switch 112 is provided between the input terminal of the picture memory 111 and the output terminal of the inter-pixel filter 114.

The inter-pixel filter 114 is, for example, a deblocking filter that filters decoded image data to remove block distortion, which is high-frequency noise near the boundary between blocks, and has different filtering intensities, such as filter A114a, filter B114b, and filter. It has a C114c and a filter D114d. As for the filter A114a, the filter B114b, the filter C114c, and the filter D114d, the filter A114a has the strongest filtering strength, the filter B114b, the filter C114c, and the filter D114d have the weakest filtering strength in this order, and the filter D114d has the weakest filtering strength. In addition, the amount of arithmetic processing for filtering differs depending on the filtering strength. The illustrated configurations such as the switch 112 and the switch 113 may be implemented as hardware or software.

FIG. 2 is an explanatory diagram showing the order of the pictures in the picture memory 101, and is an explanatory diagram showing (a) the input order and (b) the rearranged order. Here, the vertical line indicates a picture, and the symbol shown at the lower right of each picture is the picture number in the order of display time, with the alphabet of the first letter indicating the picture type (I, P, or B) and the numbers after the second letter. Is shown. Further, the P picture uses the nearby I picture or P picture in the front in the display time order as the reference picture, and the B picture refers to the nearby I picture or P picture in the front in the display time order and the rear in the display time order. It is assumed that one I picture or P picture in the vicinity of is used as a reference picture.

FIG. 3 is an explanatory diagram of a picture and a reference index. The reference index is used to uniquely identify the reference picture stored in the picture memory 111, and is a number associated with each picture as shown in FIG. The reference index is also used to indicate the reference picture to use when coding the block by inter-picture prediction.

First, in the information indicating the display order, the value of the first reference index is assigned a value starting from "0" in the order closer to the coded target picture with respect to the reference picture before the coded target picture. If a value starting with "0" is assigned to all the reference pictures before the coded target picture, the subsequent values are assigned to the reference pictures after the coded target picture in order from the closest to the coded target picture. ..

As the value of the second reference index, first, in the information indicating the display order, a value starting from "0" is assigned to the reference picture after the coded picture to be closer to the coded picture. When a value starting with "0" is assigned to all the reference pictures after the coded target picture, the subsequent values are assigned to the reference pictures before the coded target picture in order from the closest to the coded target picture. ..

For example, in FIG. 3, when the first reference index Ridx1 is "0" and the second reference index Ridx2 is "1", the forward reference picture is the B picture of picture number 7, and the backward reference picture is picture number 9. It is a P picture of. Here, the picture number is a number indicating the display order. Note that FIG. 3 shows an example of how to allocate the reference index, and the allocation method is not limited to the example of FIG.

Next, the operation of the moving image coding device configured as described above will be described. The input images are input to the picture memory 101 in picture units in the order of display time, for example, as shown in FIG. 2A. When the picture type to be encoded is determined, each picture input to the picture memory 101 is sorted in the order in which the coding is performed, for example, as shown in FIG. 2 (b). This sorting to the coding order is performed based on the reference relationship in the inter-picture predictive coding, and the pictures used as the reference pictures are sorted so as to be encoded before the pictures used as the reference pictures. .. As for the determination of the picture type, for example, a method of periodically assigning the picture type is generally used.

Each picture rearranged in the picture memory 101 is read out in macroblock units divided into groups of 16 horizontal pixels and 16 vertical pixels, for example. In addition, motion compensation and motion vector extraction are performed, for example, in block units divided into groups of 8 horizontal pixels and 8 vertical pixels.

The target macroblock read from the picture memory 101 is input to the motion vector detection unit 107 and the difference calculation unit 102.

The motion vector detection unit 107 uses the decoded image data stored in the picture memory 111 as a reference picture, and detects the motion vector for each block in the macro block. Then, the motion vector detection unit 107 outputs the detected motion vector and the reference index indicating the reference picture to the motion compensation coding unit 109.

The motion compensation coding unit 109 determines the coding mode of the macroblock by using the motion vector and the reference index detected by the motion vector detection unit 107. Here, the coding mode is, for example, in the case of a B picture, in-picture coding, inter-picture predictive coding using a forward motion vector, inter-picture predictive coding using a backward motion vector, and two motion vectors. It is possible to select which method to encode from the inter-picture predictive coding and the direct mode used.

Here, the inter-picture prediction method in the direct mode will be described with reference to FIG. FIG. 4 is an explanatory diagram showing a motion vector in the direct mode, and shows a case where the block a of the picture B8 is encoded in the direct mode. In this case, the motion vector c having the block b at the same position as the block a in the picture P9, which is the reference picture behind the picture B8, is used. This motion vector c is the motion vector used when the block b was encoded, and refers to the picture P5. Block a uses the motion vector d for picture P5, which is a forward reference picture, and the motion vector e for picture P9, which is a backward reference picture, obtained by scaling the motion vector c, to form picture P5 and picture P9. Motion compensation is performed from.

The motion compensation coding unit 109 generates predicted image data based on the determined coding mode, and outputs the predicted image data to the difference calculation unit 102 and the addition calculation unit 106. When the motion compensation coding unit 109 selects the direct mode, the motion vector of the block in the back reference picture at the same position as the target block is used as the reference motion vector as described above. The compensation coding unit 109 reads out the reference motion vector and the reference index from the motion vector storage unit 108. Further, when the motion compensation coding unit 109 selects in-picture coding, the predicted image data is not output. Further, the motion compensation coding unit 109 filters the determined coding mode, motion vector, and reference index information to the control control unit 110 and the code string generation unit 104, and filters the value of the reference index indicating the reference picture. Output to the control unit 110.

The difference calculation unit 102 in which the predicted image data is input from the motion compensation coding unit 109 calculates and predicts the difference between the predicted image data and the macroblock image data of the picture B11 read from the picture memory 101. Residual image data is generated and output to the predicted residual coding unit 103.

The predicted residual coding unit 103 to which the predicted residual image data is input performs coding processing such as orthogonal conversion and quantization on the predicted residual image data, generates the coded data, and generates a code string. Output to unit 104 and predictive residual decoding unit 105. The coding string generation unit 104 to which the coding data is input performs variable length coding or the like on the coding data, and further, motion vector information and coding mode information input from the motion compensation coding unit 109. Etc. are added to generate a code string and output it. For macroblocks encoded in the direct mode, motion vector information is not added to the encoded sequence.

On the other hand, the predictive residual decoding unit 105 performs decoding processing such as inverse quantization and inverse orthogonal conversion on the input encoded data, generates the decoded difference image data, and sends it to the addition calculation unit 106. Output. The addition calculation unit 106 generates the decoded image data by adding the decoded difference image data and the predicted image data input from the motion compensation coding unit 109, and the interpixel filter 114 via the switch 113. Output to.

The inter-pixel filter 114 into which the decoded image data is input filters or filters the decoded image data by any of the filter A114a, the filter B114b, the filter C114c, or the filter D114d selected by the switches 112 and 113. First, it is stored in the picture memory 111 via the switch 112. At this time, the control for switching the terminals "1" to "5" of the switch 112 and the switch 113 is performed by the filter processing control unit 110 as follows.

FIG. 5 is a flow chart showing a method of determining the filtering intensity in the filter processing control unit 110.

The filtering control unit 110 determines the filtering intensity required for both vertical and horizontal block boundaries in the decoded image data. The determination for selecting the strength of the filter used for this filtering is executed at the block boundary of two adjacent blocks p and q as in the conventional case shown in FIG. 13 (step S201). First, the filtering control unit 110 determines whether or not these blocks p or q are coded in the picture (intra) based on the coding mode of each macroblock output from the motion compensation coding unit 109. Is determined (step S202). Here, when one of the block p or the block q is intra-encoded (Yes in step S202), the filter processing control unit 110 determines whether or not the block boundary becomes a macroblock boundary (Yes). Step S203).

As a result of this determination, when the block boundary becomes a macroblock boundary, that is, when two blocks are not included in the same macroblock, the filtering control unit 110 sets the filter A114a (Bs = 4) having the strongest filtering intensity. Select (Yes in step S203). That is, the filter processing control unit 110 controls to switch each terminal of the switch 112 and the switch 113 to 1. On the other hand, when the block boundary does not become the macroblock boundary, that is, when these two blocks are included in the same macroblock, the filtering control unit 110 selects the second intensity filter B114b (Bs 3). (No in step S203). That is, the filter processing control unit 110 controls to switch each terminal of the switch 112 and the switch 113 to 2. Here, Bs 3 indicates that Bs is at least a value of 3 or more under the conditions shown in this flowchart, and whether Bs = 3 or Bs is a value larger than 3 is not disclosed here. It is determined by the conditions of. Hereinafter, the formula including this inequality sign indicates a range that can be determined under conditions not disclosed in the present invention.

As a result of the above determination (step S202), when both the block p and the block q are not intra-encoded (No in step S202), the filter processing control unit 110 determines that either the block p or the block q is used. It is determined whether or not a coefficient indicating the spatial frequency component after orthogonal conversion is included (step S204). Here, when any of the two blocks contains a coefficient (Yes in step S204), the filtering control unit 110 selects the filter C114c (Bs 2) having the third strongest filtering intensity. That is, the filter processing control unit 110 controls to switch each terminal of the switch 112 and the switch 113 to 3.

On the other hand, when both of the two blocks do not include the coefficient, that is, when the coefficient is not encoded in both the block p and the block q (No in step S204), the filtering control unit 110 sets the block p and the block q. It is determined whether the included picture is a P picture or a B picture (step S205). Here, when the picture including the block p and the block q is a P picture, the filtering control unit 110 is input from the motion compensation coding unit 109 and the reference index value input from the motion vector storage unit 108. Based on the motion vector, the reference picture referenced by block p and block q is the same, and the vertical components (V (p, y) and V (q, y)) of the motion vector of block p and block q and It is determined whether or not the difference between the horizontal components (V (p, x) and V (q, x)) is less than one pixel (step S208). That is, whether or not all of the following equations (A), (B), and (C) are satisfied.

<maths num="1"><img file="JP4580901B2_D0001.tif" /></maths>

Here, Ref (p) and Ref (q) indicate reference pictures referred to by blocks p and q , respectively .

As a result of this determination, when the reference pictures referenced by the blocks p and the block q are the same, and the difference between the vertical and horizontal motion vectors of the blocks p and the block q is less than one pixel (Yes in step S208). , The filtering control unit 110 selects not to perform filtering (Bs = 0). That is, the filter processing control unit 110 controls to switch each terminal of the switch 112 and the switch 113 to 5. On the other hand, in other cases (No in step S208), the filter processing control unit 110 selects the filter D114d (Bs 1) having the weakest filtering intensity. That is, the filter processing control unit 110 controls to switch each terminal of the switch 112 and the switch 113 to 4.

As a result of the above determination (step S205), when the picture including the block p and the block q is a B picture, the macroblock coding mode sets the inter-picture predictive coding using the forward motion vector and the backward motion vector. Either the inter-picture predictive coding used, the inter-picture predictive coding using two motion vectors, or the direct mode. For example, if block p uses only forward prediction and block q uses prediction that references two pictures, the number of reference pictures in block p is "1" and the number of reference pictures in block q. Is "2". Therefore, the filter processing control unit 110 determines whether or not the number of reference pictures referenced by the block p and the block q is the same (step S206). As a result, when the number of reference pictures referenced by the block p and the block q are different (No in step S206), the filter processing control unit 110 selects the filter D114d (Bs 1) having the weakest filtering intensity.

On the other hand, when the number of reference pictures referenced by the blocks p and q is the same (Yes in step S206), the filtering control unit 110 is based on the value of the reference index input from the motion compensation coding unit 109. , It is determined whether or not the reference pictures referenced by the blocks p and q are exactly the same (step S207). As a result, when even one reference picture referred to by the block p and the block q is different (No in step S207), the filtering control unit 110 selects the filter D114d (Bs 1) having the weakest filtering intensity. ..

On the other hand, when the reference pictures referenced by the blocks p and the block q are exactly the same (Yes in step S207), the filtering control unit 110 has the same weighting (ABP) coefficient of the weighting prediction in the block p and the block q. It is determined whether or not there is (step S209). As a result, when the weighting coefficients of the block p and the block q are different (No in step S209), the filter processing control unit 110 selects the filter D114d (Bs 1) having the weakest filtering intensity. Here, the weighted prediction is a prediction method in which the predicted pixel value is obtained by multiplying the pixel value of the reference image by the first weighting coefficient α and further adding the second weighting coefficient β in the inter-picture prediction. That is.

On the other hand, when the weighting coefficients of the block p and the block q are the same (Yes in step S209), the filtering control unit 110 determines that the vertical and horizontal motion vectors of the block p and the block q are different from each other. It is determined whether or not the number of pixels is less than one pixel (step S210). That is, whether or not all of the following equations (D) to (G) are satisfied.

<maths num="2"><img file="JP4580901B2_D0002.tif" /></maths>

Here, Vf and Vb indicate motion vectors in each of the block p and the block q, and for example, when there is one reference picture, there is only one.

As a result of this determination, when the difference between the vertical and horizontal motion vectors for all the motion vectors of the blocks p and q is less than one pixel (Yes in step S210), the filtering control unit 110 performs filtering. Select None (Bs = 0). On the other hand, in other cases (No in step S210), the filter processing control unit 110 selects the filter D114d (Bs 1) having the weakest filtering intensity.

As described above, the macro block of the B picture can be predicted by using the direct mode. When the direct mode is used, the motion vector of the target block is derived from the motion vector of the block corresponding to the same position as the target block in the reference picture in which the second reference index Ridx2 is 0. In this case, the forward reference picture of the target block is a reference picture referenced by the motion vector of the corresponding block, and the backward reference picture of the target block is a reference picture in which the second reference index Ridx2 is 0. Therefore, the filtering control unit 110 uses the derived motion vector and the reference picture to determine the intensity of filtering.

As described above, when the picture including the block p and the block q is the B picture, whether or not the number of the reference pictures referred to by the block p and the block q is the same, and the reference pictures to be referred to are exactly the same. Since it is determined whether or not it is, the optimum filtering intensity can be determined even when predictive coding that refers to two pictures is used. Therefore, the moving image can be encoded so as to improve the image quality of the decoded moving image.

FIG. 6 is a block diagram showing a configuration of a moving image decoding apparatus using the method for determining the filtering intensity according to the present invention.

The moving image decoding device is a device that decodes the code string encoded by the moving image coding device, and as shown in FIG. 6, the code string analysis unit 201, the predicted residual decoding unit 202, and the motion compensation decoding unit. A unit 203, a motion vector storage unit 204, a filter processing control unit 205, a picture memory 206, an addition calculation unit 207, switches 208 and 209, and an inter-pixel filter 210 are provided.

The code string analysis unit 201 extracts various data such as coding mode information and motion vector information used at the time of coding from the input code string. The predictive residual decoding unit 202 decodes the input predictive residual coded data and generates the predictive residual image data. The motion compensation decoding unit 203 acquires image data from the reference picture stored in the picture memory 206 based on the coding mode information at the time of coding, the motion vector information, and the like, and generates motion compensation image data. To do. The motion vector storage unit 204 stores the motion vector extracted by the code string analysis unit 201. The addition calculation unit 207 adds the predicted residual image data input from the predicted residual decoding unit 202 and the motion compensation image data input from the motion compensation decoding unit 203 to generate the decoded image data. .. The picture memory 206 stores the filtered decoded image data.

The filter processing control unit 205 selects the filtering intensity of the inter-pixel filter 210, that is, which of the filter A210a, the filter B210b, the filter C210c, the filter D210d, and the non-filtering (skip) is used, and switches 208 and Control switch 209. The switch 208 and the switch 209 are switches for selectively connecting any of the terminals "1" to "5" under the control of the filter processing control unit 205, respectively. The switch 209 is provided between the output terminal of the addition calculation unit 207 and the input terminal of the inter-pixel filter 210. Further, the switch 208 is provided between the input terminal of the picture memory 206 and the output terminal of the inter-pixel filter 210.

The inter-pixel filter 210 is, for example, a deblocking filter that filters decoded image data to remove block distortion, which is high-frequency noise near the boundary between blocks, and has different filtering intensities, such as filter A210a, filter B210b, and filter. It has a C210c and a filter D210d. Of the filter A210a, the filter B210b, the filter C210c, and the filter D210d, the filter A210a has the strongest filtering strength, the filter B210b, the filter C210c, and the filter D210d have the weakest filtering strength, and the filter D210d has the weakest filtering strength. In addition, the amount of arithmetic processing for filtering differs depending on the filtering strength.

Next, the operation of the moving image decoding apparatus configured as described above will be described. The code string analysis unit 201 extracts various data such as coding mode information and motion vector information from the input code string. The code sequence analysis unit 201 outputs the extracted coding mode information to the motion compensation decoding unit 203 and the filter processing control unit 205, and outputs the motion vector information and the reference index to the motion vector storage unit 204. Further, the code string analysis unit 201 outputs the extracted predicted residual coded data to the predicted residual decoding unit 202. The predicted residual decoding unit 202 into which the predicted residual coded data is input decodes the predicted residual coded data, generates the predicted residual image data, and outputs the predicted residual image data to the addition calculation unit 207.

On the other hand, the motion compensation decoding unit 203 has a picture memory 206 based on the coding mode information input from the code string analysis unit 201, the value of the reference index, and the motion vector information read from the motion vector storage unit 204. The motion compensation image data is generated by referring to the reference picture stored in. Next, the motion compensation decoding unit 203 outputs the generated motion compensation image data to the addition calculation unit 207, and outputs the value of the reference index indicating the reference picture to the filter processing control unit 205. The addition calculation unit 207 adds the motion compensation image data and the predicted residual image data input from the predicted residual decoding unit 202 to generate the decoded image data, and the interpixel filter 210 via the switch 209. Output to.

The inter-pixel filter 210 to which the decoded image data is input filters or filters the decoded image data by any of the filter A210a, the filter B210b, the filter C210c, or the filter D210d selected by the switches 208 and 209. First, it is stored in the picture memory 206 via the switch 208. At this time, the control for switching the terminals "1" to "5" of the switch 208 and the switch 209 is performed by the filter processing control unit 205 in the same manner as the operation of the filter processing control unit 110 of the moving image coding device.

As described above, when the picture including the block p and the block q is the B picture, whether or not the number of the reference pictures referred to by the block p and the block q is the same, and the reference pictures to be referred to are exactly the same. Since it is determined whether or not it is, the optimum filtering intensity can be determined even when predictive coding that refers to two pictures is used. Therefore, it is possible to improve the image quality of the moving image and perform decoding.

(Embodiment 2) In the present embodiment, the method for determining the filtering intensity in the filter processing control unit 110 described in the first embodiment is partially different. The configuration is the same as that of the first embodiment, and detailed description thereof will be omitted. Further, regarding the determination of the filtering strength in the filter processing control unit 110, the same part as in the first embodiment will not be described. Further, in the case of a moving image decoding device, it is a method of determining the filtering strength in the filter processing control unit 205.

FIG. 7 is a flow chart showing a method for determining the filtering intensity in the second embodiment. As a result of the determination (step S304) performed by the filtering control unit 110 whether either the block p or the block q contains a coefficient indicating the spatial frequency component after orthogonal conversion, one of the two blocks has a coefficient. (Yes in step S304), the following processing is performed.

The filter processing control unit 110 determines whether the picture including the block p and the block q is a P picture or a B picture (step S311). Here, when the picture including the block p and the block q is a P picture, the filtering control unit 110 selects the filter C114c (Bs (p) 2) having the third strongest filtering intensity. On the other hand, when the picture including the block p and the block q is a B picture, the filtering control unit 110 has Bs (b) (Bs (b)) having a stronger filtering intensity than Bs (p) in the case of a P picture. )&gt; Bs (p)).

As described above, when either block p or block q contains a coefficient indicating the spatial frequency component after orthogonal conversion, it is determined whether the picture including block p and block q is a P picture or a B picture. Therefore, the optimum filtering intensity can be determined even when predictive coding that refers to two pictures is used. Therefore, the moving image can be encoded so as to improve the image quality of the decoded moving image.

In each of the above embodiments, when the filter processing control unit 110 selects not to perform filtering (Bs = 0), the filter D114d (which has the weakest filtering intensity) is used instead of not performing filtering (skip). A filter having a weaker filtering intensity than Bs 1) may be used.

Further, in each of the above embodiments, the filter processing control unit 110 does not need to execute all the steps shown in the flowchart of FIG. 5 or FIG. 8, and the processing of some steps may be omitted. For example, if the result of the determination in step S207 (S307) is that the reference pictures referenced by the blocks p and the block q are exactly the same (Yes in step S207 (S307)), the determination process in step S209 (S309) is performed. , The determination process of step S210 (S310) may be performed without performing this process. Further, the execution order of each step may be changed.

Further, in each of the above embodiments, although the description has been made using a picture as a unit of coding, a field or a frame may be used.

(Embodiment 3) Further, by recording a program for realizing the configuration of the moving image coding method or the moving image decoding method shown in each of the above embodiments on a storage medium such as a flexible disk, each of the above-mentioned embodiments is carried out. The processing shown in the embodiment can be easily performed in an independent computer system.

FIG. 8 is an explanatory diagram of a storage medium for storing a program for realizing the moving image coding method and the moving image decoding method of each embodiment by a computer system.

FIG. 8 (b) shows the appearance, cross-sectional structure, and flexible disc of the flexible disc when viewed from the front, and FIG. 8 (a) shows an example of the physical format of the flexible disc, which is the main body of the recording medium. The flexible disk FD is built in the case F, and a plurality of track Trs are concentrically formed on the surface of the disk from the outer circumference toward the inner circumference, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk in which the program is stored, the moving image coding method as the program is recorded in the area allocated on the flexible disk FD.

Further, FIG. 8 (c) shows a configuration for recording / reproducing the above program on the flexible disk FD. When recording the above program on the flexible disk FD, the moving image coding method or the moving image decoding method as the above program is written from the computer system Cs via the flexible disk drive FDD. When the moving image coding method is constructed in the computer system by the program in the flexible disk, the program is read from the flexible disk by the flexible disk drive and transferred to the computer system.

In the above description, a flexible disk is used as the recording medium, but an optical disk can also be used in the same manner. The recording medium is not limited to this, and any recording medium such as an IC card or ROM cassette that can record a program can be used in the same manner.

Further, here, an application example of the moving image coding method and the moving image decoding method shown in the above embodiment and a system using the same will be described.

FIG. 9 is a block diagram showing the overall configuration of the content supply system ex100 that realizes the content distribution service. The communication service provision area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.

This content supply system ex100 is, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera via the Internet service provider ex102 and the telephone network ex104, and the base stations ex107 to ex110 on the Internet ex101. Each device such as the mobile phone ex115 with is connected.

However, the content supply system ex100 is not limited to the combination as shown in FIG. 9, and any combination may be used for connection. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110, which are fixed radio stations.

The camera ex113 is a device capable of shooting moving images such as a digital video camera. In addition, the mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone. Alternatively, it may be PHS (Personal Handyphone System) or the like.

Further, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution based on the coded data transmitted by the user using the camera ex113 becomes possible. The captured data may be encoded by the camera ex113 or by a server or the like that performs data transmission processing. Further, the moving image data taken by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images such as a digital camera. In this case, the moving image data may be encoded by either the camera ex116 or the computer ex111. Further, the coding process is performed by the LSI ex117 of the computer ex111 and the camera ex116. The moving image coding / decoding software may be incorporated into some storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium that can be read by a computer ex111 or the like. Further, the moving image data may be transmitted by the mobile phone ex115 equipped with a camera. The moving image data at this time is the data encoded by the LSI of the mobile phone ex115.

In this content supply system ex100, the content photographed by the user with the camera ex113, the camera ex116, etc. (for example, a video of a live music) is encoded and transmitted to the streaming server ex103 in the same manner as in the above embodiment. On the other hand, the streaming server ex103 streams the above content data to the requested client. Clients include a computer ex111, a PDAex112, a camera ex113, a mobile phone ex114, and the like, which can decode the encoded data. By doing so, the content supply system ex100 can receive the encoded data at the client and play it back, and further realize personal broadcasting by receiving it in real time at the client, decoding it, and playing it back. It is a system that makes it possible.

For the coding and decoding of each device constituting this system, the moving image coding device or the moving image decoding device shown in each of the above embodiments may be used.

A mobile phone will be described as an example. FIG. 10 is a diagram showing a mobile phone ex115 using the moving image coding method and the moving image decoding method described in the above embodiment. The mobile phone ex115 is an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, images of a CCD camera, etc., a camera unit ex203 capable of taking still images, an image taken by the camera unit ex203, and an antenna ex201. Display unit ex202 such as liquid crystal display that displays the decoded data of received video, etc., main unit consisting of operation key ex204 group, audio output unit ex208 such as speaker for audio output, audio input To save encoded or decoded data such as audio input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. It has a slot portion ex206 for mounting the recording media ex207 on the recording media ex207 and the mobile phone ex115. The recording medium ex207 is an EEPROM (Electrically Erasable and) which is a non-volatile memory that can be electrically rewritten or erased in a plastic case such as an SD card. It stores a flash memory element, which is a type of Programmable Read Only Memory).

Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 has a power supply circuit unit ex310, an operation input control unit ex304, and image coding for the main control unit ex311 which is designed to collectively control each part of the main body unit provided with the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, multiplex separation unit ex308, recording / playback unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via the synchronization bus ex313. Connected to each other.

The power circuit unit ex310 activates the camera-equipped digital mobile phone ex115 in an operable state by supplying power to each unit from the battery pack when the call ends and the power key is turned on by the user's operation. ..

Based on the control of the main control unit ex311 consisting of CPU, ROM, RAM, etc., the mobile phone ex115 converts the voice signal collected by the voice input unit ex205 in the voice call mode into digital voice data by the voice processing unit ex305. This is spread spectrum processed by the modulation / demodulation circuit unit ex306, digital-to-analog conversion processing and frequency conversion processing are performed by the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201. In addition, the mobile phone ex115 amplifies the received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, spectrum despread processing by the modulation / demodulation circuit unit ex306, and analog voice by the voice processing unit ex305. After converting to data, this is output via the audio output unit ex208.

Further, when the e-mail is transmitted in the data communication mode, the text data of the e-mail input by the operation of the operation key ex204 of the main body unit is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing of text data by the modulation / demodulation circuit unit ex306, digital-to-analog conversion processing and frequency conversion processing by the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image coding unit ex312 via the camera interface unit ex303. Further, when the image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

The image coding unit ex312 has a configuration including the moving image coding device described in the present invention, and the image data supplied from the camera unit ex203 is a code used in the moving image coding device shown in the above embodiment. It is converted into coded image data by compression coding according to the coding method, and this is sent to the multiple separation unit ex308. At the same time, the mobile phone ex115 transmits the sound collected by the voice input unit ex205 during imaging by the camera unit ex203 to the multiplex separation unit ex308 as digital voice data via the voice processing unit ex305.

The multiplex separation unit ex308 multiplexes the coded image data supplied from the image coding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit. Spread spectrum processing is performed by ex306, digital-to-analog conversion processing and frequency conversion processing are performed by the transmission / reception circuit unit ex301, and then transmission is performed via the antenna ex201.

When receiving the data of the moving image file linked to the homepage etc. in the data communication mode, the received data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is performed. The data is sent to the multiplex separator ex308.

Further, in order to decode the multiplexed data received via the antenna ex201, the multiplexing separator ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and synchronizes the data. The encoded image data is supplied to the image decoding unit ex309 and the audio data is supplied to the audio processing unit ex305 via the bus ex313.

Next, the image decoding unit ex309 has a configuration including the moving image decoding device described in the present invention, and is a decoding method corresponding to the coding method shown in the above embodiment for a bit stream of image data. Reproduced moving image data is generated by decoding and supplied to the display unit ex202 via the LCD control unit ex302, whereby the moving image data included in the moving image file linked to the homepage is displayed, for example. .. At the same time, the audio processing unit ex305 converts the audio data into analog audio data and then supplies the audio data to the audio output unit ex208, whereby, for example, the audio data contained in the moving image file linked to the homepage is reproduced. To.

Not limited to the above system example, digital broadcasting by satellite and terrestrial broadcasting has recently become a hot topic, and as shown in FIG. 12, the digital broadcasting system also includes at least a moving image encoding device of the above embodiment. Any of the moving image decoding devices can be incorporated. Specifically, in the broadcasting station ex409, a bit stream of video information is transmitted via radio waves to a communication or a broadcasting satellite ex410. In response to this, the broadcasting satellite ex410 transmits radio waves for broadcasting, receives the radio waves with a home antenna ex406 equipped with satellite broadcasting receiving equipment, and receives the radio waves such as TV (receiver) ex401 or set-top box (STB) ex407. The device decodes the bit stream and plays it back. Further, the moving image decoding device shown in the above embodiment can also be mounted on the playback device ex403 that reads and decodes the bit stream recorded on the storage medium ex402 such as a recording medium such as a CD or DVD. .. In this case, the reproduced video signal is displayed on the monitor ex404. It is also conceivable to mount a moving image decoding device in a set-top box ex407 connected to a cable ex405 for cable TV or an antenna ex406 for satellite / terrestrial broadcasting, and reproduce this on a TV monitor ex408. At this time, the moving image decoding device may be incorporated in the television instead of the set-top box. It is also possible for the car ex412 having the antenna ex411 to receive a signal from the satellite ex410 or the base station ex107 or the like and reproduce the moving image on the display device such as the car navigation ex413 which the car ex412 has.

Further, the image signal can be encoded by the moving image coding device shown in the above embodiment and recorded on a recording medium. Specific examples include a recorder ex420 such as a DVD recorder that records an image signal on a DVD disc ex421 and a disc recorder that records an image signal on a hard disk. It can also be recorded on the SD card ex422. If the recorder ex420 is provided with the moving image decoding device shown in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

The car navigation system ex413 may be configured by excluding the camera unit ex203, the camera interface unit ex303, and the image coding unit ex312 from the configurations shown in FIG. 11, and the same applies to the computer ex111 and the television (receiver). ) Ex401 etc. can also be considered.

In addition, terminals such as the mobile phone ex114 are implemented in three types: a transmitter / receiver terminal having both an encoder and a decoder, a transmitter terminal having only an encoder, and a receiving terminal having only a decoder. Can be considered.

As described above, it is possible to use the moving image coding method or the moving image decoding method shown in the above-described embodiment for any of the above-mentioned devices / systems, and by doing so, the above-described embodiment will be described. The effect can be obtained.

Further, the present invention is not limited to the above-described embodiment, and various modifications or modifications can be made without departing from the scope of the present invention.

<figref num="1">It is a block diagram which shows the structure of the moving image coding apparatus which concerns on this invention.</figref><figref num="2">It is explanatory drawing which shows the order of a picture in a picture memory, (a) the input order, and (b) the rearranged order.</figref><figref num="3">It is explanatory drawing of a picture and a reference index.</figref><figref num="4">It is explanatory drawing which shows the motion vector in a direct mode.</figref><figref num="5">It is a flow chart which shows the method of determining the filtering intensity in the filtering processing control part in Embodiment 1.</figref><figref num="6">It is a block diagram which shows the structure of the moving image decoding apparatus which concerns on this invention.</figref><figref num="7">It is a flow chart which shows the method of determining the filtering intensity in the filtering processing control part in Embodiment 2. FIG.</figref><figref num="8">It is explanatory drawing about the recording medium for storing the program for realizing the moving image coding method and moving image decoding method of each embodiment by a computer system, and (a) of the flexible disk which is a recording medium main body. Explanatory drawing showing an example of a physical format, (b) an explanatory view showing the appearance of the flexible disk from the front, a cross-sectional structure, and a flexible disk, and (c) a configuration for recording and reproducing the above program on the flexible disk FD. It is explanatory drawing which showed.</figref><figref num="9">It is a block diagram which shows the whole structure of a content supply system.</figref><figref num="10">It is the schematic which shows the example of the mobile phone.</figref><figref num="11">It is a block diagram which shows the structure of a mobile phone.</figref><figref num="12">It is a figure which shows the example of the system for digital broadcasting.</figref><figref num="13">It is a flow chart which shows the conventional method of determining the filtering intensity.</figref>

Code description

101, 111, 206 picture memory 102 Differential calculation unit 103 Predictive residual coding unit 104 Code sequence generator 105, 202 Predictive Residual Decoding Unit 106, 207 Addition calculation unit 107 Motion vector detector 108, 204 Motion vector storage 109 Motion compensation coding unit 110, 205 Filter processing control unit 112, 113, 208, 209 switches 114, 210 Pixel-to-pixel filter 201 Code sequence analysis unit 203 Motion compensation decoding unit