Motion picture encoding device and motion picture encoding processing program

CROSS-REFERENCE TO RELATED APPLICATION

0001This is a Continuation of U.S. application Ser. No. 11/504,021, filed Aug. 14, 2006, now U.S. Pat. No. 8,121,193 which is a Continuation Application of PCT Application No. PCT/JP2005/020689, filed Nov. 4, 2005.

BACKGROUND OF THE INVENTION

00021. Field of the Invention

0003The present invention relates to a motion picture encoding device and associated motion picture encoding processing program which compresses motion images in view of human visual characteristics.

00042. Description of the Related Art

0005In applications, such as digital television broadcasting, Internet video streaming, DVD, etc., a coding technique with a high compression ratio is required because transmission bandwidth and storage capacity are limited. For instance, an H.264 standard is known as a high compression coding technique which meets such a requirement. Hereinafter, an example of a motion picture encoding device based on the H.264 standard will be described with reference to <figref idref="DRAWINGS">FIG. 9˜FIG</figref>. <b>15</b>.

0006<figref idref="DRAWINGS">FIG. 9</figref> is a block diagram showing an outline configuration of a motion picture encoding device. A subtracter <b>10</b> generates a prediction error signal indicating luminance difference by subtracting the prediction block pixel value from the current block pixel value. A quantization/transform section <b>11</b> applies integer DCT (Discrete Cosine Transform) to the prediction error signal outputted from the subtracter <b>10</b> and a transform coefficient is obtained. Furthermore, this transform coefficient is quantized with a predetermined quantization width and coefficient data is generated. An entropy encoder section <b>12</b> performs entropy encoding of the coefficient data generated by the quantization/transform section <b>11</b> with Exponential-Golomb codes based on Variable Length Codes (VLC) and applies CABAC (Context-based Adaptive Binary Arithmetic Coding).

0007An inverse quantization/inverse transform section <b>13</b>, an adder <b>14</b>, a loop filter <b>15</b> and a frame memory <b>16</b> form a local decoding portion. The local decoding portion applies inverse quantization and inverse integer DCT to the coefficient data generated in the quantization/transform section <b>11</b>, adds the previous prediction block pixel value and generates a decoded image. Further, after the local decoding portion reduces block noise by performing loop filtering to the generated decoded image, it is temporarily stored in the frame memory <b>16</b>. An intra-frame prediction section <b>17</b> calculates the intra-frame prediction block value using the decoded image read out from the frame memory <b>16</b>.

0008A motion detection section <b>18</b> detects the motion vector of the current block. A motion compensation section <b>19</b> calculates the inter-frame prediction block value by performing motion compensation to the reference frame (decoded image read out from the frame memory <b>16</b>) corresponding to the motion vector detected by the motion detection section <b>18</b>. A selector <b>20</b> selects either the intra-frame prediction block value calculated by the intra-frame prediction section <b>17</b> or the inter-frame prediction block value calculated by the motion compensation section <b>19</b> corresponding to instructions of a determination section <b>21</b> and provides the respective value to the subtracter <b>10</b>. The determination section <b>21</b> estimates the amount of coded data at the time of intra-frame predictive coding as well as the amount of coded data of inter-frame predictive coding and directs the selector <b>20</b> to select the coding mode with the smaller amount of coded data.

0009Next, the operations of the encoding determination processing of the motion picture encoding device according to the above-mentioned configuration will be explained with reference to <figref idref="DRAWINGS">FIGS. 10 through 15</figref>. In the following, after first explaining the operations of the “encoding determination processing”, the separate operations of the “inter-prediction processing”, the “intra-prediction processing” and the “inter D&amp;Q processing” which encompass the encoding determination processing will be outlined.

0010(1) Operations of the Encoding Determination Processing

0011<figref idref="DRAWINGS">FIG. 10</figref> is a flow chart showing operations of the encoding determination processing executed for each input macroblock. This processing is initiated by inputting a 16×16 macroblock image (hereinafter, denoted as “input macroblock”). In Step SF<b>1</b>, each section of the device is initialized. Secondly, in Step SF<b>2</b>, the amount of coded data at the time of inter-frame predictive coding is estimated by the execution of inter-prediction processing. Next, in Step SF<b>3</b>, the amount of coded data for intra-frame predictive coding is estimated and further intra-frame coding (integer DCT, quantization, inverse quantization and inverse integer DCT) is performed by the execution of intra-prediction processing. Then, in Step SF<b>4</b>, the minimum Sum of Absolute Differences SADinter (this is equivalent to the amount of coded data at inter-prediction) obtained in the above-mentioned Step SF<b>2</b> is judged as to whether or not greater than the minimum Sum of Absolute Differences SADintra (this is equivalent to the amount of coded data at intra-prediction) obtained in the above-mentioned Step SF<b>3</b>.

0012When the amount of coded data at inter-prediction is greater than the amount of coded data at intra-prediction, the judgment result becomes “YES” and completes this processing. Accordingly, in this case the motion picture encoding device performs video compression with the intra-frame coding executed in the above-mentioned Step SF<b>3</b>.

0013Conversely, when the amount of coded data at the time of inter-prediction is less than the amount of coded data at the time of intra-prediction, the judgment result of the above-mentioned Step SF<b>4</b> becomes “NO” and the flow advances to Step SF<b>5</b>, which executes inter D&amp;Q processing to perform inter-frame coding (integer DCT, quantization, inverse quantization and inverse integer DCT). Accordingly, in this case the motion picture encoding device performs video compression with inter-frame coding.

0014(2) Operations of the Inter-Prediction Processing

0015Next, the operations of the inter-prediction processing will be explained with reference to <figref idref="DRAWINGS">FIGS. 11 through 13</figref>. When processing has been executed via the above-mentioned Step SF<b>2</b> (refer to <figref idref="DRAWINGS">FIG. 10</figref>), the flow advances to Step SG<b>1</b> shown in <figref idref="DRAWINGS">FIG. 11</figref> and judges whether or not processing has been completed for all of the 4×4 pixel blocks of a 16×16 pixel macroblock partitioned into 16 sub-macroblocks. When processing for all the blocks has been completed, the judgment result becomes “YES” and this processing is completed. Otherwise, the judgment result becomes “NO” and the flow advances to Step SG<b>2</b>.

0016Additionally, with regard to the association between a 16×16 pixel macroblock and the 4×4 pixel blocks, as illustrated in <figref idref="DRAWINGS">FIG. 13</figref>, the 4×4 pixel blocks within the macroblock are denoted by the block number n.

0017Next, in Step SG<b>2</b>, a 4×4 pixel block processing object (hereinafter, denoted as the “current block”) pixel value Org is calculated. Subsequently, in Step SG<b>3</b>, MV search processing is executed which searches for a motion vector (MV). In the MV search processing, as shown in Steps SH<b>1</b>˜SH<b>3</b> of <figref idref="DRAWINGS">FIG. 12</figref>, correlation of the current block is calculated while shifting pixels in the center of the reference block within a search region of the reference frame. Amongst those pixels, the pixel position with the highest correlation (similarity) is extracted as the best motion vector.

0018MV search processing estimates the correlation of the reference block and the current block with the Sum of Absolute Differences SAD between both blocks. Accordingly, when the pixel position of the highest correlation is extracted as the motion vector, the Sum of Absolute Differences SAD represents the minimum. The minimum Sum of Absolute Differences SADinter is used for Step SF<b>4</b> (refer to <figref idref="DRAWINGS">FIG. 10</figref>) for judging whether or not to execute inter D&amp;Q processing.

0019(3) Operations of the Intra-Prediction Processing

0020Next, the operations of the intra-prediction processing will be explained with reference to <figref idref="DRAWINGS">FIG. 14</figref>. When inter-prediction processing has been executed via Step SF<b>3</b> mentioned above (refer to <figref idref="DRAWINGS">FIG. 10</figref>), the flow advances to Step SJ<b>1</b> shown in <figref idref="DRAWINGS">FIG. 14</figref>. In Step SJ<b>1</b>, whether or not processing has been completed for all 4×4 pixel blocks of a 16×16 pixel macroblock partitioned into 16 sub-macroblocks is judged. When processing for all the blocks has been completed, the judgment result becomes “YES” and this processing is completed. Otherwise, the judgment result becomes “NO” and the flow advances to Step SJ<b>2</b>. In Step SJ<b>2</b>, for example in intra 4×4 mode, a prediction block value for each mode amongst a total of nine optional prediction modes is calculated, which are referred to as mode<b>0</b>˜mode<b>8</b>.

0021Subsequently, in Steps SJ<b>3</b>˜SJ<b>5</b>, the current block Sum of Absolute Differences SAD and the prediction block value for each of the above-mentioned modes is calculated, respectively, and the minimum Sum of Absolute Differences SADintra is obtained from within these results. The minimum Sum of Absolute Differences SADintra is used for Step SF<b>4</b> (refer to <figref idref="DRAWINGS">FIG. 10</figref>) for judging whether or not to execute inter D&amp;Q processing.

0022When the minimum Sum of Absolute Differences SAD has been determined, the judgment result in Step SJ<b>3</b> becomes “YES” and the flow advances to Step SJ<b>6</b>. In Step SJ<b>6</b>, intra-frame coding of the current block is performed using the mode which produces the minimum Sum of Absolute Differences SADintra. Hereinafter, the above-mentioned Steps SJ<b>1</b>˜SJ<b>6</b> are repeated until processing for all blocks is completed.

0023(4) Operations of the Inter D&amp;Q Processing

0024Next, the operations of the inter D&amp;Q processing will be explained with reference to <figref idref="DRAWINGS">FIG. 15</figref>. When processing has been executed via Step SF<b>5</b> mentioned above (refer to <figref idref="DRAWINGS">FIG. 10</figref>), the flow advances to Step SK<b>1</b> shown in <figref idref="DRAWINGS">FIG. 15</figref> and whether or not processing has been completed for all of the 4×4 pixel blocks of a 16×16 pixel macroblock partitioned into 16 sub-macroblocks is judged. If processing has not been completed for all the blocks, the judgment result becomes “NO” and the flow advances to Step SK<b>2</b>.

0025In Steps SK<b>2</b>˜SK<b>6</b>, the inter-frame prediction block value ref (i, j) is subtracted from the current block pixel value Org (i, j) and proceeds to execute transform processing (integer DCT), quantization processing Q, inverse quantization processing Q-<b>1</b> and inverse transform processing (inverse integer DCT-<b>1</b>) to the produced prediction error signal. Then, the flow advances to Step SK<b>7</b>. Because of the stepping performed in the inter-frame prediction block value ref (i, j), processing will revert to the above-mentioned Step SK<b>1</b>. Hereafter, the above-mentioned Steps SK<b>1</b>˜SK<b>7</b> are repeated until processing for all the blocks is completed.

0026As described above, the motion picture encoding device performs video compression by selectively using either intra-frame coding by correlation in a spatial domain or inter-frame coding by correlation in a temporal domain. A magnitude comparison is executed for each input macroblock between the minimum Sum of Absolute Differences SADinter that is equivalent to the amount of coded data at the time of inter-prediction and the minimum Sum of Absolute Differences SADintra that is equivalent to the amount of coded data at the time of intra-prediction. The coding mode with the smaller Sum of Absolute Differences SAD is selected and compression encoding is performed.

0027Apart from that, in order to obtain the minimum Sum of Absolute Differences SADintra equivalent to the amount of coded data at the time of intra-prediction, for example, when intra 4×4 mode is selected, it is necessary to calculate the prediction block value for each mode amongst a total of nine optional prediction modes, which are referred to as mode<b>0</b>˜mode<b>8</b>. This situation causes an exponential increase in the calculation amount. Furthermore, the coding technique by the H.264 standard has been described in the conventional prior art, for example, as disclosed by Iain E. G. Richardson “H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia”, Publisher: John Wiley &amp; Sons, Ltd. (December 2003, First Edition) “6.H.264/MPEG-4 Part 10” (p 159˜p 223) (ISBN: 0470869607).

0028There are the following issues concerning motion picture encoding devices which perform video compression by selectively using intra-frame coding and inter-frame coding mentioned above.

0029(a) In many cases intra-frame coding is not selected, except in a case where there is a noticeable difference between an inputted image as compared with a reference image, for example, when a scene changes, when a motion vector has not been detected properly or when there is a substantial luminance variation due to a camera flash, etc. For this reason, there is a problem in that the complexity and number of calculations required for intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) is simply excessive.

0030(b) Because a coding determination is performed which uniquely selects either intra-frame coding/inter-frame coding only by a magnitude comparison of the amount of coded data, human visual characteristics are not taken into consideration. This is commonly referred to as the Human Visual System (HVS), the system by which a human eye and brain perceive and interpret visual images. Therefore, there is the possibility of inviting significantly reduced image quality that can easily develop into an image with noticeable noise (luma and chroma noise), such as in the case of quantization error propagation being generated at the time of a motion vector search whereby a fast search algorithm may become ‘trapped’ in a local minimum giving a suboptimal result or during coding at a low bit rate whereby distortion increases the respective quantization value. A local minimum is the concept of a minimum value in a defined section (area) of a block. For instance, this may not be the true global minimum value and is commonly referred to as a false minimum value (false minima).

0031Consequently, the present invention has been made in view of the above-described circumstances with the purpose of providing a motion picture encoding device and associated motion picture encoding processing program, wherein wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0032Furthermore, the present invention aims at providing a motion picture encoding device and motion picture encoding processing program which can rapidly stop quantization error propagation when ‘trapped’ in a local minimum as well as avoid reduced image quality.

SUMMARY OF THE INVENTION

0033In accordance with a first aspect of the present invention, there is provided a motion picture encoding device which selects either intra-frame coding or inter-frame coding and performs video compression, comprising: a coding determination means for judging the existence of a block image having visually noticeable noise when performing inter-frame coding for a block image of a partitioned macroblock image to encode and selecting intra-frame coding if an applicable block image exists.

0034In accordance with a second aspect of the present invention, there is provided a motion picture encoding device which selects either intra-frame coding or inter-frame coding and performs video compression, comprising: a first coding determination means for judging the existence of a block image having visually noticeable noise when performing inter-frame coding for a block image of a partitioned macroblock image to encode and selecting intra-frame coding if an applicable block image exists; and a second coding determination means for judging the existence of a block image having visually noticeable noise while considering the magnitude of a motion vector obtained by inter-frame prediction and selecting intra-frame coding if an applicable block image exists.

0035In accordance with a third aspect of the present invention, there is provided a motion picture encoding device which selects either intra-frame coding or inter-frame coding and performs video compression, comprising: an error estimation means for estimating the magnitude of quantization error of an entire macroblock image to encode using an assessment value obtained by a motion vector search at the time of inter-frame prediction; a detection means for detecting flatness and quantization error for each block image when performing inter-frame coding for a block image of a partitioned macroblock image; a first coding determination means for judging the existence of a block image having visually noticeable noise from the comparison of flatness and quantization error detected by the detection means when the estimated quantization error by the error estimation means is less than a predetermined value and selecting intra-frame coding if an applicable block image exists; and a second coding determination means for judging the existence of a block image having visually noticeable noise from the estimated quantization error when the estimated quantization error by the error estimation means is greater than a predetermined value and selecting intra-frame coding if an applicable block image exists.

0036In accordance with a fourth aspect of the present invention, there is provided a motion picture encoding device which selects either intra-frame coding or inter-frame coding and performs video compression, comprising: an error estimation means for estimating the magnitude of quantization error of an entire macroblock image to encode using an assessment value obtained by a motion vector search at the time of inter-frame prediction; a first coding determination means for detecting flatness and quantization error for each block image when performing inter-frame coding by a block image of a partitioned macroblock image, judging the existence of a block image having visually noticeable noise from the comparison of detected flatness and quantization error when the estimated quantization error by the error estimation means is less than a predetermined value and selecting intra-frame coding if an applicable block image exists; and a second coding determination means for judging the existence of a block image having visually noticeable noise from an estimated quantization error when the estimated quantization error by the error estimation means is greater than a predetermined value and selecting intra-frame coding if an applicable block image exists.

0037In accordance with a fifth aspect of the present invention, there is provided a motion picture encoding device which selects either intra-frame coding or inter-frame coding and performs video compression, comprising: an error estimation means for estimating the magnitude of quantization error of an entire macroblock image to encode using an assessment value obtained by a motion vector search at the time of inter-frame prediction; a first coding determination means for calculating flatness of an entire macroblock image and judging the flatness as to whether or not there is visually noticeable noise when the estimated quantization error by the error estimation means is less than a predetermined value, wherein if there is no visually noticeable noise selects inter-frame coding and conversely, if there is visually noticeable noise selects intra-frame coding; and a second coding determination means for judging whether or not visually noticeable noise exists in a macroblock image from an estimated quantization error when the estimated quantization error by the error estimation means is greater than a predetermined value, wherein if there is no visually noticeable noise selects inter-frame coding and conversely, if there is visually noticeable noise selects intra-frame coding.

0038In accordance with a sixth aspect of the present invention, the second coding determination means judges existence of a block image having visually noticeable noise while considering the magnitude of a motion vector obtained by inter-frame prediction.

0039In accordance with a seventh aspect of the present invention, an assessment value obtained by a motion vector search at the time of the inter-frame prediction is the Sum of Absolute Differences when detecting a motion vector with the highest correlation.

0040In accordance with an eighth aspect of the present invention, there is provided a motion picture encoding device which selects either intra-frame coding or inter-frame coding and performs video compression, comprising: a detection means for detecting the magnitude of quantization error of an entire macroblock image, as well as detecting flatness and quantization error for each block image when performing inter-frame coding by a block image of a partitioned macroblock image; a first coding determination for judging the existence of a block image having visually noticeable noise from the comparison of detected flatness and quantization error when a quantization error of an entire macroblock image detected by the detection means is less than a predetermined value and selecting intra-frame coding if an applicable block image exists; and a second coding determination means for judging the existence of a block image having visually noticeable noise while considering the magnitude of a motion vector obtained by inter-frame prediction when the quantization error of an entire macroblock image detected by the detection means is greater than a predetermined value and selecting intra-frame coding if an applicable block image exists.

0041In accordance with a ninth aspect of the present invention, there is provided a motion picture encoding processing program which selects either intra-frame coding or inter-frame coding and performs video compression, wherein the motion picture encoding processing program executes encoding determination processing by a processor, comprising the steps of: judging the existence of a block image having visually noticeable noise when performing inter-frame coding by a block image of a partitioned macroblock image to encode; and selecting intra-frame coding if an applicable block image exists.

0042In accordance with a tenth aspect of the present invention, there is provided a motion picture encoding processing program which selects either intra-frame coding or inter-frame coding and performs video compression, wherein the motion picture encoding processing program executes encoding determination processing by a processor, comprising the steps of: a first encoding determination process for judging the existence of a block image having visually noticeable noise when performing inter-frame coding by a block image of a partitioned macroblock image to encode and selecting intra-frame coding if an applicable block image exists; and a second encoding determination process for judging the existence of a block image having visually noticeable noise while considering the magnitude of a motion vector obtained by inter-frame prediction and selecting intra-frame coding if an applicable block image exists.

0043In accordance with an eleventh aspect of the present invention, there is provided a motion picture encoding processing program which selects either intra-frame coding or inter-frame coding and performs video compression, wherein the motion picture encoding processing program executes encoding determination processing by a processor, comprising the steps of: an error estimation process for estimating the magnitude of quantization error of an entire macroblock image to encode using an assessment value obtained by a motion vector search at the time of inter-frame prediction; a detection process for detecting flatness and quantization error for each block image when performing inter-frame coding by a block image of a partitioned macroblock image; a first encoding determination process for judging the existence of a block image having visually noticeable noise from the comparison of flatness and quantization error detected by the detection process when the estimated quantization error by the error estimation process is less than a predetermined value and selecting intra-frame coding if an applicable block image exists; and a second encoding determination process for judging the existence of a block image having visually noticeable noise from an estimated quantization error when the estimated quantization error by the error estimation process is greater than a predetermined value and selecting intra-frame coding if an applicable block image exists.

0044In accordance with a twelfth aspect of the present invention, there is provided a motion picture encoding processing program which selects either intra-frame coding or inter-frame coding and performs video compression, wherein the motion picture encoding processing program executes encoding determination processing by a processor, comprising the steps of: error estimation process for estimating the magnitude of quantization error of an entire macroblock image to encode using an assessment value obtained by a motion vector search at the time of inter-frame prediction; a first encoding determination process for detecting flatness and quantization error for each block image when performing inter-frame coding by a block image of a partitioned macroblock image, judging the existence of a block image having visually noticeable noise from the comparison of detected flatness and quantization error when the estimated quantization error by the error estimation process is less than a predetermined value and selecting intra-frame coding if an applicable block image exists; and a second encoding determination process for judging the existence of a block image having visually noticeable noise from the estimated quantization error when the estimated quantization error by the error estimation process is greater than a predetermined value and selecting intra-frame coding if an applicable block image exists.

0045In accordance with a thirteenth aspect of the present invention, there is provided a motion picture encoding processing program which selects either intra-frame coding or inter-frame coding and performs video compression, wherein the motion picture encoding processing program executes encoding determination processing by a processor, comprising the steps of: an error estimation process for estimating the magnitude of quantization error of an entire macroblock image to encode using an assessment value obtained by a motion vector search at the time of inter-frame prediction; a first encoding determination process for calculating flatness of an entire macroblock image and judging the flatness as to whether or not there is visually noticeable noise when the estimated quantization error by the error estimation process is less than a predetermined value, wherein if there is no visually noticeable noise selects inter-frame coding and conversely, if there is visually noticeable noise selects intra-frame coding; and a second encoding determination process for judging whether or not visually noticeable noise exists in a macroblock image from an estimated quantization error when the estimated quantization error by the error estimation process is greater than a predetermined value, wherein if there is no visually noticeable noise selects inter-frame coding and conversely, if there is visually noticeable noise selects intra-frame coding.

0046In accordance with a fourteenth aspect of the present invention, the second encoding determination process judges existence of a block image having visually noticeable noise while considering the magnitude of a motion vector obtained by inter-frame prediction.

0047In accordance with a fifteenth aspect of the present invention, an assessment value obtained by a motion vector search at the time of the inter-frame prediction is the Sum of Absolute Differences when detecting a motion vector with the highest correlation.

0048In accordance with a sixteenth aspect of the present invention, there is provided a motion picture encoding processing program which selects either intra-frame coding or inter-frame coding and performs video compression, wherein the motion picture encoding processing program executes encoding determination processing by a processor, comprising the steps of: a detection process for detecting the magnitude of quantization error of an entire macroblock image, as well as flatness and quantization error for each block image when performing inter-frame coding by a block image of a partitioned macroblock image; a first encoding determination process for judging the existence of a block image having visually noticeable noise from the comparison of detected flatness and quantization error when the quantization error of an entire macroblock image detected by the detection process is less than a predetermined value and selecting intra-frame coding if an applicable block image exists; and a second encoding determination process for judging the existence of a block image having visually noticeable noise while considering the magnitude of a motion vector obtained by inter-frame prediction when a quantization error of the entire macroblock image detected by the detection process is greater than a predetermined value and selecting intra-frame coding if an applicable block image exists.

0049According to the first and ninth aspects of the present invention, when performing inter-frame coding by a block image of a partitioned macroblock image to encode intra-frame coding is selected if a block image has visually noticeable noise. Conventionally, although the frequency of usage is low, the immense number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are executed for coding determination can be omitted, whereby wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0050Also, because intra-frame coding is immediately selected if there is visually noticeable noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0051According to the second and tenth aspects of the present invention, when performing inter-frame coding by a block image of a partitioned macroblock image to encode intra-frame coding is selected if a block image has visually noticeable noise and intra-frame coding is selected if a block image has visually noticeable noise while considering the magnitude of a motion vector obtained by inter-frame prediction. Conventionally, although the frequency of usage is low, the excessive number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are performed for coding determination can be omitted, whereby wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0052Also, because intra-frame coding is immediately selected if there is visually noticeable noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0053According to the third and eleventh aspects of the present invention, the magnitude of quantization error of the entire macroblock image to encode is estimated using the assessment value obtained by a motion vector search at the time of inter-frame prediction, and flatness and quantization error for each block image are detected when performing inter-frame coding for a block image of a partitioned macroblock image. Then, existence of a block image having visually noticeable noise is judged from the detected comparison of flatness and quantization error when the estimated quantization error is less than a predetermined value and intra-frame coding is selected if an applicable block image exists.

0054Meanwhile, existence of a block image having visually noticeable noise is judged from an estimated quantization error when the estimated quantization error is greater than a predetermined value and intra-frame coding is selected if an applicable block image exists.

0055Accordingly, even though the frequency of usage has been low in the past, the excessive number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are performed for coding determination can be omitted, whereby wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0056Also, because intra-frame coding is immediately selected if there is visually noticeable, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0057According to the fourth and twelfth aspects of the present invention, the magnitude of quantization error of the entire macroblock image to encode is estimated using the assessment value obtained by a motion vector search at the time of inter-frame prediction, as well as flatness and quantization error for each block image are detected when performing inter-frame coding for a block image of a partitioned macroblock image. Then, existence of a block image having visually noticeable noise is judged from the comparison of detected flatness and quantization error when the estimated quantization error is less than a predetermined value and intra-frame coding is selected if an applicable block image exists.

0058Meanwhile, existence of a block image having visually noticeable noise is judged from an estimated quantization error when the estimated quantization error is greater than a predetermined value and intra-frame coding is selected if an applicable block image exists.

0059Accordingly, even though the frequency of usage has been low in the past, the excessive number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are performed for coding determination can be omitted, whereby wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0060Also, because intra-frame coding is immediately selected if there is visually noticeable noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0061According to the fifth and thirteenth aspects of the present invention, the magnitude of quantization error of the entire macroblock image to encode is estimated using the assessment value obtained by a motion vector search at the time of inter-frame prediction, flatness is calculated for an entire macroblock image and the flatness is judged as to whether or not there is visually noticeable noise when the estimated quantization error is less than a predetermined value, wherein if there is no visually noticeable noise selects inter-frame coding and conversely, if there is visually noticeable noise selects intra-frame coding.

0062Meanwhile, visually noticeable noise is judged as to whether or not present in the macroblock image from the estimated quantization error when the estimated quantization error is greater than a predetermined value, wherein if there is no visually noticeable noise selects inter-frame coding and conversely, if there is visually noticeable noise selects intra-frame coding.

0063Accordingly, even though the frequency of usage has been low in the past, the excessive number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are performed for coding determination can be omitted, whereby wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0064Also, because intra-frame coding is immediately selected if there is visually noticeable noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0065According to the seventh and fifteenth aspects of the present invention, as the assessment value obtained by a motion vector search at the time of the inter-frame prediction, the Sum of Absolute Differences is used when detecting a motion vector with the highest correlation. As a result, because the magnitude of quantization error of an entire macroblock image to encode is estimated, the operations which calculate the quantization error of an entire macroblock image can be omitted.

0066According to the eighth and sixteenth aspects of the present invention, the magnitude of quantization error of the entire macroblock image is detected, as well as flatness and quantization error for each block are detected when performing inter-frame coding by a block image of a partitioned macroblock image. Then, existence of a block image having visually noticeable noise is judged from the comparison of detected flatness and quantization error when the quantization error is less than a predetermined value and intra-frame coding is selected if an applicable block image exists.

0067Meanwhile, existence of a block image having visually noticeable noise is judged while considering the magnitude of a motion vector obtained by inter-frame prediction when the quantization error of the entire macroblock image detected by the detection means is greater than a predetermined value and selecting intra-frame coding if an applicable block image exists.

0068Accordingly, even though the frequency of usage has been low in the past, the excessive number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are performed for coding determination can be omitted, whereby wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0069Also, because intra-frame coding is immediately selected if there is visually noticeable noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0070Human visual characteristics are categorized into a spatial visual sense characteristic and a temporal visual sense characteristic in relation to space and time, respectively. As for the spatial visual characteristic, it is known that sensitivity relative to gradation is higher than for resolution in flat portions of an image and conversely, sensitivity relative to resolution is higher than for gradation in a non-flat portions. Accordingly, when video is regarded as a sequence of still images, it is necessary to take into consideration the spatial visual sense characteristic.

0071Incidentally, since a quantization error is produced when encoding and added to a prediction image after inverse DCT is performed, the quantization error will ultimately remain in each pixel of the decoded image obtained by adding a decoding differential signal to the prediction image. When the difference value for each pixel produced in the process of encoding and decoding is added to each block, this reflects the amount of noise generated in these blocks.

0072As mentioned above, sensitivity of gradation is higher than for resolution in flat portions of an image. When the amount of noise is compared with commensurate blocks, the noise becomes visually noticeable to the extent that flatness of the image is higher. Specifically, although the noise in flat portions, such as, “human skin”, “sky”, “ground surface”, etc. is noticeable, noise in flat portions, such as “audience seating”, “forest”, etc. is not noticeable. Accordingly, it is possible to estimate whether or not there is visually noticeable noise generated in a block by detecting the amount of noise and flatness in each block.

0073Based on such knowledge, the present invention judges whether or not visually noticeable noise exists in the current macroblock to encode and intra-frame coding is performed only if visually noticeable noise exists. In this manner, wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics. Moreover, as described later, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0074The above and further novel features of the present invention will more fully appear from the following detailed description when the same is read in conjunction with the accompanying drawings. It is to be expressly understood, however, that the drawings are for the purpose of illustration only and are not intended as a definition of the limits of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

0075<figref idref="DRAWINGS">FIG. 1</figref> is a block diagram showing the configuration of the first preferred embodiment of the present invention;

0076<figref idref="DRAWINGS">FIG. 2</figref> is a graph for illustrating the corresponding comparison between the minSAD value (x-axis) of an ordinary image, for example, a broadcast image, and the quantization error total amount Σ Noise (y-axis);

0077<figref idref="DRAWINGS">FIG. 3</figref> is a plotted graph showing macroblock flatness Flat/minSAD values with visually noticeable noise and macroblock flatness Flat/minSAD values without visually noticeable noise;

0078<figref idref="DRAWINGS">FIG. 4</figref> is a flow chart showing operations of the encoding determination processing according to the first preferred embodiment;

0079<figref idref="DRAWINGS">FIG. 5</figref> is a flow chart showing operations of the inter D&amp;Q processing according to the first preferred embodiment;

0080<figref idref="DRAWINGS">FIG. 6</figref> is a flow chart showing operations of the encoding determination processing according to the second preferred embodiment;

0081<figref idref="DRAWINGS">FIG. 7</figref> is a flow chart showing operations of the encoding determination processing according to the third preferred embodiment;

0082<figref idref="DRAWINGS">FIG. 8</figref> is a flow chart showing operations of the encoding determination processing according to the fourth preferred embodiment;

0083<figref idref="DRAWINGS">FIG. 9</figref> is a block diagram showing the configuration of a motion picture encoding device of a conventional prior art example;

0084<figref idref="DRAWINGS">FIG. 10</figref> is a flow chart showing operations of the encoding determination processing of a conventional prior art example;

0085<figref idref="DRAWINGS">FIG. 11</figref> is a flow chart showing operations of the inter-prediction processing of a conventional prior art example;

0086<figref idref="DRAWINGS">FIG. 12</figref> is a flow chart showing operations of the MV search processing of a conventional prior art example;

0087<figref idref="DRAWINGS">FIG. 13</figref> is a diagram showing the association between a 16×16 pixel macroblock and a 4×4 pixel block;

0088<figref idref="DRAWINGS">FIG. 14</figref> is a flow chart showing operations of the intra-prediction processing of a conventional prior art example; and

0089<figref idref="DRAWINGS">FIG. 15</figref> is a flow chart showing operations of the inter D&amp;Q processing of a conventional prior art example.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

0090Hereinafter, the preferred embodiments of the present invention will be described in detail with reference to the drawings.

A. First Preferred Embodiment

0091A-1. Configuration

0092<figref idref="DRAWINGS">FIG. 1</figref> is a block diagram showing the outline configuration of a motion picture encoding device according to the first preferred embodiment. In the motion picture encoding device of <figref idref="DRAWINGS">FIG. 1</figref>, the same nomenclature is appended to the constituent elements which are common in the conventional prior art example shown in <figref idref="DRAWINGS">FIG. 9</figref> and explanation is omitted.

0093The differences between the motion picture encoding device shown in <figref idref="DRAWINGS">FIG. 1</figref> and the conventional prior art example in <figref idref="DRAWINGS">FIG. 9</figref> are that a flatness calculation section <b>30</b> and a noise calculation section <b>40</b> are newly provided, and the determination section <b>21</b> selects either intra-frame coding or inter-frame coding according to each output of the flatness calculation section <b>30</b> and the noise calculation section <b>40</b>.

0094The flatness calculation section <b>30</b> calculates flatness by applying the well-known Hadamard transform (also known as Walsh-Hadamard transform) to the macroblock image (the current macroblock) inputted as a coding object. Specifically, the sum absolute value of the harmonic content coefficient of a Hadamard coefficient is obtained by performing Hadamard transform on each of the 4×4 pixel blocks of a 16×16 pixel current macroblock partitioned into 16 sub-macroblocks to calculate the flatness for each block as Flat(<b>0</b>)˜Flat (<b>15</b>). Furthermore, the current macroblock flatness Flat is obtained from the sum total for each of these blocks Flat(<b>0</b>)˜Flat (<b>15</b>).

0095In addition, flatness Flat is calculated by 4×4 pixel blocks based on the following three reasons:

00961) There are predominantly a lesser number of calculations in the way of performing Hadamard transform 16 times for 4×4 pixel blocks rather than for performing Hadamard transform for a 16×16 pixel macroblock.

00972) As for 4×4 pixel Hadamard transform, sufficient resolution can be obtained in the flatness assessment that the present invention intends although square wave resolution drops to ¼ as compared with 16×16 Hadamard transform.

00983) Since integer DCT is used for 4×4 pixel sub-blocks in orthogonal transform of the H.264 standard, the calculation buffer which temporarily stores the sub-block pixel values can also be shared for use in flatness calculation.

0099The noise calculation section <b>40</b> extracts a quantization error relative to overlapping pixel values and generates noise Noise(<b>0</b>)˜(<b>15</b>) for each block. The quantization error for each pixel is calculated from difference value An (i,j) of the current block pixel value Org (i,j) and the reference image (motion compensated locally decoded image) pixel value ref (i,j), and from difference Zn (i,j) with difference value A′n (i,j) after these are locally decoded. The sum absolute value difference Zn (i,j) becomes noise Noise(<b>0</b>)˜(<b>15</b>) for each 4×4 pixel block.

0100In the determination section <b>21</b>, an error status determination is performed using a minimum Sum of Absolute Differences minSAD (hereinafter, denoted as “minSAD value”) obtained by the motion vector search at the time of inter-prediction as a substitute for the quantization error total amount Σ Noise (noise Noise(<b>0</b>)˜(<b>15</b>) sum total). A first and second coding determination is performed corresponding to the result of this error status determination.

0101The error status determination judges whether or not the minSAD value of the current macroblock as the coding object is greater than a threshold value TH<b>1</b> (described later). When the minSAD value is less than the threshold value TH<b>1</b> (referred to as “lower” below) according to this error status determination, a first coding determination (described later) is performed. Conversely, when the minSAD is greater than the threshold value TH<b>1</b> (referred to as “higher” below), a second coding determination (described later) is performed.

0102The quantization error total amount Σ Noise will be explained with reference to <figref idref="DRAWINGS">FIG. 2</figref> concerning the reason for which the minSAD value can be substituted. <figref idref="DRAWINGS">FIG. 2</figref> is a graph for illustrating the comparison between the minSAD value (x-axis) and the quantization error total amount Σ Noise (y-axis) obtained in a 16×16 pixel macroblock of an ordinary image, such as a broadcast image, etc.

0103As shown in this graph, the minSAD value acquires the value which invariably exceeds the quantization error total amount Σ Noise bordering on the linear function K<b>1</b>. Therefore, even when the minSAD value is substituted for the quantization error total amount Σ Noise, there is no possibility of overlooking an actual quantization error and proves that a general error difference determination can be sufficiently tolerated. Moreover, an increase in the number of calculations is also avoidable by substituting the minSAD value for the quantization error total amount Σ Noise.

0104However, even when the quantization error (minSAD value) is judged to be lower by the error status determination of the determination section <b>21</b>, visually noticeable noise exists in some pixels of the macroblock and the possibility remains that this may be the result of a quantization error with other pixels in an equally lower positions.

0105Therefore, in the determination section <b>21</b>, a first coding determination is performed which takes into consideration both noise “Noise” and flatness “Flat” extracted by a 4×4 pixel block of a macroblock partitioned into 16 sub-macroblocks. Although these details will be described later, in the first coding determination, the existence of visually noticeable noise is judged from noise “Noise” and flatness “Flat”, and intra-frame coding is selected if visually noticeable noise exists.

0106On the other hand, when the quantization error (minSAD value) is judged to be higher by the error status determination, the quantization error may have been generated not only in some of the pixels but in the entire macroblock. Therefore, in the determination section <b>21</b>, a second coding determination is performed is performed which takes into consideration the “minSAD value”, flatness “Flat” and motion vector by a 16×16 pixel macroblock.

0107Here, an outline of the second coding determination will be explained with reference to <figref idref="DRAWINGS">FIG. 3</figref>. <figref idref="DRAWINGS">FIG. 3</figref> is a plotted graph showing the flatness Flat and the minSAD value of macroblocks (black squares in graph) with visually noticeable noise and the flatness Flat and the minSAD value of macroblocks (white circles in graph) without visually noticeable noise, respectively.

0108The graph shows that in the area greater than the straight dashed line A drawn in the vicinity of minSAD=“600”, it is possible to discriminate whether or not there is visually noticeable noise in the extended area using a linear approximation function shown by the straight line B. In addition, the straight dashed line A value is equivalent to the threshold value TH<b>1</b> used for the error status determination mentioned above.

0109Accordingly, the second coding determination judges whether or not the flatness Flat and the minSAD value of the current macroblock are in the upper side area of the straight line B (extended area with visually noticeable noise), and intra-frame coding is selected when in the upper side area of the straight line B.

0110Incidentally, in cases where the motion vector is large due to movement of an object, camera pan, etc., the current macroblock becomes a blurred image and the quantization error indicates no visually noticeable noise. Nevertheless, there is a tendency for the minSAD value to become higher.

0111Consequently, the second coding determination judges whether or not to perform intra-frame coding based on the following formula [1] from the fact that there is a necessity to adaptively change the determination contents mentioned above in proportion to the motion vector (u, v) of the current macroblock (Note the symbol “x” denotes multiplication and has the same meaning as the symbol “*” (asterisk) also used for multiplication.) <br />minSAD>(1<i>+a×f</i>(<i>mv</i>))×<i>k</i>×Flat  [1]

0112Additionally, in formula [1] the coefficient a is constant to adjust the contributive factor of the motion vector, and the coefficient k is constant equivalent to the inclination of the straight line B mentioned above. Function f (mv) is the magnitude (u²+v²) of the current macroblock motion vector (u, v).

0113A-2. Operations

0114Next, the encoding determination operations of the first preferred embodiment according to the above-mentioned configuration will be explained with reference to <figref idref="DRAWINGS">FIG. 4˜FIG</figref>. <b>5</b>.

0115<Operations of the Encoding Determination Processing>

0116<figref idref="DRAWINGS">FIG. 4</figref> is a flow chart showing operations of the encoding determination processing executed for each input macroblock. When processing is performed corresponding to an input macroblock, after initializing each section of the device in Step SA<b>1</b>, the flow advances to Step SA<b>2</b> and inter-prediction processing to estimate the amount of coded data at the time of inter-frame predictive coding is executed. This inter-prediction processing is same as that of the conventional prior art example shown in <figref idref="DRAWINGS">FIG. 11</figref>. MV search processing is performed to search for a motion vector among all of the 4×4 pixel blocks of the 16×16 pixel current macroblock partitioned into 16 sub-macroblocks and the minSAD value for each block is obtained.

0117Subsequently, in Step SA<b>3</b>, inter D&amp;Q processing to perform inter-frame coding (integer DCT, quantization, inverse quantization and inverse integer DCT) is executed. Next, in Step SA<b>4</b>, whether or not the minSAD value obtained in the inter-prediction processing of the above-mentioned Step SA<b>2</b> exceeds the threshold value TH<b>1</b> is judged. Accordingly, the above-mentioned error status determination is executed.

0118When the quantization error in the error status determination is judged to be lower, the judgment result becomes “NO” and the flow advances to Step SA<b>5</b>. In Step SA<b>5</b>, whether or not a flag INTRA_FLG (described later) has been set to “1” is judged. The flag INTRA_FLG is a flag generated by inter D&amp;Q processing (described later), which is set to “1” if visually noticeable noise exists in the current macroblock. Accordingly, this Step SA<b>5</b> is equivalent to the first coding determination mentioned above.

0119Because of the flag INTRA_FLG being set to “1” if visually noticeable noise exists in the current macroblock, the judgment result of Step SA<b>5</b> becomes “YES”. The flow advances to Step SA<b>7</b> and intra D&amp;Q processing to perform intra-frame coding is executed. Meanwhile, if visually noticeable noise does not exist in the current macroblock, the flag INTRA_FLG will not be set to “1”. The judgment result of Step SA<b>5</b> becomes “NO” and this processing is completed.

0120Conversely, in the error status determination of the above-mentioned Step SA<b>4</b>, when the quantization error is judged to be higher, the judgment result becomes “YES” and the flow advances to Step SA<b>6</b>. In Step SA<b>6</b>, the second coding determination is performed based on the criterion expressed in formula [1] stated above. Namely, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector. If visually noticeable noise exists, the judgment result becomes “YES,” the flow advances to Step SA<b>7</b> and executes intra D&amp;Q processing. Conversely, if visually noticeable noise does not exist, the judgment result becomes “NO” and this processing is completed.

0121<Operations of the Inter D&Q Processing>

0122Next, the operations of the inter D&amp;Q processing will be explained with reference to <figref idref="DRAWINGS">FIG. 5</figref>. When this processing is executed via Step SA<b>3</b> (refer to <figref idref="DRAWINGS">FIG. 4</figref>), the flow advances to Step SB<b>1</b> shown in <figref idref="DRAWINGS">FIG. 5</figref> and whether or not processing has been completed for all of the 4×4 pixel blocks of a 16×16 pixel current macroblock partitioned into 16 sub-macroblocks is judged. When processing has been completed for all the blocks, the judgment result becomes “YES” and this processing is completed. Otherwise, the judgment result becomes “NO” and the flow advances to Step SB<b>2</b>. In Step SB<b>2</b>, flatness Flat (n) of the current block is calculated from the sum absolute value of the harmonic content coefficient of a Hadamard coefficient obtained by Hadamard transform for each block.

0123Next, in Steps SB<b>3</b>˜SB<b>7</b>, the inter-frame prediction block value ref (i, j) (reference image pixel value) is subtracted from the current block pixel value Org (i, j) and then transform processing (integer DCT), quantization processing Q, inverse quantization processing Q-<b>1</b> and inverse transform processing (inverse integer DCT⁻¹) are applied to the generated prediction error signal (difference value An (i, j)). Subsequently, in Step SB<b>8</b>, the quantization error relative to overlapping pixel values is extracted from the above-mentioned difference value An (i, j), as well as the difference Zn (i, j) with the difference value A′n (i, j) after these are locally decoded. Further, the sum absolute value difference Zn (i, j) is calculated for the current block noise Noise (n). Also, in Step SB<b>8</b>, the quantization error maximum value max Noise (n) within the 4×4 pixels that constitute the current block is detected. Then, in Step SB<b>9</b>, stepping of the inter-frame prediction block value ref (i, j) is performed.

0124Next, in Step SB<b>10</b>, the above-mentioned first coding determination is executed for judging the existence of visually noticeable noise, namely, the current block noise Noise value (n) is greater than a threshold value TH<b>2</b>, the current block flatness Flat (n) is less than a threshold value TH<b>3</b> and the quantization error maximum value max Noise (n) is less than a threshold value TH<b>10</b>.

0125If visually noticeable noise exists, the judgment result becomes “YES” and the flow advances to Step SB<b>11</b>. After setting the flag INTRA_FLG to “1”, processing reverts to Step SB<b>1</b>. Conversely, when no visually noticeable noise exists, the judgment result becomes “NO” and processing reverts to Step SB<b>1</b>.

0126Thus, in this first preferred embodiment, the minSAD value obtained by the motion vector search at the time of inter-prediction is substituted as the assessment value which represents the quantization error total amount Σ Noise for the current macroblock made into a coding object. The error status determination roughly judges the magnitude of quantization error by whether or not this minSAD exceeds the threshold TH<b>1</b>. Subsequently, when the quantization error in this error status determination is deemed to be lower, whether or not there is visually noticeable noise in some pixels of the current macroblock is judged based on flatness Flat and noise Noise detected in each of the 4×4 pixel blocks of the current macroblock partitioned into 16 sub-macroblocks. If there is visually noticeable noise, intra-frame coding is selected. Meanwhile, when the quantization error in this error status determination is deemed to be higher, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector. If there is visually noticeable noise, intra-frame coding is selected.

0127Accordingly, although the former frequency of usage is low, the immense number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are executed for coding determination can be omitted. As a result, wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0128Also, because intra-frame coding is immediately selected when judged there is visually noticeable noise in the current macroblock based on flatness Flat and noise Noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

B. Second Preferred Embodiment

0129Next, the operations of the encoding determination processing according to the second preferred embodiment will be explained with reference to <figref idref="DRAWINGS">FIG. 6</figref>. When processing is performed corresponding to an input macroblock similar to the above-mentioned first preferred embodiment and after initializing each section of the device in Step SC<b>1</b>, the flow advances to Step SC<b>2</b> and inter-prediction processing to estimate the amount of coded data at the time of inter-frame prediction processing coding is executed. The minSAD for each of the 4×4 pixel blocks of the 16×16 pixel current macroblock partitioned into 16 sub-macroblocks is obtained.

0130Subsequently, in Step SC<b>3</b>, an error status determination is performed to judge whether or not the minSAD value obtained in the inter-prediction processing of the above-mentioned Step SC<b>2</b> exceeds the threshold value TH<b>1</b> and after all the quantization error is lower. When the quantization error is judged to be lower, the judgment result becomes “NO” and the flow advances to Step SC<b>4</b>. In Step SC<b>4</b>, while performing inter-frame coding, whether or not visually noticeable noise exists in some of the pixels of the current macroblock is judged based on flatness Flat and noise Noise detected in each of the 4×4 pixel blocks. If there is visually noticeable noise, inter D&amp;Q processing (refer to <figref idref="DRAWINGS">FIG. 5</figref>) which sets the flag INTRA_FLG to “1” is executed.

0131In Step SC<b>5</b>, whether or not visually noticeable noise has been detected in the inter D&amp;Q processing of the above-mentioned Step SC<b>4</b> is judged, namely, whether or not the flag INTRA_FLG has been set to “1”. If visually noticeable noise has been detected, the judgment result becomes “YES”. The flow advances to Step SC<b>6</b> and the intra D&amp;Q processing to perform intra-frame coding is executed. Conversely, if visually noticeable noise has not been detected, the judgment result of Step SC<b>5</b> becomes “NO” and this processing is completed.

0132Meanwhile, when the quantization error is judged to be higher, the judgment result of the above-mentioned Step SC<b>3</b> becomes “YES” and the flow advances to Step SC<b>7</b>. In Step SC<b>7</b>, similar to the first preferred embodiment, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector based on the criterion expressed in above-mentioned formula [1]. If visually noticeable noise does not exist in the current macroblock, the judgment result becomes “NO” and the flow advances to Step SC<b>8</b>.

0133In Step SC<b>8</b>, inter D&amp;Q processing (refer to <figref idref="DRAWINGS">FIG. 15</figref>) to perform inter-frame coding is executed. Meanwhile, if visually noticeable noise does exist in the current macroblock, the judgment result of the above-mentioned Step SC<b>7</b> becomes “YES”. The flow advances to Step SC<b>6</b> and intra-frame D&amp;Q processing is executed to perform intra-frame coding and this processing is completed.

0134Thus, in the second preferred embodiment, the minSAD value obtained by the motion vector search at the time of inter-prediction is substituted as the assessment value which represents the quantization error total amount Σ Noise for the current macroblock made into a coding object. The error status determination roughly judges the magnitude of quantization error by whether or not this minSAD exceeds the threshold TH<b>1</b>. Subsequently, when the quantization error in this error status determination is deemed to be lower, whether or not there is visually noticeable noise in some pixels of the current macroblock is judged based on flatness Flat and noise Noise detected in each of the 4×4 pixel blocks of the current macroblock partitioned into 16 sub-macroblocks. If there is visually noticeable noise, intra-frame coding is selected.

0135Meanwhile, when the quantization error in this error status determination is deemed to be higher, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector. If there is no visually noticeable noise, inter-frame coding is selected. Conversely, if there is visually noticeable noise, intra-frame coding is selected.

0136Accordingly, similar to the first preferred embodiment, the immense number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are executed for coding determination can be omitted. As a result, wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics. Also, because intra-frame coding is immediately selected when judged there is visually noticeable noise in the current macroblock based on flatness Flat and noise Noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

C. Third Preferred Embodiment

0137Next, the operations of the encoding determination processing according to the third preferred embodiment will be explained with reference to <figref idref="DRAWINGS">FIG. 7</figref>. When processing is performed corresponding to an input macroblock similar to the above-mentioned first preferred embodiment and after initializing each section of the device in Step SD<b>1</b>, the flow advances to Step SD<b>2</b> and inter-prediction processing to estimate the amount of coded data at the time of inter-frame prediction processing coding is executed. The minSAD for each of the 4×4 pixel blocks of the 16×16 pixel current macroblock partitioned into 16 sub-macroblocks is obtained. Subsequently, in Step SD<b>3</b>, an error status determination is performed to judge whether or not the minSAD value obtained in the inter-prediction processing of the above-mentioned Step SD<b>2</b> exceeds the threshold value TH<b>1</b>, namely, the quantization error is lower.

0138When quantization error is judged to be lower by the error status determination, the judgment result becomes “NO” and the flow advances to Step SD<b>4</b>. In Step SD<b>4</b>, the current macroblock flatness Flat is calculated from the sum absolute value of the harmonic content coefficient of a Hadamard coefficient obtained by Hadamard transform for each 4×4 pixel block. Next, in Step SD<b>5</b>, whether or not the calculated flatness Flat is less than the threshold value TH<b>3</b> is judged, namely, flatness in which there is visually noticeable noise.

0139If the flatness Flat has no visually noticeable noise, the judgment result becomes “NO” and the flow advances to Step SD<b>8</b>, which executes inter D&amp;Q processing (refer to <figref idref="DRAWINGS">FIG. 15</figref>) to perform inter-frame coding (integer DCT, quantization, inverse quantization and inverse integer DCT). Conversely, if the flatness Flat has visually noticeable noise, the judgment result of the above-mentioned Step SD<b>5</b> becomes “YES” and the flow advances to Step SD<b>6</b> and executes intra D&amp;Q processing to perform intra-frame coding.

0140Meanwhile, when the quantization error is judged to be higher, the judgment result of the above-mentioned Step SD<b>3</b> becomes “YES” and the flow advances to Step SD<b>7</b>. In Step SD<b>7</b>, similar to the first preferred embodiment, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector based on the criterion expressed in the above-mentioned formula [1]. If visually noticeable noise does not exist in the current macroblock, the judgment result becomes “NO”, the flow advances to Step SD<b>8</b> and executes inter D&amp;Q processing (refer to <figref idref="DRAWINGS">FIG. 15</figref>) to perform inter-frame coding.

0141Conversely, if visually noticeable noise does exist in the current macroblock, the judgment result of the above-mentioned Step SD<b>7</b> becomes “YES”. The flow advances to Step SD<b>6</b> and executes inter D&amp;Q processing to perform intra-frame coding and this processing is completed.

0142Thus, in the third preferred embodiment, the minSAD value obtained by the motion vector search at the time of inter-prediction is substituted as the assessment value which represents the quantization error total amount Σ Noise for the current macroblock made into a coding object. The error status determination roughly judges the magnitude of quantization error by whether or not this minSAD exceeds the threshold TH<b>1</b>. Accordingly, when the quantization error is deemed to be lower, the current macroblock flatness Flat is calculated and whether or not there is visually noticeable noise is judged. If there is no visually noticeable noise, inter-frame coding is selected. Conversely, if there is visually noticeable noise, intra-frame coding is selected.

0143Meanwhile, when the quantization error is deemed to be higher, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector. If there is no visually noticeable noise, inter-frame coding is selected. Conversely, if there is visually noticeable noise, intra-frame coding is selected.

0144Accordingly, similar to the first preferred embodiment, the immense number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are executed for coding determination can be omitted. As a result, wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics. Furthermore, because intra-frame coding is immediately selected if there is visually noticeable noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

D. Fourth Preferred Embodiment

0145Next, the operations of the encoding determination processing according to the fourth preferred embodiment will be explained with reference to <figref idref="DRAWINGS">FIG. 8</figref>. When processing is performed corresponding to an input macroblock similar to the above-mentioned first preferred embodiment and after initializing each section of the device in Step SE<b>1</b>, the flow advances to Step SE<b>2</b> and inter-prediction processing to estimate the amount of coded data at the time of inter-frame predictive coding is executed. Next, in Step SE<b>3</b>, while performing inter-frame predictive coding, whether or not visually noticeable noise exists in some of the pixels of the current macroblock is judged based on flatness Flat and noise Noise detected in each of the 4×4 pixel blocks. If there is visually noticeable noise, inter D&amp;Q processing (refer to <figref idref="DRAWINGS">FIG. 5</figref>) which sets the flag INTRA_FLG to “1” is executed.

0146Next, in Step SE<b>4</b>, the sum total of noise Noise is obtained in the inter D&amp;Q processing of the above-mentioned Step SE<b>3</b>. Specifically, whether or not the quantization error total amount Σ Noise in the current macroblock exceeds the threshold value TH<b>1</b> is judged, namely, the quantization error total amount Σ Noise is lower. When the quantization error total amount Σ Noise is judged to be lower, the judgment result becomes “NO” and the flow advances to Step SE<b>5</b>. In Step SE<b>5</b>, whether or not virtually noticeable noise has been detected in the inter D&amp;Q processing of the above-mentioned Step SE<b>3</b> is judged, namely, whether or not the flag INTRA_FLG has been set to “1”. If visually noticeable noise has been detected, the judgment result becomes “YES”. The flow advances to Step SE<b>7</b> and inter D&amp;Q processing to perform intra-frame coding is executed. Conversely, if noise visually noticeable noise has not been detected, the judgment result of Step SE<b>5</b> becomes “NO” and this processing is completed.

0147Meanwhile, when the quantization error total amount Σ Noise is judged to be higher, the judgment result of the above-mentioned Step SE<b>4</b> becomes “YES” and the flow advances to Step SE<b>7</b>. In Step SE<b>7</b>, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector, based on the criterion expressed in the following formula [2]. <br />ΣNoise>(1<i>+a×f</i>(<i>mv</i>))×<i>k</i>×Flat  [2]

0148Additionally, in formula [2] the coefficient a is constant to adjust the contributive factor of the motion vector, and the coefficient k is constant equivalent to the inclination of the straight line B mentioned above. Function f (mv) is the size (u²+v²) of the current macroblock motion vector (u, v).

0149If there is visually noticeable noise, the judgment result of the above-mention Step SE<b>6</b> becomes “YES”. The flow advances to Step SE<b>7</b> and intra D&amp;Q processing to perform intra-frame coding is executed. Conversely, if there is no visually noticeable noise, the judgment result becomes “NO” and this processing is completed.

0150Thus, in the fourth preferred embodiment, the magnitude of quantization error of the current macroblock is judged by whether or not the quantization error total amount Σ Noise exceeds the threshold TH<b>1</b>. When the quantization error, is lower, whether or not there is visually noticeable noise in some pixels of the current macroblock is judged based on flatness Flat and noise Noise detected in each of the 4×4 pixel blocks of the current macroblock partitioned into 16 sub-macroblocks. If there is visually noticeable noise, intra-frame coding is selected.

0151Meanwhile, when the quantization error is deemed to be higher, whether or not visually noticeable noise exists in the current macroblock is judged while considering the magnitude of the motion vector. If there is visually noticeable noise, intra-frame coding is selected.

0152Accordingly, similar to the first preferred embodiment, the immense number of calculations in intra-prediction processing (refer to <figref idref="DRAWINGS">FIG. 14</figref>) which are executed for coding determination can be omitted. As a result, wasteful calculations will not be performed and coding modes can be determined at a faster speed in view of human visual characteristics.

0153Also, because intra-frame coding will be immediately selected when judged there is visually noticeable noise in the current macroblock based on flatness Flat and noise Noise, quantization error propagation when ‘trapped’ in a local minimum can be rapidly stopped to avoid reduction in image quality.

0154Furthermore, although the processing program of the motion picture encoding device which is a preferred embodiment of the present invention is stored in the memory (for example, ROM, etc.) of the motion picture encoding device, this processing program is stored on a computer-readable medium and should also be protected in the case of manufacturing, selling, etc. of only the program. In that case, the method of protecting the program with a patent will be realized by the form of the computer-readable medium on which the motion picture encoding processing program is stored.

0155While the present invention has been described with reference to the preferred embodiments, it is intended that the invention be not limited by any of the details of the description therein but includes all the embodiments which fall within the scope of the appended claims.