Apparatus, method, and program for restoring moving picture information

The present invention relates to a restoration process of digital image information used in compressed moving image data transmission and storage, reproduction technology, and the like.

When a video-encoded bitstream is transmitted and decoded via a wireless line or IP network, a bit error may occur in the bitstream due to line error or packet loss, and an error correction code with sufficient accuracy for the system is used. If it is not available, there is no guarantee that the video-encoded bitstream after the actual bit error will be decoded correctly. In a video-encoded bitstream that uses Huffman coding, if a bit error occurs in the received compressed data, there is a problem that it becomes an undefined codeword or changes to another Huffman codeword. sell. If it happens to change to an undefined code at the location where the bit error occurs, a bit error can be detected as a decoding failure, but if the bit error changes the original Huffman codeword to another codeword, the changed code. Since words can also be interpreted as codes that can occur, the decoding process may be continued while decoding information different from the originally encoded information even after the part where the bit error actually occurs. As a result, there is a problem that the actual bit error portion is often overlooked and decoding is continued, and the video quality in that range is greatly deteriorated.

As a countermeasure to this problem, conventional international standard video coding methods such as MPEG and ITU-T H.26x have one or more macroblocks in order to suppress quality deterioration due to the above-mentioned bit errors. The sequence of is defined as slices, video packets, GOB (Group of Blocks), etc., and the bitstream syntax is defined so that normal decoding synchronization is ensured in these units (hereinafter, MPEG-4 in the text). These units are collectively referred to by the term "video packet" used in the visual standard (ISO / IEC 14496-2), and one frame of bitstream data is generally a set of one or more video packets. The quality deterioration caused by a bit error in a certain video packet is localized in the video packet so that the video quality deterioration caused by the decoding synchronization shift does not spread over a wide range. However, in general, since the macroblock data contained in the video packet is variable-length coded, the quality deterioration due to the decoding synchronization shift in the video packet or the change to another code word is not avoided.

<nplcit num="1"><text>MPEG-4 Visual Standard (ISO / IEC 14496-2) Annex E</text></nplcit>

<p> Even if encoding / decoding is performed for each video packet, it is still not possible to exclude the cause of video distortion due to decoding synchronization shift that occurs in the video packet or change to another code word, and the resulting quality deterioration cannot be excluded. The only way to suppress it is to repair the image of the entire video packet. Since it is difficult to determine in advance which macroblock in a video packet has the actual bit error, the method of repairing the entire video packet was successful in decoding or visually. The decoded image in the range that can be judged to be normal will also be restored. In general, the portion repaired by information that is not encoded information is merely an approximation of the image data that should be decoded, and therefore image quality deterioration cannot be completely avoided.</p><p> That is, in the method of repairing the entire video packet, there is a risk that the image is repaired in a wider range than necessary, and the effect of image quality deterioration due to the risk is increased. In addition, in the repair from the position where the error is actually detected due to the decoding failure, there is a problem that the decoding synchronization loss that causes the disturbance of the image is overlooked and the deterioration cannot be completely repaired. Since it occurs in the decoding process that can occur in syntax analysis, even if you try to detect and repair it as a decoding error, there is a problem that there is a possibility of false detection and false repair that even other normal coded data is repaired as an error. It was.</p><p> An object of the present invention is to minimize the range in which image restoration is performed and to perform accurate image restoration.</p>

<p> In the present invention, based on the syntax of the moving image coded bit stream encoded in block units, a decoding failure state in which it is impossible to normally restore the coded data from the coded bit string is detected, and each block is detected. The presence or absence of image deterioration in the block is detected from the state of the coded data of, and when the decoding failure state is detected, the image restoration range is determined based on the position in the screen of the block detected as having image deterioration. It is intended to be determined.</p>

<p> According to the present invention, the image information is repaired by tracing back to the image quality deterioration portion only when a fatal error that causes a decoding failure is included during video packet decoding. Therefore, the range of image repair is the minimum necessary. It is possible to perform accurate restoration of image information.</p>

<u style="single">Embodiment 1.</u> First, the basic image coding / decoding process will be described. In international standard video coding methods such as MPEG and ITU-T H.26x, each frame of a video signal is a unit of block data (called a macro block) that summarizes the luminance signal 16x16 pixels and the corresponding color difference signal 8x8 pixels. As a method, a compression method is adopted based on motion compensation technology and orthogonal conversion / conversion coefficient quantization technology. Motion compensation technology is a technology that reduces the redundancy of the signal in the time direction for each macroblock by utilizing the high correlation existing between video frames, and the previously encoded frame is stored in the memory as a reference image. Accumulate and search the block area with the smallest difference power from the current macroblock that is the target of motion compensation prediction within the predetermined search range in the reference image, and search the spatial position and search result of the current macroblock. This is a technique for encoding the deviation from the spatial position of a block as a motion vector. Orthogonal transform / conversion coefficient quantization is a technology that reduces the redundancy of the image signal remaining in the spatial direction, and DCT (discrete cosine transform) is widely used for orthogonal transform due to the optimal power concentration of the conversion factor. Has been done. The conversion coefficient is quantized to reduce the amount of information and is encoded. These codings are done by using a predefined Huffman code. Huffman coding performs compression as a whole by expressing those with a high probability of occurrence with a short code and those with a low probability of occurrence with a long code.

Here, when a video-encoded bit stream is transmitted and decoded via a wireless line or an IP network, a bit error may occur in the bit stream due to a line error or packet loss, which is an error with sufficient accuracy as a system. If the correction code is not available, there is no guarantee that the video-encoded bitstream after the point where the bit error actually occurred will be decoded correctly. In a video-encoded bitstream that uses Huffman coding, if a bit error occurs in the received compressed data, there is a problem that it becomes an undefined codeword or changes to another Huffman codeword. sell. If it happens to change to an undefined code at the location where the bit error occurs, a bit error can be detected as a decoding failure, but if the bit error changes the original Huffman codeword to another codeword, the changed code. Since words can also be interpreted as codes that can occur, the decoding process may be continued while decoding information different from the originally encoded information even after the part where the bit error actually occurs. As a result, there is a problem that the actual bit error portion is often overlooked and decoding is continued, and the video quality in that range is greatly deteriorated.

As a countermeasure to this problem, conventional international standard video coding methods such as MPEG and ITU-T H.26x have one or more macroblocks in order to suppress quality deterioration due to the above-mentioned bit errors. A series of is defined as slices, video packets, GOBs (Group of Blocks), etc., and bitstream syntax is defined so that normal decoding synchronization is ensured in these units. Hereinafter, these units are collectively referred to in the text using the term "video packet" used in the MPEG-4 visual standard (ISO / IEC 14496-2). That is, one frame of bitstream data is generally a collection of one or more video packets. Figure 1 shows an example of a typical bitstream syntax for a video packet. A unique synchronization code in the bitstream, called a resynchronization marker, is added to the beginning. Even if the bitstream contains a bit error at a certain point, the subsequent decoding synchronization can be recovered by detecting the next resynchronization marker. Following that, video packet header information that multiplexes various information necessary to decode the macroblock in the video packet completely independently of the previous video packet is inserted, and then individual macroblock data. Is multiplexed. By adopting the syntax of such a configuration, the macroblock can be decoded independently without requiring the information of any other video packet beyond the resynchronization marker.

That is, the quality deterioration due to the bit error that occurs in a certain video packet is localized in the video packet, and it is possible to prevent the video quality deterioration due to the decoding synchronization shift from spreading over a wide range. However, in general, since the macroblock data contained in the video packet is variable-length coded, the quality deterioration due to the decoding synchronization shift in the video packet or the change to another code word is still unavoidable. Figure 2 shows the situation. In FIG. 2, the "macroblock in which abnormal data different from the original coded data is decoded" is data within the normal range in terms of bitstream syntax, but the original coded information is a bit error. It shows a case where it appears as a disorder of the image because it has changed due to. In the figure, it is clear that normal decoding cannot be continued after the "macroblock in which the decoding actually failed and an error was detected", so the bitstream data is read up to the macroblock immediately before the next video packet, and during that time. The image data of is being restored. For restoration of image data, a method of copying the image data at the same spatial position from the reference image used for motion compensation prediction by using the inter-frame correlation of the moving image is generally used. In addition, the scope of repair may be the entire video packet including "a macroblock in which decoding actually fails and an error is detected".

In the first embodiment according to the present invention, each frame image of the video is divided into video packet units, and the video packet is further subjected to moving image compression data composed of variable-length encoded macroblock data as input. A moving image decoding device that obtains a reproduced image while repairing (restoring moving image information) image quality deterioration due to an error that occurs in image compressed data will be described. The feature of the moving image decoding device in the present embodiment is that a macroblock that is likely to cause image quality deterioration in a video packet is inferred and its position is memorized, and a clear decoding failure occurs during decoding of the video packet. Only when it is detected, the image data is restored from the position of the macro block which is likely to cause the deterioration of the image quality.

FIG. 3 shows the configuration of the moving image decoding device according to the first embodiment. The moving image decoding device in the figure has a macroblock decoding processing unit 2, an image deterioration detection unit 7, an image restoration start position determination unit 9, an image restoration processing unit 11, a switch 4, and a frame memory 14. Further, FIG. 4 shows the internal configuration of the macroblock decoding processing unit 2, and the variable length decoding unit 16 in FIG. 2 is based on the syntax of the moving image coded bit stream encoded in block units. It has a function as a syntax analysis unit that detects a decoding failure state in which it is impossible to normally restore the coded data from the coded bit string. Further, FIG. 5 shows a processing flow of the moving image decoding apparatus of FIG. Hereinafter, the operation of the moving image decoding apparatus of the present embodiment will be described with reference to these figures.

<u style="single">(1) Variable length decoding process</u> The decoding device shown in FIG. 3 receives the moving image compressed data 1 for each video packet shown in FIG. 1, and the macroblock decoding processing unit 2 performs decoding processing on each macroblock data in the video packet. When the macroblock decoding processing unit 2 receives the moving image compressed data 1 for each video packet, the variable length decoding unit 16 first decodes the video packet header information (step S1). Although not shown, the video packet header information includes in-screen position information of the first macroblock of the video packet, a quantization parameter reference value of the macroblock included in the video packet, and the like. At this time, the decoding failure detection flag 12a is reset to zero, and the image deterioration detection position information described later is set to an initial value that does not correspond to any position in the screen. Then, the process of decoding individual macroblocks is started. In each macroblock decoding process, first, the variable length decoding unit 16 extracts the macroblock encoded data from the moving image compressed data 1 (step S2). The variable-length decoding unit 16 performs syntax analysis according to the coding standard of the input video compressed data 1, and extracts and restores the coded data of each variable-length coded macroblock. The macroblock coding data includes motion vector 17, coding mode information 18, orthogonal conversion coefficient data 6, quantization step parameter 19 for each macroblock, and the like.

In addition, the variable-length decoding unit 16 detects decoding failures that occur in these syntax analysis processing processes. Decryption failure refers to a state in which normal bitstream analysis cannot be continued until the decoding synchronization is restored again. Examples of decryption failures to be detected include the following.

-When an undefined code word or undefined decoded value that cannot occur in the bitstream syntax is detected in the moving image compression standard that the input data follows-Bitstream in the moving image compression standard that the input data follows When the data whose decoding value is fixedly determined on the syntax takes another value In the moving image compression standard that the input data follows, the value range of the decoding value on the bit stream syntax is unique to that data or When the data defined under the constraint of the decrypted value of another data deviates from the normal value range When the decoded data is clearly inconsistent in the light of the decoding process up to that point (for example) , When the in-screen position of the macro block at the beginning of a video packet is decoded and the value is discontinuous with the in-screen position of the previously decoded macro block, etc.)-The input data is If the number of similar data to be decoded is limited in the following moving image compression standard, and the number of data to be decoded exceeds that number (for example, MPEG-2, Since the orthogonal conversion (DCT) adopted in standards such as MPEG-4 is applied to 8x8 pixel blocks, the maximum orthogonal conversion coefficient is only 64 per block, but this number is 64 at the time of decoding. For example)

The variable-length decoding unit 16 switches the processing flow depending on whether or not the above-mentioned decoding failure is detected in the syntax analysis processing process (step S3). During the decoding process of the video packet, unless a decoding failure is detected, the processing (steps S4 to S8) described in (2) below is continuously executed for each macroblock, and the decoding failure is detected in any of the macroblocks. If this is the case, then the processes (steps S9 to S12) described in (3) below are executed. In the configuration of the decoding device shown in FIG. 3, based on the decoding failure detection flag 12a, the decoded image 3 generated by the following process (2) and the decoded image 3 generated by the following process (3) are generated depending on whether or not the decoding failure is detected. It is illustrated that the final decoded image 5 is determined by switching from the restored image 13 to be performed, but this is to generate both the decoded image 3 and the restored image 13 regardless of the presence or absence of the decoding failure detection. It does not mean that, but is illustrated in this way for the convenience of clearly describing the gist of the present invention. When the process of (2) below is executed, the processes of the image restoration start position determination unit 9 and the image restoration processing unit 11 among the components of FIG. 3 are not executed, and the restoration image 13 is not generated. In addition, when transitioning to the processing of (3) below, the inverse quantization unit 20, the inverse orthogonal conversion unit 21, and the motion compensation in the internal configuration of the macroblock decoding processing unit 2 in FIG. 4 are applied to the subsequent macroblocks. No processing is performed in part 23, and the decoded image 3 is not generated.

<u style="single">(2) Macroblock decoding process when no decoding failure is detected</u> First, processing (steps S4 to S8) in the case of a macroblock that does not detect a decoding failure will be described. If the variable-length decoding unit 16 does not detect the above-mentioned decoding failure, the decoding failure detection flag 12a remains zero. The specific value of the decryption failure detection flag 12a may be set as long as it can identify the presence or absence of the decryption failure. In this case, according to the processing flow of FIG. 5, the image deterioration detection unit 7 determines whether or not image deterioration is observed in the macroblock (step S4). What kind of state is judged to be image deterioration will be described in detail in (4) below.

When it is determined that there is image deterioration (step S5), first, inside the image deterioration detection unit 7, it is determined whether or not the macroblock is the first image deterioration detection in the video packet. Although this judgment is not explicitly shown in the processing flow, the image deterioration detection position information 8 becomes an initial value (for example, -1) that does not correspond to any in-screen position when image deterioration is detected. If it is a QCIF image that constitutes a one-frame screen with a value indicating a certain position in the screen (for example, 176 pixels × 144 lines (11 × 9 macroblocks)), any of 0 to 98. It can be judged by whether it is (value of). Only when the image deterioration is first detected in the video packet, the image deterioration detection unit 7 sets the in-screen position of the macroblock in the image deterioration detection position information 8 and stores it (step S6). Then, the process proceeds to step S7 described below.

If it is determined that there is no image deterioration (step S5), the macroblock encoded data is used to generate a decoded image of the macroblock (step S7). In the process of step S6, functional blocks such as the inverse quantization unit 20, the inverse orthogonal conversion unit 21, and the motion compensation unit 23 in the configuration of FIG. 4 are used. First, the orthogonal conversion coefficient data 6 and the quantization step parameter 19 are sent to the inverse quantization unit 20 and the inverse orthogonal conversion unit 21, and are decoded into an image signal. The orthogonal conversion coefficient data 6 is output to the outside by the macroblock decoding processing unit 2 for use in the image deterioration detection processing described later. When the coding mode information 18 indicates the inter-frame motion prediction mode (intermode), the switch 22 is in the frame memory 14 based on the motion vector 17 and the coding mode information 18 in the motion compensation unit 23. While the predicted image 24 is generated and output from the reference image 15 of the above, 0 is output when the in-frame coding mode (intra mode) is indicated.

In the intermode, the output of the inverse orthogonal converter 21 is the prediction error image signal of the result of the motion compensation prediction described above, so that the same prediction image as that generated on the coding side using the motion vector 17 from the reference image. The decoded image 3 is reproduced by generating 24 and adding it to the output of the inverse orthogonal conversion unit 21 in the addition unit 25.

The intra mode is a case where the reference image is not used at all and is coded in the frame. In this case, since the output of the switch 22 is zero, the output of the inverse orthogonal converter 21 becomes the decoded image 3 as it is. .. When the decoding failure detection flag indicates that there is no failure (= 0), the switch 4 outputs the above-described decoded image 3 as the final decoded image 5 (as already described above, the switch 4 always changes to the decoded image 3). Restoration image 13 is not input). Since the final decoded image 5 is used to generate a predicted image of subsequent frames, it is stored in the frame memory 14. This completes the decoding process when no decoding failure is detected in the macroblock. When this processing flow is completed for all the macroblocks contained in the video packet (step S8), the processing of the video packet is terminated.

<u style="single">(3) Macroblock decoding process when a decryption failure is detected</u> When the variable-length decoding unit 16 detects a decoding failure, the variable-length decoding unit 16 sets the decoding failure detection flag 12a to 1 and notifies the external notification that the decoding failure has occurred. The resynchronization marker is then searched to restore the decoding synchronization and resume normal decoding (step S9). As described above, since the macroblock coded data from the detection of the decoding failure to the time when the resynchronization marker is found is lost, it is necessary to supplement the image data in a predetermined range by repair. One of the points of the present invention is to adaptively adjust the image restoration range based on the result of the image deterioration detection described in (2) above so as to suppress the image deterioration caused by the bit error in the video packet as much as possible. It is at the point to be decided.

After the resynchronization marker is detected, the position of the first macroblock of the next video packet in the screen is confirmed to identify the position of the macroblock in the screen where the video packet ends. As a result, the variable-length decoding unit 16 recognizes the end of the image area (image restoration area) that needs to be image-repaired in the video packet, and externally notifies the end position by the image restoration end position information 12b. The image restoration range is determined based on the image restoration end position information 12b and the image restoration start position information 10 defined by the image restoration start position determination unit 9 (step S10). In the present embodiment, the image restoration start position determination unit 9 determines the image restoration start position information 10 as follows based on the image deterioration detection position information 8 which is the output of the image deterioration detection unit 7. MBPOS_B = (MBPOS_A MBPOS_C &lt;THR)? MBPOS_C: THR In the above formula, MBPOS_A is the in-screen position of the macroblock that detected the decryption failure, MBPOS_B is the value of image repair start position information 10, and MBPOS_C is the image deterioration detection position information 8. The value, THR, is a predetermined threshold. Z = (X &lt;Y)? A: The operation B is a ternary operator defined in the C language description, meaning that if X is less than Y, then Z becomes A, otherwise Z becomes B. Also, in general, MBPOS_A&gt; = MBPOS_C because the decoding synchronization is lost due to a bit error before or after the macroblock that detects the decoding failure. However, if no image deterioration is detected in the video packet until MBPOS_A, MBPOS_C remains at the initial value that does not indicate any in-screen position. In that case, MBPOS_C == Considered as MBPOS_A. In principle, MBPOS_C cannot be before the position of the first macroblock of the video packet.

That is, in the image repair start position information 10 (MBPOS_B), the number of macroblocks that revert when going back from the in-screen position MBPOS_A of the macroblock that detected the decoding failure to the image deterioration detection position MBPOS_C is within a predetermined range (THR). If it is settled, set the image deterioration detection position MBPOS_C as the image restoration start position MBPOS_B. On the other hand, if the number of macroblocks that revert is not within the predetermined range (THR), the position that reverts by the threshold THR is set as the image restoration start position MBPOS_B. The reason for adopting such a decision method is as follows. In the present embodiment, when the image deterioration detection unit 7 detects a macroblock in which image deterioration is observed, image restoration is performed including such an image deterioration portion. This is because, as shown as an issue earlier, in the image restoration from the position where the decoding failure is detected, the decoding synchronization loss that causes the image to be disturbed before the decoding failure as shown in Fig. 2 can be overlooked and the deterioration can be completely repaired. This is to solve the problem of not having it.

However, since the repair process is executed retroactively even for the past macroblocks for which the decoded image has been generated based on the process (2) above without causing the decoding failure, such macroblocks are used. The processing is duplicated and the processing amount as a whole increases. According to the image restoration start position determination unit 9 in the present embodiment, an upper limit is given to the number of macroblocks for which restoration processing is performed retroactively by the threshold value THR, and the upper limit value of the processing amount increment due to the above duplication processing is surely given. Therefore, a decoding device that matches the system design conditions can be obtained by setting the THR to an appropriate value in advance based on the balance between the required conditions for the image deterioration repair capability of the decoding device to be designed and the allowable processing amount. It has the effect of being able to design flexibly. For example, there are the following methods for determining THR. In general, there is a correlation between the location where the decoding failure occurs and the location where the decoded image is disturbed due to the out-of-decoding synchronization with the amount of code contained between them. When the total code amount in a predetermined number of macroblocks is small, the effect of decoding synchronization loss often appears as a disorder of the image from a position well before the detection of the decoding failure, but the total code amount is large. In this case, the amount of code per macroblock is large, so even if the decoding synchronization is lost, the number of macroblocks to be decoded before it is detected as a decoding failure is not so large stochastically. Therefore, a method such as determining the THR based on the bit rate of the input moving image compressed data 1 or the quantization parameter reference value included in the header information of the video packet can be considered.

Next, the image restoration end position information 12b and the image restoration start position information 10 determined by the image restoration start position determination unit 9 described later are input to the image restoration processing unit 11, and based on these start / end position information. A process of performing image repair (repaired image generation) is executed for all macro blocks within the determined image repair range (step S11). The image restoration process is a process of generating a similar image as a restoration image from image information or encoded data in the temporal or spatial vicinity of the macroblock to be restored. For example, the following various processing methods can be considered.

[How to generate a repair image from a reference image]

-From the reference image in the frame memory, the image data at the same spatial position as the macroblock to be repaired is copied as it is. -From the reference image in the frame memory, the image data of the part shifted by a predetermined motion vector from the spatially the same position as the macroblock to be repaired is copied. If the original motion vector is lost due to decoding failure and cannot be used, for example, the alternative motion vector is obtained by the following method. Hold the motion vector of the macroblock directly above the macroblock to be repaired and use it. Hold the motion vector of the macroblock around the macroblock to be repaired, and use them to calculate and use the motion vector predicted value used in the motion vector prediction decoding processing. -When the video packet uses the data partitioning syntax specified in the MPEG-4 visual standard, for example, and the marker code inserted as the code for separating the high-importance data and the low-importance data can be correctly decoded. In addition, using the motion vector encoded as the data of high importance, the part of the reference image in the frame memory that is spatially shifted from the same position as the macroblock to be repaired by the motion vector. Copy the image data.

[How to generate a repair image using the data in the same frame]

-The DC coefficient and AC coefficient (part or all) of the peripheral macroblock in the same frame are retained, and a restored image is generated using them. -When the video packet uses the data partitioning syntax and the marker code inserted as the code for separating the high-importance data and the low-importance data can be correctly decoded, it is coded as the high-importance data. Generate a restored image using the converted DC coefficient. Or, at that time, the AC coefficient of the peripheral macroblock is also retained, and these are also taken into consideration to generate a restored image.

When the decoding failure detection flag indicates that the decoding failure detection flag is broken (= 1), the switching device 4 outputs the repaired image 13 as the final decoding image 5 (as already described above, the switching device 4 always changes to the decoding image 3). Restoration image 13 is not input). Since the final decoded image 5 is used to generate a predicted image of subsequent frames, it is stored in the frame memory 14. When the repaired images of all the macroblocks within the image repair range have been generated by the above method, the processing of the video packet is finished (step S12).

<u style="single">(4) Image deterioration detection processing</u> Hereinafter, the image deterioration determination process of step S4 performed by the image deterioration detection unit 7 will be described in detail. In the present embodiment, the image deterioration is defined as the standard in which the orthogonal conversion coefficient data 6 of the macroblock coded data analyzed / extracted and output by the variable length decoding unit 16 is followed by the input moving image compression data 1. Is allowed, but it is defined as the case where the probability of occurrence is very low in normal moving image compressed data.

The image deterioration detection unit 7 checks whether the value of the input orthogonal conversion coefficient data 6 is within the range of the threshold value for deterioration determination stored in advance, and when the range is exceeded, such orthogonality is obtained. The conversion coefficient data 6 is determined to be an image deterioration factor. Specifically, among the orthogonal conversion coefficient data 6, the predicted difference value (DC_DIFF) and AC coefficient (AC_LEVEL) of the intra DC coefficient are stored in advance inside the image deterioration detection unit 7, respectively, as threshold tables THR_TABLE_DCDIFF, Image deterioration is defined as when it is out of the range of THR_TABLE_AC_LEVEL. That is, DC_DIFF&gt; THR_TABLE_DCDIFF [Qp] [YC]

Or AC_LEVEL&gt; THR_TABLE_AC_LEVEL [M] [Qp] [YC] [FQ]

The case where is satisfied is regarded as image deterioration. In the above equation, Qp is the quantization step parameter 19, YC is the luminance signal / color difference signal type (the value of YC can be determined by the number of the 8x8 pixel block uniquely determined in the macroblock. The number of the 8x8 pixel block is 0 ~ 3 is the brightness, 4,5 is the color difference), M is the coding mode information 18 (intra mode, inter mode type), and FQ is the frequency of the AC coefficient. Regarding the deterioration judgment for DC_DIFF, for example, when the orthogonal conversion coefficient data 6 does not directly include the DC_DIFF information, the average value of the images of the peripheral macroblocks is actually obtained and held, and the deterioration is detected. It may be configured to execute the deterioration judgment after obtaining the difference from the average value of the target macroblock as DC_DIFF. The deterioration determination of DC_DIFF is particularly effective in detecting the occurrence of blocks having an abnormal average color in the surrounding image. Further, the deterioration determination of AC_LEVEL is effective in detecting an abnormal waveform pattern in a macroblock caused by emphasizing the orthogonal conversion basis of the frequency component by an abnormal AC coefficient value.

In addition, various image deterioration determination methods as shown below can be considered. -When the output obtained by inversely quantizing the orthogonal conversion coefficient data 6 by the inverse quantization unit 20 deviates from the value range defined by the standard.-The pixel value of the decoded image 3 generated by the adder 25 is the value range defined by the standard. When the effective coefficient distribution (frequency distribution) of the orthogonal conversion coefficient data 6 deviates from a predetermined value range When the number of effective coefficients of the orthogonal conversion coefficient data 6 exceeds a predetermined threshold Motion vector 17 Or when the predicted difference value of the motion vector 17 deviates from a predetermined value range When the sum of powers in the block of the orthogonal conversion coefficient data 6 deviates from a predetermined value range

Each of the above "ranges" may be read as between "mean value- (constant) x standard deviation" and "mean value + (constant) x standard deviation" using a certain positive constant. Theoretically, the difference value is considered to be symmetrical with respect to the maximum value and the minimum value centered on 0. Therefore, the "range of the difference value" may be read as the "maximum value of the absolute value of the difference value".

Further, the threshold value or the threshold value table used for the image deterioration determination as described above may be configured to be learned in the process of decoding the input moving image compressed data 1. Another configuration example of such a moving image decoding device is shown in FIG. In the figure, the statistic storage unit 26 is an initial value in which the decoding failure detection flag 12a is zero and the image restoration start position information 10 cannot indicate any in-screen position with respect to the input of the orthogonal conversion coefficient data 6. For the macro block only, the function to calculate the statistic used in the threshold / threshold table for image deterioration judgment from the orthogonal conversion coefficient data 6 and dynamically change the threshold / threshold table inside the image deterioration detection unit 7. Have. For example, at the beginning of a video packet, the value of the threshold value / threshold value table at that time is temporarily saved in another memory, and while the video packet is being decoded, the statistic storage unit 26 of the threshold value / threshold value table Try to learn. If no decoding failure is detected, the threshold / threshold table updated by learning is used when decoding the next video packet, and if a decoding failure is detected, the threshold / threshold table is updated in the video packet. Since there is a possibility of being affected by a bit error, it is conceivable to return to the state of the threshold value / threshold value table temporarily saved in another memory and perform decoding processing of the next video packet.

In general, since the image signal is a non-stationary signal, there is an effect that stable image deterioration determination can be performed according to the statistical property of the image signal being decoded by performing such learning.

Further, the image deterioration determination check in the image deterioration detection unit 7 can be configured not to be executed after the macroblock in which the image deterioration is recognized once is detected in the video packet. As a result, the image deterioration determination check process can be skipped thereafter during the video packet decoding process, which has the effect of reducing the processing amount.

According to the moving image decoding device having the above configuration, only when a fatal error that causes a decoding failure is included during video packet decoding, image restoration is performed by going back to the image quality deterioration part due to the decoding synchronization loss or the like. Therefore, it is possible to perform accurate image restoration while suppressing the range of image restoration to the minimum necessary.

Furthermore, if the video packet does not contain a fatal error that causes decoding failure, even if a macroblock that causes image deterioration is detected, the normal decoding process is executed and no image repair is performed. Even if there is a macroblock that is erroneously determined as image deterioration even though it is correctly decoded, stable decoding processing can be performed without performing image repair.

In the present embodiment, the range in which image restoration is performed is from the first image deterioration detection location in the video packet to the end of the video packet, but the positions of all macroblocks in which image deterioration is detected are retained. If this is the premise, the repair process may be performed only for the macroblock in which image deterioration is detected. With this configuration, the decoded image in the visually normal range can be excluded from the repair target, so the range for image repair can be further localized, and duplicate repair images must be generated for image repair. It has the effect of reducing the number of macroblocks that do not become.

Further, the range for performing image restoration may be set to determine the start point further back by a predetermined number of macroblocks from the value of the image restoration start position 10 determined by the image restoration start position determination unit 9. Since the point of the block recognized as image deterioration by the image deterioration detection unit 7 is not necessarily the bit error occurrence position, the bit error occurrence position is image-repaired by going back a little further from the image deterioration detection position. It has the effect of increasing the probability of being able to do it.

<figref num="1">It is explanatory drawing which showed the syntax example of the video packet data.</figref><figref num="2">It is explanatory drawing explaining the influence which bit error in a video packet has on a decoding process / decoding image.</figref><figref num="3">It is a block diagram which showed the structure of the moving image decoding apparatus in embodiment.</figref><figref num="4">It is a block diagram of the macroblock decoding processing part of a moving image decoding apparatus.</figref><figref num="5">It is a flowchart which showed the processing flow in a moving image decoding apparatus.</figref><figref num="6">It is a block diagram which showed another configuration example of the moving image decoding apparatus.</figref>

Code description

2 Macroblock decoding processing unit 7 Image deterioration detection unit 9 Image restoration start position determination unit 11 Image restoration processing unit 16 Variable length decoding unit