Image encoding apparatus, image encoding method, image encoding program, image decoding apparatus, image decoding method, and image decoding program

Image encoding device, image encoding method, image encoding program, image decoding device, image decoding method, image decoding program

The invention relates to an image encoding device, an image encoding method, an image encoding program, an image decoding device, an image decoding method, and an image decoding program.

In recent years, due to the popularization of the Internet, it has become more widely used to transmit and receive image data or store image data via the Internet. Generally speaking, the image data encoding adopts a forward encoding method that can effectively reduce the image data capacity. As an example of this forward coding method, there is the international standard H.264 animation coding method disclosed by ITU-T (for example, refer to ITU-T VCEG (Q.6/16), "H.26L Test Model Long Term Number 8 (TML-8)draft0").

In the H.264 Internet coding frame (I frame), the animation is compressed by using intra-frame prediction using intra-frame coding. In addition, in the internal frame coding of H.264, the coding target frame is divided into 16×16 portrait size micro-blocks, and the coding process is performed on each micro-block. The micro-block is further divided into blocks of 16×8 pixels or 8×8 pixels in size, and motion compensation prediction is performed on each divided block. This reduces the verbosity in the time direction of the animation.

However, for image encoding such as animation encoding or static image encoding, more efficient encoding techniques will be obtained.

Therefore, the object of the present invention is to provide an image encoding device, an image encoding method, and an image encoding program that can efficiently encode images, and to provide a bit-recoverable image decoding device and an image generated by the image encoding device of the present invention. Decoding method, image decoding program.

One aspect of the related image encoding device of the present invention is provided with: (a) The input image of the encoding object is divided into a plurality of partial areas formed by a specific size, and it is determined that the prediction assistance information must be used to generate the prediction image. 1 picture prediction process or the second picture prediction process, generate the coding mode of the predicted picture, and generate the means of determining the coding mode information used to limit the coding mode, (b) execute the part of the aforementioned plural fields, from the aforementioned Coding mode information, which restricts the part of the field in which the predicted image should be generated by the first image prediction process, extracts the prediction subsidy information for generating the predicted image in this part of the field from the generated broadcast images in other parts of the field, and based on the prediction subsidy Information is used to generate the predicted image by the first image prediction means of the first image prediction process, (c) the memory means for the playback image based on the foregoing image, and (d) the data that contains the encoded coding mode information and the prediction auxiliary information The encoding method of bit stream.

Another aspect of the image encoding method related to the present invention, wherein the determining means includes: (a) For the multiple partial areas formed by dividing the input image of the encoding object into a specific size, determine the relevant method for generating the predicted image by using the prediction auxiliary information Which of the first image prediction process or the second image prediction process, the coding mode of the predicted image is generated, and the coding mode information used to define the coding mode is generated, (b) the first image prediction means, the aforementioned plural In some domains, from the aforementioned coding mode information, the first image prediction process is used to limit the part of the domain in which the predicted image should be generated, and the broadcast image that has been generated from other parts of the domain is extracted to generate the prediction subsidy for the predicted image in this part of the domain. Information, and based on the predicted auxiliary information to generate the predicted image based on the first image prediction step of the first image prediction process, (c) the memory means to memorize the memory step of the play image based on the above image, and (d) the encoding means to generate The coding step of encoding the bit stream of the data of the coding mode information and the prediction auxiliary information.

In addition, the image coding program related to the other aspect of the present invention functionalizes the computer as: (a) For each partial area formed by dividing the input image of the coding object into a specific size, it is determined that the relevant information must be used to predict the supplementary information Which one of the first picture prediction processing to generate the predicted picture or the second picture prediction processing to generate the coding mode of the predicted picture, and the decision means for generating coding mode information used to limit the coding mode, (b) the complex number part of the field, From the foregoing coding mode information, the first image prediction process is used to limit the part of the domain in which the predicted image should be generated, and the broadcast image that has been generated from other parts of the domain is extracted to generate the prediction assistance information of the predicted image in this part of the domain, and based on this The prediction assistance information generates the first portrait prediction method of the above-mentioned first portrait prediction process of the predicted portrait, (c) the memory means that memorizes the playback portrait based on the above-mentioned portrait, (d) generates the coding mode information that has been encoded and the above-mentioned prediction assistance The program of the coding method of the bit stream of the information data. In addition, the aforementioned image encoding program and the image encoding program of the present invention described below can be provided in the form of a computer-readable recording medium, a computer data signal superimposed on a carrier wave, or a program product.

The second image prediction process mentioned above is to use the area where the pixels for which no prediction signal is generated as part of the area as a template, and the generated playback image as the reference area, and select the area with the highest correlation to the template in the reference area as the copy reference In the domain, the pixels that have not generated the predicted signal in the aforementioned template are given the pixel value of the corresponding pixel in the copied reference domain, thereby generating the processing of the predicted image.

Another aspect of the present invention relates to an image decoding device, which includes: (a) For each partial area formed by dividing a decoding target image into a complex number of a specific size, the first image prediction process used to generate a predicted image is restricted from the inclusion of coding Or the coding mode information of the second picture prediction process, and the bit stream of the prediction auxiliary information of the predicted picture generated based on the first picture prediction process, the decoding means for decoding the coding mode information and the prediction auxiliary information, (b) a complex number In some domains, from the decoding mode information, the prediction image of the part of the domain in which the prediction image should be generated by the first image prediction process is limited, and the first image prediction process by using the prediction auxiliary information from the generated broadcast image In the first image prediction method generated, (c) the plural domains, from the decoding mode information, the prediction image of the partial domain of the domain where the prediction image should be generated by the second image prediction process is limited by the second image The second image prediction method generated by the prediction process, (d) the memory method that memorizes the play image based on the predicted image; (e) In the second image prediction process, the area where the pixel that does not generate the predicted signal is a part of it is set as a template , The playback image memorized in the memory means is set as the reference field, and the field in the reference field that is most relevant to the template is selected as the copy reference field, and the corresponding pixels in the copy reference field are applied to the pixels in the template that do not generate the predicted signal The pixel value of the pixel to generate a predicted image.

Another aspect of the image decoding method related to the present invention is that: (a) For each partial area formed by dividing the image to be decoded into a complex number of specific sizes, the first image prediction process used to generate the predicted image is restricted from the inclusion of encoding Or the decoding mode information of the second picture prediction process, and the bit stream of the prediction auxiliary information of the predicted picture generated by the first picture prediction process, the decoding means decodes the coding mode information and the decoding step of the prediction auxiliary information, (b ) And the first image prediction means, among the aforementioned plural fields, from the aforementioned decoding mode information, the prediction images of the partial areas of the fields in which the prediction images should be generated by the aforementioned first image prediction processing are limited by using the aforementioned prediction auxiliary information The first image prediction step generated by the first image prediction process, (c) the second image prediction means, in the plural partial areas, from the foregoing decoding mode information, the second image prediction process should be used to generate the predicted image Predictive portraits in some domains of the domain, using the second portrait prediction process generated by the second portrait prediction process, (d) and memory means to memorize the memory step of the playback portrait based on the predicted portrait; (e) in the second portrait prediction process , The area where the pixel that does not generate the predicted signal is a part of it is set as the template, the playback image memorized in the memory means is set as the reference area, and the area with the highest template correlation in the reference area is selected as the copy reference area, in the aforementioned template The pixel value of the corresponding pixel in the copy reference area is applied to the pixel for which the prediction signal is not generated, thereby generating the prediction image.

The image decoding program related to the other aspect of the present invention functions as a computer: (a) For each partial area of the complex number formed by dividing the image to be decoded into a specific size, the first one is used to generate the predicted image from the inclusion of the encoding limit. Picture prediction processing, or decoding mode information of the second picture prediction processing, and the bit stream of the prediction auxiliary information of the predicted picture generated by the first picture prediction processing, decoding means for decoding the coding mode information and the prediction auxiliary information, (b) Among the aforementioned plural domains, from the decoding mode information, the prediction image of the partial domain of the domain where the prediction image should be generated by the first image prediction process is limited by the first image prediction process using the prediction auxiliary information In the first image prediction method generated, (c) partial areas of the plural number, from the decoding mode information, the predicted images in the partial areas of the areas where the predicted images should be generated by the second image prediction processing are limited, and predicted by the second image The second image prediction method generated by the processing, (d) the memory method based on the memory method of the play image of the predicted image. In this program (e) in the second image prediction process, the area where the pixel that does not generate the prediction signal is part of it is set as the template, the playback image stored in the memory means is set as the reference area, and the reference area and the aforementioned template are selected The area with the highest correlation is the replicated reference area. On pixels in the template that do not generate the aforementioned prediction signal, the pixel values of the corresponding pixels in the aforementioned replicated reference area are applied to generate a predicted image and the computer is functionalized. In addition, the above-mentioned image decoding program and the image decoding program of the present invention described below can be provided in the form of a computer-readable recording medium, a computer data signal superimposed on a carrier wave, or a program product.

According to the above-mentioned invention, in the second image prediction process, the generated playback image is set as the reference area, and the pixels corresponding to the copied reference area selected from the reference area are copied on the pixels where the prediction signal is not generated in the template. From the reference area, select the area that is more relevant to the template as the copy reference area. For example, selecting the field with the highest correlation value, or the field with the highest correlation value compared to a specific benchmark value, etc. Therefore, in the decoding end, for some areas that are limited to the areas where the predicted image should be generated by the second image prediction processing, the predicted image can be actively generated without using the prediction assistance information from the encoding end. Therefore, high-efficiency encoding can be achieved at the encoding end.

In the present invention related to image coding, the determining means is preferably to generate the playback images in the plural partial areas through the first image prediction process and generate them in accordance with the specific scanning order, and then select the processing in the order opposite to the specific scanning order. Compared with the partial area of the processing object, the partial area of the object is compared with the partial area of the image in the front of the scanning sequence. Compared with the partial area of the object of the processing, the partial area of the scanning sequence is located in the back. The field of the predicted image that should be generated by the first image prediction process determines the encoding mode of the play image of the partial field as the reference field. The play image of the partial field of the processing target is generated through the second image prediction process, based on the comparison of the first image prediction process. 2 The playback image of the partial area of the processing object generated by the image prediction processing, and the playback image of the partial area of the processing object generated by the first image prediction processing, determine the coding mode of the partial area of the processing object.

At this time, in the present invention related to image decoding, for a part of the area where the predicted image should be generated by the first image prediction process defined from the decoding mode information, the predicted image is generated by the first image prediction process according to a specific scanning sequence, After memorizing the playback image generated based on the predicted image in the aforementioned memory means, the second image predicting method defines the part of the area where the predicted image should be generated by the second image prediction process from the decoding mode information, according to a specific scanning sequence The predicted image is generated by the second image prediction process.

According to the present invention, after the first image prediction process generates the play image, the generated play image is set as the reference area for the second image prediction process. Therefore, the playback image at the rear in the specific scanning sequence can also be used in the second image prediction process, so the redundancy in the spatial direction can be effectively reduced.

In the present invention related to image coding, when the determining means and calculation use the playback image of the partial area of the processing target generated by the second image prediction processing and the playback image of the partial area of the processing target generated by the first image prediction processing. The coding skew or/and the cost value of the cost function derived from the amount of coded data may be based on the cost value to determine the aforementioned coding mode of the part of the processing object.

In the present invention related to image coding, in some areas with plural numbers, from the aforementioned coding mode information, the predicted image in the partial area of the area where the predicted image should be generated by the second image prediction process is limited, and predicted by the second image The second image prediction method generated by the processing; the second image prediction method sets the predicted image generated by the second image prediction processing as a broadcast image. In this case, in the present invention related to picture decoding, the second picture prediction means uses the predicted picture generated by the second picture prediction process as a broadcast picture. That is, the predicted portrait generated by the second portrait prediction method is still used as a broadcast portrait. Therefore, the playback image generated by the second image prediction process can also be used for prediction in some areas of the second process, so the redundancy can be reduced. In addition, since the information about the difference between the input image and the predicted image generated by the second image prediction process does not need to be included in the bit stream, effective coding can be achieved.

In the present invention, the input image of the encoding object can also be an animation frame. At this time, in the second image prediction process, at least one of the playback image of the encoding target frame and the playback image of the frame that is processed prior to the encoding target frame is set as the reference area. Also, in this case, in the present invention related to image decoding, the image to be decoded is an animation frame; the second image predicting means is stored in the playback image of the aforementioned memory means, and the playback image of the decoding target frame, and the corresponding At least one of the playback images of the frame to be processed prior to the decoding target frame is set as the aforementioned reference area.

According to the present invention, in the second image prediction process, both the playback image of the encoding target frame and the playback image of the processed frame that is different from the encoding target frame are set as reference areas, so the time direction can be reduced. The length of the space direction.

In the present invention related to image coding, the first image prediction process may also be motion compensation prediction process. In this case, the prediction assistance information includes the motion vector extracted by the first image prediction process. Also, in this case, in the present invention related to image decoding, the prediction assistance information also includes the motion vector used in the first image prediction process.

In the present invention, the first image prediction process can also be a process of using a play image in the same space as a part of the field of the processing object for prediction to generate a predicted image. That is, the first picture prediction process can also be a prediction process used in the encoding and decoding of a static image, or an intra-frame prediction process used in the encoding and decoding of an animation. In the first image prediction processing, from the coding mode, the connected partial area connected to the partial area of the processing object is limited to the case where the second image prediction processing should generate the partial area of the predicted image, based on the partial area not connected to the processing object The play image of the connected non-connected part of the field, and the predicted image of the part of the field of the processing object is generated.

According to the present invention, the part of the field of the processing object is the area where the predicted image should be generated by the first image prediction process, and the connected part of the field connecting the part of the process object is the area where the predicted image should be generated by the second image prediction process At this time, it is also possible to use the playback image data of the non-connected partial areas that are not connected to the partial areas of the processing object to generate the predicted image. In this way, not only the playback image of the connected part of the field of the processing object is connected, but also the playback image of the non-connected part of the field that is not connected to the part of the processing object can be used to generate a predicted image. Therefore, the reference range when generating the predicted image data will be expanded, and the redundancy of the space will be reduced, so the coding efficiency can be improved.

In addition, in the present invention related to image coding, in the first image prediction process, the prediction mode of the prediction image used in the partial area of the processing target is determined from the complex prediction mode related to the plurality of different prediction rules. The prediction mode information that defines the prediction mode is generated. The coding method can include the data of the coded prediction mode information in the bit stream. At this time, in the present invention related to picture decoding, in the case where the bit stream generates a predicted picture through the first picture prediction process, including the complex prediction mode related to the coding complex number different prediction rules, it is limited to the first picture prediction The data of the prediction mode information of the processed prediction mode; the decoding means decodes the prediction mode information from the bit stream, and in the first image prediction process, the prediction image is generated based on the prediction mode information.

In the present invention, in the first image prediction processing, from the coding mode, the connecting partial areas of the partial areas of the processing target are limited to the straight line in the prediction direction when the partial areas of the Misaki image should be generated by the second image prediction processing It is better to generate the predicted image based on the pixel value of the pixel in the partial area closest to the processing target in the playback image that exists in the non-connected part of the field in the direction of the prediction start end. In this way, from the non-connected blocks, the best play image data corresponding to each predicted image generated pattern is selected.

In the present invention related to image coding, the prediction residual image generation means performs the calculation of the difference between the prediction image generated by the first image prediction method and the input image of the encoding object to generate the prediction residual image; the encoding method uses the coding based on prediction The data of the signal generated by the residual image can also be included in the bit stream.

At this time, in the present invention related to image decoding, the bit stream includes a signal encoding a predicted image based on a partial area generated by the first image prediction process, and a predicted residual image generated by calculating the difference between the image in the partial area. And the generated data; the decoding means, included in the data of the bit stream, decode the signal from the data formed by encoding the signal generated based on the predicted residual image; the playback image generation means is based on the addition and the decoded means is decoded The signal recovery predicts the residual image and the predicted image to generate a play image.

In the present invention related to image coding, the prediction residual image generation means performs the calculation of the difference between the prediction image generated by the first image prediction method and the input image of the coding target to generate the prediction residual image; the conversion means generates the prediction residual image due to the prediction residual image The conversion process is performed to generate conversion information; the reverse conversion method generates the restored prediction residual image due to the reverse conversion process on the conversion information, and the playback image generation method restores the predicted residual image and the predicted image by addition to generate the playback image, and the encoding method encodes The data of the conversion information may also be included in the bit stream.

At this time, in the present invention related to image decoding, the bit stream includes a prediction image of a part of the field generated by the first image prediction process, and the prediction residue generated by calculating the difference between the image (input image) of the part of the field The data generated by the conversion information of the conversion processing on the image; the decoding method decodes the conversion information from the bit stream; the reverse conversion method generates the restored prediction residual image due to the reverse conversion processing on the conversion information, and the playback image generation method uses addition Respond to the predicted residual image and the predicted image, and generate a play image.

Hereinafter, the best implementation mode of the present invention will be described in detail with reference to the drawings. In addition, in each drawing, the same or equivalent parts will be marked with the same symbols.

[First Embodiment]

First, the first embodiment of the present invention will be described. Fig. 1 is a diagram showing the structure of an image encoding device related to the first embodiment. The image encoding device 1 shown in FIG. 1 can be physically set as a computer equipped with, for example, a CPU (central processing unit), a memory device such as a memory, a display device such as a display, and a communication device. In addition, the image encoding device 1 may also be a mobile communication terminal such as a mobile phone, or a DVD device. That is, the image encoding device 1 can be widely used as an information processing device.

The image encoding device 1 functionally includes an image segmentation unit (image segmentation means) 11, an encoding mode decision unit (decision means) 12, a predicted image generation unit (first image prediction means) 13, and a subtraction unit (predicted residual image generation means) 14 , Conversion part (conversion means) 15, coding part (coding means) 16, inverse conversion part (inverse conversion means) 17, addition part (play image generation means) 18, and storage part (memory means) 19.

Next, each structural element shown in FIG. 1 will be described. The portrait dividing unit 11 divides the input portrait input in the unit of a frame into blocks of a specific size (for example, a 4×4 portrait), that is, a partial area. In addition, the image segmentation unit 11 generates block position information for defining the processing target block as the encoding processing target. As the block location information, for example, each block in the frame is marked with a numbered block number from 0, 1, 2 in order of raster scanning, or a picture containing each block The upper left end of the frame is the block coordinates shown by the reference coordinates.

The coding mode determination unit 12 determines the coding mode of each block based on the specific coding mode decision rule, and generates coding mode information used to limit the coding mode.

In this embodiment, the coding modes include the predictive coding processing mode (P mode) that uses the input image of the processing target block and the predicted image corresponding to the image, and encodes the image mode of the processing target block (P mode), and does not use The input image of the processing target block and the predicted image corresponding to the image are encoded, and the filling coding processing mode (C mode) of the image mode of the processing target block. That is, when the encoding mode is the predictive encoding processing mode, the image related information of the processing target block is encoded and output. On the other hand, when the coding mode is the padding coding processing mode, the image-related information of the processing target block is not coded or output. Also, when the coding mode is the predictive coding processing mode, the prediction image at the decoder side is used to generate the required prediction auxiliary information through the first image prediction processing to generate the predicted image. When the coding mode is the padding coding processing mode, the decoding side The second image prediction processing (image filling processing), which does not require forecast subsidy information, is used to generate the predicted image.

The coding mode decision rule, for example, through the image padding process described later (refer to Figure 6, the second image prediction process), the playback image of the processing target block is generated, and the error between the input image of the processing target block and the playback image is preset If the square value is below the critical value, set it to the padding coding mode, and set it to the predictive coding processing mode for other cases. In addition, it is not necessary to compare the squared value of the error with the critical value, but also to compare the absolute value of the error with the critical value. In addition, other coding mode decision rules can also be, for example, the information of the block coded in the predictive coding processing mode and the block coded in the padding coding mode in advance, corresponding to the block position information and maintained for processing. At this time, the coding mode of the corresponding processing target block is obtained based on the block location information.

The prediction image generation unit 13 uses the first image prediction process to generate the prediction image corresponding to the input image of the processing target block when the coding mode is the prediction coding processing mode. , That is, the prediction mode, select from the nine kinds of prediction image generation patterns described later, and output the prediction mode information used to select the prediction mode. That is, the prediction image generation pattern (prediction mode) in this frame is used for the prediction assistance information required when the decoding side generates the prediction image.

The predicted image generation unit 13 uses a part of the image that is completely encoded in the image of each block, played and memorized in the memory unit 19 according to the determined predicted image generation pattern, and generates an input image corresponding to the processing target block Forecast portrait. In addition, the details of the first image prediction process when generating the predicted image will be described later.

The subtraction unit 14 subtracts the predicted image of the processing target block from the input image of the processing target block to a pixel unit to generate a predicted residual image.

The conversion unit 15 uses a specific conversion rule to convert the predicted residual image, and outputs the conversion coefficient (conversion information) obtained by the conversion. Specific conversion rules are, for example, the 2-dimensional elementary DCT with 4 rows and 4 columns, and the quadrature conversion and quantization with 4 rows and 4 columns used in H.264. In addition, the specific conversion rules are, for example, the conversion operations of Matching Pursuit, vector quantization, and wave rate conversion. Quantization is also possible.

The encoding unit 16 encodes the conversion coefficient based on the average amount of information based on a specific rule. In addition, the encoding unit 16 encodes the encoding mode information and the prediction image generation pattern (prediction mode) based on a specific rule and the average amount of information. The average amount of information coding uses, for example, arithmetic coding.

The inverse conversion unit 17 inversely converts the conversion coefficients according to a specific inverse conversion rule to generate the restored prediction residual image. This specific anti-conversion rule corresponds to the anti-conversion rule of the specific conversion rule adopted by the conversion unit 15.

The adding unit 18 adds the predicted image of the target block and the restored predicted residual image corresponding to the predicted image to generate a broadcast image. In addition, when the pixel value of the image is set to a specific range, the addition unit 18 may perform clipping processing in order to limit the pixel value to the specific range.

The memory unit 19 stores the playback image generated by the addition unit 18 in a memory not shown.

Next, referring to Fig. 2 and Fig. 3, the first image prediction process when generating a predicted image will be explained. In addition, in this embodiment, although the first image prediction process is an intra-frame prediction process, various prediction processes such as motion compensation prediction process can be applied to the first image prediction process.

First, as shown in Figure 2, the block adjacent to the upper left of the processing target block Y of 4×4 pixels is set to block X0, and the block adjacent to the upper side is set to block X1, and The block adjacent to the upper right side is set to block X2, and the block adjacent to the left is set to block X3. In addition, the block adjacent to the upper side of the block X1 is referred to as X4, the block adjacent to the upper side of the block X2 is referred to as the block X5, and the block adjacent to the left side of the block X3 is referred to as the block X6. In addition, the playback pixel values of the bottom row of block X1 are A, B, C, and D from the left, and the playback pixel values of the bottom row of block X2 are E, F, G, and H from the left. , The playback pixel values in the rightmost column of block X3 are I, J, K, L in order from the top. Also, the playback pixel value in the lower right corner of block X0 is set to M. Furthermore, the pixel values of the predicted image of the processing target block Y are set as a, b, c, d, e, f, g, h, i, j, k, l, m in accordance with the scanning order of the light spots , N, o, p.

Herein, the 9 prediction modes A0 to A8 of the 4×4 pixel block coding mode will be described with reference to FIG. 3. The prediction mode A0 shown in FIG. 3(a) is a mode in which the pixel values adjacent to the upper side of the processing target block are linearly extended below, thereby generating a prediction image. In this prediction mode A0, a prediction image is generated based on the following equation.

a=e=i=m=A b=f=j=n=B c=g=k=o=C d=h=l=p=D

The prediction mode A1 shown in FIG. 3(b) is a mode for generating a prediction image by linearly extending the pixel values adjacent to the left side of the processing target block. In this prediction mode A1, a prediction image is generated based on the following equation.

a=b=c=d=I e=f=g=h=J i=j=k=l=K m=n=o=p=L

The prediction mode A2 shown in FIG. 3(c) is a mode that uses the average value of surrounding pixels to predict only the DC component of the processing target block. In this prediction mode A2, a prediction image is generated based on the following rules. First, when A~M are all the playback pixels in the frame, all the values of a~p are set to (A+B+C+D+I+J+K+L+4)/8. In contrast, when A~D do not belong to the playback block in the frame, and I~L belong to the playback block in the frame, all values of a~p are set to (I+J+K+L+2 )/4. Also, I~L do not belong to the playback block in the frame, and A~D belong to the playback block in the frame, all values of a~p are set to (A+B+C+D+2)/ 4. In addition, A~D and I~L are not in the case of the playback block in the frame, and all the values of a~p are set to 128.

The prediction mode A3 shown in FIG. 3(d) is a mode that generates a prediction image by linearly extending the pixel values adjacent to the upper side of the processing target block and diagonally above the right side in the diagonally lower left direction. In this prediction mode A3, a prediction image is generated based on the following equation.

a=(A+2B+C+2)/4 b=e=(B+2C+D+2)/4 c=f=i=(C+2D+E+2)/4 d=g=j =m=(D+2E+F+2)/4 h=k=n=(E+2F+G+2)/4 l=o=(F+2G+H+2)/4 p=(G +3H+2)/4

The prediction mode A4 shown in FIG. 3(e) is a mode that generates a prediction image by linearly extending the pixel values adjacent to the left side, diagonally upper left, and upper side of the processing target block to the diagonally lower right. In this prediction mode A4, a prediction image is generated based on the following equation.

m=(J+2K+L+2)/4 i=n=(I+2J+K+2)/4 e=j=o=(M+2I+J+2)/4 a=f=k =p=(A+2M+I+2)/4 b=g=l=(M+2A+B+2)/4 c=h=(A+2B+C+2)/4 d=(B +2C+D+2)/4

The prediction mode A5 shown in FIG. 3(f) is a mode for generating a prediction image by linearly extending the pixel values adjacent to the left side, diagonally upper left, and upper side of the processing target block to the lower right side. In this prediction mode A5, a prediction image is generated based on the following equation.

a=j=(M+A+1)/2 b=k=(A+B+1)/2 c=l=(B+C+1)/2 d=(C+D+1)/2 f=o=(M+2A+B+2)/4 g=p=(A+2B+C+2)/4 h=(B+2C+D+2)/4 i=(M+2I+ J+2)/4 m=(I+2J+K+2)/4

The prediction mode A6 shown in FIG. 3(g) is a mode for generating a prediction image by linearly extending the pixel values adjacent to the left side, diagonally upper left, and upper side of the processing target block to the lower right side. In this prediction mode A6, a prediction image is generated based on the following equation.

a=g=(M+I+1)/2 b=h=(I+2M+A+2)/4 c=(M+2A+B+2)/4 d=(A+2B+C+ 2)/4 e=k=(I+J+1)/2 f=l=(M+2I+J+2)/40i=o=(J+K+1)/2 j=p=(I +2J+K+2)/4 m=(K+L+1)/2 n=(J+2K+L+2)/4

The prediction mode A7 shown in FIG. 3(h) is a mode for generating a prediction image by linearly extending the pixel values adjacent to the upper side and diagonally upper right of the processing target block to the lower left side. In this prediction mode A7, a prediction image is generated based on the following equation.

a=(A+B+1)/2 b=i=(B+C+1)/2 c=j=(C+D+1)/2 d=k=(D+E+1)/2 l=(E+F+1)/2 e=(A+2B+C+2)/4 f=m=(B+2C+D+2)/4 g=n=(C+2D+E+ 2)/4 h=o=(D+2E+F+2)/4 p=(E+2F+G+2)/4

The prediction mode A8 shown in FIG. 3(i) is a mode for generating a prediction image by linearly extending the pixel values adjacent to the left side of the processing target block to the upper right side. In this prediction mode A8, a prediction image is generated based on the following equation.

a=(I+J+1)/2 b=(I+2J+K+2)/4 c=e=(J+K+1)/2 d=f=(J+2K+L+2) /4 g=i=(K+L+1)/2 h=j=(K+3L+2)/4 k=l=m=n=o=p=L

The prediction image generation unit 13 does not select the prediction mode outside the frame even if the playback pixel value used when generating the prediction image in each of the above prediction modes is 1.

Here, in the present embodiment, when the block coding mode including any one of A to M is the padding coding mode, since no predicted image is generated, there is no display pixel value at this time. Therefore, when the predicted image is generated in other blocks, the pixel value of the block in the padding coding mode cannot be referred to.

Patterns in the present embodiment, in this case, the prediction mode of the prediction direction (the direction indicated by an arrow in FIG. 3) of straight present direction of the line and the predicted start side (side of the starting point of an arrow in FIG. 3) of the Among the playback pixel values in the same frame, the playback pixel value closest to the processing target block is used as the replacement pixel value of A~M to generate the predicted image. Also, when a straight line in the prediction direction passes between two pixels, the average pixel value of the two pixels is set as the substitute pixel value.

The above-mentioned substitute pixel value will be described with reference to FIGS. 4 and 5. First, FIG. 4 is a diagram showing the alternative pixel value determined when the block X1 adjacent to the upper side of the block Y is in the padding coding mode and the prediction mode is A0. As shown in Fig. 4, the playback pixel values N, O, P, and Q in the bottom row of block X4 are selected as the replacement pixel values for the playback pixel values A, B, C, and D of block X1. That is, for the playback pixel values A, B, C, D, the prediction mode A0 is on the straight line in the prediction direction, and the playback pixel values in the same frame on the prediction start side are selected, and the block closest to the processing target block is selected The playback pixel value of Y is the playback pixel value N, O, P, Q. Therefore, when the prediction image of the processing target block Y shown in FIG. 4 is generated, N, O, P, and Q will be used instead of A, B, C, and D in the formula of the above prediction mode A0. In addition, the block X4 is outside the frame or in the case of padding coding mode, and A, B, C, and D are outside the frame. That is, the prediction mode will not select A0.

Next, FIG. 5 is a diagram showing the alternative pixel value determined when the block X1 adjacent to the upper side of the block Y is in the padding coding mode and the prediction mode is A7. As shown in Figure 5, select the 2 playback pixel values N and O in the bottom row of block X4 and the 2 playback pixel values P and Q in the leftmost column of block X2 as the playback pixel value A of block X1 , B, C, D alternate pixel value candidates. Also, select N as the substitute value for A, select O as the substitute value for B, select P as the substitute value for D, and use the average value of O and P as the substitute value for C. The average value of O and P is used as the substitute value of C, because the straight line in the prediction direction passes through the middle of the two pixels O and P (located on the pixel in the lower left corner of block X5). That is, for the playback pixel values A, B, C, D, the prediction mode A7 is on a straight line in the prediction direction, and is located in the same frame in the direction of the prediction start side, and the closest to the processing target is selected The playback pixel values of block Y are N, O, (O+P)/2, Q. Therefore, when generating the prediction image of the processing target block Y shown in FIG. 5, N, O, (O+P)/2, and Q will be used to replace the formulas A, B, C, and C in the above prediction mode A7. D. In addition, in the case of calculating the average value of two pixel values in a binary system, add two pixel values, and after adding 1 to the result of this addition, one bit can be shifted to the right.

In this way, by determining the replacement pixel value, the best playback pixel corresponding to each prediction mode can be selected from non-adjacent blocks.

Next, referring to FIG. 6, an explanation will be given of the image filling process (the second image prediction process) when the filling image is generated in the above-mentioned coding mode decision rule.

In the image filling processing of this embodiment, as shown in FIG. 6, one pixel included in the processing target block Y is set as the processing target pixel P. The area containing the processing target pixel P and the pixels of the playback image (playing pixel) located near the processing target pixel P is set as a template T (template). In addition, if there are pixels (padded pixels) that have completed the image padding processing in the processing target block Y, the pixels that have completed the image padding processing may be included in the template T. In addition, the processing target block Y and the surrounding area of the processing target block Y are referred to as the target area R.

First, the processing target block Y is scanned based on a specific scanning rule, thereby selecting the processing target pixel P from the plurality of pixels included in the processing target block Y. Next, the template T is determined based on the selected pixel P to be processed. Next, in the target area P, among the areas having the same shape as the template T, the relevant area S that has the greatest correlation with the pixels of the part of the processing target pixel P removed from the template T is selected. Secondly, the pixel value of the playback pixel Q corresponding to the processing target pixel P in the related field S is set as the padded pixel value of the processing target pixel P. The same processing as the above processing is performed on each pixel included in the processing target block Y in the scanning order. In this way, a filling image corresponding to the processing target block Y is generated. Here, the correlation calculation method when selecting the above-mentioned related field S, for example, the method where the square of the difference between the corresponding pixel values is the smallest is set to the maximum correlation, or the absolute difference of the corresponding pixel values The sum of the values is the smallest, and the method of setting the maximum correlation is also acceptable. Any other method that can measure the correlation is also applicable. In addition, when the encoding target image is an animation, by using the decoded pixels of the decoded frame and the padding image as the target area R, the padding image can be performed more efficiently.

Next, referring to FIG. 7, the operation of the image encoding device 1 for image encoding processing will be described. This image coding process is performed in the unit of the read block from a frame according to a specific scanning sequence (such as light spot scanning).

First, the image dividing unit 11 divides the input image input in the frame unit into blocks of a specific size (for example, 4×4 pixels), and generates block position information used to define each processing target block (step S1 ).

Next, the coding mode determination unit 12 determines whether the coding mode when encoding the image of the target block based on the specific coding mode decision rule is either the predictive coding processing mode (P mode) or the padding coding mode (C mode), And output coding mode information used to define the coding mode (step S2). The block coding mode information is output to the image dividing unit 11, the predicted image generating unit 13, and the coding unit 16.

Next, the prediction image generation unit 13 determines whether the coding mode of the processing target block determined by the coding mode determination unit 12 is a predictive coding processing mode (step S3). If it is judged as negative (step S3; NO), the processing moves to step S11 in order to perform the image encoding processing of the next processing target block.

On the other hand, in the judgment of step S3, it is judged that the coding mode of the processing target block is the predictive coding processing mode (step S3; YES), the prediction image generation unit 13 determines the prediction mode, and adopts the prediction mode according to the determined prediction mode A part of the playback image that has been encoded and stored in the memory unit 19 is generated to generate a predicted image corresponding to the image of the processing target block (step S4). That is, the predicted image generation unit 13 performs the above-mentioned first image prediction process, and generates a prediction based on the playback image of the connected block connected to the processing target block and the non-connected block not connected to the processing target block. portrait. This predicted image is output to the subtraction unit 14 and the addition unit 18.

Next, the subtraction unit 14 subtracts the predicted image corresponding to the image of the processing target block in pixel units from the image of the processing target block (input image) to generate a predicted residual image (step S5). This predicted residual image is output to the conversion unit 15.

Furthermore, the conversion unit 15 converts the predicted residual image generated by the subtraction unit 14 according to a specific conversion rule, and calculates the conversion coefficient (conversion information) obtained by the conversion (step S6). This conversion coefficient is output to the encoding unit 16 and the inverse conversion unit 17.

Next, the encoding unit 16 encodes the conversion coefficient calculated by the conversion unit 15 on the average amount of information based on a specific rule (step S7). In addition, the encoding unit 16 averages the amount of information to encode the encoding mode information determined in step S2 based on a specific rule, and at the same time, based on the specific rule, averages the amount of information to encode the predicted image generation pattern selected in step S4. The average amount of information encodes these coded data, makes compressed data (bit stream) and outputs it to an external image decoding device.

Next, the inverse conversion unit 17 uses an inverse conversion rule corresponding to the specific conversion rule adopted by the conversion unit 15, and inversely converts the conversion coefficient calculated by the conversion unit 15 to generate a restored prediction residual image (step S8). This restored prediction residual image is output to the adding unit 18.

Furthermore, the adding unit 18 adds the predicted image generated by the predicted image generating unit 13 and the restored prediction residual image generated by the inverse conversion unit 17 to generate a broadcast image (step S9). The playback image is stored in the memory by the memory unit 19 and stored (step S10).

Next, it is judged whether the processing is finished for all the blocks (step S11), and when all the blocks are finished (step S11; YES), the image encoding process is finished. On the other hand, when all the blocks are not finished (step S11; NO), the process moves to step S2.

Secondly, a description will be given of the image encoding program related to the present invention and the computer-readable recording medium (hereinafter referred to as the recording medium) on which the image encoding program is recorded. The recording medium described here is a reading device equipped with the hardware resources of the computer. According to the description content of the program, it causes the change of the state of energy such as magnetism, light, electricity, etc., in the form of a signal corresponding to this The description content of the program is communicated to the reading device. As related recording media, for example, floppy disks, optical discs, CD-ROMs, and computer built-in memory, etc.

Fig. 8 is a diagram showing the structure of a recording medium related to the first embodiment. The recording medium 100 is shown in FIG. 8 and has a program area 101 of a recording program. The image coding program 102 is recorded in this program area 101.

FIG. 14 is a diagram showing the computer hardware structure used to execute the program recorded on the recording medium, and FIG. 15 is an oblique view of the computer used to execute the program recorded on the recording medium. As shown in FIG. 15, the computer 110 is provided with a reading device 112 such as a floppy disk drive device, a CD-ROM drive device, and a DVD drive device, and a working memory (RAM) 114 that usually exists in an operating system, and a memory recording medium The memory 116 for the program memorized in 100, the display device 118 for the display, etc., the mouse 120 and keyboard 122 for the input device, the communication device 124 for receiving and transmitting data, and the CPU 126 for controlling the execution of the program.

If the recording medium 100 is inserted into the reading device 112, the computer 110 can access the image encoding program 102 stored in the recording medium 100 from the reading device 112, and can use the image encoding program 102 to set the image encoding device 1 to operate .

As shown in FIG. 15, the image encoding program 102 can also be provided as a computer data signal 130 superimposed on a carrier wave via a network. At this time, the computer 110 can store the image encoding program 102 received by the communication device 124 in the memory 116 and execute the image encoding program 102.

The image encoding program 102 includes an image segmentation module 102a, an encoding mode determination module 102b, a prediction image generation module 102c, a subtraction module 102d, a conversion module 102e, an encoding module 102f, an inverse conversion module 102g, and an addition module 102h and memory module 102i. Here, by using the image division module 102a, the encoding mode determination module 102b, the predicted image generation module 102c, the subtraction module 102d, the conversion module 102e, the encoding module 102f, the inverse conversion module 102g, and the addition module The functions realized by the respective actions of 102h and memory module 102i are the same as those of the image dividing unit 11, the coding mode determining unit 12, the predicted image generating unit 13, the subtracting unit 14, the converting unit 15, the encoding unit 16, and the inverse image coding device 1 described above. The conversion unit 17, the addition unit 18, and the storage unit 19 each have the same function.

With the image coding device 1 of the first embodiment, the coding mode is the field of the padding coding mode, since it is not necessary to include the data generated based on the prediction auxiliary information in the bit stream, the bit stream with high coding efficiency is generated. flow.

In addition, when the coding mode of the connected block connected to the processing target block is the padding coding mode, the predicted image generating unit 13 may use the playback image of the non-connected block not connected to the processing target block to generate the predicted image. As a result, since not only the playback image of the connected block connected to the processing target block, but also the playback image of the non-connected block not connected to the processing target block can be used to generate the image, the reference range when generating the predicted image is expanded , And can reduce the redundant length in space, and at the same time can improve the coding efficiency. In addition, by more effectively limiting the image of the filling coding mode, the reference range when generating the predicted image will be more effectively expanded, and the spatial redundancy will be reduced.

[Second Implementation Type]

Next, the second embodiment of the present invention will be described. The image decoding device receives the compressed data (including coded data) output by the image encoding device of the first embodiment, that is, the bit stream, and decodes the received bit stream to generate playback image data.

Fig. 9 is a diagram showing the structure of an image decoding device related to the second embodiment. The image decoding device 2 shown in FIG. 9 can be physically set as a computer equipped with, for example, a CPU (Central Processing Unit), a memory device such as a memory, a display device such as a display, and a communication device. In addition, the image decoding device 2 may also be a mobile communication terminal such as a mobile phone, or a DVD device. That is, the image decoding device 2 can be widely applied to devices capable of information processing.

The image decoding device 2 shown in FIG. 9 has a decoding unit (decoding means) 21, an encoding mode judgment unit 22, a predicted image generation unit (the first image prediction means) 23, an inverse conversion unit (inverse conversion means) 24, and an addition unit (Play image generation means) 25, memory unit (memory means) 26, switch 27, and fill-in image generation unit (second image prediction means) 28.

Next, each structural element shown in FIG. 9 will be described. The decoding unit 21 receives input image information (compressed data) related to the input image of the decoding process object divided into blocks of a specific size. The decoding unit 21 decodes the received input image information based on a specific rule average amount of information. By decoding the average information amount, the conversion coefficient, coding mode information, and prediction mode information are decoded. These conversion coefficients, coding mode information, and prediction mode information are the same as the conversion coefficients, coding mode information, and prediction mode information of the differential image data described with respect to the image coding device 1, so the description is omitted.

The coding judgment unit 22 judges whether the coding mode is a predictive coding processing mode or a padding coding mode from the input coding mode information according to a specific scanning sequence (for example, a spot scanning sequence).

Here, the processing performed by the judgment result of the code judgment unit 22 differs depending on whether the scan in the frame is the first or second pass. Specifically, when the scan in the frame is the first time, it is determined that the encoding mode is the predictive encoding processing mode, and the processing target block is subjected to the predictive image decoding processing including the first image prediction processing described above; and the encoding mode is determined To fill the coding mode, read the next block. On the other hand, when the scan in the frame is the second time, it is judged that the coding mode is the padding coding mode, and the processing target block is subjected to padding image decoding processing including the above-mentioned image padding processing; the coding mode is judged to be predictive coding In the case of processing mode, read the next block. That is, in the first round, only the processing target block in the predictive coding processing mode is subjected to prediction image decoding processing including image prediction processing, and in the first round, only the processing target block in the padding coding processing mode is subjected to image padding. Processing of filling image decoding processing.

The prediction image generation unit 23, in accordance with the prediction mode defined by the decoded prediction mode information, completely decodes the image of each block, and uses part of the playback image memorized by the storage unit 26 to generate the processing target block of the decoding processing target The prediction portrait. The image prediction process used to generate this predicted image is the same as the first image prediction process described above (refer to Figure 2 and Figure 3), so the description is omitted.

The inverse conversion unit 24 uses an inverse conversion rule corresponding to the specific conversion rule adopted by the conversion unit 15 to reverse the decoded conversion coefficients, and generates the restored prediction residual image obtained by the inverse conversion.

The adding unit 25 adds the predicted image and the restored predicted residual image corresponding to the predicted image to generate a broadcast image. In addition, when the adding unit 25 sets a specific range for the pixel value of the image, it may also limit the pixel value within the specific range and perform clipping processing (clipping).

The memory 26 stores the playback image generated by the addition unit 25 in a memory not shown.

The switch 27 is adapted to the encoding mode of the processing target block to switch the transmission location of the play image stored in the memory unit 26. That is, the switch 27 is switched when the encoding mode is the predictive encoding processing mode, so that the playback image stored in the storage unit 26 can be transmitted to the predictive image generating unit 23. On the other hand, when the encoding mode is the padding encoding processing mode, the switch is switched so that the playback image stored in the storage unit 26 can be transmitted to the padding image generating unit 28.

The filling image generation unit 28 uses the decoded playback image near the processing target block to generate the filling image. Here, the image filling process when the filling image is generated is the same as the image filling process described above (refer to FIG. 6, the second image prediction process), so the description is omitted.

Next, referring to FIG. 10 to FIG. 12, the operation of the image decoding device 2 and the image decoding process will be described. First, referring to FIG. 10, the outline of the operation of the image decoding process will be described. As shown in Fig. 10, in the image decoding process, firstly, by the decoding unit 21, the input image information of 1 frame part received from the image encoding device 1 of the first embodiment is decoded based on the average amount of information according to a specific rule to generate Conversion coefficient, coding mode information, and prediction mode information (step S20).

Next, perform the prediction image decoding process performed in the first in-frame scan (step S30), and then perform the fill-in image decoding process performed in the second in-frame scan (step S40).

Hereinafter, each operation of the predicted picture decoding process (step S30) and the padding picture decoding process (step S40) will be explained in detail in each process.

First, referring to FIG. 11, the detailed operation of the prediction image decoding process will be described. First, the code determining unit 22 determines whether the coding mode defined by the input code information is a predictive coding processing mode according to a specific scanning sequence (for example, a spot scanning sequence) (step S31). If this determination is NO (step S31; NO), the process is shifted to step S36, which will be described later.

On the other hand, in the judgment of step S31, it is judged that the coding mode is the predictive coding processing mode (step 31; YES), and the prediction image generation unit 23 is defined in accordance with the prediction mode information decoded in step S20 (refer to FIG. 10) In the prediction mode, a part of the playback image completely decoded from the image of each block and stored in the memory unit 26 is used to generate the predicted image of the processing target block (step S32). Furthermore, the method of generating this predicted portrait is based on the first portrait prediction process described above. In addition, this predicted image is output to the adding unit 25.

Secondly, the inverse conversion unit 24 uses an inverse conversion rule corresponding to the specific conversion rule used in the conversion unit 15 with respect to the conversion coefficient decoded in the above step S20 (refer to FIG. 10), performs the inverse conversion, and generates the The recovered predicted residual image obtained by the reverse conversion (step S33). This restored prediction residual image is output to the adding unit 25.

Secondly, the adding unit 25 adds the predicted image generated by the predicted image generating unit 23 and the reverse-converted return predicted residual image by the inverse conversion 24 to generate a broadcast image. The playback image is stored in the memory by the memory unit 26 and stored (step S35).

Next, it is judged whether the processing is finished for all the blocks (step S36), and when all the blocks are finished (step S36; YES), the prediction picture decoding process is finished. On the other hand, when all the blocks are not finished (step S36; NO), the process is shifted to the above-mentioned step S31.

With reference to FIG. 12, the detailed operation of the padding image decoding process (step S40 in FIG. 10) will be described. First, the code determining unit 22 determines whether the code mode defined by the input code information is a padding code mode according to a specific scanning sequence (for example, a light spot scanning sequence) (step S41). If this determination is NO (step S41; NO), the process moves to step S44 described later.

On the other hand, in the judgment of step S41, it is judged that the coding mode is the padding coding mode, (step 41; YES), the padding image generation unit 28, in each processing target pixel included in the processing target block, is located in each The playback image around the processing target pixel is obtained, and the playback pixel value with the largest correlation is obtained, and the padding pixel value is calculated to generate the padding image corresponding to the processing target block (step S42). Furthermore, the method of generating this filled portrait is based on the above-mentioned portrait filling process (second portrait prediction process).

Next, the filling image generated by the filling image generating unit 28 is used as a playback image, and is stored in the memory by the memory unit 26 and stored (step S43).

Next, it is judged whether the processing is finished for all the blocks (step S44), and when all the blocks are finished (step S44; YES), the padding image decoding process is finished. On the other hand, when all the blocks are not finished (step S44; NO), the process moves to the above-mentioned step S41.

Fig. 13 is a diagram showing the structure of a recording medium related to the second embodiment. The recording medium 100 is shown in FIG. 13 and has a program area 201 of a recording program. In this program area 201, an image decoding program 202 is recorded.

If the recording medium 100 is inserted into the reading device 112, the computer 110 (refer to FIG. 14 and FIG. 15) will be able to access the image decoding program 202 stored in the recording medium 100 from the reading device 112, and with the image decoding program 202, It is assumed that the image decoding device 2 operates.

As shown in FIG. 15, the image decoding program 202 can also be provided as a computer data signal 130 superimposed on a carrier wave via a network. At this time, the computer 110 can store the image decoding program 202 received by the communication device 124 into the memory 116 and execute the image decoding program 202.

The image decoding program 202 includes a decoding module 202a, an encoding mode judgment module 202b, a predicted image generation module 202c, an inverse conversion module 202d, an addition module 202e, a memory module 202f, a switch module 202g, and fill image generation It is composed of module 202h. Here, the decoding module 202a, the encoding mode judgment module 202b, the predicted image generation module 202c, the inverse conversion module 202d, the addition module 202e, the memory module 202f, the switch module 202g, and the filling image are generated The functions realized by the respective actions of the module 202h are the same as the decoding unit 21, decoding mode judgment unit 22, predicted image generation unit 23, inverse conversion unit 24, addition unit 25, storage unit 16, and switch 27 of the image decoding device 2 described above. The functions of the filling image generating units 28 are the same.

With the image decoding device 2 of the first implementation type, the image effectively encoded by the image encoding device 1 can be effectively decoded. That is, when the coding mode of the connected block connected to the processing target block is the padding coding mode, the predicted image generation unit 23 can use the playback image of the non-connected block not connected to the processing target block to generate the predicted image. As a result, since not only the playback image of the connected block connected to the processing target block, but also the playback image of the non-connected block not connected to the processing target block can be used to generate the image, the reference range when generating the predicted image is expanded , And can reduce the length of space.

In addition, in the case of judging that the coding mode is the padding coding mode, a fully decoded playback image can be used to generate a padding image that fills the playback image. Therefore, even if the picture in the padded coding mode is not transmitted during encoding, it is possible to generate a playback picture.

In addition, in each of the above embodiments, although the intra-frame coding process has been described as the image prediction of the predicted image generating units 13, 23, it is not limited to this. For example, motion compensation prediction may be included in the image. predict. In this case, it is also possible to average the amount of information to encode the motion vector information. In addition, as this motion compensation prediction, the same method as the traditional MPEG-4 or H.264 motion compensation prediction can be used. In addition, various other motion compensation prediction methods may be applied.

In addition, in each of the foregoing implementation types, the prediction of the spatial domain used in H.264 is used as the intra-frame prediction, but the intra-frame prediction is not limited to this. For example, predictions in the frequency domain made by MPEG-4 can also be applied.

In this case, in the case of encoding or decoding of the block connected to the top or left, all or part of the conversion coefficient of the connected block is used as the predicted value of the conversion coefficient of the target block for encoding or decoding. When the blocks connected to the top and left are both coded or decoded, it shows the 1-bit data of which block conversion coefficient is used as the predicted value in the blocks connected to the top and the left, and is set to the prediction mode Information is encoded. In addition, if any one of the blocks connected to the top or the left is coded or decoded, all or part of the orthogonal conversion coefficients of the coded or decoded block are used as the predicted value.

On the other hand, if the encoding or decoding of any one of the blocks connected to the upper or left side is not completed, all the orthogonal conversion coefficients of the encoded or decoded block located at the distance above or to the left of the block are used. Or part of it, as the predicted value. However, if the coded or decoded block located closest to the upper and left distances is also at the same distance as the upper and left, the blocks located at the upper and the left show all the conversion coefficients of which block is used Or part of the 1-bit data as the predicted value is set as the prediction mode information to be coded. In addition, when all the blocks above and to the left of the coding or decoding target block are coded or not decoded, only the DC component is set to a set value (for example, 128) for prediction.

In addition, in each of the above implementation types, the prediction value can also be set to 0 by intra-frame prediction, and the intra-frame prediction is not performed and the operation is performed. In this case, since the redundancy of the space is reduced by introducing the image filling mode, the coding efficiency can also be improved.

In addition, in each of the above-mentioned implementation types, although the image fill-in process uses the play image in the same space as the processed object frame to generate the predicted image, it can also be used in a frame that is different from the processed object frame. Finish the picture fill-in process of the picture frame that generates the picture frame of the picture.

In addition, in the first embodiment described above, the predicted image generated by the image padding process is still used as the playback image, but in terms of encoding, the predicted image and the predicted residual image of the input image can also be converted and averaged as described above. The amount of information is coded and included in the bit stream. In this case, on the decoding side, by decoding and inversely converting the data of the processed bit stream, a restored prediction residual image is generated, and by adding the restored prediction residual image and the predicted image generated by the image padding process, the playback is generated portrait.

[The third implementation type]

Hereinafter, the third embodiment of the present invention will be described. Fig. 16 is a structural diagram showing an animation encoding device related to the third embodiment of the present invention. The animation encoding device 310 shown in FIG. 16 can be physically set as a computer equipped with, for example, a CPU (Central Processing Unit), a memory device such as a memory, a display device such as a display, and a communication device. In addition, the animation encoding device 310 may also be a mobile communication terminal such as a mobile phone, or a DVD device. In other words, the image encoding device 310 can be widely applied to devices capable of information processing.

The animation encoding device 310 functionally includes a domain division unit 312, an encoding mode determination unit (deciding means) 314, an image prediction unit 316, a subtraction unit 318, a conversion unit (conversion means) 320, an encoding unit (coding means) 322, and an inverse conversion unit (Reverse conversion means) 324, addition part (play image generation means) 326, and image memory part (memory means) 328.

The area dividing unit 312 sequentially sets each frame of the input image (animation) as an encoding target frame, and divides the encoding target frame into a plurality of partial areas of a specific size (hereinafter referred to as micro-blocks). The specific size may be, for example, the size of 16×16 pixels, but is not limited to this, and may be other sizes.

The domain dividing unit 312 generates micro-block location information used to define the micro-blocks. For example, the location information of the micro-blocks can be used in each micro-block in the frame with an integer number marked from large to small in accordance with the scanning order of the light spots.

In addition, the position information of the micro-blocks can be based on the scanning order of the light spots, using the pixel coordinates at the front end of the micro-blocks.

The coding mode determination unit 314 obtains the coding mode of each micro-block, and outputs the coding mode information that defines the coding mode to the image prediction unit 316 and the coding unit 322. In this embodiment, the coding mode information is used to define the information of the predicted image of the micro-block that should be generated by the first image prediction process or the second image prediction process. Hereinafter, the first picture prediction process is set to motion compensation prediction, and this embodiment is described. In addition, in the following, in this specification, the coding mode when processing micro-blocks in the first picture prediction process should be set to P mode, and the coding mode when processing micro-blocks in the second picture prediction process should be set to C mode. In addition, the first image prediction processing can employ, for example, processing such as well-known Internet frame prediction. The details of the first image prediction process and the second image prediction process will be described later.

The image prediction unit 316 uses one of the first image prediction process or the second image prediction process based on the encoding mode information output by the encoding mode determination unit 314 to generate a predicted image of the micro-block of the processing target. The image prediction unit 316 processes the micro-blocks by the first image prediction process, and outputs the motion vector of the prediction auxiliary information to the encoding unit 322. The details of the portrait prediction section 316 will be described later.

The subtraction unit 318 performs the calculation of the difference between the predicted image of the processing target micro-block generated by the image prediction unit 316 and the input image of the processing target micro-block output from the field segmentation unit 312 to generate a predicted residual image.

The conversion unit 320 converts the predicted residual image to generate conversion information. For this conversion, for example, DCT (Discrete Cosine Transform) and other conversion processing and dequantization processing can be used. In this case, the conversion information includes a set of quantization coefficients. DCT can be a 2-dimensional DCT with 4 rows and 4 columns, or a 2-dimensional DCT with 8 rows and 8 columns. In addition, this conversion can adopt any conversion processing such as 4 rows and 4 columns integer straight conversion and quantization, MP method (Matiching Pursuit), or vector quantization and wavelet transformation (Wavelet Transform) used in H.264. Quantum processing.

The encoding unit 322 uses the average amount of information to encode the micro-block position information from the field segmentation unit 312, the encoding mode information from the encoding mode determination unit 314, the motion vector from the image prediction unit 316, and the conversion from the conversion unit 320 The encoding process of information, etc. is encoded, and the encoding result is included in the bit stream and output.

The inverse conversion unit 324 inversely converts the conversion information from the conversion unit 320 to generate a restored prediction residual image that restores the predicted residual image. The conversion information includes the quantization coefficient group, and the inverse conversion unit 324 performs inverse quantization processing and inverse conversion processing on the quantum coefficients to generate a restored prediction residual image that restores the predicted residual image.

The adding unit 326 generates a broadcast image by adding the restored predicted residual image from the inverse conversion unit 324 and the predicted image from the image predicting unit 316, and combines this into the playing frame to be memorized in the image memory unit 328. In addition, in the case of the pixel value setting range of the image data, the addition unit 326 may also capture the pixel value of the addition result in order to condense within this range.

The image storage unit 328 stores the playback frame and reference frame of the playback image completed by grouping into the micro-blocks of the encoding target frame. This reference frame is a playback frame that is different from the encoding target. In this embodiment, it is a playback frame before a frame.

The encoding mode determination unit 314 will be described below. Fig. 17 is a diagram for explaining the processing of the coding mode determination unit. Regarding Figure 17, the quadrangular areas are represented as blocks, and the solid and dotted lines marked with arrows represent the scanning order.

As shown in (a) of this figure, the coding mode determining unit 314 first generates a first image prediction process (motion compensation prediction process) in a specific scanning sequence (for example, a spot scanning sequence) of the coding target frame The predicted images of all the micro-blocks (that is, the predicted images are generated in P mode), and the playback frame formed by the playback images of all the micro-blocks of the encoding target frame is generated.

As shown in (b) of this figure, the coding mode determination unit 314 generates the coding target image by generating predicted images of all micro-blocks in the coding mode target frame of the second image prediction process in the reverse scanning order of the above. Play images of all micro-blocks in the frame.

When the coding mode determination unit 314 generates the predicted image by using the second image prediction process, it uses the micro-blocks of the lithium object to use the playback image of the micro-blocks located in the front in a specific scanning order, and uses a specific scan The sequence is located in the rear micro-block, the coding mode is the playback image of the P-mode micro-block, and the reference frame stored in the image memory 328. For example, as shown in this figure (c), if the micro-block marked "A" is set as the micro-block to be processed, the micro-block in the playback frame will be used instead of the one in front of block A. The playback image of the micro-block, compared to the playback image of the micro-block behind A and whose encoding mode is P mode (the micro-block marked "P" in the figure), and the reference frame memorized in the image memory section 328 , Perform the second image prediction process. The detailed content related to the second portrait prediction process will be described later.

Each time the coding mode determination unit 314 generates a replay picture through the second picture prediction process, it determines by calculating the respective costs of the replay picture generated by the first picture prediction process and the replay picture generated by the second picture prediction process. Encoding mode.

This cost is, for example, the number of bits R required for the micro-block coding of the processing object, the square D of the error between each pixel of the playback image of the micro-block and each image of the input image of the micro-block, and the preset When the coefficient is set to λ, use D+λ. R is calculated. The coding mode determining unit 314 selects the P mode or the C mode, and the mode with the lower cost is set as the coding mode. In addition, if the cost can show coding efficiency or/and portrait, it can also be calculated from various functions.

The following describes the portrait prediction unit 316 in detail. FIG. 18 is a block diagram showing the structure of the portrait prediction unit 316. The image prediction unit 316 includes a mode switching unit 330, a first image prediction unit (first image prediction means) 332, and a second image prediction unit (second image prediction means) 334.

The mode switching unit 330 receives the coding mode information from the coding mode determination unit 314, and activates the first picture predicting unit 332 or the second picture predicting unit 334 mode switching unit 330 according to the coding mode defined by the coding mode information, and the coding mode is In the case of the P mode, the first picture prediction unit 332 is activated, and when the coding mode is the C mode, the second picture prediction unit 334 is activated.

In the animation encoding device 310, the mode switching unit 330 initially generates the predicted images of all the micro-blocks of the coding target frame by the first image prediction unit 332, and the coding mode is the predicted image of all the micro-blocks of the P mode. After that, the mode switching unit 330 generates the predicted images of all the micro-blocks in the coding target frame through the second image prediction unit 334, and the coding mode is the predicted image of all the micro-blocks in the C mode.

As described above, the first picture prediction unit 332 generates a broadcast picture through the first picture prediction process, that is, the motion compensation prediction process. The so-called motion compensation prediction processing is to perform block matching between the input image of the micro-block of the processing object and any area of the reference frame, using the part of the image in the reference frame area with the highest correlation as the playback image, and generate The processing of the movement vector to the field. This relationship uses, for example, the difference square D between the input image of the micro-block of the object processing and the playback image of the field of the matching object, the data length R required for the movement vector coding from the micro-block of the processing object to the field, and The predetermined coefficient λ is calculated from the reference function of D+λ R. In addition, motion compensation prediction processing can also be performed by using upsampling reference frame of 2 times or 4 times the image. In addition, the motion compensation prediction process can also be implemented when the micro-block is divided into smaller blocks. At this time, the block division mode representing the motion vector of each small block and the type of micro block division is output.

The second picture prediction unit 334 uses the second picture prediction process to generate a prediction picture of a micro-block whose coding mode is determined to be C mode. Fig. 19 is a diagram for explaining the second image prediction process. In the second image prediction process, select the microblock Y that has not generated a play image. In this micro-block Y, select unprocessed pixels that are not applied to the pixel value (prediction signal) of the image to be played, and set a template T that includes the pixel in a part of it. In this embodiment, although the template T based on the above-mentioned unprocessed pixels is set, if it includes processed pixels and unprocessed pixels, the template T can have any size and shape. Moreover, for the micro-block Y, the reference area R of any size is set relative to the playback frame F1. In addition, for the reference frame F2 stored in the image storage unit 328, the reference area R is also set. The reference area R of the reference frame F2 can also be set at a position equivalent to the reference frame F2 located at the position of the reference area R of the playback frame F1. Alternatively, the reference area R of the reference frame F2 can also be set at a position that deviates from the movement vector generated by the median of the movement vector of the micro-block around the micro-block Y.

In the second image prediction process, the correlation between the candidate area S of the generated play image and the template T located at any position of the reference area R is calculated, and the candidate area with the highest correlation is selected to copy the reference area Sc. For example, the corresponding position between the template T and the candidate area S generates the pixel value of the playback image, which is set as a valid pixel. The square of the difference between the pixel value of the effective pixel of the template T and the candidate area S , That is, the matching error M is calculated from the value divided by the effective pixel number U. In addition, if the correlation can show the pixel similarity between the template T and the candidate area S, it can also be calculated by various calculation methods.

In the second image prediction process, the pixel value of the corresponding pixel in the reference area Sc is copied on the unprocessed pixel in the template T that has not been given the pixel value of the image to be played, as the fill-in pixel value. In FIG. 19, only the part of the play portrait of the generated template T is marked with oblique lines. In FIG. 19, the area in the template T where no play image is generated (the lower part in the template T) shows the pixel value of the lower part of the reference area Sc given to the copy. In the second image prediction process, the relevant micro-blocks are repeated until the unprocessed pixels of the pixel value of the image to be played are not applied, and the pixels are repeatedly selected and the image to be played is generated.

Hereinafter, the operation of the animation encoding device 310 will be described. At the same time, the animation coding method related to the implementation of the present invention is described. Fig. 20 is a flowchart showing the animation encoding method related to the third embodiment.

As shown in FIG. 20, in the encoding process of the animation encoding device 310, the area segmentation unit 312 divides the encoding target frame into a plurality of micro-blocks (step S301). In addition, the above-mentioned micro-block location information is generated by the domain dividing unit 312.

Next, an encoding mode determination process for determining the encoding mode of each micro-block by the encoding mode determination unit 314 is performed (step S302). Fig. 21 shows a flowchart of the encoding mode determination process of the animation encoding method related to the third embodiment. As shown in Figure 21, in the encoding mode determination process, first, according to the scanning sequence (spot scanning sequence), through the first image prediction process (motion compensation prediction process), the playback image of all micro-blocks is generated. The play frame (step S302-1).

Next, according to the scanning order, the last micro-tile is selected as the micro-tile to be processed (step S302-2), and the second image prediction process described above is executed. In the second image prediction process, as shown above, the play frame generated through step S302-1 is used to scan the image of the micro-block of the processing target in the front of the micro-block in the scanning order, and process The micro-blocks of the object are located in the rear micro-blocks in the scanning order, the coding mode is the playback image of the P-mode micro-blocks, and the reference frame is used to generate the playback image.

Secondly, the cost of the playback portrait of the micro-block of the processing target generated in step S302-3 and the cost of the playback portrait of the micro-block of the processing target generated in step S302-1 are calculated separately using the above cost function ( Step S302-4). Next, in the first picture prediction process, that is, the P mode, or the second picture prediction process, that is, the C mode, a mode with a lower cost is selected as the coding mode (step S302-5).

Furthermore, it is tested whether the processing of all micro blocks is finished (step S302-6). When all micro-block processing is completed (Yes), the encoding mode determination processing is ended. On the other hand, in the case that all micro-block processing is not completed (No), follow the reverse scanning sequence to select the micro-block whose encoding mode has not been determined (step S302-7), and repeat from step S302-3 to step S302 -6 treatment.

Returning to FIG. 20, next, in the video encoding device 310, the encoding mode information is encoded by the encoding unit 322 (step S303). Then, the mode switching unit 330 selects the micro-block as the processing target according to a specific scanning sequence (such as the light spot scanning sequence), and tests whether the coding mode of the micro-block of the processing target is P mode (the first image) Prediction processing) (step S304). If the coding mode is not P mode (No), the process moves to step S312. On the other hand, when the coding mode is P mode (Yes), the mode switching unit 330 activates the first image prediction unit 332, and generates a predicted image of the micro-block to be processed through the first image prediction process (step S305).

Furthermore, the subtraction unit 318 generates a predicted residual image formed by the difference between the input image and the predicted image of the micro-block of the processing target (step S306). Next, the prediction residual image is converted into conversion information by the conversion unit 320 (step S307). Then, the encoding unit 322 averages the amount of information to encode the motion vector and the conversion information (step S308).

Secondly, the inverse conversion unit 324 performs inverse conversion on the conversion information, and generates a restored predicted residual image that restores the predicted residual image (step S309). Furthermore, the adding unit 326 adds the restored predicted residual image and the predicted image from the image predicting unit 316, thereby generating a playback image of the micro-block of the processing target (step S310). The playback picture is grouped into the playback picture frame and stored in the picture storage unit 328 (step S311).

Furthermore, it is tested whether the processing of all micro blocks is finished (step S312). In the case that all micro-tile processing is not completed (No), select the unprocessed micro-tile (step S302-7), and repeat the processing from step S304 to step S312. On the other hand, if the processing of all micro blocks is completed (Yes), the process moves to step S313.

In step S313, the mode switching unit 330 selects the micro-block as the processing object according to a specific scanning sequence (such as the light spot scanning sequence), and tests whether the coding mode of the micro-block of the processing object is the C mode ( The second image prediction process). When the coding mode is not C mode (No), the process moves to step S312.

On the other hand, when the coding mode is the C mode (Yes), the mode switching unit 330 activates the second image prediction unit 334, and generates a predicted image of the micro-block to be processed through the second image prediction process (step S314). Fig. 22 shows a flowchart of the second image prediction process of the animation coding method related to the third embodiment.

As shown in Fig. 22, in the second picture prediction process, a block in which no replay picture has been generated is selected. In this embodiment, the micro-blocks whose coding mode is C mode are set as the micro-blocks to be processed, and they are selected according to a specific scanning sequence (step S314-1).

Next, the reference area relative to the micro-block of the processing target is set as described above (step S314-2). Next, select a pixel in the micro-block that does not generate a playback image (step S314-3), and set a template that includes the pixel in a part of it (step S314-4).

The correlation between this template and the candidate area of the reference area is calculated as described above (step S314-5), and the candidate area with the highest correlation is selected as the copy reference area (step S314-6).

Secondly, copy the unprocessed pixels in the template where the pixel value of the playback image is not applied, and copy the pixel value of the corresponding pixel in the reference area (step S314-7). Next, it is tested whether there are unprocessed pixels that have not been given the pixel value of the playback pixel in the micro-block of the processing object (step S314-8). If there is an unprocessed pixel (Yes), select the unprocessed pixel (step S314-9), and repeat the processing of step S314-4 to step S314-8. On the other hand, if there is no unprocessed pixel in the processing target microblock, the second image prediction process is ended (step S314).

Returning to FIG. 20, in the animation encoding device 310, the subtracting unit 318 generates the input image of the micro-block of the processing target, and the predicted residual image formed by the difference between the predicted image generated by the second image prediction process (step S315).

Next, the predicted residual image is converted into conversion information by the conversion unit 320 (step S316). Then, the information is coded and converted by the average amount of information by the coding unit 322 (step S317).

The inverse conversion unit 324 performs inverse conversion on the conversion information, and generates a restored predicted residual image that restores the predicted residual image (step S318). Furthermore, the adding unit 326 adds the restored predicted residual image and the predicted image from the image predicting unit 316 to generate a play image of the micro-block of the processing target (step S319). The playback picture is grouped into the playback picture frame and stored in the picture storage unit 328 (step S320).

Furthermore, it is tested whether the processing of all micro blocks is finished (step S312). In the case that all micro-tile processing is not completed (No), select the unprocessed micro-tile (step S302-7), and repeat the processing from step S304 to step S312. On the other hand, if all micro-blocks are processed (Yes), the encoding process will end.

Hereinafter, an animation coding program that uses a computer as the animation coding device 310 and makes it operate will be described. Fig. 23 is a diagram showing the structure of the animation coding program related to the third implementation type and the recording medium co-presented. The animation encoding program 340 shown in FIG. 23 is stored in the recording medium 100 for use.

The recording medium 100 is, for example, a recording medium such as a magnetic disk, CD-ROM, DVD, or ROM, or a semiconductor memory.

If the recording medium 100 is inserted into the reading device 112, the computer 110 (refer to FIGS. 14 and 15) will be able to access the animation encoding program 340 stored in the recording medium 100 from the reading device 112, and with the animation encoding program 340, It operates as an animation encoding device 310.

As shown in FIG. 15, the animation encoding program 340 can also be provided as a computer data signal 130 superimposed on a carrier wave via a network. At this time, the computer 110 can store the animation encoding program 340 received by the communication device 124 in the memory 116 and execute the animation encoding program 340.

As shown in FIG. 23, the animation encoding program 340 is equipped with a main module 341 for general processing, a domain division module 342, an encoding mode determination module 344, an image prediction module 346, a subtraction module 348, a conversion module 350, and an encoding module. Module 352, inverse conversion module 354, addition module 356 and image memory module 358. The image prediction module 346 includes a mode switching module 360, a first image prediction module 362, and a second image prediction module 364.

Domain division module 342, encoding mode determination module 344, image prediction module 346, subtraction module 348, conversion module 350, encoding module 352, inverse conversion module 354, addition module 356, image memory module 358 , The mode switching module 360, the first image prediction module 362, and the second image prediction module 364 each perform the functions of a computer, and the above-mentioned domain division unit 312, encoding mode determination unit 314, image prediction unit 316, and subtraction unit 318 , The conversion unit 320, the encoding unit 322, the inverse conversion unit 324, the addition unit 326, the image storage unit 328, the mode switching unit 330, the first image prediction unit 332, and the second image prediction unit 334, the corresponding parts are the same.

Hereinafter, the function and effect of the animation encoding device 310 will be described. If the image encoding device 310 of the third implementation type is used to encode the micro-blocks of mode C, that is, the micro-blocks of the predicted image generated by the second image prediction process, it is not necessary to use the information based on the prediction auxiliary information. The generated data is contained in the bit stream, so a bit stream with high coding efficiency is generated.

In the animation encoding device 310, the second image prediction unit 334 uses the reference frame and the playback frame of the encoding target frame to generate a predicted image. This play frame contains the play image completely generated through the first image prediction process and the second image prediction process. That is, the second image prediction unit 334 uses the reference frame formed by the playback image of the frame different from the encoding target frame and the playback frame formed by the playback item of the encoding target frame to generate the predicted image . Therefore, the coded data that can reduce the redundant length in the time direction and the space direction is generated.

In addition, in the second picture prediction process, a play frame including a play picture completely generated through the first picture prediction process and the second picture prediction process is used. Therefore, it is also possible to use the playback image at the back due to the scanning order for prediction, so the redundancy in the spatial direction can be more effectively reduced.

[Fourth Implementation Type]

Hereinafter, the animation decoding device of the fourth embodiment of the present invention will be described. Fig. 24 is a structural diagram showing an animation decoding device related to the fourth embodiment of the present invention. The animation decoding device 370 shown in FIG. 24 can be physically set as a computer equipped with, for example, a CPU (central processing unit), a memory device such as a memory, a display device such as a display, and a communication device. In addition, the animation decoding device 370 may also be a mobile communication terminal such as a mobile phone, or a DVD device. That is, the animation decoding device 370 can be widely applied to devices capable of information processing.

As shown in FIG. 24, the animation decoding device 370 has a decoding unit (decoding means) 372, an inverse conversion unit (inverse conversion means) 374, an image prediction unit 376, an addition unit (play image generation means) 378, and an image storage unit (memory means) )380.

The decoding unit 372 receives the input bit stream and decodes the bit stream. Upon receiving the bit stream generated by the animation encoding device 310, the decoding unit 372 decodes the bit stream to generate micro-block position information, coding mode information, motion vector, and conversion information.

The inverse conversion unit 374 receives the conversion information from the decoding unit 372, and performs inverse conversion on the processing target micro-block conversion information, thereby generating a restored prediction residual image of the processing target micro-block. The inverse conversion unit 374 outputs this restored prediction residual image to the addition unit 378. In addition, the processing performed by the reverse conversion unit 374 is the same as the processing performed by the reverse conversion unit 324 of the animation encoding device 310.

The image prediction unit 376 generates a prediction image of the micro-block of the processing target, and outputs the prediction image to the addition unit 378. The image prediction unit 376 includes a mode switching unit 382, a first image prediction unit (first image prediction means) 384, and a second image prediction unit (second image prediction means) 386. The mode switching unit 382 activates the first picture predicting unit 384 or the second picture predicting unit 386 based on the coding mode information from the decoding unit 372. The first image prediction unit 384 generates the predicted image of the micro-block of the processing target by the first image prediction process; the second image prediction unit 386 generates the predicted image of the micro-block of the processing target by the second image prediction process. The processing performed by these elements included in the image prediction unit 376 is the same as the processing performed by the corresponding elements of the image prediction unit 316 of the animation encoding device 310.

The adding unit 378 adds the predicted image from the image predicting unit 376 and the restored predicted residual image from the inverse conversion unit 374 to generate the predicted image of the micro-block of the processing target and store it in the image memory unit 380. Furthermore, when the pixel value of the image data is set to a specific range, the addition unit 378 may also limit the pixel value within this range for capturing processing.

The image storage unit 380 memorizes the playback frame of the decoded target frame of the generated playback image and the reference frame. The reference frame, as shown above, is a playback frame that is different from the decoded target frame. In this embodiment, it is the playback frame before one of the decoded target frames.

Hereinafter, the operation of the animation decoding device 370 will be described. Fig. 25 is a flowchart showing the animation decoding method related to the fourth embodiment.

As shown in FIG. 25, in the decoding process of the animation decoding device 370, the decoding unit 372 first decodes the coding mode information of all micro-blocks of the target frame and decodes it from the bit stream (step S331).

Then, the mode switching unit 382 selects the micro-block as the processing object according to a specific scanning sequence (such as the light spot scanning sequence), and tests whether the coding mode of the micro-block of the processing object is the P mode (the first image prediction Processing) (Step S332). If the coding mode is not P mode (No), the process moves to step S338. On the other hand, when the encoding mode is the P mode (Yes), the decoding unit 372 decodes the conversion information and the motion vector of the micro-block to be processed from the average information amount of the bit stream (step S333). Furthermore, the mode switching unit 382 activates the first image prediction unit 384, and generates a predicted image of the micro-block of the processing target through the first image prediction process (step S34).

Next, the inverse conversion unit 374 performs inverse conversion on the conversion information to generate a restored prediction residual image (step S335). Next, by the adding unit 378, the recovered predicted residual image and the predicted image are added, and a playback image is generated (step S336). The play portrait is grouped into the play frame and stored in the portrait storage unit 380 (step S337).

Next, it is tested whether the processing of all micro blocks is finished (step S338). If all the block processing is not completed (No), select the unprocessed micro-block, and repeat the processing from step S332 to step S338.

On the other hand, if the processing of all blocks is completed (Yes), the process moves to step S339. In step S339,

With the mode switching unit 82, in accordance with a specific scanning sequence (such as a light spot scanning sequence), select a micro-block as the processing object, and test whether the coding mode of the micro-block of the processing object is the C mode (second image prediction processing ) (Step S339). When the coding mode is not C mode (No), the process moves to step S345. On the other hand, when the coding mode is the C mode (Yes), the decoding unit 372 decodes the conversion information of the micro-block to be processed from the average information amount of the bit stream (step S340). Furthermore, the mode switching unit 382 activates the second image prediction unit 386, and generates a predicted image of the micro-block of the processing target through the second image prediction process (step S341).

Next, the inverse conversion unit 374 performs inverse conversion on the conversion information, and generates a restored prediction residual image (step S342). Furthermore, the adding unit 378 adds the recovered predicted residual image and the predicted image to generate a broadcast image (step S343). The play portrait is grouped into the play frame, and stored in the portrait storage unit 380 (step S344).

Furthermore, it is tested whether the processing of all micro blocks is finished (step S345). In the case that all micro-block processing is not completed (No), select the unprocessed micro-block, and repeat the processing from step S339 to step S345. On the other hand, if all micro-block processing is finished (Yes), the decoding process is finished.

Hereinafter, an animation decoding program that uses a computer as the animation decoding device 370 and operates it will be described. Fig. 26 is a diagram showing the structure of the animation decoding program related to the fourth embodiment and the co-presentation of the recording medium. The animation decoding program 390 shown in FIG. 26 is stored in the recording medium 100 for use. The recording medium 100 is, for example, a recording medium such as a magnetic disk, CD-ROM, DVD, or ROM, or a semiconductor memory.

If the recording medium 100 is inserted into the reading device 112, the computer 110 (refer to FIGS. 14 and 15) will be able to access the animation decoding program 390 stored in the recording medium 100 from the reading device 112, and with the animation decoding program 390, It operates as the animation decoding device 370.

As shown in FIG. 15, the animation decoding program 390 can also be provided as a computer data signal 130 superimposed on a carrier wave via a network. At this time, the computer 110 can store the animation encoding program 340 received by the communication device 124 into the memory 116 and execute the animation decoding program 390.

As shown in FIG. 26, the animation decoding program 390 includes a main module 391, a decoding module 392, an inverse conversion module 394, an image prediction module 396, an addition module 398, and an image memory module 400 for general processing. The image prediction module 396 includes a mode switching module 402, a first image prediction module 404, and a second image prediction module 406.

Decoding module 392, reverse conversion module 394, image prediction module 396, addition module 398 and image memory module 400, mode switching module 402, first image prediction module 404, and second image prediction module 406 Functions executed on a computer, and the above-mentioned decoding unit 372, inverse conversion unit 374, image prediction unit 376, addition unit 378, image storage unit 380, mode switching unit 382, first image prediction unit 384, and second image prediction unit 386 , The corresponding elements have the same function.

As shown in the above description, the animation decoding device 370 can recover the animation based on the bit stream generated by the animation encoding device 310. In addition, the animation decoding device 370 can generate the micro-blocks whose coding mode is C mode, that is, the micro-blocks of the predicted picture generated by the second picture prediction process, without obtaining the prediction auxiliary information such as the motion vector from the coding side. Forecast portrait.

In addition, the present invention is not limited to the third and fourth embodiments described above, and various changes are possible. For example, in the third embodiment, in the second image prediction process, the playback frame formed by the playback image of the encoding target frame is referred to. Instead of playing the frame here, and referring to the encoding target frame, that is, the input image body, the second image prediction process can also be implemented.

In addition, the predicted image generated by the second image prediction processing can still be used as the broadcast image. At this time, the coding data of the predicted residual image formed by the difference between the predicted image generated by the second image prediction process and the input image does not need to be included in the bit stream, so a bit stream with higher coding efficiency can be generated.

In addition, regarding the characteristics of the animation, the more moving parts and the less moving background parts are known. By referring to the table of preset coding modes of each micro-block, etc., the process of determining the coding mode can also be saved.

As described above, as shown in the description of the best embodiment of the present invention, if the present invention is used, an image encoding device, an image encoding method, an image encoding program that can effectively encode an image can be provided, and the image encoding device that can be generated from the image encoding device of the present invention can be provided. Image decoding device, image decoding method, and image decoding program for bitstream recovery images.

<p>1Image coding device</p><p>11Portrait Division</p><p>12Encoding Mode Decision Section</p><p>13Predictive image production department</p><p>14Subtraction Department</p><p>15Conversion Department</p><p>16Coding Department</p><p>17Reverse Conversion Department</p><p>18Addition Department</p><p>19Memory Department</p><p>TTemplate</p><p>PProcessing target pixel</p><p>QPlay pixel</p><p>SRelated Fields</p><p>YProcessing target block</p><p>RTarget area</p><p>S1Split input portrait</p><p>S2Decide the coding mode</p><p>S3Predictive coding processing mode</p><p>S4Generate predictive portraits</p><p>S5Generate predictive residual images</p><p>S6Calculation conversion factor</p><p>S7Average message volume coding</p><p>S8Generate and play differential images</p><p>S9Generate and play images</p><p>S10Save and play portrait</p><p>S11All blocks completed</p><p>101Program field</p><p>102aPortrait segmentation module</p><p>102bEncoding mode decision module</p><p>102cPredictive image generation module</p><p>102dSubtraction Module</p><p>102eConversion Module</p><p>102fCoding Module</p><p>102gReverse conversion module</p><p>102hAddition Module</p><p>102iMemory Module</p><p>2Image decoding device</p><p>21Decoding Department</p><p>22Encoding Mode Judgment Unit</p><p>23Predictive image production department</p><p>24Reverse Conversion Department</p><p>25Addition Department</p><p>26Memory Department</p><p>27Toggle switch</p><p>28Fill the image production department</p><p>S20Average message volume decoding</p><p>S30Predictive image coding processing</p><p>S40Filling image coding processing</p><p>S31Predictive coding processing mode</p><p>S32Generate predictive portraits</p><p>S33Generate predictive residual images</p><p>S34Generate and play images</p><p>S35Save and play portrait</p><p>S36Are all blocks over?</p><p>S41Padded coding processing mode?</p><p>S42Generate filling portrait</p><p>S43Save and fill the image</p><p>S44Are all blocks over?</p><p>201Programming area</p><p>202Image Decoding Program</p><p>202aDecoding Module</p><p>202bEncoding Mode Judgment Module</p><p>202cPredictive image generation module</p><p>202dReverse Conversion Module</p><p>202eAddition Module</p><p>202fMemory Module</p><p>202gToggle switch module</p><p>202hPadded image generation module</p><p>110Computer</p><p>100Recording Media</p><p>112Reading device</p><p>114Work memory</p><p>116Memory</p><p>126CPU</p><p>118Display</p><p>120Mouse</p><p>122Keyboard</p><p>124Communication device</p><p>130Computer data signal</p><p>310Animation image coding device</p><p>312Field Division</p><p>314Encoding Mode Decision Section</p><p>316Portrait Prediction Department</p><p>318Subtraction Department</p><p>320Conversion Department</p><p>322Coding Department</p><p>324Reverse Conversion Department</p><p>326Addition Department</p><p>328Portrait Memory Department</p><p>330Mode switch</p><p>332The first portrait prediction department</p><p>334Second Portrait Prediction Department</p><p>F1Play frame</p><p>F2Reference frame</p><p>S(Sc)Replication reference area</p><p>S301Split the field</p><p>S302Decide the coding mode</p><p>S303Encoding mode information</p><p>S304The first image prediction processing?</p><p>S305The first image prediction processing</p><p>S306Generate predictive residual images</p><p>S307Conversion</p><p>S308Average message volume coding</p><p>S309Reverse conversion</p><p>S310Addition</p><p>S311Memory Play Portrait</p><p>S312Are all blocks over?</p><p>S313The second image prediction processing?</p><p>S314Second image prediction processing</p><p>S315Generate predictive residual images</p><p>S316Conversion</p><p>S317Average message volume coding</p><p>S318Reverse conversion</p><p>S319Addition</p><p>S320Memory playback portrait</p><p>S321Are all blocks over?</p><p>STARTEncoding mode decision processing</p><p>S302-1Perform the first image prediction processing in the scanning order</p><p>S302-2Select the last block</p><p>S302-3Second image prediction processing</p><p>S302-4Cost calculation</p><p>S302-5Select coding mode</p><p>S302-6Are all blocks over?</p><p>S302-7Select the next block in reverse scan order</p><p>STARTSecond image prediction processing</p><p>S314-1Select a block that does not generate a play image</p><p>S314-2Set reference area</p><p>S314-3Select unprocessed pixels in the block</p><p>S314-4Setting template</p><p>S314-5Calculation related</p><p>S314-6Select the area of replication</p><p>S314-7The corresponding pixel in the unprocessed pixel copy area of the template</p><p>Are there any unprocessed pixels in the S314-8 block?</p><p>S314-9Select unprocessed pixels</p><p>340Animation image coding program</p><p>341Main Module</p><p>342Domain Segmentation Module</p><p>344Encoding Mode Decision Module</p><p>346Portrait prediction module</p><p>360Mode switch module</p><p>362The first portrait prediction module</p><p>364The second portrait prediction module</p><p>348Subtraction Module</p><p>350Conversion Module</p><p>352Coding Module</p><p>354Reverse Conversion Module</p><p>356Addition Module</p><p>358Image memory module</p><p>370Animation image decoding device</p><p>372Decoding Department</p><p>374Reverse Conversion Department</p><p>376Portrait Prediction Department</p><p>378Addition Department</p><p>380Portrait Memory Department</p><p>382Mode switch</p><p>S331Decoding and encoding mode</p><p>S332The first image prediction processing?</p><p>S333Average information decoding</p><p>S334The first image prediction processing</p><p>S335Reverse conversion</p><p>S336Addition</p><p>S337Save and play portrait</p><p>S338Are all blocks over?</p><p>S339The second image prediction processing?</p><p>S340Average message volume decoding</p><p>S341Second image prediction processing</p><p>S342Reverse conversion</p><p>S343Addition</p><p>S344Memory Play Frame</p><p>S345Are all blocks over?</p><p>384The first portrait prediction department</p><p>386Second Portrait Prediction Department</p><p>390Animation image decoding program</p><p>391Main Module</p><p>392Decoding Module</p><p>394Reverse Conversion Module</p><p>396Portrait prediction module</p><p>398Addition Module</p><p>400Image memory module</p><p>402Mode switch module</p><p>404The first portrait prediction module</p><p>406The second portrait prediction module</p>

Fig. 1 is a diagram showing the structure of an image encoding device related to the first embodiment.

Fig. 2 is a diagram for explaining the content of image prediction processing.

Fig. 3 is a diagram for explaining the content of image prediction processing.

Fig. 4 is a diagram showing an alternative pixel value.

Fig. 5 is a diagram illustrating the replacement pixel value.

Fig. 6 is a diagram illustrating the content of the image filling process.

Fig. 7 is a flowchart showing the operation of the image encoding device for image encoding processing.

Fig. 8 is a diagram showing the structure of a recording medium for recording an image encoding program.

Fig. 9 is a diagram showing the structure of an image decoding device related to the second embodiment.

Fig. 10 is a flowchart showing the outline operation of the image decoding process.

Fig. 11 is a flowchart showing the operation of the prediction picture decoding process.

Fig. 12 is a flowchart showing the operation of the padding picture decoding process.

Figure 13 shows the structure of the recording medium of the recorded image decoding program.

FIG. 14 is a diagram showing a computer hardware structure used to execute a program stored in a recording medium.

Fig. 15 is an oblique view of a computer used to execute a program stored in a recording medium.

Fig. 16 is a diagram showing the structure of an animation encoding device related to the third embodiment.

Fig. 17 is a diagram for explaining the processing of the coding mode determination unit.

Fig. 18 is a block diagram showing the structure of the portrait prediction unit.

Fig. 19 is a diagram for explaining the second image prediction image processing.

Fig. 20 is a flowchart showing the animation encoding method related to the third embodiment.

Fig. 21 is a flowchart showing the encoding mode determination process of the animation encoding method related to the third embodiment.

Fig. 22 is a flowchart showing the second image prediction process.

Fig. 23 is a diagram showing the structure of the animation coding program related to the third implementation type and the recording medium co-presented.

Fig. 24 is a diagram showing the structure of an animation decoding device related to the fourth embodiment.

Fig. 25 is a flowchart showing the animation decoding method related to the fourth embodiment.

FIG. 26 is a diagram showing the co-presentation of the animation decoding program and the recording medium related to the fourth embodiment.