Method and apparatus for encoding and decoding image by using large transform unit

Technical Field] [0001] Exemplary embodiments are related to an image decoding apparatus.

[Background of the Invention] [0002] In most methods and apparatus for coding and decoding an image, an image from the pixel domain is transformed into the frequency domain, and the transformed image is coded to compress the image. Discrete cosine transform (DCT) is a well-known technology used to compress audio / video data (A / V). Recently, many attempts have been made to find more efficient coding methods. In audio coding, parametric coding works better than DCT, and with two-dimensional data, the Karhunen Loeve transform (KLT) has the smallest bit size, but it has a large excess size.

[0003] WIEGAND T ET AL document, "Overview of the H.264 / AVC video coding standard" ("Overview of the H.264 / AVC video coding standard"), IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE

SERVICE CENTER, PISCATAWAY, NJ, US, vol. 13, no. 7, pages 560 - 576, describe profiles and applications for this standard, and discuss the history of the standardization process. The main goals of the H.264 / AVC standardization approach were to increase the efficiency of compression and to provide a "network-friendly" representation of video directed to "conversational" applications (video conversations), and "non-inviting" (storage, broadcast or streaming) [Disclosure of invention] [Solution to the problem] [0004] Exemplary embodiments provide a device for decoding an image using efficient conversion.

[Advantageous Effects of the Invention] [0005] According to exemplary embodiments, the image is more compressed and coded efficiently, because the transform unit may be set to have a size larger than the prediction unit, and the conversion may be performed on the transform unit.

[Brief Description of the Drawings] [0006] The above and / or other aspects will become more apparent by detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

FIG. 1 is a block diagram of an image coding apparatus;

FIG. 2 is a block diagram of an image decoding apparatus according to an exemplary embodiment;

FIG. 3 shows hierarchical coding units

FIG. 4 is a block diagram of an image encoder based on the coding unit;

FIG. 5 is a block diagram of an image decoder based on a coding unit according to an exemplary embodiment;

FIG. 6 shows the maximum coding unit, coding sub-unit and prediction unit;

FIG. 7 shows a coding unit and a transformation unit;

FIG. 8A, 8B, 8C and 8D show the division shapes of the coding unit, the prediction unit and the transformation unit;

FIG. 9 is a block diagram of an image coding apparatus;

FIG. 10 is a diagram for describing a prediction method;

FIG. 11 is a block diagram of a transforming system;

FIG. 12A to 12C are diagrams of the types of transforming units;

FIG. 13A to 13D are diagrams of the types of transforming units;

FIG. 14 is a schematic of other transformation units;

FIG. 15 is a block diagram of an image decoding apparatus according to another exemplary embodiment;

FIG. 16 is a flow diagram of an image coding method; and FIG. 17 is a flowchart of an image decoding method.

[Best Mode for Applying the Invention] [0007] According to another aspect of an exemplary embodiment, an image decoding apparatus is provided, such as set forth in the appended claim 1.

[Mode of the invention] [0008] Specific exemplary embodiments are described in more detail below, with reference to the accompanying drawings. Expressions such as "at least one of" when they precede the list of elements, change the entire list of items, but do not change individual items from this list. In the present description, "image" means a still image for video, or a moving image, i.e. the video itself.

[0009] In the following description, like reference numerals are used for similar elements, even in various figures of the drawings. The issues defined in the description, such as detailed structure and elements, are provided to facilitate in-depth understanding of exemplary embodiments. However, exemplary embodiments may be implemented without these specific issues.

[0010] FIG. 1 is a block diagram of an apparatus 100 for coding an image for encoding an image. The image coding apparatus 100 may be implemented as a hardware device, such as for example a computer processor or a computer system. The image coding apparatus 100 may also be implemented as a program module located in a computer system.

[0011] Referring to FIG. 1, the image encoding apparatus 100, comprising a separation element 110 of a maximum coding unit, a coding depth element 120, an image data encoder 130, and a coder information encoder 140 that may be implemented, e.g. as hardware or program modules located within the device. 100 for coding the image or separately from the image encoding device 100.

[0012] The distribution element 110 of the maximum coding unit may separate the current frame or slice based on the maximum coding unit, which is the largest size coding unit. That is, the distribution element 110 of the maximum coding unit may separate the current frame or section into at least one maximum coding unit.

[0013] The coding unit may be represented using a maximum coding unit and depth. As described above, the maximum coding unit indicates the coding unit having the largest size among the coding units of the current frame, and the depth indicates the degree of hierarchical reduction of the coding unit. With increasing depth, the coding unit may decrease from the maximum coding unit to the minimum coding unit, where the depth of the maximum coding unit is defined as the minimum depth, and the depth of the minimum coding unit is defined as the maximum depth. Because the size of the coding unit decreases from the maximum coding unit with increasing depth,

[0014] In accordance with the increase in the size of the frame to be encoded, encoding an image in a larger coding unit may give a higher image compression rate. However, if the larger coding unit is constant, the image can not be efficiently encoded by reflecting the ever-changing image characteristics.

[0015] For example, if a smooth area such as sea or sky is coded, then the larger the coding unit, the more the coding rate may increase. However, if a complex area such as people or buildings is coded, the smaller the encoding unit, the more the compression ratio can increase.

[0016] Accordingly, different maximum image coding units and different maximum depths are set for each frame or slice. Since the maximum depth means the maximum number of times that the coding unit can be reduced, the size of each maximum coding unit included in the maximum coding unit of the image can be variably set according to the maximum depth. The maximum depth can be set differently for each frame or slice, or each maximum coding unit.

[0017] The coding depth determining element 120 determines the shape of the division of the maximum coding unit. The division shape can be determined based on the calculation of rate-distortion (RD). The determined shape of the division of the maximum coding unit is provided to the encoder 140 of the coding information, and the image data according to the maximum coding units are provided to the encoder 130 of the image data.

[0018] The maximum coding unit may be divided into sub-coding units having different sizes according to different depths, and coding sub-units having different sizes that are included in the maximum coding unit may be subject to frequency prediction or transformation based on units. processing having different sizes. In other words, the image encoding apparatus 100 may perform a plurality of processing operations to encode the image based on processing units having different sizes and different shapes. To encode image data, processing operations such as prediction, transformation, and entropy coding are performed in which processing units having the same size or different sizes can be used for each operation.

[0019] For example, the image encoding apparatus 100 may select a processing unit that is different from the coding unit to predict the coding unit.

[0020] If the size of the coding unit is 2N x 2N (where N is a positive integer), then the processing units may have 2N x 2N, 2N x N, N x 2N, and N x N. In other words, the motion prediction may be performed on a processing unit having a shape in which at least one of the height and width of the coding unit is evenly divided by two. Henceforth, the processing unit that forms the basis of the prediction is referred to as the prediction unit.

[0021] The prediction mode may be at least one of the intra-frame mode, the interframe mode, and the skip mode, and the specific prediction mode may be implemented only on the prediction unit having a particular size and specific shape. For example, the intra-frame prediction mode can be performed only for prediction units having sizes 2N x 2N or N x N, and the shape of a square. In addition, the skip mode can only be performed on a prediction unit having the size 2N x 2N. If multiple prediction units exist in the coding unit, the prediction mode with the least number of coding errors can be selected after the prediction has been performed for each prediction unit.

[0022] Alternatively, the image encoding apparatus 100 may perform frequency conversion on image data based on processing units having a size other than the size of the coding unit. For frequency conversion in a coding unit, the frequency conversion may be performed based on a processing unit having a size equal to or less than that of the coding unit. Henceforth, the processing unit that forms the basis of the frequency conversion is referred to as the transforming unit. The frequency conversion may be a discrete cosine transform (DCT) or a Karhunen Loeve transform (KLT).

[0023] The coding depth determining element 120 may determine the coding sub-unit included in the maximum coding unit using the RD optimization based on the Lagrange multiplier. In other words, the coding depth determining element 120 can determine the shape of the numerous coding sub-units extracted from the maximum coding unit, where the coding sub-units have different sizes according to the depth of the coding sub-units. The image data processor 130 derives the bit stream to the output by encoding the maximum coding unit based on the distribution shapes determined by the coding depth determining element 120.

[0024] The encoder information encoder 140 encodes information regarding the coding mode of the maximum coding unit determined by the coding depth determining element 120. In other words, the coding information encoder 140 outputs a bit stream to the output by encoding the information regarding the shape of the split of the maximum coding unit, information about the maximum depth, and information about coding mode of the coding subunit for each depth. The coding mode information of the coding sub-unit may include information about the prediction unit of the coding sub-unit, information about the prediction mode for each prediction unit, and information about the coding sub-unit of the coding unit.

[0025] The information regarding the shape of the division of the maximum coding unit may be marker information, indicating whether each coding unit is split. For example, if the maximum coding unit is split and coded, information is coded indicating whether the maximum coding unit is split. Thus, if the coding sub-unit separated from the maximum coding unit is split and coded, information is coded indicating whether the coding sub-unit is split and coded.

[0026] Since there are coding sub-units having different sizes for the maximum coding unit and the coding mode information is determined for each coding subunit, information about at least one coding mode may be set for one maximum coding unit.

[0027] The image encoding apparatus 100 may generate the coding sub-unit by equally dividing the height and width of the maximum coding unit by two according to the increase in depth. That is, when the size of the coding unit at which depth is 2N x 2N, the size of the coding unit o (k + 1) -th depth is N x N.

[0028] Accordingly, the image encoding apparatus 100 may determine the optimal division shape for each maximum coding unit based on the sizes of the maximum coding units and the maximum depth taking into account the image characteristics. By varying the size of the maximum coding unit by varying the image characteristics and coding of the image by dividing the maximum coding unit into sub-coding units with different depths, images having different resolutions may be more efficiently coded.

[0029] FIG. 2 shows an image decoding device 200 for decoding an image according to an exemplary embodiment. The image decoding apparatus 200 may be implemented as a hardware device, such as, for example, a computer processor, or a computer system. The image decoding device 200 may also be implemented as a program module located in a computer system.

[0030] Referring to FIG. 2, the image decoding apparatus 200 includes an image data acquiring unit 210, an encoding information element 220, and an image data decoder 230, which may be implemented, for example, as hardware or software modules embedded in the image decoding device 200, or separately with respect to the image encoding device 200.

[0031] The image data acquiring unit 210 acquires image data in accordance with the coding units by analyzing the bit stream received by the image decoding apparatus 200 and outputs the image data to the image data decoder 230. The image data acquiring unit 210 may extract information about the maximum coding unit of the current frame or slice from the header of the current frame or slice. In other words, the image data acquisition unit 210 divides the bit stream into a maximum coding unit such that the image data decoder 230 may decode the image data according to the maximum coding units.

[0032] Extracting coding information element 220 extracts information about the maximum coding unit, maximum depth, shape of the maximum division of the coding unit, coding mode of the coding sub-units from the header of the current frame by analyzing the bit stream received by the image decoding device 200. Information about the shape of the division and information about the coding mode are provided to the decoder 230 of the image data.

[0033] The information about the shape of the division of the maximum coding unit may include information about the coding sub-entities having different dimensions to depth and contained in the maximum coding unit, and may be flag information indicating whether each coding unit is split.

[0034] The coding mode information may include information about the prediction unit according to the coding sub-units, the information about the prediction mode, and the information about the transforming unit.

[0035] The image data decoder 230 restores the current frame by decoding the image data of each maximum coding unit based on the information retrieved by the coding information extraction element 220.

[0036] The image data decoder 230 may decode the coding sub-units included in the maximum coding unit based on the information about the shape of the maximum division of the coding unit. The decoding process may include prediction processing including intra-frame prediction and motion compensation, and an inverse conversion process.

[0037] The image data decoder 230 may perform intra-frame prediction or inter-frame prediction based on the prediction unit information and information of the prediction mode to predict the prediction unit. The image data decoder 230 may also perform an inverse translation for each sub-unit coding based on the information about the coding sub unit of the coding unit. [0038] FIG. 3 shows hierarchical coding units.

[0039] Referring to FIG. 3, these hierarchical coding units may include coding units whose widths and heights are 64^χ 64, 32 ^χ 32, 16 ^χ 16, 8 ^χ 8, and 4 ^χ 4. In addition to these coding units having exactly square shapes, there may also be coding units whose widths and heights are 64 χ

32, 32 χ 64, 32 χ 16, 16 χ 32, 16 χ 8, 8 χ 16, 8 χ 4, and 4 χ 8.

[0040] Referring to FIG. 3, for a set of 310 image data whose resolution is 1920 × 1080, the size of the maximum coding unit is set to 64 χ 64, and the maximum depth is set to 2.

For image data set 320, the resolution of which is 1920 × 1080, the size of the maximum coding unit is set to 64 × 64 and the maximum depth is set to 3. For the image data set 330, the resolution of which is

352 * 288, the size of the maximum coding unit is set to 16 * 16, and the maximum depth is set to 1.

[0042] If the resolution is high or the amount of data is large, the maximum size of the coding unit may be relatively large not only to increase the compression ratio and to more accurately render the image characteristics. Accordingly, as the maximum size of the coding unit of sets 310 and 320 of the video image data having a higher resolution than the image data set 330, 64 * 64 may be selected.

[0043] The maximum depth indicates the total number of layers in the hierarchical coding units. Since the maximum depth of the image data set 310 is 2, the image data set 310 may 315 comprise a maximum coding unit whose longer axis is 64, and sub-coding units whose longer axis dimensions are 32 and 16, respectively, to increase in depth.

[0044] On the other hand, since the maximum depth of the image data set 330 is 1, the image data unit 330 coding unit 335 may comprise a maximum coding unit whose longer axis is 16, and sub-coding units whose size of the long axis is 8, according to the depth increase.

[0045] However, since the maximum depth of the image data set 320 is 3, the image data set 320 may be a maximum encoding unit 320 whose long axis is 64, and sub-coding units whose size of the long axis is 32, 16, 8 and 4, according to the depth increase. Because the image is encoded based on a smaller sub-unit of coding with increasing depth, these examples are useful for coding an image comprising multi-minute scenes.

[0046] FIG. 4 is a block diagram of an image encoder 400 based on coding units. The image coder 400 may be implemented as a hardware device, such as, for example, a computer processor, or a program module housed in a computer system.

[0047] The intra-frame prediction system 410 performs intra-frame prediction on the intra-frame prediction unit in the current frame 405, and the motion estimation system 420 and the motion compensator 425 perform inter-frame prediction and motion compensation on the interframe-mode prediction units. using the current frame 405 and the reference frame 495. The intra-frame prediction system 410, motion estimation system 420, motion compensator 425 and reference frame 495 may be implemented, for example, in hardware or as program modules integrated with the image encoder 400, or independently of the video encoder 400.

[0048] The residual values ​​are generated based on the prediction units derived from the output from the intra-frame prediction system 410, the traffic estimation system 420, and the motion compensator 425. These generated residual values ​​are outputted as quantized transform coefficients by passing them through the transforming element 430 and the quantization element 440.

These quantized transform coefficients are restored to residual values ​​by passing them through the reverse quantizing element 460 and reverse transform system 470, and these restored residual values ​​are subjected to post-processing by passing them through the de-blocking unit 480 and the loop filtering unit 490 and outputted as a reference frame 495. These quantized transform coefficients can be outputted as bit stream 455 by passing through the entropy encoder 450.

[0050] To carry out coding based on the coding method, intra-frame prediction system 410, motion estimation system 420, motion compensating circuit 425, transformation element 430, quantization element 440, entropy coder 450, reverse quantization element 460, inverse element 470 the transformer, the block de-blocking unit 480 and the loop filter coder unit 490 performs the image coding process based on the maximum coding unit, the sub-coding unit corresponding to the depth, the prediction unit and the transforming unit.

[0051] FIG. 5 is a block diagram of a picture decoder 500 based on a coding unit according to an exemplary embodiment. The image decoder 500 may be implemented as a hardware device, such as, for example, a computer processor or as a software module located in a computer system.

The bit stream 505 goes through the parser 510 so that the coded image data to be decoded and the coding information necessary for decoding are extracted. The encoded image data is outputted as inverse-quantized data by passing through the entropy decoder 520 and the inverse quantizing element 530 and restored to the residual values ​​by passing through the inverse transforming element 540. These residual values ​​are restored to the coding units by adding them to the intra-frame prediction result from the intra-frame prediction system 550 or the motion compensation result from the motion compensator 560. These restored 585 units, The coding 595 is used to predict the next coding units or the next frame by going through the block removal unit 570 and filtering unit 580 in the loop. Parser 510, entropy decoder 520, reverse quantizing element 530, reverse transform element 540, intrafra frame prediction system 550, compensator 560, block removal unit 570, and loop filtering unit 580 can be implemented, for example, in hardware or as program modules located inside the image decoder 500 or independently of the image decoder 500.

[0053] To perform decoding based on a decoding method according to an exemplary embodiment, a parser 510, entropy decoder 520, inverse quantizing element 530, inverse transforming element 540, intra-frame prediction system 550, motion compensator 560, block removal unit 570, and the filter unit 580 in the image decoder 500 loop performs image decoding processes based on a maximum coding unit, a coding sub-unit corresponding to the depths, a prediction unit, and a transforming unit. [0054] In particular, the intra-frame prediction system 550 and the motion compensator 560 determine the prediction unit and the prediction mode in the coding sub-unit by including the maximum coding unit and depth,

[0055] FIG. 6 shows the maximum coding unit, coding sub-unit and prediction unit.

[0056] The image encoding apparatus 100 shown in FIG. 1 and the image decoding device 200 shown in FIG. 2 use hierarchical coding units to perform coding and decoding taking into account the image characteristics. The maximum coding unit and maximum depth can be adaptively adjusted to the image characteristics or set differently according to the user's requirements.

[0057] In FIG. 6, the hierarchical structure 600 of the coding unit has a maximum coding unit 610, which is a maximum coding unit, whose height and width are 64, and the maximum depth is 4. The depth grows along the vertical axis of the hierarchical structure of the coding unit 600, and with increasing depth, height and the widths of the coding sub-units from 620 to 650 are decreasing. The prediction units of the maximum coding unit 610 and the coding sub-units 620 to 650 are shown along the horizontal axis of the hierarchical structure 600 of the coding unit.

[0058] The maximum coding unit 610 has a depth of 0 and the size of the coding unit, or height and width of 64 χ 64. The depth increases along the vertical axis, and there exists a first coding sub-unit 620, whose size is 32 χ 32 and the depth is 1 , a second coding sub-unit 630, the size of which is 16 χ 16 and the depth is 2, the third encoder sub-unit 640, whose size is 8 χ 8 and the depth is 3, and the minimum coding unit 650, which size is 4 χ 4 a the depth is 4. The minimum coding unit 650, whose size is 4 χ 4 and the depth is 4 is the minimum coding unit, and this minimum coding unit can be divided into prediction units, each of which has a size smaller than the minimum coding unit.

[0059] Referring to FIG. 6, examples of prediction units are shown along a horizontal axis corresponding to each depth. That is, the prediction unit of the maximum coding unit 610, whose depth is 0, may be a prediction unit whose size is equal to the size 64 χ 64 of the maximum coding unit, or a prediction unit 612, whose size is 64 χ 32, or a prediction unit 614 of which size is 32 χ 64, or a prediction unit 616, whose size is 32 χ 32, which is smaller in size than the one for the maximum coding unit, whose size is 64 χ 64.

[0060] The prediction unit of the first coding sub-unit 620, the depth of which is 1, and the size 32 χ 32 may be a prediction unit whose size is equal to 32 χ 32 of the first coding sub-unit, or a prediction unit 622 whose size is 32 χ 16, a prediction unit 624, whose size is 16 χ 32, or a prediction unit 626, whose size is 16 χ 16, which is smaller than that for the first sub-coding unit 620, whose size is 32 χ 32.

[0061] The prediction unit of the second coding subunit 630, whose depth is 2, and the size is 16 χ 16, may be a prediction unit whose size is 16 χ 16 of the second coding sub-unit 630, or a prediction unit 632 whose size is is 16 χ 8, a prediction unit 634, whose size is 8 χ 16, or a prediction unit 636, whose size is 8 χ 8, which is smaller than that for the second coding sub-unit 630, whose size is 16 χ 16.

[0062] The prediction unit of the third coding sub-unit 640, whose depth is 3, and the size is 8 χ 8 may be a prediction unit whose size is equal to size 8 χ 8 of the third coding sub-unit 640, or a prediction unit 642, whose size is 8 χ 4, a 644 prediction unit whose size is 4 χ 8, or a 646 prediction unit whose size is 4 χ 4, which is smaller than that for the third coding sub-unit 640, whose size is 8 χ 8.

[0063] The minimum coding unit 650, whose depth is 4, and the size is 4 χ 4 is the minimum coding unit with maximum depth. The prediction unit of the minimum coding unit 650 may be a prediction unit 650 whose size adds 4 χ 4, a prediction unit 652 having a size 4 χ 2, a prediction unit 654 having a size 2 χ 4, or a prediction unit 656 having a size 2 χ 2.

[0064] FIG. 7 shows the coding unit and the transformation unit.

[0065] The image encoding apparatus 100 shown in FIG. 1 and the image decoding device 200 shown in FIG. 2, perform coding and decoding of the image at a maximum coding unit or with coding sub units that have a size equal to or less than the maximum coding unit extracted from the maximum coding unit. In the coding and decoding process, the size of the transformation unit for the frequency transformation is selected larger than the corresponding coding unit. For example, if the current encoding unit 710 has a size of 64x64, the frequency conversion may be performed using a transformation unit 720 having a size of

32x32.

[0066] FIG. 8A, 8B, 8C and 8D show the division shapes of the coding unit, the prediction unit and the transformation unit.

[0067] FIG. 8A and 8B, respectively, represent the coding unit and the prediction unit.

[0068] FIG. 8A shows a split shape selected by the image encoding apparatus 100 shown in FIG. 1, for coding a maximum coding unit 810. The image encoding apparatus 100 separates the maximum coding unit 810 into different shapes, performs coding, and selects the optimal partition shape by comparing the coding results of the different partition shapes to each other based on the RD rating. If it is optimal that the maximum coding unit 810 is to be coded, the maximum coding unit 810 may be encoded without dividing the maximum coding unit 810, as shown in FIG. from 8A to 8D.

[0069] Referring to FIG. 8A, the maximum coding unit 810, the depth of which is 0, is coded by dividing the maximum coding unit 810 into the coding subunits 812, 854 whose depths are equal to or greater than 1. This means that the maximum coding unit 810 is separated into 4 sub-coding units whose depth is 1, and all or some of these coding sub-units, whose depth is 1, are separated into sub-units 814, 816, 818, 828, 850, and 852 of coding, the depth of which is 2.

[0070] The coding sub-unit located in the upper-right portion and the coding sub-unit located in the bottom-left of these coding sub-units, whose depth is 1, are separated into sub-coding units whose depths are equal to or greater than than 2. Some of the coding sub-units whose depths are equal to or greater than 2 can be further separated into sub-units 820, 822, 824, 826, 830, 832, 840, 842, 844, 846, and 848 coding whose depth is equal to or greater than 3. [0071] FIG. 8B shows the shape of the division of the prediction unit for the maximum coding unit 810.

[0072] Referring to FIG. 8B, the prediction unit 860 for the maximum coding unit 810 may be separated differently from the maximum coding unit 810. In other words, the prediction unit for each of the coding sub-units may be smaller than the corresponding coding sub-unit.

[0073] For example, the prediction unit for the coding sub-unit 854 located in the bottom-right part of these coding sub-elements 812, 854, whose depth is 1 may be smaller than the coding sub-unit 854. In addition, the prediction units for the sub-units 814, 816, 850, and 852 of the coding of the sub-units 814, 816, 818, 828, 850, and 852 of the coding, whose depths are 2 can be, respectively, smaller than those of the sub-units 814 , 816, 850, and 852 coding.

[0074] In addition, the prediction units for the coding sub-units 822, 832, and 848, whose depths are 3 may be smaller than those for the coding sub-entities 822, 832 and 848, respectively. These prediction units may have a shape whereby the respective units are equally divided by two in height or width directions, or have a shape, as a result of which the respective coding sub-units are equally divided by four in the height and width directions.

[0075] FIG. 8C and 8D show the prediction unit and the transformation unit.

[0076] FIG. 8C shows the shape of the division of the prediction unit for the maximum coding unit 810 shown in FIG. 8B, and FIG. 8D shows the shape of the division of the transformation unit of the maximum coding unit 810.

[0077] Referring to FIG. 8D, the division shape of the transformation unit 870 may be set differently than the prediction unit 860.

[0078] For example, even if the prediction unit for the coding sub-unit 854, whose depth is 1, has a selected shape as a result of which the height of the sub-unit

854 of the coding is equally divided by two, the transforming unit may be selected with the original size of the coding sub-unit 854. Similarly, even if the prediction units for the sub-units 814 and 850 of the coding, whose depth is 2, are selected in a shape, as a result of which the height of each of these sub-units 814 and 850 of coding is equally divided by two, then the transformation unit may be selected with the same size as the original size of each of these 814 and 850 coding sub-units.

[0079] The transforming unit may be selected with a smaller size than the prediction unit. For example, if the prediction unit for the coding sub-unit 852, the depth of which is 2, is selected in a shape, as a result of which the width of the sub-unit coding 852 is equally divided by two, the transforming unit may be selected in a shape as a result of the coding sub unit 852 is equally divided by four in the height and width directions, which is smaller in size than the shape of the prediction unit.

[0080] Alternatively, as will be described with reference to FIG. 13A to 13D, the transforming unit may be set to have a size larger than the coding unit, regardless of the coding unit.

[0081] FIG. 9 is a block diagram of a video encoding device 900.

[0082] Referring to FIG. 9, the image encoding apparatus 900 includes a prediction unit 910, a transformation element 920, a quantizer 930, and an entropy encoder 940.

[0083] The prediction unit 910 generates residual values ​​by performing intra-frame prediction or inter-frame prediction on one or more coding units. As will be described later, the residual values ​​contained in the multiple prediction units can be grouped into one unit of transformation and then converted to the frequency domain, and thus residual values ​​are generated by predicting that one or more coding units based on these numerous ones. prediction units. The transformation to the frequency domain can be DCT or KLT.

[0084] As described above with reference to FIG. 8A, in the picture decoding method, one coding unit may comprise a plurality of prediction units. Therefore, the prediction unit 910 can predict each of these prediction units, and generate residual values ​​of these prediction units included in the one coding unit.

[0085] Alternatively, the prediction unit 910 may perform the prediction of a plurality of coding units simultaneously. As will be described later, the plurality of prediction units included in a plurality of coding units can be grouped into one transforming unit, and thus residual values ​​are generated by predicting each of the prediction units included in these coding units. For example, all of the coding sub-units included in one maximum coding unit may be predicted to generate residual values ​​of these coding units.

[0086] According to traditional technology, since the transformation (e.g., DCT, or KLT) is carried out at a size smaller than or equal to the prediction unit, the predetermined prediction unit is independently coded, restored and then used to predict the next prediction. prediction units. However, according to the picture coding method which will be described hereinafter, because the transformation is carried out by grouping the prediction units included in one or more coding units into one transformation unit, the predetermined prediction unit can not be independently coded and restored. This will be described in detail with reference to FIG. 10.

[0087] FIG. 10 is a diagram for describing a prediction method.

[0088] Referring to FIG. 10, one coding unit 1000 may comprise a plurality of coding units from 1010 to 1040. If the transformation is performed at a size smaller than or equal to the prediction unit, as in traditional technology, these prediction units 1010 to 1030 may be coded and restored before the entity is coded. 1040 predictions in the bottom-right part.

Accordingly, if the prediction unit 1040 is to be predicted by intra-frame prediction according to traditional technology, the prediction unit 1040 is subjected to intra-frame prediction using pixels adjacent to the prediction unit 1040, among the pixels generated by the coding and then restoring these prediction units from 1010 to 1030.

[0090] On the other hand, here, a number of the prediction units are grouped into one transformation unit, and then the transformation is performed. Here, if these prediction units 1010 to 1040 in FIG. 10 are grouped into one transformation unit, then the prediction unit in the bottom-right portion is encoded with other prediction units from 1010 to 1030, and thus these prediction units 1010 to 1030 are not coded before the prediction unit 1040 is coded. Accordingly, the prediction unit 1040 can not be subjected to intra-frame prediction by using the pixels generated by the coding and then restoring these prediction units from 1010 to 1030.

[0091] Consequently, the prediction unit 910 of FIG. 9 may predict the prediction unit 1040 by using prediction values ​​of these prediction units 1010 to 1030. The lower right-hand prediction unit 1040 is predicted using those prediction values ​​of these prediction units from 1010 to 1030 instead of the pixels generated by the coding, and then restoring these prediction units from 1010 to 1030.

In other words, if there is a first prediction unit predicted by intra-frame prediction, among the prediction units grouped in one transformation unit, the first prediction unit may be subjected to intra-frame prediction using the prediction value of the at least one adjacent prediction unit. .

[0093] Alternatively, these prediction units grouped in one transforming unit may all be predicted by inter-frame prediction. As described with reference to FIG. 10, because the prediction unit that is predicted by inter-frame prediction is problematic when grouping multiple prediction units into one transformation unit, all the prediction units grouped in the transform unit can be predicted using only inter-frame prediction.

[0094] Referring again to FIG. 9, the transformation member 920 receives the image processing unit in the pixel domain, and converts the image processing unit into the frequency domain. The transformation element 920 converts the residual values ​​generated by the prediction unit 910 to the frequency domain.

[0095] As described above, the transforming element 920 groups these prediction units to one transformation unit and performs DCT or KLT according to that transformation unit. These residual values ​​may be residual values ​​of the multiple prediction units included in one or more coding units. Frequency component coefficients are generated as a result of the conversion of the pixel domain into a frequency domain.

[0096] The frequency domain conversion can be performed via DCT or KLT, and discrete cosine coefficients are generated as a result of DCT or KLT. However, any conversion for image conversion in the pixel domain to the frequency domain can be used.

[0097] FIG. 11 is a block diagram of a transformation system 920.

[0098] Referring to FIG. 11, the transformation system 920 includes a dial element 1110 and a processing circuit arrangement 1120.

[0099] Selection element 1110 sets one transformation unit by selecting a plurality of adjacent prediction units. According to the traditional image coding devices described above, intra-frame prediction or inter-frame prediction is performed according to a predetermined prediction unit, and DCT or KLT is performed with a size smaller than or equal to a predetermined prediction unit. In other words, conventional image coding devices perform DCT or KLT based on a transformation unit having a size smaller than or equal to the prediction unit.

[0100] However, the coding rate of the image coding is degraded because the added redundancy increases the size of the transform unit is decreased due to the header information added to each transformation unit. Accordingly, the image encoding apparatus 900 groups these adjacent prediction units to one transformation unit, and then performs DCT or KLT according to the transformation unit. In particular, since these adjacent prediction units are likely to have similar residual values, the compression ratio for coding can be significantly increased when DCT or KLT is performed according to the transformation unit generated by the grouping of these adjacent prediction units.

Accordingly, the dialing element 1110 selects these prediction units to be grouped into one transformation unit on which the DCT or KLT is to be performed. These prediction units may be adjacent to each other. This will be described in detail with reference to FIG. 12A to 12C, and from 13A to 13D. [0102] FIG. 12A to 12C are diagrams of unit types from 1230 to 1250 transformation.

[0103] Referring to FIG. 12A to 12C, the prediction unit 1220 may have a shape whereby the coding unit 1210 is equally divided by two in the width direction. The coding unit 1210 may be a maximum coding unit as described above, or a coding unit having a size smaller than the maximum coding unit.

[0104] Even when the coding unit 1210 and the prediction unit 1220 are identical, the transformation units 1230 to 1250 may be different. The size of the transformation unit 1230 may be smaller than that for the prediction unit 1220, as shown in FIG. 12A, or the size of the transformation unit 1240 may be the same as that for the prediction unit 1220, as shown in FIG. 12B. Alternatively, the size of the transformation unit 1250 may be larger than that for the prediction unit 1220, as shown in FIG. 12C.

[0105] These prediction units grouped into one transformation unit may be a plurality of prediction units included in different coding units as shown in FIG. 12A to 12C, or may be numerous prediction units included in different coding units. In other words, the plurality of prediction units included in the at least one coding unit may be grouped into one transforming unit and then converted into the frequency domain. [0106] FIG. 13A to 13D are diagrams of the types of transforming units. [0107] One maximum coding unit 1300 may be divided into coding sub units from 1302 to 1308 with different sizes and then encoded as shown in FIG. 13A, and each of these coding sub-units 1302 to 1308 may include at least one prediction unit from 1310 to 1340, as shown in FIG. 13B.

[0108] Dial element 1110 can group the prediction units 1310 to 1340 shown in FIG. 13B to one transformation unit 1350 provided in FIG. 13C, and then convert this transformation unit 1350 to the frequency domain.

[0109] Alternatively, dial element 1110 may group these units 1310 and 1330 to 1339 predict these sub coding units 1302 to 1306 left to one transformation unit 1360, and group these prediction units from 1320 to 1328 and 1340 of these. the coding sub units 1304 and 1308 located to the right to one transformation unit 1362, as shown in FIG. 13D.

[0110] Referring again to FIG. 11, for the selection element 1110 selecting a plurality of adjacent prediction units is not limited. As described above, in one example, the selecting element 1110 may select a depth conversion unit. As described above, the depth indicates the degree of hierarchical reduction of the coding unit from the maximum coding unit of the current sector or frame to the coding sub-units. As described above with reference to FIG. 3 and 6, as the depth increases, the size of the coding sub-unit decreases, and thus the size of the prediction unit contained in the coding sub-unit decreases. Here, if DCT or KLT is carried out according to a transform unit having a size smaller than or equal to the prediction unit, this coding ratio of the image coding is reduced because the header information is added to each transformation unit as described above. Accordingly, the prediction units included in the coding sub-unit, the depth of which is equal to or above the predetermined value can be grouped into one transformation unit, and then DCT or KLT can be performed on this transforming unit. For this reason, the dial element 1110 may position the transform unit based on the depth of the coding sub-unit. For example, if the depth of the coding unit 1210 of FIG. 12C is higher than k, then dial element 1110 groups these prediction units 1220 into one transformation unit 1250. [0112] Alternatively, if the maximum coding unit has a number of coding subunits, whose depths are equal to or above a predetermined value, then the selecting element 1110 may group the prediction units of these coding sub-units into one transformation unit. FIG. 13C illustrates an example of grouping prediction units belonging to coding sub-units whose depth is greater than the maximum coding unit, i.e. the depth is greater than 1, to one transformation unit.

[0113] According to another example, the selection element 1110 may set a number of adjacent coding units on which the prediction according to the same type of prediction is performed to one transformation unit. These adjacent prediction units that are predicted using intra-frame prediction or inter-frame prediction are grouped into one transformation unit.

Because it is highly likely that these adjacent prediction units that are predicted according to the same type of prediction prediction have similar residual values, then DCT or KLT can be performed by grouping these adjacent prediction units to one transformation unit. [0114] When the selection element 1110 sets up the transforming unit, the processing execution circuit 1120 converts these adjacent prediction units to the frequency domain according to the set transformation unit. Frequency domain factors (e.g., discrete cosine coefficients) are generated by converting these selected prediction units to one transformation unit.

[0115] Referring again to FIG. 9, the quantizing element 930 quantizes the frequency component coefficients generated by the transformation element 920. The quantizing element 930 can quantize these input coefficients according to a predetermined quantization process.

[0116] The entropy encoder 940 encodes entropically those coefficients quantized by the quantizing element 930. Here, these discrete cosine coefficients can be entropy encoded using context-adaptive binary arithmetic coding (CABAC) or variable-length encoding variable-length encoding (CABAC). context-adaptive variable length coding) (CAVLC).

[0117] The image encoding apparatus 900 may encode tag information indicative of whether the transforming unit generated by grouping these adjacent prediction units includes these coefficients. If there are no coefficients to be entropy encoded, i.e. if these quantized coefficients are all "0" then tag information is coded indicating that the transform units do not contain these coefficients, and these quantized coefficients are not entropy coded separately.

[0118] The picture encoding device 900 of the present example may determine the optimal transforming unit by repeatedly transforming, quantizing, and entropy coding on different transformation units. The optimal transformation unit can be determined by mechanical repetition of the selection process of multiple prediction units using different methods, instead of selecting these prediction units based on a predetermined criterion, such as depth, or the type of prediction mode itself. The optimal transformation unit may be determined based on the calculation of the RD rating, and will be described in detail with reference to FIG. 14.

[0119] FIG. 14 is a diagram of another example of transforming units from 1430 to 1460.

[0120] Referring to FIG. 14, the image coding unit 900 repeatedly encodes different transformation units from 1430 to 1460.

[0121] As shown in FIG. 14, the coding unit 1410 may be predicted and encoded based on the prediction unit 1420 having a size smaller than the coding unit 1410. DCT or KLT is carried out on residual values ​​generated as a prediction result, and here, DCT or KLT can be performed based on different transformation units from 1430 to 1460, as shown in FIG. 14.

[0122] The transformation unit 1430 has the same size as the coding unit 1410, and is generated by grouping all the prediction units included in the coding unit 1410.

[0123] The transformation units 1440 have a size as a result of which the coding unit 1410 is equally divided by two in the width direction, and are generated by grouping those prediction units that are adjacent in the vertical direction.

[0124] The transformation units 1450 are sized so that the coding unit 1410 is equally divided by two in the height direction, and is generated by grouping those prediction units that are adjacent in the horizontal direction.

[0125] The transformation units 1460 have the same size as the prediction units 1420.

[0126] The image encoding apparatus 900 may determine the optimal transforming unit by repeatedly transforming, quantizing, and entropy coding on those transforming units from 1430 to 1460.

[0127] Alternatively, the image encoding apparatus 900 may encode the flag information indicative of whether the transform unit is generated by grouping a plurality of prediction units included in one or more coding units. For example, if the transforming unit is set by grouping a plurality of prediction units included in one coding unit, as shown in FIG. from 12A to 12C, the flag information is set to & quot; 0 & quot ;, and if the transforming unit is set by grouping the plurality of prediction units included in the plurality of coding units, as shown in FIG. 13A to 13D, the tag information is set to "1".

[0128] FIG. 14 shows an example of determining the optimal coding unit if one transforming unit is set by grouping the prediction units included in one coding unit. However, the optimal transformation unit can be determined by repeatedly performing DCT, quantization, and entropy coding on different transformation units, as shown in FIG. 14, even if one unit of transformation is set by grouping the prediction units contained in the plurality of coding units.

[0129] FIG. 15 is a block diagram of an apparatus 1500 for decoding an image, according to another exemplary embodiment.

[0130] Referring to FIG. 15, the image decoding apparatus 1500 includes an entropy decoder 1510, an inverse quantizing element 1520, a reverse transforming element 1530, and a restoration circuit 1540.

[0131] The entropy decoder 1510 decodes entropyly the frequency component coefficients of a predetermined transformation unit. As described above with reference to FIG. 12A to 12C and 13A to 13D, the transforming unit may be generated by grouping a plurality of prediction units. As described above, these prediction units may be adjacent to each other, and may be included in one coding unit or in a plurality of different coding units.

[0132] As described above with respect to the image encoding apparatus 900, the transform unit may be generated by grouping a plurality of adjacent prediction units based on depth, or by grouping a plurality of contiguous prediction units on which the prediction is performed according to the same type of mode. prediction, i.e. according to intra-frame or interframe mode. Alternatively, as described with reference to FIG. 14, the optimal transforming unit may be selected by repeatedly transforming, quantizing, and entropy decoding on different transformation units by mechanically repeating the grouping process of the plurality of prediction units.

[0133] If the transforming unit does not include coefficients (e.g., discrete cosine coefficients), then the entropy decoder 1510 can not entropyly decode the quantized coefficients separately. If the transforming unit does not include these quantized coefficients, then these quantized coefficients are not entropyally coded separately by reference to predetermined tag information.

[0134] The inverse quantizing element 1520 quantizes those frequency component coefficients that are entropy decoded by the entropy decoder 1510. These frequency component coefficients that are entropy decoded according to the quantization step used in the coding of the transforming unit are quantized inversely.

[0135] The reverse transforming element 1530 reverses the inverse quantized frequency component coefficients to the pixel domain.

Reverse DCT or inverse KLT is carried out on the inversely quantized discrete cosine coefficients to reproduce the pixel transformation unit. As a result of the inverse conversion, the residual values ​​of the transformation unit are restored.

[0136] The reconstructed unit of transformation comprises a plurality of prediction units, and as described above, these prediction units may be included in one coding unit or in a plurality of different coding units.

[0137] The restoration circuit 1540 generates prediction values ​​by performing predictions of the plurality of prediction units included in the reconstituted unit of transformation. Prediction values ​​of one coding unit are generated if these prediction units grouped in one transformation unit are included in one coding unit, and the prediction values ​​of coding units are generated if these prediction units grouped in one transformation unit are included in a plurality of coding units. One coding unit or a plurality of coding units are reproduced by adding these generated prediction values ​​and residual values ​​restored by the reverse transformation element 1530.

[0138] Whether prediction values ​​are generated for one coding unit or for a plurality of coding units may be determined based on tag information indicating whether the image encoding device 900 generated a unit of transformation by grouping a plurality of prediction units included in one coding unit or in coding units.

[0139] According to one example, if the prediction units grouped into one transformation unit comprise a prediction unit that has been subjected to intra-frame prediction, the intra-frame prediction can be performed based on the prediction values ​​of the at least one adjacent prediction unit, as described with reference to FIG. 10. Alternatively, multiple prediction units grouped into one transforming unit can be predicted using inter-frame prediction.

[0140] FIG. 16 is a diagram illustrating a picture coding method.

[0141] Referring to FIG. 16, the image encoding apparatus generates residual values ​​by performing predictions on one or more coding units in operation 1610.

[0142] Numerous prediction units grouped into one transformation unit may be included in one coding unit or in a plurality of coding units. Accordingly, when these prediction units are included in one coding unit, these residual values ​​are generated by performing predictions on one coding unit, and when these prediction units are included in a number of coding units, these residuals are generated by prediction on numerous coding units.

[0143] The method of generating residual values ​​by subjecting prediction units of prediction simultaneously has been described above with reference to FIG. 10.

[0144] In operation 1620, the device sets one transformation unit by selecting a plurality of prediction units. These prediction units may be included in one coding unit or in a plurality of coding units. These adjacent prediction units may be selected based on depth, or adjacent contingency units may be selected on which prediction is performed in the same type of prediction mode.

[0145] In operation 1630, the device converts these prediction units to the frequency domain according to the transformation units set in operation 1620. The frequency field coefficients are generated by performing a transform on the transform unit set by grouping these prediction units.

[0146] In operation 1640, the device quantizes the frequency component coefficients, i.e. those discrete cosine coefficients generated in operation 1630 according to a predetermined quantization process.

[0147] In operation 1650, the device entropy encodes those frequency component factors quantized in operation 1640. The entropy coding is performed via CABAC or CAVLC.

[0148] As described with reference to FIG. 14, the method may further comprise setting the optimal transforming unit by repeating operations 1610 to 1640 on different transformation units. The optimal transformation unit can be set by repeatedly transforming, quantizing, and entropy coding on these different transformation units, as shown in FIG. 14. [0149] FIG. 17 is a flowchart illustrating a picture decoding method. [0150] Referring to FIG. 17, the device decodes entropyly the frequency component coefficients of a predetermined transforming unit in operation 1710. These frequency component coefficients may be discrete cosine coefficients. The transformation unit can be set by grouping multiple prediction units. As described above, these prediction units may be adjacent to each other, and may be included in one coding unit or in a plurality of different coding units.

[0151] In operation 1720, the device inverse quantizes those frequency component coefficients that have been inversely quantized in operation 1710. Discrete cosine coefficients are quantized inversely using a quantization step that is used during coding.

[0152] In operation 1730, the device reverses the frequency component factors that are inversely quantized in the pixel pixel sector 1720 operation to reproduce the transforming unit. The reconstructed unit of transformation is set by grouping numerous prediction units. The residual values ​​contained in the transformation unit are reproduced. The residual values ​​of one coding unit are reproduced if these prediction units are included in one coding unit and the residual values ​​of a number of coding units are restored if these prediction units are included in these coding units.

[0153] As described above, the transforming unit may be set by grouping adjacent prediction units based on depth, or by grouping adjacent prediction units on which the prediction is performed, according to the same prediction mode.

[0154] In operation 1740, the device restores one or more coding units based on residual values ​​included in the reconstitution unit reproduced in operation 1730. Prediction values ​​are generated by subjecting predictions to one or more coding units, and this one, or a larger number of coding units are restored by adding these generated prediction values ​​and residual values ​​restored in operation 1730. The method of predicting prediction values ​​contained in one or more coding units has been described above with reference to FIG. 10. If the transformation unit is set by grouping these prediction units contained in one coding unit, one coding unit is restored,

[0155] According to exemplary coding forms, the image is more efficiently compressed and coded, because the transform unit may be set to be larger than the prediction unit, and the conversion may be performed on the transform unit.

[0156] Although the present invention has been thoroughly described and described in relation to its exemplary embodiments, it will be apparent to the average person skilled in the art that various changes in form and detail may be made therein without departing from the scope of the invention. as defined in the following claims.

[0157] A device for encoding an image or for decoding an image or an encoder or image decoder shown in FIG. 1, 2, 4, 5, 9, 11, or 15 may include a bus connected to each unit of this device or encoder or decoder, at least one processor that is connected to this bus to perform instructions, and memory connected to the bus to store orders, received messages, and messages generated.