Method and apparatus for encoding and decoding image by using large transform unit

Technical field] [0001] Exemplary embodiments relate to image decoding methods.

[Background of the invention] [0002] In most methods and apparatus for encoding and decoding an image, a pixel image is converted to a frequency domain, and the transformed image is encoded to compress the image. Discrete cosine transform (DCT) is a well-known technology used to compress audio / video (A / V) data. Recently, many attempts have been made to find more efficient coding methods. In audio coding, parametric coding works better than DCT, and for two-dimensional data, the Karhunen Loeve Transform (KLT) has the smallest bit size, but has a large redundant size.

[0003] Document WIEGAND T ET AL, "Overview of the H.264 / AVC video coding standard", IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, IEEE

SERVICE CENTER, PISCATAWAY, NJ, US, vol. 13, No. 7, pages 560 - 576, describes the profiles and applications for this standard, and discusses the history of the standardization process.

The main goals of the H.264 / AVC standardization approach were to increase compression efficiency and provide a "network-friendly" video representation focused on "conversational" (video conversations) and "non-conversational" (storage, broadcast, or streaming) [Disclosure of Invention] [Solution to the problem] [0004] Exemplary embodiments provide a method for decoding an image using efficient transformation.

[Advantageous Effects of the Invention] [0005] According to exemplary embodiments, the image is compressed and encoded more efficiently because the transform unit can be set to be larger than the prediction unit and the transform can be performed on the transform unit.

[Brief description of the drawing figures] [0006] The above and / or other aspects will become more apparent by detailed description of exemplary embodiments with reference to the attached drawing figures, in which:

FIG. 1 is a block diagram of an image encoding device;

FIG. 2 shows a block diagram of an image decoding device,

FIG. 3 shows hierarchical coding units

FIG. 4 is a block diagram of an image encoder based on a coding unit;

FIG. 5 shows a block diagram of an image decoder based on a coding unit,

FIG. 6 shows the maximum coding unit, sub-coding unit and prediction unit;

FIG. 7 shows a coding unit and a transformation unit;

FIG. 8A, 8B, 8C and 8D show the division shapes of a coding unit, a prediction unit and a transformation unit;

FIG. 9 is a block diagram of an image encoding device;

FIG. 10 is a diagram for describing the prediction method;

FIG. 11 is a block diagram of a transformation element;

FIG. 12A to 12C are diagrams of the types of transformation units;

FIG. 13A to 13D are diagrams of the types of transformation units;

FIG. 14 is a diagram of other transformation units;

FIG. 15 is a block diagram of an image decoding device,

FIG. 16 is a flowchart of an image coding method; and

FIG. 17 is a flowchart of an image decoding method according to an exemplary embodiment.

[Best Mode for Using the Invention] [0007] In accordance with another aspect of the exemplary embodiment, an image decoding method is provided, such as as set out in the appended claim 1.

[Mode of the invention] [0008] Specific exemplary embodiments are described in more detail below with reference to the accompanying drawings. Expressions such as "at least one of" when they precede a list of items, change the entire list of items, but do not change individual items from that list. As used herein, "image" means a still image for video, or a moving image, i.e., video only.

[0009] In the following description, similar reference numerals are used for similar elements, even in the various figures. Issues defined in the description, such as detailed construction and components, are provided to facilitate in-depth understanding of exemplary embodiments. However, exemplary embodiments may be implemented without these specific issues.

[0010] FIG. 1 is a block diagram of an image encoding apparatus 100 for encoding the image. The image encoding device 100 may be implemented as a hardware device, such as, for example, a computer processor or computer system. The image encoding device 100 may also be implemented as a program module arranged in a computer system.

[0011] Referring to FIG. 1, the image coding apparatus 100 includes a maximum coding unit separation element 110, a coding depth determining element 120, an image data encoder 130, and an encoding information encoder 140 that can be implemented, e.g., as hardware or program modules disposed within the device 100 for image encoding or separately from the image encoding device 100.

[0012] The maximum coding unit separation element 110 may split the current frame or slice based on the maximum coding unit, which is the largest coding unit. That is, the maximum coding unit separation element 110 may split the current frame or slice into at least one maximum coding unit.

[0013] The coding unit may be represented using the maximum coding unit and depth. As described above, the maximum coding unit indicates the coding unit having the largest size among the coding units of the current frame, and the depth indicates the degree of hierarchical reduction of the coding unit. As the depth increases, the coding unit may decrease from the maximum coding unit to the minimum coding unit, where the depth of the maximum coding unit is referred to as the minimum depth and the depth of the minimum coding unit is referred to as the maximum depth. Because the size of the coding unit decreases from the maximum coding unit as the depth increases, the coding sub-unit with the k -th depth can contain multiple coding sub-units with the (k + n) -th depth (k and n are integers equal to or greater than 1).

[0014] According to the increase in the size of the frame to be encoded, the coding of the image in a larger coding unit may give a higher image compression ratio. However, if the larger coding unit is fixed, the image cannot be efficiently encoded by reflecting the ever-changing characteristics of the image.

[0015] For example, if a smooth area such as sea or sky is coded, then the larger the coding unit, the more the coding rate may increase. However, if a complex area, such as people or buildings, is coded, then the smaller the coding unit, the more the compression ratio can increase.

[0016] Accordingly, different maximum image coding units and different maximum depths are set for each frame or slice. Since the maximum depth means the maximum number of times by which the coding unit can be reduced, the size of each maximum coding unit contained in the maximum image coding unit can be varied according to the maximum depth. The maximum depth can be set differently for each frame or slice, or for each maximum coding unit.

[0017] The coding depth determining element 120 sets the split shape of the maximum coding unit. The shape of the split can be determined based on the calculation of the rate-distortion (RD). A predefined split shape of the maximum coding unit is provided to the coding information encoder 140, and image data according to the maximum coding units is provided to the image data encoder 130.

[0018] The maximum coding unit may be divided into sub-coding units having different sizes according to different depths, and the coding sub-units having different sizes that are included in the maximum coding unit may be subjected to prediction or frequency transformation based on units processing having different sizes. In other words, the image encoding device 100 may perform a number of processing operations to encode the image based on processing units having different sizes and different shapes. To encode image data, processing operations such as prediction, transformation, and entropy coding are performed in which processing units having the same size or different sizes can be used for each operation.

[0019] For example, the image coding apparatus 100 may select a processing unit that is different from the coding unit to predict the coding unit.

[0020] If the size of the coding unit is 2N x 2N (where N is a positive integer), then the processing units may have 2N x 2N, 2N x N, N x 2N, and N x N. In other words, motion prediction can be performed on a processing unit having a shape in which at least one of the height and width of the coding unit is equally divided by two. From now on, the processing unit that forms the basis of the prediction is referred to as the prediction unit.

[0021] The prediction mode may be at least one of the intra-frame mode, the inter-frame mode, and the skip mode, and the specific prediction mode can only be implemented on a prediction unit having a specific size and shape. For example, the intra-frame prediction mode can be implemented only for prediction units having sizes 2N x 2N or N x N, and a square shape. In addition, the skip mode can only be implemented on a prediction unit having a size of 2N x 2N. If there are multiple prediction units in a coding unit, then the prediction mode with the least number of coding errors can be selected after performing a prediction for each prediction unit.

[0022] Alternatively, the image coding apparatus 100 may perform a frequency conversion on the image data based on processing units having a size other than the size of the coding unit. For frequency conversion in a coding unit, frequency conversion may be performed based on a processing unit having a size equal to or less than that of the coding unit. From now on, the processing unit that forms the basis of the frequency transformation is referred to as the transformation unit. The frequency transformation can be a discrete cosine transform (DCT) or a Karhunen Loeve transform (KLT).

[0023] The coding depth determining element 120 may determine the coding sub-units contained in the maximum coding unit using RD optimization based on Lagrange multiplier. In other words, the coding depth determining element 120 may shape the number of coding sub-units separated from the maximum coding unit, wherein these coding sub-units have different sizes according to the depth of the coding sub-units. The image data encoder 130 outputs the bit stream by coding the maximum coding unit based on partition shapes determined by the coding depth determining element 120.

[0024] The coding information encoder 140 encodes information regarding the coding mode of the maximum coding unit determined by the coding depth determining element 120. In other words, the coding information encoder 140 outputs the bit stream by encoding information regarding the shape of the division of the maximum coding unit, information about the maximum depth, and information about the coding mode of the coding subunit for each depth. Information about the coding mode of the coding sub-unit may include information about the prediction unit of the coding sub-unit, information about the prediction mode for each prediction unit, and information about the transformation unit of the coding sub-unit.

[0025] The information regarding the split shape of the maximum coding unit may be marker information indicating whether each coding unit is split. For example, if the maximum coding unit is split and coded, information indicating whether the maximum coding unit is split is coded. Thus, if the sub-coding unit separated from the maximum coding unit is divided and coded, information indicating whether the coding sub-unit is divided and coded is coded.

[0026] Since for the maximum coding unit there are sub-coding units having different sizes, and information about the coding mode is set for each coding sub-unit, information on the at least one coding mode can be set for one maximum coding unit.

[0027] The image coding apparatus 100 may generate sub-coding units by equally dividing the height and width of the maximum coding unit by two according to the increase in depth. This means that when the coding unit size is 2N x 2N depth, the coding unit size (k + 1) depth is N x N. [0028] Accordingly, the image coding apparatus 100 may determine the optimal split shape for each maximum coding unit based on the sizes of the maximum coding units and the maximum depth taking into account the characteristics of the image. By varying the size adjustment of the maximum coding unit taking into account the image characteristics and image coding by dividing the maximum coding unit into sub coding units of different depths, images having different resolutions can be coded more efficiently. [0029] FIG. 2 shows an image decoding device 200 for decoding an image. The image decoding device 200 may be implemented as a hardware device, such as, for example, a computer processor, or computer system. The image decoding device 200 can also be implemented as a program module located in a computer system.

[0030] Referring to FIG. 2, the image decoding device 200 includes an image data acquisition unit 210, an encoding information extraction element 220, and an image data decoder 230 that can be implemented, e.g., as hardware or software modules embedded in the image decoding device 200, or separately relative to the image encoding device 200.

[0031] The image data acquisition unit 210 acquires image data according to the coding units by analyzing the bit stream received by the image decoding apparatus 200 and outputs this image data to the image data decoder 230. The image data acquisition unit 210 may extract information about the maximum coding unit of the current frame or slice from the header of the current frame or slice. In other words, the image data acquisition unit 210 divides the bit stream in a maximum coding unit such that the image data decoder 230 can decode image data according to the maximum coding units.

[0032] The coding information extraction element 220 extracts information about the maximum coding unit, maximum depth, split shape of the maximum coding unit, coding mode of the sub-coding units from the header of the current frame by analyzing the bit stream received by the image decoding apparatus 200. Partition shape information and coding mode information is provided to the image data decoder 230.

[0033] The information about the split shape of the maximum coding unit may include information about the coding sub-units having different sizes according to the depth and included in the maximum coding unit, and may be marker information indicating whether each coding unit is split. [0034] The coding mode information may include prediction unit information corresponding to coding sub-units, prediction mode information, and transformation unit information.

[0035] The image data decoder 230 restores the current frame by decoding the image data of each maximum coding unit based on the information extracted by the coding information extraction element 220.

[0036] The image data decoder 230 may decode the coding sub-units contained in the maximum coding unit based on the information on the partition shape of the maximum coding unit. The decoding process may include predictive processing including intra-frame prediction and motion compensation as well as the inverse transformation process.

[0037] The image data decoder 230 may perform intra-frame prediction or inter-frame prediction based on the prediction unit information and the prediction mode information to predict the prediction unit. The image data decoder 230 may also perform reverse transforming for each coding sub-unit based on the transformation information sub-coding unit information. [0038] FIG. 3 shows hierarchical coding units.

[0039] Referring to FIG. 3, these hierarchical coding units may include coding units whose widths and heights are 64 x 64, 32 x 32, 16 x 16, 8 x 8, and 4 x 4. In addition to these coding units having exactly square shapes, there may also be units coding whose widths and heights are 64 x 32, 32 x 64, 32 x 16, 16 x 32, 16 x 8, 8 x 16, 8 x 4, and 4 x 8.

[0040] Referring to FIG. 3, for image data set 310 whose resolution is 1920 x 1080, the size of the maximum coding unit is set to 64 x 64, and the maximum depth is set to 2.

[0041] For an image data set 320 whose resolution is 1920 x 1080, the size of the maximum coding unit is set to 64 x 64 and the maximum depth is set to 3. For an image data set 330 whose resolution is 352 x 288, the maximum size the coding unit is set to 16 x 16 and the maximum depth is set to 1.

[0042] If the resolution is high or the amount of data is large, the maximum size of the coding unit can be relatively large not only to increase the compression ratio and to more accurately convey the characteristics of the image. Accordingly, as the maximum size of the coding unit of the video image data sets 310 and 320 having a higher resolution than the image data set 330, 64 64 64 may be selected.

[0043] Maximum depth indicates the total number of layers in hierarchical coding units. Since the maximum depth of the image data set 310 is 2, the coding unit 315 of the image data set 310 may include a maximum coding unit whose long axis is 64, and coding sub-units whose long axis sizes are 32 and 16, according to the increase in depth.

[0044] On the other hand, since the maximum depth of the image data set 330 is 1, the coding unit 335 of the image data set may include a maximum coding unit whose long axis is 16, and coding sub-units whose dimensions of the long axis are 8, according to the increase in depth.

[0045] However, since the maximum depth of the image data set 320 is 3, the coding unit 325 of the image data set may include a maximum coding unit whose long axis is 64, and coding sub-units whose long axis dimensions are 32, 16, 8 and 4, as the depth increases. Since the image is encoded based on the smaller coding sub-unit as the depth increases, these examples are useful for encoding an image containing multi-minute scenes.

[0046] FIG. 4 is a block diagram of an image encoder 400 based on coding units. The image encoder 400 may be implemented as a hardware device, such as, for example, a computer processor, or program module disposed on a computer system.

[0047] Intra-frame prediction element 410 performs intra-frame prediction per intra-frame mode prediction unit in current frame 405, and motion estimation element and motion compensator 425 perform inter-frame prediction and motion compensation on inter-frame mode prediction units using current frame 405 and reference frame 495. Intra-frame prediction element 410, motion estimation element 420, motion compensator 425 and reference frame 495 may be implemented, for example, in hardware form or as software modules integrated with the image encoder 400, or independent of the image encoder 400. [0048] The residual values ​​are generated based on the prediction units output from the in-frame prediction element 410, the motion estimation element 420 and the motion compensator 425. These generated residual values ​​are output as quantized transform coefficients by passing them through transform element 430 and quantization element 440.

[0049] These quantized transformation coefficients are restored to residual values ​​by passing them through inverse quantization element 460 and inverse transformation element 470, and these restored residual values ​​are post-processed by passing them through a block removal unit 480 and a loop filter unit 490 and output as the reference frame 495. These quantized transform coefficients can be output as bit stream 455 by passing through an entropy encoder 450.

[0050] In order to carry out the coding based on the coding method, element 410 intra-frame prediction, traffic estimation element 420, 425 motion compensator, transformation element 430, quantization element 440, 450 entropy coder, reverse quantization element 460, reverse transformation element 470, block removal unit 480 and loop filtering unit 490 perform image encoding process based on the maximum coding unit, pod7 coding unit appropriate for the depth, the prediction unit and the transformation unit.

[0051] FIG. 5 is a block diagram of an image decoder 500 based on a coding unit. The image decoder 500 may be implemented as a hardware device, such as, for example, a computer processor or as a program module arranged in a computer system.

[0052] Bit stream 505 passes through the parser 510 so that the encoded image data to be decoded and the encoding information necessary for decoding are extracted. The encoded image data is output as inverse quantized data by passing through entropy decoder 520 and inverse quantization element 530 and restoring to residual values ​​by passing through inverse transformation element 540. These residual values ​​are restored to the coding units, respectively, by adding them to the intra-frame prediction result from the intra-frame prediction element 550 or the motion compensation result from the motion compensator 560. These restored coding units 585, 595 are used to predict next coding units or the next frame by passing through the block removal unit 570 and the loop filtering unit 580. Syntax analyzer 510, entropy decoder 520, inverse quantization element 530, inverse transformation element 540, in-frame prediction element 550, compensator 560, block removal unit 570, and loop filtering unit 580 can be implemented, for example, in hardware or as program modules located inside the image decoder 500 or independently of the image decoder 500.

[0053] To perform decoding based on a decoding method according to an exemplary embodiment, 510 parser 520 entropy decoder inverse quantization element 530, inverse transformation element 540, element 550 intra-frame prediction, 560 motion compensator, block removal unit 570, and the filtering unit 580 in the image decoder loop 500 performs image decoding processes based on the maximum coding unit, coding sub-unit corresponding to the depth, a prediction unit and a transformation unit. In particular, intra-frame prediction element 550 and motion compensator 560 determine the prediction unit and prediction mode in the coding sub-unit by taking into account the maximum coding unit and depth, and inverse transform element 540 performs the inverse transformation considering the size of the transform unit.

[0055] FIG. 6 shows the maximum coding unit, sub-coding unit and prediction unit.

[0056] The image encoding device 100 shown in FIG. 1 and the image decoding device 200 shown in FIG. 2 use hierarchical coding units to perform coding and decoding taking into account image characteristics. The maximum coding unit and maximum depth can be adaptively set according to the characteristics of the image or set differently according to user requirements.

[0057] In FIG. 6, the hierarchical structure 600 of the coding unit has a maximum coding unit 610, which is the maximum coding unit whose height and width are 64 and the maximum depth is 4. The depth increases along the vertical axis of the hierarchical structure 600 coding units, and as the depth, height and width of the coding sub-units increase from 620 to 650 they decrease. The prediction units of the maximum coding unit 610 and the coding sub-units 620 to 650 are shown along the horizontal axis of the hierarchical structure 600 of the coding unit. [0058] The maximum coding unit 610 has a depth of 0 and a size of coding unit, or a height and width of 64 x 64. The depth increases along the vertical axis, and there are the first coding sub-unit 620, which size is 32 x 32 and the depth is 1, the second coding sub-unit 630, which size is 16 x 16 and the depth is 2, the third coding sub-unit 640 whose size is 8 x 8 and depth is 3, and the minimum coding unit 650, which size is 4 x 4 and depth is 4. The minimum coding unit 650, the size of which is 4 x 4 and the depth is 4, is the minimum coding unit, and this minimum coding unit can be divided into prediction units, each of which is smaller than the minimum coding unit.

[0059] Referring to FIG. 6, examples of prediction units are shown along the horizontal axis for each depth. That is, the prediction unit of the maximum coding unit 610 whose depth is 0 may be a prediction unit whose size is equal to the size of 64 x 64 maximum coding unit, or a prediction unit 612 which size is 64 x 32, or a prediction unit 614 which the size is 32 x 64, or a 616 prediction unit, whose size is 32 x 32, which is smaller in size than that of the maximum coding unit, which size is 64 x 64.

[0060] The prediction unit of the first coding sub-unit 620, whose depth is 1, and the size is 32 x 32 can be a prediction unit, whose size is equal to the size of 32 x 32 of the first sub-coding unit, or a 622 prediction unit, which size is 32 x 16 a unit of 624 predictions, which size is 16x32, or a 626 prediction unit, which size is 16 x 16 which is smaller than the one for the first coding sub-unit 620, whose size is 32 x 32.

[0061] The prediction unit of the second coding sub-unit 630, whose depth is 2, and the size is 16 x 16 can be a prediction unit, whose size is equal to the size of 16 x 16 of the second sub-unit 630 coding, or a 632 prediction unit, which size is 16 x 8, 634 prediction unit which size is 8 x 16 or a 636 prediction unit, whose size is 8 x 8, which is smaller than the one for the second coding sub-unit 630, whose size is 16x16.

[0062] The prediction unit of the third coding sub-unit 640 whose depth is 3 and the size is 8 x 8 may be a prediction unit whose size is equal to the size of 8 x 8 of the third coding sub-unit 640, or a prediction unit 642 whose size is 8 x 4, a 644 prediction unit, which is 4 x 8, or a 646 prediction unit, which is 4 x 4, which is smaller than that of the third coding subunit 640, which is 8 x 8.

[0063] The minimum coding unit 650 whose depth is 4 and the size is 4 x 4 is the minimum coding unit with maximum depth. The prediction unit of the minimum coding unit 650 may be a prediction unit 650 whose size is 4 x 4, a prediction unit 652 having a size 4 x 2, a prediction unit 654 having a size 2 x 4, or a prediction unit 656 having a size 2 x 2.

[0064] FIG. 7 shows a coding unit and a transformation unit.

[0065] The image encoding device 100 shown in FIG. 1 and the image decoding apparatus 200 shown in FIG. 2, carry out the coding and decoding of the image at the maximum coding unit or at coding sub-units that are equal to or smaller than the maximum coding unit, extracted from the maximum coding unit. In the coding and decoding process, the size of the transformation unit for the frequency transformation is selected larger than the corresponding coding unit. For example, if the current coding unit 710 has a size of 64x64, then the frequency transformation may be performed using a conversion unit 720 having a size of 32x32.

[0066] FIG. 8A, 8B, 8C and 8D show split shapes of coding unit, prediction unit and transformation unit.

[0067] FIG. 8A and 8B, respectively, represent a coding unit and a prediction unit.

[0068] FIG. 8A shows the split shape selected by the image encoding device 100 shown in FIG. 1, to code the maximum coding unit 810. The image coding apparatus 100 splits the maximum coding unit 810 into different shapes, performs coding, and selects the optimal split shape by comparing the results of coding the various split shapes with each other based on an RD rating. If it is optimal that the maximum coding unit 810 is to be encoded, then the maximum coding unit 810 may be encoded without dividing the maximum coding unit 810 as shown in FIG. from 8A to 8D.

[0069] Referring to FIG. 8A, the maximum coding unit 810, whose depth is 0, is coded by dividing the maximum coding unit 810 into coding sub-units 812, 854 whose depths are equal to or greater than 1. This means that the maximum coding unit 810 is split into 4 coding sub-units whose depths are 1, and all or some of these coding sub-units whose depths are 1 are separated into 814, 816, 818 , 828, 850, and 852 encodings, whose depths are 2.

[0070] The sub-coding unit located in the upper-right part and the sub-coding unit located in the lower-left part among those coding sub-units whose depths are 1, are divided into coding sub-units whose depths are equal or greater than 2. Some of these coding sub-units whose depths are equal to or greater than 2 may be further separated into 820, 822, 824, 826, 830, 832, 840, 842, 844, 846, and 848 coding sub-units are equal to or greater than 3. [0071] FIG. 8B shows the shape of the prediction unit division for the maximum coding unit 810.

[0072] Referring to FIG. 8B, the prediction unit 860 for the maximum coding unit 810 may be split differently from the maximum coding unit 810. In other words, the prediction unit for each of the coding sub-units may be smaller than the corresponding sub-coding unit.

[0073] For example, the prediction unit for the coding sub-unit 854 located in the lower-right part of those coding sub-units 812, 854 whose depths are 1 may be smaller than the coding sub-unit 854. In addition, the prediction units for the coding sub-units 814, 816, 850, and 852 of the coding sub-units 814, 816, 818, 828, 850, and 852 of the coding, whose depths are 2 may be smaller, respectively, than these sub-units 814 , 816, 850, and 852 coding.

In addition, the prediction units for the coding sub-units 822, 832, and 848 whose depths are 3 may be smaller than those for the coding sub-units 822, 832 and 848, respectively. These prediction units may have a shape as a result of which the respective units are equally divided by two in height or width direction, or have a shape as a result of which the corresponding coding sub-units are equally divided by four in height and width directions.

[0075] FIG. 8C and 8D represent the prediction unit and the transformation unit.

[0076] FIG. 8C shows a prediction unit split shape for the maximum coding unit 810 shown in FIG. 8B and FIG. 8D shows the split shape of the transformation unit of the maximum coding unit 810.

[0077] Referring to FIG. 8D, the split shape of the transformation unit 870 may be set differently from the prediction units 860.

[0078] For example, even if the prediction unit for coding sub-unit 854 whose depth is 1 has a selected shape, as a result of which the height of coding sub-unit 854 is equally divided by two, the transform unit may be selected with the original sub-unit coding size of 854. Similarly, even if the prediction units for the coding sub-units 814 and 850, whose depths are 2, are selected in a shape that results in the height of each of these coding sub-units 814 and 850 being divided equally by two, the transformation unit can be selected of the same size as the original size of each of these 814 and 850 coding sub-units.

[0079] The transformation unit may be selected with a smaller size than the prediction unit. For example, if the prediction unit for coding sub-unit 852 whose depth is 2 is selected with a shape by which the width of coding sub-unit 852 is evenly divided by two, then the transform unit may be selected with a shape as a result of of which, the coding sub-unit 852 is equally divided by four in height and width directions, which is smaller in size than the shape of the prediction unit.

[0080] Alternatively, as will be described with reference to FIG. from 13A to 13D, the transformation unit may be set to have a size larger than the coding unit, regardless of the coding unit.

[0081] FIG. 9 shows a block diagram of an image encoding device 900.

[0082] Referring to FIG. 9, the image coding apparatus 900 includes a prediction unit 910, a transform element 920, a quantization element 930, and an entropy encoder 940.

[0083] Prediction unit 910 generates residual values ​​by performing intra-frame prediction or inter-frame prediction on one or more coding units. As will be described later, residual values ​​contained in multiple prediction units can be grouped into one transformation unit and then transformed into the frequency domain, and thus residual values ​​are generated by prediction of one or more coding units based on these numerous units prediction units. The frequency domain transformation can be DCT or KLT.

[0084] As described above with reference to FIG. 8A, in an image decoding method, one coding unit may include a plurality of prediction units. Therefore, the prediction unit 910 can predict each of these prediction units, and generate residual values ​​of these prediction units contained in that one coding unit.

[0085] Alternatively, the prediction unit 910 may perform the prediction of multiple coding units simultaneously. As will be described later, many prediction units contained in multiple coding units may be grouped into one transformation unit, and thus residual values ​​are generated by predicting each of the prediction units contained in these coding units. For example, all coding sub-units contained in one maximum coding unit may be predicted to generate residual values ​​of these coding units.

[0086] According to traditional technology, since the transformation (for example, DCT, or KLT) is performed at a size less than or equal to the prediction unit, the predetermined prediction unit is independently coded, restored, and then used to make the next prediction prediction units. However, according to the method of image coding, which will be described later, since the transformation is performed by grouping the prediction units contained in one or more coding units into one transformation unit, the predetermined prediction unit cannot be independently coded and restored. This will be described in detail with reference to FIG. 10.

[0087] FIG. 10 is a diagram for describing the prediction method.

[0088] Referring to FIG. 10, one coding unit 1000 may contain a plurality of coding units from 1010 to 1040. If the transformation is performed at a size less than or equal to the prediction unit, as in traditional technology, then these prediction units from 1010 to 1030 can be coded and restored before coding the unit 1040 predictions in the lower right.

[0089] Accordingly, if the prediction unit 1040 is to be predicted via intra-frame prediction according to traditional technology, the prediction unit 1040 is subjected to intra-frame prediction using pixels neighboring the prediction unit 1040 from among the pixels generated by coding and then restoring these prediction units from 1010 to 1030.

[0090] On the other hand, here, many of the prediction units are grouped into one transformation unit, followed by transformation. Here, if these prediction units from 1010 to 1040 in FIG. 10 are grouped into one transformation unit, then the 1040 prediction unit in the lower right portion is encoded with the other 1010 to 1030 prediction units, and therefore these 1010 to 1030 prediction units are not coded before the 1040 prediction unit coding. Accordingly, the prediction unit 1040 cannot be subjected to intra-frame prediction using these pixels generated by coding and then restoring these prediction units from 1010 to 1030.

[0091] Consequently, the prediction unit 910 of FIG. 9 can predict the prediction unit 1040 by using the prediction values ​​of these prediction units from 1010 to 1030. The prediction unit 1040 in the lower right section is predicted using these prediction values ​​of these prediction units from 1010 to 1030, instead of those pixels generated by coding, and then restoring these prediction units from 1010 to 1030.

[0092] In other words, if there is a first prediction unit predicted via intra-frame prediction from among the prediction units grouped in one transformation unit, the first prediction unit may be subjected to intra-frame prediction using the prediction value of at least one adjacent prediction unit .

[0093] Alternatively, these prediction units grouped in one transformation unit may all be predicted via inter-frame prediction. As described with reference to FIG. 10, because the prediction unit that is predicted via inter-frame prediction is problematic when grouping multiple prediction units into one transformation unit, all prediction units grouped in the transformation unit can be predicted using only inter-frame prediction.

[0094] Referring again to FIG. 9, the transform element 920 receives an image processing unit in the pixel domain, and converts the image processing unit into the frequency domain. Transformer element 920 transforms these residual values ​​generated by the prediction unit 910 into the frequency domain.

[0095] As described above, transformation element 920 groups these prediction units into one transformation unit and performs DCT or KLT according to that transformation unit. These residual values ​​may be residual values ​​of numerous prediction units contained in one or more coding units. Frequency component coefficients are generated as a result of converting a pixel domain into a frequency domain.

[0096] Conversion to the frequency domain can be performed via DCT or KLT, and discrete cosine coefficients are generated as a DCT or KLT result. However, any transformation can be used to transform an image in the pixel domain into the frequency domain.

[0097] FIG. 11 is a block diagram of a transformation element 920.

[0098] Referring to FIG. 11, transformation element 920 includes a selection element 1110 and a transformation element 1120.

[0099] Dial element 1110 sets one transformation unit by selecting a plurality of adjacent prediction units. According to the traditional image coding devices described above, intra-frame prediction or inter-frame prediction is performed according to a predetermined prediction unit, and DCT or KLT is performed at a size less than or equal to the predetermined prediction unit. In other words, traditional image coding devices perform DCT or KLT based on a transformation unit having a size smaller than or equal to the prediction unit.

[0100] However, the image coding compression rate is impaired because the added redundancy increases the size of the transformation unit is reduced due to the header information added to each transformation unit. Accordingly, the image encoding device 900 groups these adjacent prediction units into one transformation unit, and then performs DCT or KLT according to the transformation unit. In particular, since these adjacent prediction units are highly likely to have similar residual values, the compression rate for coding can be significantly increased when DCT or KLT is performed according to the transformation unit generated by the grouping of these adjacent prediction units.

[0101] Accordingly, the dialing element 1110 selects these prediction units to be grouped into one transformation unit on which DCT or KLT is to be performed. These prediction units can be adjacent to each other. This will be described in detail with reference to FIG. 12A to 12C, and from 13A to 13D. [0102] FIG. 12A to 12C are diagrams of the types of transformation units from 1230 to 1250.

[0103] Referring to FIG. 12A to 12C, the prediction unit 1220 may have a shape whereby the coding unit 1210 is equally divided by two in the width direction. The coding unit 1210 may be a maximum coding unit as described above, or a coding sub-unit having a size smaller than the maximum coding unit.

[0104] Even when the coding unit 1210 and the prediction unit 1220 are identical, the transformation units 1230 to 1250 may be different. The size of the conversion unit 1230 may be smaller than that of the prediction unit 1220 as shown in FIG. 12A, or the size of the conversion unit 1240 may be the same as that of the prediction unit 1220, as shown in FIG. 12B. Alternatively, the size of the conversion unit 1250 may be larger than that of the prediction unit 1220, as shown in FIG. 12C.

[0105] These prediction units grouped into one transformation unit may be multiple prediction units contained in different coding units as shown in FIG. 12A to 12C, or may be multiple prediction units contained in different coding units. In other words, the multiple prediction units contained in the at least one coding unit may be grouped into one transformation unit and then transformed into the frequency domain. [0106] FIG. 13A to 13D are diagrams of the types of transformation units. [0107] One maximum coding unit 1300 may be divided into coding subunits 1302 to 1308 of different sizes, and then encoded as shown in FIG. 13A, and each of these sub-coding units 1302 to 1308 may include at least one prediction unit from 1310 to 1340, as shown in FIG. 13B.

[0108] Dial element 1110 may group these prediction units 1310 to 1340 shown in FIG. 13B to one transformation unit 1350 shown in FIG. 13C, and then transform this 1350 transform unit to the frequency domain.

[0109] Alternatively, the dial element 1110 may group these units 1310 and 1330 to 1339 predictions of these sub-coding units 1302 to 1306 to the left to one transform unit 1360, and group these prediction units from 1320 to 1328 and 1340 these coding sub-units 1304 and 1308 located to the right of one conversion unit 1362, as shown in FIG. 13D.

[0110] Referring again to FIG. 11, for the selection element 1110, the selection criterion of a plurality of adjacent prediction units is not limited. As described above, in one example, the dial member 1110 may select a transformation unit based on depth. As described above, the depth indicates the degree of hierarchical reduction of the coding unit from the maximum coding unit of the current slice or frame to the coding sub units. As described above with reference to FIG. 3 and 6, as the depth increases, the size of the coding sub-unit decreases, and therefore the size of the prediction unit contained in the coding sub-unit decreases. Here, if DCT or KLT is carried out according to a transformation unit having a size smaller than or equal to the prediction unit, the image coding compression ratio is reduced because the header information is added to each transformation unit as described above. [0111] Accordingly, the prediction units contained in the coding sub-unit whose depth is equal to or above a predetermined value can be grouped into one transformation unit, and then DCT or KLT can be carried out on that transformation unit. Therefore, the dial element 1110 may set the transformation unit based on the depth of the sub-coding unit. For example, if the depth of the coding unit 1210 of FIG. 12C is higher than k, then dial member 1110 groups these prediction units 1220 into one transformation unit 1250. [0112] Alternatively, if the maximum coding unit comprises a plurality of coding subunits whose depths are equal to or above a predetermined value, then the dial element 1110 may group the prediction units of these coding sub-units into one transformation unit. FIG 13C illustrates an example of grouping prediction units belonging to sub-coding units whose depth is greater than the maximum coding unit, i.e. the depth is greater than 1, into one transformation unit.

[0113] According to another example, the dial element 1110 may set a plurality of adjacent coding units on which a prediction is performed according to the same type of prediction to one transformation unit. Those adjacent prediction units that are predicted using intra-frame prediction or inter-frame prediction are grouped into one transformation unit. Since it is highly likely that those adjacent prediction units that are predicted according to the same type of prediction mode prediction have similar residual values, DCT or KLT can be performed by grouping these adjacent prediction units into one transformation unit. [0114] When the dial element 1110 sets the transform unit, the transform element 1120 transforms these adjacent prediction units into the frequency domain according to the set transform unit. Frequency domain coefficients (e.g., discrete cosine coefficients) are generated by converting these selected prediction units into one transformation unit.

[0115] Referring again to FIG. 9, quantization element 930 quantizes the frequency component coefficients generated by transform element 920. The quantization element 930 may quantize these input coefficients according to a predetermined quantization process.

[0116] The entropy encoder 940 entropy encodes these coefficients quantized by the quantization element 930. Here, these discrete cosine coefficients can be entropy encoded using context-adaptive binary arithmetic coding (CABAC) or context-sensitive variable-length coding (CABAC). context-adaptive variable length coding) (CAVLC).

[0117] The image encoding apparatus 900 may encode marker information indicating whether the transformation unit generated by grouping these adjacent prediction units contains these coefficients. If there are no coefficients to be entropy encoded, i.e., if these quantized coefficients are all "0" then tag information is coded indicating that the transformation units do not contain these coefficients, and these quantized coefficients are not entropy coded separately.

[0118] The image coding apparatus 900 of the current example may determine the optimal transformation unit by performing transformation, quantization, and entropy coding repeatedly on different transformation units. The optimal transformation unit can be determined by mechanically repeating the process of selecting multiple prediction units in a variety of ways instead of selecting these prediction units based on a predetermined criterion, such as depth, or the type of prediction mode itself. The optimal transformation unit can be determined based on the calculation of the RD rating, and this will be described in detail with reference to FIG. 14.

[0119] FIG. 14 is a diagram of another example of transformation units from 1430 to 1460.

[0120] Referring to FIG. 14, the image coding unit 900 repeatedly encodes various transformation units from 1430 to 1460.

[0121] As shown in FIG. 14, the coding unit 1410 may be predicted and encoded based on the prediction unit 1420 having a size smaller than the coding unit 1410. DCT or KLT is performed on residual values ​​generated as a result of the prediction, and here, DCT or KLT can be performed based on various transformation units from 1430 to 1460, as shown in FIG. 14.

[0122] The transform unit 1430 has the same size as the coding unit 1410, and is generated by grouping all prediction units contained in the coding unit 1410.

[0123] The transformation units 1440 have a size that the coding unit 1410 is equally divided by two in the width direction, and are generated by grouping those prediction units that are adjacent in the vertical direction.

[0124] Transformation units 1450 have a size that results in coding unit 1410 being equally divided by two in height direction, and is generated by grouping those prediction units that are adjacent in the horizontal direction.

[0125] The transformation units 1460 have the same sizes as the prediction units 1420.

[0126] The image coding apparatus 900 may determine the optimal transformation unit by performing transformation, quantization, and entropy coding repeatedly on these transformation units from 1430 to 1460.

[0127] Alternatively, the image coding apparatus 900 may encode marker information indicating whether the transformation unit is generated by grouping a plurality of prediction units contained in one or more coding units. For example, if the transformation unit is set by grouping a plurality of prediction units contained in one coding unit, as shown in FIG. from 12A to 12C, the tag information is set to "0", and if the transformation unit is set by grouping a plurality of prediction units included in the plurality of coding units, as shown in FIG. 13A to 13D, the marker information is set to "1".

[0128] FIG. 14 shows an example of determining an optimal coding unit if one transformation unit is set by grouping the prediction units contained in one coding unit. However, the optimal transformation unit can be determined by repeatedly performing DCT, quantization, and entropy coding on various transformation units, as shown in FIG. 14, even if one transformation unit is set by grouping the prediction units contained in the multiple coding units.

[0129] FIG. 15 is a block diagram of an image decoding device 1500.

[0130] Referring to FIG. 15, the image decoding apparatus 1500 includes an entropy decoder 1510, a reverse quantization element 1520, a reverse transform element 1530, and a restore system 1540.

[0131] The entropy decoder 1510 entrodes the frequency component coefficients of the predetermined transformation unit entropy. As described above with reference to FIG. 12A to 12C and 13A to 13D, a transformation unit may be generated by grouping multiple prediction units. As described above, these prediction units may be adjacent to each other, and may be contained in one coding unit or in a number of different coding units.

[0132] As described above with respect to the image coding apparatus 900, the transformation unit may be generated by grouping a plurality of adjacent prediction units based on depth, or by grouping a plurality of adjacent prediction units on which prediction is performed according to the same type of mode prediction, i.e. according to intra-frame or inter-frame mode. Alternatively, as described with reference to FIG. 14, the optimal transformation unit can be selected by performing transformation, quantization, and entropy decoding repeatedly on various transformation units by mechanically repeating the grouping process of multiple prediction units.

[0133] If the transformation unit does not contain coefficients (e.g., discrete cosine coefficients), entropy decoder 1510 cannot separately decode entropy quantized coefficients. If the transformation unit does not contain these quantized coefficients, then these quantized coefficients are not separately entropy coded by referring to predetermined marker information.

[0134] Inverse quantization element 1520 inversely quantizes those frequency component coefficients that are entropy decoded by entropy decoder 1510. Those frequency component coefficients that are entropy decoded according to the quantization step used when coding the transformation unit are inverse quantized.

[0135] The inverse transform member 1530 transforms these inverse quantized frequency component ratios into the pixel domain. Reverse DCT or reverse KLT is performed on these inverse quantized discrete cosine coefficients to reproduce the transformation unit in the pixel domain. As a result of inverse transformation, the residual value of the transformation unit is restored.

[0136] The reconstituted transformation unit comprises a plurality of prediction units, and as described above, these prediction units can be contained in one coding unit or in a number of different coding units.

[0137] The restore system 1540 generates prediction values ​​by performing a prediction of a plurality of prediction units contained in the reconstituted transformation unit. The prediction values ​​of one coding unit are generated if these prediction units grouped in one transformation unit are contained in one coding unit, and the prediction values ​​of the coding units are generated if these prediction units grouped in one transforming unit are contained in multiple coding units. One coding unit or multiple coding units are reproduced by adding these generated prediction values ​​and residual values ​​restored by the inverse transform element 1530.

[0138] Whether prediction values ​​are generated for one coding unit or for multiple coding units can be determined based on marker information indicating whether the image coding apparatus 900 has generated a transformation unit by grouping multiple prediction units contained in one coding unit or in multiple coding units.

[0139] According to one example, if the prediction units grouped to one transformation unit contain a prediction unit that has been subjected to intra-frame prediction, then the intra-frame prediction may be performed based on the prediction values ​​of at least one adjacent prediction unit as described with reference to FIG. 10. Alternatively, many prediction units grouped into one transformation unit can be predicted using inter-frame prediction.

[0140] FIG. 16 is a diagram illustrating how to encode an image.

[0141] Referring to FIG. 16, the image coding apparatus generates residual values ​​by performing prediction on one or more coding units in operation 1610.

[0142] Multiple prediction units grouped into one transformation unit may be included in one coding unit or in multiple coding units. Accordingly, when these prediction units are contained in one coding unit, these residual values ​​are generated by performing prediction on one coding unit, and when these prediction units are contained in multiple coding units, these residual values ​​are generated by performing prediction on numerous coding units.

[0143] A method of generating residual values ​​by predicting prediction units simultaneously, has been described above with reference to FIG. 10.

[0144] In operation 1620, the device sets one transformation unit by selecting multiple prediction units. These prediction units can be included in one coding unit or in multiple coding units. These adjacent prediction units can be selected based on depth, or adjacent prediction units on which the prediction is performed in the same type of prediction mode can be selected.

[0145] In operation 1630, the device transforms these prediction units into a frequency domain according to the transformation units set in operation 1620. Frequency domain coefficients are generated by performing transformation on a transformation unit set by grouping these prediction units.

[0146] In operation 1640, the device quantizes the frequency component coefficients, i.e., the discrete cosine coefficients generated in operation 1630 according to a predetermined quantization process.

[0147] In operation 1650, the device entropy encodes these frequency component coefficients quantized in operation 1640. Entropy coding is performed via CABAC or CAVLC.

[0148] As described with reference to FIG. 14, the method may further include setting the optimal transformation unit by repeating operations 1610 to 1640 on the various transformation units. The optimal transformation unit can be set by performing transformation, quantization, and entropy coding repeatedly on these various transformation units, as shown in FIG. 14. [0149] FIG. 17 is a flowchart illustrating a method of decoding an image, according to an exemplary embodiment.

[0150] Referring to FIG. 17, the device decodes entropy the frequency component coefficients of the predetermined transformation unit in operation 1710. These frequency components may be discrete cosine factors. The transformation unit can be set by grouping multiple prediction units. As described above, these prediction units may be adjacent to each other, and may be contained in one coding unit or in a number of different coding units.

[0151] In operation 1720, the device inversely quantizes those frequency component coefficients that were inverse quantized in operation 1710. Discrete cosine coefficients are inverse quantized using a quantization step that is used during coding.

[0152] In operation 1730, the device inversely converts frequency component coefficients that are inversely quantized in operation 1720 to the pixel domain to reproduce the transformation unit. The restored transformation unit is set by grouping multiple prediction units. Residual values ​​contained in the transformation unit are restored. Residual values ​​of one coding unit are reproduced if these prediction units are contained in one coding unit, and residual values ​​of multiple coding units are restored if these prediction units are contained in these coding units.

[0153] As described above, the transformation unit may be set by grouping adjacent prediction units based on depth, or by grouping adjacent prediction units on which the prediction is performed according to the same prediction mode.

[0154] In operation 1740, the device restores one or more coding units based on the residual values ​​contained in the transformation unit reconstructed in operation 1730. Prediction values ​​are generated by predicting one or more coding units, and the one or more coding units are restored by adding these generated prediction values ​​and residual values ​​restored in operation 1730. A method of predicting prediction values ​​contained in one or more coding units has been described above with reference to FIG. 10. If the transformation unit is set by grouping these prediction units contained in one coding unit, then one coding unit is restored, and if the transformation unit is set by grouping these prediction units contained in multiple coding units, multiple coding units are restored.

[0155] According to exemplary coding forms, the image is more efficiently compressed and encoded because the transform unit can be set to be larger than the prediction unit and the transform can be performed on the transform unit.

[0156] Although the present invention has been thoroughly illustrated and described with reference to its exemplary embodiments, it will be understood by one of ordinary skill in the art that various changes in form and detail can be made therein without departing from the scope of the invention. as set out in the following claims. In addition, exemplary embodiments may also be implemented as computer-readable codes on a computer readable recording medium.

[0157] The image encoding or image decoding device or the image encoder or decoder shown in FIG. 1, 2, 4, 5, 9, 11, or 15 may contain a bus connected to each unit of this device or encoder or decoder, at least one processor that is connected to this bus to carry out orders, and memory connected to the bus to store orders, messages received, and messages generated.

[0158] The computer readable recording medium is any data storage device that can store data so that it can then be read by a computer system. Examples of computer recording media include read-only memory (ROM), random access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical storage media. The computer-readable recording medium can also be distributed through networked computer systems, so that the code read by the computer is stored and executed in a distributed manner. Alternatively, exemplary embodiments may be embodied in the form of transmission means read by the computer on carrier waves or signals for transmission over a network, such as the Internet.