Image decoding method
Abstract
This record has no abstract on file.
Term
2.8 yearsto projected expiry
Projected expiry 2 July 2029, counted from filing; an application has no term until it is granted.
- Priority
- Filed
- Published
- Today
- Projected expiry
1 claim: 1 independent, 0 dependent
- 1Patent claims Zastrzeżenia patentowe 1. A method of decoding an image, the method comprising:1. Sposób dekodowania obrazu, przy czym sposób obejmuje: determining coding units in a hierarchical structure using partition information indicating whether to split the coding unit, syntactically parsed from the received bit stream;ustalanie jednostek kodowania w hierarchicznej strukturze przy użyciu informacji o podziale wskazującej, czy należy podzielić jednostkę kodowania, przeanalizowaną składniowo z odebranego strumienia bitów;determining the prediction unit in the current coding unit by dividing the current coding unit into at least one prediction unit;obtaining the first predicted pixel value of the current prediction unit using the average of adjacent pixels of the current prediction unit;and obtaining a second predicted pixel value located on the top and left edges of the current prediction unit using a weighted average of the obtained first predicted value and at least one are a seven pixel, wherein the second prediction value for the current prediction unit is obtained using at least one adjacent pixel and the weighted value of the obtained first with value prediction and the second prediction value of the upper left pixel of the current prediction unit is obtained using the first neighbor pixel located in the same a column as the top left pixel and a second adjacent pixel located in the same row as the top left pixel, wherein the image is divided into numerous maximum coding units according to ustalanie jednostki predykcji w bieżącej jednostce kodowania przez podział bieżącej jednostki kodowania na co najmniej jedną jednostkę predykcji;uzyskiwanie pierwszej z predykcją wartości pikseli bieżącej jednostki predykcji przy użyciu średniej z sąsiednich pikseli bieżącej jednostki predykcji;oraz uzyskiwanie drugiej z predykcją wartości pikseli usytuowanych na górnej krawędzi i lewej krawędzi bieżącej jednostki predykcji przy użyciu średniej ważonej z uzyskanej pierwszej z predykcją wartości i co najmniej jednego są siedniego piksela, przy czym druga z predykcją wartość dla bieżącej jednostki predykcji jest uzyskiwana przy użyciu co najmniej jednego sąsiedniego piksela i wartości ważonej z uzyskanej pierwszej z predykcją wartości oraz druga z predykcją wartość górnego lewego piksela bieżącej jednostki predykcji jest uzyskiwana przy użyciu pierwszego sąsiedniego piksela usytuowanego w tej samej kolumnie co górny lewy piksel oraz drugiego sąsiedniego piksela usytuowanego w tym samym wierszu co górny lewy piksel, przy czym obraz jest podzielony na liczne maksymalne jednostki kodowania odpowiednio do 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 -80 information on the maximum size of the coding unit, the maximum coding unit is hierarchically divided into one or more coding units for individual depths, according to partition information, the coding unit for the current depth is one of the rectangular data units obtained from splitting the higher level coding unit depth when the split information indicates the split for the current depth, the coding unit for the current depth is divided into smaller depth coding units, regardless of the adjacent coding units, and when the split information indicates no split for the current depth, at least one prediction unit is obtained from the coding unit for the current depth. -80informacji o maksymalnym rozmiarze jednostki kodowania, maksymalna jednostka kodowania jest hierarchicznie podzielona na jedną lub większą liczbę jednostek kodowania dla poszczególnych głębokości, odpowiednio do informacji o podziale, jednostka kodowania dla bieżącej głębokości jest jedną spośród prostokątnych jednostek danych, uzyskaną z podzielenia jednostki kodowania wyższego poziomu głębokości, kiedy informacje o podziale wskazują podział dla bieżącej głębokości, jednostka kodowania dla bieżącej głębokości jest podzielona na jednostki kodowania mniejszej głębokości, niezależnie od sąsiednich jednostek kodowania, oraz kiedy informacje o podziale wskazują brak podziału dla bieżącej głębokości, co najmniej jedna jednostka predykcji jest uzyskiwana z jednostki kodowania dla bieżącej głębokości. Samsung Electronics Co., Ltd Samsung Electronics Co., Ltd Pełnomocnik: Proxy: 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 81 FIG. 1 - 81 FIG. 1 100 100 FIG. 2 FIG. 2 200 200 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 82 64 - 82 64 FIG. 3 FIG. 3 315 315 RESOLUTION: 1920x1080 MAXIMUM CODING SIZE: 64 MAXIMUM DEPTH = 2 ROZDZIELCZOŚĆ: 1920x1080 MAKSYMALNY ROZMIAR JEDNOSTKI KODOWANIA: 64 MAKSYMALNA GŁĘBOKOŚĆ = 2 4X4 4X4 57P3582iPL00 57P3582iPL00 EP 2 7i3 6i8 Bi EP 2 7i3 6i8 Bi - 83 o o - 83 oo 'TT 'TT A kO AND kABOUT H H CQ o er CQ on er 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 84 ιΌ - 84 ιΌ FIG. FIG. 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 FIG. 6 FIG. 6 600 600 MAKSYMALNA JEDNOSTKA MAKSYMALNA WYSOKOŚĆ I MAXIMAL UNIT MAXIMUM HEIGHT DEEPER GŁĘBSZE JEDNOSTKI UNITS ENCODE KODOWANIA 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 86 2N - 86 2N FIG. 7 FIG. 7 CODING UNIT (710) JEDNOSTKA KODOWANIA (710) 64x64 64x64 2N ^ 302 2N ^302 CL) _0 CL)_0 64X64 64X64 JEDNOSTKA UNIT PRZEKSZTAŁCANIA (720) TRANSFORMATION (720) 32X32 32X32 FIG. 8 FIG. 8 PARTITION TYPE (800) RODZAJ PARTYCJI (800) 2N ^, 804 2N^, 804 2N 2N 806 806 808 808 N N PREDICTION MODE (810) TRYB PREDYKCJI (810) ROZMIARY JEDNOSTKI PRZEKSZTAŁCANIA (820) TRANSFORMATION UNIT SIZES (820) 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 87 DEPTH = 0 - 87 GŁĘBOKOŚĆ =0 FIG. 9 FIG. 9 PARTITION TYPE RODZAJ PARTYCJI 910 910 DIVISION, 920 PODZIAŁ ,920 2N_0 2N_0 2N.0 2N.0 914 914 N_0 N_0 DEPTH = 1 GŁĘBOKOŚĆ =1 2N 2N N 1 N 1 NJ) NJ) 916 916 2N_1 2N_1 2N_1 2N_1 2N_1 2N_1 T934 T934 936 936 -η r: j l -η r:j l. 530 DIVISION 530 PODZIAŁ 922 ,924 922 ,924 .942 , 944! .942 , 944! L_ L_ DEPTH = d-1 GŁĘBOKOŚĆ =d-1 2N_ {d-1) 2N_ (d-1) 2N_{d-1) 2N_(d-1) N_ (d-1) 0 N_(d-1) 0 2NJ (H) 2NJ(H) 954 954 DIVISION PODZIAŁ 2N_ (d-1) 2N_(d-1) DEPTH = d GŁĘBOKOŚĆ =d 2N d 2N d 2N_d -960 2N_d —960 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 89 O - 89 O Eli in Eli w about o CM 10 CM lO O o Oh LT3 LT3 ABOUT O and. i. > in < >w < ABOUT O CO o WHAT about waee-γηηπ · waee-γηηπ· MO MO OJ co o OJ what about CC ' CC' M o M o o > o> Q Q LU LU OT OT CL CL Cd cd LD LD ABOUT O 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 90 FIG. 11 - 90 FIG. 11 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 91 FIG. 12 - 91 FIG. 12 L J LJ FIG. 13 FIG. 13 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 * * DIRECTIONS OF PREDICTION MODES KIERUNKI TRYBÓW PREDYKCJI ABOUT O 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 93 FIG. 14C - 93 FIG. 14C 57P3582iPL00 57P3582iPL00 EP 2 7i3 6i8 Bi EP 2 7i3 6i8 Bi - 94 FIG (PIONOWY) (POZIOMY) - 94 FIG (VERTICAL) (HORIZONTAL) (DC) (DC) (DOWN, WITH A JAW TO THE RIGHT) (W DÓŁ, Z ODCHYLENIEM W PRAWO) 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 95 OOOOOO oooooo oooooo oooooo oooooo oooooo ooooo ooooo - 95 OOOOOO oooooo oooooo oooooo oooooo oooooo ooooo ooooo OOOOPi ooo oooooo ooopoo ooopoo ooo ooo ooo ooo ooo OOOOPi ooo oooooo ooopoo ooopoo ooo ooo ooo ooo ooo FIG. 16 FIG. 16 NEIGHBORHOOD PIXEL SĄSIEDNI PIKSEL CODE OF THE CURRENT CODING UNIT ooo ooo οοώο PIKSEL BIEŻĄCEJ JEDNOSTKI KODOWANIA ooo ooo οοώο 160 ooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo 160 ooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo oooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo ooo 57P3582iPL00 57P3582iPL00 EP 2 7i3 6i8 Bi EP 2 7i3 6i8 Bi - 96 FIG. 17 ή - 96 FIG. 17 ή 177 S 177 S Λ ł Λ ł ł ł ł ł s and £ s iz£ 176 176 -a-. -and-. 74 74 Φ1 Φ1 ---<"—170 ---<"—170 171 171 W1 W1 W2 W2 Ί72 >h2 Ί72> h2 B ;SĄSIEDNI PIKSEL B;NEIGHBORHOOD PIXEL 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 FIG. 18 FIG. 18 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 99 FIG. 21 - 99 FIG. 21 END KONIEC 57P35821PL00 57P35821PL00 EP 2 713 618 B1 EP 2 713 618 B1 - 100 FIG. 22 - 100 FIG. 22 2210 2210 2220 2220 2230 2230 2240 2240 2250 2250 2260 2260
364 paragraphs in 145 sections, as filed
TECHNICAL FIELD [0001] One or more aspects of the present invention relate to a video encoding method and apparatus, and a video decoding method and apparatus that can improve the efficiency of video compression by post-processing video data with prediction.
BACKGROUND OF THE INVENTION [0002] In a method of image compression, for example, the Moving Picture Experts Group (MPEG) -1, MPEG-2, MPEG-4 or H.264 / MPEG-4 Advanced Video Coding (AVC), the image is split into macro blocks to encode the image. Each of the macro-blocks is encoded in all coding modes that can be used in inter-picture prediction or intra-picture prediction, and then is coded in a coding mode that was selected according to the bit rate used to encode the macroblock and the degree of distortion of the decoded macro -block relative to the original macro-block.
[0003] Since the supply and reproduction of high definition or high quality video content devices has been developed and started, there has been a growing demand for a video codec that effectively encodes or decodes high definition or high quality video content. In a traditional video codec, video is encoded in units of macro blocks having a predetermined size.
57P35821PL00
EP 2 713 618 B1
BRIEF DESCRIPTION OF THE DRAWINGS [0004]
Fig. 1 shows video coding according to the invention.
Fig. 2 shows video decoding of the present invention.
a device block diagram for an embodiment of the present device block diagram for an embodiment
Fig. 3 is a diagram describing hierarchical coding units according to an embodiment of the present invention.
concept of an example
Fig. 4 is a block diagram of an image encoder based on coding units according to an embodiment of the present invention.
Fig. 5 shows a block diagram of an image decoder based on coding units according to an embodiment of the present invention.
Fig. 6 is a diagram illustrating deeper coding units by depth and a prediction unit according to an embodiment of the present invention.
Fig. 7 is a diagram describing the relationship between a coding unit and a transformation unit, according to an embodiment of the present invention.
Fig. 8 is a diagram describing the coding information of the coding unit according to the coding depth, according to an embodiment of the present invention.
Fig. 9 is a diagram of deeper coding units according to depth, according to an embodiment of the present invention.
57P35821PL00
EP 2 713 618 B1
Figs. 10A and 10B are diagrams illustrating the relationship between a coding unit, a prediction unit, and a transformation unit, according to an embodiment of the present invention.
Fig. 11 is a table showing coding information regarding each coding unit according to an embodiment of the present invention.
Fig. 12 shows a block diagram of an intra-image prediction device according to an embodiment of the present invention.
Fig. 13 is a table showing the number of in-picture prediction modes according to the size of the coding unit, according to an embodiment of the present invention.
Figures 14A to 14C are diagrams for explaining in-picture prediction modes that can be performed in a coding unit having a predetermined size, according to an embodiment of the present invention.
Fig. 15 shows drawings to explain intra-image prediction modes that can be performed in a coding unit having a predetermined size, according to other embodiments of the present invention.
Fig. 16 is a reference diagram for explaining inter-picture prediction modes that have different orientations according to an embodiment of the present invention.
Fig. 17 is a reference diagram for explaining a two-line mode according to an embodiment of the present invention.
57P35821PL00
EP 2 713 618 B1
Fig. 18 is a reference diagram for explaining post-processing of the first prediction coding unit according to an embodiment of the present invention.
Fig. 19 is a reference diagram to explain the operation of the end processor according to an embodiment of the present invention.
Fig. 20 is a reference diagram for explaining the concept of adjacent pixels used by the end processor according to an embodiment of the present invention.
Fig. 21 is a flowchart illustrating a video coding method according to an embodiment of the present invention.
Fig. 22 is a flowchart illustrating a video decoding method according to an embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
TECHNICAL PROBLEM [0005] One or more aspects of the present invention provide a video encoding method and apparatus and a video decoding method and apparatus for improving the efficiency of video compression.
TECHNICAL SOLUTION [0006] According to an aspect of the present invention, a new prediction block is produced by changing the value of each pixel contained in the prediction block by postprocessing the block with the prediction.
57P35821PL00
EP 2 713 618 B1
BENEFIT EFFECT [0007] According to an aspect of the present invention, a new prediction block is produced by changing the value of each pixel contained in the prediction block by postprocessing the block with the prediction, thereby improving the efficiency of video compression.
BEST MODE FOR IMPLEMENTING THE INVENTION [0008] According to an aspect of the present invention, a video coding method is provided, the method comprising: producing a first coding unit with a prediction of the current coding unit to be encoded; producing a second prediction coding unit by changing the value of each pixel of the first prediction coding unit by using each pixel of the first prediction coding unit and at least one adjacent pixel of each pixel and coding the difference between the current coding unit and the second prediction coding unit.
[0009] According to another aspect of the present invention, a video coding device is provided, the device comprising: a prediction system for producing a first coding unit with a prediction of the current coding unit to be encoded; an end processor for generating the second prediction coding unit by changing the value of each pixel of the first prediction coding unit using each pixel of the first prediction coding unit and at least one adjacent pixel of each pixel; and an encoder for coding the difference between the current coding unit and the second prediction coding unit.
57P35821PL00
[0010] According to another aspect of the present invention, a video decoding method is provided, the method comprising: extracting from the received stream of information bits relating to the prediction mode of the current decoding unit to be decoded; playing the first decoding unit with a prediction of the current decoding unit, based on the extracted information related to the prediction mode; extracting from the bit stream information related to the operating mode in which: each pixel of the first prediction decoding unit and adjacent pixels of each pixel are used; playing the second prediction decoding unit by changing the value of each pixel of the first prediction decoding unit using each pixel of the first prediction decoding unit and adjacent pixels of each pixel, based on the extracted information related to the operating mode; extracting from the bit stream a residual block, which is the difference between the current decoding unit and the second prediction decoding unit, and restoring the residual block; and decoding the current decoding unit by adding the residual block to the second prediction decoding unit.
[0011] According to another aspect of the present invention, a video decoding device is provided, the device comprising: an entropy decoder, for extracting from the bit stream information related to the prediction mode of the current decoding unit to be decoded and information related to the mode working area in which: each pixel of the first decoding unit with a prediction of the current decoding unit and adjacent pixels of each pixel of the first predicted decoding unit; system
57P35821PL00
A prediction, for reproducing a first prediction decoding unit based on extracted information relating to a prediction mode; an end processor for reproducing the second prediction decoding unit by changing the value of each pixel of the first prediction decoding unit by using each pixel of the first prediction decoding unit and adjacent pixels of each pixel of the first prediction decoding unit based on the extracted information relating to operating mode; an inverse transformation and inverse quantization unit for reproducing a residual block from the bit stream, which is the difference between the current decoding unit and the second prediction decoding unit; and an adder to decode the current decoding unit by adding the residual block to the second prediction decoding unit.
MODE OF THE INVENTION [0012] In the following, the video coding method and apparatus and the video decoding method and apparatus according to exemplary embodiments of the present invention will be described with reference to the accompanying drawings.
[0013] Fig. 1 is a block diagram of a video encoding device 100 according to an embodiment of the present invention. The video coding apparatus 100 includes a circuit 110 dividing the image into maximum coding units, a coding depth determining circuit 120, an image data encoder 130 and an encoding information encoder 140.
[0014] The image dividing system 110 into maximum coding units may divide the current image or image segment based on the maximum coding unit. The current image or segment can be divided into at least one
57P35821PL00
EP 2 713 618 B1 maximum coding unit. The partition result may be passed to the coding depth determining system 120 according to at least one maximum coding unit.
[0015] According to an embodiment of the present invention, the coding unit may be characterized by a maximum coding unit and a depth. The maximum coding unit is the largest coding unit among the coding units of the current image. Depth means how many times the maximum coding unit has been hierarchically divided to get the coding unit. As the depth increases, deeper coding units, depending on the depth, can be obtained from dividing the maximum coding unit up to the minimum coding unit. The depth of the maximum coding unit may be the largest depth, and the depth of the minimum coding unit may be the smallest depth. Since the size of the coding unit corresponding to each depth decreases as the depth of the maximum coding unit deepens, the coding unit having a depth value "k" may contain a plurality of coding units, each having a depth value greater than "k + 1".
[0016] As described above, the current image data is divided into maximum coding units according to the maximum coding unit size, and each of the maximum coding units may include deeper coding units that are divided according to depth. Because the maximum coding unit according to an embodiment of the present invention is divided according to the depth, the spatial domain image data contained in the maximum coding unit,
57P35821PL00
EP 2 713 618 B1
10 can be hierarchically classified according to depth.
[0017] The maximum depth and maximum size of the coding unit, which limit the total number of times the hierarchical division of the height and width of the maximum coding unit, can be predetermined. The maximum coding unit and maximum depth can be set in image or segment units. That is, each of the image units or segments may have a different maximum coding unit and a different maximum depth included, set depth, minimum coding unit size, can be a maximum coding unit, variable according to maximum, because the maximum coding unit and maximum depth can be set in a way variable for each image unit or segment, video compression efficiency can be improved by encoding an image having a flat region using a maximum coding unit having a relatively large size, and encoding an image with a high complexity using a coding unit having a relatively small size.
[0018] The coding depth determining system 120 sets the maximum depth in such a way that different maximum depths are assigned to the maximum coding units, respectively. The maximum depth can be determined based on the cost of distortion rate (RD) - (Rate - Distortion). The determined maximum depth is transmitted to the encoder information encoder 140, and the image data of the maximum coding unit is transmitted to the image data encoder 130.
57P35821PL00
EP 2 713 618 B1
11 coding encoding.
it is determined in one whether or not [0019] The image data in the maximum coding unit is encoded based on deeper coding units, corresponding to at least one depth, equal to or lower than the maximum depth, and the coding results of the image data are compared based on each of the deeper coding units. The depth having the smallest error can be selected after comparing the errors. At least one coding depth can be selected for each maximum coding unit.
[0020] The size of the maximum coding unit is divided when the coding unit is hierarchically divided according to depth and as the number of coding units increases. Also, even if the coding units correspond to the same maximum coding unit, divide each of the coding units corresponding to the same depth to a greater depth by measuring the coding error of the image data of each coding unit separately. Accordingly, even when the image data is contained in one maximum coding unit, the image data is divided into regions according to depth and coding errors may be different in different regions in one maximum coding unit, and therefore the coding depth may differ according to regions in the data image. Thus, one or more coding may be determined in the coding unit and the image data of the maximum coding unit may be divided according to the coding units of at least one coding depth.
[0021] Also, the coding subunits having different number of depths of one maximum size, the codings that are included can be the maximum prediction unit or transformed based on processing units having different sizes,
57P35821PL00
EP 2 713 618 B1
-12odpowiednio. In other words, the video encoding device 100 may perform a number of operations to encode the video, based on processing units having different sizes and shapes. Video data coding involves various operations, e.g., prediction, transformation and entropy coding. Processing units having the same sizes can be used in all different operations, or processing units having different sizes can be used in different operations, respectively.
[0022] For example, the video coding apparatus 100 may select a processing unit that is different from the coding unit so as to be able to predict the coding unit. If the coding unit size is 2Nx2N, then the processing unit size may be, for example, 2Nx2N, 2NxN, Nx2N, or NxN. Here, N is a positive integer. In other words, motion prediction can be performed in processing units obtained by dividing at least one element of the group comprising the height and width of the coding unit into two equal parts. In the following, the data unit for which the prediction is performed is referred to as the "prediction unit".
[0023] The prediction mode may include at least one member of the group consisting of internal mode, external mode and skip mode. A given prediction mode can be performed only in prediction units, each of which has a predetermined size or shape. For example, internal mode can only be performed on 2Nx2N or NxN prediction units. Also, the skip mode can only be performed on 2Nx2N prediction units. If multiple prediction units are included in the coding unit, the prediction can be performed independently in multiple prediction units
57P35821PL00
EP 2 713 618 B1
13and can choose the prediction mode that has the smallest coding error.
[0024] The video coding apparatus 100 may also perform image data conversion in the coding unit based not only on the coding unit for encoding the image data, but also based on a data unit that is different from the coding unit. To perform the transformation in a coding unit, the transformation may be performed based on a data unit having a size less than or equal to the size of the coding unit. The data unit used as the transformation base will now be referred to as the "transformation unit".
[0025] The coding depth determining system 120 may determine the form into which the maximum coding unit should be divided in such a way that the maximum coding unit may have an optimal coding error, by measuring the coding errors of the coding units corresponding to different depths using speed optimization - distortion Lagrange based on the multiplier. In other words, the coding depth determining system 120 may determine the type of coding subunits into which the maximum coding unit is divided. Here, the size of each coding subunit varies depending on the respective depth.
[0026] The image data encoder 130 encodes the image data of the maximum coding unit based on the at least one coding depth determined by the coding depth determining system 120 and transmits the coding result in the bit stream. Since the coding has already been performed by the coding depth determining system 120, therefore, to measure the smallest coding error, the coded data stream can be sent using the coding result.
57P35821PL00
EP 2 713 618 B1
[0027] The coding information encoder 140 codes the coding mode information for each depth for each maximum coding unit respectively, based on at least one coding depth determined by the coding depth determining circuit 120, and then outputs the coding result in the bit stream. The coding mode information according to each depth may include information relating to at least one coding depth, partition type of the coding unit prediction unit having the given at least one coding depth, prediction mode of each prediction unit, and transformation unit size.
[0028] The coding depth information may be determined using depth-specific partition information that indicates whether coding is performed in coding units of greater depth instead of the current depth. If the current depth of the current coding unit is coding depth, the image data in the current coding unit is encoded and broadcast, and thus the split information may indicate that the current coding unit should not be split to a greater depth. Alternatively, if the current depth of the current coding unit is not the coding depth, the coding is performed in the coding unit of greater depth, and thus partition information may indicate that the current coding unit should be split to obtain coding units of greater depth.
[0029] If the current depth is not the coding depth, the coding is performed in a coding unit that is divided into coding units to a greater depth. Since at least one coding unit of greater depth exists in one coding unit of the current depth, the coding is repeated in
57P35821PL00
EP 2 713 618 B1
15 each coding unit of greater depth, and therefore coding may be performed recursively for coding units having the same depth.
[0030] Since at least one coding depth should be set for one maximum coding unit and information about at least one coding mode should be set for each coding depth, information on at least one coding mode can be set for one maximum coding unit. Also, the coding depth of the image data of the maximum coding unit may be different, depending on the position, since the image data is hierarchically divided according to the depth, and therefore information about the coding depth and coding mode can be determined for the image data.
[0031] Accordingly, in accordance with an embodiment of the present invention, the coding information encoder 140 may determine the coding information about the coding depth of each minimum unit included in the maximum coding unit. That is, the coding unit having the coding depth includes at least one minimum coding unit that contains the same coding information. Thus, if adjacent minimum coding units have the same coding information according to depth, adjacent minimum coding units may be minimum coding units contained in the same maximum coding unit.
[0032] In the video coding apparatus 100, the deeper coding unit may be a coding unit obtained by dividing the height or width of the coding unit of a smaller depth that is one level up into two parts. In other words, when the coding unit size of the current depth k is 2Nx2N,
57P35821PL00
EP 2 713 618 B1
-16 the coding unit size of greater depth (k + 1) is NxN. Also, a coding unit of a given depth having the size 2Nx2N may contain a maximum of 4 coding units of greater depth having the size NxN.
[0033] Accordingly, the video coding apparatus 100 may determine the optimal split form for each maximum coding unit based on the size of the maximum coding unit and the maximum determined depth, taking into account the characteristics of the current image. Also, since coding can be performed at any maximum coding unit using any of the different prediction and transformation modes, an optimal coding mode can be determined taking into account the characteristics of the coding unit of different image sizes.
[0034] If an image having a high resolution or a large amount of data is encoded in a traditional macro block, the number of macro blocks per image increases excessively.
Accordingly, the number of compressed information items generated for each macro-block increases, and therefore it is difficult to send compressed information and the data compression efficiency decreases. However, using video encoding device 100 can increase the efficiency of image compression because the coding unit is matched when taking into account the image characteristics when increasing the maximum size of the coding unit when taking into account the size of the image.
[0035] Fig. 2 is a block diagram of a video decoding device 200 according to an embodiment of the present invention.
[0036] The video decoding apparatus 200 includes an image data acquisition unit 210, an encoding information extractor 220 and an image data decoder 230.
57P35821PL00
EP 2 713 618 B1
[0037] The image data acquisition unit 210 parses the bit stream received by the video decoding device 200 to obtain the image data in maximum coding units and transmits the image data to the image data decoder 230. The image data acquisition unit 210 may extract information relating to the maximum coding units for the current image or segment from the header of the current image or segment. According to an embodiment of the present invention, the video decoding device 200 decodes image data in maximum coding units.
[0038] The coding information extractor 220 parses the bit stream to extract coding depth information and coding mode for each maximum coding unit from the header of the current image or segment. The extracted information about the coding depth and coding mode is sent to the image data decoder 230.
[0039] Information about the coding depth and coding mode according to the maximum coding unit may be determined for the information about the at least one coding unit according to the coding depth, and the coding mode information may include information about the partition type of the respective coding unit according to the coding depth, on the prediction mode and the size of the transformation unit. Also information about the division according to depth can be extracted as information on the coding depth.
[0040] Information relating to the form into which each of the maximum coding units is divided may include information related to the coding subunits having different sizes according to the depth of each maximum coding unit.
57P35821PL00
EP 2 713 618 B1
18 coding unit coding
Information related to the coding mode may include information related to the prediction unit for each of the coding subunits, information related to the prediction mode, information about the transformation unit, etc.
[0041] The image data decoder 230 reproduces the current image or segment by decoding the image data of each of the maximum coding units based on the information extracted by the coding information extractor 220. The image data decoder 230 decodes the coding subunits contained in each of the maximum coding units based on the information relating to the form into which each of the maximum coding units is divided. Decoding can include intra-image prediction, motion estimation, which includes motion compensation and inverse transformation.
[0042] The image data decoder 230 reproduces the current image by decoding the image data in each maximum coding unit based on the depth information and coding mode corresponding to the maximum In other words, the image data decoder 230 can decode the encoded image data based on the extracted type information partition, prediction mode and transformation unit for each coding unit, contained in each maximum coding unit. The decoding process may include prediction, including intra-image prediction and motion compensation, and inverse transformation.
[0043] The image data decoder 230 may perform in-image prediction or motion compensation according to the partition and the prediction mode of each coding unit, based on information about the partition type and the prediction mode of the coding unit prediction unit according to the coding depth. Also, the decoder 230
57P35821PL00
EP 2 713 618 B1
19 of the image data may perform inverse transformation according to each transformation unit in the coding unit, based on information about the size of the transformation unit of the coding unit according to the coding depth, so as to perform the reverse transformation according to the maximum coding units.
[0044] The image data decoder 230 may determine at least one coding depth of the current maximum coding unit by using the split information according to the depth. If the split information indicates that the image data is no longer split in the current depth, the current depth is the coding depth. Accordingly, the image data decoder 230 can decode the encoded data of at least one coding unit according to each coding depth in the current maximum coding unit using information about the type of partition of the prediction unit, the prediction mode and the size of the transformation unit for each coding unit according to the coding depth and output image data of the current maximum coding unit.
[0045] The video decoding device 200 can obtain information about the at least one coding unit that generates the least coding error when the coding is performed recursively for each maximum coding unit and can information for decoding the current image.
optimal coding units in each maximum coding unit can be decoded. Accordingly, even if the image data has a high resolution and large amount of data, the image data can be effectively decoded and reproduced using the size of the coding unit and the coding mode, which are adaptively determined according to the characteristics of the image data, in other words,
57P35821PL00
EP 2 713 618 B1
-20 by using information about the optimal encoding mode received from the encoder.
[0046] Fig. 3 is a diagram for describing the concept of hierarchical coding units according to an embodiment of the present invention.
[0047] Referring to Fig. 3, the hierarchical coding units according to the current embodiment may include a 64x64 coding unit, a 32x32 coding unit, a 16x16 coding unit, an 8x8 coding unit and a 4x4 coding unit. However, the present invention is not limited to this, and the size of the coding unit may be e.g. 64x32, 32x64, 32x16, 16x32, 16x8, 8x16, 8x4 or 4x8.
[0048] In the video data 310, the resolution is 1920x1080, the maximum coding unit size is 64, the maximum depth is 2. In the video data 320, the resolution is 1920x1080, the maximum size of the coding unit is 64, and the maximum depth is 4. In 330 video data, the resolution is 352x288, the maximum size of the coding unit is 16, and the maximum depth is 2.
[0049] If the resolution is high and the amount of data is large, the maximum size of the coding unit can be large not only to increase coding efficiency, but also to accurately render the characteristics of the image. Accordingly, the maximum size of the video coding unit 310 and 230 having a higher resolution than the video data 330 may be equal to 64.
[0050] Since the maximum video data depth 310 is 2, the coding units 315 of the video data 310 may include a maximum coding unit having a long axis size of 64 and coding units having a long axis size of 32 and 16 because the depths include
57P35821PL00
EP 2 713 618 B1
-21 two layers by dividing the maximum coding unit twice. At the same time, since the maximum video data depth 330 is 2, the coding units 335 video data 330 may include a maximum coding unit having a long axis size of 16 and coding units having a long axis size of 8 or 4, because the depths include two layers as a result dividing the maximum coding unit by two.
[0051] Since the maximum video data depth 320 is 4, the coding units 325 of the video data 320 may include a maximum coding unit with a long axis size of 64 and coding units having a long axis size of 32, 16, 8 and 4, because the depths include 4 layers by dividing the maximum coding unit four times. As you increase the depth, you can accurately express information about the details.
[0052] Fig. 4 is a block diagram of an image encoder 400 based on coding units, according to an embodiment of the present invention.
[0053] Referring to Fig. 4, intra-picture prediction system 410 performs intra-picture prediction in internal mode in coding units from current frame 405, and motion estimator 420 and motion compensator 425 perform external estimation and motion compensation in external mode in coding units from current frame 405 using the current frame 405 and reference frame 495.
[0054] Data from the intra-image prediction system 410, motion estimator 420 and motion compensator 425 are transmitted as a quantized transformation coefficient through a transformer 430 and a quantization system 440. In particular, as will be described later with reference to Fig. 12,
57P35821PL00
EP 2 713 618 B1
The intra-image prediction system 410 may perform post-processing in which the value of each pixel of the coding unit with the intra-image prediction is changed using adjacent pixels. Residual values, which are the differences between the values of the coding unit after post-processing and the original coding unit, can be sequentially sent to transformer 430 and quantization system 440, and then ultimately emitted as a quantized transform factor.
[0055] The quantized transform coefficient is reproduced as spatial domain data by inverse quantization system 460 and inverse transformer 470, and the reconstructed spatial domain data is emitted as reference frame 495 after final processing by blocking de-blocking unit 480 and loop filtering unit 490. The quantized transformation coefficient may be emitted as a 455 bit stream by the entropy encoder 450.
[0056] In order for the image encoder 400 to be used in the video encoding apparatus 100, all elements of the image encoder 400, i.e. intra-image prediction system 410, motion estimator 420, motion compensator 425, transformer 430, quantization system 440, entropy encoder, inverse quantization system 460, inverse transformer 470, blocking de-blocking unit 480 and filtering unit in loop 490, perform operations based on each a coding unit from among coding units having a tree structure, when considering the maximum depth of each maximum coding unit. In particular, the intra-image prediction system 410, motion estimator 420 and motion compensator 425 determine the partitions and the prediction mode of each
57P35821PL00
EP 2 713 618 B1
Encoding units among the coding units having a tree structure, when considering the maximum size and maximum depth of the current maximum coding unit, and transformer 430 sets the size of the transformation unit in each coding unit among the coding units having a tree structure.
[0057] Fig. 5 is a block diagram of an image decoder 500 based on coding units according to an embodiment of the present invention.
[0058] The syntax analyzer 510 parses the decoded encoded image data and coding information required to decode the bit stream 505. The encoded image data is transmitted as inverse quantized data by entropy decoder 520 and inverse quantization system 530, and inverse quantized data are rendered as spatial domain image data by a 540 inverse transformer. Image data in the spatial domain that has passed through the 1900 intra-image prediction system with the prediction and motion compensator 560 can be transmitted as the reproduced frame 595 after final processing by the blocking removal unit 570 and the loop filtering unit 580. Also, image data that is subjected to final processing by the de-blocking unit 570 and the loop filtering unit 580 may be emitted as reference frame 585.
[0059] For the image decoder 500 to be used in the video decoding method according to an embodiment of the present invention, all elements of the image decoder 500, i.e. parse analyzer 510, entropy decoder 520, inverse quantization system 530, inverse transformer 540, in-picture prediction system 1900 with prediction, motion compensator 560, removal unit 570
57P35821PL00
EP 2 713 618 B1
The blocking and loop filtering unit 580 performs operations based on coding units having tree structures for each maximum coding unit. In particular, the 1900 in-prediction prediction system and motion compensator 560 perform partition-based operations and prediction mode for each coding unit having a tree structure, and the inverse transformer 540 performs operations based on the size of the transformation unit for each coding unit.
[0060] Fig. 6 is a diagram illustrating deeper depth coding units and partitions according to an embodiment of the present invention.
Video encoding device 100, the video decoding device 200 uses hierarchical coding units, taking into account image characteristics. The maximum and maximum depth of the adaptively set image, or they can be height, the maximum width of the coding units can be set differently by the user according to the characteristics. The sizes of the coding units according to the depth can be set according to the predetermined maximum size of the coding unit.
[0061] In coding, the invention, the hierarchical structure
600 the implementation units of this embodiment and the maximum width according to the example, the maximum height of the coding units is 64 and the maximum depth is 4. Because the depth increases along the vertical axis of the hierarchical structure 600, the deeper coding units have fractional values of height and width. Along the horizontal axis of the hierarchical structure 600, the prediction unit and partitions, which are bases for coding with the prediction of each deeper coding unit are also shown.
57P35821PL00
EP 2 713 618 B1
[0062] In other words, the maximum coding unit 610 is the maximum coding unit in the hierarchical structure 600, wherein the depth is 0 and the size, i.e. height in width is 64x64. The depth increases along the vertical axis and there are units: coding unit 620 size 32x32 and depth 1, coding unit 630 size 16x16 and depth 2, coding unit 640 size 8x8 and depth 3, coding unit 650 size 4x4 and depth 4.
The 650 coding unit 4x4 size and depth 4 is the minimum coding unit.
[0063] Also, referring to Fig. 6, the partitions of each coding unit are set as the prediction units of the coding unit according to depth and along a horizontal axis. In other words, the prediction units of the maximum coding unit 610 of size 64x64 and depth 0 may include the maximum coding unit of 610 of size 64x64 and the partitions contained in the maximum coding unit of 610, i.e. 612 partitions 64x32 size, 614 partitions 32x64 size, and 616 partitions 32x32 size. [0064] Similarly, the coding unit 620 of the 32x32 size and depth 1 coding unit may be partitioned included in the coding unit 620, i.e. partition 620 size 32x32, partition size 622 size 32x16, partition size 624 size 16x32 and partition size 626 16x16 size.
[0065] Similarly, the prediction unit of the coding unit 630 of size 16x16 and depth 2 may be divided into partitions contained in the coding unit 630, i.e. partition size 16x16 contained in coding unit 630, partitions 632 size 16x8, partitions 634 size 8x16 and 836 8x8 partitions.
57P35821PL00
EP 2 713 618 B1
[0066] Similarly, the prediction unit of the coding unit 640 of 8x8 size and depth 3 may be partitioned into partitions contained in the coding unit 640, i.e. partition 8x8 size contained in coding unit 640, partitions 642 size 8x4, partitions 644 size 4x8 and 646 partitions 4x4 size.
[0067] The coding unit 650 with size 4x4 and depth 4 is the minimum coding unit and coding unit with the largest depth. The 650 coding unit prediction unit is assigned only to 4x4 partitions.
[0068] To determine at least one coding depth of the coding units forming the maximum coding unit 610, the coding depth determining system 120 of the video coding apparatus 100 performs coding for the coding units for each depth included in the maximum coding unit 610 respectively.
[0069] The number of deeper coding units corresponding to the depth, containing data in the same range and the same size, increases with increasing depth. For example, four coding units corresponding to depth 2 are required to cover data that is contained in one coding unit corresponding to depth 1. Accordingly, to compare coding results of the same data according to depth, a coding unit corresponding to depth 1 and four coding units corresponding to depth 2 are coded.
[0070] To perform coding for the current depth among different depths, the smallest coding error can be selected for the current depth by performing coding for each prediction unit in coding units corresponding to the current depth along the horizontal axis of the hierarchical structure 600.
57P35821PL00
EP 2 713 618 B1
Alternatively, the smallest coding error can be found by comparing the smallest coding errors according to depth, by performing coding for each depth when the depth increases along the vertical axis of the hierarchical structure 600. The depth and partitions having the smallest coding error in the maximum coding unit 610 can be selected as the coding depth and partition type of the maximum coding unit 610.
[0071] Fig. 7 is a diagram for describing the relationship between coding unit 710 and transformation units 720, according to an embodiment of the present invention.
[0072] A video encoding device 100 or device
200 for video decoding, encodes or decodes an image, respectively, for coding units with sizes smaller than or equal to the maximum coding unit for each maximum coding unit. Sizes of transformation units for transforming during coding may be selected based on data units that are not larger than the corresponding coding unit. For example, in the video encoding apparatus 100 or in the video decoding apparatus 200, if the size of the coding unit 710 is 64x64, the transformations may be performed using the 32x32 size 720 transformation units. Also, the 64x64 coding unit 710 data may be encoded by performing a transformation for each 32x32, 16x16, 8x8 and 4x4 transformation unit that are smaller than 64x64, and then the transformation unit with the smallest coding error can be selected.
57P35821PL00
EP 2 713 618 B1
[0073] Fig. 8 is a diagram for describing coding information of coding units according to coding depth, according to an embodiment of the present invention. The encoder 140 of the video coding information device 100 may encode and transmit partition type information 800, prediction mode information 810, transformation unit information 820 for each coding unit according to the coding depth as coding mode information.
[0074] Information 800 indicates partition shape information obtained by splitting the prediction unit of the current coding unit, the partition being a data unit for coding with a prediction of the current coding unit. For example, the current CU_0 2Nx2N encoding unit may be partitioned into any of the 2Nx2N 802 partitions, 2NxN 804 partition, Nx2N 806 partition and NxN 808 partition. Here, information about the partition type 800 is set to indicate the 804 partition size 2NxN, 806 partition size Nx2N and 808 partition size NxN.
[0075] Information 810 indicates the prediction mode of each partition. For example, information 810 may indicate a prediction coding mode performed in the partition indicated by information 800, i.e., internal mode 812, external mode 814, and skip mode 816.
[0076] Information 820 indicates the transformation unit as the base unit when performing transforms on the current coding unit. For example, the transformation unit may be the first internal transformation unit 822, the second internal transformation unit 824,
57P35821PL00
EP 2 713 618 B1
The first external transformation unit 826, or the second internal transformation unit 828.
[0077] The coding information extractor 220 of the video decoding device 200 may extract and use the information 800, 810 and 820 for decoding, respectively, for each deeper coding unit.
[0078] Fig. 9 is a diagram of deeper coding units according to depth, according to an embodiment of the present invention. Split information can be used to indicate depth changes. The split information indicates whether the current depth coding unit is divided into larger depth coding units.
[0079] The prediction unit 910 for coding with motion prediction of coding unit 900 with a depth of 0 and size 2N_0x2N_0, may contain partitions with partition type 912 of size 2N_0x2N_0, with partition type 914 with size 2N_0xN_0, with partition type 916 with size N_0x2N_0, and partition type 918 with size N_0xN_0.
[0080] Motion predictive coding is repeatedly performed in one 2N_0x2N_0 partition, two 2N_0xN_0 partitions, two N_0x2N_0 partitions, and four N_0xN_0 partitions for each partition type, respectively. Internal mode and coding with motion prediction in external mode can be performed in partitions 2N_0x2N_0, N_0x2N_0, 2N_0xN_0 and N_0xN_0. Encoding with prediction in skip mode is only performed on partitions 2N_0x2N_0.
[0081] If the coding error is the smallest in partition type 918, size N_0xN_0, the depth is changed from "0" to "1" to divide partition type 918 in operation 920 and the coding is repeated many times in
57P35821PL00
EP 2 713 618 B1
30 coding units 922, 924, 926 and 928 having a depth of 2 and size N_0xN_0 to search for the smallest coding error.
[0082] Since the coding is repeatedly performed in coding units 922, 924, 926 and 928 with the same depth, coding of a coding unit having a depth of 1 using a coding unit from among coding units 922, 924, 926 and 928 will be described. Prediction unit 930 for motion prediction coding unit depth 1 and size 2N_1x2N_1 (= N_0xN_0), may contain partitions with partition type 932 with size 2N_1x2N_1, partition type 934 with size 2N_1xN_1, partition type 936 with size N_1x2N_1, and partition type 938 with the size N_1xN_1. The encoding is repeatedly performed in one 2N_1x2N_1 partition, two 2N_1xN_1 partitions, two N_1x2N_1 partitions, and four N_1xN_1 partitions, for each partition type and using traffic estimation.
[0083] If the coding error is the smallest in partition type 938 of size N_1xN_1, the current depth is increased from "1" to "2" in operation 940 and the coding is cyclically performed in coding units 942, 944, 946 and 948 having a depth of 2 and size N_2xN_2 to find the smallest encoding error.
[0084] If the maximum depth is "d", then partition information, depending on the depth, can be set for depth (d-1). That is, the prediction unit 950 for motion prediction in a coding unit of depth d-1 and size 2N_ (d-1) x2N_ (d-1), can contain partitions with partition type 952 with size 2N_ (d-1) x2N_ (d1 ), with partition type 954 2N_ (d-1) xN_ (d-1), o
57P35821PL00
EP 2 713 618 B1
-31 partition type 956 N_ (d-1) x2N (d-1) and partition type 958 N_ (d-1) xN_ (d-1).
[0085] The encoding is repeatedly carried out in one partition size 2N_ (d-1) x2N_ (d-1), two partitions size 2N_ (d-1) xN_ (d-1), two partitions size N_ (d- 1) x2N_ (d-1), and four partitions size N_ (D-1) xN_ (d-1), for each type of partition and using motion estimation. Because the maximum depth is "d", coding unit 952 with depth (d-1) is no longer divided.
[0086] The video coding apparatus 100 according to an embodiment of the present invention compares errors according to depth with each other and selects the depth corresponding to the smallest coding error to determine the coding depth for partition type 912. For example, for a coding unit with a depth of 0, partition types 912, 914, 916 and 918 are individually coded by performing motion estimation and the coding unit with the least coding error is selected from partition types 912, 914, 916 and 918. Similarly, the unit prediction, having the smallest coding error, can be set for each depth 0, 1, ..., d-1. For depth d, the coding error can be determined by performing motion estimation based on a prediction unit of 960, i.e. a coding unit of size 2N_dx2N_d. As described above, the smallest coding errors corresponding to depths 0, 1, ..., d-1 are compared to each other and the depth having the smallest coding error is selected as the coding depth from among the smallest coding errors. The coding depth and prediction unit corresponding to the coding depth can be coded and transmitted as information relating to the coding mode. Also because
57P35821PL00
EP 2 713 618 B1
- the coding unit should be divided from depth 0 to coding depth, the only partition information relating to the coding depth is set to "0" and the partition information related to other depths is set to "1".
[0087] The coding information extractor 220 of the video decoding device 200 may extract and use the coding depth and prediction unit information of the coding unit 900 to decode partition 912. The video decoding apparatus 200 may determine the depth, the split information being 0 as the coding depth, using the split information according to the depth and using the coding mode information for the appropriate depth for decoding.
[0088] Figs. 10A and 10B are diagrams for describing the relationship between coding units 1010, prediction units 1060, and transformation units 1070, according to an embodiment of the present invention.
[0089] The coding units 1010 are coding units corresponding to the coding depths determined by the video coding apparatus 100 for the maximum coding unit. The prediction units 1060 are the partitions of the prediction units of each of the 1010 coding units, and the 1070 transformation units are the transformation units of each of the 1010 coding units.
[0090] When the depth of the maximum coding unit is 0 in coding units 1010, the depths of coding units 1012 and 1054 are equal to 1, the depths of coding units 1014, 1016, 1018, 1028 and 1052 are equal to 2, the depths of coding units 1020, 1022, 1024, 1026, 1030, 1032 and 1048 are equal to 3, and the depths of coding units 1040, 1042, 1044 and 1046 are equal to 4.
57P35821PL00
EP 2 713 618 B1
33 size types [0091] In prediction units 1060, certain prediction units 1014, 1016, 1022, 1032, 1048, 1050, 1052 and 1054 are obtained by splitting coding units in coding units 1010. In other words, prediction units 1014, 1022, 1050 and 1054 are 2NxN partition types, the 1016, 1048 and 1052 prediction units are Nx2N partitions, and the 1032 prediction unit is the NxN partition type. Prediction units and partitions of the 1010 coding units are less than or equal to each coding unit.
[0092] Conversion or inverse transformation is performed on image data of coding units 1052 and 1054 in processing units 1070 in a data unit that is smaller than coding units 1052 and 1054. Also, transformation units 1014, 1016, 1022, 1032, 1048,
1050 and 1052 in 1070 transformation units are different from units in 1060 coding units in size and shape. In other words, the video encoding apparatus 100 and the video decoding apparatus 200 can perform in-picture prediction, motion estimation, motion compensation, transformation and inverse transformation individually on data units in the same coding unit.
[0093] Fig. 11 is a table showing coding information relating to each coding unit according to an embodiment of the present invention.
[0094] The coding information encoder 140 of the video encoding device 100 shown in Fig. 1 can encode the coding information relating to each coding unit, and the coding information extractor 220 of the video decoding device 200 shown in Fig. 2 can extract coding information related to each coding unit.
57P35821PL00
EP 2 713 618 B1
[0095] The coding information may include partition information relating to each coding unit, information related to the partition type of each coding unit (hereinafter referred to as "partition type information"), prediction mode and transformation unit size. The coding information shown in Fig. 11 are only an example of coding information that the video encoding device 100 and the video decoding device 200 can determine, and thus the innovative concept is not limited to them.
[0096] Partition information may indicate the coding depth of the respective coding unit. that is, since the coding depth is a depth at which it cannot be divided according to partition information, partition type information, prediction mode, and transformation unit size can be defined relative to the coding depth. When the current depth is divided again according to the partition information, the coding can be performed individually on four coding units corresponding to a greater depth. [0097] In the split type information, the split type of the transformation unit of the coding unit having the given coding depth may be represented as one of 2Nx2N, 2NxN, Nx2N and NxN. In the prediction mode, the motion estimation mode can be represented as one of the elements of the group consisting of internal mode, external mode and skip mode. Internal mode can only be specified if the partition type includes 2Nx2N and NxN. The bypass mode can only be specified if the partition type includes 2Nx2N. The size of the transformation unit can be set such that two sizes are set in internal mode and two sizes are set in external mode.
57P35821PL00
EP 2 713 618 B1
[0098] Each minimum coding unit contained in a coding unit may include coding information relating to each coding unit corresponding to its coding depth. Thus, it is possible to determine if the current coding unit is one of the coding units belonging to the same coding depth by checking the coding information in adjacent minimum coding units. Also, the coding units corresponding to the current coding depth can be checked using the coding information of the minimum coding unit. Accordingly, the coding depth distribution can be determined in the maximum coding unit.
[0099] The in-picture prediction that is performed by the in-picture prediction system 410 of the video encoding device 100 of Fig. 1 and the in-picture prediction system 1900 with the prediction of the video decoding device of Fig. 2 according to the embodiments of the present invention will now be described in detail. In the following descriptions, it should be noted that the term "coding unit" refers to an image coding process and is referred to as "decoding unit" in relation to an image decoding process. That is, in the following descriptions, the terms "coding unit" and "decoding unit" indicate the same thing and differ only in whether the coding or decoding process is performed. For consistency of definitions, except in a special case, the coding unit and the decoding unit may be referred to as a coding unit in both encoding and decoding processes.
[00100] Fig. 12 is a block diagram of an intra-image prediction device 1200 according to an embodiment of the present invention. Referring to Fig. 12, the intra-image prediction device 1200 includes
57P35821PL00
EP 2 713 618 B1
The prediction system 1210 and the end processor 1220. The prediction system 1210 subjects the current coding unit within the image using the intra-image prediction modes determined according to the size of the current coding unit and performs the first prediction coding unit. End processor 1220 performs post-processing using adjacent pixels among the pixels that form the first prediction coding unit to change the pixel value of the first prediction coding unit, and then generates a second prediction coding unit that has undergone post-processing.
[00101] Fig. 13 is a table showing the number of in-picture prediction modes according to the size of the coding unit, according to an embodiment of the present invention. According to an embodiment of the present invention, the number of in-picture prediction modes can be set according to the size of the coding unit (decoding units for the decoding process). Referring to Fig. 13, if the size of the coding unit to be subjected to intra-picture prediction is equal, for example, NxN, then the number of intra-picture prediction modes that should actually be performed in coding units having sizes 2x2, 4x4, 8x8, 16x16, 32x32, 64x64 and 128x128 may be equal to 5, 9, 9, 17, 33, 5 and 5, respectively (Example 2). The number of intra-picture prediction modes that should actually be performed is determined according to the size of the coding unit, since the overhead of coding the prediction mode information varies depending on the size of the coding unit. In other words, although the small-size coding unit occupies a small area throughout the image, additional overhead for overhead
57P35821PL00
EP 2 713 618 B1
Information, e.g., prediction mode, regarding the small coding unit may be large. Thus, when a small-size coding unit is encoded using too many prediction modes, the number of bits can increase, degrading the compression efficiency. A coding unit of large size, e.g. a coding unit of size 64x64 or more, is chosen with high probability as a coding unit for a flat image region. Compression efficiency can also be compromised when the large-size coding unit chosen to encode such a flat region is coded using too many prediction modes.
[00102] Thus, according to an embodiment of the present invention, the coding unit size can be roughly classified into at least three sizes: N1xN1 (2 <N1 <8, N1 is an integer), N2xN2 (16 <N2 <32, N2 is an integer ) and N3xN3 (64 <N3, N3 is an integer). If the number of intra-picture prediction modes that should be performed in each coding unit having the size N1xN1 is equal to A1 (A1 is a positive integer), the number of intra-picture prediction modes to be performed in each coding unit of size N2xN2 is equal to A2 (A2 is positive integer) and number of intra-picture prediction modes, which should be performed in each coding unit of size N3xN3 is equal to A3 (A3 is a positive integer), then the number of intra-picture prediction modes to be performed according to the size of the coding unit can be set so that "A3 <A1 <A2". That is, if the current image is divided into a small size coding unit, a medium size coding unit and a large coding unit
57P35821PL00
EP 2 713 618 B1
38 sizes, then the number of prediction modes to be performed in the medium size coding unit may be greater than the number of prediction modes to be performed in the small size coding unit and the large size coding unit. However, the present invention is not limited to this and a large number of prediction modes can also be adopted for performing in small- and medium-sized coding units. The numbers of prediction modes according to the size of each coding unit are shown in Fig. 13 only as an example and may therefore be different.
[00103] Figures 14A to 14C are drawings for explaining intra-picture prediction modes that can be performed in a coding unit of a predetermined size according to an embodiment of the present invention. In particular, Fig. 14A is a table showing the intra-picture prediction modes performed in a coding unit of size according to an embodiment of the present invention. Referring to Fig. 13 and 14, for example, if a 4x4 coding unit is subject to intra-picture prediction, vertical mode (mode 0), horizontal mode (mode 1), direct current mode (DC) (mode 2), left diagonal mode may be performed down (mode 3), diagonal right down (mode 4), vertical mode with a right tilt (mode 5), horizontal mode with a tilt down (mode 6), vertical mode with a tilt left (mode 7) , or horizontal mode with upward bias (mode 8).
[00104] Fig. 14B illustrates the directions of the intra-picture prediction modes shown in Fig. 14A, according to an embodiment of the present invention. In Fig. 14B, the values assigned to the arrows indicate the values of the modes when the prediction is made in the indicated directions that can be predetermined
57P35821PL00
EP 2 713 618 B1
39 by arrows, respectively. Here, mode 2 is a DC prediction mode that has no direction and is therefore not shown in Fig. 14B.
[00105] Fig. 1C illustrates the intra-picture prediction modes that can be performed in the coding unit shown in Fig. 14A, according to an embodiment of the present invention. Referring to Fig. 14C, the prediction coding unit is generated using adjacent pixels A to M of the current coding unit according to the available in-picture prediction mode, determined according to the size of the current coding unit. For example, a coding method will be described predicting the current coding unit of 4x4 size according to the vertical mode (mode 0) of Fig. 14A. First, the pixel values A to D adjacent to the top of the 4x4 coding unit are predicted as the 4x4 coding unit values. In particular, pixel A values are predicted as four pixel values in the first column of the 4x4 coding unit, pixel B values are predicted as four pixel values in the second column of the 4x4 coding unit, pixel C values are predicted as four pixel values in the third column 4x4 coding units, and pixel D values are predicted as four pixel values in the fourth column of the current 4x4 coding unit. Then, the error values are calculated and coded between the actual pixel values contained in the 4x4 prediction coded unit, predicted using pixels from A to D, and the original 4x4 coding unit.
57P35821PL00
EP 2 713 618 B1
4033 prediction modes [00106] Fig. 15 is drawings to explain intra-image prediction modes that can be performed in a coding unit of a predetermined size according to other embodiments of the present invention. Referring to Figs. 13 and 15, for example, if a 2x2 coding unit is subject to intra-picture prediction, it can perform all five modes, e.g. vertical mode, horizontal mode, DC mode, plane mode and downward diagonal mode.
[00107] As shown in Fig. 13, if a 32x32 coding unit has intra-picture, then the directions of 33 in-picture prediction modes must be established. According to an embodiment of the present invention, the prediction direction for selection of adjacent pixels used as reference pixels, based on the pixels contained in the coding unit, is determined by using the "dx" parameter and the "dy" parameter to determine in-picture prediction modes having different orientations , in addition to the intra-image prediction modes described above with reference to Figs. 14 and 15. For example, when each of the 33 prediction modes is specified as N mode (N is an integer from 0 to 32), mode 0, mode 1, mode 2, and mode 3 are set as vertical mode, horizontal mode, DC mode, and the plane mode, respectively, and each of the modes from 4 to 31 can be set as a prediction mode having a tan orientation<sup>-1</sup>(dy / dx) using the (dx, dy) parameter, expressed in one way from: (1, -1), (1,1), (1,2), (2, 1), (1, -2 ), (2,1), (1, -2), (2, -1), (2, -11), (5, -7), (10, -7), (11,3), ( 4.3), (1.11), (1, -1), (12, -3), (1, -11), (1, -7), (3, -10), (5, - 6), (7, -6), (7, -4), (11.1), (6.1), (8.3), (5.3), (5.7), (2, 7), (5, -7) and (4, -3) as shown in Table 1.
57P35821PL00
EP 2 713 618 B1
41 Table 1
<td>Mode No.</td><td>dx</td><td>dy</td><td>Mode No.</td><td>dx</td><td>dy</td>
<td>mode 4</td><td> 1</td><td> -1</td><td>mode 18</td><td> 1</td><td> -11</td>
<td>mode 5</td><td> 1</td><td> 1</td><td>mode 19</td><td> 1</td><td> -7</td>
<td>mode 6</td><td> 1</td><td> 2</td><td>mode 20</td><td> 3</td><td> -10</td>
<td>mode 7</td><td> 2</td><td> 1</td><td>mode 21</td><td> 5</td><td> -6</td>
<td>mode 8</td><td> 1</td><td> -2</td><td>mode 22</td><td> 7</td><td> -6</td>
<td>mode 9</td><td> 2</td><td> -1</td><td>mode 23</td><td> 7</td><td> -4</td>
<td>mode 10</td><td> 2</td><td> -11</td><td>mode 24</td><td> 11</td><td> 1</td>
<td>mode 11</td><td> 5</td><td> -7</td><td>mode 25</td><td> 6</td><td> 1</td>
<td>mode 12</td><td> 10</td><td> -7</td><td>mode 26</td><td> 8</td><td> 3</td>
<td>mode 13</td><td> 11</td><td> 3</td><td>mode 27</td><td> 5</td><td> 3</td>
<td>mode 14</td><td> 4</td><td> 3</td><td>mode 28</td><td> 5</td><td> 7</td>
<td>mode 15</td><td> 1</td><td> 11</td><td>mode 29</td><td> 2</td><td> 7</td>
<td>mode 16</td><td> 1</td><td> -1</td><td>mode 30</td><td> 5</td><td> -7</td>
<td>mode 17</td><td> 12</td><td> -3</td><td>mode 31</td><td> 4</td><td> -3</td>
<td colspan="6">Mode 0, mode 1, mode 2, mode 3 and mode 32 mean vertical mode, horizontal mode, DC mode, plane mode and two-line mode, respectively</td>
[00108] Mode 32 may be set as a two-line mode that uses two-line interpolation, as will be described later, with reference to Fig. 17.
[00109] Fig. 16 is a reference diagram for explaining inter-picture prediction modes having different orientations according to embodiments of the present invention. As previously described with reference to Table 1, each of the intra-image prediction modes according to embodiments of the present invention may have a tan orientation<sup>-1</sup>(dy / dx) by using multiple parameters (dx, dy).
[00110]
Referring to Fig. 16, adjacent pixels A and B on line 160 that runs from the current pixel P in the current coding unit that is predicted for tan<sup>-1</sup> (dy / dx), determined by the value of the parameter (dx, dy) according to the mode shown in the Table
57P35821PL00
EP 2 713 618 B1
-421, can be used as the prediction elements for the current pixel P. In this case, the adjacent pixels A and B can be pixels that have been previously encoded and reproduced and belong to the previous coding units, located above and to the left of the current coding unit. Also, when line 160 does not pass along adjacent pixels in places each of which has an integer value, but passes between adjacent pixels, adjacent pixels closer to line 160 can be used as predictions for the current pixel P. If there are two pixels that touch the line 160, e.g. the adjacent pixel A, located above the current pixel P and the adjacent pixel B, located to the left of the current pixel P, the average values of the adjacent pixels A and B can be used as predictions for the current pixel P "Dy" is a positive value, the adjacent pixel A can be used, and if the product of "dx and" dy "is a negative value, the adjacent pixel B can be used
[00111] Intra-picture prediction modes having different orientations, shown in Table 1, can be predetermined by the coding page and the decoding page and only the intra-picture prediction mode index of each coding unit can be transmitted.
[00112] Fig. 17 is a reference diagram for explaining a two-line mode according to an embodiment of the present invention. Referring to Fig. 17, in bilinear mode, the geometric mean is calculated by taking into account the value of the current pixel P 170 in the current coding unit being predicted, the pixel values on the top, bottom, left and right edges of the current coding unit and distances
57P35821PL00
EP 2 713 618 B1
43 in-line two-line mode, between the current P 170 pixel and the top, bottom, left and right edges of the current coding unit, and is then used as the prediction element for the current P 170 pixel. For example, in two-line mode, the geometric mean calculated using the virtual pixel A 171, virtual pixel B 172, pixel D 176 and pixel E 177, located from the top, bottom, left and right of the current pixel P 170 and the distance between the current P 170 pixel and the top, bottom, left and right edges of the current coding unit, is used as the prediction element for the current P 170 pixel. Because it is one of the prediction modes, adjacent pixels that have been previously encoded and rendered and belong to previous coding units are used as reference pixels for the prediction. Thus, the values in the current coding unit are not used, for example pixel A 171 and pixel B 172, but virtual values calculated using adjacent pixels located on the top and left edges of the current coding unit are used as pixel A 171 and pixel B 172 .
[00113] In particular, the first value of the virtual pixel C 173 located at the lower rightmost point of the current coding unit is calculated by calculating the average of the value of the adjacent pixel (upper right pixel) 174 at the upper rightmost point of the current coding unit and neighboring pixel (bottom pixel, leftmost) 175, located at the bottom, leftmost point of the current coding unit, as expressed in the following equation:
C = 0.5 (Left Bottom Pixel + Right Top Pixel) "(1)
57P35821PL00
EP 2 713 618 B1
[00114] Then, the value of the virtual pixel A 171, located on the left edge of the current coding unit, when the current pixel P 170 is extended downwards by taking into account the distance W1 between the current pixel P 170 and the left edge of the current coding unit and the distance W2 between the current with pixel P 170 and the right edge of the current coding unit, is calculated using the following equation:
A = (C * W1 + Left Bottom Pixel * W2) / (W1 + W2) ... (2) [00115] Similarly, the value of the virtual pixel B 172, located on the right edge of the current coding unit, when the current pixel P 170 is extended in right by considering the distance h1 between the current pixel P 170 and the top edge of the current coding unit and the distance h2 between the current pixel P 170 and the bottom edge of the current coding unit, is calculated using the following equation:
B = (C * h1 + Right Top Pixel * h2) / (h1 + h2). (3) When the values of the virtual pixels A and B are determined using equations from (1) to (3), the average of the values of pixel A 171, pixel B 172, pixel D 176 and pixel E 177 can be used as a prediction element for the current pixel P 170. As described above, in a two-line mode, a coding unit with a prediction of the current coding unit can be obtained by performing a two-line prediction on all pixels contained in the current coding unit.
[00117] According to an embodiment of the present invention, the prediction coding is performed according to one of
57P35821PL00
EP 2 713 618 B1
-45 different intra-image prediction modes, determined according to the size of the coding unit, thus enabling effective video compression based on image characteristics.
[00118] As described above, the prediction coding unit, manufactured using the intra-image prediction mode, determined according to the size of the current coding unit by the prediction system 1210 of the intra-image prediction device of Fig. 12, is oriented according to the intra-image prediction mode. Orientation in a prediction coding unit can lead to improved prediction efficiency when the pixels of the current coding unit to be predicted have predetermined orientations, but can lead to degradation of prediction efficiency when pixels do not have predetermined orientations. Thus, end processor 1220 can improve prediction efficiency by generating new prediction coding units by changing the value of pixels in the prediction coding unit by using pixels in the prediction coding unit and at least one adjacent pixel as post processing for the prediction coding unit produced by intra-image prediction.
[00119] The method of processing the final coding unit with prediction by the end processor 1220 of Fig. 12 will now be described.
[00120] End processor 1220 generates a second prediction coding unit by changing the pixel values forming the first prediction coding unit produced by the prediction system 1210, by performing operations using the pixels of the first prediction coding unit and at least one neighbor
57P35821PL00
EP 2 713 618 B1
-46piksela. Here, the prediction system 1210 produces a first prediction coding unit using an intra-image prediction mode, determined according to the size of the current coding unit, as described above.
[00121] Fig. 18 is a reference diagram for explaining the post-processing of the first prediction coding unit according to an embodiment of the present invention. In Fig. 18, reference numerals 1810 to 1860 illustrate the process of changing the value of pixels in the first coding unit predicted by the end processor 1220 in chronological order.
[00122] Referring to Fig. 18, end processor 1220 changes the pixel values in the first prediction coding unit 1810 by calculating the weighted average of the pixel value in the first prediction coding unit 1810 to be changed and the neighboring pixels with the given pixel. For example, referring to Fig. 18, if the value of pixel 1821 of the first coding unit 1810 with the prediction to be changed is equal to f [1] [1], the value of pixel 1822 located above pixel 1821 is f [0] [1], value of pixel 1823, to the left of pixel 1821 is f [1] [0], and the result of changing the value of f [1] [1] of pixel 1821 is f '[1] [1], then f' [1] [1] can be calculated using using the following equation:
f [ι] ι] = <sup>f [0</sup>]<sup>[1</sup>]+ <sup>f [l] [o]</sup>+ <sup>2</sup> * <sup>f [l</sup>[00123] As shown in Fig. 18, end processor 1220 changes the pixel values contained in the first coding unit 1810 with prediction by calculation
57P35821PL00
EP 2 713 618 B1
Weighted average of the value of each pixel of the first prediction coding unit and the pixels located above and to the left of the pixel in the direction from the top point, the leftmost first coding unit to the bottom point, the rightmost first coding unit with prediction. However, such post-processing operation according to the present invention is not limited to the operations described and can be sequentially performed for the pixels of the first coding unit with a prediction from the top point, the far right of the first coding unit with the bottom point, the extreme left of the first coding unit with prediction or from the bottom point, the rightmost first coding unit with prediction to the top rightmost left first coding unit with prediction. For example, if the end processor 1220 changes the pixel values of the first coding unit with prediction in the direction from the top, far right to bottom, far left, opposite to what is shown in Fig. 18, then the pixel values of the first prediction coding unit are changed by calculating the weighted average of the values of each pixel of the first prediction coding unit and the pixels located to the right and right of the first prediction coding unit.
[00124] Figs. 19 and 20 are reference diagrams for explaining the operation of the end processor 1220 of Fig. 12 according to embodiments of the present invention. In Fig. 19, reference numeral 1910 means the first pixel of the first coding unit 1900 with prediction to be changed, and reference numerals 1911 to 1918 indicate adjacent pixels with the first pixel 1910.
57P35821PL00
[00125] In the current embodiment (first embodiment) of Fig. 19, adjacent pixels with the first pixel 1910 are not limited to those located above and to the left in the first prediction coding unit, in contrast to as shown in Fig. 18. Referring to Fig. 19, end processor 1220 may perform post-processing of the first pixel 1910 by using a predetermined number of contiguous pixels selected from contiguous pixels from 1911 to 1918. That is, referring to Fig. 20, the predetermined number of pixels is selected from adjacent pixels P1 to P8 relative to the first pixel c in the current coding unit and the value of the first pixel c is changed by performing a predetermined operation for the selected adjacent pixels and the first pixel c. For example, if the size of the first 1900 coding unit with prediction is mxn, the value of the first 1910 pixel to be changed and located in the i-th column and in the j-th row of the first prediction 1900 coding unit is equal to f [i] [ j], the values of n pixels selected from pixels from 1911 to 1918 adjacent to the first pixel 1910 to perform post-processing of the first pixel 1910 are from f1 to fn, respectively, then the end processor 1220 changes the value of the first 1910 pixel from f [i] [j] to f '[i] [j] using the following equation. Here, m is a positive integer, n is "2" or "3", and is an integer from 0 to m-1, and j is an integer from 0 to n-1.
<img file="PL2713618T3_D0001.tif" />
57P35821PL00
EP 2 713 618 B1
-49<a name="caption1"></a>/[]']= <sup>f 1</sup> + <sup>f 2</sup> + { <sup>3</sup> + <sup>f [Φ]</sup> (= 3). (2, [00126] End processor 1220 produces a second prediction coding unit by changing the value of all pixels contained in the first prediction coding unit 1900 using equation (2). Equation (2) uses three adjacent pixels, but the present invention is not limited to this and end processor 1220 may perform post-processing using four or more adjacent pixels.
[00127] According to a second embodiment of the present invention, end processor 1220 produces a second prediction coding unit by changing the value of each pixel in the first prediction coding unit 1900 using a weighted harmonic average of the pixel value of the first prediction coding unit 1900 to be changed and neighboring pixels with a given pixel.
[00128] For example, end processor 1220 changes the pixel value in the i-th column and i-th row of the first coding unit 1900 with a prediction from f [i] [j] to f '[i] [j], using adjacent pixels located above and to the left of the pixel as shown in the following equation:
α + β + γ α β γ. (3) f [-1]]<sup>+</sup>f [[-1]<sup>+</sup>f [φ] wherein α, β and γ are positive integers and, for example, α = 2, β = 2 and γ = 1.
[00129] According to a third embodiment of the present invention, end processor 1220 generates a second coding unit with prediction by changing the value of each pixel
<img file="PL2713618T3_D0002.tif" />
57P35821PL00
EP 2 713 618 B1
-50 the first coding unit 1900 with prediction, using the geometric weighted average of the pixel value of the first coding unit 1900 with the prediction to be changed and the neighboring pixels with the given pixel.
[00130] For example, end processor 1220 changes the pixel value in the i-th column and i-th row of the first coding unit 1900 with a prediction from f [i] [j] to f '[i] [j] using adjacent pixels located above and to the left of the pixel as shown in the following equation:
./l/l/]=<sup>(</sup>[»<sup>-1</sup>1/<sup>]</sup> *(1/1/<sup>-1]</sup> * (ML /<sup>]</sup>Ά +<sup>λ)</sup> .. (4) where α, β and γ are positive integers and, for example, α = 1, β = 1 and γ = 2. In equations (2) to (4) a relatively large weight is assigned to the value f [i] [j] pixel to be changed.
[00131] As described above, in the first to third embodiments of the present invention, the end processor 1220 may perform post-processing using not only the adjacent pixels located above and to the left of the pixel to be changed, but also a predetermined number of adjacent pixels selected from neighboring pixels from 1911 to 1918, as shown in Fig. 19.
[00132] According to a fourth embodiment of the present invention, the end processor 1220 produces a second prediction coding unit by changing the value of each pixel in the first prediction coding unit, using the average of the pixel value in the first coding unit
57P35821PL00
EP 2 713 618 B1
-51 with the prediction to be changed and one pixel selected from neighboring pixels with the given pixel.
[00133] For example, the end processor 1220 changes the pixel value in the i-th column and i-th row of the first coding unit 1900 with the prediction zf [i] [j] to f '[i] [j] using adjacent pixels located above given pixel as shown in the following equation:
<a name="caption2"></a>/[/ <sup>f - 1 f]</sup> (5) [00134] Similarly, according to the fifth embodiment of the present invention, the end processor 1220 produces a second prediction coding unit by changing the value of each pixel in the first prediction coding unit, using the mean of the pixel value in the first prediction coding unit that has be changed and neighboring pixels located to the left of the pixel.
[00135] In other words, the end processor 1220 changes the pixel value in the i-th column and j-th row of the first coding unit 1900 with the prediction zf [i] [j] to f '[i] [j] as shown in the equation below :
<img file="PL2713618T3_D0003.tif" />
(6) [00136] According to a sixth embodiment of the present invention, end processor 1220 generates a second prediction coding unit by changing the value of each pixel in the first prediction coding unit, using the median between the pixel values of the first prediction coding unit to be changed , a
57P35821PL00
EP 2 713 618 B1 adjacent pixels with a given pixel. Referring to Fig. 19, for example, it was assumed that the value of f [i] [j] of the first pixel 1910 in the i-th column and j-th row of the first coding unit 1900 with prediction, the value f [i] [j-1 ] of the second pixel 1912, and the value of f [i-1] [j] of the third pixel 1911 satisfy the relationship f [i] [j-1]> f [i1] [j]> f [i] [j]. In this case, processor 1220 changes the value of f [i] [j] of the first pixel 1910 to the median f [i-1] [j] among the pixels from the first 1910 to the third 1912.
In the seventh to ninth embodiments of the present invention, the end processor 1220 generates a second prediction coding unit using the previous coding units adjacent to the current coding unit that have been previously encoded and rendered, rather than adjacent pixels with a pixel that has be changed.
[00138] Referring back to Fig. 19, in the seventh embodiment of the present invention, end processor 1220 changes the value of the first pixel 1910 to f '[i] [j] by calculating the average of the value of the first pixel 1910 in the i-th column and j-th line of the first coding unit 1900 with prediction ą of the 1921 pixel value, which is located in the same column as the first 1910 pixel and is located in the coding unit adjacent to the top of the current coding unit, as shown in the following equation:
f '[s [] = <sup>f [Φ]</sup>+<sup>f [-</sup> '<sup>] ]</sup>
- (7) with f [-1] [j] representing the 1921 pixel value.
57P35821PL00
EP 2 713 618 B1
[00139] Similarly, in the eighth embodiment of the present invention, the end processor 1220 changes the value of the first pixel 1910 to f '[i] [j] by calculating the average of the value of the first pixel 1910 in the ith column and the jth row of the first coding unit 1900 with prediction and 1922 pixel values, which is located on the same line as the first 1910 pixel and is located in the coding unit adjacent to the left of the current coding unit, as shown in the following equation:
<img file="PL2713618T3_D0004.tif" />
(8) with f [i] [- 1] representing the 1922 pixel value.
[00140] In the ninth embodiment of the present invention, the end processor 1220 changes the value of the first pixel 1910 to f '[i] [j] by calculating the weighted average of the value of the first pixel 1910 in the i-th column and the i-th row of the first 1900 predicted coding unit, 1921 pixel, located in the same column as the first 1910 pixel and located in the coding unit adjacent from the top of the current coding unit and 1922 pixel, located on the same line as the first pixel 1910 and located on the coding unit adjacent to the left of the current coding unit, as shown in the following equation:
f [Φ] + f [- H /] + f fr l · -1] (9)
<img file="PL2713618T3_D0005.tif" />
57P35821PL00
EP 2 713 618 B1
[00141] In a tenth embodiment of the present invention, the end processor 1220 changes the value of the first pixel 1910 of the first coding unit 1900 with the prediction to be changed, from f [i] [j] to f '[i] [j], using the following equations:
f <sup>'</sup>[i] [j] = min (f [i] [j] + i, 255)
... (10)
Ζ [Φ '] =<sup>min</sup>C / '<sup>[</sup>/<sup>]</sup>Ld + /<sup>,255</sup>). (ii) f<sup>'</sup>[i] [j] = max (f [i] [j] -i, 0). (12) f <sup>'</sup>[] [j] = max (f [] [j] - j, 0). (i3) [00i42] In equation (10), the values of the first coding unit 1900 with prediction are changed in order to gradually increase from top to bottom, in column units of the first 1900 coding unit with prediction. In equation (11), the values of the first prediction coding unit 1900 are changed to gradually increase to the right in the row units of the first prediction coding unit 1900. In equation (12), the values of the first prediction coding unit 1900 are changed to gradually decrease from top to bottom, in column units of the first prediction coding unit 1900. In equation (13), the values of the first prediction coding unit 1900 are changed to gradually decrease to the right in the row units of the first prediction coding unit 1900.
In the eleventh embodiment of the present invention, if the value of the first pixel 1910 which is located in the i-th column and i-th row of the first coding unit 1900 with prediction and is to be changed,
57P35821PL00
EP 2 713 618 B1
-55 is equal to f [i] [j], the value of the pixel located in the upper, leftmost point of the first coding unit 1900 with prediction is f [0] [0], the value of the pixel located in this row, every first pixel 1910 and in the leftmost point the first coding unit 1900 with prediction is f [0] [j], the pixel value located in the i-th column as the first pixel 1910 and at the highest point of the first coding unit with prediction is f [i] [0], and G [ i] [j] = f [i] [0] + f [0] [j] -f [0] [0] then the end processor 1220 changes the value of the first pixel 1910 to f '[i] [j] as shown in the following equation:
<img file="PL2713618T3_D0006.tif" />
[00144] Equation (14) is based on a wave equation in which the value of each pixel in the first coding unit 1900 with prediction is changed by calculating the value of G [i] [j] by determining the value of the pixels in the top row and the pixels in the leftmost column the first 1900 coding unit with prediction as limit values to smooth the value of each pixel in the first 1900 coding unit with prediction, and then calculating the average of the G [i] [j] and f [i] [j] values.
[00145] The costs of the bit streams containing the coding results of the second prediction coding units produced using the different operating modes according to the first to eleventh embodiments above, respectively, are compared with each other. Then, the operating mode with the minimum cost is added to the bit stream header from among the various operating modes. When draft mode is added to the stream
57P35821PL00
EP 2 713 618 B1
56 bits, it is possible to represent different operating modes, distinguished from each other using variable length coding, coding subunits and coding subunits, to which a small number of bits is assigned to the operating mode, which is most often used, based on the distribution of the operating mode determined after the coding is completed a predetermined number of coding units. For example, if the operating mode corresponding to the first embodiment of the present invention is an optimal operation, leading to the minimum cost of most coding units, the minimum number of bits is assigned to an index indicating this operating mode so that this operating mode can be distinguished from other operating modes.
[00146] When the coding unit is divided into prediction, the second prediction coding unit can be generated by applying different operating modes to the coding subunits, respectively, or by applying the same operating mode to the coding subunits belonging to the same coding unit, to simplify calculations and reduce overhead.
[00147] A method for optimizing speed - distortion can be used as a cost to determine the optimal operating mode. Since the video coding method of the embodiment of the present invention is performed in an coding unit with intra-picture prediction, used as reference data for another coding unit, the cost can be calculated by allocating a lot of weight to distortion, compared to the method of speed optimization - distortion. That is, in the way of speed optimization - distortion, the cost is calculated based on the distortion, which is the difference
57P35821PL00
EP 2 713 618 B1
57 between the encoded image and the original image and the generated bit rate as shown in the following equation:
Cost = Distortion + Bit Transmission Rate. (15) [00148] On the contrary, in the video coding method according to an embodiment of the present invention, the optimal post-processing mode is determined by allocating high weight to distortion, compared to the method of optimizing the speed - distortion, as shown in the following equation:
Cost = a * distortion + BitTransmission Rate (α is a real number, equal to or greater than '2'). (16) [00149] Fig. 21 is a flowchart illustrating a video coding method according to an embodiment of the present invention. Referring to Fig. 21, in operation 2110, a first coding unit is produced with a prediction of the current coding unit to be encoded. The first coding unit with prediction is a block with intra-picture prediction, produced by performing a general method of intra-picture prediction and one of different intra-picture prediction modes having different orientations, which is set according to the size of the coding unit.
[00150] In operation 2120, the second prediction coding unit is generated by changing the value of each pixel of the first prediction coding unit using the pixel from the first prediction coding unit and at least one adjacent pixel. How
57P35821PL00
EP 2 713 618 B1
As described above in the first to eleventh embodiments, regarding terminal processor operations 1220, a second prediction coding unit may be generated by changing the value of each pixel in the first prediction coding unit by performing one of different operating modes relative to the pixel of the first prediction coding unit which has to be changed and the neighboring pixels with it.
[00151] In operation 2130, the residual block, i.e. the difference between the current coding unit and the second prediction coding unit, is transformed, quantized and coded with respect to entropy to generate a bit stream. Information related to the operating mode used to produce the second prediction coding unit may be added to the predetermined region of the generated bit stream such that the decoding device can reproduce the second coding unit with prediction of the current coding unit.
[00152] Fig. 22 is a flowchart illustrating a video decoding method according to an embodiment of the present invention. Referring to Fig. 22, in operation 2210, information regarding the prediction mode for the current decoding unit to be decoded is extracted from the received bit stream. [00153] In operation 2220, the first decoding unit with a prediction of the current decoding unit is played back according to the extracted information.
[00154] In operation 2230, information relating to an operating mode in which: each pixel of the first prediction decoding unit and adjacent pixels of each pixel are extracted from the bit stream.
57P35821PL00
EP 2 713 618 B1
[00155] In operation 2240, the second prediction decoding unit is reproduced by changing the value of each pixel of the first prediction decoding unit by using each pixel of the first prediction decoding unit and its adjacent pixels in the pressure for information pertaining to the operating mode . [00156] In operation 2250, the residual block, i.e. the difference between the current decoding unit and the second prediction decoding unit, is extracted from the bit stream and is reconstructed.
[00157] In operation 2260, the residual block and the second prediction decoding unit are combined to decode the current decoding unit.
[00158] The present invention may also be implemented as a computer readable code in a computer readable recording medium. A computer readable recording medium is any data storage device that can store data that can then be read by a computer system. Examples of computer-readable recording media include read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tapes, flexible disks, and optical data storage devices. The computer-readable recording medium may also be distributed over a network connecting computer systems such that the computer-readable code is stored and executed in a distributed manner. [00159] Although the present invention has been shown and described in particular with reference to examples in a given field, the details and scope of the invention may be embodied, one of ordinary skill in the art will recognize that various changes in form and are made without departing from the invention as defined in the following claims. Exemplary embodiments of the invention should be analyzed in
57P35821PL00
EP 2 713 618 B1
- descriptive sense, and not for the purpose of limiting the scope of the invention. Thus, the scope of the invention is defined not by the detailed description of the invention but by the following claims and all differences within the scope are to be regarded as included in the present invention.
[00160] Attention should be drawn to all publications and documents that are registered simultaneously with or prior to this description with respect to this application and which are available for public control with this description, and the content of all such publications and documents are incorporated herein by reference.
[00161] All the features disclosed herein (including any appended claims, abbreviations and drawings) and / or all steps of any method or process so disclosed may be combined in any combination, except combinations in which at least some of said features and / or stages are mutually exclusive.
[00162] Any feature disclosed herein (including the appended claims, abbreviations and drawings) may be replaced by alternative features for the same, equivalent or similar purpose, unless the contrary is explicitly stated. Thus, unless expressly stated otherwise, each feature disclosed is only one example from a series of equivalent or similar features.
[00163] The invention is not limited to the details of the above embodiments. The invention extends to any novel or participating in any innovative combinations, features disclosed herein (including any appended claims, abbreviations and drawings), or any innovative, or
57P3582iPL00
EP 2 7i3 6i8 Bi
Participating in any innovative combinations, steps of any method or process so disclosed.
<td>[00i64]</td><td>following</td><td>aspects / examples</td><td>realizations</td>
<td>invention</td><td>they are also</td><td>provided:</td><td></td>
<td>Example</td><td>1. Way</td><td>video coding,</td><td>what a way</td>
<td>It includes:</td><td></td><td></td><td></td>
<td colspan="2">production</td><td>first unit</td><td>coding with</td>
a prediction of the current coding unit to be encoded;
producing a second prediction coding unit by changing the value of each pixel of the first prediction coding unit by using each pixel of the first prediction coding unit and at least one adjacent pixel of each pixel; and encoding the difference between the current coding unit and the second prediction coding unit.
Example 2. The method of Example 1, wherein the production of the first prediction coding unit includes:
dividing the current image into at least one coding unit based on the maximum coding unit and depth, which is hierarchical partition information regarding the maximum coding unit; and generating the first prediction coding unit by performing intra-image prediction in at least one coding unit.
57P35821PL00
EP 2 713 618 B1
Example 3. The method of Example 1, wherein the production of the second prediction coding unit involves changing the value of each pixel of the first prediction coding unit by calculating a weighted average of the value of at least one adjacent pixel of each pixel of the first predictive coding unit and each pixel of the first unit coding with prediction.
Example 4. The method of Example 3, wherein, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i ] [j], and the values of two adjacent pixels selected from adjacent pixels f [i] [j-1], f [i-1] [j], f [i + 1] [j], f [i] [j +1], f [i-1] [j + 1], f [i + 1] [j-1], f [i-1] [j-1] and f [i + 1] [j + 1 ] the first pixel is equal to f1 and f2, respectively, then producing the second prediction coding unit involves changing the value of the first pixel to f '[i] [j], using the following equation:
fw] =<sup>f 1</sup>+<sup>f 2</sup> +<sub>4</sub><sup>2</sup>*<sup>f [φ]</sup>, where m and n are positive integers.
Example 5. The method of Example 3, wherein, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i] [j], and the values of three adjacent pixels selected from adjacent pixels f [i] [j-1], f [i-1] [j], f [i + 1] [j],
57P35821PL00
EP 2 713 618 B1
-63f [i] [j + 1], f [i-1] [j + 1], f [i + 1] [j-1], f [i-1] [j-1] and f [i +1] [j + 1] of the first pixel are equal to f1, f2 and f3, respectively, then producing the second coding unit with prediction involves changing the value of the first pixel to f '[i] [j] using the following equation:
(Ll /] = (<sup>1</sup>+(<sup>2</sup>+ f<sub>f</sub><sup>3</sup>+(<sup>[</sup>φ<sup>1</sup>, where m and n are positive integers.
Example 6. The method of Example 1, wherein producing a second prediction coding unit involves changing the value of each pixel of the first prediction coding unit to a weighted harmonic average from the value of at least one adjacent pixel of each pixel of the first prediction coding unit and each pixel of the first coding unit with prediction.
Example 7. The method of Example 6, wherein, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i ] [j], the value of the second pixel located to the left of the first pixel is equal to f [i] [j-1], and the value of the third pixel located above the first pixel is equal to f [i-1] [j], then producing the second prediction coding unit involves changing the value of the first pixel to f '[i] [j] using the following equation:
57P35821PL00
EP 2 713 618 B1
<img file="PL2713618T3_D0007.tif" />
<img file="PL2713618T3_D0008.tif" />
-64α + β + γ <sup>α</sup> , <sup>β</sup> , γ <sup>, </sup>and -1]]<sup>+</sup>f [] ['-1]<sup>+</sup> where m and n are positive integers, and α, β and γ are positive integers. Example 8. The method of Example 1, wherein producing the second prediction coding unit involves changing the value of each pixel of the first prediction coding unit to a geometric weighted average from the value of at least one adjacent pixel of each pixel of the first prediction coding unit and each pixel of the first coding unit with prediction.
Example 9. The method of Example 8, wherein, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i ] [j], the value of the second pixel located to the left of the first pixel is equal to f [i] [j-1], and the value of the third pixel located above the first pixel is equal to f [i-1] [j], then producing the second prediction coding unit involves changing the value of the first pixel to f '[i] [j] using the following equation:
f '[i] ['] = (f [i-1] '((f [i (-1] * f [i]'])), where m and n are positive integers, and α, β and γ are positive integers.
57P35821PL00
EP 2 713 618 B1
Example 10. The method of Example 1, wherein producing a second prediction coding unit involves changing the value of each pixel of the first prediction coding unit to an average of at least one of the pixels located above and to the left of each pixel of the first predictive coding unit and each pixel the first coding unit with prediction.
Example 11. The method of Example 1, wherein producing the second prediction coding unit comprises changing the value of each pixel of the first prediction coding unit to a median between the value of each pixel of the first prediction coding unit and the values of adjacent pixels of each pixel of the first prediction coding unit.
Example 12. The method of Example 1, wherein, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i ] [j], and the pixel value, which is located on the same jth row as the first pixel among the pixels contained in the edge of the region of the coding unit adjacent from the top of the current coding unit, is equal to f [-1] [j], then producing the second coding unit with prediction involves changing the value of the first pixel to f '[i] [j], using the following equation:
<img file="PL2713618T3_D0009.tif" />
where m and n are positive integers.
57P35821PL00
EP 2 713 618 B1
-66 Example 13. The method of Example 1, wherein, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i ] [j], and the pixel value that is located in the same i-column as the first pixel among the pixels contained in the edge of the region of the coding unit adjacent to the left of the current coding unit, is equal to f [i] [- j], then producing the second coding unit with prediction involves changing the value of the first pixel to f '[i] [j] using the following equation:
<img file="PL2713618T3_D0010.tif" />
where m and n are positive integers.
Example 14. The method of Example 1, wherein, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i ] [j], the pixel value that is located on the same jth row as the first pixel among the pixels contained in the edge of the region of the coding unit adjacent to the top of the current coding unit is equal to f [-i] [j], and the value of the pixel that is located in the same i-column as the first pixel among the pixels contained in the edge of the coding unit region adjacent to the left of the current coding unit is equal to f [i] [- j], then the production of the second unit coding with prediction
57P35821PL00
EP 2 713 618 B1
-67 includes changing the value of the first pixel to f '[i] [j] using the following equation:
f - [] ·] = ^<sup>f</sup> · HL ' <sup>]</sup>+<sup>f</sup> H<sup>[</sup>~ <sup>l]</sup>+<sup>f [-</sup> Φ<sup>]</sup> where m and n are positive integers.
Example 15. The method of Example 1, wherein, if the size of the first prediction coding unit is mxn, and the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i] [j], then producing the second coding unit with prediction involves changing the value of the first pixel to f '[i] [j], using the following equation:
f<sup>'</sup>[] [·] = Min (f [] [·] +, 255), f<sup>'</sup>[] [·] = Min (f [] [·] + ·, 255), f<sup>'</sup>[] [·] = Max (f [] [·] -, 0), and f<sup>'</sup>[] [·] = Max (f [] [·] - ·, 0), with m and n being positive integers.
Example 16. The method of Example 1, in which, if the size of the first prediction coding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction coding unit is equal to f [i] [j], the pixel value located at the leftmost point of the first prediction coding unit is
57P35821PL00
EP 2 713 618 B1
-68 equal f [0] [0], the value of the pixel located in the same jth row as the first pixel and located in the far left column of the first coding unit with prediction, is equal to f [0] [j], the value of the pixel located in the same and -th column every first pixel and located in the top row of the first coding unit with prediction, is equal to f [i] [0], and G [i] [j] = f [i] [0] + f [0] [j ] -f [0] [0] then producing the second prediction coding unit involves changing the value of the first pixel to f '[i] [j] using the following equation:
<img file="PL2713618T3_D0011.tif" />
where m and n are positive integers.
Example 17 The method of Example 1, wherein the coding of the difference between the current coding unit and the second prediction coding unit comprises comparing the cost of bit streams containing the coding results of the second prediction coding units produced using different operating modes with each other, and adding information related to the operating mode used to produce the second coding unit with a minimum cost prediction to a predetermined bit stream region.
Example 18. The method of Example 16, wherein the coding of the difference between the current coding unit and the second prediction coding unit includes representing different operating modes, distinguished from each other by assigning a small number of bits to the operating mode that is most commonly used among the various modes
57P35821PL00
EP 2 713 618 B1
- based on the distribution of a fixed operating mode when predetermined numbers of coding units are coded.
Example 19. A video encoding device, the device comprising:
a prediction system for producing the first coding unit with a prediction of the current coding unit to be coded;
an end processor for generating the second prediction coding unit by changing the value of each pixel of the first prediction coding unit using each pixel of the first prediction coding unit and at least one adjacent pixel of each pixel; and an encoder for coding the difference between the current coding unit and the second prediction coding unit.
Example 20. A method of video decoding, the method comprising:
extracting from the received bit stream information relating to the prediction mode of the current decoding unit to be decoded;
playing the first decoding unit with a prediction of the current decoding unit, based on the extracted information related to the prediction mode;
extracting from the bit stream information related to the operating mode in which each pixel of the first unit is used
57P35821PL00
EP 2 713 618 B1
- 70 predicted decoding and adjacent pixels of each pixel;
playing the second prediction decoding unit by changing the value of each pixel of the first prediction decoding unit using each pixel of the first prediction decoding unit and adjacent pixels of each pixel, based on the extracted information related to the operating mode;
extracting from the bit stream a residual block, which is the difference between the current decoding unit and the second prediction decoding unit, and restoring the residual block; and decoding the current decoding unit by adding the residual block to the second prediction decoding unit.
Example 21. The method of Example 20, wherein reproducing the second prediction decoding unit comprises changing the value of each pixel of the first prediction decoding unit to a weighted average from the value of at least one adjacent pixel of each pixel of the first prediction decoding unit and each pixel of the first decoding unit from prediction.
Example 22. The method of Example 21, wherein if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction decoding unit is equal to f [i] [j], and the values of two adjacent pixels selected from adjacent pixels f [i] [j-1], f [i-1] [j], f [i + 1] [j], f [i] [j + 1], f [i-1] [j + 1], f [i + 1] [j-1], f [i-1] [j-1] and
57P3582iPL00
EP 2 7i3 6i8 Bi
<td></td><td colspan="3">-7i-</td>
<td>f [i + 1] [j + 1]</td><td>first</td><td>pixels are</td><td>equal to f1 and f2,</td>
<td>respectively,</td><td>then</td><td>reproduction</td><td>second unit</td>
<td>decoding from</td><td>prediction</td><td>includes change</td><td>value you first</td>
pixels on f '[i] [j] using the following equation:
Ζ [φ '] = <sup>f 1</sup>+' <sup>2</sup> +<sub>4</sub><sup>2</sup>*<sup>f [φ</sup>'<sup>]</sup>, where m and n are positive integers.
Example 23. The method of Example 21, wherein, if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction decoding unit is equal to f [i ] [j], and the values of three adjacent pixels selected from adjacent pixels f [i] [j-1], f [i-1] [j], f [i + 1] [j], f [i] [j +1], f [i-1] [j + 1], f [i + 1] [j-1], f [i-1] [j-1] and f [i + 1] [j + 1 ] the first pixel is equal to f1, f2 and f3, respectively, then reproducing the second prediction decoding unit includes changing the value of the first pixel to f '[i] [j], using the following equation:
<img file="PL2713618T3_D0012.tif" />
f 1 + f 2 + f 3 + f [φ] 4 with m and n being positive integers.
in which the first prediction for a weighted average of one neighbor
Example 24. The method of Example 20, reproducing a second decoding unit with comprises changing the value of each pixel of the decoding unit with harmonic prediction with a value of at least
57P35821PL00
EP 2 713 618 B1
72 pixels of each pixel of the first predictive decoding unit and each pixel of the first predictive decoding unit.
Example 25. The method of Example 24, in which, if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction decoding unit is equal to f [i ] [j], the value of the second pixel located to the left of the first pixel is equal to f [i] [j-1], and the value of the third pixel located above the first pixel is equal to f [i-1] [j], then, reproducing the second prediction decoding unit includes changing the value of the first pixel to f '[i] [j] using the following equation:
α + β + γ <sup>α</sup> , <sup>β</sup> , γ <sup>, </sup>i -1]] + f [i 1 / -1] + where m and n are positive integers, and α, β and γ are positive integers.
<img file="PL2713618T3_D0013.tif" />
<img file="PL2713618T3_D0014.tif" />
in which the first prediction for a weighted average
Example 26. The method of Example 20, reproducing a second decoding unit with comprises changing the value of each pixel of the decoding unit with geometric prediction from the value of at least one adjacent pixel of each pixel of the first prediction decoding unit and each pixel of the first prediction decoding unit.
57P35821PL00
EP 2 713 618 B1
-73 Example 27. The method of Example 26, wherein, if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction decoding unit is equal to f [i ] [j], the value of the second pixel located to the left of the first pixel is equal to f [i] [j-1], and the value of the third pixel located above the first pixel is equal to f [i-1] [j], then, reproducing the second prediction decoding unit includes changing the value of the first pixel to f '[i] [j] using the following equation:
fw] = (f [-1] '((f [Φ -'] '* f [Φ])), where m and n are positive integers, and α, β and γ are positive integers.
in which the first prediction
Example 28. The method of Example 20, reproducing a second decoding unit with comprises changing the value of each pixel of a prediction decoding unit to an average of at least one of the pixels located above and to the left of each pixel of the first predictive decoding unit and each pixel of the first decoding unit from prediction.
Example 29. The method of Example 20, wherein reproducing the second prediction decoding unit comprises changing the value of each pixel of the first decoding unit with a prediction to a median between the value of each pixel of the first decoding unit from
57P35821PL00
The prediction and adjacent pixel values of each pixel of the first prediction decoding unit.
Example 30. The method of Example 20, wherein, if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first predicted decoding unit is equal to f [i ] [j], and the pixel value, which is located in the same jth row as the first pixel among the pixels contained in the edge of the region of the decoding unit adjacent from the top with the current decoding unit, is equal to f [-1] [j], then reproducing the second decoding unit with prediction involves changing the value of the first pixel to f '[i] [j] using the following equation:
f / φ] = (<sup>[</sup>φ <sup>]</sup>+(<sup>[-</sup> 1<sup>1</sup> where m and n are positive integers.
Example 31. The method of Example 20, wherein, if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first predicted decoding unit is equal to f [i ] [j], and a pixel value that is located in the same i-column as the first pixel among the pixels contained in the edge of the region of the decoding unit adjacent to the left with the current decoding unit, is equal to f [i] [- j], then the second prediction decoding unit includes the change
57P35821PL00
EP 2 713 618 B1
-75 values of the first pixel on f '[i] [j], using the following equation:
<img file="PL2713618T3_D0015.tif" />
where m and n are positive integers.
Example 32. The method of Example 20, wherein, if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first prediction decoding unit is equal to f [i ] [j], the pixel value that is located on the same jth row as the first pixel among the pixels contained in the edge of the region of the decoding unit adjacent from the top with the current decoding unit, is equal to f [-i] [j], and the pixel value, which is located in the same column as the first pixel among the pixels contained in the edge of the region of the decoding unit adjacent to the left of the current decoding unit, is equal to f [i] [-j], then reproducing the second prediction decoding unit includes changing the value of the first pixel to f '[i] [j] using the following equation:
f - [] ·] = ^<sup>f</sup> · HL ' <sup>]</sup>+<sup>f</sup> H<sup>[</sup>~ <sup>l]</sup>+<sup>f [-</sup> Φ<sup>]</sup> where m and n are positive integers.
Example 33. The method of Example 20, the size of the first decoding unit, mxn, and the value of the first pixel, in which, if it is to be predicted
57P35821PL00
EP 2 713 618 B1
-76 changed and located in the i-th column and in the j-th row of the first prediction decoding unit, is equal to f [i] [j], then the reproduction of the second prediction decoding unit involves changing the value of the first pixel to f '[i] [ j] using the following equation:
f<sup>'</sup>[i] [j] = minf f [i] [j] + i, 255), f<sup>'</sup>[i] [j] = minf f [i] [j] + j, 255), f<sup>'</sup>[i] [j] = maxf f [i] [j] - i, 0), and f<sup>'</sup>[i] [j] = maxf f [i] [j] - j, 0), where m and n are positive integers. Example 34. The method of Example 20, wherein, if the size of the first prediction decoding unit is mxn, the value of the first pixel to be changed and located in the i-th column and in the j-th row of the first predicted decoding unit is equal to f [i ] [j], the value of the pixel located at the leftmost point of the first prediction decoding unit is equal to f [0] [0], the value of the pixel located in the same jth row as the first pixel and located in the leftmost column of the first prediction decoding unit is equal to f [0] [j], the value of the pixel located in the same i-th column as the first pixel and located in the highest the first row of the decoding unit with prediction, is equal to f [i] [0], and G [i] [j] = f [i] [0] + f [0] [j] -f [0] [0], then, reproducing the second prediction decoding unit includes changing the value of the first pixel to f '[i] [j] using the following equation:
57P35821PL00
EP 2 713 618 B1
<img file="PL2713618T3_D0016.tif" />
where m and n are positive integers.
Example 35. A video decoding device, the device comprising:
entropy decoder, used to extract from the bit stream information related to the prediction mode of the current decoding unit to be decoded and information related to the operating mode in which each pixel of the first decoding unit is used with prediction of the current decoding unit and adjacent pixels of each pixels of the first prediction decoding unit;
a prediction system for reproducing the first prediction decoding unit based on the extracted information relating to the prediction mode;
an end processor for reproducing the second prediction decoding unit by changing the value of each pixel of the first prediction decoding unit by using each pixel of the first prediction decoding unit and there are adjacent pixels of each pixel of the first prediction decoding unit based on the extracted information, relating to working mode;
inverse transformation and inverse quantization unit, used to restore the residual block bit stream, which is the difference between
57P35821PL00
EP 2 713 618 B1
A current decoding unit and a second prediction decoding unit; and an adder to decode the current decoding unit by adding the residual block to the second prediction decoding unit.
Example 36. A computer readable recording medium with a program code stored thereon for performing the method of any of Examples 1 to 18.
Example 37. A computer readable recording medium with a program code stored thereon for performing the method of any of Examples 20 to 34.
57P35821PL00
EP 2 713 618 B1
Contents145
98 members in 15 offices
Priority claims8
| Document | Office | Kind | Date |
|---|---|---|---|
| 7759208 | United States of America | P | |
| 20080085914 | Republic of Korea | A | |
| 09773750 | European Patent Office (EPO) | A | |
| 13196512 | European Patent Office (EPO) | A | |
| EP20090773750 | – | – | – |
| EP20130196512 | – | – | – |
| KR20080085914 | – | – | – |
| US20080077592P | – | – | – |
Members98
| Document | Office | Kind | |
|---|---|---|---|
| WO2010002214A2 | World Intellectual Property Organization (WIPO) | A2 | |
| KR20100004037A | Republic of Korea | A | |
| WO2010002214A3 | World Intellectual Property Organization (WIPO) | A3 | |
| EP2309756A2 | European Patent Office (EPO) | A2 | |
| US2011103475A1 | United States of America | A1 | |
| CN102144393A | China | A | |
| JP2011526770A | Japan | A | |
| US2012147957A1 | United States of America | A1 | |
| US8311110B2 | United States of America | B2 | |
| EP2309756A4 | European Patent Office (EPO) | A4 | |
| US2013077686A1 | United States of America | A1 | |
| US2013083849A1 | United States of America | A1 | |
| US2013083850A1 | United States of America | A1 | |
| CN103220527A | China | A | |
| EP2640076A1 | European Patent Office (EPO) | A1 | |
| JP5314136B2 | Japan | B2 | |
| US8611420B2 | United States of America | B2 | |
| US8649435B2 | United States of America | B2 | |
| EP2309756B1 | European Patent Office (EPO) | B1 | |
| EP2712198A1 | European Patent Office (EPO) | A1 | |
| EP2712199A2 | European Patent Office (EPO) | A2 | |
| EP2712200A2 | European Patent Office (EPO) | A2 | |
| EP2712199A3 | European Patent Office (EPO) | A3 | |
| EP2712200A3 | European Patent Office (EPO) | A3 | |
| EP2713617A1 | European Patent Office (EPO) | A1 | |
| EP2713618A1 | European Patent Office (EPO) | A1 | |
| US2014105287A1 | United States of America | A1 | |
| US2014105296A1 | United States of America | A1 | |
| CN102144393B | China | B | |
| US8824549B2 | United States of America | B2 | |
| EP2640076B1 | European Patent Office (EPO) | B1 | |
| US8837590B2 | United States of America | B2 | |
| CN104052993A | China | A | |
| CN104053003A | China | A | |
| CN104053004A | China | A | |
| CN104113755A | China | A | |
| CN104125462A | China | A | |
| US8879626B2 | United States of America | B2 | |
| DK2640076T3 | Denmark | T3 | |
| US8902979B2 | United States of America | B2 | |
| PT2640076E | Portugal | E | |
| ES2524555T3 | Spain | T3 | |
| SMT201400178B | San Marino | B | |
| US2015016525A1 | United States of America | A1 | |
| HRP20141157T1 | Croatia | T1 | |
| ES2524555T9 | Spain | T9 | |
| PL2640076T3 | Poland | T3 | |
| SI2640076T1 | Slovenia | T1 | |
| EP2713618B1 | European Patent Office (EPO) | B1 | |
| EP2712198B1 | European Patent Office (EPO) | B1 | |
| EP2712200B1 | European Patent Office (EPO) | B1 | |
| PT2713618E | Portugal | E | |
| DK2712198T3 | Denmark | T3 | |
| DK2712200T3 | Denmark | T3 | |
| DK2713618T3 | Denmark | T3 | |
| ES2532828T3 | Spain | T3 | |
| PT2712198E | Portugal | E | |
| PT2712200E | Portugal | E | |
| ES2533305T3 | Spain | T3 | |
| ES2533333T3 | Spain | T3 | |
| SMT201500060B | San Marino | B | |
| SMT201500073B | San Marino | B | |
| SMT201500075B | San Marino | B | |
| KR101517768B1 | Republic of Korea | B1 | |
| HRP20150299T1 | Croatia | T1 | |
| DK2712200T5 | Denmark | T5 | |
| PL2712198T3 | Poland | T3 | |
| PL2712200T3 | Poland | T3 | |
| PL2713618T3This record | Poland | T3 | |
| HRP20150346T1 | Croatia | T1 | |
| SI2712198T1 | Slovenia | T1 | |
| SI2713618T1 | Slovenia | T1 | |
| HRP20150321T1 | Croatia | T1 | |
| US9118913B2 | United States of America | B2 | |
| SI2712200T1 | Slovenia | T1 | |
| EP2713617B1 | European Patent Office (EPO) | B1 | |
| DK2713617T3 | Denmark | T3 | |
| ES2548227T3 | Spain | T3 | |
| PT2713617E | Portugal | E | |
| HRP20151005T1 | Croatia | T1 | |
| SI2713617T1 | Slovenia | T1 | |
| SMT201500227B | San Marino | B | |
| US2015326879A1 | United States of America | A1 | |
| PL2713617T3 | Poland | T3 | |
| EP2712199B1 | European Patent Office (EPO) | B1 | |
| HUE025473T2 | Hungary | T2 | |
| US9402079B2 | United States of America | B2 | |
| CN103220527B | China | B | |
| CY1115740T1 | Cyprus | T1 | |
| CY1116121T1 | Cyprus | T1 | |
| CY1116128T1 | Cyprus | T1 | |
| CY1116521T1 | Cyprus | T1 | |
| CY1116726T1 | Cyprus | T1 | |
| CN104052993B | China | B | |
| CN104053004B | China | B | |
| CN104053003B | China | B | |
| CN104113755B | China | B | |
| CN104125462B | China | B |
Numbers
- Publication, DOCDB
- 2713618
- Publication, EPODOC
- PL2713618T
- Application
- 20130196512
- Application, DOCDB
- 13196512
- Application, EPODOC
- PL20130196512T
Titles2
- English
- Image decoding method
- Polish
- Sposób dekodowania obrazu
Classification
- CPC, 12
- H04N19/13
- H04N19/11
- H04N19/117
- H04N19/119
- H04N19/176
- H04N19/182
- H04N19/33
- H04N19/50
- H04N19/593
- H04N19/82
- H04N19/198
- H04N19/91
- IPC, 10
- H04N19 00
- H04N19 11
- H04N19 117
- H04N19 119
- H04N19 176
- H04N19 182
- H04N19 33
- H04N19 50
- H04N19 593
- H04N19 82