Coding and decoding device for dynamic image
2 claims: 2 independent, 0 dependent
- 1【特許請求の範囲】 【請求項1】 符号化部と復号化部とから構成される動画像の符号化・復号化装置において、前記符号化部は、入力画像から被写体領域の輪郭線データを検出する手段と、前記輪郭線データを参照して前記入力画像上に複数の動き検出代表点を設定する手段と、前フレームの復号画像を参照して前記代表点でのフレーム間の動き量を計算し、動きデータとして出力する手段と、前記入力画像の画素毎に、前記輪郭線データを参照して該当画素と同じ被写体領域上にある前記代表点の一部を選択し、前記選択された複数の代表点での動きデータから前記該当画素の動き量を内挿計算し、前記画素毎に得た動き量のデータを用いて前記前フレームの復号画像から動き補償フレーム間予測を行い、符号化予測データを出力する手段と、前記入力画像と前記符号化予測データとの差分データを出力する手段と、前記差分データを量子化し、量子化データを出力する手段と、前記量子化データを逆量子化する手段と、前記逆量子化したデータと前記符号化予測データとを加算して復号画像を再生する手段と、前記再生した復号画像を保持し、次フレームの符号化時に出力する手段と、前記輪郭線データと前記量子化データと前記複数の代表点での動きデータとを符号化し、符号化データとして復号化部へ出力する手段とを備え、 前記復号部は、前記符号化部から供給された前記符号化データを復号し、量子化データと輪郭線データと複数の代表点での動きデータとを出力する手段と、前記復号した量子化データを逆量子化する手段と、前記復号した輪郭線データを参照して前記復号した複数の代表点が何れの領域に属するかを分類し、画素毎の動き量を該当画素と同じ領域上にある複数の代表点での動きデータから内挿計算して求め、前記画素毎に求めた動き量のデータを用いて前フレームでの復号画像から動き補償フレーム間予測を行い、復号予測データを出力する手段と、前記逆量子化したデータと前記復号予測データとを加算して復号画像を再生し、外部へ出力する手段と、前記再生した復号画像を保持し、次フレームの復号時に出力する手段とを備えることを特徴とする動画像の符号化・復号化装置。
- 2【請求項2】 符号化部と復号化部とから構成される動画像の符号化・復号化装置において、前記符号化部は、前フレームの復号画像から被写体領域の輪郭線データを検出する手段と、前記輪郭線データを参照して前記前フレームの復号画像上に複数の動き検出代表点を設定する手段と、前記入力画像を参照して前記代表点でのフレーム間の動き量を計算し、動きデータとして出力する手段と、前記入力画像の画素毎に、前記輪郭線データを参照して該当画素と同じ被写体領域上にある前記代表点の一部を選択し、前記選択された複数の代表点での動きデータから前記該当画素の動き量を内挿計算し、前記画素毎に得た動き量のデータを用いて前記前フレームの復号画像から動き補償フレーム間予測を行い、符号化予測データを出力する手段と、前記入力画像と前記符号化予測データとの差分データを出力する手段と、前記差分データを量子化し、量子化データを出力する手段と、前記量子化データを逆量子化する手段と、前記逆量子化したデータと前記符号化予測データとを加算して復号画像を再生する手段と、前記再生した復号画像を保持し、次フレームの符号化時に出力する手段と、前記量子化データと前記複数の代表点での動きデータとを符号化し、符号化データとして復号化部へ出力する手段とを備え、前記復号化部は、前記符号化部から供給された前記符号化データを復号し、量子化データと複数の代表点での動きデータとを出力する手段と、前記復号した量子化データを逆量子化する手段と、前フレームの復号画像から被写体領域の輪郭線データを検出する手段と、復号化部で前記検出した輪郭線データを参照して前記復号した複数の代表点が何れの領域に属するかを分類し、画素毎の動き量を該当画素と同じ領域上にある複数の代表点での動きデータから内挿計算して求め、前記画素毎に求めた動き量のデータを用いて前フレームでの復号画像から動き補償フレーム間予測を行い、復号予測データを出力する手段と、前記逆量子化したデータと前記復号予測データとを加算して復号画像を再生し、外部に出力する手段と、前記再生した復号画像を保持し、次フレームの復号時に出力する手段とを復号化部に備えることを特徴とする動画像の符号化・復号化装置。
Independent claims2
116 paragraphs, as filed
Description: TECHNICAL FIELD [Detailed description of the invention]
【0001】
[Industrial application field]
The present invention relates to a moving image coding / decoding device.
【0002】
[Conventional technology]
In the conventional technique, prediction data is generated from the decoded image of the previous frame without considering the subject structure of the image, and the difference between the input image and the prediction data is encoded.
【0003】
For example, in the method described in IE90-106 "Basic study of motion compensation by triangular patch" of IEICE Technical Report, first, multiple motion detection representative points are set on the input image, and between frames on these representative points. The motion vector is detected. The setting positions of the representative points are set at predetermined equal intervals, and the subject structure of the image is not taken into consideration. Next, the motion vector is interpolated and calculated for each pixel of the input image. The interpolation calculation refers to a plurality of representative points in the vicinity of the pixel of interest. Motion compensation frame-to-frame prediction is performed pixel by pixel using the motion vector obtained by interpolation calculation.
【0004】
In this method, the change of the motion vector value between pixels becomes smooth. Therefore, block-like distortion does not occur on the decoded image, and visually good code decoding can be realized. In addition, even when the movement changes continuously inside the subject area, it can be easily dealt with, and there is an effect of improving the coding efficiency.
【0005】
[Problems to be Solved by the Invention]
In the conventional video coding and decoding method, the motion vector is interpolated without considering the subject structure of the image. For this reason, even in the part where the movement is actually changed discontinuously, the interpolation process is uniformly performed, and the coding efficiency may be impaired.
【0006】
A case where a problem occurs will be described using (a) and (b) of FIG.
【0007】
FIG. 7A shows a case where the subject is moving in front of a stationary background. In FIG. 7A, the left side of the contour line is the moving subject area, and the right side is the background. In the conventional method, the motion detection representative point is set regardless of the subject contour line, and the motion vector is detected. In FIG. 7A, a motion vector having a value other than 0 is detected at a representative point inside the subject area, and a motion vector having a value of 0 is detected at a representative point outside the subject area. That is, on the actual image, the motion vector changes sharply with the contour line as the boundary.
【0008】
FIG. 7 (b) shows the result of interpolating the motion vector for each pixel from the motion vector of the representative point detected in FIG. 7 (a) according to the conventional method. In (b) of Fig. 7, the sharp change of the motion vector is lost near the contour line. As a result, the accuracy of motion compensation frame-to-frame prediction is lowered in the vicinity of the contour line, which may hinder the coding efficiency. Further, since the motion vector on the contour line has a value different from the actual value, there is also a problem that the contour shape of the subject area is deformed.
【0009】
An object of the present invention is to provide a coding / decoding device having excellent coding efficiency even in a portion where movement is discontinuously changed by performing motion vector interpolation processing in consideration of the subject structure of an image. It is to be realized.
【0010】
[Means for solving problems]
The first invention is a moving image coding / decoding device composed of a coding unit and a decoding unit, wherein the coding unit is a means for detecting contour line data of a subject area from an input image. The means for setting a plurality of motion detection representative points on the input image with reference to the contour line data, and the motion amount between frames at the representative points are calculated with reference to the decoded image of the previous frame, and the motion data For each pixel of the input image, a part of the representative points on the same subject area as the corresponding pixel is selected with reference to the contour line data, and the selected representative points are used. The movement amount of the corresponding pixel is interpolated from the movement data of the above, and the movement compensation frame-to-frame prediction is performed from the decoded image of the previous frame using the movement amount data obtained for each pixel, and the coded prediction data is output. A means for outputting the difference data between the input image and the coded prediction data, a means for quantizing the difference data and outputting the quantization data, and a means for dequantizing the quantization data. , A means for reproducing the decoded image by adding the dequantized data and the encoded prediction data, a means for holding the reproduced decoded image and outputting it at the time of encoding the next frame, and the contour line data. A means for encoding the quantization data and the motion data at the plurality of representative points and outputting the coded data to the decoding unit, the decoding unit having the code supplied from the coding unit. A means for decoding the decrypted data and outputting the quantization data, the contour line data, and motion data at a plurality of representative points, a means for dequantizing the decoded quantization data, and the decoded contour line data. With reference to this, the decoded multiple representative points are classified into which region, and the movement amount for each pixel is obtained by interpolating calculation from the movement data at the plurality of representative points on the same region as the corresponding pixel. A means for performing motion compensation frame-to-frame prediction from the decoded image in the previous frame using the motion amount data obtained for each pixel and outputting the decoding prediction data, and the inversely quantized data and the decoding prediction data. Is added to reproduce the decoded image and output to the outside, and the reproduced decoded image is retained and the next frame is displayed.It is characterized in that it is provided with a means for outputting when decoding the game.
【0011】
The second invention is a moving image coding / decoding device composed of a coding unit and a decoding unit, in which the coding unit detects contour line data of a subject area from a decoded image of a previous frame. Means, means for setting a plurality of motion detection representative points on the decoded image of the previous frame with reference to the contour line data, and calculation of the amount of motion between frames at the representative points with reference to the input image. Then, the means for outputting as motion data and a part of the representative points on the same subject area as the corresponding pixel are selected by referring to the contour line data for each pixel of the input image, and the selected plurality of points are selected. The movement amount of the corresponding pixel is interpolated from the movement data at the representative point of the above, and the movement compensation frame-to-frame prediction is performed from the decoded image of the previous frame using the movement amount data obtained for each pixel, and encoded. A means for outputting the prediction data, a means for outputting the difference data between the input image and the coded prediction data, a means for quantizing the difference data and outputting the quantization data, and an inverse quantum of the quantization data. Means for converting, means for reproducing the decoded image by adding the dequantized data and the coded prediction data, means for holding the reproduced decoded image and outputting it at the time of encoding the next frame. A means for encoding the quantization data and the motion data at the plurality of representative points and outputting the coded data to the decoding unit is provided, and the decoding unit includes the code supplied from the coding unit. Means for decoding the decrypted data and outputting the quantization data and motion data at a plurality of representative points, means for dequantizing the decoded quantization data, and contour lines of the subject area from the decoded image of the previous frame. The means for detecting the data and the decoding unit refer to the detected contour line data to classify which region the plurality of decoded representative points belong to, and the amount of movement for each pixel is the same region as the corresponding pixel. Interpolation calculation is performed from the motion data at a plurality of representative points above, and motion compensation frame-to-frame prediction is performed from the decoded image in the previous frame using the motion amount data obtained for each pixel, and the decoding prediction data. And the means for outputting the data, the inversely quantized data, and the decoding prediction data are added to reproduce the decoded image.The decoding unit is characterized in that the decoding unit is provided with a means for outputting to the outside and a means for holding the reproduced decoded image and outputting it at the time of decoding the next frame.
【0012】
[Example]
FIG. 1 is a block diagram showing an embodiment of a moving image coding / decoding device of the first invention. This moving image coding / decoding device is composed of a coding unit and a decoding unit. The coding unit includes a contour detection circuit 100 that detects the contour line data 122 of the subject area from the input image 121, and a representative point detection circuit that sets a plurality of motion detection representative points on the input image with reference to the contour line data 122. 101, a motion detection circuit 102 that detects motion data 124 between frames at the representative point with reference to the decoded image 123 of the previous frame, contour line data 122, motion data 124, and a decoded image 123 of the previous frame. With reference to the motion compensation circuit 103 that makes motion compensation frame-to-frame prediction for each pixel of the input image and outputs the coded prediction data 125, and the diffifier 104 that takes the difference between the input image 121 and the coded prediction data 125. A quantization circuit 105 that quantizes the difference data and outputs the quantization data 126, an inverse quantization circuit 106 that dequantizes the quantization data 126, and the inverse quantization data and the coding prediction data 125. An adder 107 that adds and reproduces the decoded image, a decoded image memory 108 that holds the reproduced decoded image and outputs it when the next frame is encoded, contour line data 122, motion data 124, and quantization data 126. Is encoded and the code conversion circuit 109 outputs the encoded data 127 to the decoding unit.
【0013】
The decoding unit decodes the coded data 127 supplied from the coding unit, and dequantizes the quantization data 128, the inverse code conversion circuit 110 that outputs the contour line data 129, and the motion data 130, and the quantization data 128. Motion compensation circuit 112 that refers to the inverse quantization circuit 111, the contour line data 129, the motion data 130, and the decoded image 131 of the previous frame, makes motion compensation frame-to-frame prediction for each pixel, and outputs the decoding prediction data 132. The decrypted image 133 is reproduced by adding the dequantized data and the decoding prediction data 132, and the decoder 113 and the decoded image 133 that are output to the outside are held and output at the time of decoding the next frame. It consists of memory 114.
【0014】
The operation of this embodiment will be described.
【0015】
In the coding unit, the contour detection circuit 100 first detects the contour line data 122 of the subject area from the input image 121. Examples of the contour line data detection method are shown in FIGS. 3 (a), (b), (c), and (d), and each of them will be described.
【0016】
(A), (b), (c), and (d) of FIG. 3 are block diagrams of an example of a device that realizes detection of contour line data.
【0017】
In FIG. 3A, first, the difference between the input image 302 of the previous frame held in the image memory and the current input image 301 is obtained. The difference data is 0 in the non-moving area, and often takes a value other than 0 in the moving area. The boundary detection circuit separates a region having many 0s and a region having many non-zero values from the distribution of the values of the difference data, and outputs the region boundary line as the contour line data 303 of the subject area.
【0018】
In FIG. 3B, first, the contour line data 312 of the entire subject area is detected from the input image 311. Next, the difference between the contour line data 313 in the previous frame and the current contour line data 312 stored in the contour line memory is taken. The contour line component existing only on the current input image is output as valid contour line data 314. As shown in (b) of FIG. 3, only the contour line of the moving subject area on the image can be selected.
【0019】
In (c) of FIG. 3, first, the difference between the background image 322 held in the background image memory and the input image 321 is obtained. The difference is 0 in the background area and often takes a non-zero value in the subject area other than the background. In the boundary detection circuit, a region having many 0s and a region having many non-zero values are separated from the distribution of the difference values, and the region boundary line is output as contour line data 323 of the subject area. The background image to be stored in the background image memory may be set to an appropriate image in advance, or may be generated from an input image. By using the method (c) in FIG. 3, accurate contour detection is possible even when the background is complicated. In addition, there are few false detections of contour lines even in the uncovered area that appears from behind the subject area.
【0020】
In FIG. 3D, the motion detection circuit first detects the motion vector 333 for each pixel of the input image 331. At this time, the input image 332 of the previous frame held in the image memory is referred to. The boundary detection circuit detects and outputs contour line data 334 of a subject area having different movements from the value of the motion vector detected for each pixel. It is utilized that the difference between the motion vectors between adjacent pixels is small inside the subject with continuous motion, and the value of the motion vector changes abruptly at the boundary portion of the subject with different motions. By using the method (d) of FIG. 3, when there are a plurality of subjects having different movements other than the background portion, it is possible to detect the contour line for each subject. The representative point setting circuit 101 sets a motion detection representative point on the input image. The position of the motion detection representative point is set with reference to the contour line data 122.
【0021】
(A), (b), and (c) of FIG. 4 show an example of motion detection representative points when the contour line of the subject area is detected.
【0022】
(A) in FIG. 4 is an example in which motion detection representative points are set in a square grid at equal intervals. FIG. 4B is an example of the final setting in which the motion detection representative points are tentatively set at predetermined small intervals, the representative points close to the contour line are left, and the representative points far from the contour line are thinned out. In (c) of Fig. 4, after the motion detection representative points are tentatively set in a square grid at equal intervals, the representative points near the contour line move on the contour line, and the representative points far from the contour line are left as they are. This is an example of the final setting. In FIG. 4 (c), among the motion detection representative points initially set in a square grid at equal intervals, the motion detection representative points P1, P2, P3, P4, P5, and P6 close to the contour line are moved onto the contour line. Shows the case. When the distance between the square grids is 2S and the distance between the initially set motion detection representative point and the contour line is smaller than S, it is judged that the distance between the two is close and the motion detection representative point is moved.
【0023】
Further, the motion detection representative points are not limited to the square grid-like settings as in the examples of (a), (b), and (c) in FIG. If the representative point arrangement common to the coding unit and the decoding unit is determined in advance, any arrangement may be used.
【0024】
The motion detection circuit 102 detects motion data at a motion detection representative point set on the input image. At this time, if the motion detection representative points are set to adapt to the contour lines of the subject area as shown in FIGS. 4 (b) and 4 (c), it is possible to detect the motion at the boundary of the subject in detail. become.
【0025】
Motion data detection can be realized by block matching by grouping pixels in the vicinity of the representative point of interest into blocks. Alternatively, the luminance gradient method may be used.
【0026】
In the motion compensation prediction circuit 103, first, the contour line data 122 and the motion data 124 of the motion detection representative point are referred to, and the motion vector is interpolated for each pixel of the input image 121. When the contour line does not exist in the vicinity of the pixel of interest, the motion vector is interpolated from the motion data of a plurality of representative points in the vicinity of the pixel of interest. When the contour line exists in the vicinity of the pixel of interest, the motion vector is interpolated from the motion data of a plurality of representative points in the vicinity of the pixel and on the same subject area side.
【0027】
Figures (a) and (b) of Fig. 5 show an example of the motion vector interpolation method near the contour line.
【0028】
In FIG. 5A, motion data is detected at motion detection representative points P1 and P2 inside the contour line of the subject area and motion detection representative points P3 and P4 outside. At this time, the motion vector of the pixel X1 inside the contour line is obtained by interpolation calculation from only the motion data at P1 and P2. The motion vector of the pixel position X2 outside the contour line is obtained by interpolation calculation only from the motion data at P3 and P4. That is, the motion vector at an arbitrary pixel position is interpolated only from the motion data of the motion detection representative points in the vicinity on the same side with the contour line as the boundary. In FIG. 5 (a), the case where there are only two representative points for motion detection in the vicinity is shown, but any number of representative points of one or more may be referred to.
【0029】
In FIG. 5B, motion data is detected at the motion detection representative point R1 inside the contour line of the subject area, R2 and R3 on the contour line, and R4 outside the contour line. At this time, the contour line itself is regarded as the inside of the subject area. The motion vector of the pixel position Y1 inside the contour line is obtained by interpolation calculation from the motion data of R1, R2, and R3. The motion vector of the pixel position Y2 outside the contour line is obtained by interpolation calculation from the motion data of P4.
【0030】
Next, the motion compensation prediction circuit 103 refers to the value of the motion vector obtained by interpolation calculation, and performs motion compensation frame-to-frame prediction for each pixel of the input image 121. The pixel value is read from the motion compensation position of the decoded image 123 of the previous frame, and the coding prediction data 125 of the pixel is obtained. The diffifier 104 takes the difference between the input image 121 and the coded prediction data 125. The quantization circuit 105 quantizes the difference data and outputs the quantization data 126. In the dequantization circuit 106, the quantization data 126 is dequantized. The adder 107 adds the inverse quantized data and the coded prediction data 125 to reproduce the decoded image. The decoded image memory 108 holds the decoded image and outputs it when the next frame is encoded. The code conversion circuit 109 encodes the contour line data 122, the motion data 124, and the quantization data 126, and transmits the coded data 127 to the decoding unit.
【0031】
The decoding unit first decodes the coded data 127 supplied from the coding unit by the inverse code conversion circuit 110, and obtains the quantization data 128, the contour line data 129, and the motion data 130. The dequantization circuit 111 dequantizes the quantization data 128. The motion compensation prediction circuit 112 refers to the contour line data 129, the motion data 130, and the decoded image 131 of the previous frame, and obtains and outputs the decoded prediction data 132 for each pixel. The generation of the decoding prediction data 132 in the motion compensation prediction circuit 112 can be realized by the same method as the generation of the coding prediction data 125 in the motion compensation prediction circuit 103 of the coding unit. The adder 113 adds the inverse quantized data and the decoding prediction data 132, reproduces the decoded image 133, and outputs the decoded image 133 to the outside. At the same time, the reproduced decoded image 133 is held in the decoded image memory 114 and output when the next frame is decoded.
【0032】
FIG. 2 is a block diagram showing an embodiment of the moving image coding / decoding apparatus of the second invention. This moving image coding / decoding device is composed of a coding unit and a decoding unit.
【0033】
The coding unit sets a contour detection circuit 200 that detects the contour line data 223 of the subject area from the decoded image 222 of the previous frame, and sets a plurality of motion detection representative points on the decoded image of the previous frame with reference to the contour line data 223. The representative point setting circuit 201, the motion detection circuit 202 that detects the motion data 224 between frames at the representative point with reference to the input image 221, the contour line data 223, the motion data 224, and the decoded image 222 of the previous frame. A differ that takes the difference between the motion compensation prediction circuit 203, which makes motion compensation frame-to-frame prediction for each pixel of the input image and outputs the coded prediction data 225, and the input image 221 and the coded prediction data 225. 204, a quantization circuit 205 that quantizes the difference data and outputs the quantization data 226, an inverse quantization circuit 206 that dequantizes the quantization data 226, and the inverse quantization data and the coding prediction data. The adder 207 that adds 225 to reproduce the decoded image, the decoded image memory 208 that holds the reproduced decoded image and outputs it when the next frame is encoded, and the motion data 224 and the quantization data 226 are coded. It is composed of a code conversion circuit 209 that converts the data and outputs the coded data 227 to the decoding unit.
【0034】
The decoding unit decodes the coded data 227 supplied from the coding unit, outputs the quantization data 228 and the motion data 229, and the inverse code conversion circuit 210, and the inverse quantization that dequantizes the quantization data 228. With reference to the circuit 211, the contour detection circuit 212 that detects the contour line data 231 of the subject area from the decoded image 230 of the previous frame, the motion data 229, the contour line data 231 and the decoded image 230 of the previous frame, the input image The motion compensation prediction circuit 213 that makes motion compensation frame-to-frame prediction for each pixel and outputs the decoding prediction data 232, and the inverse quantization data and the decoding prediction data 232 are added to reproduce the decoded image 233 and to the outside. It is composed of an adder 214 for output and a decoded image memory 215 which holds the decoded image 233 and outputs it when decoding the next frame.
【0035】
The operation of this embodiment will be described.
【0036】
In the coding unit, the contour detection circuit 200 detects the contour line data 223 of the subject area from the decoded image 222 of the previous frame. The detection of the contour line data 223 can be realized by the same method as described above with reference to FIG. 3 in the embodiment of the code decoding apparatus for the first moving image of the present invention.
【0037】
In the representative point setting circuit 201, a plurality of motion detection representative points are set on the decoded image 222 of the previous frame with reference to the contour line data 223. The setting of the motion detection representative point can be realized by the same method as described above with reference to FIG. 4 in the embodiment of the first invention.
【0038】
The motion detection circuit 202 refers to the input image 221 and detects the motion data 224 at the motion detection representative point. In the motion compensation prediction circuit 203, the contour line data 223 and the motion data 224 are referred to, and the motion vector is interpolated and calculated for each pixel of the decoded image 222 of the previous frame. Interpolation of the motion vector can be realized in the embodiment of the first invention in the same manner as described above with reference to FIG.
【0039】
Next, with reference to the value of this motion vector, the motion compensation position on the input image is calculated for each pixel of the decoded image 222 of the previous frame. The value of the pixel becomes the coded prediction data 225 of the pixel of the motion compensation position on the input image. In this method, pixels may be generated on the input image in which the motion compensation frame-to-frame correspondence between the input image 221 and the decoded image 222 is not provided. In these pixels, motion data is interpolated from the motion data of nearby pixels for which motion compensation frame-to-frame correspondence is given.
【0040】
Figure 6 shows an example of a method of interpolating motion data. In FIG. 6, pixels P1 and P2 on the decoded image of the previous frame are associated with pixels C1 and C2 on the input image by motion compensation frame-to-frame prediction. On the other hand, the pixel Cx in the vicinity of C1 and C2 on the input image is not given a corresponding point by motion compensation frame-to-frame prediction. Therefore, the motion data of C1 and C2 are interpolated and the motion data of Cx is obtained. The motion data of C1 and C2 can be obtained only by reversing the positive and negative signs of the motion data of P1 and P2. By the above method, the motion compensation frame-to-frame correspondence position Px on the decoded image of the previous frame of Cx is determined. The Cx coding prediction data is the Px pixel value.
【0041】
In the diffifier 204, the difference between the input image 221 and the coded prediction data 225 is taken. The quantization circuit 205 quantizes the difference data and outputs the quantization data 226. In the dequantization circuit 206, the quantization data 226 is dequantized, and the adder 207 adds it to the coded prediction data 225 to reproduce the decoded image. The decoded image is held in the decoded image memory 208 and output when the next frame is encoded. The code conversion circuit 209 encodes the motion data 224 and the quantization data 226 and transmits the coded data 227 to the decoding unit.
【0042】
The decoding unit first decodes the coded data 227 supplied from the coding unit by the inverse code conversion circuit 210 to obtain the quantization data 228 and the motion data 229. In the dequantization circuit 211, the quantization data 228 is dequantized. The contour detection circuit 212 detects the contour line data 231 of the subject area from the decoded image 230 of the previous frame. The detection of the contour line data 231 can be realized by the same method as the contour detection circuit 201 of the coding unit. The motion compensation prediction circuit 213 refers to the motion data 229, the contour line data 231 and the decoded image 230 of the previous frame, and obtains the decoding prediction data 232 for each pixel by the motion compensation inter-frame prediction processing. The generation of the decoding prediction data 232 in the motion compensation prediction circuit 213 can be realized by the same method as the generation of the coding prediction data 225 in the motion compensation prediction circuit 203 of the coding unit.
【0043】
The adder 214 adds the inverse quantized data and the decoding prediction data 232, reproduces the decoded image 233, and outputs the decoded image 233 to the outside. At the same time, the reproduced decoded image 233 is held in the decoded image memory 215 and output when the next frame is decoded.
【0044】
[Effect of the invention]
As described above, in the present invention, the contour lines of subjects having different movements are used as boundaries, and the motion vectors are interpolated separately for each subject area. Therefore, it is possible to suppress unnecessary distortion of the motion vector near the contour line of the subject. An example of this effect will be described with reference to FIGS. 8 (a) and 8 (b). FIG. 8A shows a case where the subject is moving in front of a stationary background. In FIG. 8A, the left side of the contour line is the moving subject area, and the right side is the background. In the apparatus of the present invention, the motion detection representative point is set with reference to the subject contour line, and the motion vector is detected. In FIG. 8A, the representative points are initially set by a predetermined method, and the final set positions in which the representative points near the contour line are moved onto the contour line are shown. A motion vector having a value other than 0 is detected at a representative point inside the subject area and on the contour line, and a motion vector having a value of 0 is detected at a representative point outside the subject. FIG. 8B shows the result of interpolating the motion vector for each pixel according to the apparatus of the present invention from the motion vector detected at the representative point of FIG. 8A.
【0045】
In the apparatus of the present invention, since the motion vector is interpolated by referring only to the representative points on the same region with the contour line as the boundary, a steep change of the motion vector in the vicinity of the contour line can be reproduced more accurately. As a result, the efficiency of motion compensation frame-to-frame prediction can be significantly improved as compared with the conventional case in which motion vector interpolation processing is performed without considering the subject structure of the image. Further, by setting the motion detection representative point on the contour line, the contour shape of the subject can be reproduced more accurately, and visually good coding and decoding of the moving image can be realized.
[Simple explanation of drawings]
[Figure 1]
It is a block diagram which shows one Example of 1st invention.
[Figure 2]
It is a block diagram which shows one Example of the 2nd invention.
[Fig. 3]
It is a figure explaining the method of detecting the contour line of a subject area from an image.
[Fig. 4]
It is a figure explaining the method of setting the motion detection representative point with reference to the contour line data.
[Fig. 5]
It is a figure explaining the method of interpolating the motion vector of an arbitrary pixel position from the motion data detected at the motion detection representative point with reference to the contour line data.
[Fig. 6]
In the embodiment of the second invention, it is a figure explaining the method of interpolating motion data into the pixel which is not given the motion compensation frame-to-frame correspondence.
[Fig. 7]
It is a figure explaining an example of a problem in a conventional method.
[Fig. 8]
It is a figure explaining an example of the effect of this invention.
[Explanation of symbols]
100, 200, 212 contour detection circuit 101, 201 Representative point setting circuit 102, 202 motion detection circuit 103, 112, 203, 213 Motion compensation prediction circuit 104, 204 diff 105, 205 Quantization circuit 106, 111, 206, 211 Inverse quantization circuit 107, 113, 207, 214 adder 108, 114, 208, 215 decrypted image memory 109, 209 Code conversion circuit 110, 210 sign inverse conversion circuit 121,221 Input image 122, 129, 223, 231 contour data 123, 131, 222, 230 Decoded image of the previous frame 124, 130, 224, 229 Motion data 125, 225 Coded prediction data 126, 128, 226, 228 Quantized data 127, 227 Coded data 132, 232 Decryption prediction data 133,233 Decrypted image
8 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US6862369B2 | Cited by | United States of America | Applicant |
2 priority claims, no other members on record
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 33399293 | Japan | A | |
| JP19930333992 | – | – | – |
1 legal event, as the office reported them to INPADOC
Events
| Event | Code | |
|---|---|---|
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 |
Numbers
- Publication
- 2616552
- Publication, DOCDB
- 2616552
- Publication, EPODOC
- JP2616552B
- Application
- 5333992
- Application, DOCDB
- 33399293
- Application, EPODOC
- JP19930333992
Titles2
- Japanese
- 【発明の名称】動画像の符号化・復号化装置
- English
- INDUSTRIAL APPLICABILITY: Moving image coding / decoding device
Classification
- CPC, 2
- G06T9/20
- H04N19/51
- IPC, 8
- G06T9 00
- G06T9 20
- H04N19 00
- H04N19 50
- H04N19 503
- H04N19 51
- H04N19 537
- H04N19 543
