Method and device for converting encoded video signal

The present invention relates to a coding method and a conversion method and an apparatus for converting a bit rate of a coded video signal encoded by a certain coding method.

Due to the rapid development of moving image processing technology in recent years, there are a wide variety of objects and devices that handle moving images. Further, since the amount of moving image data is generally very large, when handling moving images as digital data, it is common to perform compression coding before use.

Conventionally, compression coding is used according to the purpose of use and the target device, but for the purpose of sharing data between different devices, moving image data compressed and coded by a certain coding method is coded differently. Transcoding is being performed to convert to the conversion method. Further, the compression coding method is not changed, and the size and frame rate of the moving image are changed, the compression rate is increased, and the bit rate is converted to a bit rate suitable for the intended use.

For such conversion, a method is adopted in which the conversion source bitstream is once decoded and then re-encoded in a desired encoding format. However, in order to reduce the load of conversion processing at that time, intermediate information at the time of transcoding is adopted. A method of re-encoding using is proposed (see Patent Document).

<patcit num="1"><text>Japanese Patent Application Laid-Open No. 2003-264840</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 2003-116104</text></patcit><patcit num="3"><text>Japanese Unexamined Patent Publication No. 2003-61089</text></patcit><patcit num="4"><text>Japanese Unexamined Patent Publication No. 2002-320228</text></patcit><patcit num="5"><text>Japanese Unexamined Patent Publication No. 2001-268578</text></patcit>

<p> However, in the transcoding of the above-mentioned prior art, the first frame is decoded, and the intermediate information obtained at the time of the first decoding is immediately used for encoding the first frame, so that the first frame is still decoded. The intermediate information of the second and third frames, which was not used, could not be used for encoding the first frame.</p><p> An object of the present invention is to provide a method and an apparatus for making intermediate information of the second and third frames available when re-encoding the transcoding of the first frame.</p>

<p> A coded video signal conversion method that converts a first coded video signal into a second coded video signal, in which the input signal is compressed by the first compression coding method and then by the first compression coding method. The coded intermediate information generated before the compressed coded video signal is converted by the second compression coding method is stored, and the coded video signal converted by the first compression coding method is subjected to the second compression coding. When converting to the method, the stored coded intermediate information will be used.</p>

<p> High-quality transcoding and rate conversion can be realized.</p>

The configuration according to the first embodiment of the present invention is shown in FIG.

The video recording / playback device compresses the input digital video signal into a first encoded video signal by a first compression coding method (for example, MPEG2) and records it in a storage such as a hard disk. Further, stream conversion is realized by decoding the first encoded video signal and encoding it by a second compression coding method (for example, MPEG4).

The recording / playback device 100 includes an encoder 101, a transcoder 102, an intermediate information storage memory 105, a stream storage memory 106, and a user interface 110. The user interface 110 is not essential and is used as an interface when the user controls the operation of the transcoder 102. The intermediate information storage memory 105 and the stream storage memory 106 may be configured by independent storage media or may be configured by the same storage medium.

The encoder 101 encodes a digital video signal input from the outside of the recording / playback device 100 by the first compression coding method, and outputs the first encoded video signal. The output first encoded video signal is output to the stream storage memory 106 and stored. Further, the intermediate information generated when the digital video signal input from the outside is encoded by the encoder 101 is output to and stored in the intermediate information storage memory 105. Here, the intermediate information refers to a motion vector obtained at the time of encoding, a quantization value or a quantization scale, a statistical value of a change in brightness between frames before and after in time, a temporal transition of a bit rate, and the like.

The transcoder 102 includes a decoder 103 and an encoder 104. The first coded video signal read from the stream storage memory 106 is supplied to the decoder 103. The decoder 103 decodes the supplied first encoded video signal and outputs the digital video signal. Further, the decoder 103 has a function of changing the image size and frame rate of the decoded digital video signal, and the digital video signal output from the decoder 103 is the same as the decoded first encoded video signal. It may be the image size or frame rate, or it may be scaled to an appropriate size inside the decoder 103 or adjusted to an appropriate frame rate. The encoder 104 encodes the digital video signal output from the decoder 103 by the second compression coding method, and outputs the second coded video signal. Here, the intermediate information storage memory 105 stores the intermediate information when encoding is performed by the first compression coding method, and when the encoder 104 encodes by the second compression coding method, the intermediate information storage memory 105 stores the intermediate information. Is used when the intermediate information is read from the intermediate information storage memory 105, processed into a usable format, and encoded by the second compression coding method.

For example, when the second compression coding method is MPEG4, a large amount of calculation is required to obtain a motion vector at the time of coding. Further, in order to obtain the motion vector, the pixel values of the frames before and after the time are required, and it is necessary to secure a memory for holding the pixel values. In this method, the motion vector, which is the intermediate information when compressed by the first compression coding method, is read from the intermediate information storage memory, and this is suitable for the image size and frame rate when the second compression coding is performed. Scale and fine-tune in this way, and use it for the second compression coding. By this method, it is possible to reduce the amount of calculation for obtaining the motion vector and save the frame holding memory.

In another example, the bit rate control when performing the second compression coding by using the statistic of the temporal change of the luminance value, which is the intermediate information when the compression is performed by the first compression coding method. Can also be used. Specifically, a method of controlling the rate so as to obtain a visually high-quality image by allocating the code amount to the part where the brightness value changes drastically without allocating a large code amount to the part where the brightness value changes drastically. There is.

In another example, bit rate control when performing the second compression coding by using the quantization value, the quantization scale, or the bit rate transition when compressed by the first compression coding method. Can also be done. In this embodiment, since the intermediate information generated when encoding the video signal input from the outside already exists, more optimal bit rate control can be performed by feeding back the bit rate transition and the quantization value transition. It is also possible to do. This makes it possible to control the rate so as to follow the bit rate transition when compressed by the first compression coding method without delay.

The transcoder 102, the decoder 103, and the encoder 104 can be controlled from the user side via the user interface 110 so as to perform desired operations. Specifically, the bit rate setting of the encoder 104, the output image size of the decoder 103, the frame rate, and the like can be set.

The first compression coding method and the second compression coding method described above may be the same method or different methods.

Further, the encoder 101 and the encoder 104 may be partially or wholly shared.

Further, the intermediate information when compressed by the first compression coding method may always be recorded at the time of coding, or when the set bit rate at the time of coding exceeds a certain threshold, that is, it is encoded with high image quality. Intermediate information may be recorded only if it is the case. The user may specify whether or not to record intermediate information using the user interface 110.

The configuration according to the second embodiment of the present invention is shown in FIG.

The video recording / playback device compresses the input digital video signal into a first encoded video signal by a first compression coding method (for example, MPEG2) and records it in a storage such as a hard disk. At the same time, the input digital video signal itself is recorded in the storage in a compressed or uncompressed manner. Further, stream conversion is realized by encoding the digital video signal obtained by decoding the first coded video signal or the digital video signal recorded in the storage by the second compression coding method (for example, MPEG4).

The recording / playback device 100 includes an encoder 101, a transcoder 102, an intermediate information storage memory 105, a stream storage memory 106, a compression / decompression means 107, an input image storage memory 108, and a user interface 110. The user interface 110 is not essential, and is used as an interface when the user controls the operation of the transcoder 102 and the compression / decompression means 107. The intermediate information storage memory 105, the stream storage memory 106, and the input image storage memory 108 may be configured by independent storage media, or may be configured by the same storage medium.

Since the operations of the encoder 101, the intermediate information storage memory 105, and the stream storage memory 106 are the same as those of the first embodiment of the present invention, description thereof will be omitted here. The input digital video signal supplied to the encoder 101 is also supplied to the compression / decompression means 107 at the same time. The compression / decompression means 107 stores the input digital video signal in the input image storage memory 108 in the uncompressed state or after compressing it by a certain compression method. Here, the compression method is a method of entropy-encoding the difference between the pixel value with the previous frame, a method of entropy-encoding the difference with the left pixel value, a lossless compression method such as LZH or ZIP, JPEG, GIF. , PNG, or any other lossy compression method may be used.

Further, a mechanism may be provided in which the characteristics of the input digital video signal are determined by an analysis means such as a filtering operation, and an appropriate compression method is selected according to the characteristics. For example, the Sobel filter calculation for edge extraction is performed on the input digital signal, and for images with a large number of edge detection points, that is, images with many sharp changes in brightness, a compression method such as GIF or PNG is used to determine the number of edge detection points. For images with a small number of images, that is, images with smooth changes in brightness, a mechanism for switching such as compression by the JPEG method may be incorporated.

Further, the selection of compression / uncompression, the selection of the compression method, the selection of the compression rate, and the like when the input digital video signal is stored in the input image storage memory 108 can be controlled by the user via the user interface 110.

During the transcoding operation, the input digital video signal stored in the input image storage memory 108 is read by the compression / decompression means 107, and if the signal is compressed, it is decompressed and supplied to the encoder 104. When the signal is uncompressed, it is supplied to the encoder 104 as it is. Here, the compression / decompression means 107 has a function of changing the image size and frame rate of the input digital video signal, and the digital video signal output from the compression / decompression means 107 decodes the first encoded video signal. It may be the same image size or frame rate as it is, or it may be scaled to an appropriate size or adjusted to an appropriate frame rate by the compression / decompression means 107.

Since the operation of the decoder 103 included in the transcoder 102 is the same as that of the first embodiment of the present invention, the description thereof will be omitted here.

The configuration of the encoder 104 is shown in FIG. The encoder 104 includes input switching means 1041, encoding means 1042, and intermediate information use / non-use determination means 1043. The input digital video signal supplied from the compression / decompression means 107 and the decoded digital video signal supplied from the decoder 103 are supplied to the input switching means 1041, and are appropriately switched and output to the encoding means 1042. The input switching control can be directly controlled by the user via the user interface 110.

The encoding means 1042 encodes the digital video signal supplied from the input switching means 1041 by the second compression coding method, and outputs the second encoded video signal. Here, the intermediate information storage memory 105 stores the intermediate information when encoding is performed by the first compression coding method, and when the encoder 104 encodes by the second compression coding method, the intermediate information storage memory 105 stores the intermediate information. Is used when the intermediate information use / non-use determination means 1043 reads the intermediate information from the intermediate information storage memory 105, processes it into a usable format, and encodes it by the second compression coding method. Since this specific example is the same as that of the first embodiment of the present invention, the description thereof will be omitted. The intermediate information use / non-use determination means 1043 switches between coding using the intermediate information and coding without referring to the intermediate information. The control of switching can be controlled from the user side via the user interface 110.

Here, considering the case where the compression / decompression means 107 handles the input digital video signal in uncompressed or reversible compression, the input switching means 1041 selects the input digital video signal supplied from the compression / decompression means 107. In this case, since the digital video signal supplied to the encoding means 1042 does not deteriorate in image quality due to compression / decompression, it is possible to realize transcoding with very high image quality. Further, as described in the first embodiment of the present invention, the amount of processing required for transcoding can be reduced by using the data stored in the intermediate information storage memory 105.

In addition, a mechanism is provided to determine whether or not the intermediate information when compressed by the first compression coding method exists in the intermediate information storage memory 105, and to record the intermediate information only when the intermediate information does not exist. You may.

FIG. 4 shows a configuration according to a third embodiment of the present invention.

In this embodiment, the first coded video signal encoded by the first coding method input to the recording / playback device is decoded and coded by the second coding method to obtain a second code. Generates a video signal.

The recording / playback device 100 includes a transcoder 102, an intermediate information storage memory 105, a display means 109, and a user interface 110.

The first encoded video signal is input from the outside of the recording / playback device 100, for example, the stream storage memory 106, and is supplied to the transcoder 102. In addition to the above, the first encoded video signal may be supplied from an external storage such as a DVD medium, the Internet, or the like.

The transcoder 102 includes a decoder 103 and an encoder 104. The first coded video signal read from the stream storage memory 106 is supplied to the decoder 103. The decoder 103 decodes the supplied first encoded video signal and outputs a digital video signal. Further, the decoder 103 has a function of changing the image size and frame rate of the decoded digital video signal, and the digital video signal output from the decoder 103 is the same as the decoded first encoded video signal. It may be the image size or frame rate, or it may be scaled to an appropriate size inside the decoder 103 or adjusted to an appropriate frame rate. Further, the decoder 103 also outputs the intermediate information extracted from the first encoded video signal at the time of decoding at the same time, and stores the intermediate information in the intermediate information storage memory 105. Here, the intermediate information is the motion vector, the quantization value or the quantization scale extracted from the first coded video signal at the time of decoding, the statistical value of the brightness change between the frames before and after the time, and the bit rate. Refers to temporal transitions. In this embodiment, the decoder 103 is treated as a component of the transcoder 102, but instead, a decoder is connected to the outside of the transcoder 102, and the decoder is used to decode the first encoded video signal. May be.

The encoder 104 encodes the digital video signal output from the decoder 103 by the second compression coding method, and outputs the second coded video signal. Here, the intermediate information storage memory 105 stores the intermediate information extracted from the first compressed coded signal, and when the encoder 104 encodes with the second compressed coding method, the intermediate information is stored. Information is read from the intermediate information storage memory 105, processed into a usable format, and used when encoding with the second compression coding method. Since this specific method is the same as that of the first embodiment of the present invention, the description thereof will be omitted.

The timing at which the decoder 103 decodes the first compressed coded signal and simultaneously extracts the intermediate information and writes it to the intermediate information storage memory 105 is arbitrary. For example, the first compression code stored in the stream storage memory 106. The intermediate information may be stored in the background in which the conversion signal is decoded and reproduced and displayed via the display means 109, or the intermediate information may be extracted only without being displayed by the display means 109. Further, the idling state in which the user does not perform any operation may be automatically determined, and the intermediate information may be extracted and stored in the background.

Further, when the reproduction / display is not particularly performed, the system may perform decoding at the highest possible speed and extract and store the intermediate information.

Since it is unknown whether the intermediate information of the stream to be transcoded exists in the intermediate information storage memory 105, the encoder 104 has a function of determining whether or not the intermediate information of the stream to be transcoded exists in the intermediate information storage memory 105. It may be provided. In this configuration, if there is intermediate information as a result of determination, transcoding is performed using the intermediate information. If there is no intermediate information, encoding may be performed without using the intermediate information, or transcoding may be performed using the intermediate information after automatically generating the intermediate information. Further, the configuration may include the means for determining the use / non-use of the intermediate information in FIG. 3 and allow the user to select the use / non-use of the intermediate information. The flow of the above judgment is shown in Fig. 5. As an application example of this embodiment, when transcoding is performed, it is sufficient that intermediate information for the data desired for transcoding is prepared in advance, and further, decoding at the time of transcoding is re-encoded. By performing several frames to several tens of frames ahead of the frame, future intermediate information can be used, and for example, it is possible to follow the transition of the bit rate without delay.

<figref num="1">It is a figure which showed the 1st Embodiment of this invention.</figref><figref num="2">It is a figure which showed the 2nd Embodiment of this invention.</figref><figref num="3">It is a figure which showed the internal structure of the encoder 104.</figref><figref num="4">It is a figure which showed the 3rd Embodiment of this invention.</figref><figref num="5">It is a flowchart which showed the flow of judgment in the intermediate information use / non-use determination means 1043.</figref>

Code description

100 ...... Recording / playback device 101 ...... Encoder 102 ...... Transcoder 103 ...... Decoder 104 ...... Encoder 1041 ...... Input switching means 1042 ...... Encoding means 1043 ...... Intermediate information use / non-use determination means 105 ...... Intermediate information storage memory 106 ...... Stream storage memory 107. ..... compression / decompression means 108 ...... input image storage memory 109 ...... display means 110 ...... user interface