Multi-rate transcoder for digital streams

Multi-rate code converter for digital stream

BACKGROUND OF THE INVENTION The present invention relates to multi-rate encoding conversion of data signals such as video, audio, and/or multimedia, and is particularly useful for data streaming from a server to a client via a network (such as the Internet or a satellite or cable television system).

Background technique

Although the present invention is described here in connection with a multi-rate transcoder for compressed video applications, it should be understood that these inventive concepts can also be applied to any digital data stream that needs to provide the same data at multiple rates. Accordingly, the present invention and the appended claims are not limited to the video applications specifically introduced here.

Most network applications can be divided into two main parts. This is the client and server. We can find countless examples of clients and servers. Specifically, streaming includes sending movies or other content (such as audio and/or other multimedia formats) from a server to a client via a network (such as the Internet). Streaming is different from simple file transfer. The difference between it and file transfer is that the client plays the movie (or other content) as if it came from the network, instead of waiting for the entire movie to be received before playing. Real-time streams can be sent one-to-one (unicast) or one-to-many (multicast). In unicast, the client contacts the server to request a movie. The server then responds to the client with information about the requested movie. Then the actual movie stream is sent to the client. In multicast, a copy of the movie stream is "broadcast" to every branch of the network. The client receives the stream by "joining" the broadcast. Not all routers support multicast.

Clients (such as cable or satellite TV users) usually communicate with only one server at a time. For example, the client terminal sends a request (unicast) to the server and receives a stream from the server (that is, it can be unicast or multicast). From the perspective of the server at any given moment, the server usually communicates with multiple clients. For example, in unicast, the server may send streams to different clients at the same time, and in multicast, multiple clients receive the streams broadcast by the server.

With the recent development of video compression technology, through the use of compression technologies (such as MPEG1, 2, 4 (here MPEG is a standard issued by the Motion Picture Experts Group) and ITU-TH.261, 3, 3+, or L( Here ITU is a standard issued by the International Telecommunication Union)) can significantly reduce the size of the original video material without reducing the picture quality. Therefore, it is hoped that more pre-compressed video materials can be provided on the server side in the future. Therefore, the server may need to have the ability to adjust the rate of the pre-compressed video material stored on the server in order to adapt to the bandwidth requirements of different clients. A code converter is an option to solve this problem. According to its definition, a transcoder is a device used to convert a pre-compressed bit stream into another bit stream with a different format. These different formats can include resolution, bit rate, and other variable parameters. The present invention provides a code converter, which provides rate conversion of a signal input stream to provide multiple output streams of different rates.

For example, when a client (for example, a set-top box used to receive TV services in a user's home) requests a movie, it often also provides information about the network situation on the client side at the same time. The server can then convert the pre-compressed stream encoding of the requested movie into a form suitable for the bandwidth of the client. However, a problem is that the server may receive multiple requests for the same movie from different clients. In addition, the bandwidth on the client side can vary greatly. For example, a 56Kb/s telephone modem and a few Mb/s cable modem have different capabilities to receive data from the server. Therefore, it is very useful to encode a compressed video bitstream on the server side into several different new streams with different rates. A straightforward solution is to use different codecs to convert the same input video stream at different rates in response to user requests. However, this solution is not feasible because it requires the server to have the same number of transcoders as the number of clients served by the server, which is extremely expensive.

Another solution is that every time a different client makes a request (but only one at a time), a transcoder on the server side processes the same video stream. In other words, before the server completes a client's request, other clients will have to wait. Therefore, in order to solve the problem of one server and multiple services, either the system processor will have to install more transcoders, or the client will have to wait for their turn to receive the requested service (e.g., movie). In addition, in the above two solutions, for each request for the same video bitstream, all modules in the encoding processor need to perform operations.

It is beneficial to provide a reliable and reasonably priced transcoding solution that can deliver services to multiple users (clients) at different rates according to the client's request. The present invention provides a multi-rate transcoding scheme with the above and other advantages.

Summary of the invention

In the present invention, a code converter device is provided for the coded bit stream. The data processor extracts overhead data from the bit stream. A decoder for at least partially decoding a bitstream is provided. The rate control processor re-encodes the at least partially decoded bit stream at different rates to generate a plurality of re-encoded bit streams with different rates. The multiplexer combines the overhead data with each re-encoded bitstream to provide multiple versions of the encoded bitstream at different rates.

In a preferred embodiment, the multiplexer provides multiple versions of the encoded bitstream completely simultaneously. The encoded bitstream may be a compressed video bitstream, and the transcoder is located on the server of the video stream, and is used to provide multiple versions of the video bitstream to different clients at different rates at the same time.

In an application where the encoded bitstream is a compressed video bitstream, the overhead data may include video object sequence (VOS), video object (VO), video object layer (VOL), video object plane (VOP), At least one of the video object plane group (GOV) and motion vector (MV) data. For example, the overhead data can be extracted from the packet header contained in the encoded bit stream. It should be understood that although the terms used herein (such as VO, VOL, VOP, and GOV) may be specific to a specific standard (such as MEPG-4), the concepts contained therein can also be found in other standards. of. Therefore, the terms in a specific standard used here and in the claims are not meant to be limited to this standard, but should be broadly interpreted as terms used in any standard or technology related to the present invention.

In one embodiment, the rate control processor re-encodes the at least partially decoded bitstream multiple times to sequentially provide multiple re-encoded bitstreams. More specifically, the rate control processor may re-encode at least part of the decoded bit stream to obtain a plurality of re-encoded bit streams.

Generally, the encoded bit stream is received at the first rate. The rate control processor may operate at a second rate that is N times the first rate, where N is the number of bit streams that have been re-encoded. In this way, the original compressed video bitstream is used to provide all re-encoded bitstreams at the same time.

In a preferred embodiment, the first function that does not affect the rate of the re-encoded bitstream is performed only once on the encoded bitstream. In order to obtain each re-encoded bit stream, the second function that affects the rate is performed separately.

In an embodiment where the encoded bitstream is a compressed video bitstream, the first function may include at least one of variable-length decoding and dequantization. The second function may include at least one of requantization, variable-length coding, and motion compensation.

The rate control processor may include multiple encoders working in parallel to generate multiple re-encoded bit streams. The re-encoded bit stream can be provided in the form of a variable bit rate stream and/or a constant bit rate stream according to the request of the client.

In order to effectively provide multiple output streams of different rates, the processor cycles of the rate control processor can be monitored, and at least one processing step can be skipped if the number of available processing cycles for completing the rate control operation is insufficient. For example, when the encoded bit stream composes the compressed video bit stream, in the case that the number of available processing cycles for completing the rate control operation is insufficient, the bi-directionally predicted (B) frame (bi-directionally predicted (B) Frame) can skip at least one of the motion compensation step and the discrete cosine transform (DCT) step.

In another optimized technique, the rate control processor may re-encode the at least partially decoded bit stream multiple times to provide multiple re-encoded bit streams in sequence, and no more than all the bit streams are generated. During the processing of multiple re-encoded bitstreams, at least one processing step is selectively skipped.

The invention also provides a corresponding method.

Description of the drawings

Figure 1 is a block diagram of a client-server network; Figure 2 illustrates the concept of the same server providing services to multiple clients via a network (such as the Internet); Figure 3 is a block diagram of a prior art implementation, where Multiple code converters are used to provide multiple output streams from the compressed video bit stream; Figure 4 is a block diagram of the prior art implementation, in which one code converter is used to process the same Compressed video bitstream; Figure 5 is a block diagram illustrating the present invention, in which multiple streams of different rates are simultaneously generated from the compressed video bitstream as input; and; Figure 6 is a block diagram of an embodiment, in which The macroblock is decoded only once and re-encoded by different encoders to generate multiple streams of different rates.

detailed description

In the present invention, a code converter device is provided for the coded bit stream for simultaneously providing multiple coded bit streams of different rates from a single input bit stream (such as a compressed video bit stream).

The basic principle of the present invention is to provide a multi-rate transcoder. This transcoder can obtain a single compressed video bitstream as input and generate multiple streams of different rates completely at the same time according to the requests of different clients.

Figure 1 illustrates a simple network application in which a client 12 communicates with a server via a communication path 14. This client-server network is well known in this technology. Although the client generally communicates with only one server at a time, it usually sends a request to the server and receives the information data stream in the response, but the server usually communicates with multiple clients at the same time. This is illustrated in Figure 2 where the server 10 communicates with multiple clients 12A, 12B...12C via the Internet 16, for example. The client can be any of a variety of different devices, including, but not limited to: personal computers (PC), wireless Internet devices, TV set-top boxes, point-of-sale terminals, wageng terminals, user equipment (consumer appliance), photocopiers, printers, industrial equipment, automated systems, and similar equipment. Generally, different clients (even those devices of the same general type (such as set-top boxes)) have different processing capabilities and requirements for receiving data from the server. These data are usually communicated from the server to the client in the form of packetized data streams that comply with industry-wide standards. An example of such a standard is the standard for the communication of compressed digital video signals proposed by the Moving Picture Experts Group, that is, MPEG-2. There are other existing standards and new standards under development, and the present invention is not limited to any such transmission standard.

In an embodiment of a television set as an application of the present invention, a client (for example, a cable or satellite television user) may request a movie. As part of the request, the server may receive information about the network status of the client side of the communication path, and then can encode and convert the pre-compressed stream carrying the requested movie according to the bandwidth of the client side. When the server receives multiple requests for the same movie from different clients (each client has a different bandwidth requirement), the transcoder is required to provide multiple instances of the movie according to different requested bandwidths.

As shown in Figure 3, a straightforward solution is to encode and convert the same input video stream at different rates requested by different clients. One disadvantage of this solution is that the server needs to have as many transcoders as the clients it serves. This is unrealistic.

Another solution is that the server's transcoder processes the same video stream every time it is requested by a different user (but there is only one request at a time). In such an embodiment, other clients will have to wait for their turn to watch the requested movie or other service request before the server finishes processing the client's request. Such an embodiment is shown in FIG. 4, where the transcoder 20 first provides stream 1 from the compressed video bitstream, and then stream 2 after a delay 22, and then at another delay 22 (or one based on the request interval). Stream N is provided after a series of delays), and so on. Similarly, since there is no real-time service for the request, this is also unrealistic.

The present invention solves the above-mentioned problems by providing a multi-rate transcoder that processes a single compressed bit stream (such as a video bit stream) and simultaneously responds to requests from multiple clients. Generate multiple streams at different rates. Figure 5 shows a possible structure of the multi-rate transcoder in the present invention. Assume that the pre-compressed video bitstream arrives. This stream is applied to the header processor 30 and the variable-length decoder 32 (assuming the bit stream contains variable-length packets). At the header processor 30, only the header segments of the pre-compressed video bitstream (such as video object sequence (VOS), video object (VO), video object layer (VOL), video object plane group (GOV) And the Video Object Plane (VOP) is scanned once for subsequent multiplexing in the multiplexer 60 and generating streams of various rates.

Then, at the macro block (MB) level of video processing (for example, MB is defined in the widely available MPEG-2 standard), since the motion vector (MV) field is reused, they can simply be copied Into the new stream. This is shown in module 30, which treats the MV domain in a similar way to the header information described above.

For illustrative purposes, assume that N streams need to be generated at N different rates requested by N clients. At the MB/module level, those shaded modules (including: quantizer (Q(i)) 42, dequantizer (DeQ(i)) 46, variable length encoder ( VLC) 44, rate control subsystem 36, discrete cosine transform processor (DCT) 56 and inverse DCT (IDCT) 50, and motion compensation engine (MC) 54) will be executed N times, once for each requested rate . As shown in FIG. 5, the processor 40 controls the rate control function. As is well known in the art, the processor may be a microprocessor. The authorized US patent application No. 09/198,867 filed on November 28, 1998 for "rate control of MPEG transcoder without prior knowledge of picture type" discloses the rate that can be used in conjunction with the present invention. An example of the application of the control processor.

In the present invention, simple function modules such as variable length decoder (VLD) 32 and non-quantizer (DeQ) 38 are executed only once. In particular, each input MB in the compressed bit will pass through VLD32 and DeQ38. If the MB is in intraframe mode, it is directly transmitted to the quantization module (Q(i)) 42 via the gate circuit 40. If it is in the inter-frame mode, the MB in the DCT domain first performs motion compensation through the gate circuit 40. The gate circuit is processed by DeQ(i) 46, summer 48, and IDCT50, and the motion compensator (MC ) 54 and DCT 56 receive the quantization error from the previous frame stored in the frame buffer 52 before performing subsequent processing in a conventional manner. For each new rate request of client i, the rate control 36 associated with the signal from the processor 34 will assign a new quantization factor Q(i) to MB. The quantized coefficient error coded at 44 is variable length, which generates a new stream at a new rate. In the feedback loop for the motion compensation stage, the unquantized coefficient error at 46 is subtracted from the coefficient error at 48 and stored in the frame buffer 52, each of which is used for a different rate of streaming . It should be noted that motion compensation uses the same motion vector (MV) decoded from the VLD, but uses different reference (quantization error) frames for different new streams.

The various new functions of the proposed multi-rate transcoder are listed below: 1. The multi-rate transcoder can be applied to almost all video coding standards, such as MPEG1, 2, 4 and H.261, 3. 3+, L.

2. The principle of the proposed multi-rate transcoder is to perform one decoding operation and multiple encoding operations. This principle can be used in almost all transcoders (such as cascaded transcoders with or without reuse of the MV domain, and various shortcut transcoders). For example, in FIG. 6, the input MB is decoded only once by the decoder 62, but it is re-encoded by different encoders 64A, 64B, ... 64N, so that it is combined with the data source from the multiplexer 60 after passing through the multiplexer 60. The common head data of the head processor 30 is multiplexed to generate different streams.

3. The output stream can be either constant bit rate (CBR) or variable bit rate (VBR), depending on the client's request.

4. In a limited CPU cycle, multi-rate code converters can choose to abandon some of the functional modules in Figure 5. For example, if there is a lack of CPU cycles, one or more modules such as DCT56, MC54, frame buffer 52, IDCT50, adder 48, and/or DeQ(i)46 may not be executed when processing bidirectional prediction (B) frames. Can significantly save CPU cycles. Since the B frame is never used as a reference frame, it will not have much impact on the quality of the entire video and will not propagate errors. Multi-rate code converters can also selectively stop executing some functional modules, depending on the priority.

The following lists the advantages of the invented multi-rate transcoder compared with the prior art: 1. The server can provide multiple new streams of different rates at the same time.

2. Only scan the header (VOS, VO, VOL, GOV and VOP) once and copy it directly to a different new stream.

3. It only needs to execute decoding function modules (such as VLD and DeQ) once for different streams of different rates.

4. For different streams, the multi-rate transcoder can choose to execute all the functional modules and can also choose to execute some of the functional modules.

It should be understood that the present invention provides a multi-rate transcoder, which provides multiple output streams of different rates from a single input signal stream. The output stream can be provided at the same time without repeating the existing coding structure. The common processing of input streams is provided for the possibility of expansion of all different output streams. Processing functions that are different for all N output streams can be executed at N times the rate of processing a single stream, so that all streams can be provided in real time. Compared with the multi-rate transcoder solution of the prior art, the hardware and software requirements of the present invention are reduced, and the processing requirements are also reduced compared with the single transcoder solution.

Although the present invention is described in conjunction with specific embodiments, it should be understood that various changes and modifications can be made to it without departing from the scope of the present invention set forth in the following claims.