Transmitting and receiving real-time data

ES 2 331 111 T3

DESCRIPTION

Transmission and reception of data in real time.

The invention is in the field of time-sensitive data management over packet-switched networks, and more particularly of the transmission and reception of video data over the Internet.

The invention relates to a method of providing a streaming video service to a customer over a packet network while reducing the startup delay usually associated with preparing a data buffer while maintaining the use of a data buffer. buffer memory. The invention also relates to a method of controlling the transmission speed of streaming video to adapt to network congestion.

Traditionally, the Internet has supported traffic such as FTP, email, and web page browsing, in which the overall delay does not inherently detract from the final presentation of the media. The advent of faster-processing multimedia PCs has driven the distribution of multimedia, including video, over the Internet. However, time-sensitive applications require continuous data channels, with guaranteed quality of service, and high bandwidth, which apparently does not match the packet-based nature of the Internet and has the potential to disrupt transmissions. with unacceptable jitter of the packets, that is, the variation in the times between packet arrivals, caused by changeable routing and variability of delivery speeds due to congestion. Today's commercial streaming technologies overcome jitter by building a large buffer (5 to 30 seconds) before starting to play video material. This startup delay is not optimal for a user, who may have to wait for this period before realizing that the requested content is incorrect; and generally distracts users from the multimedia presentation experience.

WO00 / 01151 describes a system in which the frame rate of a sequence of images can be varied. In one embodiment, the image sequence is encoded and stored at different frame rates (eg, 30fps, 25fps, 20fps). In an alternative, only the motion information, eg motion vectors, is stored for the other frame rates.

US Patent No. 5,822,524, a client machine requests multimedia files, such as compressed video clips, from a server. The transmission uses digital data packets. In the case of video files, the packet headers identify the video frame and the sequence number of each packet derived from the frame. The transmission timing is not based on a continuous and stable stream of bytes or an average value of bytes to be transmitted. Instead, in the case of video, the frame rate determines normal transmission and one frame is transmitted during each frame time. The client agent has a normal packet buffer, typically containing 1 to 5 video frames. The baud rate is adjusted to keep that buffer full within its normal range. The timing information required for transmission is stored in a separate index file, associated with each media file.

In US Patent No. 5,918,020, a pacing system is implemented in a data processing system to allow a client to regulate the pacing of a streaming server in a stable manner such that a level of The filling of a client buffer oscillates around an individual threshold value. In implementing the pacing mechanism, the streaming server transmits data at a slightly faster rate than it was encoded. Subsequently, a decoder circuit at the client, or receiver, uses the transmitted data at the scrambled rate. This will gradually increase the use of buffers on the client. When the buffer utilization reaches a threshold level, the client provides a pacing message to the server. When the pacing message is received, the server delays sending data for a period of time sufficient to cause the client's buffer utilization to drop below a threshold level.

According to a first aspect of the present invention, there is provided a method of operating a real-time communication apparatus as set forth in claim 1.

According to a second aspect of the present invention, there is provided a real-time communication apparatus as set forth in claim 4.

Other preferred features are as set forth in the dependent claims.

It is desirable to use as much of the available bandwidth of a link as possible to transmit data since higher video data bit rates result in better quality playback. However, the loss of data on the network causes a severe deterioration in service - clearly outweighing the benefits of increasing the bit rate. For example, with predictive coding schemes such as H.263 and MPEG, reception of half of a 500 kbit video stream is likely^-1 deliver much worse quality than the entirety of a 250 kbit stream^-1. For this reason it is important to reduce the transmission speed in a controlled way, rather than letting data be lost on the network. The Internet protocol TCP has a built-in control mechanism with which the data transmission speed is increased uniformly.

ES 2 331 111 T3 until packet loss is detected, after which the data rate is reduced. Then the data rate is increased again until packet loss occurs again. A variable transmission rate is said to be elastic, and applications that can control the speed of data transmission in response to network conditions are said to be compatible with TCP. It is desirable to provide video data in a TCP-compliant manner so that the greatest available amount of bandwidth is utilized at any particular time. An additional benefit of TCP-compliant data delivery is that network congestion is managed as individual applications themselves reduce data rates until each has a fair share of bandwidth.

Conventional compression technologies, such as MPEG4 or H.263, can be manipulated to display TCP-compatible behavior, see, for example, applicant's pending patent application number GB 9928023.2. However, this solution requires a dedicated, high-speed PC for streaming video. Transcoding a stream of encoded data from a high bit rate to a low bit rate when network congestion is detected further suffers from the problem of being computationally demanding. Another approach is to use a layered arrangement of video streams, whereby quality adjustment is achieved by adding or removing layers of the video stream. The disadvantage of this method is that it is inefficient, since a certain proportion of the available bandwidth must be allocated to instructions to integrate the layers.

Embodiments of the invention will now be described, by way of example only, with reference to the figures, in which:

Figure 1 is a schematic overview of the relationship between the encoder, the video streaming device, and clients;

Figure 2 shows the arrangement of the video streaming device;

Figure 3 shows the layout of a customer; and Figure 4 shows the stepwise operation of an embodiment of the present invention.

As shown in Figure 1, a first embodiment of the present invention consists of a compressed video data source, encoder 1, which encodes data at both a low bit rate R_L, which can have a value of, for example, 500 kbits^-1, as at a high bit rate R_H, of, for example, 1,500 kbits ¹. The compression codec used is H.263 although it can also be any other codec, such as MPEG4. The encoder takes "live" video data as input, for example a broadcast of a sporting event.

The two encoded data streams are transmitted via independent logical connections to the video stream device 2 at a transmission rate T_AND. The video streaming device 2 may be in the same premises as the encoder 1 and connected via an intranet. The video streaming device 2 runs on a server computer, for example one comprising a 700 MHz Pentium III, with 256 MB RAM, and having Internet access.

A video viewer, so far referred to as a client, running on a PC (a, b, c, etc., in Figure 1) properly configured for Internet access, can be connected to the device streaming video over the Internet, and thus the customer can access the content. A suitable PC terminal is a 266 MHz Pentium II notebook PC. The video streaming device 2 can support a large number of clients (typically up to 1,000) viewing the same video stream. For a live broadcast broadcast, encoder 1 will transmit at a transmission rate TE which is real time. The two data streams RL and RH encoded at different bit rates offer different quality video playback, although each data stream has the same bit rate, TE. Data must be decoded at this rate for the program to play in real time.

Figure 2 shows the arrangement of the video streaming device 2. Low-quality encoded video data, encoded at a low bit rate RL, and high-quality encoded video data, encoded at a high bit rate R, are received at the input connections 21 and 22 respectively._H, from the encoder 2, and they are fed respectively to the buffers 23 and 24. It should be noted that one buffer is provided per channel of encoded video data that is received by the video streaming device 2. From each buffer 23, 24 encoded video data is read through a switch 26 which selects which stream of encoded video data is to be sent to the output connection 27. A buffer manager 25 is provided which is capable of controlling the rate at which data is read from each of the buffers 23, 24 and thus defines the transmission speed T_S video streaming device 2. The buffer manager is also in connection with switch 26 and is also capable of receiving signals from connection 28. TS is selected by varying the time delay between transmission of each packet, such that TS can be less than , equal to or greater than the encoder transmission rate TE. The experts in the field will perceive that the limiting factor on the sustainability of the transmission in which T_S> T_AND is the size of the buffer 23, 24 such that a memory

ES 2 331 111 T3 intermediate of size S kbits will be able to support a transmission speed of T_S = 2T_AND for a time twice as long as a buffer size S / 2 kbits. Through the control of both switch 26 and the baud rate T_S, the buffer manager can control the bit rate that is output from the video streaming device 2 at two levels: by adjusting the transmission rate T_Sfine control of the bit rate is achieved, and by switching between the two streams of data encoded at the bit rates R_L and R_H bit rate control can be achieved at a rough level. The buffer manager 25 makes adjustments to TS or switches the output between buffers in response to signals received from connection 28.

Figure 3 shows the client layout running on a PC 3a, b, c, and so on. The encoded video data that is sent from the video streaming device 2 is received at the client through a connection 27 and its integrity is checked by means of a packet loss detector 31. The data is then sent to a client buffer 32 that is sized to absorb fluctuations in network throughput. The client buffer 32 is directly connected to a decoder 33 and, from there, decoded data is sent to be displayed on the client screen (not shown). A client status monitor 34 is connected to the packet loss detector 31 and to the client buffer 32. The client status monitor 34 can send signals through the connection 28.

The packet loss detector 31 monitors incoming packets. If a packet loss is detected, then a signal is sent to the client status monitor 34, which informs the buffer manager at the video streaming device 2 over connection 28. The lost packets they can be re-transmitted. The buffer manager 25 uniformly increases the transmission rate TS until a consistent pattern of packet loss occurs, indicating that the maximum bandwidth is being used. In the interest of maintaining a congestion-free network, the transmission speed T_S can be exponentially reduced. The client status monitor 34 monitors the volume of data in the client buffer 32 such that a signal is sent through the connection 28 to the buffer manager 25 in the video streaming device 2 when the client buffer 32 becomes sufficiently full of data.

The system of video streaming device 2 and client 3 as described above allows an attractive video stream for the user, that is, the client buffer 32 enables the existence of a certain quality of the video in spite of variations in network conditions, which, on the other hand, could have a detrimental effect on the overall perceived quality of the media.

The operation of the present embodiment of the invention will now be described with reference to Figure 4.

The video streaming device 2 is initialized, which involves filling the buffers 23, 24 with an amount of data from the encoder 1. For a live broadcast, data is constantly fed to the buffers 23 , 24 and are subsequently discarded after an amount of time defined by the size of the buffer and the quality of data being received.

A PC running browser software can be used to browse web pages on the Internet to select a link for, for example, a live broadcast on a site hosted by the entity that provides streaming video. The user interested in viewing the particular clip or broadcast clicks (selects) the link. The navigation software detects that streaming video data has been requested and launches the video display client software embodying client 3. Client 3 issues a "send data" command via connection 28 to the manager Buffer 25, which sets switch 26 to read encoded video data from low bit rate data buffer 23 and requests a transmission rate of T_S = 2T_AND. The data is transmitted to the data connection 27 and from there to the client 3. Using the coding bit rate of the example, cited above, of 500 kbits^-1 stop_L, data flows over the network towards the client at a rate of 1,000 kbits^-1.

The client 3 receives the encoded video data and sends it through the packet loss detector 31 to the client buffer 32 which is fed at the rate of 2TE. When data is detected in buffer 32, the encoded video data is immediately read to decoder 33 at a rate of T_AND. Therefore, the buffer 32 fills at a TE rate while the decoded data from the decoder 33 is displayed. Thus, the user is provided with video images without having to wait for the client buffer 32 to fill. .

Client monitor 34 waits for the amount of RL data in client buffer 32 to reach a specified level, after which a "switch buffer" command is sent to buffer manager 25 on device 2 streaming video through connection 28. Next, the buffer manager 25 switches the data flow from the low bit rate data buffer 23 to the high bit rate data buffer 24 and commands the transmission at a rate TS = TE. Using the example coding rate cited above, data is transmitted at 1,500 kbit / s over the network^-1.

ES 2 331 111 T3

Next, the client buffer 32 will begin to fill with high quality data that will lag behind the low quality data. After a period of time, the data in R_H they will begin to be read to the decoder 33, after which the user will perceive an increase in image quality. At this point, client 3 has a full buffer and the user is viewing images of a quality that is consistent with the capacity of the network link.

The video streaming device 2 can support multiple clients (typically 1,000). Each client is initially given a unique read point for the boot phase, after which, after the client's buffer 32 equilibrium has been reached and the video streaming device 2 is supplying data from high bit rate from buffer 24, the read point can be amalgamated with other client read points. Read points may need to be transferred as discrepancies in network capacity demand increase or decrease transmission speed for a particular customer.

Those skilled in the art will appreciate that the low bit rate data buffer 23 should be of a size that allows data to be read from it at a 2T rate_AND for a period of time that is long enough to provide the client buffer 32 with an adequate amount of data. For example, to buffer 500 kbit data^-1 for 5 seconds on client 3, video streaming device 2 should supply 1,000 kbits of data for 5 seconds, of which 500 kbits will be consumed by decoder 33 per second and 500 kbits will be accumulated in memory intermediate per second until 5 seconds have elapsed. Therefore, the low bit rate data buffer must be able to hold at least 5 Mbits of data (5 x 1000 kbits), or simply more than 0.5 Mb.

Those of skill will appreciate that there are problems associated with "tapping" in a stream of encoded data when data is initially being read from a buffer. The compression technology typically used by encoder 1 involved the encoding of a frame of video data, called the anchor frame or I frame, and from this frame an estimate is made about what the next frame will look like, called frame B a this esteemed plot. In this way, the amount of data representing a series of frames can be greatly reduced. However, if the first frame to be read from any of the data buffers 23, 24 is a B frame, then the first decoded data frames may be unintelligible as the decoder tries to reconstruct frames based on an estimate. In a further embodiment of the invention, an additional data buffer is supplied in parallel with the data buffers 23, 24 consisting only of I frames. The first frame to be transmitted is read from the I frame buffer and thus provides the decoder with a reliable point from which to start decoding. Data is then switched to be read from any of the data buffers 23, 24.

The system enables user-friendly video streaming, that is, video quality does not fluctuate rapidly as network conditions vary, which can have a detrimental effect on the overall perceived quality of the media. In the event that the client reports a packet loss, the system can exponentially reduce its transmission speed. This does not necessarily result in immediate switching of the video source, as there may be data buffered on the client. Immediately after packet loss, the transmission rate may be lower than the encoding rate, and the client is supplementing received data with buffered data in order to meet the demands of the video decoder, with the result that the client's buffer is getting empty. In the event of isolated packet loss, the system can gradually raise the transmission rate again, initially slowing down the rate at which the client buffer is being flushed before finally returning to a full state of the client.

Those skilled in the art will appreciate that the ability to transmit data at varying rates over a period of time allows streaming data to be elastic and enables TCP-compatible transmission. A sustained packet loss detection by the packet loss detector 31 is indicative of network congestion. The buffer manager 25 in the video streaming device 2 reacts to a packet loss notification by ordering a reduction in the data rate from the high bit rate data buffer 24. The high bit rate data buffer 24 should be of an appropriate size to cope with such an event. If the packet loss persists at the reduced transmission rate for longer than the high bit rate data buffer can handle, then the buffer manager 25 will switch to supply data from the data buffer 23. low bit rate. Efficient management protocols are necessary to avoid rapid switching between data buffers 23 and 24 when the data capacity of the network fluctuates, as this will cause changes in the perceived quality of the reproduced video. Although a user will tolerate poor quality playback, rapid changes in quality can be irritating to the user.

There is no limit on the number of encoded data streams that can be provided to the video streaming device. In this way, maximum bandwidth utilization can be achieved: starting with reading data from a low bit rate data buffer, the transmission speed is increased. After observing that no packets are lost at this baud rate, the output is switched to a higher bit rate data buffer, after which the baud rate is increased. If this transmission speed does not meet any obstacle, then it can be switched to a

ES 2 331 111 T3 data buffer of an even higher bit rate, and so on until the maximum bandwidth is used.

The buffer manager 25 located in the video streaming device 2 is empowered to decide how to adjust the transmission speed T_S and when to switch buffers. In the same way, instructions can be sent from the client 3 to the video streaming device 2 about the transmission rate TS and from which buffer to feed data. The location of the buffer manager 25 in the described embodiments has been selected because it is practical to locate the control center close to the center that is responsible for billing for the service, which in this case is the ISP.

The video data example is selected as a multimedia data example to illustrate the above embodiments. The invention is equally suitable for any other form of time-sensitive data, such as audio data or a multimedia presentation.

In the embodiment described above, encoder 1 supplies data. Similarly, a library of program data files, for example a feature film library, which can be accessed when required, may contain compressed video data.

The video streaming device 2 may be remote from the encoder 1 such that the video streaming device 2 and the encoder 1 are connected via the Internet. The video streaming device 2 is likely to be exploited by an Internet Service Provider (ISP) and a remote connection of the video streaming device 2 and encoder 1 would allow the ISP to make content available for the user. client from many encoders.