Distributed internet protocol-based real-time multimedia streaming architecture

Description: TECHNICAL FIELD [Detailed description of the invention]

【0001】

[Technical field to which the invention belongs]

The present invention generally relates to a distributed media transmission system, which is a networked multimedia system. In particular, the present invention relates to a distributed media transmission system that transmits media selections to one or more media clients on a multicasting network.

【0002】

[Problems to be solved by conventional techniques and inventions]

With the explosive development of the Internet, the Internet and other Internets are used to transmit multimedia selections that point to audio and video when one medium is selected from multiple media that are material data such as video and audio. There is also increasing interest in using protocol-based networks. Interactive television, movie-on-demand, and other multimedia push technologies are among the very promising applications.

【0003】

The Internet is a connectionless network that provides best effort transmission services. Multiple packets of data are routinely routed as datagrams carrying the address of the specified receiver. No specific connection between the transmitter and receiver is required as all nodes on the network include the original function of routing datagrams from node to node until transmission is complete. This datagram packet scheme is constructed as a best effort transmission system where the transmission of datagram packets is not guaranteed. Datagram packets may be transmitted over different routes with different efforts to increase the likelihood of transmission. Therefore, if one node on the network is experiencing congestion, the next datagram may be routed by another route to avoid the congested node. This means that the datagram packet of data does not actually have a guaranteed arrival time. Even multiple packets corresponding to a single message may be received in the wrong order. This fact clearly affects how certain multimedia data is transmitted.

【0004】

In many cases, multimedia data requires real-time transmission. In the case of audio or video data, a data stream representing a particular media selection will allow the user to "live" the data of the audio or video selection when the audio or video selection is transmitted. Or it needs to be transmitted in an appropriate time series so that it can be played back in real time). Clearly, if a datagram packet is improperly transmitted because it takes a different transmission route, playback on a multimedia client (eg, the user's interactive TV) will be confusing.

【0005】

Real-Time Protocol (or RTP) is the current de facto standard protocol for transmitting real-time content over the Internet. The real-time protocol replaces the traditional Transmission Control Protocol (or TCP) with a framework that real-time applications can use directly to transfer data. Currently, the RTP standard supports the first type of message, that is, the one that transmits the content data or streaming data of the media. Typically, an independent protocol, such as Real-Time Control Protocol (or RTCP), works with RTP to pass control messages for managing sessions, adjusting rates, and so on. Used.

【0006】

In addition, conventional systems use a hierarchical coding method that treats some components with greater importance than other components. Therefore, traditional systems typically need to spend considerable resources to ensure that more important components are always transmitted.

【0007】

In addition, real-time protocols can be used to carry multimedia streaming data over computer networks, while existing architectures use best-effort network services, such as network services provided by the Internet. It does not provide sufficient robustness to provide high quality presentations.

【0008】

An object of the present invention is to solve the above problems and to realize media selection, audio and / or video, etc. in real time with higher service quality without relying on transmission retry technology that causes delay. An object of the present invention is to provide a distributed media transmission system capable of transmitting multimedia data.

【0009】

[Means for solving problems]

The present invention solves the above problems by using a distributed media push architecture that is capable of supplying duplicate streaming data from multiple sources and streaming data across multiple distributed paths. The media push engine has an associated media storage unit that stores streaming data as a non-hierarchical set of substream components. The above components may be reconstructed from less than all components into a reconstructed stream so that as the number of components used for reconstruction increases, so does the quality of the stream being reconstructed. it can.

【0010】

Further, the media transmission system of the present invention uses multiple description coding (also referred to as MDC), which is a non-hierarchical coding method and treats all components equally. Therefore, no special resources need to be allocated to ensure that a given set of substream components is transmitted. Of course, as the number of components transmitted increases, so does the quality achieved, on the other hand, unlike traditional hierarchical coding methods, the missing one packet gives a sharp signal quality. It will not fall.

【0011】

The distributed media transmission system also uses a distributed authorization control system. The media client communicates with a single authorization control unit and requests a given media selection, after which the authorization control decisions are handled in a distributed manner by the media push engine itself. The authorization control unit communicates the request to a plurality of media push engines distributed over the network, which individually determine whether they can participate in a multicasting session. Therefore, each of the individual media push engines evaluates local traffic congestion and determines whether the requested data stream can be delivered. Therefore, the authorization control unit is not required to directly determine which media push engine should participate in the multicast group session. The authorization control unit simply assigns a multicast group session address and then allows the authorization process to proceed autonomously in a distributed manner.

【0012】

The distributed media transmission system according to claim 1 according to the present invention is a distributed media transmission system that transmits a media selection to a media client on a multicasting network, and the distributed media transmission system is the network. A plurality of media push engines accessible via the above and a permission control system accessible via the above network, the plurality of media push engines include the first and second media push engines, and the first And the second media push engine each have an associated media storage unit that stores streaming data representing a given media selection available for transmission, the media storage unit substreaming the streaming data. The non-hierarchical set of components in the substream is configured to be stored as a non-hierarchical set of, so that the quality of the reconstructed stream increases as the number of components used for reconstruction increases. It is possible to reconstruct one reconstructed stream from less than the number of all components, and each of the above substream components requires a combination with another important substream component. Without having to be able to contribute equally to providing a given media selection, the authorization control system includes a catalog that stores the identifiers of the media selections available for transmission by each of the media push engines. , The authorization control system determines that the predetermined media selection is available by comparing the request with the contents of the catalog in response to a request from the media client for the predetermined media selection. And, it operates to open a multicast group session between the media client, the first media push engine, and the second media push engine, thereby causing the multicast group session. participateEach of the plurality of media push engines uses the substream component corresponding to the predetermined media selection to the media client without using the authorization control system as an intermediary to transmit the substream component. The first media push engine and the second media push engine provide the substream components via the network to the network so that they can be transmitted to and reconfigured by the media client. Connected to the network at different addressable locations selected to establish the availability of different routes to send to the media client, thereby reducing network congestion to the performance of the media transmission system. It is characterized by reducing the effect on the network.

【0013】

【0014】

Further, the distributed media transmission system according to claim 2 is characterized in that, in the distributed media transmission system according to claim 1, the plurality of media push engines communicate with the network via different communication paths. ..

【0015】

The distributed media transmission system according to claim 3 is the distributed media transmission system according to claim 1, wherein the network is a connectionless network that provides the best effort transmission service.

【0016】

Further, the distributed media transmission system according to claim 4 is the distributed media transmission system according to claim 1, wherein the network is the Internet.

【0017】

Further, in the distributed media transmission system according to claim 5, the media client and the plurality of media push engines participating in the multicast group session transfer data in the distributed media transmission system according to claim 1. It features the use of a real-time transport protocol (RTP) for.

【0018】

Further, in the distributed media transmission system according to claim 6, the media client and the plurality of media push engines participating in the multicast group session manage the sessions in the distributed media transmission system according to claim 1. It features the use of a real-time control protocol (RTCP) for.

【0019】

The distributed media transmission system according to claim 7 is characterized in that, in the distributed media transmission system according to claim 1, at least a part of the components of the substream is replicated over several media push engines. And.

【0020】

【0021】

Further, the distributed media transmission system according to claim 8 further includes a data buffer system related to the media client that stores substream components before reconstruction in the distributed media transmission system according to claim 1. It is characterized by being prepared.

【0022】

Further, the distributed media transmission system according to claim 9 is the distributed media transmission system according to claim 1, wherein the permission control system further responds to a request from the media client to terminate a multicast group session. It is characterized in that it operates so as to instruct all media push engines participating in the multicast group session to end the multicast group session.

【0023】

Further, the distributed media transmission system according to claim 10 is the distributed media transmission system according to claim 1, wherein the permission control system is a multicast session address used to call a multicast group session. Includes a permission control unit that holds a pool, wherein the permission control unit assigns a specified multicast session address selected from the pool for use by the multicast group session. And.

【0024】

Further, in the distributed media transmission system according to claim 11, the authorization control unit responds to a request from the media client to terminate a multicast group session in the distributed media transmission system according to claim 10. , It is characterized in that it further operates to return the specified multicast session address to the above pool.

【0025】

【0026】

Further, the distributed media transmission system according to claim 12 is the distributed media transmission system according to claim 1 between the media client and all media push engines participating in the multicast group session. It is characterized by the existence of a unicast flow of datagrams containing real-time stream component data.

【0027】

For a more complete understanding of the present invention, its purpose and its advantages, refer to the following embodiments of the invention and the accompanying drawings.

【0028】

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments according to the present invention will be described with reference to the drawings.

【0029】

With reference to FIG. 1, an example of a distributed network multimedia system is illustrated with reference numeral 10. Multiple media push engines 12a, 12b, 12c, 12d, 12e are accessible via the multicasting network 14 (or network 14). The preferred embodiment is the Internet Protocol (Internet). Protocol; or IP. ) Is designed to operate on the network 14, but the principles of the embodiments according to the present invention can also be easily extended for networking using other protocols. The network 14 can also be evaluated by one or more multimedia clients 16a and 16b, as shown. The authorization control unit 18 accessible via the network 14 performs the exact authorization control function and starts or releases the multicast group session first. The authorization control unit 18 includes a catalog service system 20. The catalog service system 20 includes a database record showing which multimedia selections are available for transmission by the media push engines 12a-12e. The permission control process is included in releasing the multicast group session, but is actually executed by the distribution method described in detail later.

【0030】

The distributed media transmission system 10 responds to a transmission request from the multimedia client 16a or 16b with the multimedia client 16a or 16b and those media pushes having the requested media selection available for transmission. Respond by opening a multimedia group session between engines 12a-12e. Typically, a number of media push engines 12a-12e participate in the simultaneous transmission of streaming data corresponding to the required selection data. The multimedia client 16a or 16b is a user host that performs the presentation function. It reconstructs one final stream from the various stream components transmitted by the participating media push engines described above. Each media push engine 12a-12e has its own data storage for stream components, and their data storage system is suitable distributed to provide wearable and transparent storage and retrieval capabilities. Controlled by the file system. Here, the distributed file system supplies a file storage service to the media push engine. Each media push engine 12a to 12e has its own data storage function (data storage unit) as described above, but those data storage functions are assisted by the distributed file system.

【0031】

An important concept of the distributed media transmission system 10 is how streaming data is stored in a plurality of media push engines 12a to 12e. Instead of resembling conventional systems that store multimedia data in a hierarchical manner, one embodiment of the invention uses a non-hierarchical coding system referred to herein as Multiple Description Coding (MDC). To do. Multiple descriptive coding decomposes video and / or audio streams into substreams called components. Each component is then encoded and transmitted over the network independently of all other components. The multimedia client 16a and 16b client software can build one reconstructed stream from any subset of multiple components. Therefore, the reconstructed stream can be constructed from fewer components than all components. The higher the number of components used in the reconstruction, the higher the quality of the reconstructed stream.

【0032】

By using this non-hierarchical coding method to transmit streaming data over essentially unreliable networks, it provides incredibly robust media transmission, especially with numerous media push engines. Provides incredibly robust media transmission when participating in transmission. More fully, the media push engines 12a-12e control the authorization process of their own multicast group sessions in a distributed manner and media if needed to maintain higher quality of service. The push engine is added to or removed from the multicast group session. Therefore, when the multicasting network 14 shows low traffic congestion, only a few media push engines may be needed to supply all the components of the MDC-encoded stream. Even if some components are not transmitted from time to time, the multimedia client will nevertheless be able to reconstruct the stream for presentation (albeit with slightly lower quality). If the traffic congestion on network 14 is very high, the media push engine will negotiate with another media push engine to add an additional media push engine. Since the authorization control process is distributed, the individual media push engines 12a-12e can determine their own local traffic congestion and therefore, depending on the local traffic situation, the group. Participate in or do not participate in the session.

【0033】

FIG. 2 is a network diagram showing in more detail how the multiple description coding method works. In FIG. 2, the two multimedia streams represented by the symbols X and Y are stored across the plurality of media push engines 12a-12e. These streams are distinguished by a subscript X₁, X₂, ..., X_nAnd Y₁, Y₂, ..., Y_nIt is decomposed into substream components represented by. Note that the substream components stored across multiple media push engines 12a-12e are not necessarily the same for each media push engine 12a-12e. Therefore, the media push engine 12a is component X₁, X₆And Y₁Remember. Similarly, the media push engine 12b is component X₂, X₇And Y₂Remember.

【0034】

The multimedia clients 16a and 16b reconstruct the data stream of interest by combining the appropriate substreams in the appropriate order. Thus, the multimedia client 16a reconfigures stream X as shown in FIG. 2, while the multimedia client 16b reconfigures stream Y as shown in FIG. For multimedia clients 16a and 16b, it is not critical for individual substream components to arrive from different media push engines via different paths.

【0035】

A preferred multiplex coding method is constructed as illustrated in FIG. The original multimedia data stream (eg, video and / or audio data) is decomposed into a number of sub-signals, each of which is then individually compressed. As mentioned above, one accessible signal is reproduced from any one sub-signal, further improvements are achieved with additional sub-signals, and a complete reconstruction of the original signal is all sub-signals. Decomposition into sub-signals is done non-hierarchically so that is achieved when is received correctly. In addition, it is preferable to maximize the overall compression gain while the above three criteria are met.

【0036】

One way to decompose the original signal is to construct each sub-signal as a reduced resolution representation of the original signal. As shown, this is done by splitting the original signal, the input stream X, by the splitter 40, passing the decomposed signal through the lowpass filters 51 to 53, respectively, and then by the downsamplers 61 to 63. Performed by downsampling. Here, the downsampler 61 performs downsampling based on the sampling interval t1 generated by the clock generator 71, and similarly, the downsampler 62 performs downsampling based on the sampling interval t2 generated by the clock generator 72. The downsampler 63 performs downsampling based on the sampling interval tn generated by the clock generator 73. Therefore, each sub-signal differs only in the sampling position. If desired, such decomposition can be performed by a filter bank containing a pretreatment filter and its shifted version.

【0037】

The pre-processing filter should be selected to suppress the alienating component in the downsampled sub-signal. This helps reduce the bit rate required to encode the sub-signals and allows for acceptable image reproduction from a single sub-signal.

【0038】

The pretreatment filter should also be selected so that it does not completely remove high frequency components. If the high frequency components are completely removed, there is no way to reproduce those components of the original signal, even if all the sub-signals are present. Therefore, the above filter suppresses high frequency components but should not be completely removed.

【0039】

Mathematically, rebuilding multiple substream components into one reconstructed stream is the inverse of the determinant that associates the sample data in all substreams with the sample data in the original stream. including. Typically, this involves a large determinant with inverse matrix computations that use a lot of computational complexity and memory space. One way to treat this computational burden as a problem is to use the block recursive reconstruction method. At each step of the induction process, a block of 2x2 sample data in the original stream is reproduced based on up to four corresponding sample data in the received subcomponent stream. Of course, other computational techniques may be used to achieve the same result. For more information on coding and encoding in non-hierarchical methods using multi-descriptive coding, the prior art document Yao Wang et al. al.), Robust Image Coding and Transport in Wireless Networks Using a Non-Hierarchical Decomposition, Mobile Multimedia Communications; Goodman, Plenum Press. I want to be.

【0040】

The MDC-encoded substream is transmitted over the multicasting network 14 as a datagram using Real-Time Protocol (RTP) for transmitting content and Real-Time Transport Protocol (RTCP) for performing flow control. Will be done. These protocols can, of course, co-exist with common TCP / IP used by many Internet applications.

【0041】

Figure 4 presents a review of these protocols with an example showing how the five entities 30,32,34,36,38 communicate with each other. In FIG. 4, entities 30-38 communicate with each other using a hierarchical architecture generalized by the Internet. Of course, it will be understood that Figure 4 is shown only to show how the preferred Real-Time Protocol (RTP) fits into one feasible structural schema. Real-time protocols are preferred, but they are not shown as limiting embodiments of the present invention in a broader manner. In other words, other message transmission protocols may be used appropriately.

【0042】

In FIG. 4, each of the communicating entities 30 to 38 is illustrated as a hierarchical architecture using the physical layer at the bottom layer and the application layer at the top layer. Entity 30 communicates with Entity 32 using the Ethernet protocol at the physical level. Entity 32 and Entity 34 communicate using the ATM protocol at the physical level. In a similar manner, entity 34 communicates with entity 36 using the Ethernet protocol and entity 36 communicates with entity 38 using the PPP protocol. Again, the physical layer communication protocol selected here for illustration does not limit embodiments of the invention as described in the appended claims.

【0043】

The upper layer of the physical layer is the Internet Protocol (IP) layer. The IP protocol blocks the physical or transport layer from the application layer. The IP protocol supports connectionless communication in which packets of information are transmitted and received as datagrams. Note that in the hierarchical architecture illustrated in Figure 4, all communication entities use the IP protocol.

【0044】

Two different higher-level protocols, one higher than the IP protocol shown, are the TCP protocol and the UDP protocol. Again, FIG. 4 is only shown as an example of a feasible form. UDP protocol or user datagram protocol represents a simple transfer protocol. It does not protect the message strings transmitted by the UDP protocol. TCP protocols or transmission control protocols provide a higher level of reliability and also ensure that datagrams are transmitted in the proper sequence. The TCP protocol uses a recognition system to ensure that all datagrams are transmitted in the proper sequence. The TCP protocol includes a mechanism for retransmitting unrecognized packets. This recognition and retransmission technique guarantees proper packet transmission, but it does not guarantee real-time packet transmission. Therefore, the TCP protocol is generally not suitable for transmitting real-time data such as multimedia video and / or audio data.

【0045】

Real-time Transport Protocol (RTP) replaces TCP's complex forwarding protocol with a simple framework that applications can use directly. Without implementing a missing data detection and retransmission mechanism that can introduce transmission delays, the RTP protocol simply ignores the missing data. The RTP protocol is also typically unrelated to the packet transmission sequence. The protocol assumes that the application layer, which is the layer above it, corrects any data in an incorrect order. The RTP protocol is compatible with several different coding standards such as MPEG, JPEG and H.261.

【0046】

In the illustrated example of FIG. 4, entities 30 and 38 are both processed using the RTP protocol. Therefore, streaming data can be supplied from entity 30 to entity 38 via a network composed of entities 30, 32, 34, 36 and 38.

【0047】

The RTP protocol is designed for multicast operation. Multicasting is a form of message broadcasting in which multiple messages are transmitted to many different receivers in one specified set. Multicast addresses identify multiple sets of interfaces, often including multiple interfaces belonging to different systems. When a message has one multicast designated address, the network strives to transmit it to all interfaces in the above set. This feature allows the system to generate a message once and have that message transmitted to many different receivers.

【0048】

In addition to transmitting datagram packets to multiple receivers, multicasting networks typically also support feedback from the receiver of the message. Typically, all participating devices in a multi-casting group session can receive these feedback messages. Such feedback messages are commonly used for real-time traffic control, followed by the associated real-time control protocol (RTCP). In that respect, RTCP is an optional extension of RTP. The RTCP packet is used by the preferred embodiment for transmitting flow control and session management information between entities participating in the group multicast session.

【0049】

Figure 5 illustrates the RTP packet format. Note that the packet contains the sequence number and time stamp used to reconstruct the packet in the proper time sequence.

【0050】

6 and 7 show in detail how the multimedia client 16, authorization control unit 18, and media push engines 12a-12e communicate with another device during a multimedia group session. In particular, FIG. 6 is a diagram showing a basic message flow and communication sequence of a preferred embodiment. FIG. 7 is a detailed diagram of how the RTP and RTCP protocols are used when routing substream component datagrams from the media push engines 12a to 12e to the multimedia client 16.

【0051】

First, referring to FIG. 6, the multimedia client 16 sends a unicast TCP protocol message to the authorization control unit 18 to request a transmission start signal for a particular media selection. The authorization control unit 18 seeks information from the catalog service system 20 of the authorization control unit 18 to determine whether the requested media selection (ie, the requested stream) exists on the network. Assuming the stream exists, the authorization control unit 18 sends a stream release message to those media push engines 12a-12e that have at least some substream components of the requested selection. This stream release request is transmitted to all media push engines 12a to 12e. The media push engine, which has detected that it can supply the requested stream component itself, has a multicasting session management and flow control session between those hosts that supply that particular stream and the host that receives it. Enter together. The participating media push engine and the participating multimedia client 16 obtain the required multimedia address from the authorization control unit 18 to so control the multicast group session. The authorization control unit 18 then effectively leaves the above session and modifies the next session management and flow control message only between the multimedia client 16 and all corresponding media push engines. To enable. This reduces the overhead of the authorization control unit 18.

【0052】

The authorization control unit 18 generates the address of the multicast class D used by the multicast session. This address may be selected from a pool of available multicast address entities. The authorization control unit 18 is therefore responsible for managing the allocation of multicast addresses. At the end of the multicast session, authorization control unit 18 returns the multicast address to the pool of available multicast address entities.

【0053】

Therefore, once a multimedia group session is initiated, those media push engines that can supply substream components do so by sending unicast RTP session stream data to the multimedia client 16. Execute. With the RTCP protocol, these media push engines 12a-12e can be combined with another device to join or leave a multicast group session if required to maintain higher quality of service. You may communicate.

【0054】

Referring to FIG. 7, the media push engine 12 and the multimedia client 16 communicate over the network at two different levels. As shown by the dotted line, the Unicast RTP session signal sends multimedia streaming data to the multimedia client 16. At the same time, if requested, the media push engine 12 and multimedia client 16 send multicast RTCP report signals to each other, in particular any suitable flow control command or other session management command (eg, push start, The RTCP transmitter report signal and receiver report signal are transmitted in the same manner as (pause, continue). The RTCP control signal is shown by the bidirectional solid line in FIG.

【0055】

In essence, after a stream release message has been sent by authorization control unit 18, each media push engine 12 has to determine whether it can supply the required stream components. Ask the relevant media storage system 20 for information. If so, Media Push Engine 12 joins a specific multicast group. Otherwise, it will not participate in the multicast session (despite subsequent requests). Once the Media Push Engine 12 joins the multicast group, it joins the communication using the RTCP protocol, which changes the statistics of the transmitted and received data among the members of the group. .. As mentioned above, the authorization control unit 18 does not need to participate in these communications, and thus unless requested for another session or until the current session is requested to be terminated. , It keeps waiting.

【0056】

Effectively, the system implements a decentralized permit control system, where the participating members of the group make collective and decentralized permit control decisions. One advantage of this distributed approach is that this embodiment prevents network congestion and quality of service, despite the fact that the multicasting network is a best effort network with no guarantee of real-time transmission. It is possible to incorporate an intellectual mechanism to improve.

【0057】

Of the best-intensity networks, in particular those that lack complex traffic and user control policies often experience congestion. Such congestion results in real-time data loss or substantial delay. As mentioned above, real-time data that is transmitted with a delay is actually treated as not being transmitted. The continuous influx of data into the congested nodes of the network tends to further exacerbate the congestion. Therefore, when an overload (congestion) occurs due to excessive information, the overloaded node reduces the number of components transmitted by the node. By reducing the number of components, the amount of information is reduced and the cause of overload is reduced. Therefore, in the embodiment of the present invention, when congestion is detected, time is detected and transmitted as an instruction signal required by the given node in order to reduce the number of components transmitted. Use the RTCP transmitter report signal and the reception report signal when performing.

【0058】

FIG. 7 illustrates how this is achieved. Multimedia client 16 has substream component X₁, X₂, X₃And X₄Requests a real-time data stream X consisting of. Assume that the media push engine 12 in FIG. 7 is experiencing local traffic congestion such that the substream components arrive late at the multimedia client 16. The RTCP receiver report signal of the multimedia client 16 notifies the media push engine 12 (and other media push engines participating in the group session) of a few percent of the components from the media push engine 12. The media push engine 12 parses these report signals and stops sending the selected component, in this case substream component X.₃Stops the transmission of, thereby reducing the amount of traffic flowing through the location of its congestion. Therefore, after adjustment, Media Push Engine 12 is component X₁, X₂, X₄Only feed multimedia client 16. Other media push engines participating in the multicast group session receive the same transmitter and receiver report signals and component X from media push engine 12.₃Missing may be interpolated if other media push engines can supply this missing packet. Otherwise, the quality of service will be reduced slightly as mentioned above.

【0059】

FIG. 8 shows how the data stream can be effectively redistributed by making local adjustments to the substream components being transmitted. In the example of FIG. 8, it is assumed that there is local congestion somewhere in the path of the data feeding the substream component from the media push engine 12b. Therefore, the RTCP transmitter report signal and receiver report signal point out that some parts of the components previously transmitted by the media push engine 12b to the multimedia client 16 are missing or delayed due to local congestion. .. In the illustrated example, the missing component also happens to be present in the storage system of the media push engine 12a. The media push engine 12a can retransmit the payload of the missing component to the multimedia client 16 or easily adjust the set of components to be transmitted in future real-time data transactions. If the missing payload is retransmitted by another media push engine, it allows the missing component to be rebuilt with the previously transmitted component before the stream is reconstructed and presented to the user. In order to do so, a sufficient amount of buffer should be provided by the multimedia client 16. If the system simply modifies the set of components for future transmissions, such changes constitute a redistribution mechanism for the components of the scalable server. This mechanism enhances the improved quality of service by improving the presentation of stream data to multimedia clients.

【0060】

The embodiments described above are generally suitable for most media transmission applications, but there are some systems that cannot tolerate even the slightest degradation in quality. Such systems include very high quality broadcast video signal distribution. In these demanding applications, the systems of the preferred embodiments described above can be modified to use additional reliability mechanisms for real-time components. In this case, the real-time protocol may be modified or increased to allow the retransmission of the missing real-time payload. This "reliable RTP" is illustrated in FIG. The media push engine uses a real-time protocol to communicate with the RTP stack memory. In this case, it is assumed that the first and third components are received, but the second component is missing. There is an immediate negative response signal (NACK) from the RTP stack memory telling the media push engine that the second payload was not received. The media push engine then retransmits the required payload and the RTP stack memory stores the required payload at the exact location in the data buffer memory. The client application then reads the data from the data buffer memory. Any replicated packet may be dropped (removed) and any highly delayed packet may also be dropped (removed).

【0061】

From the above, it will be understood that the embodiments according to the present invention provide a media transmission system architecture that uses distributed networking techniques for transmitting streaming data over a best effort network. Since the complexity of the server increases linearly with the number of clients, the architecture may easily increase or decrease the number of components. Therefore, the architecture is a fully distributed and closely connected parallel architecture that can provide simplicity and robust service.

【0062】

By using multiple description coding (MDC) and multiple path transmissions, embodiments according to the invention can provide higher quality of service without relying on delay-causing transmission retry techniques. .. Therefore, one embodiment of the present invention facilitates cooperation with existing real-time transport protocols (RTPs) for data transfer and real-time control protocols (RTCP) for session management, rate adaptation, and the like. be able to. When congestion is encountered, the presentation is degraded without interruption of transmission due to multiple descriptive coding and the way in which group session participants are added to or excluded from the group. There is. For example, if the signal being transmitted is a high quality signal (eg, a signal with a large number of frames per second) and it is not possible to transmit the high quality signal due to congestion, the signal is It is dropped to a lower quality signal without interrupting transmission. Those watching broadcasters may experience slightly degraded images when this happens, but the transmission of information will not be interrupted. Stream flow control can also be controlled by these same mechanisms to eliminate or reduce the congestion once the network congestion is detected.

【0063】

One embodiment of the present invention is therefore theoretically suitable for transmitting multimedia selections as well as video and audio streaming data. The embodiments according to the present invention can easily support a large number of bit rate data streams and can provide services at both constant bit rate and variable bit rate.

【0064】

While the embodiments according to the present invention have been described in preferred embodiments, modifications and modifications of the embodiments according to the present invention are made without departing from the concept of the embodiments according to the present invention by the scope of the appended claims. It will be understood that it is possible to form an example.

【0065】

[Effect of the invention]

Each of the plurality of media push engines participating in the loop session corresponds to the predetermined media selection without using the authorization control system as an intermediary to transmit the components of the substream. To the network so that it can be transmitted to the media client and reconfigured by the media client, and the first media push engine and the second media push engine are the components of the substream. Connected to the network at different addressable locations selected to establish the availability of different routes for sending to the media client over the network, thereby causing network congestion. Reduce the effect on the performance of the media transmission system. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology. It is connected to the network at selected, differently addressable locations, thereby reducing the effect of network congestion on the performance of the media transmission system. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology. It is connected to the network at selected, differently addressable locations, thereby reducing the effect of network congestion on the performance of the media transmission system. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0066】

【0067】

Further, in the distributed media transmission system according to claim 2, in the distributed media transmission system according to claim 1, the plurality of media push engines communicate with the network via different communication paths. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0068】

Further, in the distributed media transmission system according to claim 3, in the distributed media transmission system according to claim 1, the network is a connectionless network that provides the best effort transmission service. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0069】

Further, in the distributed media transmission system according to claim 4, the network is the Internet in the distributed media transmission system according to claim 1. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0070】

Further, in the distributed media transmission system according to claim 5, in the distributed media transmission system according to claim 1, the media client and the plurality of media push engines participating in the multicast group session are of data. Use real-time transport protocol (RTP) for transfer. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0071】

Further, in the distributed media transmission system according to claim 6, in the distributed media transmission system according to claim 1, the media client and the plurality of media push engines participating in the multicast group session are of the session. Use real-time control protocol (RTCP) for management. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0072】

Further, in the distributed media transmission system according to claim 7, at least a part of the components of the substream is replicated over several media push engines in the distributed media transmission system according to claim 1. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0073】

【0074】

Further, in the distributed media transmission system according to claim 8, in the distributed media transmission system according to claim 1, a data buffer system related to the media client that stores substream components before reconstruction is provided. Further prepare. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0075】

Further, in the distributed media transmission system according to claim 9, in the distributed media transmission system according to claim 1, the permission control system further responds to a request from the media client to terminate the multicast group session. Then, it operates so as to instruct all media push engines participating in the multicast group session to end the multicast group session. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0076】

Further, in the distributed media transmission system according to claim 10, in the distributed media transmission system according to claim 1, the permission control system is a multicast session address used to call a multicast group session. Includes a permission control unit that holds a pool of, where the permission control unit assigns a specified multicast session address selected from the pool for use by the multicast group session. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0077】

Further, in the distributed media transmission system according to claim 11, in the distributed media transmission system according to claim 10, the permission control unit responds to a request from the media client to terminate the multicast group session. Then, it further operates to return the specified multicast session address to the above pool. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

【0078】

【0079】

Further, in the distributed media transmission system according to claim 12, in the distributed media transmission system according to claim 1, between the media client and all media push engines participating in the multicast group session. , There is a unicast flow of datagrams containing real-time stream component data. Therefore, a delay is caused by communicating the stream using a non-hierarchical set of substream components from the plurality of media push engines and controlling the media client and the media push engine by the authorization control system. It is possible to transmit multimedia data such as media selections and audio and / or video in real time with higher quality of service without relying on transmission retry technology.

[Simple explanation of drawings]

[Figure 1]

It is a network diagram which illustrates the preferable embodiment of this invention.

[Figure 2]

It is a network diagram showing in more detail how two different data streams (X and Y) are distributed across a network using non-hierarchical multi-description coding.

[Fig. 3]

It is a block diagram which shows one Embodiment of multiplex description coding (MDC) used in this embodiment.

[Fig. 4]

It is a hierarchical diagram showing the TCP / IP architecture and how the RTP architecture is incorporated into an IP-based system.

[Fig. 5]

It is a format diagram which illustrates the packet format according to the real-time protocol (RTP) in more detail.

[Fig. 6]

It is a network diagram which shows the call permission and session management by this Embodiment.

[Fig. 7]

It is a network diagram which shows the flow of information between a media push engine 12 and a multimedia client 16 on a multimedia pushing IP network 14 in more detail.

[Fig. 8]

It is a network diagram which illustrates the redistribution processing of a source component server on the multicasting IP network 14 of FIG.

[Fig. 9]

It is a protocol diagram showing how Real Time Protocol (RTP) is modified to increase its reliability.

[Explanation of symbols]

10 ... Distributed Network Multimedia System, 12,12a, 12b, 12c, 12d, 12e ... media push engine, 14 ... IP Multicasting Network, 16,16a, 16b ... multimedia client, 18 ... permission control unit, 20 ... Catalog service system, 30,32,34,36,38 ... Entity, 40 ... splitter, 51,52,53 ... Low Pass Filter, 61,62,63 ... Down sampler, 71,72,73 ... Clock generator.