Useful data format of real-time transport protocol

FIELD OF THE INVENTION

The present invention relates to a real-time transport protocol (RTP), and more specifically to the format of an RTP wire for streaming multimedia data (e.g. audio-video) over a network such as the Internet.

BACKGROUND

The following description assumes that the reader is familiar with the standard contained in the Request for Comments (RFC) 1889 Task Force on Engineering Support Internet (IETF): "RTP: A transport protocol for the application of real-time" (hereinafter referred to as the standard RFC 1889 IETF) and with the standard contained in the Request for Comments (RFC) 1890 Task Force for engineering support of the Internet: "The parameters of RTP for use in audio and video conferences with minimal control" (hereinafter referred to as the standard RFC 1890 IETF).

Real Time Transport Protocol (RTP) according to how it is defined in RFC 1889, provides through network transport functions suitable for applications of systems that transmit data having a real-time, such as audio, video or data simulation on network Group services or personal mailings. These transport functions provide services through the delivery of data with real-time characteristics, such as interactive audio and video. Such services include payload type identification, serial numbering, timestamping assignment and control of the delivery. The RTP protocol supports data transfer in many destination addresses using multicast, if such delivery is provided in your network.

RFC 1889 standard does not provide any mechanism to ensure timely delivery of data or provide other quality of service guarantees, but relies on the services of a lower level. It does not guarantee delivery or prevent improper delivery, but it also does not suggest that you use a network smoothly and delivers the packets one by one. Sequence numbers contained in the protocol RTP, allow the receiver to reconstruct the sequence of the packet from the sender, but sequence numbers might also be used to determine the proper location of the package, such as video decoding without having to decode packets in order.

A typical application of RTP refers to streaming data, where packets of audiovisual (AB) data format advanced (advanced) systems (ASF) are sent in packets created protocol RTP (RTP-packets) over a network from a server to a client or a peer device to another peer device. Audio and video data presented in the ASF format, are stored together in one ASF packet format (ASF-package). For this reason, RTP-packet can contain both audio and video data.

According to the RTP protocol as defined in RFC 1889, lack of flexibility in regard to integrating multiple payloads into one RTP-packet and separation of a payload between multiple RTP-packets. Also standard RFC 1889 does not define a format in which each module with payload data in the RTP-packet can be delivered metadata. Another disadvantage of the RFC 1889 standard is the lack of a mechanism streaming encrypted blocks of data across a network while maintaining a block boundary of each encrypted block such that the recipient of these units could decrypt the encrypted blocks of data. Providing this kind of flexibility by improving the mechanism for streaming RTP protocol would be a step forward in the art. Consequently, there is a need for improved methods, computer-readable medium, data structures, devices and computing devices that can provide such flexibility.

SUMMARY OF THE INVENTION

In one embodiment, packets of audio-visual (AB) data of Advanced Systems Format (ASF) re-packetized into packets Real Time Transport Protocol (RTP-packets), and in response to a request for streaming AB-data sent over the network from the server to the client or network communication means from one of the peer to peer. AB-data is encrypted to form encryption units. The re-packetization includes packetizing encryption units in RTP-packets, each of which comprises a header RTP-packet, one or more payloads overall data flow and format header payload sent over the protocol RTP (hereinafter header RTP payload format or RTP PF header), for each payload. The RTP PF header contains a payload boundary for the corresponding encryption units. Payload in RTP-packet may be one or more encryption units or a fragment of the encryption section. After the RTP-packets forwarded through the network, encryption units contained in the received RTP-packets rebuilds. Rebuilding process uses the payloads contained in the RTP-packets, and the frontier contained in the relevant PF header RTP. Rebuild the section of encryption can be decrypted for later playback. Each RTP PF header can have attributes for its composition corresponding payload and these attributes can be used when playing back useful data.

In a variation of Your eprivedennogo embodiment for the formation of RTP-packets use data in a format different from the format ASF. In another variety of the foregoing embodiment RTP-packets are formed so as to contain payloads of plaintext.

In another embodiment, the invention provides a wire format for streaming over a network as part of RTP-packets of encrypted blocks of data protected by digital rights management protocol environments Windows® (WM DRM) (e.g., for streaming protected with WM DRM content). Each RTP-packet contains a header in the incoming data intended for storing encryption block boundaries so that each encryption unit can be decrypted by the recipient. After decryption using the WM DRM protocol streaming data can be played by the recipient.

LIST OF FIGURES

1 - an example of an exemplary process according to an embodiment of the present invention, for the conversion of two (2) packets of audiovisual (AB) data presented in Advanced Systems Format (ASF), four (4) RTP-packet, wherein the video data and audio data are packetized in the resultant RTP-packets separately from each other, and the block boundary of each payload are preserved such that original sample (selected for delivery portion) AB-information that were encrypted and packetized in the two ASF-packets can be reconstructed by decryption.

2 - an example of alternative exemplary processes corresponding to various embodiments of the present invention to convert two (2) packets of video data presented in the ASF format, one (1) RTP-packet, wherein the another alternate process places the payloads of packets ASF- into separate payloads in the RTP-packet, and the other alternative process combines the payloads of the ASF-packets into a combined payload in the RTP-packet, where block boundaries for each payload are preserved such that original video sample that was zapaketirovany encrypted and in two-ASF packets can be reconstructed by a decryption mechanism.

Figures 3a-3b - respective payloads of data structures corresponding to the embodiment of the present invention for RTP-header packet and a corresponding payload header.

4 - block diagram sootvetstsvuyuschaya embodiment depicting a network client-server system in which streaming can be performed by one server to the client or peer to peer.

5 - a block diagram according to an embodiment of the present invention, illustrating communications between a server (or client) and a client, where the server (or client) to the client delivers the requested media data stream that the client can reproduce.

Figure 6 - a block diagram according to an embodiment of the present invention illustrating a networked computer that can be used to perform the role of a server or a client.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the invention disclosed herein define a wire format for delivering streams of similar or dissimilar data such as multimedia data from media Windows® operating system, through real-time transport protocol (RTP). This delivery can be between server and client, as well as in the context of transferring data from one peer to peer (such as audio-visual conference software environment Windows® Messenger ™).

Wire format, in various embodiments, extends IETF standard RFC 1889, providing a data delivery protocol RTP greater flexibility. Embodiments of the invention provide a mechanism for streaming being in RTP-packets the audio data, which are separated from the RTP-packet of video data. Embodiments also provide a wire format in which with each payload in RTP-packets can be delivered metadata, wherein the metadata supplied extensive information describing the payload. Other embodiments of the invention provide a mechanism for streaming encrypted blocks of data network while maintaining a block boundary of each encrypted block in such a way that the recipient is able to decrypt the encrypted blocks of data blocks. In another embodiment, a wire format provides delivery of data protected using digital rights management protocol environments Windows® (WM DRM), in such a way that the delivered data may be decrypted for playback.

Various embodiments disclosed herein provide repackage data in a sequence of media packets that are included in a system layer bit stream. These data are packetized into RTP-packets that are compliant with the RFC 1889 standard, while expanding its capabilities, so the bit stream is converted to the system-level (a bit-wise map of RTP) in RTP. The transformed to a form of data, each media packet contains one or more payloads. In some system layer bit streams may be packages of diverse media, having in its composition data such as audio data, video data, program data, the data in the format of the Joint Expert Group on the picture (JPEG), data Hypertext Markup Language (HTML), these digital Musical instrument interface (MIDI), etc. Package heterogeneous multimedia data is a media packet where two or more of its payloads belong to different media streams.

Various embodiments of the invention relate to a system layer bit streams where each media packet is a single media packet. The package of single media data, all of the payloads contained in the media packet belong to the same media stream. Other embodiments of the invention relate to a system layer bit streams where each media stream always contains only one (1) payload. In further embodiments, the size of the "payload header" in the multimedia packet is zero, it is possible if each media packet only contains a single payload, but could also take place in the presence of many payloads if a media packet header contains information about the size of each payload.

Figures 1-2 depict illustrative embodiments of the invention, in which system layer bit streams include a series of packets of Advanced Systems Format (ASF), each of which is composed of data. These data are packetized into RTP-packets compliant RTP in 1889, although the possibility of extending it. Essentially, system layer bit streams include a series of media packets that are ASF-packets and payload in each ASF packet-is ASF payloads. Although illustrative purposes used ASF-packs, in other embodiments disclosed herein, the creation of RTP-packet is not limited to using the data format ASF, but rather can use other formats in which the data intended for streaming . These other formats, as well as format ASF, generally described herein as system layer bit streams that include a plurality of media packets, each of which is composed of data while the data are converted to the bit map RTP in various embodiments.

1 shows the stream of audiovisual (AB) data format 100 ASF. Streaming data AB-100 ASF format, which includes audio data 102 and video 104 are packetized in ASF packet A 106-and-ASF packet B 108. ASF packet A 106-includes a first ASF header-packet header module ASF payload data, audio data 102, a second ASF header, and video data A fragment of video data 104. ASF composition-package 108 includes an ASF header-packet payload header, and ASF video data B fragment of video data structure 104.

Streaming data AB-100 ASF format, represented as-ASF packet A 106 and ASF packet B-108, in one embodiment, can be packetized into a plurality of RTP-packets. As shown in FIG. 1, these packets comprise RTP-packet A 110, RTP packets from packet-112 (1) and RTP-packet 112 (N) inclusive, and RTP-packet D 116. Each RTP-packet in accordance with the RFC 1889 standard has RTP-header packet payload and payload format header (PF) RTP. The RTP PF header in the sense in which it is used herein, is a payload header in the RTP-packet. The RTP-packet contains only one (1) type of media. In other words, RTP-packet contains payloads heterogeneous multimedia data. In the embodiment illustrated in Figure 1, video data A of ASF packet A 106-are too large to fit into a single RTP-packet. By virtue of this video data A of ASF packet-A-106 divided between packets from RTP-packet 112 (1) and RTP-packet 112 (N) inclusive. Size RTP-packet may be a function of the physical characteristics of your network, on which shall be transmitted RTP-packets or administrative policy regarding the size of the package, which can install your network administrator, or estimates of bandwidth used by the network.

In accordance with the process of stacking in RTP-packets, illustrated in Figure 1, audio data 102 is included in RTP packet A-110, and video data B of ASF packet B-108 are included in RTP packet D 116. Each RTP PF header of each protocol RTP-packet can contain information relating to the separation of video and audio data respectively in separate RTP-packets. Thus, the data stream sampling A / B-data 124 can be reconstructed from the audio data contained in RTP-packet A 110, video data ranging from the track 1 video data A and fragment N video data and including, contained in RTP-packets with 112 (1) and 112 (N) inclusive, and the video data B, contained in the RTP-packet D 116. Once the reconstruction of the data stream sampling A / B-completed information contained therein audio sample data 120 and the video sample data A + B 122 can be played back in a streaming context. In view of the foregoing, Figure 1 illustrates a wire format in which smaller RTP packets are created from a-larger-ASF packets, where the packetization puts a payload of different data streams into separate packets each with its own RTP PF header. 1 also illustrates an embodiment of a wire format in which block boundaries for each payload are preserved such that original audio and video samples that were encrypted and packetized in ASF-packets can be reconstructed by a decryption mechanism, producing operations RTP-packets.

2 illustrates the data stream AB-200 ASF format. Streaming data AB-200 ASF format, including video data 202 have been stacked in-ASF packet A 208 and ASF packet-B 210. ASF packet A 208-includes ASF-header packet payload header, and ASF A video data 204. ASF packet B-210 includes a header ASF-packet payload header, and ASF video data B 206. Figure 2 shows two (2) alternatives stream packetization AB-200 data represented in ASF format, in RTP-packets that are compliant with the RFC 1889 standard, while expanding its capabilities.

In the first alternative, following arrow 250, video data A 204 and video data B 206 are packetized into a single RTP packet alternative A-212 having an RTP-header packet. Each of the modules: A video module 204 and video data B 206, preceded by PF header RTP. An RTP packet alternative A 212 in accordance with RFC 1889 is composed of a header RTP-packet, multiple payloads, and respective RTP PF headers.

In the second alternative, also following arrow 250, video data A 204 and video data B 206, from respective ASF packets are packaged-in-RTP packet alternative B 214 having an RTP-header packet. Video data A 204 and video data B 206 are collected together as a continuous payload in RTP packet alternative B-214 payload is preceded by RTP PF header. -RTP packet alternative B 214, in accordance with the RFC 1889 standard is composed of a header RTP-packet payload, and one RTP PF header.

In accordance with the process of packets in RTP packetizer, shown in Figure 2, video data A and B (204, 206) comprises either a part-RTP packet alternative A 212 or in the RTP-packet alternative B 214. Each RTP PF header can contain information relating to the corresponding payload. Each of the alternative RTP-packets 212, 214 contain sufficient data to reconstruct ASF packet A 208-and-ASF packet B 210 so as to receive therein video data A and B (204, 206). Once their re-creation is complete, the video sample data 222 can be played back in a streaming context. Based on the above FIG. 2 illustrates the format of a wired communication protocol RTP, which larger RTP-packets are created from small ASF-packets and where block boundaries for each payload are preserved such that the data source samples videoinformtsii that were encrypted and packetized in the two ASF-packet It can be reconstructed by a decryption conducting operations on RTP-packets.

3a shows a payload data structure for RTP protocol header fields. RTP packet header is more fully described in the standard RFC 1889. timestamp field in RTP-header packet should be set at the time of submission of the sample contained in the RTP packet protocol. In one embodiment, the clock frequency is 1 kHz unless means independent of RTP protocol, established by another value.

Eight bits from the beginning of the packet header RTP-bit field is interpreted as a marker (M). M bit is set to zero, but is set to one ("1") whenever the corresponding RTP-packet has payload that is not a fragment of a sample, it contains the last fragment of a sample, or is one of a plurality of complete samples in the RTP-packet . M bit may be used by the receiver to identify the complete sample receiving to decode and playback. Thus, the M bit in the RTP-header packet may be used to mark the packet stream to meaningful events (e.g., video sample frame boundaries).

3b shows one embodiment of a header payload format (PF) Header or RTP payload. The RTP PF header is a fixed length portion of a size of sixteen (16) bits, followed by a variable length part. RTP PF header field, illustrated in Figure 3b, comprise eight-bit string, character fields designated as "SGLRTDXZ" field length / offset field, a relative timestamp field, the time compression (decompression), a duration field, a length field expansion payload (RAP) and the corresponding data field of the RAP, each of which is explained below.

The S field has a length of one (1) bit and is set to one ("1") if the corresponding payload (e.g., sample, fragment of a sample, or combination of samples) is a key sample, i.e. sample intra-coded or I-frame. Otherwise it is set to zero. S bit in all RTP PF headers, preceding fragments of the same sample must be set to the same value.

Field G has a length of one (1) bit and is used to group sub-samples in a corresponding data payload, which is of a single sample. Minutes of digital rights management environments Windows® (WM DRM) encrypts content, measured from the boundaries of "payload format ASF". In order to allow proper decryption of the content, the boundaries of sub-samples can be communicated to the client that is to receive the payload. For example, the encryption unit can be packetized such that would be broken into a plurality of transmission units (e.g., placed in separate packets) that must be transmitted. Before scattered plurality of transmission units can be decrypted at the receiving client they have to be reassembled into the original encrypted form. As in other decryption methodologies and algorithms in order to properly reconstruct the encrypted encryption units in preparation for decryption of encrypted content, the client can use the boundaries. As such each border "a payload format ASF" should precede consideration here PF header RTP.

To indicate that an encrypted "section" was broken into fragments, G field shall be set to zero ("0"). If ASF format is used, the encryption unit will ASF payload and the bit is set to zero ("0") on all fragmented ASF payloads, except the last ASF payload. In this case, the question of whether the sample is fragmented or not, does not matter. If ASF is not being used, the encryption unit is a media sample, in which case the G bit is set to zero ("0") on all fragmented media data samples except the last sample. In this second case, the question of whether the fragmented ASF payloads or not, is not relevant, since ASF is not used.

Field L has a length of one (1) bit and is set to one ("1") if the field "Length / Offset" contains the length. Otherwise it is set to zero ("0") and the field "Length / Offset" contains an offset. Bit L should be set to one ("1") in all PF headers RTP, preceding complete (unfragmented) sample in the corresponding payload and must be set to zero in all PF headers RTP, that precede a payload containing a fragmented sample.

R field has a length of one (1) bit and is set to one ("1") if the RTP PF header contains a relative timestamp. Otherwise it is set to zero. R bit in all headers preceding fragments of the same sample must be set to the same value.

T field has a length of one (1) bit and is set to one ("1") if the RTP PF header contains a decompression time. Otherwise it is set to zero. The T bit in all RTP PF headers, which precede a payload containing fragments of the same sample must be set to the same value.

Field D is a length of one (1) bit and is set to one ("1") if the RTP PF header contains a sample duration. Otherwise it is set to zero. D bit in all RTP PF headers, which precede a payload containing fragments of the same sample must be set to the same value.

X field has a length of one (1) bit and is designed for optional use unspecified. Transmitter RTP-packet should set this bit to zero, and the receiver of the packet can ignore this bit.

Z field has a length of one (1) bit and is set to one ("1") if the RTP PF header contains extension data payload (RAP), which can be metadata relating to the corresponding payload. Otherwise the Z field is set to zero. Bit field Z would be zero for all headers PF RTP, whose M-bit to zero, but it should be set for all headers PF RTP, whose M-bit is set to one ("1") if the corresponding payload has associated data RPD.

The field "Length / Offset" has a length of twenty-four (24) bits, and determines the value of the length or offset of the sample, which was divided into fragments arranged in many RTP-packets. Bit L is set to zero, and the field "Length / Offset" contains the byte offset of the first byte of this fragment from the beginning of the corresponding payload (e.g., sample or fragment thereof). If the RTP packet contains one or more complete samples, the L bit is set to one ("1") in each RTP PF header, and the field "Length / Offset" sample contains the length of the sample (including the RTP PF header length).

The "Relative Timestamp" has a length of thirty-two (32) bits and is present only if the R bit is set to one ("1"). It contains the relative timestamp for the corresponding sample with respect to the timestamp in the corresponding RTP-header packet. As used in this time scale is the same as the time scale used for the timestamp in the RTP-header packet. The "Relative Timestamp" is defined as the number of 32 bits having a mark, which allows for negative offsets from the timestamp RTP-header packet. In the case where the "Relative Timestamp" is missing, it can be used relative timestamp default is zero.

The "decompression" has a length of thirty-two (32) bits and is present only if the T bit is set to one ("1"). It contains the decompression time relative to the timestamp in the RTP-header packet. As used in this time scale is the same as the time scale used for the timestamp in the RTP-header packet. This field is defined as the number of 32 bits having a mark, which allows for negative offsets from the timestamp in the RTP-header packet.

The field "Duration" has a length of thirty-two (32) bits and is present only if the D bit is set to one ("1"). It contains the length of the respective sample. As used in this time scale is the same as the time scale used for the timestamp in the RTP-header packet. The field "Duration" in all RTP PF headers, preceding fragments of the same sample must be set to the same value. In the case when this field is absent, the default duration value is taken implicitly or explicitly from the data of the sample itself. If this is not practicable, then a default value is taken equal to the difference between the timestamp of the sample and the time stamp of the next sample.

The "length of the extension data payload (RAP)" has a length of sixteen (16) bits and is present only if the Z-bit is set to one ("1"). It contains the number of data bytes RAP contained after the fixed part of the RTP PF header. These RAPs are of variable length and contain one or more attributes that describe the corresponding payload, which they precede. Data length field immediately follows the RAP fixed part of the payload header and indicates the number of bytes that contain the actual data RPD. The data structure of the RAP is transmitted between client and server (or peer-to peer), for example, by describing the SDP. In one embodiment, for information protected by using DRM protocol WM, there may be at least 4 bytes of DUE data format, representing the payload protocol identifier WM DRM, associated with each sample.

Although FIG. 3a-3b for RTP-packet header and RTP PF header provides a variety of fields, arranged in a different order, not all fields are required, and their order can be changed. In some embodiments, the required fields and their sequence, therefore, may be consistent with the flexibility with RFC 1889, while expanding the possibilities. Although the illustration in FIG. 3a-3b are used ASF-packs, in other embodiments disclosed herein, the creation of RTP-packets, RTP PF headers and payloads for them is not limited to using the data format ASF, but, in contrast, allows the use of other formats, which store data to be streamed.

The overall structure of the network

4 illustrates a network system 400 is a client / server network environment of the present invention. Typically, the system 460 includes one or more (m) network multimedia servers 402 and one or more (k) network clients 404. Computers communicate with each other over a data network, which in Figure 4 includes a wired or wireless network 406. The data network might also include the Internet or local area networks and private wide-area networks. Servers 402 and clients 404 communicate with each other by means of any protocol from a wide variety of known protocols, such as Transmission Control Protocol (TCP) or user datagram protocol (UDP).

Multimedia servers / clients 402/404 have access to streaming media data in various multimedia data streams. These media streams can be individual media streams (e.g., audio, video, graphical, simulation, etc.) or, alternatively, the composite multimedia data streams include many such individual data streams. Some media streams might be stored as files 408 in a database (e.g., file format ASF) or other file storage system, while other media streams 410 data could be supplied to the multimedia server 402 or client 404 "live" from other components that generate data, via dedicated communication channels or through the Internet itself.

Media streams received from servers 402 or from clients 404 are played at the client 404 as a multimedia presentation, which can include media streams from one or more servers / clients 402/404. These different media streams can include one or more, same or different types of media streams. For example, a multimedia presentation may include two video streams, one audio stream and one stream of graphical images. The user interface (UI) at the client 404 may provide users with a variety of management tools, for example, allow a user to increase or decrease the speed at which play a multimedia presentation.

Exemplary Computer Environment

In the following explanation of the invention will be described in the general context of computer-executable instructions, such as program modules, executed by one or more conventional personal computers. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Furthermore, those skilled in the art will recognize that the invention may be practiced with other computer system configurations, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers and similar systems. In a distributed computing environment, program modules may be located in both local and remote memory storage devices. Alternatively, the invention could be implemented through hardware or combinations of hardware, software and / or firmware. For example, to implement the invention could be programmed, special purpose integrated circuits (ASIC).

As shown in Figure 4, the network system of the present invention includes network server (network servers) and client 402, 404, which may be received from a plurality of multimedia data streams. In some cases, the media streams stored on a real server (s) and / or client 402, 404. In other cases, server (s) and / or client (s) 402, 404 may receive media streams from other network sources or devices . Typically, the network clients 404 responsive to user input request for the multimedia data stream corresponding to the selected multimedia content. In response to a request for multimedia data stream corresponding to multimedia content, server (s) and / or clients 402, 404 send the requested media streams to the requesting network client 404 in accordance with the format of a wired communication protocol RTP. The client 404 decrypts the payloads that are in the corresponding protocol stack RTP, and displays the resulting decrypted data streams for the requested multimedia content.

5 illustrates the input and storage of streaming A / V data on the server 402 or client 404 (e.g., peers). 5 also illustrates communications between server and client (402-404) or between peer devices (404-404) in accordance with various embodiments. Broadly speaking, the server or client 402, 404 receives inputting streaming A / B information from input device 502. The server or client 402, 404 encodes the information entered using the coder belongs to the codec. The encoding can, but need not, be performed on ASF format data. If the data is used in the ASF format, the encoding is performed on ASF-packets, each of which includes a header, and ASF-packet payload header, and ASF payload containing the AB-information (audio and / or video). The encoding can include encryption, such as where WM DRM protocol is used. ASF-packets stored server / client to service future requests to them.

The client asks the server / client flow AB-relevant data. The server / client retrieves and transmits to the client the corresponding flow AB-data that the server / client previously retained. Upon receipt of the client decodes the stream of AB-data and reconstructs and decrypts encrypted and broken into pieces sample AB-stream data using the border, reported in the corresponding PF headers RTP. The client can implement the AB-playback data transmitted stream.

Figure 5 shows the flow of data between and among blocks 504-530. At block 504, input device 502 provides the server / client 402/404 input that includes the streaming A / V data. As an example of streaming A / V-data might be supplied to server / client 402/404 input device 502 "live" through dedicated communications channels or through the Internet. Stream A / B information at block 504 is supplied to the encoder to put data into ASF-packets. In block 506 it is made non-binding encryption protocol WM DRM, and ASF-packets are stored in the memory of the server / client 402/404. As a result, the encryption protocol WM DRM and packaging may be that the encryption unit is broken down into a number of separate packages. Before the broken plurality of transmission units can be decrypted at the receiving client they have to be reassembled at the client into the source of the encryption units. Due to this border broken into pieces of transmission units stored in block 506 Titles ASF payloads.

At block 508, the client 404 submits a request for stream A / V data that is transmitted to server / client as indicated by arrow 510 in Figure 5. In block 512, server / client 402/404 receives the request. Retrieves the appropriate ASF-packets containing the requested stream of the A / V data. At block 514, the payloads from the audio and video information contained in the ASF-packets are logically separated so that they can be separately packetized into RTP-packets. Thus defined boundaries for each logically separate payload with audio information and video information.

Determined by the bandwidth of the network on which should be transmitted RTP-packets. This definition is used to calculate the predetermined-sized RTP packet. If ASF-size smaller than a predetermined packet size RTP-packet payloads of the same type may be combined into a single RTP-packet. If ASF-size package is larger than the predetermined RTP-packet size, ASF payloads can be broken into fragments for placing each fragment as a payload in an RTP-packet. Using corresponding logically separate audio and video units ASF payload boundary-defined packets for each RTP payload.

At step 516 each packet is completed with RTP-RTP-packet header, RTP PF header, and respective payload. In fact, the formed set of RTP-packets, which represents a plurality of ASF-packets, wherein the packets contain ASF-flow A / V data requested by the client 404. RTP-packets are transmitted for playback at the client 404 from server / client 402/404 via the function transmission provided for in section 518.

Arrow 520 in Figure 5 shows a transmission-RTP packets from server / client 402/404 to the client 404. At block 522 the client 404 receives the RTP-packets. In block 524 the decoder protocol RTP, is available at the client 404 decodes each received RTP-packet, including the header and the RTP-packet PF header RTP. In block 526 the process performs defragmentation and reconstruction of ASF-packets containing the requested stream of the A / V data. The defragmentation and reconstructing packets uses boundaries set forth in the RTP PF header for each corresponding payload containing, for example, sample or fragment thereof.

In block 528 reconstructed ASF packets are decrypted for-play unit 520. RTP PF header, included in the RTP-packet extension data may comprise useful data (RAPs), which describes the corresponding payload. These RAPs can thus provide metadata that can be used when playing unit 530 payload data contained in the corresponding RTP-packet. Blocks 522-530 are repeated for each RTP-packet received from the client 404, thereby completing the streaming A / V data from server / client 402/404 for rendering.

Figure 6 shows a general example of a computer 642, which can be used according to the invention. Computer 642 is shown as an example of a computer capable of performing the functions of any of clients 402 or servers 404 of FIG. 4-5. Computer 642 includes one or more processors or processing units 644, a system memory 646, and a system bus 648 that couples various system components including the system memory 646, processor 644.

Bus 648 represents one or more buses, belonging to any of several types of bus structures including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. The system memory includes read only memory (ROM, ROM) 650 and random access memory (RAM, RAM) 652. A cache 675 have levels L1, L2 and L3 and may be included in RAM 652. A basic input / output system (BIOS ) 654, containing the basic routines that help to transfer information between elements within computer 642, such as during startup is stored in the ROM 650. In addition, computer 642 includes a hard drive 656, magnetic disk for reading from the hard magnetic disk (not shown ) and writing to the disk drive 658 a magnetic disk for reading a removable magnetic disk 660 and writing to, and drive 662 for the optical disk, for reading a removable optical disk 664 such as a compact disc (CD ROM) or other optical media, or write to it.

Any of the devices: a hard disk (not shown), a drive 658, a magnetic disk drive 662 for the optical disk, a removable optical disk 664 may be a data carrier having recorded on it data. The information storage medium has a data area for recording stream data using stream packets each of which includes a certain area of ​​the package having one or more data packets. As an example, each data packet is encoded and decoded by the codec contained in the application programs 672 executed by processor 644. In fact, the encoder distributes the stream data fields of data packets within the stream packets so that the distributed stream data using encoding algorithms stored in the field of data packets. Alternatively, encoding and decoding of data packets can be performed as a function of the operating system 670 executed in the processor 644.

656 drive a hard disk drive 658 a magnetic disk drive 662 and optical disk connected to the system bus 648 by a SCSI interface 666 or some other suitable for this interface. The drives and their associated computer readable media provide nonvolatile storage of computer readable instructions, data structures, program modules, and other data. Although the exemplary environment described herein employs a magnetic hard disk, a removable magnetic disk 660 and removable optical disk 664, those skilled in the art should recognize that in the exemplary operating environment may also be used and other types of computer readable media which can store data, accessible by a computer, such as magnetic cassettes, flash memory cards, digital video disks, random access memories (RAMs), read only memories (ROM) and similar types of media.

The magnetic hard disk, magnetic disk 660, optical disk 664, ROM 650, RAM 652 can hold a number of program modules including operating system 670, one or more application programs 672 (which may include the Codec), other program modules 674 and program data 676. A user can enter commands and information into the computer 642 through input devices such as a keyboard 678 and pointing device 680. Other input devices (not shown) may include a microphone, joystick, game pad, satellite dish, scanner, or the like. These and other input devices are connected to the processor 644 via interface 682, coupled to the system bus. Also to the system bus 648 via an interface, such as a video adapter may be connected to a monitor 684 or other type of display device 686. In addition to the monitor, personal computers typically include other peripheral output devices (not shown), such as speakers and printers.

Computer 642 operates in a networked environment using logical connections to one or more remote computers, such as a remote computer 688. The remote computer 688 may be a personal computer, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to the computer 642, although Figure 6 illustrates only a memory storage device 690. The logical connections depicted in Figure 6 include a local area network (LAN) 692 and a wide area network (WAN) 694. Such networking environments It is commonplace in offices, enterprise-wide computer networks, intranets and the Internet. In the disclosed embodiment, remote computer 688 executes the Web-browser of the Internet, such as Web-browser Internet Explorer®, manufactured and distributed by "Microsoft Corporation" Redmond, Washington.

When used in a LAN networking environment, the computer 642 is connected to the LAN 692 through a network interface or adapter 696. When used in a WAN networking environment, the computer 642 typically includes a modem 698 or other means for establishing communications over the WAN 694, such as the Internet. The modem 698, which may be internal or external, is connected to the system bus 648 via the serial port interface 668. In a networked environment, program modules depicted relative to the personal computer 642, or portions thereof, may be stored in the remote memory storage device. It will be appreciated that the network connections shown are exemplary and may be used and other means of establishing a communications link between the computers.

Typically, the data processors computer 642 are programmed by means of instructions stored at different times in the various computer-readable storage media. Programs and operating systems are typically distributed, for example, on floppy disks or CD-ROM (CD-ROM). C these carriers they are installed or loaded into the secondary memory of a computer. At execution, they are loaded at least partially into the computer's primary electronic memory. The invention described herein includes these and other types of computer readable storage media when such media contain instructions or programs for implementing the steps described below in conjunction with a microprocessor or other data processor. The invention may also comprise the computer itself when programmed according to the methods and techniques described below. Furthermore, certain sub-components of the computer may be programmed to perform the functions and steps described below. The invention includes such sub-components when they are programmed in the manner described. In addition, the invention described herein includes data structures, described below, as embodied in various types of storage media.

For purposes of illustration, programs and other executable program components such as the operating system are illustrated herein as discrete blocks, although it is known that such programs and components reside at various times in various components of the computer, and are executed by a processor (CPU) data processing incoming (inbound) in the computer.

Conclusion

Disclosed herein in embodiments of the invention define a wire format that can be used for delivery of multimedia data between server and client and from one peer to peer via RTP protocol. To deliver the data using the RTP wire format allows for greater flexibility than the currently accepted standard IETF RFC 1889. Embodiments of the wire format provide streaming of encrypted data, provide a mechanism for delivery by the RTP metadata associated with each data sample and provide streaming data protected with WM DRM protocol.

While the invention has been described in language specific structural features and / or methodological acts, it is to be understood that the invention set forth in the appended claims is not necessarily limited to the described specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claimed invention.