RTP Payload Format
Abstract
This record has no abstract on file.
Term
Term ended
Expired 5 July 2024, 2.2 years ago.
- Priority
- Filed
- Granted
- Expired
- Today
62 claims: 52 independent, 10 dependent
- 1A means of encrypting a data stream having an arbitrary block size to form a plurality of cryptographic units,A means of converting packets of Advanced Systems Format (ASF) audio and video data into RTP packets, wherein the audio and video data are packetized separately.A means of storing data to hold a cryptographic block boundary for each payload, said cryptographic block boundary for each payload as a means determined by the corresponding separate audio and video payloads in ASF format. ,A means of writing payloads for different data streams in separate RTP packets, A device including means for packetizing the plurality of cryptographic units into a plurality of RTP packets, and the plurality of RTP packets are each packetized.(a)RTP packet header and(b-1)With one or more cryptographic units,(b-2)With one crypto unit fragment(B) One or more payloads of a common data stream, selected from a group of(c)With one RTP payload format header for each payloadIncludingThe RTP payload format header has an arbitrary block size for the corresponding cryptographic unit.Data to hold encrypted block boundariesIncludingMi,There is a separation of audio data and video data in the RTP packet so as not to include the mixed media payload.The RTP packet header of each packet contains information about the separation of the audio data and the video data, and the data for holding the encrypted block boundary for each payload is the original encrypted and packetized audio. And the video data is saved so that it can be restored by decryptionA device characterized by that. 任意のブロックサイズを有するデータストリームを暗号化して、複数の暗号ユニットを形成する手段と、拡張システム形式(ASF)の音声および映像のデータのパケットをRTPパケットに変換する手段であって、前記音声および映像のデータは、別個にパケット化される手段と、各ペイロードに対する暗号化ブロック境界を保持するためのデータを保存する手段であって、各ペイロードに対する前記暗号化ブロック境界は、ASFフォーマットの対応する別個の音声ペイロードと映像ペイロードとにより決定される手段と、別個のRTPパケットに異なるデータストリームのペイロードを書き込む手段と、 前記複数の暗号ユニットを、複数のRTPパケットにパケット化する手段と を備える装置であって、前記複数のRTPパケットはそれぞれ、(a)RTPパケットヘッダと、(b-1)1つまたは複数の暗号ユニットと、(b-2)1つの暗号ユニットのフラグメントとからなるグループから選択される、(b)共通データストリームの1つまたは複数のペイロードと、(c)各ペイロードに対する1つのRTPペイロード形式ヘッダとを含み、前記RTPペイロード形式ヘッダは、対応する前記暗号ユニットに対して、任意のブロックサイズの暗号化ブロック境界を保持するためのデータを含み、混合メディアペイロードを含まないように、前記RTPパケット内に音声データと映像データとの分離があり、各パケットの前記RTPパケットヘッダは、前記音声データと映像データとの前記分離に関する情報を含み、各ペイロードに対する前記暗号化ブロック境界を保持するためのデータは、暗号化されパケット化された元の音声および映像のデータを暗号解除によって復元することができるように、保存されることを特徴とする装置。
- 6A device comprising means for logically separating the media data types in a data stream, including a plurality of media data types, and means for forming a plurality of RTP packets from the data stream, and each RTP packet is a device. Corresponding to only one media data type, one RTP packet header, one or more variable length RTP payload format headers, each with one or more attributes, and each RTP payload format header, said With the RTP packet described by the above one or more attributes in each RTP packet format headerThere is a separation of audio data and video data in the RTP packet so as to include and not include the mixed media payload, and the RTP packet header of each packet contains information about the separation of the audio data and video data. , The information is stored so that the original packetized audio and video data can be restored.A device characterized by that. 複数のメディアデータタイプを含む、データストリーム中の前記メディアデータタイプを論理的に分離する手段と、 前記データストリームから複数のRTPパケットを形成する手段と を備える装置であって、各RTPパケットは、 ただ1つのメディアデータタイプと、 1つのRTPパケットヘッダと、 それぞれが1つまたは複数の属性を有する、1つまたは複数の可変長RTPペイロード形式ヘッダと、 前記各RTPペイロード形式ヘッダに対応し、前記各RTPペイロード形式ヘッダ中の前記1つまたは複数の属性によって記述されるRTPペイロードと を含み、混合メディアペイロードを含まないように、前記RTPパケット内に音声データと映像データとの分離があり、各パケットの前記RTPパケットヘッダは、前記音声データと映像データとの前記分離に関する情報を含み、前記情報は、パケット化された元の音声および映像のデータを復元することができるように保存されることを特徴とする装置。
- 11The steps of encrypting a data stream with an arbitrary block size to form multiple cryptographic units,A step of converting a packet of Advanced Systems Format (ASF) audio and video data into an RTP packet, wherein the audio and video data is packetized separately.A step of storing data for holding a cryptographic block boundary for each payload, wherein the cryptographic block boundary for each payload is determined by ASF's corresponding separate audio and video payloads. It is a step of packetizing the plurality of cryptographic units into a plurality of RTP packets, and the plurality of RTP packets are each packetized.(a)RTP packet header and(b-1)With one or more cryptographic units,(b-2)Selected from a group consisting of fragments of one crypto unit(b)With one or more payloads of a common data stream,(c)One RTP payload format header for each of the above payloadsAnd with one RTP payload format header containing data to hold the cryptographic block boundaries of any block size for the corresponding cryptographic unit. IncludingWith steps、With the step of writing the payload of different data streams to separate RTP packetsIs a way to prepareThere is a separation of audio data and video data in the RTP packet so as not to include the mixed media payload.The RTP packet header of each packet contains information about the separation of the audio data and the video data, and the data for holding the encrypted block boundary for each payload is the original encrypted and packetized audio. And the video data is saved so that it can be restored by decryptionA method characterized by that. 任意のブロックサイズを有するデータストリームを暗号化して、複数の暗号ユニットを形成するステップと、拡張システム形式(ASF)の音声および映像のデータのパケットをRTPパケットに変換するステップであって、前記音声および映像のデータは、別個にパケット化されるステップと、各ペイロードに対する暗号化ブロック境界を保持するためのデータを保存するステップであって、各ペイロードに対する前記暗号化ブロック境界は、ASFの対応する別個の音声ペイロードと映像ペイロードとにより決定されるステップと、 前記複数の暗号ユニットを、複数のRTPパケットにパケット化するステップであって、前記複数のRTPパケットはそれぞれ、(a)RTPパケットヘッダと、(b-1)1つまたは複数の暗号ユニットと、(b-2)1つの暗号ユニットのフラグメントと からなるグループから選択される(b)共通データストリームの1つまたは複数のペイロードと、(c)前記各ペイロードに対する1つのRTPペイロード形式ヘッダであって、対応する前記暗号ユニットに対して、前記任意のブロックサイズの暗号化ブロック境界を保持するためのデータ含む1つのRTPペイロード形式ヘッダと を含むステップと、別個のRTPパケットに異なるデータストリームのペイロードを書き込むステップとを備える方法であって、混合メディアペイロードを含まないように、前記RTPパケット内に音声データと映像データとの分離があり、各パケットの前記RTPパケットヘッダは、前記音声データと映像データとの前記分離に関する情報を含み、各ペイロードに対する前記暗号化ブロック境界を保持するためのデータは、暗号化されパケット化された元の音声および映像のデータを暗号解除によって復元することができるように、保存されることを特徴とする方法。
- 12A step of reassembling the plurality of cryptographic units using the payload in the plurality of RTP packets and the respective boundary for the arbitrary block size in the respective RTP payload format headers, and the plurality. A claim further comprising the step of decrypting the cryptographic unit of the data stream to form the data stream.11The method described in. 前記複数のRTPパケット中の前記ペイロードと、 前記それぞれのRTPペイロード形式ヘッダ中の前記任意のブロックサイズに対するそれぞれの前記境界と を使用して、前記複数の暗号ユニットを再アセンブルするステップと、 前記複数の暗号ユニットを暗号解除して、前記データストリームを形成するステップと をさらに備えることを特徴とする請求項11に記載の方法。
- 13Each RTP payload format header further comprises one or more attributes of the corresponding payload, and the attributes of the corresponding payload are used to render the formed data stream.The rendering step is performed by the processor included in the client computing device to create the requested multimedia content.A claim characterized in that12The method described in. 前記各RTPペイロード形式ヘッダは、対応する前記ペイロードの1つまたは複数の属性をさらに含み、 対応する前記ペイロードの前記属性を用いて、前記形成されたデータストリームをレンダリングするステップであって、前記レンダリングするステップは、クライアントコンピューティングデバイスが備えるプロセッサによって、要求されたマルチメディアコンテンツを作成するために行われるステップをさらに備えることを特徴とする請求項12に記載の方法。
- 14A claim characterized in that the attribute in each RTP payload format header is selected from a group consisting of timing information and video compression frame information.12The method described in. 前記各RTPペイロード形式ヘッダ中の前記属性は、 タイミング情報と、 映像圧縮フレーム情報と からなるグループから選択されることを特徴とする請求項12に記載の方法。
- 15Prior to the reassembling step, the reassembling is performed on the client.Over the networkA claim comprising further comprising a step of transmitting the plurality of RTP packets.12The method described in. 前記再アセンブルするステップに先立って、前記再アセンブルが実施されるクライアントに、ネットワークを介して前記複数のRTPパケットを伝送するステップをさらに備えることを特徴とする請求項12に記載の方法。
- 16When executed, claims11A computer-readable medium comprising a machine-readable instruction that implements the method described in. 実行されると、請求項11に記載の方法を実施する機械可読命令を備えることを特徴とするコンピュータ可読媒体。
- 17A method that comprises the steps of forming multiple RTP packets from a single data stream containing multiple media data types, where each RTP packet has only one media data type and one RTP packet header, respectively. One or more variable length RTP payload format headers with one or more attributes, and corresponding to each RTP payload format header, described by the one or more attributes in each RTP payload format header. With RTP payloadThere is a separation of audio data and video data in the RTP packet so as to include and not include the mixed media payload, and the RTP packet header of each packet contains information about the separation of the audio data and video data. , The information is stored so that the original packetized audio and video data can be restored.A method characterized by that. 複数のメディアデータタイプを含む1つのデータストリームから、複数のRTPパケットを形成するステップを備える方法であって、各RTPパケットは、 ただ1つのメディアデータタイプと、 1つのRTPパケットヘッダと、 それぞれが1つまたは複数の属性を有する、1つまたは複数の可変長RTPペイロード形式ヘッダと、 各RTPペイロード形式ヘッダに対応し、前記各RTPペイロード形式ヘッダ中の前記1つまたは複数の属性によって記述されるRTPペイロードと を含み、混合メディアペイロードを含まないように、前記RTPパケット内に音声データと映像データとの分離があり、各パケットの前記RTPパケットヘッダは、前記音声データと映像データとの前記分離に関する情報を含み、前記情報は、パケット化された元の音声および映像のデータを復元することができるように保存されることを特徴とする方法。
- 18A step of extracting the payload from the plurality of RTP packets and a step of rendering each payload in the plurality of RTP packets using the one or more attributes in the corresponding RTP payload format header.The rendering step is performed by the processor included in the client computing device to create the requested multimedia content.A claim characterized by further comprising17The method described in. 前記複数のRTPパケットから前記ペイロードを抽出するステップと、 前記複数のRTPパケット中の各ペイロードを、対応する前記RTPペイロード形式ヘッダ中の前記1つまたは複数の属性を用いてレンダリングするステップであって、前記レンダリングするステップは、クライアントコンピューティングデバイスが備えるプロセッサによって、要求されたマルチメディアコンテンツを作成するために行われるステップと をさらに備えることを特徴とする請求項17に記載の方法。
- 19A claim characterized in that the attribute in each RTP payload format header is selected from a group consisting of timing information and video compression frame information.18The method described in. 各RTPペイロード形式ヘッダ中の前記属性は、 タイミング情報と、 映像圧縮フレーム情報と からなるグループから選択されることを特徴とする請求項18に記載の方法。
- 20The step of extracting the payload from the plurality of RTP packets is to adjacent the plurality of parts, which are one of the media data types, to each RTP payload including the plurality of parts, which is one of the media data types. For each RTP payload that includes a portion that is one of the media data types, a step that assembles the portion that is one of the media data types into an adjacent payload, and the media. For each RTP payload that contains one fragment of a portion of one of the data types, further includes assembling all of the fragments of that portion of one of the media data types into adjacent payloads. A claim characterized by18The method described in. 前記複数のRTPパケットから前記ペイロードを抽出するステップは、 前記メディアデータタイプの1つである複数の部分を含む各RTPペイロードごとに、前記メディアデータタイプの1つである前記複数の部分を、隣接するペイロードにアセンブルするステップと、 前記メディアデータタイプの1つである一部分を含む各RTPペイロードごとに、前記メディアデータタイプの1つである前記一部分を、隣接するペイロードにアセンブルするステップと、 前記メディアデータタイプの1つである一部分のうち1つのフラグメントを含む各RTPペイロードごとに、前記メディアデータタイプの1つである前記一部分の前記フラグメントすべてを、隣接するペイロードにアセンブルするステップと をさらに含むことを特徴とする請求項18に記載の方法。
- 21The steps of assembling the adjacent payloads in their respective time-series order corresponding to the plurality of media data types of the media file, and the adjacent payloads of the media file in the time-series order, which are the plurality of media data types of the media file. A claim characterized by further including steps to render at the same time.20The method described in. メディアファイルの前記複数のメディアデータタイプに対応するそれぞれの時系列順に、前記隣接するペイロードをアセンブルするステップと、 前記メディアファイルの前記複数のメディアデータタイプである、前記時系列順に隣接するペイロードを、同時にレンダリングするステップと をさらに備えることを特徴とする請求項20に記載の方法。
- 23On the processorWhen executed, claims18Carry out the method described inComputerComputer readable, characterized by having readable instructionsRecordMedium. プロセッサで実行されると、請求項18に記載の方法を実施するコンピュータ可読命令を備えることを特徴とするコンピュータ可読記録媒体。
- 24A method comprising the step of converting multiple single media packets into one mixed packet, where each single media packet is encrypted and has a payload of one data stream with an arbitrary block size. A payload header for the payload.SaidAny block sizeAgainstThe mixed packet corresponds to the plurality of single media packets and is one or more payloads of a similar data stream, including a payload header including a boundary, and each of the plurality of single media packets. Includes one or more payloads corresponding to the payload and a payload profile format header for each payload in the mixed packet, the payload profile format header corresponding to the payload header of the plurality of single media packets. The payload profile format header is the payload boundary of each payload in the mixed packet.Data to holdAnd said payload boundaryData to holdIdentify the order of the payload in the plurality of single media packetsAnd the original encrypted data is saved so that it can be restored by decryptionA method characterized by that. 複数の単一メディアパケットを1つの混合パケットに変更するステップを備える方法であって、 各単一メディアパケットは、 暗号化されており、任意のブロックサイズを有する、1つのデータストリームのペイロードと、 前記ペイロードに対するペイロードヘッダであって、前記任意のブロックサイズに対する境界を含むペイロードヘッダと を含み、前記混合パケットは、前記複数の単一メディアパケットに対応し、 同様のデータストリームの1つまたは複数のペイロードであって、前記複数の単一メディアパケットのそれぞれのペイロードに対応する1つまたは複数のペイロードと、 前記混合パケット中の各ペイロードに対するペイロードプロファイル形式ヘッダであって、前記複数の単一メディアパケットの前記ペイロードヘッダに対応するペイロードプロファイル形式ヘッダと を含み、前記ペイロードプロファイル形式ヘッダは、前記混合パケット中のそれぞれのペイロードのペイロード境界を保持するためのデータを有し、前記ペイロード境界を保持するためのデータは、前記複数の単一メディアパケット中の前記ペイロードの順序を識別し、暗号化された元のデータを暗号解除によって復元することができるように、保存されることを特徴とする方法。
- 25The mixed packet is a plurality of single media packets.For eachA packet header corresponding to a packet header and a constituent unit are further included, and the constituent unit includes a plurality of payloads each having a corresponding payload profile format header, and one payload and a corresponding payload profile format header. A claim characterized in that it is selected from a group consisting of24The method described in. 前記混合パケットは、 前記複数の単一メディアパケットの各々に対するパケットヘッダに対応する1つのパケットヘッダと、 構成単位と をさらに含み、前記構成単位は、 それぞれが、対応するペイロードプロファイル形式ヘッダを有する複数のペイロードと、 1つのペイロードおよび対応するペイロードプロファイル形式ヘッダと からなるグループから選択されることを特徴とする請求項24に記載の方法。
- 26Each single media packet has the physical characteristics of the underlying network, the packet size management policy, andSaidA claim characterized in that it is smaller than a predetermined size, which is a function selected from a group consisting of an evaluation of the transmission bandwidth of the underlying network.24The method described in. 各単一メディアパケットは、 基底ネットワークの物理特性と、 パケットサイズに関する管理方針と、前記基底ネットワークの伝送帯域幅の評価と からなるグループから選択される関数である所定のサイズより小さいことを特徴とする請求項24に記載の方法。
- 27A claim, wherein the data stream is selected from a group consisting of audio data, video data, program data, JPEG data, HTML data, and MIDI data.24The method described in. 前記データストリームは、音声データ、映像データ、プログラムデータ、JPEGデータ、HTMLデータ、およびMIDIデータからなるグループから選択されることを特徴とする請求項24に記載の方法。
- 28Each mixed media packetAdvanced Systems Format (ASF)It contains a portion of the data stream, an ASF packet header, and at least one ASF payload header, each single media packet comprising an RTP packet header, one RTP payload format header, and a portion of the RTP data stream. Claim24The method described in. 各混合メディアパケットは、拡張システム形式(ASF)のデータストリームの一部分、ASFパケットヘッダ、および少なくとも1つのASFペイロードヘッダを含み、 各単一メディアパケットは、RTPパケットヘッダ、1つのRTPペイロード形式ヘッダ、およびRTPデータストリームの一部分を含むことを特徴とする請求項24に記載の方法。
- 29A claim characterized in that the payload profile format header includes a fixed length portion and a variable length portion, the variable length portion including the corresponding attributes of the payload.24The method described in. 前記ペイロードプロファイル形式ヘッダは、固定長部分および可変長部分を含み、 前記可変長部分は、対応する前記ペイロードの属性を含むことを特徴とする請求項24に記載の方法。
- 31On the processorWhen executed, claims24Carry out the method described inComputerComputer readable, characterized by having readable instructionsRecordMedium. プロセッサで実行されると、請求項24に記載の方法を実施するコンピュータ可読命令を備えることを特徴とするコンピュータ可読記録媒体。
Independent claims21
66 paragraphs, as filed
The present invention relates to Real Time Transport Protocol (RTP), and more particularly to RTP wire formats for streaming media (eg, audio, video) over networks such as the Internet.
The following description assumes that the reader is knowledgeable about Non-Patent Document 1 and Non-Patent Document 2.
Real-time Transport Protocol (RTP), as defined in Non-Patent Document 1, is an end suitable for applications that transmit real-time data, such as voice, video, or simulation data, over multicast or unicast network services. Provides two-end network transport capabilities. These transport functions provide end-to-end delivery services for data with real-time characteristics, such as interactive audio and video. Such services include payload type identification, sequence numbering, time stamping, and delivery monitoring. RTP, when provided by an underlying network, uses multicast distribution to support data transfer to multiple destinations.
Non-Patent Document 1 does not provide any mechanism for ensuring timely delivery and does not guarantee other quality of service, but relies on lower layer services to do so. Non-Patent Document 1 does not guarantee delivery or prevent random delivery, and does not assume that the underlying network is reliable and that packets are delivered in order. The sequence number contained in the RTP allows the receiver to restore the packet sequence on the sender, but the sequence number does not necessarily decode the packet in the sequence, for example in video decoding, but is appropriate for the packet. It can also be used to determine position.
Common RTP applications include streaming data, in which packets of extended system format (ASF) audio-visual (AV) data are sent from the server to the client over the network in the form of RTP packets. Or it is sent peer-to-peer. ASF audio and video data can be stored together in a single ASF packet. In this way, the RTP packet can include both audio and video data.
RTP does not have the flexibility to group multiple payloads together into a single RTP packet and split one payload into multiple RTP packets, as defined in Non-Patent Document 1. In addition, Non-Patent Document 1 does not define a format in which metadata can be delivered together with each payload in an RTP packet. Another drawback of Non-Patent Document 1 is a mechanism for streaming data encryption blocks throughout the network, and each encryption block allows the receiver of the encryption block to decrypt the data encryption block. There is no mechanism to hold the block boundary of. Providing such flexibility as an enhancement to RTP streaming would be an advance in the field.
<nplcit num="1"><text>IETF RFC 1889 standard --RTP: A Transport Protocol for Real-Time Applications</text></nplcit><nplcit num="2"><text>IETF RFC 1890 standard --RTP Profile for Audio and Video Conferences with Minimal Control</text></nplcit>
<p> Therefore, there is a need for improved methods, computer-readable media, data structures, devices, and computing devices that can provide such flexibility.</p>
<p> In one implementation, packets of enhanced system format (ASF) audio-visual (AV) data are repacketized into real-time transport protocol (RTP) packets in response to a request to stream the AV data. , Sent from server to client over the network, or by peer-to-peer network communication. AV data is encrypted to form a cryptographic unit. The repacketization process involves packetizing the cryptographic unit into RTP packets, where each RTP packet has an RTP packet header, one or more payloads of a common data stream, and an RTP payload format (PF) header for each payload. including. RTP The PF header contains boundaries for the payload for the corresponding crypto unit. The payload in an RTP packet can be one or more cryptographic units or a fragment of the cryptographic unit. After the RTP packet is sent over the network, the cryptographic units contained in the received RTP packet are reassembled. The reassembly process uses the payload in the RTP packet and its boundaries in each RTP PF header. The reassembled cryptographic unit can be decrypted for rendering. Each RTP PF header can have a payload attribute that corresponds to each RTP PF header that can be used to render the payload.</p><p> In a variant of the above implementation, data in a format other than ASF is used to form the RTP packet. In yet another variant of the above implementation, the RTP packet is formed to include an unencrypted payload.</p><p> In yet another implementation, encrypted blocks of Windows® Media Digital Rights Management (WM DRM) protected data are streamed across the network in the form of RTP packets (for example, WM DRM protected). A wire format for (streaming content) is provided. Each RTP packet contains header data to hold the cryptographic block boundary, so that each cryptographic unit can be decrypted by the receiver of the cryptographic unit. Decryption using the WM DRM protocol allows the stream data to be rendered by the receiver.</p>
The embodiments disclosed herein define a wire format for delivering single and mixed data streams, such as Windows® media data, over the Real-Time Transport Protocol (RTP). .. Such distribution may be between the server and the client, or in a peer-to-peer situation (eg, a Windows® Messenger® audiovisual conferencing software environment).
The wire format extends Non-Patent Document 1 in various implementations to make RTP distribution more flexible. Some implementations provide a mechanism for streaming audio data in an RTP packet, which is separate from the video data in the RTP packet. Some implementations also provide a wire format in which metadata that provides a wealth of information describing the payload can be delivered with each payload in an RTP packet. Yet another embodiment streams the encrypted blocks of data across the network while preserving the block boundaries of each encrypted block so that the recipient of the encrypted blocks can decrypt the encrypted blocks of the data. Provides a mechanism. In another embodiment, the wire format provides delivery of data protected by Windows® Media Digital Rights Management (WM DRM) so that the delivery of data does not have to be encrypted for rendering. ..
The various implementations disclosed herein repackage the data into a series of media packets contained within the system layer bitstream. Such data is packetized into RTP packets in a way that further enhances it in accordance with Non-Patent Document 1 so that the system layer bitstream is mapped to RTP. In this mapping, each media packet contains one or more payloads. Within some system layer bitstreams, there may be mixed media packets containing data such as audio data, video data, program data, JPEG data, HTML data, MIDI data and the like. A mixed media packet is a media packet in which two or more of its payloads belong to different media streams.
Various implementations apply to system tier bitstreams, where each media packet is a single media packet. In a single media packet, all payloads in the media packet belong to the same media stream. Other implementations apply to system tier bitstreams where each media packet always contains only one payload. In yet another embodiment, the size of the "payload header" in the media packet is zero, which is often the case when each media packet contains only a single payload, where the media packet header is each payload. It can also occur if there are multiple payloads containing information about the size of.
Figures 1 and 2 show an exemplary implementation in which a system layer bitstream contains a series of Advanced Systems Format (ASF) packets, each of which has data. Such data is packetized into RTP packets in a manner further enhanced in accordance with Non-Patent Document 1. Thus, the system layer bitstream contains a series of media packets that are ASF packets, and the payload in each ASF packet is the ASF payload. Although ASF packets are used as an example, the creation of RTP packets is not limited to the use of ASF format data in other implementations disclosed herein and contains streamed data. You can use other formats that are used. These other formats, as well as the ASF format, are generally described herein as system layer bitstreams containing multiple media packets, each of which has data, and such data is mapped to RTP in various implementations.
ASF stream audiovisual (AV) data 100 is shown in FIG. The ASF stream AV data 100 includes audio data 102 and video data 104, and is packetized into ASF packet A106 and ASF packet B108. The ASF packet A106 includes a first ASF header, an ASF payload header, audio data 102, a second ASF header, and a video data A fragment of video data 104. The ASF packet B108 includes an ASF header, an ASF payload header, and a video data B fragment of the video data 104.
The ASF stream AV data 100 represented by the ASF packet A106 and the ASF packet B108 can be packetized into a plurality of RTP packets in one implementation. As seen in FIG. 1, these plurality of RTP packets include RTP packet A110, RTP packet 112 (1) to RTP packet 112 (N), and RTP packet D116. Each RTP packet has an RTP packet header, a payload, and an RTP payload format (PF) header, according to Non-Patent Document 1. RTP used herein The PF header is the payload header in the RTP packet. There is only one type of media in the RTP packet. In other words, RTP packets do not contain mixed media payloads. In the implementation shown in FIG. 1, the video data A of the ASF packet A106 is too large to fit in a single RTP packet. Therefore, the video data A of the ASF packet A106 is divided into the RTP packet 112 (1) and the RTP packet 112 (N). The size of an RTP packet is the physical characteristics of the base network through which the RTP packet should be transmitted, or the packet size management policy such that it can be created by the base network administrator, or the transmission bandwidth of the base network. It can be a function of evaluation.
Continuing the description of the RTP packetization shown in FIG. 1, the voice data 102 is included in the RTP packet A110, and the video data B of the ASF packet B108 is included in the RTP packet D116. Each RTP PF header in each RTP packet can contain information about the separation of audio and video data into separate RTP packets. Therefore, the A / V stream sample data 124 includes the audio data in the RTP packet A110, the video data A fragment 1 to the video data A fragment N in the respective RTP packets 112 (1) to 112 (N), and the RTP packet. It can be restored from the video data B in D116. When the restoration of the A / V stream sample data 124 is completed, the audio sample data 120 and the video sample data A + B 122 contained therein can be rendered in the streaming situation. Given the above, Figure 1 shows a wire format in which smaller RTP packets are created from larger ASF packets, in which the packetization results in its own RTP. Payloads for different data streams are written to separate packets, each with a PF header. Figure 1 preserves the block boundaries for each payload so that the original audio and video samples encrypted and packetized into ASF packets can be restored by the decryption mechanism implemented on the RTP packet. The implementation form of the wire format to be used is also shown.
The ASF stream AV data 200 is shown in Figure 2. The ASF stream AV data 200 includes video data 202 and is packetized into ASF packet A208 and ASF packet B210. The ASF packet A208 includes an ASF header, an ASF payload header, and video data A204. The ASF packet B210 includes an ASF header, an ASF payload header, and video data B206. FIG. 2 shows two alternative forms of packetizing ASF stream AV data 200 into RTP packets in a way that further enhances it in accordance with Non-Patent Document 1.
In the first alternative, following arrow 250, video data A204 and video data B206 are packetized into a single RTP alternative packet A212 with an RTP header. There is an RTP PF header before each of the video data A204 and the video data B206. According to Non-Patent Document 1, the RTP alternative packet A212 has an RTP header, a plurality of payloads, and each RTP PF header.
In the second alternative form, also following the arrow 250, the video data A204 and the video data B206 from the respective ASF packets are packetized into an RTP alternative packet B214 with an RTP header. The video data A204 and the video data B206 are assembled adjacently as a payload in the RTP alternative packet B214. This payload is preceded by the RTP PF header. The RTP alternative packet B214 has an RTP header, a payload, and one RTP PF header, according to Non-Patent Document 1.
Following the RTP packetization shown in FIG. 2, the video data A and B (204, 206) are included in either the RTP alternative packet A212 or the RTP alternative packet B214. Each RTP PF header can contain information about the corresponding payload. Alternate RTP packets 212, 214 contain sufficient data to restore ASF packet A208 and ASF packet B210, respectively, to capture video data A and B (204, 206) therein. When the restoration of the video sample data 222 is completed, the video sample data 222 can be rendered in a streaming situation. Given the above, Figure 2 shows an RTP wire format in which larger RTP packets are created from smaller ASF packets, in which the original encrypted and packetized into two ASF packets. The block boundaries for each payload are preserved so that the video sample can be restored by the decryption mechanism performed on the RTP packet.
Figure 3a shows a layout of the data structure for the fields in the RTP header. The RTP header is more fully described in Non-Patent Document 1. The timestamp field in the RTP header should be set to the presentation time of the sample contained in the RTP packet. In one implementation, the clock frequency is 1 kHz unless specified by any other value by RTP-independent means.
The eighth bit from the beginning of the RTP header is interpreted as a marker (M) bit field. The M-bit is set to zero, but the corresponding RTP packet has a payload that is not a fragment of the sample, a payload that contains the final fragment of the sample, or a payload that is one of several complete samples in an RTP packet. Whenever it is set to 1. The M bits can be used by the receiver to detect the receipt of a complete sample for decoding and presentation. Therefore, the M bits in the RTP header can be used to mark critical events in the packet stream (eg, video sample frame boundaries).
Figure 3b shows an implementation of the RTP Payload Format (PF) header, i.e. the payload header. The RTP PF header has a 16-bit fixed length portion followed by a variable length portion. The fields in the RTP PF header shown in Figure 3b are the character field "SGLRTDXZ", the length / offset field, the relative timestamp field, the decompression time field, the duration field, and the payload extension (PE) length field and their corresponding. It contains an 8-bit string represented by PE data fields, each of which is described below.
The S field is 1 bit in length and 1 if the corresponding payload (eg, sample, sample fragment, or combination of samples) is a key sample, that is, an intracoded sample, or an I-frame. Is set to. Otherwise, it is set to zero. The S bits in all RTP PF headers that precede fragments of the same sample must be set to the same value.
The G-field, which is 1 bit in length, is used to group the lower samples in the corresponding payload that make up a single sample. Windows® Media Digital Rights Management (WM DRM) encrypts content based on "ASF payload" boundaries. To ensure that this content is properly decrypted, the boundaries of the lower samples in the payload can be propagated to the client that should receive the payload. For example, a cryptographic unit can be packetized so that it is separated into multiple transmission units to be transmitted (eg, placed in separate packets). The delimited transmission units must be reassembled into their original form of encryption before they can be decrypted by the receiving client. As with other decryption methods and mechanisms, clients can use boundaries to properly restore encrypted cryptographic units in preparation for decryption of encrypted content. Therefore, this RTP PF header must precede each "ASF payload".
The bit in the G field should be set to zero ("0") to indicate that the encrypted "unit" is fragmented. When ASF is used, the cryptographic unit is the ASF payload and its bits are set to zero ("0") in all fragmented ASF payloads except the last ASF payload. In this case, it doesn't matter if the sample is fragmented. If ASF is not used, the crypto unit is the media sample, in which case the G bit is set to zero ("0") in all fragmented media samples except the last sample. In the latter case, the question of whether the ASF payload is fragmented is not appropriate because ASF is not used.
The L field is set to 1 if the length is 1 bit and the Length / Offset field contains the length. Otherwise, it is set to zero ("0") and the length / offset field contains the offset. The L bit must be set to 1 in all RTP PF headers that precede the complete (non-fragmented) sample in the corresponding payload and all that precede the payload that contains the fragmented sample. Must be set to zero in the RTP PF header.
The R field is set to 1 if it is 1 bit in length and the RTP PF header contains a relative timestamp. Otherwise, it is set to zero. The R bits in all headers that precede fragments of the same sample must be set to the same value.
The T field is set to 1 if it is 1 bit in length and the RTP PF header contains the decompression time. Otherwise, it is set to zero. The T bits in all RTP PF headers that precede a payload containing the same sample fragment must be set to the same value.
The D field is set to 1 if it is 1 bit in length and the RTP PF header contains a sample duration. Otherwise, it is set to zero. The D bits in all RTP PF headers that precede a payload containing the same sample fragment must be set to the same value.
The X field is 1 bit in length and is for optional or unspecified use. The sender of the RTP packet should set this bit to zero, and the receiver can ignore this bit.
The Z field is set to 1 if the RTP PF header contains payload extension (PE) data that is 1 bit in length and can be metadata about the corresponding payload. Otherwise, the Z field is set to zero. The bits in the Z field can be zero for all RTP PF headers where the M bits are zero, but if the corresponding payload has associated PE data, then all M bits are set to 1. Should be set for the RTP PF header of.
The length / offset field represents the length or offset value of a single sample that is 24 bits in length and is fragmented into multiple RTP packets. The L bit is set to zero and the length / offset field contains the byte offset of the first byte of this fragment (eg, sample or sample fragment) from the beginning of the corresponding payload. If the RTP packet contains one or more complete samples, the L bit is set to 1 in each RTP PF header and the sample length / offset field is the sample length (including the RTP PF header). including.
The Relative Timestamp field is only shown if it is 32 bits in length and the R bit is set to 1. This field contains the relative time stamp for the corresponding sample with respect to the time stamp in the corresponding RTP header. The timescale used is the same as that used for the timestamp in the RTP header. The relative timestamp field is specified as a signed 32-bit number to allow a negative offset from the timestamp of the RTP header. If the relative timestamp field does not exist, you can use the default relative timestamp, which is zero.
The Decompression Time is shown only if it is 32 bits in length and the T bit is set to 1. The decompression time includes the decompression time relative to the time stamp in the RTP header. The timescale used is the same as that used for the timestamp in the RTP header. This field is specified as a signed 32-bit number to allow a negative offset from the time stamp in the RTP header.
The Duration field is shown only if it is 32 bits in length and the D bit is set to 1. This field contains the duration of the corresponding sample. The timescale used is the same as that used for the timestamp in the RTP header. The duration field should be set to the same value in all RTP PF headers that precede fragments of the same sample. If this field does not exist, the default duration is taken automatically or explicitly from the sample data. If this is not possible, the default will be different for this sample timestamp and the next sample timestamp.
The Payload Extension (PE) Data Length field is shown only if the length is 16 bits and the Z bit is set to 1. This field contains the number of bytes of PE data contained after the fixed part of the RTP PF header. The PE data is variable length and contains one or more attributes that describe the corresponding payload preceded by the PE data. The PE data length field immediately follows the fixed part of the payload header and is a few bytes containing the actual PE data. The structure of PE data is communicated between the client and server (or peer-to-peer), for example via a description of SDP. In one implementation for WM DRM protected content, there can be at least 4 bytes of DUE data representing the WM DRM payload ID associated with every sample.
Figures 3a-3b show the various fields for the RTP and RTP PF headers in different orders, but not all fields are required and the order can be reordered. Therefore, in some implementations, the required fields and order can be further extended according to the flexibility of Non-Patent Document 1. Although ASF packets are used for the purposes of FIGS. 3a-3b, the creation of RTP packets, RTP PF headers and payloads is therefore the use of ASF-formatted data in other embodiments disclosed herein. You can also use other formats that store the data to be streamed, rather than being limited to.
General network structure FIG. 4 shows a client / server network system 400 and environment according to the present invention. In general, system 400 includes one or more (m) network multimedia servers 402 and one or more (k) network clients 404. The computers communicate with each other via a data communication network including the wired / wireless network 406 in FIG. The data communication network 406 can also include the Internet, or a local area network and a dedicated wide area network. The server 402 and client 404 communicate with each other via any of a wide variety of known protocols, such as Transmission Control Protocol (TCP) and User Datagram Protocol (UDP).
The multimedia server 402 / client 404 has access to the contents of the streaming media in various media stream formats. Such a media stream may be a separate media stream (eg, audio, video, image, simulation, etc.) or a composite media stream containing a plurality of such separate streams. Some media streams can be stored as files 408 in a database (for example, an ASF file) or other file storage system, while other media streams 410 can be stored on the multimedia server 402 or client from other data source components. The 404 can be supplied "live" via a dedicated communication channel or the Internet itself.
The media stream received from the server 402 or client 404 is rendered on the client 404 as a multimedia representation that can contain media streams from one or more of the server 402 / client 404. These various media streams can include one or more of the same or different types of media streams. For example, one multimedia representation can include two video streams, one audio stream, and one graphic image stream. The user interface (UI) on the client 404 can give the user various controls, for example, allowing the user to speed up or slow down the rendering of the media representation.
Illustrative computer environment In the following description, the present invention will be described in the general context of computer-executable instructions such as program modules executed by one or more conventional personal computers. In general, a program module includes routines, programs, objects, components, data structures, etc. that perform a particular task or implement a particular abstract data type. Further, the present invention may be implemented with other computer system configurations such as handheld devices, multiprocessor systems, microprocessor-based home appliances or programmable home appliances, network PCs, minicomputers, mainframe computers, and the like. Those who can do it will understand. In a distributed computer environment, program modules can be located in both local and remote memory storage. Alternatively, the invention can be implemented in hardware or in a combination of hardware, software, and / or firmware. For example, one or more special purpose integrated circuits (ASICs) can be programmed to carry out the present invention.
As shown in FIG. 4, the network system according to the invention includes a network server (group) 402 and a client 404 from which multiple media streams are available. In some cases, the media stream is actually stored by network server (group) 402 and client 404. Otherwise, network server (group) 402 and client (group) 404 can obtain media streams from other network sources or devices. Generally, the network client 404 is responsible for user input for requesting a media stream corresponding to the selected multimedia content. In response to a request for a media stream corresponding to multimedia content, the network server (group) 402 and client 404 stream the requested media stream to the requesting network client 404 according to the RTP wire format. Client 404 decrypts the payload in each RTP packet and renders the resulting unencrypted data stream to create the requested multimedia content.
Figure 5 shows the input and storage of A / V stream data to server 402 or client 404 (eg, peer). Figure 5 also shows communication between the server and client (402-404) or peer-to-peer (404-404) in various implementations. As an overview, server 402 or client 404 receives input of A / V stream data from input device 503. The server 402 or client 404 uses the codec encoder to encode the input. Encoding can be performed on ASF format data, but it does not have to be performed. When ASF format data is used, encoding is performed on ASF packets, each containing an ASF header, an ASF payload header, and an AV (audio and / or video) payload. Encoding can include encryption, for example if WM DRM is used. ASF packets are stored by server 402 / client 404 to serve future requests for the same.
The client then requests the corresponding AV data stream from the server / client. The server / client retrieves the corresponding AV stream stored by the server / client and sends it to the client. Upon receipt, the client decodes the AV data stream and restores and decrypts the encrypted delimited AV data stream sample using the boundaries carried in the corresponding RTP PF header. The client can then perform rendering of the streamed AV data.
The data flow can be seen in blocks 504-530 in FIG. At block 504, the input device supplies (502) an input containing A / V stream data to the server 402 / client 404. As an example, A / V stream data may be supplied (502) to server 402 / client 404 via a dedicated communication channel or the Internet in a "live" state by an input device. The A / V stream data is supplied at block 504 to the encoder to place the data in the ASF packet. At block 506, optional WM DRM encryption is performed and ASF packets are stored on server 402 / client 404. As a result of WM DRM encryption and packetization, cryptographic units can be separated into multiple separate packets. The delimited transmission units must be reassembled to the original cryptographic unit at the receiving client before they can be decrypted at the receiving client. In this way, the delimited transmission unit boundaries are stored in the ASF payload header at block 506.
At block 508, client 404 requests an A / V data stream to be sent to server 402 / client 404, as seen by arrow 510 in Figure 5. At block 512, server 402 / client 404 receives this request. The corresponding ASF packet containing the requested A / V data stream is retrieved. At block 514, the audio and video payloads in the ASF packet are logically separated so that they can be packetized separately into RTP packets. The boundaries of each logically distinct audio and video payload are identified.
Determines the bandwidth of the network on which RTP packets should be transmitted. This determination is used to derive a given RTP packet size. If the ASF packet size is smaller than a given RTP packet size, similar payloads can be combined into a single RTP packet. If the ASF packet size is larger than a given RTP packet size, the ASF payload can be fragmented because it is placed as one payload in a single RTP packet. Boundaries for each RTP payload are determined using the corresponding logically separate audio and video payloads of the ASF packet.
In step 516, the RTP header, the RTP PF header, and their respective payloads are assembled for each RTP packet. In this way, a plurality of RTP packets representing a plurality of ASF packets are formed, and the ASF packet includes the A / V data stream requested by the client 404. The RTP packet is streamed from server 402 / client 404 via a transfer function at block 518 for being rendered at client 404.
Arrow 520 in FIG. 5 indicates the transmission of RTP packets from server 402 / client 404 to client 404. At block 522, client 404 receives an RTP packet. At block 524, the client 404's RTP decoder decodes each received RTP packet, including the RTP header and RTP PF header. At block 526, processing performs defragmentation and defragmentation of the ASF packet containing the requested A / V data stream. Defragmentation and restoration utilize the boundaries set in the RTP PF header for each corresponding payload, for example, a sample or a fragment of a sample.
At block 528, the restored ASF packet is decrypted for rendering at block 530. The RTP PF header in an RTP packet can contain payload extension (PE) data that describes the corresponding payload. Therefore, PE data can provide metadata that can be used during the rendering of the payload in the corresponding RTP packet at block 530. Blocks 522-530 are repeated for each RTP packet received by client 404, thereby achieving streaming of A / V data from server 402 / client 404 for rendering.
FIG. 6 shows a general example of a computer 642 that can be used in accordance with the present invention. Computer 642 is shown as an example of a computer capable of performing any of the functions of client 404 or server 402 in Figures 4-5. The computer 642 includes one or more processors or processors 644, a system memory 646, and a system bus 648 that connects various system components such as the system memory 646 to the processor 644.
Bus 648 is one of several types of bus structures, including memory buses or memory controllers that use any of the various bus architectures, peripheral buses, accelerated graphics ports, and processor or local buses. Represents one or more. System memory includes read-only memory (ROM) 650 and random access memory (RAM) 652. Cache 675 has levels L1, L2, and L3 and can be included in RAM 652. The basic input / output system (BIOS) 654 contains, for example, basic routines that help transfer information between elements inside computer 642 during boot, and is stored in ROM 650. The computer 642 includes a hard disk drive 656 that reads from and writes to a hard disk (not shown), a magnetic disk drive 658 that reads from or writes to a removable magnetic disk 660, and a CD ROM. It further includes an optical disk drive 662 that reads from or writes to a removable optical disk 664, such as another optical medium.
The hard disk (not shown), magnetic disk drive 658, optical disk drive 662, or removable optical disk 664 may all be information media having recorded information. Such an information medium has a data area for recording stream data using stream packets, each of which contains a packet area containing one or more data packets. As an example, each data packet is encoded and decoded by the codec of application program 672 that runs on processing device 644. Therefore, the encoder distributes the stream data to the data packet area in the stream packet, so that the distributed stream data is recorded in the data packet area using the encoding algorithm. Alternatively, the encoding and decoding of data packets can be performed as a function of operating system 670 running on processor 644.
Hard disk drive 656, magnetic disk drive 658, and optical disk drive 662 are connected to system bus 648 by SCSI interface 666 or any other suitable interface. The drive and associated computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer 642. The exemplary environment described herein uses a hard disk, a removable magnetic disk 660, and a removable optical disk 664, but other types that can store computer-accessible data. It will be appreciated by those skilled in the art that computer readable media such as magnetic cassettes, flash memory cards, digital video disks, random access memory (RAM), read-only memory (ROM), etc. can also be used in exemplary operating environments. ..
Several program modules, such as operating system 670, one or more application programs 672 (which can contain codecs), other program modules 674, and program data 676, have hard disks, magnetic disks 660, disk disks 664, ROM 650. , Or can be stored in RAM652. The user can enter commands and information into the computer 642 through input devices such as the keyboard 678 and the pointing device 680. Other input devices (not shown) may include microphones, joysticks, gaming pads, satellite dishes, scanners, and the like. These input devices and other input devices are connected to the processing device 644 via an interface 682 coupled to the system bus. A monitor 684 or other type of display is also connected to system bus 648 via an interface such as a video adapter 686. In addition to monitors, personal computers typically include other peripheral output devices (not shown), such as speakers and printers.
Computer 642 operates in a networked environment using a logical connection to one or more remote computers, such as remote computer 688. The remote computer 688 may be another personal computer, server, router, network PC, peer device, or other common network node, usually containing many or all of the above-mentioned elements associated with computer 642, but with memory storage. Only device 672 is shown in Figure 6. The logical connections shown in Figure 6 include a local area network (LAN) 690 and a wide area network (WAN) 692. Such network environments are common in enterprises, enterprise-scale computer networks, intranets, and the Internet. In a described embodiment of the present invention, the remote computer 688 runs an Internet web browser program, eg, an Internet Explorer® web browser manufactured and sold by Microsoft Corporation in Redmond, Washington.
When used in a LAN network environment, the computer 642 is connected to the local network 690 via a network interface or adapter 694. When used in a WAN network environment, the computer 642 typically includes a modem 696, or other means of establishing communication over a wide area network 692, such as the Internet. Modem 696 may be internal or external and can be connected to system bus 648 via serial port interface 668. In a networked environment, the program module illustrated in connection with the personal computer 642 or a portion thereof can be stored in a remote memory storage device. It will be appreciated that the network connections shown are exemplary and other means of establishing communication links between computers can also be used.
Generally, the data processor of a computer 642 is programmed using instructions stored at different times on various computer-readable storage media of the computer. Programs and operating systems are typically distributed, for example, on floppy (registered trademark) disks or CD-ROMs. From there, it is installed or loaded into your computer's auxiliary storage. At runtime, it is at least partially loaded into the computer's electronic main memory. If such media includes instructions or programs that implement the steps described in the claims in connection with a microprocessor or other data processor, the invention described herein is such computer readable. Includes storage media and various other types of computer-readable storage media. The present invention also includes the computer itself when programmed according to the methods and techniques described in the claims. In addition, certain subcomponents of the computer can be programmed to perform the functions and steps described in the claims. The present invention includes such subcomponents, if programmed as described. In addition, the invention described herein includes the data structures described in the claims as embodied on various types of storage media.
For purposes of illustration, programs such as operating systems and other executable program components are described herein as separate blocks, but such programs and components are different storage elements of the computer. It will be understood that it exists within and is executed by a computer's data processor (s).
Conclusion The implementations disclosed herein define a wire format that can be used in the delivery of multimedia data over RTP, between servers and clients, and peer-to-peer. This wire format allows for greater flexibility than Non-Patent Document 1 currently adopted for RTP distribution. The wire format implementation implements streaming of encrypted data, provides a mechanism for delivering sample metadata via RTP, and implements WM DRM-protected data streaming.
Although the present invention is described in terms specific to operation for structural features and / or methods, the invention, as defined in the claims, also works for the particular features described. Please understand that is not always limited. Instead, these particular features and behaviors are disclosed as exemplary embodiments of the claimed invention.
<figref num="1">It is a figure which shows the exemplary process which converts two packets of the audio-video (AV) data of extended system format (ASF) into four RTP packets by embodiment of this invention. The audio and video data are packetized separately into the resulting RTP packets, and the block boundaries for each payload are encrypted into two ASF packets and the original packetized AV sample is restored by the decryption mechanism. It is saved so that it can be done.</figref><figref num="2">It is a figure which shows the exemplary alternative processing which converts two packets of ASF video data into one RTP packet by the different embodiment of this invention. One alternative process moves the payload of the ASF packet to a separate payload in the RTP packet, and the other alternative process combines the payloads of the ASF packet to create the combined payload in the RTP packet, each The block boundaries for the payload are stored so that the original video sample, encrypted and packetized into two ASF packets, can be restored by the decryption mechanism.</figref><figref num="3a">It is a layout drawing which shows the data structure of the RTP header by embodiment of this invention.</figref><figref num="3b">It is a layout drawing which shows the data structure of the payload header corresponding to the RTP header of FIG. 3a according to the Embodiment of this invention.</figref><figref num="4">FIG. 6 is a block diagram showing a networked client / server system that can be streamed to a client by a server or peer-to-peer according to an embodiment of the present invention.</figref><figref num="5">FIG. 5 is a block diagram showing communication between a server (or client) and a client according to an embodiment of the present invention. The server (or client) provides the client with a requested audio-video data stream that the client can render.</figref><figref num="6">FIG. 6 is a block diagram showing a networked computer that can be used to implement either a server or a client according to an embodiment of the invention.</figref>
Code description
100 ASF stream A / V data 102 Voice data 104 Video data 106 ASF packet A 108 ASF packet B 110 RTP packet A 112 (1) RTP packet 112 (N) RTP packet 116 RTP packet D 120 audio sample data 122 Video sample data A + B 124 A / V sample data 200 ASF stream video data 202 Video data 204 Video data A 206 Video data B 208 ASF packet A 210 ASF packet B 212 RTP Alternate Packet A 222 Video sample data 406 Wired / Wireless Network (Group) 408 stream data file 410 Stream data 402 Multimedia server (1) 402 Multimedia server (m) 404 client (1) 404 client (k) 503 input device 644 Processing equipment 646 system memory 648 bus 670 operating system 666 SCSI interface 668 Serial port 672 Application program (eg codec) 674 Other modules 675 cache 676 Program data 678 keyboard 682 keyboard / mouse interface 686 video adapter 690 Local Area Network 692 Wide area network 694 network interface 696 modem
Every citation, both waysCites: the store holds 3 of 4
| Document | Relation | Office |
|---|---|---|
| JP2000287192A | Cites | Japan |
| JP2002044135A | Cites | Japan |
| WO02051096A1 | Cites | World Intellectual Property Organization (WIPO) |
| Abdelhamid Nafaa,Toufik Ahmed,Yassine Hadjadj Aoul,Ahmed Mehaoua,RTP4MUX:A NOVEL MPEG-4 RTP PAYLOAD FOR MULTICAST VIDEO COMMUNICATIONS OVER WIRELESS IP,IEEE-PV 2003,13TH INTERNATIONAL PACKET VIDEO WORKSHOP,2003年 4月28日 | Non-patent | – |
| メディア配信技術,NTT R&D 第52巻 第1号,2003年 1月10日 | Non-patent | – |
5 priority claims, no other members on record
Priority claims5
| Document | Office | Kind | Date |
|---|---|---|---|
| 10612851 | United States of America | – | |
| 61285103 | United States of America | A | |
| 61285103 | United States of America | A | |
| 2003612851 | – | – | – |
| US20030612851 | – | – | – |
22 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Cancellation because of no payment of annual feesLAPS | LAPS | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Written notification of registration of transferJAPANESE INTERMEDIATE CODE: R350R350 | R350 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Request for change of ownership or part of ownershipJAPANESE INTERMEDIATE CODE: R313113S111 | S111 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Renewal fee payment (event date is renewal date of database)FPAY | FPAY | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Renewal fee payment (event date is renewal date of database)FPAY | FPAY | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 4504749
- Publication, DOCDB
- 4504749
- Publication, EPODOC
- JP4504749B
- Application
- 198165
- Application, DOCDB
- 2004198165
- Application, EPODOC
- JP20040198165
Titles2
- Japanese
- RTPペイロード形式
- English
- RTP payload format
Classification
- CPC, 18
- H04L63/0457
- H04R1/32
- H04N21/2347
- H04N21/2381
- H04N21/4143
- H04N21/42623
- H04N21/472
- H04N21/4788
- H04N21/6437
- H04L69/03
- H04L65/65
- H04L65/70
- H04R1/02
- H04R2201/02
- H04L65/60
- H04N21/00
- H04L9/40
- H04L65/1101
- IPC, 6
- H04L12 56
- H04N7 08
- H04N7 081
- H04L47 43
- H04N7 167
- H04N21 6437