Receiver driven up-switching in video telephony
25 claims: 12 independent, 13 dependent
- 1データを処理する方法であって、 第1のビットレートでネットワークを介してデータを送信することと、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定することと、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する ための 回復ビットレートを決定することと、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機器が、 前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定することと、 前記回復ビットレート及び 前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定することとを備え、 ここにおいて、前記回復ビットレートを決定することが、前記ネットワークリンクレートの前記低下の大きさに基づくアンダーシュート係数を使用して前記回復ビットレートを決定することを備える、 方法。
- 2前記第1のネットワークリンクレートと前記第2のネットワークリンクレートとの間の差として前記アンダーシュート係数を決定することと、前記第1のネットワークリンクレートによって前記差を除することとを更に備える、請求項1に記載の方法。
- 3前記回復ビットレートを決定することが、1と前記アンダーシュート係数との間の差と乗じられた前記第2のネットワークリンクレートとして前記回復ビットレートを決定することを備える、請求項2に記載の方法。
- 4前記回復 ビット レート 及び前記バッファリング 時間長 に基づいて回復レート時間長 を決定することが、式 ΔT u =ΔT(R 0 -R 1 )/(f U R 1 )に基づいて前記回復レート時間長を 決定することを備え、ここにおいてΔT u が前記回復レート時間長を表し、ΔTが 前記バッファリング時間長を表し、R 0 が前記第1のネットワークリンクレートを表し、R 1 が前記第2のネットワークリンクレートを表し、f U が前記ネットワークリンクレートの前記低下 の前記大きさ に基づく前記アンダーシュート係数を表す、請求項1に記載の方法。
- 5データを処理する方法であって、 第1のビットレートでネットワークを介してデータを送信することと、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定することと、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定することと、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定することと、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定することとを備え、ここにおいて、前記回復ビットレートを決定することが最小ビットレートより大きい回復ビットレートを決定することを備え、前記最小ビットレートがオーディデータ及びビデオデータの少なくとも1つのデータを符号化するように構成されるエンコーダの符号化ビットレートに基づく、方法。
- 6データを処理する方法であって、 第1のビットレートでネットワークを介してデータを送信することと、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定することと、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定することと、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定することと、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定することと、 前記回復レート時間長の間に前記データを送信する間に、前記第2のネットワークリンクレートから第3のネットワークリンクレートへの前記ネットワークリンクレートの増加を特定することと、 前記ネットワークリンクレートの前記増加を特定したことに応答して、前記回復ビットレートより高いビットレートへ前記回復ビットレートを増加することと、 を備える、方法。
- 7データを処理する方法であって、 第1のビットレートでネットワークを介してデータを送信することと、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定することと、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定することと、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定することと、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定することとを備え、ここにおいて、前記バッファリング時間長を決定することが、受信機機器から送信機機器へのダウンリンク遅延と関連付けられるデータ、レート制御反応遅延と関連付けられるデータ、混雑制御の反応遅延と関連付けられるデータ又はメッセージ生成遅延と関連付けられるデータの少なくとも1つに基づいて、前記バッファリング時間長を決定することとを備える、方法。
- 8データを処理する方法であって、 第1のビットレートでネットワークを介してデータを送信することと、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定することと、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定することと、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定することと、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定すること、 それらの間で前記データが送信される、送信機機器から受信機機器への順方向チャネルの推定された最大ビットレートを示すデータを受信することと、 受信品質フィードバックを示すデータを受信することとを、備え、 前記バッファリング時間長を決定することが、前記推定された最大ビットレートを示す前記データ及び前記受信品質フィードバックを示す前記データに基づいて、前記バッファリング時間長を決定することを備える、方法。
- 9前記推定された最大ビットレートを示す前記データを受信することが、一時最大メディアストリームビットレート要求(TMMBR)メッセージを受信することを備え、前記受信品質フィードバックを示す前記データを受信することが、受信機報告(RR)メッセージを受信することを備える、請求項8に記載の方法。
- 10前記推定された最大ビットレートを示す前記データを受信し、前記受信品質フィードバックを示す前記データを受信することが、前記推定された最大ビットレートを示す前記データと、前記受信品質フィードバックを示す前記データとを含む単一のメッセージを受信することを備える、請求項8に記載の方法。
- 11前記推定された最大ビットレートを示す前記データ及び前記受信品質フィードバックを示す前記データに基づいて前記バッファリング時間長を決定することが、第1のメッセージが受信機機器に送信される時間と、前記推定された最大ビットレートを示す前記データ及び前記受信品質フィードバックを示す前記データが受信される時間との間の差を決定することを備える、請求項8に記載の方法。
- 12データを処理する方法であって、 第1のビットレートでネットワークを介してデータを送信することと、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定することと、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定することと、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定することと、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定することとを備え、ここにおいて前記データが符号化されたビデオデータを備え、前記方法は、決定された前記回復レート時間長の間前記回復ビットレートで前記データを送信することを更に備え、決定された前記回復レート時間長の間前記回復ビットレートで前記データを送信することが、前記符号化されたビデオデータの前記回復ビットレートを決定された前記回復レート時間長の間の前記回復ビットレートに下げるようにレート制御を実行することを備える、方法。
- 13データを処理するための 機器 であって、 データを記憶するように構成されたメモリと、 1つ以上のプロセッサとを備え、前記1つ以上のプロセッサが、 第1のビットレートでネットワークを介して 前記 データを送信し、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定し、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定し、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、 前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定し、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定し、 前記回復ビットレートを決定するために、前記ネットワークリンクレートの前記低下の大きさに基づくアンダーシュート係数を使用して前記回復ビットレートを決定する ように構成される、機器。
- 14前記1つ以上のプロセッサが更に、前記第1のネットワークリンクレートと前記第2のネットワークリンクレートとの間の差として前記アンダーシュート係数を決定し、前記第1のネットワークリンクレートによって前記差を除するように構成される、請求項13に記載の機器。
- 15前記回復ビットレートを決定するために、前記1つ以上のプロセッサが、1と前記アンダーシュート係数との間の差と乗じられた前記第2のネットワークリンクレートとして前記回復ビットレートを決定するように構成される、請求項14に記載の機器。
- 16前記回復ビットレート及び前記バッファリング時間長に基づいて前記回復レート時間長を決定するために、前記1つ以上のプロセッサが、式 ΔT u = ΔT (R 0 -R 1 ) / (f U R 1 ) に基づいて前記回復レート時間長を決定するように構成され、ΔT u が前記回復レート時間長を表し、ΔTが前記バッファリング時間長を表し、R 0 が前記第1のネットワークリンクレートを表し、R 1 が前記第2のネットワークリンクレートを表し、f U が前記ネットワークリンクレートの前記低下の前記大きさに基づく前記アンダーシュート係数を表す、請求項13に記載の機器。
- 17データを処理するための機器であって、 データを記憶するように構成されたメモリと、 1つ以上のプロセッサとを備え、前記1つ以上のプロセッサが、 第1のビットレートでネットワークを介して前記データを送信し、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定し、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定し、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定し、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定し、ここにおいて、前記回復ビットレートを決定するために、前記1つ以上のプロセッサが、最小ビットレートより大きい回復ビットレートを決定するように構成され、前記最小ビットレートがオーディオデータ及びビデオデータの少なくとも1つのデータを符号化するように構成されるエンコーダの符号化ビットレートに基づく、機器。
- 18データを処理するための機器であって、 データを記憶するように構成されたメモリと、 1つ以上のプロセッサとを備え、前記1つ以上のプロセッサが、 第1のビットレートでネットワークを介して前記データを送信し、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定し、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定し、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定し、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定し、 前記回復レート時間長の間に前記データを送信する間に、前記第2のネットワークリンクレートから第3のネットワークリンクレートへの前記ネットワークリンクレートの増加を特定し、 前記ネットワークリンクレートの前記増加を特定したことに応答して、前記回復ビットレートより高いビットレートへ前記回復ビットレートを増加するように構成される、 機器。
- 19データを処理するための機器であって、 データを記憶するように構成されたメモリと、 1つ以上のプロセッサとを備え、前記1つ以上のプロセッサが、 第1のビットレートでネットワークを介して前記データを送信し、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定し、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定し、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定し、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定するように構成され、 前記バッファリング時間長を決定するために、前記1つ以上のプロセッサが、受信機機器から送信機機器へのダウンリンク遅延と関連付けられるデータ、レート制御反応遅延と関連付けられるデータ、混雑制御の反応遅延と関連付けられるデータ又はメッセージ生成遅延と関連付けられるデータの少なくとも1つに基づいて、前記バッファリング時間長を決定するように構成される、機器。
- 20データを処理するための機器であって、 データを記憶するように構成されたメモリと、 1つ以上のプロセッサとを備え、前記1つ以上のプロセッサが、 第1のビットレートでネットワークを介して前記データを送信し、 第1のネットワークリンクレートから第2のネットワークリンクレートへの前記ネットワークのネットワークリンクレートの低下を特定し、 前記ネットワークリンクレートの前記低下を特定したことに応答して、前記ネットワークを介して前記データを送信する際に用いるべき回復ビットレートを決定し、ここにおいて、前記回復ビットレートは前記第2のネットワークリンクレートより低い、 送信機機器が、前記ネットワークリンクレートの前記低下の前記特定の時間と、前記ネットワークリンクレートの前記低下の推定される実際の時間との間の差に基づいて、バッファリング時間長を決定し、 前記回復ビットレート及び前記バッファリング時間長に基づいて、前記回復ビットレートで前記データを送信すべき回復レート時間長を決定し、 それらの間で前記データが送信される、送信機機器から受信機機器への順方向チャネルの推定された最大ビットレートを示すデータを受信し、 受信品質フィードバックを示すデータを受信し、 前記バッファリング時間長を決定するために、前記推定された最大ビットレートを示す前記データ及び前記受信品質フィードバックを示す前記データに基づいて、前記バッファリング時間長を決定するように構成される、機器。
- 21前記推定された最大ビットレートを示す前記データを受信するために、前記1つ以上のプロセッサが、一時最大メディアストリームビットレート要求(TMMBR)メッセージを受信するように構成され、前記受信品質フィードバックを示す前記データを受信するために、前記1つ以上のプロセッサが、受信機報告(RR)メッセージを受信するように構成される、請求項20に記載の機器。
- 22前記推定された最大ビットレートを示す前記データを受信し、前記受信品質フィードバックを示す前記データを受信するために、前記1つ以上のプロセッサが、前記推定された最大ビットレートを示す前記データと、前記受信品質フィードバックを示す前記データとを含む単一のメッセージを受信するように構成される、請求項20に記載の機器。
- 23前記推定された最大ビットレートを示す前記データ及び前記受信品質フィードバックを示す前記データに基づいて前記バッファリング時間長を決定するために、前記1つ以上のプロセッサが、第1のメッセージが受信機機器に送信される時間と、前記推定された最大ビットレートを示す前記データ及び前記受信品質フィードバックを示す前記データが受信される時間との間の差を決定するように構成される、請求項20に記載の機器。
- 24前記データが符号化されるビデオデータを備え、前記1つ以上のプロセッサが更に、決定された前記回復レート時間長の間前記回復ビットレートで前記データを送信するように構成され、決定された前記回復レート時間長の間前記回復ビットレートで前記データを送信するために、前記1つ以上のプロセッサが、前記符号化されたビデオデータの前記回復ビットレートを決定された前記回復レート時間長の間前記回復ビットレートに下げるようにレート制御を実行するように構成される、請求項13に記載の機器。
- 25集積回路、 マイクロプロセッサ、又は ワイヤレス通信機器の少なくとも1つを備える、請求項13に記載の機器。
Independent claims25
147 paragraphs, as filed
[0001] The application claims the benefit of US Provisional Application No. 62 / 030,513 filed July 29, 2014, the entire contents of which are incorporated herein by reference.
[0002] The present disclosure relates to the processing of video data.
[0003] Video telephony (VT) involves real-time communication of packets carrying audio and video data. VT equipment includes a video encoder that acquires video from an imaging device such as a video camera or video archive to generate video packets. Similarly, the audio encoder in the VT device acquires audio from an audio capture device such as a microphone or speech synthesizer and generates an audio packet. Video packets and audio packets are placed in wireless link protocol (RLP) queues. The medium access control (MAC) layer unit generates a medium access control (MAC) layer packet from the contents of the RLP queue. MAC layer packets are converted to physical (PHY) layer packets for transmission across the communication channel to another VT device.
[0004] In mobile VT applications, the VT device receives physical layer packets via a wireless forward link (FL) (or "downlink") from the base station to the VT device as a wireless terminal. The VT device transmits a PHY layer packet over a wireless reverse link (RL) (or "uplink") to the base station. Each VT device includes a PHY layer and a MAC layer to convert received PHY layer packets and MAC layer packets and reassemble the packet payload into audio and video packets. The video decoder in the VT device decodes the video data for presentation to the user via the display device. The audio decoder in the VT device decodes the audio data for output through the audio speaker.
[0005] The technique of the present disclosure relates to determining the bit rate for encoding data based on network conditions. For example, aspects of the present disclosure relate to reducing the transmit bit rate (or simply also referred to as the rate) from a first rate to a second rate in response to a decrease in network link rate. According to aspects of the present disclosure, identifying a reduction in network link rate allows the transmitter device to reduce the transmission rate to a reduced rate below the second network link rate, eg, a second network. You can undershoot the link rate. The transmitter device is based on the reduced rate, and in the meantime data is buffered in the transmitter device or another device associated with the network, identifying a decrease in network link rate and decreasing the network link rate. The transmission rate can be maintained at a reduced rate for a period of time, based on the length of time between reacting to. In this way, the transmitter device can reduce the amount of data buffered during the decrease in network link rate relatively quickly without unduely affecting the user experience.
[0006] Aspects of the present disclosure also relate to increasing the transmission rate in cases where the network link rate is not fully utilized. For example, according to aspects of the present disclosure, the receiver device fully utilizes the network link rate based on the fact that the data is received by the receiver device before the time scheduled for the data to be played back. It can be determined that it has not been done. The receiver device can determine the acceptable excess delay parameter based on the time difference between the time the data is received and the time the data is scheduled to be reproduced. The receiver device can determine the increase in transmission rate according to the acceptable excess delay parameter. The transmitter device can better utilize the network link rate without exceeding the network link rate, since the receiver device can, in some cases, send an instruction to increase the transmission rate to the transmitter device. be able to.
[0007] In one example, the method of processing data is to send data over the network at the first bit rate and the network link rate of the network from the first network link rate to the second network link rate. Identifying the degradation and determining the recovery bit rate to use when transmitting data over the network in response to identifying the degradation of the network link rate, where the recovery bit rate is the second. Determining the buffering time length and the recovery bit rate based on the difference between the specific time of the network link rate drop and the estimated actual time of the network link rate drop, which is lower than the network link rate. And to determine the recovery rate time length for which data should be transmitted at the recovery bit rate based on the buffering time length.
[0008] In another example, the device for processing the data includes a memory configured to store the data and one or more processors, the one or more processors being the first. Sending data over the network at a bit rate, identifying a decrease in the network link rate of the network from the first network link rate to the second network link rate, and in response to identifying the decrease in the network link rate, Determine the recovery bit rate to use when transmitting data over the network, where the recovery bit rate is lower than the second network link rate, the specific time of the network link rate decline and the estimation of the network link rate decline. Determine the buffering time length based on the difference from the actual time to be done, and determine the recovery rate time length for which data should be transmitted at the recovery bit rate based on the recovery bit rate and buffering time length. It is configured as follows.
[0009] In another example, the device for processing data is a means for transmitting data over a network at a first bit rate and a network from a first network link rate to a second network link rate. A means for identifying a decrease in the network link rate and a means for determining the recovery bit rate to be used when transmitting data over the network in response to the identification of the decrease in the network link rate. In, the recovery bit rate is lower than the second network link rate, the buffering time length based on the difference between the specific time of the network link rate decline and the estimated actual time of the network link rate decline. A means for determining the recovery rate time length for transmitting data at the recovery bit rate based on the recovery bit rate and the buffering time length.
[0010] In another example, a non-temporary computer-readable medium, when executed, causes one or more processors to transmit data over a network at a first bit rate, starting with a first network link rate. Identify the decrease in the network link rate of the network to the network link rate of 2, and in response to the identification of the decrease in the network link rate, determine the recovery bit rate to be used when transmitting data over the network, here. In, the recovery bit rate is lower than the second network link rate, the buffering time length based on the difference between the specific time of the network link rate decline and the estimated actual time of the network link rate decline. Is stored, and an instruction is stored to determine the recovery rate time length for transmitting data at the recovery bit rate based on the recovery bit rate and the buffering time length.
[0011] In another example, the method of processing the data is by the receiver device between the time the received data is received by the receiver device and the time the received data is scheduled to be replayed. The acceptable excess delay parameter is determined based on the difference between, and here the acceptable excess delay parameter indicates the amount of delay that can be accommodated by the channel between the transmitter equipment and the receiver equipment. Determining an increase in the transmitter bit rate to increase the bit rate to be used when data is transmitted from the transmitter device to the receiver device, based on the allowable excess delay parameter determined by the device. Includes sending instructions to the transmitter equipment to increase the transmitter bit rate.
[0012] In another example, the receiver device for processing the data includes a memory configured to store the data and one or more processors, where the one or more processors are the data. Determines an acceptable excess delay parameter based on the difference between the time received by the receiver device and the time the data is scheduled to be regenerated, where is acceptable. The excess delay parameter indicates the amount of delay that can be accommodated by the channel between the transmitter device and the receiver device, and data is transmitted from the transmitter device to the receiver device based on a determined acceptable excess delay parameter. It is configured to determine an increase in the transmitter bit rate to increase the bit rate to be used in the event and send an instruction to increase the transmitter bit rate to the transmitter equipment.
[0013] In another example, the device for processing the data is the difference between the time the received data is received by the receiver device and the time the received data is scheduled to be replayed. Based on the means for determining the acceptable excess delay parameter, where the acceptable excess delay parameter is determined, indicating the amount of delay that can be accommodated by the channel between the transmitter and receiver equipment. Means for determining the increase in the transmitter bit rate to increase the bit rate to be used when data is transmitted from the transmitter device to the receiver device, and the transmitter bit, based on the acceptable excess delay parameters. Includes means for transmitting rate increase instructions to the transmitter equipment.
[0014] In another example, a non-temporary computer-readable medium, when executed, replays the received data to one or more processors at the time the received data is received by the receiver device and the received data. The allowable excess delay parameter is determined based on the difference from the scheduled time so that the acceptable excess delay parameter can be accommodated by the channel between the transmitter device and the receiver device. Based on a determined allowable excess delay parameter that indicates the amount of delay, let the transmitter bit rate increase to increase the bit rate that should be used when data is transmitted from the transmitter device to the receiver device. , The instruction to increase the transmitter bit rate is sent to the transmitter device, and the instruction is stored.
[0015] Details of one or more examples of the present disclosure are set forth in the accompanying drawings and in the description below. Other features, objectives and advantages will become apparent from the description, drawings and claims.
<figref num="1">[0016] A block diagram showing an audio / video coding and decoding system for video telephone (VT) applications.</figref><figref num="2">[0017] A block diagram illustrating a video coding system capable of performing video source rate matching according to the techniques of the present disclosure.</figref><figref num="3">[0018] A block diagram illustrating a video decoding system capable of performing video source rate matching according to the techniques of the present disclosure.</figref><figref num="4A">[0019] A graph showing a video source rate matching technique according to the techniques of the present disclosure.</figref><figref num="4B">The graph which shows the video source rate adaptation technique by the technique of this disclosure.</figref><figref num="5">[0020] A conceptual diagram showing the determination of the buffering time length by the technique of the present disclosure.</figref><figref num="6A">[0021] A graph showing a decrease in network link rate.</figref><figref num="6B">A graph showing the corresponding delay time.</figref><figref num="7A">[0022] A graph showing a decrease in network link rate.</figref><figref num="7B">A graph showing the corresponding delay time.</figref><figref num="8">[0023] A flow chart illustrating an exemplary process for downswitching the rate at which data is transmitted, according to the techniques of the present disclosure.</figref><figref num="9">[0024] A flow chart illustrating an exemplary process for upswitching the rate at which data is transmitted, according to the techniques of the present disclosure.</figref>
[0025] Video telephone (VT) devices may be connected via a wired or wireless network for conducting VT sessions (eg, transmitting audio and / or video data between VT devices). A VT device that is processing audio and / or video data for transmission to another VT device may be referred to as a transmitter device. Similarly, a VT device processing received audio and / or video data (eg, for presentation to a user of the VT device) may be referred to as a receiver device.
[0026] The transmitter device can encode audio data and / or video data at a particular rate (which may be interchangeably referred to herein as a bit rate). The transmitter device can select the rate based on the network conditions. For example, the transmitter device can select a rate based on the maximum (or close to) network link rate supported by the network used for the VT session. In this way, the transmitter device can prepare the data for transmission using the relatively higher quality supported by the network without exceeding the limits of the network.
[0027] In some cases, the network link rate connecting VT devices can vary, especially when using VT over wireless networks such as Wi-Fi® or cellular networks. In some cases, network equipment may use buffers to deal with fluctuations in link rates and / or to perform queue management. For example, a transmitter device may include a buffer for buffering encoded audio and / or video data before transmitting the data to the receiver device. A sudden drop in network link rate can cause bottlenecks that can adversely affect VT sessions. For example, a transmitter device for accumulating encoded video data in a buffer when the network link rate drops, which can cause interruptions and / or jerkiness in a VT session at the receiver device.
[0028] Transmitter equipment is the rate at which video data is transmitted in response to a decrease in network link rate (sometimes referred to herein as transmission rate, rate refers to the bit rate throughout the disclosure. (Used) can be changed. In some examples, the transmitter device can change the transmission rate by changing the rate at which the audio and / or video data are encoded. However, there may be a delay in the reaction when lowering the rate due to a delay in the congestion control feedback of the receiver device, a delay in the return path from the receiver device to the transmitter device, a delay in the rate matching reaction, and the like. Therefore, the transmission rate can remain well above the network link rate for a period of time from the decline in the network link rate. Inconsistencies between transmit rates and network link rates can lead to increased buffer levels at bottleneck links, thus increased end-to-end delays (and even packet loss), which is a quality experience for VT sessions. May have an adverse effect on.
[0029] In addition, the accumulated delay may remain for some time even after the transmitter equipment has reduced the transmission rate in response to the decrease in network link rate. For example, delay can generally refer to the time between the time data is available for transmission across a network link and the time the data is actually transmitted to the network. Therefore, delay can be associated with data buffering. For example, an increase in delay results in an increase in buffer levels, because the data must be stored after encoding and before transmission to the network.
[0030] Depending on the difference between the transmit rate and the bottleneck network link rate, the transmitter equipment may reduce the quality of the buffered data relatively slowly. That is, if the difference between the reduced transmit rate and the network link rate is relatively small, the transmitter equipment may reduce the accumulated delay relatively slowly, leaving an impact on the VT session. ..
One approach to reducing the amount of buffered data is to reduce the transmit rate below the estimated network link rate. Using relatively conservative techniques, such as transmission rates well below the estimated network link rate, can result in inadequate use of links and an overall reduction in the video quality experience at the receiver device. .. However, such a conservative approach can also reduce the buffer of bottleneck links relatively quickly. Conversely, a relatively aggressive approach, such as simply reducing the transmit rate to the network link rate, can result in full use of the link and higher quality encoded data. However, as mentioned above, such techniques may cause the data to remain in the buffer for a relatively long period of time.
[0032] Aspects of the present disclosure relate to determining a transmission rate (eg, a bit rate for encoding audio and / or video data in a transmitter device) based on network conditions. Specifically, the technique involves reducing the transmit rate in response to a decrease in the network link rate. According to aspects of the present disclosure, identifying a reduced network link rate allows the transmitter device to reduce the transmission rate to a rate below the network link rate. In some examples, the receiver device can then request a reduced transmission rate implemented by the transmitter device. Reducing the transmit rate below the network link rate is sometimes referred to as undershooting the network link rate.
The technique also involves determining the amount of time to maintain the transmit rate at a reduced rate. In some examples, aspects of the present disclosure recover based on buffering time length, magnitude of network link rate reduction, rate reduction factor and / or other factors, as described in more detail below. Includes determining the rate time length (also called the undershoot period). In this way, the technique can be used to determine the optimal undershoot period. For example, a transmitter device maintains a reduced transmission rate only if necessary to reduce the amount of buffered data before returning to the increased transmission rate supported by the network. be able to. The technique can achieve a balance between the conservative and aggressive techniques described above, so the amount of buffered data is relatively fast without overly impacting the user experience. Can be reduced.
[0034] Aspects of the present disclosure also include signaling delayed data associated with processing encoded audio and / or video data. The techniques of the present disclosure include generating data for use in determining buffering time lengths in transmitter equipment. The buffering time length can be associated with a delay between the actual decrease in network link rate and the time that the decrease in network link rate is detected (eg, the transmitter and / or receiver equipment is of network link rate. Suppose you do not immediately recognize the decline and respond to it). During this delay time, the transmitter equipment usually buffers the prepared / encoded data at the original transmission rate, which is transmitted in real time (or near real time) at the reduced network link rate. Is not possible. Data buffering creates a delay in the receiver device during which no data is received. As mentioned above, the buffering time can be used to determine the amount of data buffered and / or the recovery rate time length in the transmitter equipment.
[0035] Another aspect of the present disclosure may relate to increasing the transmission rate in cases where the network link rate is not fully utilized. For example, the transmitter device can increase the data transmission rate in order to improve the quality of the user experience case where the transmission rate is less than the link rate that can be supported by the network connecting the transmitter device to the receiver device. Increasing the bit rate at which data is encoded can be referred to herein as upswitching. However, upswitching the transmit rate with an increment that is too large can result in an overshoot of the network link rate, which can adversely affect the user experience as described. Conversely, upswitching the transmit rate with an increment that is too small can result in continued undershoot of the network link rate, which results in a lower quality user experience than what the network link rate can support. Sometimes.
[0036] According to aspects of the present disclosure, the receiver device is underutilizing the network link rate based on the fact that the data is received before the time scheduled for the data to be regenerated. Can be determined. The receiver device can determine the acceptable excess delay parameter based on the difference between the time the data is received and the time the data is scheduled to be played, and the receiver device is acceptable. The increase in transmission rate can be determined according to the excess delay parameter. The receiver device can, in some cases, send an instruction to increase the transmission rate to the transmitter device, so that the transmitter device has a better network link rate without overshooting the network link rate. It can be used.
[0037] Accordingly, aspects of the present disclosure are for controlling video flow originating from a transmitter device and transmitted over a network channel (also referred to as a network link) with a time-varying bandwidth to the receiver device. Includes rate matching or congestion control techniques. Specifically, the technique involves up-switching the average bit rate of the video flow in a controlled manner to improve the user experience without causing congestion on the network. Such rate matching techniques can avoid a significant increase in end-to-end delays that can result in packet loss.
[0038] For example, according to aspects of the present disclosure, the receiver device inspects the received video packet and the data arrives early or on time with respect to the scheduled playback of the data. You can decide if you are arriving late or if you are arriving late. If the data arrives later than the intended playback, the receiver device can determine that the network link rate is lower than the transmission rate (eg, the coding rate implemented in the transmitter device). Therefore, the receiver device can transmit a request for lowering the transmission rate to the transmitter device. In some examples, the receiver device has an initial rate lower than the sustainable rate of network link rate (eg, available bandwidth) to allow the system to clear network channel congestion. Can be requested.
[0039] In some cases, the technique described herein is a Multimedia Telephony Service for IP Multimedia Subsystem (MTSI) device for the IP Multimedia Subsystem (IMS). Can be performed by. For example, MTSI equipment can perform bit rate matching and / or congestion control using the techniques described herein.
[0040] FIG. 1 is a block diagram showing a coding and decoding system 10. As shown in FIG. 1, the system 10 includes an encoder system 12 and a decoder system 14 connected by a transmission channel 16. In the example of FIG. 1, the encoder system 12 is associated with a first video communication device, with an audio source 17, a video source 18, a video encoder 20, an audio encoder 22, and a real-time transport protocol (RTP). / Real-time Transport Protocol (RTCP) / User Datagram Protocol (UDP) / Internet Protocol (IP) / Point-to-Point Protocol (PPP) Conversion Unit 26, Wireless Link Protocol (RLP) Queue 28, and MAC Layer Unit 30 , Includes PHY layer unit 32. The decoder system 14 is associated with another video communication device, the PHY layer unit 34, the MAC layer unit 36, the RLP queue 38, the RTP / RTCP / UDP / IP / PPP conversion unit 40, the video decoder 42, and so on. It includes an audio decoder 44, an audio output device 46, and a video output device 48.
[0041] As described in more detail below, the encoder system 12 and / or the decoder system 14 can use the techniques of the present disclosure to adjust the coding rate based on network conditions. For example, the video encoder 20 can control the video source coding rate, at least in part, as a function of network bandwidth. Specifically, the video encoder 20 can reduce the coding rate of video data and / or audio data in response to a decrease in network link rate. Similarly, the video encoder 20 can increase the coding rate of video data and / or audio data in response to instructions for inadequate use of network link rates.
[0042] System 10 can provide two-way video and audio transmission, for example for video telephones over transmission channel 16. Therefore, a generally compatible coding, decoding and conversion unit may be provided on the opposite side of channel 16. In some embodiments, the encoder system 12 and the decoder system 14 can be embodied within a video communication device such as a wireless mobile terminal that supports video streaming, video telephones, or both. Mobile terminals can support VT according to packet switching standards such as RTP, RTCP, UDP, IP or PPP.
[0043] For example, in the encoder system 12, the RTP / RTCP / UDP / IP / PPP conversion unit 26 is suitable for audio data and video data received from the video encoder 20 and the audio encoder 22. Add IP / PPP header data and place the data in RLP queue 28. An exemplary bitstream may include a MAC header, an IP header, a UDP header, an RTCP header, and payload data. In some examples, RTP / RTCP runs on UDP, UDP runs on IP, and IP runs on PPP. In some examples, as described herein, RFC 3550: RTP: A Transport Protocol for Real-Time Applications, H. et al. Schulzrinne et al., July 2003, "RFC 5104: Codec Control Messages in the RTP Audio-Visual Provide with Feedback (AVPF), S. Wenger et al., February 2008 (hereafter RFC 5104) and / or others for real-time or near-real-time transport of data. RTP / RTCP / UDP / IP / PPP conversion unit 26 that conforms to a specific standard, such as applicable standards of. The MAC layer unit 30 generates a MAC RLP packet from the contents of the RLP queue 28. The PHY layer unit 32 translates MAC RLP packets into PHY layer packets for transmission over channel 16. [0044] The PHY layer unit 34 and the MAC layer unit 36 of the decoder system 14 operate with each other. The PHY layer unit 34 converts the PHY layer packet received from channel 16 into a MAC RLP packet. MAC layer unit 36 is a MAC Place the RLP packet on the RLP queue 38. The RTP / RTCP / UDP / IP / PPP conversion unit 40 steals header information from the data in the RLP queue 38 and reassembles the video data and audio data for distribution to the video decoder 42 and audio decoder 44, respectively. ..
System 10 includes code division multiple access (CDMA), frequency division multiple access (FDMA), time division multiple access (TDMA) or orthogonal frequency division multiplexing (OFDM) or another suitable wireless technique. It may be designed to support one or more wireless communication technologies. The above wireless communication technology can be realized according to any of various wireless access technologies. For example, CDMA can be implemented according to cdma2000 or wideband CDMA (WCDMA®) standards. TDMA can be implemented according to the Global System for Mobile Communications (GSM®) standard. The Universal Mobile Telecommunications System (UMTS) standard enables GSM or WCDMA operation. Typically, for VT applications, System 10 can be designed to support high data rate (HDR) technology.
[0046] The video encoder 20 produces video data encoded according to one video compression method, such as MPEG-4, High Efficiency Video Coding (HEVC) or another video coding standard. Other video compression methods include the International Telecommunication Union (ITU) H.263, ITU H.264 or MPEG-2 methods. The audio encoder 22 encodes the audio data associated with the video data. The video source 18 can be an imaging device, such as one or more video cameras, one or more video archives or a combination of video cameras and video archives.
The audio data can be encoded according to an audio compression method, such as Adaptive Multi-Rate Narrow Band (AMR-NB) or other techniques. The audio source 17 can be an audio capture device, such as a microphone or speech synthesizer device. For VT applications, video makes the VT conference visible to the participant, and audio makes the participant hear what they are saying.
[0048] In operation, the RTP / RTCP / UDP / IP / PPP conversion unit 26 acquires video data packets and audio data packets from the video encoder 20 and the audio encoder 22. As mentioned earlier, the RTP / RTCP / UDP / IP / PPP conversion unit 26 adds the appropriate header information to the audio packet and inserts the resulting data into the RLP queue 28. Similarly, the RTP / RTCP / UDP / IP / PPP conversion unit 26 adds appropriate header information to the video packet and inserts the obtained data into the RLP queue 28. The MAC layer unit 30 extracts data from the RLP queue 28 and forms a MAC layer packet. Each MAC layer packet carries RTP / RTCP / UDP / IP / PPP header information and audio packet data or video packet data contained in the RLP queue 28. Audio packets can be inserted into RLP queue 28 independently of video packets.
[0049] In some cases, the MAC layer packet generated from the contents of the RLP queue 28 carries only the header information and the video packet data. In other cases, the MAC layer packet carries only header information and audio packet data. In many cases, the MAC layer packet carries header information, audio packet data, and video packet data according to the contents of the RLP queue 28. MAC layer packets may be configured according to Radio Link Protocol (RLP) and may be referred to as MAC RLP packets. The PHY layer unit 32 translates MAC RLP audio-video packets into PHY layer packets for transmission across channel 16.
[0050] Channel 16 carries the PHY layer packet to the decoder system 14. Channel 16 can be any physical connection between the encoder system 12 and the decoder system 14. For example, channel 16 can be a wired connection, such as a local area or wide area wired network. Alternatively, as described herein, channel 16 can be a wireless connection, such as a cellular connection, satellite connection, or optical connection. Channel conditions can be a challenge for wired and wireless channels, but can be particularly relevant for mobile VT applications running through wireless channel 16, where channel conditions are exacerbated by fading or congestion. Sometimes. Channel 16 may support a particular network link rate (eg, a particular bandwidth), which may vary according to channel conditions. For example, channel 16 can be characterized by a reverse link (RL) with throughput that varies according to channel conditions.
[0051] In general, the PHY layer unit 34 of the decoder system 14 identifies a MAC layer packet from a PHY layer packet and reassembles its contents into a MAC RLP packet. The MAC layer unit 36 then reassembles the contents of the MAC RLP packet to provide the video and audio packets for insertion into the RLP queue 38. The RTP / RTCP / UDP / IP / PPP unit 40 removes the accompanying header information and provides the video packet to the video decoder 42 and the audio packet to the audio decoder 44. The video decoder 42 decodes a video data frame to generate a stream of video data for use in driving the display device. The audio decoder 44 decodes audio data, for example, to generate audio information for presentation to the user via an audio speaker.
[0052] As mentioned above, the system 10 can provide two-way video and audio transmission, for example for video telephones over transmission channel 16. In some examples, problems can occur when the network link rate of channel 16 changes, which can occur in Wi-Fi, cellular or other network links. One or more buffers may be included in the network equipment to handle rate fluctuations and, in some cases, perform queue management, as described in more detail with respect to FIG.
[0053] For example, a VT flow with a transmission rate (eg, the coding rate used by the video encoder 20) may encounter a sudden drop in link rate, which can create a flow bottleneck. .. Delays in the response of the encoder system 12 to this drop in link rate (eg, this is caused by delays in receiver congestion control feedback, delays on the return path from transmitter to receiver, delays in rate matching reactions, etc. Due to (possible), the transmission rate can remain well above the link rate for a period of time. This results in increased buffer levels at the bottleneck link, and thus can result in increased end-to-end delay (and even packet loss) between the encoder system 12 and the decoder system 14. , May adversely affect the quality experience of VT sessions.
[0054] After the encoder system 12 has reduced the bit rate as data is transmitted through channel 16 (eg, reduced transmission rate), the accumulated delay may remain for some time. For example, in some cases, the length of time that the accumulated delay remains can depend on the difference between the transmit rate and the reduced link rate (eg, the link rate that causes the bottleneck). If the reduction in transmission rate is too small, the accumulated delay will decrease relatively slowly, which can affect the user experience in the decoder system 14. A conservative transmit rate approach is to consistently transmit at a rate much lower than the estimated link rate. However, this approach can result in inadequate use of links on channel 16 and a diminished overall video quality experience.
[0055] According to the techniques described in the present disclosure, the video encoder 20 can encode video from a video source 18 based on the conditions of channel 16. Specifically, the video encoder 20 can reduce the coding rate (also referred to herein as the transmission rate) based on the reduction in bandwidth on channel 16. Lowering the coding rate can be referred to herein as downswitching. The encoder system 12 receives a video after a significant drop in the link rate on channel 16 is detected, for example, after a receiver-side congestion control feedback message generated by the decoder system 14 is received by the encoder system 12. The transmission rate of the encoded data in the encoder 20 can be temporarily lowered. [0056] In one example, according to aspects of the present disclosure, the encoder system 12 can initially transmit data through channel 16 at a first bit rate. The encoder system 12 can identify a decrease in the network link rate on channel 16 from the first network link rate to the second network link rate. In some examples, the encoder system 12 can identify a decrease in network link rate based on one or more reports received from the decoder system 14.
[0057] According to aspects of the present disclosure, in response to identifying a decrease in network link rate, the encoder system 12 may determine the recovery bit rate to be used when transmitting data through channel 16. , Here the recovery bit rate is lower than the second network link rate. The encoder system 12 can also determine the buffering time length, including the difference between the specific time of network link rate decline and the estimated actual time of network link rate decline. For example, as mentioned above, there may be some reaction time associated with identifying the delay and adjusting the rate at which the video encoder 20 encodes the data. The encoder system 12 is encoded by the video encoder 20 at or near the initial (higher) network link until the video encoder 20 has time to identify and adjust to a lower rate. The converted data can be buffered.
[0058] The encoder system 12 can determine the recovery rate time length at which data should be transmitted at the recovery bit rate, based on the recovery bit rate and the buffering time length. The encoder system can then transmit data at the recovery bit rate for a determined recovery rate duration. In this way, the technique can reduce the accumulated end-to-end delay relatively quickly, by using the link rate available after the end-to-end delay has been reduced (eg, by using the link rate available). The quality of the user experience can be preserved (rather than maintaining the transmit rate at a reduced rate for a longer period of time). Although the encoder system 12 has been described for illustration purposes, it should be understood that some of the techniques described above may be performed by the decoder system 14 in addition or alternatives.
Still other techniques of the present disclosure include techniques for up-switching (eg, increasing) the rate at which data is encoded, based on network conditions. For example, in the announcement of "Discussion on Upswitch Principals", SA4 MTSI SWG Conference Call No. 4 on End-to-End Video Rate Adaptation of E2EMTSI-S4, S4-AHM215, June 24, 2014 ("AHM215") Several problems with upswitching have been identified. "Report from SA4 MTSI SWG Conference Call No. 4 on End-to-End Video Rate Adaptation of E2EMTSI-S4 (June 24, 2014) ", Tdoc S4 (14) 0768, a conference call before agreeing on the principles of upswitching. It seemed that further discussion was needed to study the new ideas from.
[0060] In general, the model presented at AHM215 relies on a ramp-up probing model, which can have the disadvantage that probing can cause delays in the system when the probe does not match the channel conditions. .. A more robust model is to allow a receiver, such as the decoder system 14, to passively measure the state of channel 16 to determine if the system may have extra capacity. Based on this, the decoder system 14 can make a more accurate estimate of the sustainable rate of the system.
The model presented at AHM215 also proposes a two-step approach, in which the encoder system 12 first probes the channel to see if additional capabilities are possible. If the probing phase is successful, the video encoder 20 can increase the rate more aggressively during the "ramp-up phase". Such a model can result in a relatively large amount of congestion in the system, because a successful probe with a slight increase in data rate cannot imply that the system can handle a very large increase thereafter. Is. In fact, when increasing the rate of the video encoder 20 to match the capabilities of the system, a more robust approach is to first increase the rate relatively significantly and then the sustainable rate supported by channel 16. It is to make the steps smaller as the rate converges to.
[0062] To follow a potentially more robust approach that converges to a sustainable rate in the manner described above, the entity that drives the fit (eg, transmitter (encoder system 12) or receiver (decoder system 14). )) Must have an estimate of the sustainable rate of the system. The transmitter may rely on RTCP receiver reporting to detect end-to-end channel conditions and can calculate the overall throughput, but there is still some measurement delay due to RTCP reporting. The receiver can calculate both the overall throughput and the amount of additional delay that can be tolerated, in which the packet arrival at the decoder system 14 is not considered too late for the scheduled playback. Therefore, if the relevant measures calculated at the receiver are transmitted directly to the transmitter, the fit model driven by the receiver may be realized and more robust, determining the minimum fit to be performed. Should be used in doing so.
[0063] According to aspects of the present disclosure, the decoder system 14 can implement a rate-up switching technique driven by a receiver if it determines that bandwidth is not fully utilized on channel 16. For example, according to aspects of the present disclosure, the decoder system 14 can provide the encoder system 12 with data that prompts the video encoder 20 to increase the coding rate.
[0064] In some examples, according to aspects of the present disclosure, the decoder system 14 has a time between the time the data is received by the decoder system 14 and the time the received data is scheduled to be reproduced. The acceptable excess delay parameter can be determined based on the difference between. The acceptable excess delay parameter can indicate the amount of delay that channel 16 can handle, for example, the user experience is not affected, for example, the data arrives not too late to be decoded and played at the appropriate time. it can. The decoder system 14 also increases the transmitter bit rate to increase the bit rate to be used when data is transmitted from the encoder system 12 to the decoder system 14 based on the determined allowable excess delay parameters. Can be decided. The decoder system 14 can also transmit instructions to increase the transmitter bit rate to the encoder system 12.
[0065] Thus, the decoder system 14 can control the average bit rate of the video flow in a controlled manner in order to improve the user experience without causing congestion on the network. This technique can avoid significantly increasing the end-to-end delay that can result in packet loss.
FIG. 2 is a block diagram showing an encoder system 12 capable of performing video source rate matching according to the techniques of the present disclosure. As shown in FIG. 2, the video encoder 20 includes a video coding engine 50, a video buffer 52, and a video rate controller 54. The video encoder 20 also receives network link rate information 56, which may be prepared by the decoder system 14 (discussed in more detail below).
The video coding engine 50 obtains video data from the video source 18 and encodes the video data at a rate controlled by the video rate controller 54. The video coding engine 50 then stores the encoded video in the video buffer 52. The video rate controller 54 can monitor the fullness of the video buffer 52 and control the video coding rate applied by the video coding engine 50 based on at least a portion of the fullness. In addition, as described in more detail below, the video rate controller 54 controls the rate based on network link rate information 56 and / or other data associated with the conditions of channel 16 (FIG. 1). be able to.
[0068] In some examples, the video encoder 20 can provide a video source rate control scheme that is generally codec independent. For example, the video encoder 20 may be adapted for video coding according to HEVC, MPEG4, ITU H.263, or ITU H.264. In addition, the video encoder 20 may be sensitive to implementation within the DSP or embedded logic core. In some embodiments, the video encoder 20 (eg, the video rate controller 54 of the video encoder 20) can apply model-based rate control, eg, video block rate control can be applied in the rho region. it can. For example, when a frame bit plan is established for a particular video frame, the frame bit plan is a rho region rate between the video blocks within the frame, such as coding units (CU) and / or macroblocks (MB). Can be allocated using control. The rho region values for individual MBs can then be mapped to quantization parameter (QP) values.
[0069] According to aspects of the present disclosure, the video rate controller 54 can perform rate down switching based on network conditions. For example, the video coding engine 50 can initially encode data at a first bit rate for transmission through a transport medium such as channel 16 (FIG. 1). The video rate controller 54 can identify a decrease in the network link rate from the first network link rate to the second network link rate. In some examples, the video rate controller 54 can identify the decrease in network link rate from the feedback in the video encoder 20. In another example, the video rate controller 54 can identify a decrease in the network link rate based on the network link rate information 56.
[0070] In response to identifying a decrease in network link rate, the video rate controller 54 determines the recovery bit rate for the video encoder 20, which is lower than the second (reduced) network link rate. Can be done. The recovery rate can be used to reduce the amount of data buffered between the actual time of the network link rate decline and the identification of the network link rate decline. Reducing such buffered data helps ensure that the user experience is unaffected in the receiver device. Therefore, the video rate controller 54 determines the recovery bit rate for use in the video encoder 20 that undershoots the reduced network link rate to reduce the amount of data buffered in the video encoder 20. Can be done.
[0071] According to aspects of the present disclosure, the video rate controller 54 can determine the recovery rate based on the undershoot coefficient. The video rate controller 54 can determine the undershoot coefficient based on the difference between the first network link rate and the reduced network link rate. That is, the video rate controller 54 can determine the undershoot coefficient, which has a magnitude that changes based on the magnitude of the decrease in the network link rate. Therefore, if the decrease in network link rate is relatively large, the video rate controller 54 may determine a relatively large undershoot coefficient. Similarly, if the decrease in network link rate is relatively small, the video rate controller 54 may determine a relatively small undershoot coefficient.
[0072] In some examples, the video rate controller 54 can determine the undershoot factor that can be applied to the reduced network link rate to determine the recovery rate. For example, the video rate controller 54 can determine a decimal undershoot coefficient and apply the decimal undershoot coefficient to the reduced network link rate to determine the recovery rate. In one example, the video rate controller 54 can determine the undershoot coefficient based on the ratio of the magnitude of the decrease in network link rate to the first network link rate.
[0073] According to aspects of the present disclosure, the video rate controller 54 displays how much data is video between a particular time of network link rate decline and an estimated actual time of network link rate decline. How long to maintain the recovery rate based on how much data is buffered in the encoder 20 (or more generally, how much data is buffered in the transmitter equipment including the video encoder 20). Can be determined. The time associated with buffering data in a transmitter device may be referred to herein as the buffering time length (or buffering time period), but the time length at which the recovery rate should be maintained is the recovery rate time. It may be referred to herein as long (or reduced rate time period). In some cases, the recovery rate time length is also referred to as the undershoot time length or duration, because the rate at which the data is encoded during the recovery rate time length is lower than the network link rate. ..
[0074] As described in more detail with respect to FIG. 5 below, the video rate controller 54 can determine the buffering time length in a variety of ways. For example, the video rate controller 54 may include round trip time (RTT), downlink delay (eg, receiver-to-transmitter delay), rate matching response between transmitter equipment and receiver equipment that incorporates the video encoder 20. The buffering time length is estimated by estimating the buffering time length from the network link rate information 56, such as data related to the delay of, congestion control reaction delay (for example, link rate estimation), and message generation delay (RTCP packet). Can be decided. The network link rate information 56 may be available in the video encoder 20 or may be signaled to the video encoder 20 by the receiver device.
[0075] According to the aspects of the present disclosure, the video rate controller 54 can determine the recovery rate time length based on the magnitude of the recovery rate and based on the buffering time length. In some examples, the video rate controller 54 determines the magnitude of the network link rate drop (eg, as indicated by the recovery rate) and the network link rate drop (eg, as indicated by the buffering time length). The length of buffering time can be determined, which is proportional to the amount of time associated with reacting to. That is, if the network link rate drop is relatively large and / or the time required to react to the network link rate drop is relatively long, the video rate controller 54 will have a proportionally longer recovery rate time. The length can be determined. Similarly, if the network link rate drop is relatively small and / or the time required to react to the network link rate drop is relatively short, the video rate controller 54 will have a proportionally shorter recovery rate. The time length can be determined.
[0076] According to another aspect of the present disclosure, the video rate controller 54 may, in addition or alternative, perform rate upswitching based on network conditions. For example, the video rate controller 54 can receive network link rate information 56 from a receiver device such as a device including the decoder system 14 (FIG. 1). The video rate controller 54 uses the received network link rate information 56 to upswitch the transmit rate (eg, the code rate) used by the video coding engine 50 to encode the data. be able to.
[0077] In some examples, the received network link rate information 56 may include certain requested transmission rates (eg, coding rates) being implemented by the video coding engine 50. In another example, the received network link rate information 56 may include an increase in rate steps that should be added to the current transmission rate (eg, transmission rate step). In either case, the received network link rate information 56 was received at the receiver device before the packet was scheduled to be replayed, as described in more detail below with respect to FIG. It can be obtained based on the excess delay parameter indicating that. In such cases, the video rate controller 54 increases the transmit rate used by the video coding engine 50 until the arrival time of the packet more closely matches the scheduled playback time of the packet in the receiver device. Can be done.
[0078] The technique of FIG. 2 is described as being performed by a particular component of FIG. 2 (eg, video rate controller 54, etc.), but such technique is additionally or alternative to video telephone equipment. It should be understood that it can be performed by one or more of the other components of. As an example, MTSI equipment can perform some of the techniques described above to perform rate matching and / or congestion control. In this example, the MTSI instrument can then provide data to the video rate controller 54 to perform appropriate rate control in the video encoder.
FIG. 3 is a block diagram showing a video decoder system 14 capable of performing video source rate matching according to the techniques of the present disclosure. As shown in FIG. 3, the video decoder 42 receives the encoded data and the network link rate information 60, and generates the video decoding engine 62, the reproduction determination unit 64, and the rate control data 68. Includes unit 66 and.
[0080] The video decoding engine 62 receives the encoded data and the network link rate information 60, and decodes the video data. In some examples, the video decoding engine 62 may conform to one or more video coding standards. As mentioned above, exemplary video coding standards include HEVC, MPEG4, ITU H.263, or ITU H.264.
The rate at which the video data is received can be controlled by the video rate controller 54 of the video encoder 20 (FIG. 2). According to aspects of the present disclosure, the rate control unit 66 can prepare rate control data 68 for use in adjusting the coding rate and transmit it to the video encoder 20. In some examples, rate control data 68 may include data for performing downswitching in the transmitter equipment. In another example, in addition or alternative, rate control data 68 may include data for performing upswitching in the transmitter equipment. The rate control unit 66 can prepare data that allows the transmitter equipment to determine an appropriate bit rate, or can request a specific bit rate from the transmitter equipment.
[0082] With respect to preparing data for downswitching, according to aspects of the present disclosure, the rate control unit 66 has a recovery rate, buffering time, in a manner similar to that described above with respect to FIG. The length and / or recovery rate time length can be determined. In another example, the rate control unit 66 can be used by a transmitter device (such as the encoder system 12) to determine the recovery rate, buffering time length and / or recovery rate time length, generating data, and / Or you can send a message.
[0083] In one example, the rate control unit 66 requests an RTCP temporary maximum media stream bit rate (TMMBR) from the transmitter device with an estimated maximum bit rate for the forward channel to indicate a decrease in network link rate. : Temporary Maximum Media Stream Bit Rate Request) message can be generated. Generally, as described in RFC 5104 mentioned above, the receiver, converter or mixer requires the transmitter to limit the maximum bit rate of the media stream to a given value or less. You can use TMMBR (called "timber") to do this. Temporary Maximum Media Stream Bitrate Notification (TMMBN) is the most restrictive of the restrictions defined in TMMBR received by the media transmitter to help participants suppress TMMBR, which would not further constrain the media transmitter. Includes the current view of media transmitters in a general subset.
[0084] According to aspects of the present disclosure, a change in the estimated maximum bit rate of the forward channel from the first rate to the second lower rate indicates a decrease in the network link rate. In some examples, rate control unit 66 may send TMMBR as soon as congestion is detected, or almost immediately (eg, there may be a message generation delay associated with generating TMMBR messages). .. Although the TMMBR message is explained for illustration purposes, it should be understood that various other messages indicating delay / congestion may be used.
[0085] To assist the transmitter equipment by estimating the buffering time length described herein, the rate control unit 66 generates and sends an RTCP receiver report (RR) message. You can also do it. For example, as described in RFC 3550 mentioned above, several RTCP packet types can be used to carry various control information. A transmitter report (SR) can be used for transmission and reception statistics from participants who are operating transmitters. RR can be used in combination with SR for reception statistics from participants who are not operating transmitters and for reporting operating transmitters for more than 32 sources. ..
[0086] According to aspects of the present disclosure, the rate control unit 66 may generate and transmit an RR message after the TMMBR message, eg, immediately after, or approximately immediately after the TMMBR message. In this example, the transmitter device can receive the TMMBR message and the RR message, the transmission time of the SR referenced in the RR by the time stamp (LSR data) of the last SR contained in the RR, and the transmission. The upper limit of the buffering time length can be determined as the time difference from the time when the RR is received by the machine / equipment. In other words, the rate control unit 66 has a first piece of data (eg, a TMMBR message) indicating a request for a bit rate limit and a second piece of data (eg, an LSR data) indicating the time the message was generated. Can be sent. LSR data is the latest RTCP from the source It may contain 32 bits in the middle of the 64-bit Network Time Protocol (NTP) timestamp received as part of the SR packet. The LSR timestamp field can be set to 0 if the SR has not yet been received. The transmitter equipment can receive the data described above and can use that data to determine the buffering time length, which can be used during downswitching.
[0087] In another example, instead of sending two separate contiguous messages (eg, TMMBR and RTCP RR), rate control unit 66 combines TMMBR data and RTCP RR data into a single RTCP message. Can be grouped with and a single message can be sent to the transmitter device. At a minimum, the rate control unit 66 can transmit LSR data, which allows the transmitter equipment to estimate the buffering time length. In this example, the message size can be reduced compared to sending two separate messages.
[0088] In some examples, rate control unit 66 uses the LSR of the last received RTCP SR to be transmitted to the transmitter device, even if it previously transmitted an RTCP RR with the same LSR. can do. If rate control unit 66 has not yet transmitted RR, rate control unit 66 can combine the full RR with TMMBR. In another example, to reduce the message size, the rate control unit 66 can only send the LSR data along with the TMMBR, and the transmitter equipment receives the TMMBR to determine the RTT. Can be used. In yet another example, if the rate control unit 66 had already transmitted the RR, the transmitter equipment would rate the time it received the last received RR and the new RR (eg, after congestion was detected). The buffering time length can be calculated more accurately as the time difference from the time when the RR) transmitted by the control unit 66 is received.
[0089] The rate control unit 66 can also determine the recovery rate time length and / or generate data and transmit it to the transmitter equipment to determine the recovery rate time length. For example, instead of or in addition to the techniques described above, rate control unit 66 (or transmitter equipment) inter-arrival RTCP RR to determine when the recovery rate duration should end. -arrival jitter data) can be monitored. In general, inter-arrival jitter data can provide an estimate of the statistical variance of the time between arrivals of RTP data packets, measured in timestamp units and expressed as an unsigned integer. The inter-arrival jitter J can be defined as the standard deviation (smoothed absolute value) of the packet gap difference D at the receiver compared to the transmitter for a pair of packets. As shown in equation (1) below, this is equivalent to the difference between the "relative transmission times" of the two packets, where the relative transmission time is the arrival measured in the same units as the packet's RTP timestamp. It is the difference between the time of the receiver and the clock of the receiver. If Si is the RTP timestamp from packet i and Ri is the time of arrival in RTP timestamp units for packet i, then for two packets i and j, D can be expressed as:
D (i, j) = (Rj --Ri)-(Sj --Si) = (Rj --Sj)-(Ri --Si) (1) [0090] According to aspects of the present disclosure, if the inter-arrival jitter is 0 or less than the threshold, the transmitter equipment can end the rate reduction (eg, the transmitter equipment is approximately from the reduced rate). The transmission rate can be increased to the network link rate). The threshold may be a constant or may adapt to changes in network conditions. In some examples, the transmitter equipment can end the rate reduction if the inter-arrival jitter is maintained at 0 or below the threshold for the minimum time period. In some cases, the more frequently the rate control unit 66 signals RTCP SR and RR, the more accurately the transmitter equipment can monitor on-arrival jitter.
In yet another example, transmitter equipment (such as encoder system 12) can monitor delays (eg, RTT) and transmitters are reduced in transmission rate until the delay is sufficiently reduced. Can be maintained at a high rate. For example, a transmitter device can maintain a reduced transmission rate until the amount of data stored in the buffer falls below a threshold level.
According to other techniques of the present disclosure, the replay decision unit 64 inspects the received video packet and the received data arrives early or on time for the scheduled replay. You can decide if you are or are arriving late. Scheduled playback timing can be indicated using encoded data. If the packet arrives late (eg, there is a playback time before the packet is received / inspected), the rate control unit 66 can request the transmitter equipment to lower the transmit bit rate. .. In some examples, rate control unit 66 can send TMMBR messages at the selected rate.
[0093] According to some embodiments, the rate control unit 66 determines the amount of excess delay that needs to be removed and the data rate of the arriving video as measured by the rate control unit 66 and this excess. By multiplying the delay parameters, the amount of remaining data (eg, the data buffered in the transmitter device) can be estimated. In other words, the rate control unit 66 can determine the delay based on the difference between the time the data is received / inspected and the playback time indicated with the data. The rate control unit 66 can then multiply the bit rate at which the data is being received by this delay time to determine the amount of data buffered by the transmitter.
[0094] In some examples, the rate control unit 66 is a sustainable rate of transport path between the video decoder 42 and the transmitter equipment to allow the system to clear channel congestion. An initial rate lower than (eg, the available bandwidth of the network link) can be requested (eg, in a TMMBR message). In one example, rate control unit 66 can choose an initial rate that is low enough to allow the system to clear channel congestion in a fixed amount of time (indicated by the variable T_decongest). If the variable R_sustain is equal to the sustainable rate of the channel, then the variable ΔDelay is equal to the amount of delay that needs to be removed, and the rate control unit 66 first data in bitrate R according to equation (2) below. Can be requested from the transmitter equipment to encode.
R = R_sustain (1-ΔDelay / T_decongest) (2) After sending a message containing the requested bit rate (eg, a TMMBR message), the rate control unit 66 can wait for the congestion resolution time (T_decongest) to elapse. The rate control unit 66 can then send another requested bit rate (eg, an additional TMMBR message) at a sustainable rate (R_sustain) over the network link, thus ending the decontamination period.
[0095] In another example, the rate control unit 66 does not have to send another message (eg, an additional TMMBR message) to increase the rate. In this example, the rate control unit 66 can simply start measuring the amount of acceptable delay (eg, delay less than a predetermined threshold). If rate control unit 66 determines that the amount of delay is less than required (eg, packets are arriving earlier than required for properly scheduled playback), then rate control unit The 66 can send another message (eg, another TMMBR message) to increase / ramp up the encoding rate of the transmitter device.
[0096] With respect to upswitching, the channel between the transmitter device and the receiver device (such as channel 16 between the encoder system 12 and the decoder system 14 (Fig. 1)) is not fully utilized by the transmitter device. When the delivery of a video packet to the video decoder 42 occurs before such a video packet actually needs to be played (eg, received before the playback time indicated with the data). There is a high possibility that it will be done. In such cases, the transmitter rate may be increased and some additional delay may be introduced to the system without adversely affecting the user experience.
[0097] In cases where the excess bits that can be brought into the transmit path are equal to the average receive rate measured by the rate control unit 66 (eg, in the worst case where there is no spare channel bandwidth available). , Can be calculated according to the following equation (3).
excess_bits = rate_increase_step * (RTT + receiver_detection_delay) (3) Where excess_bits indicates the additional bits brought to the system, rate_increase_step indicates the increase in the coding rate, RTT indicates the round trip time, and receiver_detection_delay indicates the delay associated with detecting the system delay by the receiver. Show (this can be determined according to any of the techniques described herein).
[0098] In some examples, the rate control unit 66 determines the corresponding worst-case excess delay (excess_delay) due to the resulting excess bits (excess_bits) according to equation (4) below. Can be done.
excess_delay = rate_increase_step * (RTT + receiver_detection_delay) / avg_receiving_rate (Four) Here, excess_delay indicates the amount of delay caused in the transmitter equipment, rate_increase_step indicates the increase in the coding rate, RTT indicates the round trip time between the transmitter equipment and the video decoder 42, and receiver_detection_delay indicates the rate control. Indicates the delay associated with detecting system delays by unit 66, where avg_receiving_rate indicates the rate at which the data is being received by the video decoder 42.
[0099] Thus, in some examples, according to aspects of the present disclosure, the rate control unit 66 of the video decoder 42 has the number of excess bits associated with a particular rate increase (eg, excess_bits) and the excess bits. The delay associated with contamination (eg, excess_delay) can be determined.
[0100] The rate control unit 66 can calculate the amount of rate increase based on the acceptable excess delay parameter. For example, the rate control unit 66 can determine how much the transmit rate can be increased by the transmitter equipment without causing congestion and / or delay to the system according to equation (5) below.
rate_increase_step = allowable_excess_delay * avg_receiving_rate / (RTT + receiver_detection_delay) (5) Here, rate_increase_step indicates the amount by which the transmit rate can be increased (which can be referred to as increase in transmitter bit rate), allowable_excess_delay indicates the acceptable excess delay parameter (as described in more detail below), and avg_receiving_rate. Indicates the average rate at which data was received before determining the rate increase, RTT indicates the round trip time between the video decoder 42 and the transmitter equipment (such as the video encoder 20), and receiver_detection_delay indicates at the video decoder 42. Shows the amount of time required to identify the delay. In some cases, the receiver detection delay parameter is implementation Can be dependent) and can be estimated or measured in offline testing. If such a receiver detection delay is not available, the rate control unit 66 may be configured to use the estimated reaction delay, which is necessary for the rate control unit 66 to identify the delay. It can be a relatively conservative estimate of the alleged time.
[0101] Since the one-way delay from the transmitter device to the receiver device, including the video decoder 42, is generally unknown to the receiver device, the receiver device typically calculates an acceptable excess delay parameter (allowable_excess_delay). Because of this cannot be used. Alternatively, according to aspects of the present disclosure, the rate control unit 66 can determine the amount of excess delay that can be tolerated from the received video packets. For example, the rate control unit 66 can determine when a video packet is received and / or processed by the video decoder 42. The rate control unit 66 can also determine when the video data associated with the video packet is designed to be played (eg, displayed to the user). The rate control unit 66 can determine the acceptable excess delay parameter based on the difference between the time the packet is received and / or evaluated and the playback time).
[0102] The acceptable excess delay parameter can generally indicate the amount of time available to the transmitter equipment as a basis for increasing the bit rate without affecting the user experience. For example, the acceptable excess delay parameter is supported by channel 16 so that the data arrives too late in the video decoder 42, for example, to be decoded and played at the appropriate time without affecting the user experience. It can indicate the amount of time that can be used to increase the transmit rate without increasing the transmit rate to a rate that is not possible. The acceptable excess delay measure may be more accurate from the user experience perspective, as the acceptable excess delay parameter degrades the video information in the received packet (eg, jitter, stuttering, etc.) Or it directly indicates whether it can actually be displayed to the user without (such as frame loss).
[0103] According to aspects of the present disclosure, based on the above analysis, rate control unit 66 may impose the following requirements on receiver and transmitter equipment to perform upswitching. The receiver shall inspect the arrival of the packet and compare it to the normal scheduled playback time to determine if there is an acceptable amount of delay in the transmission path: allowable_excess_delay. To do. The receiver shall inspect the arrival of packets to calculate the average reception rate: avg_receiving_rate. The receiver shall calculate the round trip time: RTT. -The receiver shall calculate rate_increase_step as follows.
rate_increase_step = allowable_excess_delay * avg_receiving_rate / (RTT + receiver_detection_delay) When permitted by the Audio Visual Provide with Feedback (AVPF) RTCP transmission rules, the receiver When detecting rate_increase_step> 5% × avg_receiving_rate, a temporary maximum media stream bit rate request (TMMBR) should be sent, When it is detected that rate_increase_step> 15% × avg_receiving_rate, TMMBR shall be transmitted. -When sending a TMMBR message, the requested_rate in TMMBR is avg_receiving_rate + rate_increase_step Must be equal to avg_receiving_rate + 0.80 (rate_increase_step) <= requested_rate <= avg_receiving_rate + rate_increase_step Suppose that
[0104] In addition, according to aspects of the present disclosure, the following requirements may be imposed on the transmitter equipment to perform upswitching. Upon receiving a Temporary Maximum Media Stream Bitrate Request (TMMBR), the video transmitter should ramp up the transmit rate to requested_rate within 500ms and within 1 second.
[0105] The "requirements" mentioned above are given for illustration purposes and it is understood that the techniques of the present disclosure may be applied using values different from the particular values mentioned above. I want to. In addition, certain techniques have been attributed to specific units in FIG. 3 (eg, rate control unit 66) for illustration purposes, but one or more other units of the video decoder 42 have such techniques. Please understand that you can be responsible for doing. Moreover, since VT is often a two-way communication flow, similar techniques can be applied to both forward and reverse network paths, for example, transmitter equipment (devices incorporating the video encoder 20 in FIG. 2). Etc.) and may be applied by both devices designated herein as receiver devices (such as devices incorporating the video decoder 42 in FIG. 3).
[0106] FIGS. 4A and 4B are graphs showing the video source rate matching technique according to the present disclosure. For example, FIG. 4A generally shows the bit rate of encoded data in a transmitter device (eg, encoder system 12) during the time including the decrease in network link rate. Figure 4B generally shows the resulting delay associated with lower network link rates. Although the techniques of FIGS. 4A and 4B are described for the encoder system 12, it should be understood that this technique can be performed by various other transmitter devices with various other components.
[0107] In the example of Figure 4A, time t<sub>0</sub>In, the link rate (also called network link rate or bandwidth) is R as indicated by line 80.<sub>0</sub>From R<sub>1</sub>It drops to, where the transmission rate = link rate). In response to the decrease in network link rate, the encoder system 12 can decrease the transmission rate. However, as shown in the example of Figure 4A, t associated with lowering the transmit rate, as shown by the dashed line 82.<sub>0</sub>From t<sub>1</sub>There is a response delay (ΔT) to. Response delays may also be described herein as buffering time lengths, during which the transmit rate overshoots the network link rate and the encoder system 12 buffers data that cannot be addressed by the network link. Responsible for ringing.
[0108] As shown by line 84 in the example of FIG. 4B, the delay (eg, the time between the time when the encoded data becomes available and the time when the encoded data is actually transmitted). ) Is D during the response delay (ΔT)<sub>0</sub>From D<sub>1</sub>Increases relatively quickly. That is, the delay is the time t of the decrease in the network link rate.<sub>0</sub>And a specific time of network link rate drop t<sub>1</sub>Between and D<sub>0</sub>From D<sub>1</sub>Increases relatively quickly. This delay can be proportional to the amount of data buffered in the encoder system 12.
[0109] time t<sub>1</sub>In, the examples of FIGS. 4A and 4B show that the transmit rate technique is branched. For example, solid line 80 shows a first example in which the encoder system 12 maintains the transmit rate at the network link rate. For example, if a decrease in network link rate is identified, the encoder system 12 will use the transmit rate as the original rate R.<sub>0</sub>New reduced network link rate from R<sub>1</sub>Lower to. In this example, the corresponding delay remains relatively large, as indicated by solid line 88. That is, the transmission rate is the network link rate R<sub>1</sub>Since it is set to, there is no excess bandwidth used to reduce the amount of buffered data.
[0110] Dashed lines 82 and 86 indicate that the encoder system 12 has the original rate R.<sub>0</sub>From network link rate R<sub>1</sub>Lower reduced rate R<sub>U</sub>Here is a second example of reducing transmissions. This can be referred to as "undershooting" the network link rate. In this example, the encoder system 12 has a determined recovery rate time length (ΔT).<sub>u</sub>), Reduced rate R<sub>u</sub>Can be maintained. During this time, the encoder system 12 has time t, as indicated by the dashed line 90.<sub>2</sub>In D<sub>1</sub>From D<sub>2</sub>To reduce the delay.
[0111] As described herein, the encoder system 12 has a reduced rate (RU), a buffering time length (ΔT), and a recovery rate time length (ΔT).<sub>u</sub>) And can be determined using various techniques. In one example, the encoder system 12 is expressed in equations (1-f).<sub>U</sub>) × R<sub>1</sub>Reduced rate R based on<sub>U</sub>Can be determined, where f<sub>U</sub>Is the undershoot coefficient, R<sub>1</sub>Is the reduced link rate, f<sub>U</sub>Is the rate undershoot coefficient (1-f<sub>U</sub>) Is determined and 0 <f<sub>U</sub><1, and the rate undershoot coefficient links the transmission rate to the link rate R<sub>1</sub>Associate with. In some examples, f<sub>U</sub>May depend on the magnitude of the decrease in network link rate, which is the equation ΔR = (R<sub>0</sub>-R<sub>1</sub>) Can be represented. In this example, R<sub>0</sub>Is the first network link rate before it is lowered. If the magnitude of the decrease in network link rate ΔR is large, f<sub>U</sub>Can grow proportionally. In another example, if ΔR is small, f, as shown in Eq. (6) below.<sub>U</sub>Can be proportionally smaller.
f<sub>U</sub> = ΔR / R<sub>0 </sub>(6) [0112] If the encoder system 12 buffers all of the bits during the buffering time length (ΔT) and contributes to the delay, the encoder system 12 will recover the recovery rate time based on Eq. (7) below. Length (ΔT<sub>u</sub>) Can be determined.
ΔT<sub>u</sub> = ΔT (R<sub>0</sub>-R<sub>1</sub>) / (F<sub>U</sub> R<sub>1</sub>) (7) Where ΔT<sub>u</sub>Is the recovery rate time length, ΔT has the buffering time length, and R<sub>0</sub>Has a first network link rate, R<sub>1</sub>Has a second reduced network link rate, f<sub>U</sub>Has a rate reduction factor.
[0113] In some examples, the encoder system 12 can apply the minimum bit rate requirements. The minimum bit rate requirements can be based on the capabilities of the video encoder 20, the minimum system requirements for the user experience, and so on. In the example where the encoder system 12 applies the minimum bit rate requirement, the video encoder 20 R the minimum bit rate requirement.<sub>U</sub>Therefore, the undershoot coefficient f<sub>U</sub>Can also be applied to. For example, the encoder system 12 has a reduced rate R.<sub>U</sub>And undershoot coefficient f<sub>U</sub>The following equations (8) and (9) can be applied to determine and.
R<sub>U</sub> > = R<sub>min</sub> (8) f<sub>U</sub> <= 1-(R<sub>min</sub>/ R<sub>1</sub>) With R<sub>1</sub> > R<sub>min</sub> (9) Where R<sub>U</sub>Is the reduced bit rate, R<sub>min</sub>Is the minimum code rate, R<sub>1</sub>Is the second reduced network link rate, f<sub>U</sub>Is the undershoot coefficient.
[0114] Recovery rate time length (ΔT<sub>u</sub>), New rate value R<sub>2</sub>A TMMBR message is received by the encoder system 12 and R<sub>2</sub>Is R<sub>1</sub>Much larger (eg R<sub>2</sub>Is R<sub>1</sub>The encoder system 12 can reduce the recovery rate time length (more than multiplying by 1.2). On the contrary, R<sub>2</sub>Is R<sub>1</sub>If lower, the encoder system 12 can determine additional or extended recovery rate durations.
[0115] In general, as mentioned above with respect to FIGS. 2 and 3, the encoder system 12 has knowledge of RTP, downlink delay (eg, receiver to transmitter), rate control response delay, congestion. The buffering time length (ΔT) can be estimated from network information such as control response delay (eg, link rate estimation), message generation delay (eg, delay associated with generating RTCP packets). This network information may be available on the transmitter side or may be signaled to the encoder system 12 by a receiver device such as a device incorporating a video decoder 42 (FIG. 3).
[0116] The examples in FIGS. 4A and 4B are stepwise changes (eg, R) for illustration purposes.<sub>0</sub>And R<sub>U</sub>Although showing a single rate change between, it should be understood that this technique can be applied repeatedly so that the shape of the undershoot is smoother.
[0117] FIG. 5 is a conceptual diagram showing that the buffering time length is determined by the technique of the present disclosure. In the example of FIG. 5, a transmitter device (eg, encoder system 12) can transmit an RTCP transmitter report (SR) to a receiver device (eg, decoder system 14) at time 120. For example, as described in RFC 3550 as described above, some RTCP packets can be used to carry a variety of control information. Transmitter reporting (SR) can be used for transmission and reception statistics from participants who are operating transmitters. Similarly, Receiver Report (RR) is for receiving statistics from participants who are not working transmitters, and for reporting working transmitters for more than 32 sources. Can be used in combination with SR. The receiver device can receive the RTCP SR at time 122.
[0118] The receiver device can send an RTCP TMMBR message to the transmitter device at the estimated maximum bit rate of the forward channel at time 124. In some examples, there may be a delay associated with generating the message, but the receiver device can send the TMMBR message immediately after detecting congestion. Although the TMMBR message is described for illustration purposes, various other messages that may indicate delay / congestion may be used.
[0119] To assist the transmitter device by estimating the buffering time length (ΔT), the receiver device can also send an RTCP RR message at time 124. According to aspects of the present disclosure, the receiver device may transmit an RR message immediately after the TMMBR message. In this way, the transmitter device can receive the TMMBR message and the RR message at time 126 and transmit the SR referenced in the RR by the last SR time stamp (LSR data) contained in the RR. The upper limit of the buffering time length (ΔT) 128 can be calculated as the time difference between that and the time when the RR is received. In other words, the receiver device sends first data indicating a request for bit rate limitation (eg, TMMBR message) and second data indicating the time the message was generated (eg, LSR data). can do. LSR data is the latest RTCP from the source It may contain 32 bits in the middle of the 64-bit Network Time Protocol (NTP) timestamp received as part of the SR packet. The LSR timestamp field can be set to 0 if the SR has not yet been received.
[0120] In another example, instead of sending two separate contiguous messages (eg, TMMBR and RTCP RR) with the data mentioned above, the TMMBR and RTCP RR data are single RTCP. Can be grouped into messages. At a minimum, the receiver device can transmit LSR data, which allows the transmitter device to estimate the buffering time length (ΔT) 128. In this example, the message size can be reduced.
[0121] The receiver device may use the LSR of the last received RTCP SR message, even if it has previously sent an RTCP RR message with the same LSR. If the receiver device has not yet transmitted the RR, the receiver device can combine the complete RR message with the TMMBR message. In another example, to reduce the message size, the receiver device may only send LSR data along with the TMMBR message, which can be used to calculate the RTT.
[0122] In another example, if the receiver device had already transmitted the RR, the transmitter device would receive the last received RR message and (eg, the receiver after congestion was detected). The buffering time length (ΔT) 128 can be calculated more accurately as the time difference from the new RR message (sent by).
[0123] In yet another example, the transmitter equipment can monitor the delay (eg, RTT), and the transmitter equipment has the transmission rate reduced until the delay is sufficiently reduced.<sub>u</sub>Can be kept in. For example, a transmitter device has a rate R whose transmission rate has been reduced until the amount of data stored in the transmitter device's buffer falls below a threshold level.<sub>u</sub>Can be maintained at.
[0124] FIGS. 6A and 6B are graphs showing the decrease in network link rate and the corresponding delay time, respectively. The graph of FIG. 6A can be associated with line 80 of FIG. 4A, while the graph of FIG. 6B can be associated with line 88 of FIG. 4B. For example, Figure 6A shows the network bandwidth 140 (also known as the network link rate) indicated by the broken line and the transmit rate 142 (also referred to as the coded bit rate) indicated by the solid line (eg, measured in kilobytes per second (KBPS)). ) And. As shown in FIG. 6A, a transmitter device (eg, an encoder system 12) can encode data at a transmit rate 142 of the same or similar rate as the bandwidth 140. Thus, when bandwidth 140 is reduced at time 144, the transmitter equipment can reduce transmission rate 142 to about the same value as bandwidth 140.
[0125] After a decrease in bandwidth 140, the delay in the encoder system 12 is at the first level (eg, measured in milliseconds (MS)), as shown in the corresponding delay graph in Figure 6B. Can be raised from 146 to the second level 148. As described herein, the lower the bandwidth, the higher the delay, because there is a reaction time associated with lowering the transmit rate 142 to the level of the bandwidth 140. The encoder system 12 can buffer the buffer data encoded at the original (higher) rate before reducing the transmit rate 142 to match the bandwidth 140. As shown in Figure 6B, delays can remain for a relatively long time if techniques for reducing delays are not applied.
[0126] FIGS. 7A and 7B are graphs showing the decrease in network link rate and the corresponding delay time, respectively. The graph of FIG. 7A can be associated with lines 82 and 86 of FIG. 4A, while the graph of FIG. 7B can be associated with the dashed line 90 of FIG. 4B. For example, Figure 7A shows the network bandwidth 160 (also known as the network link rate) indicated by the broken line and the transmit rate 162 (also referred to as the coded bit rate) indicated by the solid line (for example, measured in kilobytes per second (KBPS)). ) And. As shown in FIG. 7A, a transmitter device (eg, encoder system 12) can initially encode data at a transmit rate 162 of the same or similar rate as the bandwidth 160.
[0127] According to aspects of the present disclosure, when bandwidth 160 is reduced at time 164, the transmitter equipment can reduce the transmission rate to a reduced rate lower than bandwidth 160. That is, the transmitter device can determine a transmission rate 162 that undershoots the bandwidth 140 in order to reduce the delay associated with the reduction in the bandwidth 160. As described herein, the transmitter equipment can determine the buffering time length, reduced rate and / or recovery rate time length according to the techniques of the present disclosure.
[0128] After a reduction in bandwidth 160, the delay in the encoder system 12 is at the first level (eg, measured in milliseconds (MS)), as shown in the corresponding delay graph in Figure 7B. Can be raised from 166 to the second level 168. As mentioned above, the lower the bandwidth, the higher the delay, because there is a reaction time associated with lowering the transmit rate 162 in response to the lower bandwidth 160. However, by reducing the transmit rate 162 to a reduced rate (undershooting the bandwidth 160), the encoder system 12 can reduce the delay faster than the example shown in FIG. 6B.
[0129] FIG. 8 is a flow diagram illustrating an exemplary process for downswitching the rate at which data is transmitted. The example of FIG. 8 will be described with respect to the encoder system 12 for illustrative purposes. However, it should be understood that the process of Figure 8 can be performed by a variety of other equipment and / or processors.
[0130] The encoder system 12 can encode and transmit data over the network at the first rate (180). While transmitting data at the first rate, the encoder system 12 may identify a decrease in the network link rate from the first rate to the second rate (182). For example, the encoder system 12 may monitor network conditions and / or receive one or more messages indicating a decrease in network link rate.
[0131] The encoder system 12 can determine a recovery bit rate that is lower than the second (reduced) network link rate (184). For example, the encoder system 12 can determine the bit rate for encoding the data, which undershoots the new network link rate. According to aspects of the present disclosure, the encoder system 12 can determine the recovery bit rate based on the difference between the first network link rate and the reduced network link rate. For example, if the network link rate drop is relatively large, the encoder system 12 can determine a relatively aggressive (eg, undershoot the reduced rate with a significant margin) recovery bit rate. Similarly, if the network link rate drop is relatively small, the encoder system 12 can determine a relatively conservative (eg, undershoot the reduced rate with a relatively small margin) recovery bit rate. ..
[0132] The encoder system 12 can also determine the buffering time length based on the response delay (eg, the time associated with lowering the transmit rate in response to a lower network link rate) (186). .. The encoder system 12 can determine the buffering time length in various ways. For example, the encoder system 12 buffers network information such as round trip time (RTT) between the encoder system 12 and the receiver device, downlink delay, rate matching response delay, congestion control response delay, and message generation delay. The buffering time length can be determined by estimating the time length. The encoder system 12 can independently determine the network information or can receive the network information from the receiver device.
[0133] The encoder system 12 can then determine the recovery rate duration for which the recovery bit rate should be maintained (188). In some examples, the encoder system 12 can determine the recovery rate time length based on the magnitude of the recovery rate and based on the buffering time length. In some examples, the encoder system 12 is responsible for the magnitude of the network link rate drop (eg, as indicated by the recovery rate) and the network link rate drop (eg, as indicated by the buffering time length). The length of buffering time can be determined, which is proportional to the amount of time associated with the reaction.
[0134] The encoder system 12 is capable of transmitting data at the recovery bit rate for the duration of the recovery rate time (190). In some examples, if the network link rate rises during the recovery rate duration, the encoder system 12 can terminate the recovery rate duration earlier and upswitch to a higher transmission rate. As an example, some actions or events of any of the techniques described with respect to FIG. 8 may be performed in a different order, may be added, may be integrated, or may be completely excluded. It should be understood that (for example, not all described actions or events are necessary for the practice of the technique).
[0135] FIG. 9 is a flow diagram illustrating an exemplary process for upswitching the rate at which data is transmitted. The example of FIG. 9 will be described with respect to the decoder system 14 for illustrative purposes. However, it should be understood that the process of Figure 9 can be performed by a variety of other equipment and / or processes.
[0136] The decoder system 14 can determine when data is received (200). For example, in some cases, the decoder system 14 can identify the time when data is received and stored in the decoder system 14. In another case, the decoder system 14 can identify the time at which data is processed (eg, decoded) by the decoder system 14.
[0137] The decoder system 14 can also determine the replay time of the received data (202). For example, the received data may include an indication of the time the data is intended to be output for display to the user. Therefore, the playback time indication can assist the decoder system 14 in organizing the data for output.
[0138] The decoder system 14 can determine the acceptable excess delay parameters (204). For example, the decoder system 14 can determine an acceptable excess delay parameter based on the difference between the time the data is received and the time the received data is scheduled to be replayed. As described herein, delay can refer to the time between the time the data is available for transmission across the network link and the time the data is actually transmitted to the network at the transmitter equipment. Thus, the acceptable excess delay parameter can indicate the amount of delay that can be accommodated by the system so that the user experience is not affected. That is, the acceptable excess delay parameter can generally indicate the amount of time available to the transmitter device as a basis for increasing the bit rate without affecting the user experience.
[0139] The decoder system 14 can then determine the increase in transmitter bit rate (206). For example, according to aspects of the present disclosure, the decoder system 14 can determine an increase in transmitter bit rate based on acceptable excess delay parameters. That is, the decoder system 14 can determine how much the transmission rate can be increased by the transmitter equipment without causing congestion to the system.
[0140] In some examples, the decoder system 14 can determine a stepwise rate increase to be added to the transmit rate. For example, the decoder system 14 has a transmitter bit based on an acceptable excess delay parameter and the current average transmission rate at which the data was received (eg, the current reception rate) before determining the increase in transmission rate. You can determine the rate increase. In this example, the decoder system 14 is currently without increasing the rate beyond the sustainable link rate (eg, to the rate at which the packet arrives at the decoder system 14 after the scheduled playback time of the packet). It is possible to determine how much the transmission rate of the packet can be increased.
[0141] In some examples, the decoder system 14 is also the amount of time required to send a message between the decoder system 14 and the transmitter equipment (eg, round trip time) and / or the decoder. The delays associated with identifying delays in system 14 can be considered. For example, the decoder system 14 determines the transmitter bit rate based on the ratio of the receiver rate multiplied by the acceptable excess delay parameter to the sum of the round trip time and the time to detect the delay in the decoder system 14. The increase can be determined.
[0142] The decoder system 14 can then send instructions to increase the transmitter rate (208). For example, the decoder system 14 can transmit data representing a stepwise increase in transmitter rate relative to the transmitter device that the transmitter device should add to the transmit rate. In another example, the decoder system 14 can transmit data representing the requested transmission rate, including increasing the transmission rate of the transmitter equipment.
[0143] In some examples, the decoder system 14 may transmit instructions for increasing the transmitter bit rate only when the increase in the transmitter bit rate exceeds a threshold amount. For example, the decoder system 14 can compare the increase in transmitter bit rate to a predetermined threshold. In one example, the decoder system 14 may transmit an instruction to increase the transmitter bit rate only when the increase in the transmitter bit rate exceeds about 5 percent of the receive rate. In another example, the decoder system 14 may transmit an instruction to increase the transmitter bit rate only when the increase in the transmitter bit rate exceeds about 15 percent of the receive rate. Other thresholds or percentages are also possible.
[0144] Depending on the example, some actions or events of any of the techniques described with respect to FIG. 9 may be performed, added, integrated, or completely excluded in a different order. It should be understood that it may be done (eg, not all described actions or events are necessary for the practice of the technique).
[0145] Although some of the examples described herein have been described with respect to a particular point of view (eg, being performed by a "transmitter device" or "receiver device"), the techniques of the present disclosure are: Please understand that it is not limited in this way. For example, as mentioned above, VT is often a two-way communication flow. Thus, similar techniques can be applied on both forward and reverse network paths, eg, by both "transmitter equipment" and "receiver equipment". Moreover, although some devices have been shown and described with respect to some viewpoint for purposes of illustration, it is understood that the devices described herein may have more or fewer components than those shown. I want to be. As an example, a transmitter device can incorporate both a video encoder 20 (FIG. 2) and a video decoder 42 (FIG. 3), and each of the techniques described can be performed on them.
[0146] In one or more examples, the functions described may be performed on hardware, software, firmware, or any combination thereof. When executed in software, a function may be stored on a computer-readable medium as one or more instructions or codes, or transmitted via a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium that corresponds to a tangible medium such as a data storage medium, or optionally allows the transfer of a computer program from one location to another according to a communication protocol, for example. It may include a communication medium including the medium of. In this way, the computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium such as a signal or carrier wave. The data storage medium can be accessed by one or more computers or one or more processors to retrieve instructions, codes and / or data structures for the implementation of the techniques described in this disclosure. It can be an available medium. Computer program products may include computer-readable media.
[0147] By way of example, but not limited to, such computer-readable storage media include RAM, ROM, EEPROM®, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage, flash. It may include memory, or any other medium used to store the desired program code in the form of instructions or data structures and accessible by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, instructions are transmitted from a website, server or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, wireless and microwave. If so, wireless technologies such as coaxial cable, fiber optic cable, twisted pair, DSL or infrared, wireless and microwave are included in the definition of medium. However, it should be understood that computer-readable and data storage media do not include connections, carriers, signals or other temporary media, but instead target non-temporary tangible storage media. The discs and discs used herein are compact discs (CDs), laser discs (registered trademarks) (discs), optical discs, and digital versatile discs (discs). Includes DVD), floppy (registered trademark) disc (disk) and Blu-ray (registered trademark) disc (disc), where the disc (disk) typically reproduces data magnetically and is a disc. Optically reproduces the data with a laser. The above combinations should also be included in the scope of computer readable media.
[0148] Instructions are for one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs) or other equivalent integrated or discrete logic circuits. It can be run by one or more processors, such as. Thus, the term "processor" as used herein can refer to either the structure described above or any other structure suitable for implementing the techniques described herein. In addition, in some embodiments, the functionality described herein is provided within a dedicated hardware and / or software unit or module configured for coding and decoding, or to a composite codec. Can be incorporated. Also, the technique can be fully implemented in one or more circuits or logic elements.
[0149] The techniques of the present disclosure can be implemented in a wide variety of devices or devices, including wireless handsets, integrated circuits (ICs) or sets of ICs (eg, chipsets). Although various components, modules or units have been described in this disclosure to emphasize the functional aspects of the equipment configured to perform the disclosed techniques, those components, modules or units are described. It does not necessarily require implementation by different hardware units. Rather, as described above, the various units, along with the appropriate software and / or firmware, are combined or mutually combined in the codec hardware unit, including the one or more processors described above. It can be given by a set of operational hardware units.
[0150] Various examples have been explained. These and other examples are within the scope of the following claims.<u style="single">The inventions described in the original claims of the present invention are described below.</u><u style="single">[C1]</u><u style="single">It s a way to process data,</u><u style="single">Sending data over the network at the first bit rate,</u><u style="single">Identifying a decrease in the network link rate of the network from the first network link rate to the second network link rate, and</u><u style="single">In response to identifying the decrease in the network link rate, determining the recovery bit rate to be used when transmitting the data over the network, where the recovery bit rate is the second. Lower than the network link rate of</u><u style="single">Determining the buffering time length based on the difference between the particular time of the decrease in the network link rate and the estimated actual time of the decrease in the network link rate.</u><u style="single">A method comprising determining a recovery rate time length at which the data should be transmitted at the recovery bit rate, based on the recovery bit rate and the buffering time length.</u><u style="single">[C2]</u><u style="single">The method of C1, wherein determining the recovery bit rate comprises determining the recovery bit rate using an undershoot coefficient based on the magnitude of the reduction in the network link rate.</u><u style="single">[C3]</u><u style="single">It further comprises determining the undershoot coefficient as a difference between the first network link rate and the second network link rate, and dividing the difference by the first network link rate. The method described in C2.</u><u style="single">[C4]</u><u style="single">The method of C3, wherein determining the recovery bit rate comprises determining the recovery bit rate as the second network link rate multiplied by the difference between 1 and the undershoot coefficient.</u><u style="single">[C5]</u><u style="single">It is possible to determine the reduced rate time period based on the undershoot rate and the buffering time length.</u><u style="single">ΔT</u><sub><u style="single">u</u></sub><u style="single"> = ΔT (R</u><sub><u style="single">0</u></sub><u style="single">-R</u><sub><u style="single">1</u></sub><u style="single">) / (F</u><sub><u style="single">U</u></sub><u style="single"> R</u><sub><u style="single">1</u></sub><u style="single">)</u><u style="single">The recovery rate time length is determined based on ΔT.</u><sub><u style="single">u</u></sub><u style="single">Represents the recovery rate time length, ΔT represents the buffering time length, and R</u><sub><u style="single">0</u></sub><u style="single">Represents the first network link rate, and R</u><sub><u style="single">1</u></sub><u style="single">Represents the second reduced network link rate, f</u><sub><u style="single">U</u></sub><u style="single">The method according to C2, wherein represents the undershoot coefficient based on the decrease in the network link rate.</u><u style="single">[C6]</u><u style="single">C1 is based on the encoded bit rate of the encoder, wherein determining the recovery bit rate comprises determining a recovery bit rate greater than the minimum bit rate, the minimum bit rate being configured to encode the data. The method described in.</u><u style="single">[C7]</u><u style="single">Identifying an increase in the network link rate from the second network link rate to the third network link rate while transmitting the data during the recovery rate duration.</u><u style="single">The method of C1, further comprising increasing the recovery bit rate to a bit rate higher than the recovery bit rate in response to identifying the increase in the network link rate.</u><u style="single">[C8]</u><u style="single">Determining the buffering time length is the round trip time (RTT) data, the data associated with the downlink delay from the receiver device to the transmitter device, the data associated with the rate control response delay, and the congestion control response delay. The method of C1, wherein the buffering time length is determined based on at least one of the data associated with or the data associated with the message generation delay.</u><u style="single">[C9]</u><u style="single">Receiving data indicating the estimated maximum bit rate of the forward channel from the transmitter device to the receiver device, to which the data is transmitted between them,</u><u style="single">Further prepared to receive data indicating reception quality feedback,</u><u style="single">The buffering time length is determined according to C1, wherein determining the buffering time length is based on the data indicating the estimated maximum bit rate and the data indicating the reception quality feedback. the method of.</u><u style="single">[C10]</u><u style="single">Receiving the data indicating the estimated maximum bit rate comprises receiving a temporary maximum media stream bit rate request (TMMBR) message, and receiving the data indicating the reception quality feedback receives. The method described in C9, which comprises receiving a machine report (RR) message.</u><u style="single">[C11]</u><u style="single">Receiving the data indicating the estimated maximum bit rate and receiving the data indicating the reception quality feedback is the data indicating the estimated maximum bit rate and the data indicating the reception quality feedback. The method described in C9, which comprises receiving a single message containing and.</u><u style="single">[C12]</u><u style="single">Determining the buffering time length based on the data indicating the estimated maximum bit rate and the data indicating the reception quality feedback is the time when the first message is transmitted to the receiver device and the said. The method according to C9, comprising determining the difference between the time at which the data indicating the estimated maximum bit rate and the data indicating the reception quality feedback are received.</u><u style="single">[C13]</u><u style="single">The data comprises encoded video data, further comprising transmitting the data at the recovery bit rate for the determined recovery rate duration, and the recovery bit for the determined recovery rate duration. Transmitting the data at a rate comprises performing rate control to reduce the bit rate of the encoded video data to the recovery bit rate for a determined recovery rate duration of time, C1. The method described in.</u><u style="single">[C14]</u><u style="single">A device for processing data</u><u style="single">With memory configured to store data,</u><u style="single">With one or more processors, said one or more processors</u><u style="single">Sending the data over the network at the first bit rate,</u><u style="single">Identifying a decrease in the network link rate of the network from the first network link rate to the second network link rate,</u><u style="single">In response to identifying the decrease in the network link rate, a recovery bit rate to be used when transmitting the data over the network is determined, wherein the recovery bit rate is the second network. Lower than the link rate,</u><u style="single">The buffering time length is determined based on the difference between the particular time of the decrease in the network link rate and the estimated actual time of the decrease in the network link rate.</u><u style="single">Based on the recovery bit rate and the buffering time length, the recovery rate time length for transmitting the data at the recovery bit rate is determined.</u><u style="single">A device that is configured to be.</u><u style="single">[C15]</u><u style="single">To determine the recovery bit rate, the one or more processors are configured to determine the recovery bit rate using an undershoot coefficient based on the magnitude of the reduction in the network link rate. Equipment described in C14.</u><u style="single">[C16]</u><u style="single">The one or more processors further determine the undershoot coefficient as the difference between the first network link rate and the second network link rate, and divide the difference by the first network link rate. The equipment described in C15, which is configured to be.</u><u style="single">[C17]</u><u style="single">To determine the recovery bit rate, the one or more processors may determine the recovery bit rate as the second network link rate multiplied by the difference between 1 and the undershoot coefficient. The equipment described in C16, which is configured.</u><u style="single">[C18]</u><u style="single">To determine the reduced rate time period based on the undershoot rate and the buffering time length, the one or more processors formulate.</u><u style="single">ΔT</u><sub><u style="single">u</u></sub><u style="single"> = ΔT (R</u><sub><u style="single">0</u></sub><u style="single">-R</u><sub><u style="single">1</u></sub><u style="single">) / (F</u><sub><u style="single">U</u></sub><u style="single"> R</u><sub><u style="single">1</u></sub><u style="single">)</u><u style="single">It is configured to determine the recovery rate time length based on</u><sub><u style="single">u</u></sub><u style="single">Represents the recovery rate time length, ΔT represents the buffering time length, and R</u><sub><u style="single">0</u></sub><u style="single">Represents the first network link rate, and R</u><sub><u style="single">1</u></sub><u style="single">Represents the second reduced network link rate, f</u><sub><u style="single">U</u></sub><u style="single">The device according to C15, wherein represents the undershoot coefficient based on the decrease in the network link rate.</u><u style="single">[C19]</u><u style="single">To determine the recovery bit rate, the one or more processors are configured to determine a recovery bit rate greater than the minimum bit rate, the minimum bit rate being configured to encode the data. The device according to C14, based on the encoded bit rate of the encoder.</u><u style="single">[C20]</u><u style="single">The one or more processors further</u><u style="single">While transmitting the data during the recovery rate duration, the increase in the network link rate from the second network link rate to the third network link rate is identified.</u><u style="single">The device according to C14, which is configured to increase the recovery bit rate to a bit rate higher than the recovery bit rate in response to identifying the increase in the network link rate.</u><u style="single">[C21]</u><u style="single">To determine the buffering time length, the one or more processors associate round trip time (RTT) data, data associated with a downlink delay from a receiver device to a transmitter device, and rate control response delay. The device according to C14, which is configured to determine the buffering time length based on at least one of the data to be generated, the data associated with the congestion control response delay, or the data associated with the message generation delay.</u><u style="single">[C22]</u><u style="single">The one or more processors further</u><u style="single">Receive data indicating the estimated maximum bit rate of the forward channel from the transmitter device to the receiver device to which the data is transmitted between them.</u><u style="single">Configured to receive data that indicates receive quality feedback,</u><u style="single">To determine the buffering time length, the one or more processors determine the buffering time length based on the data indicating the estimated maximum bit rate and the data indicating the reception quality feedback. The equipment described in C14, which is configured to be.</u><u style="single">[C23]</u><u style="single">To receive the data indicating the estimated maximum bit rate, the one or more processors are configured to receive a temporary maximum media stream bit rate request (TMMBR) message to indicate the received quality feedback. The device according to C22, wherein in order to receive the data, the one or more processors are configured to receive a receiver report (RR) message.</u><u style="single">[C24]</u><u style="single">In order to receive the data indicating the estimated maximum bit rate and receive the data indicating the reception quality feedback, the one or more processors have received the data indicating the estimated maximum bit rate and the data indicating the estimated maximum bit rate. The device according to C22, configured to receive a single message comprising said data indicating said reception quality feedback.</u><u style="single">[C25]</u><u style="single">In order to determine the buffering time length based on the data indicating the estimated maximum bit rate and the data indicating the reception quality feedback, the one or more processors, the first message is the receiver device. 22. C22, which is configured to determine the difference between the time transmitted to the data and the time the data indicating the estimated maximum bit rate and the data indicating the reception quality feedback are received. machine.</u><u style="single">[C26]</u><u style="single">The data is encoded and the one or more processors are further configured and determined to transmit the data at the recovery bit rate for the determined recovery rate time length. During the recovery rate time length, in order to transmit the data at the recovery bit rate, the one or more processors determined the bit rate of the encoded video data. A device according to C14 that is configured to perform rate control to lower the recovery bit rate.</u><u style="single">[C27]</u><u style="single">Integrated circuit,</u><u style="single">Microprocessor, or</u><u style="single">A device described in C14 that includes at least one of the wireless communication devices.</u><u style="single">[C28]</u><u style="single">A device for processing data</u><u style="single">A means for transmitting data over the network at the first bit rate,</u><u style="single">A means for identifying a decrease in the network link rate of the network from the first network link rate to the second network link rate, and</u><u style="single">Means for determining the recovery bit rate to be used when transmitting the data over the network in response to identifying the decrease in the network link rate, where the recovery bit rate is said. Lower than the second network link rate,</u><u style="single">A means for determining the buffering time length based on the difference between the particular time of the decrease in the network link rate and the estimated actual time of the decrease in the network link rate.</u><u style="single">An apparatus comprising a means for determining a recovery rate time length for transmitting the data at the recovery bit rate based on the recovery bit rate and the buffering time length.</u><u style="single">[C29]</u><u style="single">When executed, on one or more processors,</u><u style="single">Send data over the network at the first bit rate,</u><u style="single">Identify a decrease in the network link rate of the network from the first network link rate to the second network link rate.</u><u style="single">In response to identifying the decrease in the network link rate, the recovery bit rate to be used when transmitting the data via the network is determined, wherein the recovery bit rate is the second network. Lower than the link rate,</u><u style="single">The buffering time length is determined based on the difference between the particular time of the decrease in the network link rate and the estimated actual time of the decrease in the network link rate.</u><u style="single">Based on the recovery bit rate and the buffering time length, the recovery rate time length for transmitting the data is determined at the recovery bit rate.</u><u style="single">A non-transitory computer-readable medium that stores instructions.</u>
12 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12
Every citation, both waysCites: the store holds 2 of 3
| Document | Relation | Office |
|---|---|---|
| JP2004007361A | Cites | Japan |
| JP2012530469A | Cites | Japan |
| Ericsson,Requirements for end-to-end video rate adaptation,3GPP TSG-SA4 Meeting #79 S4-140686,2014年 5月15日,pp.1-11 | Non-patent | – |
35 members in 11 offices
Priority claims10
| Document | Office | Kind | Date |
|---|---|---|---|
| 201462030513 | United States of America | P | |
| 201462030513 | United States of America | P | |
| 62030513 | United States of America | – | |
| 14811491 | United States of America | – | |
| 201514811491 | United States of America | A | |
| 201514811491 | United States of America | A | |
| 14811491 | – | – | – |
| 62030513 | – | – | – |
| US201462030513P | – | – | – |
| US201514811491 | – | – | – |
Members35
| Document | Office | Kind | |
|---|---|---|---|
| CA2953711A1 | Canada | A1 | |
| US2016037125A1 | United States of America | A1 | |
| US2016037128A1 | United States of America | A1 | |
| WO2016019015A1 | World Intellectual Property Organization (WIPO) | A1 | |
| WO2016019019A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US9426418B2 | United States of America | B2 | |
| US9438853B2 | United States of America | B2 | |
| AU2015296540A1 | Australia | A1 | |
| KR20170018458A | Republic of Korea | A | |
| CN106537855A | China | A | |
| KR20170040216A | Republic of Korea | A | |
| KR101727450B1 | Republic of Korea | B1 | |
| CN106576081A | China | A | |
| EP3175583A1 | European Patent Office (EPO) | A1 | |
| EP3175584A1 | European Patent Office (EPO) | A1 | |
| JP2017530584A | Japan | A | |
| JP2017531345A | Japan | A | |
| BR112017001040A2 | Brazil | A2 | |
| BR112017001510A2 | Brazil | A2 | |
| CA2953711C | Canada | C | |
| EP3175584B1 | European Patent Office (EPO) | B1 | |
| CN106537855B | China | B | |
| ES2673701T3 | Spain | T3 | |
| HUE037155T2 | Hungary | T2 | |
| JP2018139407A | Japan | A | |
| JP6420006B2This record | Japan | B2 | |
| EP3175583B1 | European Patent Office (EPO) | B1 | |
| CN106576081B | China | B | |
| HUE043634T2 | Hungary | T2 | |
| AU2015296540B2 | Australia | B2 | |
| JP6602842B2 | Japan | B2 | |
| ES2739527T3 | Spain | T3 | |
| KR102366630B1 | Republic of Korea | B1 | |
| BR112017001510B1 | Brazil | B1 | |
| BR112017001040B1 | Brazil | B1 |
11 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for extension of timeJAPANESE INTERMEDIATE CODE: A601A601 | A601 |
Numbers
- Publication
- 6420006
- Publication, DOCDB
- 6420006
- Publication, EPODOC
- JP6420006B
- Application
- 28790
- Application, DOCDB
- 2018028790
- Application, EPODOC
- JP20180028790
Titles2
- Japanese
- ビデオ電話における遅延の低減
- English
- Reducing delays in video calls
Classification
- CPC, 20
- H04L47/12
- H04L47/263
- H04L43/0852
- H04L47/2416
- H04L47/283
- H04L43/0864
- H04L43/0894
- H04L43/106
- H04L43/16
- Y02D30/50
- H04N19/40
- H04N7/147
- H04N21/2365
- H04N21/23655
- H04N19/61
- H04N7/148
- H04N7/15
- H04N7/152
- H04N21/2662
- H04N21/4347
- IPC, 4
- H04L47 56
- H04N7 14
- H04L47 2416
- H04L12 875
