Bitrate adaptation of a voice-over-ip communication session
Claim Score by NHIP
Abstract
A method for adapting an encoding bitrate of real-time signals of a real-time communication session between sender devices and receiver devices of communication terminals. A sender device includes a multi-bitrate encoder using a set of discrete bitrates. The method includes a test step of increasing the encoding bitrate at the sender device by transmitting at least one redundant packet according to selected transmission parameters. A method is also provided for determining a request to adapt the encoding bitrate of real-time signals in order to implement a test of increasing the encoding bitrate at the sender device by transmitting at least one redundant packet according to selected transmission parameters. A sender device and a receiver device implementing the methods are provides as well as a terminal containing these devices.

Term
12.7 yearsto projected expiry
Projected expiry 3 June 2039, counted from filing; an application has no term until it is granted.
- Priority
- Filed
- Published
- Today
- Projected expiry
15 claims: 7 independent, 8 dependent
- 1A method comprising:adapting an encoding bitrate of real-time signals of a real-time communication session between a sender device and a receiver device of communication terminals, the sender device comprising a multi-bitrate encoder using a set of discrete bitrates, wherein the adapting comprises: a test step of increasing the encoding bitrate at the sender device by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
- 10A method comprising:determining a request to adapt an encoding bitrate of real-time signals of a real-time communication session between a sender device and a receiver device of communication terminals, the sender device comprising a multi-bitrate encoder using a set of discrete bitrates, wherein the determining comprises: the receiver device estimating an available bandwidth ( 511 ) for the receiver device;and of the receiver device constructing an adaptation request ( 517 ) to be transmitted to the sender device and comprising a request to implement a test step of increasing the encoding bitrate by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
- 11Broadest claimClaim Score 76, broad(NHIP)A sender device of a communication terminal able to implement real-time communication sessions and comprising:a multi-bitrate encoder using a set of discrete bitrates, which comprises a packet transmission unit configured to implement a test step of increasing an encoding bitrate by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
- 12A receiver device of a communication terminal able to implement real-time communication sessions and comprising:a multi-bitrate encoder using a set of discrete bitrates;and an estimation module configured to estimate an available bandwidth;and a module configured to construct and transmit to a sender device an adaptation request containing a request to implement a test step of increasing an encoding bitrate by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
- 15A non-transitory computer-readable storage medium storing a computer program comprising instructions for executing an adaptation method when the instructions are executed by a processor of a sending device, wherein the instructions configure the sending device to:adapt an encoding bitrate of real-time signals of a real-time communication session between a sender device and a receiver device of communication terminals, the sender device comprising a multi-bitrate encoder using a set of discrete bitrates, by: implementing a test step of increasing the encoding bitrate at the sender device by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
Independent claims5
197 paragraphs in 6 sections, as filed
CROSS-REFERENCE TO RELATED APPLICATIONS
0001This application is a Section 371 National Stage Application of International Application No. PCT/FR2019/051301, filed Jun. 3, 2019, the content of which is incorporated herein by reference in its entirety, and published as WO 2019/234338 on Dec. 12, 2019, not in English.
FIELD OF THE DISCLOSURE
0002The present invention relates to the field of telecommunications, and more particularly to the field of packet-switched communication networks. In this type of network, it is possible to route data streams associated with real-time services.
0003The Internet Protocol, called IP, developed by the IETF, for “Internet Engineering Task Force”, is implemented on packet-switched communication networks to support both non-real-time services, such as data transfer services, looking up web pages, electronic messaging, and real-time or conversational services, such as telephony over IP, video telephony over IP or even video broadcasting over IP.
0004The invention relates more particularly to adapting the encoding/decoding bitrate of real-time signals such as voice or video signals during a real-time communication session between two communication terminals.
0005This bitrate adaptation and the related mechanisms are suitable for the transmission of digital signals, such as audio-frequency signals (speech, music or the like), but they apply to other real-time signals, such as video.
BACKGROUND OF THE DISCLOSURE
0006One example of an existing voice over IP communication system is described with reference to <figref idref="DRAWINGS">FIG. 1</figref>. This figure describes a bidirectional voice over IP (VoIP) communication system with two telephony terminals (<b>100</b> and <b>150</b>) connected by an IP packet-switched network (<b>125</b>). The “signaling plane” is not shown in this figure, but the possible solutions for establishing and managing calls may be based on various known protocols, such as for example: <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0007">SIP/SDP (for “Session Initiation Protocol/Session Description Protocol”) in accordance with the IETF RFC 3261 and RFC 4566 specifications—as in multimedia services on IMS (for “IP Multimedia SubSystem”). To facilitate signaling exchanges on the media capacities and the associated offers/responses, it is also possible to use “SDPcapneg” in accordance with the IETF RFC 5939 specification.</li><li id="ul0002-0002" num="0008">JSEP (for “JavaScript Session Establishment Protocol”), which uses SDP syntax to define WebRTC session descriptions (with an exchange via WebSocket or another means).</li></ul></li></ul>
0009<figref idref="DRAWINGS">FIG. 1</figref> is a simplified view of the “media plane” and of the audio chain used when the call is established between 2 terminals (<b>100</b> and <b>150</b>) connected by an IP network (<b>125</b>). There is a limit here to the case of a mono audio signal, and the ambient acoustic signal is captured for example by a microphone (<b>101</b> and <b>151</b>) on each side of the communication. It will be noted that the case of a mono input/output signal may easily be generalized to a multichannel case in which a plurality of microphones and/or loudspeakers are used. Likewise, microphones and loudspeakers could be replaced with cameras and screens in the case of video signals.
0010The remote signal is rendered on a loudspeaker (<b>102</b> and <b>152</b>). The audio signals that are captured and rendered generally undergo various pre-processing/post-processing operations at sending and at reception (<b>103</b> and <b>153</b>) such as for example: <ul id="ul0003" list-style="none"><li id="ul0003-0001" num="0000"><ul id="ul0004" list-style="none"><li id="ul0004-0001" num="0011">At sending: analog-to-digital conversion, gain control, noise reduction, echo cancellation, etc.</li><li id="ul0004-0002" num="0012">At reception: digital-to-analog conversion, gain control, etc.</li></ul></li></ul>
0013The audio signal preprocessed at sending is encoded in successive frames—typically with a frame length of 20 ms—this length is generally between 10 to 60 ms for conversational applications. The encoded frames are formatted as IP packets (<b>104</b> and <b>154</b>). The packets are typically transported by the RTP (for “Real Time Protocol”) protocol, described in the IETF RFC 3550 specification; this protocol is located above the IP/UDP (for “User Datagram Protocol”) transport protocols. It will be noted that the UDP protocol may be replaced with another transport protocol, for example with TCP (for “Transmission Control Protocol”) in order in particular to facilitate the traversing of networks with network address translation NAT, proxies or firewalls.
0014At reception (<b>105</b> and <b>155</b>), the packets are received in a jitter buffer aimed at compensating the variations in the reception times, and the signal is decoded (by compensating any frame losses), and finally the reconstructed signal is post-processed (<b>103</b> and <b>153</b>) and rendered.
0015Communication is assumed here to be bidirectional, and the communication system thus forms a looped system with feedback. The feedback may be conveyed in two ways: <ul id="ul0005" list-style="none"><li id="ul0005-0001" num="0000"><ul id="ul0006" list-style="none"><li id="ul0006-0001" num="0016">“Out-of-band” (that is to say contained in packets that form an additional stream with respect to the RTP media stream). The RTCP (for Real-Time Control Protocol) protocol is typically used as feedback channel. RTCP allows transmissions of control packets in separate packets of the RTP stream. It is recalled that RTCP packets may have a substantial size and therefore result in a non-negligible additional bitrate; in addition, the possible packet loss may be problematic, because if RTCP is used for adaptation, the feedback that is conveyed may be lost. The sending of packets using the RTCP protocol may be discontinuous and non-repetitive, which may make the adaptation less reactive and dependent on network conditions (transmission delay, packet losses, etc.). In general, the application should support and use the AVPF (for “Audio Video Profile with Feedback”) RTP profile in order for RTCP to actually be usable for media adaptation. In addition, voice over IMS applications at present only allow the AVP (for “Audio Video Profile”) profile, which restricts the use of RTCP; media adaptation in terminals—if used—may be based on “monitoring” network conditions, including the information contained in the RTCP RR (Receiver Report) reports about packet loss or jitter that are sent on average every 5 seconds.</li><li id="ul0006-0002" num="0017">“In-band” (that is to say in the RTP media stream). This type of feedback may be considered more robust than RTCP because it is possible to repeat one and the same request in a plurality of successive packets. It should be noted that, in some situations (call waiting, listening to voicemail, etc.), it is possible for the RTP packets to be sent in just one direction, however it is assumed here that there is bidirectional sending of RTP packets so that the feedback works. One example of in-band signaling is the use of a CMR (for “Codec Mode Request”) field used to transmit an encoding adaptation request.</li></ul></li></ul>
0018In general, several types of degradation may potentially affect the quality of voice over IP: <ul id="ul0007" list-style="none"><li id="ul0007-0001" num="0000"><ul id="ul0008" list-style="none"><li id="ul0008-0001" num="0019">variable bandwidth/network congestion</li><li id="ul0008-0002" num="0020">packet loss, desequencing and repetition</li><li id="ul0008-0003" num="0021">packet delay (transmission, queuing, processing, etc.), delay variation (jitter)</li><li id="ul0008-0004" num="0022">clock drift between terminals</li></ul></li></ul>
0023There are various solutions for mitigating these different degradations, including adaptation solutions in terminals. There are two types of adaptation in VoIP terminals: sender-based adaptation and receiver-based adaptation. There are also variants in which the adaptation decision is assisted or taken by the network, but this case is not dealt with here because it goes beyond the scope of the invention.
0024In the case of a “sender-based” adaptation, in order for the sender to be able to make an optimal end-to-end adaptation decision, it must receive feedback from the remote receiver indicating the quality perceived at the end of the chain, with for example indicators such as the observed loss rate or the available bandwidth estimated by the remote receiver.
0025In the case of a “receiver-based” adaptation, the adaptation decision is made by the remote receiver (<b>155</b>) and transmitted by feedback to the local sender (<b>104</b>) (for example the choice of the mode or bitrate to use)—this feedback is transmitted via the remote sender (<b>154</b>) and then the local receiver (<b>105</b>).
0026<figref idref="DRAWINGS">FIG. 1</figref> indicates this feedback through dotted arrows for the direction from the receiver <b>155</b> to the sender <b>104</b>. Of course, this feedback may take place in the other communication direction, with feedback from the receiver <b>105</b> to the sender <b>154</b> via the blocks <b>104</b> and <b>155</b>; this opposite direction is not shown in <figref idref="DRAWINGS">FIG. 1</figref> so as not to overload this figure.
0027What is of more particular interest here is a bitrate adaptation in order to overcome for example bandwidth changes and network congestion.
0028In one example of adaptation solutions, in order to control a bandwidth variation or network congestion, the techniques described below may be implemented.
0029When a codec operates at a fixed rate, one adaptation solution for modifying the effective bitrate on the network is that of varying the number of consecutive signal frames in a packet (“frame bundling”) and thus varying the packet rate and the relative bitrate of the IP/UDP/RTP protocol headers (and the lower layers).
0030When a codec is multi-rate, is it also possible to change the bitrate of the codec. This bitrate change may be performed on the basis of the available bandwidth estimated by the remote receiver and received through feedback (“sender-based” decision) or on the basis of a bitrate change request received through feedback (“receiver-based” decision).
0031<figref idref="DRAWINGS">FIG. 2</figref> describes an advanced example of bitrate control of a multi-rate codec called iSAC (for “Internet Speech Audio Codec”). The iSAC codec is a proprietary codec developed by GIPS (Global IP Solutions). The iSAC source code has been available in the Google Chromium™ open source WebRTC project since 2011.
0032The codec operates in two different modes: <ul id="ul0009" list-style="none"><li id="ul0009-0001" num="0000"><ul id="ul0010" list-style="none"><li id="ul0010-0001" num="0033">Wideband (WB) mode encodes an audio band from 0 to 8 kHz with a frame length of 30 or 60 ms.</li><li id="ul0010-0002" num="0034">Super Wideband (SWB) mode encodes an audio band from 0 to 12 kHz or from 0 to 16 kHz with a frame length of 30 ms.</li></ul></li></ul>
0035The bitrate of the iSAC codec is variable; the average bitrate may range from 10 to 32 kbit/s in wideband mode and from 10 to 56 kbit/s in super wideband mode. The iSAC codec is able to operate in two transmission modes on the channel: <ul id="ul0011" list-style="none"><li id="ul0011-0001" num="0000"><ul id="ul0012" list-style="none"><li id="ul0012-0001" num="0036">“Channel Adaptive”</li><li id="ul0012-0002" num="0037">“Channel Independent”</li></ul></li></ul>
0038The “Channel Adaptive” mode takes into account the bandwidth estimates made at reception by the iSAC decoder and makes it possible to adapt the encoding bitrate, whereas the “Channel Independent” mode simply follows a target bitrate.
0039In comparison with <figref idref="DRAWINGS">FIG. 1</figref>, the audio processing blocks (<b>103</b> and <b>153</b>) and the network (<b>125</b>) have been intentionally omitted from <figref idref="DRAWINGS">FIG. 2</figref> in order to simplify the figure. The audio capturing and rendering elements (blocks <b>101</b>, <b>102</b> and <b>151</b>, <b>152</b>) may be seen.
0040The encoding bitrate (blocks <b>201</b>, <b>251</b>) is adapted in the iSAC codec (blocks <b>202</b>, <b>252</b>) based on the downlink bitrate indication, estimated at the receiver (blocks <b>205</b>, <b>255</b>).
0041Specifically, the receiver (blocks <b>205</b>, <b>255</b>) estimates the bandwidth available in the downlink direction upon each reception of packets from the following information: <ul id="ul0013" list-style="none"><li id="ul0013-0001" num="0000"><ul id="ul0014" list-style="none"><li id="ul0014-0001" num="0042">size of the packet (payload);</li><li id="ul0014-0002" num="0043">arrival time;</li><li id="ul0014-0003" num="0044">sequence number;</li><li id="ul0014-0004" num="0045">RTP sending time.</li></ul></li></ul>
0046The estimated bitrate is then made available to the sender via a shared structure (blocks <b>207</b> and <b>257</b>). The sender (blocks <b>204</b>, <b>254</b>) sends a field called BEI (for “Bandwidth Estimate Index”) whose values range from 0 to 23 in order to indicate the available bandwidth estimated by the receiver; the BEI field indicates an estimated available bandwidth value (“bottleneck”) from among values defined between the minimum and maximum target bitrates of the iSAC codec (from 0 to 11 in WB mode and from 0 to 23 in SWB mode). The BEI field is decoded at the other end of the chain (blocks <b>203</b>, <b>253</b>).
0047It will also be noted that the bandwidth estimate in the iSAC codec is also accompanied by an estimate of the jitter, which is not described here but which is used to estimate the size of the padding described later on; the estimated jitter is represented on 1 bit to indicate a low or high value, and it is transmitted in the BEI field in WB mode (0 if the BEI is between 0 and 11 and 1 if it is between 12 and 23) or encoded in the binary train (payload of the current frame) in SWB mode.
0048For the transmission direction from <b>200</b> to <b>250</b>, the bitrate adaptation is implemented on the sender side. The encoding performed at <b>201</b> has a bitrate updated by the block <b>202</b> based on a BEI field decoded at <b>203</b>. This decoded field was encoded at <b>254</b> based on an available bandwidth estimate made by block <b>255</b> of the receiver <b>155</b>.
0049The transmission in the other direction takes place in the same way, with the corresponding blocks.
0050A padding method using random bits, at the end of the packet, is performed in order to test whether it is possible to increase the encoding bitrate (“bandwidth probing”). This simple method specifically makes it possible to artificially increase the size of certain packets at sending (<b>210</b> or <b>260</b> respectively) in order to be able to estimate, at the end of the reception chain (<b>205</b> or <b>255</b> respectively), whether the bandwidth available on the transmission channel allows use of a bitrate higher than the current bitrate. It will however be noted that this method uses random bits with no usefulness other than for the bandwidth test.
0051The size and the decision to send the bit padding are determined using a rate model. At the start of the call (typically the first 10 seconds), the bitrate is kept to a minimum bitrate. Three consecutive packets containing padding bits are then sent in order to test the channel, with a maximum interval of typically 500 ms. The size of these consecutive packets is fixed so that the instantaneous bitrate associated with the packet i is given by (1+γ<sub>i</sub>) R<sub>i</sub>, where R<sub>i </sub>is the estimated available bandwidth for packet i and the adaptive term γ<sub>i </sub>(of higher value >0) is calculated “heuristically”. The principle is that of sending regular bitrate peaks (“bursts”) on the channel in order to test a bitrate increase with respect to the current bitrate.
0052The rate model implemented in blocks <b>210</b> and <b>260</b> is based on a bottleneck model limiting the bandwidth available in the network. This rate model is based on a plurality of internal states defined at the receiver, including binary detection of bandwidth overuse by the last packet received with a measurement of the time elapsed since the last detected bandwidth overuse (in ms), as well as information on the queue modeling the “bottleneck” (number of packets still to be transmitted, increase in the delay).
0053The bitrate estimate performed at the receiver is generally effective in reducing the encoding bitrate, that is to say detecting that the available bandwidth is lower than the current bitrate and that the bitrate should be lowered. By contrast, the method of padding using random bits in order to check that the current bitrate is able to be increased is not optimal, in fact it introduces random bits that are not used and not checked. On the other hand, the iSAC codec produces an average bitrate that is able to be adjusted relatively finely between a minimum bitrate and a maximum bitrate (for example 10 to 32 kbit/s or 10 to 56 kbit/s), whereas a multi-rate codec operating in accordance with a defined set of discrete bitrates, such as an AMR, AMR-WB or EVS codec, does not work over a continuous range of bitrates.
0054Furthermore, the bitrate adaptation method for the iSAC codec, using a bitrate estimate at reception and a channel test using bit padding, assumes the use of a field specific to the iSAC codec (BEI). This specific field does not exist for other types of codec, in particular for multi-rate AMR, AMR-WB and EVS codecs. It will be noted that the CMR field of the AMR, AMR-WB and EVS codecs is different from the BEI field of the iSAC codec, because one (BEI) represents information on the available bandwidth estimated at the receiver and assumes “sender-based” adaptation, and the other (CMR) makes it possible to encode an adaptation request indicating a maximum bitrate and assumes a “receiver-based” adaptation.
0055Another more advanced method—using the same principles—of estimating available bandwidth and adapting the encoding bitrate is described in the article “Making Google Congestion Control robust over Wi-Fi networks using packet grouping” by G. Carlucci, L. De Cicco, S. Holmer, S. Mascolo, published in ACM, IRTF & ISOC, Applied Networking Research Workshop, 2016. This article describes how the GCC (for “Google Congestion Control”) algorithm operates for video congestion control; the GCC adaptation algorithm is divided into two parts: a sender part and a receiver part. The GCC algorithm assumes the use of the AVPF RTP profile to send RTCP REMB (RTCP message for Receiver Estimated Maximum Bitrate) messages every second or as soon as the estimated bandwidth has dropped by 3%.
0056The sender part is based on the loss rate (“loss based”). It transmits, in the headers of the RTP packets, absolute sending times called “abs-send-time”, which are used to estimate the bandwidth available in the receiver part. It uses the RTCP REMB reports indicating the estimated bitrate A<sub>r </sub>and the frame loss f<sub>t </sub>observed by the remote receiver to adapt the encoding bitrate: the target encoding bitrate A<sub>s </sub>is respectively increased, maintained or decreased when the loss rate is negligible, low or high:
0000<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mrow><mrow><msub><mi>A</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow></mrow><mo>=</mo><mrow><mo>{</mo><mtable><mtr><mtd><mrow><mrow><mo>(</mo><mrow><mn>1</mn><mo>-</mo><mrow><mrow><mn>0</mn><mo>.</mo><mn>5</mn></mrow><mo></mo><msub><mi>f</mi><mi>l</mi></msub></mrow></mrow><mo>)</mo></mrow><mo></mo><mrow><msub><mi>A</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></mrow></mtd><mtd><mrow><mrow><mrow><mi>if</mi><mo></mo><mstyle><mspace width="0.2em" height="0.2ex" /></mstyle><mo></mo><msub><mi>f</mi><mi>l</mi></msub></mrow><mo>></mo><mn>0</mn></mrow><mo>,</mo><mn>1</mn></mrow></mtd></mtr><mtr><mtd><mrow><mrow><mn>1</mn><mo>.</mo><mn>0</mn></mrow><mo></mo><mn>5</mn><mo></mo><mrow><msub><mi>A</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow><mo></mo><mstyle><mspace width="0.2em" height="0.2ex" /></mstyle></mrow></mtd><mtd><mrow><mrow><mrow><mi>if</mi><mo></mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo></mo><msub><mi>f</mi><mi>l</mi></msub></mrow><mo><</mo><mn>0</mn></mrow><mo>,</mo><mrow><mn>0</mn><mo></mo><mn>2</mn></mrow></mrow></mtd></mtr><mtr><mtd><mrow><msub><mi>A</mi><mi>s</mi></msub><mo></mo><mrow><mo>(</mo><mrow><mi>i</mi><mo>-</mo><mn>1</mn></mrow><mo>)</mo></mrow></mrow></mtd><mtd><mi>otherwise</mi></mtd></mtr></mtable></mrow></mrow></math></maths>
0057The actual encoding bitrate is obtained as min(A<sub>s</sub>,A<sub>r</sub>), that is to say the minimum between this target encoding bitrate and the last available bandwidth value estimated by the remote receiver (received by RTCP).
0058The receiver part is based on the variation in the unidirectional delay (“delay-based”). It uses a finite state machine with 3 states (decrease, maintain, increase) and an estimate at reception of the available bandwidth using Kalman filtering. The bandwidth estimate is based on a model of the variation in the unidirectional delay (“one-way delay”), defined as
0000<br /><i>d</i><sub>m</sub>(<i>i</i>)=(<i>t</i><sub>r</sub>(<i>i</i>)−<i>t</i><sub>r</sub>(<i>i−</i>1))−(<i>t</i><sub>s</sub>(<i>i</i>)−<i>t</i><sub>s</sub>(<i>i−</i>1))
0059Where t<sub>s</sub>(i) and t<sub>r</sub>(i) are the sending times (according to the “abs-send-time” information sent in the RTP header) and reception times of the packet i, in the form of two components:
0000<br /><i>d</i><sub>m</sub>(<i>i</i>)=<i>m</i>(<i>i</i>)+<i>n</i>(<i>i</i>)
0060Where m(i) is the variation in the queue delay—estimated using Kalman filtering—and n(i) is network jitter—considered to be noise.
0061Overuse of the available bandwidth is detected by applying an adaptive threshold to the estimated variation in the queue delay m(i) in order to control the transitions between the 3 states. The bitrate estimate at reception is adapted according to the states: <ul id="ul0015" list-style="none"><li id="ul0015-0001" num="0000"><ul id="ul0016" list-style="none"><li id="ul0016-0001" num="0062">Decrease: A<sub>r</sub>(i)=αR(i) where α∈[0,85,0,95] and R(i) is the bitrate measured over the last 500 ms</li><li id="ul0016-0002" num="0063">Maintain: A<sub>r</sub>(i)=A<sub>r</sub>(i−1)</li><li id="ul0016-0003" num="0064">Increase: A<sub>r</sub>(i)=ηA<sub>r</sub>(i−1) where η∈[1,005, 1,3]</li></ul></li></ul>
0065This adaptation method uses heuristics for adaptation, and it assumes that it is possible to adapt the bitrate finely with multiplicative factors (1.05 and η) specific for the bitrate increase. It is difficult to apply to voice codecs such as EVS, which are multi-rate with a set of discrete bitrates. For example, the ratio between successive bitrates for the codec ranges from 1.11 to 1.5 for the EVS codec with fixed bitrates from 7.2 to 128 kbit/s. Furthermore, the use of RTCP packets (of REMB type or the like) is not always possible in VoIP applications restricted to an RTP profile such as AVP (for Audio Video Profile, defined in IETF RFC 3551), which limits the benefit of RTCP for adaptation purposes.
0066For current VoLTE (for “Voice over LTE (Long Term Evolution)”) telephony applications, denoting a voice transport technique on 4G LTE mobile telephony networks specified in GSMA IR.92 and VoWifi (for “Voice over Wi-Fi”) telephony applications, denoting a technique for transporting voice over the Wi-Fi network specified in GSMA IR.65, encoding adaptation mechanisms are described in chapter 10 of the 3GPP TS 26.114 specification (use of a=bw-info, ECN, ANBR, definition of CMR and RTCP-APP, etc.) with general recommendations but no adaptation obligation in the service. One example of an (informative) adaptation algorithm for VoIP is described in Annex C of the TS 26.114 specification. In practice, only the AVP RTP profile is authorized, and adaptation for voice over IMS is at present rarely authorized and configured by mobile network operators, who prefer to dimension the service with a guaranteed fixed bitrate (GBR for “Guaranteed Bit Rate”).
0067In addition, the TS 26.114 specification contains recommendations (see clause 7.5.2.1.6) on the initial encoding bitrate (ICM for “initial codec mode”) to be used at the start of a call in order to avoid link congestion and that it is recommended—if no bitrate control information has been received for a certain period of time—to gradually increase the encoding mode (bitrate) at most to the value of the ICM corresponding to the authorized encoding bitrate; if no poor quality is detected or in the absence of bitrate control information, it is recommended for the sender to increase its bitrate in progressive increments with a certain waiting time of the order of 500-600 ms. This adaptation approach is highly heuristic. The bitrate increase is based on switching to the immediately higher encoding mode, with monitoring of the quality indicators or waiting for a request to be received by the receiver (typically CMR or RTCP-APP) in order to validate the bitrate increase. This method is not optimal because it relies on increasing heuristics in gradual increments and leads to abrupt bitrate change jumps with “blind testing” of the bitrate increase.
SUMMARY
0068There is thus a need for an encoding/decoding bitrate adaptation method that overcomes the abovementioned drawbacks.
0069To this end, the invention proposes a method for adapting an encoding bitrate of real-time signals of a real-time communication session between sender devices and receiver devices of communication terminals, a sender device comprising a multi-bitrate encoder using a set of discrete bitrates. The method is such that it comprises a test step of increasing the encoding bitrate at the sender device by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
0070Testing the bitrate increase using at least one copy of a previous frame, also called “redundant packets” below, makes it possible to increase the bitrate without abrupt jumps, since the transmission parameters are able to be defined so as to avoid blindly testing the higher discrete bitrate and risking degrading the quality without being able to compensate the consequences of a direct bitrate increase. This test step makes it possible to check whether the available bandwidth is sufficient.
0071In addition, the redundancy of the packets may be used for example to correct any frame losses if the bitrate increase is not justified and if it creates a congestion problem. The redundant packets are then used both to test the bitrate increase and to correct frame losses.
0072According to one embodiment, the test step is implemented for as long as no bitrate adaptation request is received from a receiver device.
0073Thus, when the receiver device implements a bandwidth estimate, it may inform the sender device of a change to a higher or lower bitrate. In both cases, the test stops in order to perform the requested bitrate change.
0074In one particular embodiment, the test step is implemented after a bitrate change to the lower discrete bitrate.
0075This process thus makes it possible to modulate the bitrate increase even further (less abrupt increase), but it may however introduce degradations into the transmitted signal.
0076In one embodiment, the test step is implemented when a time delay has reached a threshold since the last adaptation request received from a receiver device.
0077Thus, if the encoding bitrate has not been adapted for a certain time, then a bitrate increase is undoubtedly possible and an increase test is then appropriate.
0078In one embodiment, the time delay for triggering the increase test is adapted on the basis of information on available bandwidth estimated at the receiver device.
0079Thus, depending on the state of the network and the available bandwidth, it is possible to define a shorter time to attempt a bitrate increase if for example the available bandwidth tends to increase or, on the other hand, to define a longer time when the estimated bandwidth tends to decrease.
0080In one possible embodiment, the test step is implemented on the basis of obtained information on the evolution of the encoding bitrate.
0081If the trend of the evolution of the encoding bitrate is more downward, then there is no need to perform an increase test on the encoding bitrate.
0082In one advantageous embodiment, the information on the evolution of the encoding bitrate is obtained from a history of available bandwidth estimated at the receiver device.
0083Thus, the various estimates of available bandwidth at a receiver device make it possible to obtain a trend of the evolution of this estimate and to deduce therefrom information on the evolution of the available bandwidth.
0084In another embodiment, the test step is implemented after reception of an adaptation request received from a receiver device and comprising a request to transmit redundant packets according to selected transmission parameters.
0085Thus, in this embodiment, it is the receiver device that determines the relevance of the implementation of a test step according to the bandwidth estimates made. The adaptation request is determined for example at the receiver device, on the basis of the estimated available bandwidth.
0086The present invention also targets a method for determining a request to adapt the encoding bitrate of real-time signals of a real-time communication session between sender devices and receiver devices of communication terminals, a sender device comprising a multi-bitrate encoder using a set of discrete bitrates. The method is such that it comprises a step of estimating an available bandwidth for a receiver device and of constructing an adaptation request to be transmitted to a sender device and comprising a request to implement a test step of increasing the encoding bitrate by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
0087Thus, from a bandwidth estimate made at a receiver device of a terminal, the latter is able to determine whether it is relevant to perform a test step at a sender device in order to increase the encoding bitrate. An adaptation request is then constructed accordingly.
0088The invention also targets a sender device of a communication terminal able to implement real-time communication sessions and comprising a multi-bitrate encoder using a set of discrete bitrates. The sender device is such that it comprises a packet transmission unit able to implement a test step of increasing the encoding bitrate by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
0089This device has the same advantages as the adaptation method described above that it implements.
0090The invention also targets a receiver device of a communication terminal able to implement real-time communication sessions and comprising a multi-bitrate encoder using a set of discrete bitrates. The device is such that it comprises an estimation module able to estimate an available bandwidth and a module for constructing and transmitting an adaptation request, able to construct and transmit, to a sender device, a request to implement a test step of increasing the encoding bitrate by transmitting a current frame and at least one copy of a previous frame with a chosen offset.
0091This device has the same advantages as the method for determining an adaptation request described above that it implements.
0092The invention also targets a communication terminal comprising a sender device as described and/or a receiver device as described.
0093The invention targets a computer program comprising code instructions for implementing the steps of the adaptation method as described and/or the steps of the method for determining a request as described when these instructions are executed by a processor.
0094The invention relates lastly to a storage medium, able to be read by a processor, storing a computer program comprising instructions for executing the adaptation method as described and/or the steps of the method for determining a request as described.
BRIEF DESCRIPTION OF THE DRAWINGS
0095Other features and advantages of the invention will become more clearly apparent on reading the following description, given purely by way of nonlimiting example and with reference to the appended drawings, in which:
0096<figref idref="DRAWINGS">FIG. 1</figref> illustrates a VoIP communication system known from the prior art and described above;
0097<figref idref="DRAWINGS">FIG. 2</figref> illustrates a bitrate adaptation method known from the prior art and used with the iSAC codec described above;
0098<figref idref="DRAWINGS">FIG. 3</figref> illustrates one embodiment of a voice over IP communication system according to one embodiment of the invention;
0099<figref idref="DRAWINGS">FIGS. 4<i>a </i>to 4<i>c </i></figref>illustrate, in the form of flowcharts, the steps of a method for adapting an encoding bitrate in a first embodiment of the invention;
0100<figref idref="DRAWINGS">FIGS. 5<i>a </i>and 5<i>c </i></figref>illustrate, in the form of flowcharts, the steps of a method for adapting an encoding bitrate in a second embodiment of the invention as well as the steps of a method for determining a bitrate adaptation request according to one embodiment of the invention;
0101<figref idref="DRAWINGS">FIG. 6</figref> illustrates a hardware example of a communication terminal incorporating a sender device and a receiver device, according to one embodiment of the invention.
DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS
0102With reference to <figref idref="DRAWINGS">FIG. 3</figref>, a bidirectional communication system with bitrate adaptation and use of redundancy according to one embodiment of the invention is now described.
0103The communication is performed between 2 terminals A and B. The audio capturing and rendering elements (blocks <b>101</b>, <b>102</b> and <b>151</b>, <b>152</b>) already presented with reference to <figref idref="DRAWINGS">FIG. 1</figref> may be seen. Without loss of generality, it is assumed here that the encoding is performed using the EVS codec restricted to the “EVS Primary SWB” modes over a range of possible bitrates ranging from 9.6 to 128 kbit/s. In some variants of the invention, the EVS codec could be used over a more restricted bitrate range, for example from 9.6 to 24.4 kbit/s, or a possible change of audio band could be considered, with fixed bitrates for example from 7.2 (NB+WB) to 24.4 (NB+WB+SWB) kbit/s using the maximum encoded band at each bitrate. In some variants, it is also possible to use other codecs, such as for example AMR or AMR-WB over a bitrate range ranging respectively from 4.75 to 12.2 kbit/s (for AMR) and 6.6 to 23.85 kbit/s (for AMR-WB).
0104The encoding bitrate at the sender (blocks <b>301</b>, <b>351</b>) is adapted according to the invention by preferably using in-band signaling to indicate the adaptation requests with a CMR field present in the payload of the packets for the EVS codec. This type of in-band signaling is robust—in the sense that it is able to be repeated in successive RTP packets if necessary—and does not depend on the constraints on the sending times of packets for RTCP. For codecs not having a CMR field in their payload, it is possible in some variants to define a field of equivalent functionality or to use out-of-band signaling using RTCP (for example RTCP REMB); it is assumed hereinafter that the adaptation request signaling takes place in-band using CMR.
0105It is assumed that, when the call is established, the RTP payload of the EVS codec is configured with a “header-full” mode (hf-only=1) and a systematic CMR (cmr=1). The reader is referred to the specification of the RTP payload of EVS in the specification 3GPP TS 26.445 Annex A for the definition of the SDP parameters (cmr, hf-honly, etc.) and also regarding the “packetization” modes (Compact or Header-Full) of the EVS codec. It is recalled that the CMR code called NO_REQ of the EVS codec indicates that the CMR does not contain a request, and therefore that the CMR information may be ignored; this therefore makes it possible to send packets without a request, even when the sending of the CMR is systematic. In some variants of the invention, it will be possible to use other SDP configurations, such as cmr=0 (sending CMRs on demand, only when a CMR has to be sent) or hf-only=0 (use of Compact mode except when Header-Full mode is necessary, for example when a redundant frame is added to the current packet).
0106Moreover, in one preferred embodiment, it is assumed that discontinuous transmission (DTX), in which the inactive frames are transmitted on average every 160 ms by silence descriptors (SID for Silence Description), is deactivated by the SDP parameter dtx=−1 of the EVS codec. This makes it possible to ensure continuous testing of the bandwidth on the channel. However, in some variants of the invention, the DTX mode will be activated for the EVS codec, which means that the invention will be applied only in the active signal periods, since it is not relevant to modify the discontinuous transmission mode of SID frames in order to preserve maximum efficiency of the DTX mode when this is activated. It is recalled that, for the AMR and AMR-WB codecs, it is not possible to control the DTX mode at SDP level, and so this DTX mode is always activated by default.
0107When the negotiation of the call uses the SDP protocol, this being the case in the embodiments described here, the use of redundancy at the application level (“application layer redundancy”) depends on the SDP parameter “max-red”. Typically, the “max-red” parameter gives the maximum duration (in ms)—at the sender—between the transmission of a frame (called primary frame) and the transmission of a redundant version; this parameter therefore makes it possible to set a maximum delay when redundancy is used. For example, “max-red=20” indicates that it is possible to use redundancy and that a redundant frame may be transmitted up to 20 ms after the original frame. In general, when “max-red” is set to 0, this is tantamount to deactivating the use of redundancy, and if “max-red” is not present as a signaling parameter (SDP attribute), this indicates that there is no limit on the use of redundancy—as long as the overall bandwidth specified by the parameter SDP “b=AS:” in accordance with IETF RFC 4566 and the encoding modes authorized in the session are complied with. It is assumed here that the SDP parameter “max-red” is not defined or that its value is sufficient to be able to apply the invention (for example max-red=220).
0108The receiver (blocks <b>308</b> and <b>358</b>) receives the successive RTP packets. As soon as a new packet is received, the CMR field—if present—is extracted (blocks <b>307</b> and <b>357</b>) and the content of this CMR field is written—except in the case of “NO-REQUEST” (NO_REQ) or if the CMR field is not present—to a structure called “CMR_Req” in <b>306</b> and <b>356</b>, which is shared between the sender and the receiver in one and the same terminal. In one exemplary embodiment, the CMR_Req structure will comprise a plurality of entries: <ul id="ul0017" list-style="none"><li id="ul0017-0001" num="0000"><ul id="ul0018" list-style="none"><li id="ul0018-0001" num="0109">a Boolean entry called “updated”, which indicates whether a new CMR has been received (taking the values “true” or “false”)</li><li id="ul0018-0002" num="0110">an entry called “requested_bitrate”, which indicates the (maximum) encoding bitrate requested in the CMR <br /> It will be noted that blocks <b>307</b> and <b>357</b> are entitled “Ext. CMR_A/B” in order to cover the general case of an extended CMR request used in the second embodiment; in the first embodiment, the CMR request will be a conventional request. <br /> In some variants, these entries may be supplemented with other information such as “requested_bandwidth” in order to indicate the requested encoded audio band (NB, WB, SWB, FB) in the CMR, and entries called for example “activate_ca_mode”/“ca_fec_mode”/“ca_offset” in order to respectively control the activation of “channel-aware mode” and the parameters of channel-aware mode (“FEC mode” having the value LO or HI and “offset” having the value −1, 0, 2, 3, 5 or 7). </li></ul></li></ul>
0111In one typical embodiment, the sender and the receiver are executed in parallel (for example in different execution queues or “threads”); this shared structure is then necessary, and this structure is typically accessed in a critical section with the use of a mutex to manage parallel access.
0112The CMR field to be sent is constructed and encoded by the CMR field encoding modules <b>302</b> and <b>352</b>. The received CMR field is extracted by the extraction modules <b>307</b> and <b>357</b>. The CMR field may be either, according to a first embodiment of the invention, a conventional CMR field with codes defined for codecs such as AMR, AMR-WB or EVS or, according to a second embodiment of the invention, an extended CMR field as described later in the second embodiment. In this second embodiment, an extended CMR is used to indicate the activation of redundant transmission.
0113In the preferred embodiment, use of the EVS codec is restricted to Primary SWB mode in order to simplify the description. The encoding and sending parameters adapted according to the invention are in this case the encoding bitrate and the use of 100% redundancy. In some variants, more sending parameters could be considered for the adaptation, for example bandwidth (NB, WB, SWB or FB) in the case of the EVS codec—if the bitrate range used for the adaptation also allows the encoded band to be changed —, the type of redundancy (for example partial redundancy, or redundancy greater than 100%).
0114Redundancy is defined here as a transmission mode with repetition of encoded frames at the application level, as described in chapter 10 of specification 26.114. Consideration is given more particularly to the use of what is called 100% redundancy, which means that the current packet contains the payload of the current frame and the payload of a previous frame shifted by a predetermined offset. This type of redundancy is thus tantamount to (approximately) doubling the instantaneous transmission bitrate (assuming that the bitrate of the current frame and that of the redundant frame are identical) by copying a previous packet in a one-off manner, in one embodiment.
0115The reader is referred to chapter 10 of the 3GPP TS 26.114 specification for illustrations of cases of redundancy with the AMR, AMR-WB and EVS codecs, where the encoded frame N is repeated in a following packet with a distance called “offset” and denoted K here. When K=1, the packet N contains the frame N and the previous frame N−1, whereas when K=2 the packet N contains the frame N and the frame N−2 as well as an empty frame (NO_DATA). For a larger offset K, the number of empty frames (NO_DATA) “inserted” between the current frame of index N and the redundant frame of index N−K is K−1. It will be noted that it is possible to combine redundancy with the aggregation of frames, but this case is not described here (see 3GPP TS 26.114 FIGS. 10.12 and 10.13).
0116According to the invention, the sending parameters (bitrate, activation of redundancy) are provided by block <b>305</b>, <b>355</b> to the encoder/sender (block <b>301</b>, <b>351</b>), which applies these parameters when encoding and transmitting the next encoded frame (or the next encoded frames) until a new CMR request is received.
0117It is assumed here that the initial encoding bitrate is set to the lowest bitrate authorized in the session, assuming that there is no quality of service (QoS) guarantee. However, in some variants, this initial bitrate may be defined as the maximum authorized bitrate (if there is information on a guaranteed bitrate “GBR”) or a predetermined intermediate bitrate between the minimum bitrate and the maximum bitrate.
0118Upon reception of a new RTP packet, a bandwidth estimate is performed (blocks <b>304</b> and <b>354</b>). In one embodiment, this estimate will be similar to the estimate performed in the iSAC codec based on information on the last packet received (having kept the history of the analysis performed based on the previous packets): <ul id="ul0019" list-style="none"><li id="ul0019-0001" num="0000"><ul id="ul0020" list-style="none"><li id="ul0020-0001" num="0119">size of the packet (RTP payload excluding protocol headers)</li><li id="ul0020-0002" num="0120">arrival time</li><li id="ul0020-0003" num="0121">sequence number (RTP field)</li><li id="ul0020-0004" num="0122">timestamp (RTP field)</li></ul></li></ul>
0123This available bandwidth estimate is performed each time a new RTP packet is received by the receiver. In some variants, it is possible to use an estimate different from the estimate of the bandwidth at the receiver, for example the estimate of the GCC congestion control algorithm, or other methods of estimating the available bandwidth from the same information on the reception of packets or derived information, such as estimated loss rate or estimated jitter.
0124It is assumed here that the history of the estimated bandwidth upon reception of the packets over a predetermined duration, for example 500 ms, including the packet that has just been received, is stored in block <b>304</b> and <b>354</b>, and that a shared structure called “BW_info” at <b>303</b> and <b>353</b> is defined in order to be able to communicate to the sender that a new CMR should be sent and the information associated with this CMR. In one exemplary embodiment, the structure “BW_info” comprises the following entries: <ul id="ul0021" list-style="none"><li id="ul0021-0001" num="0000"><ul id="ul0022" list-style="none"><li id="ul0022-0001" num="0125">a Boolean entry called “updated”, which indicates whether a new CMR will need to be sent (taking the values “true” or “false”)</li><li id="ul0022-0002" num="0126">an entry called “requested_bitrate”, which gives the encoding bitrate corresponding to the available bandwidth estimated by the receiver</li></ul></li></ul>
0127In some variants of the invention, it will be possible to supplement these entries with additional information in order to construct a CMR request to change the encoded audio band, to activate the “channel-aware mode” of the EVS codec. In the second embodiment, the structures “BW_info” and “CMR_Req” will also comprise a field called “burst_uplink” associated with an extended CMR request, as explained later in the second embodiment.
0128From the history of available bandwidth estimated at <b>304</b>, <b>354</b>, it is possible to take into account not only the instantaneous value of the bandwidth (updated when the last packet is received), but also its trend over the time horizon defined by the history (here for example 500 ms). It is considered here by way of example that the trend of the evolution of the available bandwidth is estimated through simple linear regression. It is thus possible to calculate the slope of the linear model y=f(x), where x is the arrival time of the packets (in seconds or ms) and y is the estimated available bandwidth. This slope (also called directional coefficient) is given for example analytically in the form: Σ(x<sub>i</sub>−<o ostyle="single">x</o>)(y<sub>i</sub>−<o ostyle="single">y</o>)/Σ(x<sub>i</sub>−<o ostyle="single">x</o>)<sup>2 </sup>where (x<sub>i</sub>,y<sub>i</sub>) are the reception time and the estimated bandwidth upon reception of the packet i, <o ostyle="single">x</o> and <o ostyle="single">y</o> are the averages of x<sub>i </sub>and y<sub>i </sub>over the time horizon under consideration. In some variants, it is possible to use robust linear regression variants, with regularization according to the L1 or L2 norm. In some variants, it is also possible to use a trend estimate derived from the available bandwidth estimate (which is possible for example if Kalman filtering is used as in the GCC algorithm).
0129Before encoding the next frame, the extracted request is read and converted (blocks <b>305</b> and <b>355</b>) into encoding/sending parameters, which are listed below: <ul id="ul0023" list-style="none"><li id="ul0023-0001" num="0000"><ul id="ul0024" list-style="none"><li id="ul0024-0001" num="0130">the encoding bitrate to be used</li><li id="ul0024-0002" num="0131">the use of 100% redundancy</li></ul></li></ul>
0132It should be noted that the size of the RTP payload is often slightly greater than the “net payload” associated with the encoding bitrate. For example, for EVS encoding with a “Header-full” transport mode and with a systematic CMR (hf-only=1, cmr=1), two bytes are systematically added to the encoded frame in order to form the payload.
0133An additional overhead may also be present in a transmission context, such as for example with WebRTC technology, in which RTP header extensions are used, thereby adding additional bytes to the RTP packets. For example, for voice communication, 12 bytes may be added to the RTP header if the following configuration is used: <ul id="ul0025" list-style="none"><li id="ul0025-0001" num="0000"><ul id="ul0026" list-style="none"><li id="ul0026-0001" num="0134">A “one-byte” header extension in accordance with RFC 5285 (with a preamble 0xBEDE on 2 bytes and a length field indicating “length=2” on 2 bytes to signal that 2 types of extension are added), for a subtotal of 4 bytes.</li><li id="ul0026-0002" num="0135">A “one-byte” extension on 2 bytes, of the type: <ul id="ul0027" list-style="none"><li id="ul0027-0001" num="0136">a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level</li></ul></li><li id="ul0026-0003" num="0137">A “one-byte” extension on 4 bytes, of the type: <ul id="ul0028" list-style="none"><li id="ul0028-0001" num="0138">a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time</li></ul></li><li id="ul0026-0004" num="0139">Padding of 2 null bytes. <br /> In another configuration, a single type of extension (for example the above “ssrc-audio-level” extension) could be used, which would add only 8 additional bytes. </li></ul></li></ul>
0140If this additional bitrate is not taken into account (subtracted from the size of the RTP packet) in the available bandwidth estimate at the receiver, this is tantamount to testing a bitrate higher than the current encoding mode. Thus, the bandwidth actually used for a given frame (without 100% redundancy) will be biased toward a higher encoding bitrate value, because of the additional bytes in the RTP header if the RTP headers are only taken into account after estimating bandwidth. In the preferred embodiment, this bias will be eliminated by giving the estimate of available bandwidth (block <b>304</b>, <b>354</b>) only the size of the payload without the RTP headers. In some variants of the invention, there will be provision not to eliminate this bias, and it will be assumed that the receiver will compensate this bias before forming the CMR to be sent.
0141A first embodiment is first of all presented in which the sender and the receiver act in a coordinated manner and in which the decision to activate redundancy is taken by the sender based on information on the reception of CMR and the current bitrate. The current bitrate is in particular used to check whether it corresponds to the maximum bitrate authorized in the session, in which case redundancy is not activated. According to this first embodiment, the sender decides to test the channel by sending packets with redundancy, after a certain delay before triggering a bitrate increase test. In a first embodiment, consideration is given first of all to the case where each packet resulting from this test contains redundancy. In another more optimized embodiment, the redundancy will be sent only intermittently in order to limit the average bitrate peak caused by the channel test. In this first embodiment, the initiative to decide on redundancy returns to the sender.
0142It is recalled that the “normal” logic of the CMR field for AMR, AMR-WB and EVS codecs is to indicate that a maximum bitrate indicated by the CMR should not be exceeded; according to the invention; the sender therefore normally has to comply with this constraint, which is particularly important in cases of interoperability with mobile systems in circuit-switched mode, which may have a maximum encoding bitrate (for example 12.65 kbit/s for AMR-WB) lower than that authorized in voice over LTE (for example 23.05 kbit/s). Thus, in order to retain interoperability with other systems (including interoperability gateways), provision will be made in this first embodiment to define an SDP parameter “adapt”, the purpose of which will be to check that the two communicating terminals are indeed compatible with the invention. In this case, the logic of the CMR field may be modified in order to allow the sender to temporarily exceed the limit imposed by the last CMR received, in order to test a bitrate increase (which by nature does not comply with the maximum bitrate constraint of the last CMR received). If a terminal implementing the invention communicates with another terminal that is not compatible with the SDP parameter “adapt”, the invention cannot be used because it assumes that the remote terminal uses CMR to send a bitrate corresponding to an estimated bandwidth at reception, which would not be the case.
0143In this embodiment, when the available bandwidth estimated at the receiver is different from the current bitrate observed by the receiver (based on the last packet received), a CMR is sent to change the bitrate. This approach is in particular sufficient to reduce the bitrate, because the bandwidth estimate according to the prior art generally works well enough to detect that the current bitrate should be lowered when it exceeds the capacity of the channel. Specifically, at the onset of congestion, the receiver will observe an increase in the queue delay or in the loss rate. However, when the current bitrate is less than the available bandwidth, the bandwidth is generally underestimated if a bitrate increase test is not implemented.
0144Thus, in order to be able to increase the bitrate, the sender, according to the invention, sends packets containing redundancy to test the channel. If losses are caused by this bitrate increase, they may be compensated by redundancy. Using redundancy makes it possible to test the channel before actually increasing the bitrate and to proactively compensate induced losses.
0145In this embodiment, the following adaptation parameters are set and define the characteristics of the redundancy packets or “bursts” according to a following example of values: <ul id="ul0029" list-style="none"><li id="ul0029-0001" num="0000"><ul id="ul0030" list-style="none"><li id="ul0030-0001" num="0146">(maximum) redundancy duration: L<sub>burst</sub>=100 (defined as a number of frames);</li><li id="ul0030-0002" num="0147">redundancy offset: K<sub>burst </sub>(its value is set to a value >0, for example to 2, during redundancy, and it has a value of −1 when redundancy is deactivated), it will be seen later that the offset may also take a value that will be a function of a redundancy sending frequency parameter;</li><li id="ul0030-0003" num="0148">elapsed time (“timer”) since the last CMR received: T<sub>CMR </sub>reinitialized at 0 upon each reception of CMR (with a given bitrate, other than NO_REQ);</li><li id="ul0030-0004" num="0149">delay for triggering the redundancy bitrate increase test: T<sub>burst </sub>(defined as a number of frames) set at 250</li><li id="ul0030-0005" num="0150">adjustment factor controlling the delay for triggering the test: f<sub>burst </sub>(its value is adapted but reinitialized at 1.0);</li><li id="ul0030-0006" num="0151">parameter for adapting the factor f<sub>burst</sub>:φ<sub>burst</sub>=1.5. <br /> In some variants, other values may be assigned to the above parameters. </li></ul></li></ul>
0152<figref idref="DRAWINGS">FIGS. 4<i>a</i>, 4<i>b </i>and 4<i>c </i></figref>show one exemplary implementation of a bitrate adaptation method with redundancy according to the invention in the form of flowcharts, implemented in a bidirectional system.
0153<figref idref="DRAWINGS">FIG. 4<i>a </i></figref>shows the steps implemented by the receiver according to a first embodiment of the invention, upon reception of a packet.
0154In this embodiment, the communication is bidirectional and the following steps may be applied in both terminals.
0155Upon reception of a packet (in <b>401</b>), two types of information are extracted: <ul id="ul0031" list-style="none"><li id="ul0031-0001" num="0000"><ul id="ul0032" list-style="none"><li id="ul0032-0001" num="0156">transmission information for the current packet, allowing estimation of the available bandwidth (at <b>402</b>) according to one of the estimation methods described above. The extraction of this information is followed by the steps X described with reference to <figref idref="DRAWINGS">FIG. 4</figref><i>b; </i></li><li id="ul0032-0002" num="0157">information on the type of adaptation requested from the CMR request sent by the other end of the communication (at <b>403</b>). The extracted CMR is indicated in the shared structure “CMR_req”, in particular in order to signal that a new CMR has been received if this is present and different from NO_REQ. The “updated” entry of the CMR_Req structure is set to the value “true” when a CMR (other than NO_REQ) has been extracted.</li></ul></li></ul>
0158It is recalled that, for the EVS codec, the CMR field is encoded on one byte called “CMR byte” and that it is constructed from 3 fields: H (1-bit “header”), T (“type” on 3 bits) and D (“data” on 4 bits) according to tables A.2 and A.3 of Annex A of the 3GPP TS 26.445 specification. It is therefore possible to extract the requested bitrate D<sub>CMR </sub>and possibly other encoding parameters (such as the encoded band) by decoding the CMR (if this is received and other than the value NO_REQ corresponding to T=111 and D=1111 in binary).
0159No description is given here of the steps known to those skilled in the art, which consist in extracting the headers (CMR and ToC for “Table of Content” fields if present), in demultiplexing and decoding the encoded frames from the payload of the received packet for the EVS codec. In particular, in the event of packet losses, the possible redundancy will be used by the receiver to correct losses if the lost current frame has been duplicated in another packet with a given offset.
0160<figref idref="DRAWINGS">FIG. 4<i>b </i></figref>describes the steps X implemented at the receiver after extracting the information on the current packet (<b>402</b>). The bandwidth is estimated using one of the methods described above (at <b>411</b>). Next, mapping between the estimated bandwidth and the discrete EVS bitrates is performed (at <b>412</b>) in order to determine whether there is a need to change the current reception bitrate and whether it is possible to change from one discrete bitrate to another.
0161In the preferred embodiment, this mapping is implemented using the pseudo-code given in Annex 1 in order to obtain the bitrate D from the available bandwidth B.
0162In some variants, this mapping could be modified by taking for example the discrete higher or lower bitrate of the EVS codec closest to the estimated bandwidth.
0163If the estimated bitrate is equivalent to the current bitrate (N at <b>413</b>), no bitrate information (block <b>414</b>) is written to the shared structure “BW_info” and the “updated” entry of this structure is set to the value “false”. If the estimated bitrate is different from the current bitrate (Y at <b>413</b>), the necessary information (the value of the “requested-bitrate” entry) is written to the shared structure “BW_Info” in step <b>415</b> so that the sender is able to encode the CMR with the next frame to be sent, also setting the value of the “updated” entry to “true”. A conventional request to change the bitrate to a corresponding discrete bitrate will then be constructed and encoded as in the case of an EVS codec from the prior art.
0164<figref idref="DRAWINGS">FIG. 4<i>c </i></figref>describes the steps, implemented at the sender (including the encoder), of a method for adapting an encoding bitrate according to a first embodiment of the invention.
0165Upon receiving a new signal frame to be encoded (step <b>420</b>), the sender checks whether a CMR has been received in the shared structure “CMR_Req” at <b>421</b> (by checking whether the “updated” entry has changed to “true”). If this is the case, it adapts the encoding and sending parameters (at <b>424</b>), if a CMR containing an adaptation request exists, then it sets the value of the “updated” entry of the structure CMR_Req to “false”. Additionally, if a CMR was received with a requested bitrate different from the current encoding bitrate, a CMR reception indication is stored in memory and the frame counter from the last received CMR is reset to 0 (T<sub>CMR</sub>=0) in step <b>424</b>.
0166At <b>425</b>, it is checked whether a bitrate increase test with redundancy is in progress, by checking whether N<sub>burst</sub>≥0. If this is the case (Y at <b>425</b>), then this test is deactivated at <b>426</b>. The redundancy parameters are reinitialized at <b>426</b>, with N<sub>burst</sub>=−1 (N<sub>burst </sub>representing the number of frames with redundancy still to be transmitted) and K<sub>burst</sub>=−1. In addition, the delay for triggering the bitrate increase test is adapted depending on the bitrate adaptation requested in the CMR request.
0167Thus; <ul id="ul0033" list-style="none"><li id="ul0033-0001" num="0000"><ul id="ul0034" list-style="none"><li id="ul0034-0001" num="0168">If the current encoding bitrate is higher than the received CMR (R<sub>s</sub>>R<sub>CMR</sub>), the receiver asks to lower the bitrate. It is then decided to increase the time delay for triggering a bitrate increase test. To this end, the adjustment factor is increased by setting f<sub>burst</sub>=f<sub>burst</sub>+φ<sub>burst</sub>. The time delay for triggering a test is, as explained later, dependent on the factor f<sub>burst </sub>according to the following formula:</li></ul></li></ul>
0000<br />τ<sub>CMR</sub><i>=f</i><sub>burst</sub>·τ<sub>burst </sub><ul id="ul0035" list-style="none"><li id="ul0035-0001" num="0000"><ul id="ul0036" list-style="none"><li id="ul0036-0001" num="0169">If not, the remote receiver asks to increase the bitrate (R<sub>s</sub><R<sub>CMR</sub>) and it is chosen to reduce the time delay for triggering a bitrate increase test by adapting the factor according to the formula f<sub>burst</sub>=f<sub>burst</sub>/φ<sub>burst</sub>. It is also possible to add a minimum time interval condition with for example the additional step: f<sub>burst</sub>←max(f<sub>burst</sub>,0.15). <br /> This adaptation logic follows the AIMD (“Additive Increase Multiplicative Decrease”) congestion control principle. </li></ul></li></ul>
0170If there is no CMR received in the shared structure or if the CMR does not contain an adaptation request (NO_REQ), then step <b>422</b> is implemented.
0171At <b>422</b>, the sender checks whether a bitrate increase test should be implemented. It is checked that the conditions for activating an encoding bitrate increase test are met. More precisely, in the preferred embodiment, it is decided to test a bitrate increase at <b>422</b> if the following conditions are simultaneously met: <ul id="ul0037" list-style="none"><li id="ul0037-0001" num="0000"><ul id="ul0038" list-style="none"><li id="ul0038-0001" num="0172">If the current bitrate R<sub>s </sub>is less than the maximum bitrate R<sub>s</sub><sup>max </sup>authorized in the session. If not, if R<sub>s</sub>=R<sub>s</sub><sup>max</sup>, no test is implemented (<b>429</b>). It is also possible to reinitialize f<sub>burst</sub>=1.</li><li id="ul0038-0002" num="0173">If the time elapsed since the last CMR request received T<sub>CMR </sub>(defined as the number of frames since the last CMR) is greater than a predetermined time delay τ<sub>CMR </sub>(for example 500 ms). τ<sub>CMR</sub>=f<sub>burst</sub>·τ<sub>burst </sub>is adopted here; in some variants, it is possible for example to set τ<sub>CMR</sub>=250 frames of 20 ms (which corresponds to 5 seconds). If not, if the time required is insufficient (T<sub>CMR</sub>≤τ<sub>CMR</sub>), no test is implemented (<b>429</b>). <br /> In some variants, it is also possible to add an additional condition: </li><li id="ul0038-0003" num="0174">If the evolution trend of the bitrate is stable with a positive trend compared to the bitrate history. It is possible for example to use simple linear regression on the estimated bandwidth as a function of the arrival time of the packets over a period of 500 ms to calculate a slope (see the estimate of the slope described above or the associated variants). If the slope exceeds a positive or zero threshold (for example 0), a redundancy test may be started in order to test the channel. If not, if the slope is negative, no test is implemented (<b>429</b>).</li></ul></li></ul>
0175If a bitrate increase test (“burst”) is to be activated (Y at <b>422</b>), then the redundancy parameters to be applied at <b>423</b> are defined. It is checked: <ul id="ul0039" list-style="none"><li id="ul0039-0001" num="0000"><ul id="ul0040" list-style="none"><li id="ul0040-0001" num="0176">Whether the encoder does not have redundancy in progress (N<sub>burst</sub><0; N<sub>burst </sub>representing the frame counter with redundancy) when encoding the current frame. If this is the case, then the following parameters are set: the offset of the redundancy at K<sub>burst </sub>(for example at 2) and the frame counter with redundancy is initialized at N<sub>burst</sub>=L<sub>burst</sub>+K<sub>burst </sub>with for example L<sub>burst </sub>at 100.</li><li id="ul0040-0002" num="0177">If the encoder has redundancy in progress (N<sub>burst</sub>≥0) when encoding the current frame, then it is checked: <ul id="ul0041" list-style="none"><li id="ul0041-0001" num="0178">Whether the end of the current applied redundancy has been reached (N<sub>burst</sub>=0). In this case, K<sub>burst</sub>=−1 is set and τ<sub>CMR</sub>=0 is reinitialized.</li><li id="ul0041-0002" num="0179">If not: the frame counter with redundancy is decremented with N<sub>burst</sub>=N<sub>burst</sub>−1 <br /> At E427, the sender checks whether a CMR should be transported in the packet associated with the current frame to be encoded. If so, the data from the CMR in the “BW_Info” structure are extracted for subsequent encoding of the CMR. </li></ul></li></ul></li></ul>
0180The sender then creates the header of the current RTP payload, if the CMR and ToC fields are to be inserted into the current packet. For the CMR field, the reader is referred to tables A.2 and A.3 of Annex A of the 3GPP TS 26.445 specification for the construction and the encoding of the CMR field for the EVS codec based on the information on the CMR request (bitrate, encoded band, etc.) to be encoded. If a CMR field should be sent but the “updated” entry of the structure “BW_Info” has the value “false”, the code NO_REQ will be used, which corresponds to T=111 and D=1111 (in binary). Similarly, the encoding of the ToC field is described in figure A.4 and tables A.4 and A.5 of Annex A of TS 26.445. According to the invention, if 100% redundancy is used in the current packet, the ToC field will simultaneously describe the bitrate of the current frame, the bitrate of the redundant frame and the redundancy offset K<sub>burst </sub>(using the appropriate number of ToCs associated with NO_DATA). It is recalled that examples of a redundant frame structure are given in chapter 10 of the 3GPP TS 26.114 specification and the construction of the ToC field follows the principles given in this specification for the AMR, AMR-WB and EVS codecs.
0181The current frame is encoded at <b>427</b> and added to the data of the packet.
0182If redundancy is activated (Y at <b>422</b>), the redundant encoded frame corresponding to the offset K<sub>burst </sub>(stored in a queue or other data structure) is also added to the data of the current packet to be transmitted at <b>428</b>.
0183The packet thus formed with the headers (if present) and encoded data is transmitted (<b>428</b>) using the information relating to the size of the RTP payload and the number of frames encoded since the reception of the last CMR is incremented by 1 (T<sub>CMR</sub>←T<sub>CMR</sub>+1). The current encoded frame is stored in a queue for possible later use as a redundant frame.
0184It will be noted that it is not always possible to double the encoding bitrate, if the SDP session (with the parameter b=AS) or the quality of service parameters (GBR and MBR on a VoLTE mobile network) limit the maximum bitrate usable by the codec.
0185In some variants of the invention, the definition of an SDP parameter of the type b=AS limiting the maximum bitrate usable by the codec will for example be taken into account. For example, in the case of the EVS SWB codec, it is possible to have a parameter b=AS limiting the bitrate to 24.4 kbit/s (at transmission of a single frame of 20 ms). In this case, according to the invention, the use of redundancy may be limited in two ways: <ul id="ul0042" list-style="none"><li id="ul0042-0001" num="0000"><ul id="ul0043" list-style="none"><li id="ul0043-0001" num="0186">if the network does not reject the packets on the basis of the instantaneous bitrate (for example a bitrate greater than 24.4 kbit/s) but using a sliding average of the bitrate over a time window (for example of a duration of 2 seconds as indicated in section 7.5.5.1 of the 3GPP TS 26.114 specification), it is possible to keep the embodiment of alternate transmission of packets with or without redundancy, that is to say use intermittent redundant transmission for the bitrate increase test. The transmission frequency of the redundant packets is then adapted to this maximum bitrate constraint. One variant of the first embodiment is described below for this case.</li><li id="ul0043-0002" num="0187">if no packet may exceed a size limit corresponding to the maximum bitrate (for example a size corresponding to 24.4 kbit/s), it is possible to test for example 2×9.6 kbit/s but not 2×13.2 kbit/s, and it is thus possible to change to the bitrates of 13.2 or 16.4 kbit/s, which are between 9.6 and 2×9.6 kbit/s; on the other hand, it will be difficult to test the change to 24.4 kbit/s, unless alternating between encoding at 9.6 and encoding at 13.2 and associating an encoded frame and a redundant frame for a bitrate of around 9.6+13.2 kbit/s. In any case, this variant means that, before applying redundancy, a reduction of the bitrate to a lower discrete bitrate should potentially be performed (for example to test whether it is possible to change from 13.2 to 16.4 or from 16.4 to 24.4 kbit/s). A bitrate (for example 9.6 kbit/s) lower than the current bitrate (for example 13.2 or 16.4 kbit/s) will be used. Specifically, it would not be possible to use 2×13.2 kbit/s, because this would exceed the bitrate of 24.4 kbit/s and only the cases 2×9.6 kbit/s and 9.6+13.2 kbit/s will be authorized.</li></ul></li></ul>
0188In some variants of the invention according to the first embodiment, the activation of redundancy may be authorized only during active signal periods in order to minimize the impacts of the bitrate increase.
0189A description has been given above of a bitrate increase test with 100% redundancy, that is to say redundancy for each packet transmitted during a determined number of frames (L<sub>burst</sub>). In some variant embodiments, such as for the second embodiment described below, redundancy will be activated only intermittently. For the sake of simplification, redundancy “frequency” will be the name given to the period with which a redundant packet is used—thus a frequency of x means that a redundant frame is inserted every x packets.
0190Rather than (approximately) doubling the encoding bitrate, redundancy is used for example every 2 or 3 packets, thereby making it possible to obtain on average a relative increase in fractional bitrate, in order to test (on average) the change to the immediately higher discrete bitrate and not double the bitrate.
0000For example, if the bitrate is 9.6 kbit/s, activating redundancy increases the instantaneous peak bitrate to approximately 2×9.6 kbit/s, but the actual average bitrate on the channel will be (freq+1)/freq×9.6, that is to say 2×9.6 (approx. 19.2) if freq=1, 1.5×9.6 (approx. 14.4) if freq=2, 4/3×9.6 (approx. 12.8) if freq=3.
0191The redundancy offset is set to the same value as the frequency parameter (freq) so that redundancy is able to be used.
0192This other parameter defining the redundancy may be determined in step <b>423</b> of <figref idref="DRAWINGS">FIG. 4<i>c </i></figref>on the basis of information on the current bitrate and the immediately higher bitrate. For example, if the 9.6 to 24.4 kbit/s bitrate range is able to be used in a session with the EVS codec in Super-Wideband mode, it is possible to use: <ul id="ul0044" list-style="none"><li id="ul0044-0001" num="0000"><ul id="ul0045" list-style="none"><li id="ul0045-0001" num="0193">starting from a bitrate of 9.6 kbit/s, the bitrate increase test may be performed with a redundant frame encoded at 9.6 kbit/s every 2 packets (so as to arrive at an average bitrate on the channel close to 14.4 kbit/s) or every 3 packets (approximately 12.8 kbit/s)</li><li id="ul0045-0002" num="0194">starting from a bitrate of 13.2 kbit/s, the bitrate increase test may be performed with a redundant frame encoded at 13.2 kbit/s every 3 packets (approx. 17.6 kbit/s) or every 4 packets (approx. 16.5 kbit/s)</li><li id="ul0045-0003" num="0195">starting from a bitrate of 16.4 kbit/s, the bitrate increase test may be performed with a redundant frame encoded at 16.4 kbit/s every 2 packets (approx. 24.6 kbit/s)</li></ul></li></ul>
0196The redundancy sending frequency is therefore adaptive, depending on the current bitrate. It is 2 or 3 at 9.6 kbit/s, 3 or 4 at 13.2 kbit/s, 2 at 16.4 kbit/s.
0197The redundancy offset will preferably be defined as being equal to the sending frequency of the redundant packets in order to compensate possible losses of larger packets. In some variants, it will be possible to use another offset to repeat for example the frame that immediately follows the packet with redundancy.
0198More generally, the frequency may be chosen on the basis of the current discrete bitrate D<b>0</b> and the immediately higher discrete bitrate D<b>1</b> for a given codec, such as the integer value closest to 1/(D<b>1</b>/D<b>0</b>−1).
0199By way of example, consideration is given to the following cases (Table 1), which will be used preferably in this variant of the first embodiment:
0000<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="5"><colspec colname="offset" colwidth="21pt" align="left" /><colspec colname="1" colwidth="28pt" align="center" /><colspec colname="2" colwidth="63pt" align="center" /><colspec colname="3" colwidth="42pt" align="center" /><colspec colname="4" colwidth="63pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="4" rowsep="1">TABLE 1</entry></row><row><entry /><entry namest="offset" nameend="4" align="center" rowsep="1" /></row><row><entry /><entry>D0</entry><entry>D1</entry><entry>1/(D1/D0-1)</entry><entry>freq</entry></row><row><entry /><entry namest="offset" nameend="4" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="5"><colspec colname="offset" colwidth="21pt" align="left" /><colspec colname="1" colwidth="28pt" align="char" char="." /><colspec colname="2" colwidth="63pt" align="char" char="." /><colspec colname="3" colwidth="42pt" align="char" char="." /><colspec colname="4" colwidth="63pt" align="char" char="." /><tbody valign="top"><row><entry /><entry>9.6</entry><entry>13.2</entry><entry>2.66</entry><entry>3</entry></row><row><entry /><entry>13.2</entry><entry>16.4</entry><entry>4.125</entry><entry>4</entry></row><row><entry /><entry>16.4</entry><entry>24.4</entry><entry>2.05</entry><entry>2</entry></row><row><entry /><entry namest="offset" nameend="4" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
0200A second embodiment is presented next, in which the sender and the receiver still act in a coordinated manner but in which this time the receiver sends an explicit request to activate 100% redundancy in order to test a bitrate increase, by way of an extended CMR request. This description assumes a maximum bitrate of 24.4 kbit/s and a value of b=AS corresponding to this maximum bitrate.
0201In this embodiment, the existing CMR codes of the EVS codec (as defined in table A.3 of the 3GPP TS 26.445 specification) are used to send a request to a given encoding bitrate, when this involves lowering or increasing the bitrate on the basis of the estimated available bandwidth. However, a non-conventional CMR, called extended CMR, is necessary to send a redundancy activation request, because the existing CMR codes of the EVS codec only allow adaptation in terms of bitrate, audio band and control of a special partial redundancy mode at 13.2 kbit/s (called “channel-aware mode”). It is assumed that the SDP session includes an additional parameter called “adapt”, without loss of generality, in order to signal and authorize the use of such an extended CMR, and that both terminals are compatible with the SDP parameter “adapt”. To indicate the activation of a bitrate increase test step through transmission of redundant packets, for example, 100% redundancy, in the case of the EVS codec, according to this embodiment, CMR codes are used that are free, left for future use or reserved and that are not currently used in the specifications of the EVS codec.
0202By way of example, consideration is given to the following codes (Table 2):
0000<tables id="TABLE-US-00002" num="00002"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="14pt" align="left" /><colspec colname="1" colwidth="63pt" align="left" /><colspec colname="2" colwidth="140pt" align="left" /><thead><row><entry /><entry namest="offset" nameend="2" rowsep="1">TABLE 2</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row><row><entry /><entry>CMR code</entry><entry /></row><row><entry /><entry>(HTD)</entry><entry>EVS request</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>1 111 0000</entry><entry>RED 2 × 9.6-SWB, freq. 1, offset = 1</entry></row><row><entry /><entry>1 111 0001</entry><entry>RED 2 × 9.6-SWB, freq. 2, offset = 2</entry></row><row><entry /><entry>1 111 0010</entry><entry>RED 2 × 9.6SWB, freq. 3, offset = 3</entry></row><row><entry /><entry>1 111 0011</entry><entry>RED 2 × 13.2-SWB, freq. 1, offset = 1</entry></row><row><entry /><entry>1 111 0100</entry><entry>RED 2 × 13-2-SWB, freq. 2, offset = 2</entry></row><row><entry /><entry>1 111 0101</entry><entry>RED 2 × 13.2-SWB, freq. 3, offset = 3</entry></row><row><entry /><entry>1 111 0110</entry><entry>reserved</entry></row><row><entry /><entry>1 111 0111</entry><entry>reserved</entry></row><row><entry /><entry>1 111 1000</entry><entry>reserved</entry></row><row><entry /><entry>1 111 1001</entry><entry>reserved</entry></row><row><entry /><entry>1 111 1010</entry><entry>reserved</entry></row><row><entry /><entry>1 111 1011</entry><entry>reserved</entry></row><row><entry /><entry>1 111 1100</entry><entry>reserved</entry></row><row><entry /><entry>1 111 1101</entry><entry>reserved</entry></row><row><entry /><entry>1 111 1110</entry><entry>reserved</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>
0203in which 100% redundancy is activated by the sender intermittently every 1, 2 or 3 packets (depending on the sending frequency) in order to increase the bitrate.
0204For example, if the bitrate is 9.6 kbit/s, activating redundancy increases the instantaneous peak bitrate to approximately 2×9.6 kbit/s, but the actual average bitrate on the channel will be (freq+1)/freq×9.6, that is to say 2×9.6 (approx. 19.2) if freq=1, 1.5×9.6 (approx. 14.4) if freq=2, 4/3×9.6 (approx. 12.8) if freq=3.
0205The offset is set to the same value as the frequency (freq) so that redundancy is able to be used. In some variants, another convention may be defined to set the offset value.
0206The details of CMR construction or extraction are not described here according to table 2. However, it is assumed that the structure “CMR_Req” contains a field called “burst_uplink” that is set to “1”, “2” or “3” when extended CMR codes according to table 2 are used, otherwise its value is set to “−1”. Thus, by combining the “requested_bitrate” entry and this “burst_uplink” entry, it is possible to completely signal the type of request to be sent or request received.
0207One important feature of this second embodiment over the conventional use of 100% redundancy is that the redundancy is preferably intermittent: rather than (approximately) doubling the encoding bitrate, the redundancy is used adaptively for example every 1, 2 or 3 packets, thereby making it possible to obtain on average a fractional bitrate increase, in order to test (on average) the change to the immediately higher discrete bitrate and not double the bitrate. More generally, the frequency may be chosen on the basis of the current discrete bitrate D<b>0</b> and the immediately higher discrete bitrate D<b>1</b> for a given codec, such as the integer value closest to 1/(D<b>1</b>/D<b>0</b>−1).
0208If the instantaneous peak bitrate may not exceed the maximum bitrate authorized in the session (for example 24.4 kbit/s), only requests associated with 9.6 kbit/s with different frequency values will be used (freq=1, 2 or 3), in the knowledge that it will be difficult in this case to test a bitrate close to 24.4 kbit/s because the frequency freq=1 gives a bitrate around 19.2 kbit/s. If not, it is also possible to authorize requests associated with the bitrate of 13.2 kbit/s, thereby making it possible to test bitrates of approximately (freq+1)/freq×13.2, that is to say 2×13.2 (approx. 26.4) if freq=1, 1.5×13.2 (approx. 19.8) if freq=2, 4/3×13.2 (approx. 17.6) if freq=3.
0209Starting from the estimated bandwidth—instantaneously or over a past horizon given by the history—and potentially using additional information such as the packet loss rate measured at reception (where the estimated bandwidth will be multiplied by a factor <1 such as 0.9 as soon as the rate is for example >10%), a bitrate adaptation request is determined and encoded in a CMR code (blocks <b>302</b> and <b>352</b> of <figref idref="DRAWINGS">FIG. 3</figref>). This CMR request is added by the local sender to the next packet to be transmitted.
0210According to the invention, three types of adaptation request decision to be sent for the bitrate adaptation are defined: <ul id="ul0046" list-style="none"><li id="ul0046-0001" num="0000"><ul id="ul0047" list-style="none"><li id="ul0047-0001" num="0211">“SET_RATE” decision: in this case, a CMR is sent that indicates an encoding bitrate that corresponds to the estimated bandwidth if the current bitrate is not within an interval around this estimated bandwidth;</li><li id="ul0047-0002" num="0212">“NO_REQ” decision: the current bitrate is kept if the bitrate to be used remains unchanged</li><li id="ul0047-0003" num="0213">in this case, a CMR indicating the current bitrate may be sent;</li><li id="ul0047-0004" num="0214">“USE_RED” decision: a CMR is sent indicating that 100% redundancy should be activated at sending—in one preferred embodiment, redundancy is used adaptively—with an adaptive offset linked to the encoding bitrate—to test the change to the bitrate immediately higher than the current bitrate (with a variable sending frequency); in some variants it may be contemplated to activate redundancy for continuous use—with freq=1 and offset=1—until quality problems are detected. This allows a rapid rise in the bitrate, but preference will generally be given to a gradual rise in increments in order to test the higher bitrates one by one.</li></ul></li></ul>
0215For AMR and AMR-WB codecs which work with in-band feedback through CMR, the meaning of CMR NO_REQ is different from that of EVS, NO_REQ indicates the authorized maximum bitrate. In some variants using these codecs, the NO_REQ decision will therefore be replaced with a SET_RATE decision to indicate the current bitrate. In some variants of the invention using EVS, it is possible in the same way to replace the NO_REQ decision with a SET_RATE decision to indicate the current bitrate.
0216According to this second embodiment, reserved CMR codes are used, which justifies the name extended CMR (Ext. CMR) in blocks <b>307</b> and <b>357</b> in <figref idref="DRAWINGS">FIG. 3</figref>. The extended CMR codes are used to indicate an adaptation request to increase the encoding bitrate, this request tells the remote sender to test a bitrate higher than this bitrate given in the request—in this case the sender will activate transmission at this given bitrate with 100% redundancy by transmitting redundant packets.
0217This request assumes the use of redundancy on the sender side. In one simplified embodiment, the sender may execute the request from the CMR as in the first embodiment of the invention, by checking that the “updated” input is at “true” in the structure “CMR_Req” and by applying the adaptation parameters contained in “CMR_Req”; after retrieving the parameters, the value of “updated” is reset to “false”. In one preferred embodiment, the sender will be allowed sufficient flexibility to execute the redundancy activation request on the basis of the signal currently being encoded. As explained below, the bitrate increase may result in congestion, with packet loss or an increase in jitter, and it is therefore important, when the available bandwidth is not known and tested blind (to estimate the “bottleneck” in the downlink direction), not to exceed this limit excessively on the channel; moreover, the impact of losses and jitter is different depending on the types of encoded frames; for the inactive parts or the less sensitive parts of the active speech, these degradations may be tolerated more easily.
0218For this second embodiment, <figref idref="DRAWINGS">FIG. 5<i>a </i></figref>picks up, from the first embodiment, from the extraction steps performed at the receiver. Thus, upon reception of a packet at <b>501</b>, two items of information are extracted at the receiver: <ul id="ul0048" list-style="none"><li id="ul0048-0001" num="0000"><ul id="ul0049" list-style="none"><li id="ul0049-0001" num="0219">transmission information for the current packet, allowing estimation of the available bandwidth (at <b>502</b>) according to one of the estimation methods described above. The extraction of this information is followed by the steps X described with reference to <figref idref="DRAWINGS">FIG. 5</figref><i>b; </i></li><li id="ul0049-0002" num="0220">information on the type of adaptation requested from the CMR request sent by the other end of the communication (at <b>503</b>). <br /> The main difference with <figref idref="DRAWINGS">FIG. 4<i>a </i></figref>relates to the fact that the CMR request may correspond to an extended CMR, whereas in <figref idref="DRAWINGS">FIG. 4<i>a </i></figref>a “conventional” CMR request is assumed; moreover, steps X also include the possibility of deciding to send an extended request to activate redundancy using the “burst_uplink” entry. </li></ul></li></ul>
0221<figref idref="DRAWINGS">FIG. 5<i>b </i></figref>describes the steps X implemented by the receiver device after extracting the information on the current packet (<b>502</b>).
0222In the same way as for the first embodiment, the bandwidth is estimated according to one of the methods described above (at <b>511</b>). Next, mapping between the estimated bandwidth and the discrete EVS bitrates is performed (at <b>512</b>) as described above in order to determine whether it is necessary to change the current reception bitrate and whether it is possible to change from one discrete bitrate to another.
0223If the bitrate determined according to the bandwidth estimated in the last packet received is different from the current bitrate (Y at <b>513</b>), the “SET_RATE” decision is taken to change the bitrate according to this new bitrate. A conventional request to change the bitrate to a corresponding discrete bitrate is defined in step <b>514</b>; the associated data are written to the shared structure “BW_info”, so that the sender is then able to encode the mapping request with the frame to be sent.
0224If the new bitrate is the same as the current bitrate, then a step of checking test application conditions is performed at <b>516</b>.
0225This checking step <b>516</b> checks the bandwidth estimate history and thus measures a variation trend in the estimated bandwidth. If this trend is positive (Y at <b>516</b>), that is to say if the estimated bandwidth tends to increase, then a “USE_RED” decision may be taken and step <b>517</b> is then implemented.
0226If not (N at <b>516</b>), if the trend is negative, then there is no sending of a request, the decision “NO_REQ” is taken and the data associated with the CMR are written to the shared structure “BW_info”.
0227To measure the variation trend, it is possible for example to use simple linear regression on the estimated bandwidth as a function of the arrival time of the packets over a period of 500 ms to calculate a slope, as described previously.
0228The time for which this “NO-REQ” decision remains selected may be measured in order to decide whether or not a bitrate increase test should be activated. If this elapsed time is less than a threshold (for example 500 ms), then a “NO_REQ” decision is taken (N at <b>516</b>). The “NO-REQ”mode was not long enough. In this case, no request (no CMR) or a CMR indicating NO_REQ will typically be sent at <b>519</b>—in the latter, the data associated with the CMR NO_REQ are written to the shared structure “BW_info”.
0229In some variants, it is possible to reuse the criteria for triggering (activating) the bitrate increase test of the first embodiment.
0230In some variants, the NO_REQ decision will be replaced with a SET_RATE decision at the current bitrate, in an equivalent manner. It will be noted that, for AMR and AMR-WB codecs, the CMR is systematically present in each packet, and the NO_REQ case may be replaced with the current bitrate.
0000Conversely (Y at <b>516</b>), when the elapsed time is greater than the threshold, the “USE_RED” decision is taken in order to request a bitrate increase test. In this case, in step <b>517</b>, an extended CMR request is prepared, by writing the data associated with the extended CMR to the shared structure “BW_info”.
0231<figref idref="DRAWINGS">FIG. 5<i>c </i></figref>describes the operation of the sender upon reception of a CMR request (normal or extended, other than NO_REQ). The steps that have remained identical to the sender with reference to <figref idref="DRAWINGS">FIG. 4<i>c </i></figref>are not repeated here for the sake of simplification.
0232Two possible cases arise in the step of checking whether a CMR field has been received. Thus, if the CMR field contains a conventional bitrate adaptation request or an extended CMR request, the encoding and sending parameters are set as in the first embodiment. However, the sender may be allowed to decide when to activate redundancy.
0233With reference to <figref idref="DRAWINGS">FIG. 5<i>c</i></figref>, in step <b>521</b>, it is checked whether a conventional bitrate change CMR has been received (“updated” at “true” and “burst_uplink”=−1) and whether a test step through transmission of redundancy packets is in progress (N<sub>burst</sub>≥0). If so (Y at <b>521</b>), the test step is stopped and redundancy is deactivated at <b>522</b>, as described above in the first embodiment.
0234If there is no test step in progress (N at <b>521</b>)—if N<sub>burst</sub><0—, then step <b>523</b> is implemented.
0235In step <b>523</b>, it is checked whether an extended CMR request has been received (“updated” at “true” and “burst_uplink>0” in the structure “CMR_Req”). If so (Y at <b>523</b>), the sender implements a bitrate increase test step by transmitting redundant packets at <b>524</b> on the basis of the information contained in the structure “CMR_Req”. The bitrate is also set in order to encode the next frame on the basis of the received CMR (at <b>525</b>).
0236In some variants, the above algorithm may be supplemented so as also to use the packet loss rate. This adds additional conditions in order in particular to lower the bitrate in the event of significant losses (>=10%) and/or switch to a more robust mode (such as channel-aware mode) if it is less than or equal to the current bitrate, or avoid the USE_RED decision in case of losses >=10%.
0237In some variants, it is possible to use partial redundancy, as explained above, instead of 100% redundancy, and it is also possible to use higher redundancy, not limited to 100%, for example 200 or 300% redundancy.
0238In some variants, it is also possible to use audio or video codecs that do not have the “in-band” CMR signaling mechanism as defined above, and in this case it is possible to use RTCP packets to indicate requests equivalent to the CMR used in the embodiments of the invention.
0239<figref idref="DRAWINGS">FIG. 6</figref> illustrates a hardware example of a communication terminal TA comprising a receiver device and a sender device able to implement the methods for bitrate adaptation and for determining an adaptation request according to the various embodiments of the invention.
0240The terminal TA comprises a storage space <b>11</b>, for example a memory MEM, a processing unit <b>10</b> comprising a processor P, driven by a computer program PG, stored in the memory <b>11</b> and implementing the steps of the bitrate adaptation method and/or of the method for determining a bitrate adaptation request within the meaning of the invention, and in particular the test step of increasing the encoding bitrate implemented by a sender device by transmitting at least one redundant packet according to selected transmission parameters when these instructions are executed by the processor P.
0241Typically, the description of <figref idref="DRAWINGS">FIGS. 4<i>a </i>to 4<i>c </i></figref>and of <figref idref="DRAWINGS">FIGS. 5<i>a </i>to 5<i>c </i></figref>picks up from the steps of the algorithms of such computer programs.
0242On initialization, the code instructions of the program PG are for example loaded into a RAM memory (not shown) before being executed by the processor P of the processing unit <b>10</b>. The program instructions may be stored on a storage medium such as flash memory, a hard disk or any other non-transient storage medium.
0243The terminal TA comprises a communication module <b>12</b> able to receive the voice packets from an IP network and to transmit voice packets to the IP network with redundancy in order to test a bitrate increase according to the invention.
0244The terminal comprises a sender device comprising a packet transmission unit able to implement a test step of increasing the encoding bitrate by transmitting at least one redundant packet according to selected transmission parameters.
0245The terminal also comprises a receiver device that, according to one embodiment, comprises an estimation module able to estimate an available bandwidth and a module for constructing and transmitting an adaptation request, able to construct and transmit, to a sender device, a request to implement a test step of increasing the encoding bitrate by transmitting redundant packets according to selected transmission parameters.
0246These modules are as described with reference to <figref idref="DRAWINGS">FIG. 3</figref>.
0247The term module may correspond equally to a software component or to a hardware component or to a set of software and hardware components, a software component itself corresponding to one or more computer programs or subroutines or, more generally, to any element of a program able to implement a function or a set of functions such as described for the modules in question. In the same way, a hardware component corresponds to any element of a hardware assembly able to implement a function or a set of functions for the module in question (integrated circuit, chip card, memory card, etc.).
0248The terminal is for example a telephone, a smartphone, a tablet, a computer, a home gateway or a connected object.
0249ANNEX 1: <ul id="ul0050" list-style="none"><li id="ul0050-0001" num="0000"><ul id="ul0051" list-style="none"><li id="ul0051-0001" num="0250">EVS_D (0 to 8)=(9600, 13200, 16400, 24400, 32000, 48000, 64000, 96000, 12800)</li></ul></li></ul>
0251For i=0 to 8: <ul id="ul0052" list-style="none"><li id="ul0052-0001" num="0000"><ul id="ul0053" list-style="none"><li id="ul0053-0001" num="0252">If EVS_D(i)>B−(EVS_D(i+1)−EVS_D(i))/2: <ul id="ul0054" list-style="none"><li id="ul0054-0001" num="0253">Exit the “For” loop</li></ul></li><li id="ul0053-0002" num="0254">End If</li><li id="ul0053-0003" num="0255">i=i+1</li></ul></li></ul>
0256End For
0257If i==9: D=EVS_D(i−1)
0258Else D=EVS_D(i)
0259Although the present disclosure has been described with reference to one or more examples, workers skilled in the art will recognize that changes may be made in form and detail without departing from the scope of the disclosure and/or the appended claims.
Contents6
13 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US2024007365A1 | Cited by | United States of America | Search report |
| US11936707B2 | Cited by | United States of America | Search report |
| US2021409475A1 | Cited by | United States of America | Search report |
| US2023362733A1 | Cited by | United States of America | Search report |
| KR20230141738A | Cited by | Republic of Korea | Search report |
| US11824737B2 | Cited by | United States of America | Search report |
| JP2025535551A | Cited by | Japan | Search report |
| US2025039099A1 | Cited by | United States of America | Search report |
| US2021075698A1 | Cited by | United States of America | Search report |
| US2025104723A1 | Cited by | United States of America | Search report |
| US12363010B2 | Cited by | United States of America | Search report |
| US2023318980A1 | Cited by | United States of America | Search report |
| CN117062143A | Cited by | China | Search report |
| US12004011B2 | Cited by | United States of America | Search report |
5 members in 4 offices
Priority claims3
| Document | Office | Kind | Date |
|---|---|---|---|
| 1855048 | France | – | |
| 1855048 | France | A | |
| 2019051301 | France | W |
Members5
| Document | Office | Kind | |
|---|---|---|---|
| WO2019234338A1 | World Intellectual Property Organization (WIPO) | A1 | |
| FR3082386A1 | France | A1 | |
| EP3804243A1 | European Patent Office (EPO) | A1 | |
| US2021258363A1 | United States of America | A1 | |
| US11349898B2 | United States of America | B2 |
62 transactions on the USPTO file
Allowed after 1 non-final rejection and 1 final rejection.
- Non-final rejections
- 1
- Final rejections
- 1
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Payment of Maintenance Fee, 4th Year, Large EntityM1551 | M1551 | |
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Patent Issue Date Used in PTA CalculationAllowedPTAC | PTAC | |
| Email NotificationEML_NTR | EML_NTR | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Email NotificationEML_NTR | EML_NTR | |
| Filing Receipt - CorrectedFLRCPT.C | FLRCPT.C | |
| Response to Reasons for AllowanceREAS | REAS | |
| Miscellaneous Incoming LetterLET. | LET. | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Reasons for AllowanceEX.R | EX.R | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| PILOT- Request for After Final Consideration ProgramRAFC | RAFC | |
| Response after Final ActionA.NE | A.NE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Final Rejection (PTOL - 326)Final rejectionMCTFR | MCTFR | |
| Final RejectionFinal rejectionCTFR | CTFR | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Non-Final ActionA... | A... | |
| Email NotificationEML_NTR | EML_NTR | |
| Application ready for PDX access by participating foreign officesCCRDY | CCRDY | |
| PG-Pub Issue NotificationPG-ISSUE | PG-ISSUE | |
| Email NotificationEML_NTR | EML_NTR | |
| Filing Receipt - CorrectedFLRCPT.C | FLRCPT.C | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Application Is Now CompleteCOMP | COMP | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Email NotificationEML_NTR | EML_NTR | |
| Email NotificationEML_NTR | EML_NTR | |
| Notice of DO/EO Acceptance MailedM903 | M903 | |
| Filing ReceiptFLRCPT.O | FLRCPT.O | |
| Sent to Classification ContractorPGPC | PGPC | |
| FITF set to YES - revise initial settingFTFS | FTFS | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Request for Foreign Priority (Priority Papers May Be Included)RQPR | RQPR | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| 371 Completion Date371COMP | 371COMP | |
| Patent Term Adjustment - Ready for ExaminationPTA.RFE | PTA.RFE | |
| PTO/SB/69-Authorize EPO Access to Search ResultsSREXR141 | SREXR141 | |
| Applicants have given acceptable permission for participating foreignAPPERMS | APPERMS | |
| Cleared by OIPE CSRL194 | L194 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Entity Status Set To Undiscounted (Initial Default Setting or Status Change)BIG. | BIG. | |
| Initial Exam Team nnIEXX | IEXX |
10 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Maintenance fee paymentMAFP | MAFP | |
| Information on status: patent grantGrantedPATENTED CASESTCF | STCF | |
| Information on status: patent application and granting procedure in generalPUBLICATIONS -- ISSUE FEE PAYMENT VERIFIEDSTPP | STPP | |
| Information on status: patent application and granting procedure in generalNOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONSSTPP | STPP | |
| Information on status: patent application and granting procedure in generalRESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINERSTPP | STPP | |
| Information on status: patent application and granting procedure in generalFINAL REJECTION MAILEDSTPP | STPP | |
| Information on status: patent application and granting procedure in generalRESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINERSTPP | STPP | |
| AssignmentAS | AS | |
| Information on status: patent application and granting procedure in generalNON FINAL ACTION MAILEDSTPP | STPP | |
| Fee payment procedureENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP |
Numbers
- Publication
- 20210258363
- Application
- 16972744
Titles
- English
- BITRATE ADAPTATION OF A VOICE-OVER-IP COMMUNICATION SESSION
Patent term adjustment
- Applicant delay
- −36 days
- Net adjustment
- 0 days
Classification
- CPC, 4
- H04L65/607
- H04L47/38
- H04L65/70
- H04L69/24
- IPC, 2
- H04L29 06
- H04L12 811