Error concealment in relation to decoding of encoded acoustic signals

THE BACKGROUND OF THE INVENTION AND PRIOR ART

The present invention relates generally to the concealment of errors in decoded acoustic signals caused by encoded data representing the acoustic signals being partially lost or damaged. More particularly the invention relates to a method of receiving data in the form of encoded information from a transmission medium and an error concealment unit according to the preambles of claims 1 and 38 respectively. The invention also relates to decoders for generating an acoustic signal from received data in the form of encoded information according to the preambles of claims 40 and 41 respectively, a computer program according to claim 36 and a computer readable medium according to claim 37.

There are many different applications for audio and speech codecs (codec = <u style="single">co</u>der and <u style="single">dec</u>oder). Encoding and decoding schemes are, for instance, used for bit-rate efficient transmission of acoustic signals in fixed and mobile communications systems and in videoconferencing systems. Speech codecs can also be utilised in secure telephony and for voice storage.

Particularly in mobile applications, the codecs occasionally operate under adverse channel conditions. One consequence of such non-optimal transmission conditions is that encoded bits representing the speech signal are corrupted or lost somewhere between the transmitter and the receiver. Most of the speech codecs of today's mobile communication systems and Internet applications operate block-wise, where GSM (Global System for Mobile communication), WCDMA (Wideband Code Division Multiple Access), TDMA (Time Division Multiple Access) and IS95 (International Standard -95) constitute a few examples. The block-wise operation means that an acoustic source signal is divided into speech codec frames of a particular duration, e.g. 20 ms. The information in a speech codec frame is thus encoded as a unit. However, usually the speech codec frames are further divided into sub-frames, e.g. having a duration of 5 ms. The sub-frames are then the coding units for particular parameters, such as the encoding of a synthesis filter excitation in the GSM FR-codec (FR = Full Rate), GSM EFR-codec (EFR = Enhanced Full Rate), GSM AMR-codec (AMR = Adaptive Multi Rate), ITU G.729-codec (ITU = International Telecommunication Union) and EVRC (Enhanced Variable Rate Codec).

Besides the excitation parameters, the above codecs also model acoustic signals by means of other parameters like, for instance, LPC-parameters (LPC = Linear Predictive Coding), LTP-lag (LTP = Long Term Prediction) and various gain parameters. Certain bits of these parameters represent information that is highly important with respect to the perceived sound quality of the decoded acoustic signal. If such bits are corrupted during the transmission the sound quality of the decoded acoustic signal will, at least temporarily, be perceived by a human listener as having a relatively low quality. It is therefore often advantageous to disregard the parameters for the corresponding speech codec frame if they arrive with errors and instead make use of previously received correct parameters. This error concealment technique is applied, in form or the other, in most systems through which acoustic signals are transmitted by means of non-ideal channels.

The error concealment method normally aims at alleviating the effects of a lost / damaged speech codec frame by freezing any speech codec parameters that vary comparatively slowly. Such error concealment is performed, for instance, by the error concealment unit in the GSM EFR-codec and GSM AMR-codec, which repeats the LPC-gain and the LPC-lag parameters in case of a lost or damaged speech codec frame. If, however, several consecutive speech codec frames are lost or damaged various muting techniques are applied, which may involve repetition of gain parameters with decaying factors and repetition of LPC-parameters moved towards their long-term averages. Furthermore, the power level of the first correctly received frame after reception of one or more damaged frames may be limited to the power level of the latest correctly received frame before reception of the damaged frame(s). This mitigates undesirable artefacts in the decoded speech signal, which may occur due to the speech synthesis filter and adaptive codebook being set in erroneous states during reception of the damaged frame(s).

Below is referred to a few examples of alternative means and aspects of ameliorating the adverse effects of speech codec frames being lost or damaged during transmission between a transmitter and a receiver.

The <patcit id="pcit0001" dnum="US5907822A"><text>U.S. patent 5,907,822</text></patcit> discloses a loss tolerant speech decoder, which utilises past signal-history data for insertion into missing data segments in order to conceal digital speech frame errors. A multi-layer feed-forward artificial neural network that is trained by back-propagation for one-step extrapolation of speech compression parameters extracts the necessary parameters in case of a lost frame and produces a replacement frame.

The European patent, <patcit id="pcit0002" dnum="EP0665161B1"><text>B1, 0 665 161</text></patcit> describes an apparatus and a method for concealing the effects of lost frames in a speech decoder. The document suggests the use of a voice activity detector to restrict updating of a threshold value for determining background sounds in case of a lost frame. A post filter normally tilts the spectrum of a decoded signal. However, in case of a lost frame the filtering coefficients of the post filter are not updated.

The <patcit id="pcit0003" dnum="US5909663A"><text>U.S. patent 5,909,663</text></patcit> describes a speech coder in which the perceived sound quality of a decoded speech signal is enhanced by avoiding a repeated use of the same parameter at reception of several consecutive damaged speech frames. Adding noise components to an excitation signal, substituting noise components for the excitation signal or reading an excitation signal at random from a noise codebook containing plural excitation signals accomplishes this.

The patent application <patcit id="pcit0004" dnum="WO0068934A"><text>WO-A-00/68934</text></patcit> describes an error concealment technique for an encoded audio signal.

The known error concealment solutions for narrow-band codecs generally provide a satisfying result in most environments by simply repeating certain spectral parameters from a latest received undamaged speech codec frame during the corrupted speech codec frame(s). In practice, this procedure implicitly retains the magnitude and the shape of the spectrum of the decoded speech signal until a new undamaged speech codec frame is received. By such preservation of the speech signal's spectral magnitude and the shape, it is also implicitly assumed that an excitation signal in the decoder is spectrally flat (or white).

However, this is not always the case. An Algebraic Code Excited Linear Predictive-codec (ACELP) may, for instance, produce non-white excitation signals. Furthermore, the spectral shape of the excitation signal may vary considerably from one speech codec frame to another. A mere repetition of spectral parameters from a latest received undamaged speech codec frame could thus result in abrupt changes in the spectrum of the decoded acoustic signal, which, of course, means that a low sound quality is experienced.

Particularly, wide-band speech codecs operating according to the CELP coding paradigm have proven to suffer from the above problems, because in these codecs the spectral shape of the synthesis filter excitation may vary even more dramatically from one speech codec frame to another.

SUMMARY OF THE INVENTION

The object of the present invention is therefore to provide a speech coding solution, which alleviates the problem above.

According to one aspect of the invention the object is achieved by a methods of receiving data in the form of encoded information and decoding the data into an acoustic signal as initially described, according to claim 1.

According to another aspect of the invention the object is achieved by a computer program directly loadable into the internal memory of a computer, comprising software for performing the method described in the above paragraph when said program is run on the computer.

According to a further aspect of the invention the object is achieved by a computer readable medium, having a program recorded thereon, where the program is to make the computer perform the method described in the penultimate paragraph above.

According to still a further aspect of the invention the object is achieved by an error concealment unit as initially described, according to claim 38<i>.</i>

According to yet another aspect of the invention the object is achieved by a decoder for generating an acoustic signal from received data in the form of encoded information. The decoder includes a primary error concealment unit to produce at least one parameter. It also includes a speech decoder to receive speech codec frames, the at least one parameter from the primary error concealment and to provide in response thereto an acoustic signal. Furthermore, the decoder includes the proposed error concealment unit wherein the primary reconstructed signal constitutes the decoded speech signal produced by the speech decoder and the secondary reconstructed signal constitutes an enhanced acoustic signal.

According to still another aspect of the invention the object is achieved by a decoder for generating an acoustic signal from received data in the form of encoded information. The decoder includes a primary error concealment unit to produce at least one parameter. It also includes an excitation generator to receive speech codec parameters and the at least one parameter and to produce an excitation signal in response to the at least one parameter from the primary error concealment unit. Finally, the decoder includes the proposed error concealment unit wherein the primary reconstructed signal constitutes the excitation signal produced by the excitation generator and the secondary reconstructed signal constitutes an enhanced excitation signal.

The proposed explicit generation of a reconstructed spectrum as a result of lost or received damaged data ensures spectrally smooth transitions between periods of received undamaged data and periods of received damaged data. This, in turn, provides an enhanced perceived sound quality of the decoded signal, particularly for advanced broadband codecs, for instance, involving ACELP-coding schemes.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is now to be explained more closely by means of preferred embodiments, which are disclosed as examples, and with reference to the attached drawings. <dl id="dl0001"><dt>Figure 1</dt><dd>shows a general block diagram over an error concealment unit according to the invention,</dd><dt>Figure 2</dt><dd>shows a diagram over consecutive signal frames containing encoded information representing an acoustic signal,</dd><dt>Figure 3</dt><dd>shows a decoded acoustic signal based on the encoded information in the signal frames in <figref idref="f0002">figure 2</figref>,</dd><dt>Figure 4</dt><dd>shows a set of spectra for segments of the decoded acoustic signal in <figref idref="f0002">figure 3</figref> corresponding to the signal frames in <figref idref="f0002">figure 2</figref>,</dd><dt>Figure 5</dt><dd>shows a diagram including a spectrum generated on basis of previous undamaged data, a primary reconstruction of the damaged data respective a secondary reconstruction of the damaged data according to the invention,</dd><dt>Figure 6</dt><dd>shows a block diagram over a first embodiment of an error concealment unit according to the invention,</dd><dt>Figure 7</dt><dd>shows a block diagram over a second embodiment of an error concealment unit according to the invention, and</dd><dt>Figure 8</dt><dd>illustrates in a flow diagram the general method according to the invention.</dd></dl>

DESCRIPTION OF PREFERRED EMBODIMENTS OF THE INVENTION

<figref idref="f0001">Figure 1</figref> shows a block diagram over an error concealment unit 100 according to the invention. The object of the error concealment unit 100 is to produce an enhanced signal Z_n^E decoded from received data in case the received data is damaged or lost. The enhanced decoded signal Z_n^E either represents a parameter of a speech signal, such as an excitation parameter, or the enhanced decoded signal z_n^E itself is an acoustic signal. The unit 100 includes a first transformer 101, which receives a primary reconstructed signal y_n being derived from the received data. The primary reconstructed signal y_n is regarded as a signal in the time domain and the first transformer 101 regularly produces a primary reconstructed frequency transform Y_n of a latest received time segment of the primary reconstructed signal y_n in the form of a first spectrum. Typically, each segment corresponds to a signal frame of the received signal.

The first spectrum Y_n is forwarded to a spectral correction unit 102, which produces a secondary reconstructed spectrum Z_n^E on basis of the first spectrum Y_n. The secondary reconstructed spectrum Z_n^E is produced such that it deviates less with respect to spectral shape from a spectrum of a previously reconstructed signal than a spectrum based on the primary reconstructed signal y_n.

In order to illustrate this, reference is made to <figref idref="f0002">figure 2</figref>, where consecutive signal frames F(1)-F(5) containing encoded information, which represents an acoustic signal are shown in a diagram. The signal frames F(1)-F(5) are produced by a transmitter at regular intervals t₁, t₂, t₃, t₄ respective t₅.

Nevertheless, it is not necessary that the signal frames F(1)-F(5) arrive with the same regularity to the receiver or even in the same order as long as they arrive within a sufficiently small delay so, as the receiver can re-arrange the signal frames F(1)-F(5) in the correct order before decoding. However, for reasons of simplicity, the signal frames F(1)-F(5) are in this example assumed arrive in a timely manner and in the same order as they were generated by the transmitter. The initial three signal frames F(1)-F(3) arrive undamaged, i.e. without any errors in the included information. The fourth signal frame F(4), however, is damaged, or possibly lost completely before reaching a decoding unit. The subsequent signal frame F(5) again arrives undamaged.

<figref idref="f0002">Figure 3</figref> shows a decoded acoustic signal z(t) being based on the signal frames F(1)-F(5) in <figref idref="f0002">figure 2</figref>. An acoustic signal z(t) in the time domain t is generated on basis of information contained in the first signal frame F(1) between a first time instance t₁ and a second time instance t₂. Correspondingly, the acoustic signal z(t) is generated up to a fourth time instant t₄ based the information in the second F(2) and third F(3) signal frames. In a real case, there would also be shift between the intervals t₁-t₅ on the transmitter side and the corresponding time instances t₁-t₅ on the receiver side due to i.a. encoding delay, transmission time and decoding delay. Again, for simplicity, this fact has been ignored here.

Nevertheless, at the fourth time instant t₄ there exists no (or possibly only unreliable) received information to base the acoustic signal z(t) upon. Therefore, the acoustic signal z'(t₄)-z'(t₅) is based on a reconstructed signal frame F_rec(4) produced by a primary error concealment unit between the fourth time instant t₄ and a fifth time instant t₅. As illustrated in the <figref idref="f0002">figure 3</figref> the acoustic signal z(t) derived from the reconstructed signal frame F_rec(4) exhibits different waveform characteristics than the parts of the acoustic signal z(t) derived from the adjacent signal frames F(3) and F(5).

<figref idref="f0002">Figure 4</figref> shows a set of spectra Z₁, Z₂, Z₃, Z'₄ and Z₅, which correspond to the respective segments z(t₁)-z(t₂), z(t₂)-z(t₃), z(t₃)-z(t₄) and z'(t₄)-z'(t₅) of the decoded acoustic signal z(t) in <figref idref="f0002">figure 3</figref>. The decoded acoustic signal z(t) is comparatively flat in the time domain t between the third time instance t₃ and the fourth time instance t₄ and therefore has a relatively strong low frequency content, which is represented by a corresponding spectrum Z₃ having the majority of its energy located in the low-frequency region. In contrast to this, the spectrum of the acoustic signal z'(t₄)-z'(t₅) based on the reconstructed signal frame F_rec(4) contains considerably more energy in the highfrequency band and the signal z'(t₄)-z'(t₅) in the time domain t shows relatively fast amplitude variations. The contrasting spectral shapes of the spectrum Z₃ of the decoded acoustic signal based on the latest received undamaged signal frame F(3) and the spectrum Z'₄ of the decoded acoustic signal based on the reconstructed signal frame F_rec(4) leads to undesired artefacts in the acoustic signal and a human listener perceives a low sound quality.

<figref idref="f0002">Figure 5</figref> shows a diagram in which an enlarged version of the spectrum Z₃ of the decoded acoustic signal based on the latest received undamaged signal frame F(3) and the spectrum Z'₄ of the decoded acoustic signal based on the reconstructed signal frame F_rec(4) are outlined as respective solid lines. A secondary reconstructed spectrum Z_n^E generated by the spectral correction unit 102 is shown in the diagram by means of a dashed line. The spectral shape of the latter spectrum Z_n^E deviates less from the spectrum Z₃ of the decoded acoustic signal based on the latest received undamaged signal frame F(3) than the spectrum Z'₄ of the decoded acoustic signal based on the reconstructed signal frame F_rec(4). For instance, the spectrum Z_n^E is more shifted towards the low-frequency region.

Returning to <figref idref="f0001">figure 1</figref>, a second transformer 103 receives the secondary reconstructed spectrum Zn ^E, performs an inverse frequency transform and provides a corresponding secondary reconstructed signal Z_n^E in the time domain constituting the enhanced decoded signal. <figref idref="f0002">Figure 3</figref> shows this signal z^E(t₄)-z^E(t₅) as a dashed line, involving a waveform characteristics, which is more similar to the acoustic signal z(t₃)-z(t₄) decoded from the latest received undamaged signal frame F(3) than the acoustic signal z'(t₄)-z'(t₅) based on the reconstructed signal frame F_rec (4).

The secondary reconstructed spectrum Z_n^E is produced by multiplying the phase of the first spectrum Y_n, i.e. Y_n/|Y_n| (where Y_n denotes the first spectrum and |Y_n-| denotes the magnitude of the first spectrum), corresponding to the reconstructed signal frame F_rec(4) with a correction spectrum C_n. In practice, this can be performed according to the expression: Z_n^E = C_n • Y_n/|Y_n|.

According to a preferred embodiment of the invention, the correction spectrum C_n is generated from previously received undamaged data F(n-1) according to the following. The spectral correction unit 102 first generates a previous spectrum Y_n-1 of a signal produced from the previously received undamaged data F(n-1), corresponding to Z₃ in <figref idref="f0002">figures 4 and 5</figref> respective F(3) in <figref idref="f0002">figure 3</figref>. Then, the spectral correction unit 102 produces a magnitude spectrum |Y_n-1| of the previous spectrum Y _n-1.

According to another preferred embodiment of the invention the correction spectrum C_n is generated by producing a previous spectrum Y_n-1 of a signal produced from the previously received undamaged data F(n-1). The resulting spectrum is then filtered into a filtered previous spectrum H(Y_n-1). Finally, a magnitude spectrum |H(Y_n-1)| of the filtered previous spectrum H(Y_n-1) is produced.

The filtering may involve many alternative modifications of the previous spectrum Y_n-1. The overall purpose of the filtering is, however, always to create a signal with corresponding spectrum, which is a smoothed repetition of the spectrum of the signal decoded from the previous undamaged signal frame. Low-pass filtering therefore constitutes one reasonable alternative. Another alternative would be smoothing in the cepstral domain. This could involve transforming the previous (possibly logarithmic) magnitude spectrum | Y_n-1 | into the cepstral domain, discarding of cepstral coefficients of a particular order, (say 5-7) and above, and back transforming into the frequency domain. Another non-linear filtering alternative is to divide the previous spectrum Y_n-1 into at least two frequency sub-bands f₁-f_M and calculate an average coefficient value of the original spectral coefficients, within the respective frequency sub-band f₁-f_M. Finally, the original spectral coefficients are replaced by the respective average coefficient value. As a result, the overall frequency band is smoothed. The frequency sub-bands f₁-f_M may either be equidistant, i.e. divide the previous spectrum Y_n-1 into segments of equal size, or be non-equidistant (e.g. according to the Bark or Mel scale band division). A non-equidistant logarithmic division of the spectrum Y_n-1 is preferable, since also the human hearing is approximately logarithmic with respect to frequency resolution and loudness perception.

Furthermore, the frequency sub-bands may partly overlap each other. Resulting coefficient values in overlapping regions are in this case derived by first, multiplying each frequency sub-band with a window function and second, adding coefficient values of neighbouring windowed frequency sub-bands in each region of overlap. The window function shall have a constant magnitude in non-overlapping frequency regions and a gradually declining magnitude in an upper and a lower transition region where neighbouring frequency sub-bands overlap.

According to another preferred embodiment of the invention, the spectrum of the secondary reconstructed signal Z_n^E is produced by reducing the dynamic range of the correction spectrum C_n relative a so-called target muting spectrum |Y₀|. The target muting spectrum |Y₀| may, for instance, represent a long term average value of the acoustic source signal.

A dynamic reduction of the range of the correction spectrum C_n in relation to the target muting spectrum |Y₀| can be performed according to the relationship: <maths id="math0001" num=""><math display="block"><msub><mi mathvariant="normal">C</mi><mi mathvariant="normal">n</mi></msub><mo mathvariant="normal">=</mo><mrow><mo mathvariant="normal">(</mo><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup></mrow><mo mathvariant="normal">+</mo><mi>comp</mi><mo>⁢</mo><mfenced separators=""><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup><mo mathvariant="normal">-</mo><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup></mfenced><mo>⁢</mo><msup><mrow><mo mathvariant="normal">)</mo></mrow><mmultiscripts><msub><mo mathvariant="normal">/</mo><mi mathvariant="normal">k</mi></msub>&lt;mprescripts /&gt;&lt;none /&gt;<mn mathvariant="normal">1</mn></mmultiscripts></msup></mrow></math><img file="EP1327242B1_D0001.tif" /></maths> where Y_n-1 denotes the spectrum of the previously reconstructed signal frame (N.B. this frame need not necessarily be an undamaged signal frame, but may in turn be an earlier reconstructed damaged or lost signal frame), |Y₀| denotes the target muting spectrum, k denotes an exponent, e.g. 2, and comp(x) denotes a compression function. The compression function is characterised by having a smaller absolute value than the absolute value of the input variable, i.e. | comp(x) | &lt; |x|. Thus, a decaying factor η &lt; 1 constitutes a simple example of a compression function comp(x) = η·x.

The decaying factor η is preferably given by a state machine, which, as in the GSM AMR-standard, may have seven different states. The decaying factor η can thus be described as a function of a state variable s, η (s), having the following values: <tables id="tabl0001" num="0001"><table frame="none"><tgroup cols="8"><colspec colnum="1" colname="col1" colwidth="16mm" /><colspec colnum="2" colname="col2" colwidth="10mm" /><colspec colnum="3" colname="col3" colwidth="12mm" /><colspec colnum="4" colname="col4" colwidth="12mm" /><colspec colnum="5" colname="col5" colwidth="12mm" /><colspec colnum="6" colname="col6" colwidth="12mm" /><colspec colnum="7" colname="col7" colwidth="12mm" /><colspec colnum="8" colname="col8" colwidth="10mm" colsep="0" /><thead><row><entry align="right" valign="top">state (s)</entry><entry align="center" valign="top">0</entry><entry align="center" valign="top">1</entry><entry align="center" valign="top">2</entry><entry align="center" valign="top">3</entry><entry align="center" valign="top">4</entry><entry align="center" valign="top">5</entry><entry align="center" valign="top">6</entry></row></thead><tbody><row rowsep="0"><entry align="right">η (s)</entry><entry align="center">1</entry><entry align="center">0,98</entry><entry align="center">0,98</entry><entry align="center">0,98</entry><entry align="center">0,98</entry><entry align="center">0,98</entry><entry align="center">0,7</entry></row></tbody></tgroup></table></tables>

The state variable is set to 0 at reception of an undamaged piece of data. In case of reception of a first piece of damaged data, it is set to 1. If subsequent pieces of damaged data are received after reception of the first piece of damaged data the state variable s is incremented one state for each piece of received damaged data up to a state 6. In the state 6 and at reception of yet another piece of damaged data the state variable remains in state 6. If a piece of an undamaged data is received in the state 6 the state variable is set to state 5, and if in this sate 5 a subsequent piece of undamaged data is received the state variable is reset to 0.

According to another preferred embodiment of the invention, the spectrum of the secondary reconstructed signal Z_n^E is instead produced by reducing the dynamic range of the correction spectrum C_n in relation to a normalised target muting spectrum. This can be effectuated by a calculation of the expression: <maths id="math0002" num=""><math display="block"><msub><mi mathvariant="normal">C</mi><mi mathvariant="normal">n</mi></msub><mo mathvariant="normal">=</mo><mo mathvariant="normal">‖</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo mathvariant="normal">‖</mo><mo mathvariant="normal">⋅</mo><msub><msup><mi mathvariant="normal">C</mi><mi mathvariant="normal">s</mi></msup><mi mathvariant="normal">n</mi></msub><mo mathvariant="normal">/</mo><mo mathvariant="normal">‖</mo><msub><msup><mi mathvariant="normal">C</mi><mi mathvariant="normal">s</mi></msup><mi mathvariant="normal">n</mi></msub><mo mathvariant="normal">‖</mo></math><img file="EP1327242B1_D0002.tif" /></maths> where ∥Y_n-1∥ denotes an L_k-norm of the spectrum of the previously reconstructed signal frame. The L_k-norm ∥Y_n-1∥ of a vector Y_n-1 = {y₁, y₂, ... , y_m} is given by the expression: <maths id="math0003" num=""><math display="block"><mo mathvariant="normal">‖</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo mathvariant="normal">‖</mo><mo mathvariant="normal">=</mo><mrow><mo mathvariant="normal">(</mo><mfrac><mn mathvariant="normal">1</mn><mi mathvariant="normal">m</mi></mfrac><mstyle displaystyle="true"><munderover><mo mathvariant="normal">∑</mo><mrow><mi mathvariant="normal">i</mi><mo mathvariant="normal">=</mo><mn mathvariant="normal">1</mn></mrow><mi mathvariant="normal">m</mi></munderover></mstyle><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">y</mi><mi mathvariant="normal">l</mi></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup></mrow><msup><mrow><mo>)</mo></mrow><mmultiscripts><msub><mo mathvariant="normal">/</mo><mi mathvariant="normal">k</mi></msub>&lt;mprescripts /&gt;&lt;none /&gt;<mn mathvariant="normal">1</mn></mmultiscripts></msup></mrow></math><img file="EP1327242B1_D0003.tif" /></maths> where k is an exponent and y_i is the i:th spectral coefficient of Y_n-1. Furthermore, C^s_n is derived according to the relationship: <maths id="math0004" num=""><math display="block"><msub><msup><mi mathvariant="normal">C</mi><mi mathvariant="normal">s</mi></msup><mi mathvariant="normal">n</mi></msub><mo>=</mo><mo mathvariant="normal">(</mo><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup></mrow><mo>/</mo><msup><mrow><mo>‖</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>‖</mo></mrow><mi mathvariant="normal">k</mi></msup><mo mathvariant="normal">+</mo><mi>comp</mi><mo>⁢</mo><mfenced separators=""><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup><mo>/</mo><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo>-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup><mo>-</mo><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mi mathvariant="normal">k</mi></msup></mrow><mo>/</mo><msup><mrow><mo>‖</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>‖</mo></mrow><mi mathvariant="normal">k</mi></msup></mfenced><mo>⁢</mo><msup><mrow><mo mathvariant="normal">)</mo></mrow><mmultiscripts><msub><mo mathvariant="normal">/</mo><mi mathvariant="normal">k</mi></msub>&lt;mprescripts /&gt;&lt;none /&gt;<mn mathvariant="normal">1</mn></mmultiscripts></msup></math><img file="EP1327242B1_D0004.tif" /></maths> where |Y₀| denotes the target muting spectrum, ∥Y₀∥^k denotes the power of the target muting spectrum according to the L_k-norm used, k is an exponent, e.g. 2, and comp(x) denotes a compression function.

According to a preferred embodiment of the invention the correction spectrum C_n is generated by compressing the magnitude of the spectrum of the previously reconstructed signal frame with respect to a target power ∥Y₀∥^k according to a linear norm L_k, where the exponent k, for instance, equals 2.

In the general case, this compression is achieved by calculating the expression: <maths id="math0005" num=""><math display="block"><msub><mi mathvariant="normal">C</mi><mi mathvariant="normal">n</mi></msub><mo mathvariant="normal">=</mo><mfenced open="|" close="|"><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub></mfenced><mo mathvariant="normal">/</mo><mo mathvariant="normal">‖</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo mathvariant="normal">‖</mo><mo mathvariant="normal">⋅</mo><mrow><mo mathvariant="normal">(</mo><mo mathvariant="normal">‖</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>⁢</mo><msup><mo mathvariant="normal">‖</mo><mi mathvariant="normal">k</mi></msup><mo mathvariant="normal">+</mo><mi>comp</mi><mo>⁢</mo><mfenced separators=""><mo mathvariant="normal">‖</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub><mo>⁢</mo><msup><mo mathvariant="normal">‖</mo><mi mathvariant="normal">k</mi></msup><mo mathvariant="normal">-</mo><mo mathvariant="normal">‖</mo><msub><mi mathvariant="normal">Y</mi><mn mathvariant="normal">0</mn></msub><mo>⁢</mo><msup><mo mathvariant="normal">‖</mo><mi mathvariant="normal">k</mi></msup></mfenced><mo>⁢</mo><msup><mrow><mo mathvariant="normal">)</mo></mrow><mmultiscripts><msub><mo mathvariant="normal">/</mo><mi mathvariant="normal">k</mi></msub>&lt;mprescripts /&gt;&lt;none /&gt;<mn mathvariant="normal">1</mn></mmultiscripts></msup></mrow></math><img file="EP1327242B1_D0005.tif" /></maths> where | Y_n-1 | denotes the magnitude of the spectrum of the previously reconstructed signal frame, ∥Y₀∥^k denotes the target muting power according to an L_k-norm, where k is an exponent, e.g. 2, and comp(x) denotes a compression function.

According to a preferred embodiment of the invention the correction spectrum C_n is described by the relationship: <maths id="math0006" num=""><math display="block"><msub><mi mathvariant="normal">C</mi><mi mathvariant="normal">n</mi></msub><mo>=</mo><mi mathvariant="normal">η</mi><mo>⋅</mo><mfenced open="|" close="|"><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo>-</mo><mn>1</mn></mrow></msub></mfenced><mn>.</mn></math><img file="EP1327242B1_D0006.tif" /></maths> where η denotes a decaying factor &lt; 1, and | Y_n-1 | denotes the magnitude of the spectrum of the previously reconstructed signal frame.

Also in this case the decaying factor η is preferably given by a state machine having seven different states, 0 - 6. Furthermore, the same values of η (s) and rules of the state machine as above may be applied.

According to a preferred embodiment of the invention the correction spectrum C_n is generated by first producing the spectrum Y _n-1 of the previously reconstructed signal frame. Then, producing the corresponding magnitude spectrum |Y_n-1|, and finally multiplying a part m (i.e. an m:th sub-band) of the magnitude spectrum |Y_n-1| with an adaptive muting factor γ_m. One simple example is to use only one band (i.e. m = 1) containing the complete spectrum.

The adaptive muting factor γ_m may in turn be derived from the previously reconstructed signal frame and the received damaged data F(n) according to the expression: <maths id="math0007" num=""><math display="block"><msub><mi mathvariant="normal">γ</mi><mi mathvariant="normal">m</mi></msub><mo>=</mo><mfrac><msqrt><mstyle displaystyle="true"><munderover><mo mathvariant="normal">∑</mo><mrow><mi mathvariant="normal">k</mi><mo mathvariant="normal">=</mo><mi>low</mi><mfenced><mi mathvariant="normal">m</mi></mfenced></mrow><mrow><mi>high</mi><mfenced><mi mathvariant="normal">m</mi></mfenced></mrow></munderover></mstyle><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mi mathvariant="normal">n</mi></msub><mfenced><mi mathvariant="normal">k</mi></mfenced><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mn mathvariant="normal">2</mn></msup></mrow></msqrt><msqrt><mstyle displaystyle="true"><munderover><mo mathvariant="normal">∑</mo><mrow><mi mathvariant="normal">k</mi><mo mathvariant="normal">=</mo><mi>low</mi><mfenced><mi mathvariant="normal">m</mi></mfenced></mrow><mrow><mi>high</mi><mfenced><mi mathvariant="normal">m</mi></mfenced></mrow></munderover></mstyle><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo>-</mo><mn>1</mn></mrow></msub><mfenced><mi mathvariant="normal">k</mi></mfenced><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mn mathvariant="normal">2</mn></msup></mrow></msqrt></mfrac></math><img file="EP1327242B1_D0007.tif" /></maths> where "low(m)" denotes a frequency coefficient index corresponding to a lower frequency band boundary of a sub-band f_m of a spectrum of the signal having been decoded from reconstructed data, "high(m)" denotes a frequency coefficient index corresponding to an upper frequency band boundary of a sub-band f_m of a spectrum of the signal having been decoded from reconstructed data, | Y_n (k) | denotes the magnitude of a coefficient representing a k:th frequency component in the first spectrum, and | Y_n-1 (k) | denotes the magnitude of a coefficient representing a k:th frequency component in the previous spectrum.

Moreover, it is not necessary to sub-divide the spectrum. Thus, the spectrum may only comprise one sub-band f_m, having coefficient indices corresponding to the boundaries of the entire frequency band of the signal decoded from reconstructed data. If, however, a sub-band division is made, it should preferably accord with the Bark scale band division or the Mel scale band division.

According to a preferred embodiment of the invention, the correction spectrum C_n exclusively influences frequency components above a threshold frequency. For reasons of implementation, this threshold frequency is chosen such as it corresponds to a particular threshold coefficient. The correction spectrum C_n can hence be described by the expressions: <maths id="math0008" num=""><math display="block"><msub><mi mathvariant="normal">C</mi><mi mathvariant="normal">n</mi></msub><mfenced><mi mathvariant="normal">k</mi></mfenced><mo mathvariant="normal">=</mo><mfenced open="|" close="|" separators=""><msub><mi mathvariant="normal">Y</mi><mi mathvariant="normal">n</mi></msub><mfenced><mi mathvariant="normal">k</mi></mfenced></mfenced></math><img file="EP1327242B1_D0008.tif" /></maths> for k ≤ the threshold coefficient <maths id="math0009" num=""><math display="block"><msub><mi mathvariant="normal">C</mi><mi mathvariant="normal">n</mi></msub><mfenced><mi mathvariant="normal">k</mi></mfenced><mo mathvariant="normal">=</mo><mi mathvariant="normal">γ</mi><mo>⋅</mo><mfenced open="|" close="|" separators=""><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo>-</mo><mn>1</mn></mrow></msub><mfenced><mi mathvariant="normal">k</mi></mfenced></mfenced></math><img file="EP1327242B1_D0009.tif" /></maths> for k &gt; the threshold coefficient where C_n(k) denotes the magnitude of a coefficient k representing a k:th frequency component in the correction spectrum C_n, | Y_n (k) | denotes the magnitude of a coefficient k representing a k:th frequency component in the first spectrum, | Y_n-1 (k) | denotes the magnitude of a coefficient representing a k:th frequency component in the precious spectrum and γ denotes an adaptive muting factor &lt; 1.

The adaptive muting factor γ may, for instance, be chosen as the square-root of the ratio between the power |Y_n|² of the first spectrum Y_n and the power | Y_n-1 |² of the previous spectrum Y_n-1, i.e.: <maths id="math0010" num=""><math display="block"><mi mathvariant="normal">γ</mi><mo mathvariant="normal">=</mo><mfrac><msqrt><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mi mathvariant="normal">n</mi></msub><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mn mathvariant="normal">2</mn></msup></mrow></msqrt><msqrt><msup><mfenced open="|" close="|"><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo mathvariant="normal">-</mo><mn mathvariant="normal">1</mn></mrow></msub></mfenced><mn mathvariant="normal">2</mn></msup></msqrt></mfrac></math><img file="EP1327242B1_D0010.tif" /></maths>

The adaptive muting factor γ, may also be derived for a particular frequency band according to the expression: <maths id="math0011" num=""><math display="block"><mi mathvariant="normal">γ</mi><mo>=</mo><mfrac><msqrt><mstyle displaystyle="true"><munderover><mo mathvariant="normal">∑</mo><mrow><mi mathvariant="normal">k</mi><mo mathvariant="normal">=</mo><mi>low</mi></mrow><mi>high</mi></munderover></mstyle><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mi mathvariant="normal">n</mi></msub><mfenced><mi mathvariant="normal">k</mi></mfenced><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mn mathvariant="normal">2</mn></msup></mrow></msqrt><msqrt><mstyle displaystyle="true"><munderover><mo mathvariant="normal">∑</mo><mrow><mi mathvariant="normal">k</mi><mo mathvariant="normal">=</mo><mi>low</mi></mrow><mi>high</mi></munderover></mstyle><mrow><mo mathvariant="normal">|</mo><msub><mi mathvariant="normal">Y</mi><mrow><mi mathvariant="normal">n</mi><mo>-</mo><mn>1</mn></mrow></msub><mfenced><mi mathvariant="normal">k</mi></mfenced><mo>⁢</mo><msup><mrow><mo mathvariant="normal">|</mo></mrow><mn mathvariant="normal">2</mn></msup></mrow></msqrt></mfrac></math><img file="EP1327242B1_D0011.tif" /></maths> where "low" denotes a frequency coefficient index corresponding to a lower frequency band boundary of the spectrum of a signal having been decoded from reconstructed data, "high" denotes a frequency coefficient index corresponding to an upper frequency band boundary of the spectrum of a signal having been decoded from reconstructed data, |Y_n(k)| denotes the magnitude of a coefficient representing a k:th frequency component in the first spectrum, and |Y_n-1(k)| denotes the magnitude of a coefficient representing a k:th frequency component in the previous spectrum. Typically, the lower frequency band boundary may be 0 kHz and the upper frequency band boundary 2 kHz. The threshold frequency in the expressions for describing the correction spectrum C_n(k) above may, but need not, coincide with the upper frequency band boundary. According to a preferred embodiment of the invention the threshold frequency is instead 3 kHz.

Since the primary error concealment unit generally is most effective in the lower part of the frequency band, the proposed muting action is also most effective in this band. Thus, by in the first spectrum Y_n forcing the ratio between the high frequency band power and the low frequency band power to be identical to the corresponding ratio of the previous signal frame the muting from the primary error concealment unit can be extended also to the higher part of the frequency band.

It is a common feature in state-of-the-art error concealment methods to limit the power level of the first frame after a lost or damaged frame to the power level of the latest received undamaged signal frame before the error / loss occurred. Also according to the present invention it is advantageous adapt a similar principle and thus limit the power of a sub-band of the correction spectrum C_n to the power of a corresponding sub-band of a previously received undamaged data F(n-1). The sub-bands can, for example, be defined as coefficients representing frequency components above a threshold frequency (represented by the threshold coefficient k). Such magnitude limitation namely ensures that the high to low frequency band energy ratio is not falsified in the first frame after a frame erasure. The magnitude limitation can be described by the expression: <maths id="math0012" num=""><math display="block"><msub><mi mathvariant="normal">C</mi><mi mathvariant="normal">n</mi></msub><mfenced><mi mathvariant="normal">k</mi></mfenced><mo mathvariant="normal">=</mo><mi>min</mi><mfenced><mn mathvariant="normal">1</mn><mo>⁢</mo><mfrac><msub><mi mathvariant="normal">σ</mi><mrow><mi mathvariant="normal">h</mi><mo mathvariant="normal">,</mo><mi>prevgood</mi></mrow></msub><msub><mi mathvariant="normal">σ</mi><mrow><mi mathvariant="normal">h</mi><mo mathvariant="normal">,</mo><mi mathvariant="normal">n</mi></mrow></msub></mfrac></mfenced><mo mathvariant="normal">⋅</mo><mfenced open="|" close="|" separators=""><msub><mi mathvariant="normal">Y</mi><mi mathvariant="normal">n</mi></msub><mfenced><mi mathvariant="normal">k</mi></mfenced></mfenced></math><img file="EP1327242B1_D0012.tif" /></maths> for k &gt; the threshold coefficient where σ_h,prevgood denotes the root of the power of a signal frame derived from the latest received undamaged signal frame F(N-1), σ_h,_n denotes the root of the power of a signal frame derived from a current signal frame and | Y_n (k) | denotes the magnitude of a coefficient k representing a k:th frequency component in a spectrum derived from the current signal frame.

Since the invention is mainly intended to be used in relation to encoding of speech signals the primary reconstructed signal is preferably an acoustic signal. Furthermore, the encoded speech data is segmented into signal frames, or more precisely so-called speech codec frames. The speech codec frames may also be further divided into speech codec sub-frames, which likewise may constitute the basis for the operation of the error concealment unit according to the invention. Damaged data is then determined on basis of whether a particular speech codec or speech codec sub-frame is lost or received with at least one error.

<figref idref="f0001">Figure 6</figref> shows a block diagram over a CELP-decoder including an error concealment unit 100 to which an acoustic signal a is fed as the primary reconstructed signal y.

The decoder includes a primary error concealment unit 603, which produces at least one parameter P₁ in case a damaged speech frame F is received or if a speech frame F is lost. A data quality determining unit 601 checks all incoming speech frames F, e.g. by performing to a cyclic redundancy check (CRC), to conclude whether a particular speech frame F is correctly or erroneously received. Undamaged speech frames F are passed through the data quality determining unit 601 to a speech decoder 602, which generates an acoustic signal a on its output and via a closed switch 605.

If the data quality determining unit 601 detects a damaged or lost speech frame F the unit 601 activates the primary error concealment unit 603 that produces at least one parameter p₁ representing a basis for a first reconstruction of the damaged speech frame F. The speech decoder 602 then generates the first reconstructed speech signal a in response to the reconstructed speech frame. The data quality determining unit 601 also activates the error concealment unit 100 and opens the switch 605. Thus, the first reconstructed speech signal a is passed as a signal y to the error concealment unit 100 for further enhancement of the acoustic signal a according to the proposed methods above. A resulting enhanced acoustic signal a is delivered on the output as a signal z_E, being spectrally adjusted such that its spectrum deviates less with respect to spectral shape from an acoustic signal a produced from a previously received undamaged speech frame F than the spectrum of the first reconstructed speech signal.

<figref idref="f0001">Figure 7</figref> shows a block diagram over another application of an error concealment unit according to the invention. Here, a data quality determining unit 701 receives incoming parameters S representing important characteristics of an acoustic source signal. In case the parameters S are undamaged (determined e.g. by CRC), they are passed on to an excitation generator 702. The excitation generator 702 delivers an excitation signal e via a switch 705 to a synthesis filter 704, which generates an acoustic signal a.

If, however, the data quality determining unit 701 finds that the parameters S are damaged or lost it activates a primary error concealment unit 703, which produces at least one parameter p₂. The excitation generator 702 receives the at least one parameter p₂ and provides in response thereto a first reconstructed excitation signal e. The data quality determining unit 701 also opens the switch 705 and activates the error concealment unit 100. As a consequence of this, the excitation signal e is received by the error concealment unit 100 as a primary reconstructed signal y. The error concealment unit 100 generates in response thereto a secondary reconstructed signal z_E, being spectrally adjusted such that its spectrum deviates less with respect to spectral shape from an excitation signal e produced from a previously received undamaged speech frame F than the spectrum of the first reconstructed excitation signal.

According to preferred embodiment of the invention, the primary error concealment unit 703 also passes at least one parameter c_i to the error concealment unit 100. This transfer is controlled by the data quality determining unit 701.

In order to sum up, the general method of the invention will now be described with reference to a flow diagram in <figref idref="f0003">figure 8</figref>. Data is received in a first step 801. A subsequent step 802 checks whether the received data is damaged or not, and if the data is undamaged the procedure continues to a step 803. This step stores the data for possible later use. Then, in a following step 804, the data is decoded into an estimate of either the source signal itself, a parameter or a signal related to the source signal, such as an excitation signal. After that, the procedure returns to the step 801 for reception of new data.

If the step 802 detects that the received data is damaged the procedure continues to a step 805 where the data previously stored in step 803 is retrieved. Since, in fact, many consecutive pieces of data may be damaged or lost, the retrieved data need not be data that immediately precede the currently lost or damaged data. The retrieved is nevertheless the <i>latest</i> received undamaged data. This data is then utilised in a subsequent step 806, which produces a primary reconstructed signal. The primary reconstructed signal is based on the currently received data (if any) and at least one parameter of the stored previous data. Finally, a step 807 generates a secondary reconstructed signal on basis of the primary reconstructed signal such that the spectral shape deviates less from a spectrum of the previously received undamaged data than a spectrum of the primary reconstructed signal. After that, the procedure returns to the step 801 for reception of new data.

Another possibility is to include a step 808, which generates and stores data based on the presently reconstructed frame. This data can be retrieved in step 805 in case of a further immediately following frame erasure.

The method above, as well as any of the other described embodiments, of the invention may be performed by a computer program directly loadable into the internal memory of a computer. Such a program comprises software for performing the proposed steps when said program is run on the computer. The computer may naturally also be stored onto any kind of readable medium.

Moreover, it is envisaged to be advantageous to co-locate an error concealment unit 100 according to the invention with a so-called enhancement unit for speech codecs, which performs filtering in the frequency domain. Both these units namely operate in a similar manner in the frequency domain and involve a reverse frequency transformation into the time domain.

Even though the secondary reconstructed signal above has been proposed to be produced by use of a correction magnitude spectrum C_n obtained by performing filtering operations in the frequency domain the same filtering may, of course, equally well be performed in the time domain by instead using a corresponding time domain filter. Any known design method is then applicable to derive such a filter having a frequency response, which approximates the correction magnitude spectrum C_n.

The term "comprises/comprising" when used in this specification is taken to specify the presence of stated features, integers, steps or components. However, the term does not preclude the presence or addition of one or more additional features, integers, steps or components or groups thereof.