Speech enhancement

<u style="single">FIELD OF THE INVENTION</u>

0001The present invention relates to a method and apparatus for enhancing speech quality. Although the present invention is suitable for a wide scope of applications, it is particularly suitable for enhancing the quality of speech effectively.

<u style="single">BACKGROUND OF THE INVENTION</u>

0002Generally, various kinds of methods for enhancing a quality of speech have been proposed. A spectral subtraction method (SSM) is representative one of the various kinds of methods. The spectral subtraction method (SSM) is explained with reference to FIG. 1 as follows.

0003The SMM is a method of estimating a short-time spectral magnitude directly. In the SSM, speech is modeled into a form to which a noise, represented by an uncorrelated random variable, is added. The speech modeling is expressed by Formula 1 as follows.

0004<maths id="math0001" num="[Formula 1]"><math display="block"><mtable><mtr><mtd><mi mathvariant="italic">y</mi><mfenced open="[" close="]"><mi mathvariant="italic">n</mi></mfenced><mo mathvariant="italic">=</mo><mi mathvariant="italic">s</mi><mfenced open="[" close="]"><mi mathvariant="italic">n</mi></mfenced><mo mathvariant="italic">+</mo><mi mathvariant="italic">d</mi><mfenced open="[" close="]"><mi mathvariant="italic">n</mi></mfenced></mtd></mtr></mtable></math><img file="EP1632935B1_D0001.tif" /></maths>

0005In Formula 1, <i>y[n]</i> is an input speech. Furthermore, it is assumed that <i>d[n]</i> is an uncorrelated noise to <i>s[n].</i> Hence, power spectral density is found according to Formula 2 as follows.

0006<maths id="math0002" num="[Formula 2]"><math display="block"><mtable><mtr><mtd><msub><mi>S</mi><mi>y</mi></msub><mfenced><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><mi>ω</mi></mrow></msup></mfenced><mo>=</mo><msub><mi>S</mi><mi>s</mi></msub><mfenced><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><mi>ω</mi></mrow></msup></mfenced><mo>+</mo><msub><mi mathvariant="italic">S</mi><mi mathvariant="italic">d</mi></msub><mfenced><msup><mi mathvariant="italic">e</mi><mi mathvariant="italic">jω</mi></msup></mfenced></mtd></mtr></mtable></math><img file="EP1632935B1_D0002.tif" /></maths>

0007In Formula 2, <i>S_y(e^jω)</i> is represented by Formula 3 via a short-time Discrete-Time Fourier Transform (DTFT).

0008<maths id="math0003" num="[Formula 3]"><math display="block"><mtable><mtr><mtd><msub><mi>S</mi><mi>y</mi></msub><mfenced><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><mi>ω</mi></mrow></msup></mfenced><mo>=</mo><msup><mfenced open="|" close="|" separators=""><mi>Y</mi><mfenced><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><mi>ω</mi></mrow></msup></mfenced></mfenced><mn>2</mn></msup></mtd></mtr></mtable></math><img file="EP1632935B1_D0003.tif" /></maths>

0009A phase is known to find a spectrum of a speech frame itself. Moreover, it is proven that there is no large difference in determining the phase of the speech frame using a phase of noisy speech that is substantially mixed with noise. <nplcit id="ncit0001" npl-type="s"><text>D. L. Wang and J. S. Lim, "The unimportance of phase in speech enhancement," IEEE Trans. on Acoust. Speech, and Signal Processing, vol-ASSP. 30, pp. 679-681, 1982</text></nplcit>.

0010In case of determining the phase of the speech frame using the phase of the noisy speech, the short-time DTFT to be sought can be found by Formula 4.

0011<maths id="math0004" num="[Formula 4]"><math display="block"><mtable><mtr><mtd><mover accent="true"><mi>S</mi><mo stretchy="true">^</mo></mover><mfenced><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><mi>ω</mi></mrow></msup></mfenced><mo>=</mo><msup><mfenced open="|" close="|" separators=""><msub><mi>S</mi><mi>y</mi></msub><mfenced><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><mi>ω</mi></mrow></msup></mfenced><mo>−</mo><msub><mover accent="true"><mi>S</mi><mo stretchy="true">^</mo></mover><mi>d</mi></msub><mfenced><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><mi>ω</mi></mrow></msup></mfenced></mfenced><mrow><mn>1</mn><mo>/</mo><mn>2</mn></mrow></msup><mo>⁢</mo><msup><mi>e</mi><mrow><mi>j</mi><mo>⁢</mo><msub><mi mathvariant="italic">ϕ</mi><mn mathvariant="italic">1</mn></msub><mfenced><mi>ω</mi></mfenced></mrow></msup></mtd></mtr></mtable></math><img file="EP1632935B1_D0004.tif" /></maths>

0012<i>S_y</i> (<i>e^jω</i>) in Formula 4 is found from Formula 2. And <i>φ_y</i> (<i>e^jω)</i> uses the phase of the noisy speech. Therefore, an estimated value of <i>ŝ</i>[<i>n</i>] to be sought is found from Formula 4. If there is no speech, <i>Ŝ_d</i> (<i>e^jω</i>) is estimated from the noise.

0013One of the various speech quality enhancing methods such as an Adaptive Line Enhancer (ALE) is explained with reference to FIG. 2 as follows. First, use of a general adaptive filter is explained because of the ALE's evolution from a scheme using the adaptive filter.

0014When using the adaptive filter, after receiving inputs of two microphones, i.e., receiving a noise speech as an input of one microphone and a pure noise as an input of the other microphone, a transfer function and the like are generated due to a distance between the two microphones and the like. However, the adaptive filter removes the transfer function to attain a clean speech.

0015The method using the adaptive filter is very effective in some cases and has been successfully used for a practical purpose. Yet, the method requires installation of a pair of microphones. Also, there is a structural difficulty in deciding how far the pair of microphones should be spaced apart from each other. Hence, it is difficult to apply the method to a user equipment such as a mobile terminal.

0016The ALE (Adaptive Line Enhancer) is an improvement of the method employing the adaptive filter and is a scheme for performing adaptive filtering on signals s[n] and d[n] attained from the same microphone by leaving a difference equivalent to a pitch period in between the signals. Here, the pitch period corresponds to a period of a voiced speech part of a speech signal.

0017For the voiced speech, a periodic impulse train excites a vocal tract. Hence, the ALE exerts a considerable effect on the voiced speech. However, for an unvoiced speech, the corresponding speech is crushed.

0018One of the various speech quality enhancing methods such as a scheme for using an adaptive comb filter is explained as follows. First, when using an adaptive comb filter, a corresponding scheme similar to the ALE has a better effect on a voiced speech.

0019In case of the voiced speech, an excitation signal is a periodic signal. Even if a Fourier Transform is performed on an impulse train, the result indicates that the impulse train appears in a frequency domain. Hence, in case of the voiced speech, a peak periodically appears at a portion where a pitch frequency becomes multiple. It is a matter of course that a contour of an overall spectrum is represented by a resonance of a vocal tract called a formant.

0020When a noisy speech is represented by <i>y</i>[<i>n</i>], a speech is represented by <i>s</i>[<i>n</i>], and the speech of which noise is removed is estimated to be represented by <i>ŝ</i>[<i>n</i>], the speech enhanced by an adaptive comb filter is expressed by Formula 5.

0021<maths id="math0005" num="[Formula 5]"><math display="block"><mtable><mtr><mtd><mover accent="true"><mi>s</mi><mo stretchy="true">^</mo></mover><mfenced open="[" close="]"><mi>n</mi></mfenced><mo>=</mo><mstyle displaystyle="true"><munderover><mo>∑</mo><mrow><mi>i</mi><mo>=</mo><mo>−</mo><mi>L</mi></mrow><mi>L</mi></munderover><msub><mi>c</mi><mi>i</mi></msub><mo>⁢</mo><mi>y</mi><mo>⁢</mo><mfenced separators=""><mi>n</mi><mo>−</mo><mi>i</mi><mo>⁢</mo><msub><mi>T</mi><mn>0</mn></msub></mfenced></mstyle></mtd></mtr></mtable></math><img file="EP1632935B1_D0005.tif" /></maths>

0022In Formula 5, <i>T</i>₀ represents an extracted pitch period and <i>c_i</i> represents a comb filter coefficient. Here, a small value (1~6) is generally used as a value of L. Meanwhile, since a noise is not generally periodic, the adaptive comb filter is effective in removing the noise. However, the related art speech quality enhancing methods have the following problems or disadvantages.

0023First, if there is no speech, <i>Ŝ_d</i> (<i>e^jω</i>) is estimated from the noise in the SSM. However, it is unable to measure the <i>Ŝ_d</i> (<i>e^jω</i>) reliably. Namely, it is able to estimate the <i>Ŝ_d</i> (<i>e^jω</i>) only if it is assumed that the noise <i>d</i>[<i>n</i>] is a stationary signal. Even if it is actually so, it is unable to avoid a spectrum variation according to a time. Specifically, in case of a mobile terminal or the like, it is unable to measure the <i>Ŝ_d</i> (<i>e^jω</i>) reliably since circumferential environments keep changing.

0024Second, the ALE or the scheme using the adaptive comb filter shows excellent performance on the voiced speech. However, these schemes or methods are applicable to the voiced signal only. In case of applying the ALE or the scheme using the adaptive comb filter to an unvoiced signal, performance is reduced due to a slight misalignment of a voiced/unvoiced (V/UV) decision.

0025Third, in case of a certain speech, a voiced characteristic appears in a low frequency or an unvoiced characteristic appears in a high frequency, whereby the performance of the ALE is degraded.

0026Yoo CD et al describe in "Speech enhancement based on the generalized dual excitation model with adaptive analysis window", ICASSP' 95, Detroit, USA, May 1995, an enhancement method whereby the voiced component is enhanced by removing any harmonic amplitudes below the effective noise model, with the unvoiced component being enhanced by spectral subtraction.

<u style="single">SUMMARY OF THE INVENTION</u>

0027The present invention is directed to enhancing speech quality.

0028To achieve this, the present invention is embodied in a method for enhancing a quality of speech, the method comprising dividing an input speech into a voiced speech and an unvoiced speech, performing an adaptive line enhancer process using adaptive filtering on the voiced speech to remove a noise of the voiced speech, and performing spectral subtraction on the unvoiced speech.

0029An average value of noise spectrums estimated from prescribed frames corresponding to a previous voiced speech by the adaptive line enhancer process may be used for the spectral subtraction. The adaptive filtering may use a pitch period extracted from a frame corresponding to the voiced speech.

0030The method may further comprise performing at least one of low pass filtering and high pass filtering on the input speech and performing adaptive comb filtering on an output of the high pass filtering to remove a noise of the output. Preferably, the adaptive comb filtering is performed when the output of the high pass filtering corresponds to the voiced speech. An output of the low pass filtering may be divided into the voiced speech and the unvoiced speech.

0031Preferably, noise spectral data obtained from a section of the voiced speech is used for the spectral subtraction. Furthermore, the noise spectral data may be a value resulting from averaging noise spectrums estimated from prescribed frames corresponding to a previous voiced speech by the adaptive filtering.

0032In accordance with another embodiment of the present invention, an apparatus for enhancing a quality of speech comprises a decision block for dividing an input speech into a voiced speech and an unvoiced speech, an adaptive line enhancer (ALE) block for performing an adaptive line enhancer process on the voiced speech to remove a noise of the voiced speech, and a spectral subtraction (SS) block for performing spectral subtraction on the unvoiced speech.

0033Preferably, the apparatus further comprises a low pass filter for performing low pass filtering on the input speech to output to the decision block and a high pass filter for performing high pass filtering on the input speech.

0034The apparatus may further comprise an adaptive comb filter for removing a noise from an output of the high pass filter if the output of the high pass filter corresponds to the voiced speech. Preferably, the adaptive comb filter uses a pitch period extracted from the voiced speech.

0035The apparatus may further comprise a pitch extractor for extracting a pitch period from the voiced speech, wherein the pitch extractor provides the extracted pitch period to the ALE block.

0036Preferably, the SS block uses a noise spectrum estimated by the ALE block. Furthermore, the SS block may use an average value of noise spectrums estimated from prescribed frames corresponding to a previous voiced speech by the ALE block.

0037It is to be understood that both the foregoing general description and the following detailed description of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

<u style="single">BRIEF DESCRIPTION OF THE DRAWINGS</u>

0038The accompanying drawings, which are included to provide a further understanding of the invention, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention..

0039FIG. 1 is a block diagram illustrating a general spectral subtraction method (SSM).

0040FIG. 2 is a block diagram illustrating a general adaptive line enhancer (ALE).

0041FIG. 3 is a block diagram of an apparatus for enhancing a quality of speech in accordance with one embodiment of the present invention.

0042FIG. 4 is a flow diagram illustrating a method for enhancing a quality of speech in accordance with one embodiment of the present invention.

<u style="single">DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS</u>

0043The present invention relates to enhancing a quality of speech.

0044Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

0045In a method of enhancing a quality of speech according to one embodiment of the present invention, a prescribed speech quality enhancing process is performed on a voiced speech and a spectral subtraction method (SSM) is performed on an unvoiced speech using a noise spectrum attained from performing the prescribed speech quality enhancing process.

0046An apparatus for enhancing a quality of speech in accordance with one embodiment of the present invention is explained with reference to FIG. 3.

0047Referring to FIG. 3, an apparatus for enhancing a quality of speech comprises a low pass filter (LPF) 51 performing low pass filtering on an inputted speech y[n] and a high pass filter (HPF) 50 performing high pass filtering on the inputted speech y[n].

0048The apparatus further comprises an adaptive comb filter 56 for processing a high frequency component. The apparatus also comprises a voiced/unvoiced (V/UV) decision block 52, a pitch extractor 53 and a spectral subtraction block 55 to process a low frequency component. Moreover, the apparatus comprises an adaptive line enhancer (ALE) block 54. Alternatively, the ALE block 54 may be replaced by a means for employing a different speech quality enhancing scheme.

0049An output of the HPF 50 is inputted to an adaptive comb filter 56. An output of the LPF 51 passes through a path using either the ALE or SSM according to a voiced or unvoiced speech. The V/UV decision block 52 decides whether the speech having passed through the LPF 51 corresponds to the voiced or unvoiced speech. It is then decided whether to use the ALE or SSM according to the decision result of the V/UV decision block 52.

0050Preferably, the V/UV decision block 52 delivers a frame corresponding to the unvoiced speech of the speech having passed through the LPF 51 to the spectral subtraction block 55 using the SSM. Alternatively, a frame corresponding to the voiced speech of the speech having passed through the LPF 51 is delivered to the path using the ALE. The path using the ALE comprises the pitch extractor 53 and the ALE block 54.

0051The pitch extractor 53 extracts a pitch period To from the frame corresponding to the voiced speech and then provides the extracted pitch period To to the adaptive comb filter 56. The pitch extractor 53 also provides the extracted pitch period to the ALE block 54, wherein the ALE block 54 uses the pitch period To for the ALE to enhance a quality of speech for the frame corresponding to the voiced speech.

0052As mentioned in the foregoing description, the present invention uses the ALE block 54 as the means for enhancing the quality of speech in accordance with one embodiment of the present invention.

0053Because a frequency range, within which a pitch frequency exists, corresponds to 50~400Hz, a cutoff frequency of the LPF 51 is determined to sufficiently include the frequency range and to allow a portion of the speech having the most dominant influence on the pitch period to pass through. Preferably, the cutoff frequency is set to about 800Hz.

0054In one embodiment of the present invention, when applying the ALE, the speech having a bandwidth of 0~4kHz may be obtained by recombination with a range of 400~N4,000Hz. This corresponds to a case having an 8kHz sampling rate. To prepare for the case, the present invention further uses the adaptive comb filter 56.

0055The adaptive comb filter 56 of the present invention removes noises lying between portions seeming like an impulse train represented by a pitch component in a high frequency. Preferably, the adaptive comb filter 56 operates if a clear signal corresponding to the voiced speech exists in the high frequency component.

0056Meanwhile, the spectral subtraction block 55 employing the SSM uses noise spectral data obtained from a section of the voiced speech. Preferably, the spectral subtraction block 55 uses a value resulting from averaging noise spectrums estimated in a prescribed frame of the previous voiced speech. In other words, the noise spectral data is obtained from averaging noise spectrum data sequences of a predetermined number of frames each time the noise spectrum is obtained from the voiced speech. Therefore, the speech <i>ŝ</i>[<i>n</i>] can be obtained in a manner of removing noises from the outputs of the spectral subtraction block 55 and the adaptive comb filter 56.

0057FIG. 4 is a block diagram of a method for enhancing a quality of speech in accordance with one embodiment of the present invention. Referring to FIG. 4, once a prescribed speech y[n] is inputted (S1), low pass filtering (S2) and high pass filtering (S3) are carried out on the inputted speech y[n].

0058A frequency range, in which a pitch frequency exists, is generally 50~400Hz. Accordingly, a portion of the speech, which sufficiently includes the frequency range and which has the most dominant influence on a pitch period, undergoes low pass filtering. Preferably, a cutoff frequency of the low pass filtering is set to about 800Hz.

0059Subsequently, it is identified whether an output of the low pass filtering corresponds to a voiced speech or an unvoiced speech (S4). If the output of the low pass filtering corresponds to the voiced speech, a prescribed speech quality enhancing method is carried out on a frame corresponding to the voiced speech. Preferably, ALE is used as the speech quality enhancing method for the voiced speech. Hence, an ALE process is carried out on the frame corresponding to the voiced speech (S6).

0060Prior to the ALE process, it is a matter of course that a pitch period is extracted from the frame corresponding to the voiced speech (S5). The extracted pitch period is used for adaptive comb filtering (S8) as well as for the ALE process (S6).

0061However, if the output of the low pass filtering corresponds to the unvoiced speech, spectral subtraction is carried out on a frame corresponding to the unvoiced speech (S9). In carrying out the spectral subtraction, a value obtained from averaging noise spectrums estimated from a prescribed frame of the previous voiced speech by the ALE process is used. Preferably, a value obtained from averaging noise spectrum data sequences of a predetermined number of frames each time a noise spectrum is obtained from the voiced speech by the ALE process is used. The corresponding value is the noise spectral data obtained from the voiced speech.

0062Adaptive comb filtering is carried out on an output resulting from performing high pass filtering on the inputted speech y[n] to remove noise of the output (S8). In doing so, the pitch period extracted from the voiced speech of the output from the low pass filtering (S5) is used in carrying out the adaptive comb filtering. However, prior to the adaptive comb filtering, it is decided whether the output from the high pass filtering corresponds to the voiced speech (S7). If a clear signal corresponding to the voiced speech exists, the adaptive comb filtering is carried out.

0063Therefore, the speech <i>ŝ</i>[<i>n</i>] can be obtained in a manner of removing noises from the results of the spectral subtraction and the adaptive comb filtering. According to the above-described present invention, performance better than that of the ALE or SSM is expected.

0064In the present invention, after the ALE is performed on the low frequency component having the strong pitch characteristic, the adaptive comb filter is further used when the high frequency component corresponds to the voiced speech. Hence, the present invention provides effective performance if the low and high frequencies have the voiced and unvoiced characteristics, respectively.

0065Because the quality of speech is enhanced based on the pitch characteristic, which is the generic characteristic of the speech, the present invention is more tenacious against babble noise and the like than other speech quality methods (e.g., Wiener filtering, spectral subtraction method). Accordingly, the present invention is useful for noise removal using a single microphone of a mobile terminal and for noise removal when recording speech with a portable recorder. The present invention is further useful for noise removal in a general wire/wireless phone or for recording speech in a PDA or the like.