Voice/unvoice determination and dialogue enhancement

0001The present invention relates to A signal processing method comprising receiving an input signal and performing linear prediction coding on the input signal.

0002Commonly, a dialogue enhancing system improves the intelligibility of a dialogue degraded by background noise. A conventional dialogue enhancing system uses equalizers and clipping circuits to increase only a voice volume. However, the equalizers and clipping circuits amplify the dialogue and the background noise together.

0003A conventional dialogue enhancing system is disclosed in US-A-5459813.

0004As shown in Figure 1, a known dialogue enhancing system includes a voice/unvoice determinator 90, a spectrum analyzer 42, a voltage controlled amplifier (VCA) unit 50, a combining unit 60, and a combiner 108.

0005Referring to Figure 1, the voice/unvoice determinator 90 determines whether an input signal is a voice signal or a non-voice signal using a low pass filter. The spectrum analyzer 42 includes 30 filter banks and determines formants by analyzing frequency components of the input signal. The VCA unit 50 controls the amplitudes of the formants by applying a gain stored in a gain table to the formants, according to the voice/unvoice signal determined by the voice/unvoice determinator 90. The combining unit 60 combines frequency components of the formants, whose amplitudes are controlled by the VCA unit 50, and other frequency bands.

0006Since the known dialogue enhancing system uses a number of filter banks to analyze frequencies in the spectrum analyzer 42, the analysis is computationally intensive and, since gains for the formants are controlled by the VCA unit 50, the voice signal envelope becomes distorted.

0007In order to enhance dialogue it is necessary to do the voice/unvoice detection.

0008A signal processing method, according to the present invention, is characterised by calculating line spectrum pair coefficients on the basis of the result of said linear prediction coding and determining whether a voice signal is comprised in said input signal on the based of the calculated line spectrum pair coefficients.

0009Thus, the present invention provides a new method that can for voice/unvoice detection and similar functions.

0010The present invention can be applied to dialogue enhancement by selectively boosting a formant from the linear prediction coding result in dependence on the determination of a voice signal being comprised in said input signal.

0011Preferably, the method is performed on a frame-by-frame basis with, for example, each frame having duration in the range 5 to 30ms, preferably in the range 10 to 20ms.

0012According to the present invention, there is also provided an apparatus comprising means for performing a method according to the present invention. Such an apparatus may be a computer, for example a desktop computer or an embedded device in a telephony apparatus.

0013According to the present invention, there is also provided electric or electromagnetic signal representing program codes for controlling a computer to perform a method according to the present invention.

0014According to the present invention, there is further provided a data carrier carrying a record of a signal according to the present invention.

0015Additional preferred and optional features are set forth in claims 8 to 27 appended hereto.

0016Embodiments of the present invention will now be described, by way of example, with reference to Figures 2 to 7 of the accompanying drawings, in which: <ul id="ul0001" list-style="none" compact="compact"><li>Figure 1 is a block diagram of a known dialogue enhancing system;</li><li>Figure 2 is a block diagram of a dialogue enhancing apparatus according to the present invention;</li><li>Figure 3 is a block diagram of the signal combiner in Figure 2;</li><li>Figure 4 is a block diagram of the boost filter coefficient extractor in Figure 2;</li><li>Figure 5 is a flowchart of a dialogue enhancing method according to the present invention;</li><li>Figure 6 is a graph of a spectrum envelope of a voice for p discontinuous frequencies; and</li><li>Figure 7 is a graph of a spectrum envelope of a voice signal passing through a boost filter of the first and second processing units in Figure 2.</li></ul>

0017Referring to Figure 2, a signal combiner 210 combines signals input via left and right channels to generate a combined signal. Here, the left and right channel signals include voice signals and background noise.

0018A boost filter coefficient extractor 220 extracts formants by calculating line spectrum pair (LSP) coefficients and linear prediction coding (LPC) coefficients from the combined signal, extracts boost filter coefficients from the formants, determines whether voice zones exist in the input signals on the basis of proximity of the LSP coefficients, and generates an enhancing select mode (mode select signal) by boosting the input signals according to a determination of whether voice zones exist.

0019A first signal processing unit 230 includes a boost filter with 4 bands, to which the boost filter coefficients extracted by the boost filter coefficient extractor 220 are applied,, and enhances the left input signal by controlling the left input signal to pass through the 4-band boost filter according to the enhancing select mode.

0020A second signal processing unit 240 includes a boost filter with 4 bands, to which the boost filter coefficients extracted by the boost filter coefficient extractor 220 are applied,, and enhances the right input signal by controlling the right input signal to pass through the 4-band boost filter according to the enhancing select mode.

0021Figure 3 is a block diagram of the signal combiner 210 of Figure 2.

0022Referring to Figure FIGS. 2 and 3, dialogue components exist equally in the left and right channels by comparison with acoustic components. Therefore, the input signals of the left and right channels are multiplied by 0.5 in a first multiplier 310 and a second multiplier 320 respectively. Then, the signals are added in an adder 330.

0023Figure 4 is a block diagram of the boost filter coefficient extractor 220 of Figure 2.

0024Referring toFigures 2 to 4, the dialogue components have principal frequency components within 4 KHz. A downsampler 420 performs 1/5 downsampling of the combined signal with a sampling frequency 44.1 KHz.

0025An LPC extractor 430 extracts the LPC coefficients to express the spectrum envelope of a voice component with respect to the signal downsampled by the downsampler 420. Here, four formants exist within the 4 KHz in the spectrum of the voice component.

0026An LSP converter 440 converts the LPC coefficients, extracted by the LPC extractor 430, into LSP coefficients. Here, two LSP coefficients represent one formant. Also, the sharper and higher the formant is, the narrower the gap between the two LSPs.

0027A voice zone determinator 450 determines whether or not a voice zone exists, by comparing the gap between the LSPs, provided by the LSP converter 440, with a threshold value. That is, if the LSP gap is larger than the threshold value, the voice zone determinator 450 determines that there is no voice zone, and generates a bypass signal and, if the LSP gap is smaller than the threshold value, the voice zone determinator 450 determines that there is a voice zone, and generates a boost filtering mode signal (mode select signal).

0028A boost filter coefficient generator 460 calculates center frequencies of first, second, third and fourth formants from the LSP coefficients, provided by the LSP converter 440, and generates booster filter coefficients having boost gains from the center frequencies of the first, second, third and fourth formants.

0029Figure 5 is a flowchart of a dialogue enhancing method according to the present invention.

0030Referring toFigures 2 to 4, the signals input via the left and right channels are combined in operation 510. Here, the left and right channel signals include the center signal.

0031Therefore, the left (L) and right (R) channel signals can be represented as L = Lt + Ct and R = Rt + Ct. Here, Lt is the true L channel signal, Rt is the true R channel signal and Ct is a true center component. Therefore, the combined input signal can be represented as Xinput = 0.5*Lt + 0.5*Rt + Ct. Here, Lt ≠ Rt.

0032When a sound signal is expressed in the frequency domain, most frequency components exist within 6 KHz and several frequency bands are dominant. A voice formant is applicable to a dominant band in the frequency domain. Commonly, four formants are observed in a voice signal. Also, the formants are placed every 1 KHz. Therefore, first, second, third and fourth formants exist within 4 KHz. Accordingly, 1/5 downsampling of the combined signal using a sampling frequency of 44.1 KHz is performed to reduce the computational load in operation 520.

0033The LPC coefficients are extracted from the down sampled signal using an LPC method in operation 530. Here, the LPC method, which is a method of modelling characteristics of a vocal tract among voice generating organs with digital filters having all-pole structures, is to predict coefficients of digital filters from frames (short zones) with 10-20 ms of the voice signal under a presumption that the voice signal is stationary in the 10-20 ms frames. Here, the voice signal s(n) can be represented by Equation 1.<maths id="math0001" num=""><img file="EP1533791A2_D0001.tif" /></maths><maths id="math0002" num=""><img file="EP1533791A2_D0002.tif" /></maths>

0034Here, <i>a</i>_i is a linear filter coefficient modelling the vocal tract, G is a gain and u(n) is an excitation signal.

0035The linear filter coefficients represent frequency characteristics of a voice signal frame and, more particularly, well represent information with respect to a resonant frequency (formant) of the vocal tract, which is a meaningful acoustic characteristic.

0036The LPC coefficients are calculated as shown in Equations 2 to 8 using, for example, a Durbin method using autocorrelation coefficients.<maths id="math0003" num=""><math display="block"><mrow><mtext>[Equation 2]</mtext><mspace linebreak="newline" /><msup><mrow><mtext mathvariant="italic">E</mtext></mrow><mrow><mtext>0</mtext></mrow></msup><mtext> = </mtext><mtext mathvariant="italic">r</mtext><mtext>(0)</mtext></mrow></math><img file="EP1533791A2_D0003.tif" /></maths>

0037Here, <i>E</i>^<i>0</i> is an energy of an input signal and <i>r</i>(0) is a first value of the autocorrelation coefficients.<maths id="math0004" num=""><img file="EP1533791A2_D0004.tif" /></maths>

0038Here, <i>k</i>_i is an ith reflection coefficient and <i>r</i>(<i>i</i>) is an ith autocorrelation coefficient. Therefore, linear filter coefficients are calculated using Equations 4 and 5.<maths id="math0005" num=""><math display="block"><mrow><mtext>[Equation 4]</mtext><mspace linebreak="newline" /><msubsup><mrow><mtext> α</mtext></mrow><mrow><mtext mathvariant="italic">i</mtext></mrow><mrow><mtext>(i)</mtext></mrow></msubsup><mtext>=</mtext><msub><mrow><mtext mathvariant="italic">k</mtext></mrow><mrow><mtext>i</mtext></mrow></msub></mrow></math><img file="EP1533791A2_D0005.tif" /></maths><maths id="math0006" num=""><math display="block"><mrow><mtext>[Equation 5]</mtext><mspace linebreak="newline" /><msubsup><mrow><mtext mathvariant="italic">α</mtext></mrow><mrow><mtext mathvariant="italic">j</mtext></mrow><mrow><mtext>(</mtext><mtext mathvariant="italic">i</mtext><mtext>)</mtext></mrow></msubsup><msubsup><mrow><mtext> = α</mtext></mrow><mrow><mtext mathvariant="italic">j</mtext></mrow><mrow><mtext>(i</mtext><mtext mathvariant="italic">-</mtext><mtext>1)</mtext></mrow></msubsup><msub><mrow><mtext> - k</mtext></mrow><mrow><mtext>i</mtext></mrow></msub><msubsup><mrow><mtext>α</mtext></mrow><mrow><mtext>i-j</mtext></mrow><mrow><mtext>(</mtext><mtext mathvariant="italic">i-</mtext><mtext>1)</mtext></mrow></msubsup><mtext>,   1 ≤ j ≤ i-1</mtext></mrow></math><img file="EP1533791A2_D0006.tif" /></maths><maths id="math0007" num=""><math display="block"><mrow><mtext>[Equation 6]</mtext><mspace linebreak="newline" /><mtext></mtext><msup><mrow><mtext mathvariant="italic">E</mtext></mrow><mrow><mtext>i</mtext></mrow></msup><mtext> = (1-</mtext><msubsup><mrow><mtext mathvariant="italic">k</mtext></mrow><mrow><mtext mathvariant="italic">i</mtext></mrow><mrow><mtext>2</mtext></mrow></msubsup><mtext>)</mtext><msup><mrow><mtext mathvariant="italic">E</mtext></mrow><mrow><mtext>(</mtext><mtext mathvariant="italic">i</mtext><mtext>-1)</mtext></mrow></msup></mrow></math><img file="EP1533791A2_D0007.tif" /></maths>

0039Here, an autocorrelation coefficient r(m) is calculated in advance using Equation 7.<maths id="math0008" num=""><img file="EP1533791A2_D0008.tif" /></maths>

0040Here, s(n) is a voice signal.

0041Eventually, the LPC coefficients can be finally represented as shown in Equation 8.<maths id="math0009" num=""><math display="block"><mrow><mtext>[Equation 8]</mtext><mspace linebreak="newline" /><msub><mrow><mtext> α</mtext></mrow><mrow><mtext mathvariant="italic">m</mtext></mrow></msub><mtext> = LPC coefficients </mtext><mtext mathvariant="italic">=</mtext><msubsup><mrow><mtext> α</mtext></mrow><mrow><mtext mathvariant="italic">m</mtext></mrow><mrow><mtext>(</mtext><mtext mathvariant="italic">P</mtext><mtext>)</mtext></mrow></msubsup><mtext>,   1 ≤ m ≤ p</mtext></mrow></math><img file="EP1533791A2_D0009.tif" /></maths>

0042In order to indicate frequency spectrum information of the voice signal, the LSP coefficients are extracted on the basis of the LPC coefficients in operation 540. The line spectrum pair (LSP) indicates the voice spectrum envelope for p discontinuous frequencies as shown in Figure 6. That is, the LSP is obtained from an LPC model using coefficients based on linear prediction and suggested as another expression type of the LPC coefficients by Itakura-Saito LPC spectral distance.

0043As shown in Equation 1, the voice signal s(n) can be represented as a filter transfer function H(z) = 1/A(z) which performs modelling of a vocal structure. Here, A(z) is equal to Equation 9.<maths id="math0010" num=""><math display="block"><mrow><mtext>[Equation 9]</mtext><mspace linebreak="newline" /><mtext mathvariant="italic">A</mtext><mtext>(</mtext><mtext mathvariant="italic">z</mtext><mtext>) = 1 + </mtext><msub><mrow><mtext mathvariant="italic">a</mtext></mrow><mrow><mtext>1</mtext></mrow></msub><msup><mrow><mtext mathvariant="italic">z</mtext></mrow><mrow><mtext>-1</mtext></mrow></msup><mtext> + ... + </mtext><msub><mrow><mtext mathvariant="italic">a</mtext></mrow><mrow><mtext mathvariant="italic">P</mtext></mrow></msub><msup><mrow><mtext mathvariant="italic">z</mtext></mrow><mrow><mtext>-</mtext><mtext mathvariant="italic">P</mtext></mrow></msup></mrow></math><img file="EP1533791A2_D0010.tif" /></maths>

0044Here, <i>a</i>_<i>p</i> is a pth grade LPC coefficient.

0045The LSP can be defined using A(z) as presented in Equations 10 and 11.<maths id="math0011" num=""><math display="block"><mrow><mtext>[Equation 10]</mtext><mspace linebreak="newline" /><mtext mathvariant="italic">P</mtext><mtext>(</mtext><mtext mathvariant="italic">z</mtext><mtext>) = </mtext><mtext mathvariant="italic">A</mtext><mtext>(</mtext><mtext mathvariant="italic">z</mtext><mtext>) + </mtext><msup><mrow><mtext mathvariant="italic">z</mtext></mrow><mrow><mtext>-(</mtext><mtext mathvariant="italic">P+</mtext><mtext>1)</mtext></mrow></msup><mtext mathvariant="italic">A</mtext><mtext>(</mtext><msup><mrow><mtext mathvariant="italic">z</mtext></mrow><mrow><mtext>-1</mtext></mrow></msup><mtext>)</mtext></mrow></math><img file="EP1533791A2_D0011.tif" /></maths><maths id="math0012" num=""><math display="block"><mrow><mtext>[Equation 11]</mtext><mspace linebreak="newline" /><mtext mathvariant="italic">Q</mtext><mtext>(</mtext><mtext mathvariant="italic">z</mtext><mtext>) = </mtext><mtext mathvariant="italic">A</mtext><mtext>(</mtext><mtext mathvariant="italic">z</mtext><mtext>) </mtext><msup><mrow><mtext mathvariant="italic">- z</mtext></mrow><mrow><mtext>-(</mtext><mtext mathvariant="italic">P+</mtext><mtext>1)</mtext></mrow></msup><mtext mathvariant="italic">A</mtext><mtext>(</mtext><msup><mrow><mtext mathvariant="italic">z</mtext></mrow><mrow><mtext>-1</mtext></mrow></msup><mtext>)</mtext></mrow></math><img file="EP1533791A2_D0012.tif" /></maths>

0046Roots of the two defined polynominal expressions P(z) and Q(z) are defined as the LSP.

0047The LSP coefficients can be obtained from the LPC coefficients and the LPC coefficients can be obtained from the LSP coefficients.

0048Also, since the polynominal expression P(z) is an even function and the polynominal expression Q(z) is an odd function, a power spectrum |<i>A</i>(<maths id="math0013" num=""><math display="inline"><mrow><mover accent="true"><mrow><mtext>ω</mtext></mrow><mo>¯</mo></mover></mrow></math><img file="EP1533791A2_D0013.tif" /></maths>)|² can be represented as shown in Equation 12.<maths id="math0014" num=""><math display="block"><mrow><mtext>[Equation 12]</mtext><mspace linebreak="newline" /><mtext> |A(</mtext><mover accent="true"><mrow><mtext>ω</mtext></mrow><mo>¯</mo></mover><msup><mrow><mtext>)|</mtext></mrow><mrow><mtext>2</mtext></mrow></msup><mtext> = </mtext><mfrac><mrow><mtext>1</mtext></mrow><mrow><mtext>4</mtext></mrow></mfrac><mtext> [|</mtext><mtext mathvariant="italic">P</mtext><mtext>(</mtext><mover accent="true"><mrow><mtext>ω</mtext></mrow><mo>¯</mo></mover><msup><mrow><mtext>)|</mtext></mrow><mrow><mtext>2</mtext></mrow></msup><mtext></mtext><mtext mathvariant="italic">+</mtext><mtext> |</mtext><mtext mathvariant="italic">Q</mtext><mtext>(</mtext><mover accent="true"><mrow><mtext mathvariant="italic">ω</mtext></mrow><mo>¯</mo></mover><msup><mrow><mtext>)|</mtext></mrow><mrow><mtext>2</mtext></mrow></msup><mtext>]</mtext></mrow></math><img file="EP1533791A2_D0014.tif" /></maths>

0049Equation 12 shows that a root of A(z) is closely correlated with the roots of P(z) and Q(z). That is, a formant frequency is represented by gathering 2 or 3 LSP frequencies. Also, a bandwidth of a formant can be expressed according to the proximity of a line pair of the LSP. That is, referring to Figure 6, a greater proximity indicated by a gap between a solid line and a dotted line shows a formant with a narrower bandwidth and a greater amplitude.

0050Whether the voice zones exist is determined using the LSP coefficients in operation 550. In a voice, a formant has a narrow bandwidth and a great amplitude. Therefore, whether the voice zones exist is determined using the proximity of the LSP. That is, if the LSP gap is smaller than the threshold value, it is determined that there is a voice zone, and if the gap of the LSP is larger than the threshold value, it is determined that there is no voice zone.

0051If it is determined that there is no voice zone using the proximity of the LSP in operation 560, the input stereo signal is bypassed as it is in operation 582.

0052If it is determined that there are voice zones using the proximity of the LSP in operation 560, operations 572, 574 and 576 of the boosting of voice formants is performed as follows.

0053That is, if it is determined that there are voice zones in the input signal, center frequencies of first, second, third, and fourth formants are determined using the LSP coefficients in operation 572.

00544-band boost filter coefficients with boost levels are obtained using the center frequencies of the first, second, third and fourth formants in operation 574. Here, the boost levels of the formants are all the same so that a spectrum envelope of the voice signal is not varied.

0055An input stereo signal, e.g., the left or right channel signal, passes through a 4-band boost filter to which the boost filter coefficients are applied in operation 576. Figure 7 shows an LPC spectrum of a signal having the same boost gains at the first, second, third, and fourth formant bands 710, 720, 730, and 740.

0056Finally, as shown in Figure 7, voice zones of the input stereo signal are improved by passing the 4-band boost filter.

0057The present invention can also be embodied as computer readable codes stored on a computer readable recording medium. The computer readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks and optical data storage devices. The codes may also be transmitted as electric or electromagnetic signals either as baseband signals or carried by carrier waves. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

0058As described above, according to the present invention, the computational amount of a voice detecting/enhancing operation can be reduced by predicting formants using LPC coefficients. Also, since an envelope of a voice signal is not distorted by setting the predetermined gains in first, second, third, and fourth formant bands of the voice signal, a timbre is not varied.