Enhancement method for high-frequency reconstruction techniques combining frequency translation or folding with spectral envelope adjustment adjusting patched subband signal according to desired spectral envelope

SWEDEN (12) PATENT (13) C2 tu) 523 883

<img file="SE523883C2_D0001.tif" />

2004-05-25

2002-11-22

2002-11-22

2001-05-23

<img file="SE523883C2_D0002.tif" />

(19) SE (si)

International class ¹G10L 19/00

PATENT AND REGISTRATION (45) (41) (22) (24) (62) (86)

Patent filed Application widely available The patent application was submitted on expiration date

Tribal application number (83)

International filing date 2001-05-23 Filing date for European patent application Deposit of microorganism (21) Patent application number (J203468-4

Application received as:

Swedish patent application completed international patent application with number PCT / SE01 / 01171 converted European patent application with number (30)

2000-05-23 SE 0001926-5 (73) (72) (74) (54) (56) (57)

PATENT HOLDER Coding Technologies AB, Döbelnsgatan 64 113 52b Stockholm SE INVENTOR Lars Gustaf Liljeryd, Solna SE, Per Ekstrand, Stockholm SE,

Fredrik Henn, Bromma SE, Kristofer Kjörling, Solna SE OMBUD Swedpatent AB

NAME Improved spectral translation / folding in the subband area

CALLED PUBLICATIONS: - SUMMARY:

The present invention relates to a new method and apparatus for improved High Frequency Reconstruction (HFR) technology utilizing frequency translation or folding or a combination thereof. The proposed invention can be applied to audio source coding systems, and offers significantly reduced computational complexity. This is accomplished by frequency translation or folding in the subband region, preferably integrated with envelope alignment in the same domain. The concept of dissonance protection band filtration is also presented. The proposed invention offers a mid-range low-complexity HFR method useful in coding applications for speech and natural audio.

• bco OOO oOo

Ooo

Ooo

Ooo o-0-o β O-0-o ooo

<img file="SE523883C2_D0003.tif" />

The numbers in brackets indicate international identification code, INID code. Letters in clamps indicate international document code.

523 883

SUMMARY

The present invention relates to a new method and apparatus for improved High Frequency Reconstruction (HFR) technology utilizing frequency translation or folding or a combination thereof. The proposed invention can be applied to audio source coding systems, and offers significantly reduced computational complexity. This is accomplished by frequency translation or folding in the subband area, preferably integrated with envelope alignment in the same domain. The concept of dissonance protection band filtration is also presented. The proposed invention offers a low complex intermediate quality HFR method useful in speech and natural audio coding applications.

523 883

The present invention relates to a new method and apparatus for improving High Frequency Reconstruction (HFR) technology, applicable to audio source coding systems. Significantly reduced computational complexity is achieved by the new method. This is accomplished by frequency translation or folding in the subband region, preferably integrated with the spectral envelope alignment process. The invention also improves the noticeable audio quality through the concept of dissonant protective band filtration. The proposed invention offers a low complex intermediate quality HFR method and relates to the PCT patent Spectral Band Replication (SBR) [WO 98/57436].

Schemes in which original audio information above a certain frequency are replaced by noise canceling or manipulated low band information are collectively referred to as High Frequency Reconstruction (HFR) methods. Previously known HFR methods, apart from interference introduction and nonlinearities such as rectification, are generally utilizing so-called copying techniques for generating the high band signal. These techniques utilize mainly broadband linear frequency shifts, ie translations, or frequency inverted linear shifts, ie folds. Previously known HFR methods have primarily been intended to improve the performance of speech codecs. However, recent developments in high-band conversion that utilize accurate perception methods have, however, made HFR methods successfully applicable even to natural audio codecs, coding of music or other complex program material, PCT patent [WO 98/57436]. Under certain conditions, simple copying technology has proven to be adequate even when coding complex program material. These techniques have been found to produce reasonable results for medium-quality applications, and especially for codec implementations as there are serious limitations to the computational complexity of the overall system.

The human voice and most musical instruments generate quasi-stationary tonal signals emanating from oscillating systems. According to Fourier theory, each periodic signal can be expressed as a sum of sine components with the frequencies f, 2f, 3f, 4f, 5f, etc., where f is the fundamental frequency. The frequencies form a harmonic series. Total affinity refers to the relationships between the perceivable tones or the harmonies. In natural sound reproduction, such tonal affinity is controlled and given by the different types of voice and instrument used. The general idea of ​​HFR technology is to replace original high frequency information with information created from available low band and subsequently add spectral envelope adjustment to this information. Previously known HFR methods create high band signals where tonal affinity is often uncontrolled and impaired. The methods generate non-harmonic frequency components that cause perceptible deviations when used for complex program material. Such deviations are referred to in the coding literature as coarse "sound and are perceived by

523 883 the listener as distortion.

Sensible dissonance (roughness), as opposed to consonance (pleasantness), occurs when nearby tones or parts interfere. The dissonance theory has been explained by many researchers, including Plomp and Levelt [Tonal Consonance and Critical Bandwidth ”R Plomp, WJM Levelt JASA, Vol 38,1965], and states that two parts are considered dissonant if the frequency difference is within about 5 to 50% of the bandwidth. for the critical band in which the parts are located. A bark equals a frequency range on a critical band. For reference, the function z &lt;/) = - ^ - 0.53 [BaA] (1) + -f can be used to convert from frequency (f) to bark scale (z). Plomp states that the human perception system cannot discriminate two parts if they differ in frequency by approximately less than five percent of the critical band in which they are located, or equivalent, are separated less than 0.05 Bark in frequency. On the other hand, if the distance between the parts is more than about 0.5 Bark, then they are perceived as separate tones.

The dissonance theory partly explains why previously known methods give unsatisfactory results. A set of consonant parts translated upward in frequency can be dissonant. In addition, in transition areas between moments of transposed bands and the low band, the parts may interfere, as they may not be within the bounds of acceptable deviation according to the dissonance rules.

WO 98/57436 demonstrates how frequency transposing is accomplished by multiplication by a transposition factor M. Consecutive channels from an analysis filter bank are frequency translated into synthesis filter bank channels, but which are separated by two intermediate reconstruction range channels, when the multiplication factor M is different, when the multiplication factor M is equal to 2. Alternatively, amplitude and phase information from two different analyzing channels can be combined. The amplitude signals are connected so that the sizes of the consecutive filter bank consecutive channels are frequency translated to the sizes of subband signals associated with consecutive synthesis channels. The phases of subband signals from the same channels are subjected to frequency transposing using a factor M.

It is an object of the present invention to provide a concept for obtaining an envelope-adjusted and frequency-translated signal through high-frequency spectral reconstruction as well as a concept for decoding utilizing high-frequency spectral reconstruction, resulting in better quality reconstruction.

523 883

This intention is achieved by a method according to claims 1 and 13 and 23 or a device according to claims 19 and 20 or a decoder according to claim 21.

The present invention demonstrates a new method and apparatus for improving translation or folding techniques in a source coding system. The purpose includes substantial reduction of the computational complexity and reduction of perceptible perceptible deviations. The invention demonstrates a new implementation for a sub-sampled digital filter bank as a frequency translation or folding device, in addition to offering improved transition accuracy between the low band and translated or folded band. In addition, the invention shows that transition areas, to avoid detectable dissonance, are improved by filtering. The filtered areas are designated dissonance protection bands, and the invention offers the opportunity to reduce dissonant portions in an uncomplicated and accurate manner utilizing the subsampled filter bank.

The new filter bank-based translation or folding process can advantageously be integrated with the spectral envelope adjusting process. The filter bank utilized for envelope adjustment is thus also utilized for the frequency translation or folding process, thus eliminating the need to use a separate filter bank or process for spectral envelope adjustment. The proposed invention offers a unique and flexible filter bank design at a low computational cost, thus creating a very efficient translation / folding / envelope adjusting system.

In addition, the proposed invention is advantageously combined with the method of Adaptive Noise-Floor Addition described in PCT patent [SE00 / 00159]. This combination improves the perceptible quality under difficult program material conditions.

The proposed subband domain-based translation of folding technique includes the following steps:

filtering a low band signal through a filter bank's analysis portion to obtain a set of subband signals:

repackaging a number of the subband signals from consecutive low band channels to consecutive high band channels in the synthesis portion of a digital filter bank; adjusting the patched subband signals, in accordance with a desirable spectral envelope; and filtering the adjusted subband signals through the synthesis portion of a digital filter bank, to obtain an envelope-adjusted and frequency-translated or folded signal in a very efficient manner.

Attractive applications for the proposed invention relate to improvements in various types of mid-quality codec applications such as MPEG 2 Layer III, MPEG 2/4 AAC, DOLBY AC-3,

NTT TwinVQ, AT &amp; T / Lucent PAC etc., when such codecs are used at low bit rate.

523 883

The invention is also very useful in various speech codecs such as G. 729 MPEG-4 CELP and H VXC etc. to improve perceptual quality. The above codecs are widely used in multimedia, in the telephone industry, on the Internet as well as in professional multimedia applications.

The present invention is described in the form of illustrative examples, not limiting the scope or scope of the invention, with reference to the accompanying drawings, in which:

Fig. 1 illustrates filter bank-based translation or folding integrated into a coding system according to the present invention;

Fig. 2 shows a base structure for a maximum decimated filter bank;

Fig. 3 illustrates spectral translation according to the present invention;

Fig. 4 illustrates spectral folding according to the present invention;

Fig. 5 illustrates spectral translation utilizing protective bands of the present invention.

Digital filter bank-based translation and folding

New filterbank-based technology for translation or folding will now be described. The signal under consideration is divided into a series of subband signals by the filter bank's analysis section. The subband signals are then repatched, by reconnecting analysis and synthesis subband channels, to obtain spectral translation or folding or a combination thereof.

Fig. 2 shows the basic structure of a maximally decimated filter bank analysis / synthesis system. The analyzer bank 201 divides the input signal into a plurality of subband signals. The synthesis filter bank 202 combines the subband samplings with the intention of recreating the original signal. Implementations utilizing maximum decimated filter banks will drastically reduce calculation costs. It will be appreciated that the invention can be implemented using several types of filter banks or converters, including cosine or complex exponent-modulated filter banks, filter bank translations of wave conversion, other non-uniform filter banks or conversions, and multi-dimensional filter banks or conversions.

In the illustrative, but not limiting, description below, it is assumed that an L-channel filter bank divides the input signal x (n) into L subband signals. The input signal, with the sampling frequency f_s, is band limited to the frequency f_c. The analysis filters at a maximally decimated filter bank (Fig. 2) are designated H 2 (z) 203, where k = 0.1 ..... L1. The subband signals v_k(n) each having the sampling frequency ffL, after passing the decimers 204.

523 883

The synthesis section, with the synthesis filters designated F / µm, reconstitutes the subband signals after interpolation 205 and filtering 206 to produce x (n). In addition, the present invention performs a spectral reconstruction of x (n), which provides an improved signal y (n).

The start channel of the reconstruction area, denoted M, is determined by

<img file="SE523883C2_D0004.tif" />

(2)

The number of source area channels is denoted S (1 sSs M). Implementation of spectral reconstruction by translation into x (n) of the present invention, in combination with envelope alignment, is accomplished by repatching the subband signals such as^vM + kW = eM + k (”) ^vMS-P + k (.ⁿ) · (3) o + p where ke [0, S-1], (-1) = 1, ie S + P is an even number, P is an integer offset (0 &lt;

P &lt;MS) as well as e_{M + K}(n) is the envelope correction. In addition, spectral reconstruction by folding on x (n) of the present invention is accomplished by repatching the subband signals such as^vM + k (ⁿ) = ^eM + k (ⁿ) ^V* M —PSk (”)&gt;

(4) where ke [0, S-1], (-1)^{p + p} = -1, ie S + P is an odd integer number, P is an integer offset (1-SP &lt;M-2S + 1) and e_{M + t}fn) is the envelope correction. The operator [*] denotes complex conjugation. Usually, the repatching process is repeated until the intended size of high frequency bandwidth is reached.

It should be noted that by using subband domain-based translation and folding, improved transition accuracy between the low band and existing translated or folded bands is achieved, as all signals are filtered through the filter bank which has matched frequency responses.

If the frequency f_c for x (n) is too high, or equivalent f_s is too low to allow efficient spectral reconstruction, ie M + S&gt; L, the number of subband channels can be increased after the analysis filtration. Filtering of the subband signals with a QL channel synthesis filter bank, where only the L low band channels are utilized and the sampling factor Q is selected so that QL is an integer value, will result in an output with the sampling frequency Qf_s. The extended filter bank will thus function as an L-channel filter bank followed by a collector. Since in this case the L (Q-1) high band filters are unused (fed with zeros), so the

523 883 bandwidth - the filter bank will only reconstruct a sampled version of x (n). However, if the L subband signals are repatched to the high band signals, according to Eq. (3) or (4), the bandwidth of x (n) will be increased. Using this scheme, the sampling process is integrated into the synthesis filtration. It should be noted that any size of the synthesis filter bank can be utilized, resulting in different sampling rates for the output signal.

Referring to Fig. 3, consider the subband signals from a 16-channel analysis filter bank. The input signal x (n) has frequency content up to the Nyqvist frequency (f_c= FF2). In the first iteration, the 16 sub-bands are extended to 23 sub-bands, and frequency translation according to Eq. (3) is used with the following parameters: M = 16, S = 7 and P = 1. This illustration is illustrated by the repatching of subband from point a to b of the figure. In the next iteration, the 23 subbands are extended to 28 subbands, and Eq. (3) is utilized with the new parameters: M = 23, S = 5 and P = 3. This operation is illustrated by the repatching of subband from point b to c. The subbands thus created can then be synthesized using a 28-channel filter bank. This would create a critically sampled output with the sampling frequency 28 / 16f_s = 1.75f_s. The subband signals could also be synthesized using a 32-channel filter bank, where the four top channels are fed with zeros, illustrated by dashed lines in the figure, resulting in an output with the sampling frequency 2f_s.

Using the same analysis filter bank and an input with the same frequency content, Fig. 4 illustrates repatching using frequency folding according to Eq. (4) in two iterations.

In the first iteration, M = 16, S = 8 and P = -7, and the 16 subbands are increased to 24.1, the second iteration is M = 24, S = 8 and P = -7, and the number of subbands is increased from 24 to 32. The subband is synthesized with a 32-channel filter bank. In the output signal, sampled at frequency 2f_s, this repatch results in two reconstructed frequency bands - a band resulting from repeating the subband signal to channels 16 to 23, which is a weighted version of the bandpass signal extracted through channels 8 to 15, and a band resulting from repatching to channels 24 to 31, which is a translated version of the same bandpass signal.

Skvddsband in high frequency reconstruction

Detectable dissonance can occur in the translation or folding process through interference from neighboring bands, i.e. interference between parts in the vicinity of the transition area between moments of translated bands and the low band. This type of dissonance is more common in harmonically rich, multi-tuned program material. In order to reduce dissonance, protective bands are inserted and preferably consist of small frequency bands with zero energy, ie the transition range between the low band signal and replicated spectral band is filtered using a band stop or notch filter. Reduced perceptive deterioration is achieved if dissonant reduction with the use of protective bands is carried out. The bandwidth of the protective bands should preferably be about 0.5 Bark. If less, then dissonance may arise, and if broader, so can

523 883 comb filter-like sounds emerge.

In the case of filter bank-based translation or folding, protective bands can be inserted and preferably consist of one or more subband signals set to zero. Use of protective straps changes Eq. (3) to^vM + D + k (ⁿ) = ^eM + D + k (ⁿ)^vMS-P + k (ⁿ(5) and Eq. (4) to^VM + D + k (ⁿ) = ^eM + D + k (ⁿ) ^v* MPSk («) · (6)

D is a small integer and represents the number of filter bank channels utilized as protective bands. Now P + S + D should be an even integer in Eq. (5) and an odd integer in Eq. (6). P occupies the same value as before. Fig. 5 shows repatching of a 32-channel filter bank using Eq. (5). The input signal has frequency content up to f_c = 5 / 16f_s, which makes Af = 20 in the first iteration. The number of source channels is selected as S = 4 and P = 2. In addition, D should preferably be selected such that the bandwidth of the protective bands is 0.5 Bark. Here D is equal to 2, which makes the protective bands // 32 Hz wide. In the second iteration, the parameters selected as Af = 26, S = 4 and P = 0.1 figure are illustrated by the protective bands through the sub-bands with the dashed connections.

In order to make the spectral envelope continuous, the protective bands against dissonance can be partially reconstructed using an arbitrary white noise signal, ie the sub bands are fed with white noise instead of being zero. The preferred method utilizes the Adaptive Noisefloor Addition (ANA) as described in PCT patent application [SE00 / 00159]. This method estimates the interference floor of the original signal's high band and adds synthetic noise in a well-defined way to the recreate high band in the decoder.

Practical implementations

The present invention can be implemented in various systems for storing or transmitting audio signals using arbitrary codecs. Fig. 1 shows the decoder for an audio coding system. The demultiplexer 101 separates envelope data and other HFR-related control signals from the bitstream and feeds the relevant portion to the arbitrary low band decoder 102. The low band decoder produces a digital signal which is fed to the analysis filter bank 104. Envelope data is decoded in envelope decoder 103, and the resulting spectral envelope information is fed together with subband samples from the analysis filter bank to the integrated translator or fold and envelope adjusting filter bank unit 105. This unit translates or folds the low band signal, to form a broadband signal, to transmit a broadband signal. envelope. They dealt

523 The 883 subband samples are then fed to the synthesis filter bank 106, which may be of a different size than the analysis filter bank. The digital broadband output 107 is finally converted into an analog output signal.

The above described embodiments are merely illustrative of the principles of the present invention to improve High Frequency Reconstruction (HFR) technology utilizing filter bank based frequency translation or folding. It will be appreciated that modifications and variations of the embodiments and details described herein will be apparent to those skilled in the art. The intention is therefore to be limited only by the scope of protection for the associated claims, and not by the specific details presented here by the description and explanations of embodiments.

523 883