Encoder apparatus, decoder apparatus, methods thereof and associated audio system

[0001]

The present invention relates to a method and apparatus for processing a stereo signal obtained from an encoder, which encoder encodes the N-channel audio signal to the left and right signals and spatial parameters. The invention also relates to a coding device comprising such an encoder and such a device.

[0002]

The present invention also relates to a method and apparatus for processing a stereo signal obtained by means of this method and this apparatus for processing a stereo signal obtained from an encoder. The invention also relates to a decoder comprising such a device for processing a stereo signal.

[0003]

The present invention also relates to an audio system comprising such a coding device and a decoding device.

[0004]

Stereo reproduction of music, for example in a home environment, has been widespread for a long time. In 1970, some experiments were carried out with the reproduction of four channels using home music equipment.

[0005]

In larger rooms like cinema halls, multi-channel sound reproduction existed through _® long period of time. The Dolby Digital system and other systems have been expanded to provide a realistic and effective experience when reproducing sound in large rooms.

[0006]

Such multi-channel systems have been introduced to home cinema and are gaining a lot of interest. That is why systems with five full-range channels and one partially-range or low frequency effects (LFE) channel, so-called 5.1 systems, are common on the market today. There are also other systems such as 2.1, 4.1, 7.1 and even 8.1.

[0007]

With the introduction of SACD and DVD, the reproduction of multi-channel sound is gaining further interest. Many consumers already have the option of multi-channel playback in their homes, and multi-channel source material is becoming popular.

[0008]

Due to the growing popularity of multi-channel material, efficient coding of multi-channel material is becoming increasingly important, which is also recognized by standardization bodies such as MPEG.

[0009]

Previously known encoders often did not use effective methods to encode multi-channel audio. Input channels can be mainly coded individually (probably after matrixing), thus requiring a high transmission speed due to the large number of channels.

[0010]

However, the multi-channel audio encoder can generate a 2-channel downmix, which is compatible with 2-channel reproduction systems, while allowing high quality multi-channel reconstruction on the decoder side. The high quality of the reconstruction is controlled by the transmitted P parameters that control the stereo mixing process to multiple channels. These parameters contain information describing, among others coefficients of the front signal relative to the spatial signal that is present in the down-mix two-channel signal. Using this approach, the decoder can control the amount of the front signal relative to the spatial signal in the mixing process. In other words, the parameters describe the important spatial properties of the sound field that were present in the original multi-channel signal, but which are lost in the stereo mix due to the downmixing process.

[0011]

An example of a multi-channel encoder and decoder is disclosed in patent application filed in PCT procedure WO2004 / 008805.

[0012]

The present invention relates to the possibility of using this parameterized spatial information to use a dependent parameter, preferably invert, to be converted into a 2-channel downmix to enhance the mixing process, such as the perceived quality of their spatial properties.

[0013]

The object of the present invention is to allow processing of the processed downmix after coding based on parameters as determined in the multi-channel encoder and still maintain the possibility of multi-channel decoding without affecting subsequent processing.

[0014]

This object has been achieved by means of a method and apparatus for processing a stereo signal obtained from an encoder which encodes the N-channel signal (N> 2) into left and right signals and spatial parameters. The method includes processing said left and right signal channels to provide a processed stereo signal. Processing is controlled depending on the spatial parameters mentioned. The general idea is to use spatial parameters obtained from the N-channel to the stereo encoder to control some subsequent algorithm processing. In this way, the stereo signal obtained from the encoder can be processed, e.g. to improve the spatial impression.

[0015]

In an embodiment of the invention, the processing is controlled by a first parameter for each input channel, i.e. for each of the left and right signals whose first parameter is dependent on the spatial parameters. The first parameter can be a function of time and / or frequency. In this way, the system may have a variable number of subsequent processing, of which the actual number of subsequent processing depends on the spatial parameters. Subsequent processing can be performed separately in different frequency bands. The encoder provides independent spatial parameters describing the spatial image for a set of frequency bands. In this case, the first parameter may depend on the frequency.

[0016]

In another embodiment of the invention, post-processing includes adding the first, second and third signals to obtain said channel processing signals.

The first signal includes the first input signal, i.e. the left or right signal, modified by the first transfer function, the second signal contains the first input signal modified by the second transfer function, and the third signal contains the second input signal, i.e. right or left signal, modified by the third function transfer. The second transfer function may include said first parameter and first filter function. The first transfer function may include a second parameter, wherein the sum of said first parameter and the second parameter may be equal. The third transfer function may include said first parameter of the second input signal and a second filter function.

[0017]

Filter functions can be unchanged over time.

[0018]

In one particular embodiment, the signals can be described by the equation:

<img file="PL1735779T3_D0001.tif" />

where "a" is constant.

[0019]

Using this representation, the filtering effect of the filter functions H1, H2, H3 and H4 is different by changing the parameters in and wr. If both parameters have zero values, the processed L0w, R0w signals are essentially equal to the pair of stereo input L0, R0. On the other hand, if the parameters are +1, the subsequent processing of stereo L0w, R0w pairs is fully processed by the filter functions H1, H2, H3 and H4. The invention allows controlling the actual amount of filtering, i.e. parameter values ​​w1 and wr through spatial parameters P.

[0020]

According to an embodiment, the filter functions and parameters are selected such that the transfer function matrix is ​​inverted. This makes reconstruction of the original stereo signal possible.

[0021]

In another aspect of the invention, the invention includes a device for processing a stereo signal in accordance with the above-mentioned methods, and an encoder device comprising such a device.

[0022]

In another aspect of the invention there is provided a method and apparatus for inverting processing in accordance with the above-mentioned methods, and a decoding device comprising such an inverting device.

[0023]

In yet another aspect of the invention there is provided an audio system comprising such an encoder device and such a decoder device.

[0024]

Further objects, features and advantages of the invention will be apparent from the following detailed description of the invention with reference to examples of its embodiment and with reference to the accompanying drawings, in which:

Fig. 1 is a block diagram of an audio system encoder / decoder, including post-processing and inverse processing after post-processing in accordance with the present invention.

Fig. 2 shows a detailed block diagram of an embodiment of the device for post-processing the stereo signal obtained from the multi-channel encoder.

Fig. 3 shows a block diagram of another embodiment of the device for post-processing the stereo signal obtained from a multi-channel decoder.

Fig. 4 shows a block diagram of an embodiment for inverse processing after post-processing a stereo signal, including left and right signals.

[0025]

Fig. 1 is a block diagram of an encoder / decoder system in which the invention is to be used. In an audio system 1, an N-channel audio signal is provided to encoder 2, where

N is an integer that is greater than 2. Encoder 2 converts the N-channel audio signal to L0 and R0 signals and P decoder parametric information, with which the decoder can decode information and estimate the original N-channel signals that will be played by the decoder. The spatial set of parameters P is most preferably dependent on time and / or frequency. N-channel signals can be 5.1 system signals, consisting of a central channel, two front channels, two surround sound channels and an LFE channel.

[0026]

The pair of encoder stereo signals L0 and R0 and the spatial information of the P decoder are transmitted to the user in an appropriate manner, e.g. via CD, DVD, VHS Hi-Fi, teletransmission, laser disk, DBS, digital cable, Internet or any other transmission or system distribution line indicated by circle line 4 in FIG. 1. Because the left and right signals are transmitted, the system is compatible with the overwhelming number of receiving devices that can play stereo signals. If the receiver includes a decoder, the decoder can decode N-channel signals and provide an estimate thereof based on information in a pair of stereo signals L0 and R0 as well as information signals of the spatial decoder or P. spatial parameters.

[0027]

However, due to the reduction in the number of reproduced signals, stereo signals contain little spatial information compared to N-channel signals or other properties that may be desirable in some situations. Thus, according to the present invention, there is provided a postprocessor 5 that processes the stereo signal before transmission / distribution to the receiver. Post-processing can be position-dependent "adding" bass or reverb, or deleting vocals (karaoke with vocals in the center of the channel).

[0028]

Other examples of post-processing are stereo base extension, which can be performed by using knowledge of spatial signal composition, such as front / rear, because the contribution of individual input signals is known from the P decoder information signals. Basically, stereo extension can already be used in the encoder , but it is not generally reversible, because only two signals are available in the decoder, instead ofN, inverting is generally not possible. But in addition to stereo widening, other post-processing techniques are also possible for individual multi-channel contributions.

[0029]

According to the invention, the post-processing signals are transmitted to the receiver as shown in circle 6 in Fig. 1. The innovative device for processing the stereo signal obtained from the encoder comprises a postprocessor 5. The coding device according to the present invention consists of encoder 2 and postprocessor 5.

[0030]

The received signal can be used directly, e.g. when the receiver does not contain a multi-channel decoder. This can take place in a computer receiving signal 6 via the Internet or in a receiver equipped with only two columns. This kind of received signal is perceived as a high quality signal because the spatial impression or other features as specified in this processing by the encoder and postprocessor have been improved.

[0031]

If the signal should be used for decoding in conventional N-channel decoders 3, it must first be subjected to inverse post-processing by inverse postprocessor 7 to reproduce the original stereo L0 and R0 signal pairs, which together with the decoder information or spatial parameters P produce the estimated signal Nkanałowy. According to the invention, such reconstruction is possible on a multi-channel mix when the reconstruction almost does not concern post-processing. Also, post-processing at the decoder is possible for stereo playback as part of the user selection, without having to specify the multi-channel signal first. The innovative stereo signal processing device including left and right signals contains an inverse postprocessor 7. The decoding device according to the present invention comprises a decoder 3 and an inverse postprocessor 7.

[0032]

Without downstream processing, the downmix is ​​comparable to a standard ITU downmix. The method of the invention, however, can significantly improve the downmix.

[0033]

The method according to the invention is able to determine in the downmix the influence of the original channels in a multi-channel mix by means of a fixed spatial parameter P in the encoder. In this way, post-processing can be applied to individual channels of a multi-channel mix, e.g. widening of the stereo base of the rear channels, while other channels are not changed. Post-processing does not affect final multi-channel reconstruction if post-processing is reversible. It can also be used for improved stereo playback without the need for initial reconstruction of the multi-channel mix.

[0034]

This method differs from existing post-processing techniques in that it uses the information of the original multi-channel mix, i.e. the established spatial parameters P.

[0035]

Encoder 2 works as follows:

Let's take the N-channel audio signal as the input signal to encoder 2, in which z1 [n], z2 [n], ....

zN [n] describe discrete time domain curves from N channels. These N signals are divided using common segmentation, preferably using overlapping analysis windows. Then, each segment is transformed into a frequency domain using a complex transformation (e.g., FFT). However, complex filter structures may also be appropriate to obtain a time / frequency scale. This process results in segmentation, a representation of the sub-ranges of the input signals, which will be denoted by Z1 [k], Z2 [k], ZN [k], with "k" being the frequency index.

[0036]

From these N channels, 2 downmix channels are created as L0 [k] and R0 [k]. Each downmix channel is a linear combination of N input signals:

and₀[L] = e₍from_f[*] ł = J λ_ο[Α] = ΣΜ [Μ

II [0037]

Parameters a_and and e_and are selected such that a stereo signal consisting of L₀[k] and R₀[k] has a good stereo image. In the case of a 5-channel input signal including Lf, Rf, C, LS, and RS (for channels: front left, front right, center, left spatial, right spatial), the corresponding downmix can be obtained according to:

<img file="PL1735779T3_D0002.tif" />

<img file="PL1735779T3_D0003.tif" />

[0038]

L and R signals can be obtained using equations:

I [t] = i_/[i] + Ą [i] / 72

J? [*] = Ą [*] + Ą [*] / T2 [0039]

In addition, the spatial parameters P are extracted to enable the perceptual reconstruction of Lf, Rf, C, Ls and Rs signals from the L0 and R0 channels.

[0040]

In an embodiment, the set of parameters P includes inter-channel differences in intensity (IIDs) and values ​​of possible inter-channel cross-correlations (ICCs) between signal pairs (Lf, Ls) and (Rf, RS). IID and ICC between pairs Lf and Ls are obtained according to the equations:

<img file="PL1735779T3_D0004.tif" />

[0041]

Here, (*) means complex coupling. For other signal pairs, similar equations can be used. Thus, the IIDl parameter specifies the relative amount of power between the left front and left spatial channels, and the ICCl parameter describes the amount of mutual correlation between the left front and left spatial channels. These parameters basically describe the perception of the corresponding parameters between the front and back spatial channels.

[0042]

Parameterization of the amount of the center signal that is present in L0, R0 can be obtained by estimating two predictive parameters c1 and c2. These two predictive parameters determine the 2x3 matrix that controls the process of mixing the decoder from L0, R0 to L, C and R:

<img file="PL1735779T3_D0005.tif" />

[0043]

The implementation of the M mixing matrix is ​​described by:

<img file="PL1735779T3_D0006.tif" />

[0044]

In the above example, the parameter set P contains {c1, c2, IIDl, ICCl, IIDr, ICCr} for each time / frequency scale.

[0045]

On the resulting stereo signal pair (L0, R0), post-processing can be used in such a way that it mainly relates to the contribution of Zi [k], for example Ls and Rs in the stereo mix. Fig. 1 shows the position of this block in a codec.

[0046]

Fig. 2 is a detailed view of the postprocessor 5 of Fig. 1, according to an embodiment of the invention. The post-processing of the left L0w signal is the sum of three signals, namely the left L0 signal modified by HA transfer function, the left L0 signal modified by HB transfer function and the right R0 signal modified by HD transfer function. In the same way, post-processing the right signal R0w is the sum of three signals, namely the right R0 signal modified by the HF transfer function, the right R0 signal modified by the HE transfer function and the left L0 signal modified by the HC transfer function. The HA - HF transfer function can be implemented as FIR or IIR filters, or it can simply be a (complex) scale factor that can be frequency dependent. In addition, the HA transfer function may be multiplied with the second parameter (1-in) and the HB transfer function may contain the first wl with this parameter wl determining the amount of stereo stereo processing.

[0047]

This is shown in Fig. 3. The parameter wl determines the amount of post-processing L0 [k] and wr with R0 [k]. When wl is 0, then L0 [k] is unchanged, and when wl is 1, L0 [k] is changed as much as possible. The same applies to wr relative to R0 [k].

[0048]

The following equations apply to the post-processing of parameters wl and wr:

w, = fi (IID |, ICC |, cl, c2) in_r= f_r(IID_r, ICC_r, cl, c2) [0049]

The blocks H1, H2, H3 and H4 in Fig. 3 are filter functions that can be different types of filters, for example stereo widening filters, as shown below.

[0050]

The result is:

<img file="PL1735779T3_D0007.tif" />

, wherein:

<img file="PL1735779T3_D0008.tif" />

where a is any constant (e.g. +1) [0051]

If the H1, H2, H3 and H4 filter functions are selected properly, the H matrix transfer function can be inverse. Moreover, in order to enable the calculation of the inverse matrix in the decoder, the filter functions H1, H2, H3 and H4 and the parameters wl and wr should be recognized in the decoder. This is possible because wl and wr can be calculated from the transmitted parameters. In this way, the original stereo signal L0, R0 will be available again, which is necessary for decoding a multi-channel mix.

[0052]

Another option is to transfer the original stereo signal and apply post-processing in the decoder to enable better stereo playback without having to specify the multi-channel mix first.

[0053]

An embodiment of post-processing is described in detail below. However, the invention is not limited to specific details, but may vary within the scope of the invention as defined in the appended claims.

[0054]

In post-processing, the parameters or weights wl and wr are a function of the transmitted spatial parameters:

<img file="PL1735779T3_D0009.tif" />

[0055]

The f function is designed in such a way that w1 increases when the L0 signal contains more power from the left spatial signal compared to the front left or center signals. In a similar way, wr increases with increasing power ratio in the right spatial signal found in R0. wl and wr have been well described by:

», = FMMJD,) w, = / (^) / (// 0,)

<img file="PL1735779T3_D0010.tif" />

and

<img file="PL1735779T3_D0011.tif" />

[0056]

For the H1, H2, H3 and H4 filter functions, the following formula functions were then selected (in the z domain):

H, (z) = H &lt;(z) = 0.8 (1.0 + 0.2z '' + 0.2z '²)

H₂(z) = Hj (z) = 0.8 (-1 .Oz '- 0.2z'²).

[0057]

This invention can be integrated into a multi-channel audio encoder device, which creates a compatible stereo downmix. The overall scheme of such a multi-channel parametric audio encoder, which is amplified by a post-processing system as described above, can be described as follows:

- conversion of the multi-channel input signal to a frequency domain, either by segmentation and transformation or by applying a set of filters;

- extraction of spatial parameters P and frequency downmix generation;

- application of a frequency domain post-processing algorithm; Conversion of post-processing signals to a time domain;

- stereo coding using conventional coding techniques such as those specified in MPEG;

- multiplexing the stereo bit stream with the encoded P parameters to form the total output bit stream.

[0058]

The corresponding multi-channel decoder device (i.e., a decoder with integrated inverse post-processing) can be represented as follows:

- demultiplexing the bit stream parameter to obtain the P parameters and the encoded stereo signal;

- stereo signal decoding;

- conversion of the encoded stereo signal into a frequency domain;

- applying reverse post-processing based on P parameters;

- mixing a stereo signal to a multi-channel output signal based on P parameters;

- conversion of the multi-channel output signal to the time domain.

[0059]

Since post-processing and inverse post-processing are performed in the frequency domain, the filter functions H1 to H4 are preferably transformed or estimated in the frequency range by simple (real or complex values) scale factors that can be frequency dependent.

[0060]

Those skilled in the art may understand that one or more processing steps, as described above, may be combined as one processing step.

[0061]

Another application of the invention is to use post-processing to stereo only at the decoder side (i.e., without post-processing at the encoder side). Using this method, the decoder can generate an amplified stereo signal from an un-amplified stereo signal.

[0062]

Additional information can be obtained in the bit stream that indicates whether or not post-processing has been performed, and the functions of parameters f1, f2 and which filter functions H1, H2, H3, and H4 have been used, which allows reverse post-processing.

[0063]

The filter function can be described as a frequency domain multiplier. Since the parameters are available for individual frequency bands, the invention can be implemented as a simple, comprehensive improvement in place of filters that are used individually in different frequency bands. In this case, the L0w and R0w frequency bands are obtained by simple (2x2) matrix multiplication with the corresponding frequency bands from (L0, R0). The actual matrix entries are determined by the parameters and frequency representation with the H filter function, i.e. consisting of a gain constant over time H and variables controlled by the time / frequency parameters of gain wl and wr. Because the filters are scalar for each band, inversion is possible.

[0064]

Post-processing in the encoder can be described by the following matrix equation:

<img file="PL1735779T3_D0012.tif" />

where

Γ^Α“ ^{ft |}2l_i (^1_Wl) "⁺W ^ l W^HAnd a,% JL [0065]

This matrix equation is used for each frequency band. The H matrix contains all scalars. The use of scalars makes post-processing and inverse post-processing relatively easy.

[0066]

Parameters wl and wr are scalars and functions of the P parameter set. These 2 parameters determine the amount of post-processing from the input channels.

[0067]

Parameters H1 ... H4 are complex filter functions.

[0068]

Reversing this process can also be done by simply multiplying the matrix by frequency bands. The following equation for frequency bands is used:

<img file="PL1735779T3_D0013.tif" />

where

<img file="PL1735779T3_D0014.tif" />

Μ and

AJ ^ 11 ^ 23^-^12^21

AND

<img file="PL1735779T3_D0015.tif" />

[0069]

Matrix H.^-1 contains only scalars. Elements from H.^-1, k1 K4, are also functions of the parameter set P. When functions in the matrix H, h11 ...... h22, and parameters P are known in the decoder, then the post-processing can be inverted.

[0070]

A block diagram of an inverse postprocessor 3 that performs this type of inverse postprocessing is depicted in Fig. 4.

[0071]

This inversion is possible when the determinant of the matrix H is not zero. The determinant of H is equal to:

<img file="PL1735779T3_D0016.tif" />

[0072]

When the corresponding functions h11 h22 are selected, det (H) will be uneven zero, so the process is reversible.

[0073]

It is mentioned that the term "including" does not exclude other elements or stages, and the use of the singular does not exclude the use of a plurality of elements. The reference marks in the claims should not be considered as restricting the scope of protection included in the claims.

[0074]

Although the present invention has been described in connection with some embodiments, they are not intended to limit the invention to the specific form described herein. On the contrary, the scope of the present invention is limited only by the appended claims. In addition, although a feature may appear which can be described in connection with particular embodiments, one of ordinary skill in the art will know that the various features of the described embodiments can be combined in accordance with the present invention.