Multi-channel audio energy loss compensation

IS 2 292 147 T3

DESCRIPTION

Multi-channel reconstruction based on multi-parameterization.

Technical field

The present invention relates to the multichannel reconstruction of audio signals based on an available stereo signal and additional control data.

Background of the invention

Recent development in audio coding has made available the ability to recreate a multi-channel representation of an audio signal based on a stereo (or mono) signal and corresponding control data. These methods differ substantially from the old matrix-based solution such as Dolby Prologic in that additional control data is transmitted to control the recreation, also known as upmix (conversion of mono or stereo signal into surround signal), of surround channels based on on the transmitted mono or stereo channels.

Thus, parametric multichannel audio decoders reconstruct N channels based on M transmitted channels, where N> M, and the additional control data. The additional control data represents a significantly lower data rate than transmitting the additional NM channels, making the encoding very efficient while at the same time ensuring compatibility with both M-channel and N-channel devices.

These parametric surround encoding methods typically comprise a parameterization of the surround signal based on IID (Inter channel Intensity Difference) and ICC (Inter Channel Coherence). These parameters describe the correlation and power relationships between pairs of channels in the upmix process. Additional parameters also used in the prior art comprise prediction parameters used to predict intermediate or output channels during the upmix procedure.

One of the most attractive uses of the prediction-based method as described in the prior art is for a system that recreates the 5.1 channel from two transmitted channels. In this configuration a stereo transmission is available on the decoder side, which is a downmix (conversion of surround signal to mono or stereo signal) of the original 5.1 multichannel signal. In this context, it is particularly interesting to be able to extract the central channel of the stereo signal as precisely as possible, since a downmix is ​​normally carried out on the central channel with both the left and right downmix channels. This is done by estimating two prediction coefficients that describe the amount of each of the two transmitted channels used to construct the center channel. These parameters are estimated for different frequency regions in a similar way to the IID and ICC parameters above.

However, since the prediction parameters do not describe a power relationship of two signals, but are based on matching the waveform in a least squared error sense, the method becomes inherently sensitive to any modification of the stereo waveform after calculation of the prediction parameters.

Further developments in audio coding over the last few years have introduced high frequency reconstruction methods as a very useful tool in low bit rate audio codecs. An example is SBR (Spectral Band Replication) [WO 98/57436], which is used in standardized MPEG codecs such as MPEG-4 high performance AAC. It is common for these methods that they recreate the high frequencies on the decoder side from a narrow band signal encoded by the underlying core codec and a small amount of additional guide information. Similar to the case of parametric reconstruction of multichannel signals based on one or two channels, the amount of control data required to recreate the unknown signal components (in the case of SBR, the high frequencies), is significantly smaller than the amount of data that would be required to encode the entire signal with a waveform codec.

However, it should be understood that the recreated high-band signal is in its perception equal to the original high-band signal, while the actual waveform differs significantly. Furthermore, for waveform encoders encoding stereo signals at low bit rate stereo preprocessing is normally used, which means that limiting is made on the side signal of the mid / side representation of the stereo signal.

When multichannel representation is desired based on a stereo codec signal using MPEG-4 high-efficiency AAC or any other codec that uses high-frequency reconstruction techniques, these and other aspects of the codec used to encode the stereo signal over the line must be considered. that has been downmixed.

IS 2 292 147 T3

The article “Compatibility matrixing of multi-channel bit-rate-reduced audio signals” (Ten Kate WR Th, Journal of the Audio Engineering Society, New York, United States, vol. 44, n ° 12, December 1996, pages 1104 to 1119) discloses a variable matrix: in each time frame the optimal matrix for the minimum bits required is determined.

Still further, it is common for a recording available as a multichannel audio signal to have a dedicated stereo mix available, which is not an automated downmix version of the multichannel signal. This is commonly referred to as an "artistic downmix". This downmix cannot be expressed as a linear combination of multichannel signals.

Summary of the invention

It is an object of the present invention to provide a multichannel downmix / encoder or upmix / decoder concept, which results in a better quality of the reconstructed multichannel output.

This objective is achieved by a multichannel synthesizer according to claim 1, an encoder for processing a multichannel input signal according to claim 18, a method of generating at least three output channels according to claim 31, a processing method according to claim 32 or a multi-channel encoded signal according to claim 33.

The present invention is based on the finding that different parametric representations for different frequency or time portions of a signal are useful to obtain a coding or decoding situation that adapts to different situations. These situations can result from encoder events such as performing an SBR information calculation or an energy measurement calculation used for power loss compensation or any other event. Other situations that may result in different parametric representations may include upmix quality, downmix bit rate, computational efficiency on the encoder side or on the decoder side, or for example the power consumption of for example battery powered devices, so that, for a certain subband or frame, the first parameterization is better than the second parameterization. Naturally, the objective function can also be a combination of different individual objectives / events as discussed above.

Preferably, a parametric representation includes parameters for a predictive upmix based on modifying the waveform of the downmixed multichannel signal. This includes when the downmixed signal is encoded by a codec that performs stereo preprocessing, high-frequency reconstruction, and other encoding schemes that significantly modify the waveform. Furthermore, the invention addresses the problem that arises when predictive upmix techniques are used for an artistic downmix, that is, a downmix signal that is not automatically derived from the multichannel signal.

Preferably, the present invention comprises the following features:

- estimation of the prediction parameters based on the modified waveform rather than the downmixed waveform;

- use of prediction-based methods only in frequency ranges where it is advantageous;

- correction of energy loss and non-precise correlation between channels introduced in the prediction-based upmix procedure.

Brief description of the drawings

The present invention will now be described by way of illustrative examples, which do not limit the scope of the invention, with reference to the accompanying drawings, in which:

Figure 1 illustrates a three-channel prediction-based reconstruction from two channels; Figure 2 illustrates a predictive upmix with power compensation; Figure 3 illustrates an energy offset in the predictive upmix;

Figure 4 illustrates a prediction parameter estimator on the encoder side with power compensation of the downmix signal;

Figure 5 illustrates a predictive upmix with correlation reconstruction;

Figure 6 illustrates a mixing module for mixing the decorrelated signal with the upmixed signal in the upmix with correlation reconstruction;

Figure 7 illustrates an alternative mixing module for mixing the decorrelated signal with the upmixed signal in the upmix with correlation reconstruction;

Figure 8 illustrates an estimation of prediction parameters on the encoder side; Figure 9 illustrates an estimation of prediction parameters on the encoder side; Figure 10 illustrates an inventive multi-parameter scenario;

Figure 11 illustrates an upmix device;

Figure 12 illustrates an energy graph showing the result of an upmix introducing energy loss and the preferred offset;

Figure 13 is a table of energy compensation methods;

Figure 14a shows a schematic diagram of a preferred multichannel encoder;

Figure 14b shows a flow chart of the method performed by the device of Figure 14a;

Figure 15a shows a multichannel encoder having spectral band replication functionality to generate a different parameterization compared to the device of Figure 14a;

Figure 15b shows a table illustration of frequency selective generation and transmission of parametric data; and Figure 16a shows a decoder illustrating the calculation of upmix matrix coefficients; Figure 16b shows a detailed description of a parameter calculation for the predictive upmix; Figure 17 shows a transmitter and a receiver of a transmission system; and Fig. 18 shows an audio recoder having an encoder and an audio player having a decoder.

Description of the preferred embodiments

The embodiments described below are merely illustrative of the principles of the present invention. It is understood that modifications and variations to the arrangements and details described herein will be apparent to others skilled in the art. Therefore, it is intended to be limited only by the scope of the impending patent claims and not by the specific details presented by way of description and explanation of the embodiments herein.

It is emphasized that the application, upmixing, downmixing, parameter calculation and any other subsequent action can be performed on a selective basis for frequency band, that is, for subbands in a filter bank.

In order to explain the advantages of the present invention, a more detailed description of a predictive upmix as known from the prior art is first given. Suppose a three channel upmix based on two downmix channels, as shown in Figure 1, in which 101 represents the left original channel, 102 represents the center original channel, 103 represents the right original channel, 104 represents the module. parameter extraction and downmix on the encoder side, 105 and 106 represent prediction parameters, 107 represents the channel on which the left downmix has been performed, 108 represents the channel on which the right downmix has been performed, 109 represents the predictive upmix module, and 110, 111 and 112 represent, respectively, the reconstructed left, center and right channel.

Assume the following definitions where X is a 3 x L matrix containing the three signal segments l (k), r (k), c (k), k = 0, ..., L-1 as rows.

Likewise, the two signals l₀(k), r₀(k) on which the downmix has been made form the rows of X₀. The downmix process is described by

<img file="ES2292147T3_D0001.tif" />

ES 2 292 147 T3 where the downmix matrix is ​​defined by

<img file="ES2292147T3_D0002.tif" />

A preferred choice of downmix matrix is

D.-P ° “1 '(or 1« J (3) which means that the signal l₀(k) from left downmix will contain only l (k) and ac (k), and r₀(k) will contain only r (k) and ac (k). This downmix matrix is ​​preferred since it allocates an equal amount of the center channel to the left and right downmix, and since it does not allocate any of the original right channel to the left downmix or vice versa.

The upmix is ​​defined by

X = CX, (4) where C is a 3 x 2 upmix matrix.

Predictive upmix as known from the prior art is based on the idea of ​​solving the overdetermined system

CXo = X (5) for C in the least squares sense. This leads to the normal equations

CXeX ^ XX;

(6)

Multiplying (6) to the left by D gives DCX₀X *₀= X₀X *₀, which, in the generic case where X₀X₀* = DXX * D * is non-singular, it implies

<img file="ES2292147T3_D0003.tif" />

where i_n denotes the identity matrix n. This relationship reduces parameter space C to dimension two.

Given the above, the matrix C =

<img file="ES2292147T3_D0004.tif" />

upmix can be fully defined on the decoder side if downmix matrix D is known, and two elements of matrix C are transmitted, for example c_B and c₂₂. The residual signals (prediction error) are given by

<img file="ES2292147T3_D0005.tif" />

Multiplying left by D leads to

DX, = (D-DCD) X = 0 (9) due to (7). From this it follows that there is a signal x_r row vector 1 x L such that

<img file="ES2292147T3_D0006.tif" />

ES 2 292 147 T3 where v is a 3x1 unit vector spanning the kernel (null space) of D. For example, in the case of downmix (3), you can use

-to

<img file="ES2292147T3_D0007.tif" />

In general, when v = [Vi, V_r, V_C]^T, and the X l (k), r (k), c (k) this simply means that, up to a weighting factor, the residual signal is common for all three channels.

r {k) = r (k) + v, x_r(k) (12) c (*) = c (¿) + v_andx, (?)

Due to the orthogonality principle, the x_r(k) residual is orthogonal to the three predicted signals l (k), t (k), £ (k).

Solved Problems and Improvements Obtained by Preferred Embodiments of the Present Invention

Obviously the following problems arise when using upmix based on the prediction according to the prior art as explained above:

• The method is based on matching the waveform in one sense of the least mean square errors, which does not work for systems where the waveform of the downmixed signals is not maintained.

• The method does not provide the correct correlation structure between the reconstructed channels (as will be discussed later).

• The method does not rebuild the correct amount of energy in the rebuilt channels.

Power compensation

As mentioned above, one of the problems with prediction-based multichannel reconstruction is that the prediction error corresponds to a loss of energy from the three reconstructed channels. The theory for this energy loss and a solution as taught by the preferred embodiments will be discussed later. First, the theoretical analysis is performed, and subsequently a preferred embodiment of the present invention is given according to the theory discussed below.

Let E, E, and E_r the sum of the energies of the original signals at X, the predicted signals at X, and the prediction error signals at Xr, respectively. From the orthogonality, it follows that

<img file="ES2292147T3_D0008.tif" />

_AND

The total prediction gain can be defined as ρ = - but from now on it will be more convenient to consider the parameter

Er

<img file="ES2292147T3_D0009.tif" />

Therefore, ρ² e [0,1] measures the total relative energy of the predictive upmix.

Given this ρ, it is possible to readjust each channel by applying a compensation gain, z_g(k) = g_ZZ (k), such that || Z_g||² = || z ||² for z = l, r, c. Specifically, the target energy is given by (12),

<img file="ES2292147T3_D0010.tif" />

IS 2 292 147 T3

<img file="ES2292147T3_D0011.tif" />

(16) for what needs to be solved

In this case, since v is a unit vector,

<img file="ES2292147T3_D0012.tif" />

(17) and from the definition (14) of ρ and (13) it follows that

<img file="ES2292147T3_D0013.tif" />

P (18)

Putting all this together, you get to profit

<img file="ES2292147T3_D0014.tif" />

(19)

It is evident that with this method, in addition to transmitting ρ, the energy distribution of the decoded channels has to be calculated in the decoder. Furthermore only the energies are reconstructed correctly, while the off-diagonal correlation structure is ignored.

It is possible to derive a gain value that guarantees that the total energy is conserved, whereas the energy of the individual channels is not guaranteed to be correct. A common gain for all g channels_z = g which guarantees that the total energy is conserved is derived through the definition equation g²E = E. That is,

<img file="ES2292147T3_D0015.tif" />

(20)

By linearity, this gain can be applied at the encoder to downmixed signals, so that no additional parameters have to be transmitted.

Figure 2 depicts a preferred embodiment of the present invention that recreates the three channels while maintaining the correct energy of the output channels. The signs l₀ yr₀ those that have been downmixed are entered in the upmix module 201, together with the cyc parameters₂ prediction. The upmix module recreates the upmix matrix C based on the knowledge about the downmix matrix D and the received prediction parameters. The three output channels of 201 are entered in 202 along with the adjustment parameter ρ. All three channels are gain adjusted as a function of the transmitted ρ parameter and the energy corrected channels are output.

A more detailed embodiment of the adjustment module 202 is shown in Figure 3. The three channels on which upmix has been carried out are introduced in the adjustment module 304, as well as in the module 301, 302 and 303 respectively. The energy estimation modules 301-303 estimate the energy of the three upmixed signals and input the measured energy into the adjustment module 304. The control signal ρ (representing the prediction gain) received from the encoder is also input to 304. The tuning module implements equation (19) as discussed above.

In an alternative implementation of the present invention the power correction can be done on the encoder side. Figure 4 illustrates an implementation of the encoder in which the signals l₀ 107 yr₀ 108 that have been downmixed are adjusted in gain by means of 401 and 402 according to a gain value calculated by 403. The gain value is derived according to equation (20) above. As discussed above this is an advantage of this embodiment of the present invention, since it is not necessary to calculate the energy of the three recreated channels from the predictive upmix. However, this only guarantees that the total energy of the three recreated channels is correct. It does not guarantee that the energy of the individual channels is correct.

IS 2 292 147 T3

A preferred example for a downmix matrix corresponding to equation (3) is indicated below the downmix element in figure 4. However, the downmix element can apply any general downmix matrix as discussed in equation ( 2).

As will be discussed later, for the present case of a downmix element that presents, as input, three channels, and, that presents, as output, two channels, at least two parameters ci, c are required₂ additional upmix. When a downmix matrix D is variable or not fully known to a decoder, additional information about the used downmix also has to be transmitted from the encoder side to the decoder side, in addition to parameters 105 and 106.

Correlation structure

One of the problems with the upmix procedure described by the prior art is that it does not reconstruct the correct correlation between the recreated channels. Therefore, as discussed above, the center channel is predicted as a linear combination of the left downmix channel and the right downmix channel, and the left and right channels are reconstructed by subtracting the predicted center channel from the left downmix channels. and right. Clearly, the prediction error will result in remnants of the original center channel in the predicted left and right channel. This implies that the correlations between the three channels are not the same for the reconstructed channels as they were for the original three channels.

A preferred embodiment teaches that the three predicted channels should be combined with decorrelated signals according to the measured prediction error.

The basic theory to get the correct correlation structure is discussed below. The special remainder structure can be used to reconstruct the complete 3 x 3 correlation structure XX * by substituting an x ​​signal_d uncorrelated by the rest in the decoder.

First, note that the normal equations (6) lead to X_rX *₀ = 0 so

X, X '= 0, xx' = o

Therefore, since X = X + X_r,

XX '= XX * + χχ = χχ · + vv £, where (10) and (17) were applied for the last equality.

(21) (22)

Let x_d a decoded signal of all signals 1, 1, C decoded in such a way that Xx *_r = 0. The enhanced signal

Y = X + vx_¿ (23) then has the correlation matrix

YY '= XX' + w '|| x, f (24)

In order to fully reproduce the original correlation matrix (22), it is sufficient that

<img file="ES2292147T3_D0016.tif" />

(25)

<img file="ES2292147T3_D0017.tif" />

for a gain γ then it should be considered that

ES 2 292 147 T3 (26)

This gain can be calculated at the encoder. However, if the parameter p2 e [0,1] better defined from (14) is to be used, the estimation of E y + r_Q it has to be done in the decoder. In view of this, a more attractive alternative is to generate x_d using three decorrelators ^ = Ζ · (4, {/) + 4₂Η + 4, {έ}) (26a) since then || x_d||² = γ² E, so (25) is satisfied by choosing

<img file="ES2292147T3_D0018.tif" />

Figure 5 illustrates an embodiment of the present invention for three-channel predictive upmix from two downmix channels, while maintaining the correct correlation structure between the channels. In Figure 5 the modules 109, 110, 111 and 112 are the same as in Figure 1 and will not be explained further at this time. The three signals on which the upmix has been carried out, which are the output of 109, are introduced into the decorelation modules 501, 502 and 503. These generate mutually uncorrelated signals. The uncorrelated signals are added and fed into mixing modules 504, 505, and 506, where they are mixed with the 109 output. The mix of signals on which predictive upmix has been performed with the uncorrelated versions of the same is a essential feature of the present invention. An embodiment of mixing modules 504, 505, and 506 is shown in Figure 6. In this embodiment of the invention the level of the decorrelated signal is adjusted by 601 based on the control signal γ. The decorrelated signal is subsequently added to the signal that has been predictively upmixed at 602.

A third preferred embodiment uses decorrelators 501, 502, 503 for the upmixed channels. A de-correlated signal can also be generated by a de-correlator 501 ', which receives, as an input signal, the downmix channel or even all downmix channels. Furthermore, in case of more than one downmix channel, as shown in figure 5, the decorelation signal can also be generated by separate decorelators for channel 1₀ left base and r channel₀ right base and combining the output of these separate de-correlators. This possibility is substantially the same as the possibility shown in figure 5, but presents a difference with respect to the possibility shown in figure 5 in that the base channels are used before performing upmix.

Furthermore, it is commented in connection with figure 5 that the mixing modules 504, 505 and 506 not only receive the factor γ, which is the same for the three channels, since this factor only depends on the measure ρ of energy, but also they also receive the channel-specific factor v1, vc and vr, which is determined as discussed in connection with equations (10) and (11). However, this parameter does not have to be transmitted from an encoder to a decoder when the decoder knows the down-mix used in the encoder. Instead, these parameters in matrix v as shown in equations (10) and (11) are preferably preprogrammed in mixing modules 504, 505, and 506 such that these weighting factors do not have to be transmitted. channel-specific (but can of course be streamed when required).

In FIG. 6, it is shown that the weighting device 601 adjusts the energy of the decorrelated signal using the product of γ and the channel-specific downmix-dependent parameter vz, where z means 1, r or c. In this context, it is observed that equation (26a) guarantees that the energy of x_d it is equal to the sum energy of the left, right and central channels on which the upmix has been performed in a predictive way. Therefore, device 601 can be implemented simply as a scaling element using the scaling factor GI. However, when the decorrelated signal is alternately generated, the mixing module 504, 505, 506 has to perform an absolute energy adjustment of the decorrelated signal added by the addition device 602 such that the signal energy added in adder 602 is equal to the energy of the residual signal, for example, the energy that is lost by the predictive upmix that does not conserve energy.

With respect to the channel-specific downmix dependent parameter vz, the same observations also apply as discussed above with respect to figure 6 for the embodiment of figure 7.

IS 2 292 147 T3

Furthermore, it is to be noted at this time that the embodiment of Figure 6 and Figure 7 are based on the recognition that at least a portion of the energy loss in the predictive upmix is ​​added using a decorrelation signal. In order to have correct signal energies and correct parts of the signal component of the dry signal (uncorrelated) signal and the "wet" (uncorrelated) signal component, it has to be ensured that the "dry" signal input to the Mix module 504 has not been previously scaled. When, for example, the base channels have been previously corrected on the decoder side (as shown in figure 4) then this previous correction of figure 4 has to be compensated by multiplying the channel by the measure ρ of energy (relative) before entering the channel into mixer box 504, 505 or 506. Additionally, the same procedure has to be performed when such a power correction has been performed on one side of the decoder prior to inputting the downmix channels into the upmix element 109 as shown in FIG. 5.

When only a part of the residual energy is to be covered by an uncorrelated signal, the pre-correction only has to be partially removed by pre-scaling the signal input in the mixing box 504, 505, 506 by a factor dependent on ρ, which however, it is closer to one than the factor ρ itself. Naturally, this partially compensating prescaling factor will depend on the signal input κ generated by the encoder at 605 in Figure 7. When such a partial scaling has to be performed, then the applied weighting factor in G₂ there's no need. Instead, then the branch from input 604 to adder 602 will be the same as in Figure 6.

Control the degree of decorrelation

A preferred embodiment of the invention teaches that the amount of decorrelation added to predicted upmixed signals can be controlled from the encoder, while still maintaining the correct output energy. This is because in an example of a typical "interview" of dry voice in the environment and center channel on the left and right channels, substituting the decorrelated signal for prediction error on the center channel may be undesirable.

According to a preferred embodiment of the present invention, an alternative mixing procedure to that shown in Figure 5 can be used. It will be shown later how, according to the present invention, the issues of total energy conservation and actual correlation reproduction can be separated and the amount of decorrelation can be separated. controlled by parameter K.

It will be assumed that a gain compensation (20) has been performed that conserves total energy on the downmixed signal, such that the decoded X / p signal is obtained first. From this, a decorrelated d signal is produced with the same total energy || d ||² = E / ρ², for example by using three de-correlators as in the previous section. The total upmix is ​​then defined according to

<img file="ES2292147T3_D0019.tif" />

P (29) where κ e [ρ, 1] is a transmitted parameter. The choice κ = 1 corresponds to the conservation of total energy without the addition of decorrelated signal and κ = ρ corresponds to the reproduction of the complete 3 x 3 correlation structure. It has

<img file="ES2292147T3_D0020.tif" />

(30) so the total energy is conserved for all κ e [ρ, 1], as can be seen by calculating the traces (sum of the diagonal values) of the matrices in (30). However, the correct individual energy is only obtained for κ = ρ.

Figure 7 illustrates an embodiment of the mixing modules 504, 505 and 506 of Figure 5 according to the theory discussed above. In this alternative of the mixing modules, the control parameter γ is entered in 702 and 701. The gain factor used for 702 corresponds to κ according to equation (29) above, and the gain factor used for 701 corresponds to V1 - κ² according to equation (29) above.

The above-described embodiment of the present invention allows the system to employ a detection mechanism on the encoder side, which estimates the amount of decorrelation to be added in the prediction-based upmix. The implementation described in FIG. 7 will add the indicated amount of decorrelated signal, and apply the energy correction such that the total energy of the three channels is correct, while still being able to substitute an arbitrary amount of the prediction error for decorrelated signal.

This means that for an example with three ambient signals, for example a piece of classical music, with a lot of ambient noise, the encoder can detect the lack of a “dry” center channel, and allow the decoder to substitute the complete prediction error for signal decorrelated, thus recreating the sound environment of the three channels in a way that would not be possible with prior art prediction-based methods alone.

IS 2 292 147 T3

In addition, for a signal with a dry center channel, for example speech in the center channel and ambient sounds in the left and right channels, the encoder detects that substituting the prediction error for a decorrelated signal is not psychoacoustically correct and instead allows the The decoder adjusts the levels of the three reconstructed channels such that the energy of the three channels is correct. Obviously the above extreme examples represent two possible consequences of the invention. It is not limited to covering only the extreme cases discussed in the previous examples.

Adapt prediction coefficients to modified waveforms

As previously commented, the prediction parameters are estimated by minimizing the root mean square error given the three original X channels and a downmix matrix D. However, in many situations it cannot be relied upon that the downmixed signal can be described as a downmix matrix D multiplied by a matrix X describing the original multichannel signal.

An obvious example for this is when a so-called “artistic downmix” is used, that is, the two-channel downmix cannot be described as a linear combination of the multichannel signal. Another example is when the downmixed signal is encoded using a perceptual audio codec that uses stereo preprocessing or other tools for improved encoding efficiency. It is commonly known in the prior art that many perceptual audio codecs are based on mid / side stereo coding, in which the side signal is attenuated under a limited bit rate condition, producing an output that presents a more stereo image. narrow than that of the signal used for encoding.

Figure 8 shows a preferred embodiment of the present invention in which extracting parameters on the encoder side other than the multichannel signal also has access to the modified downmix signal. The modified downmix is ​​generated in this case by 801. If only two parameters of the matrix C are transmitted, a knowledge of the matrix D is needed on the decoder side in order to be able to perform the upmix, and to achieve the minimum mean square error for all the channels on which it has been performed. upmix. However, the present embodiment teaches that the downmixed signals l0 and r0 on the encoder side can be replaced by the downmixed signals l'0 and r'0 obtained using a matrix. D from downmix which is not necessarily the same as the one assumed in the decoder. Using alternative downmix for parameter estimation on the encoder side only guarantees correct center channel reproduction on the decoder side. By transmitting additional information from the encoder to the decoder a more accurate upmix of the three channels can be obtained. In an extreme case, all six elements of matrix C can be transmitted. However, the present embodiment teaches that a subset of matrix C can be transmitted if it is accompanied by information 802 about the used downmix matrix D.

As mentioned above perceptual audio codecs employ mid / side encoding for stereo encoding at low bit rates. Furthermore, stereo preprocessing is commonly employed in order to reduce the side signal energy under limited bit rate conditions. This is done based on the psychoacoustic feeling that a stereo signal reduction of the width of the stereo signal is a preferred coding artifact over audible quantization distortion and bandwidth limitation.

Therefore, if stereo preprocessing is used, the downmix equation (3) can be expressed as

<img file="ES2292147T3_D0021.tif" />

where γ is the attenuation of the lateral signal. As discussed above, the matrix D on the decoder side needs to be known in order to be able to correctly reconstruct the three channels. Therefore, the present embodiment teaches that the attenuation factor should be sent to the decoder.

Figure 9 shows another embodiment of the present invention in which the downmix signal output l0 and r0 from 104 is input to a stereo preprocessing device 901 limiting the signal (l₀ - r₀) lateral of the mid / lateral representation of the downmix signal by a factor γ. This parameter is transmitted to the decoder. Parameterization for HFR codec signals

If prediction-based upmixing is used with high-frequency reconstruction methods such as SBR [W0 98/57436], the estimated prediction parameters on the encoder side will not match the recreated high-band signal on the decoder side. The present embodiment teaches the use of an alternative non-waveform based upmix structure for recreation of three channels from two. The proposed upmix procedure is designed to recreate the correct energy of all upmixed channels in the case of uncorrelated noise signals.

IS 2 292 147 T3

Suppose the matrix D is used_to downmix as defined in (3). And that next the matrix C of upmix will be defined. So the upmix is ​​defined by

<img file="ES2292147T3_D0022.tif" />

Striving to only recreate the correct energy of the signal l (k), r (k) and c (k) on which the upmix has been carried out, in which the energies are L, R and C, the upmix matrix of such that the diagonal elements of XX * and XX * are the same, according to:

<img file="ES2292147T3_D0023.tif" />

The corresponding expression for the downmix matrix will be. ÍL + a²C a²C Ί

Χ.Χ&gt; Γ,,, ta²CR + a²CJ (36)

<img file="ES2292147T3_D0024.tif" />

Setting the diagonal element of XX * equal to the diagonal element of XX * leads to three equations that define the relationship between the elements in C and L, R and C

Lc ^₃+ Rí ^_i+ Ca¹(c_or+ c_tl)^t = L

L ^ + Rt ^ + Co ^^ + c ^)¹ = R (38) ¿cj, + R &lt;i + Ca² (cj, + Cj,) '= C

Based on the above, an upmix matrix C can be defined. It is preferable to define an upmix matrix that does not add the channel on which the right downmix has been performed to the channel on which the left upmix has been performed and vice versa. Therefore, a suitable upmix matrix can be

<img file="ES2292147T3_D0025.tif" />

This gives a matrix C according to:

<img file="ES2292147T3_D0026.tif" />

It can be shown that the elements of matrix C can be recreated on the decoder side from the two transmitted parameters

<img file="ES2292147T3_D0027.tif" />

IS 2 292 147 T3

Figure 10 depicts a preferred embodiment of the present invention. In this case 101 to 112 are the same as in Fig. 1 and will not be explained further at this time. The three original signals 101 to 103 are entered L + R c, = ^ —-

<img file="ES2292147T3_D0028.tif" />

produced in the estimation module 1001. This module estimates two parameters, for example from which matrix C can be derived on the decoder side. These parameters along with the parameters issued from 104 are input to the selection module 1002. In a preferred embodiment, the selection module 1002 outputs the parameters from 104 if the parameters correspond to a frequency range that is encoded by a waveform codec, and outputs the parameters from 1001 if the parameters correspond to a frequency range. reconstructed using HFR. The selection module 1002 also outputs information 1005 about which parameterization is used for the different frequency ranges of the signal.

On the decoder side, the module 1004 takes the transmitted parameters and directs them to the predictive upmix 109 or to the energy-based upmix 1003 as above, depending on the indication given by the parameter 1005. The energy-based upmix 1003 implements the matrix C of upmix according to equation (40).

The upmix matrix C as represented in equation (40) has equal weights (δ) to obtain the signal c (k) (of decoder) estimated from the two signals l₀(k), r₀(k) on which has been downmixed. Based on the observation that the relative amount of the signal c (k) may differ in the two signals l₀(k), r₀(k) on those that have been downmixed (that is, C / L not equal to C / R), the following generic upmix matrix could also be considered:

c - fl (Cj J c,) f \ (pl&gt; ^C1) (41)

In order to estimate c (k), this embodiment also requires the transmission of two control parameters c1 and c2, which are for example equal to c = a²C / (L + a²X) and c₂ = a²X / (R + a²C). A possible implementation of the functions f, of the upmix matrix is ​​given by (42) (43)

<img file="ES2292147T3_D0029.tif" />

(44)

The signaling of the different parameterization for the SBR interval according to the present invention is not limited to SBR. The aforementioned parameterization can be used in any frequency range where the prediction error of the prediction-based upmix is ​​considered too large. Thus, the module 1002 can output the parameters from 1001 or 104 depending on a multitude of criteria, such as method of encoding the transmitted signals, prediction error, etc.

A preferred method for improved prediction-based multichannel reconstruction includes, on the encoder side, extracting different multichannel settings for different frequency ranges and, on the decoder side, applying these settings to the frequency ranges in order to reconstruct the multichannels. .

A further preferred embodiment of the present invention includes a method for multi-channel reconstruction based on improved prediction that includes, on the encoder side, extracting information about the used downmix process and subsequently sending this information to a decoder and, on the decoder side , apply an upmix based on extracted prediction parameters and information about the downmix in order to reconstruct the multichannels.

A further preferred embodiment of the present invention includes a method for improved prediction-based multichannel reconstruction, wherein, on the encoder side, the energy of the downmix signal is adjusted according to a prediction error obtained for the predictive upmix parameters. extracted.

A further preferred embodiment of the present invention relates to a method for multi-channel reconstruction based on improved prediction, in which, on the decoder side, an energy loss due to prediction error is compensated by applying a gain to the channels over the that upmix has been performed.

IS 2 292 147 T3

A further embodiment of the present invention relates to a method for multi-channel reconstruction based on improved prediction, in which, on the decoder side the energy loss due to a prediction error is replaced by a decoded signal.

A further preferred embodiment of the present invention relates to a method for improved prediction-based multichannel reconstruction, in which, on the decoder side, a part of the energy loss due to a prediction error is replaced by a decoded signal , and a part of the energy loss is replaced by applying a gain to the upmixed channels. This part of the power loss is preferably signaled from an encoder.

A further preferred embodiment of the present invention is an apparatus for improved prediction-based multichannel reconstruction comprising means for adjusting the energy of the downmix signal according to the prediction error obtained for the extracted predictive upmix parameters.

A further preferred embodiment of the present invention is an apparatus for improved prediction-based multichannel reconstruction comprising means for compensating for energy loss due to prediction error by applying a gain to the upmixed channels.

A further preferred embodiment of the present invention is an apparatus for improved prediction-based multichannel reconstruction comprising means for replacing the energy loss due to prediction error with a de-correlated signal.

A further preferred embodiment of the present invention is an apparatus for improved prediction-based multichannel reconstruction comprising means for replacing part of the energy loss due to prediction error with a decorrelated signal, and part of the energy loss by applying a gain to the channels on which upmix has been carried out.

A further preferred embodiment of the present invention is an encoder for improved prediction-based multichannel reconstruction that includes adjusting the energy of the downmix signal according to the prediction error obtained for the extracted predictive upmix parameters.

A further preferred embodiment of the present invention is a decoder for improved prediction-based multichannel reconstruction that includes compensating for a power loss due to prediction error by applying a gain to the upmixed channels.

A further preferred embodiment of the present invention relates to a decoder for multi-channel reconstruction based on improved prediction that includes replacing the energy loss due to prediction error with a deco-related signal.

A further preferred embodiment of the present invention is a decoder for improved prediction-based multichannel reconstruction that includes replacing a part of the energy loss due to prediction error with a deco-related signal, and a part of the energy loss by applying a gain to the channels on which the downmix has been carried out.

Figure 11 shows a multichannel synthesizer for generating at least three output channels 1100 using an input signal having at least one base channel 1102, the at least one base channel being derived from an original multichannel signal. The multichannel synthesizer as shown in Figure 11 includes an upmix device 1104, which may be implemented as shown in any of Figures 2 to 10. Generally, the upmix device 1104 functions to upmix the at least one base channel using an upmix rule such that the at least three output channels are obtained. The upmix element 1104 functions to generate the at least three output channels in response to an energy measurement 1106 and at least two different upmix parameters 1108 using an upmix rule that introduces energy loss such that the at least three Output channels have an energy that is greater than an energy of signals that result from only the upmix rule that introduces energy loss. Therefore, regardless of an energy error depending on the upmix rule that introduces energy loss, the invention results in an energy compensated result, in which energy compensation can be performed by scaling and / or by means of the addition of an uncorrelated signal. The at least two different upmix parameters 1108 and the energy measure 1106 are included in the input signal.

Preferably, the energy measure is any measure related to an energy loss introduced by the upmix rule. It can be an absolute measure of the energy error introduced by the upmix or the energy of the upmix signal (which is normally lower in energy than the original signal), or it can be a relative measure such as a ratio of the energy of the signal original and the energy of the upmix signal or a relationship between the energy error and the energy of the original signal or even a relationship between the energy error and the energy of the upmix signal. A relative energy measure can be used as a correction factor, but it is nevertheless an energy measure since it depends on the energy error introduced in the upmix signal generated by an upmix rule that introduces energy loss or, expressed in others. words, an upmix rule that does not conserve energy.

IS 2 292 147 T3

An example upmix rule that introduces power loss (upmix rule that does not conserve power) is an upmix that uses transmitted prediction coefficients. In case of a non-perfect prediction of a frame or sub-band of a frame, the upmix output signal is affected by a prediction error, corresponding to a loss of energy. Naturally, the prediction error varies from frame to frame, since in the case of a near perfect prediction (a low prediction error) only a small compensation has to be made (by scaling or adding a decorrelated signal) while the in case of a larger prediction error (a non-perfect prediction) more compensation has to be made. Therefore, the energy measurement of the invention also varies between a value that indicates no or only a small offset and a value that indicates a large offset.

When the energy measurement is considered as a coherence value between channels (ICC), the consideration of which is natural, when the compensation is performed by adding a decorelated signal scaled depending on the energy measurement, the relative energy measurement (p) used preferably normally ranges between 0.8 and 1.0, 1.0 indicating that the upmixed signals are uncorrelated as required or that no uncorrelated signal has to be added or that the energy of the predictive upmix result is equal to the energy of the original signal or that the error prediction is zero.

However, the present invention is also useful in connection with other upmix rules that introduce energy loss, that is, rules that are not based on waveform coincidence but are based on other techniques, such as using codebook, spectrum matching, or any other upmix rule that doesn't address energy conservation.

Generally, power compensation can be done before or after applying the upmix rule that introduces power loss. As an alternative, compensation for energy loss can even be included in the upmix rule, for example by altering the coefficients of the original matrix using the energy measure in such a way that a new upmix rule is generated and used by the element of upmix. This new upmix rule is based on the upmix rule which introduces energy loss and energy measurement. In other words, this embodiment refers to a situation in which the energy compensation is "mixed" in the "enhanced" upmix rule such that the energy compensation and / or the addition of a decorrelated signal is performed by applying one or more upmix matrices to an input vector (the one or more base channels) to obtain (after the one or more matrix operations) the output vector (the reconstructed multichannel signal presenting at least three channels).

Preferably, the upmix device receives two channels l₀, r₀ base and outputs three reconstructed channels 1, r and c.

Reference is then made to FIG. 12 to show an example power situation at different positions in an encoder-to-decoder path. Block 1200 displays an energy of a multichannel audio signal such as a signal having at least one left channel, one right channel, and one center channel, as shown in FIG. 1. For the embodiment in Figure 12, it is assumed that the input channels 101, 102, 103 in Figure 1 are completely uncorrelated, and that the downmix element conserves energy. In this case, the energy 1202 of the one or more base channels indicated by block 1202 is identical to the energy 1200 of the original multichannel signal. When the original multichannel signals are correlated with each other, the base channel energy 1202 may be less than the energy of the original multichannel signal, when, for example, the left and right cancel each other (partially).

For further discussion, however, it is assumed that the energy 1202 of the base channels is the same as the energy 1200 of the original multichannel signal.

With 1204 the energy of the upmix signals is illustrated, when the upmix signals (for example, 110, 111, 112 of figure 1) are generated using an upmix that does not conserve energy or a predictive upmix, as discussed in connection with figure 1. Since, as will be discussed later with respect to Figures 14a and 14b, such a predictive upmix introduces an energy error Er, the energy 1204 of the upmix result will be less than the energy 1202 of the base channels.

The upmix element 1104 operates to emit output channels, which have an energy that is greater than the energy 1204. Preferably, the upmix device 1104 performs a full compensation such that the upmix result 1100 in Figure 11 presents a energy as shown in 1206.

Preferably, the upmix result, the energy of which is displayed at 1204, is not simply scaled up as shown in Figure 2, or is individually scaled up as shown in Figure 3 or is scaled up on the encoder side as shown in Figure 4. Instead, the remaining energy Er, which corresponds to the error due to the predictive upmix, is "filled in" using a decorelated signal. In another preferred embodiment, this energy error Er is only partially covered by a de-correlated signal, while the remainder of the energy error is supplemented by upscaling the result of the upmix. The complete coverage of the energy error by a de-correlated signal is shown in Figure 5 and Figure 6, while the solution "in part" is illustrated by Figure 7.

Figure 13 shows a plurality of energy compensation methods, for example, methods which have in common the characteristic that, based on an energy measure that depends on the energy error, the energy of

ES 2 292 147 T3 output channels is higher than the pure result of the predictive upmix, that is, the result of the upmix rule (uncorrected) that introduces energy loss.

The number 1 of the table in figure 13 refers to the power compensation on the decoder side, which is done after the upmix. This option is shown in Figure 2 and further explained in connection with Figure 3, which shows channel-specific upscaling factors gz, which not only depend on the energy measure ρ, but also which, additionally, depend on the channel-dependent downmix factors vz, where z is 1, ro c.

Number 2 of figure 13 includes the method of energy compensation on the encoder side, which is performed after the downmix, which is illustrated in figure 4. This embodiment is preferable because the measurement ρ of energy does not have to be transmitted from the encoder to the decoder.

The number 3 of the table in figure 13 refers to the power compensation on the decoder side, which is done before the upmix. When considering Figure 2, the energy correction 202, which is performed after the upmix in Figure 2, would be performed before the upmix block 201 in Figure 2. This embodiment results, compared to Figure 2, in an easier implementation, since channel specific correction factors are not required as shown in Figure 3, although quality losses could occur.

Number 4 in Figure 13 refers to a further embodiment, in which a correction is made on the encoder side before downmixing. When considering Figure 1, channels 101, 102, 103 would be scaled up by a corresponding compensation factor such that the output of the downmix element is increased after downmixing as shown at 1208 in the figure 12. Thus, embodiment number four in FIG. 13 has the same consequence for base channel output via an encoder as embodiment number two of the present invention.

The number 5 of the table of figure 13 refers to the embodiment in figure 5, when de-correlated signal is derived from the channels generated by the upmix rule 109 that does not conserve energy in figure 5.

Embodiment number 6 in the table in Fig. 13 refers to the embodiment in which only part of the residual energy is covered by the decorrelated signal. This embodiment is illustrated in Figure 7.

Embodiment number 8 of FIG. 13 is similar to embodiment number 5 or 6, but the de-correlated signal is derived from the base channels prior to downmixing, as set forth by frame 501 'in FIG. 5.

A preferred embodiment of the encoder is described in detail below. Figure 14a illustrates an encoder for processing a multichannel input signal 1400 having at least two channels and preferably having at least three channels l, c, r.

The encoder includes an energy measurement calculator 1402 to calculate an error measurement that depends on an energy difference between an energy of the multichannel input signal 1400 or at least one base channel 1404 and a signal 1406 on which it has been performed. upmix generated by an upmix operation 1407 that does not conserve energy.

In addition, the encoder includes an output interface 1408 to output the at least one base channel after it has been scaled (401, 402) by a scaling factor 403 that depends on the energy measurement or to output the measurement itself. of energy.

In a preferred embodiment, the encoder includes a downmix element 1410 for generating the at least one base channel 1404 from the original multichannels 1400. To generate the upmix parameters, a difference calculator 1414 and a parameter optimizer 1416 are also present. These items work to find the best match upmix 1412 parameters. At least two of this set of best fit upmix parameters are output through the output interface as the parameter output in a preferred embodiment. The difference calculator preferably operates to perform a least root mean square error calculation between the original multichannel signal 1400 and the upmix signal generated by the upmix element for parameter input on parameter line 1412. This parameter optimization procedure can be performed by several different optimization procedures, all of which are guided by the goal of obtaining a best match upmix result 1406 using a certain upmix matrix included in the upmix element 1407.

The functionality of the encoder of Figure 14a is shown in Figure 14b. After a downmix stage 1440 performed by the downmix element 1410, the base channel or the plurality of base channels may be output as illustrated by 1442. Then, an upmix parameter optimization stage 144 is performed which, depending on a certain optimization strategy can be an iterative or non-iterative procedure. However, iterative procedures are preferred. Generally, the upmix parameter optimization procedure can be implemented in such a way that the difference between the upmix result and the original signal is as small as possible. Depending on the implementation, this difference can be an individual channel-related difference or a combined difference. Generally, upmix parameter optimization stage 1444 works for mini16

ES 2 292 147 T3 mize any cost function, which can be derived from individual channels or from combined channels in such a way that, for a channel, a larger difference (error) is accepted, when, for example, a much better match for the other two channels.

Then, when the best set of tuning parameters has been found, for example, the best tuning upmix matrix, at least two upmix parameters from the set of parameters generated by step 1444 are output to the output interface as shown. indicated by step 1446.

In addition, after the upmix parameter optimization step 1444 has been completed, the energy measurement can be calculated and output as indicated by step 1448. Generally, the energy measurement will depend on the energy error 1210. In a preferred embodiment, the energy measure is the factor ρ which depends on the ratio of the energy of the upmix result 1406 and the energy of the original 1400 signal as shown in the figure

2. Alternatively, the calculated and emitted energy measure may be an absolute value for the energy error 1210 or it may be the absolute energy of the upmix result 1406 which, of course, depends on the energy error. In this context, it is to be noted that the energy measure as output via the output interface 1408 is preferably quantized and again preferably entropy encoded using any widely known entropy encoder such as an arithmetic encoder, an encoder Huffman or a blank segment encoder, which is especially useful when there are many subsequent identical energy measurements. Alternatively or additionally, energy measurements for subsequent time slices or frames may be differentially encoded, this differential encoding preferably being performed prior to entropy encoding.

Reference is now made to Figure 15a, which shows an alternative downmix element embodiment, which, according to a preferred embodiment of the present invention, is combined with the encoder of Figure 14a. The embodiment of Figure 15a covers an SBR implementation, although this embodiment can also be used in cases where spectral band replication is not performed, but where the full bandwidth of the base channels is transmitted. The encoder of FIG. 15a includes a downmix element 1500 to downmix the original signal 1500 to obtain at least one base channel 1504. In a non-SBR embodiment, the at least one base channel 1504 is input to a core encoder 1506, which may be an AAC encoder for mono signals in the case of a single base channel, or which may be any stereo encoder in the case of a single base channel. for example two stereo base channels. On the output of the core encoder 1506, a bit stream is output that includes a scrambled base channel or that includes a plurality of scrambled base channels (1508).

When the embodiment of FIG. 15a exhibits SBR functionality, the at least one base channel 1504 is low-pass filtered 1510 before entering the core encoder. Naturally, the functionalities of blocks 1510 and 1506 can be implemented by a single encoding device, which performs low-pass filtering and core encoding within a single encoding algorithm.

The base channels encoded on the output 1508 only include a low band of the base channels 1504 in encoded form. Information on the high band is calculated by an SBR spectral envelope calculator 1512, which is connected to an SBR information encoder 1514 to generate and output SBR-side information encoded at an output 1516.

The original signal 1502 is input into an energy calculator 1520, which generates channel energies (for a certain period of time of the original channels l, c, r, the channel energies being indicated by L, C, R, emitted by the block 1520). The channel energies L, C, R are input into a parameter calculator block 1522. The parameter calculator 1522 outputs two upmix parameters c1, c2 which may, for example, be the c parameters_B c₂, indicated in figure 15a. Of course, other power combinations (eg linear) involving the energies of all input channels can be generated by the parameter calculator 1522, for transmission to a decoder. Naturally, different transmitted upmix parameters will result in a different way of calculating the remaining elements of the upmix matrix. As indicated in connection with equation (40) or equations (41 to 44), the upmix matrix for the realization of figure 15 directed to energy presents at least four elements that are not zero, being the elements in the third row equal to each other. Therefore, the parameter calculator 1522 can use any combination of energies L, C, R, for example, from which the four elements in the upmix matrix can be derived such as the indication (40) or (41) of the upmix matrix.

The embodiment of Figure 15a illustrates an encoder that operates to perform upmix that conserves energy or, expressed generally, energy derivative for the entire bandwidth of a signal. This means that, on the encoder side, which is illustrated in FIG. 15a, the parametric representation output by the parameter calculator 1522 is generated for the entire signal. This means that, for each subband of the scrambled base channel, a corresponding set of parameters is calculated and output. When, for example, the scrambled base channel is considered, which is, for example, a full-bandwidth signal having ten subbands, the parameter calculator could output ten parameters cyc₂ for each subband of the scrambled base channel. When, however, the coded base channel was a low-band signal in an SBR environment, for example covering only the three lower subbands, then the parameter calculator 1522 would output a set of parameters for each of the five lower subbands and, additionally, for each of the five upper subbands, even if the signal at output 1508 does not include a corresponding subband. This is due to the fact that such a subband would be recreated on the decoder side, as will be described later in connection with Fig. 16a.

IS 2 292 147 T3

Preferably, however, and as described in connection with FIG. 10, the energy calculator 1520 and the parameter calculator 1522 only work for the high band portion of the original signal, while the parameters for the band portion Low original signal are calculated by predictive parameter calculator 104 in FIG. 10, which would correspond to predictive upmix element 109 in FIG. 10.

Figure 15b shows a schematic representation of a parametric representation emitted by the selection module 1002 in Figure 10. Therefore, a parametric representation according to the present invention includes (with or without the coded base channel (s) and optionally even without the energy measurement) a set of predictive parameters for the low band, for For example, for sub-bands 1 ai and parameters per sub-band for the high band, for example, for sub-bands i + 1 to N. Alternatively, the predictive parameters and the energy-type parameters may be mixed, for example, a sub-band having energy-type parameters may be located between sub-bands having predictive parameters.

Furthermore, a frame that has only predictive parameters can follow a frame that has only energy-type parameters. Therefore, expressed generally, the present invention as discussed in connection with Figure 10, refers to different parameterizations, which may be different in the frequency direction as shown in Figure 15b or which may be different in time direction, when a frame presenting only predictive parameters is followed by a frame presenting only energy-type parameters. Naturally, the distribution or parameterization of subbands can change from frame to frame, such that, for example, subband i presents a first set of parameters (for example predictive) as shown in figure 15b in the first frame, and presents a second set of parameters (for example of energy type) in another frame.

Furthermore, the present invention is also useful when using parameterizations other than predictive parameterization, as shown in Figure 14a, or energy-type parameterization as shown in Figure 15a. Additional examples can also be used for parameterization other than predictive or energy type as soon as any target parameter or target event indicates upmix quality, downmix bit rate, encoder side computational efficiency or on the decoder side or eg the power consumption of eg battery powered devices etc. Suppose, for a certain subband or frame, for the first parameterization it is better than for the second parameterization. Naturally, the objective function can also be a combination of different individual objectives / events as discussed above. An example event would be a high band reconstructed by SBR, etc.

Furthermore, it is to be noted that the frequency or time selective calculation and the transmission of parameters can be explicitly signaled as shown at 1005 in Figure 10. Alternatively, the signaling may also be performed implicitly as discussed in connection with Figure 16a. In this case, predefined rules are used for the decoder, for example that the decoder automatically assumes that the transmitted parameters are energy-type parameters for subbands belonging to the high band in figure 15b, for example, for subbands that have been reconstructed by means of a high frequency regeneration or spectral band replication technique.

Furthermore, it is to be noted that the calculation on the inventive encoder side of one, two or even more different parameterizations and the selection on the encoder side, which parameterization is transmitted based on a decision using any information available on the side encoder (the information can be an actually used objective function or signaling information used for other reasons such as SBR signaling and processing), it can be done with or without transmitting the energy measure. Even when the preferred energy correction is not performed at all, for example, when the upmix result that does not conserve energy (predictive upmix) is not energy corrected, or when no corresponding precompensation is performed on the encoder side, the Switching of the invention between different parameterizations is useful to obtain a better multi-channel output quality and / or lower bit rate.

In particular, the switching of the invention between different parameterizations depending on the information available on the encoder side can be used with or without the addition of a completely de-correlated signal or that at least partially covers the energy error performed by the predictive upmix as shown. shown in connection with Figures 5 to 7. In this context, the addition of a decorrelated signal as described in connection with Figure 5 is only performed for subbands / frames, for which predictive upmix parameters are transmitted, while different measures for decorrelation are used for those. subbands or frames in which energy-type parameters have been transmitted. Such measures are, for example, downscaling the wet signal and generating a decorrelated signal and scaling the decorrelated signal such that a required amount of decorrelation is obtained as required, for example, by a measure of transmitted inter-channel correlation such as ICC, when appropriately scaled decorrelated signals are added to the dry signal.

Figure 16a is discussed below to illustrate an implementation on the decoder side of the upmix block 201 of the invention and the corresponding power correction at 202. As discussed in connection with Figure 11, upmix parameters 1108 are extracted transmitted from a received input signal. These transmitted upmix parameters are preferably entered into a calculator 1600 to calculate the upmix parameters.

ES 2 292 147 T3 remaining, when the upmix matrix 1602 that includes energy compensation is to perform a predictive upmix and a previous or later energy correction. The procedure for calculating the remaining upmix parameters is discussed below in connection with Figure 16b.

The calculation of the upmix parameters is based on the equation in figure 16b, which is also repeated as equation (7). In the three input signal / two output signal embodiment, the downmix matrix D has six variables. Additionally, the upmix matrix C also presents six variables. However, on the right hand side of equation (7), there are only four values. Therefore, in the case of an unknown downmix and an unknown upmix, there would be twelve unknown variables from matrices D and C and only four equations to determine these twelve variables. However, the downmix is ​​known, in such a way that the number of variables that are not known is reduced to the coefficients of the upmix matrix C, which presents six variables, although there are still four equations to determine these six variables. Therefore, the optimization method as discussed in connection with step 1444 in Figure 14b and as illustrated in Figure 14a is used to determine at least two variables from the upmix matrix, which are preferably c₁₁ and c2₂. Now, since there are four unknown, for example, c₁₂, c₂₁, c₃₁ and c₃₂ And since there are four equations, eg, one equation for each element in the I identity matrix on the right hand side of the equation in Figure 16b, the remaining unknown variables of the upmix matrix can be calculated in a straightforward way. This calculation is performed in the 1600 calculator to calculate the remaining upmix parameters.

The upmix matrix in device 1602 is set according to the two transmitted upmix parameters as forwarded by the dashed line 1604 and by the remaining four upmix parameters calculated by block 1600. This upmix matrix is ​​then applied to the input of base channels through line 1102. Depending on the implementation, an energy measure for a low band correction is forwarded through line 1106 such that a corrected upmix can be generated and output. When the predictive upmix is ​​only performed for the low band as indicated, for example, implicitly through line 1606, and when there are energy-type upmix parameters on line 1108 for the high band, this fact is signaled, for a corresponding subband, to calculator 1600 and upmix matrix device 1602. In the case of the energy type, it is preferred to calculate the upmix matrix elements of the upmix matrix (40) or (41). For this purpose, the transmitted parameters as indicated below equation (40) or the corresponding parameters as indicated below equation (41) are used. In this embodiment, the parameters c₁, c₂ transmitted upmix cannot be used directly for an upmix coefficient, but the upmix coefficients of the upmix matrix, as shown in equation (40) or (41), have to be calculated using the upmix parameters ci and c2 transmitted.

For the high band, an upmix matrix as determined for the energy-based upmix parameters is used to upmix the high band portion of the multichannel output signals. Subsequently, the low band part and the high band part are combined in a high / low combiner 1608 to output the reconstructed full bandwidth output channels 1, r, c. As illustrated in figure 16a, the high band of the base channels is generated using a decoder to decode the transmitted low band base channels, this decoder being a mono decoder for a mono base channel, and a stereo decoder for two channels. stereo base. This (these) decoded lowband base channel (s) are input into an SBR device 1614, which additionally receives envelope information as calculated by device 1512 in FIG. 15a. Based on the low-band portion and the high-band envelope information, the high-band of the base channels is generated to obtain full-bandwidth base channels on line 1102, which are forwarded to the matrix device 1602. upmix.

The inventive methods or devices or computer programs can be implemented or included in various devices. Figure 17 shows a transmission system having a transmitter including an inventive encoder and having a receiver including an inventive decoder. The transmission channel can be a wireless or wired channel. Furthermore, as shown in Fig. 18, the encoder can be included in an audio recorder or the decoder can be included in an audio player. Audio recordings from the audio recorder can be distributed to the audio player over the Internet or through a distributed storage medium using mail or messaging resources or other possibilities to distribute storage media such as memory cards, CDs or DVDs. .

Depending on certain implementation requirements of the inventive methods, the inventive methods can be implemented in hardware or software. The implementation can be done using a digital storage medium, a particular disk or CD having electronically readable control signals stored thereon, which can co-operate with a programmable computer system in such a way that the inventive methods are performed. In other words, the inventive methods are therefore a computer program that presents program code to perform the inventive methods, when the computer program is run on a computer.