Loudness measurement with spectral modifications

Technical field [0001] The invention relates to audio signal processing. In particular, the invention relates to measuring the perceived loudness of the audio signal by modifying the spectral representation of the audio signal as a function of the reference spectral shape so that the spectral representation of the audio signal is more closely tuned to the reference spectral shape, and calculating the perceived loudness of the modified spectral representations of the audio signal.

Appeals and incorporation by reference [0002] Certain techniques for objectively measuring perceived (psychoacoustic) loudness useful in better understanding aspects of the invention are described in published international patent specification WO 2004/111994 A2, published by Alan Jeffrey Seefeldt et al, published December 23, 2004 r., entitled "Method, apparatus and computer program for calculating and adjusting the perceived loudness of the sound signal", in the resulting patent publication published as US 2007/0092089, published on April 26, 2007, and in "A New Objective Measurement of Perceived Loudness" by Alan Seefeldt et al, Materials of the Convention of the Sound Engineering Association 6236, San Francisco, October 28, 2004. .

Background Art [0003] There are many ways to objectively measure the perceived loudness of audio signals. Examples of these methods include A-, B- and C-weighted power measurements and psychoacoustic loudness models such as those described in "Acoustics - methods for calculating loudness levels," ISO 532 (1975) and WO 2004/111994 A2 and US 2007 / 0092089. Weighted power measurements operate by using the audio input signal, a task of a known filter that enhances the frequencies felt with greater sensitivity while reducing the frequencies felt with less sensitivity, and then averages the power of the filtered signal over a pre-set period of time. Psychoacoustic methods are typically more complex and tend to better model the functioning of the human ear. Such psychoacoustic methods divide the signal into frequency bands that mimic the frequency response and sensitivity of the ear, and then manipulate and combine these bands taking into account psychoacoustic phenomena, such as frequency and time masking, and non-linear loudness perception at changing signal strength. The purpose of such methods is to obtain a numerical measurement that closely corresponds to the subjective perception of the sound signal.

[0004] The inventor has found that the described objective loudness measurements are not able to accurately reproduce the subjective impressions for certain types of audio signals. In WO 2004/111994 A2 and US 2007/0092089 such problematic signals were

- 2 described as "narrowband", which means that most of the signal energy is concentrated in one or more small parts of the audible spectrum. These descriptions disclose a way of dealing with such signals, involving the modification of a traditional psychoacoustic loudness perception model so as to incorporate two increases in loudness function: one for "wideband" signals and the other for "narrowband" signals. The patent application descriptions WO 2004/111994 A2 and US 2007/0092089 show the interpolation between these two functions based on the "narrowband" signal measure.

[0005] Although this interpolation method improves the efficiency of objective loudness measurement relative to subjective impressions, the inventor has since developed an alternative psychoacoustic loudness perception model, which he believes explains and resolves differences between objective and subjective loudness measurements for the problem of "narrow" signals band. " The use of such an alternative model for objective loudness measurement is an aspect of the invention.

Description of the drawings [0006]

FIG. 1 is a simplified schematic block diagram of aspects of the invention.

FIGURES 2A, B, and C illustrate in an exemplary manner an example of using spectral modifications, according to aspects of the invention, to an idealized sound spectrum that comprises predominantly bass frequencies.

FIGURES 3A, B, and C show in an exemplary manner an example of using spectral modifications, according to aspects of the invention, to an idealized sound spectrum that is similar to the reference spectrum.

FIG. 4 shows a set of critical bandpass filter responses useful for calculating the excitation signal for a psychoacoustic loudness model.

FIG. 5 shows the contours of equal loudness according to ISO 226. The horizontal scale is the frequency in hertz (logarithmic scale at base 10), and the vertical scale is the sound pressure level in decibels.

FIG. 6 is a graph comparing objective loudness measures from an unmodified psychoacoustic model to subjective loudness measures for a sound recording database.

FIG. 7 is a graph comparing objective loudness measures from a psychoacoustic model using aspects of the invention to subjective loudness measures for the same sound recording database.

Disclosure of the Invention [0007] The invention is defined by the independent claims. The dependent claims relate to optional functions of some embodiments of the invention.

[0008] According to aspects of the disclosure, the method of measuring the perceived loudness of an audio signal includes obtaining a spectral representation of the audio signal, modifying the spectral representation as a function of the reference spectral shape so that the spectral representation of the audio signal is more closely tuned with the reference spectral shape, and calculating the perceived loudness of the modified spectral representation of an audio signal. Modifying the spectral representation as a function of the reference spectral shape may include minimizing the function of differences between the spectral representation and the reference spectral shape and setting the reference level of the spectral shape in response to minimizing. Minimizing the difference function can minimize the weighted average of the differences between the spectral representation and the reference spectral shape. Minimizing the difference function may further include using compensation to change the differences between the spectral representation and the reference spectral shape. Compensation can be fixed compensation. Modifying the spectral representation as a function of the reference spectral shape may further include adopting a maximum level of spectral representation of the audio signal of the reference spectral shape at a predetermined level. The spectral representation of an audio signal can be an excitation signal that approximates the energy distribution on the base membrane of the inner ear.

[0009] According to further aspects of the disclosure, the method of measuring the perceived loudness of an audio signal includes obtaining a representation of the audio signal, comparing the representation of the audio signal with a reference representation to determine how closely the representation of the audio signal corresponds to the reference representation, modifying at least a portion of the audio signal representation, that the resulting modified representation of the audio signal corresponds more closely to the reference representation and determining the perceived loudness of the audio signal from the modified representation of the audio signal. Modifying at least a portion of the audio signal representation may include adjusting the level of the reference representation relative to the audio signal level. The level of reference representation can be adjusted to minimize the function of the differences between the level of reference representation and the level of representation of the audio signal. Modifying at least a portion of the audio signal may include increasing the level of the audio signal portion.

[0010] According to yet further aspects of the disclosure, the method of determining the perceived loudness of an audio signal includes obtaining an audio signal representation, comparing the spectral shape of the audio signal with the reference spectral shape, adjusting the reference level of the spectral shape so as to to match the shape of the spectral representation of the audio signal so that the differences between the spectral shape of the audio signal representation and the reference spectral shape are reduced, forming a modified shape of the spectral representation of the audio signal by increasing a portion of the shape of the spectral representation of the audio signal to further improve the fit between the spectral shape of the signal representation sound and reference shape

- 4 spectral, and determining the perceived loudness of the sound signal based on the modified shape of the spectral representation of the sound signal. Customization may include minimizing the function of differences between the spectral shape of the audio signal representation and the reference spectral shape and setting the reference level of the spectral shape in response to minimization. Minimizing the difference function may minimize the weighted average of the differences between the spectral representation of the audio signal representation and the reference spectral shape. Minimizing the difference function may further include using compensation to change the differences between the spectral shape of the audio signal representation and the reference spectral shape. Compensation can be fixed compensation. Modifying the spectral representation as a function of the reference spectral shape may further include adopting a maximum level of spectral representation of the audio signal of the reference spectral shape at a predetermined level.

[0011] According to further aspects and still further aspects of the disclosure, the sound signal representation may be an excitation signal that approximates the energy distribution on the inner ear's primary membrane.

[0012] Other aspects of the invention include an apparatus performing any of the above-listed methods and a computer program recorded on a computer-readable medium to cause the computer to perform any of the above-listed methods.

Best Mode for Carrying Out the Invention [0013] In a general sense, all objective loudness measurements mentioned earlier (both weighted power measurements and psychoacoustic models) can be seen as integrating over the frequency of a certain representation of the audio signal spectrum. In the case of weighted power measurements, this spectrum is the signal power spectrum multiplied by the power spectrum of the selected weighting filter. In the case of the psychoacoustic model, this spectrum can be a non-linear function of power within a series of successive critical bands. As mentioned earlier, it has been found that such objective loudness measures provide limited efficiency for audio signals having the spectrum previously described as "narrowband".

[0014] Instead of looking at such signals as narrowband, the inventor has developed a simpler and more intuitive explanation based on the premise that such signals are unlike the average spectral shape of ordinary sounds. It can be argued that most of the sounds encountered in everyday life, in particular speech, have a spectral shape that does not differ much from the average "expected" spectral shape. This average spectral shape shows an overall increase in energy with an increase in frequency that is passed band-wise between the lowest and highest audible frequencies. The artist puts forward the thesis that when someone assesses the loudness of a sound having a spectrum that deviates significantly from such an average spectral shape, it cognitively "fills" to some extent those areas of the spectrum in which there is no expected energy. Overall impression

- loudness is then obtained by integration over the frequency of the modified spectrum, which includes the "cognitively filled" spectral portion, rather than the actual spectrum of the signal. For example, if someone was listening to a music piece in which only the bass guitar plays, then one would generally expect that eventually other instruments will join this guitar and fill the spectrum. The creator is of the opinion that instead of assessing the overall loudness of the solo bass based only on its spectrum, part of the overall perception of loudness is attributed to the missing frequencies that can be expected to accompany the bass. Analogs can be drawn with the well-known "missing fundamental component" effect in psychoacoustics. If someone hears a series of harmonically related sounds, but there is no fundamental frequency of that series, they still perceive this series as having a pitch corresponding to the frequency of the missing fundamental component.

[0015] To comply with aspects of the invention, the hypothetical subjective phenomenon described above is embedded in the objective measure of perceived loudness. FIG. 1 describes an overview of aspects of the invention that relate to each of the objective measures already mentioned (i.e., both weighted power models and psychoacoustic models). As a first step, the audio signal x can be converted into a spectral representation of X proportional to the specific objective loudness measure that is used. The fixed reference spectrum Y represents the hypothetical averaged spectral shape discussed above. This reference spectrum can be pre-calculated, for example, by averaging the spectra of representative databases of ordinary sounds. As a next step, the Y reference spectrum can be "matched" to the X spectrum of the signal to generate a fixed spectrum Y reference level_M. By matching is meant that Y_M is generated as a Y level scaling so that the level of the matched Y reference spectrum_M was aligned with X, where equalization is a function of the level difference between X and Y over frequency. Level alignment may include minimizing the weighted or unweighted difference between X and Y by frequency. Such weighing can be defined in any number of ways, but it can be chosen that the parts of the X spectrum that deviate most from the reference spectrum Y receive the largest weights. In this way, the most "unusual" parts of the X spectrum of the signal are most closely aligned with Y_M. Then the modified X spectrum_c signal is generated by modifying X to be close to the matched Y reference spectrum_M according to the modification criterion. As will be detailed below, this modification may take the form of a simple maximum selection from X and Y_M by frequency, which simulates the cognitive "filling" discussed above. Finally, the modified X spectrum_c the signal can be processed according to the chosen objective loudness measure (i.e., some type of frequency integration) to produce an objective loudness value L.

[0016] FIGURES 2A-C and 3A-C describe an example of calculating modified X spectra, respectively_c signal for two different original X spectra of the signal. In FIG. 2A, the original X spectrum of the signal, represented by a solid line, contains most of its energy at bass frequencies. Compared to the presented reference spectrum Y, represented by dashed lines, the shape of the spectrum X of the signal is considered to be

- 6 "extraordinary". In FIG. 2A the reference spectrum is initially shown as any initial level (upper dashed line) at which it is above the X spectrum of the signal. The Y reference spectrum can then be scaled down as far as level to match the X spectrum of the signal to form the matched Y reference spectrum_M (bottom dashed line). It can be seen that Y_M is most closely matched to the bass frequencies

X, which may be considered the "unusual" part of the signal spectrum compared to the reference spectrum. In FIG. 2B, those parts of the X spectrum of the signal that fall below the matched reference Y spectrum_M are equated with Y_M, thus modeling the cognitive "filling" process. In FIG. 2C shows the result where the modified X spectrum_Csignal, represented by a dashed line, is equal to the maximum of X and Y_M by frequency. In this case, the use of spectral modification has added a significant amount of energy to the original signal spectrum at higher frequencies. As a result, the volume calculated from the modified X spectrum_C signal is larger than if it were calculated from the original X spectrum of the signal, which is the desired effect.

[0017] In FIGURES 3A-C, the X spectrum of the signal has a similar shape as the reference spectrum

Y. As a result, the matching reference spectrum Y_M may be below the X spectrum of the signal at all frequencies, and the modified XC spectrum of the signal may be equal to the original X spectrum of the signal. In this example, the modification has no effect on subsequent loudness measurements. For most signals, their spectra are sufficiently close to the modified spectrum, as in FIGURES 3 AC, so that no modification is applied and thus no change in loudness calculation occurs. Preferably, only "unusual" spectra are modified, as in FIGURES 2A-C.

[0018] In WO 2004/111994 A2 and US 2007/0092089, Seefeldt et al discloses, inter alia, an objective measure of perceived loudness based on a psychoacoustic model. A preferred embodiment of the invention may use the described spectral modification for such a psychoacoustic model. This model without modification is first reviewed and then the details of the modification application are presented.

[0019] From the sound signal x [n], the psychoacoustic model first calculates the excitation signal E [b, t] approximating the energy distribution on the inner membrane of the inner ear in the critical band b during the time block t. This excitation can be calculated from the short-time discrete Fourier transform (STDFT) sound signal as follows

Elb, a = Λ, ερ, ι -1] + (i - (i) b

where X [k, t] represents STDFT x [n] in the time block ti of the reservoir k, where k is the frequency reservoir indicator in the transform, T [k] represents the frequency response of the filter simulating the transmission of sound through the outer and middle ear, and C_b[k] represents the frequency response of the base membrane at a location corresponding to critical band b. FIG. 4 represents the corresponding set

- 7 critical band filter responses in which forty bands are evenly spaced on a scale equivalent to a rectangular bandwidth (ERB) defined by Moore and Glasberg (BCJ Moore, B. Glasberg, T. Baer, ​​"Model for predicting threshold values, volume and partial volume, "Journal of the Audio Engineering Society, Vol. 45, No. 4, April 1997, pp. 224-240). The shape of each filter is described by a rounded exponential function, and the bands are spread using 1 ERB spacing. Finally, the smoothing time constant λβ in (1) can be advantageously selected in proportion to the integration time of loudness perception by a human within the b band.

[0020] Using equal loudness envelopes such as those shown in FIG. 5, the excitation in each band is converted to an excitation level that would generate the same loudness at 1 kHz. Specific loudness, a measure of perceptual loudness distributed in the frequency and time range, is then calculated from the transformed excitation E_1kHz[b, t], through elastic nonlinearity. One such suitable function for calculating the specific loudness N [b, t] is given by:

<img file="PL2162879T3_D0001.tif" />

where TQ 1kHz is the threshold value in silence at 1 kHz, and the constants β and α are chosen to match the subjective impression of volume increase for a sound with a frequency of 1 kHz. Although it was found that 0.24 for β and 0.045 for α are appropriate, these values ​​are not critical. Finally, the total loudness, L [t], is calculated in units of sons by adding the specific loudness in the bands:

= (3) [0021] In this psychoacoustic model, there are two intermediate spectral representations of sound before the calculation of total loudness: excitation E [b, t] and specific loudness N [b, t]. In the case of the invention, spectral modification can also be used, but the use of modification rather for excitation instead of specific loudness simplifies the calculation. This is because the excitation shape over the frequency is invariant relative to the overall audio level. This is reflected in the way the spectra maintain the same shape at varying levels, as shown in FIGURES 2A-C and 3A-C. This is not the case for specific loudness, due to the non-linearity in equation 2. Therefore, in the examples given here, spectral modifications are used to represent the spectral excitation.

[0022] When applying the spectral modification to excitation, it is assumed that there is a constant reference excitation Y [b]. In practice, Y [b] can be created by averaging the excitations calculated from the sound database containing a large number of speech signals. The source of the reference excitation spectrum Y [b] does not matter

- critical to the invention. Using the modification, it is useful to work with decibel representations of signal excitation E [b, t] and reference excitation Y [b]:

<img file="PL2162879T3_D0002.tif" />

In the first stage, the decibel reference excitation YdB [b] can be matched to the decibel excitation of the EdB signal [b, t] to generate a matching decibel reference excitation YdB_M[b], where YdB_M[b] is represented as scaling (or added compensation when using dB) of the reference excitation:

= + (5)

Matching compensation Δ_M is calculated as a function of the difference, A [b], between EdB [b, t] and YdB [b]:

Δ [ΐ&gt;] = EdB {b, t} -Y (lB [b} (6)

From this excitation difference, A [b], the weight W [b] is calculated as the excitation difference, normalized to have a minimum at the zero point, and then raised to the power γ:

W [ó] = (δ [Ζ&gt;] - min {A [Z&gt;]}) '(7)

In practice, the γ = 2 setting works well, although this value is not critical and other weights can be used, or no weights (i.e., γ = 1). Matching compensation A._M it is then calculated as a weighted average excitation difference, A [b], plus tolerance compensation, A_tol:

£ ^ [ό] Δ [ό] ^{Δλ =} * £&gt; Φ&gt;] ^{+ Δϊω} (8)

The weighting factor in Equation 7, if it is greater than one, causes that those parts of the EdB signal excitation [b, t] differing most from the reference excitation YdB [b] contribute most to the matching compensation A_M. Tolerance compensation A_tol affects the amount of "fill" that occurs when a modification is applied. In practice, setting A works well_tol= -12dB, causing most audio spectra to remain unmodified with this modification. (In FIGURES 3A-C, this is a negative value of A_tol which causes the matched reference spectrum to fall completely below instead of being

- 9 proportional to the signal spectrum and therefore does not result in any adjustment of the signal spectrum).

[0023] Once the matched reference excitation is calculated, the modification is used to generate the modified signal excitation by adopting the maximum value of EdB [b, t] and YdB_M[b] in the bands:

EDB_c[b, t} = max {Zu / £ [ó, /], In "[ó]} (9)

The decibel representation of the modified excitation is then converted back to the linear representation.

E_c[6, / J = io ^¹^¹⁰ (10)

This is a modified E signal excitation_C[b, t] then replaces the original signal excitation E [b, t] in the remaining stages of calculating the loudness according to the psychoacoustic model (i.e. calculating the specific loudness and summing the specific loudness in the bands, as given in equations. 2 and 3).

[0024] To demonstrate the practical utility of the disclosed invention, Figures 6 and 7 describe data showing how unmodified and suitably modified psychoacoustic models predict a subjectively rated loudness of a sound recording database. For each test recording in the database, subjects were asked to adjust the sound volume to match the volume of some fixed reference recordings. For each test recording, subjects could immediately switch back and forth between the test recording and the reference recording to assess the volume difference. For each subject, final adjusted loudness gain in dB was recorded for each test recording, and then these gains were averaged for many subjects to generate subjective loudness measures for each test recording. Both unmodified and modified psychoacoustic models were then used to generate an objective loudness measure for each of the recordings in the database, and these objective measures are compared with the subjective measures in FIGURES 6 and 7. In both FIGURES, the horizontal axis represents the subjective measure in dB, and the vertical axis represents the objective measure in dB. Each point in the figure represents a recording in the database, and if the objective measure were to be perfectly matched to the subjective measure, then each point would be exactly on the diagonal line.

[0025] For the unmodified psychoacoustic model in Fig. 6, it can be seen that most of the data points are near the diagonal line, but there are a significant number of isolated values ​​above this line. Such isolated values ​​represent the problem signals discussed earlier, and the unmodified psychoacoustic model assesses them too quiet compared to the average subjective rating. For the entire database, the average absolute error (AAE) between the objective and subjective measures is 2.12 dB, which is quite low, but the maximum absolute error reaches very high

- 10 values ​​of 10.2 dB.

[0026] FIG. 7 shows the same data for a modified psychoacoustic model. Here, most data points remained unchanged compared to those shown in Fig. 6, except for isolated values ​​that were brought to the line with other points centered around the diagonal. Compared to the unmodified psychoacoustic model, AAE is slightly reduced to 1.43 dB, and MAE is significantly reduced to 4 dB. The advantage of the disclosed spectral modifications for previously outgoing signals is easily discernible.

Implementation [0027] Although in principle the invention can be practically used in either the analog domain or in the digital domain (or in some combination thereof), in practical embodiments of the invention the audio signals are represented by samples in data blocks and the processing is carried out in the digital domain .

[0028] The invention may be implemented in hardware or in software, or by a combination of both of these methods (eg, programmable logic arrays). Unless otherwise specified, the algorithms and processes incorporated as part of the invention are not inseparably associated with any computer or other apparatus. In particular, various general-purpose machines can be used with programs written according to the principles set out herein, or it may be more convenient to build more specialized apparatus (e.g. integrated circuits) to perform the required method steps. Hence, the invention may be implemented in one or more computer programs executed on one or more programmable computer systems, each of which includes at least one processor, at least one data storage system (including volatile and non-volatile memory and / or memory elements mass), at least one device or input port, and at least one device or output port. The program code is applied to the input data to perform the functions described herein and generate output information. Output information is applied to one or more output devices in a known manner.

[0029] Any such program can be implemented in any desired computer language (including machine language, assembler, or procedural logical or object-oriented high-level programming languages). In any case, the language may be a compiled or interpreted language.

[0030] Each such computer program is preferably stored or downloaded to media or recording devices (e.g., memory or semiconductor media, or magnetic or optical media), readable by a general or special purpose programmable computer, for configuring and operating the computer when the media or storage device is read by the computer system to perform the procedures described herein. The inventive system can also be considered to be implemented as a computer-readable recording medium, configured by means of a computer program, where the recording medium configured in this way causes

- 11 operation of the computer system in a special and predetermined manner to perform the functions described herein. A number of embodiments of the invention have been described. However, it will be appreciated that various modifications may be made without departing from the scope of the invention as defined by the claims. For example, some of the steps described herein may be independent of the order and therefore may be performed in a different order than described.

Dorota Rzążewska Patent Attorney