Voice mixing device, and its noise suppressing method and program

(Related Application) This application claims priority based on Japanese Patent Application 2007-205443 filed on August 7, 2007, and the entire Japanese application is incorporated into this document by citation. The present invention relates to a voice mixing device, a noise suppression method thereof, and a program, and more particularly to a technique for suppressing noise superimposed on voice in a multipoint conference.

In the multipoint conference service, the voice data of each participant is transmitted to the multipoint conference server. The multipoint conference server transmits to each participant an audio signal that is a mixture of the audio signals of all participants other than itself. FIG. 4 is a diagram showing a typical configuration example of an audio mixing device for a multipoint conference (see Patent Documents 1 and 2 and the like). In Fig. 4, the number of participants in the conference is M (M is a positive integer of 2 or more), and the voice signals for the 1st, 2nd, ... Mth speakers are voice input terminals 501, 502, ... 50M, respectively. Enter from. Each of the input audio signals is input to the full adder 520 and the adders 531, 532, ... 53M, respectively. The full adder 520 outputs an audio signal obtained by adding all the input audio signals to the adders 531, 532, ... 53M. The adders 531 and 532, ... 53M subtract the input audio signals from the audio signals input from the full adder 520, and the subtracted audio signals are the audio output terminals 571, 572, ... Output to 57M. With such an audio mixing device, it is possible to mix and listen to audio signals of a person other than oneself.

By the way, in a multipoint conference service, background noise is often superimposed on a voice signal, especially when a participant is using a mobile phone terminal. Noise suppression processing is effective in such mobile phone terminals and the like. As a noise suppression process, for example, a method is known in which a frequency domain analysis by Fourier transform or the like is performed on an input voice, then the superimposed noise is estimated and a noise component is subtracted from the input voice (. For example, see Non-Patent Document 1).

<patcit num="1"><text>Japanese Patent Application Laid-Open No. 2005-269347</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 10-75310</text></patcit><nplcit num="1"><text>Yamato Kazuhiro, Sugiyama Akihiko, Kato Masanori, Post-Processing Noise Suppressor with Adaptive Gain-Flooring for Cell-Phone Handsets and IC Recorders, International Conference on Consumer Electronics (ICCE), 2007, pp.1-2.</text></nplcit>

<p> The above disclosure items of Patent Documents 1 and 2 and Non-Patent Document 1 shall be renormalized and described in this document. The following is an analysis of related techniques according to the present invention. The following analysis is given in the present invention.</p><p> In multipoint conference services, background noise is often superimposed on voice signals, especially when participants are using mobile terminals. Therefore, it is preferable to perform noise suppression processing on the received voice signals of each participant and then perform voice mixing. In the noise suppression process, background noise is estimated and smoothed based on the retained past information.</p><p> In a conventional multipoint conference, noise suppression processing is performed on the voices of all participants, so that the amount of calculation for noise suppression processing may increase as the number of participants increases. As a method for solving this, it is conceivable to reduce the number of noise suppression processes to be operated by detecting the speaker who is speaking and mixing only the voice signal of the detected speaker.</p><p> However, if noise suppression processing is applied to this method, abnormal noise may occur due to inconsistency with the past noise suppression information held by the noise suppression processing when the speaker is switched. There is.</p>

<p> The voice mixing device according to one aspect (side surface) of the present invention is a voice mixing device that mixes a plurality of voice signals, and is a speaker selection means for selecting at least one voice signal from the plurality of voice signals, and a speaker selection means. All signal addition means for adding all at least one audio signal selected in the speaker selection means, each subtraction means for subtracting only one voice signal selected from the addition result of all signal addition means, and all signal addition means. Based on the selection result of the common noise suppressing means for suppressing the noise of the common voice signal which is the addition result of the above, the individual noise suppressing means for suppressing the noise of the individual voice signal which is the subtraction result of the subtracting means, and the speaker selection means. The memory switching means for copying the noise suppression information obtained by the common noise suppression means to the noise suppression information in the individual noise suppression means is provided.</p><p> In the voice mixing device of the present invention, the voice data switching means for switching and outputting the noise-suppressed voice signal obtained by either the common noise suppressing means or the individual noise suppressing means based on the selection result of the speaker selecting means. , You may be prepared further.</p><p> The audio mixing device of the present invention may further include pre-noise suppression means that can select whether or not noise suppression is performed for each of a plurality of audio signals.</p><p> In the voice mixing device of the present invention, the speaker selection means may select at least one voice signal based on the power of the voice signal.</p><p> In the voice mixing device of the present invention, the speaker selection means may select at least one voice signal based on whether the voice signal is sounded or not.</p><p> The multipoint conference system of the present invention may include the above-mentioned audio mixing device.</p><p> The noise suppression method according to another aspect of the present invention is a noise suppression method in a voice mixing device that mixes a plurality of voice signals, and is a first method of selecting at least one voice signal from the plurality of voice signals. A step, a second step of adding all at least one audio signal selected in the first step, a third step of subtracting only one audio signal selected from the addition result of the second step, and a second step. The 4th step of suppressing the noise of the common audio signal which is the addition result, the 5th step of suppressing the noise of the individual audio signal which is the subtraction result of the 3rd step, and the 4th step based on the selection result of the 1st step. The sixth step of copying the noise suppression information obtained in the fifth step to the noise suppression information in the fifth step is included.</p><p> The noise suppression method of the present invention further includes a seventh step of switching and outputting the noise suppression audio signal obtained in any of the fourth step and the fifth step based on the selection result in the first step. May be good.</p><p> In the noise suppression method of the present invention, an eighth step of selecting whether or not to perform noise suppression for each of a plurality of audio signals may be further included before the first step.</p><p> In the noise suppression method of the present invention, at least one audio signal may be selected based on the power of the audio signal in the first step.</p><p> In the noise suppression method of the present invention, in the first step, at least one voice signal may be selected based on whether the voice signal is sounded or not.</p><p> The program according to still another aspect of the present invention includes a first step of selecting at least one audio signal from a plurality of audio signals in a computer constituting an audio mixing device for mixing a plurality of audio signals. In the second step of adding all the at least one audio signal selected in the first step, the third step of subtracting only one audio signal selected from the addition result of the second step, and the addition result of the second step. Obtained in the 4th step of suppressing the noise of a certain common audio signal, the 5th step of suppressing the noise of the individual audio signal which is the subtraction result of the 3rd step, and the 4th step based on the selection result of the 1st step. Perform the sixth step of copying the noise suppression information to the noise suppression information in the fifth step.</p><p> In the program of the present invention, the seventh step of switching and outputting the noise-suppressed audio signal obtained in either the fourth step or the fifth step is further executed based on the selection result in the first step. May be good.</p><p> In the program of the present invention, the eighth step of selecting whether or not to suppress noise for each of the plurality of audio signals may be further executed before the first step.</p><p> In the program of the present invention, in the first step, at least one audio signal may be selected based on the power of the audio signal.</p><p> In the program of the present invention, in the first step, at least one voice signal may be selected based on whether the voice signal is sounded or not.</p>

<p> According to the present invention, when the noise suppression means is switched according to the switching of the speaker, the noise suppression information after the switching is made the same as the noise suppression information before the switching, so that the generation of abnormal noise is reduced. be able to.</p>

The voice mixing device according to the embodiment of the present invention includes M power calculation unit, speaker selection unit, M voice signal input switch, all signal adder, common noise suppressor, M adder, and M. It is composed of three noise suppression units, M memory switchers, and M output audio data switchers, and performs audio mixing operations including noise suppression processing in multipoint conferences.

The power calculation unit calculates the power for each input voice signal for M people and outputs it to the speaker selection unit. The speaker selection unit selects a speaker who is uttering based on the input power of each speaker. Next, the input audio signals of all the selected speakers are mixed by the all signal adder, and the mixed audio signals are input to the common noise suppression unit.

The common noise suppression unit outputs an audio signal that suppresses noise contained in the input audio signal. On the other hand, a voice signal obtained by subtracting the voice signal of each speaker selected from the mixing voice by the adder is input to the noise suppression unit. The noise suppression unit outputs an audio signal that suppresses noise contained in the input audio signal. The memory switch copies the past information of the common noise suppression unit to the noise suppression unit based on the selection result of the speaker selection unit. The output voice data switch switches the output voice signal based on the selection result of the speaker selection unit.

According to the voice mixing device as described above, when the noise suppressing means is switched according to the switching of the speaker, the noise suppressing means before the switching has information on the past background noise of the noise suppressing means after the switching. Make it the same as the information. Therefore, it is possible to prevent the generation of abnormal noise, which is a change in noise due to switching. Such a device is suitable for a multipoint conference system or a multipoint conference server.

Hereinafter, a detailed description will be given with reference to the drawings according to an embodiment.

<figref num="1">It is a figure which shows the structure of the audio mixing apparatus which concerns on 1st Example of this invention.</figref><figref num="2">It is a flowchart which shows the operation of the audio mixing apparatus which concerns on 1st Embodiment of this invention.</figref><figref num="3">It is a figure which shows the structure of the audio mixing apparatus which concerns on 2nd Example of this invention.</figref><figref num="4">It is a figure which shows the structure of the conventional audio mixing apparatus.</figref>

Code description

101, 102, 10M audio input terminal 111, 112, 11M Power calculation unit 121, 122, 12M audio signal input switch 131, 132, ... 13M adder 141, 142, 14M, 191, 192, 19M Noise suppressor 151, 152, 15M memory switch 161, 162, 16M voice data switch 171, 172, 17M audio output terminal 181, 182, 18M noise suppression switch 200 Speaker selection section 300 full signal adder 400 Common noise suppressor

FIG. 1 is a block diagram showing a configuration of an audio mixing device according to a first embodiment of the present invention. In FIG. 1, the multipoint conference system has voice input terminals 101, 102, ... 10M, power calculation unit 111, 112, ... 11M, speaker selection unit 200, voice signal input switch 121, 122, ... 12M, all signal adder 300, adder 131, 132, 13M, common noise suppression unit 400, noise suppression unit (individual noise suppression unit) 141, 142, 14M, memory switch 151, It is equipped with 152, ... 15M, audio data switchers 161, 162, ... 16M, and audio output terminals 171, 172, ... 17M.

The power calculation units 111, 112, ... 11M are audio signals input from the audio input terminals 101, 102, ... 10M corresponding to speaker 1, speaker 2, ..., and speaker M, respectively. Is calculated and output to the speaker selection unit 200. The power P is calculated using the following equation (1) every 20 milliseconds (160 samples) for, for example, an input voice signal s (n) sampled at 8 kHz.<img file="JP5045751B2_D0001.tif" />Here, L = 160.

The speaker selection unit 200 selects a voice signal corresponding to the speaker who is uttering using the calculated power of each speaker, and determines the selection result, that is, whether or not the voice signal is selected, respectively. Output to 121, 122, ... 12M, memory switch 151, 152, ... 15M, and audio data switch 161, 162, ... 16M.

Here, as a method of selecting the speaker who is uttering, a method of selecting a predetermined top N speaker (1 N &lt;M) in descending order of the power of the voice signal, or a power exceeding a predetermined threshold value is used. There is a way to select the audio signal of. Also, instead of using the input power as it is, it is conceivable to save the power input in the past and use the value obtained by leak integration at a certain time. Further, the audio signal may be selected based on whether the audio signal is sounded or silent.

The audio signal input switching devices 121, 122, ... 12M switch whether or not to output each input audio signal based on the selection result of the speaker selection unit 200. The audio signal selected by the speaker selection unit 200 is output to the corresponding adders 131, 132, ... 13M, and is output to the full signal adder 300, respectively.

The full signal adder 300 adds all the audio signals for the speaker selected by the speaker selection unit 200 and outputs them to the adders 131, 132, ... 13M, and the common noise suppression unit 400.

The adders 131, 132, ... 13M each subtract the selected speaker's own audio signal from the audio signal output from the full signal adder 300 and output it. That is, the adders 131, 132, ... 13M input audio signals from the audio signals input from the all-signal adder 300 only for the audio signals corresponding to the speaker selected by the speaker selection unit 200. The audio signals output from the switches 121, 122, ... 12M are subtracted and output to the noise suppression units 141, 142, ... 14M, respectively.

The common noise suppression unit 400 suppresses noise superimposed on the audio signal output from the all signal adder 300, and outputs the noise to the audio data switchers 161, 162, ... 16M.

The noise suppression units 141, 142, ... 14M suppress the noise superimposed on the audio signals output from the adders 131, 132, ... 13M, respectively. That is, the noise suppression units 141, 142, ... 14M suppress the noise superimposed on the audio signal only when the audio signal is input from the adders 131, 132, ... 13M, respectively, and each audio data. Output to switches 161, 162, ... 16M.

The memory switches 151, 152, ... 15M have the information that the common noise suppression unit 400 has as noise suppression information, respectively, based on the selection result of the speaker selection unit 200, noise suppression units 141, 142, ... Copy to 14M. That is, the memory switches 151, 152, ... 15M are included in the common noise suppression unit 400 only when the speaker selection unit 200 transitions from a state in which the speaker is not selected as a speaker to a state in which the speaker is selected. The past information is output to the noise suppression units 141, 142, ... 14M, respectively.

The voice data switchers 161, 162, ... 16M switch the voice data to be output based on the selection result of the speaker selection unit 200, and from the voice output terminals 171, 172, ... 17M, respectively. Output. That is, when the voice data switchers 161, 162, ... 16M are selected as the speaker uttered by the speaker selection unit 200, the voice signals input from the noise suppression units 141, 142, ... 14M, respectively. Is output, and when the speaker is not selected as the speaker uttered by the speaker selection unit 200, the voice signal input from the common noise suppression unit 400 is output.

FIG. 2 is a flowchart showing the operation of the audio mixing device according to the first embodiment of the present invention.

In step S1, the power calculation units 111, 112, ... 11M calculate the power of the voice signal corresponding to each speaker.

In step S2, the speaker selection unit 200 selects an audio signal corresponding to the speaker (s) speaking based on each power.

In step S3, the audio signal input switches 121, 122, ... 12M switch whether or not to output the audio signal of each speaker based on the speaker selection result.

In step S4, the full signal adder 300 adds all the audio signals of the selected speaker (one or more) and outputs the addition result.

In step S5, the adders 131, 132, ... 13M subtract the audio signals of each selected speaker from the total addition result, and output the subtraction result.

In step S6, the common noise suppression unit 400 suppresses the noise as a result of the total addition.

In step S7, the noise suppression units 141, 142, ... 14M suppress the noise as a result of the output of the adders 131, 132, ... 13M, respectively.

In step S8, the memory switches 151, 152, ... 15M copy the results of the common noise suppression unit 400 to the noise suppression units 141, 142, ... 14M, respectively, based on the speaker selection results.

In step S9, the voice data switchers 161, 162, ... 16M output the noise suppression units 141, 142, ... 14M or the common noise suppression unit 400, respectively, based on the speaker selection result for each speaker. Switch and output audio data.

In steps S1 and S2, the audio signal may be selected based on whether the audio signal is sounded or silent.

Further, the audio signal of the speaker instructed separately may be mixed.

Further, a computer constituting the voice mixing device for mixing a plurality of voice signals as described above may execute a program to realize the voice mixing device.

FIG. 3 is a block diagram showing a configuration of an audio mixing device according to a second embodiment of the present invention. In FIG. 3, noise suppression switches 181, 182, ... 18M, and noise suppression units (pre-noise suppression units) 191, 192, ... 19M are added to the audio mixing device of FIG. Hereinafter, for the sake of brevity, only the differences from the first embodiment will be described.

Does the noise suppression switch 181, 182, ... 18M output the audio signals input from the audio input terminals 101, 102, ... 10M to the noise suppression units 191, 192, ... 19M, respectively? , The power calculation units 111, 112, ... 11M, and the audio signal input switch 121, 122, ... 12M, respectively, are switched. In this switching, for example, when the terminal used by the participant is a mobile phone terminal, it is considered that the background noise is superimposed, so that the input signal is input to the noise suppression unit for each participant. Set to independently. Specifically, if the first three numbers of the telephone number are 090 or 080 indicating the mobile phone terminal, the method may be such that it is determined to be the mobile phone terminal. Alternatively, a method of inputting the input signal of the speaker selected by the speaker selection unit 200 to the noise suppression unit can be considered.

The noise suppression units 191, 192, ... 19M suppress the noise of the input audio signal only when the audio signal is input, and the noise-suppressed audio signals are converted into the power calculation units 111, 112, respectively. Output to 11M and audio signal input switch 121, 122, 12M.

Since the subsequent operations are the same as those described in the first embodiment, the description thereof will be omitted.

According to this embodiment, since the input audio signal is noise-suppressed before mixing, the noise reduction effect can be further enhanced.

Although the above description has been made based on Examples, the present invention is not limited to the above Examples. Within the framework of the entire disclosure (including the scope of claims) of the present invention, it is possible to change or adjust the embodiments or examples based on the basic technical idea thereof. Further, within the scope of the claims (claims) of the present invention, various combinations, substitutions or selections of various disclosed elements are possible. Further problems / purposes and development forms of the present invention will be clarified from all the disclosure matters of the present invention including claims.