Hearing aid

The present invention relates to a hearing aid device.

Patent Document 1 discloses a hearing aid that directs the microphone array toward the speaker to clarify the sound picked up by the microphone. Further, in Patent Document 2 and Patent Document 3, the rotation angle of the head of the head of the head of the head is detected by a sensor such as a digital vibration gyro or a camera, and a virtual sound image is generated even if the head of the head of the head of the head is rotated. A sound image localization technique that does not move is disclosed. Patent Document 4 also discloses a method of detecting the rotation angle of the head by a head tracker.

When the sound image localization technique disclosed in Patent Document 2 and the hearing aid device disclosed in Patent Document 1 are combined, for example, the hearing aid device as shown in FIG. 10 can be realized. FIG. 10 is a block diagram showing a configuration of a conventional hearing aid device. The conventional hearing aid shown in FIG. 10 includes an external microphone array 900 and a hearing aid 800.

The hearing aid 800 includes a binaural speaker 801, a virtual sound image rotation unit 803, an inverse mapping rule storage unit 805, a direction reference setting unit 809, a head rotation angle sensor 811 and a direction estimation unit 813.

The head rotation angle sensor 811 is composed of, for example, a digital vibrating gyro, and detects the rotation angle of the head of a person wearing a hearing aid. The direction reference setting unit 809 includes a direction reference setting switch. The direction reference setting unit 809 can set the reference direction that determines the direction of the virtual sound source or reset the head rotation angle sensor 811 by operating the direction reference setting switch by the person wearing the hearing aid 800. it can.

The head rotation angle sensor 811 detects the rotation of the head of the person wearing the hearing aid 800. The direction estimation unit 813 integrates the rotation angles detected by the head rotation angle sensor 811 in the opposite direction, and determines the direction of the virtual sound source to be localized as an angle from the reference direction set by the direction reference setting switch. The reverse mapping rule storage unit 805 stores the reverse mapping rule for converting the angle determined by the direction estimation unit 813 into a directional component.

The virtual sound image rotation unit 803 rotates the sound image of the speaker's voice separated by the sound source separation unit 902, which will be described later, in the direction determined by the direction estimation unit 813, with reference to the inverse mapping rule. The binaural speaker 801 expresses and outputs the virtual sound image rotating unit 803 and the sound image of the speaker's voice rotated by the virtual sound image rotating unit 803 as an acoustic signal for the left ear and an acoustic signal for the right ear, respectively.

The external microphone array 900 includes a sound source input unit 901 and a sound source separation unit 902. The sound source input unit 901 is composed of a plurality of microphones arranged in a predetermined arrangement, and captures sound from the outside in multiple channels. The sound source separation unit 902 separates the speaker's voice by directing the external microphone array 900 toward the speaker. The separated speaker's voice is transferred to the virtual sound image rotating unit 803 described above.

In the conventional hearing aid as described above, a reverse mapping rule for converting the angle determined by the direction estimation unit 813 into a directional component is held in advance, and the wearer can refer to this reverse mapping rule. It is possible to determine the direction of the sound image of the speaker's voice with respect to.

<p><patcit num="1"><text>Japanese Unexamined Patent Publication No. 9-140000</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 8-9490</text></patcit><patcit num="3"><text>Japanese Unexamined Patent Publication No. 2004-23180</text></patcit><patcit num="4"><text>Special Table 2006-503526</text></patcit></p>

<p> In the conventional hearing aid as described above, the frequency characteristics expressed by the transfer function, the interaural volume difference, and the interaural time difference are used as directional components that are used as clues when a person perceives the direction of arrival of sound. It was necessary to obtain in advance the mapping relationship with the direction of arrival of the sound perceived by humans, and to localize the sound image from the inverse mapping.</p><p> An object of the present invention is to provide a hearing aid device capable of enhancing the clarity of the voice spoken by the speaker while reproducing the direction in which the voice spoken by the speaker arrives without using the inverse mapping rule. Is.</p>

<p> The present invention separates a sound source input unit that inputs sound coming from a sound source and converts it into a first acoustic signal, and the first acoustic signal converted by the sound source input unit into sound source signals corresponding to each sound source. A sound source separation unit, a binaural microphone that is arranged near the left and right ears and inputs the sound coming from the sound source and converts it into a second acoustic signal, and the left and right second acoustic signals converted by the binaural microphone. From the directional component calculation unit that calculates the directional component representing the directional sense of the sound source with the binaural microphone as a base point, and the left and right output acoustic signals are generated based on the sound source signal and the directional component. It includes an output signal generation unit and a binaural speaker that outputs the left and right output audio signals generated by the output signal generation unit.<u style="single">The directional component calculation unit uses the transmission characteristic between the sound source signal from the sound source separation unit and the left and right second acoustic signals from the binaural microphones as the directional component for each sound source. calculate</u>A hearing aid is provided.</p><p> According to the hearing aid device of the present invention, it is possible to enhance the clarity of the voice spoken by the speaker while reproducing the direction in which the voice spoken by the speaker arrives without using the inverse mapping rule.</p><p><u style="single">Also,</u>According to the above configuration, it is possible to generate a binaural signal difference in consideration of the frequency characteristic included in the transmission characteristic, and it is possible to realize a realistic sense of direction.</p><p> In the hearing aid, the directional component calculation unit detects an utterance section for each sound source from the sound source signal acquired from the sound source separation unit, and the directional component calculation unit simultaneously performs the utterance sections of a plurality of sound sources. When detected, the immediately preceding value is used as the transmission characteristic.</p><p> According to the above configuration, it is possible to prevent the clarity from being lowered when the estimation error of the transmission characteristic is large due to the simultaneous utterance.</p><p> In the hearing aid, the directional component calculation unit estimates the position of each sound source based on the transmission characteristics, and the directional component calculation unit estimates that the wearer of the binaural microphone is the position of the sound source. If so, the output signal generation unit outputs the second acoustic signal to the binaural speaker.</p><p> According to the above configuration, when it is determined that the sound source is the hearing aid wearing itself, the acoustic signal from the binaural microphone closer to the sound source is output, so that the hearing aid wearer's own voice can be clearly heard.</p>

<p> According to the hearing aid device according to the present invention, it is possible to enhance the clarity of the voice spoken by the speaker while reproducing the direction in which the voice spoken by the speaker arrives without using the inverse mapping rule.</p>

<figref num="1">The block diagram which shows the structure of the hearing aid device of Embodiment 1.</figref><figref num="2">A block diagram showing the configuration of the hearing aid device of the first embodiment in detail.</figref><figref num="3">The figure which shows the use example 1 of the hearing aid device of Embodiment 1.</figref><figref num="4">The figure which shows the use example 2 of the hearing aid device of Embodiment 1.</figref><figref num="5">The configuration diagram of the hearing aid device of the first embodiment and the configuration diagram of the conference system using the hearing aid device.</figref><figref num="6">A modified example of the hearing aid 100 shown in FIG.</figref><figref num="7">The block diagram which shows the structure of the hearing aid device of Embodiment 2.</figref><figref num="8">A block diagram showing the configuration of the hearing aid device of the second embodiment in detail.</figref><figref num="9">The figure which shows the use example of the hearing aid device of Embodiment 2.</figref><figref num="10">Block diagram showing the configuration of a conventional hearing aid</figref>

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1) FIG. 1 is a block diagram showing a configuration of the hearing aid device of the first embodiment. As shown in FIG. 1, the hearing aid device of the first embodiment includes a hearing aid 100 and an external microphone array 300. FIG. 3 is a diagram showing a usage example 1 of the hearing aid device of the first embodiment, and FIG. 4 is a diagram showing a usage example 2 of the hearing aid device of the first embodiment.

FIG. 2 is a block diagram showing in detail the configuration of the hearing aid device shown in FIG. The components to which the same reference numbers as those in FIG. 1 are assigned in FIG. 2 have the same functions as the components in FIG.

With reference to FIG. 1, the configuration of the hearing aid 100 that constitutes a part of the hearing aid device of the first embodiment will be described. The hearing aid 100 includes a right unit to be worn on the right ear and a left unit to be worn on the left ear. Each of the left and right units is composed of a microphone for each ear of the binaural microphone 101, a directional component calculation unit 103, an output signal generation unit 105, and a speaker for each ear of the binaural speaker 107. Wireless communication is performed between the left and right units of the hearing aid 100. The left and right units of the hearing aid 100 may be configured to communicate by wire.

The binaural microphone 101 is composed of a right-ear microphone 101A that forms a part of the right unit and a left-ear microphone 101B that forms a part of the left unit. The binaural microphone 101 inputs the sound coming from the sound source to the wearer of the hearing aid 100 at the left and right ears of the wearer of the hearing aid 100 and converts it into an acoustic signal.

The directional component calculation unit 103 determines the time difference between the two ears and the volume difference between the two ears from the acoustic signal converted by the binaural microphone 101, and determines the direction of arrival of the sound coming from the sound source to the wearer of the binaural microphone. It is calculated as a sense of direction component felt by the wearer of the hearing aid 100. That is, the directional component represents the directional sense of the sound source with the wearer of the binaural microphone 101 as the base point.

When calculating the interaural time difference as the directional component, the directional component calculation unit 103 determines the time of the right acoustic signal converted by the right ear microphone 101A and the time of the left acoustic signal converted by the left ear microphone 101B. Calculate the cross-correlation value while shifting with. Then, the time at which the cross-correlation value is maximized is defined as the interaural time difference. When calculating the volume difference between both ears as the direction component, the direction component calculation unit 103 uses the time of the right acoustic signal converted by the right ear microphone 101A and the time of the left ear microphone 101B by the amount of the time difference between both ears. The power ratio of the left and right acoustic signals is obtained by shifting the converted left acoustic signal. Then, the directional component calculation unit 103 sets the power ratio of the left and right acoustic signals as the volume difference between both ears.

As described above, the directional component calculation unit 103 directly calculates the directional component of the sound arriving from the sound source from the sound arriving at the binaural microphone 101 from the sound source. Therefore, the hearing aid device of the first embodiment can faithfully reproduce the direction of the sound coming from the sound source. The directional component calculation unit 103 may calculate either the interaural time difference or the interaural volume difference as the directional component, or directs both the interaural time difference and the binaural volume difference. It may be calculated as a sensory component.

The output signal generation unit 105 outputs the left and right acoustic signals to be output from the left and right speakers from the direction feeling component calculated by the direction feeling component calculation unit 103 and the sound source signal received from the external microphone array 300 described later. Generate. The output signal generation unit 105 determines which of the left unit and the right unit is far from the sound source from the interaural time difference, which is one of the directional components.

The output signal generation unit 105 delays the sound source signal received from the sound source separation unit 303 of the external microphone array 300, which will be described later, by the time difference between the ears for the unit farther from the sound source. Further, the output signal generation unit 105 controls the unit farther away from the sound source so that the volume of the binaural speaker 107 of the unit is reduced by the difference in volume between the binaural ears.

Further, the output signal generation unit 105 outputs the sound source signal received from the sound source separation unit 303 to the binaural speaker 107 as it is for the unit close to the sound source among the left and right units.

The binaural speaker 107 is composed of a right ear speaker 107A which forms a part of the right unit and a left ear speaker 107B which forms a part of the left unit. The binaural speaker 107 outputs the sound source signal generated by the output signal generation unit 105 as the left and right acoustic signals at the left and right ears of the wearer of the hearing aid 100.

Next, the configuration of the external microphone array 300 that constitutes a part of the hearing aid device of the first embodiment will be described with reference to FIG. The external microphone array 300 includes a sound source input unit 301 and a sound source separation unit 303. In the hearing aid of Embodiment 1, the external microphone array 300 is installed closer to the binaural microphone 101 of the hearing aid 100. The external microphone array 300 wirelessly communicates with the left and right units of the hearing aid 100. The external microphone array 300 may be configured to communicate with the left and right units of the hearing aid 100 by wire.

The sound source input unit 301 inputs the sound coming from the sound source to the external microphone array 300 and converts it into an acoustic signal. The sound source input unit 301 is composed of a plurality of microphones. The acoustic signal of each microphone converted by the sound source input unit 301 is transferred to the sound source separation unit 303.

The sound source separation unit 303 detects the direction of the sound source with the external microphone array 300 as the base point by using the difference in the arrival time of the sound coming from the sound source to each microphone.

Based on the spatial arrangement of each microphone, the sound source separation unit 303 adds the sound delay time for each microphone and adds the acoustic signals of each microphone, so that the sound source separation unit 303 uses the external microphone array 300 as a base point. A sound source signal that has been directionally processed in the direction of the sound source is generated and wirelessly transmitted to the output signal generation unit 105 of the hearing aid 100.

Here, the sound source signal generated by the sound source separation unit 303 emphasizes (directivity processing) the sound coming from the target sound source with the external microphone array 300 as the base point. Therefore, in the sound source signal generated by the sound source separation unit 303, sounds other than the sound of the target sound source are suppressed, and the sound of the target sound source becomes clear. When the position of the external microphone array 300 is closer to the position of the sound source than the position of the binaural microphone 101, the sound of the target sound source is further clarified in the sound source signal generated by the sound source separation unit 303.

Next, an operation example 1 of the hearing aid device of the first embodiment will be described with reference to FIG.

(Operation example 1) As shown in FIG. 3, a person A, a person B, and a person C wearing a hearing aid 100 are having a meeting around a round table 700 in which an external microphone array 300 is installed near the center. In FIG. 3, while person B is speaking, person A looks at person B diagonally to the right and listens to person B.

First, the sound spoken by person B is input from the two microphone systems and converted into an acoustic signal. The first microphone system is a plurality of microphones constituting the sound source input unit 301 of the external microphone array 300, and the second microphone system is the binaural microphone 101 of the hearing aid 100.

(1st microphone system) In the sound source input unit 301 of the external microphone array 300, the sound (arrow 1) arriving at the external microphone array 300 is input from the speaking person B and converted into an acoustic signal. Each of the plurality of microphones constituting the sound source input unit 301 of the external microphone array 300 picks up the sound of the utterance of the person B arriving from the person B who is the sound source. The acoustic signal converted by the sound source input unit 301 is transferred to the sound source separation unit 303.

The sound source separation unit 303 detects the sound source direction indicating the direction of the sound source with respect to the external microphone array 300 by utilizing the difference in the arrival time of the utterance sound of the person B arriving at each microphone. In the sound source separation unit 303, the acoustic signals of each microphone are added in consideration of the delay time of the sound for each microphone based on the spatial arrangement of each microphone, and the directivity processing is performed in the direction of the sound source with the external microphone array 300 as the base point. Will be done. Then, the directionally processed audio signal is wirelessly transmitted to the output signal generation unit 105 of the hearing aid 100 as a directionally processed sound source signal in the direction of the sound source with the external microphone array 300 as the base point.

(Second microphone system) In the right-ear microphone 101A and the left-ear microphone 101B constituting the binaural microphone 101 of the hearing aid 100, the sounds (arrows 2A and 2B) arriving from the speaking person B to the binaural microphone 101 are converted into acoustic signals, respectively. ..

The left and right acoustic signals converted by the right-ear microphone 101A and the left-ear microphone 101B are transferred to the directional component calculation unit 103.

In the directional component calculation unit 103, at least one of the interaural time difference and the interaural volume difference is based on the wearer of the binaural microphone 101 from the left and right acoustic signals converted by the binaural microphone 101. It is calculated as a directional component indicating the direction of the sound source. In the operation example 1 shown in FIG. 3, since the person A sees the person B, which is the sound source, on the right side, the binaural time difference based on the right ear microphone 101A is a positive value, and the binaural volume difference (power ratio). Is less than or equal to 1 (arrow 2B is longer than arrow 2A). The direction sense component calculated by the direction sense component calculation unit 103 is transferred to the output signal generation unit 105.

The output signal generation unit 105 outputs from the binaural speaker 107 from the direction sensation component calculated by the direction sensation component calculation unit 103 and the sound source signal directionally processed in the direction of the sound source with the external microphone array 300 as the base point. Left and right acoustic signals are generated.

In operation example 1 shown in FIG. 3, the left ear of the person A is farther from the person B than the right ear of the person A. Therefore, in the output signal generation unit 105, the left acoustic signal output from the left ear speaker 107B of the person A is delayed by the time difference between the two ears, which is a directional component.

Further, the output signal generation unit 105 controls the left ear speaker 107B so that the volume of the left ear speaker 107B for outputting the left acoustic signal is reduced by the difference in volume between the two ears.

Further, in the output signal generation unit 105, the sound source signal received from the sound source separation unit 303 is transferred to the right ear speaker 107A as a right acoustic signal for output from the right ear speaker 107A.

As described above, the acoustic signals output from the left-ear speaker 107B and the right-ear speaker 107A of the binaural speaker 107 are calculated by (1) the directional component calculation unit 103, and the wearer of the binaural microphone 101 is used as the base point. The direction component that expresses the sense of direction of the sound source faithfully reproduces the direction in which the sound of the speech of person B, which is the sound source, arrives, and (2) directional processing in the direction of the sound source with the external microphone array 300 as the base point. The generated sound source signal enhances the clarity of the voice of the person B who is the sound source.

Next, operation example 2 of the hearing aid device of the first embodiment will be described with reference to FIG.

(Operation example 2) As shown in FIG. 4, it is assumed that the person A, the person B, and the person C wearing the hearing aid 100 are having a meeting around the round table 700 in which the external microphone array 300 is installed near the center. In FIG. 4, from the state shown in FIG. 3, the person B stopped uttering and the person A was looking at the external microphone array 300 in front, but the person C who started the utterance was turned to the front and turned to the person. I'm listening to C's utterance.

First, the sound spoken by person C is input from the two microphone systems and converted into an acoustic signal. The first microphone system is a plurality of microphones constituting the sound source input unit of the external microphone array 300, and the second microphone system is the binaural microphone 101 of the hearing aid 100.

(1st microphone system) In the sound source input unit 301 of the external microphone array 300, the sound arriving at the external microphone array 300 from the speaking person C (arrow 3) is input and converted into an acoustic signal. Each of the plurality of microphones constituting the sound source input unit 301 of the external microphone array 300 picks up the sound of the utterance of the person C arriving from the person C who is the sound source.

The sound source separation unit 303 detects the sound source direction indicating the direction of the sound source with respect to the external microphone array 300 by utilizing the difference in the arrival time of the utterance sound of the person C arriving at each microphone.

In the sound source separation unit 303, the acoustic signals of each microphone are added based on the spatial arrangement of each microphone in consideration of the delay time of the sound for each microphone, and the directivity is directed toward the sound source with the external microphone array 300 as the base point. It is processed. Then, the directionally processed audio signal is wirelessly transmitted to the output signal generation unit 105 of the hearing aid 100 as a directionally processed sound source signal in the direction of the sound source with the external microphone array 300 as the base point.

(Second microphone system) In the right-ear microphone 101A and the left-ear microphone 101B constituting the binaural microphone 101 of the hearing aid 100, the sounds (arrows 4A and 4B) arriving at the binaural microphone 101 from the speaking person C are input to the acoustic signal, respectively. Be converted. The left and right acoustic signals converted by the right-ear microphone 101A and the left-ear microphone 101B are transferred to the directional component calculation unit 103.

In the directional component calculation unit 103, at least one of the interaural time difference and the interaural volume difference is based on the wearer of the binaural microphone 101 from the left and right acoustic signals converted by the binaural microphone 101. It is calculated as a directional feeling component representing the directional feeling of the sound source. In the operation example 2 shown in FIG. 4, the person A turns from the direction in which the person C is viewed from the left to the direction in which the person C is viewed from the front. Therefore, the interaural time difference is positive when the left ear microphone 101B is used as a reference. The value changes from 0 to 0, and the interaural time difference (power ratio) changes from less than 1 to 1 (arrows 4A and 4B have equal lengths). The direction sense component calculated by the direction sense component calculation unit 103 is transferred to the output signal generation unit 105.

The output signal generation unit 105 outputs from the binaural speaker 107 from the direction sensation component calculated by the direction sensation component calculation unit 103 and the sound source signal directionally processed in the direction of the sound source with the external microphone array 300 as the base point. Left and right acoustic signals are generated.

The left and right acoustic signals synthesized by the output signal generation unit 105 are output from the left ear speaker 107B and the right ear speaker 107A of the binaural speaker 107.

In operation example 2 shown in FIG. 4, while the person A turns from the direction in which the external microphone array 300 is viewed in the front to the direction in which the person C is viewed in the front, the output signal generation unit 105 changes the time difference between the two ears, which is a directional component. Changes from the value calculated from the measured value to zero. Further, the output signal generation unit 105 controls the right ear speaker 107A so as to reduce the volume of the right ear speaker 107A by the difference in volume between the two ears, and gradually makes the volume equal to the left. Therefore, when the person A is looking at the external microphone array 300 in front, the right ear speaker 107A of the right ear delays the utterance of the person C compared to the left ear speaker 107B of the left ear, and a small sound is output. .. However, as the person A turns from the direction of looking at the external microphone array 300 from the front to the direction of looking at the person C from the front, the speech of the person C is not delayed not only from the left ear speaker 107B but also from the right ear speaker 107A of the right ear. It changes so that the same loudspeaker is output. Then, when the person A sees the person C from the front, the person A can hear the utterance of the person C from the front.

In other words, the sound image of the person C's utterance to the person A does not move in response to the movement of the person A who is the wearer of the hearing aid 100.

As described above, in the operation example 2, in the hearing aid device of the first embodiment, the sound image of the person C's utterance to the person A does not move according to the movement of the person A who wears the hearing aid 100.

Further, the acoustic signals output from the left ear speaker 107B and the right ear speaker 107A of the binaural speaker 107 are calculated by (1) the directional component calculation unit 103, and the sound source is based on the wearer of the binaural microphone 101. The directional component indicating the direction faithfully reproduces the direction in which the voice of the person C, who is the sound source, arrives, and (2) the sound source signal directionally processed in the direction of the sound source with the external microphone array 300 as the base point. As a result, the clarity of the voice of the speech of the person C, which is the sound source, is enhanced. Therefore, the hearing aid device of the first embodiment can enhance the clarity of the voice spoken by the speaker while reproducing the direction in which the voice spoken by the speaker arrives.

FIG. 5 shows a configuration diagram of the hearing aid device of the first embodiment and a configuration diagram of a conference system using the hearing aid device.

Hearing aids include a hearing aid 100 and an external microphone array 300. The hearing aid 100 includes a hearing aid body 110, a right-ear microphone 101A and a right-ear speaker 107A, and a left-ear microphone 101B and a left-ear speaker 107B, which are connected to each other by wire. The external microphone array 300 includes a speakerphone main body 310 and two external microphones 320, and the two external microphones 320 and the speakerphone main body 310 are connected by a wired L1. The speakerphone body 310 includes four built-in microphones 330. The hearing aid body 110 included in the hearing aid 100 and the speakerphone body 310 included in the external microphone array 300 are connected by a wired L2.

The hearing aid body 110 and the speakerphone body 310 include a power supply, a DSP (Digital Signal Processor), a communication unit, a storage unit, and a control unit, respectively.

As shown in FIG. 5, a conference system using a hearing aid is composed of a hearing aid, a desk 710, and a plurality of chairs 720. The plurality of chairs 720 are installed around the desk 710. The voice of the speaker sitting on the chair 720 is input to the external microphone array 300, and the right ear microphone 101A and the left ear microphone 101B. The speaker's voice is output to the binaural speaker 107 as a highly clear voice component via the external microphone array 300. Further, the speaker's voice is output to the binaural speaker 107 as a directional component via the right ear microphone 101A and the left ear microphone 101B. The user of the hearing aid device can hear the speaker's voice clearly and perceive the direction of arrival based on the highly clear voice component and the direction sense component.

In the above description, it is assumed that each part is connected by wires L1 and L2, but each part may be connected by wireless. For example, a right-ear unit 110R with a right-ear microphone 101A and a right-ear speaker 107A, a left-ear unit 110L with a left-ear microphone 101B and a left-ear speaker 107B, and an external microphone array 300, respectively, have power supplies, DSPs, and It may include a communication unit, a storage unit, a control unit, and the like, and communicate with each other wirelessly.

Further, as shown in FIG. 6, in the conference system using the hearing aid shown in FIG. 5, the remote control unit 130 may be added to the hearing aid 100. In FIG. 6, the part that communicates wirelessly is shown by a broken line. The basic function of the remote control unit 130 is to be controlled by the user, such as changing the output volume of the hearing aid 100, but by implementing a microphone array consisting of four microphones 131, it can also be used as an external microphone array 300. become. The remote control unit 130 can be mounted on the mobile phone 150, for example.

Regardless of whether it is wired or wireless, and the configuration of each unit included in the hearing aid, in any case, the information processing in the hearing aid takes into consideration the processing delay and power consumption due to communication. It is desirable to have proper distribution among the plurality of units included in the hearing aid 100 and the external microphone array 300.

For example, in FIG. 5, according to the block configuration of FIG. 1, the DSP built in the speakerphone main body 310 may perform sound source input processing and sound source separation processing, and the DSP built in the hearing aid main body 110 may perform other processing. As a result, the communication signal between the external microphone array 300 and the hearing aid 100 need only include the separated audio signal, which has the effect of reducing the communication capacity. Further, by performing the sound source separation with a large amount of processing by the speakerphone main body 310 that can use the AC adapter, there is an effect that the power consumption of the hearing aid main body 110 can be suppressed.

Further, for example, in FIG. 6, the processing delay associated with wireless communication becomes more remarkable than that with wired communication, so it is necessary to consider the amount of communication.

If the volume difference between both ears is used as the directional component, the volume of the left and right output signals can be determined by using the difference between each of the left and right volumes and a predetermined reference volume. As a result, there is no processing delay due to the transmission of the signal from the left and right units of the hearing aid main body 110 to the remote control unit 130, so that there is an effect that the directional component is kept as it is. Furthermore, since it is not necessary to directly compare the left and right volumes, the right output signal is generated in the right unit of the hearing aid body 110, the left output signal is generated in the left unit of the hearing aid body 110, and the left and right are processed independently. Since it is possible to perform the above, there is an effect that the processing delay due to the left and right communication does not occur.

The shape of the hearing aid 100 of the hearing aid of the first embodiment is not particularly limited. However, for example, if the shape of the hearing aid 100 of the hearing aid of the first embodiment is a canal type, the hearing aid of the first embodiment is not only the direction of the head of the wearer of the binaural microphone 101, but also the hearing aid. It is possible to generate a sense of direction component that reflects the influence of reflection depending on the size and shape of each part (auricle, shoulder, torso) of 100 wearers.

In the hearing aid device of the first embodiment, the external microphone array 300 is installed near the center of the round table 700, but the present invention is not limited to this. Each speaker may wear a headset-type external microphone array 300. In this case, the external microphone array is composed of the sound source input unit 301, and the sound source separation unit 303 is not required.

In the hearing aid device of the first embodiment, the binaural speaker 107 may be built in the headphones, for example.

In the hearing aid device of the first embodiment, the binaural microphone 101 may be built in the headphones, for example.

In the hearing aid device of the first embodiment, the sound source input unit 301 of the external microphone array 300 may be configured by one microphone, and the external microphone array 300 may be arranged closer to the sound source than the binaural microphone 101. ..

(Embodiment 2) FIG. 7 is a block diagram showing a configuration of the hearing aid device according to the second embodiment. Further, FIG. 8 is a block diagram showing in detail the configuration of the hearing aid device according to the second embodiment. As shown in FIG. 7, the hearing aid device of the second embodiment includes a hearing aid 200 and an external microphone array 400. FIG. 9 is a diagram showing a usage example of the hearing aid device of the second embodiment.

With reference to FIG. 7, the configuration of the hearing aid 200 that constitutes a part of the hearing aid device of the second embodiment will be described. The binaural microphone and the binaural speaker of the hearing aid of the second embodiment have the same configuration as the binaural microphone 101 and the binaural speaker 107 of the first embodiment. Therefore, it has the same reference number as in FIG.

The hearing aid 200 includes a right unit worn on the right ear and a left unit worn on the left ear. Each of the left and right units is composed of a binaural microphone 101, an output signal generation unit 205, a binaural transmission characteristic measurement unit 207, a sound source position estimation unit 209, a binaural speaker 107, and a voice detection unit 211. .. Wireless communication is performed between the left and right units of the hearing aid 200. The left and right units of the hearing aid 200 may be configured to communicate by wire.

The binaural microphone 101 is composed of a right-ear microphone 101A that forms a part of the right unit and a left-ear microphone 101B that forms a part of the left unit. The binaural microphone 101 inputs the sound coming from the sound source to the wearer of the hearing aid 200 at the left and right ears of the wearer of the hearing aid 200 and converts it into an acoustic signal. Then, the converted acoustic signal is transferred to the binaural transmission characteristic measurement unit 207 in order to obtain the transfer function of the left and right ears of the hearing aid 200 wearer.

As will be described later, the voice detection unit 211 receives each sound source signal separated by the sound source separation unit 403 of the external microphone array 400, and detects the voice of the person speaking from the sound source signal. The voice detection unit 211 obtains the power in a predetermined time interval for each sound source signal separated for each sound source. Then, a sound source whose power in a predetermined time interval is equal to or higher than the threshold value is detected as the voice of the person who is speaking. In addition to power, the voice detection unit 211 uses a parameter representing a tuning structure (for example, power by a comb-shaped filter assuming pitch) as an element of a sound source signal used when detecting the voice of a person who is speaking. And wideband power ratio) may be used.

The binaural transmission characteristic measurement unit 207 is between the sound source signal (hereinafter referred to as the voice signal) detected by the voice detection unit 211 as the voice of the person speaking and the right acoustic signal picked up from the right ear microphone 101A. The transfer function of (hereinafter referred to as the transfer characteristic on the right) is obtained. Similarly, the binaural transmission characteristic measurement unit 207 obtains a transfer function (hereinafter referred to as the left transmission characteristic) between the voice signal and the left acoustic signal picked up from the left ear microphone 101B. The binaural transmission characteristic measurement unit 207 associates the transmission characteristics of each ear with the direction indicating the direction of the sound source with respect to the external microphone array 400 (hereinafter referred to as the sound source direction). Therefore, even when there are a plurality of voice signals detected as voice, the binaural transmission characteristic measurement unit 207 can express the sound source direction of each sound source.

In the hearing aid of the second embodiment, the directional component in the first embodiment corresponds to the transmission characteristics of each ear obtained by the binaural transmission characteristic measurement unit 207.

When a plurality of speakers are speaking at the same time, that is, when the voice detection unit 211 simultaneously detects a plurality of sound source signals separated for each sound source, the binaural transmission characteristic measurement unit 207 is used for each. Stop measuring the transfer function of the ear. In that case, the sense of direction of the sound source of each person can be maintained by using the transfer function immediately before stopping the measurement of the transfer function of each ear.

The sound source position estimation unit 209 can estimate the position of each sound source based on the transfer functions of the left and right ears associated with the sound source direction, which are obtained by the binaural transmission characteristic measurement unit 207.

First, the sound source position estimation unit 209 reaches the sound arrival time from the external microphone array 400 to the binaural microphone 101 from the time having the first peak on the impulse response of the transfer function of each ear associated with the sound source direction. Ask for. From this arrival time, the perspective of each sound source from the wearer of the hearing aid 200 can be estimated. Further, the sound source position estimation unit 209 calculates the cross-correlation value while shifting the time from the impulse response of the transfer functions of the left and right ears, and obtains the time at which the cross-correlation value is maximized as the interaural time difference.

Then, the sound source position estimation unit 209 sets the sound source having the minimum arrival time among the plurality of sound sources and having a time difference between the two ears close to 0 as the utterance of the hearing aid 200 itself. Therefore, the sound source position estimation unit 209 can estimate the position of each sound source based on the transfer functions of the left and right ears associated with the sound source direction, which are obtained by the binaural transmission characteristic measurement unit 207. Then, the estimation result of the sound source position estimation unit 209 is referred to by the output signal generation unit 205. As described above, in the hearing aid device of the second embodiment, the voice detection unit 211, the binaural transmission characteristic measurement unit 207, and the sound source position estimation unit 209 have the same functions as the directional component calculation unit of the first embodiment. It has.

The output signal generation unit 205 outputs the left and right transmission characteristics measured by the binaural transmission characteristic measurement unit 207 and the left and right audio signals from the right ear speaker 107A and the left ear speaker 107B of the binaural speaker 107, respectively. To generate left and right acoustic signals for. The output signal generation unit 205 generates the left and right acoustic signals by convolving the impulse response of the transfer function representing the left and right transmission characteristics with the voice signal of the first microphone system.

The output signal generation unit 205 refers to the estimation result of the sound source position estimation unit 209 as necessary, and determines whether or not the sound sources of the left and right audio signals are the wearer of the hearing aid 200. When the sound source position estimation unit 209 determines that the sound source is the hearing aid 200 attached, the output signal generation unit 205 does not output the audio signal of the first microphone system to the binaural speaker 107, and the second microphone system. The audio signal of is output to the binaural speaker 107. As a result, the wearer's self-confidence voice is clear and the voice with little time delay can be heard without discomfort.

The binaural speaker 107 is composed of a right ear speaker 107A which forms a part of the right unit and a left ear speaker 107B which forms a part of the left unit. The binaural speaker 107 outputs the sound source signal generated by the output signal generation unit 205 as the left and right acoustic signals at the left and right ears of the wearer of the hearing aid 200.

Next, the configuration of the external microphone array 400 that constitutes a part of the hearing aid device of the second embodiment will be described with reference to FIGS. 7 and 8. In the hearing aid of the second embodiment, the sound source input unit 301 of the external microphone array has the same configuration as the sound source input unit of the external microphone array of the first embodiment. Therefore, it has the same reference number as in FIG.

The external microphone array 400 includes a sound source input unit 301 and a sound source separation unit 403. In the hearing aid of the second embodiment, the external microphone array 400 is installed closer to the speakers B and C than the binaural microphone 101 of the hearing aid 200. The external microphone array 400 wirelessly communicates with the left and right units of the hearing aid 200. The external microphone array 400 may be configured to communicate with the left and right units of the hearing aid 200 by wire.

The sound source input unit 301 inputs the sound coming from the sound source to the external microphone array 400 and converts it into a sound signal. The sound source input unit 301 is composed of a plurality of microphones. The acoustic signal of each microphone converted by the sound source input unit 301 is transferred to the sound source separation unit 403.

The sound source separation unit 403 detects the direction of the sound source with the external microphone array 400 as the base point by using the difference in the arrival time of the sound coming from the sound source to each microphone.

The sound source separation unit 403 adds the acoustic signals of each microphone, taking into account the delay time of the sound for each microphone, based on the spatial arrangement of each microphone. Then, the sound source separation unit 303 generates a sound source signal that has been directionally processed in the direction of the sound source with the external microphone array 400 as the base point, and wirelessly transmits the sound source signal to the sound detection unit 211 of the hearing aid 200.

Here, the sound source signal generated by the sound source separation unit 403 emphasizes (directivity processing) the sound coming from the target sound source with the external microphone array 400 as the base point. Therefore, in the sound source signal generated by the sound source separation unit 403, sounds other than the sound of the target sound source are suppressed, and the sound of the target sound source becomes clear. When the position of the external microphone array 400 is closer to the position of the sound source than the position of the binaural microphone 101, the sound of the target sound source is further clarified in the sound source signal generated by the sound source separation unit 303.

The sound source separation unit 403 may perform sound source separation by independent component analysis. At that time, in order to utilize the power in the voice detection unit 211, the power information is restored by multiplying each independent component by the diagonal elements of the inverse matrix of the separation matrix.

(Operation example) As shown in FIG. 9, it is assumed that the person A, the person B, and the person C wearing the hearing aid 200 are having a meeting around the round table 700 in which the external microphone array 400 is installed near the center. In FIG. 9, while the person B and the person C are speaking, the person A looks at the person B in front and listens to the person B.

The sounds of the utterances of person B, person C, and person A are input from the two microphone systems and converted into left and right acoustic signals. The first microphone system is a plurality of microphones constituting the sound source input unit of the external microphone array 400, and the second microphone system is the binaural microphone 101 of the hearing aid 200.

(1st microphone system) In the sound source input unit 301 of the external microphone array 400, the sound (arrow 5) arriving at the external microphone array 400 from the person B is input and converted into an acoustic signal. Similarly, in the sound source input unit 301 of the external microphone array 400, the sound (arrow 7) arriving from the person C to the external microphone array 400 is converted into an acoustic signal. Further, in the sound source input unit 301 of the external microphone array 400, the sound (arrow 9) reaching the external microphone array 400 from the person A is also converted into an acoustic signal. Each of the plurality of microphones constituting the sound source input unit 301 of the external microphone array 400 collects the sounds of utterances arriving from the sound sources, person B, person C, and person A, respectively. The acoustic signal converted into the acoustic signal by the sound source input unit 301 is transferred to the sound source separation unit 403.

In the sound source separation unit 403, for example, the sound source direction indicating the direction of the sound source with the external microphone array 400 as the base point is detected by using the difference in the arrival time of the utterance sound of the person B arriving at each microphone.

In the sound source separation unit 403, the acoustic signals of each microphone are added in consideration of the delay time of the sound for each microphone based on the spatial arrangement of each microphone, and the directivity processing is performed in the direction of the sound source with the external microphone array 400 as the base point. Will be done. Then, the directionally processed audio signal is wirelessly transmitted to the voice detection unit 211 of the hearing aid 200 as a directionally processed sound source signal in the direction of the sound source with the external microphone array 400 as the base point.

(Second microphone system, hearing aid 200) In the left and right microphones 101A and 101B of the binaural microphone 101 of the hearing aid 200, the sound of the speech of each person (person B or person C or person A) coming from each sound source (arrow 6A, arrow 8A, arrow 10A, arrow 6B, Arrows 8B and 10B) are input and converted into acoustic signals, respectively. The converted acoustic signal of each sound source is transferred from the microphones 101A and 101B to the binaural transmission characteristic measurement unit 207.

Further, the voice detection unit 211 detects the voices of each person B, C, and A from each sound source signal received from the sound source separation unit 403 of the external microphone array 400.

Further, the voice detection unit 211 obtains the power of a predetermined time interval for each sound source signal separated for each sound source. Then, a sound source whose power in a predetermined time interval is equal to or higher than the threshold value is detected as the voice of the person who is speaking. The detected voice of the speaking person is very clear because it is detected from the sound source signal directionally processed by the sound source separation unit 403.

Each sound source signal (hereinafter referred to as a voice signal) in which the voice of the person speaking is detected is transferred to the binaural transmission characteristic measurement unit 207.

In the binaural transmission characteristic measurement unit 207, transmission between each of the voice signals of each sound source (person B or person C or person A) transferred from the voice detection unit 211 and the acoustic signal transferred from the right ear microphone 101A. The function is required. Similarly, in the binaural transmission characteristic measurement unit 207, transmission between each of the voice signals of each sound source (person B or person C) transferred from the voice detection unit 211 and the acoustic signal transferred from the left ear microphone 101B. The function is required.

Further, in the binaural transmission characteristic measurement unit 207, the transmission characteristics of each ear of each sound source (person B, person C, person A) are associated with the sound source direction indicating the direction of the sound source with the external microphone array 400 as the base point. Has been done.

When two or more persons are speaking at the same time, the binaural transmission characteristic measurement unit 207 stops the measurement of the transfer function of each ear. In that case, the transfer function immediately before stopping the measurement of the transfer function of each ear is used.

The transmission characteristics of each ear of each sound source associated with the sound source direction are transferred to the output signal generation unit 205 and the sound source position estimation unit 209.

In the sound source position estimation unit 209, each sound source is based on the transfer function of the left and right ears, which is obtained by the binaural transmission characteristic measurement unit 207 and is associated with the sound source direction indicating the direction of the sound source with respect to the external microphone array 400. The position of can be estimated.

In FIG. 9, the utterance of the person A who wears the hearing aid 200 has the smallest arrival time value among the plurality of sound sources (the difference in length between the arrows 10B and the arrow 9 is the arrow 6B and the arrow). It is detected as a sound source with a difference in length of 5 or smaller than the length of arrow 8B and arrow 7) and a time difference between both ears close to 0 (the lengths of arrow 10A and arrow 10B are almost equal).

In the output signal generation unit 205, the impulse response of the transmission function representing the transmission characteristics of each ear of each sound source associated with the sound source direction is convoluted in each of the left and right audio signals of each sound source, and the binaural speaker 107 The left and right acoustic signals are combined for output from the right ear speaker 107A and the left ear speaker 107B. In FIG. 9, when the sound source position estimation unit 209 detects the utterance of the person A who is the wearer of the hearing aid 200, the output signal generation unit 205 outputs the audio signal of the second microphone system to the binaural speaker 107. To do.

In the binaural speaker 107, the left and right acoustic signals synthesized by the output signal generation unit 205 are output from the right ear speaker 107A and the left ear speaker 107B, respectively.

As described above, in the hearing aid device of the second embodiment, the left and right audio signals in which the sound of each sound source processed by the external microphone array 400 is clear, and the binaural transmission characteristic measurement unit 207 of the hearing aid 200 are obtained. The left and right acoustic signals generated from the left and right transmission functions associated with the sound source directions are output from the binaural speaker 107. Therefore, the hearing aid device of the second embodiment can enhance the clarity of the voice spoken by the speaker while reproducing the direction in which the voice spoken by the speaker arrives.

Further, in the hearing aid of the second embodiment, the shape of the hearing aid 200 is not particularly limited, but when a canal type is used, for example, the left and right acoustic signals synthesized by the output signal generation unit 205 are uttering. The left and right transmission characteristics include not only the orientation of the head on which the person is wearing the hearing aid 200, but also the influence of reflection from the size and shape of each part (auricle, shoulder, torso) of the person speaking. Therefore, in the hearing aid device of the second embodiment, the wearer of the hearing aid 200 can feel the direction of the sound output from the binaural speaker 107 in real time.

As for the hearing aid device in the second embodiment, the configuration diagram of the hearing aid device and the configuration diagram of the conference system shown in FIG. 5 in the first embodiment can be applied.

This application is based on a Japanese patent application filed on January 22, 2009 (Japanese Patent Application No. 2009-012292), the contents of which are incorporated herein by reference.

The hearing aid device according to the present invention has an effect that the clarity of the voice spoken by the speaker can be enhanced while reproducing the direction in which the voice spoken by the speaker arrives without using the inverse mapping rule. However, it is useful as a hearing aid or the like.

100, 200, 800 hearing aids 101 Binaural microphone 101A right ear microphone 101B left ear microphone 103, 203 Directional component calculation unit 105, 205 Output signal generator 107, 801 Binaural speaker 107A right ear speaker 107B Left ear speaker 110 Hearing aid body 130 remote control unit 207 Binaural transmission characteristic measurement unit 209 Sound source position estimation unit 211 Audio detector 300, 400, 900 External Microphone Array 301, 901 Sound source input section 303, 403, 902 Sound source separator 310 Speakerphone body 320 external microphone 700 Round table 710 desk 720 multiple chairs 803 Virtual sound image rotating part 805 Reverse mapping rule storage 807 Head angle sensor 809 Direction reference setting unit 813 Direction estimation unit