Method and apparatus for rendering audio in wireless hearing instruments

CLAIM OF PRIORITY

0001The present application claims the benefit of priority under 35 U.S.C. § 119(e) of <patcit id="pcit0001" dnum="US61927669A"><text>U.S. Provisional Patent Application Serial No. 61/927,669, filed on January 15, 2014</text></patcit>, which application is incorporated herein by reference in its entirety.

TECHNICAL FIELD

0002This document relates generally to audio systems and more particularly to a method and apparatus for rendering audio in one or more hearing instruments coupled to an audio source via a wireless communication link.

BACKGROUND

0003One or more hearing instruments may be worn on one or both sides of a person's head to deliver sounds to the person's ear(s). An example of such hearing instruments includes a pair of hearing aids that are used to assist a patient suffering hearing loss by transmitting amplified sounds to ear canals. In some examples, the one or more hearing instruments receive audio from an audio source via a wireless communication link. To ensure quality of the sound delivered to the person's ear(s), there is a need to coordinate the one or more hearing instruments for processing the wirelessly received audio under various practical conditions.

SUMMARY

0004An audio system such as a hearing assistance system includes an audio source device wirelessly communicating with one or more audio sink (destination) devices. The audio source and sink devices each include a sample clock for processing audio signal that is digitized and wirelessly transmitted. The sample clock of each of the one or more audio sink devices is synchronized to the sample clock of the audio source device to prevent audio sample over-runs or under-runs causing undesirable audio artifacts. In one embodiment, because audio packets may be lost in transmission, packet loss concealment techniques are used to improve audio reproduction. In one embodiment, link layer information is used for audio phase lock loop jitter reduction in the audio system.

0005This Summary is an overview of some of the teachings of the present application and not intended to be an exclusive or exhaustive treatment of the present subject matter. Further details about the present subject matter are found in the detailed description and appended claims. The scope of the present invention is defined by the appended claims and their legal equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

0006<ul id="ul0001" list-style="none" compact="compact"><li><figref idref="f0001">FIG. 1</figref> is a block diagram illustrating an embodiment of an audio system including audio source and audio sink devices.</li><li><figref idref="f0002">FIG. 2</figref> is a block diagram illustrating an embodiment of a pair of audio source and audio sink devices with synchronized audio sample clocks.</li><li><figref idref="f0003">FIG. 3</figref> is a block diagram illustrating an embodiment of a hearing assistance system including a host device and a hearing assistance device.</li><li><figref idref="f0004">FIG. 4</figref> is a block diagram illustrating an embodiment of a pair of hearing aids of the hearing assistance system.</li><li><figref idref="f0005">FIG. 5</figref> is an illustration of an embodiment of operation of an algorithm for reducing jitter caused by disruption of packet arrival times due to interference.</li></ul>

DETAILED DESCRIPTION

0007The following detailed description of the present subject matter refers to subject matter in the accompanying drawings which show, by way of illustration, specific aspects and embodiments in which the present subject matter may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present subject matter. References to "an", "one", or "various" embodiments in this disclosure are not necessarily to the same embodiment, and such references contemplate more than one embodiment. The following detailed description is demonstrative and not to be taken in a limiting sense. The scope of the present subject matter is defined by the appended claims, along with the full scope of legal equivalents to which such claims are entitled.

0008The present document discusses an audio system, such as a hearing assistance system, that includes an audio source device wirelessly communicating with one or more audio sink (destination) devices. The audio source and sink devices each include a sample clock for acoustic signal processing. The sample clock of each of the one or more audio sink devices is synchronized to the sample clock of the audio source device to prevent audio sample over-runs or under-runs causing undesirable audio artifacts. In various embodiments, because audio packets may be lost in transmission, packet loss concealment techniques are used to improve audio reproduction.

0009In various embodiments, the sample clock of an audio sink device is synchronized to the sample clock of an audio source device using a radio employing a packet based method of transmission based on a receive time slot. The synchronization is maintained by timing the arrival of an audio packet using an adjustable oscillator and making adjustments to that oscillator. In various embodiments, the adjustable oscillator includes an RC type oscillator with an adjustable resistance or capacitance, a voltage-controlled oscillator (VCO), a current-controlled oscillator (ICO), or a digital numerically controlled oscillator.

0010In various embodiments, the link layer of a radio is used to determine when a packet is missed, sent with errors or is empty, thereby aiding the signal processor of an audio sink device in maintaining accurate synchronization using an adjustable oscillator. In various embodiments, the adjustable oscillator includes an RC type oscillator with an adjustable resistance or capacitance, a VCO, an ICO, or a digital numerically controlled oscillator.

0011In various embodiments, the link layer of a radio is used to determine when a packet is missed, sent with errors or is empty for the purpose of employing a packet loss concealment strategy for the missing, empty, or packet received in error.

0012In various embodiments, the link layer of a radio is used to determine whether a packet received was on the primary transmission or on one of multiple retransmission attempts, thus allowing successful maintenance of an output sample rate regardless of the arrival time of the packet and on which retransmission a successful reception took place.

0013<figref idref="f0001">FIG. 1</figref> is a block diagram illustrating an embodiment of an audio system 100 including an audio source device 102, audio sink devices 104, and a wireless communication network 106. In the illustrated embodiment, audio sink devices 104 includes audio sink device 104A coupled to audio source device 102 via wireless communication link 106A, audio sink device 104B coupled to audio source device 102 via wireless communication link 106B, ... , audio sink device 104N coupled to audio source device 102 via wireless communication link 106N. In various embodiments, N is an integer equal to or greater than 1. In various embodiments, audio sink devices 104 include a hearing instrument configured to be worn by a person (hearing instrument wearer) to deliver sound to that person. Audio source devices 102 transmits an acoustic signal to each of audio sink device 104A-N via wireless communication link 106A-N, such as a Bluetooth® link, respectively.

0014Modern hearing instruments employ digital processing techniques to enhance the intelligibility of incoming acoustic information for the hearing instrument wearer. In addition, modern hearing instruments have wireless connectivity to audio sources such as cellphones, computers including desktop, laptop, and tablet computers, and other devices that can send and receive digital audio information. Other devices such as wireless headphones process digital audio information in a similar manner. Most wireless standards such as Bluetooth employ audio compressor-decompressors (CODECs) to compress audio information before sending it over the air to limit the amount of data to be sent wirelessly to remote audio sink devices and to enhance the performance of the wireless link. In order for the wireless audio sink devices receiving the information to playback the transmitted information without unwanted audio artifacts, such devices employ digital and analog techniques to synchronize the output sample clock with the input sample clock at the audio source device. In addition, these wireless audio sink devices must cope with the possibility that the transmitted information may be lost in the wireless communication link due to interference, multipath fading, and/or the like. To improve synchronization, several techniques such as digital multi-rate filtering, clock tracking, packet loss concealment and the like are employed. Digital techniques may require significant over sampling or resampling based on polyphase multi-rate filtering techniques to achieve "artifact free" audio playback.

0015In various embodiments in which audio sink devices 104 include hearing aids, the hearing instrument wearer uses two hearing instruments each rendering left and right audio channels independently. To maintain a comfortable listening environment it is essential that these two channels are rendered coherent with each other within a small timing offset. This further increases the need for audio sample accuracy between the two independent audio sink devices.

0016In various embodiments, audio source device 100 and audio sink devices 104 each include a sample clock for processing audio signal that is digitized and transmitted via wireless communication network 106. The sample clock of each of audio sink devices 104A-N is synchronized to the sample clock of the audio source device 102 to prevent audio sample over-runs or under-runs causing undesirable audio artifacts. In various embodiments, because audio packets may be lost in transmission, packet loss concealment techniques are used to improve audio reproduction.

0017<figref idref="f0002">FIG. 2</figref> is a block diagram illustrating an embodiment of an audio system 200 including an audio source device 202 and an audio sink device 204. System 200 represents an example of system 100 for the purpose of illustrating portions of a circuit of audio source and sink devices. Audio source device 202 represents an example of audio source device 102. Audio sink device 204 represents an example of one of audio sink devices 104A-N. While one pair of audio source and sink devices is illustrated in <figref idref="f0002">FIG. 2</figref>, system 200 may include two or more audio sink devices each operating as discussed for audio sink device 204.

0018The portions of the circuit of audio source device 202 include a processing circuit 212 and a wireless communication circuit 214. The processing circuit 212 is configured to process an acoustic signal for transmission to audio sink device 204 via wireless communication link 206 and includes an audio reference oscillator 222 and a sample clock 224. Sample clock 224 times the input sampling of the acoustic signal based on the frequency generated by audio reference oscillator 222. Wireless communication circuit 214 provides an interface to wireless communication link 206 and includes a radio circuit with a communication reference oscillator (also referred to as RF oscillator) 226 for generating an RF (radio frequency) signal for the wireless transmission of the digitized acoustic signal.

0019The portions of the circuit of audio sink device 204 include a processing circuit 216 and a wireless communication circuit 218. The processing circuit 216 is configured to process the acoustic signal transmitted from audio source device 202 via wireless communication link 206 and includes an audio reference oscillator 228 and a sample clock 230. Sample clock 230 times the output sampling of the acoustic signal based on the frequency generated by audio reference oscillator 228. Wireless communication circuit 218 provides an interface to wireless communication link 206 and includes a radio circuit a communication reference oscillator (also referred to as RF oscillator) 232 for demodulating the received acoustic signal.

0020In various embodiments, processing circuit 216 includes a digital signal processor (DSP). Low power DSPs typically employ clock sources based on RC oscillators, voltage controlled oscillators (VCOs) and current controlled oscillators (ICOs). Such oscillators drive the system clocks on these DSPs and ultimately drive the output sample rate (from sample clock 230). Radios operating at high frequencies such as 2.4 GHz require much greater frequency stability and accuracy in order to perform successful over the air reception and transmission of data. Such devices use high accuracy frequency references such as crystal oscillators, resonators, or MEMS type oscillators. The DSP in a hearing instrument may, depending on the application, benefit from the addition of such accurate frequency references. Given the size constraints of a hearing instrument worn on or in the ear it is somewhat impractical to have two crystal reference oscillators within the device, one for the RF reference and one for the audio reference. In other words, using crystal reference oscillators for both communication reference oscillator 232 and audio reference oscillator 228 may not be a practical solution to the problem of providing frequency stability and accuracy.

0021It should be noted that in most cases, sample clock 224 and communication reference oscillator 214 of audio source device 202 are not necessarily locked together, and sample clock 230 and communication reference oscillator 232 of audio sink device 204 are not necessarily locked together, either. Consequently, audio sink device 204 should make no assumption about the relationship between communication (RF) clock and audio clock in standardized communication such a Bluetooth. An audio source device may derive its sample rate from highly accurate reference such as crystal oscillators and the like but do not necessarily have their RF radio frequency and packet timing based on the same oscillator that the audio sample clock is derived from.

0022Therefore it is necessary for the audio sink device to lock its audio sample rate to audio sample rate of the audio source device to avoid dropping or adding samples periodically which would cause audio artifacts. In system 200, audio sink device 204 can adjust its sample clock 230 based on time stamps between packet arrivals and the number of encoded words received in each incoming packet. Audio source device 202 may send a variable number of encoded words in each packet since the timing for the RF packet arrival is not synchronized with the audio encoder. Audio sink device 204, knowing how many system clock signals it should expect between packet arrivals when adjusted for the number of arrived encoded samples, can form an error signal that it can use to drive audio reference oscillator 228. In various embodiments, audio reference oscillator 228 includes an RC oscillator with an adjustable resistance or adjustable capacitance, a VCO, an ICO, or a digital numerically controlled oscillator, such that the output frequency is controllable using the error signal.

0023These adjustments eventually lead to a lock condition where sample clock 230 of audio sink device 204 has reached a tolerable error from audio source device 202. This time to lock depends on the initial offset and the loop bandwidth of the closed loop control system. In order to minimize the initial offset, system 200 uses communication reference oscillator 232 to "preposition" audio reference oscillator 228 prior to receiving any samples from audio source device 202. By sending empty words over the same serial bus used to send the audio when there is active transmission from audio source device 202, the DSP of processing circuit 216 can adjust its controllable audio reference oscillator 228 for a small PPM (parts per million) error in synchronizing to audio reference oscillator 222 of audio source device 202. Once audio source device 202 starts sending data, it will be a short time to achieve the lock since audio sink device 204 has already "trained" its audio reference oscillator 228 to within an acceptable error using communication reference oscillator 232 as a reference.

0024Using this process will also obviate the need for manufacturing calibration which is commonly used to tune each device's operating frequency thus lowering production costs. This process can also compensate for long term drift of the oscillator frequency caused by device aging. Eliminating the need for factory calibration of oscillators will surely save time and money in production of hearing instruments. All of this is made possible by the addition of the highly accurate reference needed for RF wireless communication.

0025In various embodiments, synchronization may have to survive events such as lost packets while maintaining a close relationship between sample clocks 224 and 230. To do this, processing circuit 216 can be aided from the radio of wireless communication circuit 218 by being updated as to the status of a packet. This may include information from the radio as to whether a packet was empty, missed, or received with errors. Such link layer information can be sent through an interface between the radio of wireless communication circuit 218 and the DSP of processing circuit 216. The DSP can use this information to determine whether a packet loss concealment strategy should be employed to substitute the missing audio with silence, a previously received packet, or an interpolated packet based on linear prediction. In addition, this link layer information can be used to maintain the phase and frequency of sample clock 230. In hearing aid applications, in this way relative phase lock can be maintained between the left and right hearing aids (each being an audio sink device) with respect to each other and to the audio source device. Missed packets are much more easily identified by the radio which has employed time division multiple access techniques. These techniques have inherently good timing mechanisms with which to schedule packet arrivals. Should a scheduled receiving event take place without a received packet, the radio will immediately recognize this event and be able to inform the DSP of the missing packet.

0026Packets may be resent from audio source device 202 in an effort to improve the overall packet error rate performance of wireless link 206. These retransmissions can happen as a result of audio source device failing to receive an acknowledgement from audio sink device 204, or they can be sent unconditionally with a number of retransmissions as in the case of a one way or broadcast mode of transmission. In this case, it will be necessary for the link layer to inform the DSP that the successful reception was on a retransmission that may take place at a different time interval from the primary transmission. The DSP can adapt its synchronization timer with information from the link layer of the radio informing the DSP as to which of the transmissions the packet was successfully received. The transmission may be successful on the primary transmission or on one of the retransmission attempts. Without knowing this information from the link layer, the phase and frequency lock may be disturbed since the time of arrival or the packet at the DSP is used as the error signal to maintain the output sample clock.

0027In various embodiments, the radio of wireless communication circuit 218 sends information to the DSP of processing circuit 216 that a packet was missing or was sent as an empty packet to allow the DSP to insert a packet loss concealment (PLC) frame of information. This may be silent packet, replay or Linear predicted frame based on previous frames. In these embodiments, the radio is making this determination instead of relying on the DSP and its first-in-first-out (FIFO) pointer and crude RC timing to determine if a packet did not arrive.

0028In an alternative embodiment, one could use digital techniques to oversample the incoming signals and then interpolate and or decimate the output of a polyphase filter. The drawback is that it may require a higher sample rate and a multistage filter which require additional instructions or operations, thus increasing power consumption of the system. In another alternative embodiment, one could also use the signal processor to determine a missing packet event but with much less accuracy if its adjustable oscillator is prone to inaccuracy, which is the case with most adjustable oscillators.

0029<figref idref="f0003">FIG. 3</figref> is a block diagram illustrating an embodiment of a hearing assistance system 300. System 300 represents an example of system 100 or system 200 and includes a host device 302, a hearing assistance device 304, and communication link 206 providing for wireless communication between host device 302 and hearing assistance device 304. In various embodiments, host device 302 and hearing assistance device 304 may each include one or more devices. For example, host device 302 may include a computer or a computer connected to a communicator, and hearing assistance device 304 may include a single device or a pair of devices such as a pair of left and right hearing aids. In various embodiments, host device 302 includes any device that transmits acoustic signals to hearing assistance device 304. Examples of host device 302 include a hearing aid programmer, a computer, phone, and any devices capable of streaming audio to hearing assistance device 304. In one embodiment, communication link 206 includes a Bluetooth wireless connection.

0030In one embodiment, host device 302 includes an audio streaming device that streams audio to hearing assistance device 304. Examples of the audio streaming device include computer, television set, telephone, and any music or other audio player. In the illustrated embodiment, host device 302 includes a user interface 310, a processing circuit 312, and a wireless communication circuit 314. In various embodiments, user interface 310 includes a presentation device including at least a display screen and an input device. In various embodiments, the presentation device may also include various audial and/or visual indicators, and the user input device may include a computer mouse, a touchpad, a trackball, a joystick, a keyboard, a keypad, and/or various forms of switches. In one embodiment, user interface 310 includes an interactive screen such as a touchscreen functioning as both the presentation device and the input device. Processing circuit 312 represents an example of processing circuit 212 and is configured to perform various functions including those of processing circuit 212 as discussed in this document. Wireless communication circuit 314 represents an example of wireless communication circuit 214 and is configured to perform various functions including those of wireless communication circuit 214 as discussed in this document.

0031Hearing assistance device 304 includes a processing circuit 316 and a communication circuit 318. Processing circuit 316 represents an example of processing circuit 216 and is configured to perform various functions including those of processing circuit 216 as discussed in this document. Wireless communication circuit 318 represents an example of wireless communication circuit 218 and is configured to perform various functions including those of wireless communication circuit 218 as discussed in this document.

0032<figref idref="f0004">FIG. 4</figref> is a block diagram illustrating an embodiment of a pair of hearing aids 404 representing an example of hearing assistance device 304. Hearing aids 404 include a left hearing aid 404L and a right hearing aid 404R. Left hearing aid 404L includes a microphone 440L, a wireless communication circuit 418L, a processing circuit 416L, and a receiver (also known as a speaker) 442L. Microphone 440L receives sounds from the environment of the listener (hearing aid wearer). Wireless communication circuit 418L represents an example of wireless communication circuit 318 and wirelessly communicates with host device 302 and/or right hearing aid 404R, including receiving signals from host device 302 directly or through right hearing aid 404R. Processing circuit 416L represents an example of processing circuit 316 and processes the sounds received by microphone 440L and/or an audio signal received by wireless communication circuit 418L to produce a left output sound. Receiver 442L transmits the left output sound to the left ear canal of the listener.

0033Right hearing aid 404R includes a microphone 440R, a wireless communication circuit 418R, a processing circuit 416R, and a receiver (also known as a speaker) 442R. Microphone 440R receives sounds from the environment of the listener. Wireless communication circuit 418R represents an example of communication circuit 318 and wirelessly communicates with host device 302 and/or left hearing aid 404L, including receiving signals from host device 302 directly or through left hearing aid 404L. Processing circuit 416R represents an example of processing circuit 316 and processes the sounds received by microphone 440R and/or an audio signal received by wireless communication circuit 418R to produce a right output sound. Receiver 442R transmits the right output sound to the right ear canal of the listener.

0034In various embodiments, link layer information is used by processing circuit 216 (including its various examples discussed in this document) for audio phase lock loop (PLL) jitter reduction in system 100 and its various embodiments such as those discussed in this document. In an audio streaming device, maintaining the phase between the transmit clock and the audio rendering clock (such as sample clock 224 and sample clock 230, respectively) is very important. Not only does it ensure high audio quality by keeping the audio FIFO at the receiver (such as one of audio sink devices 104 including its various examples discussed in this document) at a suitable depth, but it also guarantees a defined phase difference between two decoders in a wireless stereo system. Maintaining the phase difference between the two ears is extremely important in providing the user with a high degree of audio fidelity. Differential phase disturbance between the two ears is perceived as a spatial change in sound location which can be quite annoying.

0035In one example, each audio packet is transmitted up to two times. If the first (primary) packet is not acknowledged by the receiver, then audio packet is sent again.

0036At the receiver, the audio PLL measures the time of arrival of the audio packet and uses that as an "error" variable to control its timing loop. Under ideal, error free conditions, the PLL loop will see a train of clean timing events generated by primary packets. These can be used to control the timing loop as defined by a classic PLL. When interference is encountered, the clean timing signal is replaced with a signal which has the following elements added to it: <ol id="ol0001" compact="compact" ol-style=""><li>1. Secondary packets (the audio packet is received at a time (T/2) after the normal expected arrival time of a packet); and</li><li>2. Missing packet (neither primary nor the secondary packet is received).</li></ol> In a classic PLL, these timing errors cause jitter in the loop. To reduce this jitter, a loop filter with a suitable time constant is used. In the present system, such as system 100 and its various embodiments as discussed in this document, link layer information (such as information derived from the link layer of a Bluetooth device) is used to reduce jitter and maintain a fast loop response. To do this, the link layer is configured to provide the following information: <ol id="ol0002" compact="compact" ol-style=""><li>1- Packet type: primary or secondary; and</li><li>2- Missing packet, i.e., neither primary nor secondary packet has been received. Since the delay of each packet arrival time with respect to the expected arrival time is known, the PLL subtracts the known delay of the packet from the actual arrival time to arrive at an "ideal" error value. This is then used to control the loop in the normal manner. <figref idref="f0005">FIG. 5</figref> shows the operation of such an algorithm for a case where packet arrival times are disrupted due to interference and corrected by applying link layer information.</li></ol>

0037It is understood that variations in communications protocols, antenna configurations, and combinations of components may be employed without departing from the scope of the present subject matter. Hearing assistance devices typically include an enclosure or housing, a microphone, hearing assistance device electronics including processing electronics, and a speaker or receiver. It is understood that in various embodiments the receiver is optional. Antenna configurations may vary and may be included within an enclosure for the electronics or be external to an enclosure for the electronics. Thus, the examples set forth herein are intended to be demonstrative and not a limiting or exhaustive depiction of variations.

0038It is further understood that any hearing assistance device may be used without departing from the scope and the devices depicted in the figures are intended to demonstrate the subject matter, but not in a limited, exhaustive, or exclusive sense. It is also understood that the present subject matter can be used with a device designed for use in the right ear or the left ear or both ears of the wearer.

0039It is understood that the processing circuit of the hearing aids referenced in this document (e.g., processing circuit 216 and its various examples) may be a digital signal processor (DSP), microprocessor, microcontroller, other digital logic, or combinations thereof. The processing of signals referenced in this application can be performed using the processing circuit. Processing may be done in the digital domain, the analog domain, or combinations thereof. Processing may be done using subband processing techniques. Processing may be done with frequency domain or time domain approaches. Some processing may involve both frequency and time domain aspects. For brevity, in some examples drawings may omit certain blocks that perform frequency synthesis, frequency analysis, analog-to-digital conversion, digital-to-analog conversion, amplification, audio decoding, and certain types of filtering and processing. In various embodiments the processor is adapted to perform instructions stored in memory which may or may not be explicitly shown. Various types of memory may be used, including volatile and nonvolatile forms of memory. In various embodiments, instructions are performed by the processor to perform a number of signal processing tasks. In such embodiments, analog components are in communication with the processor to perform signal tasks, such as microphone reception, or receiver sound embodiments (i.e., in applications where such transducers are used). In various embodiments, different realizations of the block diagrams, circuits, and processes set forth herein may occur without departing from the scope of the present subject matter.

0040The present subject matter is demonstrated for hearing assistance devices, including hearing aids, including but not limited to, behind-the-ear (BTE), in-the-ear (ITE), in-the-canal (ITC), receiver-in-canal (RIC), invisible-in-the-canal (IIC) or completely-in-the-canal (CIC) type hearing aids. It is understood that behind-the-ear type hearing aids may include devices that reside substantially behind the ear or over the ear. Such devices may include hearing aids with receivers associated with the electronics portion of the behind-the-ear device, or hearing aids of the type having receivers in the ear canal of the user, including but not limited to receiver-in-canal (RIC) or receiver-in-the-ear (RITE) designs. The present subject matter can also be used in hearing assistance devices generally, such as cochlear implant type hearing devices and such as deep insertion devices having a transducer, such as a receiver or microphone, whether custom fitted, standard, open fitted or occlusive fitted. It is understood that other hearing assistance devices not expressly stated herein may be used in conjunction with the present subject matter.

0041This application is intended to cover adaptations or variations of the present subject matter. It is to be understood that the above description is intended to be illustrative, and not restrictive. The scope of the present subject matter should be determined with reference to the appended claims, along with the full scope of legal equivalents to which such claims are entitled.