Multi-orientation playback device microphones
20 claims: 3 independent, 17 dependent
- 1プロセッサによって実行されると、再生デバイスに機能を実行させる命令を記憶した、有形の非一時的コンピュータ読取り可能媒体であって、 当該機能は、 再生デバイスの向きを特定するステップ、ここで、再生デバイスは少なくとも1つのマイクロフォンアレイを備える 、 複 数のマイクロフォン訓練応答から 、特定した再生デバイスの向きに対応する、 再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップ、ここで、複数のマイクロフォン訓練応答は測定値であ り、それぞれのマイクロフォン訓練応答は、少なくとも1つのマイクロフォンアレイから得られた、水平向きに関する第1グループの応答と、鉛直向きに関する第2グループの応答とを含む 、 少なくとも1つのマイクロフォンアレイを介して、音声入力を検出するステップ、 特定された少なくとも1つのマイクロフォン訓練応答および検出された音声入力に基づいて、音声入力の音源の位置を特定するステップ、 特定された音源の位置に基づいて、少なくとも1つのマイクロフォンアレイにおける指向性の焦点を調整するステップ、 少なくとも1つのマイクロフォンアレイにおける調整された指向性の焦点に基づいて、音声入力を捕捉するステップ、を含む、コンピュータ読取り可能媒体。
- 2機能は、再生デバイスの向きに基づいて、音声の捕捉を可能とする少なくとも1つのマイクロフォンアレイを特定するステップをさらに含む、請求項1に記載のコンピュータ読取り可能媒体。
- 3再生デバイスの向きに基づいて、複数のマイクロフォン訓練応答から再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップは、 複数のマイクロフォン訓練応答から、再生デバイスの向きに対応する、少なくとも1つのマイクロフォンアレイに関するマイクロフォン訓練応答のセットを特定するステップを含む、請求項1に記載のコンピュータ読取り可能媒体。
- 4少なくとも1つのマイクロフォン訓練応答および検出された音声入力に基づいて、音声入力の音源の位置情報を特定するステップは、 検出された音声入力をマイクロフォン訓練応答のセットと比較するステップと、 少なくとも1つのマイクロフォンアレイそれぞれに対する検出された音声入力のそれぞれの方向を特定するステップと、を含む、請求項3に記載のコンピュータ読取り可能媒体。
- 5検出された音声入力をマイクロフォン訓練応答のセットと比較するステップは、 マイクロフォン訓練応答のセットが有する大きさおよび位相の少なくとも一方を、検出された音声入力が有する大きさおよび位相の少なくとも一方と比較するステップを含む、請求項4に記載のコンピュータ読取り可能媒体。
- 6再生デバイスは第1マイクロフォンアレイと第2マイクロフォンアレイとを備え、 機能は、音声入力を捕捉するための第1マイクロフォンアレイを選択するステップをさらに含み、 再生デバイスの向きに基づいて、複数のマイクロフォン訓練応答から再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップは、 第1マイクロフォンアレイおよび再生デバイスの向きに対応するマイクロフォン訓練応答のセットを特定するステップを含む、請求項1に記載のコンピュータ読取り可能媒体。
- 7再生デバイスは第1マイクロフォンアレイと第2マイクロフォンアレイとをさらに含み、 方法はさらに、第1マイクロフォンアレイの第1向きおよび第2マイクロフォンアレイの第2向きに基づいて、第1マイクロフォンアレイおよび第2マイクロフォンアレイを選択するステップをさらに含む、請求項1に記載のコンピュータ読取り可能媒体。
- 8プロセッサと、 プロセッサによって実行されると、再生デバイスに機能を実行させる命令を記憶するメモリと、を備え、 当該機能は、 再生デバイスの向きを特定するステップ、ここで、再生デバイスは少なくとも1つのマイクロフォンアレイを備える 、 複 数のマイクロフォン訓練応答から 、特定した再生デバイスの向きに対応する、 再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップ、ここで、複数のマイクロフォン訓練応答は測定値であ り、それぞれのマイクロフォン訓練応答は、少なくとも1つのマイクロフォンアレイから得られた、水平方向に関する第1グループの応答と、鉛直向きに関する第2グループの応答とを含む 、 少なくとも1つのマイクロフォンアレイを介して、音声入力を検出するステップ、 特定された少なくとも1つのマイクロフォン訓練応答および検出された音声入力に基づいて、音声入力の音源の位置を特定するステップ、 特定された音源の位置に基づいて、少なくとも1つのマイクロフォンアレイにおける指向性の焦点を調整するステップ、 少なくとも1つのマイクロフォンアレイにおける調整された指向性の焦点に基づいて、音声入力を捕捉するステップ、を含む、再生デバイス。
- 9機能は、再生デバイスの向きに基づいて、音声の捕捉を可能とする少なくとも1つのマイクロフォンアレイを特定するステップをさらに含む、請求項8に記載の再生デバイス。
- 10再生デバイスの向きに基づいて、複数のマイクロフォン訓練応答から再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップは、 複数のマイクロフォン訓練応答から、再生デバイスの向きに対応する、少なくとも1つのマイクロフォンアレイに関するマイクロフォン訓練応答のセットを特定するステップを含む、請求項8に記載の再生デバイス。
- 11少なくとも1つのマイクロフォン訓練応答および検出された音声入力に基づいて、音声入力の音源の位置情報を特定するステップは、 検出された音声入力をマイクロフォン訓練応答のセットと比較するステップと、 少なくとも1つのマイクロフォンアレイそれぞれに対する検出された音声入力のそれぞれの方向を特定するステップと、を含む、請求項10に記載の再生デバイス。
- 12検出された音声入力をマイクロフォン訓練応答のセットと比較するステップは、 マイクロフォン訓練応答のセットが有する大きさおよび位相の少なくとも一方を、検出された音声入力が有する大きさおよび位相の少なくとも一方と比較するステップを含む、請求項11に記載の再生デバイス。
- 13再生デバイスは第1マイクロフォンアレイと第2マイクロフォンアレイとを備え、 機能は、音声入力を捕捉するための第1マイクロフォンアレイを選択するステップをさらに含み、 再生デバイスの向きに基づいて、複数のマイクロフォン訓練応答から再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップは、 第1マイクロフォンアレイおよび再生デバイスの向きに対応するマイクロフォン訓練応答のセットを特定するステップを含む、請求項8に記載の再生デバイス。
- 14再生デバイスは第1マイクロフォンアレイと第2マイクロフォンアレイとをさらに含み、 方法はさらに、第1マイクロフォンアレイの第1向きおよび第2マイクロフォンアレイの第2向きに基づいて、第1マイクロフォンアレイおよび第2マイクロフォンアレイを選択するステップをさらに含む、請求項8に記載の再生デバイス。
- 15再生デバイスの向きを特定するステップであって、再生デバイスは少なくとも1つのマイクロフォンアレイを備える、ステップと 、 複 数のマイクロフォン訓練応答から 、特定した再生デバイスの向きに対応する、 再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップと、ここで、複数のマイクロフォン訓練応答は測定値であ り、それぞれのマイクロフォン訓練応答は、少なくとも1つのマイクロフォンアレイから得られた、水平向きに関する第1グループの応答と、鉛直向きに関する第2グループの応答とを含む 、 少なくとも1つのマイクロフォンアレイを介して、音声入力を検出するステップと、 特定された少なくとも1つのマイクロフォン訓練応答および検出された音声入力に基づいて、音声入力の音源の位置を特定するステップと、 特定された音源の位置に基づいて、少なくとも1つのマイクロフォンアレイにおける指向性の焦点を調整するステップと、 少なくとも1つのマイクロフォンアレイにおける調整された指向性の焦点に基づいて、音声入力を捕捉するステップと、を含む、方法。
- 16機能は、再生デバイスの向きに基づいて、音声の捕捉を可能とする少なくとも1つのマイクロフォンアレイを特定するステップをさらに含む、請求項15に記載の方法。
- 17再生デバイスの向きに基づいて、複数のマイクロフォン訓練応答から再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップは、 複数のマイクロフォン訓練応答から、再生デバイスの向きに対応する、少なくとも1つのマイクロフォンアレイに関するマイクロフォン訓練応答のセットを特定するステップを含む、請求項15に記載の方法。
- 18少なくとも1つのマイクロフォン訓練応答および検出された音声入力に基づいて、音声入力の音源の位置情報を特定するステップは、 検出された音声入力をマイクロフォン訓練応答のセットと比較するステップと、 少なくとも1つのマイクロフォンアレイそれぞれに対する検出された音声入力のそれぞれの方向を特定するステップと、を含む、請求項17に記載の方法。
- 19検出された音声入力をマイクロフォン訓練応答のセットと比較するステップは、 マイクロフォン訓練応答のセットが有する大きさおよび位相の少なくとも一方を、検出された音声入力が有する大きさおよび位相の少なくとも一方と比較するステップを含む、請求項18に記載の方法。
- 20再生デバイスは第1マイクロフォンアレイと第2マイクロフォンアレイとを備え、 方法は、音声入力を捕捉するための第1マイクロフォンアレイを選択するステップをさらに含み、 再生デバイスの向きに基づいて、複数のマイクロフォン訓練応答から再生デバイスに関する少なくとも1つのマイクロフォン訓練応答を特定するステップは、 第1マイクロフォンアレイおよび再生デバイスの向きに対応するマイクロフォン訓練応答のセットを特定するステップを含む、請求項15に記載の方法。
Independent claims20
127 paragraphs, as filed
Cross-reference of related applications
This application is a continuation application claiming priority based on U.S. Patent Application No. 15 / 282,554 "Multidirectional Playback Device Microphone" filed on September 30, 2016, and the content of the U.S. Patent Application is referenced. This is incorporated herein by reference in its entirety.
This application relates to consumer products, in particular to methods, systems, products, features, services, and other elements directed at media reproduction, and some aspects thereof.
In 2003, Sonos Incorporated filed a patent application entitled "How to Synchronize Audio Playback Between Multiple Network Devices", one of the first patent applications, and sold the media playback system in 2005. Until it started, the options for accessing and auditioning digital audio in the outloud settings were limited. The Sonos Wireless HiFi system allows people to experience music from many sources via one or more network playback devices. Through software control applications installed on smartphones, tablets, or computers, people can play the music they want in any room equipped with a network playback device. You can also use a controller, for example, to stream different songs to each room with a playback device, group multiple rooms for synchronous playback, or synchronize in all rooms. You can also listen to the same song.
Given the growing interest in digital media so far, there is a need to further open up consumer-accessible technologies that can further improve the listening experience.
The features, embodiments, and advantages of the techniques disclosed herein are more easily understood with reference to the following description, the appended claims, and the accompanying drawings.
<figref num="1">The figure which shows the structure of the exemplary media reproduction system which can be carried out in a certain embodiment.</figref><figref num="2">Functional block diagram of an exemplary playback device</figref><figref num="3">Functional block diagram of an exemplary control device</figref><figref num="4">Diagram showing an exemplary controller interface</figref><figref num="5">Diagram showing exemplary network devices</figref><figref num="6">Functional block diagram of an exemplary network microphone device</figref><figref num="7A">Perspective view showing the orientation of the playback device</figref><figref num="7B">Perspective view showing the orientation of the playback device</figref><figref num="8">An exemplary flow diagram of how to process audio input based on the orientation of the playback device</figref><figref num="9">Top view of an exemplary calibration setting</figref>
The drawings are intended to illustrate some exemplary embodiments, but it is understood that the invention is not limited to the arrangements and means shown in the drawings.
I. Overview Some embodiments described herein include adjusting the directional focus of at least one microphone array based on the orientation of the playback device. Microphone arrays for multi-directional devices may have different sensitivity and response profiles for audio coming from the same direction across different orientations of the device. When acquiring training response values (eg, calibration response values, measurement response values) for microphone arrays in different playback device orientations, it is possible to create general response profiles for different playback device orientations. When comparing a typical response profile to the response profile of a received voice input response, the location of the sound source of the voice input (eg, orientation in one or more dimensions) can be identified or estimated.
Examples provided herein include methods, playback devices, and systems. The method includes determining the orientation of the playback device, including at least one microphone array, and determining at least one microphone training response for the playback device from multiple microphone training responses based on the orientation of the playback device. .. At least one microphone array can detect the voice input, and the location information of the sound source of the voice input can be determined based on at least one microphone training response and the detected voice input. The directional focus of at least one microphone array can be adjusted based on the location information of the sound source, and the audio input can be captured based on the adjusted directional focus.
In another aspect, a non-temporary computer-readable medium is provided. A non-transient computer-readable medium stores instructions that cause a computing device to perform a function when executed by the computing device. The function includes determining the orientation of a playback device containing at least one microphone array and determining at least one microphone training response for the playback device from multiple microphone training responses based on the orientation of the playback device. .. At least one microphone array can detect the voice input, and the location information of the sound source of the voice input can be determined based on at least one microphone training response and the detected voice input. The directional focus of at least one microphone array can be adjusted based on the location information of the sound source, and the audio input can be captured based on the adjusted directional focus.
In yet another aspect, the device is provided. The device includes a processor and memory. Memory stores instructions that cause the system to perform functions when executed by the device. The function includes determining the orientation of a playback device containing at least one microphone array and determining at least one microphone training response for the playback device from multiple microphone training responses based on the orientation of the playback device. .. At least one microphone array can detect the voice input, and the location information of the sound source of the voice input can be determined based on at least one microphone training response and the detected voice input. The directional focus of at least one microphone array can be adjusted based on the location information of the sound source, and the audio input can be captured based on the adjusted directional focus.
Some examples described herein refer to functions performed by a given actor, such as a "user" and / or other entity, but this is for illustration purposes only. Unless explicitly required by the words of the claims themselves, it should not be construed as requiring action by such exemplary actors. Those skilled in the art will appreciate that many other embodiments are included in this disclosure.
II. Examples of Operating Environment Figure 1 shows an exemplary configuration of a media playback system 100 that can be implemented or implemented in one or more embodiments disclosed herein. As illustrated, the media playback system 100 is associated with an exemplary home environment having multiple rooms and spaces, such as a master bedroom, an office, a dining room, and a living room. As shown in the example of FIG. 1, the media playback system 100 includes playback devices 102-124, control devices 126 and 128, and a wired or wireless network router 130.
Further, a description of the different components of the exemplary media playback system 100 and how the different components work to provide the user with a media experience is provided in the sections below. Although the description herein generally refers to the media playback system 100, the techniques described herein are not limited to the home environment applications shown in FIG. For example, the techniques described herein include environments where multi-zone audio is desired, such as commercial environments such as restaurants, malls, or airports, sport utility vehicles (SUVs), buses or cars. It is useful in environments such as vehicles, ships, or boards and airplanes.
Exemplary Zone Players FIG. 2 shows a functional block diagram of an exemplary playback device 200 that comprises one or more of the playback devices 102-124 of the media playback system 100 of FIG. The reproduction device 200 may include a processor 202, a software component 204, a memory 206, an audio processing component 208, an audio amplifier 210, a speaker 212, and a network interface 214. Network interface 214 includes wireless interface 216, wired interface 218 and microphone 220. In some cases, the reproduction device 200 does not include the speaker 212, but may include a speaker interface for connecting the reproduction device 200 to an external speaker. In other cases, the reproduction device 200 does not include the speaker 212 or the audio amplifier 210, but may include an audio interface for connecting the reproduction device 200 to an external audio amplifier or audiovisual receiver.
In one example, processor 202 may be a clock-driven computer component configured to process input data based on instructions stored in memory 206. The memory 206 may be a non-temporary computer-readable recording medium configured to store instructions that can be executed by the processor 202. For example, the memory 206 may be data storage capable of loading one or more of the software components 204 that can be executed by the processor 202 to perform a function. In one example, the function may include the step of the playback device 200 reading audio data from an audio source or another playback device. In another example, the function may include the step of the playback device 200 transmitting audio data to another device or playback device on the network. In yet another example, the function may include pairing the playback device 200 with one or more playback devices to create a multi-channel audio environment.
One feature comprises the step of the playback device 200 synchronizing the playback of audio content with one or more other playback devices. While synchronizing the playback, it is preferable that the listener is unaware of the delay between the playback of the audio content by the playback device 200 and the playback by one or more other playback devices. US Pat. No. 8,234,395, entitled "Systems and Methods for Synchronizing Operations Between Multiple Independent Clock Digital Data Processing Devices," is incorporated herein by reference, which may synchronize audio playback between playback devices. It provides a more detailed example as described.
Further, the memory 206 may be configured to store data. The data may be stored in, for example, a playback device 200 such as a playback device 200, an audio source accessible by the playback device 200, or a playback device 200 (or other playback) included as part of one or more zones and / or zone groups. It is associated with a play queue, which can be associated with the device). The data may be updated periodically and stored as one or more state variables indicating the state of the playback device 200. The memory 206 may also contain data associated with the state of other devices in the media system, and by sharing between the devices at any time, one or more devices may have near-most recent data associated with the system. Can have. Other embodiments are also possible.
The audio processing component 208 may include one or more digital-to-analog converters (DACs), audio processing components, audio enhancement components, digital signal processors (DSPs), and the like. In certain embodiments, the audio processing component 208 may be a subcomponent of the processor 202. In certain embodiments, the audio content may be processed and / or deliberately modified by the audio processing component 208 to generate an audio signal. The generated audio signal is transmitted to the audio amplifier 210, amplified, and reproduced through the speaker 212. In particular, the audio amplifier 210 may include a device configured to amplify the audio signal to a level capable of driving one or more speakers 212. The speaker 212 may include a complete speaker system including an independent transducer (eg, a "driver") or a housing containing one or more drivers. Certain drivers included in the speaker 212 may include, for example, a subwoofer (eg, for low frequencies), a middle range driver (eg, for intermediate frequencies), and / or a tweeter (for high frequencies). In some cases, each converter of one or more speakers 212 may be driven by the corresponding individual audio amplifiers of the audio amplifier 210. In addition to generating an analog signal to be played on the playback device 200, the audio processing component 208 processes the audio content and sends the audio content for playback by one or more other playback devices.
Audio content processed and / or played by the playback device 200 is received via an external source, such as an audio line-in input connection (eg, an auto-detecting 3.5 mm audio line-in connection) or network interface 214. May be good.
The network interface 214 may be configured to allow data flow between the playback device 200 and one or more other devices on the data network. Thus, the playback device 200 is a data network from one or more other playback devices communicating with the playback device, a network device within a local area network, or an audio content source on a wide area network such as, for example, the Internet. It may be configured to receive audio content via. In one example, audio content and other signals transmitted and received by the playback device 200 may be transmitted in the form of digital packets containing source addresses based on Internet Protocol (IP) and destination addresses based on IP. In such a case, the network interface 214 can appropriately receive and process the data addressed to the reproduction device 200 by the reproduction device 200 by analyzing the digital packet data.
As shown, the network interface 214 may include a wireless interface 216 and a wired interface 218. The wireless interface 216 provides network interface functionality for the playback device 200 and is wireless, including communication protocols such as the wireless standards IEEE802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 802.11ac, 4G mobile communication standards, etc. Wireless communication with other devices (eg, other playback devices, speakers, receivers, network devices, control devices in the data network associated with the playback device 200) based on any of the standards) good. The wired interface 218 provides a network interface function for the playback device 200, and may communicate via a wired connection with another device based on a communication protocol (eg, IEEE802.3). Although the network interface 214 shown in FIG. 2 includes both the wireless interface 216 and the wired interface 218, the network interface 214 may include only the wireless interface or only the wired interface in certain embodiments. good.
The microphone 220 may be configured to detect sounds in the environment of the playback device 200. The microphone may be attached, for example, to the outer wall of the housing of the reproduction device. The microphone may be any type of microphone currently known or later developed, such as condenser microphones, electret condenser microphones, or dynamic microphones. The microphone may be sensitive to a portion of the frequency range of the speaker 220. One or more of the speakers 220 may operate in the reverse of the microphone 220. In some embodiments, the reproduction device 200 may not include the microphone 220.
In one example, the playback device 200 and other playback devices may be paired to play two separate audio components of audio content. For example, the playback device 200 may be configured to play the left channel audio component, while the other playback device may be configured to play the right channel audio component. This can generate or enhance the stereo effect of the audio content. The paired playback device (also referred to as a "combined playback device") may further play audio content in synchronization with another playback device.
In another example, the reproduction device 200 may be acoustically integrated with one or more other reproduction devices to form a single integrated reproduction device (integrated reproduction device). The integrated playback device can be configured to process and reproduce sound differently than a non-integrated playback device or a paired playback device. This is because the integrated playback device can add speakers to play audio content. For example, if the playback device 200 is designed to play low frequency range audio content (eg, a subwoofer), the playback device 200 is designed to play full frequency range audio content. It may be integrated with the device. In this case, the full frequency range playback device may be configured to play only the mid-high frequency components of the audio content when integrated with the low frequency playback device 200. On the other hand, the low frequency range playback device 200 plays low frequency components of audio content. Further, the integrated playback device may be paired with a single playback device or even another integrated playback device.
As an example, Sonos Incorporated currently plays including "PLAY: 1", "PLAY: 3", "PLAY: 5", "PLAYBAR", "CONNECT: AMP", "CONNECT", and "SUB". We sell and offer devices. Any other past, present, and / or future reproduction device may be additionally or optionally implemented and used in the reproduction device of the embodiments disclosed herein. Further, it is understood that the reproduction device is not limited to the particular example shown in FIG. 2 or the Sonos product provided. For example, the playback device may include wired or wireless headphones. In another example, the playback device may include or interact with a docking station for a personal mobile media playback device. In yet another example, the reproduction device may be integrated with another device or component, such as a television, luminaire, or some other device for indoor or outdoor use.
b. Illustrative Playback Zone Configuration Returning to the media playback system of Figure 1, the environment has one or more playback zones, each playback zone containing one or more playback devices. .. The media reproduction system 100 may be formed of one or more reproduction zones, and one or more zones may be added or deleted later to form an exemplary configuration shown in FIG. Each zone may be given a name based on a different room or space, such as an office, bathroom, master bedroom, bedroom, kitchen, dining room, living room, and / or balcony. In some cases, a single regeneration zone may include multiple rooms or spaces. In other cases, a single room or space may include multiple regeneration zones.
As shown in Figure 1, each of the balcony, dining room, kitchen, bathroom, office, and bedroom zones has one playback device, while each of the living room and master bedroom zones has multiple playback devices. Has. The living room zone has playback devices 104, 106, 108, and 110 as separate playback devices, one or more combined playback devices, one or more integrated playback devices, or any of these. The combination may be configured to play audio content in synchronization. Similarly, in the case of the master bedroom, the playback devices 122 and 124 may be configured to play audio content synchronously as separate playback devices, combined playback devices, or integrated playback devices. ..
In one example, one or more playback zones in the environment of FIG. 1 are playing different audio content. For example, the user can listen to hip-hop music played by the playback device 102 while grilling in the balcony zone. Meanwhile, another user can listen to classical music played by the playback device 114 while preparing a meal in the kitchen zone. In another example, the playback zone may play the same audio content in sync with another playback zone. For example, if the user is in the office zone, the office zone playback device 118 may play the same music that is being played on the balcony playback device 102. In such cases, the playback devices 102 and 118 are playing the rock music synchronously so that the user can seamlessly (or at least at least) play the audio content out-loud as they move between different playback zones. You can enjoy it almost seamlessly). Synchronization between reproduction zones may be performed in the same manner as synchronization between reproduction devices as described in US Pat. No. 8,234,395 above.
As described above, the zone configuration of the media playback system 100 may be dynamically changed, and in certain embodiments, the media playback system 100 supports a plurality of configurations. For example, if the user physically moves one or more playback devices to or out of the zone, the media playback system 100 may be reconfigured to accommodate the changes. For example, if the user physically moves the playback device 102 from the balcony zone to the office zone, the office zone may include both the playback device 118 and the playback device 102. If desired, the playback devices 102 may be paired, grouped into office zones, and / or renamed via control devices, such as control devices 126 and 128. On the other hand, when one or more playback devices are moved to a certain area in a home environment in which a playback zone has not yet been set, a new playback zone may be formed in that area.
Further, the different playback zones of the media playback system 100 may be dynamically combined into zone groups or divided into separate playback zones. For example, by combining the dining room zone and the kitchen zone 114 into a zone group for a dinner party, the playback devices 112 and 114 can play audio content in synchronization. On the other hand, if one user wants to watch TV while another user wants to listen to music in the living room space, the living room zone is the TV zone containing the playback device 104 and the listening zone containing the playback devices 106, 108 and 110. And may be divided into.
c. Illustrative Control Device Figure 3 shows a functional block diagram of an exemplary control device 300 that constitutes one or both of the control devices 126 and 128 of the media playback system 100. As shown, the control device 300 may include a processor 302, a memory 304, a network interface 306, a user interface 308, a microphone 310, and a software component 312. In one example, the control device 300 may be a control device dedicated to the media playback system 100. In another example, the control device 300 is a network device with the media playback system controller application software installed, such as an iPhone®, iPad®, or any other smartphone, tablet or network device (eg). , PC or network computer such as Mac®).
Processor 302 may be configured to perform functions related to enabling user access, control, and configuration of the media playback system 100. The memory 304 may be a data storage that can be loaded with one or more software components that are executed and function by the processor 302. The memory 304 may also be configured to store media playback system controller application software and other data associated with the media playback system 100 and the user.
In one example, network interface 306 is an industrial standard (eg, infrared, wireless, wired standards such as IEEE802.3, wireless standards such as IEEE802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 802.11ac, 802.11ac, 4G communication. It may be based on standards, etc.). In the network interface 306, the control device 300 may provide a means for communicating with other devices in the media playback system 100. In one example, data and information (eg, state variables) may be communicated between the control device 300 and other devices via network interface 306. For example, the configuration of playback zones and zone groups in the media playback system 100 may be received by the control device 300 from the playback device or another network device, or by the control device 300 via the network interface 306. It may be transmitted to a playback device or a network device. In some cases, the other network device may be another control device.
Playback device control commands such as volume control and audio playback control may be communicated from the control device 300 to the playback device via network interface 306. As described above, the configuration of the media playback system 100 can be changed by the user using the control device 300. Configuration changes include adding one or more playback devices to a zone, removing one or more playback devices from a zone, adding one or more zones to a zone group, one or more. Zones may include removing a zone from a zone group, forming a combined player or integrated player, dividing the combined player or integrated player into one or more playback devices, and the like. As described above, the control device 300 may be referred to as a controller, and the control device 300 may be a dedicated controller in which the media playback system controller application software is installed, or a network device.
The control device 300 may include a microphone 310. The microphone 310 may be configured to detect sounds in the environment of the control device 300. The microphone 310 may be any type of microphone currently known or developed in the future, such as condenser microphones, electret condenser microphones, and dynamic microphones. The microphone may be sensitive to some frequency range. Two or more microphones 310 may be provided to obtain location information of a sound source (eg, voice, audible sound) and / or to assist in filtering background noise.
The user interface 308 of the control device 300 may be configured to allow user access and control of the media playback system 100 by providing a controller interface such as the controller interface 400 shown in FIG. .. The controller interface 400 includes a reproduction control area 410, a reproduction zone area 420, a reproduction status area 430, a reproduction queue area 440, and an audio content source area 450. The illustrated user interface 400 is merely an example of a user interface provided with a network device (and / or control devices 126 and 128 of FIG. 1) such as the control device 300 of FIG. 3, and is played back by the user. It is accessed to control a media playback system such as System 100. Alternatively, various formats, styles, and interactive sequences may be implemented in the interface of another user on one or more network devices to provide similar control access to the media playback system.
Playback control area 410 may include selectable icons (eg, by touch or cursor). This icon allows playback devices in the selected playback zone or zone group to play or stop, fast forward, rewind, then skip, skip before, shuffle mode on / off, repeat mode on / off, cross. Turns fade mode on / off. The reproduction control area 410 may include another selectable icon. Other selectable icons may change other settings such as equalization settings, playback volume, etc.
The playback zone region 420 may include a display of playback zones within the media playback system 100. In certain embodiments, the graphic display of the reproduction zone may be selectable. Playback zones within the media playback system can be managed or configured by moving additional selectable icons. Other management or configuration can be performed, for example, creating joined zones, creating zone groups, splitting zone groups, and renaming zone groups.
For example, as illustrated, a "group" icon may be provided for each of the graphic displays of the playback zone. The "Group" icon in the graphic display of a zone may be selectable so that one or more zones in the media playback system can be selected to give an option to group with the zone. Once grouped, the playback devices in one zone and the grouped zone are configured to play audio content in sync with the playback devices in the zone. Similarly, the "group" icon may be provided within the graphic display of the zone group. In this case, the Group icon can be selected to give the option to deselect one or more zones in the zone group in order to remove one or more zones in the zone group from the zone group. There may be. Other interactions for grouping and ungrouping zones are possible and can be performed via a user interface such as the user interface 400. The display of the reproduction zone in the reproduction zone area 420 may be dynamically updated when the reproduction zone or zone group configuration is changed.
The playback status area 430 provides a graphic display of the currently played audio content, the previously played audio content, or the next audio content scheduled to be played within the selected playback zone or zone group. It may be included. Selectable play zones or groups may be visually distinguished on the user interface, eg, within play zone areas 420 and / or play status areas 430. The graphic display contains other relevant information that is useful to the user when controlling the track title, artist name, album name, album year, track length, and media playback system through the user interface 400. May be good.
The play queue area 440 may include a graphic display of audio content in the play queue associated with the selected play zone or zone group. In certain embodiments, each play zone or zone group may be associated with a play queue containing information corresponding to zero or more audio items played by the play zone or play group. For example, each audio item in the play queue contains a URI (URI), a URL (URL), or another identifier that can be used by a play zone or playback device in a zone group. You may. These allow audio items to be found and / or retrieved from a local audio content source or a network audio content source and played by a playback device.
In some examples, playlists may be added to the play queue. In this case, the information corresponding to each audio item in the playlist may be added to the play queue. In another example, the audio items in the play queue may be saved as playlists. In yet another example, when the playback device continues to play streaming audio content, eg, an internet radio that plays continuously unless stopped, rather than an audio item that does not play continuously due to having a play time. The replay queue may be empty or "unused" but filled. In another embodiment, the play queue may contain internet radio and / or other streaming audio content items and shall be "unused" when the play zone or zone group is playing those items. Can be done. Other examples are possible.
When a replay zone or zone group is "grouped" or "ungrouped", the replay queue associated with the affected replay zone or zone group may be cleared or reassociated. You may. For example, if the first replay zone containing the first replay queue is grouped with the second replay zone containing the second replay queue, the formed zone group may have an associated replay queue. .. The associated play queue is initially empty, contains audio items in the first play queue (for example, if a second play zone is added to the first play zone), or contains an audio item in the first play zone (for example, first play). If a zone is added to the second play zone), it can contain audio items in the second play queue, or it can be combined with both audio items in the first play queue and the second play queue. If the formed zone group is then ungrouped, the ungrouped first play zone may be associated again with the previous first play queue or with an empty new play queue. Alternatively, it may be associated with a new play queue that contains audio items from the play queue that were associated with the zone group before it was ungrouped. Similarly, the ungrouped second play zone may be associated again with the previous second play queue, with an empty new play queue, or before the zone group is ungrouped. May be associated with a new play queue that contains audio items from the play queue that was associated with the zone group. Other examples are possible.
Returning to the user interface 400 in Figure 4, the graphic display of the audio content in the play queue area 440 shows the track title, artist name, track length, and other relevant information associated with the audio content in the play queue. It may be included. In one example, the graphic display of audio content can be moved by selecting additional selectable icons. This allows you to manage and / or manipulate the audio content displayed in the play queue and / or play queue. For example, the displayed audio content may be removed from the play queue, moved to a different position in the play queue, played immediately, or played after the currently playing audio content. It may be selected for, or it may perform other actions. The replay queue associated with a replay zone or zone group is the memory of one or more replay devices within the replay zone or zone group, the memory of replay devices not within the replay zone or zone group, and / or other designations. It may be stored in the memory of the device.
The audio content source area 450 may include a graphic display of selectable audio content sources. In this audio content source, the audio content may be retrieved and played by the selected playback zone or zone group. A description of audio content sources can be found in the following sections.
d. Illustrative Audio Content Sources As previously illustrated, one or more playback devices within a zone or zone group play audio content (eg, based on the corresponding URI or URL of the audio content). ) May be configured to retrieve from multiple available audio content sources. In one example, audio content may be retrieved directly from the corresponding audio content source (eg, line-in connection) by the playback device. In another example, the audio content may be provided to a playback device on the network via one or more other playback devices or network devices.
An exemplary audio content source may include the memory of one or more playback devices in the media playback system. Examples of the media playback system include the media playback system 100 of FIG. 1, a local music library on one or more network devices (eg, a control device, a network-enabled personal computer, or network-attached storage (NAS)). A streaming audio service that provides audio content over the Internet (eg, the cloud), or an audio source that is connected to a media playback system via a line-in input connection on a playback device or network device, or any other possible system. You may.
In certain embodiments, the audio content source may be periodically added to or removed periodically from a media playback system such as the media playback system 100 of FIG. In one example, audio items may be indexed each time one or more audio content sources are added, removed, or updated. Indexing audio items may include scanning identifiable audio items in all folders / directories shared on the network. Here, the network is accessible by the playback device in the media playback system. Indexing of audio items also includes creating or updating an audio content database that contains metadata (eg, title, artist, album, track length, etc.) and other relevant information. good. Other relevant information may include, for example, a URI or URL for finding each identifiable audio item. Other examples for managing and maintaining audio content sources are possible.
The above description of playback devices, control devices, playback zone configurations, and media content sources is only a partial example of an operating environment in which the features and methods described below can be implemented. With respect to media playback systems, playback devices, and network devices, other operating environments and configurations not expressly described herein are similarly applicable and suitable for implementing this feature and this method. there is a possibility.
e. Multiple Illustrative Network Devices Figure 5 shows a plurality of exemplary devices 500 configured to provide an audio playback experience based on voice control. Those skilled in the art will appreciate that the devices shown in FIG. 5 are for illustrative purposes only and variations including different and / or additional devices may be feasible. As shown, the plurality of devices 500 include computing devices 504, 506, and 508, network microphone devices (NMD) 512, 514, and 516, and playback devices (PBD) 532, 534, 536, and 538. And the control device (CR) 522.
Each of the multiple devices 500 is NFC, Bluetooth® over one or more types of networks, such as wide area networks (WANs), local area networks (LANs), and personal area networks (PANs). , Ethernet, and IEEE802.11 and other network protocols, may be network-enabled devices capable of establishing communication with one or more other devices in multiple devices.
As shown, the computing devices 504, 506, and 508 may be part of the cloud network 502. The cloud network 502 may include additional computing devices. In one example, the computing devices 504, 506, and 508 may be different servers, and in another example, two or more of the computing devices 504, 506, and 508 are modules of a single server. You may. Similarly, each of the computing devices 504, 506, and 508 may include one or more modules or servers. For ease of illustration herein, each of the computing devices 504, 506, and 508 may be configured to perform a particular function within the cloud network 502. For example, the computing device 508 may be a source of audio content for a music streaming service.
As shown, the computing device 504 may be configured to interface with the NMDs 512, 514, and 516 via a communication path 542. NMD512, 514, and 516 may be components of one or more "smart home" systems. In some cases, the NMDs 512, 514, and 516 may be physically located throughout the home, similar to the device placement shown in FIG. In other cases, the two or more NMDs 512, 514, and 516 may be physically placed so that they are relatively close to each other. The communication path 542 may include one or more types of networks, such as WAN, LAN, and / or PAN and others, including the Internet.
In one example, one or more of the NMDs 512, 514, and 516 may be devices configured primarily to perform speech detection. In another example, one or more of NMD512, 514, and 516 may be components of the device with various major utilities. For example, as described above in connection with FIGS. 2 and 3, one or more of the NMDs 512, 514, and 516 may be the microphone (s) 220 of the playback device 200 or the microphone (s) of the network device 300. ) 310 may be. In some cases, one or more of the NMDs 512, 514, and 516 may be the playback device 200 or the network device 300. In one example, one or more of the NMDs 512, 514, and / or 516 may include multiple microphones arranged in a microphone array.
As shown, the computing device 506 may be configured to interface with the CR522 and the PBD 532, 534, 536, and 538 via the communication path 544. In one example, the CR522 may be a network device such as the network device 200 of FIG. Therefore, the CR522 may be configured to provide the controller interface 400 of FIG. Similarly, PBD 532, 534, 536, and 538 may be playback devices such as the playback device 300 of FIG. For this reason, PBDs 532, 534, 536, and 538 may be physically located throughout the home as shown in FIG. For purposes of illustration, PBD 536 and 538 may be part of the binding zone 530, while PBD 532 and 534 may be part of their respective zones to which they belong. As mentioned above, PBD 532, 534, 536, and 538 may be dynamically combined, grouped, uncoupled, and ungrouped. The communication path 544 may include one or more types of networks such as WAN, LAN, and / or PAN and others, including the Internet.
In one example, like NMD512, 514, and 516, CR522 and PBD532, 534, 536, and 538 may also be components of one or more "smart home" systems. In some cases, PBD 532, 534, 536, and 538 may be located in the same household as NMD 512, 514, and 516. Further, as described above, one or more of PBD 532, 534, 536, and 538 may be one or more of NMD 512, 514, and 516.
The NMD 512, 514, and 516 may be part of the local area network, and the communication path 542 links the local area network to which the NMD 512, 514, and 516 belong to the computing device 504 over the WAN (communication). The route may include an access point (not shown). Similarly, each of NMD512, 514, and 516 may communicate with each other via such an access point.
Similarly, the CR522 and PBD532, 534, 536, and 538 may be part of a local area network and / or a local playback network, as described in the previous section, and the communication path 544 may be a CR522 and / or a local playback network. It may include an access point that links the local area network and / or the local playback network to which the PBD 532, 534, 536, and 538 belong to the computing device 506 over the WAN. Therefore, CR522 and each of PBD532, 534, 536, and 538 may also communicate with each other via such access points.
In one example, communication paths 542 and 544 may have the same access point. In one example, each of NMD512, 514, and 516, CR522, and PBD532, 534, 536, and 538 may access cloud network 502 via the same home access point.
As shown in FIG. 5, each of the NMD 512, 514, and 516, CR522, and PBD 532, 534, 536, and 538 also communicates directly with one or more of the other devices via the means of communication 546. You may. The means of communication 546 described herein may include one or more forms of communication between devices by one or more network protocols over one or more types of networks and / or. It may include communication via one or more other network devices. For example, the communication means 546 may include, for example, one or more of Bluetooth (IEEE802.15), NFC, Wireless Direct, and / or proprietary radio and the like.
In one example, the CR522 may communicate with the NMD 532 over Bluetooth and with the PBD 534 over another local area network. In another example, the NMD 514 may communicate with the CR522 over another local area network and with the PBD 536 over Bluetooth. In yet another example, PBD532, 534, 536, and 538 may each communicate with each other over the local replay network according to the Spanning Tree Protocol, while having a local area network that is different from the local replay network. It may communicate with CR522 via each. Other examples are possible.
In some cases, the means of communication between NMD512, 514, and 516, CR522, and PBD532, 534, 536, and 538 will vary depending on the type of communication between devices, network state, and / or latency requirements. May be good. For example, the communication means 546 may be used when the NMD 516 is first introduced into the home with the PBD 532, 534, 536, and 538. In some cases, the NMD516 may send the identification information corresponding to the NMD516 to the PBD538 via NFC, and the PBD538 may respond accordingly to the local area network information via NFC (or some other form of communication). May be sent to NMD516. However, once the NMD516 is installed in the home, the means of communication between the NMD516 and the PBD538 may change. For example, the NMD 516 may communicate with the PBD 538 continuously via the communication path 542, the cloud network 502, and the communication path 544. In another example, NMD and PBD may never communicate via local communication means 546. In yet another example, the NMD and PBD may communicate primarily via local communication means 546. Other examples are possible.
In an exemplary example, the NMDs 512, 514, and 516 may be configured to receive voice inputs to control the PBDs 532, 534, 536, and 538. The control commands available may include controls of any media playback system described above, such as playback volume control, playback transport control, music source selection, and grouping and the like. For example, the NMD 512 may receive voice input to control one or more of the PBD 532, 534, 536, and 538. In response to receiving the voice input, the NMD 512 may send the voice input to the computing device 504 for processing purposes via the communication path 542. In one example, the computing device 504 may translate the voice input into an equivalent text command and parse the text command to identify the command. The computing device 504 may then subsequently send its text command to the computing device 506. In another example, the computing device 504 may translate the voice input into an equivalent text command and then subsequently send the text command to the computing device 506. The computing device 506 may then parse the text command to identify one or more playback commands.
For example, if the text command is "Play" Track 1 "by" Artist 1 "from" Streaming Service 1 "in" Zone 1 "", the compute device 506 will be from (i) "Streaming Service 1". You may identify the available "Artist 1" URL for "Track 1" and (ii) at least one playback device in "Zone 1". In this example, the URL of "Track 1" by "Artist 1" from "Streaming Service 1" may be the URL pointing to the computing device 508, and "Zone 1" may be the combined zone 530. .. Therefore, once the URL is identified with one or both of the PBD 536 and 538, the computing device 506 may send the identified playback URL to one or both of the PBD 536 and 538 over the communication path 544. .. One or both of PBD 536 and 538 will accordingly retrieve audio content from the computing device 508 according to the received URL and initiate playback of "Track 1" by "Artist 1" from "Streaming Service 1". May be good.
In yet another example, the computing device 504 identifies a user's relevant commands and intents and performs a process of providing information about media content associated with voice input to the computing device 506. The computing device 504, for example, performs a "speech to text" on a voice input and analyzes the command or intent of the voice input along with other information about how to execute the command (eg, play, pause, stop, etc.). Volume up, volume down, skip, next, group, ungroup). The computing device 504 or computing device 506 determines which PBD command corresponds to the command or intent determined by the computing device 504. A command or intent determined from voice input and / or other information related to the execution of the command is transmitted from the computing device 504 to the computing device 506. Processing on the computing device 504 is performed by applications, modules, add-on software, integration with the native network microphone system software platform, and / or by the native network microphone system software platform.
Those skilled in the art will appreciate that the above is merely exemplary and that other embodiments are feasible. In some cases, as described above, the operation performed by one or more of the plurality of devices 500 may be performed by one or more of the other devices in the plurality of devices 500. For example, the conversion of voice input to text commands may be performed alternativeally, partially or completely, by other devices such as NMD512, computing device 506, PBD536, and / or PBD538. Similarly, URL identification may be performed, partially or completely, instead by another device or devices such as NMD512, computing device 504, PBD536, and / or PBD538.
f. Illustrative Network Microphone Device Figure 6 shows a functional block diagram of an exemplary network microphone device 600 that constitutes one or more of the NMDs 512, 514, and 516 in FIG. As shown, the network microphone device 600 includes a processor 602, a memory 604, a microphone array 606, a network interface 608, a user interface 610, a software component 612, and a speaker (s) 614, as long as it is a trader. Understand that the configuration and placement of network microphone devices is also possible. For example, a network microphone device may, as an alternative, exclude the speaker (s) 614 or have a single microphone in place of the microphone array 606.
Processor 602 may include one or more processors and / or controllers in the form of a general purpose processor or controller or a dedicated processor or controller. For example, the processing unit 602 may include a microprocessor, a microcontroller, an application-specific integrated circuit, a digital signal processor, and the like. The memory 604 may be a data storage that can contain one or more software components that are executed and function by the processor 602. Thus, memory 604 includes one or more non-temporary computer-readable recording media such as random access memory, registers, cache, etc., as well as read-only memory, hard disk drives, solid state drives, flash memory, and / Or may include one or more non-volatile recording media such as an optical storage device or the like.
The microphone array 606 may be a plurality of microphones configured to detect sound in the environment of the network microphone device 600. The microphone array 606 may include any type of microphone currently known or developed in the future, such as condenser microphones, electret condenser microphones, or dynamic microphones. In one example, the microphone array may be configured to detect audio from one or more directions with respect to the network microphone device. The microphone array 606 may be sensitive to some frequency range, and in one example, the first subset of the microphone array 606 may be sensitive to the first frequency range, while the microphone. The second subset of the array may be sensitive to the second frequency range. Further, the microphone array 606 may be provided to acquire position information of an audio source (eg, voice, audible sound) and / or to assist in filtering background noise. In particular, in certain embodiments, the microphone array may consist of only a single microphone rather than a plurality of microphones.
Network interface 608 is wireless and / or wired between various network devices within cloud network 502 such as CR522, PBD532-538, computing devices 504-508, and other network microphone devices in relation to FIG. It may be configured to facilitate communication. For this reason, the network interface 608 can take any form suitable for performing these functions, such as an Ethernet interface, a serial bus interface (eg FireWire, USB2.0, etc.), wireless. Examples include chipsets and antennas configured to facilitate communication, and / or any other interface that provides wired and / or wireless communication. In one example, the network interface 608 is an industrial standard (eg, infrared, wireless, wired standards such as IEEE802.3, wireless standards such as IEEE802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 802.11ac, 802.11ac, 4G communication. It may be based on standards, etc.).
The user interface 610 of the network microphone device 600 may be configured to facilitate user interaction with the network microphone device. In one example, the user interface 608 is a graphical interface or the like provided on a physical button, touch sensor screen (s) and / or surface (s) to allow the user to type directly into the network microphone device 600. It may contain one or more of them. User interface 610 may further include one or more lights and speakers (s) 614 to provide visual and / or auditory feedback to the user. In one example, the network microphone device 600 may be further configured to play audio content via a speaker (s) 614.
III. Multidirectional Microphone Array of Illustrative Media Playback Systems As mentioned above, the embodiments described herein can be trained to capture and process audio input based on the particular orientation of the playback device, 1 It may include one or more microphone arrays.
7A-7B show perspective views of each of the exemplary reproduction devices 700 in different orientations. The reproduction device 700 may be placed in various orientations by installing the reproduction device 700 on various sides or sides of the reproduction device 700, and FIGS. 7A and 7B show two examples regarding the orientation of the reproduction device 700. FIG. 7A shows a perspective view of the reproduction device 700 in the first orientation, and FIG. 7B shows a perspective view of the reproduction device 700 in the second orientation. For simplicity, the playback device 700 rests on one of its long sides, the first orientation shown in Figure 7A is called the horizontal direction, and the playback device 700 rests on one of its short sides. The second orientation shown in FIG. 7B may be referred to as the vertical orientation. The reproduction device 700 may be installed on a surface along either its side or the surface.
The playback device 700 has one or more microphone arrays 702 (eg, microphone array 702a, microphone array 702b, microphone array 702c, microphone array 702d) installed or mounted in the housing or main body of the playback device 700. You may. The microphone arrays 702a to 702d are shown as an example of an approximate arrangement in which the microphone array is incorporated into the playback device. More or less microphone arrays may be used, and microphone arrays may be placed along other sides or walls not shown in FIGS. 7A-7B. In addition, two or more microphone arrays may be located on the same side of the playback device.
The reproduction device 700 may be a reproduction device 200, and the microphone array 702 may be a microphone (s) 220. In some embodiments, the microphone array 702 may include, or may include, NMDs (eg, NMDs 512, 514, 516) that can be mounted or mounted on various walls or sides of the playback device 700. It may be a microphone array 606. By arranging the playback device in different orientations, the microphone array 702 also has different orientations, which allows the microphone array to each have a voice input (eg, voice input, audio, depending on their individual orientation). It becomes easier to capture input (input, tone input). Each microphone array may consist of two or more individual microphone arrays located at various points within the array. The shape of the microphone array 702 may be circular, and individual microphones may be placed around the microphone array 702 (eg, every x degrees from 0 to 360 degrees). For example, the microphone array 702a has individual microphones 704a to 704f.
As shown in FIG. 7A, the reproduction device 700 is arranged substantially or almost horizontally so that the longest side of the reproduction device 700 is parallel to one surface (eg, table, floor) along the xz plane. May be done. When oriented in this orientation, the orientation of the microphone array 702 changes in various ways. For example, the first microphone array 702a may be on top in this orientation and substantially parallel to the xz plane, while the second microphone array 702b is located substantially along the yz plane. The orientation may be substantially vertical. The microphone array 702 may be most sensitive to voice generated along a plane parallel to the microphone array. For example, a microphone array 702a that is horizontal (eg, perpendicular to gravity) may better identify the position of the sound source. Since the microphone array 702a is located on the upper surface of the reproduction device 700, it may be in a position where the voice input can be detected and captured best.
FIG. 7B shows the playback device 700 in a second orientation, in which the longest side of the playback device 700 is substantially perpendicular to the plane along the xz plane and with respect to the y-axis. It can be in a substantially vertical direction, which is substantially parallel. In this orientation, the microphone arrays 702b, 702c may be substantially horizontal and the microphone arrays 702a, 702d may be substantially vertical.
Although not shown, it can be placed in other positions, including, for example, along the front or back of the reproduction device 700. In addition, it may be possible to provide speakers of other shapes (eg, cylindrical, triangular, irregular) to orient the microphone array so that it can be oriented between various planes. In some embodiments, the microphone array may not be placed parallel to the wall of the playback device closest to it.
FIG. 8 shows an exemplary flow diagram 800 for how to process audio input based on the orientation of the playback device. The method 800 is, for example, one or more of the media playback system 100 of FIG. 1, one or more of the playback devices 200 of FIG. 2, one or more of the control devices 300 of FIG. 3, and a plurality of systems 500 of FIG. One embodiment of a method that can be performed in an operating environment, including one or more of the devices, one or more of the plurality of devices in the system 600, and the playback device 700 of FIGS. 7A-7B. Is presented. Method 800 may include one or more actions, functions, or actions. Although the blocks are shown in order in FIG. 8, these blocks may be executed simultaneously and / or in a different order than described herein. Also, the various blocks may be combined into a smaller number of blocks, divided into additional blocks, and / or removed based on the desired embodiment.
Further, for Method 800 and other processes and methods disclosed herein, this flow diagram shows the function and operation of one feasible embodiment of the present embodiment. In this regard, each block represents a portion of a module, segment, or program code that is executed by one or more processors and contains one or more instructions to perform a particular logical function or step in a process. be able to. This program code may be stored on any type of computer readable medium, for example storage devices including disks or hard drives. This computer-readable medium includes non-transitory computer-readable media such as register memory, processor cache, and computer-readable media that store short-term data such as random access memory (RAM). In addition, computer readable media such as read-only memory (ROM), optical or magnetic disks, compact disk read-only memory (CD-ROM), and other non-temporary or permanent long-term storage. Recording media can also be mentioned. Also, the computer readable medium may be any other volatile or non-volatile storage system. This computer-readable medium can be considered, for example, a computer-readable recording medium or a tangible storage device. Further, with respect to Method 800 and other processes and methods disclosed herein, each block of FIG. 8 can represent a circuit that is wired to perform a particular logical function within the process.
At block 802, microphone training may be performed on the microphone array (s) 702 of the playback device 700. This microphone training may generate a response profile for the microphone array (s) of the playback device 700. Microphone training may be completed, for example, as part of the manufacturing process or during a calibration process that may occur after manufacturing. Microphone training may include capturing individual microphone responses (eg, polar responses) to the test voice (eg, tone, audio, sound, voice, noise) played from the speaker. To generate the training response profile (s) for the microphone array 702, the playback device 700 is placed first-facing in a complete or near-complete acoustic environment (eg, an anechoic chamber) or other acoustic environment. May be good. The test audio may be reproduced from a speaker directed towards the reproduction device 700 (eg, speaker 902).
A top view showing an example of the calibration setting 900 is described for block 802 and is also shown in FIG. To make the first capture measurement, the speaker 902 may be directed to the front center of the playback device 700, which may be coaxial with the center of the microphone array 702a, and this position may be referred to as the test start position, and / Or may be specified as a 0 degree position. The test audio may be reproduced by speaker 902. The first microphone array may receive test audio and measure the respective response of the individual microphones in the array. The measured response by each individual microphone to the test voice may be stored as a set of measured responses for the first test voice position (eg, 0 degree position), and / or the measured response is the first. It may be stored as a relative value (eg, difference) between responses by different pairs in one microphone with respect to one test voice position (eg, 0 degree position). Each response value may have a magnitude and a phase component. The first set of measured responses may be stored as a matrix or table such as Table 1 shown below. The microphone array 702a may have six individual microphones identified as microphones 704a, 704b, 704c, 704d, 704e, and 704f.
<tables num="1"><img file="JP6986071B2_D0001.tif" /></tables>
In Table 1, the value R stored in (row b, column a)<sub>b b</sub>、<sub>a</sub>Is the response value measured for microphone 704b (R)<sub>b b</sub>) To the response value of the microphone 704a (R)<sub>a</sub>) May be subtracted from the response relative value of the microphone 704b to the microphone 704a. This may be repeated for all other microphones in the microphone array, one microphone at a time, according to the same rules, as shown in Table 1. This response value may be a complex number in which the magnitude of the response can be the real part of the value and the phase can be the imaginary part of the value. R inferred as a response to (row a, column b)<sub>a</sub>、<sub>b b</sub>The value of is the response R in (row b, column a)<sub>b b</sub>、<sub>a</sub>The response value may not be stored for this relative response, as it may be a duplicate of, which is shown in Table 1 with a dash (ie, "-") in the cell. The set of values in Table 1 may be associated with the location (eg, relative angle) of the speaker 902 playing the test audio.
A similar set of values associated with the second position of the speaker may be generated for the microphone array 702, which is the front center of the playback device 700 or the front center of the playback device 700 while maintaining the same distance from the center of the microphone array 702. It is obtained by changing the position of the speaker with respect to a given frequency. For example, a second set of values for the first microphone array at a relative angle of 90 degrees may move the speaker 902 to a 90 degree position or center the microphone array 702a or playback device 700 around a vertical axis (eg, yaw). The speaker 902 may be generated or acquired by being rotated to and so that the speaker 902 is aligned with the 90 degree position of the microphone array 702a. These measurements are rotated around the playback device every x degrees (eg, every 1 degree, every 5 degrees, every 10 degrees, etc.) in small increments (eg, clockwise, counterclockwise, etc.). ), And the degree depends on the desired data resolution of 0 to 360 degrees centered on the playback device 700.
The measurement process in block 802 may be repeated to collect the values of each microphone array 702 of the reproduction device 700 in each orientation. For example, when in the first position of the speaker, the response values of all the microphones in the individual microphone arrays may be specified each time the test voice is played. In some cases, the test audio may be played once for each angular position, and individual response values in all individual microphones and all microphone arrays may be specified based on the reproduction of the same test audio. Other methods of obtaining the response value may be adopted.
The measurements obtained through this process at or after the end of block 802 may include the response values of all microphone arrays to multiple orientations. These measurements may be organized into various data sets for individual response profiles, such as a collection of measurements for each microphone array. For example, the first set of measurements may include all measurements taken for the microphone array 702a and the second set of measurements may include all the measurements captured for the microphone array 702b.
In some embodiments, these response values may be associated with each other in the form of a group of response values identified for the same microphone array in the orientation of two or more different playback devices. A group of calibration measurements may be measurements of a particular microphone array acquired while the playback device is in a particular orientation. For example, the first group of calibration measurements may be the measurements of the microphone array 702a acquired while the playback device 700 is in the horizontal direction shown in FIG. 7A, and the second group of calibration measurements may be. It may be a measurement of the microphone array 702a acquired while the reproduction device 700 is in the vertical direction shown in FIG. 7B.
In another aspect, as described above, the first set of measurements may be, for example, the measurements shown in Table 1, which is the first with respect to the first microphone array when the playback device is in the first orientation. It can be a microphone measurement generated for the test voice played from an angular position. More specifically, the first set of measurements may be measurements for each of the individual microphones 704 of the microphone array 702a.
In some embodiments, these measurements may be organized as vectors. Each vector may correspond to an individual microphone, or may include measurements of individual microphones obtained when one or more speakers are in the same orientation. For example, the first vector is the value of the microphone 704a in the horizontal direction, or the difference between the microphones 704b and 704a, when measured at any angle between 0 and 360 degrees with respect to the position of the speaker 902. It may include a relative value to be represented.
In some embodiments, measurements may be obtained using the same test voice for all microphone arrays. For example, the test voice can be played once for each orientation and the response values of any or all of the individual microphones in the microphone array can be collected. The values in Table 1 can be expanded to include individual microphone response values or relative response values in two or more microphone arrays. Examples for the microphone array 702a and microphone array 702c are shown in Table 2 below. The microphone 702c array includes individual microphones 706a-706f. In the example in Table 2 below, R<sub>y y</sub>、<sub>z z</sub>Represents the response value of the microphone y minus the response value of the microphone z. For example, R<sub>706d</sub>、<sub>704c</sub>= Response value of microphone 706d-Response value of microphone 704c.
<tables num="2"><img file="JP6986071B2_D0002.tif" /></tables>
The response values described above may generally represent measurements in a plane. In other words, these values may correspond to the measurement information collected in two dimensions. For example, for the horizontal microphone array 702a shown in FIG. 7A, the measurements may represent data along a horizontal plane (eg, xz plane). This can be achieved, for example, by maintaining the same height in the installation of the speaker 902 when the test audio is played. Although this example has described collecting response data along a horizontal plane, these data are on different axes (eg, vertical and / or horizontal) while maintaining the same distance from the center point. Can be collected in a vertical plane when rotating the reproduction device 700 or speaker 902 vertically around (eg, roll and / or pitch).
In some embodiments, these calibration values are added to the information collected along, for example, a second plane (eg, a plane orthogonal to the first plane) to the information collected along the first plane. , By using a combination of data from different planes, it can be in a three-dimensional (eg, spherical) format. For example, the speaker 902 may be rotated about an x-axis to measure the response in a manner similar to that described above, thereby collecting measurements along the yz plane (eg, vertically).
The training or measurements described herein may be obtained using one or more devices in the system. For example, the playback device 700 may determine the response of the test voice received by the microphone array. The playback device 700 stores the response values locally in the memory of the playback device 700 and / or processes and / or stores these response values in a computing device (eg, a server, computer, or other measurement). Can be sent to the device).
The measured response values obtained for a given playback device model are associated with a given playback device model and / or speaker type and are associated with a given playback device model and / or microphone array model and / or individual microphones. It may be stored in a database or a server as a representative value of a component. For example, any combination of replay device identifier (eg, model number, serial number, supplier identifier) or microphone identifier (eg, model number, serial number, supplier identifier), or the same combination of replay device identifier and microphone identifier. The same values may be used for microphone arrays or individual microphones used in the same reproduction device model and provided by the same supplier, which may be identified using other calibration identifiers that indicate. The playback device is directional in its microphone array These identifiers are used to preload representative calibration data to the playback device 700 during manufacturing of the playback device or before providing it to the user so that it is ready to adjust the focus). May be good. In some embodiments, the response value of the playback device may be loaded or updated when the playback device is connected to the network. These values may be obtained by the replay device from the computing device over the network or they may be transmitted to the replay device. In some cases, loading or updating these values may occur during the playback device configuration process or other playback device calibration processes (eg, playback device adjustment based on the playback environment, playback equalizer adjustment process). good.
Block 804 may determine the orientation of the playback device. This orientation may be determined based on a sensor in the reproduction device 700 (eg, accelerometer, gyroscope, etc.). The orientation of the playback device was filed on July 19, 2011 as application number 13 / 186,249, and the specification issued on May 26, 2013 as US Pat. No. 9,042,556, and filed on April 24, 2015. It may be determined as described in the specification filed as No. 14 / 696,041 and published as US Patent Application Publication No. 2016/0315384 on October 27, 2016, both of which are in its entirety. The whole is used as a reference. The orientation of the playback device may be predetermined. For example, the predetermined orientation may correspond to the orientation in which the training response was measured in block 802.
At block 806, one or more microphone arrays may be determined for use for the purpose of capturing or processing audio input. The playback device 700 may have a single microphone array, in which case the single microphone array will be selected for use in capturing audio input. In some embodiments, the reproduction device 700 may have two or more microphone arrays, in which case one or more microphone arrays may be selected.
For the purposes of use, some microphone arrays 702 and certain microphone arrays are the location of the microphone array 702, the available processing power, the orientation of the playback device, and / or other contextual information (eg, the microphone's relative to the playback environment). It may be selected depending on various factors such as position).
When assessing the percentage of available processing power, the playback device 700 or other devices in the system may be local to the playback device 700 and one or more networks to the playback device 700 (eg, local). The evaluation can be made via (area network, wide area network) and / or based on the processing power available to the playback device 700 via one or more computing devices. For example, the microphone array selected in the playback device 700 captures the first part of the audio input and data representing that first part captured by another device (eg, a server, control device, other playback device). May be sent to and processed for location information. In some embodiments, the playback device 700 may process the data locally to identify the location information associated with the voice input.
The number of microphone arrays used may be determined during the initialization process and / or during the playback device calibration process in the playback environment (eg, adjusting the playback device based on the playback environment).
In some embodiments, the microphone array on top of the playback device may be selected. In some cases, the microphone array that is most sensitive to the audio in front of the playback device may be selected. In yet another embodiment, the microphone that is most perpendicular to gravity may be selected. The weighting or priority of the microphone array may be given based on the orientation of the playback device.
If the playback device 700 has only a single microphone array 702, or if only one of the multiple microphone arrays 702 is selected to be enabled or used for capturing or processing audio input. , One of its selected microphone arrays 702 may be enabled or activated to capture voice input. In some embodiments, it may not be necessary to adjust the directional focus in the microphone array (s), in which case the playback device 700 will enable or activate one or more. After selecting the microphone array, you may be ready to process the voice input to execute the command.
After the microphone array (s) to be enabled have been determined, the system may identify the set of response data corresponding to the orientation of the selected microphone array and playback device 700. For example, if the playback device 700 is currently oriented in the primary orientation as shown in FIG. 7A and the microphone array 702a is selected or activated for voice input, then the system (eg, the device of FIG. 5). Either) can identify a group of response values, which corresponds to the current orientation (eg, first orientation) of the microphone array 702a and playback device 700.
At block 808, voice input may be detected by the selected microphone array (s). For example, the selected microphone array (s) may continuously monitor the initiation of voice commands. Voice commands may begin with an activation word or phrase (also known as a wakeup word or hotword) and notify the system that the user issuing the command is ready to issue a command prompting the system to execute it. can. When the activation word is issued, the system can start processing the words emitted after the activation word in the voice command. The microphone array selected by detecting the start of a voice command, which may be after detecting that the activation word has been issued, or that may include detecting that the activation word has been issued, is the voice command of the voice command. Capturing (eg, recording, streaming, processing) of voice input (eg, voice input) may be initiated for processing purposes.
At block 810, the position of the sound source of the voice input (eg, voice input, audio input) may be specified (eg, a direction in one or more dimensions, a direction in one or more planes). The position of the sound source of the voice input may be specified with various accuracy. For example, this position may be per microphone array or in the exact or approximate orientation in the playback device 700, or in the selected microphone array (s) or orientation with respect to the playback device 700. .. A portion or sample of the audio input may be captured and used in locating the sound source of the audio input. For example, the voice input sample may be in the form of a response captured by each individual microphone in the selected microphone array. These response values may be the actual responses captured by each of the individual microphones, or may be relative values between different pairs in the individual microphones of the microphone array. These values can be calculated in the same way as the training response values obtained in block 802. For example, the received sample input may be processed in the same way as the test voice, in which case the differences between different pairs of individual microphones in one or more microphone arrays can be calculated. In other words, the received sample input can be organized in the same format as Table 1 or Table 2, in which case the angle association is unknown and will be specified in block 810.
The sample response value of the voice input can be compared to the training response value to determine the direction of the sound source of the input with respect to the microphone array or playback device. This comparison may include comparing each set of training response values to determine which set of values corresponds to the sample input response. Since each set of training response values corresponds to a certain angle value, the direction of the voice input may correspond to the angle value of the set of training responses corresponding to the voice input. Using a probability function (eg, probability distribution function (pdf), Gaussian distribution), which training response set or value the sample input response is associated with, and the corresponding angular position in the training response set or value. By determining, the most probable direction of the voice input can be specified.
In embodiments where a single microphone array is active or enabled, the response values of the sample input are compared to the group of training response values associated with one microphone array at all position angles during testing. , The most likely direction of the voice input may be specified. For example, you may put a set of sample response values in a probability function such as a probability density function and compare it with the set of response values in Table 1 to determine the probability that the direction of the voice input is from the 0 degree position. .. The result of the probability function may be a probability value (eg, the distance value of the probability density function) indicating the probability that the direction of the voice input is the direction associated with the set of values (eg, the 0 degree position). In some embodiments, the probability value may be a percentage, where the higher the percentage, the more likely it is that the voice input value corresponds to the direction associated with the set of values. In other embodiments, the probability value may be the distance value of the probability density function, in which case the smaller the value, the more likely it is that the voice input value corresponds to the direction associated with the set of values. Become.
The sample input response value may be compared to a subset of a group of values, or a subset of values or the entire group of values producing each probability value of the group. For example, the training response may be acquired every 5 degrees between 0 degrees and 360 degrees. Since such high resolution data may not be required, the system may compare training responses every 10 degrees between 0 and 360 degrees. In some cases, training responses may be compared every 20 degrees, and this comparison process is a constant number of degrees in the direction most likely to correspond, as indicated by the probability value (eg, 10 degrees, 15 degrees, etc.). It may be repeated every x frequency (for example, 1 degree, 5 degrees, etc.) within the plus or minus range of. For example, if the distance value of the probability density function represents the probability, the minimum or minimum distance value of the probability density function and the angle position associated with it may correspond to the most probable direction in the voice input. There is sex.
As described herein, in some embodiments, two or more microphone arrays may be active to detect and record voice input. Processing for two or more microphone arrays may be performed individually for each selected microphone array. For example, a sample input response can be compared to a group of response values for one microphone array to simultaneously generate a set of probability values for each of two or more microphone arrays. Each value in the set of probability values may represent the possibility that the sound source of the audio input is from a given direction.
Weights may be assigned to a set of probability values for a particular microphone array. This assigned weight may be the weight associated with a particular microphone array, which may vary depending on the orientation of the playback device. For example, in the vertical direction shown in Figure 7B, the values associated with the microphone array 702c can be given greater weight than the values associated with the microphone array 702a, because the microphone array 702c, This is because it can provide more useful information about the position of the sound source than a microphone in a more vertical direction.
Sample input responses may be compared simultaneously with a group of response values for each of all microphones in the microphone array. For example, voice input values may be organized as a set of input response values with relative response values for each unique combination of individual microphone pairs. These sets of input response values may be compared to one or more sets of training response values, in which case the set of training response values is a training response for each unique combination of individual microphone pairs. Includes a value.
Similar to the previous description, the response value of a particular microphone array may be weighted more heavily by weighting the corresponding probability value generated for that microphone array.
Similarly, the sample input response value may be compared to the training response value vector.
As described herein, the response value may have two components, a magnitude component and a phase component. The magnitude may indicate the magnitude or amplitude of the voice received by the microphone, and the phase may indicate the timing of the voice received by the microphone. For example, comparing the phase information between two microphones may indicate that one microphone received a particular sound before another. This phase information can be used to identify when the received sound is reflected.
In the comparison of the sample input response value and the training response value described herein, any combination of response value magnitude and phase may be compared when determining the possibility that the sound source is in a particular direction. good. For example, only the magnitude may be compared, only the phase value may be compared, or the magnitude and the phase value may be compared.
At block 812, the focus of directivity in the selected microphone array (s) may be adjusted based on the location of the sound source identified in block 810. For example, in order to improve the quality of the captured audio input, location information can be used to notify the beamforming and / or acoustic echo correction (aec) process during the audio input acquisition process.
At block 814, audio input may be captured based on the adjusted directional focus in the microphone array. For example, the selected microphone array (s) may be beam-formed in the direction specified at the location specified in block 810. In some cases, no adjustment may be necessary. For example, the system may determine that an existing or current directional focus may be appropriate or sufficient, or the system may capture audio inputs that require a directional focus. It may be determined that it may not be.
At block 816, the captured voice input may be processed to identify any voice command contained in the voice input. The captured voice input may be converted from voice to text, and block 818 may execute any command for the media playback system contained in the voice input.
Whenever the playback device 700 or another device in the system determines that its orientation and / or positioning has changed when the playback device 700 is in a media playback environment (eg, the environment shown in FIG. 1), block 804 From 806 may be repeated. For example, a sensor in the reproduction device (eg, accelerometer, gyroscope) can determine when there is movement and / or change in its orientation.
A subset of the Method 800 or Method 800 blocks are periodic, aperiodic, and / or in response to the occurrence of a particular event (eg, adjusting the playback characteristics for the playback environment, setting in the new environment). , Changing the orientation of the device) May be repeated.
IV. Conclusion This specification discloses various exemplary systems, methods, devices, products, etc., which, among other components, firmware and / or software running on hardware. include. It is understood that such examples are merely examples and should not be considered limiting. For example, some or all of these firmwares, hardware, and / or aspects or components of the software, exclusively to hardware, exclusively to software, exclusively to firmware, or to any hardware, software, and / or firmware. It is intended that the combination can be carried out. Therefore, those examples provided are not the only way to implement their systems, methods, equipment, and / or products.
In addition, references herein to "embodiments" include specific features, structures, or properties described in connection with that embodiment in at least one exemplary embodiment of the invention. Means to get. The use of this phrase in various parts of the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is exclusive to other embodiments. Thus, it will be appreciated by those skilled in the art, expressly and implicitly, that the embodiments described herein may be combined with other embodiments.
This specification is broadly described with respect to exemplary environments, systems, procedures, steps, logical blocks, processes, and other symbolic representations, which are the behavior of data processing devices that are directly or indirectly connected to a network. Is similar to. These processing descriptions and expressions are commonly used by those skilled in the art and can most efficiently convey the content of their work to those skilled in the art. Much specific content is provided to understand this disclosure. However, it will be appreciated by those skilled in the art that certain embodiments of the present disclosure may be implemented without specific, specific details. In other examples, well-known methods, procedures, components, and circuits are not described in detail to avoid unnecessarily obscuring embodiments. Therefore, the scope of the present disclosure is defined by the appended claims rather than the embodiments described above.
When any of the appended claims is simply read to cover implementation in software and / or firmware, one or more of the elements in at least one example are referred to herein as software and / or firmware. It is clearly defined to include tangible non-temporary storage media for storing firmware, such as memory, DVD, CD, Blu-ray® and the like.
12 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2008158868A | Cites | Japan |
| JP2015161551A | Cites | Japan |
| JP2007142595A | Cites | Japan |
| JP2007013400A | Cites | Japan |
21 members in 6 offices
Priority claims9
| Document | Office | Kind | Date |
|---|---|---|---|
| 15282554 | United States of America | – | |
| 201615282554 | United States of America | A | |
| 201615282554 | United States of America | A | |
| 2017054063 | United States of America | W | |
| 2017054063 | United States of America | W | |
| 15282554 | – | – | – |
| US201615282554 | – | – | – |
| US2017054063 | – | – | – |
| WO2017US54063 | – | – | – |
Members21
| Document | Office | Kind | |
|---|---|---|---|
| US9743204B1 | United States of America | B1 | |
| US2018098167A1 | United States of America | A1 | |
| WO2018064362A1 | World Intellectual Property Organization (WIPO) | A1 | |
| US2018227688A1 | United States of America | A1 | |
| US10075793B2 | United States of America | B2 | |
| US10117037B2 | United States of America | B2 | |
| US2019052985A1 | United States of America | A1 | |
| KR20190055832A | Republic of Korea | A | |
| US10313812B2 | United States of America | B2 | |
| CN109997370A | China | A | |
| EP3520436A1 | European Patent Office (EPO) | A1 | |
| US2019281399A1 | United States of America | A1 | |
| JP2019537309A | Japan | A | |
| KR102158514B1 | Republic of Korea | B1 | |
| US10873819B2 | United States of America | B2 | |
| EP3520436B1 | European Patent Office (EPO) | B1 | |
| CN109997370B | China | B | |
| US2021219082A1 | United States of America | A1 | |
| JP6986071B2This record | Japan | B2 | |
| US11516610B2 | United States of America | B2 | |
| US2023164503A1 | United States of America | A1 |
21 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Notice of transfer of a case for reconsideration by examiners before appeal proceedingsAppealJAPANESE INTERMEDIATE CODE: C21C21 | C21 | |
| Transfer to examiner for re-examination before appeal (zenchi)AppealJAPANESE INTERMEDIATE CODE: A911A911 | A911 | |
| Written invitation by the commissioner to file amendmentsJAPANESE INTERMEDIATE CODE: C11C11 | C11 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Trial request (containing other claim documents, opposition documents)OppositionJAPANESE INTERMEDIATE CODE: C60C60 | C60 | |
| Decision of refusalJAPANESE INTERMEDIATE CODE: A02A02 | A02 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for extension of timeJAPANESE INTERMEDIATE CODE: A601A601 | A601 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Written request for extension of timeJAPANESE INTERMEDIATE CODE: A601A601 | A601 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on accelerated examinationJAPANESE INTERMEDIATE CODE: A971005A975 | A975 | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 | |
| Explanation of circumstances concerning accelerated examinationJAPANESE INTERMEDIATE CODE: A871A871 | A871 |
Numbers
- Publication
- 6986071
- Publication, DOCDB
- 6986071
- Publication, EPODOC
- JP6986071B
- Application
- 2019517281
- Application, DOCDB
- 2019517281
- Application, EPODOC
- JP20190517281
Titles2
- Japanese
- 多方向再生デバイスマイクロフォン
- English
- Multi-directional playback device microphone
Classification
- CPC, 18
- H04R29/005
- G01S3/8003
- H04R3/005
- H04R2201/401
- H04R2430/20
- G01S3/803
- H04R1/406
- H04R3/12
- H04R5/04
- H04S7/301
- H04R2201/405
- H04R2205/024
- H04R2420/07
- H04R2430/21
- H04R2499/11
- H04R29/007
- H04R2227/003
- H04R2227/005
- IPC, 4
- H04R3 00
- G01H3 00
- G10K15 00
- H04R1 40
