Voice interaction method, and device
5 claims: 3 independent, 2 dependent
- 1音声対話エージェントサーバによって行われる音声対話方法であって、 前記音声対話エージェントサーバは、前記音声対話エージェントサーバおよび他の音声対話エージェントサーバと1以上のキーワードとを対応付けるエージェント情報を記憶するメモリを備え、 前記音声対話方法は、 少なくとも1つのキーワードを含む入力音声信号を受け付け、 前記入力音声信号に対して音声認識処理を行い、 前記音声認識処理の結果と前記メモリに記憶されるエージェント情報とに基づいて、当該入力音声信号に基づく処理を、前記音声対話エージェントサーバと前記他の音声対話エージェントサーバとのいずれで行うかを決定し、 前記決定において、前記音声対話エージェントサーバで行うと決定された場合、前記音声認識処理の結果に基づく処理を行い、当該処理に係る応答音声信号を生成して出力し、 前記決定において、前記他の音声対話エージェントサーバで行うと決定された場合、受信した前記入力音声信号を前記他の音声対話エージェントサーバへ転送 し 、 前記入力音声信号は機器から送信され、 前記決定において、前記音声対話エージェントサーバで行うと決定された場合、前記入力音声信号をそのまま前記他の音声対話エージェントサーバへ転送した後、前記他の音声対話エージェントサーバから前記機器との通信終了の信号を受信するまで前記音声入力信号を受け付けない、 音声対話方法。
- 2前記決定において、前記音声対話エージェントサーバで行うと決定された場合、前記音声認識処理の結果に基づく処理に係る応答テキスト信号をさらに出力する、 請求項1記載の音声対話方法。
- 3前記他の音声対話エージェントサーバは複数であり、前記エージェント情報は、前記他の音声対話エージェントサーバを識別する識別子と、前記キーワードとを対応付ける情報であり、前記音声認識処理の結果に前記キーワードのいずれかが含まれる場合に、前記エージェント情報において当該キーワードに対応付けられた識別子によって識別される他の音声対話エージェントサーバを、前記入力音声信号に基づく処理を行う他の音声対話エージェントサーバとして決定するように、前記決定を行う 請求項1記載の音声対話方法。
- 4前記識別子は、前記他の音声対話エージェントサーバのIPアドレスである、請求項 3 記載の音声対話方法。
- 5音声対話エージェントサーバであって、 前記音声対話エージェントサーバおよび他の音声対話エージェントサーバと1以上のキーワードとを対応付けるエージェント情報を記憶するメモリと、 ネットワークと接続する通信部と、 プロセッサとを備え、 前記通信部は、少なくとも1つのキーワードを含む入力音声信号を受け付け、 前記プロセッサは、 前記入力音声信号に対して音声認識処理を行い、 前記音声認識処理の結果と前記メモリに記憶されるエージェント情報とに基づいて、当該入力音声信号に基づく処理を、前記音声対話エージェントサーバと前記他の音声対話エージェントサーバとのいずれで行うかを決定し、 前記決定において、前記音声対話エージェントサーバで行うと決定された場合、前記音声認識処理の結果に基づく処理を行い、当該処理に係る応答音声信号を生成し、前記応答音声信号を前記通信部から出力させ、 前記決定において、前記他の音声対話エージェントサーバで行うと決定された場合、受信した前記入力音声信号を、前記通信部から前記他の音声対話エージェントサーバへ転送さ せ、 前記入力音声信号は機器から送信され、 前記決定において、前記音声対話エージェントサーバで行うと決定された場合、前記入力音声信号をそのまま前記他の音声対話エージェントサーバへ転送した後、前記他の音声対話エージェントサーバから前記機器との通信終了の信号を受信するまで前記音声入力信号を受け付けない、 音声対話エージェントサーバ。
Independent claims5
469 paragraphs, as filed
The present invention relates to a voice dialogue method that performs processing based on voice input in an interactive format.
Conventionally, there is known a voice dialogue system provided with a voice input interface and performing processing based on voice input interactively by a user.
For example, Patent Document 1 describes a headset equipped with a microphone that performs voice recognition processing on voice input from a microphone and performs processing based on the result of the voice recognition processing.
Further, Patent Document 2 describes a voice dialogue system having an agent that performs processing based on voice input interactively by a user.
<p><patcit num="1"><text>Japanese Unexamined Patent Publication No. 2004-233794</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 2008-90545</text></patcit></p>
<p><u style="single">By the way, in the above-mentioned prior art, there are a plurality of agents that provide a voice recognition function in a network, and when a user selects one of them and starts communication, the voice recognition function of the other agent is used. It may not be what the user wants. In this case, sufficient recognition results may not be obtained, which may be disappointing.</u></p><p><u style="single">In order to avoid such disappointment, the user must know in advance what kind of voice recognition function each of the plurality of agents existing in the network has. However, forcing the user to grasp such advance information impairs the usability of the voice recognition function.</u></p><p> An object of the present invention is<u style="single">To provide a voice dialogue method that can perform appropriate voice recognition without knowing in advance what kind of voice recognition function each of the plurality of agents existing in the network has. The purpose.</u></p>
<p><u style="single"> In order to solve the above problems, the voice dialogue method according to the present invention is a voice dialogue method performed by a voice dialogue agent server, and the voice dialogue agent server has one or more keywords with another voice dialogue agent server. The voice dialogue method includes a memory for storing the associated agent information, receives an input voice signal, performs voice recognition processing on the input voice signal, and performs the voice recognition processing result and the agent information stored in the memory. Based on the above, it is determined whether the processing based on the input voice signal is performed by the voice dialogue agent server or the other voice dialogue agent server, and in the determination, it is determined that the processing is performed by the voice dialogue agent server. If so, a process based on the result of the voice recognition process is performed, a response voice signal related to the process is generated and output, and if it is determined in the determination that the other voice dialogue agent server performs the process, the above. The input voice signal is transferred to the other voice dialogue agent server.</u></p>
<p><u style="single">According to the above-mentioned voice dialogue method, when the user is communicating with one voice dialogue agent server (voice dialogue agent server A), the voice dialogue agent server A is a voice dialogue agent other than the voice dialogue agent server A. If it is determined that the server (voice dialogue agent server B) should be the communication partner, the voice dialogue agent server of the user's communication partner is changed from the voice dialogue agent server A to the voice dialogue agent server B. You will be able to do it.</u><u style="single">As a result, the user can receive the service from a more appropriate voice dialogue agent server without knowing what kind of service is provided by each voice dialogue agent server. Further, in this case, the voice dialogue agent server A transfers the input voice to the voice dialogue agent server B as it is, so that the voice dialogue agent server B receives the voice recognition process itself for the input voice itself. It can be performed. Therefore, the user can receive more appropriate service provision.</u></p>
<figref num="1">System configuration diagram showing the configuration of the voice dialogue system 100</figref><figref num="2">Block diagram showing the functional configuration of device 140</figref><figref num="3">State transition diagram of the state managed by the control unit 210</figref><figref num="4">Block diagram showing the functional configuration of the voice dialogue agent 400</figref><figref num="5">Data structure diagram of dialogue DB500</figref><figref num="6">Flow chart of the first device processing</figref><figref num="7">Flowchart of the first voice input process</figref><figref num="8">Flowchart of 1st agent processing</figref><figref num="9">Flowchart of the first instruction execution process</figref><figref num="10">Processing procedure diagram in a specific example</figref><figref num="11">(a) Figure 1 that schematically shows the display contents of the device 140, (b) Figure 2 that shows the display contents of the device 140, and (c) Figure 3 that shows the display contents of the device 140 schematically. , (D) Figure 4 schematically showing the display contents of the device 140</figref><figref num="12">Figure 5 schematically showing the display contents of the device 140</figref><figref num="13">Block diagram showing the functional configuration of device 1300</figref><figref num="14">State transition diagram of the state managed by the control unit 1310</figref><figref num="15">Flow chart of the second device processing</figref><figref num="16">A processing procedure diagram schematically showing how a dialogue with a voice dialogue agent is performed.</figref><figref num="17">Block diagram showing the functional configuration of device 1700</figref><figref num="18">State transition diagram of the state managed by control unit 1710</figref><figref num="19">Flow chart of the third device processing</figref><figref num="20">Flowchart of the second voice input process</figref><figref num="21">A processing procedure diagram schematically showing how a dialogue with a voice dialogue agent is performed.</figref><figref num="22">Block diagram showing the functional configuration of the voice dialogue agent 2200</figref><figref num="23">Data structure diagram of target agent DB2300</figref><figref num="24">Flowchart of second agent processing</figref><figref num="25">Flow chart of the second instruction execution process</figref><figref num="26">Flowchart of first connection response processing</figref><figref num="27">Flowchart of disconnection response processing</figref><figref num="28">Flowchart of 3rd agent processing</figref><figref num="29">A processing procedure diagram schematically showing how a dialogue with a voice dialogue agent is performed.</figref><figref num="30">Block diagram showing the functional configuration of the voice dialogue agent 3000</figref><figref num="31">Data structure diagram of available service DB3100</figref><figref num="32">Flowchart of 4th agent processing</figref><figref num="33">Flowchart of the third instruction execution process</figref><figref num="34">Flowchart of second connection response processing</figref><figref num="35">A processing procedure diagram schematically showing how a dialogue with a voice dialogue agent is performed.</figref><figref num="36">(A) Schematic diagram of the operation mode in which the voice dialogue system is operated, (B) Schematic diagram of the data center operating company 3610 Part 1, (C) Schematic diagram of the data center operating company 3610 Part 2</figref><figref num="37">Schematic diagram of service type 1</figref><figref num="38">Schematic diagram of service type 2</figref><figref num="39">Schematic diagram of service type 3</figref><figref num="40">Schematic diagram of service type 4</figref><figref num="41">System configuration diagram showing the configuration of the voice dialogue system 4100</figref><figref num="42">Block diagram showing the functional configuration of the intermediary server 4150</figref><figref num="43">Block diagram showing the functional configuration of the intermediary server 4350</figref><figref num="44">(a) Image example 1 displayed on the display unit, (b) Image example 2 displayed on the display unit, (c) Image example 3 displayed on the display unit, (d) Displayed on the display unit Image example 4</figref><figref num="45">(a) Image example displayed on the display unit No. 5, (b) Image example displayed on the display unit No. 6</figref><figref num="46">Example of state transition diagram of state Part 1</figref><figref num="47">Example of state transition diagram of state Part 2</figref><figref num="48">Example of state transition diagram of state Part 3</figref><figref num="49">Example of state transition diagram of state Part 4</figref><figref num="50">Example of state transition diagram of state Part 5</figref>
<Embodiment 1> <Overview> Hereinafter, as one aspect of the voice dialogue method according to the present invention and one aspect of the device according to the present invention, a device installed in a house, a car, or the like, and a voice communicating with the device. A voice dialogue system including a dialogue agent server will be described.
In this voice dialogue system, the voice dialogue agent server realizes the voice dialogue agent by executing the stored program. This voice dialogue agent engages in voice dialogue (voice input / output) with a user who uses the voice dialogue system via a device. Then, the voice dialogue agent executes a process that reflects the content of the dialogue, and outputs the execution result by voice via the device used by the user.
When the user wants to interact with the voice dialogue agent (want to input voice to the voice dialogue agent), the user performs a predetermined voice input start operation to the devices constituting the voice dialogue system. Then, the device is in a state of being able to accept the voice input for a predetermined period after the voice input start operation is performed. Then, during the period in which the device is in a state where the voice input can be accepted, the user makes a voice input to the voice dialogue agent.
Hereinafter, the details of this voice dialogue system will be described with reference to the drawings.
<Configuration> FIG. 1 is a system configuration diagram showing the configuration of the voice dialogue system 100.
As shown in the figure, the voice dialogue system 100 includes a voice dialogue agent server 110a to a voice agent server 110b, a network 120, a gateway 130a to a gateway 130b, and a device 140a to a device 140e.
Of these, the gateway 130a and the equipment 140a to the equipment 140c are arranged in the house 180, and the gateway 130b and the equipment 140d to the equipment 140e are arranged in the car 190.
The devices 140a to 140e are wirelessly or wirelessly with the gateway 130 (the gateway 130a or the gateway 130b is simply referred to as the gateway 130, unless the gateway 130a and the gateway 130b are explicitly distinguished from each other). Voice dialogue agent except when the function of wired communication and the voice dialogue agent server 110 (voice dialogue agent server 110a and voice dialogue agent server 110b are explicitly distinguished via the gateway 130 and the network 120) are displayed. The server 110a or the voice dialogue agent server 110b has a function of communicating with the voice dialogue agent server 110).
The devices 140a to 140c arranged in the house 180 are, for example, TVs, air conditioners, recorders, washing machines, portable smartphones, etc. installed indoors, and the devices 140d arranged in the car 190. ~ The device 140e is, for example, a car air conditioner, a car navigation system, etc. installed in the car 190.
Here, instead of explaining the devices 140a to 140e individually, a virtual device 140 having a function commonly possessed by the devices 140a to 140e will be described.
FIG. 2 is a block diagram showing the functional configuration of the device 140.
As shown in the figure, the device 140 includes a control unit 210, a voice input unit 220, an operation reception unit 230, an address storage unit 240, a communication unit 250, a voice output unit 260, and a display unit 270. , Execution unit 280.
As an example, the voice input unit 220 is realized by a microphone and a processor that executes a program, is connected to the control unit 210, and is controlled by the control unit 210. Then, it has a function of accepting a voice input from a user and generating a voice signal (hereinafter, may be referred to as "input voice data").
The voice input unit 220 is controlled by the control unit 210 to either a state in which voice input can be accepted or a state in which voice input cannot be accepted. It becomes the state of.
As an example, the operation reception unit 230 is realized by a touch panel, a touch panel controller, and a processor that executes a program, is connected to the control unit 210, and is controlled by the control unit 210. Then, it has a function of accepting a predetermined contact operation performed by the user and generating an electric signal based on the received contact operation.
The predetermined contact operation performed by the user received by the operation reception unit 230 includes a predetermined voice input start operation indicating that the voice input unit 220 is used to start the voice input.
As an example of the voice input start operation, an operation of touching an icon for accepting the voice input start operation displayed on the touch panel which is a part of the operation reception unit 230 can be considered. Further, as another example, the operation reception unit 230 is configured to include a button for receiving a voice input start operation, and an operation of pressing this button can be considered.
As an example, the address storage unit 240 is realized by a memory and a processor that executes a program, and is connected to the communication unit 250. Then, a function for storing the IP (Internet Protocol) address in the network 120 of one of the voice dialogue agent servers 110 (hereinafter, this one is referred to as a "specific voice dialogue agent server") is provided. Have.
By this function, each of the devices 140 is associated with a specific voice dialogue agent server, which is one of the voice dialogue agent servers 110.
As an example of the memory built in the device 140, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, and the like can be considered.
As an example, the communication unit 250 is realized by a processor that executes a program, a communication LSI (Large Scale Integration), and an antenna, is connected to the control unit 210 and the address storage unit 240, and is controlled by the control unit 210. Will be done. It also has the following gateway communication function and voice dialogue agent server communication function.
The gateway communication function is a function for communicating with the gateway 130 wirelessly or by wire.
The voice dialogue agent server communication function is a function of communicating with the voice dialogue agent server 110 via the gateway 130 and the network 120.
Here, when the communication unit 250 communicates with any one of the voice dialogue agent servers 110, when the control unit 210 does not specify a specific one as the communication destination voice dialogue agent server 110, the address It communicates with the specific voice dialogue agent server by referring to the IP address stored in the storage unit 240.
As an example, the audio output unit 260 is realized by a processor that executes a program and a speaker, is connected to the control unit 210, and is controlled by the control unit 210. Then, it has a function of converting an electric signal sent from the control unit 210 into voice and outputting it.
As an example, the display unit 270 is realized by a touch panel, a touch panel controller, and a processor that executes a program, is connected to the control unit 210, and is controlled by the control unit 210. Then, it has a function of displaying an image, a character string, or the like based on an electric signal sent from the control unit 210.
The execution unit 280 has a function that the device 140 originally has as a device (for example, if the device 140 is a television, the execution unit 280 receives a television signal, decodes it, displays the decoded television image on the display, and decodes the television. It is a function to output sound from a speaker, for example, if the device 140 is an air conditioner, it is a function to blow cold air or hot air from the air outlet so that the room where the air conditioner is installed reaches the set temperature.) It is a functional block that realizes the above, is connected to the control unit 210, and is controlled by the control unit 210.
The execution unit 280 is realized by, for example, a television signal receiver, a television signal tuner, a television signal decoder, a display, a speaker, or the like when the device 140 is a television.
Further, the execution unit 280 does not necessarily have to be a configuration realized by including all the components thereof in one housing. For example, when the device 140 is a television, an example in which the remote controller and the display are included in separate housings can be considered. Similarly, each functional block constituting the device 140 does not have to be a configuration realized by including all the components thereof in one housing.
As an example, the control unit 210 is realized by a processor that executes a program, and includes a voice input unit 220, an operation reception unit 230, a communication unit 250, a voice output unit 260, a display unit 270, and an execution unit 280. A function to control the connected voice input unit 220, a function to control the operation reception unit 230, a function to control the communication unit 250, a function to control the voice output unit 260, and a function to control the display unit 270. In addition to the function of controlling the execution unit 280, it has the voice input unit state management function described below and the first device processing realization function.
The voice input unit state management function is a function for managing the state (hereinafter, also referred to as "state") of the voice input unit 220, which is either a voice input acceptable state or a voice input non-acceptable state.
FIG. 3 is a state transition diagram of the state managed by the control unit 210.
As shown in the figure, when the state is in the state where voice input cannot be accepted, the control unit 210 cannot accept voice input until (1) the operation reception unit 230 accepts the voice input start operation. The state is maintained, and (2) when the voice input start operation is received by the operation reception unit 230, the state is changed to the voice input acceptable state. Then, when the state is in the voice input acceptable state, (3) the voice input acceptable state is maintained until a predetermined time T1 (for example, 5 seconds) elapses after the state transitions to the voice input acceptable state. It is maintained, and (4) when the predetermined time T1 elapses, the state is changed to the voice input unacceptable state.
When the device 140 is activated, the control unit 210 starts management with the state as a voice input unacceptable state.
Returning to FIG. 2 again, the description of the control unit 210 will be continued.
The first device processing realization function is that the control unit 210 controls the voice input unit 220, the operation reception unit 230, the communication unit 250, the voice output unit 260, the display unit 270, and the execution unit 280. By causing the device 140 to execute the first device processing, which is a characteristic operation thereof, when the device 140 is (1) performed a voice input start operation by the user, (2) a voice input from the user is input. Receives and generates input voice data, (3) sends the generated input voice data to the voice dialogue agent, (4) receives the response voice data returned from the voice dialogue agent, and (5) receives the received response voice. It is a function that realizes a series of processes of outputting audio based on data.
The first device processing will be described in detail later in the item <1st device processing> using a flowchart.
Returning to FIG. 1, the description of the device 140 is continued again.
As an example, the gateway 130 is realized by a personal computer or the like having a communication function, and is connected to the network 120. Then, by executing the stored program, the function of communicating wirelessly or by wire with the device 140, the function of communicating with the voice dialogue agent server 110 via the network 120, and the device 140 and the voice dialogue agent server 110 It realizes the function of mediating communication between.
As an example, the voice dialogue agent server 110 is realized by a server having a communication function composed of one or a plurality of computer systems, and is connected to the network 120. Then, by executing the stored program, the function of communicating with other devices connected to the network 120, the function of communicating with the device 140 via the gateway 130, and the function of realizing the voice dialogue agent 400. And have.
FIG. 4 is a block diagram showing a functional configuration of the voice dialogue agent 400 realized by the voice dialogue agent server 110.
As shown in the figure, the voice dialogue agent 400 includes a control unit 410, a communication unit 420, a voice recognition processing unit 430, a dialogue DB (Date Base: database) storage unit 440, and a voice synthesis processing unit 450. , It is composed of the instruction generation unit 460.
As an example, the communication unit 420 is realized by a processor that executes a program and a communication LSI, is connected to the control unit 410, the voice recognition processing unit 430, and the voice synthesis processing unit 450, and is controlled by the control unit 410. Will be done. Then, it has a function of communicating with another device connected to the network 120 and a function of communicating with the device 140 via the gateway 130.
As an example, the voice recognition processing unit 430 is realized by a processor that executes a program, is connected to the control unit 410 and the communication unit 420, and is controlled by the control unit 410. Then, it has a function of performing voice recognition processing on the input voice data received by the communication unit 420 and converting the voice data into a character string (hereinafter, also referred to as "input text").
As an example, the voice synthesis processing unit 450 is realized by a processor that executes a program, is connected to the control unit 410 and the communication unit 420, and is controlled by the control unit 410. Then, it has a function of performing voice synthesis processing on the character string sent from the control unit 410 and converting it into voice data.
As an example, the dialogue DB storage unit 440 is realized by a memory and a processor that executes a program, is connected to the control unit 410, and has a function of storing the dialogue DB 500.
FIG. 5 is a data structure diagram of the dialogue DB 500 stored in the dialogue DB storage unit 440.
As shown in the figure, the dialogue DB 500 is configured by associating the keyword 510, the target device 520, the startup application 530, the processing content 540, and the response text 550.
The keyword 510 is a character string that is assumed to be included in the input text converted by the voice recognition processing unit 430.
The target device 520 is information that identifies a device that executes the process specified by the associated processing content 540 (described later).
Here, the voice dialogue agent 400 itself may be included in the device that executes the process specified by the process content 540.
The start-up application 530 executes an application program to be started in the device specified by the associated target device 520 in order to execute the process specified by the associated process content 540 (described later). Information to identify.
When the input text converted by the voice recognition processing unit 430 includes the associated keyword 510, the processing content 540 is executed by the device specified by the associated target device 520. Information that identifies the specified process.
The response text 550 is a character string (hereinafter, also referred to as "response text") that is determined to be generated based on the processing result when the processing specified by the associated processing content 540 is executed. .) Information indicating.
Returning to FIG. 4 again, the explanation of the voice dialogue agent 400 is continued.
As an example, the instruction generation unit 460 is realized by a processor that executes a program, is connected to the control unit 410, and is controlled by the control unit 410. Then, when a set of the target device 520, the startup application 530, and the processing content 540 is sent from the control unit 410, the application program specified by the startup application 530 in the device specified by the target device 520. Has a function of generating an instruction group for executing the process specified by the process content 540.
As an example, the control unit 410 is realized by a processor that executes a program, and is connected to a communication unit 420, a voice recognition processing unit 430, an interactive DB storage unit 440, a voice synthesis processing unit 450, and an instruction generation unit 460. In addition to the function of controlling the communication unit 420, the function of controlling the voice recognition processing unit 430, the function of controlling the voice synthesis processing unit 450, and the function of controlling the instruction generation unit 460, the following input text It has a reply function, an instruction generation function, an instruction execution function, and a first agent processing realization function.
The input text reply function controls the communication unit 420 to transmit the input voice data when the voice recognition processing unit 430 converts the input voice data received by the communication unit 420 into the input text. This is a function that causes the device 140 to return the input text.
The command generation function is included in the input text by referring to (1) the dialogue DB 500 stored in the dialogue DB storage unit 440 when the input text is sent from the voice recognition processing unit 430. From the keyword 510, the target device 520, the startup application 530, the processing content 540, and the response text 550 associated with the keyword 510 are read, and (2) the read target device 520 and the startup application are read. This is a function of sending a set of 530 and the processing content 540 to the instruction generation unit 460 so that the instruction generation unit 460 generates an instruction group.
The instruction execution function executes the instruction group generated by the instruction generation unit 460, generates the response text specified by the response text 550 based on the execution result, and sends it to the speech synthesis processing unit 450. It is a function.
When realizing the instruction execution function, the control unit 410 uses the communication unit 420 to communicate with the device specified by the target device 520, and causes the device to execute the instruction group. , The response text is generated by transmitting the execution result of the instruction group from the device.
The first agent processing realization function is that the control unit 410 controls the communication unit 420, the voice recognition processing unit 430, the voice synthesis processing unit 450, and the instruction generation unit 460 to the voice dialogue agent 400. By executing the first agent processing, which is a characteristic operation, the voice dialogue agent 400 receives (1) input voice data transmitted from the device, and (2) voice recognition for the received input voice data. It processes, generates input text and returns it to the device, (3) generates and executes a group of instructions based on the generated input text, (4) generates response text based on the execution result, and ( It is a function that realizes a series of processing of 5) converting the generated response text into response voice data, and (6) returning the response text and response voice data to the device.
The first agent processing will be described in detail later in the item <1st agent processing> using a flowchart.
Here, if, for example, the voice recognition processing unit 430 sends an input text "What is Mr. A's address?", The control unit 410 refers to the dialogue DB 500 stored in the dialogue DB storage unit 440. Then, the application program "contact" specified by the startup application 530 is started on the device "smartphone" specified by the target device 520, and the process of "finding Mr. A's address" specified by the processing content 540 is performed. Is executed, and the response text "Mr. A's address is XXXX" is generated based on the execution result of the process.
The operation performed by the voice dialogue system 100 configured as described above will be described with reference to the following drawings.
<Operation> The voice dialogue system 100 performs the first device processing and the first agent processing as its characteristic operation.
Hereinafter, these processes will be described in order.
<First device processing> The first device processing is a process performed by the device 140. When (1) the user performs a voice input start operation, (2) the voice input from the user is accepted and input voice data. Is generated, (3) the generated input voice data is sent to the voice dialogue agent, (4) the response voice data returned from the voice dialogue agent is received, and (5) the voice based on the received response voice data is output. This is the output process.
FIG. 6 is a flowchart of the first device processing.
The first device processing is started when the device 140 is started.
When the device 140 is activated, the state managed by the control unit 210 is in a state in which voice input cannot be accepted.
When the first device processing is started, the control unit 210 waits until the voice input start operation performed by the user using the voice dialogue system 100 is accepted by the operation reception unit 230 (step S600: No is repeated). ). Then, when the voice input start operation is accepted (step S600: Yes), the control unit 210 shifts the state from the voice input unacceptable state to the voice input acceptable state (step S610), and displays the display unit 270. , Display that the voice input can be accepted (step S620).
FIG. 11A is a schematic diagram showing an example in which the display unit 270 displays that the device 140 is in a state where voice input can be accepted when the device 140 is, for example, a smartphone.
In the figure, the touch panel 1110 constituting the smartphone is a part of the display unit 270, and the state is changed by blinking the lower right area 1120 (for example, turning on black and white alternately). It is displayed that voice input can be accepted.
Returning to FIG. 6 again, the explanation of the first device processing is continued.
When the process of step S620 is completed, the device 140 executes the first voice input process (step S630).
FIG. 7 is a flowchart of the first voice input process.
When the first voice input process is started, the voice input unit 220 accepts voice input by the user and generates input voice data (step S700). Then, when T1 elapses for a predetermined time after the state becomes the voice input acceptable state (step S710: Yes after repeating step S710: No), the control unit 210 changes the state from the voice input acceptable state to the voice input acceptance state. After transitioning to the impossible state (step S720), the display unit 270 ends the display indicating that the voice input can be accepted state (step S730).
Then, the control unit 210 controls the communication unit 250 to cause the voice dialogue agent 400 realized by the specific voice dialogue agent server to transmit the input voice data generated by the voice input unit 220 (step S740). ..
When the process of step S740 is completed, the device 140 ends the first voice input process.
Returning to FIG. 6 again, the explanation of the first device processing is continued.
When the first voice input process is completed, the control unit 210 waits until the input text returned from the voice dialogue agent 400 in response to the input voice data transmitted in the process of step S740 is received by the communication unit 250. (Repeat step S640: No.).
Here, the input text data is the input voice data transmitted in the process of step S740 converted into a character string by the voice dialogue agent 400.
When the input text is received by the communication unit 250 (step S640: Yes), the display unit 270 displays the input text (step S650).
FIG. 11B is a schematic diagram showing an example in which the display unit 270 displays the input text when the device 140 is, for example, a smartphone.
The figure is an example when the input text is the character string "What is the room temperature?". As shown in the figure, the input text "What is the temperature of the room?" Is displayed together with the character string "you" on the touch panel 1110, which is a part of the display unit 270.
Returning to FIG. 6 again, the explanation of the first device processing is continued.
When the process of step S650 is completed, the control unit 210 receives the response text and the response voice data returned from the voice dialogue agent 400 in response to the input voice data transmitted in the process of step S740 by the communication unit 250. Wait until it is done (step S660: No is repeated).
When the response text and the response voice data are received by the communication unit 250 (step S660: Yes), the display unit 270 displays the response text (step S670), and the voice output unit 260 displays the response voice. Convert the data to voice and output it (step S680).
FIG. 11C is a schematic diagram showing an example in which the display unit 270 displays the response text when the device 140 is, for example, a smartphone.
The figure is an example when the response text is the character string "Which room is the temperature?". As shown in the figure, the response text "Which room is the temperature?" Is displayed on the touch panel 1110, which is a part of the display unit 270, together with the character string "House agent". Has been done.
Returning to FIG. 6 again, the explanation of the first device processing is continued.
When the processing of step S680 is completed, the device 140 ends the processing of the first device.
<1st agent processing> The 1st agent processing is a process performed by the voice dialogue agent 400, which (1) receives the input voice data transmitted from the device, and (2) voices the received input voice data. Performs recognition processing, generates input text and returns it to the device, (3) generates and executes a group of instructions based on the generated input text data, and (4) generates response text based on the execution result. , (5) The generated response text is converted into response voice data, and (6) the response text and response voice data are returned to the device.
FIG. 8 is a flowchart of the first agent processing.
The first agent processing is started when the voice dialogue agent 400 is started.
When the first agent processing is started, the voice dialogue agent 400 waits until the input voice data transmitted from the device 140 is received by the communication unit 420 (step S800: No is repeated). Then, when the input voice data is received (step S800: Yes), the voice dialogue agent 400 executes the first instruction execution process (step S810).
FIG. 9 is a flowchart of the first instruction execution process.
When the first command execution process is started, the voice recognition processing unit 430 performs voice recognition processing on the input voice data received by the communication unit 420 and converts the input voice data into input text which is a character string. (Step S900).
When converted into input text, the control unit 410 controls the communication unit 420 to return the input text to the device 140 that has transmitted the input voice data (step S910).
Then, the control unit 410 refers to (1) the dialogue DB 500 stored in the dialogue DB storage unit 440, and is associated with the keyword 510 from the keyword 510 included in the input text. The device 520, the startup application 530, the processing content 540, and the response text 550 are read, and (2) the pair of the read target device 520, the startup application 530, and the processing content 540 is set as the instruction generator 460. To cause the instruction generator 460 to generate an instruction group (step S920).
When the instruction group is generated, the control unit 410 executes the generated instruction group (step S930), and based on the execution result, generates the response text specified by the response text 550 (step S940). .. Here, the control unit 410 uses the communication unit 420 to communicate with the device specified by the target device 520, causes the device to execute a part of the instruction group, and issues a command from the device, if necessary. The response text is generated by sending the execution result of the group.
When the response text is generated, the voice synthesis processing unit 450 performs voice synthesis processing on the generated response text to generate response voice data (step S950).
When the response voice data is generated, the control unit 410 controls the communication unit 420 to send the generated response text and the generated response voice data to the device 140, which is the source of the input voice data. Is sent (step S960).
When the process of step S960 is completed, the voice dialogue agent 400 ends the first instruction execution process.
Returning to FIG. 8 again, the explanation of the first agent processing is continued.
When the first instruction execution process is completed, the voice dialogue agent 400 returns to the process of step S800 and continues the process of step S800 and thereafter.
A specific example of the operation performed by the voice dialogue system 100 having the above configuration will be described below with reference to the drawings.
<Specific example> In FIG. 10, a user using the voice dialogue system 100 engages in a voice dialogue with the voice dialogue agent 400 using the device 140 (here, it is assumed to be a smartphone). It is a processing procedure diagram which shows typically the state that the voice dialogue agent 400 performs the processing which reflected the content of the dialogue.
When the user performs a voice input start operation on the device 140 (corresponding to step S1000, step S600 in FIG. 6: Yes), the state transitions to a voice input acceptable state (step S1005, step S610 in FIG. 6). The device 140 performs the first voice input processing (corresponding to step S1010 and step S630 in FIG. 6).
In FIG. 11 (a), in the first voice input process, the touch panel 1110, which is a part of the display unit 270 of the device 140, which is a smartphone, blinks the area 1120 while the state is in the voice input acceptable state. It is a figure which shows typically the example of the state which displays that the state is a state which can accept voice input by making it make it.
Returning to FIG. 10 again, the explanation of the specific example will be continued.
In this first voice input process, if the user inputs a voice saying "What is the room temperature?", The device 140 transmits the input voice data "What is the room temperature?" To the voice dialogue agent 400. (Corresponds to step S740 in Figure 7).
Then, the voice dialogue agent 400 receives the input voice data (corresponds to step S800: Yes in FIG. 8) and executes the first instruction execution process (corresponds to step S1060 in FIG. 8 and step S810 in FIG. 8).
Here, if the voice dialogue agent 400 generates the response voice data "Which room is the temperature?" In this first instruction execution process, the voice dialogue agent 400 tells the device 140 "Which room?" Send the response voice data "Is it the room temperature?" (Corresponding to step S960 in Fig. 9).
Then, the device 140 receives the response voice data (corresponding to step S660: Yes in FIG. 6) and outputs a voice "Which room is the temperature?" (Step S1015, Step S680 in FIG. 6). Corresponds to).
In the process of step S1010, when a predetermined time T1 elapses after the state transitions to the voice input acceptable state, the state transitions to the voice input unacceptable state again (corresponding to step S720 in FIG. 7). ). Therefore, the user who hears the voice "Which room temperature is it?" Output from the device 140 performs a new voice input start operation on the device 140 in order to perform a new voice input. (Corresponds to step S1020, step S600 in Figure 6: Yes). Then, the state transitions to the voice input acceptable state (step S1025, corresponding to step S610 in FIG. 6), and the device 140 performs the first voice input process (step S1030: corresponding to step S630 in FIG. 6). ..
In FIG. 11 (c), in the first voice input process, the touch panel 1110, which is a part of the display unit 270 of the device 140, which is a smartphone, blinks the area 1120 while the state is in the voice input acceptable state. It is a figure which shows typically the example of the state which displays that the state is a state which can accept voice input by making it make it.
Returning to FIG. 10 again, the explanation of the specific example will be continued.
In this first voice input process, if the user inputs the voice of "living room.", The device 140 transmits the input voice data of "living room." To the voice dialogue agent 400 (step of FIG. 7). Compatible with S740).
Then, the voice dialogue agent 400 receives the input voice data (corresponds to step S800: Yes in FIG. 8) and executes the first instruction execution process (corresponds to step S1065 in FIG. 8 and step S810 in FIG. 8).
Here, if the voice dialogue agent 400 generates the response voice data "The temperature of the living room is 28 degrees. Is there anything else?" In this first instruction execution process, the voice dialogue agent 400 will generate the response voice data. Send the response voice data to the device 140, "The living room temperature is 28 degrees. Is there anything else?" (Corresponding to step S960 in Figure 9).
Then, the device 140 receives the response voice data (corresponding to step S660: Yes in Fig. 6) and outputs the voice "The living room temperature is 28 degrees. Is there anything else?" (Step). Corresponds to S1035, step S680 in Figure 6).
In the process of step S1010, when a predetermined time T1 elapses after the state transitions to the voice input acceptable state, the state transitions to the voice input unacceptable state again (corresponding to step S720 in FIG. 7). ). For this reason, the user who hears the voice "The temperature of the living room is 28 degrees. Is there anything else?" Output from the device 140 is newly added to the device 140 in order to make a new voice input. Perform a voice input start operation (corresponding to step S1040 and step S600: Yes in Fig. 6). Then, the state transitions to the voice input acceptable state (step S1045, corresponding to step S610 in FIG. 6), and the device 140 performs the first voice input process (step S1050: corresponding to step S630 in FIG. 6). ..
FIG. 12 shows that in the first voice input process, the touch panel 1110, which is a part of the display unit 270 of the device 140, which is a smartphone, blinks the area 1120 during the period when the state is in the voice input acceptable state. , Is a diagram schematically showing an example of a state indicating that the state is in a state where voice input can be accepted.
Returning to FIG. 10 again, the explanation of the specific example will be continued.
In this first voice input process, if the user inputs the voice "No. Thank you.", The device 140 transmits the input voice data "No. Thank you." To the voice dialogue agent 400. (Corresponds to step S740 in Figure 7).
Then, the voice dialogue agent 400 receives the input voice data (corresponds to step S800: Yes in FIG. 8) and executes the first instruction execution process (corresponds to step S1070 in FIG. 8 and step S810 in FIG. 8).
Here, if the voice dialogue agent 400 generates the response voice data "terminates" in this first instruction execution process, the voice dialogue agent 400 tells the device 140 "terminates." Send the response voice data (corresponding to step S960 in Figure 9).
Then, the device 140 receives the response voice data (corresponds to step S660: Yes in FIG. 6) and outputs the voice "End" (corresponds to step S1055 and step S680 in FIG. 6).
<Discussion> According to the voice dialogue system 100 having the above configuration, the user who uses the voice dialogue system 100 sets the state of the device 140 to the state in which voice input can be accepted by performing the voice input start operation on the device 140. , Perform voice input. After that, when the predetermined time T1 elapses, the state of the device 140 is changed to the voice input unacceptable state even if the user does not perform an operation for making the state of the device 140 unacceptable. become.
Therefore, compared to the voice dialogue system in which it is necessary to perform an operation for changing the state of the device 140 to the voice input unacceptable state each time the voice input is completed, the voice dialogue system 100 accompanies the voice input. The number of operations that the user has to perform is reduced. <Embodiment 2> <Overview> Hereinafter, as one aspect of the voice dialogue method according to the present invention and one aspect of the device according to the present invention, a part of the voice dialogue system 100 according to the first embodiment is modified. 1 Describe the modified voice dialogue system.
In the voice dialogue system 100 according to the first embodiment, when the device 140 performs a voice input start operation by the user, the voice input is possible for a period from the time when the operation is performed until a predetermined time T1 elapses. It was an example of the configuration.
On the other hand, in the first modified voice dialogue system according to the second embodiment, when each device outputs a voice based on the response voice data in addition to the above period, the predetermined time T1 from the time when the output is performed. This is an example of a configuration in which voice input is possible even during the period until the elapse of.
Hereinafter, the details of the first modified voice dialogue system will be described with reference to the drawings, focusing on the differences from the voice dialogue system 100 in the first embodiment.
<Structure> The first modified voice dialogue system is modified so that the device 140 is changed to the device 1300 from the voice dialogue system 100 in the first embodiment.
The device 1300 has not been changed as hardware from the device 140 in the first embodiment, but a part of the software stored as an execution target has been changed. As a result, the device 1300 is modified from the device 140 in the first embodiment so that a part of its function is changed.
FIG. 13 is a block diagram showing a functional configuration of the device 1300.
As shown in the figure, the device 1300 is modified from the device 140 (see FIG. 2) in the first embodiment so that the control unit 210 is changed to the control unit 1310.
In the control unit 1310, the voice input unit state management function among the functions of the control unit 210 is changed from the control unit 210 in the first embodiment to the first modified voice input unit state management function described below, and the first The device processing realization function has been modified so as to be changed to the second function processing realization function described below.
The first modified voice input unit state management function is the same as the voice input unit state management function in the first embodiment, of the voice input unit 220 which is either a voice input acceptable state or a voice input non-acceptable state. It is a function of managing a state (that is, a "state"), and a part of the state transition condition is changed from the voice input unit state management function in the first embodiment.
FIG. 14 is a state transition diagram of the state managed by the control unit 1310.
As shown in the figure, in the control unit 1310, when the state is in the voice input unacceptable state, (1) the operation reception unit 230 accepts the voice input start operation, or the voice output unit 260 , The voice input unacceptable state is maintained until the voice other than the predetermined voice among the voices based on the response voice data is output, and (2) the voice input start operation is accepted by the operation reception unit 230, or , When the voice output unit 260 outputs a voice other than a predetermined voice among the voices based on the response voice data, the state is changed to the voice input acceptable state. Then, when the state is in the voice input acceptable state, (3) the voice input acceptable state is maintained until a predetermined time T1 (for example, 5 seconds) elapses after the state transitions to the voice input acceptable state. It is maintained, and (4) when T1 elapses for a predetermined time, the state is changed to a state in which voice input cannot be accepted.
Here, the predetermined voice among the voices based on the response voice data is a predetermined voice indicating that a new voice input is not required, for example, a voice saying "End." .. Hereinafter, this voice is also referred to as "dialogue end voice".
When the device 1300 is activated, the control unit 1310 starts management with the state as a voice input unacceptable state.
Returning to FIG. 13 again, the description of the control unit 1310 will be continued.
The second function processing realization function is that the control unit 1310 controls the voice input unit 220, the operation reception unit 230, the communication unit 250, the voice output unit 260, the display unit 270, and the execution unit 280. Then, by causing the device 1300 to execute the second device processing, which is a characteristic operation thereof, when the device 1300 is (1) performed a voice input start operation by the user, (2) a voice input from the user is performed. To generate input voice data, (3) send the generated input voice data to the voice dialogue agent, (4) receive the response voice data returned from the voice dialogue agent, and (5) receive the response. Outputs a voice based on voice data, and (6) if the voice is not the dialogue end voice, realizes a series of processes of (2) repeating the following processes again even if the user does not perform the voice input start operation. It is a function.
The second device processing will be described in detail later in the item <Second device processing> using a flowchart.
The operation performed by the first modified voice dialogue system configured as described above will be described below with reference to the drawings.
<Operation> As a characteristic operation of the first modified voice dialogue system, in addition to the first agent processing in the first embodiment, a part of the processing is changed from the first device processing in the first embodiment. 2nd device processing is performed.
Hereinafter, this second device processing will be described focusing on the changes from the first device processing.
<Second device processing> The second device processing is a process performed by the device 1300. When (1) the user performs a voice input start operation, (2) the voice input from the user is accepted and input voice data. Is generated, (3) the generated input voice data is sent to the voice dialogue agent, (4) the response voice data returned from the voice dialogue agent is received, and (5) the voice based on the received response voice data is output. If (6) the voice is not the dialogue end voice, the process of (2) and the following is repeated again even if the user does not perform the voice input start operation.
FIG. 15 is a flowchart of the second device processing.
The second device processing is started when the device 1300 is started.
When the device 1300 is activated, the state managed by the control unit 1310 is in a state in which voice input cannot be accepted.
In the figure, the processes of steps S1500 to S1580 are the same as the processes of steps S600 to S680 of the first device process (see FIG. 6) in the first embodiment. Therefore, these processes have already been described.
When the process of step S1580 is completed, the control unit 1310 checks whether or not the voice output by the voice output unit 260 in the process of step S1580 is the dialogue end voice (step S1585). This process is realized, for example, by checking whether the response text received in the process of step S1560: Yes is a predetermined character string (for example, the character string "End"). Will be done.
In the process of step S1585, if it is not the dialogue end voice (step S1585: No), the control unit 1310 shifts the state from the voice input unacceptable state to the voice input accepted state (step S1590), and the display unit. Display on 270 that voice input can be accepted (step S1595).
When the process of step S1595 is completed, the device 1300 returns to the process of step S1530 and repeats the process of step S1530 and the like.
In the process of step S1585, if the dialogue end voice is used (step S1585: Yes), the device 1300 ends the second device process.
A specific example of the operation performed by the first modified voice dialogue system having the above configuration will be described below with reference to the drawings.
<Specific example> In Fig. 16, a user using the first modified voice dialogue system uses the device 1300 (here, tentatively, it is a smartphone) to have a voice dialogue with the voice dialogue agent 400. It is a processing procedure diagram which shows typically how the voice dialogue agent 400 performs the processing which reflected the content of the dialogue.
Here, the dialogue end voice will be described as a voice saying "End."
In the figure, the processes of steps S1600 to S1615, the processes of steps S1630 to S1635, the processes of steps S1650 to S1655, and the processes of steps S1660 to S1670 are specific examples in the first embodiment, respectively. (See FIG. 10), the processes of steps S1000 to S1015, the processes of steps S1030 to S1035, the processes of steps S1050 to S1055, and the processes of steps S1060 to S1070 are the same. Therefore, these processes have already been described.
When the processing of step S1615 is completed, the voice "Which room is the temperature?" Is not the dialogue end voice (corresponding to step S1585: No in FIG. 15), so the state transitions to the voice input acceptable state. (Corresponding to step S1625 and step S1590 in FIG. 15), the device 1300 performs the first voice input processing (step S1630: corresponding to step S1530 in FIG. 15).
When the process of step S1635 is completed, the voice "The temperature of the living room is 28 degrees. Is there anything else?" Is not the dialogue end voice (corresponding to step S1585: No in Fig. 15), so the state is voice input. After transitioning to the acceptable state (corresponding to step S1645 in FIG. 15 and step S1590 in FIG. 15), the device 1300 performs the first voice input processing (step S1650: corresponding to step S1530 in FIG. 15).
When the processing of step S1635 is completed, the voice "End" is the dialogue end voice (corresponding to step S1585: Yes in Fig. 15), so the state does not transition to the voice input acceptable state, and the device 1300 Ends the second device processing.
<Discussion> According to the first modified voice dialogue system having the above configuration, when the device 1300 outputs a voice based on the response voice data sent from the voice dialogue agent 400, if the voice is not the dialogue end voice, Even if the user who uses the device 1300 does not perform the voice input operation, the voice input can be accepted.
Therefore, once the voice input start operation is performed on the device 1300, the user waits until the voice based on the response voice data is output until the predetermined period T1 elapses until the dialogue end voice is output. During this period, new voice input can be performed without performing a new voice input operation on the device 1300.
As described above, in the first modified voice dialogue system, the number of operations that the user needs to perform in connection with the voice input is further reduced as compared with the voice dialogue system 100 in the first embodiment. <Embodiment 3> <Overview> Hereinafter, as one aspect of the voice dialogue method according to the present invention and one aspect of the device according to the present invention, a part of the voice dialogue system 100 according to the first embodiment is modified. 2 The modified voice dialogue system will be described.
In the voice dialogue system 100 according to the first embodiment, when the device 140 performs a voice input start operation by the user, the voice input is possible for a period from the time when the operation is performed until a predetermined time T1 elapses. It was an example of the configuration.
On the other hand, in the second modified voice dialogue system according to the third embodiment, when each device performs a voice input start operation by the user, the period from the time when the operation is performed to the time when the dialogue end voice is output. , This is an example of a configuration that enables voice input.
Hereinafter, the details of the second modified voice dialogue system will be described with reference to the drawings, focusing on the differences from the voice dialogue system 100 in the first embodiment.
<Structure> The second modified voice dialogue system is modified so that the device 140 is changed to the device 1700 from the voice dialogue system 100 in the first embodiment.
The device 1700 has not been changed as hardware from the device 140 in the first embodiment, but a part of the software stored as an execution target has been changed. As a result, the device 1700 is modified from the device 140 in the first embodiment so that a part of its function is changed.
FIG. 17 is a block diagram showing a functional configuration of the device 1700.
As shown in the figure, the device 1700 is modified from the device 140 (see FIG. 2) in the first embodiment so that the control unit 210 is changed to the control unit 1710.
In the control unit 1710, the voice input unit state management function among the functions of the control unit 210 is changed from the control unit 210 in the first embodiment to the second modified voice input unit state management function described below, and the first The device processing realization function has been modified so as to be changed to the third function processing realization function described below.
The second modified voice input unit state management function is the same as the voice input unit state management function in the first embodiment and the first modified voice input unit state management function in the second embodiment, that is, the voice input acceptable state and the voice. It is a function of managing the state (that is, "state") of the voice input unit 220 which is one of the input unacceptable states, and a part of the state transition conditions is the voice input unit state management in the first embodiment. It has been changed from the function.
FIG. 18 is a state transition diagram of the state managed by the control unit 1710.
As shown in the figure, when the state is in the state where voice input cannot be accepted, the control unit 1710 cannot accept voice input until (1) the operation reception unit 230 accepts the voice input start operation. The state is maintained, and (2) when the voice input start operation is received by the operation reception unit 230, the state is changed to the voice input acceptable state. Then, when the state is in the voice input acceptable state, (3) the voice input acceptable state until the dialogue end voice (for example, the voice "Ends") is output by the voice output unit 260. (4) When the dialogue end voice is output by the voice output unit 260, the state is changed to the voice input unacceptable state.
Returning to FIG. 17, the description of the control unit 1710 will be continued.
The third function processing realization function is that the control unit 1710 controls the voice input unit 220, the operation reception unit 230, the communication unit 250, the voice output unit 260, the display unit 270, and the execution unit 280. Then, by causing the device 1700 to execute the third device process, which is a characteristic operation thereof, when the device 1700 is (1) performed a voice input start operation by the user, (2) a voice input from the user is performed. To generate input voice data, (3) send the generated input voice data to the voice dialogue agent, (4) receive the response voice data returned from the voice dialogue agent, and (5) receive the response. Outputs voice based on voice data, and (6) if the voice is not the dialogue end voice, realizes a series of processes of (2) repeating the following processes again even if the user does not perform the voice input start operation. It is a function.
The third device processing will be described in detail later using a flowchart in the item <Third device processing>.
The operation performed by the second modified voice dialogue system configured as described above will be described below with reference to the drawings.
<Operation> As a characteristic operation of the second modified voice dialogue system, in addition to the first agent processing in the first embodiment, a part of the processing is changed from the first device processing in the first embodiment. 3rd equipment processing is performed.
Hereinafter, this third device processing will be described focusing on the changes from the first device processing.
<Third device processing> The third device processing is a process performed by the device 1700. When (1) the user performs a voice input start operation, (2) the voice input from the user is accepted and input voice data. Is generated, (3) the generated input voice data is sent to the voice dialogue agent, (4) the response voice data returned from the voice dialogue agent is received, and (5) the voice based on the received response voice data is output. If the voice is output and (6) the voice is not the dialogue end voice, the process of (2) and the following is repeated again even if the voice input start operation is not performed by the user.
FIG. 19 is a flowchart of the third device processing.
The third device processing is started when the device 1700 is started.
When the device 1700 is started, the state managed by the control unit 1710 is in a state where voice input cannot be accepted.
In the figure, the processes of steps S1900 to S1920 and the processes of steps S1940 to S1980 are the processes of steps S600 to S620 and the steps of the first device process (see FIG. 6) in the first embodiment, respectively. It is the same process as the process of S640 to step S680. Therefore, these processes have already been described.
When the process of step S1920 is completed, the device 1700 executes the second voice input process (step S1930).
FIG. 20 is a flowchart of the second voice input process.
When the second voice input process is started, the voice input unit 220 receives the voice input by the user and generates the input voice data (step S2000).
Then, the control unit 1910 controls the communication unit 250 to cause the voice dialogue agent 400 to transmit the input voice data generated by the voice input unit 220 (step S2040).
When the process of step S2040 is completed, the device 1700 ends the second voice input process.
Returning to FIG. 19 again, the explanation of the third device processing is continued.
When the second voice input process is completed, the device 1900 proceeds to the process of step S1940 and performs the process of step S1940 or lower.
When the process of step S1980 is completed, the control unit 1710 examines whether or not the voice output by the voice output unit 260 in the process of step S1980 is the dialogue end voice (step S1985). This process is realized, for example, by checking whether the response text received in the process of step S1960: Yes is a predetermined character string (for example, the character string "End"). Will be done.
In the process of step S1985, if it is not the dialogue end voice (step S1985: No), the device 1900 returns to the process of step S1930 and repeats the process of step S1930 and subsequent steps.
In the process of step S1985, when the dialogue end voice is used (step S1585: Yes), the control unit 1710 shifts the state from the voice input acceptable state to the voice input non-acceptable state (step S1990).
When the processing of step S1990 is completed, the device 1900 ends the processing of the third device.
A specific example of the operation performed by the second modified voice dialogue system having the above configuration will be described below with reference to the drawings.
<Specific example> In Fig. 21, a user using the second modified voice dialogue system talks with the voice dialogue agent 400 using voice using the device 1700 (here, it is assumed to be a smartphone). It is a processing procedure diagram which shows typically how the voice dialogue agent 400 performs the processing which reflected the content of the dialogue.
Here, the dialogue end voice will be described as a voice saying "End."
In the figure, the process of step S2100, the process of step S2105, the process of step S2115, the process of step S2135, the process of step S2155, and the process of steps S2160 to S2170 are the first embodiments. In the specific example (see FIG. 10), the processing of step S1000, the processing of step S1005, the processing of step S1015, the processing of step S1035, the processing of step S1055, and the processing of steps S1060 to S1070 are the same. It is a process. Therefore, these processes have already been described.
When the processing of step S2105 is completed, the device 1700 performs the second audio processing (corresponding to step S2110 and step S1930 of FIG. 19).
In this second voice input process, if the user inputs a voice saying "What is the room temperature?", The device 1700 transmits the input voice data "What is the room temperature?" To the voice dialogue agent 400. (Corresponds to step S2040 in Figure 20).
When the processing of step S2115 is completed, the voice "Which room temperature is it?" Is not the dialogue end voice (corresponding to step S1985: No in FIG. 19), so the device 1700 performs the second voice processing (corresponding to step S1985: No in FIG. 19). Corresponds to step S2130 and step S1930 in Figure 19).
In this second voice input process, if the user inputs the voice of "living room.", The device 1700 transmits the input voice data of "living room." To the voice dialogue agent 400 (step of FIG. 20). Compatible with S2040).
When the process of step S2135 is completed, the voice "The temperature of the living room is 28 degrees. Is there anything else?" Is not the voice of the end of the dialogue (corresponding to step S1985: No in Fig. 19). Perform the second voice processing (corresponding to step S2150 and step S1930 in FIG. 19).
In this second voice input process, if the user inputs the voice "No. Thank you.", The device 1700 transmits the input voice data "No. Thank you." To the voice dialogue agent 400. (Corresponds to step S2040 in Figure 20).
When the processing of step S2135 is completed, the voice "End" is the dialogue end voice (corresponding to step S1985: Yes in FIG. 19), so the state is changed to the voice input acceptable state (Fig. 19). (Corresponding to step S1990), the device 1700 ends the third device process.
<Discussion> According to the second modified voice dialogue system having the above configuration, once the voice input start operation is performed, the device 1700 receives voice input from the time when the operation is performed until the dialogue end voice is output. Stay in a possible state.
Therefore, once the voice input start operation is performed on the device 1700, the user does not have to perform a new voice input operation on the device 1700 until the dialogue end voice is output. You can enter.
As described above, in the second modified voice dialogue system, the number of operations that the user needs to perform in connection with the voice input is further reduced as compared with the voice dialogue system 100 in the first embodiment. <Embodiment 4> <Overview> Hereinafter, as one aspect of the voice dialogue method according to the present invention and one aspect of the device according to the present invention, a part of the second modified voice dialogue system according to the third embodiment is modified. The third modified voice dialogue system will be described.
In the second modified voice dialogue system according to the third embodiment, when the device 1700 starts communication with the voice dialogue agent A, the voice dialogue agent to be communicated is the voice dialogue agent until the series of processes is completed. It was an example of a configuration limited to A.
On the other hand, in the third modified voice dialogue system in the fourth embodiment, when the device starts communication with the voice dialogue agent A, the user who uses the third modified voice dialogue system uses the device. Then, when a voice input is made to the effect that communication is desired with another voice agent B, the communication partner of the device is changed from the voice dialogue agent A to the voice dialogue agent B thereafter. It is an example.
Hereinafter, the details of the third modified voice dialogue system will be described with reference to the drawings, focusing on the differences from the second modified voice dialogue system in the third embodiment.
<Structure> The third modified voice dialogue system is modified from the second modified voice dialogue system in the third embodiment so that the voice dialogue agent 400 is changed to the voice dialogue agent 2200.
The voice dialogue agent 2200 is realized by the voice dialogue agent server 110, similarly to the voice dialogue agent 400 in the third embodiment.
A part of the software for realizing the voice dialogue agent 2200 executed by the voice dialogue agent server 110 is changed from the software for realizing the voice dialogue agent 400 in the third embodiment. As a result, the voice dialogue agent 2200 is modified from the voice dialogue agent 400 in the third embodiment so that a part of its function is changed.
FIG. 22 is a block diagram showing a functional configuration of the voice dialogue agent 2200.
As shown in the figure, in the voice dialogue agent 2200, the target agent DB storage unit 2220 is added from the voice dialogue agent 400 (see FIG. 4) in the third embodiment, and the control unit 410 is changed to the control unit 2210. It has been transformed to be.
As an example, the target agent DB storage unit 2220 has a function realized by a memory and a processor that executes a program, connected to the control unit 2210, and stores the target agent DB 2300.
FIG. 23 is a data structure diagram of the target agent DB 2300 stored in the target agent DB storage unit 2220.
As shown in the figure, the target agent DB2300 is configured by associating the keyword 2310, the target agent 2320, and the IP address 2330.
Keyword 2310 is a character string that is assumed to be included in the input text converted by the voice recognition processing unit 430.
The target agent 2320 is information for identifying one of the voice dialogue agents 2200 (hereinafter, this one is referred to as an "additional voice dialogue agent") to be communicated with the device 140.
In this example, the additional voice target agent identified by the target agent 2320 is an example of a configuration in which a car agent, a retail store agent, and a house agent are included.
Here, the car agent is a voice dialogue agent 2200 that provides relatively abundant services for the equipment mounted on the car, and the retail store agent is provided for the equipment placed in the retail store. The voice dialogue agent 2200 has a relatively good service, and the house agent is a voice dialogue agent 2200 with a relatively good service provided for the equipment placed in the house (house).
IP address 2330 is the IP address in network 120 for the voice dialogue agent server 110 that implements the additional voice dialogue agent identified by the associated target agent 2320.
As shown in FIG. 23, one target agent 2320 is associated with one or more keywords 2310. As an example, the keyword 2310 such as "inside the car", "car", "car", and "navigation" is associated with "car agent".
By associating a plurality of keywords 2310 with one target agent 2320, the voice dialogue agent 2200 can deal with input ambiguity.
For example, when a user wants to communicate with a car agent, he / she may voice-input "Connect to the voice dialogue agent of the navigation system." Or "Connect to the voice dialogue agent of the car." "In some cases, voice input is performed.
Here, since both "navigation" and "car" are associated with "car agent", the voice dialogue agent 2200 can refer to the target agent DB2300 to be the target regardless of which voice input is used. Agent 2320 can be identified as a car agent.
Returning to FIG. 22, the explanation of the voice dialogue agent 2200 is continued.
In the control unit 2210, from the control unit 410 in the third embodiment, among the functions possessed by the control unit 410, the first agent processing realization function is the second agent processing realization function described below and the third agent processing described below. It has been transformed to be changed to a realization function.
The second agent processing realization function is that the control unit 2210 controls the communication unit 420, the voice recognition processing unit 430, the voice synthesis processing unit 450, and the command generation unit 460 to the voice dialogue agent 2200. By executing the second agent processing, which is a characteristic operation, the voice dialogue agent 2200 receives (1) input voice data transmitted from the device, and (2) voice recognition for the received input voice data. Performs processing, generates input text and replies to the device, and (3) when the generated input text indicates that you want to communicate with another voice dialogue agent, the device and its It realizes communication with the voice dialogue agent, (4) otherwise generates and executes a group of instructions based on the generated input text, and (5) generates a response text based on the execution result. It is a function that realizes a series of processes of (6) converting the generated response text into response voice data, and (7) returning the response text and response voice data to the device.
The second agent processing will be described in detail later using a flowchart in the item <Second agent processing>.
The third agent processing realization function is that the control unit 2210 controls the communication unit 420, the voice recognition processing unit 430, the voice synthesis processing unit 450, and the instruction generation unit 460 to the voice dialogue agent 2200. By executing the third agent processing, which is a characteristic operation, the voice dialogue agent 2200 (1) starts communication with the device in response to a request from another voice input agent, and (2) transmits from the device. Receives the input voice data that has been received, (3) performs voice recognition processing on the received input voice data, generates input text and returns it to the device, and (4) commands based on the generated input text. Is generated and executed, (5) a response text is generated based on the execution result, (6) the generated response text is converted into response voice data, and (7) the response text and the response voice data are sent to the device. It is a function that realizes a series of processes of replying.
The third agent processing will be described in detail later using a flowchart in the item <Third agent processing>.
The operation performed by the third modified voice dialogue system configured as described above will be described below with reference to the drawings.
<Operation> As a characteristic operation of the third modified voice dialogue system, in addition to the first device processing in the third embodiment, a part of the processing is changed from the first agent processing in the third embodiment. In addition, the second agent processing and the third agent processing are performed.
Hereinafter, these second agent processing and third agent processing will be described focusing on the changes from the first agent processing.
<Second agent processing> The second agent processing is the processing performed by the voice dialogue agent 2200, which (1) receives the input voice data transmitted from the device, and (2) voices the received input voice data. Performs recognition processing, generates input text and returns it to the device, and (3) when the generated input text indicates that you want to communicate with another voice dialogue agent, the device and It realizes communication with the voice dialogue agent, (4) otherwise generates and executes a group of instructions based on the generated input text, and (5) generates a response text based on the execution result. , (6) The generated response text is converted into response voice data, and (7) the response text and response voice data are returned to the device.
FIG. 24 is a flowchart of the second agent processing.
The second agent processing is started when the voice dialogue agent 2200 is started.
When the second agent processing is started, the voice dialogue agent 2200 waits until the input voice data transmitted from the device 1700 is received by the communication unit 420 (step S2400: No is repeated). Then, when the input voice data is received (step S2400: Yes), the voice dialogue agent 2200 executes the second instruction execution process (step S2410).
FIG. 25 is a flowchart of the second instruction execution process.
In the figure, the processes of steps S2500 to S2510 and the processes of steps S2520 to S2560 are the processes of steps S900 to S910 of the first instruction execution process (see FIG. 9) in the third embodiment, respectively. The process is the same as the process of steps S920 to S960. Therefore, these processes have already been described.
When the processing of step S2510 is completed, the control unit 2210 checks whether the input text converted by the voice recognition processing unit 430 requires communication with another voice dialogue agent (step S2515).
If the process of step S2515 does not require communication with another voice dialogue agent (step S2515: No), the voice dialogue agent 2200 proceeds to the process of step S2520 and performs the processes of steps S2520 to S2560. ..
In the process of step S2515, when communication with another voice dialogue agent is requested (step S2515: Yes), the control unit 2210 refers to the target agent DB 2300 stored in the target agent DB storage unit 2220. Then (step S2517), the voice dialogue agent 2200 required as the communication destination is specified. That is, the additional voice dialogue agent specified by the target agent 2320 associated with the keyword 2310 included in the input text converted into a character string by the voice recognition processing unit 430 is set as the voice dialogue agent 2200 requested as the communication destination. Identify.
When the additional voice dialogue agent required as the communication destination is specified, the control unit 2210 starts communication between the specified additional voice target agent and the device 1700 that is the source of the input voice data. A predetermined signal (hereinafter, this signal is referred to as a "connection command") is generated (step S2565).
When the connection command is generated, the control unit 2210 controls the communication unit 420 and uses the IP address 2330 associated with the keyword 2310 to connect the generated connection to the additional voice target agent. Have the instruction sent (step S2570).
After that, the control unit 2210 waits until the connection response (described later) returned from the additional voice dialogue agent in response to the connection command transmitted in the process of step S2570 is received by the communication unit 420 (step S2575: No). repeat.).
When the connection response is received by the communication unit 420 (step S2575: Yes), the voice dialogue agent 2200 executes the first connection response processing (step S2580).
FIG. 26 is a flowchart of the first connection response process.
When the first connection response process is started, the control unit 2210 indicates that communication between the additional voice dialogue agent and the device 1700 is possible, and a predetermined response text (for example, "" Additional voice ". The string "Connected to Dialogue Agent") is generated (step S2600).
Here, in the "additional voice dialogue agent" part, the name of the voice dialogue agent 2200 specified by the target agent 2320 of the target agent DB2300 (here, among the car agent, the retail store agent, and the house agent). Either) is to be inserted.
When the response text is generated, the voice synthesis processing unit 450 performs voice synthesis processing on the generated response text to generate response voice data (step S2610).
When the response voice data is generated, the control unit 2210 controls the communication unit 420 to send the generated response text and the generated response voice data to the device 1700 that is the source of the input voice data. Is sent (step S2620).
When the processing of step S2620 is completed, the voice dialogue agent 2200 ends the first connection response processing.
Returning to FIG. 25 again, the explanation of the second instruction execution process is continued.
When the first connection response processing is completed, the voice dialogue agent 2200 waits until the disconnection response (described later) sent from the additional voice dialogue agent is received by the communication unit 420 (step S2585: No is repeated).
When the disconnect response is received by the communication unit 420 (step S2585: Yes), the voice dialogue agent 2200 executes the disconnect response process (step S2590).
FIG. 27 is a flowchart of disconnection response processing.
When the disconnect response process is started, the control unit 2210 indicates that the communication between the additional voice dialogue agent and the device 1700 has been completed with a predetermined response text (for example, "additional voice dialogue agent". The string "Disconnected. Is there anything else?") Is generated (step S2700).
Here, in the "additional voice dialogue agent" part, the name of the voice dialogue agent 2200 specified by the target agent 2320 of the target agent DB2300 (here, among the car agent, the retail store agent, and the house agent). Either) is to be inserted.
When the response text is generated, the voice synthesis processing unit 450 performs voice synthesis processing on the generated response text to generate response voice data (step S2710).
When the response voice data is generated, the control unit 2210 controls the communication unit 420 to generate the generated response text and the generated response text for the device 1700 that is the source of the input voice data in the process of step S2400. The response voice data is transmitted (step S2720).
When the processing of step S2720 is completed, the voice dialogue agent 2200 ends the disconnection response processing.
Returning to FIG. 25 again, the explanation of the second instruction execution process is continued. When the disconnection response process is completed or the process of step S2560 is completed, the voice dialogue agent 2200 ends the second instruction execution process. To do.
Returning to FIG. 24 again, the explanation of the second agent processing is continued.
When the second instruction execution process is completed, the voice dialogue agent 2200 returns to the process of step S2400 and continues the process of step S2400 and thereafter.
<Third agent processing> The third agent processing is the processing performed by the voice dialogue agent 2200, which (1) starts communication with the device in response to a request from another voice input agent, and (2) from the device. Receives the transmitted input voice data, (3) performs voice recognition processing on the received input voice data, generates input text and returns it to the device, and (4) commands based on the generated input text. Generate and execute a group, (5) generate a response text based on the execution result, (6) convert the generated response text to response voice data, and (7) device the response text and response voice data. It is a process to reply to.
FIG. 28 is a flowchart of the third agent processing.
In the figure, the processes of steps S2800 to S2810 and the processes of steps S2820 to S2860 are the processes of steps S900 to S910 of the first instruction execution process (see FIG. 9) in the first embodiment, respectively. The process is the same as the process of steps S920 to S960. Therefore, these processes have already been described.
The third agent processing is started when the voice dialogue agent 2200 is started.
When the third agent processing is started, the voice dialogue agent 2200 waits until the connection command transmitted from the other voice dialogue agent is received by the communication unit 420 (step S2811: No is repeated). Then, when the connection command is received (step S2811: Yes), the control unit 2210 controls the communication unit 420 and performs a connection process for starting communication with the device 1700 to be communicated in the connection command. Execute.
Here, this connection process includes a process of changing the destination of the input voice data transmitted by the device 1700 from the voice dialogue agent 2200 that sent the connection command to the voice dialogue agent 2200 that received the connection command. There is.
When the connection process is executed, the control unit 2210 controls the communication unit 420 to generate a connection response, which is a signal indicating that communication with the device 1700 has started, and generates the generated connection response. , Send to the voice dialogue agent that is the source of the connection command (step S2813).
After that, the control unit 2210 waits until the input voice data transmitted from the device 1700 is received by the communication unit 420 (step S2814: No is repeated). Then, when the input voice data is received (step S2814: Yes), the processes of steps S2800 to S2810 are executed.
When the processing of step S2810 is completed, the control unit 2210 checks whether or not the input text converted by the voice recognition processing unit 430 is intended to end the communication with the self-voice dialogue agent (step S2815). ..
In the process of step S2815, if it is not intended to end the communication with the self-voice dialogue agent (step S2815: No), the voice dialogue agent 2200 proceeds to the process of step S2820 and proceeds to the process of steps S2820 to S2860. I do. Then, when the process of step S2860 is completed, the voice dialogue agent 2200 returns to the process of step S2814 and performs the process of step S2814 and the like.
In the process of step S2815, when the communication with the self-voice dialogue agent is to be terminated (step S2815: Yes), the control unit 2210 controls the communication unit 420 to communicate with the device 1700. Executes the disconnection process.
Here, this disconnection process includes a process of changing the destination of the input voice data transmitted by the device 1700 from the voice dialogue agent 2200 that received the connection command to the voice dialogue agent 2200 that sent the connection command. There is.
When the disconnection process is executed, the control unit 2210 controls the communication unit 420 to generate a disconnection response, which is a predetermined signal indicating that the communication with the device 1700 is terminated, and the generated disconnection. The response is sent to the voice dialogue agent that is the source of the connection instruction (step S2890).
When the process of step S2890 is completed, the voice dialogue agent 2200 returns to the process of step S2811 and performs the process of step S2811 and the like.
A specific example of the operation performed by the third modified voice dialogue system having the above configuration will be described below with reference to the drawings.
<Specific example> In Fig. 29, a user using the third modified voice dialogue system starts a voice dialogue with a house agent, which is one of the voice dialogue agents 2200, using the device 1700, and then. , The connection command generated by the home agent initiates communication with the car agent, which is the other one of the voice dialogue agents 2200, and illustrates how the user interacts with the car agent. It is a processing procedure diagram which shows.
Here, it is assumed that the specific voice dialogue agent server of the device 1700 used by the user is the voice dialogue agent server 110 that realizes the home agent, and the dialogue end voice is the voice "Ends."
In the figure, the processes of steps S2900 to S2905 are the same as the processes of steps S2100 to S2105 of the specific example (see FIG. 21) in the third embodiment, respectively. Therefore, these processes have already been described.
When the processing in step S2905 is completed, the device 1700 performs the second audio processing (corresponding to step S2906 and step S1930 in FIG. 19).
In this second voice input process, if the user inputs the voice "Connect to the car agent.", The device 1700 inputs the input voice data "Connect to the car agent." To the house agent. (Corresponds to step S2040 in Figure 20).
Then, the house agent receives the input voice data (corresponds to step S2400: Yes in FIG. 24) and executes the second instruction execution process (corresponds to step S2410 in FIG. 24).
In the second instruction execution process, since the input text is a communication request with the car agent (corresponding to step S2515: Yes in FIG. 25), the house agent sends a connection command to the car agent (Fig. 25). Corresponds to step S2570).
The vehicle agent then receives the connection command (corresponding to step S2811: Yes in FIG. 28), initiates communication with the device 1700 (corresponds to step S2812 in FIG. 28), and responds to the home agent with a connection. (Corresponding to step S2990 and step S2813 in FIG. 28).
The home agent then receives the connection response (corresponding to step S2575: Yes in FIG. 25) and executes the first connection response processing (corresponding to step S2965 and step S2580 in FIG. 25).
Here, if the voice dialogue agent 2200 generates the response voice data "Connected to the car agent" in this first connection response processing, the voice dialogue agent 2200 tells the device 1700 "to the car agent". Send the response voice data "Connected." (Corresponding to step S2620 in Fig. 26).
Then, the device 1700 receives the response voice data (corresponding to step S1960: Yes in FIG. 19) and outputs the voice "Connected to the car agent" (step S2907, step S1980 in FIG. 19). Correspondence).
Then, since the voice "Connected to the car agent" is not the dialogue end voice (corresponding to step S1985: No in FIG. 19), the device 1700 performs the second voice processing (step S2910, step 19 in FIG. 19). Compatible with S1930).
In this second voice input process, if the user inputs a voice saying "What is the temperature inside the car?", The device 1700 transmits the input voice data "What is the temperature inside the car?" To the car agent ( Corresponds to step S2040 in Figure 20).
Then, the car agent receives the input voice data (corresponds to step S2814: Yes in FIG. 28), and the voice data does not request the end of communication (corresponds to step S2815: No in FIG. 28). , Generates and executes a group of instructions corresponding to the input voice data (corresponding to step S2994 and step S2830 in FIG. 28).
Here, in the execution of this instruction group, if the car agent generates the response voice data "The temperature inside the car is 38 degrees. Is there anything else?", The car agent tells the device 1700 that " The temperature inside the car is 38 degrees. Is there anything else? "Sends the response voice data (corresponding to step S2860 in Figure 28).
Then, the device 1700 receives the response voice data (corresponding to step S1960: Yes in Fig. 19) and outputs the voice "The temperature inside the car is 38 degrees. Is there anything else?" (Step). Corresponds to S2915, step S1980 in Figure 19).
And since the voice "The temperature inside the car is 38 degrees. Is there anything else?" Is not the dialogue end voice (corresponding to step S1985: No in Fig. 19), the device 1700 performs the second voice processing. (Corresponds to step S2930 and step S1930 in FIG. 19).
In this second voice input process, if the user inputs the voice "No. Thank you.", The device 1700 transmits the input voice data "No. Thank you." To the car agent ( Corresponds to step S2040 in Figure 20).
Then, the car agent receives the input voice data (corresponds to step S2814: Yes in FIG. 28), and the voice data requests the end of communication (corresponds to step S2815: Yes in FIG. 28). , Terminate communication with device 1700 (corresponding to step S2870 in FIG. 28) and send a disconnect response to the home agent (corresponding to step S2998 and step S2890 in FIG. 28).
Then, the house agent receives the disconnection response (corresponds to step S2585: Yes in FIG. 25) and executes the disconnection response process (corresponds to the process in step S2970 and step S2890 in FIG. 25).
Here, if the voice dialogue agent 2200 generates the response voice data "The connection with the car agent has been disconnected. Is there anything else?" In this disconnection process, the voice dialogue agent 2200 is a device. Send the response voice data to the 1700, "I have disconnected from the car agent. Is there anything else?" (Corresponding to step S2720 in Figure 27).
Then, the device 1700 receives the response voice data (corresponding to step S1960: Yes in Fig. 19) and outputs the voice "Disconnected from the car agent. Is there anything else?" (Corresponds to step S2935, step S1980 in Figure 19).
And since the voice "I disconnected from the car agent. Is there anything else?" Is not the dialogue end voice (corresponding to step S1985: No in Fig. 19), the device 1700 performs the second voice processing. (Corresponding to step S2950 and step S1930 in FIG. 19).
In this second voice input process, if the user inputs the voice "No. Thank you.", The device 1700 transmits the input voice data "No. Thank you." To the home agent ( Corresponds to step S2040 in Figure 20).
Then, the house agent receives the input voice data (corresponds to step S2800: Yes in FIG. 24) and executes the second instruction execution process (corresponds to step S2975 and step S2410 in FIG. 24).
Here, if the house agent generates the response voice data "terminates" in this second instruction execution process, the house agent outputs the response voice data "terminates" to the device 1700. Send (corresponds to step S2560 in Figure 25).
Then, the device 1700 receives the response voice data (corresponds to step S1960: Yes in FIG. 19) and outputs the voice "End" (corresponds to step S2955 and step S1980 in FIG. 19).
Since the voice "End" is the dialogue end voice (corresponding to step S1985: Yes in Fig. 19), the state is changed to the voice input acceptable state (corresponding to step S1990 in Fig. 19), and the device 1700. Ends the third device processing.
<Discussion> According to the third modified voice dialogue system having the above configuration, the user who uses the third modified voice dialogue system is more likely to use the voice dialogue agent B than the voice dialogue agent A when communicating with the voice dialogue agent A. When it is considered desirable to have the voice dialogue agent perform the processing, the voice dialogue agent to communicate and process is changed from the voice dialogue agent A to the voice dialogue agent B, and the desired processing is executed by the voice dialogue agent B. Will be able to.
Further, in this case, the voice dialogue agent A transfers the input voice to the voice dialogue agent B as it is, so that the voice dialogue agent B may perform voice recognition processing on the input voice itself. it can. Therefore, the user can receive a more appropriate service from the voice dialogue agent B. <Embodiment 5> <Overview> Hereinafter, as one aspect of the voice dialogue method according to the present invention and one aspect of the device according to the present invention, a part of the third modified voice dialogue system according to the fourth embodiment is modified. The fourth modified voice dialogue system will be described.
In the third modified voice dialogue system according to the fourth embodiment, when a device starts communication with the voice dialogue agent A, a user who uses the third modified voice dialogue system uses the device to perform another device. This was an example of a configuration in which the communication partner of the device is changed from the voice dialogue agent A to the voice dialogue agent B after the voice input to the effect that the user wants to communicate with the voice agent B is performed. ..
On the other hand, in the fourth modified voice dialogue system according to the fifth embodiment, when the device starts communication with the voice dialogue agent A and a predetermined condition is satisfied in the communication content, the voice dialogue agent A determines that another voice dialogue agent B should be the communication partner rather than the self-voice dialogue agent A as the voice dialogue agent used by the user. This is an example of a configuration in which the voice dialogue agent A is changed to the voice dialogue agent B.
Hereinafter, the details of the fourth modified voice dialogue system will be described with reference to the drawings, focusing on the differences from the third modified voice dialogue system in the fourth embodiment.
<Structure> The fourth modified voice dialogue system is modified so that the voice dialogue agent 2200 is changed to the voice dialogue agent 3000 from the third modified voice dialogue system in the fourth embodiment.
The voice dialogue agent 3000 is realized by the voice dialogue agent server 110, similarly to the voice dialogue agent 2200 in the fourth embodiment.
A part of the software for realizing the voice dialogue agent 3000 executed by the voice dialogue agent server 110 is changed from the software for realizing the voice dialogue agent 2200 in the third embodiment. As a result, the voice dialogue agent 3000 is modified from the voice dialogue agent 2200 in the fourth embodiment so that a part of its function is changed.
FIG. 30 is a block diagram showing a functional configuration of the voice dialogue agent 3000.
As shown in the figure, in the voice dialogue agent 3000, the target agent DB storage unit 2220 is deleted from the voice dialogue agent 2200 (see FIG. 22) in the fourth embodiment, and the available service DB storage unit 3020 is added. , The control unit 2210 is modified to be changed to the control unit 3010.
As an example, the available service DB storage unit 3020 has a function realized by a memory and a processor that executes a program, connected to the control unit 3010, and stores the available service DB 3100.
FIG. 31 is a data configuration diagram of the available service DB 3100 stored in the available service DB storage unit 3020.
As shown in the figure, the available service DB3100 is configured by associating the keyword 3110, the target agent 3120, the processing content 3130, the IP address 3140, and the availability 3150.
Keyword 3110 is a character string that is assumed to be included in the input text converted by the voice recognition processing unit 430.
The target agent 3120 is information that identifies an additional voice dialogue agent to be communicated with by the device 1700.
In this example, as in the fourth embodiment, the additional voice target agent specified by the target agent 2320 is an example of a configuration including a car agent, a retail store agent, and a house agent.
The processing content 3130 is determined by the additional voice dialogue agent specified by the associated target agent 3120 when the input text converted by the speech recognition processing unit 430 contains the associated keyword 3110. Information indicating the content of the process to be executed.
The IP address 3140 is the IP address in the network 120 for the voice dialogue agent server 110 that implements the additional voice dialogue agent identified by the associated target agent 3120.
The availability 3150 is information indicating whether or not the processing of the content indicated by the associated processing content 3130 can be processed by the self-voice dialogue agent.
Returning to FIG. 30, the explanation of the voice dialogue agent 3000 is continued.
The control unit 3010 is transformed from the control unit 2210 in the fourth embodiment so that the second agent processing realization function among the functions of the control unit 2210 is changed to the fourth agent processing realization function described below. There is.
The fourth agent processing realization function is that the control unit 3010 controls the communication unit 420, the voice recognition processing unit 430, the voice synthesis processing unit 450, and the instruction generation unit 460 to the voice dialogue agent 3000. By executing the 4th agent processing, which is a characteristic operation, the voice dialogue agent 2200 receives (1) input voice data transmitted from the device, and (2) voice recognition for the received input voice data. Performs processing, generates input text and returns it to the device, and (3) When the generated input text contains a predetermined keyword, the device and the target agent associated with that keyword Realize communication between, (4) otherwise generate and execute instructions based on the generated input text, (5) generate response text based on the execution result, (6) generate It is a function that converts the response text into response voice data and (7) realizes a series of processes of returning the response text and the response voice data to the device.
The fourth agent process will be described in detail later in the item <4th agent process> using a flowchart.
The operation performed by the fourth modified voice dialogue system configured as described above will be described below with reference to the drawings.
<Operation> As a characteristic operation of the fourth modified voice dialogue system, in addition to the second device processing and the third agent processing in the fourth embodiment, the second agent processing in the third embodiment thereof Performs the 4th agent processing in which a part of the processing is changed.
Hereinafter, this fourth agent processing will be described focusing on the changes from the second agent processing.
<Fourth agent processing> The fourth agent processing is the processing performed by the voice dialogue agent 3000, which (1) receives the input voice data transmitted from the device, and (2) voices the received input voice data. Performs recognition processing, generates input text and returns it to the device, and (3) When the generated input text contains a predetermined keyword, the device and the target agent associated with that keyword Realize communication between, (4) otherwise generate and execute instructions based on the generated input text, (5) generate response text based on the execution result, (6) generate This is a process of converting the response text to the response voice data and (7) returning the response text and the response voice data to the device.
FIG. 32 is a flowchart of the fourth agent processing.
The fourth agent process is started when the voice dialogue agent 3000 is started.
When the fourth agent process is started, the voice dialogue agent 3000 waits until the input voice data transmitted from the device 1700 is received by the communication unit 420 (step S3200: No is repeated). Then, when the input voice data is received (step S3200: Yes), the voice dialogue agent 3000 executes the third instruction execution process (step S3210).
FIG. 33 is a flowchart of the third instruction execution process.
In the figure, the processes of steps S3300 to S3310, the processes of steps S3320 to S3360, the processes of steps S3365 to S3375, and the processes of steps S3385 to S3390 are the processes of step S2500 in the fourth embodiment, respectively. ~ The processing of step S2510, the processing of steps S2520 to S2560, the processing of steps S2565 to S2575, and the processing of steps S2585 to S2590 are the same. Therefore, these processes have already been described.
When the processing of step S3310 is completed, the control unit 3010 refers to the available service DB 3100 stored in the available service DB storage unit 3020 (step S3312), performs the processing corresponding to the input text, and performs another voice dialogue. Determine if the agent should run it (step S3315). That is, when the input text data includes the keyword 3110 and the target agent 3120 associated with the keyword 3110 is not a self-speech dialogue agent, it is specified by another voice dialogue agent (target agent 3120). It is determined that it should be executed by the additional voice dialogue agent), and otherwise it is determined that it should not be executed by another voice dialogue agent.
If it is determined in the process of step S3315 that it should not be executed by another voice dialogue agent (step S3315: No), the voice dialogue agent 3000 proceeds to the process of step S3320 and performs the processes of steps S3320 to S3360. Do.
If it is determined in the process of step S3315 that it should be executed by another voice dialogue agent (step S3315: Yes), the voice dialogue agent 3000 proceeds to the process of step S3365 and proceeds to the process of steps S3365 to S3375. I do.
In the process of step S3375, when the connection response returned from the additional voice dialogue agent is received by the communication unit 420 (step S3375: Yes), the voice dialogue agent 3000 executes the second connection response process (step S3380). ).
FIG. 34 is a flowchart of the second connection response process.
When the second connection response process is started, the control unit 3010 controls the communication unit 420 to input the input voice data received in the process of step S3200: Yes to the additional voice specified in the process of step S3315: Yes. Transfer to the dialogue agent (step S3400).
When the processing of step S3400 is completed, the voice dialogue agent 3000 ends the second connection response processing.
Returning to FIG. 33 again, the explanation of the second instruction execution process is continued.
When the second connection response process is completed, the voice dialogue agent 3000 proceeds to the process of step S3385 and performs the processes of steps S3385 to S3390.
When the process of step S3390 is completed or the process of step S3360 is completed, the voice dialogue agent 3000 ends the third instruction execution process.
Returning to FIG. 32 again, the explanation of the fourth agent processing is continued.
When the third instruction execution process is completed, the voice dialogue agent 3000 returns to the process of step S3200 and continues the process of step S3200 and thereafter.
A specific example of the operation performed by the fourth modified voice dialogue system having the above configuration will be described below with reference to the drawings.
<Specific example> In Fig. 35, a user using the fourth modified voice dialogue system starts a voice dialogue with a house agent, which is one of the voice dialogue agents 3000, using the device 1700, and then. , The connection command generated by the home agent initiates communication with the car agent, which is the other one of the voice dialogue agents 3000, and illustrates how the user interacts with the car agent. It is a processing procedure diagram which shows.
Here, it is assumed that the specific voice dialogue agent server of the device 1700 used by the user is the voice dialogue agent server 110 that realizes the home agent, and the dialogue end voice is the voice "Ends."
In the figure, the processes of steps S3500 to S3505 are the same as the processes of steps S2900 to S2905 of the specific example (see FIG. 29) in the fourth embodiment, respectively. Therefore, these processes have already been described.
When the processing of step S3505 is completed, the device 1700 performs the second audio processing (corresponding to step S3506 and step S1930 of FIG. 19).
In this second voice input process, if the user inputs a voice saying "What is the temperature inside the car?", The device 1700 transmits the input voice data "What is the temperature inside the car?" To the house agent ( Corresponds to step S2040 in Figure 20).
Then, the house agent receives the input voice data (corresponds to step S3200: Yes in FIG. 32) and executes the third instruction execution process (corresponds to step S3210 in FIG. 32).
In the third instruction execution process, the input text contains the keywords "temperature" and "inside the car", and the target agent 3120 is not a house agent (corresponding to step S3315: No in FIG. 33). Send a connection command to the vehicle agent (corresponding to step S3370 in Figure 33).
The vehicle agent then receives the connection command (corresponding to step S2811: Yes in FIG. 28), initiates communication with the device 1700 (corresponds to step S2812 in FIG. 28), and responds to the home agent with a connection. (Corresponding to step S3590 and step S2813 in FIG. 28).
The home agent then receives the connection response (corresponding to step S3375: Yes in FIG. 33) and executes the second connection response processing (corresponding to step S3380 in FIG. 33).
In the second connection response process, the house agent transfers the input voice data "What is the temperature inside the car?" To the car agent (corresponding to step S3400 in FIG. 34).
Then, the car agent receives the input voice data (corresponds to step S2814: Yes in FIG. 28), and the voice data does not request the end of communication (corresponds to step S2815: No in FIG. 28). , Generates and executes a group of instructions corresponding to the input voice data (corresponding to step S3594 and step S2830 in FIG. 28).
Here, in the execution of this instruction group, if the car agent generates the response voice data "The temperature inside the car is 38 degrees. Is there anything else?", The car agent tells the device 1700 that " The temperature inside the car is 38 degrees. Is there anything else? "Sends the response voice data (corresponding to step S2860 in Figure 28).
Then, the device 1700 receives the response voice data (corresponding to step S1960: Yes in Fig. 19) and outputs the voice "The temperature inside the car is 38 degrees. Is there anything else?" (Step). S3507, corresponding to step S1980 in Figure 19).
And since the voice "The temperature inside the car is 38 degrees. Is there anything else?" Is not the dialogue end voice (corresponding to step S1985: No in Fig. 19), the device 1700 performs the second voice processing. (Corresponds to step S3510 and step S1930 in FIG. 19).
In this second voice input process, if the user inputs the voice "Turn on the air conditioner at 25 degrees", the device 1700 will input the input voice data "Turn on the air conditioner at 25 degrees." To the car agent (corresponding to step S2040 in Figure 20).
Then, the car agent receives the input voice data (corresponds to step S2814: Yes in FIG. 28), and the voice data does not request the end of communication (corresponds to step S2815: No in FIG. 28). , Generates and executes a group of instructions corresponding to the input voice data (corresponding to step S3594 and step S2830 in FIG. 28).
Now, if the car agent generated the response voice data "I turned on the air conditioner at 25 degrees. Is there anything else?" In the execution of this instruction group, the car agent sent the device 1700. , Send the response voice data "I turned on the air conditioner at 25 degrees. Is there anything else?" (Corresponding to step S2860 in Figure 28).
Then, the device 1700 receives the response voice data (corresponding to step S1960: Yes in Fig. 19) and outputs the voice "The air conditioner was turned on at 25 degrees. Is there anything else?" (Corresponds to step S3525 and step S1980 in FIG. 19).
And since the voice "I turned on the air conditioner at 25 degrees. Is there anything else?" Is not the dialogue end voice (corresponding to step S1985: No in Fig. 19), the device 1700 is the second voice processing. (Corresponding to step S3530 and step S1930 in FIG. 19).
In this second voice input process, if the user inputs the voice "No. Thank you.", The device 1700 transmits the input voice data "No. Thank you." To the car agent ( Corresponds to step S2040 in Figure 20).
Then, the car agent receives the input voice data (corresponds to step S2814: Yes in FIG. 28), and the voice data requests the end of communication (corresponds to step S2815: Yes in FIG. 28). , Terminate communication with device 1700 (corresponding to step S2870 in FIG. 28) and send a disconnect response to the home agent (corresponding to step S3598 in FIG. 28, step S2890 in FIG. 28).
Then, the house agent receives the disconnection response (corresponds to step S2585: Yes in FIG. 25) and executes the disconnection response process (corresponds to the process in step S2970 and step S2890 in FIG. 25).
Here, if the voice dialogue agent 2200 generates the response voice data "terminates" in this disconnection process, the voice dialogue agent 2200 tells the device 1700 the response voice data "terminates." (Corresponding to step S2720 in Figure 27).
Then, the device 1700 receives the response voice data (corresponds to step S1960: Yes in FIG. 19) and outputs the voice "End" (corresponds to step S3555 and step S1980 in FIG. 19).
Since the voice "End" is the dialogue end voice (corresponding to step S1985: Yes in Fig. 19), the state is changed to the voice input acceptable state (corresponding to step S1990 in Fig. 19), and the device 1700. Ends the 4th device processing.
<Discussion> According to the fourth modified voice dialogue system having the above configuration, when the user using the third modified voice dialogue system communicates with the voice dialogue agent A, the voice dialogue agent A is more than the voice dialogue agent A. When it is determined that the voice dialogue agent B should be the communication partner, the voice dialogue agent of the user's communication partner can be changed from the voice dialogue agent A to the voice dialogue agent B.
As a result, the user can receive the service from a more appropriate voice dialogue agent without knowing what kind of service is provided by each voice dialogue agent.
Further, in this case, the voice dialogue agent A transfers the input voice to the voice dialogue agent B as it is, so that the voice dialogue agent B performs the voice recognition process itself for the input voice itself. Can be done. Therefore, the user can receive more appropriate service provision.
<Embodiment 6> Here, an example of an operation mode in which the voice dialogue system 100 in the first embodiment is operated will be illustrated. Of course, the voice dialogue system 100 in the first embodiment may be operated in an operation mode other than those illustrated here.
FIG. 36 (A) is a schematic diagram schematically showing an operation mode in which the voice dialogue system 100 according to the first embodiment is operated.
In FIG. 36 (A), the group 3600 is, for example, a company, a group, a household, etc., regardless of its size. Group 3600 has a plurality of devices 3601 (device A, device B, etc.) and a home gateway 3602. Multiple devices 3601 include devices that can connect to the Internet (for example, smartphones, PCs, TVs, etc.) and devices that cannot connect to the Internet by themselves (for example, lighting, washing machines, refrigerators, etc.). To do. Even if the device itself cannot connect to the Internet, there may be a device that can connect to the Internet via the home gateway 3602. In addition, there are users 10 who use a plurality of devices 3601 in the group 3600. For example, each of the devices shown in Group 3600 corresponds to the device 140 in the first embodiment.
The data center operating company 3610 has a cloud server 3611. The cloud server 3611 is a virtualization server that cooperates with various devices via the Internet. It mainly manages huge data (big data) that is difficult to handle with ordinary database management tools. The data center operating company 3610 manages data, manages the cloud server 3611, and operates the data center that manages them. Details of the services performed by the data center operating company 3610 will be described later. Here, the data center operating company 3610 is not limited to a company that only manages data and operates the cloud server 3611. For example, if a device manufacturer that develops and manufactures one of multiple devices 3601 also manages data and cloud server 3611, the device manufacturer corresponds to the data center operating company 3610. (See Figure 36 (B)). In addition, the data center operating company 3610 is not limited to one company. For example, if the equipment manufacturer and other management companies jointly or share data management and operation of the cloud server 3611, both or one of them shall correspond to the data center operating company 3610 (Fig. 36 (Fig. 36). See C)). For example, the data center operating company 3610 provides a voice dialogue agent 400 (also referred to as a "first voice dialogue agent") associated with the device 140.
Service provider 3620 owns server 3621. The server 3621 referred to here includes, for example, a memory in a personal computer regardless of its scale. Also, the service provider may not own the server 3621. For example, service provider 3620 provides another voice dialogue agent 400 (also referred to as a second voice dialogue agent) connected by the first voice dialogue agent.
Next, the flow of information in the above operation mode will be described.
First, device A or device B of group 3600 transmits each log information to the cloud server 3611 of the data center 3610. The cloud server 3611 collects log information of device A or device B (Fig. 36 (a)). Here, the log information is information indicating, for example, an operating status, an operating date and time, etc. of a plurality of devices 3601. For example, TV viewing history, recorder recording reservation information, washing machine operation date / time / amount of laundry, refrigerator opening / closing date / time / opening / closing frequency, etc. Say information about. Log information may be provided directly to the cloud server 3611 from multiple devices 3601 itself via the Internet. In addition, log information may be temporarily accumulated in the home gateway 3602 from a plurality of devices 3601 and provided to the cloud server 3611 from the home gateway 3602.
Next, the cloud server 3611 of the data center operating company 3610 provides the accumulated log information to the service provider 3620 in a fixed unit. Here, it may be a unit that can organize the information collected by the data center operating company 3610 and provide it to the service provider 3620, or it may be a unit requested by the service provider 3620. Although it is described as a fixed unit, it does not have to be constant, and the amount of information to be provided may change depending on the situation. The log information is stored in the server 3621 owned by the service provider 3620 as needed (Fig. 36 (b)). Then, the service provider 3620 organizes the log information into information suitable for the service provided to the user and provides the log information to the user. The user to be provided may be a user 10 who uses a plurality of devices 3601 or an external user 20. As a method of providing a service to a user, for example, the service provider 3620 may directly provide the service to the user (Fig. 36 (e)). Further, the method of providing the service to the user may be provided to the user via the cloud server 3611 of the data center operating company 3610 again (FIGS. 36 (c) and 36 (d)). In addition, the cloud server 3611 of the data center operating company 3610 may organize the log information into information suitable for the service provided to the user and provide it to the service provider 3620.
The user 10 and the user 20 may be different or the same.
The following are some examples of the types of services that can be provided in the above operation mode.
<Service type 1: In-house data center type> Fig. 37 is a schematic diagram schematically showing service type 1 (in-house data center type).
This type is a type in which the service provider 3620 acquires information from the group 3600 and provides the service to the user. In this type, the service provider 3620 has the function of a data center operating company. That is, the service provider 3620 owns a cloud server 3611 that manages big data. Therefore, there is no data center operating company.
In this type, the service provider 3620 operates and manages the data center (cloud server 3611) (3703). In addition, the service provider 3620 manages the OS (3702) and the application (3701). Service provider 3620 provides services using the managed OS (3702) and application (3701) (3704).
<Service type 2: IaaS usage type> FIG. 38 is a schematic diagram schematically showing service type 2 (IaaS usage type). Here, IaaS is an abbreviation for Infrastructure as a Service, which is a cloud service provision model that provides the infrastructure itself for constructing and operating a computer system as a service via the Internet.
In this type, the data center operating company 3610 operates and manages the data center (cloud server 3611) (3703). In addition, the service provider 3620 manages the OS (3702) and the application (3701). Service provider 3620 provides services using the managed OS (3702) and application (3701) (3704).
<Service type 3: PaaS usage type> Fig. 39 is a schematic diagram schematically showing service type 3 (PaaS usage type). Here, PaaS is an abbreviation for Platform as a Service, and is a cloud service provision model that provides a platform that is the basis for building and operating software as a service via the Internet.
In this type, the data center operating company 3610 manages the OS (3702) and operates and manages the data center (cloud server 3611) (3703). Service provider 3620 also manages the application (3701). The service provider 3620 provides services using the OS (3702) managed by the data center operating company 3610 and the application (3701) managed by the service provider 3620 (3704).
<Service type 4: SaaS usage type> Fig. 40 is a schematic diagram schematically showing service type 4 (SaaS usage type). Here, SaaS is an abbreviation for Software as a Service. For example, a function that allows companies / individuals (users) who do not have a data center (cloud server) to use applications provided by a platform provider who owns a data center (cloud server) via a network such as the Internet. It is a cloud service provision model that it has.
In this type, the data center operating company 3610 manages the application (3701), manages the OS (3702), and operates and manages the data center (cloud server 3611) (3703). In addition, the service provider 3620 provides the service using the OS (3702) and the application (3701) managed by the data center operating company 3610 (3704).
In any of the above types, it is assumed that the service provider 3620 has performed the service provision act. Further, for example, the service provider 3620 or the data center operating company 3610 may develop an OS, an application, a database of big data, etc. by itself, or may outsource to a third party. <Supplement> As described above, as one aspect of the voice dialogue method according to the present invention and one aspect of the device according to the present invention, the five voice dialogue systems are illustrated in the first to fifth embodiments. Although the operation mode of the voice dialogue system has been described as an example in the sixth embodiment, it can be modified as follows, and the voice dialogue system illustrated in the first to sixth embodiments or its operation thereof. Of course, it is not limited to the voice dialogue method and the device as used in the form.
(1) In the first embodiment, the voice dialogue system 100 has been described as being composed of the voice dialogue agent server 110, the network 120, the gateway 130, and the device 140, as shown in FIG. On the other hand, as another example, the voice dialogue system stores the target agent DB2300 in addition to the voice dialogue agent server 110, the network 120, the gateway 130, and the device 140, and corresponds between the voice dialogue agents. It may be composed of an intermediary server 4150 having a function of attaching, switching a connection, and the like.
FIG. 41 is a system configuration diagram showing the configuration of the voice dialogue system 4100 including the intermediary server 4150.
FIG. 42 is a block diagram showing a functional configuration of the intermediary server 4150.
As shown in the figure, the intermediary server 4150 is composed of a communication unit 4220, a control unit 4210, and a target agent DB storage unit 4230.
Here, the target agent DB storage unit 4230 has a function of storing the target agent DB 2300, similarly to the target agent DB storage unit 2220 in the fourth embodiment.
Further, as another example, the voice dialogue system is configured to include the mediation server 4350 having a function of storing the available service DB3100, associating between voice dialogue agents, switching connections, etc., instead of the mediation server 4150. May be done.
FIG. 43 is a block diagram showing a functional configuration of the intermediary server 4350.
As shown in the figure, the intermediary server 4350 is composed of a communication unit 4320, a control unit 4310, and an available service DB storage unit 4330.
Here, the available service DB storage unit 4330 has a function of storing the available service DB 3100, similarly to the available service DB 3020 in the fifth embodiment.
(2) In the first embodiment, the image shown in FIG. 12 has been illustrated and described as an image displayed on the display unit 270 of the device 140.
On the other hand, as other image examples, FIGS. 44 (a) to 44 (d), 45 (a), 45 (b), and the like can be considered.
In the examples of FIGS. 12, 44 (a) to 44 (d), and 45 (b), "you", "car agent", "house agent", etc. are displayed at the beginning of each response text. A character string that identifies the subject of the utterance is displayed. Further, in FIG. 45 (a), an icon (image) for identifying the subject of the utterance is displayed.
In the examples of FIGS. 44 (a) and 44 (b), dialogues such as "interacting with a house agent" and "interacting with a car agent" are performed at the top of the screen so that the currently interacting voice dialogue agent can be seen. The character string that identifies the voice dialogue agent is displayed.
In the example of FIG. 44 (d), "interacting with home agent" and "with car agent" are displayed at the top of the corresponding response text so that the voice dialogue agent currently or in the past is interacting with (was). A string is displayed that identifies the voice dialogue agent that is (was) interacting with, such as "in dialogue." Further, in FIG. 45 (b), an icon (image) for identifying the voice dialogue agent having (was) interacting with is displayed.
In addition, these display examples are merely examples, and may indicate which voice dialogue agent is interacting with by color, and which voice dialogue agent is interacting with in the shape of the screen or in a part of the screen. May indicate if you are interacting with. Furthermore, the background color, wallpaper, etc. may be changed for each uttering subject. In this way, it suffices if the voice dialogue agent having a dialogue or the subject who speaks is displayed so as to be identifiable by the user.
(3) In the first embodiment and the above-described modified example, an example in which the talking voice dialogue agent or the uttering subject is visually identifiable is described. However, if it can be identified by the user, it is not necessarily limited to the example in which the spoken dialogue agent or the uttering subject is visually identifiable.
For example, you may output a voice saying "I am interacting with a house agent" so that you can tell which voice dialogue agent you are interacting with by voice, or you can tell which voice dialogue agent you are interacting with with a sound effect. It may be possible to understand which voice dialogue agent is interacting with by voice color, speaking speed, volume, or the like.
(4) In the first embodiment, the state management performed by the control unit 210 has been described as being managed in the manner shown in the state transition diagram of FIG. Further, in the second embodiment, the state management performed by the control unit 1310 has been described as being managed in the manner shown in the state transition diagram of FIG. Then, in the third embodiment, the state management performed by the control unit 1710 has been described as being managed in the manner shown in the state transition diagram of FIG.
On the other hand, the state management performed by the control unit is not limited to the management according to the above-mentioned aspect, and the state management by another aspect can be considered. FIGS. 46 to 50 show state transition diagrams showing examples of state management by other modes performed by the control unit.
For example, when the state is managed in the manner shown in the state transition diagram of FIG. 48, the voice output unit 260 can perform the voice output unit 260 even if a predetermined time has not elapsed since the state transitioned to the voice input acceptable state. When the voice based on the response text transmitted from the voice dialogue agent 110 is output and the voice is the dialogue end voice, the state is changed to the voice input unacceptable state. Therefore, the voice input unit 220 is in a state where voice input can be accepted when the voice signal generated by the voice dialogue agent 110 received by the communication unit 250 indicates that a new voice input is not required. Even if the predetermined time has not passed since the transition to, the voice input cannot be accepted.
(5) In the first embodiment, as an example, the display unit 270 is realized by a touch panel, a touch panel controller, and a processor that executes a program, and displays that the display unit 270 is in a state where voice input can be accepted. It was explained that the configuration is performed by blinking the lower right area 1120 (see FIGS. 11 (a), 11 (c), 12, etc.). However, as long as the user can be made aware that the voice input can be accepted, the example is not necessarily limited to the above configuration. As another example, the display unit 270 is realized by an LED (Light Emitting Diode) and a processor that executes a program, and is configured to display that it is in a state where voice input can be accepted by turning on this LED. An example is possible. However, in the example of this configuration, since the display unit 270 does not have a means for displaying the character string, the response text received by the communication unit 250 is not displayed.
(6) In the first embodiment, when the communication unit 250 does not specify a specific voice dialogue agent server 110 as the communication destination, the communication unit 250 refers to the IP address stored in the address storage unit 240 and refers to the specific voice. It was explained that the configuration communicates with the dialogue agent server. In contrast, as another example, the address storage unit 240, a particular voice interaction agent not store the IP address of the ActiveUpdate server, spoken dialogue agent specified by the voice interaction agent server, or a user specified by the user An example of a configuration for communicating with a voice dialogue agent server that realizes the above can be considered.
(7) In the first embodiment, it has been described that each of the devices 140 communicates with the voice dialogue agent 110 via the gateway 130 and the network 120.
On the other hand, even if the device 140 has a function of directly connecting to the network 120 without going through the gateway 130 and communicating with the voice dialogue agent without going through the gateway 130. I do not care. If all the devices 140 are directly connected to the network 120 without going through the gateway, the gateway 130 becomes unnecessary.
(8) A part or all of each element constituting the above-described embodiment and the above-described modification may be composed of one system LSI. A system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip. Specifically, it is a computer system including a microprocessor, ROM, RAM, and the like. Computer programs are stored in ROM, RAM, etc., and the microprocessor operates according to these computer programs, so that the system LSI realizes its functions.
(9) A part or all of the elements constituting the above-described embodiment and the above-described modification may be configured to include an IC (Integrated Circuit) card or module that can be attached to and detached from the device. These IC cards or modules are computer systems including a microprocessor, ROM, RAM, and the like. These IC cards or modules may include the above-mentioned ultra-multifunctional LSI. Computer programs are stored in ROM, RAM, etc., and the microprocessor operates according to these computer programs, so that these IC cards or modules realize their functions. These IC cards or modules may have tamper resistance.
(10) A computer-readable recording medium such as a flexible disk, a hard disk, a CD-ROM, an MD, a DVD, a DVD-ROM, or a DVD- It may be recorded in RAM, BD, semiconductor memory, or the like.
Further, the computer program or digital signal used in the above embodiment and the above modification may be transmitted via a telecommunication line, a wireless or wired communication line, a network typified by the Internet, data broadcasting, or the like. ..
Another computer by recording the computer program or digital signal used in the above embodiment and the above modification on a computer-readable recording medium and transferring it, or by transmitting it via a network or the like. It will be possible to implement it in the system.
(12) The above-described embodiment and the above-described modification may be combined.
(13) Hereinafter, the voice dialogue method and the device according to one aspect of the present invention will be described with respect to their configurations, modified examples thereof, and their effects.
(a) The voice dialogue method according to the embodiment of the present invention is a voice dialogue method performed by a voice dialogue system including a voice signal generation unit, a voice dialogue agent unit, a voice output unit, and a voice input control unit. Therefore, the voice signal generation unit receives the voice input and generates a voice signal based on the received voice input, and the voice dialogue agent unit makes a voice with respect to the voice signal generated by the voice signal generation unit. A recognition process is performed, a process based on the result of the voice recognition process is performed, a response signal related to the process is generated, and the voice output unit outputs a voice based on the response signal generated by the voice dialogue agent unit. When the voice is output by the voice output unit, the voice input control unit sets the voice signal generation unit in a acceptable state capable of accepting the voice input for a predetermined period from the output. It is a feature.
According to the above-mentioned voice dialogue method according to the embodiment of the present invention, when the voice generated by the voice dialogue agent unit is output, the user can input the voice without operating the voice dialogue system. Can be done. As a result, the number of operations that the user needs to perform in connection with the interactive voice input is reduced as compared with the conventional case.
(b) Further, the voice dialogue system includes a display unit, and when the voice signal generation unit is in the acceptable state, the display unit indicates that the voice signal generation unit is in the acceptable state. The indication may be displayed.
By doing so, the user can visually recognize whether or not the audio signal generation unit is in the acceptable state.
(c) Further, the voice dialogue system includes an additional voice dialogue agent unit, and the voice dialogue agent unit performs a process based on the result of the voice recognition process based on the result of the voice recognition process. It is determined whether to perform the voice dialogue agent unit or the additional voice dialogue agent unit, and the voice dialogue agent unit performs the voice recognition process only when it is determined in the determination that the voice dialogue agent unit performs the operation. When processing based on the result is performed and the decision is made by the additional voice dialogue agent unit, the additional voice dialogue agent unit performs voice recognition processing for the voice received by the voice signal generation unit. Is performed, a process based on the result of the voice recognition process is performed, a response signal related to the process is generated, and the voice output unit outputs a voice based on the response signal generated by the additional voice dialogue agent unit. May be good.
By doing so, it becomes possible for the additional voice dialogue agent unit to process the processing that is more appropriate for the additional voice dialogue agent unit to process than the voice dialogue agent unit processes.
(d) Further, in the display unit, when the agent unit that performs processing based on the result of voice recognition processing is the voice dialogue agent unit, the agent unit that performs processing based on the result of voice recognition processing is the voice dialogue. When the agent unit that performs the display so as to indicate that it is the agent unit and performs processing based on the result of the voice recognition processing is the additional voice dialogue agent unit, the agent that performs the processing based on the result of the voice recognition processing. The display may be performed so as to indicate that the unit is the additional voice dialogue agent unit.
By doing so, the user can visually recognize whether the agent unit that performs processing is the voice dialogue agent unit or the additional voice dialogue agent unit.
(e) Further, when the voice dialogue agent unit is determined in the determination to be performed by the additional voice dialogue agent unit, the voice dialogue agent unit generates a voice signal generated by the voice signal generation unit. It may be transferred to the additional voice dialogue agent unit, and the additional voice dialogue agent unit may perform the voice recognition process on the voice signal transferred from the voice dialogue agent unit.
By doing so, the additional voice dialogue agent unit can perform the voice recognition process by using the voice signal transferred from the voice dialogue agent unit.
(f) When the voice signal generation unit is in the acceptable state and the response signal generated by the voice dialogue agent unit indicates that it is not necessary to accept a new voice input. Even during the predetermined period, the voice input control unit may put the voice signal generation unit in an unacceptable state in which the voice input cannot be accepted.
By doing so, even if it is a predetermined period, if it is not necessary to accept the voice input, the voice signal generation unit can be put into the unacceptable state.
(g) The device according to the embodiment of the present invention has a voice signal generation unit that receives voice input and generates a voice signal based on the received voice input, and a voice signal generated by the voice signal generation unit. A transmitting unit that transmits to an external server, a receiving unit that receives a response signal generated and returned based on the voice signal by the server that has received the voice signal transmitted by the transmitting unit, and the receiving unit. When the voice is output by the voice output unit and the voice output unit that outputs the voice based on the response signal received by the voice input unit, the voice input is received by the voice signal generation unit for a predetermined period from the output. It is characterized in that it is provided with a voice input control unit that makes it possible to accept.
According to the device according to the embodiment of the present invention described above, when the voice generated by the server is output, the user can input the voice without operating the device. .. As a result, the number of operations that the user needs to perform in connection with the interactive voice input is reduced as compared with the conventional case.
The voice dialogue method and the device according to the present invention can be widely used in a voice dialogue system that performs processing based on voice input interactively by a user.
100 Voice Dialogue System 110 Voice Dialogue Agent Server 120 Network 130 Gateway 140 Equipment 210 Control Unit 220 Voice Input Unit 230 Operation Reception Unit 240 Address Storage Unit 250 Communication Unit 260 Voice Output Unit 270 Display Unit 280 Execution Unit 400 Voice Dialogue Agent 410 Control Unit 420 Communication unit 430 Speech recognition processing unit 440 Dialogue DB storage unit 450 Speech synthesis processing unit 460 Command generation unit
50 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23 Sheet 24 Sheet 25 Sheet 26 Sheet 27 Sheet 28 Sheet 29 Sheet 30 Sheet 31 Sheet 32 Sheet 33 Sheet 34 Sheet 35 Sheet 36 Sheet 37 Sheet 38 Sheet 39 Sheet 40 Sheet 41 Sheet 42 Sheet 43 Sheet 44 Sheet 45 Sheet 46 Sheet 47 Sheet 48 Sheet 49 Sheet 50
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2005266192A | Cites | Japan |
| JP2004240150A | Cites | Japan |
| JP2002116797A | Cites | Japan |
16 members in 5 offices
Priority claims5
| Document | Office | Kind | Date |
|---|---|---|---|
| 201361836763 | United States of America | P | |
| 201361836763 | United States of America | P | |
| 61836763 | United States of America | – | |
| 61836763 | – | – | – |
| US201361836763P | – | – | – |
Members16
| Document | Office | Kind | |
|---|---|---|---|
| WO2014203495A1 | World Intellectual Property Organization (WIPO) | A1 | |
| CN105144285A | China | A | |
| EP3012833A1 | European Patent Office (EPO) | A1 | |
| EP3012833A4 | European Patent Office (EPO) | A4 | |
| US2016322048A1 | United States of America | A1 | |
| US9564129B2 | United States of America | B2 | |
| JPWO2014203495A1 | Japan | A1 | |
| JP6389171B2 | Japan | B2 | |
| CN105144285B | China | B | |
| CN108806690A | China | A | |
| JP2018189984A | Japan | A | |
| JP6736617B2This record | Japan | B2 | |
| JP2020173477A | Japan | A | |
| USRE49014E | United States of America | E | |
| JP7072610B2 | Japan | B2 | |
| EP3012833B1 | European Patent Office (EPO) | B1 |
10 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 6736617
- Publication, DOCDB
- 6736617
- Publication, EPODOC
- JP6736617B
- Application
- 145020
- Application, DOCDB
- 2018145020
- Application, EPODOC
- JP20180145020
Titles2
- Japanese
- 音声対話方法、及び、音声対話エージェントサーバ
- English
- Voice dialogue method and voice dialogue agent server
Classification
- CPC, 6
- G10L15/222
- G10L15/22
- G10L15/32
- G06F3/167
- G10L15/08
- G10L2015/088
- IPC, 5
- G10L15 22
- G10L13 00
- G10L15 10
- G10L15 30
- G10L15 32
