USRE49014E

Voice interaction method, and device

Claim Score by NHIP

Abstract

A voice dialogue method performed by a voice dialog system includes: a voice signal generation unit; a voice dialog agent unit; a voice output unit; and a voice input control unit, the method including: a step of, by the voice signal generation unit, receiving a voice input and generating a voice signal based on the received voice input; a step of, by the voice dialog agent unit, performing voice recognition processing on the voice signal and performing processing based on a result of the voice recognition processing to generate a response signal; a step of, by the voice output unit, outputting a voice based on the response signal; and a step of, when the voice output unit outputs the voice, by the voice input control unit, keeping the voice signal generation unit, for predetermined period after output of the voice, a receivable state in which a voice input is receivable.

USRE49014E, drawing sheet 1 — Sheet 1 of 45

Term

7.7 yearsleft in the term

Expires 10 June 2034.

9 claims: 1 independent, 8 dependent

1
Broadest claimClaim Score 6, narrow(NHIP)A voice dialogue method that is performed by a voice dialogue system, the voice dialogue system including:a voice signal generation unit;a voice dialogue agent unit;an additional voice dialogue agent unit;a voice output unit;and a voice input control unit a first device;a second device;a first voice dialogue agent server;and a second voice dialogue agent server , wherein the first device is a computer-embedded device that is capable of connecting to a network and performing input and output via a voice with a user, the first voice dialogue agent server and the second voice dialogue agent server are each a voice dialogue agent server that is accessed by the first device via the network, and is capable of performing, as an agent for the first device, recognition of a voice inputted by the device and synthesizing of a voice to be outputted by the first device, and the voice dialogue method comprising comprises : a step of, by the voice signal generation unit, receiving a voice input and at the first device;generating a voice signal at the first device based on the received voice input , and transmitting the voice signal from the first device to the first voice dialogue agent server ;a step of, by the voice dialogue agent unit, performing voice recognition processing on the generated voice signal and at the first voice dialogue agent server to generate first text input;determining, at the first voice dialogue agent server based on a result of the voice recognition processing the generated first text input and agent information, which one of the voice dialogue agent unit first voice dialogue agent server and the additional voice dialogue agent unit second voice dialogue agent server is appropriate for performing voice-related processing that is processing based on the voice signal, the agent information being stored in a memory included in the voice dialogue agent unit first voice dialogue agent server and associating the additional voice dialogue agent unit second voice dialogue agent server with one or more keywords;a step of, when the voice dialogue agent unit determines that the voice dialogue agent unit is appropriate for performing the voice-related processing, by the voice dialogue agent unit, performing processing based on the result of the voice recognition processing to generate a response signal, and by the voice output unit, outputting a voice based on the response signal generated by the voice dialogue agent unit;a step of, when the voice dialogue agent unit determines that the additional voice dialogue agent unit is appropriate for performing the voice-related processing, by the voice dialogue agent unit, transferring the voice signal to the additional voice dialogue agent unit, by the additional voice dialogue agent unit, performing new voice recognition processing on the transferred voice signal and performing processing based on a result of the new voice recognition processing to generate a response signal, and by the voice output unit, outputting a voice based on the response signal generated by the additional voice dialogue agent unit;and a step of, when the voice output unit outputs a voice, by the voice input control unit, keeping the voice signal generation unit in a receivable state for a predetermined period after output of the voice, the receivable state being a state in which a voice input is receivable when the determining determines that the first voice dialogue agent server is appropriate for performing the voice-related processing, (i) generating, from the generated first text input, a first instruction set for the first device or another device associated with the first voice dialogue agent server, (ii) executing the generated first instruction set using the first device or the other device associated with the first voice dialogue agent server, (iii) generating a first response signal based on the execution of the generated first instruction set using the first device or the other device associated with the first voice dialogue agent server, (iv) transmitting the generated first response signal from the first voice dialogue agent server to the first device, and (v) outputting a voice at the first device based on the received first response signal generated at the first voice dialogue agent server;when the determining determines that the second dialogue agent server is appropriate for performing the voice-related processing, (i) transferring the voice signal from the first voice dialogue agent server to the second voice dialogue agent server, (ii) performing new voice recognition processing on the transferred voice signal at the second voice dialogue agent server to generate second text input, (iii) generating, from the generated second text input, a second instruction set for the second device, (iv) executing the generated second instruction set using the second device, (v) generating a second response signal based on the execution of the generated second instruction set using the second device, (vi) transmitting the generated second response signal from the second voice dialogue agent server to the first device, and (vii) outputting a voice at the first device based on the received second response signal generated at the second voice dialogue agent server;and displaying, on a screen of the first device or a screen of the second device, a text character string obtained by recognizing voice input from the user and a text character string indicating a response signal by the first device or the second device, while indicating a distinction between the user, the first voice dialogue agent server, and the second voice dialogue agent server .

594 paragraphs in 8 sections, as filed

0001This application claims benefit to the provisional U.S. Application No. 61/836,763, filed on Jun. 19, 2013.This application is a reissue of U.S. Pat. No. 9,564,129, which issued on Feb. 7, 2017 from application Ser. No. 14/777,920, filed Sep. 17, 2015, which is the National Stage of International Application No. PCT/JP2014/003097, filed Jun. 10, 2014, which claims the benefit of U.S. Provisional Application No. 61/836,763, filed Jun. 19, 2013.

TECHNICAL FIELD

0002The present invention relates to a voice dialogue method for performing processing based on a voice that is dialogically input.

BACKGROUND ART

0003There has conventionally been known a voice dialogue system that includes voice input interface and performs processing based on a voice that is dialogically input by a user.

0004For example, Patent Literature 1 discloses a headset that includes a microphone, performs voice recognition processing on a voice input through the microphone, and performs processing based on a result of the voice recognition processing.

0005Also, Patent Literature 2 discloses a voice dialogue system that includes an agent that performs processing based on a voice that is dialogically input by a user.

CITATION LIST

Patent Literature

0006[Patent Literature 1] Japanese Patent Application Publication No. 2004-233794

0007[Patent Literature 2] Japanese Patent Application Publication No. 2008-90545

SUMMARY OF INVENTION

Technical Problem

0008According to the headset disclosed in Patent Literature 1, it is necessary to perform an operation of pressing a voice recognition control button that is provided in the headset at a start time and an end time of a voice input. Accordingly, in the case where this headset is used as input means in a voice dialogue system that performs processing based on a dialogically input voice, a user of the headset needs to start a voice input by pressing the voice recognition control button and end the voice input by pressing the voice recognition control button for each voice input.

0009This sometimes makes the user to feel troublesome to perform the operation of pressing the voice recognition control button, which needs to be performed at a start time and an end time of each voice input.

0010The present invention was made in view of the problem, and aims to provide a voice dialogue method for reducing, in a voice dialogue system, the number of times that a user needs to perform an operation in accordance with a voice that is dialogically input, compared with a conventional technique.

Solution to Problem

0011In order to solve the above aim, one aspect of the present invention provides a voice dialogue method that is performed by a voice dialogue system, the voice dialogue system including: a voice signal generation unit; a voice dialogue agent unit; a voice output unit; and a voice input control unit, the voice dialogue method comprising: a step of, by the voice signal generation unit, receiving a voice input and generating a voice signal based on the received voice input; a step of, by the voice dialogue agent unit, performing voice recognition processing on the generated voice signal and performing processing based on a result of the voice recognition processing to generate a response signal; a step of, by the voice output unit, outputting a voice based on the generated response signal; and a step of, when the voice output unit outputs the voice, by the voice input control unit, keeping the voice signal generation unit in a receivable state for a predetermined period after output of the voice, the receivable state being a state in which a voice input is receivable.

Advantageous Effects of Invention

0012According to the above voice dialogue method, in the case where a voice generated by the voice dialogue agent unit is output, a user can input a voice without performing an operation with respect to the voice dialogue system. This reduces the number of times that the user needs to perform an operation in accordance with a voice that is dialogically input, compared with conventional techniques.

BRIEF DESCRIPTION OF DRAWINGS

0013<figref idref="DRAWINGS">FIG. 1</figref> is a system configuration diagram showing configuration of a voice dialogue system 100.

0014<figref idref="DRAWINGS">FIG. 2</figref> is a block diagram showing functional configuration of a device 140.

0015<figref idref="DRAWINGS">FIG. 3</figref> shows switching of a state managed by a control unit 210.

0016<figref idref="DRAWINGS">FIG. 4</figref> is a block diagram showing functional configuration of a voice dialogue agent 400.

0017<figref idref="DRAWINGS">FIG. 5</figref> is a data structure diagram showing a dialog DB 500.

0018<figref idref="DRAWINGS">FIG. 6</figref> is a flow chart of first device processing.

0019<figref idref="DRAWINGS">FIG. 7</figref> is a flow chart of first voice input processing.

0020<figref idref="DRAWINGS">FIG. 8</figref> is a flow chart of first agent processing.

0021<figref idref="DRAWINGS">FIG. 9</figref> is a flow chart of first instruction execution processing.

0022<figref idref="DRAWINGS">FIG. 10</figref> is a procedure diagram in a specific example.

0023<figref idref="DRAWINGS">FIG. 11A</figref> to <figref idref="DRAWINGS">FIG. 11D</figref> are each a pattern diagram showing contents displayed by the device 140.

0024<figref idref="DRAWINGS">FIG. 12</figref> is a pattern diagram showing contents displayed by the device 140.

0025<figref idref="DRAWINGS">FIG. 13</figref> is a block diagram showing functional configuration of a device 1300.

0026<figref idref="DRAWINGS">FIG. 14</figref> shows switching of the state managed by a control unit 1310.

0027<figref idref="DRAWINGS">FIG. 15</figref> is a flow chart of second device processing.

0028<figref idref="DRAWINGS">FIG. 16</figref> is a procedure diagram schematically showing a situation in which a dialogue with a voice dialogue agent is performed.

0029<figref idref="DRAWINGS">FIG. 17</figref> is a block diagram showing functional configuration of a device 1700.

0030<figref idref="DRAWINGS">FIG. 18</figref> shows switching of the state managed by a control unit 1710.

0031<figref idref="DRAWINGS">FIG. 19</figref> is a flow chart of third device processing.

0032<figref idref="DRAWINGS">FIG. 20</figref> is a flow chart of second voice input processing.

0033<figref idref="DRAWINGS">FIG. 21</figref> is a procedure diagram schematically showing a situation in which a dialogue with a dialogue agent is performed.

0034<figref idref="DRAWINGS">FIG. 22</figref> is a block diagram showing functional configuration of a voice dialogue agent 2200.

0035<figref idref="DRAWINGS">FIG. 23</figref> is a data structure diagram showing a target agent DB 2300.

0036<figref idref="DRAWINGS">FIG. 24</figref> is a flow chart of second agent processing.

0037<figref idref="DRAWINGS">FIG. 25</figref> is a flow chart of second instruction execution processing.

0038<figref idref="DRAWINGS">FIG. 26</figref> is a flow chart of first connection response processing.

0039FIG. 2627 is a flow chart of disconnection response processing.

0040<figref idref="DRAWINGS">FIG. 28</figref> is a flow chart of third agent processing.

0041<figref idref="DRAWINGS">FIG. 29</figref> is a procedure diagram schematically showing a situation in which a dialogue with a voice dialogue agent is performed.

0042<figref idref="DRAWINGS">FIG. 30</figref> is a block diagram showing functional configuration of a voice dialogue agent 3000.

0043<figref idref="DRAWINGS">FIG. 31</figref> is a data structure diagram showing an available service DB 3100.

0044<figref idref="DRAWINGS">FIG. 32</figref> is a flow chart of fourth agent processing.

0045<figref idref="DRAWINGS">FIG. 33</figref> is a flow chart of third instruction execution processing.

0046<figref idref="DRAWINGS">FIG. 34</figref> is a flow chart of second connection response processing.

0047<figref idref="DRAWINGS">FIG. 35</figref> is a procedure diagram schematically showing a situation in which a dialogue with a voice dialogue agent is performed.

0048<figref idref="DRAWINGS">FIG. 36A</figref> is a diagram schematically showing an operation situation of the voice dialogue system, <figref idref="DRAWINGS">FIG. 36B</figref> and <figref idref="DRAWINGS">FIG. 36C</figref> are each a diagram schematically showing a data center administration company 3610.

0049<figref idref="DRAWINGS">FIG. 37</figref> is a diagram schematically showing service type 1.

0050<figref idref="DRAWINGS">FIG. 38</figref> is a diagram schematically showing service type 2.

0051<figref idref="DRAWINGS">FIG. 39</figref> is a diagram schematically showing service type 3.

0052<figref idref="DRAWINGS">FIG. 40</figref> is a diagram schematically showing service type 4.

0053<figref idref="DRAWINGS">FIG. 41</figref> is a system configuration diagram showing configuration of a voice dialogue system 4100.

0054<figref idref="DRAWINGS">FIG. 42</figref> is a block diagram showing functional configuration of a mediation server 4150.

0055<figref idref="DRAWINGS">FIG. 43</figref> is a block diagram showing functional configuration of a mediation server 4350.

0056<figref idref="DRAWINGS">FIG. 44A</figref> to <figref idref="DRAWINGS">FIG. 44D</figref> each show an example of an image displayed by a display unit.

0057<figref idref="DRAWINGS">FIG. 45A</figref> and <figref idref="DRAWINGS">FIG. 45B</figref> each show an example of an image displayed by the display unit.

0058<figref idref="DRAWINGS">FIG. 46</figref> shows an example of switching of the state.

0059<figref idref="DRAWINGS">FIG. 47</figref> shows an example of switching of the state.

0060<figref idref="DRAWINGS">FIG. 48</figref> shows an example of switching of the state.

0061<figref idref="DRAWINGS">FIG. 49</figref> shows an example of switching of the state.

0062<figref idref="DRAWINGS">FIG. 50</figref> shows an example of switching of the state.

DESCRIPTION OF EMBODIMENTS

Embodiment 1

0063<Outline>

0064The following explains, as one aspect of the voice dialogue method relating to the present invention and one aspect of the device relating to the present invention, a voice dialogue system including devices that are disposed in a home, a car, and so on and a voice dialogue agent server that communicates the devices.

0065In the voice dialogue system, the voice dialogue agent server embodies a voice dialogue agent by executing a program stored therein. The voice dialogue agent makes a voice dialogue via a device (input and output via a voice) with a user of the voice dialogue system. The voice dialogue agent performs processing that reflects details of the dialogue, and performs a voice output of a result of the processing via the device of the user.

0066In the case where the user hopes to make a dialogue with the voice dialogue agent (hopes to perform a voice input with respect to the voice dialogue agent), the user performs a predetermined voice input start operation with respect to the device constituting the voice dialogue system. The device is switched to a state in which voice input is receivable for a predetermined period after the voice input start operation. While the device is in the state in which voice input is receivable, the user performs voice input with respect to the voice dialogue agent.

0067The following explains the details of the voice dialogue system with reference to the drawings.

0068<Configuration>

0069<figref idref="DRAWINGS">FIG. 1</figref> is a system configuration diagram showing configuration of a voice dialogue system 100.

0070As shown in the figure, the voice dialogue system 100 includes voice dialogue agent servers 110a and 110b, a network 120, gateways 130a and 130b, and devices 140a-140e.

0071While the gateway 130a and the devices 140a-140c are disposed in a home 180, the gateway 130b and the devices 140d and 140e are disposed in a car 190.

0072The gateways 130a and 130b are hereinafter just referred to as a gateway 130 except in the case of explicit distinction. Also, the voice dialogue agent servers 110a and 110b are hereinafter just referred to as a voice dialogue agent server 110 except in the case of explicit distinction. The devices 140a-140e each have a function of performing a wireless or wired communication with the gateway 130 and a function of performing a wireless or wired communication with the voice dialogue agent server 110.

0073The devices 140a-140c, which are disposed in the home 180, are each for example a television, an air conditioner, a recorder, a washing machine, a portable smartphone, or the like that is disposed in the home 180. The devices 140d-140e, which are disposed in the car 190, are each for example a car air conditioner, a car navigation system, or the like that is disposed in the car 190.

0074Here, explanation is provided on a virtual device 140 that has functions the devices 140a-140e commonly have, instead of separate explanations of the devices 140a-140e.

0075<figref idref="DRAWINGS">FIG. 2</figref> is a block diagram showing functional configuration of the device 140.

0076As shown in the figure, the device 140 includes a control unit 210, a voice input unit 220, an operation reception unit 230, an address storage unit 240, a communication unit 250, a voice output unit 260, a display unit 270, and an execution unit 280.

0077The voice input unit 220 is for example embodied by a microphone and a processor that executes programs. The voice input unit 220 is connected to the control unit 210, and is controlled by the control unit 210. The voice input unit 220 has a function of receiving a voice input from a user and generating a voice signal (hereinafter, referred to also as input voice data).

0078The voice input unit 220 is in either a voice input receivable state or a voice input unreceivable state under the control by the control unit 210. In the voice input receivable state, the voice input unit 220 is able to receive a voice input. In voice input unreceivable state, the voice input unit 220 is unable to receive a voice input.

0079The operation reception unit 230 is for example embodied by a touchpanel, a touchpanel controller, and a processor that executes programs. The operation reception unit 230 is connected to the control unit 210, and is controlled by the control unit 210. The operation reception unit 230 has a function of receiving a predetermined contact operation performed by the user and generating an electrical signal based on the received contact operation.

0080The predetermined contact operation performed by the user, which is received by the operation reception unit 230, includes a predetermined voice input start operation indicating that a voice input using the voice input unit 220 is to be started.

0081The voice input start operation is for example assumed to be an operation of contacting an icon for receiving the voice input start operation that is displayed on the touchpanel which is part of the operation reception unit 230. Also, the voice input start operation is for example assumed to be an operation of pressing a button for receiving the voice input start operation that is included in the operation reception unit 230.

0082The address storage unit 240 is for example embodied by a memory and a processor that executes programs, and is connected to the communication unit 250. The address storage unit 240 has a function of storing therein an IP (Internet Protocol) address of one of the voice dialogue agent servers 110 in the network 120. Hereinafter, the one voice dialogue agent server 110 is referred to as a specific voice dialogue agent server.

0083The device 140 is associated with the specific voice dialogue agent server, which is one of the voice dialogue agent servers 110.

0084Note that the memory included in the device 140 is for example a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, or the like.

0085The communication unit 250 is for example embodied by a processor that executes programs, a communication LSI (Large Scale Integration), and an antenna. The communication unit 250 is connected to the control unit 210 and the address storage unit 240, and is controlled by the control unit 210. The communication unit 250 has a gateway communication function and a voice dialogue agent server communication function described below.

0086The gateway communication function is a function of performing a wireless or wired communication with the gateway 130.

0087The voice dialogue agent server communication function is a function of communicating with the voice dialogue agent server 110 via the gateway 130 and the network 120.

0088Here, in communication with any one of the voice dialogue agent servers 110, in the case where the control unit 210 does not designate a specific one of the voice dialogue agent servers 110 as a voice dialogue agent server 110 that is a communication party, the communication unit 250 communicates with a specific voice dialogue agent server with reference to an IP address stored in the address storage unit 240.

0089The voice output unit 260 is for example embodied by a processor that executes programs and a speaker. The voice output unit 220 is connected to the control unit 210, and is controlled by the control unit 210. The voice output unit 260 has a function of converting an electrical signal, which is transmitted from the control unit 210, to a voice and outputting the voice.

0090The display unit 270 is for example embodied by a touchpanel, a touchpanel controller, and a processor that executes programs. The display unit 270 is connected to the control unit 210, and is controlled by the control unit 210. The display unit 270 has a function of displaying images, character strings, and the like based on the electrical signal, which is transmitted from the control unit 210.

0091The execution unit 280 is a functional block that achieves a function the device 140 as a device originally has. In the case where the device 140 is for example a television, the function is a function of receiving and decoding a television signal, displaying television images resulting from the decoding on a display, and outputting television audio resulting from the decoding via a speaker. In the case where the device 140 is for example an air conditioner, the function is a function of blowing cool air or warm air through a duct to bring a temperature in a room in which the air conditioner is disposed to a set temperature. The execution unit 280 is connected to the control unit 210, and is controlled by the control unit 210.

0092In the case where the device 140 is for example a television, the execution unit 280 is embodied by a television signal receiver, a television signal tuner, a television signal decoder, a display, a speaker, and so on.

0093Also, the execution unit 280 does not necessarily need to have a configuration in which all compositional elements thereof are included in a single housing. In the case where the device 140 is for example a television, the execution unit 280 is assumed to have for example a configuration in which a remote controller and the display are included in separate housings. Similarly, functional blocks of the device 140 each do not need to have a configuration in which all compositional elements thereof are included in a single housing.

0094The control unit 210 is for example embodied by a processor that executes programs. The control unit 210 is connected to the voice input unit 220, the operation reception unit 230, the communication unit 250, the voice output unit 260, the display unit 270, and the execution unit 280. The control unit 210 has a function of controlling the voice input unit 220, a function of controlling the operation reception unit 230, a function of controlling the communication unit 250, a function of controlling the voice output unit 260, a function of controlling the display unit 270, and a function of controlling the execution unit 280. The control unit 210 further has a voice input unit state management function and a first device processing execution function described below.

0095The voice input unit state management function is a function of managing the state of the voice input unit 220, which is either the voice input receivable state or the voice input unreceivable state.

0096<figref idref="DRAWINGS">FIG. 3</figref> shows switching of the state managed by the control unit 210.

0097As shown in the figure, in the case where the state is the voice input unreceivable state, (1) the control unit 210 keeps the state to the voice input unreceivable state until the operation reception unit 230 receives a voice input start operation. (2) After the reception of the voice input start operation by the operation reception unit 230, the control unit 210 switches the state to the voice input receivable state. Then, in the case where the state is the voice input receivable state, (3) the control unit 210 keeps the state to the voice input receivable state until a predetermined period T1 (for example, five seconds) has lapsed after the switching of the state to the voice input receivable state. (4) After the lapse of the predetermined period T1, the control unit 210 switches the state to the voice input unreceivable state.

0098Note that upon bootup of the device 140, the control unit 210 starts managing the state as the voice input unreceivable state.

0099Returning to <figref idref="DRAWINGS">FIG. 2</figref>, the explanation on the control unit 210 is continued.

0100The first device processing execution function is a function performed by the control unit 210 controlling the voice input unit 220, the operation reception unit 230, the communication unit 250, the voice output unit 260, the display unit 270, and the execution unit 280 to cause the device 140 to execute the first device processing as its characteristic operation to execute a sequence of processing described below. In the sequence of processing, (1) when the user performs a voice input start operation, (2) the device 140 receives a voice input from the user, and generates input voice data, (3) transmits the generated input voice data to a voice dialogue agent, (4) receives response voice data returned from the voice dialogue agent, and (5) outputs a voice based on the received response voice data.

0101Note that the first device processing is explained in detail in section <First Device Processing> later with reference to a flow chart.

0102Referring back to <figref idref="DRAWINGS">FIG. 1</figref>, the explanation on the device 140 is continued.

0103The gateway 130 is for example embodied by a personal computer or the like having a communication function, and is connected to the network 120. The gateway 130 has the following functions achieved by executing programs stored therein: a function of performing a wireless or wired communication with the device 140; a function of communicating with the voice dialogue agent server 110 via the network 120; and a function of relaying communication between the device 140 and the voice dialogue agent server 110.

0104The voice dialogue agent server 110 is for example embodied by a server, which is composed of one or more computer systems and has a communication function. The voice dialogue agent server 110 is connected to the network 120. The voice dialogue agent server 110 has the following functions achieved by executing programs stored therein: a function of communicating with another device which is connected to the network 120; a function of communicating with the device 140 via the gateway 130; and a function of embodying the voice dialogue agent 400.

0105<figref idref="DRAWINGS">FIG. 4</figref> is a block diagram showing functional configuration of the voice dialogue agent 400 embodied by the voice dialogue agent server 110.

0106As shown in the figure, the voice dialogue agent 400 includes a control unit 410, a communication unit 420, a voice recognition processing unit 430, a dialogue DB (Date Base) storage unit 440, a voice synthesizing processing unit 450, and an instruction generation unit 460.

0107The communication unit 420 is for example embodied by a processor that executes programs and a communication LSI. The communication unit 420 is connected to the control unit 410, the voice recognition processing unit 430, and the voice synthesizing processing unit 450, and is controlled by the control unit 410. The communication unit 420 has a function of communicating with another device which is connected to the network 120 and a function of communicating with the device 140 via the gateway 130.

0108The voice recognition processing unit 430 is embodied by a processor that executes programs. The voice recognition processing unit 430 is connected to the control unit 410 and the communication unit 420, and is controlled by the control unit 410. The voice recognition processing unit 430 has a function of performing voice recognition processing on input voice data received by the communication unit 420 to convert the voice data to a character string (hereinafter, referred to also as an input text).

0109The voice synthesizing processing unit 450 is for example embodied by a processor that executes programs. The voice synthesizing processing unit 450 is connected to the control unit 410 and the communication unit 420, and is controlled by the control unit 410. The voice synthesizing processing unit 450 has a function of performing voice synthesizing processing on a character sting transmitted from the control unit 410 to convert the character string to voice data.

0110The dialogue DB storage unit 440 is for example embodied by a memory and a processor that executes programs. The dialogue DB storage unit 440 is connected to the control unit 410, and has a function of storing therein a dialog DB 500.

0111<figref idref="DRAWINGS">FIG. 5</figref> is a data structure diagram showing the dialog DB 500 stored in the dialogue DB storage unit 440.

0112As shown in the figure, the dialog DB 500 includes keyword 510, target device 520, startup application 530, processing details 540, and response text 550 that are associated with each other.

0113The keyword 510 indicates a character string that is assumed to be included in an input text converted by the voice recognition processing unit 430.

0114The target device 520 indicates information for specifying a device that is to execute processing specified by the associated processing details 540, which are described later.

0115Here, the device specified by the target device 520 may be the voice dialogue agent 400.

0116The startup application 530 is information for specifying an application program to be started up in a device specified by the associated target device 520 in order to cause the specified device to execute processing specified by the associated processing details 540, which are described later.

0117The processing details 540 are information for specifying, in the case where a character string indicated by the associated keyword 510 is included in an input text that is converted by the voice recognition processing unit 430, processing that is determined to be executed by a device that is specified by the associated target device 520.

0118The response text 550 is information for indicating, in the case where processing specified by the associated processing details 540 is executed, a character string that is determined to be generated based on a result of the processing (hereinafter, referred to also as a response text).

0119Referring back to <figref idref="DRAWINGS">FIG. 4</figref>, the explanation on the voice dialogue agent 400 is continued.

0120The instruction generation unit 460 is for example embodied by a processor that executes programs. The instruction generation unit 460 is connected to the control unit 410, and is controlled by the control unit 410. The instruction generation unit 460 has a function of, upon reception of a group of the target device 520, the startup application 530, and the processing details 540 transmitted from the control unit 410, starting up an application program that is specified by the startup application 530 included in a device that is specified by the target device 520, and generating an instruction set for causing the specified device to execute processing that is specified by the processing details 540.

0121The control unit 410 is for example embodied by a processor that executes programs. The control unit 410 is connected to the communication unit 420, the voice recognition processing unit 430, the dialogue DB storage unit 440, the voice synthesizing processing unit 450, and the instruction generation unit 460. The control unit 410 has a function of controlling the communication unit 420, a function of controlling the voice recognition processing unit 430, a function of controlling the voice synthesizing processing unit 450, and a function of controlling the instruction generation unit 460. The control unit 410 further has an input text return function, an instruction generation function, an instruction execution function, and a first agent processing execution function described below.

0122The input text return function is a function of controlling, in the case where input voice data received by the communication unit 420 is converted to an input text by the voice recognition processing unit 430, the communication unit 420 to return the input text to the device 140 which has transmitted the input voice data.

0123The instruction generation function is a function of, upon reception of the input text transmitted from the voice recognition processing unit 430, controlling the instruction generation unit 460 to generate an instruction set: by (1) referring to the dialog DB 500 stored in the dialogue DB storage unit 440 to read, based on the keyword 510 included in the input text, the target device 520, the startup application 530, the processing details 540, and the response text 550, which are associated with the keyword 510; and (2) transmitting a group of the read target device 520, startup application 530, and processing details 540 to the instruction generation unit 460.

0124The instruction execution function is a function of executing an instruction set generated by the instruction generation unit 460, generating a response text specified by the response text 550 based on an execution result of the instruction set, and transmitting the generated response text to the voice synthesizing processing unit 450.

0125In execution of the instruction execution function, the control unit 410 generates a response text by communicating with a device specified by the target device 520 with use of the communication unit 420 to cause the specified device to execute the instruction set and transmit an execution result of the instruction set.

0126The first agent processing execution function is a function performed by the control unit 410 controlling the communication unit 420, the voice recognition processing unit 430, the voice synthesizing processing unit 450, and the instruction generation unit 460 to cause the voice dialogue agent 400 to execute first agent processing that is its characteristic operation to execute a sequence of processing described below. In the sequence of processing, (1) the voice dialogue agent 400 receives input voice data transmitted from a device, (2) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text to the device, (3) generates an instruction set based on the generated input text, and executes the generated instruction set (4) generates a response text based on an execution result of the instruction set, (5) converts the generated response text to response voice data, and (6) returns the response text and the response voice data to the device.

0127Note that the first agent processing is explained in detail in section <First Agent Processing> later with reference to a flow chart.

0128Here, assume a case for example where an input text “Where is Mr. A's address?” is transmitted from the voice recognition processing unit 430. In this case, with reference to the dialog DB 500 stored in the dialogue DB storage unit 440, the control unit 410 causes a device “smartphone” specified by the target device 520 to start up an application program “Contact information” specified by the startup application 530 and execute processing of “Check Mr. A's address” specified by the processing details 540, and generates a response text “Mr. A's address is XXXX.” based on an execution result of the processing.

0129The following explains the operation of the voice dialogue system 100 having the above configuration, with reference to the drawings.

0130<Operation>

0131The voice dialogue system 100 performs, as its characteristic operation, the first device processing and the first agent processing.

0132Explanation is given below on the processing in order.

0133<First Device Processing>

0134The first device processing is processing performed by the device 140. In the first device processing, (1) when the user performs a voice input start operation, (2) the device 140 receives a voice input from the user, and generates input voice data, (3) transmits the generated input voice data to a voice dialogue agent, (4) receives response voice data returned from the voice dialogue agent, and (5) outputs a voice based on the received response voice data.

0135<figref idref="DRAWINGS">FIG. 6</figref> is a flow chart of the first device processing.

0136Upon bootup of the device 140, the first device processing is started.

0137At a time of bootup of the device 140, the state managed by the control unit 210 is the voice input unreceivable state.

0138When the first device processing is started, the control unit 210 stands by until the operation reception unit 230 receives a voice input start operation performed by a user of the voice dialogue system 100 (Step S600: Repetition of No). When the operation reception unit 230 receives the voice input start operation (Step S600: Yes), the control unit 210 switches the state from the voice input unreceivable state to the voice input receivable state (Step S610), and causes the display unit 270 to display that the state is the voice input receivable state (Step S620).

0139<figref idref="DRAWINGS">FIG. 11A</figref> is a pattern diagram showing an example of a situation in which in the case where the device 140 is for example a smartphone, the display unit 270 displays that the state is the voice input receivable state.

0140In the figure, a touchpanel 1110 that constitutes the smartphone is part of the display unit 270. The touchpanel 1110 displays that the state is the voice input receivable state by blinking a region 1120 that is positioned at the lower right in the touchpanel 1110 (for example by alternately lighting black color and white color in the region 1120).

0141Referring back to <figref idref="DRAWINGS">FIG. 6</figref>, the explanation on the first device processing is continued.

0142After the end of the processing in Step S620, the device 140 executes first voice input processing (Step S630).

0143<figref idref="DRAWINGS">FIG. 7</figref> is a flow chart of the first voice input processing.

0144When the first voice input processing is started, the voice input unit 220 receives a voice input from a user, and generates input voice data (Step S700). Then, when a predetermined period T1 has lapsed after switching of the state to the voice input receivable state (Step S710: Yes after repetition of No), the control unit 210 switches the state from the voice input receivable state to the voice input unreceivable state (Step S720), and causes the display unit 270 to stop displaying that the state is the voice input receivable state (Step S730).

0145Then, the control unit 210 controls the communication unit 250 to transmit the input voice data, which is generated by the voice input unit 220, to the voice dialogue agent 400 which is embodied by a specific voice dialogue agent server (Step S740).

0146After the end of the processing in Step S740, the device 140 ends the first voice input processing.

0147Referring back to <figref idref="DRAWINGS">FIG. 6</figref> again, the explanation on the first device processing is continued.

0148After the end of the first voice input processing, the control unit 210 stands by until the communication unit 250 receives an input text that is returned from the voice dialogue agent 400 in response to the input voice data transmitted in the processing in Step S740 (Step S640: Repetition of No).

0149Here, the input text is a character string resulting from conversion of the input voice data transmitted in the processing in Step S740 performed by the voice dialogue agent 400.

0150When the communication unit 250 receives the input text (Step S640: Yes), the display unit 270 displays the input text (Step S650).

0151<figref idref="DRAWINGS">FIG. 11B</figref> is a pattern diagram showing an example of a situation in which in the case where the device 140 is for example a smartphone, the display unit 270 displays an input text.

0152In the figure, an example is shown in which the input text is a character string “What is room temperature?”. As shown in the figure, the input text, which is the character string “What is room temperature?” is displayed on the touchpanel 1110, which is part of the display unit 270, together with a character string “You”.

0153Referring back to <figref idref="DRAWINGS">FIG. 6</figref> again, the explanation on the first device processing is continued.

0154After the end of the processing in Step S650, the control unit 210 stands by until the communication unit 250 receives a response text and response voice data that are returned from the voice dialogue agent 400 in response to the input voice data transmitted in the processing in Step S740 (Step S640: Repetition of No).

0155When the communication unit 250 receives the response text and the response voice data (Step S660: Yes), the display unit 270 displays the response text (Step S670), and the voice output unit 260 converts the response voice data to a voice and outputs the voice (Step S680).

0156<figref idref="DRAWINGS">FIG. 11C</figref> is a pattern diagram showing an example of a situation in which in the case where the device 140 is for example a smartphone, the display unit 270 displays a response text.

0157In the figure, an example is shown in which the response text is a character string “Which room?”. As shown in the figure, the response text, which is the character string “Which room?”, is displayed on the touchpanel 1110 which is part of the display unit 270, together with a character string “Home agent”.

0158Referring back to <figref idref="DRAWINGS">FIG. 6</figref> again, the explanation on the first device processing is continued.

0159After the end of the processing in Step S680, the device 140 ends the first device processing.

0160<First Agent Processing>

0161The first agent processing is processing performed by the voice dialogue agent 400. In the first agent processing, (1) the voice dialogue agent 400 receives input voice data transmitted from a device, (2) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text to the device, (3) generates an instruction set based on the generated input text, and executes the generated instruction set, (4) generates a response text based on an execution result of the instruction set, (5) converts the generated response text to response voice data, and (6) returns the response text and the response voice data to the device.

0162<figref idref="DRAWINGS">FIG. 8</figref> is a flow chart of the first agent processing.

0163Upon bootup of the voice dialogue agent 400, the first agent processing is started.

0164When the first agent processing is started, the voice dialogue agent 400 stands by until the communication unit 420 receives input voice data transmitted from the device 140 (Step S800: Repetition of No). When the communication unit 420 receives the input voice data (Step S800: Yes), the voice dialogue agent 400 performs first instruction execution processing (Step S810).

0165<figref idref="DRAWINGS">FIG. 9</figref> is a flow chart of the first instruction execution processing.

0166When the first instruction execution processing is started, the voice recognition processing unit 430 performs voice recognition processing on the input voice data, which is received by the communication unit 420, to convert the input voice data to an input text that is a character string (Step S900).

0167After the conversion to the input text, the control unit 410 controls the communication unit 420 to return the converted input text to the device 140 which has transmitted the input voice data (Step S910).

0168The control unit 410 controls the instruction generation unit 460 to generate an instruction set by: (1) referring to the dialog DB 500 stored in the dialogue DB storage unit 440 to read, based on the keyword 510 included in the input text, the target device 520, the startup application 530, the processing details 540, and the response text 550, which are associated with the keyword 510; and (2) transmitting a group of the read target device 520, startup application 530, and processing details 540 to the instruction generation unit 460.

0169After the generation of the instruction set, the control unit 410 executes the generated instruction set (Step S930), and generates a response text specified by the response text 550 based on an execution result of the instruction set (Step S940). Here, the control unit 410 generates a response text by communicating with a device specified by the target device 520 with use of the communication unit 420 to cause the specified device to execute part of the instruction set and transmit an execution result of the part of the instruction set.

0170After the generation of the response text, the voice synthesizing processing unit 450 performs voice synthesizing processing on the generated response text to generate response voice data (Step S950).

0171After the generation of the response voice data, the control unit 410 controls the communication unit 420 to transmit the generated response text and response voice data to the device 140 which has transmitted the input voice data (Step S960).

0172After the end of the processing in Step S960, the voice dialogue agent 400 ends the first instruction execution processing.

0173Referring back to <figref idref="DRAWINGS">FIG. 8</figref>, the explanation on the first agent processing is continued.

0174After the end of the first instruction execution processing, the voice dialogue agent 400 returns to the processing in Step S800 to perform the processing in Step S800 and the subsequent steps.

0175The following explains a specific example of the operation performed by the voice dialogue system 100 having the above configuration, with reference to the drawing.

0176<Specific Example>

0177<figref idref="DRAWINGS">FIG. 10</figref> is a procedure diagram schematically showing a situation in which the user of the voice dialogue system 100 makes a voice dialogue with the voice dialogue agent 400 with use of the device 140 (here, a smartphone), and the voice dialogue agent 400 performs processing that reflects details of the dialogue.

0178When the user performs a voice input start operation (Step S1000, corresponding to Step S600: Yes in <figref idref="DRAWINGS">FIG. 6</figref>), the state is switched to the voice input receivable state (Step S1005, corresponding to Step S610 in <figref idref="DRAWINGS">FIG. 6</figref>), and the device 140 performs first voice input processing (Step S1010, corresponding to Step S630<figref idref="DRAWINGS">FIG. 6</figref>).

0179<figref idref="DRAWINGS">FIG. 11A</figref> is a diagram schematically showing an example of a situation in which, while the state is the voice input receivable state in the first voice input processing, the touchpanel 1110, which is part of the display unit 270 included in the device 140 which is a smartphone, displays that the state is the voice input receivable state by blinking the region 1120.

0180Referring back to <figref idref="DRAWINGS">FIG. 10</figref>, the explanation on the specific example is continued.

0181In the first voice input processing, in the case where the user inputs a voice “What is room temperature?”, the device 140 transmits input voice data “What is room temperature?” to the voice dialogue agent 400 (corresponding to Step S740 in <figref idref="DRAWINGS">FIG. 7</figref>).

0182Then, the voice dialogue agent 400 receives the input voice data (corresponding to Step S800: Yes in <figref idref="DRAWINGS">FIG. 8</figref>), and performs first instruction execution processing (Step S1060, corresponding to Step S810 in <figref idref="DRAWINGS">FIG. 8</figref>).

0183Here, in the first instruction execution processing, in the case where the voice dialogue agent 400 generates response voice data “Which room?”, the voice dialogue agent 400 transmits the response voice data “Which room?” to the device 140 (corresponding to Step S960 in <figref idref="DRAWINGS">FIG. 9</figref>).

0184Then, the device 140 receives the response voice data (corresponding to Step S660: Yes in <figref idref="DRAWINGS">FIG. 6</figref>), and outputs a voice “Which room?” (Step S1015, corresponding to Step S680 in <figref idref="DRAWINGS">FIG. 6</figref>).

0185In the processing in Step S1010, when the predetermined period T1 has lapsed after the switching of the state to the voice input receivable state, the state is switched again to the voice input unreceivable state (corresponding to Step S720 in <figref idref="DRAWINGS">FIG. 7</figref>). Accordingly, the user, who has heard the voice “Which room?” which is output from the device 140, performs a new voice input start operation with respect to the device 140 to newly input a voice (Step S1020, corresponding to Step S600: Yes in <figref idref="DRAWINGS">FIG. 6</figref>). Then, the state is switched to the voice input receivable state (Step S1025, corresponding to Step S610 in <figref idref="DRAWINGS">FIG. 6</figref>), and the device 140 performs first voice input processing (Step S1030, corresponding to Step S630 in <figref idref="DRAWINGS">FIG. 6</figref>).

0186<figref idref="DRAWINGS">FIG. 11C</figref> is a diagram schematically showing an example of a situation in which, while the state is the voice input receivable state in the first voice input processing, the touchpanel 1110, which is part of the display unit 270 included in the device 140 which is a smartphone, displays that the state is the voice input receivable state by blinking the region 1120.

0187Referring back to <figref idref="DRAWINGS">FIG. 10</figref> again, the explanation on the specific example is continued.

0188In the first voice input processing, in the case where the user inputs a voice “Living room.”, the device 140 transmits input voice data “Living room.” to the voice dialogue agent 400 (corresponding to Step S740 in <figref idref="DRAWINGS">FIG. 7</figref>).

0189Then, the voice dialogue agent 400 receives the input voice data (corresponding to Step S800: Yes in <figref idref="DRAWINGS">FIG. 8</figref>), and performs first instruction execution processing (Step S1065, corresponding to Step S810 in <figref idref="DRAWINGS">FIG. 8</figref>).

0190Here, in the first instruction execution processing, in the case where the voice dialogue agent 400 generates response voice data “Living room temperature is 28 degrees C. Do you need any other help?”, the voice dialogue agent 400 transmits the response voice data “Living room temperature is 28 degrees C. Do you need any other help?” to the device 140 (corresponding to Step S960: Yes in <figref idref="DRAWINGS">FIG. 9</figref>).

0191Then, the device 140 receives the response voice data (corresponding to Step S660: Yes in <figref idref="DRAWINGS">FIG. 6</figref>), and outputs a voice “Living room temperature is 28 degrees C. Do you need any other help?” (Step S1035, corresponding to Step S680 in <figref idref="DRAWINGS">FIG. 6</figref>).

0192In the processing in Step S1010, when the predetermined period T1 has lapsed after the switching of the state to the voice input receivable state, the state is switched again to the voice input unreceivable state (corresponding to Step S720 in <figref idref="DRAWINGS">FIG. 7</figref>). Accordingly, the user, who has heard the voice “Living room temperature is 28 degrees C. Do you need any other help?” which is output from the device 140, performs a new voice input start operation with respect to the device 140 to newly input a voice (Step S1040, corresponding to Step S600: Yes in <figref idref="DRAWINGS">FIG. 6</figref>). Then, the state is switched to the voice input receivable state (Step S1045, corresponding to Step S610 in <figref idref="DRAWINGS">FIG. 6</figref>), and the device 140 performs first voice input processing (Step S1050, corresponding to Step S630 in <figref idref="DRAWINGS">FIG. 6</figref>).

0193<figref idref="DRAWINGS">FIG. 12</figref> is a diagram schematically showing an example where, in the first voice input processing, while the state is the voice input receivable state, the touchpanel 1110, which is part of the display unit 270 included in the device 140 which is a smartphone, displays that the state is the voice input receivable state by blinking the region 1120.

0194Referring back to <figref idref="DRAWINGS">FIG. 10</figref> again, the explanation on the specific example is continued.

0195In the first voice input processing, in the case where the user inputs a voice “No. Thank you.”, the device 140 transmits input voice data “No. Thank you.” to the voice dialogue agent 400 (corresponding to Step S740 in <figref idref="DRAWINGS">FIG. 7</figref>).

0196Then, the voice dialogue agent 400 receives the input voice data (corresponding to Step S800: Yes in <figref idref="DRAWINGS">FIG. 8</figref>), and performs first instruction execution processing (Step S1070, corresponding to Step S810 in <figref idref="DRAWINGS">FIG. 8</figref>).

0197Here, in the first instruction execution processing, in the case where the voice dialogue agent 400 generates response voice data “This ends dialogue.”, the voice dialogue agent 400 transmits the response voice data “This ends dialogue.” to the device 140 (corresponding to Step S960: Yes in <figref idref="DRAWINGS">FIG. 9</figref>).

0198Then, the device 140 receives the response voice data (corresponding to Step S660: Yes in <figref idref="DRAWINGS">FIG. 6</figref>), and outputs a voice “This ends dialogue.” (Step S1055, corresponding to Step S680 in <figref idref="DRAWINGS">FIG. 6</figref>).

0199<Consideration>

0200According to the voice dialogue system 100 having the above configuration, the user switches the state of the device 140 by performing a voice input start operation with respect to the device 140, and inputs a voice. Then, when the predetermined period T1 has lapsed, the state of the device 140 is switched to the voice input unreceivable state even if the user does not perform any operation for switching the state of the device 140 to the voice input unreceivable state.

0201According to the voice dialogue system 100, therefore, a reduced number of operations need to be performed by the user in accordance with a voice input, compared with a voice dialogue system in which each time a voice input ends, it is necessary to perform an operation for switching the state of the device 140 to the voice input unreceivable state.

Embodiment 2

0202<Outline>

0203The following explains, as one aspect of the voice dialogue method relating to the present invention and one aspect of the device relating to the present invention, a first modified voice dialogue system that is a partial modification of the voice dialogue system 100 in Embodiment 1.

0204The voice dialogue system 100 in Embodiment 1 has been explained as an example of the configuration in which when the user performs a voice input start operation, the device 140 is in the voice input receivable state for a period from performance of the voice input start operation to lapse of the predetermined period T1.

0205Compared with this, the first modified voice dialogue system in Embodiment 2 is an example of configuration in which in the case where a device outputs a voice based on response voice data, the device is in the voice input receivable state for a period from output of the voice to lapse of the predetermined period T1, in addition to the above period.

0206The following explains the details of the first modified voice dialogue system, focusing on different points from the voice dialogue system 100 in Embodiment 1, with reference to the drawings.

0207<Configuration>

0208The first modified voice dialogue system is modified from the voice dialogue system 100 in Embodiment 1 so as to include a device 1300 instead of the device 14.

0209The device 1300 is not modified from the device 140 in Embodiment 1 in terms of hardware, but is partially modified from the device 140 in terms of software to be stored as an execution target. Accordingly, the device 1300 is modified from the device 140 in Embodiment 1 in terms of part of functions.

0210<figref idref="DRAWINGS">FIG. 13</figref> is a block diagram showing functional configuration of the device 1300.

0211As shown in the figure, the device 1300 is modified from the device 140 in Embodiment 1 (see <figref idref="DRAWINGS">FIG. 2</figref>) so as to include a control unit 1310 instead of the control unit 210.

0212The control unit 1310 is modified from the control unit 210 in Embodiment 1 so as to have a first modified voice input unit state management function and a second device processing execution function, which are described below, instead of the voice input unit state management function and the first device processing execution function of the control unit 210.

0213Similarly to the voice input unit state management function in Embodiment 1, the first modified voice input unit state management function is a function of managing the state of the voice input unit 220, which is either the voice input receivable state or the voice input unreceivable state, and conditions for switching the state are partially modified from those in the voice input unit state management function in Embodiment 1.

0214<figref idref="DRAWINGS">FIG. 14</figref> shows switching of the state managed by the control unit 1310.

0215As shown in the figure, in the case where the state is the voice input unreceivable state, (1) the control unit 1310 keeps the state to the voice input unreceivable state until the operation reception unit 230 receives a voice input start operation or the voice output unit 260 outputs a voice included in voices based on response voice data except a predetermined voice. (2) After the reception of the voice input start operation by the operation reception unit 230 or output of the voice included in voices based on the response voice data by the voice output unit 260, the control unit 1310 switches the state to the voice input receivable state. Then, in the case where the state is the voice input receivable state, (3) the control unit 1310 keeps the state to the voice input receivable state until a predetermined period T1 (for example, five seconds) has lapsed after the switching of the state to the voice input receivable state. (4) After the lapse of the predetermined period T1, the control unit 1310 switches the state to the voice input unreceivable state.

0216Here, the predetermined voice included in the voices based on response voice data is a voice that indicates unnecessity of a new voice input, such as a voice “This ends dialogue.”. Hereinafter, this voice is referred to also as a dialogue end voice.

0217Note that upon bootup of the device 1300, the control unit 1310 starts managing the state as the voice input unreceivable state.

0218Referring back to <figref idref="DRAWINGS">FIG. 13</figref>, the explanation on the control unit 1310 is continued.

0219The second device processing execution function is a function performed by the control unit 1310 controlling the voice input unit 220, the operation reception unit 230, the communication unit 250, the voice output unit 260, the display unit 270, and the execution unit 280 to cause the device 1300 to execute the second device processing that is its characteristic operation to execute a sequence of processing described below. In the sequence of processing, (1) when the user performs a voice input start operation, (2) the device 1300 receives a voice input from the user, and generates input voice data, (3) transmits the generated input voice data to a voice dialogue agent, (4) receives response voice data returned from the voice dialogue agent, (5) outputs a voice based on the received response voice data, and (6) in the case where the output voice is not a dialogue end voice, the device 1300 repeats the processing (2) and the subsequent processing even if the user does not perform a voice input start operation.

0220Note that the second device processing is explained in detail in section <Second Device Processing> later with reference to a flow chart.

0221The following explains the operation of the first modified voice dialogue system having the above configuration, with reference to the drawings.

0222<Operation>

0223The first modified voice dialogue system performs second device processing as its characteristic operation, in addition to the first agent processing in Embodiment 1. The second device processing is partially modified from the first device processing in Embodiment 1.

0224Explanation is given on the second device processing below, focusing on different points from the first device processing.

0225<Second Device Processing>

0226The second device processing is processing performed by the device 1300. In the second device processing, (1) when the user performs a voice input start operation, (2) the device 1300 receives a voice input from the user, and generates input voice data, (3) transmits the generated input voice data to a voice dialogue agent, (4) receives response voice data returned from the voice dialogue agent, (5) outputs a voice based on the received response voice data, and (6) in the case where the output voice is not a dialogue end voice, the device 1300 repeats the processing (2) and the subsequent processing even if the user does not perform a voice input start operation.

0227<figref idref="DRAWINGS">FIG. 15</figref> is a flow chart of the second device processing.

0228Upon bootup of the device 1300, the second device processing is started.

0229At a time of bootup of the device 1300, the state managed by the control unit 1310 is the voice input unreceivable state.

0230In the figure, processing in Steps S1500-S1580 is the same as the processing in Steps S600-S680 in the first device processing in Embodiment 1 (see <figref idref="DRAWINGS">FIG. 6</figref>), and is accordingly regarded as having been already explained.

0231After the end of the processing in Step S1580, the control unit 1310 checks whether or not the voice, which is output from the voice output unit 260 in the processing in Step S1580, is a dialogue end voice (Step S1585). This processing is executed by for example checking whether or not the response text, which is received in the processing in Step S1560: Yes, is a predetermined character string (for example, a character string “This ends dialogue.”).

0232In the processing in Step S1585, in the case where the response text is not a dialogue end voice (Step S1585: No), the control unit 1310 switches the state from the voice input unreceivable state to the voice input receivable state (Step S1590), and causes the display unit 270 to display that the state is the voice input receivable state (Step S1595).

0233After the end of the processing in Step S1595, the device 1300 returns to the processing in Step S1530 to perform the processing in Step S1530 and the subsequent steps.

0234In the processing in Step S1585, in the case where the response text is a dialogue end voice (Step S1585: Yes), the device 1300 ends the second device processing.

0235The following explains a specific example of the operation performed by the first modified voice dialogue system having the above configuration, with reference to the drawing.

0236<Specific Example>

0237<figref idref="DRAWINGS">FIG. 16</figref> is a procedure diagram schematically showing a situation in which the user of the first modified voice dialogue system performs a voice dialogue with the voice dialogue agent 400 with use of the device 1300 (here, assumed to be a smartphone), and the voice dialogue agent 400 performs processing that reflects details of the dialogue.

0238Here, the explanation is given based on the assumption that a dialogue end voice is a voice “This ends dialogue.”.

0239In the figure, processing in Steps S1600-S1615, processing in Steps S1630-S1635, processing in Steps S1650-S1655, and processing in Steps S1660-S1670 are respectively the same as the processing in Steps S1000-S1015, the processing in Steps S1030-S1035, the processing in Steps S1050-S1055, and the processing in Steps S1060-S1070 in the specific examples in Embodiment 1 (see <figref idref="DRAWINGS">FIG. 10</figref>). Accordingly, the processing in the figure is regarded as having been already explained.

0240After the end of the processing in Step S1615, since a voice “Which room?” is not a dialogue end voice (corresponding to Step S1585: No in <figref idref="DRAWINGS">FIG. 15</figref>), the state is switched to the voice input receivable state (Step S1625, corresponding to Step S1590 in <figref idref="DRAWINGS">FIG. 15</figref>). The device 1300 performs first voice input processing (Step S1630, corresponding to Step S1530 in <figref idref="DRAWINGS">FIG. 15</figref>).

0241After the end of the processing in Step S1635, since a voice “Living room temperature is 28 degrees C. Do you need any other help?” is not a dialogue end voice (corresponding to Step S1585: No in <figref idref="DRAWINGS">FIG. 15</figref>), the state is switched to the voice input receivable state (Step S1645, corresponding to Step S1590 in <figref idref="DRAWINGS">FIG. 15</figref>). The device 1300 performs first voice input processing (Step S1650, corresponding to Step S1530 in <figref idref="DRAWINGS">FIG. 15</figref>).

0242After the end of the processing in Step S1635, since a voice “This ends dialogue.” is a dialogue end voice (corresponding to Step S1585: Yes in <figref idref="DRAWINGS">FIG. 15</figref>), the state is not switched to the voice input receivable state. The device 1300 ends the second device processing.

0243<Consideration>

0244According to the first modified voice dialogue system having the above configuration, in the case where the device 1300 outputs a voice based on response voice data transmitted from the voice dialogue agent 400 and the output voice is not a dialogue end voice, the state of the device 1300 is switched to the voice input receivable state even if the user does not perform a voice input start operation.

0245Accordingly, once the user performs a voice input start operation with respect to the device 1300, the user can newly input a voice without newly performing a voice input operation with respect to the device 1300, for a period from output of the voice based on the response voice data to lapse of the predetermined period T1 until a dialogue end voice is output.

0246According to the first modified voice dialogue system, as described above, a further reduced number of operations need to be performed by the user in accordance with a voice input, compared with the voice dialogue system 100 in Embodiment 1.

Embodiment 3

0247<Outline>

0248The following explains, as one aspect of the voice dialogue method relating to the present invention and one aspect of the device relating to the present invention, a second modified voice dialogue system that is partially modified from the voice dialogue system 100 in Embodiment 1.

0249The voice dialogue system 100 in Embodiment 1 has been explained as an example of the configuration in which when the user performs a voice input start operation with respect to the device 140, the device 140 is in the voice input receivable state for a period from performance of the voice input start operation to lapse of the predetermined period T1.

0250Compared with this, the second modified voice dialogue system in Embodiment 3 is an example of configuration in which once a user performs a voice input start operation with respect to a device, the device is in the voice input receivable state for a period from performance of the voice input start operation to output of a dialogue end voice.

0251The following explains the details of the second modified voice dialogue system, focusing on different points from the voice dialogue system 100 in Embodiment 1, with reference to the drawings.

0252<Configuration>

0253The second modified voice dialogue system is modified from the voice dialogue system 100 in Embodiment 1 so as to include a device 1700 instead of the device 140.

0254The device 1700 is not modified from the device 140 in Embodiment 1 in terms of hardware, but is partially modified from the device 140 in terms of software to be stored as an execution target. Accordingly, the device 1700 is modified from the device 140 in Embodiment 1 in terms of part of functions.

0255<figref idref="DRAWINGS">FIG. 17</figref> is a block diagram showing functional configuration of the device 1700.

0256As shown in the figure, the device 1700 is modified from the device 140 in Embodiment 1 (see <figref idref="DRAWINGS">FIG. 2</figref>) so as to include the control unit 1710 instead of the control unit 210.

0257The control unit 1710 is modified from the control unit 210 in Embodiment 1 so as to have a second modified voice input unit state management function and a third device processing execution function, which are described below, instead of the voice input unit state management function and the first device processing execution function of the functions of the control unit 210, respectively.

0258Similarly to the voice input unit state management function in Embodiment 1 and the first modified voice input unit state management function in Embodiment 2, the second modified voice input unit state management function is a function of managing the state of the voice input unit 220, which is either the voice input receivable state or the voice input unreceivable state, and conditions for switching the state are partially modified from those in the voice input unit state management function in Embodiment 1.

0259<figref idref="DRAWINGS">FIG. 18</figref> shows switching of the state managed by the control unit 1710.

0260As shown in the figure, in the case where the state is the voice input unreceivable state, (1) the control unit 1710 keeps the state to the voice input unreceivable state until the operation reception unit 230 receives a voice input start operation, and (2) after the reception of the voice input start operation by the operation reception unit 230, the control unit 210 switches the state to the voice input receivable state. Then, in the case where the state is the voice input receivable state, (3) the control unit 1710 keeps the state to the voice input receivable state until the voice output unit 260 outputs a dialogue end voice (for example, a voice “This ends dialogue.”), and (4) after the output of the dialogue end voice by the voice output unit 260, the control unit 1710 switches the state to the voice input unreceivable state.

0261Referring back to <figref idref="DRAWINGS">FIG. 17</figref>, the explanation on the control unit 1710 is continued.

0262The third device processing execution function is a function performed by the control unit 1710 controlling the voice input unit 220, the operation reception unit 230, the communication unit 250, the voice output unit 260, the display unit 270, and the execution unit 280 to cause the device 1700 to execute the third device processing, as its characteristic operation, to execute a sequence of processing described below. In the sequence of processing, (1) when the user performs a voice input start operation, (2) the device 1700 receives a voice input from the user, and generates input voice data, (3) transmits the generated input voice data to a voice dialogue agent, (4) receives response voice data returned from the voice dialogue agent, (5) outputs a voice based on the received response voice data, and (6) in the case where the output voice is not a dialogue end voice, repeats the processing (2) and the subsequent processing even if the user does not perform a voice input start operation.

0263Note that the third device processing is explained in detail in section <Third Device Processing> later with reference to a flow chart.

0264The following explains the operation of the second modified voice dialogue system having the above configuration, with reference to the drawings.

0265<Operation>

0266The second modified voice dialogue system performs third device processing as its characteristic operation, in addition to the first agent processing in Embodiment 1. The third device processing partially modified from the first device processing in Embodiment 1.

0267Explanation is given on the third device processing below, focusing on different points from the first device processing.

0268<Third Device Processing>

0269The third device processing is processing performed by the device 1700. In the third device processing, (1) when the user performs a voice input start operation with respect to the device 1700, (3) the device 1700 receives a voice input from the user, and generates input voice data, (3) transmits the generated input voice data to a voice dialogue agent, (4) receives response voice data returned from the voice dialogue agent, (5) outputs a voice based on the received response voice data, and (6) in the case where the output voice is not a dialogue end voice, repeats the processing (2) and the subsequent processing even if the user does not perform a voice input start operation.

0270<figref idref="DRAWINGS">FIG. 19</figref> is a flow chart of the third device processing.

0271Upon bootup of the device 1700, the third device processing is started.

0272At a time of bootup of the device 1700, the state managed by the control unit 1710 is the voice input unreceivable state.

0273In the figure, processing in Steps S1900-S1920 and processing in Steps S1940-S1980 is respectively the same as the processing in Steps S600-S620 and the processing in Steps S640-S680 in the first device processing in Embodiment 1 (see <figref idref="DRAWINGS">FIG. 6</figref>). Accordingly, the processing in the figure is regarded as having been already explained.

0274After the end of the processing in Step S1920, the device 1700 executes second voice input processing (Step S1930).

0275<figref idref="DRAWINGS">FIG. 20</figref> is a flow chart of the second voice input processing.

0276When the second voice input processing is started, the voice input unit 220 receives a voice input from a user, and generates input voice data (Step S2000).

0277Then, the control unit 1910 controls the communication unit 250 to transmit the input voice data, which is generated by the voice input unit 220, to the voice dialogue agent 400 (Step S2040).

0278After the end of the processing in Step S2040, the device 1700 ends the second voice input processing.

0279Referring back to <figref idref="DRAWINGS">FIG. 19</figref>, the explanation on the third device processing is continued.

0280After the end of the second voice input processing, the device 1700 proceeds to processing in Step S1940 to perform the processing in Step S1940 and processing in subsequent steps.

0281After the end of the processing in Step S1980, the control unit 1710 checks whether or not the voice, which is output from the voice output unit 260 in the processing in Step S1980, is a dialogue end voice (Step S1985). This processing is executed by for example checking whether or not the response text, which is received in the processing in Step S1960: Yes, is a predetermined character string (for example, a character string “This ends dialogue.”).

0282In the processing in Step S1985, in the case where the output voice is not a dialogue end voice (Step S1985: No), the device 1700 returns to the processing in Step S1930 to repeat the processing in Step S1930 and the subsequent steps.

0283In the processing in Step S1985, in the case where the output voice is a dialogue end voice (Step S1585: Yes), the control unit 1710 switches the state from the voice input receivable state to the voice input unreceivable state (Step S1990).

0284After the end of the processing in Step S1990, the device 1700 ends the third device processing.

0285The following explains a specific example of the operation performed by the second modified voice dialogue system having the above configuration, with reference to the drawing.

0286<Specific Example>

0287<figref idref="DRAWINGS">FIG. 21</figref> is a procedure diagram schematically showing a situation in which the user of the second modified voice dialogue system performs a voice dialogue with the voice dialogue agent 400 with use of the device 1700 (here, assumed to be a smartphone), and the voice dialogue agent 400 performs processing that reflects the dialogue.

0288Here, the explanation is given based on the assumption that a dialogue end voice is a voice “This ends dialogue.”.

0289In the figure, processing in Step S2100, processing in Step S2105, processing in Step S2115, processing in Step S2135, processing in Step S2155, and processing in Steps S2160-S2170 are respectively the same as the processing in Step S1000, the processing in Step S1005, the processing in Step S1015, the processing in Step S1035, the processing in Step S1055, and the processing in Steps S1060-S1070 in the specific examples in Embodiment 1 (see <figref idref="DRAWINGS">FIG. 10</figref>). Accordingly, the processing in the figure is regarded as having been already explained.

0290After the end of the processing in Step S2105, the device 1700 performs second voice input processing (Step S2110, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0291In the second voice input processing, in the case where the user inputs a voice “What is room temperature?”, the device 1700 transmits input voice data “What is room temperature?” to the voice dialogue agent 400 (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0292After the end of the processing in Step S2115, since the voice “Which room?” is not a dialogue end voice (corresponding to Step S1985: No in <figref idref="DRAWINGS">FIG. 19</figref>), the device 1700 performs second voice input processing (Step S2130, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0293In the second voice input processing, in the case where the user inputs a voice “Living room.”, the device 1700 transmits input voice data “Living room.” to the voice dialogue agent 400 (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0294After the end of the processing in Step S2135, since the voice “Living room temperature is 28 degrees C. Do you need any other help?” is not a dialogue end voice (corresponding to Step S1985: No in <figref idref="DRAWINGS">FIG. 19</figref>), the device 1700 performs second voice input processing (Step S2150, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0295In the second voice input processing, in the case where the user inputs a voice “No. Thank you.”, the device 1700 transmits input voice data “No. Thank you.” to the voice dialogue agent 400 (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0296After the end of the processing in Step S2135, since a voice “This ends dialogue.” is a dialogue end voice (corresponding to Step S1585: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), the state is switched to the voice input receivable state (corresponding to Step S1990 in <figref idref="DRAWINGS">FIG. 19</figref>). The device 1700 ends the third device processing.

0297<Consideration>

0298According to the second modified voice dialogue system having the above configuration, once voice input start operation is performed, the device 1700 keeps in the voice input receivable state for a period from performance of the voice input start operation to output of a dialogue end voice.

0299Accordingly, once the user performs a voice input start operation with respect to the device 1700, the user can newly input a voice without newly performing a voice input operation with respect to the device 1700 until a dialogue end voice is output.

0300According to the second modified voice dialogue system, as described above, a further reduced number of operations need to be performed by the user in accordance with a voice input, compared with the voice dialogue system 100 in Embodiment 1.

Embodiment 4

0301<Outline>

0302The following explains, as one aspect of the voice dialogue method relating to the present invention and one aspect of the device relating to the present invention, a third modified voice dialogue system that is partially modified from the second modified voice dialogue system in Embodiment 3.

0303The second modified voice dialogue system in Embodiment 3 has been explained as an example of the configuration in which once the device 1700 starts communication with a voice dialogue agent A, a voice dialogue agent as a communication party is limited to the voice dialogue agent A until a series of processing ends.

0304Compared with this, the third modified voice dialogue system in Embodiment 4 is an example of configuration in which in the case where a device starts communication with a voice dialogue agent A and a user of the third modified voice dialogue system inputs, with use of the device, a voice indicating that the user hopes to communicate with another voice dialogue agent B, a communication party of the device is changed from the voice dialogue agent A to the voice dialogue agent B.

0305The following explains the details of the third modified voice dialogue system, focusing on different points from the second modified voice dialogue system in Embodiment 3, with reference to the drawings.

0306<Configuration>

0307The third modified voice dialogue system is modified from the second voice dialogue system in Embodiment 3 so as to include a voice dialogue agent 2200 instead of the voice dialogue agent 400.

0308Similarly to the voice dialogue agent 400 in Embodiment 3, the voice dialogue agent 2200 is embodied by the voice dialogue agent server 110.

0309Software for embodying the voice dialogue agent 2200, which is executed by the voice dialogue agent server 110, is partially modified from the software for embodying the voice dialogue agent 400 in Embodiment 3. Accordingly, the voice dialogue agent 2200 is modified from the voice dialogue agent 400 in Embodiment 3 in terms of part of functions.

0310<figref idref="DRAWINGS">FIG. 22</figref> is a block diagram showing functional configuration of the voice dialogue agent 2200.

0311As shown in the figure, the voice dialogue agent 2200 is modified from the voice dialogue agent 400 in Embodiment 3 (see <figref idref="DRAWINGS">FIG. 4</figref>) so as to additionally include a target agent DB storage unit 2220 and include a control unit 2210 instead of the control unit 410.

0312The target agent DB storage unit 2220 is for example embodied by a memory and a processor that executes programs. The target agent DB storage unit 2220 is connected to the control unit 2210, and has a function of storing therein a target agent DB 2300.

0313<figref idref="DRAWINGS">FIG. 23</figref> is a data structure diagram showing the target agent DB 2300 stored in the target agent DB storage unit 2220.

0314As shown in the figure, the target agent DB 2300 includes keyword 2310, target agent 2320, and IP address 2330 that are associated with each other.

0315The keyword 2310 indicates a character string that is assumed to be included in an input text converted by the voice recognition processing unit 430.

0316The target agent 2320 is information for specifying, as a communication party of the device 140, one of a plurality of voice dialogue agents 2200. Hereinafter, this one of the voice dialogue agents 2200 is referred to as an additional voice dialogue agent.

0317In this example, the additional voice dialogue agent specified by the target agent 2320 is a car agent, a retailer agent, or a home agent.

0318Here, the car agent indicates one of voice dialogue agents 2200 that provides a relatively satisfactory service relating to devices in mounted in a car. The retailer agent indicates one of voice dialogue agents 2200 that provides a relatively satisfactory service relating to devices in mounted in a retailer. The home agent indicates one of voice dialogue agents 2200 that provides a relatively satisfactory service relating to devices in mounted in a residence (home).

0319The IP address 2330 indicates an IP address in the network 120 relating to the voice dialogue agent server 110 that embodies an additional voice dialogue agent specified by the associated target agent 2320.

0320As shown in <figref idref="DRAWINGS">FIG. 23</figref>, each of the additional voice dialogue agents specified by the target agent 2320 is associated with one or more character strings indicated by the keyword 2310. For example, the car agent is associated with character strings indicated by the keyword 2310, such as character strings “in-car”, “car”, “vehicle”, and “navigation system”.

0321Since each of the additional voice dialogue agents, which is specified by the target agent 2320, is associated with one or more character strings, which are indicated by the keyword 2310, the voice dialogue agent 2200 can respond an ambiguous input.

0322For example, in the case where the user hopes to communicate with the car agent, the user sometimes inputs a voice “Connect to voice dialogue agent of navigation system.”, and sometimes inputs a voice “Connect to voice dialogue agent of car.”.

0323Here, the character strings indicated by the keyword 2310 “navigation system” and “car” are each associated with the car agent. Accordingly, both in the case where a voice “navigation system” is input and in the case where a voice “car” is input, it is possible to specify the car agent as the additional voice dialogue agent 2200, which is specified by the target agent 2320, by referring to the target agent DB 2300.

0324Referring back to <figref idref="DRAWINGS">FIG. 22</figref>, the explanation on the voice dialogue agent 2200 is continued.

0325The control unit 2210 is modified from the control unit 410 in Embodiment 3 so as to have a second agent processing execution function and a third agent processing execution function, which are described below, instead of the first agent processing execution function of the control unit 410.

0326The second agent processing execution function is a function performed by the control unit 2210 controlling the communication unit 420, the voice recognition processing unit 430, the voice synthesizing processing unit 450, and the instruction generation unit 460 to cause the voice dialogue agent 2200 to execute second agent processing as its characteristic operation to execute a sequence of processing described below. In the sequence of processing, (1) the voice dialogue agent 2200 receives input voice data transmitted from a device, (2) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text to the device, (3) in the case where the generated input text indicates that the user hopes to communicate with another voice dialogue agent, establishes communication between the device and the other voice dialogue agent, (4) otherwise, generates an instruction set based on the generated input text, and executes the generated instruction set, (5) generates a response text based on an execution result of the instruction set, (6) converts the generated response text to response voice data, and (7) returns the response text and the response voice data to the device.

0327Note that the second agent processing is explained in detail in section <Second Agent Processing> later with reference to a flow chart.

0328The third agent processing execution function is a function performed by the control unit 2210 controlling the communication unit 420, the voice recognition processing unit 430, the voice synthesizing processing unit 450, and the instruction generation unit 460 to cause the voice dialogue agent 2200 to execute third agent processing as its characteristic operation to execute a sequence of processing described below. In the sequence of processing, (1) the voice dialogue agent 2200 starts communication with a device in response to a request from another voice dialogue agent, (2) receives input voice data transmitted from the device, (3) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text, (4) generates an instruction set based on the generated input text, and executes the generated instruction set, (5) generates a response text based on an execution result of the instruction set, (6) converts the generated response text to response voice data, and (7) returns the response text and the response voice data to the device.

0329Note that the third agent processing is explained in detail in section <Third Agent Processing> later with reference to a flow chart.

0330The following explains the operation of the third modified voice dialogue system having the above configuration, with reference to the drawings.

0331<Operation>

0332The third modified voice dialogue system performs second agent processing and third agent processing as its characteristic operation, in addition to the first agent processing in Embodiment 1. The second agent processing and the third agent processing is partially modified from the first agent processing in Embodiment 3.

0333Explanation is given on the second agent processing and the third agent processing below, focusing on different points from the first agent processing.

0334<Second Agent Processing>

0335The second agent processing is processing performed by the voice dialogue agent 2200. In the second agent processing, (1) the voice dialogue agent 2200 receives input voice data transmitted from a device, (2) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text to the device, (3) in the case where the generated input text indicates that the user hopes to communicate with another voice dialogue agent, establishes communication between the device and the other voice dialogue agent, (4) otherwise, generates an instruction set based on the generated input text, and executes the generated instruction set, (5) generates a response text based on an execution result of the instruction set, (6) converts the generated response text to response voice data, and (7) returns the response text and the response voice data to the device.

0336<figref idref="DRAWINGS">FIG. 24</figref> is a flow chart of the second agent processing.

0337Upon bootup of the voice dialogue agent 2200, the second agent processing is started.

0338When the second agent processing is started, the voice dialogue agent 2200 stands by until the communication unit 420 receives input voice data transmitted from the device 1700 (Step S2400: Repetition of No). When the communication unit 420 receives the input voice data (Step S2400: Yes), the voice dialogue agent 2200 performs second instruction execution processing (Step S2410).

0339<figref idref="DRAWINGS">FIG. 25</figref> is a flow chart of the second instruction execution processing.

0340In the figure, processing in Steps S2500-S2510 and processing in Steps S2520-S2560 is respectively the same as the processing in Steps S900-S910 and the processing in Steps S920-S960 in the first instruction execution processing in Embodiment 3 (see <figref idref="DRAWINGS">FIG. 9</figref>). Accordingly, the processing in the figure is regarded as having been already explained.

0341After the end of the processing in Step S2510, the control unit 2210 checks whether or not the input text, which is converted by the voice recognition processing unit 430, requests to communicate with another voice dialogue agent (Step S2515).

0342In the processing in Step S2515, in the case where the input text does not request communication with another voice dialogue agent (Step S2515: No), the voice dialogue agent 2200 proceeds to the processing in Step S2520 to perform the processing in Steps S2520-S2560.

0343In the processing in Step S2515, in the case where the input text requests to communicate with another voice dialogue agent (Step S2515: Yes), the control unit 2210 specifies a voice dialogue agent 2200 that is requested as a communication party, with reference to the target agent DB 2300 stored in the target agent DB storage unit 2220 (Step S2517). In other words, the control unit 2210 specifies, as the voice dialogue agent 2200 requested as a communication party, an additional voice dialogue agent that is specified by the target agent 2320 associated with a character string that is indicated by the keyword 2310 included in the input text, which is converted by the voice recognition processing unit 430.

0344After the specification of the additional voice dialogue agent requested as a communication party, the control unit 2210 generates a predetermined signal indicating to start communication between the specified additional voice dialogue agent and the device 1700 which has transmitted the input voice data (Step S2565). Hereinafter, this signal is referred to as a connection instruction.

0345After the generation of the connection instruction, the control unit 2210 controls the communication unit 420 to transmit the generated connection instruction to the additional voice dialogue agent, with use of an IP address indicated by the IP address 2330 which is associated with the character string indicate by the keyword 2310 (Step S2570).

0346Then, the control unit 2210 stands by until the communication unit 420 receives a connection response (described later) that is returned from the additional voice dialogue agent in response to the connection instruction that is transmitted in the processing in Step S2570 (Step S2575: Repetition of No).

0347When the connection response is received by the communication unit 420 (Step S2575: Yes), the voice dialogue agent 2200 executes first connection response processing (Step S2580).

0348<figref idref="DRAWINGS">FIG. 26</figref> is a flow chart of the first connection response processing.

0349When the first connection response processing is started, the control unit 2210 generates a predetermined response text indicating that communication becomes available between the additional voice dialogue agent and the device 1700 (Step S2600). The predetermined response text is for example a character string “Connection to [Additional voice dialogue agent] has been established.”.

0350Here, in part [Additional voice dialogue agent] in the character string, a name of the voice dialogue agent 2200 (here, either of the car agent, the retailer agent, or the home agent), which is specified by the target agent 2320 included in the target agent DB 2300, is inserted.

0351After the generation of the response text, the voice synthesizing processing unit 450 performs voice synthesizing processing on the generated response text to generate response voice data (Step S2610).

0352After the generation of the response voice data, the control unit 2210 controls the communication unit 420 to transmit the generated response text and response voice data to the device 1700 which has transmitted the input voice data (Step S2620).

0353After the end of the processing in Step S2620, the voice dialogue agent 2200 ends the first connection response processing.

0354Referring back to <figref idref="DRAWINGS">FIG. 25</figref>, the explanation on the second instruction execution processing is continued.

0355After the end of the first connection response processing, the voice dialogue agent 2200 stands by until the communication unit 420 receives a disconnection response (described later) that is transmitted from the additional voice dialogue agent (Step S2585: Repetition of No).

0356When the communication unit 420 receives the disconnection response (Step S2585: Yes), the voice dialogue agent 2200 executes disconnection response processing (Step S2590).

0357<figref idref="DRAWINGS">FIG. 27</figref> is a flow chart of the disconnection response processing.

0358When the disconnection response processing is started, the control unit 2210 generates a predetermined response text indicating that the communication ends between the additional voice dialogue agent and the device 1700 (Step S2700). The predetermined response text is for example a character string “Connection to [Additional voice dialogue agent] has been terminated. Do you need any other help?”.

0359Here, in part [Additional voice dialogue agent] in the character string, a name of the voice dialogue agent 2200 (here, either of the car agent, the retailer agent, or the home agent), which is specified by the target agent 2320 included in the target agent DB 2300, is inserted.

0360After the generation of the response text, the voice synthesizing processing unit 450 performs voice synthesizing processing on the generated response text to generate response voice data (Step S2710).

0361After the generation of the response voice data, the control unit 2210 controls the communication unit 420 to transmit the generated response text and response voice data to the device 1700 which has transmitted the input voice data (Step S2720).

0362After the end of the processing in Step S2720, the voice dialogue agent 2200 ends the disconnection response processing.

0363Referring back to <figref idref="DRAWINGS">FIG. 25</figref> again, the explanation on the second instruction execution processing is continued.

0364After the end of the disconnection response processing, or after the end of the processing in Step S2560, the voice dialogue agent 2200 ends the second instruction execution processing.

0365Referring back to <figref idref="DRAWINGS">FIG. 24</figref>, the explanation on the second agent processing is continued.

0366After the end of the second instruction execution processing, the voice dialogue agent 2200 returns to the processing in Step S2400 to perform the processing in Step S2400 and the subsequent steps.

0367<Third Agent Processing>

0368The third agent processing is processing performed by the voice dialogue agent 2200. In the third agent processing, (1) the voice dialogue agent 2200 starts communication with a device in response to a request from another voice dialogue agent, (2) receives input voice data transmitted from the device, (3) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text, (4) generates an instruction set based on the generated input text, and executes the generated instruction set, (5) generates a response text based on an execution result of the instruction set, (6) converts the generated response text to response voice data, and (7) returns the response text and the response voice data to the device.

0369<figref idref="DRAWINGS">FIG. 28</figref> is a flow chart of the third agent processing.

0370In the figure, processing in Steps S2800-S2810 and processing in Steps S2820-S2860 is respectively the same as the processing in Steps S900-S910 and the processing in Steps S920-S960 in the first instruction execution processing in Embodiment 1 (see <figref idref="DRAWINGS">FIG. 9</figref>). Accordingly, the processing in the figure is regarded as having been already explained.

0371Upon bootup of the voice dialogue agent 2200, the third agent processing is started.

0372When the third agent processing is started, the voice dialogue agent 2200 stands by until the communication unit 420 receives a connection instruction transmitted from another voice dialogue agent (Step S2811: Repetition of No). When the communication unit 420 receives the connection instruction (Step S2811: Yes), the control unit 2210 controls the communication unit 420 to execute connection processing of starting communication with the device 1700 that is a communication party requested by the connection instruction.

0373Here, the connection processing includes processing of changing a transmission destination of input voice data to be transmitted from the device 1700 from the voice dialogue agent 2200, which has transmitted the connection instruction, to the voice dialogue agent 2200, which has received the connection instruction.

0374After the execution of the connection processing, the control unit 2210 controls the communication unit 420 to generate a connection response that is a signal indicating that communication with the device 1700 has started, and transmits the generated connection response to the voice dialogue agent which has transmitted the connection instruction (Step S2813).

0375Then, the control unit 2210 stands by until the communication unit 420 receives the input voice data transmitted from the device 1700 (Step S2814: Repetition of No). When the communication unit 420 receives the input voice data (Step S2814: Yes), the control unit 2210 performs the processing in Steps S2800-S2810.

0376After the end of the processing in Step S2810, the control unit 2210 checks whether or not the input text, which is converted by the voice recognition processing unit 430, requests to terminate communication with the voice dialogue agent 2200 (Step S2815).

0377In the processing in Step S2815, in the case where the input text does not indicate to terminate the communication with the voice dialogue agent 2200 (Step S2815: No), the voice dialogue agent 2200 proceeds to the processing in Step S2820 to perform the processing in Steps S2820-S2860. After the end of the processing in Step S2860, the voice dialogue agent 2200 returns to the processing in Step S2814 to perform the processing in Step S2814 and the subsequent steps.

0378In the processing in Step S2815, in the case where the input text indicates to terminate the communication with the voice dialogue agent 2200 (Step S2815: Yes), the control unit 2210 controls the communication unit 420 to execute disconnection processing of terminating the communication with the device 1700.

0379Here, the disconnection processing includes processing of changing the transmission destination of input voice data to be transmitted from the device 1700 from the voice dialogue agent 2200, which has received the connection instruction, to the voice dialogue agent 2200, which has transmitted the connection instruction.

0380After the execution of the disconnection processing, the control unit 2210 controls the communication unit 420 to generate a disconnection response that is a predetermined signal indicating that the communication with the device 1700 has been terminated, and transmits the generated disconnection response to the voice dialogue agent which has transmitted the connection instruction (Step S2890).

0381After the end of the processing in Step S2890, the voice dialogue agent 2200 returns to the processing in Step S2811 to perform the processing in Step S2811 and the subsequent steps.

0382The following explains a specific example of the operation performed by the third modified voice dialogue system having the above configuration, with reference to the drawing.

0383<Specific Example>

0384<figref idref="DRAWINGS">FIG. 29</figref> is a procedure diagram schematically showing a situation in which the user of the third modified voice dialogue system starts, with use of the device 1700, a voice dialogue with a home agent, which is one of the voice dialogue agents 2200, and then starts communication with the car agent, which is one of the voice dialogue agents 2200, in response to a connection instruction generated by the home agent, and performs a dialogue with the car agent.

0385Here, the explanation is given based on the assumption that a specific voice dialogue agent server for the device 1700 used by the user is the voice dialogue agent server 110 that embodies the home agent, and a dialogue end voice is a voice “This ends dialogue.”.

0386In the figure, processing in Steps S2900-S2905 is respectively the same as the processing in Steps S2100-S2105 in the specific example in Embodiment 3 (see <figref idref="DRAWINGS">FIG. 21</figref>). Accordingly, the processing in the figure is regarded as having been already explained.

0387After the end of the processing in Step S2905, the device 1700 performs second voice input processing (Step S2906, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0388In the second voice input processing, in the case where the user inputs a voice “Connect to car agent.”, the device 1700 transmits input voice data “Connect to car agent.” to the home agent (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0389Then, the home agent receives the input voice data (corresponding to Step S2400: Yes in <figref idref="DRAWINGS">FIG. 24</figref>), and performs second instruction execution processing (corresponding to Step S2410 in <figref idref="DRAWINGS">FIG. 24</figref>).

0390In the second instruction execution processing, since the input text requests to communicate with the car agent (corresponding to Step S2515: Yes in <figref idref="DRAWINGS">FIG. 25</figref>), the home agent transmits a connection instruction to the car agent (corresponding to Step S2570 in <figref idref="DRAWINGS">FIG. 25</figref>).

0391Then, the car agent receives the connection instruction (corresponding to Step S2811: Yes in <figref idref="DRAWINGS">FIG. 28</figref>), and starts communication with the device 1700 (corresponding to Step S2812 in <figref idref="DRAWINGS">FIG. 28</figref>), and transmits a connection response to the home agent (Step S2990, corresponding to Step S2813 in <figref idref="DRAWINGS">FIG. 28</figref>).

0392The home agent receives the connection response (corresponding to Step S2575: Yes in <figref idref="DRAWINGS">FIG. 25</figref>), and performs first connection response processing (Step S2965, corresponding to Step S2580 in <figref idref="DRAWINGS">FIG. 25</figref>).

0393Here, in the first connection response processing, in the case where the voice dialogue agent 2200 generates response voice data “Connection to car agent has been established.”, the voice dialogue agent 2200 transmits response voice data “Connection to car agent has been established.” to the device 1700 (corresponding to Step S2620 in <figref idref="DRAWINGS">FIG. 26</figref>).

0394Then, the device 1700 receives the response voice data (corresponding to Step S1960: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), and outputs a voice “Connection to car agent has been established.” (Step S2907, corresponding to Step S1980 in <figref idref="DRAWINGS">FIG. 19</figref>).

0395Since the voice “Connection to car agent has been established.” is not a dialogue end voice (corresponding to Step S1985: No in <figref idref="DRAWINGS">FIG. 19</figref>), the device 1700 performs second voice input processing (Step S2910, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0396In the second voice input processing, in the case where the user inputs a voice “What is temperature in car?”, the device 1700 transmits input voice data “What is temperature in car?” to the car agent (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0397Then, the car agent receives the input voice data (corresponding to Step S2814: Yes in <figref idref="DRAWINGS">FIG. 28</figref>). Since the input voice data does not request to terminate the communication (corresponding to Step S2815: No in <figref idref="DRAWINGS">FIG. 28</figref>), the car agent generates an instruction set corresponding to the input voice data, and executes the generated instruction set (Step S2994, corresponding to Step S2830 in <figref idref="DRAWINGS">FIG. 28</figref>).

0398Here, in execution of the instruction set, in the case where the car agent generates response voice data “Temperature in car is 38 degrees C. Do you need any other help?”, the car agent transmits the response voice data “Temperature in car is 38 degrees C. Do you need any other help?” to the device 1700 (corresponding to Step S2860: Yes in <figref idref="DRAWINGS">FIG. 28</figref>).

0399Then, the device 1700 receives the response voice data (corresponding to Step S1960: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), and outputs a voice “Temperature in car is 38 degrees C. Do you need any other help?” (Step S2915, corresponding to Step S1980 in <figref idref="DRAWINGS">FIG. 19</figref>).

0400Since the voice “Temperature in car is 38 degrees C. Do you need any other help?” is not a dialogue end voice (corresponding to Step S1985: No in <figref idref="DRAWINGS">FIG. 19</figref>), the device 1700 performs second voice input processing (Step S2930, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0401In the second voice input processing, in the case where the user inputs a voice “No. Thank you.”, the device 1700 transmits input voice data “No. Thank you.” to the car agent (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0402Then, the car agent receives the input voice data (corresponding to Step S2814: Yes in <figref idref="DRAWINGS">FIG. 28</figref>). Since the input voice data requests to terminate the communication (corresponding to Step S2815: Yes in <figref idref="DRAWINGS">FIG. 28</figref>), the car agent terminates the communication with the device 1700 (corresponding to Step S2870 in <figref idref="DRAWINGS">FIG. 28</figref>), and transmits a disconnection response to the home agent (Step S2998, corresponding to Step S2890 in <figref idref="DRAWINGS">FIG. 28</figref>).

0403Then, the home agent receives the disconnection response (corresponding to Step S2585: Yes in <figref idref="DRAWINGS">FIG. 25</figref>), and performs disconnection response processing (Step S2970, corresponding to Step S2890 in <figref idref="DRAWINGS">FIG. 25</figref>).

0404Here, in the disconnection processing, in the case where the voice dialogue agent 2200 generates response voice data “Connection to car agent has been terminated. Do you need any other help?”, the voice dialogue agent 2200 transmits the response voice data “Connection to car agent has been terminated. Do you need any other help?” to the device 1700 (corresponding to Step S2720 in <figref idref="DRAWINGS">FIG. 27</figref>).

0405Then, the device 1700 receives the response voice data (corresponding to Step S1960: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), and outputs a voice “Connection to car agent has been terminated. Do you need any other help?” (Step S2935, corresponding to Step S1980 in <figref idref="DRAWINGS">FIG. 19</figref>).

0406Since the voice “Connection to car agent has been terminated. Do you need any other help?” is not a dialogue end voice (corresponding to Step S1985: No in <figref idref="DRAWINGS">FIG. 19</figref>), the device 1700 performs second voice input processing (Step S2950, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0407In the second voice input processing, in the case where the user inputs a voice “No. Thank you.”, the device 1700 transmits input voice data “No. Thank you.” to the home agent (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0408Then, the home agent receives the input voice data (corresponding to Step S2800: Yes in <figref idref="DRAWINGS">FIG. 24</figref>), and performs second instruction execution processing (Step S2975, corresponding to Step S2410 in <figref idref="DRAWINGS">FIG. 24</figref>).

0409Here, in the second instruction execution processing, in the case where the home agent generates response voice data “This ends dialogue.”, the home agent transmits the response voice data “This ends dialogue.” to the device 1700 (corresponding to Step S2560 in <figref idref="DRAWINGS">FIG. 25</figref>).

0410Then, the device 1700 receives the response voice data (corresponding to Step S1960: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), and outputs a voice “This ends dialogue.” (Step S2955, corresponding to Step S1980 in <figref idref="DRAWINGS">FIG. 19</figref>).

0411Since the voice “This ends dialogue.” is a dialogue end voice (corresponding to Step S1585: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), the state is switched to the voice input receivable state (corresponding to Step S1990 in <figref idref="DRAWINGS">FIG. 19</figref>). The device 1700 ends the third device processing.

0412<Consideration>

0413According to the third modified voice dialogue system having the above configuration, in the case where the user of the third modified voice dialogue system, who is communicating with the voice dialogue agent A, hopes to cause the voice dialogue agent B rather than the voice dialogue agent A to perform processing, it is possible to change the voice dialogue agent that is appropriate for performing the processing via communication from the voice dialogue agent A to the voice dialogue agent B to cause the voice dialogue agent B to perform desired processing.

0414Also, in this case, since the voice dialogue agent A transfers input voice data that is not modified to the voice dialogue agent B, the voice dialogue agent B performs voice recognition processing on the input voice data. As a result, the user can receive a more appropriate service from the voice dialogue agent B.

Embodiment 5

0415<Outline>

0416The following explains, as one aspect of the voice dialogue method relating to the present invention and one aspect of the device relating to the present invention, a fourth modified voice dialogue system that is partially modified from the third modified voice dialogue system in Embodiment 4.

0417The third modified voice dialogue system in Embodiment 4 has been explained as an example of the configuration in which in the case where a device starts communication with the voice dialogue agent A and the user of the third modified voice dialogue system inputs, with use of the device, a voice indicating that the user hopes to communicate with another voice dialogue agent B, a communication party of the device is changed from the voice dialogue agent A to the voice dialogue agent B.

0418Compared with this, the fourth modified voice dialogue system in Embodiment 5 is an example of configuration in which in the case where a device starts communication with a voice dialogue agent A and predetermined condition is satisfied for the communication, the voice dialogue agent A determines that the voice dialogue agent B rather than the voice dialogue agent A is appropriate as a communication party, and a communication party of the device is changed from the voice dialogue agent A to the voice dialogue agent B.

0419The following explains the details of the fourth modified voice dialogue system, focusing on different points from the third modified voice dialogue system in Embodiment 4, with reference to the drawings.

0420<Configuration>

0421The fourth modified voice dialogue system is modified from the third voice dialogue system in Embodiment 4 so as to include a voice dialogue agent 3000 instead of the voice dialogue agent 2200.

0422Similarly to the voice dialogue agent 2200 in Embodiment 4, the voice dialogue agent 3000 is embodied by the voice dialogue agent server 110.

0423Software for embodying the voice dialogue agent 3000, which is executed by the voice dialogue agent server 110, is partially modified from the software for embodying the voice dialogue agent 2200 in Embodiment 3. Accordingly, the voice dialogue agent 3000 is modified from the voice dialogue agent 2200 in Embodiment 4 in terms of part of functions.

0424<figref idref="DRAWINGS">FIG. 30</figref> is a block diagram showing functional configuration of the voice dialogue agent 3000.

0425As shown in the figure, the voice dialogue agent 3000 is modified from the voice dialogue agent 2200 in Embodiment 4 (see <figref idref="DRAWINGS">FIG. 22</figref>) so as not to include the target agent DB storage unit 2220, and so as to additionally include an available service DB storage unit 3020 and include a control unit 3010 instead of the control unit 2210.

0426The available service DB storage unit 3020 is for example embodied by a memory and a processor that executes programs. The available service DB storage unit 3020 is connected to the control unit 3010, and has a function of storing therein an available service DB 3100.

0427<figref idref="DRAWINGS">FIG. 31</figref> is a data structure diagram showing the available service DB 3100 stored in the available service DB storage unit 3020.

0428As shown in the figure, the available service DB 3100 includes keyword 3110, target agent 3120, processing details 3130, IP address 3140, and availability 3150 that are associated with each other.

0429The keyword 3110 indicates a character string that is assumed to be included in an input text converted by the voice recognition processing unit 430.

0430The target agent 3120 is information for specifying an additional voice dialogue agent as a communication party of the device 1700.

0431In this example, the additional voice dialogue agents specified by the target agent 2320 include the car agent, the retailer agent, and the home agent, similarly to Embodiment 4.

0432The processing details 3130 are information for specifying, in the case where a character string indicated by the associated keyword 3110 is included in an input text that is converted by the voice recognition processing unit 430, processing that is determined to be executed by a device that is specified by the associated target device 3120.

0433The IP address 3140 indicates an IP address in the network 120 relating to the voice dialogue agent server 110 that embodies the additional voice dialogue agent specified by the associated target agent 3120.

0434The availability 3150 is information for specifying whether or not the voice dialogue agent can perform processing specified by the associated processing details 3130.

0435Referring back to <figref idref="DRAWINGS">FIG. 30</figref>, the explanation on the voice dialogue agent 3000 is continued.

0436The control unit 3010 is modified from the control unit 2210 in Embodiment 4 so as to have a fourth agent processing execution function, which is described below, instead of the second agent processing execution function of the control unit 2210.

0437The fourth agent processing execution function is a function performed by the control unit 3010 controlling the communication unit 420, the voice recognition processing unit 430, the voice synthesizing processing unit 450, and the instruction generation unit 460 to control the voice dialogue agent 3000 to execute the fourth agent processing, which is its characteristic operation, to execute a sequence of processing described below. In the sequence of processing, (1) the voice dialogue agent 3000 receives input voice data transmitted from a device, (2) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text to the device, (3) in the case where the generated input text includes a predetermined keyword, establishes communication between the device and a target agent associated with the predetermined keyword, (4) otherwise, generates an instruction set based on the generated input text, and executes the generated instruction set, (5) generates a response text based on an execution result of the instruction set, (6) converts the generated response text to response voice data, and (7) returns the response text and the response voice data to the device.

0438Note that the fourth agent processing is explained in detail in section <Fourth Agent Processing> later with reference to a flow chart.

0439The following explains the operation of the fourth modified voice dialogue system having the above configuration, with reference to the drawings.

0440<Operation>

0441The fourth modified voice dialogue system performs fourth agent processing as its characteristic operation, in addition to the second device processing and the third agent processing in Embodiment 4. The fourth agent processing is partially modified from the second agent processing in Embodiment 3.

0442Explanation is given on the fourth agent processing below, focusing on different points from the second agent processing.

0443<Fourth Agent Processing>

0444The fourth agent processing is processing performed by the voice dialogue agent 3000. In the fourth agent processing, (1) the voice dialogue agent 3000 receives input voice data transmitted from a device, (2) performs voice recognition processing on the received input voice data to generate an input text, and returns the generated input text to the device, (3) in the case where the generated input text includes a predetermined keyword, establishes communication between the device and a target agent associated with the predetermined keyword, (4) otherwise, generates an instruction set based on the generated input text, and executes the generated instruction set, (5) generates a response text based on an execution result of the instruction set, (6) converts the generated response text to response voice data, and (7) returns the response text and the response voice data to the device.

0445<figref idref="DRAWINGS">FIG. 32</figref> is a flow chart of the fourth agent processing.

0446Upon bootup of the voice dialogue agent 3000, the fourth agent processing is started.

0447When the fourth agent processing is started, the voice dialogue agent 3000 stands by until the communication unit 420 receives input voice data transmitted from the device 1700 (Step S3200: Repetition of No). When the communication unit 430 receives the input voice data (Step S3200: Yes), the voice dialogue agent 3000 performs second instruction execution processing (Step S3210).

0448<figref idref="DRAWINGS">FIG. 33</figref> is a flow chart of the third instruction execution processing.

0449In the figure, processing in Steps S3300-S3310, processing in Steps S3320-S3360, processing in Steps S3365-S3375, and processing in Steps S3385-S3390 are respectively the same as the processing in Steps S2500-S2510, the processing in Steps S2520-S2560, the processing in Steps S2565-S2575, and the processing in Steps S2585-S2590 in Embodiment 4. Accordingly, the processing in the figure is regarded as having been already explained.

0450After the end of the processing in Step S3310, the control unit 3010 refers to the available service DB 3100 stored in the available service DB storage unit 3020 (Step S3312) to determine whether or not another voice dialogue agent is appropriate for performing processing corresponding to the input text data (Step S3315). In other words, in the case where the input text data includes a character string indicated by the keyword 3110 and an additional voice dialogue agent specified by the target agent 3120 associated with the keyword 3110 is not the voice dialogue agent 3000 which is currently performing the third instruction execution processing, the control unit 3010 determines that the other voice dialogue agent (another additional voice dialogue agent specified by the target agent 3120) is appropriate for performing the processing. Otherwise, the control unit 3010 determines that the other voice dialogue agent is not appropriate for performing the processing.

0451In the processing in Step S3315, in the case where the control unit 3010 determines that the other voice dialogue agent is not appropriate for performing the processing (Step S3315: No), the voice dialogue agent 3000 proceeds to the processing in Step S3320 to perform the processing in Steps S3320-S3360.

0452In the processing in Step S3315, in the case where the control unit 3010 determines that the other voice dialogue agent is appropriate for performing the processing (Step S3315: Yes), the voice dialogue agent 3000 proceeds to the processing in Step S3365 to perform the processing in Steps S3365-S3375.

0453In the processing in Step S3375, when the communication unit 420 receives the connection response returned from the additional voice dialogue agent (Step S3375: Yes), the voice dialogue agent 3000 performs second connection response processing (Step S3380).

0454<figref idref="DRAWINGS">FIG. 34</figref> is a flow chart of the second connection response processing.

0455When the second connection response processing is started, the control unit 3010 controls the communication unit 420 to transfer the input voice data, which is received in the processing in Step S3200: Yes, to the additional voice dialogue agent, which is specified by the processing in Step S3315: Yes (Step S3400).

0456After the end of the processing in Step S3400, the voice dialogue agent 3000 ends the second connection response processing.

0457Referring back to <figref idref="DRAWINGS">FIG. 33</figref>, the explanation on the second instruction execution processing is continued.

0458After the end of the second connection response processing, the voice dialogue agent 3000 proceeds to Step S3385 to perform the processing in Steps S3385-S3390.

0459After the end of the processing in Step S3390, or after the end of the processing in Step S3360, the voice dialogue agent 3000 ends the third instruction execution processing.

0460Referring back to <figref idref="DRAWINGS">FIG. 32</figref>, the explanation on the fourth agent processing is continued.

0461After the end of the third instruction execution processing, the voice dialogue agent 3000 returns to the processing in Step S3200 to perform the processing in Step S3200 and the subsequent steps.

0462The following explains a specific example of the operation performed by the fourth modified voice dialogue system having the above configuration, with reference to the drawing.

0463<Specific Example>

0464<figref idref="DRAWINGS">FIG. 35</figref> is a procedure diagram schematically showing a situation in which the user of the fourth modified voice dialogue system starts, with use of the device 1700, a voice dialogue with the home agent, which is one of the voice dialogue agents 3000, and then starts communication with the car agent in response to a connection instruction generated by the home agent, and performs a dialogue with the car agent.

0465Here, the explanation is given based on the assumption that a specific voice dialogue agent server for the device 1700 used by the user is the voice dialogue agent server 110 that embodies the home agent, and a dialogue end voice is a voice “This ends dialogue.”.

0466In the figure, processing in Steps S3500-S3505 is respectively the same as the processing in Steps S2900-S2905 in the specific example in Embodiment 4 (see <figref idref="DRAWINGS">FIG. 29</figref>). Accordingly, the processing in the figure is regarded as having been already explained.

0467After the end of the processing in Step S3505, the device 1700 performs second voice input processing (Step S3506, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0468In the second voice input processing, in the case where the user inputs a voice “What is temperature in car?”, the device 1700 transmits input voice data “What is temperature in car?” to the home agent (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0469Then, the home agent receives the input voice data (corresponding to Step S3200: Yes in <figref idref="DRAWINGS">FIG. 32</figref>), and performs third instruction execution processing (corresponding to Step S3210 in <figref idref="DRAWINGS">FIG. 32</figref>).

0470In the third instruction execution processing, since the input text includes keywords “temperature” and “in-car” and an additional voice dialogue agent specified by the target agent 3120 is not the home agent (corresponding to Step S3315: No in <figref idref="DRAWINGS">FIG. 33</figref>), the home agent transmits a connection instruction to the car agent (corresponding to Step S3370 in <figref idref="DRAWINGS">FIG. 33</figref>).

0471Then, the car agent receives the connection instruction (corresponding to Step S2811: Yes in <figref idref="DRAWINGS">FIG. 28</figref>), and starts communication with the device 1700 (corresponding to Step S2812 in <figref idref="DRAWINGS">FIG. 28</figref>), and transmits a connection response to the home agent (Step S3590, corresponding to Step S2813 in <figref idref="DRAWINGS">FIG. 28</figref>).

0472The home agent receives the connection response (corresponding to Step S3375: Yes in <figref idref="DRAWINGS">FIG. 33</figref>), and performs second connection response processing (corresponding to Step S3380 in <figref idref="DRAWINGS">FIG. 33</figref>).

0473In the second connection response processing, the home agent transmits input voice data “What is temperature in car?” to the car agent (corresponding to Step S3400 in <figref idref="DRAWINGS">FIG. 34</figref>).

0474Then, the car agent receives the input voice data (corresponding to Step S2814: Yes in <figref idref="DRAWINGS">FIG. 28</figref>). Since the input voice data does not request to terminate the communication (corresponding to Step S2815: No in <figref idref="DRAWINGS">FIG. 28</figref>), the car agent generates an instruction set corresponding to the input voice data, and executes the generated instruction set (Step S3594, corresponding to Step S2830 in <figref idref="DRAWINGS">FIG. 28</figref>).

0475Here, in execution of the instruction set, in the case where the car agent generates response voice data “Temperature in car is 38 degrees C. Do you need any other help?”, the car agent transmits the response voice data “Temperature in car is 38 degrees C. Do you need any other help?” to the device 1700 (corresponding to Step S2860: Yes in <figref idref="DRAWINGS">FIG. 28</figref>).

0476Then, the device 1700 receives the response voice data (corresponding to Step S1960: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), and outputs a voice “Temperature in car is 38 degrees C. Do you need any other help?” (Step S3507, corresponding to Step S1980 in <figref idref="DRAWINGS">FIG. 19</figref>).

0477Since the voice “Temperature in car is 38 degrees C. Do you need any other help?” is not a dialogue end voice (corresponding to Step S1985: No in <figref idref="DRAWINGS">FIG. 19</figref>), the device 1700 performs second voice input processing (Step S3510, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0478In the second voice input processing, in the case where the user inputs a voice “Turn on air conditioner with 25 degrees C. of temperature setting.”, the device 1700 transmits input voice data “Turn on air conditioner with 25 degrees C. of temperature setting.” to the car agent (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0479Then, the car agent receives the input voice data (corresponding to Step S2814: Yes in <figref idref="DRAWINGS">FIG. 28</figref>). Since the input voice data does not request to terminate the communication (corresponding to Step S2815: No in <figref idref="DRAWINGS">FIG. 28</figref>), the car agent generates an instruction set corresponding to the input voice data, and executes the generated instruction set (Step S3594, corresponding to Step S2830 in <figref idref="DRAWINGS">FIG. 28</figref>).

0480Here, in execution of the instruction set, in the case where the car agent generates response voice data “Air conditioner is turned on with 25 degrees C. of temperature setting. Do you need any other help?”, the car agent transmits the response voice data “Air conditioner is turned on with 25 degrees C. of temperature setting. Do you need any other help?” to the device 1700 (corresponding to Step S2860 in <figref idref="DRAWINGS">FIG. 28</figref>).

0481Then, the device 1700 receives the response voice data (corresponding to Step S1960: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), and outputs a voice “Air conditioner is turned on with 25 degrees C. of temperature setting. Do you need any other help?” (Step S3525, corresponding to Step S1980 in <figref idref="DRAWINGS">FIG. 19</figref>).

0482Since the voice “Air conditioner is turned on with 25 degrees C. of temperature setting. Do you need any other help?” is not a dialogue end voice (corresponding to Step S1985: No in <figref idref="DRAWINGS">FIG. 19</figref>), the device 1700 performs second voice input processing (Step S3530, corresponding to Step S1930 in <figref idref="DRAWINGS">FIG. 19</figref>).

0483In the second voice input processing, in the case where the user inputs a voice “No. Thank you.”, the device 1700 transmits input voice data “No. Thank you.” to the car agent (corresponding to Step S2040 in <figref idref="DRAWINGS">FIG. 20</figref>).

0484Then, the car agent receives the input voice data (corresponding to Step S2814: Yes in <figref idref="DRAWINGS">FIG. 28</figref>). Since the input voice data requests to terminate the communication (corresponding to Step S2815: Yes in <figref idref="DRAWINGS">FIG. 28</figref>), the car agent terminates the communication with the device 1700 (corresponding to Step S2870 in <figref idref="DRAWINGS">FIG. 28</figref>), and transmits a disconnection response to the home agent (Step S3598, corresponding to Step S2890 in <figref idref="DRAWINGS">FIG. 28</figref>).

0485Then, the home agent receives the disconnection response (corresponding to Step S2585: Yes in <figref idref="DRAWINGS">FIG. 25</figref>), and performs disconnection response processing (Step S2970, corresponding to Step S2890 in <figref idref="DRAWINGS">FIG. 25</figref>).

0486Here, in the disconnection processing, in the case where the voice dialogue agent 2200 generates response voice data “This ends dialogue.”, the voice dialogue agent 2200 transmits the response voice data “This ends dialogue.” to the device 1700 (corresponding to Step S2720 in <figref idref="DRAWINGS">FIG. 27</figref>).

0487Then, the device 1700 receives the response voice data (corresponding to Step S1960: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), and outputs a voice “This ends dialogue.” (Step S3555, corresponding to Step S1980 in <figref idref="DRAWINGS">FIG. 19</figref>).

0488Since the voice “This ends dialogue.” is a dialogue end voice (corresponding to Step S1985: Yes in <figref idref="DRAWINGS">FIG. 19</figref>), the state is switched to the voice input receivable state (corresponding to Step S1990 in <figref idref="DRAWINGS">FIG. 19</figref>). The device 1700 ends the fourth device processing.

0489<Consideration>

0490According to the fourth modified voice dialogue system having the above configuration, in the case where the voice dialogue agent A determines that the voice dialogue agent B rather than the voice dialogue agent A is appropriate as a communication party of the user while the user of the fourth modified voice dialogue system communicates with the voice dialogue agent A, it is possible to change a voice dialogue agent as the communication party of the user from the voice dialogue agent A to the voice dialogue agent B.

0491With this configuration, even if the user does not know the type of service provided by each of the voice dialogue agents, the user can receive a service provided by a more appropriate voice dialogue agent.

0492Also, in this case, since the voice dialogue agent A transfers input voice data that is not modified to the voice dialogue agent B, the voice dialogue agent B performs voice recognition processing on the input voice data. As a result, the user can receive a more appropriate service from the voice dialogue agent B.

Embodiment 6

0493The following exemplifies an operation situation of the voice dialogue system 100 in Embodiment 1. Note that the voice dialogue system 100 in Embodiment 1 may be of course operated in an operation situation other than the operation situation exemplified here.

0494<figref idref="DRAWINGS">FIG. 36A</figref> is a diagram schematically showing an operation situation in which the voice dialogue system 100 in Embodiment 1 is operated.

0495In <figref idref="DRAWINGS">FIG. 36A</figref>, a group 3600 is for example a company, an organization, or a family, and its size is not limited. A plurality of devices 3601 (devices A and B and so on) and a home gateway 3602 are disposed in the group 3600. The devices 3601 include not only devices that are connectable to the Internet (for example, a smartphone, a PC, and a TV) but also devices that are disconnectable from the Internet by themselves (for example, an illumination lamp, a washing machine, a refrigerator). The devices 3601 may include devices that are disconnectable from the Internet by themselves but are connectable to the Internet via the home gateway 3602. Also, the group 3600 includes a user 10 who uses the devices 3601. For example, the devices which are disposed in the group 3600 each correspond to the device 140 in Embodiment 1.

0496A cloud server 3611 is disposed in a data center administration company 3610. The cloud server 3611 is a virtual server that cooperates with various devices through the Internet. The cloud server 3611 mainly manages big data that is difficult to deal with by a normal data base management tool or the like. The data center administration company 3610 performs management of data and the cloud server 3611, and administers a data center for performing such management. Services performed by the data center administration company 3610 are described in detail later. Here, the data center administration company 3610 is not limited to a company only performing data management, administration of the cloud server 3611, and so on. For example, a device manufacturer developing and manufacturing one type of the devices 3601 may serve as the data administration center 3610 when the device manufacturer also performs data management and administration of the cloud server 3611 (see <figref idref="DRAWINGS">FIG. 36B</figref>). Also, the data center administration company 3610 does not need to be a single company. For example, when a device manufacturer and another management company perform data management and administration of the cloud server 3611 together, then either one or both of the device manufacturer and the management company may serve as the data center administration company 3610 (see <figref idref="DRAWINGS">FIG. 36C</figref>). For example, the data center administration company 3610 provides the voice dialogue agent 400 that is associated with the device 140 (hereinafter, referred to also as a first voice dialogue agent).

0497A service provider 3620 has a server 3621. The server 3621 here for example includes a memory embedded in a PC for individual use, and its size is not limited. Also, there is a case where the service provider 3620 does not have the server 3621. For example, the service provider 3620 provides another voice dialogue agent 400 that is connected to the first voice dialogue agent (hereinafter, referred to also as a second voice dialogue agent).

0498Next, an explanation is given on a flow of information in the above operation situation.

0499First, the device A or B, which is disposed in the group 3600, transmits log information to the cloud server 3611, which is disposed in the data center administration company 3610. The cloud server 3611 accumulates the log information transmitted from the device A or B (arrow (a) in <figref idref="DRAWINGS">FIG. 36A</figref>). Here, the log information is information indicating a driving situation, an operation time and date, and so on of the devices 3601. The log information includes for example a viewing history of a TV, timer recording information of a recorder, a driving time and date and a laundry amount of a washing machine, and a time and date and the number of opening and closing a refrigerator. Without limiting to the information described above, the log information includes all information that is acquirable from all the devices 3601. There is a case where the log information is provided directly from the devices 3601 to the cloud server 3611 through the Internet. Alternatively, the log information may be provided from the home gateway 3602 to the cloud server 3611 after being accumulated from the devices 3601 to the home gateway 3602.

0500Next, the cloud server 3611, which is disposed in the data center administration company 3610, provides the accumulated log information to the service provider 3620 in certain units. Here, the log information may be provided in units according to which the data center administration company 3610 can organize the accumulated log information and provide the organized log information to the service provider 3620. Alternatively, the log information may be provided in units requested by the service provider 3620. Moreover, the log information may not be provided in certain units, and alternatively an amount of the log information to be provided sometimes varies in accordance with circumstances. The log information is stored as necessary in the server 3621 of the service provider 3620 (arrow (b) in <figref idref="DRAWINGS">FIG. 36A</figref>). Then, the service provider 3620 organizes the log information so as to be adapted to a service to be provided to a user, and provides the organized information to the user. The user to which the organized information to be is provided may be the user 10 who uses the devices 3601 or an external user 20. The service may be provided for example from the service provider 3620 directly to the user (arrow (e) in <figref idref="DRAWINGS">FIG. 36A</figref>). Alternatively, the service may be provided for example to the user again via the cloud server 3611 of the data center administration company 3610 (arrows (c) and (d) in <figref idref="DRAWINGS">FIG. 36A</figref>). Moreover, the cloud server 3611 of the data center administration company 3610 may organize the log information so as to be adapted to a service to be provided to the user, and provide the organized information to the service provider 3620.

0501Note that the user 10 and the user 20 may be different or the same.

0502The following exemplifies several types of service that can be provided in the above operation situation.

0503<Service Type 1: Local Data Center Type>

0504<figref idref="DRAWINGS">FIG. 37</figref> is a diagram schematically showing service type 1 (local data center type service).

0505Here, the service provider 3620 acquires information from the group 3600, and provides a service to a user. In this type of service, the service provider 3620 has functions of a data center administration company. That is, the service provider 3620 includes a cloud server 3611 performing big data management. As such, there is no data center administration company.

0506In this type of service, the service provider 3620 administers and manages the data center (the cloud server 3611) (3703). Also, the service provider 3620 manages an OS (3702) and an application (3701). The service provider 3620 performs service provision (3704) with use of the OS (3702) and application (3701), which are managed by thereby.

0507<Service Type 2: IaaS Type>

0508<figref idref="DRAWINGS">FIG. 38</figref> is a diagram schematically showing service type 2 (IaaS (Infrastructure as a Service) type). Here, IaaS is a model in which infrastructure for constructing and operating a computer system is provided as a cloud service through the Internet.

0509In this type of service, the data center administration company 3610 administers and manages the data center (the cloud server 3611) (3703). Further, the service provider 3620 manages the OS (3702) and the application (3701). The service provider 3620 performs service provision (3704) with use of the OS (3702) and the application (3701), which are managed thereby.

0510<Service Type 3: PaaS Type>

0511<figref idref="DRAWINGS">FIG. 39</figref> is a diagram schematically showing service type 3 (PaaS (Platform as a Service) type). Here, PaaS is a model in which a platform for constructing and operating software is provided as a service through the Internet.

0512In this type of service, the data center administration company 3610 manages the OS (3702), and administers and manages the data center (the cloud server 3611) (3703). Further, the service provider 3620 manages the application (3701). The service provider 3620 performs service provision (3704) with use of the OS (3702), which is managed by the data center administration company 3610, and the application (3701), which is managed by the service provider 3620.

0513<Service Type 4: SaaS Type>

0514<figref idref="DRAWINGS">FIG. 40</figref> is a diagram schematically showing service type 4 (SaaS (Software as a Service) type). In this model, for example, an application that is provided by a platform provider having a data center (a cloud server) is provided to a business or a person (a user) without having a data center (a cloud server) as a cloud service through a network such as the Internet.

0515In this type of service, the data center administration company 3610 manages the application (3701), manages the OS (3702), and administers and manages the data center (the cloud server 3611) (3703). Further, the service provider 3620 performs service provision (3704) with use of the application (3701) and the OS (3702), which are managed by the data center administration company 3610.

0516The main actor in service provision is the service provider 3620 in all of the above service types. Further, for example, the service provider 3620 or the data center administration company 3610 may develop their own OS, application, or big data database, or may outsource any of these to a third party.

0000<Supplement>

0517One aspect of the voice dialogue method relating to the present invention and one aspect of the device relating to the present invention have been explained by exemplifying the five voice dialogue systems in Embodiments 1 to 5 and the operation situation of the voice dialogue system in Embodiment 6. However, the voice dialogue method and the device relating to the present invention are not of course limited to the voice dialogue method and the device as used in the voice dialogue system and the operation situation which are exemplified in Embodiments 1 to 6.

0518(1) In Embodiment 1, the voice dialogue system 100 has been explained to include the voice dialogue agent server 110, the network 120, the gateway 130, and the device 140 as shown in <figref idref="DRAWINGS">FIG. 1</figref>. A voice dialogue system as another example may include a mediation server 4150 in addition to the voice dialogue agent server 110, the network 120, the gateway 130, and the device 140. The mediation server 4150 has a function of storing therein the target agent DB 2300, associating between the voice dialogue agents, switching a connection destination, and so on.

0519<figref idref="DRAWINGS">FIG. 41</figref> is a system configuration diagram showing configuration of a voice dialogue system 4100 that includes the mediation server 4150.

0520<figref idref="DRAWINGS">FIG. 42</figref> is a block diagram showing functional configuration of the mediation server 4150.

0521As shown in the figure, the mediation server 4150 includes a communication unit 4220, a control unit 4210, and a target agent DB storage unit 4230.

0522Here, the target agent DB storage unit 4230 has a function of storing therein the target agent DB 2300, similarly to the target agent DB storage unit 2220 in Embodiment 4.

0523Also, a voice dialogue system as further another example may include a mediation server 4350 instead of the mediation server 4150. The mediation server 4350 has a function of storing therein the available service DB 3100, associating between the voice dialogue agents, switching a connection destination, and so on.

0524<figref idref="DRAWINGS">FIG. 43</figref> is a block diagram showing functional configuration of the mediation server 4350.

0525As shown in the figure, the mediation server 4350 includes a communication unit 4320, a control unit 4310, and an available service DB storage unit 4330.

0526Here, the available service DB storage unit 4330 has a function of storing therein the available service DB 3100, similarly to the available service DB storage unit 3020 in Embodiment 5.

0527(2) In Embodiment 1, the image shown in <figref idref="DRAWINGS">FIG. 12</figref> is exemplified as an image displayed on the display unit 270 included in the device 140.

0528Another examples of this image are shown in <figref idref="DRAWINGS">FIG. 44A</figref> to <figref idref="DRAWINGS">FIG. 44D</figref>, <figref idref="DRAWINGS">FIG. 45A</figref>, and <figref idref="DRAWINGS">FIG. 45B</figref>.

0529In the examples in <figref idref="DRAWINGS">FIG. 12</figref>, <figref idref="DRAWINGS">FIG. 44A</figref> to <figref idref="DRAWINGS">FIG. 44D</figref>, and <figref idref="DRAWINGS">FIG. 45B</figref>, displayed response texts each include, at the beginning thereof, a character string specifying a subject outputting a voice such as “You”, “Car agent”, “Home agent”, or the like. Also, in the example in <figref idref="DRAWINGS">FIG. 45A</figref>, an icon (image) specifying a subject outputting a voice is displayed.

0530In the examples in <figref idref="DRAWINGS">FIG. 44A</figref> and <figref idref="DRAWINGS">FIG. 44B</figref>, a character string specifying a voice dialogue agent with which the user currently makes a dialogue is displayed on an upper part of the screen such that the user recognizes the voice dialogue agent with which the user currently makes a dialogue. Such a character strings displayed here are “Dialogue with home agent” and “Dialogue with car agent”.

0531In the example in <figref idref="DRAWINGS">FIG. 44D</figref>, a character string specifying a voice dialogue agent with which the user currently makes a dialogue (or has made a dialogue in the past) is included in each of the displayed response texts, such that the user recognizes the voice dialogue agent with which the user currently makes a dialogue (or has made a dialogue in the past). Such a character strings displayed here are “Dialogue party is home agent” and “Dialogue party is car agent”. Also, in the example in <figref idref="DRAWINGS">FIG. 45B</figref>, an icon (image) specifying a voice dialogue agent with which the user currently makes a dialogue (or has made a dialogue in the past) is displayed.

0532These display examples are just examples. Alternatively, a voice dialogue agent with which the user currently makes a dialogue may be indicated by color, shape of the screen, shape of part of the screen, or the like. Furthermore, each subject outputting a voice may be indicated by changing a background color, a wall paper, and the like on the display. In this way, it is only necessary to display a voice dialogue agent with which the user makes a dialogue or a subject outputting a voice so as to be recognizable by the user.

0533(3) In Embodiment 1 and the modifications, the example has been explained that a voice dialogue agent with which the user makes a dialogue or a subject outputting a voice is displayed so as to be visually recognizable by the user. However, the present invention is not necessarily limited to the example where a voice dialogue agent with which the user makes a dialogue or a subject outputting a voice is displayed so as to be visually recognizable by the user, as long as the voice dialogue agent with which the user makes a dialogue or the subject outputting a voice is recognizable by the user.

0534For example, a voice “Dialogue party is home agent” may be output, such that a voice dialogue agent with which the user makes a dialogue is recognizable by the user. Alternatively, a sound effect may be output, such that the voice dialogue agent with which the user makes a dialogue is recognizable by the user. Further alternatively, the voice dialogue agent with which the user makes a dialogue may be indicated by changing voice tone, speech rate, voice volume, or the like.

0535(4) In Embodiment 1, the explanation has been provided that the state is managed by the control unit 210 in the form as shown in the switching of the state shown in <figref idref="DRAWINGS">FIG. 3</figref>. Also, in Embodiment 2, the explanation has been provided that the state is managed by the control unit 1310 in the form as shown in the switching of the state shown in <figref idref="DRAWINGS">FIG. 14</figref>. Furthermore, in Embodiment 3, the explanation has been provided that the state is managed by the control unit 1710 in the form as shown in the switching of the state shown in <figref idref="DRAWINGS">FIG. 18</figref>.

0536Management of the state performed by the control unit is not limited to be in the above forms. Alternatively, other forms for managing the state may be employed. <figref idref="DRAWINGS">FIG. 46</figref> to <figref idref="DRAWINGS">FIG. 50</figref> each show an example of switching of the state managed by the control unit in other forms.

0537For example, according to management of the state in a form shown in switching of the state in <figref idref="DRAWINGS">FIG. 48</figref>, in the case where a voice output by the voice output unit 260 based on a response text transmitted from the voice dialogue agent 110 is a dialogue end voice, the state is switched to the voice input unreceivable state even if the predetermined period has not lapsed after the switching of the state to the voice input receivable state. Accordingly, in the case where a voice signal, which is generated by the voice dialogue agent 110 and received by the communication unit 250, indicates unnecessity of a new voice input, the voice input unit 220 is switched to the voice input unreceivable state even if the predetermined period has not lapsed after the switching to the voice input receivable state.

0538(5) In Embodiment 1, the display unit 270 has been explained, for example, as being embodied by a touchpanel, a touchpanel controller, and a processor that executes programs, and having the configuration of displaying that the display unit 270 is in the voice input receivable state by blinking the region 1120 that is positioned at the lower right in the display unit 270 (see <figref idref="DRAWINGS">FIG. 11A</figref>, <figref idref="DRAWINGS">FIG. 11C</figref>, <figref idref="DRAWINGS">FIG. 12</figref>, and so on). However, the configuration of the display unit 270 is not limited to the above configuration example as long as the user can recognize that the display unit 270 is in the voice input receivable state. Another configuration example may be employed in which the display unit 270 is embodied by an LED (Light Emitting Diode) and a processor that executes programs, and displays that the display unit 270 is in the voice input receivable state by lighting the LED. In the other configuration example, the display unit 270 does not display a response text received by the communication unit 250 because of not including means for displaying character strings.

0539(6) In Embodiment 1, the explanation has been provided that the communication unit 250 has the configuration in which in the case where a specific one of the voice dialogue agent servers 110 is not designated as a voice dialogue agent server 110 that is a communication party, the communication unit 250 communicates with a specific voice dialogue agent server with reference to an IP address stored in the address storage unit 240. Alternatively, another configuration example may be employed in which the address storage unit 240 does not store therein the IP address of the specific voice dialogue agent server, and the communication unit 250 communicates with a voice dialogue agent server designated by the user or a voice dialogue agent server that embodies the voice dialogue agent designated by the user.

0540(7) In Embodiment 1, the devices 140 each have been explained as communicating with the voice dialogue agent 110 via the gateway 130 and the network 120.

0541Alternatively, another configuration may be employed in which the devices 140 may each have a function of directly connecting with the network 120 without the gateway 130 and communicate with the voice dialogue agent without the gateway 130. In the case where all the devices 140 are directly connected to the network 120 without the gateway 130, the gateway 130 is not necessary.

0542(8) Part or all of the elements constituting the above embodiments and modifications may be configured from a single system LSI. The system LSI is a super multifunctional LSI that is manufactured by integrating a plurality of components on a single chip. Specifically, the system LSI is a computer system composed of a microprocessor, a ROM, a RAM, and so on. Functions of the system LSI are achieved by the microprocessor operating in accordance with a computer program that is stored in the ROM, the RAM, or the like.

0543(9) Part or all of the elements constituting the above embodiments and modifications may be composed of an IC (Integrated Circuit) card detachable from a device or a module. The IC card or the module is a computer system composed of a microprocessor, a ROM, a RAM, and so on. The IC card or the module may include the above super multifunctional LSI. Functions of the IC card or the module are achieved by the microprocessor operating in accordance with a computer program that is stored in the ROM, the RAM, or the like. The IC card or the module may be each tamper-resistant.

0544(10) The computer program or the digital signal which is used in the above embodiments and modifications may be recorded in a computer-readable recording medium such as a flexible disk, a hard disk, a CD-ROM, an MD, a DVD, a DVD-ROM, a DVD-RAM, a BD, a semiconductor memory, or the like.

0545Also, the computer program or the digital signal which is used in the above embodiments and modifications may be transmitted through an electric communication network, a wireless or wired communication network, a network such as the Internet, data broadcasting, or the like.

0546The computer program or the digital signal which is used in the above embodiments and modifications can be implemented in another computer system, by transmitting the computer program or the digital signal which is recorded in the recording medium to the other computer system, or by transmitting the computer program or the digital signal to the other computer system via the network.

0547(12) The above embodiments and modifications may be combined with each other.

0548(13) The following further explains configurations, modifications, and effects of the voice dialogue method and the device relating to one aspect of the present invention.

0549(a) One aspect of the present invention provides a voice dialogue method that is performed by a voice dialogue system, the voice dialogue system including: a voice signal generation unit; a voice dialogue agent unit; a voice output unit; and a voice input control unit, the voice dialogue method comprising: a step of, by the voice signal generation unit, receiving a voice input and generating a voice signal based on the received voice input; a step of, by the voice dialogue agent unit, performing voice recognition processing on the generated voice signal and performing processing based on a result of the voice recognition processing to generate a response signal; a step of, by the voice output unit, outputting a voice based on the generated response signal; and a step of, when the voice output unit outputs the voice, by the voice input control unit, keeping the voice signal generation unit in a receivable state for a predetermined period after output of the voice, the receivable state being a state in which a voice input is receivable.

0550According to the voice dialogue method relating to one aspect of the present invention, in the case where a voice generated by the voice dialogue agent unit is output, a user can input a voice without performing an operation with respect to the voice dialogue system. This reduces the number of times that the user needs to perform an operation in accordance with a voice that is dialogically input, compared with conventional techniques.

0551(b) Also, the voice dialogue system may further include a display unit, and the voice dialogue method may further comprise a step of, while the voice signal generation unit is in the receivable state, by the display unit, displaying that the voice signal generation unit is in the receivable state.

0552This configuration allows the user to visually recognize whether or not the voice signal generation unit is in the receivable state.

0553(c) Also, the voice dialogue system may further include an additional voice dialogue agent unit, and the voice dialogue method may further comprise: a step of, by the voice dialogue agent unit, determining, based on the result of the voice recognition processing, which one of the voice dialogue agent unit and the additional voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing; a step of, when the voice dialogue agent unit determines that the voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing, by the voice dialogue agent unit, performing the processing based on the result of the voice recognition processing; a step of, when the voice dialogue agent unit determines that the additional voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing, by the additional voice dialogue agent unit, performing voice recognition processing on a voice received by the voice signal generation unit, performing processing based on a result of the voice recognition processing performed by the additional voice dialogue agent unit to generate a response signal; and a step of, by the voice output unit, outputting a voice based on the response signal generated by the additional voice dialogue agent unit.

0554According to this configuration, it is possible to cause the additional voice dialogue agent unit to perform processing that is appropriate for being performed by the additional voice dialogue agent unit rather than the voice dialogue agent unit.

0555(d) Also, the voice dialogue method may further comprise: a step of, when the voice dialogue agent unit determines that the voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing, by the display unit, displaying that the voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing; and a step of, when the voice dialogue agent unit determines that the additional voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing, by the display unit, displaying that the additional voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing.

0556This configuration allows the user to visually recognize which one of the voice dialogue agent unit and the additional voice dialogue agent unit is appropriate for performing the processing.

0557(e) Also, the voice dialogue method may further comprise a step of, when the voice dialogue agent unit determines that the additional voice dialogue agent unit is appropriate for performing the processing based on the result of the voice recognition processing, by the voice dialogue agent unit, transferring a voice signal generated by the voice signal generation unit to the additional voice dialogue agent unit, and by the additional voice dialogue agent unit, performing voice recognition processing on the transferred voice signal.

0558This configuration allows the additional voice dialogue agent unit to perform the voice recognition processing with use of the voice signal transferred from the voice dialogue agent unit.

0559(f) Also, the voice dialogue method may further comprise a step of, when the voice signal generation unit is in the receivable state and a response signal generated by the voice dialogue agent unit indicates that a new voice input does not need to be received, by the voice input control unit, switching the voice signal generation unit to an unreceivable state even during the predetermined period, the unreceivable state being a state in which a voice input is unreceivable.

0560According to this configuration, in the case where a voice input does not need to be received, it is possible to switch the voice signal generation unit to the unreceivable state even during the predetermined period.

0561(g) One aspect of the present invention provides a device comprising: a voice signal generation unit configured to receive a voice input and generate a voice signal based on the received voice input; a transmission unit configured to transmit the generated voice signal to an external server: a reception unit configured to receive a response signal that is returned from the server, the response signal being generated by the server based on the voice signal; a voice output unit configured to output a voice based on the received response signal; and a voice input control unit configured to, when the voice output unit outputs a voice, keep the voice signal generation unit in a receivable state for a predetermined period after output of the voice, the receivable state being a state in which a voice input is receivable.

0562According to the device relating to the one aspect of the present invention, in the case where a voice generated by the server is output, the user can input a voice without performing an operation with respect to the device. This reduces the number of times that the user needs to perform an operation in accordance with a voice that is dialogically input, compared with a conventional technique.

INDUSTRIAL APPLICABILITY

0563The voice dialogue method and the device relating to the present invention are widely utilizable for a voice dialogue system that performs processing based on a voice that is dialogically input by a user.

REFERENCE SIGNS LIST

0564100 voice dialogue system

0565110 voice dialogue agent server

0566120 network

0567130 gateway

0568140 device

0569210 control unit

0570220 voice input unit

0571230 operation reception unit

0572240 address storage unit

0573250 communication unit

0574260 voice output unit

0575270 display unit

0576280 execution unit

0577400 voice dialogue agent

0578410 control unit

0579420 communication unit

0580430 voice recognition processing unit

0581440 dialogue DB storage unit

0582450 voice synthesizing processing unit

0583460 instruction generation unit

Contents8

Every citation, both ways

Citations
Document	Relation	Office	Cited during
CN101558443A	Cites	China	Applicant
CN101689366A	Cites	China	Applicant
US10181099B2	Cites	United States of America	Search report
US10546441B2	Cites	United States of America	Search report
US10798244B2	Cites	United States of America	Search report
EP1154406A1	Cites	European Patent Office (EPO)	Search report
EP1591979A1	Cites	European Patent Office (EPO)	Search report
JP2001056225A	Cites	Japan	Applicant
US2002046023A1	Cites	United States of America	Applicant
JP2002116797A	Cites	Japan	Applicant
US2003163309A1	Cites	United States of America	Applicant
JP2003241797A	Cites	Japan	Applicant
US2004044516A1	Cites	United States of America	Applicant
JP2004233794A	Cites	Japan	Applicant
JP2004240150A	Cites	Japan	Applicant
US2005033582A1	Cites	United States of America	Search report
US2005256712A1	Cites	United States of America	Search report
JP2005266192A	Cites	Japan	Applicant
JP2006178175A	Cites	Japan	Applicant
US2007265831A1	Cites	United States of America	Search report
JP2008090545A	Cites	Japan	Applicant
US2008208584A1	Cites	United States of America	Search report
WO2009145796A1	Cites	World Intellectual Property Organization (WIPO)	Applicant
US2009150156A1	Cites	United States of America	Applicant
US2010076751A1	Cites	United States of America	Applicant
US2011172994A1	Cites	United States of America	Search report
US2011208525A1	Cites	United States of America	Applicant
JP2011232619A	Cites	Japan	Applicant
US2012221502A1	Cites	United States of America	Search report
JP2013114020A	Cites	Japan	Applicant
US2014095173A1	Cites	United States of America	Search report
US2014304365A1	Cites	United States of America	Search report
US2020105082A1	Cites	United States of America	Search report
US2020193748A1	Cites	United States of America	Search report
US6229880B1	Cites	United States of America	Search report
US6249720B1	Cites	United States of America	Applicant
US6480599B1	Cites	United States of America	Search report
US6636831B1	Cites	United States of America	Search report
US7003079B1	Cites	United States of America	Search report
US7039166B1	Cites	United States of America	Search report
US7117051B2	Cites	United States of America	Search report
US7177402B2	Cites	United States of America	Search report
US7228275B1	Cites	United States of America	Search report
US7460652B2	Cites	United States of America	Search report
US7903792B2	Cites	United States of America	Search report
US8150020B1	Cites	United States of America	Search report
US9263058B2	Cites	United States of America	Search report
US9558745B2	Cites	United States of America	Search report
US9564129B2	Cites	United States of America	Search report
JPH1137766A	Cites	Japan	Applicant
US20020046023A1	Cites	United States of America	Applicant
US20030163309A1	Cites	United States of America	Applicant
US20040044516A1	Cites	United States of America	Applicant
US20050033582A1	Cites	United States of America	Search report
US20050256712A1	Cites	United States of America	Search report
US20070265831A1	Cites	United States of America	Search report
US20080208584A1	Cites	United States of America	Search report
US20090150156A1	Cites	United States of America	Applicant
US20100076751A1	Cites	United States of America	Applicant
US20110172994A1	Cites	United States of America	Search report
US20110208525A1	Cites	United States of America	Applicant
US20120221502A1	Cites	United States of America	Search report
US20140095173A1	Cites	United States of America	Search report
US20140304365A1	Cites	United States of America	Search report
US20200105082A1	Cites	United States of America	Search report
US20200193748A1	Cites	United States of America	Search report
CN101558443	Cites	China	Applicant
CN101689366	Cites	China	Applicant
JP1137766	Cites	Japan	Applicant
JP200156225	Cites	Japan	Applicant
JP2002116797	Cites	Japan	Applicant
JP2003241797	Cites	Japan	Applicant
JP2004233794	Cites	Japan	Applicant
JP2004240150	Cites	Japan	Applicant
JP2005266192	Cites	Japan	Applicant
JP2006178175	Cites	Japan	Applicant
JP200890545	Cites	Japan	Applicant
JP2011232619	Cites	Japan	Applicant
JP2013114020	Cites	Japan	Applicant
WO2009145796	Cites	World Intellectual Property Organization (WIPO)	Applicant
International Search Report dated Sep. 9, 2014 in corresponding International Application No. PCT/JP2014/003097 (with English translation).	Non-patent	–	Applicant
Extended European Search Report dated Jun. 1, 2016 in European Application No. 14814417.3.	Non-patent	–	Applicant
Office Action issued Jun. 1, 2018 in Chinese Application No. 201480021678.6 (with English translation of Search Report).	Non-patent	–	Applicant
Lin et al., (1999). “A distributed architecture for cooperative spoken dialogue agents with coherent dialogue state and history.” In: Proc. Workshop Automatic Speech Recognition and Understanding.	Non-patent	–	Search report
Lin et al., (1999). “A distributed architecture for cooperative spoken dialogue agents with coherent dialogue state and history.” In: Proc. Workshop Automatic Speech Recognition and Understanding.	Non-patent	–	Search report
International Search Report dated Sep. 9, 2014 in corresponding International Application No. PCT/JP2014/003097 (with English translation).	Non-patent	–	Applicant
Extended European Search Report dated Jun. 1, 2016 in European Application No. 14814417.3.	Non-patent	–	Applicant
Office Action issued Jun. 1, 2018 in Chinese Application No. 201480021678.6 (with English translation of Search Report).	Non-patent	–	Applicant

16 members in 5 offices

This recordMember, by publication datePriority claim

Priority claims14

Priority claims
Document	Office	Kind	Date
201361836763	United States of America	P	2013-06-19
201361836763	United States of America	P	2013-06-19
2014003097	Japan	W	2014-06-10
2014003097	Japan	W	2014-06-10
201414777920	United States of America	A	2014-06-10
201414777920	United States of America	A	2014-06-10
201416268938	United States of America	A	2014-06-10
14777920	–	–	–
61836763	–	–	–
PCTJP2014003097	–	–	–
US201361836763P	–	–	–
US201414777920	–	–	–
US201416268938	–	–	–
WO2014JP03097	–	–	–

Members16

Family members, oldest first
Document	Office	Kind
WO2014203495A1	World Intellectual Property Organization (WIPO)	A1	2014-12-24
CN105144285A	China	A	2015-12-09
EP3012833A1	European Patent Office (EPO)	A1	2016-04-27
EP3012833A4	European Patent Office (EPO)	A4	2016-06-29
US2016322048A1	United States of America	A1	2016-11-03
US9564129B2	United States of America	B2	2017-02-07
JPWO2014203495A1	Japan	A1	2017-02-23
JP6389171B2	Japan	B2	2018-09-12
CN105144285B	China	B	2018-11-02
CN108806690A	China	A	2018-11-13
JP2018189984A	Japan	A	2018-11-29
JP6736617B2	Japan	B2	2020-08-05
JP2020173477A	Japan	A	2020-10-22
USRE49014EThis record	United States of America	E	2022-04-05
JP7072610B2	Japan	B2	2022-05-20
EP3012833B1	European Patent Office (EPO)	B1	2022-08-10

91 transactions on the USPTO file

Allowed after 2 non-final rejections, 2 final rejections and 2 RCEs.

Non-final rejections: 2
Final rejections: 2
RCEs: 2
Appeals: 0

Over time

Point at a mark for the transaction

Transactions

Prosecution transactions, newest first
	Event	Code
2024-07-19	Payment of Maintenance Fee, 8th Year, Large EntityM1552	M1552
2022-04-05	Recordation of Patent Grant MailedPGM/	PGM/
2022-03-17	Email NotificationEML_NTR	EML_NTR
2022-03-16	Issue Notification MailedAllowedWPIR	WPIR
2022-03-09	Dispatch to FDCD1935	D1935
2022-03-09	Application Is Considered Ready for IssuePILS	PILS
2022-03-03	Issue Fee Payment VerifiedN084	N084
2022-03-03	Issue Fee Payment ReceivedIFEE	IFEE
2022-01-20	Email NotificationEML_NTR	EML_NTR
2022-01-20	Mailing Corrected Notice of AllowabilityMCNOA	MCNOA
2022-01-13	Examiner's Amendment CommunicationEX.A	EX.A
2022-01-13	Reasons for AllowanceEX.R	EX.R
2022-01-13	Corrected Notice of AllowabilityCNOA	CNOA
2022-01-04	Pubs Case Remand to TCPUBTC	PUBTC
2021-12-21	Electronic ReviewELC_RVW	ELC_RVW
2021-12-21	Email NotificationEML_NTF	EML_NTF
2021-12-21	Mail Notice of AllowanceAllowedMN/=.	MN/=.
2021-12-17	Notice of Allowance Data Verification CompletedAllowedN/=.	N/=.
2021-12-16	Reasons for AllowanceEX.R	EX.R
2021-05-12	Disposal for a RCE / CPA / R129AbandonedABN9	ABN9
2021-05-12	Date Forwarded to ExaminerFWDX	FWDX
2021-05-10	Request for Continued Examination (RCE)RCEX	RCEX
2021-05-10	Workflow - Request for RCE - BeginBRCE	BRCE
2021-04-27	Email NotificationEML_NTR	EML_NTR
2021-04-27	Mail Advisory Action (PTOL - 303)MCTAV	MCTAV
2021-04-22	Advisory Action (PTOL-303)CTAV	CTAV
2021-04-13	Date Forwarded to ExaminerFWDX	FWDX
2021-04-09	Response after Final ActionA.NE	A.NE
2021-04-09	PILOT- Request for After Final Consideration ProgramRAFC	RAFC
2021-02-10	Electronic ReviewELC_RVW	ELC_RVW
2021-02-10	Email NotificationEML_NTF	EML_NTF
2021-02-10	Mail Final Rejection (PTOL - 326)Final rejectionMCTFR	MCTFR
2021-02-02	Final RejectionFinal rejectionCTFR	CTFR
2020-12-14	Date Forwarded to ExaminerFWDX	FWDX
2020-12-11	Response after Non-Final ActionA...	A...
2020-11-24	Email NotificationEML_NTR	EML_NTR
2020-11-24	Mail Examiner Interview Summary (PTOL - 413)MEXIN	MEXIN
2020-11-19	Interview Summary - Applicant Initiated - TelephonicEXAT	EXAT
2020-11-19	Interview Summary RecordEXIN	EXIN
2020-10-30	Electronic request for Examiner InterviewM865E	M865E
2020-09-18	Electronic ReviewELC_RVW	ELC_RVW
2020-09-18	Email NotificationEML_NTF	EML_NTF
2020-09-18	Mail Non-Final RejectionNon-final rejectionMCTNF	MCTNF
2020-09-10	Non-Final RejectionNon-final rejectionCTNF	CTNF
2020-07-15	Disposal for a RCE / CPA / R129AbandonedABN9	ABN9
2020-07-15	Date Forwarded to ExaminerFWDX	FWDX
2020-07-14	Request for Continued Examination (RCE)RCEX	RCEX
2020-07-14	Workflow - Request for RCE - BeginBRCE	BRCE
2020-07-02	Email NotificationEML_NTR	EML_NTR
2020-07-02	Mail Advisory Action (PTOL - 303)MCTAV	MCTAV
2020-06-26	Advisory Action (PTOL-303)CTAV	CTAV
2020-06-22	Date Forwarded to ExaminerFWDX	FWDX
2020-06-19	Response after Final ActionA.NE	A.NE
2020-06-19	PILOT- Request for After Final Consideration ProgramRAFC	RAFC
2020-04-20	Electronic ReviewELC_RVW	ELC_RVW
2020-04-20	Email NotificationEML_NTF	EML_NTF
2020-04-20	Mail Final Rejection (PTOL - 326)Final rejectionMCTFR	MCTFR
2020-04-10	Final RejectionFinal rejectionCTFR	CTFR
2020-04-07	Information Disclosure Statement consideredIDSC	IDSC
2020-04-01	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2020-02-19	Date Forwarded to ExaminerFWDX	FWDX
2020-02-14	Response after Non-Final ActionA...	A...
2020-02-10	Mail Interview Summary - Applicant Initiated - TelephonicMEXAT	MEXAT
2020-02-04	Interview Summary - Applicant Initiated - TelephonicEXAT	EXAT
2020-01-28	Electronic request for Examiner InterviewM865E	M865E
2019-11-22	Mail Post CardPST_CRD	PST_CRD
2019-11-14	Email NotificationEML_NTF	EML_NTF
2019-11-14	Mail Non-Final RejectionNon-final rejectionMCTNF	MCTNF
2019-11-08	Non-Final RejectionNon-final rejectionCTNF	CTNF
2019-11-08	Information Disclosure Statement consideredIDSC	IDSC
2019-11-08	Information Disclosure Statement consideredIDSC	IDSC
2019-08-23	Information Disclosure Statement (IDS) FiledM844	M844
2019-08-23	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2019-04-30	Case Docketed to Examiner in GAUDOCK	DOCK
2019-03-19	Notice of Reissue Published in Official GazetteNRE.	NRE.
2019-03-18	Paralegal Reissue Review CompletePRIR	PRIR
2019-02-22	Case Docketed to Examiner in GAUDOCK	DOCK
2019-02-22	Case Docketed to Examiner in GAUDOCK	DOCK
2019-02-22	Email NotificationEML_NTR	EML_NTR
2019-02-22	Application ready for PDX access by participating foreign officesCCRDY	CCRDY
2019-02-22	Application Is Now CompleteCOMP	COMP
2019-02-22	Filing ReceiptFLRCPT.O	FLRCPT.O
2019-02-21	Application Dispatched from OIPEOIPE	OIPE
2019-02-21	FITF set to YES - revise initial settingFTFS	FTFS
2019-02-07	Cleared by OIPE CSRL194	L194
2019-02-06	Information Disclosure Statement (IDS) FiledM844	M844
2019-02-06	PTO/SB/69-Authorize EPO Access to Search ResultsSREXR141	SREXR141
2019-02-06	Applicants have given acceptable permission for participating foreignAPPERMS	APPERMS
2019-02-06	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2019-02-06	Entity Status Set To Undiscounted (Initial Default Setting or Status Change)BIG.	BIG.
2019-02-06	Initial Exam Team nnIEXX	IEXX

2 legal events, as the office reported them to INPADOC

Over the term

Point at a mark for the event

Events

Legal events, newest first
	Event	Code
2024-07-19	Maintenance fee paymentMAFP	MAFP
2019-02-06	Fee payment procedureENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP	FEPP