Agent device

The present invention relates to an agent device, and relates to, for example, an agent device having a communication function capable of having a conversation in a vehicle with an anthropomorphic agent.

Conventionally, a radio and a cassette tape player are installed in a vehicle to improve the driving environment for the driver. In addition, there are also vehicles that improve the driving environment by enjoying conversations with acquaintances outside the vehicle by using wireless communication devices such as amateur radios and mobile phones mounted on the vehicle.

Patent Document 1 presents a technique for transmitting information to a driver by means of human facial expressions and movements.

<patcit num="1"><text>Japanese Patent Application Laid-Open No. 9-102098</text></patcit>

<p> Radios and the like in conventional vehicles only present information to the driver in one direction, and cannot have two-way conversations. On the other hand, in the case of using a mobile phone or the like, it is possible to have a conversation, but it is necessary to search for a call partner by waiting for a call or dialing. Even if the other party was found, it did not give an appropriate conversation according to the driver's one-sided convenience such as the situation of the vehicle. In this way, conventional vehicles do not have anthropomorphic agents according to the history of the vehicle such as the past state and the state of the driver, so it is just a tool as a vehicle that the vehicle does not attach to. In some cases, it only had a role.</p><p> Further, in the technique described in Patent Document 1, the display does not change depending on the history of the driver's response in the past, and the same display is always displayed when the same situation occurs. That is, the same display is always performed for a limited sensor output, and it should fall into the category of conventional instruments with improved visibility.</p><p> An object of the present invention is to provide a vehicle capable of anthropomorphic agents performing actions (actions and voices) according to a situation and communicating with a driver.</p>

<p>(1) An agent device that displays an image of an anthropomorphic agent that acts according to the situation of the vehicle on an image display device, and an image storage means that stores a plurality of images corresponding to the action of the agent, and the above. Corresponding to the display device that displays the image of the agent, the voice output means that outputs the sound of the agent, and the proposal of turning on the radio by the agent, the image of the corresponding agent is read out from the image storage means. It is characterized by including an image sound control means for displaying on the display device and outputting the sound to be proposed from the sound output means. (2) In the invention according to claim 2, in the agent device according to claim 1, when the response acquisition means for acquiring the driver's response to the proposal and the acquired response are the responses received. It is characterized by comprising a radio control means for turning on the power of the radio. (3) In the invention according to claim 3, in the agent device according to claim 2, the image / voice control means obtains an image of the corresponding agent from the image storage means as an agent's reply to the acquired response. It is characterized in that it is read out and displayed on the display device, and the voice of the reply is output from the voice output means.</p>

<p> According to the present invention, it is possible to communicate in response to a proposal by an agent to turn on the radio.</p>

Hereinafter, preferred embodiments of the agent device of the present invention will be described in detail with reference to FIGS. 1 to 11. (1) Outline of the Embodiment In the agent apparatus of the present embodiment, an anthropomorphic agent is made to appear in the vehicle by an image (planar image, three-dimensional image such as holography, etc.). Then, the situation of the vehicle including the vehicle itself, the driver, the passenger, the oncoming vehicle, etc. is judged and learned (including not only the learning of the situation but also the response and reaction of the driver), and the vehicle situation at each time point is determined. Based on the learning results up to that point, the agent responds to the driver and vehicle with various variations (action = action and voice). As a result, the driver can interact (communicate) with his / her own agent in the vehicle, and can make the environment in the vehicle comfortable. Here, the agent that appears in the vehicle is a pseudo-personalized (virtual personalized) subject that judges and learns in the same way as a human being. Therefore, even in the same vehicle situation, the content of the agent's communication differs depending on the past learning content and the like. Occasionally, there is a judgment error within the range not related to the running of the vehicle, and an unnecessary (dumb) response may be made due to this judgment error. Then, based on the driver's response, it is determined whether or not there is a judgment error, and learning is performed. The appearance displayed as an agent does not have to be a human appearance, for example, the appearance of an animal itself such as a chick, a dog, a cat, a frog, or a mouse, or a humanly stylized (illustrated) animal. It may be in appearance, or it may be in a robot-like appearance. In addition, the age of the agent does not have to be constant, and as a learning function of the agent, the appearance of the child is initially set, and then the appearance changes as it grows over time (changes to the appearance of an adult and then to the elderly). It may change to the appearance of). The appearance and voice of the agent can be selected from a plurality of appearances and voices.

(2) Details of the Embodiment FIG. 1 is a block diagram showing a configuration of an agent device in the present embodiment. In the present embodiment, the overall processing unit 1 that controls the entire communication function is provided. This overall processing unit searches for the route to the set destination and guides it by voice or image display 10. The overall processing unit learns the vehicle conditions and past responses by the driver, and makes appropriate conversations and controls. Agent processing unit 11 to perform, I / F unit 12 for navigation processing unit 10 and agent processing unit 11, image processing unit 13 to process image output and input images such as agent images and map images, agent voice, route guidance voice, etc. It has a voice control unit 14 that controls voice output and input voice, and a status information processing unit 15 that processes detection data of various situations related to the vehicle and the driver.

The navigation processing unit 10 and the agent processing unit 11 are a CPU (central processing unit) that controls data processing and the operation of each unit, and ROM, RAM, and timer connected to this CPU by a bus line such as a data bus or a control bus. Etc. are provided. Both processing units 10 and 11 are connected to a network so that they can acquire each other's processing data. ROM is a read-only memory in which various data and programs for control by the CPU are stored in advance, and RAM is a random access memory used by the CPU as a working memory.

In the navigation processing unit 10 and the agent processing unit 11 of the present embodiment, the CPU reads various programs stored in the ROM and executes various processes. The CPU reads a computer program from an external recording medium set in the recording medium drive device 23, and stores (installs) it in another storage device such as an agent storage device 29, a navigation data storage device, or a hard disk (not shown). , A necessary program or the like may be read (loaded) from this storage device into RAM and executed. Further, a necessary program or the like may be directly read into the RAM from the recording medium driving device 23 and executed.

The current position detection device 21 and the navigation data storage device 30 are connected to the navigation processing unit 10, the agent data storage device 29 is connected to the agent processing unit 11, and the input device 22 and the storage medium are connected to the I / F unit 12. The drive device 23 and the communication control device 24 are connected, the display device 27 and the image pickup device 28 are connected to the image processing unit 13, the voice processing device 25 and the microphone 26 are connected to the voice control unit 14, and the situation information processing unit is connected. A situation sensor unit 40 is connected to 15.

The current position detection device 21 is for detecting the absolute position (depending on latitude and longitude) of the vehicle, and has a GPS (Global Positioning System) receiver 211 that measures the position of the vehicle using an artificial satellite and an orientation. A sensor 212, a steering angle sensor 213, a distance sensor 214, a beacon receiving device 215 that receives position information from a beacon arranged on the road, and the like are used. The GPS receiver 211 and the beacon receiver 215 can measure the position independently, but in places where reception by the GPS receiver 211 or the beacon receiver 215 is not possible, both the orientation sensor 212 and the distance sensor 214 are used. The current position is detected by speculative navigation. The orientation sensor 212 includes, for example, a geomagnetic sensor that detects geomagnetism and obtains the orientation of the vehicle, a gyro such as a gas rate gyro or an optical fiber gyro that detects the rotational angular velocity of the vehicle and integrates the angular velocity to obtain the orientation of the vehicle, left and right. Wheel sensors, etc. are used in which the wheel sensors of the above are arranged and the displacement amount of the orientation is calculated by detecting the turning of the vehicle by the output pulse difference (difference in the moving distance). The steering angle sensor 213 detects the steering angle α by using an optical rotation sensor attached to the rotating portion of the steering, a rotation resistance volume, or the like. As the distance sensor 214, various methods are used, for example, one that detects and counts the number of rotations of a wheel, or detects an acceleration and integrates it twice.

The input device 22 is a predetermined driving environment (transmission condition) of a vehicle that wants to send a request for information such as congestion information to the current location (departure point) and destination (arrival point) at the start of travel in the navigation process, and the information providing station. This is for inputting the type (model) of the mobile phone 6. The input device 22 is also one means for the driver to respond to an agent inquiry or the like according to the present embodiment. As the input device 22, various devices such as a touch panel (functioning as a switch), a keyboard, a mouse, a light pen, a joystick, a remote controller using infrared rays, and a voice recognition device can be used. Further, a remote controller using infrared rays or the like and a receiving unit for receiving various signals transmitted from the remote controller may be provided. On the remote controller, various keys such as a menu specification key (button) and a numeric keypad are arranged in addition to a joystick for moving the cursor displayed on the screen.

The recording medium driving device 23 is a driving device used for reading a computer program for the navigation processing unit 10 and the agent processing unit 11 to perform various processes from an external recording medium. The computer program recorded on the recording medium includes various programs, data, and the like. Here, the recording medium means a recording medium on which a computer program is recorded, and specifically, a magnetic recording medium such as a flexible disk, a hard disk, or a magnetic tape, a semiconductor recording medium such as a memory chip or an IC card, or a CD-. Recording media such as ROM, MO, PD (phase change rewritable optical disc) that can read information optically, paper cards and paper tapes, and paper (and paper) such as printed matter for reading programs using a character recognition device. A recording medium using a medium having a corresponding function) and a recording medium on which a computer program is recorded by various other methods are included.

In addition to reading computer programs from these various recording media, the recording medium drive device 23 also performs navigation processing unit 10 and agent processing when the recording medium is a writable recording medium such as a flexible disk or an IC card. It is possible to write the RAM of the unit 11 and the data of the storage devices 29 and 30 to the recording medium. For example, you can store the learning content (learning item data, response data) related to the agent function in the IC card, and even when driving another vehicle, you can use this stored IC card to suit your taste (past). It becomes possible to communicate with the learned agent (depending on the situation of the reception). This makes it possible for an agent unique to the driver to appear in the vehicle instead of an agent for each vehicle.

A mobile phone made of various wireless communication devices is connected to the communication control device 24. The communication control unit 24 is used for communication with an information providing station that provides data on traffic information such as road congestion and traffic regulation, and karaoke data used for communication karaoke in a car, in addition to telephone line calls. It is possible to communicate with the information providing station that provides the information. It is also possible to send and receive learning data related to the agent function via the communication control device 24.

The voice output device 25 is composed of a plurality of speakers arranged in the vehicle, and is composed of voices controlled by the voice control unit 14, for example, guidance voices when route guidance is performed by voices, voices according to the actions of agents, and voices. Sound is output. The audio output device 25 may also be used as an audio speaker. The voice control device 14 can control the tone color, accent, and the like of the voice output from the output device 25 in response to the input of the tuning instruction of the driver. The microphone 26 is used as a voice input means for inputting / outputting the voice to be recognized by the voice control unit 14, for example, the input voice of the destination in the navigation process, the driver's conversation (response, etc.) with the agent, and the like. Function. The microphone 26 may also be used as a microphone for performing karaoke such as online karaoke, and a dedicated directional microphone is used to accurately collect the driver's voice. May be good. A hands-free unit may be formed by the voice output device 25 and the microphone 26 so that a telephone communication can be made without going through a mobile phone.

The display device 27 displays a road map and various image information for route guidance by the processing of the navigation processing unit 10, and displays various actions (videos) of the agent by the agent processing unit 11. .. Further, the images inside and outside the vehicle captured by the image pickup apparatus 28 are also displayed after being processed by the image processing unit 13. As the display device 27, various display devices such as a liquid crystal display device and a CRT are used. The display device 5 may have a function as the input device 2, such as a touch panel.

The image pickup device 28 is composed of a camera equipped with a CCD (charge-coupled device) for capturing an image, and captures the front, rear, right side, and left side of the vehicle in addition to the in-vehicle camera that captures the driver. Each outside camera is arranged. The image captured by each camera of the image pickup apparatus 28 is supplied to the image processing unit 13, processing such as image recognition is performed, and each recognition result is used by the agent processing unit 11 to determine the program number. There is.

The agent data storage device 29 is a storage device that stores various data (including programs) necessary for realizing the agent function according to the present embodiment. For the agent data storage device 29, for example, various recording media such as a flexible disk, a hard disk, a CD-ROM, an optical disk, a magnetic tape, an IC card, and an optical card, and a driving device thereof are used. In this case, for example, the learning item data 292 and the response data are composed of an easy-to-carry IC card or flexible disk, and the other data is composed of a hard disk. , Those driving devices may be used as the driving device.

The agent data storage device 29 stores the agent program 290, the program selection table 291, the learning item data 292, the response data, the image data 294 for displaying the appearance and behavior of the agent illustrated in FIG. 4, and other data. Has been done. The learning item data 292 and the response data 293 are data that store the results learned by the agent according to the driver's driving operation and response. Therefore, the learning item data 292 and the response data 293 are stored / updated (learned) for each driver. The appearance stored in the image data 294 does not have to be a human (male, female) appearance, for example, the appearance of the animal itself such as a chick, a dog, a cat, a frog, a mouse, or a human design ( It may be the appearance of an animal (illustrated), or it may be a robot-like appearance, the appearance of a specific character, or the like. In addition, the age of the agent does not have to be constant, and as a learning function of the agent, the appearance of the child is initially set, and then the appearance changes as it grows over time (changes to the appearance of an adult and then to the elderly). It may change to the appearance of). Images of the appearance of these various agents are stored in the image data 294, and can be selected from the input device 22 and the like according to the driver's preference.

In the agent program 290, an agent processing program for realizing the agent function and a detailed action when the agent and the driver communicate with each other are displayed on the display device 27 as an image, and a conversation corresponding to the action is output as a voice output device. Communication programs for output from 25 are stored in order of program number. The agent program 290 stores a plurality of types of voice data for the voice of each program number, and the driver can select the voice from the input device 22 or the like in addition to selecting the appearance of the agent. It has become like. Agent voices include male voices, female voices, children's voices, mechanical voices, animal voices, voices of specific voice actors and actors, voices of specific characters, etc. The driver chooses. The voice and the selection of the appearance can be changed in a timely manner.

The program selection table 291 is a table for selecting a communication program stored in the agent program 290. FIG. 2 shows the program selection table 291 and FIG. 3 shows the contents of the agent's actions (actions and vocalizations) corresponding to each program number selected in the program selection table 291. The program numbers shown in FIGS. 2 and 3 match the numbers of each communication program stored in the agent program 290.

FIG. 4 shows several screens of the agent's smart bowing behavior displayed on the display device 27 by the program numbers 00001 to 00002 in FIGS. 2 and 3. As shown in Fig. 4, Agent E is expressed as a clever bow by bowing while tightening his mouth and putting his hand on his knee. The words (vocalization) spoken by Agent E along with this action can be changed depending on the vehicle situation, learning situation, and so on.

When the cooling water temperature of the engine is low, the action "sleepy ..." is selected according to the condition of the engine. Sleepy expressions include a lowered eyelid expression, yawning or stretching, then taking certain actions (bowing, etc.), rubbing the eyes first, or slowing movements and vocalizations. Can be represented by. These sleepy expressions are not always the same, but the expressions are changed as appropriate by learning the number of actions and the like. For example, rub your eyes once every three times (behavior A), yawn once every ten times (behavior B), and otherwise make your eyelids look down (behavior C). These changes are realized by combining the additional programs of Action B and Action C with the basic program of Action A. Then, as for which action to combine, the number of times the program of the basic action A is executed is counted as a learning item, and the additional programs are combined according to the number of times. In addition, when expressing the action "energically", it is expressed by increasing the intonation of the voice or making the agent E appear on the screen while running.

Each item displayed in FIG. 2 represents a selection condition for selecting a program number, and is determined from various situations of the vehicle and the driver detected by the state sensor 40 (time, starting place). , Cooling water temperature, etc.) and items determined from the learning content stored in the learning item data 292 and response data 293 (today's IG ON count, elapsed time from the previous end, total startup count, etc.). In the program selection table, the program that satisfies all of these items is always uniquely determined. In the table, the "" mark indicates an item that must be satisfied in order for the program number to be selected, and the "-" mark indicates an item that is not considered in the selection of the program.

Figures 2 and 3 describe actions and selection conditions related to communication (greeting) when the ignition is turned on, but a program for selecting a program that regulates various other actions (actions and vocalizations). Various numbers and selection conditions are also specified. For example, a program is also stipulated in which an agent takes actions such as "sticking" or stepping on "tatara" on the condition that a sudden brake is applied, and makes a surprise voice. The choice of each action by the agent should be changed by learning for sudden braking, for example, from the first sudden braking to the 3rd time, "tatara" is applied, and from the 4th to 10th time, "tatara" is stepped on, 10 From the second time onward, take the action of "stepping on one foot just one step forward" so that the agent gradually gets used to sudden braking. Then, if there is a one-week interval from the last sudden braking, move back one step.

The learning item data 292 and the response data 293 in FIG. 1 are both data stored and updated by the learning of the agent, and their contents are conceptually shown in FIGS. 5 and 6, respectively. As shown in FIG. 5, the learning item data 292 includes the total number of start-ups for determining the selection conditions of the program selection table 291 (Fig. 2), the previous end date and time, the number of times the ignition is turned on today, and the remaining 5 times before refueling. The amount and the like are stored, and the number of rests / date and time, the default value, and other data for deciding whether or not to start the program selected by the selection condition (whether or not to take a rest) are stored.

The total number of activations stores the total number of times the ignition has been activated, and is counted up each time the ignition is turned on. In the previous end date and time, the date and time is stored each time the ignition is turned off. Today's ignition ON count stores the number of ignition ON counts for that day and the end time of the day. It counts up every time the ignition is turned on, but when the day ends, the data is initialized to "0". The end time of the day is stored as a default value of 24:00, and this time can be changed according to the life pattern of the user (driver). If the time is changed, the changed time is stored.

The remaining amount of fuel detected just before refueling the fuel (gasoline) is stored in the remaining amount of refueling of the previous 5 times, and each data is shifted to the left with each new refueling (oldest leftmost). (Data will be deleted) The remaining amount just before refueling this time is stored on the far right. In this data, when the detection value G1 of the fuel detection sensor 415, which will be described later, becomes equal to or less than the average value G2 (G1 G2) of the remaining amount of refueling for all five times, Agent E appears on the display device 27 and refuels. The prompting action is displayed on the display device 27, and a voice such as "I'm hungry! I want gasoline!" Is output from the voice output device 25.

In the number of days off / date and time, even if the corresponding communication program is selected, the number of times of rest without execution is stored for each program number. This number of days off / date and time is stored for the agent action in which the rest item is set as a learning item, for example, the action of the agent proposing to stop the air conditioner (program number 00123) described later. If the driver's response to the agent's suggestion or conversation is rejected (rejected) or ignored (or not responded), "rest" is selectively set according to the communication program.

The default value stores the initial setting values for each item such as time, number of times, temperature, vehicle speed, date and time, and the value changed in the learning items such as the end time of the day described above is the initial value. Used when returning to. Other data stored in the learning item data 292 include, for example, birthdays of drivers and their affiliates (this is a user input item), holidays and their so-called events such as Christmas, Valentine's Day, White Day, etc. The day etc. are stored. There is also a special menu communication program for each event day, for example, on Christmas Eve, an agent disguised as Santa Claus will appear.

In the response data 293 of FIG. 6, the history of the user's response to the action of the agent is stored for each communication program number whose learning item is the user response. The user response data includes communication program numbers 00123 and 125 in FIG. 6 (A), in which the latest response date and time and response contents are stored for a predetermined number of times (program number 00123 is two times), and program number 00124. Only the latest response content is stored once (so it is updated each time there is a response), only the latest response content is stored for a predetermined time, and the latest date and time and response content are stored once. Some of them are stored, and some of them store only the latest date and time for one time or a predetermined time. The symbols A, B, and C displayed in FIG. 6 (A) represent the response contents, and as shown in FIG. 6 (B), when the symbol A is ignored or the symbol B is rejected, Represents the case where the symbol C is accepted. The driver's response content is determined from the result of voice recognition for the driver's voice input from the microphone 26 and the input result by the input device. In this embodiment, the driver's response is classified into three putters, ignoring, rejecting, and accepting, but "strongly rejected", "angry", and "pleased" are newly added. May be good. In this case, the learning item data 292 (for example, the number of rests, etc.) and the response data 293 are additionally changed according to the newly added response.

FIG. 7 shows the contents of the data file stored in the navigation data storage device 30 (FIG. 1). As shown in FIG. 7, the navigation data storage device 30 has communication area data file 301, drawing map data file 302, intersection data file 303, node data file 304, and road as various data files used for route guidance and the like. Data file 305, search data file 306, and photo data file 307 are stored. As the navigation data storage device 4, for example, various recording media such as a flexible disk, a hard disk, a CD-ROM, an optical disk, a magnetic tape, an IC card, and an optical card, and a driving device thereof are used. The navigation data storage device 4 may be composed of a plurality of different types of recording media and a driving device. For example, the search data file 46 is composed of a readable and writable recording medium (for example, a flash memory), other files are composed of a CD-ROM, and those driving devices are used as the driving device.

In the communication area data file 301, the area where the mobile phone used in the vehicle connected to or not connected to the communication control device 24 can communicate from the inside of the vehicle is displayed on the display device 5, and the area where the communication can be performed is searched for a route. Communication area data to be used at the time is stored for each type of mobile phone. Each communication area data for each type of mobile phone is numbered and managed for easy search, and the communicable area can be represented by the inside surrounded by the closed curve, so that closed curve is a short line segment. It is divided into two and specified by the position data of the bending point. The communication area data may be divided into quadrangular areas of various sizes and converted into data by coordinate data of two diagonal points. It is desirable that the contents stored in the communication area data file 301 can be updated as the area where the mobile phone can be used expands or contracts. Therefore, by using the mobile phone and the communication control device 24, the contents are stored. It is configured so that the contents of the communication area data file 301 can be updated with the latest data by communicating with the information providing station. The communication area data file 301 may be composed of a flexible disk, an IC card, or the like, and may be rewritten with the latest data. The drawn map data file 302 stores the drawn map data drawn on the display device 27. This drawn map data stores a layered map, for example, map data for each layer from the top layer to Japan, the Kanto region, Tokyo, and Kanda. A map code is attached to the map data of each layer.

The intersection data file 303 contains the intersection number that identifies each intersection, the name of the intersection, the coordinates of the intersection (latitude and longitude), the number of the road whose start and end points are the intersection, and the presence or absence of a signal as intersection data. It is stored. The node data file 304 stores node data consisting of information such as latitude and longitude that specify the coordinates of each point on each road. That is, this node data is data about one point on the road, and when the one connecting the nodes is called an arc, it is expressed by connecting each of a plurality of node rows with an arc. The road data file 305 contains a road number that identifies each road, an intersection number that is a start point or an end point, a road number that has the same start point or end point, road thickness, prohibition information such as no entry, and photo data described later. The photo number etc. are stored. The road network data consisting of the intersection data, the node data, and the road data stored in the intersection data file 303, the node data file 304, and the road data file 305 is used for the route search.

The search data file 306 stores intersection row data, node row data, and the like that constitute the route generated by the route search. The intersection sequence data includes information such as an intersection name, an intersection number, a photo number showing a characteristic landscape of the intersection, a corner, and a distance. In addition, the node column data consists of information such as east longitude and north latitude indicating the position of the node. In the photographic data file 307, photographs of each intersection and characteristic landscapes that can be seen while going straight are stored in the form of digital, analog, or negative film corresponding to the photograph numbers.

FIG. 8 shows various sensors constituting the status sensor unit 40. As shown in FIG. 8, the status sensor unit 40 includes an ignition sensor 401, a vehicle speed sensor 402, an accelerator sensor 403, a brake sensor 404, a side brake detection sensor 405, a shift position detection sensor 406, a winker detection sensor 407, and a wiper detection sensor 408. Light detection sensor 409, seat belt detection sensor 410, door open / close detection sensor 411, passenger detection sensor 412, indoor temperature detection sensor 413, outdoor temperature detection sensor 414, fuel detection sensor 415, water temperature detection sensor 416, ABS detection sensor 417, Air conditioner sensor 418, weight sensor 419, front inter-vehicle distance sensor 420, rear inter-vehicle distance sensor 421, body temperature sensor 422, heart rate sensor 423, sweating sensor 424, brain wave sensor 425, eye tracer 426, infrared sensor 427, and other sensors (tires) Air pressure drop detection sensor, belt loosening detection sensor, window open / close state sensor, crush sensor, indoor humidity sensor, outdoor humidity sensor, oil temperature detection sensor, oil pressure detection sensor, etc.) It is equipped with various sensors that detect the conditions inside the vehicle. These various sensors are arranged at predetermined positions according to their respective sensing purposes. When each of these sensors does not exist as an independent sensor, it includes a case of indirectly sensing from another sensor detection signal. For example, a tire pressure drop detection sensor indirectly detects a drop in air pressure due to fluctuations in the signal of the wheel speed sensor.

The ignition sensor 401 detects whether the ignition is ON or OFF. As the vehicle speed sensor 402, a conventionally known vehicle speed sensor such as one that detects the rotation angular velocity or the rotation speed of the speedometer cable to calculate the vehicle speed can be used without particular limitation. The accelerator sensor 403 detects the amount of depression of the accelerator pedal. The brake sensor 404 detects the amount of depression of the brake, and detects whether or not the brake is suddenly applied from the depression force, the depression speed, and the like. The side brake detection sensor 405 detects whether or not the side brake is applied. The shift position detection sensor 406 detects the shift lever position. The blinker detection sensor 407 detects the blinking direction of the blinker. The wiper detection sensor 408 detects the driving state (speed, etc.) of the wiper. The light detection sensor 409 detects the lighting state of each lamp such as a head lamp, a tail lamp, a fog lamp, and a room lamp. The seatbelt detection sensor 410 detects whether or not the driver and passengers (auxiliary seat, rear seat) are wearing seatbelts. If it is not worn, an agent will appear as appropriate (to the extent that it is not disliked) and give warnings, cautions, comments, etc. (change the degree by learning).

The door open / close detection sensor 411 detects the open / closed state of the door, and in the case of a so-called half-door, the agent notifies that fact. The door open / close detection sensor 411 can detect the open / close of each door according to the vehicle type, such as the driver's seat door, the passenger seat door, the rear driver's seat side door, and the rear passenger seat side door. The passenger detection sensor 412 is a sensor that detects whether or not a passenger is in the passenger seat or the rear seat, and detects it from the image of the inside of the vehicle captured by the image pickup device 28, or is placed in the auxiliary seat or the like. Detect with a pressure sensor or a weight scale. The indoor temperature detection sensor 413 detects the indoor air temperature, and the outdoor temperature detection sensor 414 detects the air temperature outside the vehicle. The fuel detection sensor 415 detects the remaining amount of fuel such as gasoline and light oil. The detection values for the past five times immediately before refueling are stored in the learning item data 292, and when the average value is reached, the agent notifies that it is time to refuel.

The water temperature detection sensor 416 detects the temperature of the cooling water. Immediately after the ignition is turned on, if this detection temperature is low, the agent often acts sleepy. On the other hand, if the water temperature is too high, the agent will notify you with a "sloppy" behavior before overheating. The ABS detection sensor 417 detects whether or not ABS has been activated to prevent the tires from locking due to sudden braking and to ensure maneuverability and vehicle stability. The air conditioner sensor 418 detects the operating state of the air conditioner. For example, ON / OFF of the air conditioner, set temperature, air volume, etc. are detected. The weight sensor 419 is a sensor that detects the weight of the driver. The driver is identified from this weight or from the weight and the image of the image pickup device 28, and the agent learned in relation to the driver is made to appear. That is, by using the learning item data 292 and the response data 293 learned by the agent for the specified driver, an agent dedicated to that driver is made to appear. The front inter-vehicle distance sensor 420 detects the distance to other vehicles and obstacles in front of the vehicle, and the rear inter-vehicle distance sensor 421 detects the distance to other vehicles and obstacles behind.

The body temperature sensor 422 is a heart rate sensor 423, and the sweating sensor 424 is a sensor that detects the driver's body temperature, heart rate, and sweating state, respectively. To do. Alternatively, the body temperature sensor 422 may detect the temperature distribution of each part of the driver by thermography using an infrared detection element. The brain wave sensor 425 is a sensor that detects the driver's brain wave, and detects, for example, an α wave or a β wave to check the driver's arousal state and the like. The eye tracer 426 detects the movement of the user's line of sight, and determines whether the user is searching for an object outside the vehicle, searching for an object inside the vehicle, or being awake during normal driving. The infrared sensor 427 detects the movement of the user's hand or face.

Next, the operation of the present embodiment configured as described above will be described. FIG. 9 is a flowchart showing the main operation of the agent processing according to the present embodiment. When the ignition sensor 401 detects that the ignition has been turned on, the agent processing unit 11 first performs the initial setting (step 11). As initial settings, RAM is cleared, the work area for each process is set in RAM, the program selection table 291 (Fig. 2) is loaded into RAM, the flag is set to 0, and so on. In the agent process of the present embodiment, the start of the process is set to ON, but the process may be started when, for example, the door open / close detection sensor 411 detects the opening / closing of any of the doors.

Next, the agent processing unit 11 determines the detection value supplied from each sensor of the status sensor unit 40 to the status information processing unit 15, the processing result of the image captured by the image pickup device 28, and the vehicle detected by the current position detection device 21. Data such as the current position of is acquired, stored in a predetermined area of RAM, and the current status is grasped from the stored data (step 12). For example, when the temperature of the cooling water detected by the water temperature detection sensor 416 is t1, the agent processing unit 11 stores this temperature t1 in the RAM, and if t1 is equal to or less than a predetermined threshold value t2, the current vehicle is present. It is understood that the cooling water temperature (see Fig. 2) is low as the state of. The current situation is that there are other requests from the driver who have voice-recognized based on the input from the microphone 26, such as "Call XX" or "Display restaurants around here". And requests such as "play a CD" are also grasped as the current situation. In this case, the words "CD", "kake", etc. included in the recognized voice become the selection conditions (horizontal axis items) of the program selection table 291 (Fig. 2).

Further, the agent processing unit 11 checks the learning item data 292 and the response data 293 of the agent data storage device 29 to grasp the state (learning data) that the agent has learned so far (step 13).

The agent processing unit 11 determines from the program selection table 291 shown in FIG. 2 whether or not there is a communication program (number) that can be started in the current state from the grasped current state and the checked learning data. , If there is no corresponding program (step 14; N), go back to step 12 and grasp the new state. On the other hand, if there is a communication program that can be started (step 14; Y), determine the program number. Then, the driver's response history to the determined program number is confirmed from the response data 293, and it is confirmed whether or not the start of the communication program of the program number should be absent (step 15).

When not in the resting state (step 15; N), the agent processing unit 11 activates the communication program corresponding to the determined program number to perform the actions (actions and voices) of each agent shown in FIG. The following image is displayed on the display device 27, and the sound is output from the sound output device 25 (step 16). This makes it possible to communicate not only from the current situation of the vehicle or driver, but also with the agent that reflects the learning results of the past situation and response.

Then, the agent processing unit 11 acquires the driver's response to the agent action by activating the communication program from the voice recognition result based on the input from the microphone 26 and the input result from the input device 22 (step 17). Next, the agent processing unit 11 determines whether or not the communication program started in step 16 is a controlled program (step 18). Here, whether or not the program is controlled is specified for each program. For example, "Should I turn on the radio?" Or "I will guide you to the cafeteria" when you detect a belly ringing sound at noon. A communication program or the like when an agent proposes an action that can be processed, such as "Should I?", Is defined as a controlled program.

If it is not a controlled program (step 18; N), the process proceeds to step 20, and if it is a controlled program (step 18; Y), control is performed according to the driver's response acquired in step 17 (step 19). ). For example, if a response that accepts a proposal such as "turn on", "OK", or "yes" is recognized in response to the proposal communication for turning on the power of the radio, the agent processing unit 11 controls according to the response. As a result, the agent is made to reply (act and voice) and the radio is turned on.

Then, the agent processing unit 11 causes the agent to learn by accumulating the data related to the communication program this time (step 20). As for the accumulation of data, for example, when the communication program is not started (step 15; Y), the number of times column of the corresponding program number of the learning item data 292 is counted up. However, if the number of rests stored in the number of rests / date and time column of the learning item data 292 is Ka times and the number of rests determined from the history of the response data 293 up to the previous time for the program number is Kb times, Ka = If it is Kb-1, it means that I took a specified number of times off this time. Therefore, the data (stored at the corresponding position) in the program number column of the learning item data 292 and the response data 293 is cleared. In other cases (in the case of step 18; after step 19 in the case of N), if there is a learning item in the current situation grasped in step 12, the value of the learning item data 292 is updated and the response content is stored as a history. If it is the program number to be used, the response content acquired in step 17 is stored in the response data 293 (Fig. 7). If data for a predetermined number of times specified for each program number has already been stored in the response history, the oldest data is discarded and new data is stored.

When each of the above program determination, communication, and learning processes is completed, the agent processing unit 11 confirms whether or not the flag 1 is set in the flag area initially set in the RAM in step 11 (step 21). If the flag is not set (step 21; N), it is determined whether or not the ignition sensor 401 has detected the ignition OFF (step 22), and if it is not detected (step 22; N), step 12 Go back to and grasp the new state. On the other hand, when the ignition OFF is detected (step 22; Y), the flag 1 is set in the flag area of the RAM (step 23), and then the process returns to step 12 to perform post-processing. That is, flag 1 causes the agent to perform farewell actions (behavior and voice) according to the current situation and learning data, control such as turning off lights that have been forgotten to turn off, and learn for the post-processing (step 20). After that, since the flag is 1 (step 21; Y), the agent processing is terminated.

Next, a specific example of the action by the agent processing described above will be described. Figure 10 conceptually shows the specific contents of agent processing after the ignition is turned on. As shown in FIG. 10 (A), the agent processing unit 11 has the current status of "ON" detected by the ignition sensor 401 and the current position (latitude, longitude) detected by the current position detection device 21. It is assumed that the current position obtained from is "Other" (other than home and resting place), etc. (step 12). In addition, as the learning data checked for the learning item data 292 and the response data 293, today's ignition ON count is "2nd day", the previous end date and time is "1997.7.12 10:05", and the total activation count is "30 times". It is assumed that it is checked (step 13).

From the above processing, the agent processing unit 11 selects the communication program of program number 00004 from the program selection table 291 (step 14), confirms that it is not a holiday target (step 15; N), and then determines the number. Start the communication program of (step 16). When the communication program of program number 00004 is started, the agent actions (actions and vocalizations) shown in FIG. 3 are performed as shown in FIG. 10 (B). That is, as shown in FIG. 4, the display device 27 displays a plurality of images of continuous actions in which the agent "smartly bows" (or a moving image is displayed), and the audio output device 25 displays "again, I, Lisa will serve you. Thank you. Is output.

Since program number 00004 is not a controlled program (step 18; N), step 19 is skipped and today's ignition ON count of learning item data 292 is changed from 2 times as shown in Fig. 10 (C) as agent learning. Change to 3 times and change the total number of startups from 30 to 31 (step 20). After that, if flag = 1 is not set (normally flag = 0 because the ignition is turned on) and ignition is not turned off (step 21; N, 22; N), the process returns to step 12 and communication is continued thereafter.

By the above learning, the total number of activations stored in the learning item data 292 becomes 31 or more. Therefore, after that, the communication programs 00001 to 0006 are not selected unless all the learning data (learning item data 292 and response data 293) are initialized. After that, one of the communication programs of program numbers 00007 to 00019 is selected until the total number of ignition ON times reaches 300. That is, the agent does not "bow" when the cooling water temperature is high, and "bows sleepily" when the cooling water temperature is low. The voice at that time changes depending on the program number selected depending on the startup location, the number of IGONs of the day, etc.

FIG. 11 conceptually shows the specific contents of agent processing while the vehicle is running. As shown in FIG. 11 (A), the agent processing unit 11 is currently in a state where the state detected by the air conditioner sensor 418 is "ON", and the room temperature detected by the indoor temperature detection sensor 413 and the outdoor temperature detection sensor 414. Obtain T1 and outdoor temperature T2. Also, by checking the response data 293 of each program number, the previous two driver responses to program number 0123 are C (acceptance), so the default values (T3 = 2 degrees, T4 =) are the values of the threshold values T3 and T4. 24 degrees) is read from the learning item data 292. From these thresholds T3 and T4, the final situation is the relationship between room temperature and outdoor temperature, (T1-T2) T3 is "Yes", the relationship between outdoor temperature T2 and threshold T4 T2 T4 is "Yes", etc. It is grasped in (Fig. 11 (A)).

From the above processing, the agent processing unit 11 finally selects the communication program of program number 00123 from the program selection table 291 (step 14), and confirms that it is not a holiday target (step 15; N). , Invoke the communication program with that number (step 16). By invoking the communication program of program number 00123, the agent performs the actions shown in Fig. 11 (B). That is, the display device 27 displays a plurality of images of continuous operation (or a moving image is displayed) in which the agent appears and smiles, and the audio output device 25 says, "It has become cool outside. Let the outside air in. A voice such as "Hmm?" Is output.

Get the driver's response to this agent's communication (step 17). And since program number 00123 is a controlled program (step 18; Y), if it is accepted, the power of the air conditioner is turned off as control according to the response, and the windows on the driver's seat side and the passenger's seat side are only halved. Control the opening (step 19). After the control according to the response, the agent processing unit 11 learns the agent by A if the driver's response is ignored, B if the driver's response is ignored, and B if the driver's response is accepted, as shown in FIG. 11 (C). C is stored in the response data 293 corresponding to the program number 0123 together with the response date and time. In this case, the response data of the previous two times is deleted, and the previous response and the current response become the stored contents of the response data 293. If the stored response content is ignored A, the next thresholds T3 and T4 will be lowered by 1 degree (T3-1 = 1 degree, T4-1 = 23 degrees) and will be rested once. Become. If the stored response content is rejection B, the next thresholds T3 and T4 will be lowered by 1 degree (T3-1 = 1 degree, T4-1 = 23 degrees) and will be rested 5 times. As the threshold value for one rest, the temperatures T3 = T3-1 and T4 = T4-1 after lowering are used. If the stored response content is reception C, the same values as the previously used thresholds T3 and T4 are used.

In addition, we explained the control to open the windows on the driver's seat side and the passenger's seat side by 1/2 according to the acceptance response to the agent's proposed communication, but the position and amount of opening these windows are also targeted for learning by the agent, and the next window It may be learned about open / close control. For example, the driver's response such as "Open too much", "Tighten a little more", "Open the back", etc., is not the specific data for the situation in Fig. 11 (A), but the driver's response regarding window opening and closing. It is stored in the response data 293 as general data representing the preference. In this case, the default values are, for example, the window to be opened and the opening amount; the driver's seat side window 1/2 and the passenger seat side window 1/2. If the driver's response is "Open behind", this response (voice recognition result) is grasped as the current situation (step 12), and the agent's action is performed by starting the communication program (step 16). At the same time, the rear window is also opened.

As described above, according to the present embodiment, the agent learns various past situations, and the agent performs a communication action suitable for the current situation regarding the vehicle and the driver according to the learning contents, so that the driver is in the vehicle. You can drive without getting bored. The driver can also be angry, laugh, and consult with the agent, and can respond to the proactive suggestions made by the agent in the mood of the moment. Then, since the agent communicates by appropriately judging and learning the actions of these drivers, it is possible to feel familiarity with the agent and increase the attachment to the vehicle.

Further, according to the vehicle provided with the communication function of the present embodiment, the anthropomorphic agent's action is determined from the learning result by the learning means and the current situation judged by the situation judgment means, and the agent appears by the agent appearance means. Since the agent is allowed to do it, the anthropomorphic agent can act appropriately according to the situation and communicate with the driver. That is, an agent who is anthropomorphized from learning results based on not only the current situation of the vehicle / driver but also the past history can act according to the situation and communicate with the driver. Therefore, the driver can have various controls such as talking with the agent, opening and closing windows, controlling CDs and radios, and providing detailed guidance on driving roads and facilities, and the driving environment in the car. Can be comfortable.

The agent device of this embodiment can also be configured as follows. (1) Agent appearance means for making an anthropomorphic agent appear in the vehicle, situation judgment means for judging the situation of the vehicle, learning means for learning by memorizing a predetermined situation by this situation judgment means, and this learning. From the learning result by the means and the situation determined by the situation determination means, the action determination means for determining the action of the agent and the action determined by the action determination means are performed by the agent appearing by the agent appearance means. An agent device including an agent control means. (2) The agent device according to (1) above, wherein the situation determination means is a means for determining the current or past situation. (3) The agent device according to (1) above, wherein the situation determination means is a current position current detection means for detecting the current position of the vehicle. (4) The agent device according to (1) above, wherein the situation determination means is an in-vehicle sensor that detects the situation of each part of the vehicle. (Five) The agent device according to (1) above, wherein the situation determination means is a sensor that detects an environment outside the vehicle. (6) The agent device according to (1) above, wherein the learning means outputs a predetermined learning result according to the number of times the situation means detects the same situation. (7) The agent device according to (1) above, wherein the agent appearance means includes an image display means and an agent display means for displaying an image of a specific agent with respect to the image display means. (8) The agent device according to (7) above, wherein the specific agent is selected from a plurality of agents. (9) The agent device (10) according to (1) above, wherein the agent appearance means includes a voice output device and an agent voice output means for outputting the voice of a specific agent to the voice output device. ) The agent device according to (9) above, wherein the voice output means outputs a voice selected from a plurality of agent voices. (11) The agent device according to (1) above, wherein the agent control means displays an image of an agent's action on an image display means in response to an action determined by the action determination means. (12) The agent device according to (1) above, wherein the agent control means controls the tone color of the voice output from the voice output means in response to the action determined by the action determination means. (13) The agent device according to (1) above, wherein the learning means further includes a storage means for storing at least one of a driver's instruction content and a response content as a learning target. (14) The agent device according to (13) above, further comprising a switch for inputting a driver's operation, wherein the storage means stores the time and place of the switch input. (15) The agent device according to (1) above, wherein the learning means includes a discriminating means for determining whether or not a driver has operated in response to an action executed by the agent control means.

<figref num="1">It is a block diagram which shows the structure for realizing the communication function in 1 Embodiment of this invention.</figref><figref num="2">In the same as above, it is explanatory drawing which conceptually represented the contents of the program selection table in embodiment.</figref><figref num="3">In the same as above, in the embodiment, it is explanatory drawing which showed the action (behavior and voice) of the agent corresponding to each program number.</figref><figref num="4">In the same as above, it is explanatory drawing which showed several screens about the agent's "smart bow" behavior displayed on the display device by activating the program numbers 00001 to 00002 in the embodiment.</figref><figref num="5">In the same as above, it is explanatory drawing which conceptually represented the content of the learning item data in an embodiment.</figref><figref num="6">In the same as above, it is explanatory drawing which conceptually represented the content of the response data in an embodiment.</figref><figref num="7">In the same as above, it is explanatory drawing which conceptually represented the content of the data file stored in the navigation data storage device in embodiment.</figref><figref num="8">In the same as above, it is explanatory drawing which showed various sensors constituting the situation sensor part in embodiment.</figref><figref num="9">In the same as above, it is the flowchart which showed the main operation of the agent processing by embodiment.</figref><figref num="10">In the same as above, in the embodiment, it is explanatory drawing which conceptually represented the content of the specific agent processing after ignition ON.</figref><figref num="11">In the same as above, in the embodiment, the content of the specific agent processing while the vehicle is running is conceptually expressed.</figref>

Code description

1 Overall processing unit 10 Navigation processing unit 11 Agent processing unit 12 I / F unit 13 Image processing unit 14 Voice control unit 15 Status information processing unit 21 Current position detection device 22 Input device 23 Storage medium drive device 24 Communication control device 25 Voice output Device 26 Microphone 27 Display device 28 Imaging device 29 Agent data storage device 30 Navigation data storage device 40 Status sensor unit