Remote conference system, remote conference terminal and remote conference method

The present invention relates to a remote conference system, a remote conference terminal, and a remote conference method for executing a remote conference between conference terminals provided at a plurality of locations.

Conventionally, conference terminals are provided at a plurality of bases where users exist, and a conference terminal on the calling side (hereinafter referred to as a calling terminal) and a conference terminal on the called side (hereinafter referred to as a called terminal) are connected via a network. There is a video conferencing system that holds remote conferences. In such a video conferencing system, when the user using the called terminal is absent, even if the called terminal receives a call from the calling terminal, the absent user immediately responds. Can not do it. Therefore, when a message is received from the other party registered in the list registration data on the called party terminal in the absence of the user, a video conferencing system is known in which the other party who received the message is registered at the head position of the list registration data. (See, for example, Patent Document 1).

<p><patcit num="1"><text>Japanese Unexamined Patent Publication No. 8-317364</text></patcit></p>

<p> In the video conferencing system described in Patent Document 1, when the called party terminal receives a message while the user is absent, the user who returns to the called party terminal refers to the list registration data and receives the message. Can be specified and a reply to the message can be made. However, at the time when the user replies to the message, it cannot be determined whether or not the other party using the calling terminal is present. Then, if the calling terminal makes a reply to the message even though the other party is absent, there is a problem that the reply is useless.</p><p> The present invention has been made to solve the above problems, and when a call is made from the calling side terminal to the called side terminal while the user using the called side terminal is absent, the called side is called. It is an object of the present invention to provide a remote conference system, a remote conference terminal, and a remote conference method in which a user who returns to the terminal can grasp the usage status of the calling terminal.</p>

<p> The teleconferencing system according to the first aspect of the present invention includes a plurality of teleconferencing terminals capable of acquiring at least one of image data and audio data from a predetermined range, and the teleconferencing terminals connected to each other via a network. Each of the teleconferencing systems is a teleconferencing system that executes a teleconferencing by outputting conference data including at least one of the image data and the audio data acquired by each of the teleconferencing terminals. The terminal means that the other remote conference terminal is absent from the call signal transmitting means for transmitting the call signal requesting the connection with the one remote conference terminal and the user who uses the other remote conference terminal. When the absence information shown or the status request requesting the usage status of the one remote conference terminal is received, the status determination means for determining the usage status of the one remote conference terminal and the determination of the status determination means. Based on the result, the other remote conference terminal includes a state information transmitting means for transmitting status information indicating the usage state of the one remote conference terminal to the other remote conference terminal, and the other remote conference terminal is the one remote conference terminal. Absence determination means for determining whether or not a user using the other teleconferencing terminal is absent based on the information acquired by the other teleconferencing terminal when the ringing signal is received from the terminal. When the user is determined to be absent by the absence determination means, the absence signal transmission means for transmitting the absence information or the status request to the one remote conference terminal and the one remote conference terminal When the state information is received, it is provided with a state notification means for notifying the usage state of the one remote conference terminal based on the state information.</p><p> According to the first aspect, when another teleconferencing terminal receives a ringing signal from one teleconferencing terminal, it issues an absence information or status request if a user using the other teleconferencing terminal is absent. When one teleconferencing terminal receives absence information or status request from another teleconferencing terminal, it emits status information regarding the usage status of one teleconferencing terminal. The other teleconferencing terminal notifies the usage status of one teleconferencing terminal based on the status information received from the one teleconferencing terminal. As a result, when a call is made from one teleconferencing terminal to another teleconferencing terminal in the absence of a user who uses another teleconferencing terminal, the user who returns to the other teleconferencing terminal is one remote. It is possible to grasp the usage status of the conference terminal.</p><p> Further, in the first aspect, the state determination means is used by a user who uses the one remote conference terminal based on the information acquired by the one remote conference terminal as the usage state of the one remote conference terminal. You may decide whether or not you are absent. As a result, the user who has returned to the other teleconferencing terminal can grasp whether or not the user who uses one teleconferencing terminal is absent.</p><p> Further, in the first aspect, the state determining means is based on the connection state with another remote conference terminal different from the other remote conference terminal as the usage state of the one remote conference terminal. It may be determined whether or not the teleconferencing terminal is executing the teleconferencing. As a result, the user who has returned to the other remote conference terminal can grasp whether or not one remote conference terminal is executing the remote conference.</p><p> Further, in the first aspect, when the usage state of the one remote conference terminal shown in the status information is a usage state suitable for connection with the other remote conference terminal, the state notification means is described in the above one. The absence determination means further notifies the other remote conference terminal when the notification for the connection with the one remote conference terminal is performed by the status notification means. Based on the information acquired by the terminal, it is determined whether or not there is a user who uses the other teleconferencing terminal, and the other teleconferencing terminal is determined to have a user by the absence determination means. In this case, a response signal transmitting means for transmitting a response signal permitting connection with the other remote conference terminal to the one remote conference terminal may be provided. As a result, when a user who uses another teleconferencing terminal returns, if there is a user who uses one teleconferencing terminal, both teleconferencing terminals can be automatically connected.</p><p> The remote conference terminal according to the second aspect of the present invention is a remote conference terminal capable of performing a remote conference by connecting to each other via a network, and is at least image data and audio data from a predetermined range in the remote conference terminal. At least of the image data and the audio data acquired by the plurality of remote conference terminals when the terminal information acquisition means for acquiring one and the plurality of the remote conference terminals are connected to each other via the network. A conference data output means for outputting conference data including one, a call signal transmission means for transmitting a call signal requesting connection to the other remote conference terminal, and the other remote conference terminal as a response to the call signal. When the absence information indicating that the user to be used is absent or the status request requesting the usage status of the remote conference terminal is received, the status determination means for determining the usage status of the remote conference terminal and the above-mentioned The state information transmitting means for transmitting the state information indicating the usage state of the remote conference terminal to the other remote conference terminal based on the determination result of the state determination means is provided.</p><p> According to the second aspect, when the teleconferencing terminal sends a call signal requesting connection to another teleconferencing terminal and then receives absentee information or a status request as a response, status information regarding the usage status of the teleconferencing terminal is received. To other teleconferencing terminals. As a result, when another remote conference terminal that receives a call from the remote conference terminal is absent from the user, the user who returns to the other remote conference terminal can change the usage status of the remote conference terminal on the calling side. Can be grasped.</p><p> The remote conference terminal according to the third aspect of the present invention is a remote conference terminal capable of performing a remote conference by connecting to each other via a network, and is at least image data and audio data from a predetermined range in the remote conference terminal. At least of the image data and the audio data acquired by the plurality of remote conference terminals when the terminal information acquisition means for acquiring one and the plurality of the remote conference terminals are connected to each other via the network. The remote conference is based on the information acquired by the remote conference terminal when a call signal requesting connection is received from the conference data output means for outputting the conference data including one and the other remote conference terminal. The absence determination means for determining whether or not the user who uses the terminal is absent, and the absence of the user who uses the remote conference terminal when the absence determination means determines that the user is absent. As a response to the absence information or the status request, as an absence signal transmitting means for transmitting the absence information indicating the above or the status request requesting the usage status of the other remote conference terminal to the other remote conference terminal. When status information indicating the usage status of the other remote conference terminal is received, it is provided with a status notification means for notifying the usage status of the other remote conference terminal based on the status information.</p><p> According to the third aspect, when the teleconferencing terminal receives the ringing signal from the other teleconferencing terminal, if the user who uses the teleconferencing terminal is absent, the teleconferencing terminal transmits the absence information or the status request to the other teleconferencing terminal. To do. When the status information is received as a response to the absence information or the status request, the usage status of one remote conference terminal is notified based on the status information. As a result, when the remote conference terminal that received the call from the other remote conference terminal is in the absence of the user, the user who returns to the remote conference terminal on the called side can change the usage status of the other remote conference terminal. Can be grasped.</p><p> The remote conference terminal according to the fourth aspect of the present invention is a remote conference terminal capable of performing a remote conference by connecting to each other via a network, and is at least image data and audio data from a predetermined range in the remote conference terminal. At least of the image data and the audio data acquired by the plurality of remote conference terminals when the terminal information acquisition means for acquiring one and the plurality of the remote conference terminals are connected to each other via the network. A conference data output means for outputting conference data including one, a call signal transmission means for transmitting one call signal requesting connection to the remote conference terminal, and the above-mentioned one as a response to the one call signal. When one absence information indicating that the user using the teleconferencing terminal is absent or one status request requesting the usage status of the remote conference terminal is received, the usage status of the remote conference terminal is set. A state information transmitting means for transmitting a state information indicating a usage state of the remote conference terminal to the remote conference terminal based on the determination result of the state determination means for determining, and the one. When a ringing signal requesting a connection is received from another teleconferencing terminal that is the same as or different from the teleconferencing terminal, the user who uses the teleconferencing terminal is absent based on the information acquired by the teleconferencing terminal. Absence determination means for determining whether or not the data is present, and other absence information indicating that the user using the remote conference terminal is absent when the absence determination means determines that the user is absent. Alternatively, as an absence signal transmitting means for transmitting another state request requesting the usage state of the other remote conference terminal to the other remote conference terminal, and as a response to the other absence information or the other state request. When other status information indicating the usage status of the other remote conference terminal is received, it is provided with a status notification means for notifying the usage status of the other remote conference terminal based on the status information.</p><p> According to the fourth aspect, when the teleconferencing terminal sends a call signal requesting connection to another teleconferencing terminal and then receives absentee information or a status request as a response, status information regarding the usage status of the teleconferencing terminal is received. To other teleconferencing terminals. As a result, when one remote conference terminal that receives a call from the remote conference terminal is absent from the user, the user who returns to the one remote conference terminal can change the usage status of the remote conference terminal on the calling side. Can be grasped.</p><p> Further, when the remote conference terminal receives a ringing signal from another remote conference terminal, if the user who uses the remote conference terminal is absent, the remote conference terminal transmits the absence information or the status request to the other remote conference terminal. When the status information is received as a response to the absence information or the status request, the usage status of one remote conference terminal is notified based on the status information. As a result, when the remote conference terminal that received the call from the other remote conference terminal is in the absence of the user, the user who returns to the remote conference terminal on the called side can change the usage status of the other remote conference terminal. Can be grasped.</p><p> The teleconferencing method according to the fifth aspect of the present invention includes a plurality of teleconferencing terminals capable of acquiring at least one of image data and audio data from a predetermined range, and the teleconferencing terminals connected to each other via a network. Each is a teleconferencing method processed by a teleconferencing system that executes a teleconferencing by outputting conference data including at least one of the image data and the audio data acquired by each of the teleconferencing terminals. Then, a call signal transmission step of transmitting a call signal requesting connection with the one remote conference terminal to the other remote conference terminal designated by the user at the one remote conference terminal, and the other remote conference terminal. Absence determination step for determining whether or not a user using the other teleconferencing terminal is absent based on the information acquired by the other teleconferencing terminal when the ringing signal is received by the terminal. And, when it is determined that the user is absent by the absence determination step, the absence information indicating that the user using the other remote conference terminal is absent, or the usage state of the one remote conference terminal. When the absence information or the status request is received by the one remote conference terminal and the absence signal transmission step of transmitting the status request requesting the above to the one remote conference terminal, the one remote conference terminal A state information transmission step of transmitting status information indicating the usage status of the one remote conference terminal to the other remote conference terminal based on the status determination step for determining the usage status of the above and the determination result of the status determination step. And, when the state information is received by the other remote conference terminal, the state notification step for notifying the usage state of the one remote conference terminal based on the state information is provided.</p><p> According to the fifth aspect, when a ringing signal is received from one teleconferencing terminal in another teleconferencing terminal, an absence information or a status request is issued if a user using the other teleconferencing terminal is absent. .. When the absence information or the status request is received from the other teleconferencing terminal in one teleconferencing terminal, the status information regarding the usage status of the one teleconferencing terminal is issued. In another teleconferencing terminal, the usage status of one teleconferencing terminal is notified based on the status information received from one teleconferencing terminal. As a result, when a call is made from one teleconferencing terminal to another teleconferencing terminal in the absence of a user who uses another teleconferencing terminal, the user who returns to the other teleconferencing terminal is one remote. It is possible to grasp the usage status of the conference terminal.</p>

<figref num="1">It is a figure which shows the whole structure of a video conferencing system 1.</figref><figref num="2">It is a block diagram which shows the electrical composition of the conference terminal 3, 4, 5.</figref><figref num="3">It is a block diagram which shows the electrical composition of a conference management server 6.</figref><figref num="4">It is a timing chart which shows the flow of various data between a calling side terminal 3, a conference management server 6, and a called side terminal 4.</figref><figref num="5">It is a flowchart of the constant processing executed by the conference terminals 3, 4 and 5.</figref><figref num="6">It is a flowchart of the outgoing call processing executed by the calling side terminal 3.</figref><figref num="7">It is a flowchart of the relay process executed by the conference management server 6.</figref><figref num="8">It is a flowchart of the reception processing executed in the called side terminal 4.</figref><figref num="9">It is a flowchart of a user standby process (S121).</figref><figref num="10">It is a figure which illustrates the video conference screen 280.</figref><figref num="11">It is a figure which illustrates the state notification screen 290.</figref><figref num="12">It is another figure which illustrates the state notification screen 290.</figref><figref num="13">It is a figure which illustrates the cancellation notification screen 300.</figref><figref num="14">It is a flowchart of the outgoing call processing executed by the calling side terminal 3 which concerns on a modification.</figref><figref num="15">It is a flowchart of the user standby process (S121) which concerns on a modification.</figref><figref num="16">It is a timing chart which shows the flow of various data between the calling side terminal 3, the conference management server 6, and the called side terminal 4 which concerns on a modification.</figref>

Hereinafter, embodiments embodying the present invention will be described with reference to the drawings. The drawings to be referred to are used to explain the technical features that can be adopted by the present invention, and the description of the device configuration, flowcharts of various processes, etc. are not intended to be limited thereto. , Is just an example of explanation.

The overall configuration of the video conferencing system 1 according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, in the video conferencing system 1 according to the present embodiment, a plurality of conferencing terminals provided at a plurality of bases where users exist are connected to the network 2. In the video conferencing system 1 shown in FIG. 1, the conferencing terminal 3 is provided at the base A, the conference terminal 4 is provided at the base B, and the conference terminal 5 is provided at the base C. It suffices if a plurality of conference terminals are provided. Further, although the conference management server 6 is provided at the relay base, the conference management server 6 may be provided at any place connected to the network 2.

In the video conferencing system 1 according to the present embodiment, the conference terminals 3, 4 and 5 provided at each base are connected to each other by the conference management server 6. The video and audio acquired by the conference terminals 3, 4 and 5 are transmitted and received to each other via the network 2. The conference terminals 3, 4 and 5 create and output conference data obtained by synthesizing the video and audio acquired at each base. As a result, the video and audio of the users existing at each base are combined and output in real time on the conference terminals 3, 4 and 5, and a remote conference (here, a video conference) is performed.

The electrical configurations of the conference terminals 3, 4 and 5 will be described with reference to FIG. Since the conference terminals 3, 4 and 5 all have the same configuration, only the configuration of the conference terminal 3 will be described here, and the description of the other conference terminals 4 and 5 will be omitted.

The conference terminal 3 is provided with a CPU 20 as a controller that controls the conference terminal 3. The CPU 20 is connected to a ROM 21 that stores a BIOS or the like, a RAM 22 that temporarily stores various data, and an I / O interface 30 that mediates the transfer of data. A hard disk drive 31 (hereinafter, HDD 31) having various storage areas is connected to the I / O interface 30. The I / O interface 30 contains a communication device 25 for connecting to the network 2, a mouse 27, a video controller 23, a key controller 24, a camera 34 for photographing the user, and a user's voice. A microphone 35 for capturing, a speaker 36 for outputting audio, and a CD-ROM drive 26 are connected to each other. A display 28 is connected to the video controller 23. A keyboard 29 is connected to the key controller 24.

The camera 34 is set in advance with a shooting range, focus, and the like so that a predetermined range in which the user using the conference terminal 3 is located can be suitably shot. Further, the microphone 35 is set in advance with directivity, noise cut, and the like so that the user who uses the conference terminal 3 can appropriately capture the utterance within a predetermined range located. Further, the speaker 36 is also set in the output volume and output direction of the sound so as to be suitable for listening within a predetermined range where the user who uses the conference terminal 3 is located.

The HDD31 is used to execute various data for executing a video conference (for example, video / audio acquired by each conference terminal 3, 4 and 5 and conference data obtained by synthesizing these) and for executing a video conference. The conference execution program of is stored. The conference execution program stored in the HDD 31 includes a program for executing various processes (see FIGS. 5, 6, 8, and 9) related to the interconnection of the conference terminals 3, 4 and 5, which will be described later. The above conference execution program is stored in the CD-ROM 114 inserted into the CD-ROM drive 26. In the conference terminal 3, these programs and data can be set up in the HDD 31 by reading the CD-ROM 114 from the CD-ROM drive 26.

The electrical configuration of the conference management server 6 will be described with reference to FIG. The conference management server 6 is a device that is connected to conference terminals 3, 4, 5 provided at a plurality of bases via the network 2 and realizes interconnection between the conference terminals 3, 4, 5. The conference management server 6 has a CPU 40, a ROM 41, a RAM 42, an I / O interface 50, and an HDD 51. The I / O interface 50 includes a communication device 45 for communicating with the network 2, a mouse 47, a key controller 44 that accepts key input on the keyboard 49, and a video controller 43 that controls the display of the display 48. Each is connected.

The HDD 51 contains various data for executing the interconnection of the conference terminals 3, 4, 5 (for example, the IP address or telephone number of each conference terminal, the connection information described later, etc.), and the conference terminals 3, 4, 5 A relay control program for executing and managing the interconnection is stored. The relay control program stored in the HDD 51 includes a program for executing the relay process (see FIG. 7) described later.

With reference to FIGS. 4 to 13, in the video conferencing system 1 according to the present embodiment, processing related to interconnection between the conference terminals 3, 4 and 5 that execute the video conferencing will be described. In the video conferencing system 1, after the interconnection between the conference terminals 3, 4 and 5 is established, the video conferencing proceeds by mutual communication between the conference terminals 3, 4 and 5. In the following, the conference terminal 3 of the base A will be a calling terminal that requests a connection from the conference terminal 4. Let the conference terminal 4 of the base B be the called side terminal that is required to be connected from the conference terminal 3. Then, the flow of processing until the interconnection between the calling side terminal 3 and the called side terminal 4 is established will be described. Even if any of the conference terminals 4 and 5 is used as the calling side terminal and any of the conference terminals 3 and 5 is used as the called side terminal, the processing flow until the interconnection between the terminals is established is as follows. The same is true.

First, the overall processing of the video conferencing system 1 will be described by exemplifying the timing chart of FIG. When the user at base A gives a call instruction to base B (T1), the calling terminal 3 sends a connection request for base A to the conference management server 6 (T2). When the conference management server 6 receives the connection request of the base A (T3), the connection request is transmitted to the called terminal 4 of the base B (T4). When the connection request of the base A is received by the called side terminal 4 (T5), the presence or absence of the user who uses the called side terminal 4 is determined as described later. In the example of FIG. 4, it is assumed that the user who uses the called terminal 4 is absent. In this case, the absence notification of the base B is transmitted to the conference management server 6 according to the absence of the user who uses the called terminal 4 (T6). The absence notification of the base B is a signal indicating that the user who uses the called side terminal 4 is absent.

When the conference management server 6 receives the absence notification of the base B (T7), the status request for the base A is sent to the calling terminal 3 (T8). The status request for the base A is a signal requesting the usage status of the calling terminal 3. When the calling side terminal 3 receives the status request for the base A (T9), the usage status of the calling side terminal 3 is determined as described later. Then, a signal indicating the usage status of the calling terminal 3 (hereinafter referred to as the status information of the base A) is transmitted to the conference management server 6 (T10). In the example of FIG. 4, it is assumed that the calling terminal 3 is executing a video conference with another conference terminal 5 (that is, a video conference is being executed between the bases A and C). In this case, the status information of the base A indicating that the conference is in progress is transmitted to the conference management server 6.

When the conference management server 6 receives the status information of the base A (T11), the status information is transmitted to the called terminal 4 of the base B (T12). When the callee terminal 4 receives the status information of the base A (T13), the notification is performed based on the status information of the base A (in a meeting in the example of FIG. 4) as described later. After that, it is determined whether or not the user who uses the called side terminal 4 is present or absent (in other words, whether or not the user who uses the called side terminal 4 has returned). In the example of FIG. 4, it is assumed that the user who uses the called side terminal 4 is absent when the status information is received at T13. In this case, the called terminal 4 listens for the next status information.

After a certain period of time has passed since the previous transmission of the status information (T10), the conference management server 6 sends a status request to the base A to the calling terminal 3 (T14) as in the case of T8. Similar to T9 to T13, the status information of the base A is transmitted from the calling terminal 3 to the called terminal 4 via the conference management server 6 (T15, T16). The called side terminal 4 is notified based on the status information of the base A, and it is determined whether or not the user who uses the called side terminal 4 has returned (T17). In the example of FIG. 4, it is assumed that the user who uses the called side terminal 4 is present at the time when the status information is received at T17. In this case, the called party terminal 4 is deemed to have returned the user who uses the called party terminal 4, and the connection permission is transmitted to the conference management server 6 (T18).

When the connection permission of the base B is received by the conference management server 6 (T19), the connection permission is transmitted from the conference management server 6 to the calling terminal 3 (T20). When the calling side terminal 3 receives the connection permission of the base B (T21), the calling side is subjected to the connection processing executed by each of the calling side terminal 3, the conference management server 6, and the called side terminal 4. The interconnection between the terminal 3 and the called terminal 4 is established. As a result, the same video conferencing as before can be executed between the calling side terminal 3 and the called side terminal 4 (T22).

In the following, with reference to FIGS. 5 to 8, the processing executed by each of the calling side terminal 3, the conference management server 6, and the called side terminal 4 for realizing the entire processing of the video conferencing system 1 is described. explain. Further, in order to facilitate understanding, the timing chart shown in FIG. 4 and the screen examples shown in FIGS. 10 to 13 will be described with reference to them as appropriate.

With reference to FIG. 5, the constant processing executed by each of the conference terminals 3, 4 and 5 will be described. The constant processing (FIG. 5) is continuously executed by the CPU 20 based on the conference execution program stored in the HDD 31 until the conference terminals 3, 4 and 5 are turned off. Hereinafter, the constant processing executed by the conference terminal 3 will be described, but the same applies to the constant processing executed by the other conference terminals 4 and 5.

In the constant processing shown in FIG. 5, a timer (not shown) is first reset (S1). When the timer is reset in S1, the timer starts counting the elapsed time. Next, it is determined whether or not the user's operation or voice is detected in the conference terminal 3 (S3). Specifically, when the input operation using the mouse 27 or the keyboard 29 is detected by the signal input to the CPU 20 via the I / O interface 30, it is determined that the user's operation is detected. (S3: YES). Further, when a human voice (utterance) is detected from the voice acquired from the microphone 35 by the known voice recognition, it is determined that the user's voice is detected (S3: YES). In this way, when the user's operation and voice are detected, it is considered that the user who uses the conference terminal 3 is present, so the system returns to S1, the timer is reset, and the elapsed time is counted from the beginning. To.

On the other hand, if neither the user's operation nor voice is detected (S3: NO), it is determined whether or not no operation and no voice have continued for a predetermined time (S5). Here, if the elapsed time counted by the timer is equal to or longer than a predetermined time (for example, 10 minutes), it is determined that no operation and no voice have continued for a predetermined time (S5: YES). If no operation and no voice continue for a predetermined time, it is considered that the user using the conference terminal 3 is absent. Therefore, the still image α in the shooting range is acquired from the camera 34, and the still image α is stored in the predetermined storage area of the HDD 31 (S7). As a result, the still image α (that is, the background image when the user is absent) in the shooting range when the user who uses the conference terminal 3 is absent is stored in the predetermined storage area of the HDD 31. If the past still image α is already stored in the predetermined storage area of the HDD 31, the latest still image α acquired in S7 is overwritten and saved.

If the elapsed time counted by the timer is less than a predetermined time (for example, 10 minutes) (S5: NO), there is no operation and no voice for a predetermined time, so there is a waiting time (for example, 10 seconds). After that, return to S3. That is, when the user's operation and voice are not detected, S3 to S5 are repeatedly executed until no operation and no voice continue for a predetermined time (in other words, until the absence of the user is detected). After S7 is executed, it returns to S1, the timer is reset, and the elapsed time is counted from the beginning.

The calling process of the calling terminal 3 will be described with reference to FIG. This process is continuously executed by the CPU 20 until the calling side terminal 3 is turned off based on the conference execution program stored in the HDD 31 of the calling side terminal 3. In the call process shown in FIG. 6, for convenience, the flow for calling one other base is shown, but by executing this process multiple times, it is possible to execute calls of a plurality of other bases in parallel.

In the outgoing call processing of the calling terminal 3 shown in FIG. 6, first, the input operation of the mouse 27 or the keyboard 29 is detected, and is there a call instruction of an arbitrary base (here, base B) that is the connection request destination? Whether or not it is judged (S11). If there is no call instruction for base B (S11: NO), return to S11 and wait for the call instruction. On the other hand, when there is a call instruction for base B (S11: YES), a connection request for base B is sent to the conference management server 6 (S13). In the example of the time chart shown in FIG. 4, T1 to T2 are executed by the processing of S11 to S13.

After the execution of S13, it is determined whether or not the status request for the base A has been received (S15). When a status request is received (S15: YES), the usage status of the calling terminal 3 is automatically determined by the following processing (S17 to S31), and information corresponding to the usage status is transmitted. First, it is determined whether or not base A is conducting a video conference with another base (S17). That is, when the calling terminal 3 is executing a video conference with another conference terminal via the network 2 (S17: YES), the status information of the base A indicating "in a conference" is the conference management server. Sent to 6 (S19).

If the calling terminal 3 is not executing the video conference (S17: NO), the presence or absence of the user who uses the calling terminal 3 is automatically determined. First, a still image β in the shooting range is acquired from the camera 34 (S21), and it is determined whether or not a human face is detected from the still image β (S23). Specifically, a known face recognition method is used for image analysis to determine whether or not the still image β includes a human face. If a human face is detected from the still image β (S23: YES), it is considered that the user using the calling terminal 3 is present, so the base indicating that it is in "standby". The status information of A is sent to the conference management server 6 (S29).

When a human face is not detected from the still image β (S33: NO), the above still image β is compared with the still image α stored in the HDD 31. Then, it is determined whether or not the difference between the still images α and β (in other words, the ratio of different parts occupied by comparing the still images α and β) is γ% or more (S35). That is, the latest still image β on the calling terminal 3 changes by a predetermined value or more (for example, 10% or more) with reference to the still image α in the absence of the user acquired by the constant processing (see FIG. 5). It is judged whether or not it is. If the difference between the still images α and β is γ% or more (S25: YES), even if the camera 34 does not show a human face, the change within the shooting range is large, so the user is present. The status information of the base A indicating that it is "standby" is transmitted in S29.

When the difference between the still images α and β is less than γ% (S25: NO), it is determined whether or not the user's voice (utterance) is detected from the voice acquired from the microphone 35 (S27). .. In S27, as in S3, it is determined whether or not the voice acquired from the microphone 35 includes a human voice (utterance) by known voice recognition. If the user's voice (utterance) is detected (S27: YES), it is considered that the user using the calling terminal 3 is present, so that it is "standby" in S29. The status information of the indicated base A is transmitted. On the other hand, when the user who uses the calling terminal 3 is considered to be absent (S23, S25, and S27 are all NO), the cancellation request of the base A is sent to the conference management server 6 (S31). The cancellation request of the base A is a signal instructing the cancellation of the connection between the calling side terminal 3 and the called side terminal 4.

In the example of the time chart shown in FIG. 4, T9 is executed by the processing of S15, and T10 is executed by the processing of S17 to S29. In S19 and S23, it is preferable that the notification status of the status information on the calling side terminal 3 is displayed on the display 28. For example, on the video conference screen 280 shown in FIG. 10, the notification display 281 is displayed together with the composite video of the video conference, and the calling terminal 3 notifies the connection request destination base B that "conference is in progress". Is notified.

If the status request for base A has not been received (S15: NO), or after the execution of S19 and S29, it is determined whether or not the connection permission for base B has been received (S33). If the connection permission of base B has not been received (S33: NO), it returns to S15 and waits for the next status request. In the example of the time chart shown in FIG. 4, T15 is executed by the processing of S15 to S29 in the standby of the next state request.

When the connection permission of the base B is received (S33: YES), the connection process for the base B of the connection request destination is executed (S35). The connection process of S35 is a known connection process executed by the calling terminal. Here, the process of establishing a connection to the calling side terminal 3 is performed in response to the receiving side terminal 4 receiving the connection permission of the calling side terminal 3. When S35 is executed, it returns to S11 and waits for another call instruction. In the example of the time chart shown in FIG. 4, T21 is executed by the processing of S33 to S35. Then, the calling side terminal 3 and the called side are subjected to the connection processing (S35, S73 described later, S109 or S145 described later) executed by each of the calling side terminal 3, the conference management server 6, and the called side terminal 4. Interconnection with terminal 4 is established. As a result, the same video conferencing as before can be executed between the calling side terminal 3 and the called side terminal 4 (T22).

The relay processing of the conference management server 6 will be described with reference to FIG. 7. This process is continuously executed by the CPU 40 until the conference management server 6 is turned off based on the relay control program stored in the HDD 51 of the conference management server 6. In the relay process shown in FIG. 7, for convenience, the flow for one set (two bases) of relay connections is shown, but by executing this relay process multiple times, multiple sets of relay connections can be executed in parallel. Is.

In the relay process of the conference management server 6 shown in FIG. 7, it is determined whether or not the connection request of any base (here, base A) that is the connection request source has been received (S51). If the connection request of base A has not been received (S51: NO), it returns to S51 and waits for the reception of the connection request. On the other hand, when the connection request of the base A is received (S51: YES), the connection request is transmitted to the called terminal 4 (S53). In the example of the time chart shown in FIG. 4, T3 to T4 are executed by the processing of S51 to S53.

After the execution of S53, whether or not there is a response from the base B is determined by whether or not a response signal is received from the called terminal 4 (S55). When there is a response from base B (S55: YES), it is determined whether or not the out-of-office notification of base B has been received (S57). If there is no response from base B (S55: NO), or if an out-of-office notification is received from base B (S57: YES), the connection information of base A and base B is saved in the predetermined storage area of HDD51. (S59).

When there is no response from the base B (S55: NO), for example, the response signal cannot be received due to the power off, failure or communication failure of the called side terminal 4, and the interconnection cannot be established between the two conference terminals 3 and 4. It is in a state. In addition, when the absence notification of the base B is received (S55: NO), the user who uses the called side terminal 4 is absent, and the video conference cannot proceed between the two conference terminals 3 and 4. is there. Therefore, in S59, when it is inappropriate to interconnect the calling side terminal 3 and the called side terminal 4, the combination of both conference terminals 3 and 4 and the IP address are used as connection information in the conference management server 6. Be remembered.

After the connection information is saved in the conference management server 6, the status request for the base A is sent to the calling terminal 3 (S61). In the example of the time chart shown in FIG. 4, T7 is executed by the processing of S55 to S57, the processing of S59 is performed, and then T8 is executed by the processing of S61.

After the execution of S61, it is determined whether or not the status information of the base A has been received (S63). When the status information of the base A is received (S63: YES), the status information is transmitted to the called terminal 4 (S65). If the status information of the base A has not been received (S63: NO), it is determined whether or not the cancellation request of the base A has been received (S67). When the cancellation request of the base A is received (S67: YES), the cancellation request is transmitted to the called terminal 4 (S69). After the cancellation request for base A is sent, the connection information for base A and base B stored in HDD 51 is deleted (S75), and the system returns to S51. That is, the connection between the calling side terminal 3 and the called side terminal 4 is stopped, and another connection request is awaited. In the example of the time chart shown in FIG. 4, T11 to T12 are executed by the processing of S63 to S65.

If the stop request for base A has not been received (S67: NO), it is determined whether or not the connection permission for base B has been received (S71). The connection permission of the base B is a signal transmitted to the conference management server 6 when the connection process (S109, S145 described later) is executed on the called side terminal 4. If the connection permission for base B has not been received (S71: NO), the system returns to S61 after waiting for a certain period of time (for example, 30 seconds). Even after the above-mentioned S65 is executed, after waiting for a certain period of time (for example, 30 seconds), the process returns to S61. As a result, the conference management server 6 sends a status request to the base A to the calling terminal 3 at regular intervals until it receives the cancellation request of the base A or the connection permission of the base B. In the example of the time chart shown in FIG. 4, T14 is executed by the processing of S61 and T16 is executed by the processing of S63 to S65 after a certain time has elapsed from the previous transmission of the state information (T10).

When the connection permission of the base B is received (S71: YES), the connection process between the bases A and B is executed (S73). The connection process of S73 is a known connection process executed by a relay device that mediates the connection between two terminals. Here, the connection permission of the called side terminal 4 is transmitted to the calling side terminal 3, and a process of mediating the interconnection between the calling side terminal 3 and the called side terminal 4 is performed. When S73 is executed, the connection information of base A and base B is deleted (S75) as after the execution of S69, and the system returns to S51 to wait for another connection request. In the example of the time chart shown in FIG. 4, T19 to T20 are executed by the processing of S71 to S73, and then the processing of S75 is performed.

The reception processing of the called party terminal 4 will be described with reference to FIG. This process is continuously executed by the CPU 20 until the called party terminal 4 is turned off based on the conference execution program stored in the HDD 31 of the called party terminal 4. In the response processing shown in FIG. 8, for convenience, the flow of the response to one other base is shown, but by executing this processing a plurality of times, the response to a plurality of other bases can be executed in parallel.

In the reception processing of the called side terminal 4 shown in FIG. 8, it is determined whether or not the connection request of any base (here, base A) that is the connection request source has been received (S101). If the connection request of base A has not been received (S101: NO), it returns to S101 and waits for the reception of the connection request. On the other hand, when the connection request of the base A is received (S101: YES), it is determined whether or not the automatic response is set in the called terminal 4 (S103). The user of the called terminal 4 can arbitrarily set an automatic response or a manual response in advance. The automatic response or manual response set by the user is stored in a predetermined storage area of RAM22. The judgment of S103 is made based on whether the automatic response or the manual response is stored in the RAM 22.

When the manual response is set on the called terminal 4 (S103: NO), the connection request of the base A is displayed on the display 28 (S105). For example, the display 28 displays a notification screen including the comment "There was a call from base A." Then, it is determined whether or not there is a user operation within a certain period from the screen display of S105 (S107). Specifically, for example, it is determined whether or not an input operation using the mouse 27 or the keyboard 29 is detected within 30 seconds after the notification screen is displayed. If there is a user operation within a certain period from S105 (S107: YES), it is considered that the user using the called terminal 4 is present, and the connection to the connection requesting base A is made. Processing is executed (S109). The connection process of S109 is a known connection process executed by the terminal on the called side. Here, the calling side terminal 3 transmits a signal in response to the call as a connection permission for the call being executed, and performs a process of establishing a connection to the calling side terminal 3.

If automatic response is set on the called terminal 4 (S103: YES), or if there is no user operation within a certain period from S105 (S107: NO), the called terminal 4 The presence or absence of the user who uses is automatically determined in the same manner as S21 to S27 in FIG. 6 (S111 to S117). When it is considered that the user who uses the called terminal 4 is present (YES in any of S113, S115, and S117), the connection process of S109 is executed. On the other hand, when the user who uses the called terminal 4 is considered to be absent (S113, S115, and S117 are all NO), the absence notification of the base B is sent to the conference management server 6 (S119). In the example of the time chart shown in FIG. 4, T5 is executed by the processing of S101, the processing of S103 to S117 is performed to detect the absence of the user, and the processing of S119 executes T6.

After the execution of S119, the user standby process that waits for the return of the user who uses the called side terminal 4 (that is, the usage state of the called side terminal 4 changes from the absence of the user to the presence) is executed. (S121).

As shown in FIG. 9, in the user standby process (S121), it is first determined whether or not the status information of the base A has been received (S131). When the status information of the base A is received (S131: YES), the status information is displayed on the screen of the display 28 (S133). Specifically, the display 28 displays a status notification screen 290 as shown in FIGS. 11 and 12. The status notification screen 290 includes a connection request 291 indicating a call from the base A, a usage status 292 of the calling side terminal 3 provided at the base A, and a call button 293 for connecting to the base A. The usage status 292 in FIG. 11 indicates that the status information of the base A is during a meeting. The usage status 292 in FIG. 12 indicates that the status information of the base A is standby.

After the execution of S133, it is determined on the above-mentioned status notification screen 290 whether or not the call button 293 is pressed via the mouse 27 or the keyboard 29 (S135). When the call button 293 is pressed (S135: YES), the connection process for the connection requesting base A is executed (S145). The connection process of S145 is a known connection process executed by the terminal on the called side. Here, since the call from the calling terminal 3 has already been completed, a new connection request is sent to the calling terminal 3 as a connection permission for the call to connect to the calling terminal 3. Perform the process of establishing.

If the call button 293 is not pressed (S135: NO), the same processing as S111 to S117 in FIG. 8 is performed (S137 to 143). Then, when it is considered that the user who uses the called side terminal 4 is present (any of S139, S141, and S143 is YES), the connection process of S145 is executed. That is, when the user returns to the base B, the called party terminal 4 detects the presence of the user, and the connection to the called party terminal 4 is automatically performed. On the other hand, when it is considered that the user who uses the called terminal 4 is absent (S139, S141, and S143 are all NO), the process returns to S131 and waits for the next status information.

In the example of the time chart shown in FIG. 4, T13 is executed by the processing of S131, and the notification based on the status information of the base A is performed by the processing of S133. Since the status information of the base A indicating that "meeting is in progress" is transmitted in T10, the status notification screen 290 shown in FIG. 11 is displayed on the display 28 of the called side terminal 4. When the status information is received at T13, the absence of the user is detected by the processing of S135 to S143, and the called terminal 4 waits for the next status information. Then, in the standby for the next status information, T17 is executed by the processing of S131, and the notification based on the status information of the base A is performed by the processing of S133. When the status information is received at T17, the presence of the user is detected by the processing of S135 to S143, and T18 is executed by the processing of S145.

If the status information of the base A has not been received (S131: NO), it is determined whether or not the cancellation request of the base A has been received (S147). When the cancellation request of base A is received (S147: YES), the user standby process (Fig. 9) is terminated and the process returns to S101. That is, it waits for another connection request without executing the connection with the calling terminal 3. If the cancellation request for base A has not been received (S147: NO), it returns to S131 and waits for the next status information. Here, when the cancellation request of the base A is received (S147: YES), it is preferable that the display 28 shows that the process is terminated without being connected to the calling terminal 3. .. For example, on the cancellation notification screen 300 shown in FIG. 13, the connection request 301 indicating the call from the base A and the connection cancellation 302 based on the absence of the user at the base A are displayed.

As described above, according to the video conferencing system 1 according to the present embodiment, the called side terminal 4 that has received the connection request of the calling side terminal 3 does not have a user who uses the called side terminal 4. If so, send an out-of-office notification to the conference management server 6. The conference management server 6 that has received the out-of-office notification sends an information request to the calling terminal 3. The calling terminal 3 that has received the information request transmits the status information to the called terminal 4 via the conference management server 6. The called terminal 4 notifies the usage status of the calling terminal 3 based on the status information received from the conference management server 6. As a result, when a call is made from the calling side terminal 3 to the called side terminal 4 while the user using the called side terminal 4 is absent, the user who returns to the called side terminal 4 is the calling side terminal. You can grasp the usage status of 3.

By the way, in the above embodiment, the video conferencing system 1 corresponds to the "remote conference system" of the present invention, and the conference terminals 3, 4 and 5 correspond to the "plurality of remote conference terminals" of the present invention. In the calling terminal 3, the CPU 20 that executes S13 corresponds to the "call signal transmitting means" of the present invention, and the CPU 20 that executes S17 and S21 to S27 corresponds to the "state determination means" of the present invention. The CPU 20 that executes S29 corresponds to the "state information transmitting means" of the present invention. In the called terminal 4, the CPU 20 that executes S111 to S117 corresponds to the "absence determination means" of the present invention, and the CPU 20 that executes S119 corresponds to the "absence signal transmission means" of the present invention, and executes S133. The CPU 20 that executes the S145 corresponds to the "state notification means" of the present invention, and the CPU 20 that executes S145 corresponds to the "response signal transmitting means" of the present invention.

Further, the camera 34 and the microphone 35 correspond to the "terminal information acquisition means" of the present invention, and the CPU 20 for executing the video conference corresponds to the "conference data output means" of the present invention. S13 corresponds to the "call signal transmission step" of the present invention, S111 to S117 correspond to the "absence determination step" of the present invention, S119 corresponds to the "absence signal transmission step" of the present invention, and S17 and S21. ~ S27 corresponds to the "state determination step" of the present invention, S19 and S29 correspond to the "state information transmission step" of the present invention, and S133 corresponds to the "state notification step" of the present invention.

The present invention is not limited to the above embodiment, and changes can be made without changing the gist of the invention. In the above embodiment, the conference management server 6 controls the interconnection of the conference terminals 3, 4, 5 and executes a video conference between the conference terminals 3, 4, 5, but the present invention is not limited to this. ..

For example, the conference management server 6 is connected to conference terminals 3, 4 and 5 provided at a plurality of bases via a network 2, and relays video, audio, data, etc. to hold a video conference between multiple bases. It may be a multipoint control unit to be realized. In this case, the conference terminals 3, 4 and 5 have the same configuration as that of the above embodiment (FIG. 2), but at least function as a client to participate in the video conference executed by the conference management server 6. It suffices to have a function of transmitting video and audio to the conference management server 6 and a function of outputting conference data transmitted from a multi-site connection device).

In the video conferencing system 1 having such a configuration, the moving image data generated by the conference terminals 3, 4 and 5 provided at each base is transmitted to the conference management server 6 via the network 2. The conference management server 6 creates conference data by synthesizing the video data received from each base, and transmits the conference data to the conference terminals 3, 4 and 5 via the network 2. The conference terminals 3, 4 and 5 output the conference data received from the conference management server 6. As a result, the video and audio of the users existing at each base are synthesized in real time on the conference management server 6, and the combined conference data is output on the conference terminals 3, 4 and 5, and the video conference is carried out. ..

Further, if the conference terminals 3, 4 and 5 have a communication function for directly interconnecting with other conference terminals, it is not necessary to provide the conference management server 6. For example, the transmission time processing of the calling terminal 3 shown in FIG. 14 is the same as the transmission time processing shown in FIG. 6, but the connection request to the base B, the status information of the base A, and the cancellation request of the base A are respectively. It is different in that it is transmitted to the called terminal 4 (S13, S19, S29, S31), and in S35, it is directly interconnected with the called terminal 4. Further, after the execution of S13, it is determined whether or not the absence notification of the base B has been received instead of S15 in the transmission processing shown in FIG. 6 (S201).

On the other hand, the called terminal 4 performs the same processing as the reception processing shown in FIG. 8, except that S119 is not executed. Further, the user standby process of the called side terminal 4 shown in FIG. 15 is the same as the user standby process shown in FIG. 9, but the absence notification of the base B is sent to the calling side terminal 3 before the execution of S131. Sent (S203). Then, when the cancellation request of the base A is not received (S147: NO), or when the user who uses the called terminal 4 is considered to be absent (S139, S141, and S143 are all NO), a certain period of time is reached. After waiting (for example, 30 seconds), it returns to S203. In S109 and S145, interconnection is directly performed with the called terminal 4.

By the above processing, in the example of the time chart shown in FIG. 16, when the user of the base A gives a call instruction of the base B (T51), the connection request of the base A is sent from the calling side terminal 3 to the called side terminal 4. Sent (T52). T51 to T52 are executed by the processing of S11 to S13. In the called terminal 4, if the user is absent when the connection request of the base A is received (T53), the absence notification of the base B is transmitted to the calling terminal 3 (T54). T53 to T54 are executed in the processes of S101 to S119 and S203. When the calling side terminal 3 receives the absence notification of the base B (T55), the status information of the base A is transmitted to the called side terminal 4 (T56). T55 to T56 are executed by the processes of S201 and S17 to S29.

Next, when the called side terminal 4 receives the status information of the base A (T57), the notification based on the status information of the base A is performed. If the user of the called terminal 4 is absent at T57, an out-of-office notification is sent at S203 after a certain period of time (T58). T57 to T58 are executed in the processes of S133 to S143 and S203. In the calling side terminal 3, the status information of the base A is transmitted to the called side terminal 4 in response to the absence notification of the base B (T59). T59 is executed in the processes of S201 and S17 to S29. In the called side terminal 4, notification is performed based on the status information of the base A, and if the user of the called side terminal 4 is present, the connection permission of the base B is granted from the called side terminal 4 to the calling side. Sent to terminal 3 (T60). T60 is executed in the processes of S133 to S143 and S145. Finally, the interconnection between the calling side terminal 3 and the called side terminal 4 is established, and a video conference is executed between the bases A and B (T61). T61 is executed in the processing of S33 to S35 and S145.

In this way, even when the calling side terminal 3 and the called side terminal 4 are interconnected without going through the conference management server 6, there is no user who uses the called side terminal 4 as in the above embodiment. When a call is made from the calling side terminal 3 to the called side terminal 4, the user who returns to the called side terminal 4 can grasp the usage state of the calling side terminal 3.

In the above modification (see FIGS. 14 and 15), the absence notification of the base B is transmitted to the calling terminal 3 in S203, but a status request to the base A may be transmitted instead. (See Figure 16). Then, in S201, instead of determining whether or not the absence notification of the base B has been received, it may be determined whether or not the status request for the base A has been received. In the above modification, the CPU 20 that executes S203 corresponds to the "absent signal transmitting means" of the present invention, and S203 corresponds to the "absent signal transmitting step" of the present invention.

1 Video conferencing system 2 network 3 Conference terminal 4 Conference terminal 5 Conference terminal 6 Conference management server 20 CPU 21 ROM 22 RAM 27 mouse 28 display 29 keyboard 31 hard disk drive 34 camera 35 microphone