Multi-spot video conference system

Description: TECHNICAL FIELD [Detailed description of the invention]

【0001】

[Industrial application field]

The present invention relates to a multipoint video conferencing system for performing multipoint video conferencing by connecting a plurality of video conferencing terminals to a multipoint communication control device.

【0002】

[Conventional technology]

In recent years, with the development of image compression coding technology and the spread of high-speed, high-quality digital lines represented by ISDN, recommendations for AV (audiovisual) services are ITU-T Recommendations H.320, H.242 and H. As stipulated in 221 etc., various video conferencing systems that follow these recommendations have been developed.

【0003】

As such a video conferencing system, a multipoint video conferencing system that can connect three or more points to hold a video conference has already been devised. In the multipoint video conferencing system, a plurality of video conferencing terminals participating in the multipoint video conference are connected to a multipoint communication control device (hereinafter referred to as an MCU) that controls the multipoint video conference. , A method of forming a multipoint connection via an MCU is adopted.

【0004】

As an example of multipoint connection using an MCU, Fig. 6 shows the connection form when a multipoint video conference is held at four points. In the figure, the video conferencing terminals 602 to 605 are connected to the MCU 601 via ISDN in order to participate in the multipoint video conferencing.

【0005】

According to the conventional multipoint video conferencing system that connects multipoints using the MCU601 in this way, as a method for the MCU601 to deliver video to each video conferencing terminal 602 to 605 during multipoint communication, 1) A method of identifying the current speaker terminal by comparing the audio levels from each video conference terminal 602 to 605 and distributing the video sent from the identified speaker terminal. 2) A method of sequentially distributing the video sent from the currently connected video conferencing terminals 602 to 605 by switching at regular intervals. 3) A method of synthesizing images from all video conferencing terminals connected between multiple points by MCU601 as one image and distributing the combined image, or 4) A method of distributing video from a specific video conferencing terminal according to instructions from the operation unit of the MCU601, Has been proposed.

【0006】

[Problems to be Solved by the Invention]

However, in the methods 1), 2) and 4) above, a new video conferencing terminal is connected in the middle of the multipoint video conference, or communication with the video conferencing terminal participating in the multipoint video conference is possible. When disconnected, information about the connection status of such a video conferencing terminal is not notified to other video conferencing terminals for multipoint video conferencing, so a user of a video conferencing terminal that has not been connected up to that point suddenly makes a statement. There is a problem that it feels unnatural or inconvenient, for example, even if there is a comment to the user of the video conferencing terminal that has been connected until then, there is no response to the comment.

【0007】

In addition, even with the method of 3) above, where the MCU601 synthesizes and distributes the video from each video conferencing terminal participating in the multipoint video conferencing, the video conferencing terminal newly connected during the video conferencing. If the video capability of is turned off or the video capability of the video conferencing terminal to which the composite video is delivered is turned off, the status cannot be known from other video conferencing terminals. There was a similar problem.

【0008】

The present invention has been made to solve the above problems, and when a new video conferencing terminal is connected during a multipoint video conferencing or when communication with the participating video conferencing terminal is disconnected. The purpose is to provide a multipoint video conferencing system that can eliminate the unnaturalness and inconvenience of.

【0009】

[Means for solving problems]

In order to achieve the above object, the present invention provides the multipoint communication control in a multipoint video conferencing system for performing a multipoint video conference by connecting a plurality of video conferencing terminals to a multipoint communication control device. The device is a terminal information storage means for storing terminal information of the video conferencing terminal participating in the multipoint video conference, and a terminal for monitoring the number of terminals of the video conferencing terminal participating in the multipoint video conference. Terminal information that notifies the video conferencing terminals participating in the multipoint video conference of information on the change in the number of terminals when the number monitoring means and the terminal number monitoring means detect a change in the number of terminals. It is characterized by having a notification means.

【0010】

Further, the terminal information notification means has an image synthesizing means that converts information related to the change in the number of terminals into at least one of character data and bitmap data and overlays the transmitted image.

【0011】

Further, it is also preferable that the terminal information notification means has a voice synthesis means for voice-synthesizing information on the change in the number of terminals and adding it to the transmitted voice.

【0012】

Further, the terminal information notification means voice-synthesizes the image synthesis means that converts the information on the change in the number of terminals into at least one of the character data and the bitmap data and overlays the transmitted image, and the information on the change in the number of terminals. The voice synthesis means to be added to the transmitted voice, the video ability identification means for identifying the video ability of each video conferencing terminal participating in the multipoint video conference, and the video ability identified by the video ability identification means. Based on this, it is also preferable to have a selection means for selecting either the image synthesis means or the voice synthesis means.

【0013】

Further, the information regarding the change in the number of terminals may be composed of the number of terminals after the change and the terminal information of the video conferencing terminal that is the cause of the change in the number of terminals stored by the terminal information storage means. Good.

【0014】

[Action]

According to the multipoint video conferencing system of claim 1, when a change in the number of terminals is detected by the terminal number monitoring means, information on the change in the number of terminals is transmitted to the video conference participating in the multipoint video conference. The terminal is notified.

【0015】

Further, according to the multipoint video conferencing system of claim 2, the information regarding the change in the number of terminals is converted into at least one of character data and bitmap data by the image synthesizing means and overlaid on the transmitted image.

【0016】

Further, according to the multipoint video conferencing system of claim 3, the information regarding the change in the number of terminals is voice-synthesized by the voice synthesis means and added to the transmitted voice.

【0017】

Further, according to the multipoint video conferencing system of claim 4, either the image synthesis means or the voice synthesis means is selected by the selection means based on the video ability identified by the video ability identification means.

【0018】

[Example]

Hereinafter, examples of the present invention will be described with reference to the drawings.

【0019】

FIG. 1 is a block diagram showing a functional configuration of a multipoint communication control device (MCU) according to this embodiment.

【0020】

In the figure, the terminal information storage unit 1 stores information such as a calling address and a user abbreviation as information (terminal information) of each video conference terminal (hereinafter referred to as a participating video conference terminal) participating in a multipoint video conference. Is what you do. The terminal information stored in the terminal information storage unit 1 is directly input from the operation unit (not shown) of the MCU before the multipoint video conference is held, or when the video conference terminal is connected to the MCU. , LSD (low speed data) specified in H series and T series of ITU-T recommendation shall be used and transmitted from the video conferencing terminal to the MCU.

【0021】

The terminal information storage unit 1 is connected to the terminal information notification unit 3. The terminal information notification unit 3 converts the terminal information to be notified to the participating video conference terminals into at least one of character data and bitmap data and overlays it on the transmitted image, and the terminal information to notify the participating video conference terminals. A notification method selection unit (hereinafter referred to as a selection unit) 7 that selects one of the image composition unit 4 and the voice composition unit 5 to notify the terminal information and the voice synthesis unit 5 that synthesizes the voice and adds it to the transmitted voice. And have.

【0022】

The terminal information notification unit 3 is connected to the line interface 211 that controls a plurality of ISDN communication lines via the terminal number monitoring unit 2 that monitors the number of participating video conferencing terminals. Further, the image synthesis unit 4 and the voice synthesis unit 5 are connected to the line interface 211 via the image coding / decoding unit 207, the voice coding / decoding unit 205, and the multiplexing separation unit 210, respectively. The line interface 211 is connected to a plurality of ISDN communication lines, and controls each connected line according to the ISDN user network interface.

【0023】

The selection unit 7 is a video capability identification unit 6 that identifies whether or not the video conferencing terminal to which the terminal information is notified has video capability based on the BAS (bit allocation signal) specified in the ITU-T Recommendation H series. It is connected, and one of the image synthesis unit 4 and the voice synthesis unit 5 is selected based on the identification result of the video capability identification unit 6. For example, if the video conferencing terminal to which the terminal information is notified has video capability, the image synthesizer 4 is used to notify the terminal information, and if the video conferencing terminal to which the terminal information is notified does not have video capability, voice is used. Notify terminal information using synthesis unit 5. Further, when the video conferencing terminal to which the terminal information is notified has a video capability, it is also possible to notify the terminal information by using both the image synthesis unit 4 and the voice synthesis unit 5.

【0024】

In the above configuration, the terminal number monitoring unit 2 monitors the incoming call from the video conferencing terminal to the MCU and the disconnection of communication with the participating video conferencing terminal, and when a change in the number of participating video conferencing terminals is detected, the participating video conferencing terminal The terminal information of the video conferencing terminal, which is a factor in the change in the number of conference terminals, is taken out from the terminal information storage unit 1 by the terminal information notification unit 3. The extracted terminal information is overlaid on the transmitted image by the image synthesizing unit 4 or added to the transmitted voice by the voice synthesizing unit 5 according to the selection of the selection unit 7, together with the information on the change in the number of other terminals. The image signal generated by the image synthesizer 4 or the voice signal generated by the voice synthesizer 5 is encoded by the image coding / decoding unit 207 or the voice coding / decoding unit 205, and then the multiplexing / separating unit 210. The line interface 211 notifies each participating video conferencing terminal by multiplexing into a transmission frame.

【0025】

FIG. 2 is a block diagram showing a configuration of an MCU that can realize the above-mentioned functional configuration. The operation of the MCU 200 is controlled by the CPU 201 having functions as the terminal number monitoring unit 2, the terminal information notification unit 3, the video capability identification unit 6, and the notification method selection unit 7 described above.

【0026】

The CPU 201 is a ROM (Read Only Memory) 202 that stores a program for control, and a RAM (RAM that is used as a terminal information storage unit 1 that stores data such as terminal information and is also used as a work area of the CPU 201. Random Access Memory) 203, an operation unit 204 having an input device (not shown) such as a keyboard, tablet, mouse, etc. used for inputting control information etc. for the user to control the MCU200, and generating an audio signal. Audio processing unit 206, audio coding recombining unit 205 that encodes and recombines audio signals, image processing unit 208 that generates image signals, and image coding that encodes and recombines image signals. A recombination unit 207, a channel interface 209 for connecting an external data terminal device such as a personal computer, a multiplexing separation unit 210 for performing multiplexing and separation processing of various data, and an ISDN connected to a plurality of ISDN lines. It is connected to the line interface 211, which controls each line according to the user network interface, via the bus 212.

【0027】

The audio coding / decoding unit 205 performs decoding processing of the audio signal sent from the participating video conferencing terminal and the participating video conferencing terminal in accordance with the audio signal coding / recombining algorithm specified in the ITU-T Recommendation G series. The audio signal to be transmitted to is encoded.

【0028】

The voice processing unit 206 has the function of the voice synthesis unit 5 shown in FIG. 1, synthesizes the terminal information to be notified to the participating video conference terminals by voice to generate a voice signal, and voice-codes and decodes the voice signal. After the voice signal decoded by the conversion unit 205 is added to the voice signal, it is sent to the voice coding restoration unit 205 again.

【0029】

The image coding / decoding unit 207 performs decoding processing of the image signal sent from the participating video conference terminal and the participating video conference in accordance with the image signal coding / decoding algorithm specified in ITU-T Recommendation H.261. Encodes the image signal to be transmitted to the terminal.

【0030】

The image processing unit 208 has the function of the image synthesizing unit 4 shown in FIG. 1, converts the terminal information to be notified to the participating video conferencing terminals into at least one of character data and bitmap data, and converts the converted data into an image code. After overlaying the image signals from the participating video conferencing terminals decoded by the decryption / decoding unit 207 or synthesizing the images from all the participating video conferencing terminals, the data is sent to the image coding / decoding unit 207 again.

【0031】

The multiplexing separation unit 210 sets the audio signal sent from the audio coding / decoding unit 205, the image signal sent from the image coding / decoding unit 207, and the channel interface 209 in accordance with ITU-T Recommendation H.221. Multiplexing that multiplexes the data input from the external data terminal device connected to the outside and the control data generated by the control program executed by the CPU 201 in units of transmission frames according to the current communication capacity and communication mode. The conversion process and the received frame are separated into each medium of the constituent unit, and sent to the external data terminal device or CPU 201 connected by the voice-coded recombination unit 205, the image-coding recombining unit 207, and the channel interface 209. Perform the separation process for.

【0032】

FIG. 3 is a flowchart showing a procedure for monitoring the number of participating video conferencing terminals performed by the CPU 201. The program for executing this process is stored in ROM 202 and is started when the video conferencing terminal participating in the multipoint video conferencing is first connected to the MCU200.

【0033】

First, the value of the variable n for counting the current number of participating video conferencing terminals is initialized to "1" (step S100). Next, it is determined whether or not there is a new incoming call from the video conferencing terminal, that is, whether or not the number of participating video conferencing terminals has increased (step S101). When it is determined that there is an incoming call, the value of the variable n for counting the number of participating video conferencing terminals is incremented by 1 (step S102), and the factor that changed the number of participating video conferencing terminals (here, a new video conferencing terminal). The terminal information (for example, the calling address) of the newly connected video conferencing terminal is stored in the RAM 203 (step S103), and the terminal information notification process described later is performed (step S108). When the terminal information notification process is completed, the processes after step S101 are performed again.

【0034】

On the other hand, in step S101, when it is determined that there is no incoming call from the new video conferencing terminal, it is determined whether or not there is a video conferencing terminal whose communication with the MCU 200 is disconnected among the participating video conferencing terminals (step S104). , If there is a video conferencing terminal whose communication is disconnected, the value of the variable n for counting the number of participating video conferencing terminals is decremented by 1 (step S105). Then, as in step S103, the factors that changed the number of participating video conferencing terminals (here, the communication with the video conferencing terminal was disconnected) and the terminal information (calling address, etc.) of the video conferencing terminal whose communication was disconnected It is stored in RAM 203 (step S106). Further, it is determined whether or not there is a participating video conferencing terminal that should notify the information regarding the disconnection of communication with the video conferencing terminal determined in step S104 (step S107), and there is a participating video conferencing terminal that should notify the information. If it is determined, the terminal information notification process described later is performed (step S108). When the terminal information notification process is completed, the processes after step S101 are performed again. If it is determined that there is no participating video conferencing terminal to which the information should be notified, this processing operation is terminated.

【0035】

In step S107 described above, it is determined by the value of the variable n for counting the number of participating video conferencing terminals whether or not there is a participating video conferencing terminal to which the information should be notified. That is, when it is determined in step S104 that there is a video conferencing terminal whose communication has been disconnected, the value of the variable n becomes 0 or 1 or more by the processing of step S105. Therefore, if the value of the variable n is 0, the information It is determined that there is no participating video conferencing terminal to notify, and if the variable n is 1 or more, it is determined that there is a participating video conferencing terminal to notify information.

【0036】

FIG. 4 is a flowchart showing the terminal information notification processing procedure activated in step S108.

【0037】

First, it was stored in RAM 203 in advance based on the information (calling address, etc.) about the video conferencing terminal that caused the change in the number of participating video conferencing terminals, which was stored in RAM 203 in step S103 or step S106. More detailed terminal information about the video conferencing terminal, such as a user abbreviation, is retrieved (step S201).

【0038】

Next, the variable i for counting the number of participating video conferencing terminals notified of the change in the number of participating video conferencing terminals is initialized to "0" (step S202), and the value of the variable i is set to the number of participating video conferencing terminals. By comparing with, it is determined whether or not there is a video conferencing terminal for which information regarding the change in the number of participating video conferencing terminals has not yet been notified (step S203). In step S203, if the value of the variable i is smaller than the number of participating video conferencing terminals, that is, if there is a video conferencing terminal for which information regarding the change in the number of participating video conferencing terminals has not been notified, the process proceeds to step S204. If it is determined that there is no video conference terminal for which information regarding the change in the number of participating video conference terminals has not been notified, the terminal information notification process is terminated.

【0039】

As the number of participating video conferencing terminals to be compared with the value of the variable i, the number of participating video conferencing terminals n used in the flowchart shown in FIG. 3 may be used. Since the number of participating video conferencing terminals n includes the video conferencing terminal that caused the incoming call from the new video conferencing terminal detected in step S101, the video conferencing terminal that caused the change in the number of participating video conferencing terminals Even so, you will be notified of information about changes in the number of participating video conferencing terminals.

【0040】

Next, data indicating the presence or absence of the video capability of the "i-th video conferencing terminal", which is the notification destination video conferencing terminal for notifying the information regarding the change in the number of participating video conferencing terminals, is acquired (step S204), and based on the data. The presence or absence of video capability is determined (step S205). The presence or absence of video capability is identified from the BAS (bit allocation signal) specified in the ITU-T Recommendation H series, which is sent to the MCU 200 when the video conferencing terminal is connected to the MCU 200.

【0041】

Here, when it is determined that the i-th video conferencing terminal has video capability, the terminal information extracted in step S201 is the number of participating video conferencing terminals in the transmitted video to be transmitted to the i-th video conferencing terminal. It is converted into at least one of character data and bitmap data together with other information regarding the change in the above, and is overlaid on the transmitted video and transmitted (step S206). In this way, information regarding changes in the number of participating video conferencing terminals is notified.

【0042】

If it is determined that the i-th video conferencing terminal does not have video capability, the terminal information retrieved in step S201 is voice-synthesized together with other information regarding changes in the number of participating video conferencing terminals, and i The transmitted voice transmitted to the second video conferencing terminal is added and processed to notify the change in the number of participating video conferencing terminals (step S207).

【0043】

When the processing of step S206 or step S207 is completed, the value of the variable i is incremented by 1 (step S208), and the process returns to step S203.

【0044】

Fig. 5 shows an example of the received image of the video conferencing terminal when the number of participating communication terminals increases due to an incoming call from a new video conferencing terminal and the information regarding the change in the number of terminals is notified by overlaying the transmitted video. Shown.

【0045】

In the figure, 501 is a video (speaker video) from a video conferencing terminal identified as a speaker terminal, and the speaker video 501 is transmitted to each of the current participating video conferencing terminals. Speaker video conferencing terminals are identified by comparing the audio levels of the audio signals from each participating video conferencing terminal sent to the MCU 200.

【0046】

When the MCU200 receives an incoming call from a new video conferencing terminal, the information notifying that the video conferencing terminal is newly connected is overlapped with the area 502 of the speaker video (transmitted video) 501. As shown in FIG. 5, as the information for notifying the incoming call of the new video conferencing terminal, the user abbreviation of the video conferencing terminal stored in the RAM 203 in advance and the current number of participating video conferencing terminals (the TV added by this incoming call). (Including conference terminal) is notified.

【0047】

As described above, according to the present embodiment, the number of participating video conferencing terminals due to a new connection of the video conferencing terminal or a disconnection of communication with the video conferencing terminal participating in the video conference. Information on changes in the video conference can be notified to participating video conferencing terminals. Therefore, the user participating in the multipoint video conference can grasp the information regarding the change in the number of participating video conference terminals, and the user of the video conference terminal that has not been connected up to that point suddenly makes a statement. It is possible to eliminate such unnaturalness and inconvenience.

【0048】

In addition, since the terminal information is converted into at least one of the character data and the bitmap data and overlaid on the transmitted video, the user participating in the multipoint video conference can visually display the information on the change in the number of terminals. Can be obtained.

【0049】

In addition, even if the video conferencing terminal used does not have video capability or the video capability is turned off, the user participating in the multipoint video conferencing can hear information about the change in the number of terminals. Can be obtained by

【0050】

In addition, by selecting either voice notification or video notification depending on the presence or absence of the video capability of the video conferencing terminal to be notified, all participating video conferencing terminals are notified of information on changes in the number of terminals. It becomes possible.

【0051】

In this embodiment, the case where the ISDN line is adopted as the communication line for connecting each video conferencing terminal to the MCU has been described, but PSTN (analog telephone network), CSPDN (circuit switching network) and LAN (local area network) have been described. Needless to say, it can be applied to various lines such as.

【0052】

[Effect of the invention]

As described above, according to the multipoint video conferencing system of claim 1, a multipoint video conferencing system that conducts a multipoint video conferencing by connecting a plurality of video conferencing terminals to a multipoint communication control device. In the multipoint communication control device, the terminal information storage means for storing the terminal information of the video conferencing terminal participating in the multipoint video conference and the video conference participating in the multipoint video conference. When a change in the number of terminals is detected by the terminal number monitoring means for monitoring the number of terminals and the terminal number monitoring means, information on the change in the number of terminals is participated in the multipoint video conference. Since it is configured to have a terminal information notification means for notifying the video conferencing terminal, the user participating in the multipoint video conferencing can grasp the information regarding the number of participating video conferencing terminals, for example, connecting until then. It is possible to eliminate the unnaturalness and inconvenience of suddenly making a statement from the user of the video conferencing terminal that has not been done.

【0053】

Further, according to the multipoint video conferencing system of claim 2, by providing an image synthesizing means for converting information on a change in the number of terminals into at least one of character data and bitmap data and overlaying the transmitted image. Users participating in multipoint video conferencing have the effect of being able to visually obtain information on changes in the number of terminals.

【0054】

Further, according to the multipoint video conferencing system of claim 3, the video conferencing terminal to be used has a video capability because it is provided with a voice synthesis means for synthesizing information regarding a change in the number of terminals and adding it to the transmitted voice. Even if the user does not have or the video capability is turned off, the user participating in the multipoint video conferencing can obtain the effect of obtaining information on the change in the number of terminals by voice. ..

【0055】

Further, according to the multipoint video conferencing system of claim 4, the terminal information notification means converts the information regarding the change in the number of terminals into at least one of character data and bitmap data and overlays the image on the transmitted image. The synthesis means, the voice synthesis means for synthesizing the information related to the change in the number of terminals and adding it to the transmitted voice, and the video ability for identifying the video ability of each video conference terminal participating in the multipoint video conference. By having the identification means and the selection means for selecting either the image synthesis means or the voice synthesis means based on the video ability identified by the video ability identification means, the video ability of the video conferencing terminal can be increased. Since either voice notification or video notification can be selected depending on the presence or absence, there is an effect that it is possible to notify all participating video conferencing terminals of information regarding changes in the number of terminals.

[Simple explanation of drawings]

[Figure 1]

It is a block diagram which shows an example of the functional structure of the multipoint communication control device (MCU) which concerns on this Example.

[Figure 2]

It is a block diagram which shows the configuration of the MCU which can execute the functional configuration shown in FIG.

[Fig. 3]

It is a flowchart which shows the procedure of monitoring the number of participating video conference terminals which concerns on this Example.

[Fig. 4]

It is a flowchart which shows the terminal information notification processing procedure which concerns on this Example.

[Fig. 5]

It is a figure which shows an example of the received image of a participating video conference terminal.

[Fig. 6]

It is a figure which shows an example of the multipoint connection when the MCU is used for the conventional multipoint video conferencing system.

[Explanation of symbols]

1 Terminal information storage unit 2 Number of terminals monitoring unit 3 Terminal information notification unit 4 Image composition section 5 Speech synthesizer 6 Video capability identification unit 7 Notification method selection section 201 CPU (terminal number monitoring means, video capability identification means, identification means, selection means, terminal information notification means) 203 RAM (terminal information storage means) 206 Speech processing unit (speech synthesis means) 208 Image processing unit (image composition means)