Multimedia data monitoring method

0001This application is based upon and claims the benefit of priority from <patcit id="pcit0001" dnum="JP2006132795A"><text>Japanese Patent Application No. 2006-132795 filed on May 11, 2006</text></patcit>, the content of which are incorporated by reference.

0002The present invention relates to the invention of a multimedia data monitoring system and a multimedia data monitoring method for monitoring multimedia data during communications in an interchange apparatus that connects multiple networks to provide for the communications such as TV telephone communication.

0003An interchange apparatus which connects a public switched telephone network (hereinafter referred to as a PSTN) to an IP network (Internet protocol) enables the audio communications between a terminal connected to the PSTN and a terminal connected to the IP network. The related art of the interchange apparatus is disclosed in <patcit id="pcit0002" dnum="JP2005348163A"><text>Japanese Patent Laid-open Application Publication No. 2005-348163</text></patcit>. The related art refers to an audio monitoring device for monitoring the audio during communications.

0004Another related art is disclosed in <patcit id="pcit0003" dnum="JP2004022200A"><text>Japanese Patent Laid-open Application Publication No. 2004-22200</text></patcit>. The related art discloses an interchange apparatus for enabling TV telephone communications between a third generation mobile communication (hereinafter referred to as 3G-324M) terminal capable of television telephone communications (hereinafter referred to as TV telephone communications) supporting voice and video and a session initiation protocol (hereinafter referred to as a SIP) terminal. The interchange apparatus connects a network (for example, a 3G network) to which the 3G-324M terminal is connected to a network (for example, an IP network) to which the SIP terminal is connected.

0005It is relatively easy to monitor the multimedia data quality by monitoring the voice during communications. However, it is not easy to monitor, for example, the multimedia data quality of the TV telephone communication by monitoring audio or video during communications because, when multiplexed data obtained by multiplexing audio data with video data is communicated, the audio data and the video data cannot be demultiplexed unless multiplexing information is acquired, thereby failing to monitor audio or video. Although the multiplexed data can be demultiplexed into audio data and video data, the video data cannot be decoded unless the Config information can be acquired as parameter information required to decode the video data.

0006An exemplary object of the present invention is to provide a multimedia data monitoring system capable of monitoring multimedia data between plural networks including audio data or video data in real time.

0007An exemplary aspect of the invention is a data monitoring system including a data monitoring apparatus, a monitoring terminal, a first network, a second network, a first communication apparatus that is connected to the first network and a second communication apparatus that is connected to the second network, wherein the datamonitoring apparatus includes a call processing circuit for retrieving decoding information from a call processing signal transmitted from one of the first communication apparatus and the second communication apparatus to the other and for transmitting the decoding information to the monitoring terminal, and a data processing circuit for transmitting data having voice data or image data to the monitoring terminal, wherein one of the first communication apparatus and the second communication apparatus transmits the data to the other, and wherein the monitoring terminal includes a decoding circuit for decoding the data in accordance with the decoding information.

0008According to the present invention, multimedia data between plural networks including audio data or video data can be monitored in real time. The invention is further described with reference to the drawings: <ul id="ul0001" list-style="none" compact="compact"><li>FIG. 1 is a system diagram illustrating an example of the communication system to which a multimedia data monitoring system is applied;</li><li>FIG. 2 is a system diagram illustrating a first exemplary embodiment of the data monitoring apparatus;</li><li>FIG. 3 is an explanatory view showing an example of the information held in an information table unit;</li><li>FIG. 4 is a flowchart showing the operation of the data monitoring apparatus and a monitor terminal;</li><li>FIG. 5 is a system diagram illustrating another example of the communication system to which a multimedia data monitoring system is applied; and</li><li>FIG. 6 is a system diagram illustrating a second exemplary embodiment of the data monitoring apparatus.</li></ul>

1. First Exemplary Embodiment

0009The first exemplary embodiment according to the present invention is described below by referring to FIG. 1. FIG. 1 is a system diagram illustrating an example of the communication system to which a multimedia data monitoring system according to the present invention is applied. The communication system includes a data monitoring apparatus 1 for connecting multiple networks, a monitor terminal 2 for decoding and outputting multimedia data, a 3G network 3, an IP network 4, a 3G-324M terminal 5 connected to the 3G network 3, and a SIP terminal 6 connected to the IP network 4. In the embodiments, an IP network and a 3G network are used as exemplary heterogeneous networks.

0010The data monitoring apparatus 1 functions as an interchange apparatus for connecting the 3G network 3 to the IP network 4 to realize the TV telephone communication performed by the 3G-324M terminal 5 and the SIP terminal 6. The data monitoring apparatus 1 also determines the multiplexing information (the multiplexing information can also referred to as multiplexing table information) about multimedia data and the Config information about video and transfers them to the monitor terminal 2 when the communication protocol of a call processing signal is converted. The multiplexing information about the multimedia data and the Config information about video are information required to decode multimedia data.

0011FIG. 2 is a diagram illustrating an example of the data monitoring apparatus 1 according to the present exemplary embodiment. As shown in FIG. 2, the data monitoring apparatus 1 includes a data processing circuit 11, a call processing circuit 12, an information table unit 13, a 3G network processing circuit 14, an IP network processing circuit 15, and a monitor processing circuit 16.

0012The 3G network processing circuit 14 terminates the data received from the 3G network 3. Practically, for example, the 3G network processing circuit 14 performs a process of extracting multimedia data from the received data. Then, if the data received from the 3G network 3 is a call processing signal, the 3G network processing circuit 14 passes the data to the call processing circuit 12, When the received data is multimedia data, it passes the data to the data processing circuit 11.

0013The call processing circuit 12 converts the communication protocol of the call processing signal received from one terminal of the 3G-324M terminal 5 or SIP terminal 6. For example, the call processing circuit 12 converts the call processing signal received from the 3G network processing circuit 14 into a signal on the basis of the SIP protocol as a call processing signal of the SIP terminal 6, and outputs the result to the IP network processing circuit 15. For example, when the communication protocol of an input call processing signal is converted, the call processing circuit 12 extracts the multiplexing information about multimedia data or the Config information about video from the call processing signal. Then, it transmits the extracted multiplexing information or Config information to the information table unit 13 or the data processing circuit 11. Practically, for example, it transmits the multiplexing information about the multimedia data or Config information about video determined by the negotiation on the basis of the H. 245 to the information table unit 13 or the data processing circuit 11.

0014The data processing circuit 11 converts the format of the multimedia data input from the 3G network processing circuit 14 or the IP network processing circuit 15 on the basis of the multiplexing information or the Config information transmitted from the call processing circuit 12. Then, the data processing circuit 11 outputs the format-converted multimedia data to the 3G network processing circuit 14 or the IP network processing circuit 15.

0015For example, the IP network processing circuit 15 converts a SIP protocol signal input from the call processing circuit 12 for the UDP/IP, and transmits the result to the IP network 4. Furthermore, for example, the IP network processing circuit 15 converts the multimedia data input from the data processing circuit 11 for the RTP/UDP/IP, and transmits the result to the IP network 4. Additionally, the IP network processing circuit 15 terminates the data received from the IP network 4. When the data received from the IP network 4 is a call processing signal, the IP network processing circuit 15 outputs data to the call processing circuit 12. When the IP network processing circuit 15 receives audio or video data, it outputs the data to the data processing circuit 11.

0016The call processing circuit 12 converts the call processing signal input from the IP network processing circuit 15 into a signal on the basis of the H. 245 protocol as a call processing signal of the 3G-324M terminal 5, and outputs the result to the 3G network processing circuit 14. When the call processing circuit 12 converts the communication protocol of the call processing signal, it extracts the Config information about the video of the SIP terminal 6 from the SIP protocol signal as a call processing signal of the SIP terminal 6. Then, the call process ing circuit 12 transmits the Config information to the information table unit 13 and the data processing circuit 11.

0017The data processing circuit 11 converts the format of the audio or video data input from the IP network processing circuit 15 into the data format of the 3G network 3 on the basis of the Config information about the video transmitted from the call processing circuit 12. Then, the data processing circuit 11 outputs the format-converted audio or video data to the 3G network processing circuit 14.

0018The 3G network processing circuit 14 multiplexes, for example, an H. 245 protocol signal received from the call processing circuit 12 into the H. 223, and transmits the result to the 3G network 3. The 3G network processing circuit 14 multiplexes, for example, the audio or video data received from the data processing circuit 11 into the H. 223, and transmits the result to the 3G network 3.

0019When call connection control of the TV telephone communication between the 3G-324M terminal 5 and the SIP terminal 6 is completed, the call processing circuit 12 transmits a signal as a request to transmit an intra-frame in a predetermined time period to the 3G-324M terminal 5 and the SIP terminal 6. The "intra-frame" refers to a frame for which a compressing process is performed using only the information in the same frame independent of the preceding and subsequent frames in multiple sequentially arranged images (hereinafter referred to as frames) . The 3G-324M terminal 5 and the SIP terminal 6 transmit an intra-frame when a request to transmit an intra-frame is issued from the call processing circuit 12. In the above-mentioned embodiment, the call processing circuit 12 has the configuration in which a signal of a request to transmit an intra-frame in a predetermined time period is issued, but it is not necessary that a signal of a request to transmit an intra-frame is to be issued in a predetermined time period, but it can be issued intermittently.

0020When the monitor processing circuit 16 receives a communication status confirmation instruction signal from the monitor terminal 2, it confirms the information held by the information table unit 13. Then, it transmits the data obtained by copying the channel number during the TV telephone communication to the monitor terminal 2. Furthermore, when the monitor processing circuit 16 receives the designation of the channel of a monitor object from the monitor terminal 2, it copies the multiplexing table information or the Config information about the designated channel in the information held by the information table unit 13, and transmits the copied information to the monitor terminal 2. Then, the monitor processing circuit 16 copies the multimedia data received from the 3G network 3 from the 3G network processing circuit 14, or the multimedia data received from the IP network 4 from the IP network processing circuit 15, and transfers the copied data to the monitor terminal 2. That is, it starts the process of transferring to the monitor terminal 2 the multimedia data from the 3G network 3 to the IP network 4 or the multimedia data from the IP network 4 to the 3G network 3.

0021FIG. 3 is an explanatory view showing an example of the information held in the information table unit 13. As shown in FIG. 3, the information table unit 13 stores the multiplexing information or the Config information for each communication terminal after associating the information with a channel number during the communications. Then, the information table unit 13 continues storing the information such as the multiplexing information, the Config information, during the call connection. In FIG. 3, "LCN:0 ..." indicates an example of the contents of the multiplexing information, and "000001B0080 ..." indicates an example of the contents of the Config information.

0022The monitor terminal 2 can demultiplexes multimedia data into video data and audio data according to the multiplexing information about the received multimedia data. Then, the monitor terminal 2 can decode the video data from the Config information about video, and outputs the result. Thus, the monitor terminal 2 can perform multimedia data monitor (retrieve audio data or video data, or audio data and video data from the multimedia data received from the monitor processing circuit 16, and output audio or video, or audio and video on the basis of the retrieved data) in real time. Furthermore, the monitor terminal 2 decodes an intra-frame received from the 3G-324M terminal 5 and the SIP terminal 6 in a predetermined time period. Therefore, the monitor terminal 2 can output clear videos in a predetermined time period. In addition, the monitor terminal 2 can transmit a communication status confirmation instruction signal to the data monitoring apparatus 1 to confirm whether or not there is a call during the TV telephone communication. If TV telephone communications have been started when the monitor terminal 2 transmits an instruction signal to the data monitoring apparatus 1, the monitor terminal 2 can output a clear video upon receipt of an infra-frame, and the monitoring process can be started on the multimedia data.

0023The 3G network 3 and the IP network 4 are communication networks using different communication protocols. When data is communicated between the 3G network 3 and the IP network 4, the data monitoring apparatus 1 converts the communication protocol of multimedia data, and format-converts multimedia data.

0024Described below are the operations of the data monitoring apparatus 1 and the monitor terminal 2 in the present exemplary embodiment. FIG. 4 is a flowchart of the operations of the data monitoring apparatus 1 and the monitor terminal 2 according to the present exemplary embodiment.

0025The monitor terminal 2 transmits a communication status confirmation instruction signal to the monitor processing circuit 16 to inquire whether or not there is a call during TV telephone communications (step S1). Upon receipt of the communication status confirmation instruction signal, the monitor processing circuit 16 confirms the information held by the information table unit 13. Then, the monitor processing circuit 16 copies the channel number in TV telephone communications, and transmits the copied number to the monitor terminal 2 (step S2). When there is no channel in the TV telephone communication, the monitor terminal 2 terminates the multimedia data monitor.

0026When there is a channel in the TV tel ephone communications, a user selects the channel number to be monitored and a monitor object (for example, of the 3G-324M terminal 5 or the SIP terminal 6) from among the channels in the TV telephone communications (step S3) . The monitor terminal 2 transmits to the monitor processing circuit 16 the information indicating the selected monitor object and the channel number as an instruction signal. The information indicating the monitor object refers to the multimedia data received from the 3G-324M terminal 5, or the multimedia data received from the SIP terminal 6.

0027Upon receipt of the information designating the channel number, the monitor processing circuit 16 copies the multiplexing information or the Config information about the channel number monitored from the information table unit 13, and transmits the copied information to the monitor terminal 2 (step S4). If the information indicating the monitor object refers to the 3G-324M terminal 5, the monitor processing circuit 16 confirms whether or not the multimedia data of the channel number of a monitor object in the multimedia data received from the 3G network 3 is being received. If the information indicating a monitor object refers to the SIP terminal 6, the monitor processing circuit 16 confirms whether or not the multimedia data of the channel number to be monitored in the multimedia data received from the IP network 4 is being received (step S5). When a call is disconnected, the call processing circuit 12 deletes the information about the channel of a monitor object stored in the information table unit 13. Furthermore, the information table unit 13 notifies the monitor processing circuit 16 that the call of the channel of a monitor object has been disconnected. Thus, it is confirmed that the user does not receive multimedia data.

0028If no multimedia data is received on the specified channel, the monitor processing circuit 16 notifies the monitor terminal 2 that multimedia data has not been received. If the monitor terminal 2 is notified that no multimedia data has been received, it terminates the multimedia data monitor.

0029If multimedia data is being received on the specified channel, the monitor processing circuit 16 copies the multimedia data and transmits it to the monitor terminal 2 (step S6). If the monitor object is in the 3G-324M terminal 5, the monitor processing circuit 16 copies the multimedia data from the 3G network processing circuit 14, and transfers the copied data to the monitor terminal 2. If the monitor object is in the SIP terminal 6, the monitor processing circuit 16 copies the multimedia data from the IP network processing circuit 15, and transfers the copied data to the monitor terminal 2.

0030If the monitor object is in the 3G-324M terminal 5, the monitor terminal 2 receives the multimedia data, and demultiplexes the multimedia data into audio data and video data according to the already obtained multiplexing information (step S7). Then, the monitor terminal 2 decodes the audio data. Furthermore, the monitor terminal 2 decodes video data according to the already obtained Config information (step S8). The monitor terminal 2 outputs audio on the basis of the decoded audio data, or video on the basis of the video data (step S9). The user can monitor whether or not there is no problem with the multimedia data from the output audio and video. The information table unit 13 does not store the multiplexing information about the multimedia data relating to the IP network 4. Therefore, when the SIP terminal 6 includes a monitor object, it is not necessary to perform the process in step S7.

0031The monitor processing circuit 16 copies the multimedia data from the 3G network processing circuit 14 or the IP network processing circuit 15 and continues transmitting the data until no multimedia data is received on the specified channel. Thus, the monitor terminal 2 can continue monitoring the multimedia data in real time. When no multimedia data is received on the specified channel, the monitor processing circuit 16 notifies the monitor terminal 2 that multimedia data is not received. When the monitor terminal 2 is notified that multimedia data is not received, it terminates the multimedia data monitor.

0032In this exemplary embodiment, if the multimedia data quality is degraded by the process of monitoring multimedia data in the TV telephone communication, then a suspected portion of the degradation of multimedia data can be segmented. For example, if video data being monitored is disturbed when a monitor object is in the 3G-324M terminal 5, it is considered that there is a possibility that anywhere from the 3G-324M terminal 5 to the 3G network processing circuit 14 includes a factor of disturbing the video data, and a suspected portion can be segmented. If the monitor object is in the SIP terminal 6, and the video data being monitored is disturbed, then it is considered that there is a possibility that anywhere from the SIP terminal 6 to the IP network processing circuit 15 includes a factor of disturbing the video data, and a suspected portion can be segmented.

0033In the present exemplary embodiment, both audio and video are monitored. However, a suspected portion of multimedia data degradation can be segmented by monitoring only one of them.

2. Second Exemplary Embodiment

0034Next, the second exemplary embodiment of the present invention is described by referring to FIG. 5. FIG. 5 is a system diagram illustrating another example of the communication system to which a multimedia data monitoring system is applied. The second exemplary embodiment is different from the first exemplary embodiment in that, for example, the an ISDN (Integrated Services Digital Network) 7 and PSTN 8 replace the 3G network 3, an ISDN terminal 9 replaces the 3G-324M terminal 5, and a data monitoring apparatus 1A replaces the data monitoring apparatus 1. The configuration and the operations of the monitor terminal 2, the IP network 4, and the SIP terminal 6 are the same as the configuration and the operations of the monitor terminal 2, the IP network 4, and the SIP terminal 6 according to the first exemplary embodiment.

0035In FIG. 5, the ISDN terminal 9 can perform TV telephone communications. The data monitoring apparatus 1A connects the PSTN 8 to the IP network 4 to enable the TV telephone communication between the ISDN terminal 9 and the SIP terminal 6 by way of the ISDN 7.

0036FIG. 6 is a system diagram illustrating a second exemplary embodiment of the data monitoring apparatus 1A. The second exemplary embodiment is different from the first exemplary embodiment in that a PSTN processing circuit 17 replaces the 3G network processing circuit 14 shown in FIG. 2. The configuration and the operations of the data processing circuit 11, the call processing circuit 12, the information table unit 13, the IP network processing circuit 15, and the monitor processing circuit 16 are the same as the configuration and the operations of the data processing circuit 11, the call processing circuit 12, the information table unit 13, the IP network processing circuit 15, and the monitor processing circuit 16 according to the first exemplary embodiment.

0037The PSTN processing circuit 17 terminates the data received from the PSTN 8. Practically, the PSTN processing circuit 17 performs processes such as a process of extracting multimedia data from received data. Additionally, the PSTN processing circuit 17 passes data to the call processing circuit 12 when the data received from the PSTN 8 is a call processing signal. When the received data is multimedia data, it passes the data to the data processing circuit 11. Furthermore, the PSTN processing circuit 17 multiplexes, for example, the H. 245 protocol signal input from the call processing circuit 12 into the H. 223, and transmits the result to the PSTN 8 . The PSTN processing circuit 17 also multiplexes the audio or video data input from the data processing circuit 11 into the H. 223, and transmits the result to the PSTN 8.

0038Described next is the operations according to the second exemplary embodiment of the present invention. As compared with the operations according to the first exemplary embodiment, the operations according to the second exemplary embodiment includes the ISDN terminal 9 replacing the 3G-324M terminal 5, the ISDN 7 and the PSTN 8 replacing the 3G network 3, and the PSTN processing circuit 17 replacing the 3G network processing circuit 14.

0039In this exemplary embodiment, a suspected portion of degraded multimedia data quality can be segmented when the multimedia data degradation occurs by the process of monitoring the multimedia data being used in the TV telephone communications. In the second exemplary embodiment, both audio and video are monitored, but one of them can be monitored with a suspected portion of multimedia data degradation correctly segmented.

0040In the first exemplary embodiment, the 3G network 3 and the IP network 4 are exemplified as different networks while, in the second exemplary embodiment, the ISDN 7 and PSTN, and the IP network are exemplified as different networks. However, the different networks are not limited to these types. That is, the present invention can be applied to the multimedia data monitoring system capable of performing communications such as TV telephone communications using other types of networks. For example, the present invention can also be applied to a multimedia data monitoring system capable of performing TV telephone communications by connecting the ISDN with the IP network, the ISDN with a network of an Internet service provider (ISP), or the 3G network with the ISP network. Furthermore, the portions recognized as different networks can be realized by the same types of networks.

0041Although multimedia data is described above, the present invention can also be applied to only audio data etc.

0042The previous description of these embodiments is provided to enable a person skilled in the art to make and use the present invention. Moreover, various modifications to the embodiments will be readily apparent to those skilled in the art, and the generic principles and specific examples defined herein may be applied to other embodiments without the use of inventive faculty. Therefore, the present invention is not intended to be limited to the embodiments described herein but is to be accorded the widest scope as defined by the limitations of the claims and equivalents.