Caller confirmaion system based on image

Video-based caller identification system {CALLER CONFIRMAION SYSTEM BASED ON IMAGE}

1 is a schematic configuration diagram of an IP Multimedia Subsystem (IMS);

2 is a view showing an example of a general invite message (INVITE message);

3 is a diagram illustrating an example of an invite message including a caller identification image according to the present invention;

4 is a block diagram of a transmitting and receiving terminal device according to the first embodiment of the present invention;

5 is a flowchart of an image-based caller identification method of the present invention;

6 is a block diagram of a transmitting and receiving terminal device according to a second embodiment of the present invention;

7 is a block diagram of a transmission/reception terminal device according to a third embodiment of the present invention;

<Explanation of symbols for main parts of the drawing>

100: transmitting terminal device 110: application

120: SIP configuration process module 130: video format conversion module

140: protocol stack 150: transmission module

160: camera 170: codec

200: receiving terminal device 210: application

220: SIP analysis process module 230: video format conversion module

240: protocol stack 250: receiving module

260: codec 270: display unit

<backgroundart><p>The present invention relates to a video-based caller identification apparatus and method for identifying a caller based on an image.</p><p>Thanks to the development of information and communication technology, various and rich information such as voice/music, video, stock, and weather are freely provided anytime, anywhere, from fixed terminals to mobile terminals, based on wired/wireless communication systems. . In particular, in the case of wired/wireless communication, a function is provided in the terminal so that the receiver can check information about the sender before communication connection.</p><p>For example, during a phone call, a caller's phone number display function is provided so that the receiver can check the caller's phone number before connecting the call. This function pre-displays the caller's phone number on the recipient's phone before connecting the call, making it an important consideration in determining whether the recipient will view the phone number and call or reject the call. However, since this function only displays the other party's phone number from the receiver's point of view, there is a limitation in that it is difficult to recognize the phone number by matching it with the other party.</p><p>In order to solve this problem, a function is provided on the terminal to compare the caller's phone number and the phone number registered in the phone book in the receiver's terminal and display the matching person's name (or company name, etc.) together. However, even in this case, if the person's name and phone number are not registered (stored) in the phone book of the recipient's terminal, the above-described problem remains, and immediately recalling who the person is by looking at the person's name is a normal call or contact. This only applies to people who have a high frequency, and in the case of a person who is relatively difficult to remember at first glance due to the low frequency of calls or contact, the inconvenience remains because they have to remember who they are by looking at the displayed names.</p><p>As another function provided by the mobile carrier, the caller registers text information representing himself in advance with the mobile carrier (server) and transmits it to the other party every time he makes a call. A service is also being provided to let the other party know in advance who the caller is by displaying the corresponding text. However, this service is available only for those who have subscribed to the service, and since the text information is also not necessarily a person's name, the problem described above remains as it is.</p><p>In a way to make it easier to recognize the caller on the mobile terminal, the person's name and phone number are registered together with the person's picture in the phone book inside the terminal, and the other party's phone number and terminal provided by the mobile carrier (server) There is a method in which a picture of a matching person is displayed prior to a call by comparing and searching for phone numbers stored in the phone book within. However, in this case also, since the person's photo, etc. can be displayed only for the person registered in advance, if there is no photo representing that person even if it is registered (stored) in the phone book or registered in the phone book, The problem described remains the same. In particular, in order to use this function, it is necessary to secure a photo for each person registered in the phone book on the terminal. This big one is the problem.</p><p>A caller identification video display system with a concept of allowing the receiving terminal to identify the caller before a call by transmitting the user image acquired from the sending terminal to the receiving terminal through a wired/wireless communication line in a method that can identify the caller based on the conventional image This has also been suggested.</p><p>One of the techniques proposed in this caller identification video display system is to encode and transmit the video of the person calling (caller) directly from the sending terminal. A method of extracting features of a sender's face image and transmitting only the feature information, and the receiving side uses the received facial feature information to compare it with a pre-stored face image and display the matching image as a face image for sender identification am.</p><p>In the former case, the caller's face image is directly displayed on the receiving terminal, so the identity can be immediately and accurately identified. The downside is that you can bring it. In the latter case, since only facial feature information is transmitted, there is an advantage in that the amount of data to be transmitted is small, but a means necessary for facial region extraction and recognition is required separately, and depending on the face recognition performance of the receiving terminal, an error may occur in accurate image recognition of the sender. The downside is that it is highly probable.</p></backgroundart><abstractproblem><p>The present invention provides a method for confirming the identity of a caller based on an image, particularly in an IMS (IP Multimedia Subsystem) environment, so that a terminal device can confirm the identity of the caller using a video sent by the other party before connecting a call. An object of the present invention is to provide a video-based caller identification system.</p><p>In addition, the present invention utilizes the IMS standard, so that the sending terminal can check the video transmitted by the sending terminal before the session connection, the sending terminal sends the video information to the SIP Initiation Protocol INVITE message. and the receiving terminal interprets and displays the image information included in the SIP INVITE message received from the sending terminal, so that the image information is included in the existing SIP INVITE message. It does not require a separate line for video transmission or a separate process for video transmission only, and provides a video-based caller identification system that allows the caller's identity to be accurately recognized without errors based on the video sent by the actual sender. but it has a purpose.</p></abstractproblem>

<p>The video-based caller identification method of the present invention for achieving the above object, </p><p>A system for allowing a receiver to check a video transmitted by a caller before a call connection in an IP Multimedia Subsystem (IMS) environment, the system comprising: transmitting, by a transmitting terminal, including predetermined video information in a SIP INVITE message; interpreting and displaying, by a receiving terminal, image information included in a SIP INVITE message received from the transmitting terminal; It is characterized in that it includes.</p><p>In addition, the transmitting terminal device according to the present invention is a system in which the receiver can check the video transmitted by the caller in the IMS environment before the call connection,</p><p>It is characterized in that it comprises a message generating means for composing a message by including image information in a SIP INVITE message, and a transmitting means for transmitting the configured message.</p><p>In addition, the receiving terminal device according to the present invention is a system in which the receiver can check the video transmitted by the caller in the IMS environment before the call connection,</p><p>A message receiving means for receiving a SIP INVITE message, a data analyzing means for interpreting the image information included in the received SIP INVITE message, and the analyzed image information on a screen It is characterized in that it comprises a display means for displaying.</p><p>In addition, in the present invention, the SIP INVITE message is characterized in that it is delivered via a CSCF (Call Session Control Function) of the IMS.</p><p>In addition, in the present invention, the image information is characterized in that it is recorded in the "opaque" field in the SIP INVITE message (SIP INVITE message).</p><p>In addition, in the present invention, the image information is converted into a text format and recorded in an "opaque" field in a SIP INVITE message.</p><p>In the present invention, the image information is acquired immediately before transmission, and the acquired image is compressed and included in a SIP INVITE message.</p><p>In addition, the present invention obtains the contents of the "opaque" field from the received SIP INVITE message, converts it into a binary format, and displays the restored image by restoring the converted binary file by a decoder. characterized in that</p><p>Also, in the present invention, the binary file is a JPEG image.</p><p>In addition, in the present invention, the image information is an image acquired during transmission or an image stored in advance, and for this purpose, it is characterized in that it further includes a camera and a memory for image acquisition.</p><p>In addition, in the present invention, a format conversion means for converting the compressed image into a text (TEXT) format or converting data in a text (TEXT) format into a binary format, and image data in a compressed format converted into a binary format It is characterized in that it further comprises a decoder for reconstructing . </p><p>In addition, the transmission/reception terminal device according to the present invention is a system in which a receiver can check an image transmitted by a caller before a call connection in an IMS environment,</p><p>Video format conversion means for converting video data to check the other party before a call connection into a text (TEXT) format or converting received text format video data into a binary format, and converting the video data converted into the text format into a SIP invite message (SIP INVITE message) SIP configuration and interpretation means for composing a message by writing in a predetermined field or interpreting text format video data in a corresponding field of a received invite message, and the invite message in which the text format video data is recorded It is characterized in that it comprises a transmission/reception means for transmitting and receiving, and a display means for displaying an outgoing image received and interpreted before a call connection from the other party through the transmission/reception means and restored through conversion.</p><p>An embodiment of the video-based caller identification system in the IMS environment of the present invention made as described above will be described in detail with reference to the accompanying drawings.</p><p>First, a brief look at the IMS environment.</p><p>Standard activities to support various services by connecting IP networks capable of rich services using wireless mobile communication terminals are being actively carried out centering on the 3rd Generation Partnership Project (3GPP), the European standard for 3G mobile communication. This is called IMS, and this standard mainly defines which server can be used for connection between different terminals. That is, it defines which components are required and through which sequence the IP network can be connected.</p><p>The present invention provides specific means so that a terminal device can provide a caller identification service in such an IMS environment. Since the means proposed in the present invention satisfies the IMS standard, it can be applied regardless of the type of terminal device in the IMS environment.</p><p>In the present invention, data transmitted by the transmitting terminal device before a call connection is described as an image, but this may include any still image, video or text (TEXT) format and audio stored in advance in the terminal device, etc. It should be noted that, in addition to the face image, the user can directly configure or select any image that is serviced.</p><p>1 schematically shows an IMS system.</p><p>Referring to Figure 1, IMS is an application server (Application Server) 10, HSS (Home Subscriber Server) (20), CSCF (Call Session Control Function) (30), mobile communication network (3G, GPRS (General Packet Radio Service)) , CDMA) 40 , user equipment 50 , MGCF (Media Gateway Control Function) 60 , MGW (Media Gateway) 70 , and the like.</p><p>The HSS 20 has subscriber information and provides corresponding subscriber information according to a request, and the CSCF 30 performs functions such as an incoming call gateway, call control, serving profile management and address processing. Here, the Serving-CSCF (S-CSCF) 31 serves as a register and performs a function of controlling a session of a registered endpoint. In addition, the I-CSCF (Interrogating-CSCF) 32 is an access point when an incoming call to a subscriber terminal occurs, and when a call is requested from an arbitrary terminal, it controls registration of the requested call content and session flow, etc., and the S-CSCF functions such as assigning to the SIP registration executed by the user. The P-CSCF (Proxy-CSCF) 33 serves as a proxy and a user agent.</p><p>The MGCF 60 is in charge of transmission control and the like with an external public wired communication network (PSTN), and the MGW 70 is for connection with the public wired communication network (PSTN).</p><p>GGSN (Gatway GPRS System Node) 41 is a wireless switching network and a wireless switching network so that when a wireless terminal wants to perform high-speed packet communication, it passes through a wireless switching network including a base station and then accesses a general Internet network again to perform data communication with a destination. It plays a role of a gateway for interworking with the Internet network, and a Serving GPRS Support Node (SGSN) 42 is in charge of tracking the mobile node, which is a tunnel through which packets are exchanged with the GGSN.</p><p>In the IMS shown in FIG. 1 , each terminal may be connected to a different home network, and the terminal and the home network are connected through a mobile communication network GGSN (41)/SGSN (42). In order to transmit/receive with another terminal device, a session should be connected first, and data transmission/reception may be performed after the session connection. In the present invention, when there is a call request before actual transmission/reception occurs, the receiving side must transmit an image so that the receiving side can confirm the transmitting side in advance.</p><p>In order to explain this, a standard step of connecting a session in the IMS system shown in FIG. 1 will be described.</p><p>First, the transmitting terminal must make a request for connecting a session to the receiving terminal. For this, first, the sender must be registered with the server located in the IMS network, or there must be an authentication procedure to determine whether it is already registered, and then a session connection for actual incoming and outgoing calls is performed.</p><p>The session connection steps are as follows.</p><p>First, the transmitting terminal transmits an INVITE message of the SIP to the corresponding CSCF connected to the transmitting terminal. The receiving terminal address information is included in the INVITE message. The CSCF connected to the transmitting terminal transmits an INVITE message to the CSCF connected to the receiving terminal, and the receiving CSCF receives the INVITE message and transmits an INVITE message to the receiving terminal. Then, the receiving terminal will ring a ringtone. Then, when the receiving terminal grants reception permission, the receiving terminal sends a message indicating reception permission to the sending terminal.</p><p>Although the above-described example describes a case in which a user manually connects a session, other examples such as automatic connection are described in IMS. In the present invention, the type of connection method is not limited, and since it uses a characteristic of transmitting an INVITE message in common, it can be applied to various connection methods without difficulty.</p><p>In order for the receiving terminal to receive the video for caller identification, as described above, transmission and reception of the caller identification video must be performed at the time the phone rings, so information required when the INVITE message arrives, that is, the caller identity in the present invention. The video to be transmitted by the sender must be received along with it so that it can be confirmed. To this end, in the present invention, sender identification image information is included in the SIP INVITE message.</p><p>An example of an INVITE message structure is shown in FIG. 2 . Referring to FIG. 2, a necessary address and each piece of information are described as a header, and necessary detailed description information is included.</p><p>3 shows the structure of an INVITE message including confirmation image information according to the present invention. Among the headers that can be included in the INVITE message, there is an "authorization header", and in this header field, there is an "opaque" field. In this "opaque" field, arbitrary text (TEXT) is arbitrary. length can be included.</p><p>In the present invention, an INVITE message is configured by including image information for sender identification in the "opaque" field. The part expressed in bold type in FIG. 3 is caller identification image information recorded in text (TEXT) format in the "opaque" field. To this end, in the transmitting terminal, a module that converts the image to be transmitted into text format to be included in the "opaque" field, and a SIP configuration module that includes it in the "opaque" field to complete the INVITE message. necessary. Conversely, the receiving side recognizes the "opaque" field in the INVITE message and a SIP analysis module that extracts the corresponding data, and the content (TEXT) of the "opaque" field to be read as an image again. It contains modules.</p><p>A terminal device capable of such a function is shown in FIG. 4 . The process of including, interpreting, and displaying the confirmation image during transmission/reception is shown in FIG. 5 .</p><p>As shown in FIG. 4, the transmitting terminal device 100 according to the present invention includes a mobile communication-related application (Applications) 110, a SIP configuration process module 120 for SIP configuration, and an invite message (INVITE). A video format conversion module 130 for converting video data in a binary format to a text format in order to include video information in the message), and a protocol stack for a series of control processes for communication control. ) 140 , and a transmission module 150 for data transmission. The protocol stack 140 includes a SIP 141 for session connection, a Real time Transport Protocol (RTP)/Real Time Control Protocol (RTCP) 142 that is a protocol stack required to transmit media, and a User Datagram (UDP). Protocol)/Transmission Control Protocol (TCP) 143 and Internet Protocol (IP) 144, which are well-known standard protocols, and thus will not be described in detail in the present invention. The transmitting terminal device 100 may include a memory for storing image data and the like.</p><p>As shown in Figure 4, the receiving terminal device 200 according to the present invention, </p><p>Mobile communication-related applications (Applications) 210, SIP analysis process module 220 for SIP analysis, and image information included in an INVITE message in text (TEXT) format are binary An image format conversion module 230 for converting format image data, a protocol stack 240 for a series of control processes for communication control, a receiving module 250 for data reception, and an image display and a display unit 270 . The protocol stack 240 includes a SIP 241 for session connection, a Real time Transport Protocol (RTP)/Real Time Control Protocol (RTCP) 242 that is a protocol stack required to transmit media, and a User Datagram (UDP). Protocol)/Transmission Control Protocol (TCP) 243 and Internet Protocol (IP) 244 , which are well-known standard protocols and are not described in detail in the present invention.</p><p>A method of confirming an outgoing video according to the present invention will be described with reference to the transmitting terminal device 100 and the receiving terminal device 200 of FIG. 4 and the flowchart of FIG. 5 .</p><p>First, a connection is attempted from the transmitting terminal device 100 to the receiving terminal device 200 (step S501). Then, the image for confirmation is converted into text (TEXT) (step S502). Here, the image for confirmation may be an arbitrary still image or a moving image previously stored in a memory (not shown). The image data is image data in a binary format, and the image format conversion module 130 converts it into a text format. The reason is that, as described above, arbitrary text (TEXT) can be included with an arbitrary length in the "opaque" field in the SIP INVITE message. Inserting binary format image data into the "opaque" field To convert it to text format.</p><p>The image data converted into text format by the image format conversion module 130 is inserted into an INVITE message by the SIP configuration process module 120 (step S503). That is, the image data converted to the text format is recorded in the "opaque" field in the header of the SIP INVITE message. As described above, the INVITE message in which the video data for sender identification is recorded in a text format is transmitted to the receiving terminal device 200 through the IMS network through the transmission module 150 .</p><p>The receiving terminal device 200 receives and interprets it and displays the caller identification image. First, the receiving module 250 operates to receive the INVITE message (step S505). The received invite message (INVITE message) is interpreted by the SIP interpretation process module 220 . That is, the SIP analysis process module 220 separates the outgoing video data converted into text (TEXT) in the "opaque" field of the INVITE message (step S506). Then, the video data in the text format is converted into video data in a binary format by the video format conversion module 230 (step S507). In this way, the outgoing video converted into the original video data is displayed by the display unit 270 before the call connection (step S508).</p><p>Accordingly, it is possible to check the outgoing video before connecting the call.</p><p>As described above, the image transmitted for sender identification may be an image (photo, etc.) acquired at the time of transmission, or may be any still image or moving image stored in the memory in advance. It can be automatically selected or selected in the step immediately before transmission. In the case of a still image, a JPEG format image is used in the present invention. Therefore, the receiving terminal device should further include a JPEG decoder capable of decoding and displaying the received JPEG image, and when acquiring the image, the transmitting terminal should further include a camera and a JPEG encoder, which is shown in FIG. 6 .</p><p>That is, as shown in FIG. 6 , a camera 160 for acquiring an image is included in the transmitting terminal device 100 , and codecs 170 for compressing and encoding image data acquired by the camera 160 . ) is included. Here, the codec 170 may be a JPEG encoder according to the above example. The receiving terminal device 200 includes a codec 260 for decoding and displaying an outgoing image that is compression-encoded and transmitted by the transmitting terminal device. Here, the codec 260 may be a JPEG decoder according to the previous example.</p><p>Accordingly, the transmitting terminal device 100 according to the present invention shown in FIG. 6 converts the image acquired by the camera 160 into a text format in the image format conversion module 130 at the time of transmission to receive an invite message (INVITE). message) can be included and transmitted. In addition, image data is compressed and encoded in JPEG format using the codec 170 for a previously stored image or an image acquired at the time of transmission, and the compressed-encoded image data is converted into text (TEXT) in the image format conversion module 130 It can be converted to a format and transmitted by being included in an INVITE message.</p><p>Correspondingly, the receiving terminal device restores the image data in the received and interpreted INVITE message as follows. First, the received invite message (INVITE message) is interpreted by the SIP analysis process module 220, and the image data in the text (TEXT) format of the "opaque" field separated here is converted into binary ( binary) format, and since the converted image data is compressed in JPEG, etc. on the transmitting side, it is decoded by the codec 260-JPEG decoder, restored to the original image, and displayed on the display unit 270. will be.</p><p>The transmitting terminal device and the receiving terminal device of the present invention shown in FIGS. 4 and 6 may be integrated into one transmitting/receiving terminal device. In other words, including application and protocol stack, SIP configuration process module, SIP analysis process module, binary/text and text/binary image format conversion module, and transmission/reception module are integrated into one terminal device. A mobile communication terminal device may be implemented. In addition to this, the camera can be a component. 7 shows the configuration of a mobile communication terminal according to the present invention.</p><p>As shown in FIG. 7 , the transmitting/receiving terminal device 300 according to the present invention records the mobile communication-related application 310 and the image data converted into text format in a predetermined field of the SIP INVITE message. The SIP configuration and interpretation process module 320 that composes a message or interprets the text format video data in the corresponding field of the received invite message, and converts or receives video data to check the other party before making a call into text (TEXT) format An image format conversion module 330 for converting the image data in the text format into a binary format, a protocol stack 340, and a transmission/reception module 350 for transmitting and receiving an invite message in which the image data in the text format is recorded And, a camera 360 for image acquisition, A codec (CODECs) 370 that compresses outgoing image data to be recorded in the INVITE message in text format or recovers format-converted image data separated from the received INVITE message and a display unit 380 for displaying an outgoing image received from the other party through the transmission/reception module 350 before a call is connected, interpreted and restored through conversion.</p><p>When the transmission/reception confirmation image is transmitted from the transmitting/receiving terminal device according to the present invention configured as described above, the image captured by the camera 360 or a pre-stored image is compressed into a format such as JPEG in the codec 370, and the image format conversion module ( 330), the SIP configuration and interpretation process module 320 inserts the message into the "opaque" field of the INVITE message to complete the message, and transmits the message through the transmission/reception module 350 .</p><p>Meanwhile, the INVITE message transmitted from the counterpart is received by the transmitting/receiving module 350, interpreted by the SIP configuration and interpretation process module 320, and text recorded in the "opaque" field of the INVITE message. The video data in the format is separated, the video format conversion module 330 converts it to a binary format, and the codec 370 decompresses it and displays it on the display unit 380, so that the video sent by the other party before the call is connected. can be verified.</p>

<p>The present invention provides a means for verifying a caller prior to connecting a call in a communication terminal, using an IMS environment. The present invention enables the receiver to easily recognize the identity of the caller by displaying a screen such as a face image of a person on the receiver's terminal before connecting the call to identify the caller.</p><p>In particular, the present invention can be easily implemented in an IMS environment by providing a method for a receiver to check an image transmitted by a sender before a session connection by utilizing the IMS standard.</p><p>In addition, since the present invention uses the SIP standard as it is, it can be extended and applied to all communication environments connected by SIP other than the IMS environment. And, in addition to sending and receiving video for the purpose of confirming the caller, the receiver can transmit and confirm the sender's avatar or other still image or video before connecting the call, thereby enabling more unique caller identification.</p><p>The caller identification method proposed in the present invention is easy to implement because there is no need to pre-register an image (screen) in a terminal or server, and additional costs for receiving such a service can be eliminated. In addition, the present invention provides a basis for mutually confirming the identity of a genuine caller because it can inform the user who actually made the call, not simply information about the calling phone (phone number) itself.</p>