User monitoring system

0001The present invention relates to a user monitoring system that monitors a user who operates a computer terminal.

0002Conventionally, there is a method such as Patent Document 1 as a method for confirming whether or not a user operating a computer terminal such as a personal computer has changed to an unauthorized person during the operation. In the method described in Patent Document 1, first, the (legitimate) user's face image data is registered before using the computer terminal. Then, the registered face image data and the user's face image data captured by the camera at predetermined intervals while the user is using the terminal are compared each time. If it is determined that these face image data are different, the terminal is locked so that the terminal cannot be used.

<p num="0003"><patcit num="1"><text>Japanese Unexamined Patent Publication No. 2008-97647</text></patcit></p>

<p num="0004"> However, with the method as in Patent Document 1, fraud cannot be detected in the following cases. That is, for example, when the Nth imaging is performed, the legitimate user A is operating the terminal, but before the N + 1th imaging is performed, after the unauthorized user B operates the terminal. This is the case when a legitimate user A returns and operates the terminal. That is, as long as the legitimate user A operates the terminal at the time when the Nth and N + 1th imaging are performed, even if an unauthorized user operates during that time, it cannot be detected.</p><p num="0005"> The present invention has been made in view of the above problems, and provides a user monitoring system capable of detecting even when an unauthorized user operates a terminal other than at the time of user authentication in a predetermined cycle. The purpose is.</p>

<p num="0006"> In order to solve the above problems, one aspect of the present invention is a user monitoring system that monitors a user who operates a terminal, and the user operates the terminal at a fixed cycle for at least a part of the period. Each time the authentication information acquisition unit that acquires the user's authentication information from the user and the authentication information acquisition unit acquire the user's authentication information at a certain cycle, the user's authentication information is changed to the legitimate user's authentication information. When the authentication unit that determines whether or not the authentication unit, the input device monitoring unit that detects a change in the state of the user's input operation on the terminal, and the input device monitoring unit detect the change in the state of the user's input operation, This is a user monitoring system including a timing changing unit for changing the cycle of acquiring the authentication information in the authentication information acquisition unit.</p><p num="0007"> Another aspect of the present invention is a method executed by a user monitoring system that includes an authentication information acquisition unit, an authentication unit, an input device monitoring unit, and a timing changing unit, and monitors a user who operates a terminal. There is a step in which the authentication information acquisition unit acquires the user's authentication information from the user at regular intervals during at least a part of the period in which the user operates the terminal, and the authentication unit receives the authentication information. Every time the acquisition unit acquires the user's authentication information at the fixed cycle, the step of determining whether the user's authentication information is the legitimate user's authentication information and the input device monitoring unit perform the input device monitoring unit on the terminal. When the step of detecting a change in the state of a user's input operation and the timing change unit detect a change in the state of the user's input operation by the input device monitoring unit, the authentication information acquisition unit performs the authentication information. It is a method including a step of changing the cycle to acquire.</p><p num="0008"> Another aspect of the present invention is a computer program for causing the user monitoring system to execute the above method.</p><p num="0009"> According to the above aspect of the present invention, it is possible to detect even when an unauthorized user performs an operation during the basic cycle of executing user authentication.</p>

0010<figref num="1">It is a figure which shows an example of the structure of the user monitoring system which concerns on one Embodiment of this invention.</figref><figref num="2">It is a figure which shows an example of a "process management log" file.</figref><figref num="3">It is a figure which shows an example of a "work content log" file.</figref><figref num="4">It is a figure which shows an example of the "input information log" file.</figref><figref num="5">It is a figure which shows an example of the "user ID correspondence table".</figref><figref num="6">It is a main flow diagram which shows an example of the process in the user monitoring system which concerns on one Embodiment of this invention.</figref><figref num="7">It is a flow chart which shows an example of the input device monitoring process.</figref><figref num="8">It is a flow chart which shows an example of the data acquisition process for restoration of operation contents.</figref><figref num="9">It is a flow chart which shows an example of the related work log extraction process.</figref><figref num="10">It is a figure which shows an example of the judgment method of the relevance of each log.</figref><figref num="11">It is a flow chart which shows an example of a recovery process.</figref><figref num="12">It is a figure which shows an example of the hardware composition of the terminal and the management apparatus which concerns on one Embodiment of this invention.</figref>

0011Hereinafter, embodiments of the present invention will be described with reference to the drawings. In each of the figures referred to in the following description, the same parts as those of the other figures are indicated by the same reference numerals.

0012(User monitoring system configuration) FIG. 1 is a diagram showing a configuration example of a user monitoring system according to the present embodiment. The user monitoring system 1 illustrated in FIG. 1 includes a terminal 10 operated by a user and an administrator device 20 operated by a system administrator.

0013The terminal 10 includes an imaging unit 102, an input receiving unit (input device) 104, an authentication unit 106, a storage unit 108, an input device monitoring unit 110, an imaging timing changing unit 112, and a related work log extraction unit 114. , A transmitting unit 116, a receiving unit 118, and a recovery processing unit 120. Further, the management device 20 includes a receiving unit 202, a display unit 204, a recovery processing necessity determination unit 206, and a transmitting unit 208.

0014(Terminal) The imaging unit 102 images the user's face (part or all) at regular intervals during at least a part of the period in which the user operates the terminal 10 to acquire the user's face image data (biological features). .. Further, in addition to performing imaging at a fixed cycle, the imaging unit 102 acquires the user's biological information when the input device monitoring unit 110, which will be described later, detects a change in the state of the user's input operation. It may be.

0015The input receiving unit (input device) 104 receives the user's operation input to the terminal 10. Specifically, the input receiving unit 104 may be various input devices such as a keyboard, a mouse, a touch panel, and a touch pen. Further, the input receiving unit 104 may be adapted to receive voice input.

0016Each time the imaging unit 102 acquires the user's face image data at a fixed cycle, the authentication unit 106 determines whether the acquired user's face image data is legitimate user's face image data. Then, when it is determined that the acquired face image data is not the face image data of a legitimate user, it is determined that the authentication is NG. Further, in the present embodiment, when it is determined as a result of the determination by the authentication unit 106 that the face image data acquired by the imaging unit 102 is not the face image data of a legitimate user, the input operation of the user is restricted. To do. As a method of restricting the user's input operation, only a part of the input operation may be restricted and other input operations may be allowed.

0017Further, various methods can be considered as a method for the authentication unit 106 to determine whether the face image data acquired by the imaging unit 102 is the face image data of a legitimate user. For example, only a part of the entire face image may be collated, or a plurality of locations (plural conditions) may be collated. Further, the collation result may be determined by combining these results. Further, when the matching rate of the collation target portion is lower than the predetermined threshold value, the authentication may be NG or the like.

0018The storage unit 108 has a function of storing various types of data. In this embodiment, at least the "process management log" file, "work content log" file, "input information log" file, "user ID correspondence table", and "process relevance log" file described below are stored in the storage unit. It is stored in 108.

0019FIG. 2 is a diagram showing an example of a process management log file. The process management log shown in FIG. 2 is a file for recording the processes running on the terminal 10, including the processes executed by each user based on the operation input to the terminal 10. In the process management log, for example, the execution user, the process ID for identifying the process executed by the user, the parent process ID for identifying the parent process, the process name, the start time and end time of the process, and the like are recorded. To.

0020FIG. 3 is a diagram showing an example of a work content log file. The "work content log" file is linked to the "process management log" file by the "process ID", and more detailed contents about the process specified by the process ID in the process management log are recorded. Further, FIG. 4 is a diagram showing an example of an input information log file. The "input information log" file records the details of the input device of the terminal 10 operated by the user. FIG. 5 is a diagram showing an example of a user ID correspondence table. The user ID correspondence table includes, for each user, when the user performs some processing on another device that operates in conjunction with the terminal 10 (for example, when accessing another device via the terminal 10). It is a table that stores the user ID of).

0021Returning to FIG. 1, the input device monitoring unit 110 detects a change in the state of the user's input operation on the terminal 10. The input device monitoring unit 110 may detect a change in the state of the user's input operation by statistically processing information about the input device of the terminal 10 operated by the user. Further, when the input receiving unit 104 receives the user's operation input, the input device monitoring unit 110 stores information about the input device operated by the user in the "input information log" file or the like of the storage unit 108.

0022When the input device monitoring unit 110 detects a change in the state of the input operation of the user, the imaging timing changing unit 112 changes the cycle of acquiring the face image data in the imaging unit 102 (makes it a shorter cycle). Here, a plurality of methods for changing the imaging timing can be considered. For example, the imaging cycle that starts after the imaging process that arrives after the input device monitoring unit 110 detects a change in the operation input is shortened, and the face image is imaged when the change in the operation input is detected, and the imaging cycle that starts after that. It is conceivable that the imaging cycle itself including the time when the change in the operation input is detected is shortened (the imaging timing that arrives after the change in the operation input is detected is earlier).

0023Further, the imaging timing changing unit 112 may be adapted to return the imaging cycle to the original length by triggering the satisfaction of a predetermined condition after changing the imaging timing to a short time. For example, the imaging cycle may be returned to the original length by detecting that the user has logged off from the terminal 10. At that time, the imaging cycle may be gradually lengthened and returned to the original length.

0024When the related work log extraction unit 114 determines as a result of the determination in the authentication unit 106 that the face image data acquired by the imaging unit 102 is not the face image data of a legitimate user, the storage unit 108 makes this determination. Extract a part or all of the user's input operation to the terminal 10 executed from the time when it is performed to a certain period before.

0025The transmission unit 116 has a function of transmitting various data to the management device 20.

0026The receiving unit 118 has a function of receiving various data from the management device 20. The communication between the terminal 10 and the management device 20 may be wired communication or wireless communication.

0027When the recovery processing unit 120 determines as a result of the determination by the authentication unit 106 that the face image data acquired by the imaging unit 102 is not the face image data of a legitimate user, the recovery processing unit 120 has a certain period of time from the time of this determination. Returns (rolls back) to the state before the input operation for some or all of the user's input operations on the terminal 10 executed so far.

0028(User monitoring system processing flow) Hereinafter, a processing example in the user monitoring system according to the present embodiment will be described. FIG. 6 is a main flow diagram showing processing in the user monitoring system 1 according to the present embodiment.

0029First, when the user starts operating the terminal 10, a login operation such as inputting a user ID and a password is performed in response to a login request from the user monitoring system 1 (user input is accepted by the input reception unit 104). .. At this time, the terminal 10 captures the face of the user operating the terminal by the imaging unit 102 to acquire face image data, and stores and registers the face image data in the storage unit 108 as legitimate face image data of the user. (Step S101).

0030After that, when a predetermined time (imaging cycle) elapses while the user is operating the terminal 10 (step S103: Yes), the process proceeds to step S109, and the imaging unit 102 images the user's face and obtains face image data. get. Then, the user monitoring system 1 compares this face image data with the face image data stored in the storage unit 108 in step S101, and the user operating the terminal 10 at this point is a legitimate user (login). Check if it is the same as the user at the time (step S109). Then, while the authentication result is OK (the operating user is a legitimate user), the process returns to step S103.

0031If it is determined that the operator of the terminal 10 may have changed due to the input device monitoring process even if the predetermined time (imaging cycle) has not elapsed (step S103: No), the operator of the terminal 10 may have changed (step S105: Yes). ), The imaging cycle of the face image is changed to a shorter cycle (step S107). Here, the input device monitoring process is a process executed in a process different from this main flow, and there is a possibility that the person operating the terminal 10 has changed based on the input from the operator on the terminal 10. This is a process for detecting the existence (the input device monitoring process will be described in detail later).

0032If it is determined that the operator may have changed (step S105: Yes), the imaging unit 102 further acquires the face image of the person operating the terminal 10 at that time and collates the face image. (Step S109).

0033If the result of the authentication process in step S109 is that the face image of the user stored in the storage unit 108 and the face image of the user performing the operation at this point are different (authentication NG), Lock terminal 10 so that the user cannot operate it (step S111). Then, the related work log extraction process is executed (step S113). When the "related work log extraction process" detects that the operator of the terminal 10 may have changed, the "related work log extraction process" cancels the operation that the operator may have performed and returns it to the state before the operation ( This is a process for extracting the operations performed by the operator for the purpose of (recovery process) (details will be described later).

0034Then, the log extracted in step S113 is displayed on the display unit 204 of the management device 20. The administrator using the management device 20 checks the log displayed on the display unit 204, determines whether or not to perform recovery processing on the terminal 10, and inputs the determination result to the management device 20. (Step S115). The judgment result is transmitted to the terminal 10. When the terminal 10 receives the determination result that the recovery process is necessary from the management device 20, the recovery process is executed (step S117).

0035In the above, the image of the user captured at the time of login (step S101) is registered in the storage unit 108 as a legitimate user's face image, but the user's face image is stored in advance in the storage unit 108 before the operation of the terminal 10 is started. It may be registered in 108 and this may be used as the user's legitimate face image data. Then, at the time of starting the operation (step S101) and at the time of collating the face image (step S109), the face image registered before the start of the operation is compared with the face image captured by the imaging unit 102. May be good. Further, regarding the processing of steps S103 to S107, the processing after step S107 may be performed by the elapse of a predetermined time or by receiving the notification result from the input device monitoring processing as an event (until then, waiting for the event). ..

0036(Input device monitoring process) FIG. 7 is a flow chart showing an example of the input device monitoring process (step S105 in FIG. 6).

0037When the user's operation input is detected from the input waiting state (step S1051) via the input receiving unit (input device) 104 of the terminal 10, the operation content restoration data acquisition process is executed (step S1053). Here, FIG. 8 is a flow chart showing an example of the data acquisition process for restoring the operation contents in step S1053.

0038When the user's operation input is detected, the operation content is recorded in the "work content log" file (Fig. 3) stored in the storage unit 108 before the operation content of the user is reflected in the data. In the work content log, process A creates / reads / deletes data x at time YYYY / MM / DD hh: mm: ss, and process A updates data x to data y ( Information such as (update) is recorded (step S201).

0039Next, if the operation type of the detected user operation is Update (update) or Delete (delete) (step S203), after obtaining a backup of the original data (step S205), according to the detected user operation. Execute the processing (step S207).

0040Returning to FIG. 7, after executing the operation content restoration data acquisition process (step S1053), the input information recording process is executed (step S1055). In the input information recording process, the details of the input device operated by the user are recorded in the "input information log" file (FIG. 4) stored in the storage unit 108. Specifically, for example, information such as input time, input device type (keyboard, mouse, etc.), input content (pressed key type), kana input / romaji input, and the like is recorded in the input information log.

0041Further, when a certain time elapses from the input waiting state (step S1051), the statistical processing of the user operation is executed (step S1057) (Note that this "fixed time" may be irrelevant to the imaging cycle) Statistics. The processing calculates, for example, the following based on the recording of the input information log from the current time (the time when the statistical processing is executed) to a certain time before. Number of inputs · Average and / or median input interval -Operation ratio for each input device (xx% was input with the keyboard, yy% was input with the mouse, etc.) -Number and / or percentage of specific operation keys (operation keys such as backspace, delete, tab, alt, cursor keys, home, end, etc.) Ratio of operation time by Romaji input and operation time by Kana input

0042For example, in the case of the input information example shown in FIG. 4, each of the above data can be calculated as follows. That is, the "number of inputs" is "13". The "average input interval" is the time difference between the input IDs "1" and "2", the time difference between the input IDs "2" and "3", ..., The time difference between the input IDs "12" and "13". Is average. The "median input interval" is the median of these. In addition, the "operation ratio for each input device" can be calculated as keyboard: 12/13, mouse: 1/13, and so on. The "number of specific operation keys" is backspace: 1. The "percentage of specific operation keys" is backspace: 1/13. In addition, the ratio of Romaji input operation time to Kana input operation time is that the ratio of Romaji input is (time of input ID "6"-time of input ID "1") / (input ID "13"). Time-Time of input ID "1"), Kana input ratio is (Time of input ID "12"-Time of input ID "8") / (Time of input ID "13"-Input ID "1" Time), can be calculated. It should be noted that each data and the calculation method thereof described above are merely examples, and may be other data as long as they can show the characteristics of the input operation of the user of the terminal 10. Further, in step S1057, the statistical processing of the user operation is executed when a certain time elapses from the input waiting state (step S1051), but the present invention is not limited to this. Statistical processing can be performed if sufficient data on user operations is accumulated to perform statistical processing. For example, every time the input receiving unit (input device) 104 receives an input of one character, statistical processing may be performed.

0043Returning to FIG. 7, the statistical information calculated in step S1057 is stored in the storage unit 108 (step S1059). Compare the value at the time of the previous statistical processing with the value at the time of the current statistical processing (step S1061). When the difference is less than a predetermined threshold value (that is, when it is determined that the operator of the terminal 10 has not changed), the process returns to step S1051 and the process is continued. If the difference is equal to or greater than a predetermined threshold value (that is, when it is determined that the operator of the terminal 10 has changed), a notification to that effect is given (step S1063), and steps S107 and S109 in FIG. 6 are performed. After the transition, the imaging cycle is changed and the authentication process is executed. In addition, the operator of the terminal 10 sets a plurality of threshold values stepwise for determining whether or not the terminal 10 has changed, and the difference between the value at the time of the previous statistical processing and the value at the time of the current statistical processing is between which threshold value. The imaging cycle may also be gradually shortened or lengthened (returned) depending on whether or not the image is located at.

0044(Related work log extraction process) Next, the related work log extraction process (step S113 in FIG. 6) will be described in detail. FIG. 9 is a flow chart showing an example of the related work log extraction process.

0045First, the process ID of the user to be extracted from the related work log is acquired (step S1131). Here, the user to be the target of the related work log extraction is, in the present embodiment, the user who logged in in step S101 of FIG. 6 (specified by the user ID. In this description, the target user or the target user. It is called ID.). The process ID is obtained by, for example, referring to the "process management log" file (Fig. 2) stored in the storage unit 108 and searching for the process ID in which "execution user" = "target user ID". be able to.

0046Next, the time when the authentication by collating the face image is NG and the time when the authentication was OK most recently in the past (the time between the two is hereinafter referred to as the fraudulent processing time) are acquired (step S1133).

0047Next, the operation log of another device that operates in conjunction with the terminal 10 is specified by the target user ID (step S1135). Specifically, for example, referring to the "user ID correspondence table" (Fig. 5), the "user ID" = "target user ID", the user ID on the device A, the user ID on the device B, and so on. (User ID on device XX) is searched. Then, referring to the process management log (similar to FIG. 2) for each of device A, device B, ... (Device XX) ..., each device searched from the "user ID correspondence table". The process ID is extracted using the user ID (user name) in. As a result, it is possible to identify the operation log of another device that has been operated via the terminal 10.

0048Next, the process is recursively searched based on the process ID of the user, and the operation log of the target user is extracted (step S1137). FIG. 10 is a diagram showing an example of a method for determining the relevance of each log. In the example shown in FIG. 10, in "Log A", for example, the IP address associated with the user ID = "logA_user1" is "192.168.1.1". Next, referring to "Log B", the node name associated with this IP address "192.168.1.1" is "SV01". Then, in "Log C", the process ID associated with this node "SV01" is "1111". By following this, it is possible to extract the processes related to a certain user. Returning to FIG. 9, processing corresponding to the illegal processing time is extracted from the log extracted in step S1137 (step S1139).

0049Next, referring to the "work content log" file (Fig. 3), the process executed during the illegal processing time is extracted using the process ID of the log extracted by the processing of steps S1131 to S1139 as a key (step S1141). ).

0050By the above processing, the processing executed by the terminal 10 during the illegal processing time can be extracted. The extracted log is transmitted to the management device 20 operated by the administrator, and the administrator can confirm this on the display unit 204 (step S115 in FIG. 6).

0051(Recovery process) Next, the recovery process (step S117 in FIG. 6) will be described in detail. FIG. 11 is a flow chart showing an example of recovery processing. The process shown in this flow chart is executed when a determination result that recovery process is necessary is received from the management device 20.

0052If the log extracted by the related work log extraction process in step S113 of FIG. 6 has not been completely read (step S1171: No), one log extracted by the related work log extraction process is acquired (step S1173).

0053If the operation type of the acquired log is Create (step S1175: Yes), the data to be created is deleted (step S1177). On the other hand, if the operation type of the acquired log is not Create (Create) (Step S1175: No), Update (Update) or Delete (Delete) (Step S1179: Yes), the data to be operated is selected. Restore from backup data (step S1181). If none of the above applies (step S1179: No), the operation type is Read, so nothing is done here.

0054The processes of steps S1173 to S1181 described above are repeatedly executed until the log extracted by the related work log extraction process of step S113 of FIG. 6 is completely read (step S1171: Yes).

0055(Hardware configuration example) The terminal 10 and the management device 20 can be realized by a hardware configuration similar to that of a general computer device. FIG. 12 is a diagram showing an example of the hardware configuration of the terminal 10 and the management device 20. The computer device 30 shown in FIG. 12 is, for example, a processor 301, a RAM 302, a ROM 303, a hard disk device 304, a removable memory 305, a communication interface 306, a display / touch panel 307, a speaker 308, and an input device. (Keyboard / mouse, etc.) 309 and. Each function of the terminal 10 and the management device 20 described above can be realized by, for example, the processor 301 reading a program stored in advance in the hard disk device 304 into a memory and executing the program. However, the configuration shown in FIG. 12 is merely an example and is not limited to this.

0056(Summary) As described above, according to the user monitoring system according to the present embodiment, according to the present invention, even when an unauthorized user performs an operation during the basic cycle of executing user authentication. , It is possible to detect this. Further, for example, Patent Document 1 discloses that the user authentication cycle is changed depending on the importance of data (application). However, since the prior document 1 changes the user authentication cycle depending on the importance of the data (regardless of the input situation of the user), it detects fraudulent activity in which the user changes to another person during the operation. It is difficult to do and does not contribute to the detection of such fraud. On the other hand, according to the user monitoring system according to the present embodiment, it is determined that the user has changed to another person due to the change in the operation input of the user. It is possible to detect even when a user tries to commit fraud by colluding with another user, or when another user tries to commit fraud when the user at the start of operation is temporarily absent.

0057In the above-described embodiment, the user authentication is performed by the face image of the user, but the user authentication may be performed by other biological features. For example, user authentication may be performed based on physical characteristics such as fingerprints and irises. Further, if the user's input operation is by voice such as voice, user authentication may be performed by using voice tone or voiceprint. Further, the user authentication may be performed by other authentication information other than the biological feature, which can authenticate the user. For example, instead of the imaging unit 102 of the above-described embodiment imaging the user, the user may be required to input a password at regular intervals. When the authentication information is a biological feature, it is difficult for the user to arbitrarily change or make the authentication information the same (for example, the user A makes his / her face look like the user B and the same user). It is more suitable because it is difficult to impersonate and fingerprints, irises, etc. cannot be shared by each user). Further, when the user is authenticated by photographing the user's face as described above, the user operating the terminal 10 is a legitimate user, but the information is stolen by being looked into from behind. In some cases (shoulder hacking), it is possible to detect troubles (such as by the number of faces recognized in the captured image), suspend the operation, or perform rollback.

0058Further, in the above-described embodiment, the user monitoring system 1 is provided with the terminal 10 and the management device 20, but the present invention is not limited to such a configuration. Each function provided by the terminal 10 and the management device 20 described above may be processed in a distributed manner by a plurality of devices according to other forms. On the contrary, if the terminal 10 is provided with an administrator application that only the administrator can log in to, the management terminal 20 can be provided with the function of the management device 20 without separately providing the management terminal 20.

0059Further, in the above-described embodiment, the image pickup timing changing unit 112 is designed so that the image pickup unit 102 takes an image at a fixed cycle, but may take an image at random time intervals. As a result, it becomes difficult for the operator of the terminal 10 to predict when the imaging will be performed, so that it is possible to make it difficult for an unauthorized user to operate the terminal 10 with the gap.

0060Although one embodiment of the present invention has been described so far, it goes without saying that the present invention is not limited to the above-described embodiment and may be implemented in various different forms within the scope of the technical idea.

0061The scope of the present invention is not limited to the exemplary embodiments illustrated and described, but also includes all embodiments that provide an effect equal to that intended by the present invention. Furthermore, the scope of the present invention is not limited to the combination of the features of the invention defined by each claim, but may be defined by any desired combination of the specific features of all the disclosed features. ..

006210 Computer terminal 20 Management equipment 102 Imaging unit 104 Input reception unit (input device) 106 Certification Department 108 Memory 110 Input device monitoring unit 112 Imaging timing changer 114 Related work log extraction unit 116 Transmitter 118 Receiver 120 Recovery processing unit 202 Receiver 204 Display 206 Recovery processing necessity judgment unit 208 transmitter