Methods, devices, and systems for unobtrusive mobile device user recognition

CROSS REFERENCE TO RELATED APPLICATIONS

0001This patent application claims priority under 35 U.S.C. §119(e) to U.S. Provisional Patent Application No. 61/430,549 filed Jan. 7, 2011, and under 35 U.S.C. §365(a) to PCT Patent Application No. IL2012/000006 filed Jan. 5, 2012, which are hereby incorporated by reference in their entirety.

FIELD AND BACKGROUND OF THE INVENTION

0002The present invention relates to methods, devices, and systems for unobtrusive user recognition and user authentication of mobile devices.

0003User authentication is an important component for providing secure access to a computerized system. Authentication allows a computerized system to know who the user is and to determine that the user is not impersonating the identity of an authorized user. Typically, passwords have been used to implement authentication of users to computerized systems, though other methods including biometric identification, one time tokens and digital signatures are also used.

0004A common drawback to such methods is that they are obtrusive to the user, requiring the user to remember and enter a sequence of characters, interact with a biometric device or be in possession of a device that generates one-time tokens each time that he wishes to access the computerized system.

0005In order to prevent secure information from being compromised by brute force attacks and searches, secure passwords need to be long, difficult to guess and frequently generated a fresh. These aspects of maintaining protection increase the intrusive and cumbersome nature of passwords.

0006A mobile device is typically accessed multiple times daily and typically requires spontaneous and immediate access. The intrusive nature of current authentication techniques impedes their adoption on mobile devices, forcing users to forgo security in favor of usability. Currently available authentication methods have a low adoption rate among users of mobile devices as their obtrusive nature is not suited to the frequent and immediate usage patterns mandated by mobile device usage.

0007It would be desirable to have an unobtrusive method for authenticating a user to a mobile device that can also be exploited to provide authentication for accessing other computerized systems, performing secure payments or for unlocking physical barriers.

0008Modern mobile devices are equipped with multiple sensors which can be used for tracking Human Computer Interface (HCI) behavior patterns.

0009Prior art has suggested the use of various user behavior metrics generated by keyboard, mouse or haptic events to identify and authenticate authorized users to a computer terminal or network. In a paper titled “Direct and Indirect Human Computer Interaction Based Biometrics”, Yampolskiy et al. (journal of Computers (JCP), Vol. 2(10), 2007, pp. 76-88) surveys the state of the art in direct and indirect human computer interaction based biometrics. Yampolskiy distinguishes between “Direct HCI biometrics” based on abilities, style, preference, knowledge, or strategy used by people while working with a computer and “indirect HCI-based biometrics” based on events that can be obtained by monitoring a user's HCI behavior indirectly via observable low-level actions of computer software.

0010Behavioral traits such as, voice, gait and keystroke have been suggested in the prior art for user identification. Although these behavioral traits can be used unobtrusively, there currently does not exist an economical and accurate method of user authentication using such traits.

SUMMARY OF THE INVENTION

0011It is the purpose of the present invention to provide methods, devices, and systems for unobtrusive user recognition and user authentication of mobile devices.

0012It is noted that the term “exemplary” is used herein to refer to examples of embodiments and/or implementations, and is not meant to necessarily convey a more-desirable use-case. Similarly, the terms “preferred” and “preferably” are used herein to refer to an example out of an assortment of contemplated embodiments and/or implementations, and is not meant to necessarily convey a more-desirable use-case. Therefore, it is understood from the above that “exemplary” and “preferred” may be applied herein to multiple embodiments and/or implementations.

0013Embodiments of the present invention provide methods, devices, and systems for unobtrusively recognizing an authorized user by a mobile device, maintaining the security of the mobile device and providing the mobile device with the capability to perform such recognition based on a comparison between current and previously recorded HCI behavior metrics.

0014HCI behavior patterns can be monitored and combined in different configurations to create “usage repertoires” that reflect unique behavior patterns of a specific individual user, allowing a device to recognize and identify an authorized user.

0015According to embodiments of the present invention, multiple features of a user's HCI behavior on a hand-held mobile device are continually and unobtrusively sampled, discretized and monitored during normal operation of the device. Software on the device learns the usage repertoire of an authorized user from the samples, and is able to distinguish between the authorized user of the device and an unauthorized user by comparing a current HCI characteristic to a learned repertoire in real-time and to block access to features of the device or perform other defensive actions when software on the device determines that it is in the hands of an intruder.

0016According to some embodiments, the system samples a dynamically selected subset of HCI characteristics that are accessible from a mobile device to optimize performance and maximize recognition accuracy.

0017Recognition of an authorized user by a mobile device can be exploited to facilitate provision of secure automatic authentication to services on a network, to enable performing secure payments from a mobile device, to allow a mobile device to act as a universal key for unlocking physical barriers like a door or car ignition or to identify friendly forces in a battle field environment.

0018Therefore, according to the present invention, there is provided for the first time a method for unobtrusively recognizing a user of a mobile device, the method including the steps of: (a) unobtrusively collecting motion data from the mobile device during normal device usage by monitoring standard authorized-user interaction with the device, without any form of challenge or device-specified action; (b) demarcating the motion data into user motion-sequences based on changes in a motion-state or an elapsed time-period without an occurrence of the changes, wherein the motion-state refers to a placement and speed of the mobile device at a point in time; (c) calculating user motion-characteristics from the user motion-sequences; and (d) generating a motion-repertoire from the user motion-characteristics, whereby the motion-repertoire enables unobtrusive recognition of the user.

0019Preferably, the method further includes the step of: (e) detecting unidentified motion-characteristics that are not associated with the motion-repertoire, thereby enabling unobtrusive recognition of unidentified usage.

0020Most preferably, the method further includes the step of: (f) upon the step of detecting, triggering a defensive action and/or providing authentication services, wherein the defensive action includes at least one action selected from the group consisting of: blocking access to the device, blocking access to selected applications, deleting sensitive data, encrypting sensitive data, setting off a siren, sending a notification to a designated individual associated with the device, and wherein the authentication services include at least one service selected from the group consisting of: privacy protection, authentication to an external system, use as a universal key, authorizing a payment, and identifying friendly forces in a battle-field environment.

0021Most preferably, the step of detecting is performed repeatedly during the normal device usage, thereby providing perpetual protection of the device from unauthorized usage.

0022Preferably, the motion data is obtained from at least one sensor selected from the group consisting of: a motion sensor, a haptic sensor an accelerometer, a gyroscope, and a combination thereof.

0023Preferably, the step of generating is performed repeatedly during the standard authorized-user interaction, thereby providing ongoing improvement to recognition accuracy.

0024More preferably, the method further includes the step of: (e) determining a learning-stage parameter as a degree of user recognition, wherein the step of determining is performed by at least one method selected from the group consisting of: automatically basing the learning-stage parameter on a quantity of the motion-sequences collected, time elapsed, or a statistical variation of the motion data collected; and manually setting the learning-stage parameter by the user.

0025Most preferably, the learning-stage parameter is operative to regulate a trigger for a defensive action and/or providing authentication services upon detecting unidentified motion-characteristics that are not associated with the motion-repertoire.

0026Preferably, the step of collecting is performed at a frequency based on the motion-state.

0027Preferably, the motion-state is determined by: (i) comparing at least one current motion-sensor value to at least one prior motion-sensor value; and (ii) assessing a degree of change in the motion-sensor values based on an Absolute Total Acceleration Change (ATAC).

0028Preferably, the placement has at least one designation selected from the group consisting of: a hand-held state, an on-body state, a pocket state, a flat rest-state, and a non-flat rest-state; and wherein the speed has at least one designation selected from the group consisting of: a traveling state at or above a delimited speed, a walking state, a running state, a hand-moving state, a stable state, and a motionless state.

0029Preferably, the method further includes the steps of: (e) discretizing the motion-characteristics into discrete values; and (f) selectively increasing the number of the discrete values, thereby dynamically controlling recognition accuracy.

0030According to the present invention, there is provided for the first time a device for unobtrusively recognizing a mobile-device user, the device including: (a) a processing module including: (i) a CPU for performing computational operations; (ii) a memory module for storing data; and (iii) at least one sensor for detecting interaction with the device; and (b) a recognition module, operationally connected to the processing module, configured for: (i) unobtrusively collecting motion data from the mobile device during normal device usage by monitoring standard authorized-user interaction with the device, without any form of challenge or device-specified action; (ii) demarcating the motion data into user motion-sequences based on changes in a motion-state, wherein the motion-state refers to a placement and speed of the mobile device at a point in time; (iii) calculating user motion-characteristics from the user motion-sequences; and (iv) generating a motion-repertoire from the user motion-characteristics, whereby the motion-repertoire enables unobtrusive recognition of the user.

0031According to the present invention, there is provided for the first time a non-transitory computer-readable medium, having computer-readable code embodied on the non-transitory computer-readable medium, the computer-readable code including: (a) program code for unobtrusively collecting motion data from the mobile device during normal device usage by monitoring standard authorized-user interaction with the device, without any form of challenge or device-specified action; (b) program code for demarcating the motion data into user motion-sequences based on changes in a motion-state, wherein the motion-state refers to a placement and speed of the mobile device at a point in time; (c) program code for calculating user motion-characteristics from the user motion-sequences; and (d) program code for generating a motion-repertoire from the user motion-characteristics, whereby the motion-repertoire enables unobtrusive recognition of the user.

0032According to the present invention, there is provided for the first time a method for unobtrusively recognizing a mobile-device user, the method including the steps of: (a) providing a plurality of motion-sequences demarcated from motion data of a plurality of users of mobile devices; (b) calculating user motion-characteristics from the plurality of motion-sequences; (c) aggregating user motion-characteristics to produce population motion-characteristics; and (d) generating a differentiation-template for the user by comparing the user motion-characteristics to the population motion-characteristics.

0033According to the present invention, there is provided for the first time a system for unobtrusively recognizing a mobile-device user, the system including: (a) a server including: (i) a CPU for performing computational operations; (ii) a memory module for storing data; and (b) a processing module configured for: (i) providing a plurality of motion-sequences demarcated from motion data of a plurality of users of mobile devices; (ii) calculating user motion-characteristics from the plurality of motion-sequences; (iii) aggregating user motion-characteristics to produce population motion-characteristics; and (iv) generating a differentiation-template for the user by comparing the user motion-characteristics to the population motion-characteristics.

0034According to the present invention, there is provided for the first time a non-transitory computer-readable medium, having computer-readable code embodied on the non-transitory computer-readable medium, the computer-readable code including: (a) program code for providing a plurality of motion-sequences demarcated from motion data of a plurality of users of mobile devices; (b) program code for calculating user motion-characteristics from the plurality of motion-sequences; (c) program code for aggregating user motion-characteristics to produce population motion-characteristics; and (d) program code for generating a differentiation-template for the user by comparing the user motion-characteristics to the population motion-characteristics.

0035These and further embodiments will be apparent from the detailed description and examples that follow.

BRIEF DESCRIPTION OF THE DRAWINGS

0036The present invention is herein described, by way of example only, with reference to the accompanying drawing, wherein:

0037<figref idref="DRAWINGS">FIG. 1</figref> schematically illustrates a method for user recognition based on comparing motion-characteristics of a current user to a motion-repertoire of an authorized user, according to preferred embodiments of the present invention;

0038<figref idref="DRAWINGS">FIG. 2</figref> schematically illustrates a user operating an exemplary mobile device showing various HCI—event signals for device orientation and touch screen events, according to preferred embodiments of the present invention;

0039<figref idref="DRAWINGS">FIG. 3</figref> is a simplified flowchart of process steps for performing continuous background monitoring, learning and detection by an exemplary mobile device, according to preferred embodiments of the present invention;

0040<figref idref="DRAWINGS">FIG. 4</figref> is a simplified network diagram showing the interrelations of exemplary discretized HCI-metrics data in a Bayesian network for learning a user's HCI behavior from a mobile device, according to preferred embodiments of the present invention;

0041<figref idref="DRAWINGS">FIG. 5</figref> schematically illustrates a method for user recognition based on comparing motion-characteristics of a current user to a differentiation-template of an authorized user that may be generated by a remote server, according to preferred embodiments of the present invention;

0042<figref idref="DRAWINGS">FIG. 6</figref> schematically illustrates a system in which some processing units are located at a remote server, according to an embodiment of the invention; and

0043<figref idref="DRAWINGS">FIG. 7</figref> schematically illustrates a system for user recognition, according to preferred embodiments of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

0044The present invention relates to methods, devices, and systems for unobtrusive user recognition and user authentication of mobile devices. The principles and operation for such methods, devices, and systems, according to the present invention, may be better understood with reference to the accompanying description and drawings.

0045The present invention is described herein by way of examples that demonstrate various techniques that can be used for teaching a mobile device to recognize an authorized user. Recognition is implemented by comparing the motion-characteristics of a current user interaction to a motion-repertoire comprising previously collected motion-characteristics collected while an authorized user operates a mobile device. Terminology that is used for describing the invention is defined below:

0046Motion-state refers to the placement and speed of motion of a hand-held device at some point in time, wherein placement can have values: “in user's hand”, “in pocket”, “resting on flat surface” or “other non-flat resting state not attached to a user” and speed can have values: “traveling at above 20 km/hour”, “walking or running”, “device moving in hand while user is standing or sitting”, “device stable in hand while user is standing or sitting” and “motionless”. Motion-state is extrapolated from an analysis of sensor data collected from the device.

0047A motion-sequence refers to a collection of motion data parameters extracted from one or more sensors on the device simultaneously over a period of time. Motion-sequences are demarcated from a continual stream of collected sensor data based on changes in motion-state or after a time interval has elapsed.

0048Motion-characteristics are obtained by calculating aggregations on the data contained in each motion-sequence. Aggregations calculated may include, sum, average, maximum, minimum, first, last, standard deviation and others; which are performed on individual motion parameters in a motion-sequence or on combinations of these parameters; characteristics may also be calculated on the change in value of a parameter or combination of parameters in relation to the value of a previously collected sample in a time sequence; they can also be based on aggregations calculated separately on a value change in the positive or in the negative direction.

0049A motion-repertoire refers to a collection of motion-characteristics which are captured while an individual authorized user is operating the device. The motion-repertoire of a user expands over time as new aspects of the user's motion behavior are encountered while operating the device.

0050A differentiation-template refers to the set of motion-characteristics that best differentiate an individual user from the motion-characteristics calculated for a population of users. Differentiation-templates are determined by analyzing the frequencies of occurrence of each motion-characteristic of an individual user and comparing these to the frequencies of occurrence of motion-characteristics in the population. Various statistical techniques may be used for determining the best differentiating characteristics including Bayesian analysis, clustering and others. Differentiation-templates are used by software on the device to efficiently capture, analyze and compare only motion-characteristics that best differentiate an individual user from the motion-characteristics of the population.

0051Learning-stage parameter refers to the degree of recognition by a mobile device of an authorized user. A learning-stage parameter can be determined automatically by the system or determined manually by the authorized user. Factors that can be used for automatic determination of learning-stage include the quantity of motion-sequences collected, time elapsed, statistical variation of the data, consistency of differentiation-templates over time and analysis of FAR (False Acceptance Ratio) and a FRR (False Rejection Ratio) values determined from a comparison of motion-sequences of a population to the motion-repertoire of an authorized user. The learning-stage parameter is used in various embodiments to assist in recognition, determine defensive actions and efficiently regulate the frequency that certain actions are performed to provide an optimum balance between device performance and recognition accuracy.

0052Embodiments of the present invention enable HCI metrics (and others), used in isolation or in combination via software (as well as embedded hardware or firmware) running on the device and/or on a remote server, to sample usage characteristics, and to store a representation of a user's unique, HCI behavioral usage repertoire. Implementations can be configured for security devices in which the monitoring cannot be overridden.

0053According to some embodiments HCI metrics (other or in addition to device movement and placement) can be used. Some HCI metrics that can be measured on a mobile device include: geographic location (e.g., as measured by a GPS sensor on the device), orientation and location in 3-dimensional space (e.g., as indicated by an onboard gyroscope), velocity and direction (e.g., as measured from motion detectors), type, position and frequency of touch screen events, time interval between clicks on a virtual or physical keyboard, frequency of specific key clicks, frequency of combinations of key clicks, time of day, intervals between idle and active usage, frequency of application usage, angle between current and previous cursor locations, intervals between start of phone ring and call pickup, distance between fingers touching the screen when using multi touch gestures, frequency of word occurrence in SMS messages, length of SMS messages, call durations and others.

0054HCI metrics can also include other forms of interactive control, such as pen taps, strokes, swipes and flicks, as well as “hovering” signals and visual gesture recognition (e.g., from motion based devices like Microsoft's Kinect™). Any of these metrics and others can be used in isolation or in combination by software running on the device and/or on server components to sample usage patterns and to store a representation of a user's unique HCI behavioral repertoire.

0055A method for mobile device user recognition according to an embodiment of the invention is schematically illustrated in <figref idref="DRAWINGS">FIG. 1</figref>.

0056According to one embodiment a method for mobile device user recognition includes the general steps of: collecting motion data from the mobile device (e.g., from sensors on the mobile device) while the device is held by an authorized user (<b>110</b>); demarcating a time series of motion data into motion-sequences based on changes in the motion-state of the device (<b>112</b>), wherein a motion-state describes motion and placement of the device at a point in time and may be extrapolated from ongoing analysis of the sensor data collected from the device; calculating motion-characteristics per motion-state of the authorized user by performing aggregations on a motion-sequence (<b>114</b>); generating a motion-repertoire from the motion-characteristics of the authorized user; (<b>116</b>); determining that the device is in a hand-held state which indicates the device is being held in a user's hand (<b>118</b>); demarcating motion data of an unidentified user into motion-sequences in a hand-held motion-state (<b>120</b>); aggregating data contained in each motion-sequence to calculate motion-characteristics of the unidentified user (<b>122</b>); comparing current motion-characteristics of the device in the hand held state to the motion-repertoire of an authorized user to determine if the device is being held by the authorized user (<b>124</b>); and triggering defensive actions or providing authentication services based on the comparison (<b>126</b>). E.g., triggering defensive actions if the comparison shows that the device is not being held by the authorized user and providing authentication services in the case that motion-characteristics calculated on motion parameters sampled during current operation are consistent with a prior motion-repertoire of the authorized user.

0057In the embodiment illustrated in <figref idref="DRAWINGS">FIG. 1</figref> detecting a hand held state of the device and determining motion-characteristics of the user per the hand held state is one option but other motion-states of the device may be used for detecting an authorized user.

0058According to some embodiments, determining a motion-state of a device includes obtaining a sample of motion data of the device. Motion data may include acceleration data of the device. In one embodiment at least one attribute is added to the sample of motion data.

0059For example, a background software service running on a mobile device may receive accelerometer events at a sampling frequency of, for example, ˜every 0.22 seconds. The software “embellishes” the raw accelerometer data (time, x, y, z) with additional attributes (e.g., duration of the current sample and changes in x, y, and z readings from the previous sample (dX, dY, dZ)) calculated during collection and may output a record comprising 8 attributes for each event sampled:

0000<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="9"><colspec colname="offset" colwidth="14pt" align="left" /><colspec colname="1" colwidth="21pt" align="left" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="14pt" align="center" /><colspec colname="4" colwidth="28pt" align="center" /><colspec colname="5" colwidth="35pt" align="left" /><colspec colname="6" colwidth="28pt" align="center" /><colspec colname="7" colwidth="14pt" align="center" /><colspec colname="8" colwidth="35pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="8" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>Time</entry><entry>X</entry><entry>Y</entry><entry>Z</entry><entry>Duration</entry><entry>dX</entry><entry>dY</entry><entry>dZ</entry></row><row><entry /><entry namest="offset" nameend="8" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0060Determination of motion-state is performed using moving averages calculated on the summed dX, dY, dZ attributes over, for example, 3-second periods.

0061According to one embodiment a motion-state of a device is determined by speed (acceleration*time) and using thresholds for Absolute Total Acceleration Change (ATAC) (=abs(dX)+abs(dY)+abs(dZ)) calculated on the moving averages. For example, a resting motion-state (value of 13 in Table 1) would be determined when 0.6&lt;ATAC&lt;0.1 (between threshold<b>1</b> and threshold<b>2</b>).

0062According to one embodiment, determination of placement is performed based on the ATAC and the device orientation angle extrapolated from the moving averages of the x, y, z values. For example, a static flat motion-state (value of 20 in Table 1) would be determined when ATAC<0.1 and (x>−0.22 and y<0.22 and z<−9.72 and z>−9.92).

0063For example, the background software may analyze the stream of embellished accelerometer data to determine the current motion-state of the device. It may distinguish, for example, 20 separate motion-states as shown by a combination of motion and placement indicated, as an example, in Table 1.

0000<tables id="TABLE-US-00002" num="00002"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="offset" colwidth="119pt" align="left" /><colspec colname="1" colwidth="98pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="1" rowsep="1">TABLE 1</entry></row></thead><tbody valign="top"><row><entry /><entry namest="offset" nameend="1" align="center" rowsep="1" /></row><row><entry /><entry>Placement</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="5"><colspec colname="offset" colwidth="119pt" align="left" /><colspec colname="1" colwidth="21pt" align="center" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="28pt" align="center" /><colspec colname="4" colwidth="21pt" align="center" /><tbody valign="top"><row><entry /><entry>Hand</entry><entry>Pocket</entry><entry>Stand*</entry><entry>Flat</entry></row><row><entry /><entry namest="offset" nameend="4" align="center" rowsep="1" /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="6"><colspec colname="1" colwidth="21pt" align="left" /><colspec colname="2" colwidth="98pt" align="left" /><colspec colname="3" colwidth="21pt" align="char" char="." /><colspec colname="4" colwidth="28pt" align="char" char="." /><colspec colname="5" colwidth="28pt" align="char" char="." /><colspec colname="6" colwidth="21pt" align="char" char="." /><tbody valign="top"><row><entry>Mo-</entry><entry>Traveling (speed > 20 km/hour)</entry><entry>1</entry><entry>2</entry><entry>3</entry><entry>4</entry></row><row><entry>tion</entry><entry>Walking (speed > 3 km/hour)</entry><entry>5</entry><entry>6</entry><entry>7</entry><entry>8</entry></row><row><entry /><entry>Moving (above Absolute Total</entry><entry>9</entry><entry>10</entry><entry>11</entry><entry>12</entry></row><row><entry /><entry>Acceleration Change (ATAC)</entry></row><row><entry /><entry>threshold2, see above)</entry></row><row><entry /><entry>Resting (below ATAC</entry><entry>13</entry><entry>14</entry><entry>15</entry><entry>16</entry></row><row><entry /><entry>threshold2, see above)</entry></row><row><entry /><entry>Static (below ATAC</entry><entry /><entry>18</entry><entry>19</entry><entry>20</entry></row><row><entry /><entry>threshold1, see above)</entry></row><row><entry namest="1" nameend="6" align="center" rowsep="1" /></row><row><entry namest="1" nameend="6" align="left" id="FOO-00001">*Device is held on a stand (for example in a car), or supported by some other inanimate object.</entry></row></tbody></tgroup></table></tables>

0064According to one embodiment motion-states <b>9</b> and <b>13</b> are of primary interest for unauthorized user recognition. States <b>10</b>, <b>11</b>, <b>12</b>, <b>14</b>, <b>15</b>, <b>16</b> and <b>17</b> are less relevant for user recognition. Traveling or walking motion-states of the device may be of less interest, since typically, moving and resting states imply that a user is operating the device. Thus, according to some embodiments, a motion-sequence of a device is used only if it is in a hand-held motion-state (<b>9</b> or <b>13</b>).

0065A data stream may be demarcated into motion-sequences by further embellishing the output record for each event sampled with 2 additional attributes: the motion-state determined and a counter (indicating the sequence of each event in a motion-sequence comprising samples having the same motion-state) resulting in an (exemplary) output record with the following attributes:

0000<tables id="TABLE-US-00003" num="00003"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="10"><colspec colname="1" colwidth="21pt" align="left" /><colspec colname="2" colwidth="21pt" align="center" /><colspec colname="3" colwidth="14pt" align="center" /><colspec colname="4" colwidth="21pt" align="center" /><colspec colname="5" colwidth="35pt" align="left" /><colspec colname="6" colwidth="21pt" align="center" /><colspec colname="7" colwidth="14pt" align="center" /><colspec colname="8" colwidth="21pt" align="center" /><colspec colname="9" colwidth="21pt" align="left" /><colspec colname="10" colwidth="28pt" align="left" /><thead><row><entry namest="1" nameend="10" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry>Time</entry><entry>X</entry><entry>Y</entry><entry>Z</entry><entry>Duration</entry><entry>dX</entry><entry>dY</entry><entry>dZ</entry><entry>State</entry><entry>Count</entry></row><row><entry namest="1" nameend="10" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0066Aggregations for calculating a motion-characteristic at Step <b>114</b> may include sum, average, standard deviation, minimum, maximum, first, last and others. Aggregations may be calculated on each motion-sequence on each of the x, y, z, dx, dy, dz, time and duration values. Aggregations may be calculated on various derivatives and combinations of the values.

0067A Jitter motion-characteristic, for example, can be derived by separately summing the positive and negative changes dX, dY, dZ for each motion-sequence. The difference between absolute positive and negative changes reflects actual movement in a direction on each of the X, Y and Z axes, while the remaining quantities of positive and negative acceleration change (that cancel each other out in summation) can be attributed to jitter. By comparing jitter when the device is resting on a table to jitter when the device is held by a user, jitter resulting from measurement artifacts of the device can be filtered out to derive quantities that reflect the amount of shake (jitter) by an individual's hand on each of the X, Y and Z axes while the user is holding the device.

0068According to some embodiments the step of obtaining a sample of motion data of the device (<b>110</b>) is performed at a frequency that is based on the determined motion-state of the device.

0069A user operating an exemplary mobile device showing various HCI-event signals for device orientation and touch screen events, according to an embodiment of the invention, is schematically illustrated in <figref idref="DRAWINGS">FIG. 2</figref>.

0070An exemplary mobile device <b>200</b> is shown in a user holding hand <b>202</b> and a user interacting hand <b>204</b> (<figref idref="DRAWINGS">FIG. 2</figref> is illustrative and not meant to imply left-or right-handed use). Ambidextrous use of mobile device <b>200</b> and operation of mobile device <b>200</b> with one hand only are also within the scope of the present invention.

0071User holding-hand <b>202</b> can also interact with mobile device <b>200</b> through contact (e.g., buttons, sensors and sliders); likewise, user interacting-hand <b>204</b> can also partially hold mobile device <b>200</b> when mobile device <b>200</b> is supported with both hands.

0072While mobile device <b>200</b> is depicted as a hand held device (e.g., an iPod or smartphone) with a touch screen, other mobile devices (e.g., slate computers, tablets, laptops, PDAs, still-image cameras, video cameras, remote controls, authentication keypads and e-readers) are also considered within the scope of the invention.

0073A device orientation <b>206</b> is an exemplary category of HCI-event signals which can include signals such as an azimuth <b>208</b> (i.e., in degrees representing the angle between the x-axis of the device and the north direction wherein 0&lt;azimuth≧360), a pitch <b>210</b> (i.e., in degrees representing the angle made by the y-axis of the device relative to the device's horizontal position, wherein −180&lt;pitch≧180), and a roll <b>212</b> (i.e., in degrees representing the angle made by the x-axis of the device relatively to the device's horizontal position, wherein −90&lt;roll≧90, for example).

0074Another exemplary category of HCI-event signals is a touch screen event <b>214</b> (i.e., a haptic HCI attribute) which can include signals such as an event type <b>216</b> (e.g., down, up or move; representing a user placing one or more fingers on the touch screen of the mobile device, a user lifting one or more fingers from the touch screen, or a user moving one or more fingers across the surface of the device, respectively), a screen position <b>218</b> (i.e., x-y coordinates of the area of the touch screen that was touched) and an elapsed time <b>220</b> (e.g., the interval in milliseconds elapsed since the prior touch screen event), for example.

0075Monitoring the coincidence of HCI-event signals such as device orientation <b>206</b> and touch screen event <b>214</b> provides strong differentiation between individual users, since such HCI—event signals capture physical attributes of the user that are projected in coordinated hand movements while the user manipulates the device.

0076According to one embodiment, a system samples and stores motion-characteristics in a probability table that represents the unique motion-repertoire of the device's authorized user, allowing the device to recognize and block unauthorized users from accessing device functionality or to trigger other defensive actions.

0077Modern mobile devices typically have services which trigger HCI events when the screen is touched and also allow applications to obtain orientation information from the operating system. By sampling such HCI events and using, for example, a Bayesian network to represent a user's motion-repertoire in a probability table, the device is able to “learn” the unique motion-characteristics of an authorized user for real-time monitoring.

0078Sampling and learning may take place in the background while a user is interacting with the device, providing an unobtrusive, passive training and monitoring system.

0079A mobile device typically triggers between 5 and 40 events per second; thus, several hours of usage are adequate for collecting a sufficiently large sample of HCI events that will allow the device to uniquely recognize the authorized user, according to embodiments of the invention.

0080<figref idref="DRAWINGS">FIG. 3</figref> shows a simplified flowchart of major process steps for performing continuous background monitoring learning and detection of a user's motion-characteristics from a mobile device, according to one embodiment of the invention.

0081According to one embodiment, Flowchart <b>300</b> starts with the system monitoring touch screen events registered by the device (Step <b>302</b>). When a user begins interacting with the device, a touch event is intercepted by the system (Step <b>304</b>), resulting in the system sampling parameters associated with the event (Step <b>306</b>) (e.g., event type <b>216</b>, screen position <b>218</b>, and elapsed time <b>220</b>) associated with touch screen event <b>214</b>. The system then demarcates a motion-sequence from the stream of motion data (e.g., azimuth <b>208</b>, pitch <b>210</b>, and roll <b>212</b>) associated with device orientation <b>206</b> based on the change to a hand-held motion-state as determined by the touch event <b>214</b> (Step <b>308</b>).

0082The system then calculates motion-characteristics from the values sampled for each of the parameters in the motion-sequence and “discretizes” (Step <b>310</b>). Parameter discretization, which is used in a Bayesian network for analyzing usage patterns, is discussed with reference to <figref idref="DRAWINGS">FIG. 4</figref>.

0083The discretized motion-characteristics are then compared to a Bayesian network's prior probability table comprising the motion-repertoire of all prior motion-characteristics sampled from the authorized user (Step <b>312</b>) and the system determines if the motion-characteristics of the sample are consistent with the motion-repertoire of the authorized user (Step <b>314</b>).

0084This exemplary implementation uses Bayesian techniques to match a current sample of motion-characteristics to the joint probability distribution for the set of all prior motion-characteristics collected from the authorized user and to determine a likelihood probability that the current motion-characteristics were generated while the device was held by an authorized user. Other statistical techniques may be used for determining that a set of current motion-characteristics differs significantly from a previously collected motion-repertoire.

0085In the case that the current motion-characteristics are determined by the system to be consistent with the probability table of prior motion-characteristics (the motion-repertoire), cases of a Bayesian network are then updated with the discretized values of each fresh motion-characteristic sampled (Step <b>316</b>) and the system continues monitoring touch events (Step <b>302</b>).

0086In the case that the current motion-characteristics are inconsistent with the motion-repertoire, the system determines if it is in an initial learning mode (Step <b>318</b>) based on a learning-stage parameter held by the system (see below).

0087In the case that the learning-stage parameter determines that the device is in initial learning mode (Step <b>318</b>), cases of a Bayesian network are then updated with the discretized motion-characteristics sampled (Step <b>316</b>) and the system continues monitoring touch events (Step <b>302</b>).

0088In the case that learning-stage parameter determines that the device has completed its initial learning phase (Step <b>318</b>), the system notifies the user that he is not authorized to use the device and access to protected features of the device are blocked (Step <b>320</b>).

0089According to some embodiments the system is able to dynamically determine when a sufficient number of motion-characteristics have been learned by the device to facilitate accurate recognition and to dynamically set the value of the learning-stage parameter based on this determination.

0090Automatic determination can be based on, for example, the quantity of collected motion-characteristics, a period of learned user activity, analyzing a statistical distribution of collected motion-characteristics or monitoring the rate of new motion-characteristics that are added to the motion-repertoire. Alternatively, a user can indicate directly to the system that an initial sampling period of motion-characteristics is complete. Following completion of initial learning, the system may continue to sample and learn the HCI usage pattern of the device owner but when the device detects a usage pattern that does not match the existing motion-repertoire of usage behavior, the device may block the user from accessing protected features of the device (<b>320</b>) (which can include stored information or functionality (e.g., call initiation)).

0091The threshold value for detection as well as the defensive actions triggered when unauthorized operation is detected (<b>320</b>) may be determined using the learning-stage parameter of the device. For example, in the case of a new device/user having a low learning-stage parameter, an unauthorized user will be detected when the background process detects a motion-characteristic that scores below a threshold value of 10, and a lenient defense strategy may be applied, for example, beep during the first 2 detections, lock the device on the third detection and send a message on the fourth detection. In the case of an experienced device/user having a high learning-stage parameter, a score below −20 may be required to trigger defensive actions (<b>320</b>), which may be for example: beep+lockout+message at the first instance an unauthorized user is detected.

0092<figref idref="DRAWINGS">FIG. 4</figref> is a simplified network diagram showing an exemplary motion-repertoire with the interrelations of discretized motion-characteristics in a Bayesian network for learning a user's HCI behavior pattern from a mobile device, according to an embodiment of the invention.

0093The motion-characteristic of Event type <b>216</b> (from <figref idref="DRAWINGS">FIG. 2</figref>) is discretized into three actions-up, down or move (Panel <b>400</b>). Screen position <b>218</b> (represented by x-y coordinates of the touch screen) is discretized into nine possible rectangular areas representing the area of the screen that was touched (Panel <b>402</b>). The elapsed time <b>220</b> (e.g., since a prior event) is discretized into three possible periods: less than 3000 milliseconds, between 3001 and 6000 milliseconds and more than 6001 milliseconds (Panel <b>404</b>).

0094The motion-characteristic of Azimuth <b>208</b> is discretized into 4 segments, representing angles between 0 and 90 degrees, angles between 90 and 180 degrees, angles between 180 and 270 degrees and angles between 270 and 360 degrees (Panel <b>406</b>). Pitch <b>210</b> is discretized into 4 segments, representing angles between −180 and −90 degrees, angles between −90 and 0 degrees, angles between 0 and 90 degrees and angles between 90 and 180 degrees (Panel <b>408</b>). Roll <b>212</b> is discretized into 3 segments, representing angles between −90 and −30 degrees, angles between −30 and 30 degrees and angles between 30 and 90 degrees (Panel <b>410</b>).

0095For the sake of simplicity parameters in this exemplary implementation are discretized into a small number of possible values. Note that the accuracy required to recognize differences in user motion-characteristics from the motion-repertoire of an authorized user can be calibrated by selectively increasing the number of discrete value intervals for any one or more of the motion-characteristics measured. The network diagram of <figref idref="DRAWINGS">FIG. 4</figref> shows the six motion-characteristics sampled by the exemplary system and their discretization in a Bayesian network for an authorized user A and an unauthorized user B (Panel <b>412</b>). The network represents the prior motion-repertoire of authorized user A.

0096A sample section of a probability table learned by the Bayesian network of the exemplary system to recognize motion-characteristics is shown in Table 2, indicating probabilities for coincidence of the six motion-characteristics derived from device orientation and touch screen event and the nodes learned by the Bayesian network for authorized user A and unauthorized user B.

0000<tables id="TABLE-US-00004" num="00004"><table frame="none" colsep="0" rowsep="0" pgwide="1"><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="98pt" align="left" /><colspec colname="1" colwidth="98pt" align="center" /><colspec colname="2" colwidth="70pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="2" rowsep="1">TABLE 2</entry></row></thead><tbody valign="top"><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row><row><entry /><entry>Touch screen event</entry><entry /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="5"><colspec colname="1" colwidth="98pt" align="center" /><colspec colname="2" colwidth="21pt" align="left" /><colspec colname="3" colwidth="28pt" align="center" /><colspec colname="4" colwidth="49pt" align="center" /><colspec colname="5" colwidth="70pt" align="center" /><tbody valign="top"><row><entry /><entry /><entry /><entry>Milliseconds</entry><entry /></row><row><entry>Device Orientation</entry><entry>Event</entry><entry>Screen</entry><entry>elapsed since</entry><entry>User</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="8"><colspec colname="1" colwidth="35pt" align="center" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="35pt" align="center" /><colspec colname="4" colwidth="21pt" align="left" /><colspec colname="5" colwidth="28pt" align="center" /><colspec colname="6" colwidth="49pt" align="center" /><colspec colname="7" colwidth="35pt" align="center" /><colspec colname="8" colwidth="35pt" align="center" /><tbody valign="top"><row><entry>Azimuth</entry><entry>Pitch</entry><entry>Roll</entry><entry>type</entry><entry>Position</entry><entry>previous event</entry><entry>A</entry><entry>B</entry></row><row><entry namest="1" nameend="8" align="center" rowsep="1" /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="8"><colspec colname="1" colwidth="35pt" align="center" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="35pt" align="center" /><colspec colname="4" colwidth="21pt" align="left" /><colspec colname="5" colwidth="28pt" align="center" /><colspec colname="6" colwidth="49pt" align="center" /><colspec colname="7" colwidth="35pt" align="char" char="." /><colspec colname="8" colwidth="35pt" align="char" char="." /><tbody valign="top"><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>−30 to 30 </entry><entry>down</entry><entry>3</entry><entry><3</entry><entry /><entry /></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>−30 to 30 </entry><entry>down</entry><entry>3</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>−30 to 30 </entry><entry>down</entry><entry>3</entry><entry>>6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>−30 to 30 </entry><entry>down</entry><entry>4</entry><entry><3</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>−30 to 30 </entry><entry>down</entry><entry>4</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>1</entry><entry>>6</entry><entry>87.50%</entry><entry>12.50%</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>1</entry><entry><3</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>1</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>2</entry><entry>>6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>2</entry><entry><3</entry><entry>0.00%</entry><entry>100.00%</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>2</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>3</entry><entry>>6</entry><entry>87.50%</entry><entry>12.50%</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>3</entry><entry><3</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>move</entry><entry>3</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>4</entry><entry>>6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>4</entry><entry><3</entry><entry>66.00%</entry><entry>33.00%</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>4</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>5</entry><entry>>6</entry><entry>100.00%</entry><entry>0.00%</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>5</entry><entry><3</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>5</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>6</entry><entry>>6</entry><entry>100.00%</entry><entry>0.00%</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>up</entry><entry>6</entry><entry><3</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>down</entry><entry>6</entry><entry>3 to 6</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>down</entry><entry>7</entry><entry>>6</entry><entry>83.33%</entry><entry>16.66%</entry></row><row><entry>0 to 90</entry><entry>0 to 90</entry><entry>30 to 90</entry><entry>down</entry><entry>7</entry><entry><3</entry></row><row><entry namest="1" nameend="8" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0097According to some embodiments the system samples a dynamically selected subset of motion-characteristics that are accessible from the device to optimize performance and maximize recognition accuracy. Recognition of an authorized user by the mobile device can be exploited to facilitate provision of secure automatic authentication to services on a network.

0098Artificial Intelligence techniques have been widely used to teach computers to recognize patterns. Machine learning is a branch of artificial intelligence concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data. Multiple approaches have been taken to train computers to recognize complex patterns and make intelligent decisions based on data, some of these methods include: Decision tree learning, Association rule learning, Artificial neural networks, Genetic programming, Inductive logic programming, Support vector machines, Clustering, Bayesian networks, Reinforcement learning and others. Any of these approaches may be used to practice embodiment of the present invention. According to another embodiment, a method for mobile device user recognition which is schematically illustrated in <figref idref="DRAWINGS">FIG. 5</figref>, includes the general steps of: collecting data from the mobile device (e.g., from sensors on the mobile device) of an authorized user (<b>510</b>); demarcating the time series of motion data into motion-sequences based on changes in the determined motion-state of the device (<b>512</b>), wherein a motion-state describes motion and placement of the device at a point in time and may be extrapolated from ongoing analysis of the sensor data collected from the device; calculating motion-characteristics per motion-state of the authorized user by performing aggregations on data contained in each motion-sequence (<b>514</b>); calculating motion-characteristics of a plurality of users to obtain motion-characteristics of a population (<b>516</b>); computing a differentiation score for each characteristic by comparing the frequency of occurrence of each motion-characteristic of the user to the frequency of occurrence of the equivalent motion-characteristic in the population (<b>518</b>); generating a differentiation-template for each user based on the differentiation scores (<b>520</b>), thus a differentiation-template is generated for an authorized user by comparing motion-characteristics of the authorized user to the motion-characteristics of the population; collecting motion data from the mobile device (e.g., from sensors on the device) of an unidentified user (<b>522</b>); using the differentiation-template to calculate motion-characteristics of the unidentified user (<b>524</b>); determining if the unidentified user is the authorized user based on a comparison of the calculated motion-characteristics to the differentiation-template, e.g., by comparing the motion-characteristics of the unidentified user to the motion-repertoire of the authorized user contained in the differentiation-template to determine if the user (unidentified) is the authorized user (<b>528</b>); triggering defensive actions or providing authentication services based on the determination (<b>530</b>). For example, if it is determined that that the device is not being held by the authorized user defensive actions are triggered and in the case that the motion-characteristic is consistent with the motion-repertoire of the authorized user authentication services are provided.

0099According to one embodiment, the method may include creating from the motion-characteristics of a plurality of users' motion-characteristics of a population. A frequency of occurrence of a specific motion-characteristic within the motion-characteristics of the population may be compared to the frequency of occurrence of the specific motion-characteristic within the motion-characteristics of the user; a differentiation-template per user may be thus obtained by ranking each motion-characteristic by its degree of difference to the population. The differentiation-template may be used by software running on the device to more efficiently determine if the device is being held by an authorized user.

0100An exemplary system for generating a differentiation-template according to one embodiment is described below: the system may include software on a server that groups motion-sequences collected from a plurality of users/devices (population) per motion-state, calculates motion-characteristics and converts continuous values into discrete ranges to create population motion-characteristics. For example, and as shown in Table 3, assuming that the range of measures in the population for the motion-characteristic “average of X” having motion-state=9 (device is in hand and moving at a speed that is below walking speed) is between −10.27 and 18.769, and that there are 2400 samples in the database for this motion-state for the population, and that we wish to discretize into 8 ranges, 7 threshold values for differentiating the ranges are calculated such that an approximately equal number of samples falls into each range. Thus, for example, a value of 1 may indicate aggregated values of x<=−9; a value of 2:−9<x<=−6; a value of 3:−6<x<=0; a value of 4:0<x<=5; a value of 5:5<x<=8; a value of 6:8<x<=9; a value of 7:9<x<10; and a value of 8:x>10, such that ˜300 sampled motion-characteristics fall into each range. The software may calculate discretization threshold values for the population for each motion-state for each of the motion-characteristics with discretization into 8 possible ranges for example.

0000<tables id="TABLE-US-00005" num="00005"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="4"><colspec colname="offset" colwidth="49pt" align="left" /><colspec colname="1" colwidth="42pt" align="center" /><colspec colname="2" colwidth="77pt" align="center" /><colspec colname="3" colwidth="49pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="3" rowsep="1">TABLE 3</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row><row><entry /><entry /><entry /><entry>Discretization</entry></row><row><entry /><entry>Motion-state</entry><entry>Motion-characteristic</entry><entry>thresholds</entry></row><row><entry /><entry namest="offset" nameend="3" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="4"><colspec colname="1" colwidth="49pt" align="left" /><colspec colname="2" colwidth="42pt" align="center" /><colspec colname="3" colwidth="77pt" align="center" /><colspec colname="4" colwidth="49pt" align="center" /><tbody valign="top"><row><entry>Example</entry><entry>9</entry><entry>(101) Average of X</entry><entry>−9, −6, 0, 5,</entry></row><row><entry>characteristic</entry><entry /><entry /><entry>8, 9, 10</entry></row><row><entry namest="1" nameend="4" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0101For example, software on the server may compare the frequencies of occurrence of each motion-characteristic in the data from an individual user/device to occurrence frequencies of motion-characteristics calculated for the population and assign a differentiation score for each motion-characteristic of the user. The probability table below (Table 4) demonstrates determination of a differentiation score using the example data provided above.

0000<tables id="TABLE-US-00006" num="00006"><table frame="none" colsep="0" rowsep="0" pgwide="1"><tgroup align="left" colsep="0" rowsep="0" cols="10"><colspec colname="offset" colwidth="35pt" align="left" /><colspec colname="1" colwidth="28pt" align="center" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="28pt" align="center" /><colspec colname="4" colwidth="28pt" align="center" /><colspec colname="5" colwidth="28pt" align="center" /><colspec colname="6" colwidth="28pt" align="center" /><colspec colname="7" colwidth="28pt" align="center" /><colspec colname="8" colwidth="28pt" align="center" /><colspec colname="9" colwidth="28pt" align="center" /><thead><row><entry /><entry namest="offset" nameend="9" rowsep="1">TABLE 4</entry></row><row><entry /><entry namest="offset" nameend="9" align="center" rowsep="1" /></row><row><entry /><entry>1</entry><entry>2</entry><entry>3</entry><entry>4</entry><entry>5</entry><entry>6</entry><entry>7</entry><entry>8</entry><entry>Total</entry></row><row><entry /><entry namest="offset" nameend="9" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="10"><colspec colname="1" colwidth="35pt" align="left" /><colspec colname="2" colwidth="28pt" align="center" /><colspec colname="3" colwidth="28pt" align="center" /><colspec colname="4" colwidth="28pt" align="center" /><colspec colname="5" colwidth="28pt" align="char" char="." /><colspec colname="6" colwidth="28pt" align="char" char="." /><colspec colname="7" colwidth="28pt" align="char" char="." /><colspec colname="8" colwidth="28pt" align="char" char="." /><colspec colname="9" colwidth="28pt" align="char" char="." /><colspec colname="10" colwidth="28pt" align="center" /><tbody valign="top"><row><entry>User A</entry><entry>300</entry><entry> 50</entry><entry>200</entry><entry /><entry /><entry /><entry /><entry /><entry>550</entry></row><row><entry /><entry>  55%</entry><entry>  9%</entry><entry>  36%</entry><entry /><entry /><entry /><entry /><entry /><entry>100%</entry></row><row><entry>Population</entry><entry>300</entry><entry>300</entry><entry>300</entry><entry>300</entry><entry>300</entry><entry>300</entry><entry>300</entry><entry>300</entry><entry>2400 </entry></row><row><entry /><entry>12.5%</entry><entry>12.5%</entry><entry>12.5%</entry><entry>12.5%</entry><entry>12.5%</entry><entry>12.5%</entry><entry>12.5%</entry><entry>12.5%</entry><entry>100%</entry></row><row><entry>Difference</entry><entry>  42.5</entry><entry>   3.5</entry><entry>  23.5</entry><entry>12.5</entry><entry>12.5</entry><entry>12.5</entry><entry>12.5</entry><entry>12.5</entry><entry>132</entry></row><row><entry namest="1" nameend="10" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0102For the example motion-characteristic above, the differentiation score calculated would be: (42.5+3.5+23.5+12.5+12.5+12.5+12.5+12.5)/8=16.5.

0103In one embodiment, motion-characteristics are ranked by score for each motion-state and a table comprising the top 16 ranked motion-characteristics for each motion-state (e.g., 2*16 records when only hand-held static motion-states <b>9</b> and <b>13</b> are considered) is prepared for returning to the user's device as a differentiation-template.

0104An example record in the differentiation-template that would be returned to the user's device for the example described above, would take the following form:

0000<tables id="TABLE-US-00007" num="00007"><table frame="none" colsep="0" rowsep="0" pgwide="1"><tgroup align="left" colsep="0" rowsep="0" cols="9"><colspec colname="1" colwidth="42pt" align="center" /><colspec colname="2" colwidth="77pt" align="center" /><colspec colname="3" colwidth="28pt" align="center" /><colspec colname="4" colwidth="28pt" align="center" /><colspec colname="5" colwidth="28pt" align="center" /><colspec colname="6" colwidth="28pt" align="center" /><colspec colname="7" colwidth="28pt" align="center" /><colspec colname="8" colwidth="28pt" align="center" /><colspec colname="9" colwidth="28pt" align="center" /><thead><row><entry namest="1" nameend="9" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry>Motion-state</entry><entry>Motion-characteristic Id</entry><entry>Disc-1</entry><entry>Disc-2</entry><entry>Disc-3</entry><entry>Disc-4</entry><entry>Disc-5</entry><entry>Disc-6</entry><entry>Disc-7</entry></row><row><entry namest="1" nameend="9" align="center" rowsep="1" /></row><row><entry>9</entry><entry>101 (average X)</entry><entry>−9</entry><entry>−6</entry><entry>0</entry><entry>5</entry><entry>8</entry><entry>9</entry><entry>10</entry></row><row><entry namest="1" nameend="9" align="center" rowsep="1" /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="9"><colspec colname="offset" colwidth="21pt" align="left" /><colspec colname="1" colwidth="21pt" align="center" /><colspec colname="2" colwidth="49pt" align="center" /><colspec colname="3" colwidth="21pt" align="center" /><colspec colname="4" colwidth="49pt" align="center" /><colspec colname="5" colwidth="21pt" align="center" /><colspec colname="6" colwidth="63pt" align="center" /><colspec colname="7" colwidth="21pt" align="center" /><colspec colname="8" colwidth="49pt" align="center" /><tbody valign="top"><row><entry /><entry>P1</entry><entry>P2</entry><entry>P3</entry><entry>P4</entry><entry>P5</entry><entry>P6</entry><entry>P7</entry><entry>P8</entry></row><row><entry /><entry namest="offset" nameend="8" align="center" rowsep="1" /></row><row><entry /><entry>42.5</entry><entry>−3.5</entry><entry>23.5</entry><entry>−12.5</entry><entry>−12.5</entry><entry>−12.5</entry><entry>−12.5</entry><entry>−12.5</entry></row><row><entry /><entry namest="offset" nameend="8" align="center" rowsep="1" /></row></tbody></tgroup></table></tables><br /> wherein the first two columns designate the motion-state and unique id of each motion-characteristic, columns Disc-1 to Disc-7 designate the threshold values for discretization and columns P<b>1</b> to P<b>8</b> designate scores representing the probability occurrence of values in each range in relation to population values. These are designed so that during the step of comparing the motion-characteristics of the device of an unidentified user to the authorized user's motion-repertoire to determine if the user is the authorized user (Step <b>528</b>), positive scores will be calculated when a new sample conforms to the frequency of occurrence relative to the population that was calculated for the differentiation-template and negative scores will be attributed when the new sample measure contradicts the relative frequency calculated.

0105Unexpectedly, individual differences were found between the sets of motion-characteristics that best differentiate an individual user from the population.

0106Tables 5A and 5B below show, for example, differentiation scores of the 8 top ranking motion-characteristics in motion-state <b>13</b> for 2 different users, user A and user B, illustrating these individual differences.

0000<tables id="TABLE-US-00008" num="00008"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="1"><colspec colname="1" colwidth="217pt" align="center" /><thead><row><entry namest="1" nameend="1" rowsep="1">TABLE 5B</entry></row></thead><tbody valign="top"><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row><row><entry>User B</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="28pt" align="left" /><colspec colname="1" colwidth="112pt" align="left" /><colspec colname="2" colwidth="77pt" align="center" /><tbody valign="top"><row><entry /><entry>Characteristic</entry><entry>Score</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="28pt" align="left" /><colspec colname="1" colwidth="112pt" align="left" /><colspec colname="2" colwidth="77pt" align="char" char="." /><tbody valign="top"><row><entry /><entry>X First</entry><entry>21.2</entry></row><row><entry /><entry>X Sum</entry><entry>19.5</entry></row><row><entry /><entry>X Speed</entry><entry>19</entry></row><row><entry /><entry>(acceleration * duration)</entry></row><row><entry /><entry>Y Last</entry><entry>18.8</entry></row><row><entry /><entry>Y Distance</entry><entry>18.7</entry></row><row><entry /><entry>(acceleration * duration * duration)</entry></row><row><entry /><entry>dZ Sum positive</entry><entry>18.6</entry></row><row><entry /><entry>XYZ Average</entry><entry>17.2</entry></row><row><entry /><entry>Z Standard deviation</entry><entry>17.1</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0000<tables id="TABLE-US-00009" num="00009"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="1"><colspec colname="1" colwidth="217pt" align="center" /><thead><row><entry namest="1" nameend="1" rowsep="1">TABLE 5A</entry></row></thead><tbody valign="top"><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row><row><entry>User A</entry></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="49pt" align="left" /><colspec colname="1" colwidth="49pt" align="left" /><colspec colname="2" colwidth="119pt" align="center" /><tbody valign="top"><row><entry /><entry>Characteristic</entry><entry>Score</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="offset" colwidth="49pt" align="left" /><colspec colname="1" colwidth="49pt" align="left" /><colspec colname="2" colwidth="119pt" align="char" char="." /><tbody valign="top"><row><entry /><entry>Y Average</entry><entry>18.3</entry></row><row><entry /><entry>Y Jitter</entry><entry>17.9</entry></row><row><entry /><entry>Z Average</entry><entry>17.8</entry></row><row><entry /><entry>Z Jitter</entry><entry>17.2</entry></row><row><entry /><entry>Y Last</entry><entry>16.5</entry></row><row><entry /><entry>Z Max</entry><entry>16.4</entry></row><row><entry /><entry>Z Last</entry><entry>16.1</entry></row><row><entry /><entry>X Max-Min</entry><entry>15.9</entry></row><row><entry /><entry namest="offset" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

0107In addition to blocking access by unauthorized users, the capability of a device to recognize who is using it, may be useful for automatic authentication on a network. Once the device recognizes that the device is being used by its authorized owner, various techniques can be applied (alone or in combination) for securely and automatically authenticating the device and its owner to various networked services. Such techniques may include the following methods:

0108Combining a motion-repertoire with a unique device ID to generate a unique network password representing the combination of device and user

0109Accessing encrypted network authentication credentials stored on the device when the authorized user is detected

0110Accessing a private key that can be used to apply a digital signature when the authorized user is detected

0111Exposing a private key that can be used to apply a digital signature when the device is operated by a recognized authorized user.

0112Depositing the authorized user's motion-repertoire with a 3rd party certificate authority for use as an additional identity verification credential.

0113Automatic authentication to network services can be useful for multiple applications including: authentication of a user to a corporate network, authentication to content subscription services (e.g., music, news, games, adult entertainment, etc.) authentication to banking services or for accessing any website that requires a user to login. Automatic authentication based on a device's capability to recognize who is holding it should be useful for authorizing a variety of transactions, including authorization of payment instructions and opening physical barriers.

0114A system according to embodiments of the invention, in which some processing units are located at a remote server, is schematically illustrated in <figref idref="DRAWINGS">FIG. 6</figref>.

0115In one embodiment, sampling data from a sensor of the mobile device (<b>601</b>), detecting a motion-state of the device (<b>602</b>) and embellishing the data with attributes (<b>603</b>) are performed on the mobile device <b>6001</b>. Calculating current motion-characteristics (<b>607</b>) and comparison of current motion-characteristics collected by the device to the authorized user's differentiation-template (<b>608</b>) may be performed on the device <b>6001</b> and defensive actions (<b>609</b>) may be triggered on the device.

0116In addition or alternatively embellished data may be transferred (<b>606</b>) to server components <b>6002</b> located on the Internet cloud and may be further processed at a remote server, for example, by aggregating motion-sequences (<b>612</b>) and determining differentiating motion-characteristics (<b>613</b>) based on data from a plurality of devices that is processed in parallel on the server (e.g., embellished samples of a population (<b>611</b>) that are discretized (<b>614</b>) on the server to obtain motion-characteristics of a population (<b>616</b>)).

0117Once processing on server components <b>6002</b> has detected differentiating characteristics, the data is returned to the device (<b>624</b>) as a differentiation-template (<b>625</b>), which is then used for efficient comparison of motion-characteristics collected by the device to the most differentiating motion-characteristics of the authorized user's motion-repertoire according to the differentiation-template (<b>608</b>) and defensive actions may be triggered on the device (<b>609</b>) accordingly.

0118According to one embodiment, the background software collects the stream of embellished data in a data file saved to static memory on the device (<b>603</b>). At defined intervals (for example, every 24 hours) the software performs an API call to a server component residing on the Internet cloud to transfer the collected data file of embellished accelerometer samples for analysis (<b>606</b>). Data may be stored in a database at the server (<b>610</b>).

0119In one embodiment, creating from the motion-sequences motion-characteristics includes aggregating the motion-sequences (<b>612</b>). For example, software on the server may demarcate the embellished accelerometer data into motion-sequences based on the motion-state attribute of each record and calculate aggregations for each motion-sequence. Motion-sequences that extend beyond 3 seconds, for example, are separated into 3-second chunks. In one embodiment aggregated motion-sequences can be calculated to obtain data regarding, for example, jitter of the user's hand while holding the device.

0120In one embodiment sample data from a plurality of users (<b>611</b>) is demarcated into motion-sequences, aggregated and discretized (<b>614</b>) and motion-characteristics of a population (<b>616</b>) are calculated. Motion-characteristics of the device (<b>615</b>) are compared to motion-characteristics of the population to determine the best differentiating characteristics (<b>613</b>) which may be performed as described above.

0121According to some embodiments, software on the user's device requests a differentiation-template from the server. The background software may perform an API call to the server at some intervals to collect the differentiation-template prepared for it by the server (<b>624</b>). A table comprising the template values may be loaded into a dynamic memory of the device.

0122The background software performs analysis on motion-characteristics extracted from the current data stream using the differentiation-template (<b>607</b>). In the case of an example template comprising 16 best differentiating motion-characteristics of an authorized user, aggregation and discretization is performed on each motion-sequence in the data stream for each of the 16 characteristics appearing in the differentiation-template for a current motion-state, according to threshold values defined in the differentiation-template and implementing logic that emulates the aggregation calculations performed by the server for calculating only the motion-characteristics of interest. When comparing motion-characteristics to a differentiation-template to recognize an authorized user (<b>608</b>), for each motion-sequence, each of the 16 characteristics is scored based on the probability score appearing in the differentiation-template for the motion-state and motion-characteristic. Scores for the 16 characteristics are summed to comprise a detection score. A high score may indicate the likelihood that the motion-sequence was generated while in the hands of the authorized user and a low score may indicate the probability of an unauthorized user.

0123In the case where an unauthorized user is detected, defensive actions are triggered, and possibly, progressively severe defensive actions applied (<b>609</b>).

0124For example, when the background software detects a motion-sequence that scores below a configured threshold value (for example 10), the software suspects that the device is held by an unauthorized user and logic for performing defensive actions is triggered. The software may perform a configurable combination of progressively severe defensive actions when an unauthorized user is detected, including audio alert, device lockout, deletion of files and sending a message (email or SMS) to a designated address with current location data and other information sensed by the device.

0125Alternatively, if it is determined that the device is being held by the user and not by an unauthorized user, the determination may be used in one or more of: privacy protection, authentication to an external system, use as a universal key and authorizing payments.

0126An embodiment of the present invention provides for the capability of the detection system to automatically calibrate itself to increase the accuracy of recognition using a strategy that is personalized for an individual authorized user and a device.

0127The accuracy of biometric systems is typically measured using a FAR (False Acceptance Ratio) and a FRR (False Rejection Ratio). Ideally both of these ratios should be as small as possible, but reducing one (by applying more stringent recognition criteria for example) is typically done at the expense of increasing the other.

0128The number of characteristics sampled (<b>601</b>) and the number of intervals discretized for calculating motion-characteristics (<b>607</b>) impacts device performance. Devices vary in hardware capabilities, so performance constraints may dictate sampling fewer characteristics. Individual user behavior characteristics differ such that a particular motion-characteristic or combination of characteristics may be more efficient in differentiating between an individual authorized user and an intruder.

0129The recognition accuracy of the proposed system can be dynamically controlled by software through: (1) sampling more or fewer different motion-characteristics (from the many available on a modern mobile device) and (<b>2</b>) discretizing the sample variables collected into smaller or larger intervals.

0130By monitoring FAR &amp; FRR for an individual authorized user, the system has the capability to dynamically scale itself appropriately to maximize accuracy for an individual device and user. The proposed system monitors device performance and FAR and FRR values for an individual authorized user to detect and automatically calibrate itself to provide an optimum balance between device performance and recognition accuracy for an individual authorized user.

0131Advantages offered by embodiments of the present invention include, among others, the capability for dynamic self-calibration of the system.

0132For example, triggering of acceleration events (<b>601</b>) and the subsequent processing steps performed by the background software may consume processing resources on the device and may be detrimental to battery conservation. To conserve energy, the system can regulate the frequency in which acceleration events are triggered dependent on the motion-state determined (<b>602</b>). Different states may warrant different collection frequencies, thus for example, when the device is not being held by a user, it may be adequate to briefly sample the accelerator every 10 seconds to determine if a change of state has occurred instead of continuous sampling that may be warranted when the device is in a user's hand.

0133In another example, the frequency of sending motion samples of the device to the remote server (<b>606</b>) may be based on the consistency of differentiation-templates that are obtained from the remote server. While software on the device initiates calls to the server to submit data and to collect a differentiation-template (<b>606</b> and <b>624</b>), the server may determine the timing of these calls. According to one embodiment, the response message returned by the server to each API call (<b>606</b> and <b>624</b>) includes a time field indicating to the software service on the device the time to call back with a subsequent request. This technique has several advantages:

01341. by estimating the time required for analyzing data received from the user (and from multiple other users), the server is able to throttle its work load and avoid unnecessary API calls from devices; and

01352. by comparing the similarity of the differentiation-template returned to previously transmitted templates, the server is able to gauge the degree of consistency of the template over time and to determine progressively less frequent API calls by the device to the server as the degree of consistency increases.

0136For example, at initial installation the software service on the device may be programmed to call the server every 24 hours, following several API calls, the server may determine that a weekly or monthly frequency becomes more appropriate based on the similarity of the newly learned differentiation-template to one that was previously learned.

0137In another example, a measure of overall recognition accuracy (device learning-stage) may be determined by the server. The server may determine the learning-stage of the device by running a simulation of the steps of requesting a differentiation-template from the server and detecting unauthorized use at the server on a random sample of motion-sequences comprising an equal number of records from device and population historical data. Counts of FRR (False Rejection Ratio) (motion-characteristics from the population not detected as unauthorized users) and FAR (False Acceptance Ratio) (motion-sequences from device detected as unauthorized users) are calculated and a learning-stage is determined by comparing these counts.

0138In another example, a measure of overall recognition accuracy (device learning-stage) may be determined by the server by comparing a currently generated differentiation-template of a user to prior templates generated for the same user and by gauging the degree of template consistency relative to threshold values to determine the device learning-stage of a user. The learning-stage measure may be returned to the device together with the differentiation-template for determining appropriate defensive actions for example at Step <b>609</b>.

0139According to one embodiment of the invention there is provided a system for mobile device user recognition. The system according to one embodiment of the invention is schematically illustrated in <figref idref="DRAWINGS">FIG. 7</figref>.

0140According to one embodiment, the system <b>700</b> includes a mobile device <b>70</b> being used by a user, the mobile device having one or more motion sensors <b>72</b>, such as an accelerometer, gyroscope or touch screen; and a processor <b>73</b> to receive data from the device sensors to determine, according to one embodiment, motion-states of the device based on the received data. A motion-state of the device describes the motion and placement of the device <b>70</b> at a period in time. Motion-state is extrapolated from analysis of the sensor data outputted by the device at a particular point in time.

0141Also included in the system is a unit <b>75</b> to demarcate motion-sequences based on a change in motion-state or after an elapsed time of the sequence (for example 3 seconds) and create motion-characteristics based on aggregations calculated on a motion-sequence; a unit <b>76</b> to generate a motion-repertoire of an authorized user by accumulating the motion-characteristics collected; a processor <b>77</b> to compare freshly collected motion-characteristics to the motion-repertoire of an authorized user and to determine, based on the comparison, if the device <b>70</b> is being used by the authorized user; and a unit <b>78</b> for triggering defensive actions or for enabling authentication based on the determination if the device is being operated by the authorized user.

0142According to one embodiment the motion-characteristics of the device that are created by the unit <b>75</b> are indicative of the user's hand movements while the user is holding the device, perhaps just before using the device. Thus, according to one embodiment, hand movement parameters are searched for while the user is stationary (e.g., slow moving or resting) such that the motion-characteristics are obtained from sensor input indicating that the user is moving at below walking speed.

0143According to one embodiment the system <b>700</b> may include a processor to calculate motion-sequences of a plurality of devices (not shown) to obtain motion-characteristics of a population and to compare the motion-characteristics of a user to the motion-characteristics of the population to rank motion-characteristics of a user by the degree that a distribution of occurrences of the characteristic sampled for a user differs from the distribution of occurrences in the population; thus generating a differentiation-template that can assist in identifying if a user of the device <b>70</b> is an authorized user.

0144Some of the units or processors that are part of the system <b>700</b> may be located on a remote server (as shown in <figref idref="DRAWINGS">FIG. 6</figref>). For example, the processor to calculate motion-sequences of a plurality of devices may be part of a remote server which is in communication with the mobile device <b>70</b>.

0145If it is determined that a user is not an authorized user defensive actions may be triggered by unit <b>78</b>. Defensive actions may include actions such as blocking access to features of the device. Alternatively, unit <b>78</b> may enable authentication, including, for example, privacy protection, authentication to an external system, authorizing payments, enabling use of the device as a universal key for providing physical access and identifying friendly forces in a battle field environment.

0146While the present invention has been described with respect to a limited number of embodiments, it will be appreciated that many variations, modifications, and other applications of the present invention may be made.