System and method for monitoring eye movement

This application is a continuation of application Ser. No. <b>11</b>/<b>097</b>,<b>942</b>, filed Apr. <b>1</b>, <b>2005</b>, issued as Re <b>39</b>,<b>539</b> on Apr. <b>3</b>, <b>2007</b>, which is a reissue of application Ser. No. <b>09</b>/<b>740</b>,<b>738</b>, U.S. Pat. No. <b>6</b>,<b>542</b>,<b>081</b>, which is a continuation-in-part of application Ser. No. 09/104,258, filed Jun. 24, 1998, issuing as U.S. Pat. No. 6,163,281 on Dec. 19, 2000, which is a continuation-in-part of application Ser. No. 08/978,100, filed Nov. 25, 1997, now U.S. Pat. No. 6,246,344 issued Jun. 12, 2001, which is a continuation-in-part of application Ser. No. 08/699,670, filed Aug. 19, 1996, now U.S. Pat. No. 5,748,113 issued May 5, 1998, the disclosures of which are expressly incorporated herein by reference.

Notice: More than one reissue application has been filed for the reissue of U.S. Pat. No. <b>6</b>,<b>163</b>,<b>281</b>. The reissue applications are application Ser. No. <b>11</b>/<b>097</b>,<b>942</b>, issued as Re <b>39</b>,<b>539</b>, Ser. No. <b>11</b>/<b>732</b>,<b>828</b> (the present application), which is a continuation reissue of Re <b>39</b>,<b>539</b>, and Ser. No. <b>12</b>/<b>199</b>,<b>693</b>, which is a divisional reissue of the present application.

FIELD OF THE INVENTION

The present invention relates generally to systems and methods for monitoring movement of a human eye, and more particularly to systems and methods for real-time monitoring of fatigue and other states of mind in individuals, purposeful communication, and/or controlling devices based upon movement of their eye, eyelid, and/or other components of their eye.

BACKGROUND

There have been attempts to use movement of the human eye to monitor involuntary conditions, specifically a person's wakefulness or drowsiness. For example, U.S. Pat. No. 3,863,243 discloses a device that sounds an alarm to warn a person using the device that they are beginning to fall asleep. The device includes a frame similar to a set of eyeglasses onto which is mounted a fiber optic bundle and a photocell that are directed towards the user's eye when the frame is worn. The fiber optic bundle is coupled to a source of light and a pulse generator to emit light towards the user's eye.

The photocell detects the intensity of light reflected off of the user's eye, i.e., either by the eyelid when the eye is closed or the eye surface when the eye is open. Circuitry receives a signal from the photocell, and uses a timer to distinguish between regular blinks, and an extended time period during which the eye is closed, i.e., a time period that may indicate that the person is falling asleep. When a threshold time elapses, an alarm is sounded to notify and/or wake the user. This device, however, requires running wires and fiber optic bundles from the frame to external components, e.g., the pulse generator and the required circuitry, and for this reason, the device may be awkward or inconvenient to use.

Other devices, such as those disclosed in U.S. Pat. Nos. 5,469,143 and 4,359,724, directly engage the eyelid or eyebrow of a user to detect movement of the eye and activate an alarm when a drowsiness condition is detected. These mechanical devices may be mounted directly onto the skin to detect muscle movement or may involve placing a mechanical arm against the eyelid, and consequently may be uncomfortable to wear and use.

In addition, some devices may detect eye movement, but may not be able to distinguish when the eye is opened or closed. For example, it may be desirable to measure the percentage of total time that the eyelids are closed as a function of time or the area of the palpebral fissure that is covered by the eyelid as the eye is opened or closed, commonly known as “PERCLOS,” for example during medical research or when monitoring driver alertness. Devices that merely detect eye muscle movement or eyelash movement may not be able to distinguish when the eye is open or closed, and consequently may not be able to measure PERCLOS. Similarly, such devices may not measure other parameters, such as velocity of eyelid closing or opening, acceleration or deceleration characteristics, duration of open or closed eye states, intervals between eye blinks and/or partial versus full eye blinks or eye closures.

Further, infrared cameras or other devices may be used to monitor a driver's awareness, which are typically mounted on the dashboard, roof or other fixed mounting within the user's vehicle. Such devices, however, require that the user maintain constant eye contact with the camera. In addition, they do not monitor eyelid movement if the user looks sideways or downwards, turns around, exits the vehicle or compartment in which he or she is being monitored, or if the camera moves relative to the individual. Further, such cameras may have problems seeing through eyeglasses, sunglasses, or even contact lenses, and may not operate effectively in sunlight.

Accordingly, it is believed that a more effective system and method for monitoring eye and/or eyelid movement would be considered useful.

SUMMARY OF THE INVENTION

The present invention is directed to systems and methods for monitoring eye movement. Generally, humans blink at least about 5-30 times per minute, or about 7,000-43,000 times per day. Each involuntary-reflexive blink lasts about 200-300 milliseconds, and generally averaging about 250 milliseconds, amounting to about 1,750-10,800 seconds per day of eye closure due to involuntary blinking. As tiredness or sleepiness occurs, the eye blink gets longer and slower until the eyes begin to close for short term “microsleeps,” i.e., sleep conditions that last for about 3-5 seconds or longer, or for prolonged sleep. The present invention provides systems and methods for monitoring, measuring, and/or responding to eye movement, e.g., nonpurposeful reflexive eyeblinks.

In a preferred embodiment, the system includes an emitter and a sensor in a predetermined relationship with an eye such that the emitter emits light and the sensor detects light from the emitter, the sensor producing a light intensity signal indicating when the eye is open or closed. More preferably, the emitter is directed or aimed at the eyelid and eye, while the sensor detects eyelid-reflected light, since, unlike the eyelid, the eye ball (except the retina, which may cause a “red reflex” under white light conditions or “white pupil” under infrared light) does not reflect substantial light back to the sensor. Circuitry is coupled to the sensor for converting sequential light intensity signals corresponding to eyelid movement received from the sensor into a stream of data, and a processor converts the stream of data into an understandable message.

The circuitry for converting sequential light intensity signals may compare the sequential light intensity signals with a predetermined time threshold to detect voluntary-intentional or unintentional-involuntary sequences of eyelid movements, corresponding, for example, to a predetermined binary code. Memory circuitry may be coupled to the processor for storing the stream of data and/or a communication device, such as a video monitor or synthesized voice module, may be coupled to the processor for communicating the understandable message. In addition, a control system may be coupled to the processor, and the understandable message may include a command for controlling equipment, including electrical or electronic equipment, machinery, or a computer or computer accessory devices coupled to the control system.

The system preferably also includes a transmitter, preferably a radio frequency transmitter, for wireless transmission of the stream of data to a remote location. Alternatively, other forms of wireless transmission, e.g. infrared, as well as hard-wire connections may be used. The processor, as well as the memory circuitry, communication device, and/or control system, may be located at the remote location, and a receiver may be coupled to the processor for receiving the stream of data from the transmitter.

In a preferred form, the system includes a detection device having a frame adapted to be worn on a person's head, e.g., with the frame resting on the bridge of the user's nose and/or ears. The frame has the emitter and sensor thereon such that the emitter and sensor are oriented towards the person's eye when the frame is worn on the person's head. Preferably, the emitter and sensor are a single solid state device, such as a biosensor device, that emits light within a predetermined frequency range, for example infrared light, towards the eye and detects the emitted light reflected off of the eyelid, respectively.

In another preferred embodiment, a system for monitoring a blinking cycle of a person from a remote location is provided that includes an emitter for directing light towards an eye, and a sensor in a predetermined relationship with the emitter for detecting the emitted light reflected off of the eye, the sensor producing an output signal indicating when the eye is open or closed. Depending upon the relative position of the emitter and sensor with respect to the moving eyelid, the emitter light may be reflected off of the eyelid back to the sensor, or diffused by the surface of the eyeball.

A transmitter is coupled to the sensor for wireless transmission of the output signal, and a processor is provided for comparing the output signal to a predetermined threshold to detect when the eyelid is closed for a minimum predetermined duration. A warning indicator may be coupled to the processor, the warning indicator being activated when the processor detects that the eyelid is closed for the minimum predetermined duration. For example, the warning indicator may be an audible buzzer, a visible warning light, a vibrating device, an electrical shock device, a gustatory smell device, or other device that may act as a stimulus to any sensory modality.

Similar to the previous embodiment, a receiver may be provided at the remote location coupled to the processor for receiving the wireless transmission from the transmitter. Memory circuitry may be provided for storing the output signal and/or a processor may be provided for converting the output signal into an understandable message. A communication device may be coupled to the processor for communicating the understandable message.

In another preferred embodiment, a self-contained device for detecting movement of a person's eyelid is provided that includes a frame adapted to be worn on the person's head, an emitter on the frame for directing light towards an eye of the person when the frame is worn, and a sensor on the frame for detecting light from the emitter. The sensor produces an output signal indicating when the eye is open or closed, and a transmitter on the frame is coupled to the sensor for wireless transmission of the output signal to a remote location. The frame may also include a processor for comparing the output signal to a predetermined threshold to detect drowsiness-induced eyelid movement. Similar to the previous embodiments, the emitter and sensor are preferably a solid state biosensor device for emitting and detecting infrared light, or alternatively an array of emitters and/or sensors in a predetermined configuration on the frame, e.g., in a vertical, horizontal, diagonal, or other linear or other geometric array of more than one emitter and/or sensor oriented towards one or both eyes. In particular, an array of emitters and/or sensors allows measurement of eyelid velocity, acceleration and deceleration, and calculation of “PERCLOS.”

The emitter and/or sensors may be affixed to any number of points on the frame, e.g., around the lens and preferably in the nose bridge, or alternatively anywhere along the frame, including near or on the nasal portion of the frame, the attachment of a temple piece of the frame, and/or surface mounted on the lens of an eyeglass. Alternatively, the emitter and/or sensor may be embedded in the lens of an eyeglass, or otherwise such that they operate through the lens. Thus, the emitter(s) and/or sensor(s) are fixed on an eye-frame such that they move with the wearer's head movements, and continuously focus on the user's eyes, whether the user is in a vehicle, outdoors or in any other environment.

Thus, a system in accordance with the present invention may detect eyelid movement of the user, distinguish normal blinks from other voluntary or involuntary eyelid movement, and produce a stream of data. The stream of data may be converted into an understandable message, such as a binary code, a command for controlling a piece of equipment, or an indicator of the user's physical, mental or emotional state. Thus, the system may provide a convenient and/or effective method for voluntary or involuntary communication based simply upon movement of the user's eye.

In accordance with another aspect of the present invention, a system is provided for monitoring movement of a person's eye. The system includes a device configured to be worn on a person's head and an array of emitters on the device for directing light towards an eye of the person when the device is worn. The array of emitters is configured for projecting a reference frame towards the eye. A camera is oriented towards the eye for monitoring movement of the eye relative to the reference frame. The camera may be provided on the device or may be provided remote from the device, but in relatively close proximity to the user.

Preferably, the array of emitters includes a plurality of emitters disposed in a substantially vertical arrangement on the device, and a plurality of emitters disposed in a substantially horizontal arrangement on the device. Thus, the array of emitters may project a focused set of crossed bands towards the eye for dividing a region including the eye into four quadrants.

In addition, the system preferably includes one or more scanning or nonscanning sensors on the device for detecting light from the array of emitters. The one or more sensors produce an output signal indicating when the eye is open or closed, similar to the embodiments described above. More preferably, the sensors include an array of focused sensors in a predetermined relationship with the array of focused emitters for detecting light from the array of emitters that is reflected off of respective portions of the eye or its eyelid. The emitters, because of their fixed position, produce a fixed reflection off of the surface of the eye and eyelid, appearing as a “glint,” i.e., a spot or band of light. Each sensor produces an output signal indicating when the respective portion of the eye is covered or not covered by the eyelid.

The system may also include a processor for correlating the output signal from the one or more sensors with a video signal from the camera for determining the person's level of alertness. The system may also include a warning indicator on the device, the warning indicator being activated when the processor determines a predetermined level of drowsiness has occurred.

Light from the array of emitters may be emitted towards the eye of a user wearing the device to project a reference frame onto the eye. The camera is capable of imaging light produced by the emitters, e.g., in the infrared light range, thereby detecting the projected light as a spot of light, band of light or other “glint.” Movement of the eye relative to the reference frame may be monitored with the camera. A graphical output of the movement monitored by the camera relative to the reference frame may be monitored. For example, infrared light from the emitters may be reflected off of the retina as a “red reflex” under white light, as a “white pupil” under infrared light, or as a dark pupil under subtraction, using methods known to those skilled in the art. The processor, using these methods, may detect movement of the eye's pupil may be measured relative to the reference frame. This movement may be graphically displayed, showing the movement of the eye's pupil relative to the reference frame.

In addition, the output signal from the one or more sensors may be correlated with video signals produced by the camera monitoring movement of the eye relative to the reference frame, thereby determining the person's level of drowsiness.

Other objects and features of the present invention will become apparent from consideration of the following description taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

<figref idref="DRAWINGS">FIG. 1</figref> is a perspective view of a patient in a hospital wearing a system for communication using eyelid movement in accordance with the present invention.

<figref idref="DRAWINGS">FIG. 2</figref> is an enlarged perspective view of a preferred embodiment of the system for communication using eyelid movement, shown in <figref idref="DRAWINGS">FIG. 1</figref>, including a detection device and a processing box.

<figref idref="DRAWINGS">FIG. 3</figref> is a schematic drawing of a preferred embodiment of circuitry for transmitting an output signal corresponding to a sequence of eyelid movements.

<figref idref="DRAWINGS">FIG. 4</figref> is a schematic drawing of a preferred embodiment of circuitry for controlling equipment in response to an output signal corresponding to a sequence of eyelid movements.

<figref idref="DRAWINGS">FIG. 5</figref> is a schematic drawing of a preferred embodiment of circuitry for detecting eyelid movement.

<figref idref="DRAWINGS">FIGS. 6A-6C</figref> are sectional and front views of alternate embodiments of a device for emitting light towards and detecting light reflected from a surface of an open eye.

<figref idref="DRAWINGS">FIGS. 7A-7C</figref> are sectional and front views of the devices of <figref idref="DRAWINGS">FIGS. 6A-6C</figref>, respectively, emitting light towards and detecting light reflected from a closed eyelid.

<figref idref="DRAWINGS">FIG. 8</figref> is a perspective view and block diagram of another preferred embodiment of a system for communication using eyelid movement.

<figref idref="DRAWINGS">FIG. 9</figref> is a block diagram of the components of a system for communication in accordance with the present invention.

<figref idref="DRAWINGS">FIG. 10A</figref> is a perspective view of still another preferred embodiment of a system for communication using eyelid movement.

<figref idref="DRAWINGS">FIG. 10B</figref> is a schematic detail of a portion of the system of FIG. <b>10</b>A.

<figref idref="DRAWINGS">FIG. 10C</figref> is a detail of a preferred embodiment of an array of emitters and sensors that may be provided on a nose bridge of an eye frame, such as that of FIG. <b>10</b>A.

<figref idref="DRAWINGS">FIG. 10D</figref> is a sectional view of the array of emitters and sensors of <figref idref="DRAWINGS">FIG. 10C</figref> emitting light and detecting light reflected from an eye.

<figref idref="DRAWINGS">FIG. 11A</figref> is a schematic view of a system for selectively controlling a number of devices from a remote location based upon eyelid movement.

<figref idref="DRAWINGS">FIG. 11B</figref> is a schematic view of additional devices that may be controlled by the system of FIG. <b>11</b>B.

<figref idref="DRAWINGS">FIG. 12A</figref> is a table showing the relationship between the activation of an array of sensors, such as that shown in <figref idref="DRAWINGS">FIGS. 10A-10D</figref> and an eye being monitored by the array, as the eye progresses between open and closed conditions.

<figref idref="DRAWINGS">FIG. 12B</figref> is a graph showing a stream of data provided by an array of sensors, such as that shown in <figref idref="DRAWINGS">FIGS. 10A-10D</figref>, indicating the percentage of eye coverage as a function of time (“PERCLOS”).

<figref idref="DRAWINGS">FIG. 12C</figref> is a graphical display of a number of physiological parameters, including PERCLOS, of a person being monitored by a system including a device such as that shown in <figref idref="DRAWINGS">FIGS. 10A-10D</figref>.

<figref idref="DRAWINGS">FIG. 12D</figref> is a table showing the relationship between the activation of two-dimensional arrays of sensors and an eye being monitored, as the eye progresses between open and closed conditions.

<figref idref="DRAWINGS">FIG. 13</figref> is a perspective view of another system for monitoring eye movement, in accordance with the present invention.

<figref idref="DRAWINGS">FIG. 14</figref> is a detail of a camera on the frame of FIG. <b>13</b>.

<figref idref="DRAWINGS">FIGS. 15A-15I</figref> are graphical displays of several parameters that may be monitored with the system of FIG. <b>13</b>.

<figref idref="DRAWINGS">FIG. 16</figref> is a detail of video output from a camera on the frame of FIG. <b>13</b>.

<figref idref="DRAWINGS">FIG. 17</figref> is a schematic showing circuitry for processing signals from a five-element sensor array, in accordance with an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Turning now to the drawings, <figref idref="DRAWINGS">FIG. 1</figref> shows a patient <b>10</b> in a bed <b>12</b> wearing a detection device <b>30</b> for detecting eyelid movement of the patient <b>10</b> to provide voluntary-purposeful and/or involuntary-nonpurposeful communication. The detection device <b>30</b> is coupled to a processing box <b>130</b> which converts the detected eyelid movement into a stream of data, an understandable message and/or into information, which may be communicated, for example, using a video display <b>50</b>, to a medical care provider <b>40</b>. The detection device <b>30</b> and processing box <b>130</b> together provide a system for communication <b>14</b> in accordance with one aspect of the present invention.

Turning to <figref idref="DRAWINGS">FIGS. 2</figref>, <b>6</b>A and <b>7</b>A, a preferred embodiment of a system for communication <b>14</b> is shown that includes an aimable and focusable detection device <b>30</b> that is attachable to a conventional pair of eyeglasses <b>20</b>. The eyeglasses <b>20</b> include a pair of lenses <b>21</b> attached to a frame <b>22</b>, which includes bridgework <b>24</b> extending between the lenses <b>21</b>, and side members or temple pieces <b>25</b> carrying ear pieces <b>26</b>, all of which are conventional. Alternatively, because the lenses <b>21</b> are not necessary to the present invention, the frame <b>22</b> may also be provided without the lenses <b>21</b>.

The detection device <b>30</b> includes a clamp <b>27</b> for attaching to one of the side members <b>25</b> and an adjustable arm <b>31</b> onto which is mounted an emitter <b>32</b> and a sensor <b>33</b>. Preferably, the emitter <b>32</b> and sensor <b>33</b> are mounted in a predetermined relationship such that the emitter <b>32</b> may emit a signal towards an eye <b>300</b> of a person wearing the eyeglasses <b>20</b> and the sensor <b>33</b> may detect the signal reflected from the surface of the eye <b>300</b> and eyelid <b>302</b>. As shown in <figref idref="DRAWINGS">FIGS. 6A</figref> and <b>7</b>A, the emitter <b>32</b> and sensor <b>33</b> may be mounted adjacent one another.

Alternatively, as shown in <figref idref="DRAWINGS">FIGS. 6B and 7B</figref>, the emitter <b>32</b>′ and sensor <b>33</b>′ may be mounted on the frame separately away from one another, preferably such that the emitter <b>32</b>′ and sensor <b>33</b>′ are disposed substantially laterally with respect to each other. In a further alternative, shown in <figref idref="DRAWINGS">FIGS. 6C and 7C</figref>, the emitter <b>32</b>″ and sensor <b>33</b>″ may be mounted across the eye <b>300</b> in axial alignment with another. As the eyelid <b>302</b> closes, it may break the beam <b>340</b> being detected by the sensor <b>33</b>″.

In a preferred form, the emitter <b>32</b> and sensor <b>33</b> produce and detect continuous or pulsed light, respectively, preferably within the infrared range to minimize distraction or interference with the wearer's normal vision. Preferably, the emitter <b>32</b> emits light in pulses at a predetermined frequency and the sensor <b>33</b> is configured to detect light pulses at the predetermined frequency. This pulsed operation may reduce energy consumption by the emitter <b>32</b> and/or may minimize interference with other light sources. Alternatively, other predetermined frequency ranges of light beyond or within the visible spectrum, such as ultraviolet light, or other forms of energy, such as radio waves, sonic waves and the like, may be used.

The processing box <b>130</b> is coupled to the detection device <b>30</b> by a cable <b>34</b> including one or more wires therein (not shown). As shown in <figref idref="DRAWINGS">FIG. 9</figref>, the processing box <b>130</b> preferably includes a central processing unit (CPU) <b>140</b> and/or other circuitry, such as the exemplary circuitry shown in <figref idref="DRAWINGS">FIGS. 3-5</figref>, for receiving and/or processing an output signal <b>142</b>, such as a light intensity signal, from the sensor <b>33</b>. The processing box <b>130</b> may also include control circuitry <b>141</b> for controlling the emitter <b>32</b> and/or the sensor <b>33</b>, or the CPU <b>140</b> may include internal control circuitry.

For example, in a preferred form, the control circuitry <b>141</b> controls the emitter <b>32</b> to produce a flickering infrared signal pulsed at a predetermined frequency, as high as thousands of pulses per second to as little as about 4-5 pulses per second, and preferably at least about 5-20 pulses per second, thereby facilitating detection of nonpurposeful or purposeful eyeblinks as short as about 200 milliseconds per blink. The sensor <b>33</b> may be controlled to detect light pulses only at the predetermined frequency specific to the flicker frequency of the emitter <b>32</b>. Thus, by synchronizing the emitter <b>32</b> and the sensor <b>33</b> to the predetermined frequency, the system <b>10</b> may be used under a variety of ambient conditions without the output signal <b>142</b> being substantially affected by, for example, bright sun light, total darkness, ambient infrared light backgrounds, or other emitters operating at different flicker frequencies. The flicker frequency may be adjusted to minimize the efficient measurement of the number of eye blinks per unit time (e.g. about ten to about twenty eye blinks per minute), the duration of each eye blink (e.g. about 200 milliseconds to about 300 milliseconds), and/or PERCLOS (i.e., the percentage of time that the eyelid is completely or partially closed), or to maximize efficiency of the system, while keeping power consumption to a minimum.

The control circuitry <b>141</b> and/or processing box <b>130</b> may include manual controls (not shown) for adjusting the frequency, focus, or intensity of the light emitted by the emitter <b>32</b>, to turn the emitter <b>32</b> off and on, to adjust the threshold sensitivity of the sensor <b>33</b>, and/or to allow for self-focusing with maximal infrared reflection off of a closed eyelid, as will be appreciated by those skilled in the art.

In addition, the processing box <b>130</b> also preferably includes a power source <b>160</b> for providing power to the emitter <b>32</b>, the sensor <b>33</b>, the CPU <b>144</b>, and/or other components in the processing box <b>130</b>. The processor box <b>130</b> may be powered by a conventional DC battery, e.g., a nine volt battery or a lithium battery. Alternatively, an adapter (not shown) may be connected to the processor box <b>130</b>, such as a conventional AC adapter or a twelve volt automobile lighter adapter.

Preferably, the CPU <b>140</b> includes timer circuitry <b>146</b> for comparing the length of individual elements of the output signal <b>142</b> to a predetermined threshold to distinguish between normal blinks and other eyelid movement. The timer circuitry <b>146</b> may be separate discrete components or may be provided internally within the CPU <b>140</b>, as will be appreciated by those skilled in the art. The CPU <b>140</b> converts the output signal <b>142</b> into a stream of data <b>144</b> which may be used to communicate to other persons or equipment. For example, the stream of data <b>144</b> produced by the CPU <b>140</b> may be a binary signal, such as Morse code or ASCI code. Alternatively, the CPU <b>140</b> may be capable of producing a synthesized voice signal, a control signal for a piece of equipment, or even a pictorial representation.

To facilitate communication, the processing box <b>130</b> may include a variety of output devices for using the stream of data <b>144</b>. For example, an internal speaker <b>150</b> may be provided, which may produce an alarm sound or a synthesized voice. An output port <b>148</b> may be provided to which a variety of equipment, such as the video display <b>50</b> shown in <figref idref="DRAWINGS">FIG. 1</figref>, may be directly coupled by hard-wire connections.

The processing box <b>130</b> may also include a transmitter <b>152</b> coupled to the CPU <b>144</b> for wireless communication of the stream of data <b>144</b> to a remote location. For example, as shown in <figref idref="DRAWINGS">FIG. 9</figref>, the system for communication <b>14</b> may also include a receiving and processing unit <b>154</b>, such as a computer or other control or display system. The transmitter <b>152</b> is preferably a radio frequency transmitter capable of producing a short range signal, for example, reaching as far as about one hundred feet or more, and preferably about forty five feet to fifty feet, even through walls or obstacles, although alternatively an infrared transmitter may also be effective.

The transmitter <b>152</b> may also be coupled to an amplifier (not shown) to allow the stream of data to be transmitted thousands of feet or more. For example, the amplifier and transmitter <b>152</b> may communicate via telephone communication lines, satellites and the like, to transmit the stream of data to a remote location miles away from the system. The system may include, or may be coupled to a global positioning system (GPS) for monitoring the location, movement, and state of wakefulness and safety of an individual wearing the detection device <b>30</b>.

The receiving and processing unit <b>154</b> includes a receiver <b>156</b>, preferably a radio frequency receiver, for receiving a signal <b>153</b>, including the stream of data, transmitted by the transmitter <b>152</b>. A processor <b>158</b> is coupled to the receiver <b>156</b> for translating, storing and/or using the information in the stream of data, the processor <b>158</b> being coupled to memory circuitry <b>160</b>, a communication device <b>162</b>, and/or a control system <b>164</b>. For example, the receiving and processing unit <b>154</b> may include the memory circuitry <b>160</b> therein into which the processor <b>158</b> may simply store the stream of data for subsequent retrieval and analysis.

The processor <b>158</b> may interpret the stream of data, for example, by converting a binary code in the stream of data into an understandable message, i.e., a series of letters, words and/or commands, and/or may use augmentative communication devices or software (such as KE:NX or Words Plus) to facilitate communication. The resulting message may be displayed on the communication device <b>162</b>, which may include a video display for displaying text, pictures and/or symbols, a synthesized voice module for providing electronic speech, and the like.

Alternatively, the stream of data may be displayed graphically on a computer of video screen or other electronic display device as a “real time” message signal or numerically (e.g., displaying blink rate, blink duration, PERCLOS, etc.), or displayed graphically similar to an EKG or EEG tracing. In addition, as shown in <figref idref="DRAWINGS">FIG. 12C</figref>, the stream of data may be displayed along with other physiological data (e.g. heart rate, respiratory rate, other sleep polysomnographic (PSG) or electroencephalographic (EEG) variables). Alternatively, the stream of data may be integrated with controllers which monitor automobile or mechanical functions (e.g. vehicle speed, acceleration, braking functions, torque, sway or tilt, engine or motor speed, etc.) to make intelligent decisions regarding slowing down or speeding up the vehicle depending upon road and/or vehicle conditions, as well as the state of consciousness, wakefulness or attentiveness of the driver or machine operator.

In addition, the message may be interpreted by the processor <b>158</b> for directing the control system <b>164</b> to control one or more pieces of machinery or equipment. For example, the stream of data may include a command to direct the control system <b>164</b> to control relay switches or other devices to turn off and on an electrical device, such as an appliance, electrical wheelchair, engine, light, alarm, telephone, television, computer, a tactile vibrating seat, and the like, or to operate an eye-activated computer mouse or other controller.

Alternatively, the processor <b>158</b> may use the stream of data to control PC, IBM, Macintosh and other computers and compatible computer software and/or hardware, e.g., to interact with a computer similar to a mouse, a “return” key or a “joystick.” For example, the stream of data may include commands to activate a series of menus from which sub-menus or individual items may be selected, as are used in commercially available special communications software, such as WORDS-PLUS or Ke:NX. The processor <b>158</b> may then control, scroll or select items from computer software programs, operate a printer or other peripheral device (e.g., selecting a font, paragraph, tab or other symbol operator, selecting commands, such as “edit,” “find,” “format,” “insert,” “help,” or controlling CD-ROM or disc drive operations, and/or other Windows and non-Windows functions).

Alternatively, the receiver <b>156</b> may be coupled directly to a variety of devices (not shown), such as radio or television controls, lamps, fans, heaters, motors, remote control vehicles, vehicle monitoring or controlling devices, computers, printers, telephones, lifeline units, electronic toys, or augmentative communication systems, to provide a direct interface between the user and the devices.

During use, the detection device <b>30</b> is placed on a user's head, i.e., by putting the eyeglasses <b>20</b> on as shown in FIG. <b>1</b>. The adjustable arm <b>31</b> and/or the clamp <b>27</b> may be adjusted to optimally orient the emitter <b>32</b> and sensor <b>33</b> towards the user's eye <b>300</b> (shown in <figref idref="DRAWINGS">FIGS. 6A-6C</figref> and <b>7</b>A-<b>7</b>C). The emitter <b>32</b> is activated and a beam of light <b>340</b> is directed from the emitter <b>32</b> towards the eye <b>300</b>. The intensity and/or frequency of the emitter <b>32</b> and/or the threshold sensitivity of the sensor <b>33</b> or other focus may then be adjusted (e.g. manually or automatically using self-adjusting features).

Because of the difference in the reflective characteristics of the surface of the eye <b>300</b> itself and the eyelid <b>302</b>, the intensity of the light reflected off of the eye <b>300</b> depends upon whether the eye <b>300</b> is open or closed. For example, <figref idref="DRAWINGS">FIGS. 6A and 6B</figref> illustrate an open eye condition, in which a ray of light <b>340</b> produced by the emitter <b>32</b> strikes the surface of the eye <b>300</b> itself and consequently is scattered, as shown by the rays <b>350</b>. Thus, the resulting light intensity detected by the sensor <b>33</b> is relatively low, i.e., the sensor <b>33</b> may not receive any substantial return signal.

In <figref idref="DRAWINGS">FIGS. 7A and 7B</figref>, the eye <b>300</b> is shown with the eyelid <b>302</b> closed as may occur during normal blinks, moments of drowsiness, intentional blinks, or other eyelid movement. Because the light <b>340</b> strikes the eyelid <b>302</b>, it is substantially reflected back to the sensor <b>33</b>, as shown by the ray <b>360</b>, resulting in a relatively high light intensity being detected by the sensor <b>33</b>. Alternatively, as shown in <b>7</b>C, the beam of light <b>340</b> may be broken or cut by the eyelid <b>302</b> when the eye <b>300</b> is closed.

The sensor <b>33</b> consequently produces a light intensity signal that indicates when the eye <b>300</b> is open or closed, i.e., corresponding to the time during which reflected light is not detected or detected, respectively, by the sensor <b>33</b>. Generally, the intensity of the infrared light reflected from the surface of the eyelid is not substantially affected by skin pigmentation. If it is desired to adjust the intensity of light reflected from the eyelid, foil, glitter, reflective moisturizer creams and the like may be applied to increase reflectivity, or black eye liner, absorptive or deflective creams and the like may be applied to reduce reflectivity.

Returning to <figref idref="DRAWINGS">FIG. 9</figref>, the light intensity detected by the sensor <b>33</b> results in an output signal <b>142</b> including a series of time-dependent light intensity signals (as shown, for example, in FIG. <b>12</b>B). The output signal <b>142</b> is received by the CPU <b>140</b> coupled to the sensor <b>33</b>, which compares the length of time of each light intensity signal <b>142</b>, for example, corresponding to a closed eye condition, with a predetermined threshold. The timer circuitry <b>146</b> may provide a threshold time to the CPU <b>140</b> for distinguishing normal blinks from intentional and/or other unintentional eyelid movement, which the CPU <b>140</b> may then filter out of the output signal <b>142</b>. The CPU <b>140</b> then produces a stream of data <b>144</b> which may be used for voluntary and/or involuntary communication.

In one useful application, the detection device <b>30</b> may be used to detect impending drowsiness or “micro-sleeps” (i.e., sleep intrusions into wakefulness lasting a few seconds) of a user, with the processing box <b>130</b> triggering a warning to alert the user, others in his or her presence, or monitoring equipment of the onset of drowsiness. The threshold of the timer circuitry <b>146</b> may be adjusted such that the CPU <b>140</b> detects relatively long periods of eye closure, as may occur when a person is falling asleep.

For example, because normal blinks are relatively short, the threshold may be set at a time ranging from close to zero seconds up to several seconds, preferably from about 200 milliseconds to about 300 milliseconds, and most preferably about 250 milliseconds, to distinguish normal blinks from drowsiness-induced eyelid movement. When the CPU <b>140</b> detects a drowsiness condition, i.e., detects a high light intensity signal exceeding the predetermined threshold time, it may activate a warning device. The warning device may be included within the processing box <b>130</b>, such as the speaker <b>150</b>, or alternatively on the frame, for example, by mounting a warning light (not shown) or an alarm speaker (not shown in <figref idref="DRAWINGS">FIG. 9</figref>, see <figref idref="DRAWINGS">FIG. 10C</figref>) on the frame.

Alternatively, the detection device <b>30</b> may be used to unobtrusively record or monitor drowsiness-induced eyelid movement, with the CPU <b>140</b> producing a stream of data <b>144</b> which the transmitter <b>152</b> may transmit to the receiving and processing unit <b>154</b> (FIG. <b>9</b>). For example, the device <b>30</b> may be used in conjunction with a vehicle safety system to monitor a driver's level of awareness or attentiveness. The stream of data <b>144</b> may be transmitted to a receiving and processing unit <b>154</b> mounted in a vehicle, which may store data on the driver's drowsiness and/or may use the data to make decisions and control the vehicle, e.g., adjust the vehicle's speed or even turn the vehicle's engine off. Thus, the detection device <b>30</b> may be used to monitor truck drivers, taxi drivers, ship or airline pilots, train conductors or engineers, radar or airport control tower operators, operators of heavy equipment or factory machinery, scuba divers, students, astronauts, entertainment participants or observers, and the like. The signals may be stored and analyzed in real time for trend changes measured over time to predict drowsiness effects of individuals using the device.

The detection device <b>30</b> and system <b>14</b> may also be used in a medical diagnostic, therapeutic, research or professional setting to monitor the wakefulness, sleep patterns and/or the effects of drugs, which may affect blink rate, blink velocity, blink duration, or PERCLOS of a patient or vehicle operator. Similar to the method just described, the CPU <b>140</b> produces a stream of data <b>144</b>, which the transmitter may send to a remote receiving and processing unit <b>154</b>, which may store the stream of data <b>144</b> in the memory circuitry <b>160</b> for later retrieval and analysis by researchers, medical professionals, or safety personnel (e.g., similar to the way in which flight recorder data may be stored in an aircraft's “black box” recorder). The receiving and processing unit <b>154</b> may also display the stream of data <b>144</b>, for example at a nurse's station, as an additional parameter to continually monitor a patient's physical, mental, or emotional condition. The unit <b>154</b> may store and/or produce a signal, e.g., by a series of algorithms, that must be responded to within a predetermined time (e.g., performance vigilance monitoring) to prevent false positives and negatives.

A number of medical conditions may be monitored by the detection device <b>30</b> and system <b>14</b>, such as petit mal epilepsy, in which the eyes flutter at a rate of about three cycles per second, grand mal or psychometer seizures, where the eyes may stare or close repetitively in a jerky manner, myoclonic seizures, in which the lids may open and close in a jerky manner, or tics, or other eye movements, such as encountered by people with Tourette's syndrome. The system may be used to monitor g-lock of pilots caused by g-force effects, hypoxemia of passengers or crew in aircraft due to losses in cabin pressure, nitrogen narcosis or “the bends” in divers, or the effects of gases, chemicals, or biological agents on military personnel or other individuals.

The system may also be used to monitor psychological situations, for example, to detect when a person lies (e.g., by closing their eyes when lying), during hypnosis, to monitor attentiveness, the effects of medications, e.g., L-dopa and other anti-Parkinsonian medications or anti-convulsants, drugs, alcohol, toxins, or the effects of hypoxia or ventilation, and the like. Neurological conditions may also be monitored where the innervation or mechanical function of the eyelid may be affected, such as in Parkinson's disease, muscle diseases, e.g., myotonia, myotonic muscular dystrophy, blepharospasm, photophobia or light sensitivity, encephalopathy, seizures, Bell's palsy, or where the condition may produce eyelid drooping or ptosis, such as third cranial nerve palsy or paresis, brainstem lesions or stroke, tumors, infection, metabolic diseases, trauma, degenerative conditions, e.g., multiple sclerosis, amyotrophic lateral sclerosis, polyneuropathy, myesthenia gravis, botulism, tetanus, tetany, tardive dyskinesia, brainstem encephalitis, and other primary eyelid conditions, such as exophthalmos, thyrotoxicosis or other thyroid conditions.

Similarly, the detector device <b>30</b> may be used in biofeedback applications, for example, in biofeedback, hypnosis or psychological therapies of certain conditions (e.g. tic disorders). The detector device <b>30</b> may produce a stimulus, e.g. activating a light or speaker, and monitor the user's eyelid movement in anticipation of receiving a response, e.g., a specific sequence of blinks, acknowledging the stimulus within a predetermined time. If the user fails to respond, the processor may store the response, e.g. including response time, and/or may automatically transmit a signal, such as an alarm signal.

In addition, the detection device <b>30</b> may be used to monitor individuals in non-medical settings, such as during normal activity in a user's home or elsewhere. For example, individuals with involuntary medical conditions, such as epilepsy or narcolepsy, may be monitored, or other individuals, such as, infants and children, prison inmates, demented patients (e.g., with Alzheimer's disease), law enforcement personnel, military personnel, bank tellers, cashiers, casino workers, students, swing or graveyard shift workers, and the like, may be monitored. Similar application may be applied in a sleep laboratory for monitoring sleep patients to measure parameters, such as onset of sleep, sleep latency, time of eyelid closing or opening, time of awakening during the night, etc., or to animal research where eye blinking may be a factor to be studied. Similarly, the performance and vigilance abilities of the user may be tested and analyzed as a direct function of, or in relationship to, PERCLOS.

When the CPU <b>140</b> detects the presence of particular eyelid movement, such as an extensive period of eye closure which may occur, for example, during an epileptic seizure, a syncopal episode, a narcoleptic episode, or when dozing off while driving or working, the CPU <b>140</b> may produce an output signal which activates an alarm. Alternatively, the transmitter <b>152</b> may send an output signal to shut off equipment being used, to notify medical personnel, such as by automatically activating a telephone to dial emergency services, to signal remote sites, such as police stations, ambulances, vehicle control centers, guardians, and the like.

The system for communication <b>14</b> may also find useful application for voluntary communication. A user wearing the detection device <b>30</b> may intentionally blink in a predetermined pattern, for example, in Morse code or other blinked code, to communicate an understandable message to people or equipment (e.g., to announce an emergency). The CPU <b>140</b> may convert a light intensity signal <b>142</b> received from the sensor <b>33</b> and corresponding to the blinked code into a stream of data <b>144</b>, or possibly directly into an understandable message including letters, words and/or commands.

The stream of data <b>144</b> may then be displayed on a video display <b>50</b> (see <figref idref="DRAWINGS">FIG. 1</figref>) coupled to the output port <b>148</b>, or emitted as synthesized speech on the internal speaker <b>150</b>. The stream of data <b>144</b> may be transmitted by the transmitter <b>152</b> via the signal <b>153</b> to the receiving and processing unit <b>154</b> for displaying messages, or for controlling equipment, such as household devices, connected to the control system <b>164</b>. In addition to residential settings, the system <b>14</b> may be used by individuals in hospitalized or nursing care, for example by intubated, ventilated, restrained, paralyzed or weakened patients, to communicate to attending medical staff and/or to consciously signal a nurse's station. These include all patients who have no physical ability to communicate verbally, but who retain ability to communicate using eye blinking of one or both eyes (e.g., patients with amyotrophic lateral sclerosis, transverse myelitis, locked-in syndrome, cerebravascular strokes, terminal muscular dystrophy and those intubated on ventilation).

The device may be used in any environment or domain, e.g. through water or other substantially transparent fluids. Further, the device <b>30</b> may also be used as an emergency notification and/or discrete security tool. A person who may be capable of normal speech may wear the device <b>30</b> in the event of circumstances under which normal communication, i.e., speech, is not a viable option. For example, a bank or retail employee who is being robbed or is otherwise present during the commission of a crime may be able to discretely blink out a preprogrammed warning to notify security or to call law enforcement. Alternatively, a person with certain medical conditions may wear the device in the event that they are physically incapacitated, i.e., are unable to move to call for emergency medical care, but are still able to voluntarily move their eyes. In such cases, a pre-recorded message or identifying data (e.g. name of the user, their location, the nature of the emergency, etc.) may be transmitted to a remote location by a specific set of eyeblink codes or preprogrammed message. In this manner, the detection device <b>30</b> may be used to monitor patients in an ICU setting, patients on ventilators, prisoners, elderly or disabled persons, heavy equipment operators, truck drivers, motorists, ship and aircraft pilots, train engineers, radar or airport control tower operators, or as a nonverbal or subliminal tool for communication by military guards, police bank tellers, cashiers, taxi-drivers, and the like. The detection device <b>30</b> may also be used as a recreational device, for example, as a children's toy similar to a walkie-talkie or to operate a remote control toy vehicle.

In addition, it may be desirable to have the CPU <b>140</b> perform an additional threshold comparison to ensure continued use of the detection device <b>30</b>. For example, additional timer circuitry may be coupled to the CPU <b>140</b> such that the CPU <b>140</b> may compare the light intensity signals received from the sensor <b>33</b> to a second predetermined threshold provided by the timer circuitry. Preferably, the second predetermined threshold corresponds to a time period during which a person would normally blink. If the CPU <b>140</b> fails to detect a normal blink within this time period or if the user fails to respond to a predetermined stimulus (e.g. a blinking light or sound), the CPU <b>140</b> may produce a signal, activating the speaker <b>150</b> or transmitting a warning using the transmitter <b>152</b>.

This may be useful, if, for example, the detection device <b>30</b> is removed by a perpetrator during commission of a crime, falls off because of the onset of a medical episode, as well as to prevent “false alarms,” or to measure the “state of attentiveness” of the user. Alternatively, performance vigilance tasks may be required of the user to determine whether the signal transmitted is a purposeful or “false alarm” signal, and also for measuring attention or drowsiness levels for purposes of biofeedback, and also to measure compliance of the user wearing the device.

Alternatively, the polarity of the output signal <b>142</b> may be reversed such that a stream of data is produced only when the eye is opened, for example, when monitoring patients in a sleep lab to measure onset of sleep, sleep latency, time of eyelid closure, etc., or to monitor sleeping prison inmates. For such uses, the CPU <b>140</b> may activate an alarm only when an open eye condition is detected, as will be appreciated by those skilled in the art.

Turning to <figref idref="DRAWINGS">FIG. 8</figref>, another preferred embodiment of the detection device <b>30</b> in accordance with the present invention is shown. In this embodiment, the emitter and sensor are a single solid state light emission and detecting biosensor device <b>132</b> which are mounted directly onto the eyeglasses <b>20</b>. The biosensor device <b>132</b>, which preferably produces and detects infrared light, may be as small as 2 mm×4 mm and weigh only a few grams, thereby enhancing the convenience, comfort and/or discretion of the detection device <b>30</b>. Because of the small size, the biosensor device <b>133</b> may be mounted directly in the lens <b>21</b>, as shown in <figref idref="DRAWINGS">FIG. 8</figref>, on an outside or inside surface of the lens <b>21</b>, in the bridgework <b>24</b> or at another location on the frame <b>22</b> that may facilitate detection of eye movement. The biosensor device <b>132</b> may measure less than about five millimeters by five millimeters surface area, and may weigh as little as about one ounce, thereby providing a emitter/sensor combination that may be unobtrusive to vision, portable, and may be conveniently incorporated into a light weight eye frame. Because the entire system may be self-contained on the frame, it moves with the user no matter which direction he or she looks and may operate in a variety of environments or domains, day or night, underwater, etc.

Hamamatsu manufactures a variety of infrared emitter and detector devices which may be used for the biosensor device <b>132</b>, such as Model Nos. L1909, L1915-01, L2791-02, L2792-02, L2959, and 5482-11, or alternatively, a Radio Shack infrared emitter, Model No. 274-142, may be used. Multiple element arrays, e.g., linear optical scanning sensor arrays, appropriate for use with the present invention may be available from Texas Advanced Optoelectronic Solutions, Inc. (TAOS) of Plano, Tex., such as Model Nos. TSL 201 (64 pixels×1 pixel), TSL 202 (128×1), TSL 208 (512×1), TSL 2301 (102×1). These sensors may be used in combination with lens arrays to facilitate focusing of the detected light, such as the Selfoc lens array for line scanning applications made by NSG America, Inc. of Irvine, Calif.

In addition, multiple biosensor devices <b>132</b> may be provided on the eyeglasses <b>20</b>, for example, a pair of biosensor devices <b>132</b> may be provided, as shown in <figref idref="DRAWINGS">FIG. 8</figref>, for detecting eyelid movement of each eye of the user (not shown). A cable <b>134</b> extends from each biosensor device <b>132</b> to a processing box <b>130</b>, similar to the processing box <b>130</b> described above. The CPU <b>140</b> of the processing box <b>130</b> (not shown in <figref idref="DRAWINGS">FIG. 8</figref>) may receive and compare the output signal from each biosensor device <b>132</b> to further augment distinguishing normal blinks from other eyelid movement.

The pair of biosensor devices <b>132</b> may allow use of more sophisticated codes by the user, e.g., blinking each eye individually or together, for communicating more effectively or conveniently, as will be appreciated by those skilled in the art. In one form, a blink of one eye could correspond to a “dot,” and the other eye to a “dash” to facilitate use of Morse code. The output signals from each eye could then be interpreted by the CPU <b>140</b> and converted into an understandable message.

In another form, a right eye blink (or series of blinks) may cause an electric wheelchair to move to the right, a left eye blink (or series of blinks) may move to the left, two simultaneous right and left eye blinks may cause the wheelchair to move forward, and/or four simultaneous right and left eye blinks may cause the wheelchair to move backward. Similar combinations or sequences of eye blinks may be used to control the on/off function, or volume or channel control of a television, AM/FM radio, VCR, tape recorder or other electronic or electromechanical device, any augmentative communications or controlling device, or any device operable by simple “on/off” switches (e.g., wireless television remote controls single switch television control units, universal remote controllers, single switch multi-appliance units with AC plug/wall outlet or wall switch modules, computer input adapters, lighted signaling buzzer or vibrating signal boxes, switch modules of all types, video game entertainment controller switch modules and switch-controlled electronic toys).

In additional alternatives, one or more lenses or filters may be provided for controlling the light emitted and/or detected by the biosensor device, an individual emitter and/or detector. For example, the angle of the light emitted may be changed with a prism or other lens, or the light may be columnated or focused through a slit to create a predetermined shaped beam of light directed at the eye or to receive the reflected light by the sensor. An array of lenses may be provided that are adjustable to control the shape, e.g. the width, etc., of the beam of light emitted or to adjust the sensitivity of the sensor. The lenses may be encased along with the emitter in plastic and the like, or provided as a separate attachment, as will be appreciated by those skilled in the art.

Turning now to <figref idref="DRAWINGS">FIG. 10A</figref>, another preferred embodiment of a system for communication <b>414</b> is shown, that includes a frame <b>422</b> including a biosensor device <b>432</b> with associated processor and transmitter circuitry <b>430</b> provided directly on the frame <b>422</b>, for example, to enhance the convenience and discretion of the system for communication <b>414</b>. The frame <b>422</b> may include a bridge piece <b>424</b> onto which the biosensor device <b>432</b> may be slidably and/or adjustably mounted, and a pair of ear supports <b>423</b>, <b>425</b>.

One of the supports <b>423</b> may have a larger size compared to the other support <b>425</b>, for example, to receive the processor and transmitter circuitry <b>430</b> embedded or otherwise mounted thereon. A processor <b>440</b>, similar to the CPU <b>140</b> in the processing box <b>130</b> previously described, may be provided on the frame <b>422</b>, and a power source, such as a lithium battery <b>460</b>, may be inserted or affixed to the support <b>423</b>. A radio frequency or other transmitter <b>452</b> is provided on the support <b>423</b>, including an antenna <b>453</b>, which may be embedded or otherwise fastened along the ear support <b>423</b>, in the temple piece or elsewhere in the frame <b>422</b>.

The system <b>414</b> may also include manual controls (not shown) on the ear support <b>423</b> or elsewhere on the frame <b>422</b>, for example to turn the power off and on, or to adjust the intensity and/or threshold of the biosensor device <b>432</b>. Thus, a system for communication <b>414</b> may be provided that is substantially self-contained on the frame <b>422</b>, which may or may not include lenses (not shown) similar to eyeglasses. External cables or wires may be eliminated, thereby providing a more convenient and comfortable system for communication.

In another alternative, shown in <figref idref="DRAWINGS">FIGS. 10B</figref>, <b>10</b>C, and <b>10</b>D, a linear array <b>530</b> of emitters <b>532</b> and sensors <b>533</b> may be provided, preferably in a vertical arrangement mounted on a nose bridge <b>524</b> of an eye frame <b>522</b>. A CPU <b>540</b>, battery <b>460</b>, transmitter antenna <b>543</b>, and warning indicator <b>550</b> may also be provided on the frame <b>522</b>, preferably in the temple piece <b>525</b>, similar to the previously described embodiment. An LED <b>542</b> or similar stimulus device may also be provided at a predetermined location on the eye frame <b>522</b> to allow routine biofeedback responses from the user. In addition, a receiver <b>544</b> may be provided for receiving the stream of data created by the CPU <b>540</b> and transmitted by the transmitter <b>543</b>.

As shown particularly in <figref idref="DRAWINGS">FIG. 10C</figref>, each of the sensors <b>533</b> and the emitter <b>532</b> are coupled to the CPU <b>540</b> or other control circuitry for controlling the emitter <b>532</b> and for processing the light intensity signals produced by the sensors <b>532</b>. Thus, the CPU <b>540</b> may cycle through the sensors <b>533</b> in the array <b>530</b> and sequentially process the signal from each of the sensors <b>533</b>, similar to the processors previously described. More preferably, as shown in <figref idref="DRAWINGS">FIG. 10D</figref>, the emitter <b>532</b> includes a lens <b>534</b> to focus a beam of light (indicated by individual rays <b>360</b>a, <b>360</b>b) onto the eye <b>300</b>, preferably towards the pupil <b>301</b>. The sensors <b>533</b> are embedded within the nose bridge <b>524</b> and a slit <b>535</b> is provided for each, the slits <b>535</b> having a predetermined size to control the reflected light detected by each sensor <b>533</b>. Thus, each sensor <b>535</b> may detect movement of the eyelid <b>302</b> past a particular portion of the eye <b>300</b>, e.g., to measure PERCLOS, as shown in FIG. <b>12</b>A. The sensors or emitters may have lenses or columnating devices to focus emitted or reflected light.

The linear array <b>530</b> may facilitate measurement of additional parameters related to eyelid movement in addition to mere eye closure. For example, to measure the velocity of the eyelid opening or closing, i.e., the rate of eye closure, the CPU <b>540</b> may compare the time delay between the activation of successive sensors <b>533</b>. In addition, the output signals from the sensors <b>553</b> may be processed to measure the percentage of pupil coverage of the eyelid <b>302</b>, for example, due to partial eye closure, as a function of time, e.g., to monitor when the eye is partially, but not completely, closed, and/or to monitor the percentage of time that the eye is closed (PERCLOS), as shown in <figref idref="DRAWINGS">FIGS. 12A-12C</figref>, e.g., compared to the user's baseline of maximal eye opening.

Turning to <figref idref="DRAWINGS">FIG. 12D</figref>, in a further alternative, a two-dimensional array of sensors, such as a 5×5 array <b>633</b> or a 9×11 array, <b>733</b> may be provided. Other arrays including any number of elements in the array may be provided, and the invention should not be limited to the exemplary embodiments described herein. The sensors <b>633</b>, <b>733</b> may then be used to measure surface area reflectivity of light from the emitter <b>632</b>, i.e., the processor (not shown) may process the signals from each sensor in the array <b>633</b>, <b>733</b> to create a stream of data indicating the percentage of surface area of the eye <b>300</b> covered by the eyelid <b>302</b>.

The sensors in the array <b>633</b>, <b>733</b> may be sufficiently sensitive or have sufficient resolution such that they may detect “red reflex” or the equivalent infrared “bright pupil” reflection due to the reflection of light off of the retina through the pupil <b>301</b>. Thus, the sensors may produce a light intensity signal that includes a substantially zero value, indicating no red reflex or bright pupil, a low output, indicating red reflex or white pupil reflex, and a high output, indicating reflection off of a closed eyelid <b>302</b>. The red reflex may appear as a bright white light pupil (resulting from infrared light from the emitter(s) reflecting off of the retina when the eyelid is open, or as a dark or “black pupil” if the processor uses subtraction algorithms, as is known in the art. The processor may thereby process the light intensity signals to detect when the pupil <b>301</b> is covered by the eyelid <b>302</b>, i.e., at which point the user cannot see, even though their eye <b>300</b> may not be entirely covered by the eyelid <b>302</b>, generally at a PERCLOS value of about 50-75 percent in primary gaze. Alternatively, as the eyelid, eye, and pupil descend, the sensor(s) may detect a red reflex or bright pupil even through the PERCLOS measurement may be as great as 75 -80 percent or more, e.g., where the eye may still see through a narrow slit-like palpebral fissure opening in downward gaze.

In another alternative, the processor and/or transmitter circuitry (such as the CPU <b>140</b> in the processor box <b>130</b> of <figref idref="DRAWINGS">FIG. 2</figref>, or the CPU's <b>440</b>, <b>540</b> of <figref idref="DRAWINGS">FIGS. 10A and 10B</figref>) may include identification circuitry (not shown), either as a discrete memory chip or other circuit element, or within the CPU itself. The identification circuitry may be preprogrammed with a fixed identification code, or may be programmable, for example, to include selected identification information, such as the identity of the user, the user's location, an identification code for the individual detection device, and the like.

The CPU may selectively add the identification information to the transmitted stream of data <b>553</b>, or the identification information may be automatically or periodically included in the stream of data <b>553</b>, thereby allowing the stream of data <b>553</b> to be associated with a particular detection device, individual user and/or a specific location. The identification information may be used by the processor, for example, at a remote location, to distinguish between streams of data received from a number of detection devices, which may then be stored, displayed, etc. as previously described. Thus, the detection device may not require users to consciously communicate certain identification or other standard information when the system is used.

As shown in <figref idref="DRAWINGS">FIG. 11A</figref>, the receiver <b>544</b> may allow the user to control one or more devices coupled to the receiver <b>544</b> through a single switch multi-appliance control unit <b>550</b>. The control unit <b>550</b> includes its own transmitter adapted to transmit on/off or other control signals that may be received by individual control modules <b>552</b>a-<b>552</b>fe. The user <b>10</b> may blink to create a transmitted stream of data <b>553</b> that includes commands to turn off and on, or otherwise control, selected appliances using the control unit <b>550</b> and control modules <b>552</b>a-<b>552</b>fe, such as, a radio <b>554</b>, a television <b>556</b>, a light <b>558</b>a., a light <b>562</b> controlled by a wall switch <b>560</b>, a fan <b>566</b> plugged into a wall socket <b>564</b>, and the like.

Alternatively, as shown in <figref idref="DRAWINGS">FIG. 11B</figref>, the receiver <b>554</b> may be coupled to other systems, such as a computer <b>570</b> and printer <b>572</b>, a vehicle integration system <b>574</b>, a lifeline unit <b>576</b>, a GPS or other satellite transmitter <b>578</b>, and the like. The transmitted stream of data <b>553</b> may be processed alone or along with additional data, such as other vehicle sensor information <b>573</b>, to further enhance monitoring a user, such as a long-distance truck driver.

Turning to <figref idref="DRAWINGS">FIG. 13</figref>, yet another embodiment of a system <b>810</b> for monitoring eye movement is shown. Generally, the system <b>810</b> includes a frame <b>812</b> that may include a bridge piece <b>814</b> and a pair of car supports <b>816</b>. The frame <b>812</b> may include a pair of lenses (not shown), such as prescription, shaded, or protective lenses, although they are not necessary for operation of the invention. Alternatively, the system may be provided on other devices that may be worn on a user's head, such as a pilot's oxygen mask, protective eye gear, a patient's ventilator, a scuba or swimming mask, a helmet, a hat, a head band, a head visor, and the like (not shown). The components of the system may be provided at a variety of locations on the device that generally minimize interference with the user's vision and/or normal use of the device.

An array of emitters <b>820</b> are provided on the frame <b>812</b>, preferably in a vertical array <b>820</b>a and a horizontal array <b>820</b>b. In a preferred embodiment, the emitters <b>820</b> are infrared emitters configured to emit pulses at a predetermined frequency, similar to the embodiments described above. The emitters <b>820</b> are arranged on the frame such that they project a reference frame <b>850</b> onto the region of the user's eye. In a preferred embodiment, the reference frame includes a pair of crossed bands <b>850</b>a, <b>850</b>b dividing the region into four quadrants. The intersection of the crossed bands is preferably disposed at a location corresponding substantially to the eye's pupil during primary gaze, i.e., when the user is looking generally straight forward. Alternatively, other reference frames may be provided, generally including a vertical component and a horizontal component.

An array of sensors <b>822</b> are also provided on the frame <b>812</b> for detecting light from the emitters <b>820</b> that is reflected off of the user's eyelid. The sensors <b>822</b> preferably generate output signals having an intensity identifying whether the eyelid is closed or open, similar to the embodiments described above. Preferably, the sensors <b>822</b> are disposed adjacent to respective emitters <b>820</b> for detecting light reflected off of respective portions of the eyelid. Alternatively, sensors <b>822</b> may only be provided in a vertical array, e.g., along the bridge piece <b>814</b>, for monitoring the amount of eyelid closure, similar to the embodiments described above. In a further alternative, the emitters <b>820</b> and sensors <b>822</b> may be solid state biosensors (not shown) that provide both the emitting and sensing functions in a single device.

Circuitry may be provided for measuring PERCLOS or other parameters using the signals generated by the array of sensors. For example, <figref idref="DRAWINGS">FIG. 17</figref> shows an exemplary schematic that may be used for processing signals from a five element array, e.g., to obtain PERCLOS measurements or other alertness parameters.

Returning to <figref idref="DRAWINGS">FIG. 13</figref>, the system <b>810</b> also includes a camera <b>830</b> provided on the frame <b>810</b><b>812</b>. Preferably, the camera <b>830</b> is mounted on or adjacent the bridge piece <b>814</b> such that the camera <b>830</b> is oriented towards the region surrounding one of the user's eyes while minimizing interference with the user's vision. This camera <b>830</b> preferably includes a bundle of fiberoptic cables <b>832</b> that terminate in a lens <b>834</b>, as shown in <figref idref="DRAWINGS">FIG. 14</figref>, on a first end mounted adjacent the bridge piece <b>814</b> and a second end <b>837</b> that is connected to a detector <b>838</b>, e.g., a CCD or CMOS sensor, such as those used in endoscopes, that may convert an image into a digital video signal. The camera <b>830</b> is configured to detect the frequency of light emitted by the emitters <b>820</b>, e.g., infrared light. The camera <b>830</b> may rely on the light projected by the emitters <b>820</b>, or the fiberoptic cables <b>832</b> may include emitters <b>836</b> for projecting light, e.g., infrared light, onto the user's eyes and/or face. In addition, the system <b>810</b> may include a second camera <b>840</b> oriented away from the user's head, e.g., to monitor the user's surroundings.

One of the ear supports <b>816</b> may include a panel <b>818</b> for mounting a controller or other processor <b>842</b>, a transmitter <b>844</b>, an antenna <b>845</b>, and a battery <b>846</b>. Preferably, the processor <b>840</b><b>842</b>is coupled to the emitters <b>820</b>, the sensors <b>822</b>, and/or the camera <b>830</b> for controlling their operation. The transmitter <b>844</b> may be coupled to the processor <b>842</b> for receiving the output signals from the sensors <b>822</b> and/or the video signals from the camera <b>830</b>, e.g., to transmit the signals to a remote location, as described below. Alternatively, the transmitter <b>844</b> may be coupled directly to output leads from the sensors <b>822</b> and the camera <b>830</b>. The frame <b>812</b> may also include manual controls (not shown), e.g., on the ear support <b>816</b>, for example, to turn the power off and on, or to adjust the intensity and/or threshold of the emitters <b>820</b>, the sensors <b>822</b>, and/or the camera <b>830</b>.

If desired, the system <b>810</b> may also include one or more additional sensors on the frame <b>812</b>. The sensors may be coupled to the processor <b>842</b> and/or to the transmitter <b>844</b> so that the signals from the sensors may be monitored, recorded, and/or transmitted to a remote location. For example, one or more position sensors <b>852</b>a, <b>852</b>b may be provided, e.g., for determining the spatial orientation of the frame <b>812</b>, and consequently the user's head. For example, actigraphic sensors may be provided to measure tilt or movement of the head, e.g., to monitor whether the user's head is drooping forward or tilting to the side. Acoustic sensors, e.g., a microphone <b>854</b> may be provided for detecting environmental noise or sounds produced by the user.

In addition or alternatively, the frame <b>812</b> may include one or more sensors for measuring one or more physical characteristics of the user. For example, EEG electrodes <b>856</b> may be provided on the ear support <b>816</b>, above or below the nasion, and/or other region that may contact the patient's skin to measure brain activity, e.g., waking, drowsy, or other sleep-related brain activity. An EKG electrode (not shown) may be provided that is capable of measuring cardiac activity through a skin contact site. A pulse sensor (not shown) may be used to measure cardiovascular pulsations, or an oximetry sensor <b>858</b> may be used to measure oxygen saturation levels. A thermistor or other sensor may measure of respiratory air flow, e.g., through the user's nose. A thermistor, thermocouple, or other temperature sensor (not shown) may be provided for measuring the user's skin temperature. A sweat detector (not shown) may be provided for measuring moisture on the user's skin.

In addition, the system <b>810</b> may include one or more feedback devices on the frame <b>812</b>. These devices may provide feedback to the user, e.g., to alert and/or wake the user, when a predetermined condition is detected, e.g., a state of drowsiness or lack of consciousness. The feedback devices may be coupled to the processor <b>842</b>, which may control their activation. For example, a mechanical vibrator device <b>860</b> may be provided at a location that may contact the user, e.g., on the ear support <b>816</b>, that may provide tactile vibrating stimuli through skin contact. An electrode (not shown) may be provided that may produce relatively low power electrical stimuli. A light emitter, such as one or more LED's may provided at desired locations, e.g., above the bridge piece <b>814</b>. Alternatively, audio devices <b>862</b>, such as a buzzer or other alarm, may be provided, similar to the previous embodiments. In a further alternative, aroma-emitters may be provided on the frame <b>810</b><b>812</b>, e.g., on or adjacent in the bridge piece <b>814</b>.

Alternatively, the feedback devices may be provided separate from the frame, but located in a manner capable of providing a feedback response to the user. For example, audio, visual, tactile (e.g., vibrating seat), or olfactory emitters may be provided in the proximity of the user, such as any of the devices described above. In a further alternative, heat or cold generating devices may be provided that are capable of producing thermal stimuli to the user, e.g., a remotely controlled fan or air conditioning unit.

The system <b>810</b> may also include components that are remote from the frame <b>812</b>, similar to the embodiments described above. For example, the system <b>810</b> may include a receiver, a processor, and/or a display (not shown) at a remote location from the frame <b>812</b>, e.g., in the same room, at a nearby monitoring station, or at a more distant location. The receiver may receive signals transmitted by the transmitter <b>842</b>, including output signals from the sensors <b>822</b> or any of the other sensors provided on the frame <b>812</b> and/or the video signals from the camera <b>830</b>.

A processor may be coupled to the receiver for analyzing signals from the components on the frame <b>812</b>, e.g., to prepare the signals for graphical display. For example, the processor may prepare the video signals from the camera <b>830</b> for display on a monitor, thereby allowing personal monitoring of the user. Simultaneously, other parameters may be displayed, either on a single monitor or on separate displays. For example, <figref idref="DRAWINGS">FIG. 15a</figref><figref idref="DRAWINGS">FIGS. 15A-15I</figref>shows signals indicating the output of various sensors that may be on the frame <b>812</b>, which may be displayed along a common time axis or otherwise correlated, e.g., to movement of the user's eye and/or level of drowsiness. The processor may superimpose or otherwise simultaneously display the video signal in conjunction with the other sensed parameters to allow a physician or other individual to monitor and personally correlate these parameters to the user's behavior.

In a further alternative, the processor may automatically process the signals to monitor or study the user's behavior. For example, the processor may use the output signals to monitor various parameters related to eye movement, such as eye blink duration (EBD), eye blink frequency, eye blink velocity, eye blink acceleration, interblink duration (IBD), PERCLOS, PEROP (percentage eyelid is open), and the like.

The video signals from the camera <b>830</b> may be processed to monitor various eye parameters, such as pupillary size, location, e.g., within the four quadrant defined by the crossed bands <b>850</b>, eye tracking movement, eye gaze distance, and the like. For example, because the camera <b>830</b> is capable of detecting the light emitted by the emitters <b>822</b>, the camera <b>830</b> may detect a reference frame projected onto the region of the user's eye by the emitters. <figref idref="DRAWINGS">FIG. 16</figref> shows an exemplary video output from a camera included in a system having twenty emitters disposed in a vertical arrangement. The camera may detect twenty discrete regions of light arranged as a vertical band. The camera may also detect a “glint” point, G, and/or a moving bright pupil, P. Thus, the movement of the pupil may be monitored in relation to the glint point, G, and/or in relation to the vertical band <b>1</b>-<b>20</b>.

Because the emitters <b>822</b> are fixed to the frame <b>812</b>, the reference frame <b>850</b> remains substantially stationary. Thus, the processor may determine the location of the pupil in terms of orthogonal coordinates (e.g., x-y or angle-radius) relative to the reference frame <b>850</b>. Alternatively, if the reference frame is eliminated, the location of the pupil may be determined relative to any stationary “glint” point on the user's eye. For example, the camera <b>830</b> itself may project a point of light onto the eye that may be reflected and detected by the camera. This “glint” point remains substantially stationary since the camera <b>830</b> is fixed to the frame <b>812</b>.

In addition, the video signals from a remote camera that may view the user's face from a distance may be used to monitor various facial measures, such as facial expression, yawning frequency, and the like, in addition to or alternatively, the projectinstead of the projected light reference frame from the emitters. In addition or alternatively, the parameters from other sensors may be processed and correlated, such as head orientation, tilt, body movement, physiological parameters, and the like. Preferably, the processor may correlate these parameters to generate a composite fatigue index (CFI) that is a function of two or more of these parameters. When a predetermined CFI is detected, the system <b>810</b> may activate an alarm or other notice to the user and/or to another party at a remote location. Thus, the system <b>810</b> may provide a more effective way to monitor the user's fatigue, drowsiness, alertness, mental state, and the like. In a further alternative, the system <b>810</b> may be used to generate predetermined outputs, e.g., to activate or deactivate equipment, such as a vehicle being operated by the user when a predetermined condition, e.g., CFI value, is determined by the system <b>810</b>.

Alternatively, the processor may be provided on the frame <b>812</b>, e.g. as part of processor <b>842</b>, for monitoring the parameters for a predetermined event, such as a predetermined CFI value, to occur. Although only a single lens and set of emitters, sensors, and cameras are shown, it will be appreciated that another set may be provided for the other eye of the user of the system <b>810</b>. In a further alternative, the eye tracking parameters described above may be monitored by a remote camera, e.g., in a fixed position in front of the user, such as the dashboard of a vehicle and the like. The remote camera may be coupled to the processor, either directly or via its own transmitter, as will be appreciated by those skilled in the art.

Thus, a system in accordance with the present invention may monitor or detect one or more parameters, such as those listed below in Table 1.

<tables id="TABLE-US-00001" num="00001"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="1"><colspec colname="1" colwidth="217pt" align="center" /><thead><row><entry namest="1" nameend="1" rowsep="1">TABLE 1</entry></row><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row><row><entry>Potential Biometric Measures</entry></row><row><entry namest="1" nameend="1" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /></row></tbody></tgroup><tgroup align="left" colsep="0" rowsep="0" cols="2"><colspec colname="offset" colwidth="42pt" align="left" /><colspec colname="1" colwidth="175pt" align="left" /><tbody valign="top"><row><entry /><entry>EYELID MEASURES</entry></row><row><entry /><entry>Percentage of time (t) and the amount palpebral</entry></row><row><entry /><entry>fissure is opened (PEROP-t, -d, -dt), or closed</entry></row><row><entry /><entry>(PERCLOS-t, -d, -dt),</entry></row><row><entry /><entry>lid droop</entry></row><row><entry /><entry>Eye Blink Duration (EBD)</entry></row><row><entry /><entry>Eye Blink Frequency (EBF)</entry></row><row><entry /><entry>Eye Blink Velocity (EBV)</entry></row><row><entry /><entry>Eye Blink Accelaration (EBAc)</entry></row><row><entry /><entry>Deceleration (EBDc)</entry></row><row><entry /><entry>Interblink duration (IBD)</entry></row><row><entry /><entry>Eye blink flurries</entry></row><row><entry /><entry>PUPIL MEASURES</entry></row><row><entry /><entry>Pupillary Appearance or Disappearance (with</entry></row><row><entry /><entry>eyelid movement)</entry></row><row><entry /><entry>Pupillary Size Measurement (PSM)</entry></row><row><entry /><entry>Presence and quality of Pupillary</entry></row><row><entry /><entry>Dilation or Construction</entry></row><row><entry /><entry>(including Hippus)</entry></row><row><entry /><entry>EYEGLAZE MEASURES</entry></row><row><entry /><entry>Eye Tracking Movements (ETM)</entry></row><row><entry /><entry>including Directional Nystagmus</entry></row><row><entry /><entry>Eye Gaze Distance (EGD) and Direction</entry></row><row><entry /><entry>Eye Movement Distance</entry></row><row><entry /><entry>Eye Movement Velocity (EMV)</entry></row><row><entry /><entry>Eye Movement Acceleration (EMA)</entry></row><row><entry /><entry>and Deceleration (EMD)</entry></row><row><entry /><entry>Eye Movement Frequency (EMF)</entry></row><row><entry /><entry>Phoria/eye Drill Measures (PDM)</entry></row><row><entry /><entry>HEAD ORIENTATION MEASURES</entry></row><row><entry /><entry>Head Direction or Orientation (HDir)</entry></row><row><entry /><entry>HEAD MOVEMENT MEASURES</entry></row><row><entry /><entry>Head Nodding Frequency (HNF)</entry></row><row><entry /><entry>Head Tilt (HT)</entry></row><row><entry /><entry>OTHER NON-VIDEO SENSOR METRICS</entry></row><row><entry /><entry>EEG, EKG, pulse, oxygen saturation,</entry></row><row><entry /><entry>respiration rate, body temp, skin</entry></row><row><entry /><entry>conductance, actigraphic movements, head</entry></row><row><entry /><entry>tilt sensors</entry></row><row><entry /><entry namest="offset" nameend="1" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

While the invention is susceptible to various modifications, and alternative forms, specific examples thereof have been shown in the drawings and are herein described in detail. It should be understood, however, that the invention is not to be limited to the particular forms or methods disclosed, but to the contrary, the invention is to cover all modifications, equivalents and alternatives falling within the spirit and scope of the appended claims.