Sleepiness level determination device for driver

CROSS REFERENCE TO RELATED APPLICATION

This application is based on Japanese Patent Application No. 2007-265764 filed on Oct. 11, 2007, the disclosure of which is incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to a sleepiness level determination device for a driver of a vehicle.

BACKGROUND OF THE INVENTION

A method for estimating sleepiness level of a driver of a vehicle based on low awake state is proposed to prevent the driver from dozing off at the wheel. For example, the sleepiness level is detected by detecting movement of an eye of the driver such as blink or eyelid's movement. Firstly, the movement of the eye is detected when the driver clearly awakes, for example, when the driver drives the vehicle in the early stages of the driving. The movement of the eye is compared with the early stages, so that the sleepiness level is detected.

The estimation of the sleepiness level may deviate from proper value because of individual variation and detection error. Therefore, an arousal information display device disclosed in JP-A-2006-174960 calculates detection reliability of image processing based on reliability of estimation of awake level (i.e., arousal level). Both of the detection reliability and the arousal level are presented to the driver. Thus, the driver can recognize that the arousal level is not proper when an improper arousal level is presented.

However, the above arousal information display device detects the reliability by obtaining a detection time ratio corresponding to a time for detecting the driver's eye image from an image. Therefore, the performance of the image processor and the influence of disturbance of outside light may affect the calculation of the reliability. Thus, the reliability may depend on a factor other than detection of the arousal level. Accordingly, the reliability of estimation of the arousal level may be not proper.

SUMMARY OF THE INVENTION

In view of the above-described problem, it is an object of the present disclosure to provide a sleepiness level determination device for a driver of a vehicle.

According to an aspect of the present disclosure, a sleepiness level determination device includes: a detector processing a face image of an user and for detecting an eye image of the user based on the face image; a characteristic value calculating unit calculating a characteristic value regarding the eye based on the eye image; a sleepiness level determining unit determining a sleepiness level based on the characteristic value; and a reliability calculating unit calculating reliability of the sleepiness level based on the characteristic value.

In the above device, the reliability of the sleepiness level is determined based on a statistics of the characteristic value of the eye. Thus, the reliability is calculated based on determination of the sleepiness level itself. Thus, the reliability is determined with high accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description made with reference to the accompanying drawings. In the drawings:

<figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram showing a sleepiness level determination device;

<figref idrefs="DRAWINGS">FIG. 2</figref> is a schematic view showing a camera and a projector in a compartment of a vehicle;

<figref idrefs="DRAWINGS">FIG. 3</figref> is a schematic view showing a driver's sight of the camera and the projector;

<figref idrefs="DRAWINGS">FIG. 4</figref> is a block diagram showing a control circuit;

<figref idrefs="DRAWINGS">FIG. 5</figref> is a graph showing a relationship between time and a position of an eyelid;

<figref idrefs="DRAWINGS">FIGS. 6A to 6C</figref> are graphs showing frequency distribution;

<figref idrefs="DRAWINGS">FIG. 7</figref> is a diagram showing a relationship among sleepiness level, driver's condition and behavior; and

<figref idrefs="DRAWINGS">FIG. 8</figref> is a block diagram showing a control circuit according to a modification of an example embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A sleepiness level determination device according to an example embodiment of the present disclosure will be explained. The device detects a sleepiness level of a driver of a vehicle. As shown in <figref idrefs="DRAWINGS">FIG. 1</figref>, the device includes a camera <b>10</b>, a control circuit <b>20</b>, an alarm generator <b>30</b>, a projector control circuit <b>40</b> and a projector <b>50</b>.

The camera <b>10</b> takes a picture of a face image of the driver, as shown in <figref idrefs="DRAWINGS">FIG. 2</figref>. The camera <b>10</b> is arranged around an instrument panel of a compartment of the vehicle. The camera <b>10</b> includes an imager for shooting pictures with high speed. Thus, the camera <b>10</b> can shoot a picture with a shooting time of about 300 micro seconds per one frame and a shooting interval of about 30 micro seconds.

The control circuit <b>20</b> stores image information of the face image shot by the camera <b>10</b> in an image memory (not shown). The control circuit <b>20</b> reads out the image information from the image memory, so that the control circuit <b>20</b> executes an image processing for detecting an eye image of the driver from the face image. Further, the control circuit <b>20</b> executes a process for calculating characteristic value relating to the eye based on the eye image. The projector control circuit <b>40</b> adjusts irradiation of the projector <b>50</b>. The control circuit <b>20</b> controls a shooting timing of the camera <b>10</b> in synchronization with adjustment of the irradiation of the projector <b>50</b> by the projector control circuit <b>40</b>. Furthermore, the camera <b>10</b> adjusts exposure time of the camera <b>10</b> and an output gain of the camera <b>10</b>.

The control circuit <b>20</b> determines, i.e., estimates sleepiness level of the driver based on the characteristic value regarding the eye of the driver. The reliability of determination of the sleepiness level is calculated based on the statistics of the characteristic value of the eye. The control circuit <b>20</b> determines based on the reliability of the sleepiness level whether alarm for the driver in accordance with the sleepiness level is performed.

The alarm generator <b>30</b> includes an amplifier and a speaker arranged at an appropriate position of the compartment. The alarm generator <b>30</b> generates alarm such as alarm sound and alarm voice message according to an input signal from the control circuit <b>20</b>.

The projector control circuit <b>40</b> controls light amount, i.e., strength of light from the projector and turns on and off the projector, which is arranged in parallel to the camera <b>10</b>, as shown in <figref idrefs="DRAWINGS">FIG. 3</figref>. The projector control circuit <b>40</b> controls the irradiation state of the light to irradiate the light on the face of the driver in synchronization with the shoot timing of the camera <b>10</b>.

The projector <b>50</b> includes multiple light sources for emitting light such as infrared light and near-infrared light having center wavelength in a range between 850 nm and 950 nm. Multiple light sources are aligned along with a vertical direction of the face of the driver. Each light source emits the light toward the face of the driver.

The operation of the sleepiness level determination device will be explained. <figref idrefs="DRAWINGS">FIG. 4</figref> shows functions of the control circuit <b>20</b>. An image obtaining unit <b>201</b> obtains the image information of the face image shot by the camera <b>10</b>. An image processing unit <b>202</b> executes the image processing step to detect the image of the eye of the driver from the face image, which is obtained by the image obtaining unit <b>201</b>. An eye movement, i.e., the eye motion detection unit <b>203</b> detects a height of an eyelid for detecting an opening degree of the eye and a line of sight based on the eye image detected by the image processing unit <b>202</b>. The line of sight may be defined as a part of the compartment of the vehicle, at which the driver directs his eyes. For example, the part of the vehicle is a windshield, a rearview mirror, a side mirror and the like. Here, when the part of the vehicle is the windshield, the driver directs his eyes toward the front of the vehicle.

A detection error calculating unit <b>204</b> for detecting error of the image processing step. Specifically, the calculating unit <b>204</b> generates a flag in each frame of the eye image. The flag shows whether the eye motion detection unit <b>203</b> does not detects the motion of the eye in the frame because of behavior of the driver and the influence of the outside light. In view of the flag, it is determined whether the eye motion detection unit <b>203</b> detects the eye motion in the frame.

An eye characteristic value calculating unit <b>205</b> calculates characteristic values of each blink such as a period of time c<b>1</b> from the start to the end of one blink, a time interval c<b>2</b> of closing the eye, a closing speed c<b>3</b> of the eyelid and an opening speed c<b>4</b> of the eyelid. Further, the eye characteristic value calculating unit <b>205</b> calculates a position of the line of sight, the number of blinks per unit time, average of opening degree per unit time, average of closing degree per unit time, time interval while the line of sight ceases. Thus, different characteristic values c<b>1</b> to cn (n represents natural number, i.e., n=1, 2, . . . i) regarding the eye are calculated. When the flag of one frame represents yes, the characteristic values cannot be calculated in the one frame. Thus, the characteristic values of the one frame are obtained by complementing those of adjacent frames.

A reliability calculating unit <b>206</b> for eye characteristic value calculates the reliability of the characteristic value of the eye. Specifically, the reliability calculating unit <b>206</b> calculates a detection rate of each blink. In <figref idrefs="DRAWINGS">FIG. 5</figref>, the period of time c<b>1</b> from the start to the end of one blink is defined as one blink interval. The one blink interval includes multiple frames. A detection rate E is defined as a ratio between the number of frames having the flag of “yes” and the number of the whole frames in the one blink interval. The detection rate E is in a range between 0 and 1.

A sleepiness level determining unit <b>207</b> determines, i.e., estimates a sleepiness level D based on a regression equation f having multiple explaining variables of the characteristic values c<b>1</b> to cn. The regression equation f is an estimation function. The estimation function f is expressed as follows. <br /><i>D=f</i>(<i>c</i>1,<i>c</i>2, . . . ,<i>cn</i>)  (F1)

The estimation function f is obtained by setting the characteristic values c<b>1</b>, . . . , cn based on experimental results in view of a relationship between the sleepiness level D (=1, 2, 3, 4, or 5) and a corresponding behavior of the driver. In <figref idrefs="DRAWINGS">FIG. 7</figref>, the characteristics of the driver's behavior are defined in a corresponding sleepiness level. As the sleepiness level is higher, the driver feels sleepier. The sleepiness level D is determined based on multiple different characteristic values regarding the eye of the driver, so that robustness of determination of the sleepiness level becomes high.

A reliability calculating unit <b>208</b> for the sleepiness level calculates frequency distribution of all characteristic values c<b>1</b>, . . . , cn in each sleepiness level D when the sleepiness level determining unit <b>207</b> estimates the sleepiness level D. Further, the reliability calculating unit <b>208</b> determines a representative characteristic value x<b>1</b> to xn in each characteristic value c<b>1</b> to cn. The representative characteristic value x<b>1</b> to xn represents a respective frequency distribution. The representative characteristic value x<b>1</b> to cn is a main value of the distribution such as average of the distribution and a center value of the distribution.

The reliability calculating unit <b>208</b> calculates the reliability R of the sleepiness level D based on proximity between the representative characteristic values x<b>1</b> to xn and the characteristic values c<b>1</b> to cn. The representative characteristic values x<b>1</b> to xn in each sleepiness level D are preliminarily determined. The characteristic values c<b>1</b> to cn are calculated by the eye characteristic value calculating unit <b>205</b> when the sleepiness level determining unit <b>207</b> estimates the sleepiness level D.

Specifically, as shown in <figref idrefs="DRAWINGS">FIGS. 6A to 6C</figref>, for example, when the sleepiness level D (=f(c<b>1</b>, c<b>2</b>, . . . , cn)) is three, the proximity between the characteristic value c<b>1</b> to cn and the corresponding representative characteristic value x<b>1</b> to xn in the corresponding frequency distribution. In <figref idrefs="DRAWINGS">FIGS. 6A to 6C</figref>, square dots represents the detected characteristic values in the frequency distribution. The reliability R is calculated by the following formula F2 with considering the detection ratio E. <br />R={[<i>k</i>1×(proximity between <i>c</i>1 and <i>x</i>1)+[<i>k</i>2×(proximity between <i>c</i>2 and <i>x</i>2)+ . . . +[<i>kn</i>×(proximity between <i>cn </i>and <i>xn</i>)]}×<i>E</i>  (F2)

Here, k<b>1</b> to kn are determined in proportion to contribution ratios of the characteristic values c<b>1</b> to cn when the sleepiness level D is determined. The contribution ratio is a standard of the reliability of the regression equation when the sleepiness level D is estimated with the regression equation.

The proximity between the characteristic value c<b>1</b> to cn and the corresponding representative characteristic value x<b>1</b> to xn may be described with dispersion σ. For example, when the distribution of the characteristic values c<b>1</b> to cn is a normal distribution, and the characteristic value is disposed within ±1σ, the proximity (i.e., the weight) is set to “1.” When the characteristic value is disposed within ±2σ, the proximity is set to “½.” When the characteristic value is disposed within ±3σ, the proximity is set to “⅓.” When the characteristic value is not disposed within ±3σ, the proximity is set to “0.”

The reliability R of the sleepiness level D is calculated based on the proximity (i.e., plausibility) of each characteristic value c<b>1</b> to cn when the sleepiness level D is estimated, with regard to the representative characteristic values x<b>1</b> to xn for representing the distribution of the characteristic values c<b>1</b> to cn. Thus, the reliability R properly corresponds to the reliability of the sleepiness level D.

A reliability threshold determining unit <b>209</b> determines, i.e., specifies the characteristic value c<b>1</b> to cn having low proximity, which is lower than a proximity threshold. For example, the proximity threshold is “1,” and therefore, a predetermined characteristic value is disposed within ±1σ. The proximity threshold provides a standard as high reliability. When the characteristic values c<b>1</b> to cn includes the low reliability characteristic value having the proximity smaller than the proximity threshold, a low reliability deleting unit <b>211</b> determines the sleepiness level with deleting the low reliability characteristic values. Specifically, the low reliability deleting unit <b>211</b> deletes the characteristic value having low reliability, and calculates the sleepiness level D′ by using the estimation function f without the low reliability characteristic value. For example, when the characteristic values c<b>1</b> to cn includes only one the low reliability characteristic value ci, the estimation function f is described as follows. <br /><i>D′=f</i>(<i>c</i>1,<i>c</i>2, . . . ,<i>ci−</i>1,<i>ci+</i>1, . . . ,<i>cn</i>)  (F3)

When the sleepiness level D is three, i.e., when D=f(c<b>1</b>, c<b>2</b>, . . . , cn)=3, the sleepiness level D′ without the low reliability characteristic value may be 3.5, i.e., <br /><i>D′=f</i>(<i>c</i>1,<i>c</i>2, . . . ,<i>ci−</i>1,<i>ci+</i>1, . . . ,<i>cn</i>)=3.5.

Thus, by removing the low reliability characteristic value from factors for determining the sleepiness level, the sleepiness level can be estimated with high accuracy.

An alarm determining unit <b>210</b> determines whether alarm sound and/or alarm voice message corresponding to the sleepiness level D, D′ is formed and is output to the driver. At this time, the alarm determining unit <b>210</b> determines based on the reliability R. For example, when the sleepiness level D is equal to or larger than three, and further, when the reliability R for one minute is equal to or larger than 90%, the alarm determining unit <b>210</b> determines that the alarm sound and/or the alarm voice message is output. Thus, the driver's feeling with respect to the alarm is relaxed. The alarm may be generated not only with using sound but also with using display and vibration.

The sleepiness level determination device determines the sleepiness level D based on the characteristic values c<b>1</b> to cn. Specifically, the statistics of the characteristic values c<b>1</b> to cn provides the frequency distribution. The representative characteristic values x<b>1</b> to xn represent the frequency distribution. The sleepiness level determination device calculates the reliability R of the sleepiness level D based on the proximity between the representative characteristic values x<b>1</b> to xn and the characteristic values c<b>1</b> to cn. Thus, the reliability R is calculated with respect to the determination of the sleepiness level D itself. The reliability of determination of the sleepiness level D is improved.

(Modifications)

<figref idrefs="DRAWINGS">FIG. 8</figref> shows a control circuit <b>20</b> according to a modification of the example embodiment. The circuit <b>20</b> in <figref idrefs="DRAWINGS">FIG. 8</figref> does not include the reliability threshold determining unit <b>209</b> and the low reliability deleting unit <b>211</b>. The circuit <b>20</b> includes a discrepancy determining unit <b>221</b> for the sleepiness level and a discrepancy deleting unit <b>222</b>, which determines the sleepiness level with removing the discrepancy.

The sleepiness level determining unit <b>207</b> determines the sleepiness level D from the formula F1. Further, the sleepiness level determining unit <b>207</b> estimates, i.e., determines an individual sleepiness level D<b>1</b> to Dn from each characteristic value c<b>1</b> to cn. The discrepancy determining unit <b>221</b> compares the individual sleepiness levels D<b>1</b> to Dn, so that the unit <b>221</b> determines whether an inconsistence individual sleepiness level exists. When the individual sleepiness levels D<b>1</b> to Dn includes the inconsistence individual sleepiness level, the discrepancy deleting unit <b>222</b> determines the sleepiness level D″ by using the estimation function f without the characteristic value c<b>1</b> to cn corresponding to the inconsistence individual sleepiness level D<b>1</b> to Dn. For example, the first individual sleepiness level D<b>1</b> is three, i.e., D<b>1</b>=f(c<b>1</b>)=3. The second individual sleepiness level D<b>2</b> is three, i.e., D<b>2</b>=f(c<b>2</b>)=3. The n-th individual sleepiness level Dn is one, i.e., Dn=f(cn)=1. In this case, the n-th individual sleepiness level Dn calculated based on the n-th characteristic value cn is different from the other individual sleepiness levels D<b>1</b> to Dn−1. Thus, the sleepiness level D″ is calculated by using the following formula F4. Specifically, the sleepiness level D″ is calculated by the estimation function f without using the n-th characteristic value cn. <br /><i>D″==f</i>(<i>c</i>1,<i>c</i>2, . . . ,<i>cn−</i>1)  (F4)

When the sleepiness level D is three, i.e., when D=f(c<b>1</b>, c<b>2</b>, . . . , cn)=3, the sleepiness level D″ without the inconsistence individual sleepiness level may be 3.5, i.e., D″=f(c<b>1</b>, c<b>2</b>, . . . , cn−1)=3.5.

Here, when the individual sleepiness levels D<b>1</b> to Dn are different from each other, or when it is difficult to specify the inconsistence individual sleepiness level such that, for example, the first individual sleepiness level D<b>1</b> is three, the second individual sleepiness level D<b>2</b> is three, the third individual sleepiness level D<b>3</b> is two, and the fourth individual sleepiness level D<b>4</b> is two, the inconsistence individual sleepiness level may be specified in view of a contribution rate of each characteristic value to the estimation function f.

The reliability calculating unit <b>208</b> for the sleepiness level calculates the reliability R″ based on the characteristic values c<b>1</b> to cn−1 without using the characteristic value cn corresponding to the inconsistence individual sleepiness level Dn, the contribution rate of the characteristic values c<b>1</b> to cn−1, and the detection rate E according to the following formula F5. <br />R″={[<i>k</i>1×(contribution rate of <i>c</i>1)+[<i>k</i>2×(contribution rate of <i>c</i>2)+ . . . +[<i>kn−</i>1×(contribution rate of <i>cn−</i>1)]]×<i>E </i>

The reliability R″ of the sleepiness level D″ is calculated from the contribution rate of each characteristic value c<b>1</b> to cn−1, which are used in the determination of the sleepiness level D″. The contribution ratio is a standard of the reliability of the regression equation when the sleepiness level D″ is estimated with the regression equation. Thus, the calculated reliability R″ properly corresponds to the reliability of the sleepiness level D″.

Further, the inconsistence individual sleepiness level is removed from the elements of calculation of the reliability R″ and estimation of the sleepiness level D″. Thus, the device determines the sleepiness level D″ and the reliability R″ with high accuracy.

In the above example embodiment, the reliability R of the sleepiness level D is calculated in view of the detection rate E, which is calculated by the detection error calculating unit <b>204</b>. Alternatively, the reliability R of the sleepiness level D may be calculated without using the detection rate E. Specifically, the reliability R may be calculated with using only the statistics of the characteristic values c<b>1</b> to cn. Even when the detection rate E is not used, the reliability R may properly correspond to the reliability of the sleepiness level D.

While the invention has been described with reference to preferred embodiments thereof, it is to be understood that the invention is not limited to the preferred embodiments and constructions. The invention is intended to cover various modification and equivalent arrangements. In addition, while the various combinations and configurations, which are preferred, other combinations and configurations, including more, less or only a single element, are also within the spirit and scope of the invention.