Warning method for driving vehicle and electronic apparatus for vehicle

Vehicle driving warning method and vehicle electronic device

WARNING METHOD FOR DRIVING VEHICLE AND ELECTRONIC APPARATUS FOR VEHICLE

The present invention relates to a warning mechanism, and particularly relates to a driving warning method and a vehicle electronic device based on image recognition technology.

With the development of transportation, local development has been promoted. However, traffic accidents caused by improper man-made means of transportation have become a major factor endangering social security. For example, since mobile phones have become an indispensable electronic product for modern people, more and more car drivers will use mobile phones while driving. Drivers have to hold their phones while driving, which leads to distraction and increases the accident rate. Therefore, effective and real-time monitoring of driving behavior and warning of improper driving behavior through safety systems are still problems in this field that need to be resolved.

The invention provides a driving warning method and an electronic device for a vehicle, which use image recognition technology to determine whether a driver is using a mobile device.

The driving warning method of the present invention is used in an electronic device for a vehicle. The method includes: using an image capturing unit to continuously capture an image sequence of a driver; and detecting the ear position area of a facial object in each image of the image sequence; Detect the target object in each image of the image sequence; calculate the movement trajectory of the target object according to the image sequence; and send out a prompt signal when the movement trajectory moves toward the ear position area.

In an embodiment of the present invention, the step of detecting the ear position area of the facial object in each image of the image sequence includes: obtaining the facial object by a face recognition algorithm; searching for the nostril in the facial object Object; and based on the position of the nostril object, search for the ear position area in the horizontal direction.

In an embodiment of the present invention, the step of calculating the movement trajectory of the target object according to the image sequence includes: calculating the vertical projection amount and the horizontal projection amount of the target object to obtain the size range of the target object; taking one within the size range Reference point: The movement trajectory is obtained by the position of the reference point of each image.

In an embodiment of the present invention, when the target object moves toward the ear position area, it can also be determined whether the residence time of the target object stays in the ear position area exceeds the preset time; and when the residence time exceeds the preset time To send out a reminder signal.

In an embodiment of the present invention, the step of detecting the target object includes: obtaining a Region of Interest (ROI) according to the ear measurement location area; and the respective interest of the current image and the reference image in the image sequence The region performs an image subtraction algorithm to obtain the target region image; and by referring to the interest region of the image, the noise of the target region image is filtered out to obtain the target object.

In an embodiment of the present invention, the step of filtering out the noise of the target area image by using the interest area of the reference image to obtain the target object includes: performing an edge detection algorithm and an expansion calculation on the interest area of the reference image Method to obtain the filtered area image; and perform an image subtraction algorithm on the filtered area image and the target area image to obtain the target object.

The electronic device for a vehicle of the present invention includes: an image capturing unit, which continuously captures an image sequence of the driver; a storage unit, which stores the image sequence; and a processing unit, which is coupled to the storage unit to obtain the image sequence, and performs image processing Module. The image processing module detects the ear position area of the face object in each image of the above-mentioned image sequence, and detects the target object in each image of the above-mentioned image sequence, and calculates the movement trajectory of the target object, which is based on the movement trajectory Determine whether the target object moves toward the ear position area, and when the target object moves toward the ear position area, a prompt signal is issued.

In an embodiment of the present invention, the above-mentioned image processing module includes: an ear detection module, which detects the ear position area of the face object in each image of the above-mentioned image sequence; and a target detection module, which detects The target object in each image of the above image sequence; the trajectory calculation module, which calculates the movement trajectory of the target object; the judging module, based on the movement trajectory, judging whether the target object is moving toward the ear position area; and the prompting module, where the target object is facing When the ear side position area moves, a reminder signal is issued.

In an embodiment of the present invention, the above-mentioned image processing module further includes: a face recognition module, which obtains a face object by a face recognition algorithm, and searches for nostril objects among the face objects. In addition, the above-mentioned ear detection module can be further based on nostril objects For the position of the piece, search for the area beside the ear in the horizontal direction.

In an embodiment of the present invention, the trajectory calculation module calculates the vertical projection amount and the horizontal projection amount of the target object to obtain the size range of the target object, and takes a reference point within the size range to use the image sequence The position of the reference point of each image is obtained, and the movement trajectory is obtained.

In an embodiment of the present invention, the target detection module obtains the region of interest according to the ear measurement location area, and performs an image subtraction algorithm on the respective regions of interest of the current image and the reference image of the image sequence to obtain The target area image is used to filter out the noise of the target area image by referring to the interest area of the image to obtain the target object.

In an embodiment of the present invention, the above-mentioned target detection module executes an edge detection algorithm and a dilate algorithm on the interest area of the reference image to obtain a filtered area image, and the filtered area image and The image of the target area performs an image subtraction algorithm to obtain the target object.

In an embodiment of the present invention, the above-mentioned judgment module judges whether the stay time of the target object staying in the ear position area exceeds a preset time, and when the stay time exceeds the preset time, the notification module is notified to issue a reminder signal.

Based on the above, the image recognition technology can be used to determine whether the driver is using a mobile phone in the car, and when it is determined that the driver is using the mobile phone, a reminder message is sent to avoid the driver from being distracted and causing an accident.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

<p>100Car electronic device</p><p>110Image capture unit</p><p>120Processing unit</p><p>130Storage Unit</p><p>140Image Processing Module</p><p>400Image</p><p>410Face objects</p><p>420Nostril objects</p><p>510Reference image</p><p>520Current image</p><p>530Target area image</p><p>540Filter area image</p><p>550Regional image</p><p>511, 521, RRegion of interest</p><p>551Size range</p><p>601Face recognition module</p><p>603Ear Detection Module</p><p>605Target Detection Module</p><p>607Trajectory calculation module</p><p>609Judgment Module</p><p>611Reminder Module</p><p>Btop left vertex</p><p>C1, C2The boundary of the cheek</p><p>EEar side position area</p><p>OTarget Object</p><p>S205~S225The steps of the driving warning method</p><p>S305~S355The steps of another driving warning method</p>

FIG. 1 is a schematic diagram of an electronic device for a vehicle according to an embodiment of the present invention.

Fig. 2 is a flowchart of a driving warning method according to an embodiment of the present invention.

Fig. 3 is a flowchart of another driving warning method according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of an image according to an embodiment of the invention.

5A to 5E are schematic diagrams of detecting a target object according to an embodiment of the invention.

FIG. 6 is a schematic diagram of an image processing module according to an embodiment of the invention.

FIG. 1 is a schematic diagram of an electronic device for a vehicle according to an embodiment of the present invention. Please refer to FIG. 1, the car electronic device 100 includes an image capturing unit 110, a processing unit 120, a storage unit 130 and an image processing module 140. In this embodiment, the vehicle electronic device 100 is an independent device, which is arranged in front of the driver's seat of the vehicle to capture images of the driver. In other embodiments, the vehicle electronic device 100 may also be integrated in the vehicle. For example, the above-mentioned vehicle electronic device 100 is an embedded system architecture, and can be embedded in any electronic device.

The image capturing unit 110 captures an image sequence (including one or more images) of the driver, and stores the image sequence in the storage unit 130. 110 cases of image capture unit For example, it is a video camera or camera with a charge coupled device (CCD) lens, a complementary metal oxide semiconductor transistors (CMOS) lens, or an infrared lens, but it is not limited thereto.

The processing unit 120 is, for example, a central processing unit (CPU), a graphics processing unit (GPU), or other programmable microprocessors (Microprocessors), or digital processors (Digital Signal Processors, DSPs). ) And other devices. The processing unit 120 is coupled to the storage unit 130 to obtain the image sequence captured by the image capturing unit 110. In addition, the processing unit 120 executes the image processing module 140 to perform an identification process on the above-mentioned image sequence. For example, after the image capture unit 110 captures an image, it stores the image in the storage unit 130 through its Input/Output (I/O) unit, and the processing unit 120 obtains the image from the storage unit 130 to execute the image. Processing procedures.

The storage unit 130 is, for example, a random access memory (RAM), a read-only memory (Read-Only Memory, ROM), a flash memory (Flash memory), or a magnetic disk storage device (Magnetic disk storage device). Wait.

In this embodiment, the image processing module 140 is, for example, a code fragment written in a computer programming language. The above-mentioned code fragment may be stored in the storage unit 130 (or another storage unit) and includes a plurality of commands, for example, The processing unit 120 executes the above-mentioned code fragments. In addition, in other embodiments, the above-mentioned image processing module 140 may also be a hardware component composed of one or more circuits, which is coupled to The processing unit 120 is driven by the processing unit 120.

In addition, in other embodiments, the image capturing unit 110 further has an illuminating element for supplementing light in a timely manner when the light is insufficient, so as to ensure the clarity of the captured image.

The following is a detailed description of the steps of the driving warning method in conjunction with the above-mentioned vehicle electronic device 100. Fig. 2 is a flowchart of a driving warning method according to an embodiment of the present invention. 1 and 2 at the same time, the image capturing unit 110 continuously captures the image sequence of the driver (step S205). Then, the image processing module 140 starts to perform an image processing procedure on each image of the above-mentioned image sequence.

The image processing module 140 detects the ear position area of the face object in each image of the image sequence (step S210). In order to obtain the ear side position area more accurately, in this embodiment, the image processing module 140 may also search for the nostril object in the face object after obtaining the face object, and then proceed to the horizontal direction based on the position of the nostril object Search for the area beside the ear. For example, search for the borders of the left and right cheeks on the left and right sides of the nostril object, and then, according to the relative positions of the human face and the ears, use the searched borders as a reference to obtain the left and right ear positions.

Then, the image processing module 140 detects the target object in each image of the image sequence (step S215). Here, the target object is, for example, a mobile phone. In other words, the image processing module 140 detects the mobile phone in each image. After obtaining the target object, the image processing module 140 calculates the movement track of the target object according to the image sequence (step S220). For example, take one point of the target object as the reference point, and calculate the position of the reference point in each image to obtain the moving track of the target object trace. When the image processing module 140 detects that the movement track moves toward the ear position area, the image processing module 140 will issue a prompt signal (step S225).

Here, the image processing module 140 may further determine whether the stay time of the target object staying in the ear position area exceeds a preset time (for example, 3 seconds), and issue a prompt signal when the stay time exceeds the preset time. That is to say, if the stay time of the target object in the ear position area exceeds the preset time, it means that the driver may use the mobile phone while driving.

Another embodiment will be described below.

Fig. 3 is a flowchart of another driving warning method according to an embodiment of the present invention. Please refer to FIG. 1 and FIG. 3 at the same time. First, a plurality of images of the driver are continuously captured by the image capturing mold (step S305). Then, the image processing module 140 first performs a background filtering operation (step S310). For example, the Nth image and the N+1th image are subjected to difference processing. After that, the image with the filtered back shadow can be converted into a grayscale image to perform subsequent actions.

After that, the image processing module 140 detects facial features in the above-mentioned images to obtain facial objects (step S315). For example, the storage unit 130 stores a feature database. This feature database includes facial feature samples (pattern). The image processing module 140 obtains the face object by comparing with the samples in the feature database. In a preferred embodiment, the AdaBoost algorithm or other existing face recognition algorithms (for example, the use of Haar-like features to perform face recognition actions) can be used to obtain the faces in each image, but here only For example, it is not limited to this.

Then, the image processing module 140 detects the ears of the facial objects in each image. Side position area (step S320). For example, the image processing module 140 can search for the nostril object in the face object, and then search for the boundary of the left and right cheeks on the left and right sides of the nostril object, and then use the searched boundary as a reference based on the relative position of the face and ears. Obtain the ear position area on the left and right sides. After that, the image processing module 140 can obtain a Region of Interest (ROI) according to the ear measurement location area (step S325).

For example, FIG. 4 is a schematic diagram of an image according to an embodiment of the invention. After the image processing module 140 detects the face object 410 of the image 400, it can obtain the nostril object 420, and then the nostril object 420 finds the borders C1 and C2 on the left and right sides, and obtains them based on the borders C1 and C2. The area on the side of the ear. For the convenience of description, only the boundary C1 of one cheek is used for description, however, it can be analogized to the boundary C2 of the other cheek. Using the coordinates of the boundary C1 as a reference, the ear side position area E is obtained with a preset size range. Then, according to the ear side position area E, another preset size range is used to obtain the interest area R.

Then, the image processing module 140 separates a current image and a reference image (which can be a previous image, for example, the previous image or the previous N images of the current image, or any image set in advance). Perform an image subtraction algorithm on the region of interest to obtain the target region image (step S330), and filter the noise of the target region image by referring to the region of interest in the image to obtain the target object (step S335).

For example, FIGS. 5A to 5E are schematic diagrams of detecting a target object according to an embodiment of the present invention. Here, for the sake of convenience, the gray scales of FIGS. 5A to 5E are omitted, and only the edges of the gray scales are depicted for description. Figure 5A shows the reference image 510, which shows the area of interest 511; FIG. 5B shows the current image 520 captured by the current image capturing unit 110, which shows the area of interest 521 and the ear position area E; FIG. 5C shows the target area Image 530; FIG. 5D shows the filtered area image 540, and FIG. 5E shows the area image 550 with the target object O.

Specifically, after the image subtraction algorithm is performed on the interest area 511 of the reference image 510 and the interest area 521 of the current image 520, a target area image 530 with a difference in the two images can be obtained. In other words, the target area image 530 is the result obtained by performing the image subtraction algorithm on the interest area 511 and the interest area 521. In the target area image 530, other noises of non-target objects are represented by dotted lines. In order to filter the noise to obtain the target object, an edge detection algorithm and a dilate algorithm are executed on the interest area 511 of the reference image 510 to obtain a filtered area image 540. Then, after performing the image subtraction algorithm on the target area image 530 and the filtered area image 540, the area image 550 with the target object O as shown in FIG. 5E can be obtained.

Returning to FIG. 3, after obtaining the target object, the image processing module 140 calculates the vertical projection amount and the horizontal projection amount of the target object to obtain the size range of the target object (step S340). Specifically, the image processing module 140 calculates the vertical projection of the target object to obtain the length of the target object on the vertical axis, and calculates the horizontal projection of the target object to obtain the width of the target object on the horizontal axis. Through the above-mentioned length and width, the size range of the target object can be obtained.

Then, the image processing module 140 takes a point within the size range as the reference point (step S345), and uses the same point as the target object in each subsequent image The reference point can obtain the movement trajectory based on the position of the reference point of each image (step S350).

Taking FIG. 5E as an example, the length and width of the target object O are calculated to obtain the size range 551, and the upper left vertex B of the size range 551 is taken as the reference point. The target object in other subsequent images also uses the upper left vertex of its size range as the reference point. According to this, the movement trajectory of the target object can be known from the reference points of multiple images. The above-mentioned use of the upper left vertex of the size range as the reference point is only an example, and is not limited thereto. After that, the image processing module 140 sends out a prompt message when detecting that the movement track moves to the ear side position area (step S355). In addition, the image processing module 140 can also detect whether the movement track is gradually advancing toward the ear position area, etc., so as to more clearly determine the target object (in this embodiment, a mobile phone).

An example is given below to illustrate the structure of the image processing module 140. FIG. 6 is a schematic diagram of an image processing module according to an embodiment of the invention. 4, the image processing module 140 includes a face recognition module 601, an ear detection module 603, a target detection module 605, a trajectory calculation module 607, a judgment module 609, and a prompt module 611.

The face recognition module 601 obtains the face object by the face recognition algorithm, and searches for the nostril object in the face object. For example, the face recognition module 601 uses the AdaBoost algorithm or other existing face recognition algorithms (for example, using Haar-like features to perform face recognition actions) to detect facial objects in the image.

The ear detection module 603 detects the ear position area of the face object in the image. For example, the ear detection module 603 can compare the ear features obtained from the image with the ear feature samples previously stored in the feature database to obtain the left and right sides. The ear position area. In addition, if the ears are covered by hair or other objects, and the ear features cannot be obtained, the ear detection module 603 can also obtain the position of the ear side relative to the human face based on the sample training in advance, thus directly Use the preset data to obtain the ear position areas on the left and right sides. In addition, the ear detection module 603 can also search the ear side location area in a horizontal direction based on the position of the nostril object. For example, based on the position of the nostril object, obtain the left and right boundaries of the cheek in the horizontal direction, and then obtain the left and right ear position areas with preset data.

The target detection module 605 detects the target object in the image. The target detection module 605 obtains the region of interest based on the ear measurement location area, and executes the image subtraction algorithm on the respective regions of interest of the current image and the reference image to obtain the target region image, based on the interest region of the reference image To filter out the noise of the image of the target area to obtain the target object. For detailed description, please refer to FIGS. 5A to 5E, which are omitted here.

The trajectory calculation module 607 calculates the movement trajectory of the target object. For example, the trajectory calculation module 607 uses one point of the target object as a reference point, and performs statistics on the position of the reference point in each image to obtain the movement trajectory of the target object. For example, the trajectory calculation module 607 calculates the vertical projection amount and the horizontal projection amount of the target object to obtain the size range of the target object, and takes a reference point within the size range to obtain the position of the reference point of each image Movement track.

The judging module 609 judges whether the target object has moved to the ear position area based on the movement trajectory. For example, the judging module 609 will judge whether the position is located in the ear position area according to the position of the reference point obtained by the trajectory calculation module 607. In addition, the judgment module 609 can also predict whether the above-mentioned target object will move according to the movement trajectory. To the area of the ear side. The above is only an example and not a limitation.

The prompt module 611 sends out a prompt signal when the target object moves to the ear side position area. For example, when the prompt module 611 receives an instruction or command issued by the judgment module 609, it sends out prompt signals such as voice or vibration.

The aforementioned car electronic device 100 uses an image recognition method to detect whether the driver is using a mobile phone. In addition, the aforementioned car electronic device 100 may also have a sleep detection mechanism to detect whether the driver is fatigued.

In summary, the present invention uses image recognition technology to find the target object in the image, and calculates the movement trajectory of the target object in multiple consecutive images, so as to determine whether the driver is using a mobile phone in the car. In addition, when it is determined that the driver is using the mobile phone, a prompt message is issued to avoid the driver from being distracted and causing an accident.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to those defined by the attached patent application scope.