Foreign matter attachment determining system for on-vehicle camera lens

The present invention relates to a foreign matter adhesion determination device for an in-vehicle camera lens, and more particularly to a foreign matter adhesion determination device for an in-vehicle camera lens, which is suitable for determining whether or not foreign matter is attached to a lens of an in-vehicle camera.

Conventionally, an image recognition technology based on an image taken by an in-vehicle camera has been used in a drive assist system provided with an alarm output function for preventing lane departure.

In this type of drive assist system, in order to ensure the reliability of the image recognition result, it is important to accurately determine whether or not foreign matter such as raindrops or dirt is attached to the lens of the in-vehicle camera. If the lens is covered and foreign matter adheres to the lens, the foreign matter image is included as noise in the captured image, and the system malfunctions due to this (for example, unnecessary lane departure warning output or necessary). In order to prevent improper driving operation due to lane departure warning output failure, etc.) and this malfunction, some measures (for example, implementation of image processing with noise added, alarm or image processing notifying the decrease in reliability of the recognition result) This is because it is necessary to take measures such as canceling the above.

Therefore, as a conventional technique for determining the presence or absence of foreign matter adhering to the lens of an in-vehicle camera, for example, there is a possibility that raindrops may adhere to the lens by using a raindrop sensor or a wiper operation signal. A technique for detecting is proposed.

In addition to this, for example, as shown in Patent Document 1, a method of determining a portion having little change as a foreign substance after taking a difference between captured images at regular time intervals by image processing, and a patent document. As shown in 2, a method has been proposed in which the presence or absence of a foreign substance is determined by comparing the captured image with a reference image stored in advance.

<p><patcit num="1"><text>Japanese Unexamined Patent Publication No. 2007-189369</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 2007-228448</text></patcit></p>

<p> However, when a raindrop sensor or a wiper operation signal is used, it is possible to determine the possibility of raindrops adhering to the lens, but it is not possible to determine whether or not the raindrops have actually adhered to the lens. Even in this case, it is not possible to determine the presence or absence of adhesion.</p><p> In addition, although the method of determining the presence or absence of foreign matter adhering from the difference in the captured image can be performed appropriately, the processing load is large because the process of determining the amount of change based on the difference is required for the entire image. Become.</p><p> Further, the method of comparing with the reference video requires a large memory capacity for storing the reference video.</p><p> Therefore, the present invention has been made in view of such a point, and it is possible to determine whether or not foreign matter is attached to the lens of the in-vehicle camera easily, at low cost, and appropriately. It is an object of the present invention to provide a determination device.</p>

<p> In order to achieve the above-mentioned object, the foreign matter adhesion determination device for an in-vehicle camera lens according to the present invention is for an in-vehicle camera lens for determining whether or not foreign matter is attached to the lens of the in-vehicle camera that photographs the periphery of the vehicle. A foreign matter adhesion determination device, which is a detection means for detecting a long-shaped predetermined detection object formed on a predetermined vehicle peripheral region based on an image captured by the in-vehicle camera, and a detection means detected by the detection means. A determination means for determining whether or not the foreign matter is attached to the lens based on the detection object is provided, and the determination means determines the length or formation interval of the detected detection object. The determination based on the presence or absence of the predetermined constantity, or the determination based on the presence or absence of the predetermined constantity and the presence or absence of the predetermined continuity of the shape of the detected detection object. It is characterized by. Then, according to such a configuration, the presence or absence of a predetermined constantity regarding the length or formation interval of a predetermined detection object detected based on the captured image, or a predetermined value regarding the shape of the detection object and the length of the detection object. By using both the presence or absence of continuity as the judgment criteria, it is possible to easily, lowly, and appropriately judge the presence or absence of foreign matter adhering to the lens of the in-vehicle camera.</p><p> Further, the detection target includes a lane formed on the road surface as the vehicle peripheral region, and the determination means determines that the detected lane is detected when the detection means detects the lane. When the length is constant in the longitudinal direction of the lane corresponding to the shape of the broken lane as the predetermined constant, it is determined that the foreign matter is not attached to the lens, while the detection is performed. It may be determined that the foreign matter is attached to the lens, provided that the lane is not constant in length in the longitudinal direction of the lane, as at least a part of the condition. Then, according to such a configuration, whether or not the lane detected based on the captured image has a constant length in the longitudinal direction of the lane (in other words, in the appropriately photographed broken line lane). Based on (whether or not there is), it is possible to determine whether or not foreign matter is attached to the lens of the in-vehicle camera easily, at low cost, and appropriately when driving on the road.</p><p> Further, the detection target includes a lane formed on the road surface as the vehicle peripheral region, and the determination means determines that the detected lane is detected when the detection means detects the lane. When the lens has a constant formation interval in the longitudinal direction of the lane corresponding to the shape of the broken line lane as the predetermined constant, it is determined that the foreign matter does not adhere to the lens, while the detection is performed. It may be determined that the foreign matter is attached to the lens, provided that the formed lane does not have a constant formation interval in the longitudinal direction of the lane, as at least a part of the condition. Then, according to such a configuration, whether or not the lane detected based on the captured image has a constant formation interval in the longitudinal direction of the lane (in other words, in the appropriately photographed broken line lane). Based on (whether or not there is), it is possible to determine whether or not foreign matter is attached to the lens of the in-vehicle camera easily, at low cost, and appropriately when driving on the road.</p><p> Furthermore, the determination means also applies the lens to the lens even when the detected lane has a shape continuity in the longitudinal direction of the lane corresponding to the shape of the solid lane as the predetermined continuity. It may be determined that no foreign matter is attached. In this case, the condition for determining that the foreign matter is attached to the lens includes that the detected lane does not have shape continuity in the longitudinal direction of the lane. You may. Then, according to such a configuration, whether or not the lane detected based on the captured image has the continuity of the shape in the longitudinal direction of the lane (in other words, the solid lane that is properly photographed). Based on (whether or not), it is possible to more appropriately determine whether or not foreign matter is attached to the lens of the in-vehicle camera.</p><p> Further, when a plurality of lanes are detected by the detection means, the determination means makes the determination for each of the detected lanes, and the foreign matter is contained in the lens in the determination result for each of the detected lanes. When at least one determination result indicating that the lens is attached is included, the determination result may be regarded as the determination result for the entire lens. Then, according to such a configuration, even when a plurality of lanes are detected, it is possible to appropriately determine whether or not foreign matter is attached to the lens.</p><p> Further, the detection target includes a horizontal frame line portion of each of a plurality of parking frames arranged and formed in a row on the parking area as the vehicle peripheral region, and the determination means is described by the detection means. When a horizontal frame line portion is detected, when each of the detected horizontal frame line portions has a constant length in the longitudinal direction of the horizontal frame line portion as the predetermined constantity. Determines that the foreign matter does not adhere to the lens, and on the other hand, when each of the detected horizontal frame lines does not have a constant length in the longitudinal direction, the lens is attached to the lens. It may be determined that the foreign matter is attached. Then, according to such a configuration, foreign matter to the lens of the in-vehicle camera is determined based on whether or not each horizontal frame line portion detected based on the captured image has a constant length in the longitudinal direction. It is possible to easily, at low cost, and appropriately determine the presence or absence of adhesion of the lens when parking.</p><p> Furthermore, the detection target includes the vertical frame lines of the plurality of parking frames arranged and formed in a row on the parking area as the vehicle peripheral region, and the determination means is said by the detection means. When each vertical frame line portion is detected, when each of the detected vertical frame line portions has a constant formation interval in the horizontal width direction of the vertical frame line portion as the predetermined constantness. On the other hand, if it is determined that the foreign matter does not adhere to the lens, and the detected vertical frame lines do not have a constant formation interval in the horizontal width direction, the lens is used. It may be determined that the foreign matter is attached to the lens. Then, according to such a configuration, foreign matter to the lens of the in-vehicle camera is determined based on whether or not each vertical frame line portion detected based on the captured image has a constant formation interval in the width direction. It is possible to easily, at low cost, and appropriately determine the presence or absence of adhesion of the lens when parking.</p><p> Further, the lens is a wide-angle lens and includes distortion correction means for correcting distortion on an image captured by the vehicle-mounted camera, and the detection means detects the detection based on the image captured after the distortion correction by the distortion correction means. The object may be detected. Then, according to such a configuration, the detection target can be appropriately detected in consideration of the distortion of the wide-angle lens.</p><p> Further, the detection means interlocks with the own vehicle position detection function of the in-vehicle navigation device, and obtains the detection target of the type corresponding to the own vehicle position detected by the own vehicle position detection function according to the type. It may be detected by the set detection method. Then, according to such a configuration, it is possible to detect the type of detection target according to the position of the own vehicle, so that it is possible to prevent erroneous detection or detection failure of the detection target and erroneous determination associated therewith. it can.</p><p> Furthermore, the detection means is capable of detecting a predetermined obstacle on the detection target, and when the detection means detects the obstacle, the determination means detects the detection target. It may not be used for judgment. Then, according to such a configuration, it is possible to prevent an erroneous determination caused by an obstacle.</p>

<p> According to the present invention, it is possible to determine whether or not foreign matter is attached to the lens of an in-vehicle camera easily, at low cost, and appropriately.</p>

<figref num="1">A block diagram showing an embodiment of a lens deposit determination device for an in-vehicle camera according to the present invention.</figref><figref num="2">Explanatory drawing for explaining distortion correction</figref><figref num="3">Schematic diagram showing the broken white line lane in the image used for determining foreign matter adhesion</figref><figref num="4">Schematic diagram showing a solid white lane in an image used for determining foreign matter adhesion</figref><figref num="5">Schematic diagram showing the horizontal frame line of the parking frame in the image used for determining foreign matter adhesion</figref><figref num="6">A flowchart showing an embodiment of a lens deposit determination device for an in-vehicle camera according to the present invention.</figref>

Hereinafter, embodiments of the foreign matter adhesion determination device for an in-vehicle camera lens according to the present invention will be described with reference to FIGS. 1 to 6.

As shown in FIG. 1, the foreign matter adhesion determination device 1 for an in-vehicle camera lens in the present embodiment has an in-vehicle camera 2 that photographs the periphery of the own vehicle, and the in-vehicle camera 2 is a wide-angle lens such as a fisheye lens. It is said to be a camera with a wide viewing angle equipped with 3, and it is also a digital camera equipped with a solid-state image sensor (imaging surface) such as CCD or CMOS. The in-vehicle camera 2 captures a predetermined shooting area centered on the front of the vehicle, which is attached to the front of the vehicle (for example, an emblem) so as to look down on the road surface in front of the vehicle from an oblique direction. It may be a front camera for shooting. In addition to this, the in-vehicle camera 2 is mounted on the left side of the vehicle (for example, a left door mirror, etc.) so as to look down on the road surface on the left side of the vehicle from an oblique direction. It may be a left side camera that shoots a predetermined shooting area at the center. In addition to these, the in-vehicle camera 2 is mounted on the right side of the vehicle (for example, a right door mirror, etc.) so as to look down on the road surface on the right side of the vehicle from an oblique direction. It may be a right side camera that shoots a predetermined shooting area at the center. Furthermore, in addition to these, the in-vehicle camera 2 is mounted on the rear part of the own vehicle (for example, the rear license garnish part, etc.) so as to look down on the road surface behind the own vehicle from an oblique direction. It may be a back camera that shoots a predetermined shooting area at the center.

Further, as shown in FIG. 1, a camera image acquisition unit 4 is connected to the in-vehicle camera 2, and the camera image acquisition unit 4 acquires the captured image of the in-vehicle camera 2.

Further, as shown in FIG. 1, a distortion correction unit 5 as a distortion correction means is connected to the camera image acquisition unit 4, and the in-vehicle camera 2 is connected to the distortion correction unit 5 from the camera image acquisition unit 4. The shot image of is input. Then, as shown in FIG. 2, the distortion correction unit 5 performs known distortion correction using the internal parameters (image principal point, focal length, etc.) of the in-vehicle camera 2 on the input captured image. It has become. The internal parameters may be set in the distortion correction unit 5 in advance. Further, the distortion correction unit 5 cuts out only the image of a predetermined use area (for example, a pixel area in a predetermined range excluding an unnecessary area such as an aperture or a reflection area of the own vehicle) in the input captured image. Distortion correction may be performed on the cut-out image. Further, the distortion correction unit 5 uses a so-called mapping table in which the correspondence between the coordinates (XY coordinates) of an arbitrary pixel in the image before the distortion correction and the pixel corresponding to the pixel after the distortion correction is described. Distortion correction may be performed. In this case, the correspondence between the coordinates in the mapping table may be defined by using the internal parameters set in the distortion correction unit 5.

Furthermore, as shown in FIG. 1, an object detection unit 6 as a detection means is connected to the distortion correction unit 5, and the object detection unit 6 is connected to the distortion correction unit 5 after distortion correction. The captured image of is input. Then, the object detection unit 6 detects a long predetermined detection object formed on a predetermined vehicle peripheral region by image recognition of the input image after distortion correction. ..

Here, as an example of a predetermined detection object, there is a lane formed on a road surface as an example of a predetermined vehicle peripheral region. Further, as another example of the predetermined detection object, the horizontal borders of the plurality of parking frames arranged and formed in a row on the parking area (parking lot) as another example of the predetermined vehicle peripheral area. There is a department. Further, as an example of a predetermined detection object other than these, there is a vertical frame line portion of each of a plurality of parking frames on the parking area.

Further, as shown in FIG. 1, the vehicle position detection unit 7 of the vehicle-mounted navigation device is connected to the object detection unit 6. The vehicle position detection unit 7 appropriately combines satellite navigation using a satellite positioning system (for example, GPS) and autonomous navigation based on the detection results of an autonomous navigation sensor (for example, a gyro sensor or a vehicle speed sensor). The position of the own vehicle is detected by hybrid navigation. Further, the own vehicle position detection unit 7 performs a map matching process for correcting the own vehicle position detected in this way to the position on the corresponding road in the map data stored in the map database (not shown). It has become. Then, when the map matching process is properly performed, the own vehicle position detection unit 7 uses the own vehicle position after the map matching process as the final detection result of the own vehicle position.

The detection result of the vehicle position detected in this way is input to the object detection unit 6. Then, the object detection unit 6 detects a type of detection target corresponding to the input detection result of the vehicle position by a detection method set according to the type. For example, when the detection result of the own vehicle position indicates a position on the road (that is, when the own vehicle position is matched with an arbitrary road), the object detection unit 6 detects the above-mentioned lane. You can do it like this. In this case, the object detection unit 6 may detect the lane by a method of detecting an edge corresponding to an edge of the lane. Similar to the known white line detection method, such edge detection may be performed based on the brightness difference between the lane image and the road surface image. Further, when the detection result of the position of the own vehicle indicates the position on the parking area, the object detection unit 6 detects each horizontal frame line portion or each vertical frame line portion of the plurality of parking frames described above. It should be. In this case, the detection of the horizontal border portion and the vertical border portion is basically performed by a method of detecting the edge corresponding to the end edge of the border portion, as in the case of the detection of the lane. Good. However, when detecting the horizontal frame line part and the vertical frame line part, not only the edge but also the characteristic points unique to the frame line part such as the corner part of the parking frame and the intersection of the horizontal frame line part and the vertical frame line part. Since it is necessary to detect the lane, a detection method slightly different from that in the case of lane detection is used. Regarding the fact that the vehicle position is on the parking area, for example, the vehicle position is not matched on any road, and the parked vehicle is stored in the detected coordinates of the vehicle position and the map data. It may be detected by the fact that the conversion distance between the coordinates of the representative point of the parking lot (for example, the coordinates of the entrance / exit) is equal to or less than the threshold value.

However, it is not necessary to be limited to such a configuration, and the object detection unit 6 should detect based on vehicle information other than the own vehicle position (for example, vehicle speed, shift operation information, steering angle information, etc.). If the type of the detection target object (lane / frame line portion) can be recognized, the detection target object may be detected according to the vehicle information.

Further, as shown in FIG. 1, a foreign matter adhesion determination unit 8 as a determination means is connected to the object detection unit 6, and the foreign matter adhesion determination unit 8 is to be detected by the object detection unit 6. The detection result of the object is input. Then, the foreign matter adhesion determination unit 8 has come to determine whether or not foreign matter is attached to the lens 3 (hereinafter, referred to as foreign matter adhesion determination) based on the input detection result of the detection object. There is. At this time, the foreign matter adhesion determination unit 8 has come to determine the presence or absence of foreign matter adhesion based on the presence or absence of a predetermined constantity regarding the length or formation interval of the detection object detected by the object detection unit 6. ing. In this foreign matter adhesion determination, the presence or absence of a predetermined continuity with respect to the shape of the detection object may be used as a further determination criterion.

For example, when the object detection unit 6 detects a lane, the foreign matter adhesion determination unit 8 determines that the detected lane is the length in the longitudinal direction of the lane corresponding to the shape of the broken line lane as the predetermined constantity. When the lens 3 has the constantness, it may be determined that no foreign matter is attached to the lens 3. On the other hand, if the detected lane does not have a constant length in the longitudinal direction of the lane, the foreign matter adhesion determining unit 8 causes foreign matter to adhere to the lens 3 under all of the conditions. It may be determined that it is. Alternatively, the foreign matter adhesion determination unit 8 is a solid line as the predetermined continuity even when the lane detected by the object detection unit 6 does not have a constant length in the longitudinal direction of the lane. When the shape has continuity in the longitudinal direction of the lane corresponding to the shape of the lane, it may be determined that no foreign matter is attached to the lens 3. In this case, the foreign matter adhesion determination unit 8 indicates that the lane detected by the object detection unit 6 does not have a constant length in the longitudinal direction of the lane (part of the condition) and the longitudinal lane. When both of the fact that the shape does not have continuity in the direction (a part of the condition) (all of the conditions) are satisfied, it may be determined that foreign matter is attached to the lens 3. Here, in order to improve the determination accuracy, the lane in the image used for determining the adhesion of foreign matter is the lane in the image for a series of predetermined several frames (in other words, the own vehicle) that is sequentially detected as the vehicle travels. It is desirable to set the lane to the length of the predetermined shooting time that reflects the speed).

FIGS. 3 and 4 schematically show the lanes in the image used for determining the adhesion of foreign matter.

Specifically, the lane in the image shown in FIG. 3A is a broken line white lane in the real world, and this broken line white lane is the length in the longitudinal direction of each line portion in the Road Traffic Act. Is fixed. The lane shown in FIG. 3 (a) satisfies the constant length (length of each lane portion) in the longitudinal direction of the lane, and can be recognized as an image of a dashed white lane lane taken properly. Therefore, when the lane of the aspect shown in FIG. 3A is detected, the foreign matter adhesion determining unit 8 determines that no foreign matter is attached to the lens 3. It should be noted that the satisfaction of the constant length can be determined by, for example, the following method. That is, first, the coordinates (coordinates of the image coordinate system) of the pixels at both ends (edges) in the longitudinal direction of the lane are detected for each line portion in the video. This detection may be performed in the process of detecting the lane. Next, the detected coordinates are converted from the image coordinate system to the coordinates of the three-dimensional world coordinate system on the real world by using the internal / external parameters of the in-vehicle camera 2 acquired in advance. Next, for each line portion, the difference between the three-dimensional world coordinates of both ends in the longitudinal direction of the lane (that is, the length of each line portion in the image in the real world) is calculated. Then, when it is considered that the calculation results match or approximate (within a predetermined error) between the line portions, it is determined that the constantness of the length is satisfied. When such a method is used, the correspondence data between the image coordinates and the three-dimensional world coordinates is stored in the storage unit in advance, and the coordinates of the coordinates of both ends in the longitudinal direction of the lane of each line portion are used using this correspondence data. It is efficient if the conversion is performed.

On the other hand, the lane in the image shown in FIG. 3 (b) is a broken white lane similar to that in FIG. 3 (a) in the real world, but due to raindrops adhering to the lens 3, the lane is changed. The image looks like a part of it was blocked by raindrops. Although the lane shown in FIG. 3 (b) can be recognized as a lane, it does not satisfy the constant length in the longitudinal direction of the lane and the continuity of the shape (not an intermittent lane). , The image of the broken white lane and the image of the solid white lane cannot be recognized (discriminated). The length of the lane line when it is blocked by such foreign matter such as raindrops is determined after the boundary between the lane and the foreign matter is treated as one end of the lane in the longitudinal direction of the lane. Will be done. This boundary may be detected based on the brightness difference or the like as a part of edge detection at the time of lane detection by the object detection unit 6 described above. Then, when the lane of the aspect shown in FIG. 3B is detected, the foreign matter adhesion determination unit 8 determines that the foreign matter adheres to the lens 3.

Further, the lane in the image shown in FIG. 4A is a solid white lane in the real world, and the image also faithfully reflects the real world and has a continuity of the shape in the longitudinal direction of the lane. Therefore, it can be recognized that it is a properly photographed solid white lane. Therefore, when such a lane of the aspect shown in FIG. 4A is detected, the foreign matter adhesion determination unit 8 may determine that no foreign matter is attached to the lens 3.

On the other hand, the lane in the image shown in FIG. 4 (b) is a dashed white lane similar to that in FIG. 4 (a) in the real world, but as in the case of FIG. 3 (b), the lens 3 Due to the raindrops, the image looks like a part of the lane is blocked by the raindrops. Such a lane shown in FIG. 4 (b) does not satisfy the constant length and the continuity of the shape in the longitudinal direction of the lane, as in the case of FIG. 3 (b). At this time, as described above, it may be determined that the continuity of the shape of the lane is not satisfied by detecting the boundary (edge) with the foreign matter on the lane. Then, such a lane shown in FIG. 4 (b) cannot be recognized as an image of a broken line white lane or an image of a solid white lane, as in the case of FIG. 3 (a). Therefore, when the lane of the aspect shown in FIG. 4B is detected, the foreign matter adhesion determination unit 8 determines that the foreign matter adheres to the lens 3.

The broken line white lane is defined to be constant not only for the length of each line portion but also for the interval between each line portion, that is, the formation interval. Therefore, the foreign matter adhesion determination unit 8 may use the constant formation interval in the longitudinal direction of the lane as a determination criterion instead of the constant length in the longitudinal direction of the lane.

When a plurality of lanes are detected in the image by the object detection unit 6, the foreign matter adhesion determination unit 8 determines the constant length and the continuity of the shape in the longitudinal direction of each detected lane. It suffices to judge the adhesion of foreign matter as a judgment standard. Then, when at least one judgment result indicating that foreign matter is attached to the lens 3 is included in the judgment result for each lane, the judgment result is taken as the judgment result for the entire lens 3. You should consider it.

Further, when the object detection unit 6 detects each of the horizontal frame lines of the plurality of parking frames, the foreign matter adhesion determination unit 8 sets the detected horizontal frame lines as the predetermined constantity. When the lens 3 has a constant length in the longitudinal direction of the horizontal frame line portion, it may be determined that no foreign matter is attached to the lens 3. On the other hand, if the detected horizontal frame line portion does not have a constant length in the longitudinal direction, the foreign matter adhesion determining unit 8 determines that the foreign matter adheres to the lens 3. Just do it.

FIG. 5 schematically shows each horizontal frame line portion in the image used for such determination of foreign matter adhesion.

Specifically, each horizontal frame line portion (white line) 10 in the image shown in FIG. 5 (a) satisfies the constant length in the longitudinal direction, and each horizontal frame line portion appropriately photographed. It can be recognized as an image. This is because the parking spaces in the off-street parking lot are usually arranged in rows at equal intervals. Therefore, when the horizontal frame line portion 10 of the aspect shown in FIG. 5A is detected, the foreign matter adhesion determination unit 8 determines that no foreign matter is attached to the lens 3. As a specific method for determining that the constant length is satisfied, the same method as in the case of a lane can be used.

On the other hand, in each of the horizontal borders 10 in the image shown in FIG. 5 (b), it seems that a part of the horizontal borders 10 was partially blocked by the raindrops due to the raindrops adhering to the lens 3. The image is not satisfied with the constant length in the longitudinal direction. Therefore, when the horizontal frame line portion 10 of the aspect shown in FIG. 5B is detected, the foreign matter adhesion determination unit 8 determines that the foreign matter adheres to the lens 3. As shown in FIG. 5, the object detection unit 6 detects a plurality of sets of a plurality of horizontal frame portions 10 in a row state in the image (in FIG. 5, two sets of a front side set and a back side set). If this is the case, the foreign matter adhesion determination unit 8 may perform a foreign matter adhesion determination based on the constant length of each of the detected horizontal frame portions 10. Then, when at least one set of judgment results indicating that foreign matter is attached to the lens 3 is included in the judgment results for each set, the judgment result is taken as the judgment result for the entire lens 3. You should consider it.

Furthermore, in addition to this, when the object detection unit 6 detects each of the vertical frame lines 14 (see FIG. 5) of the plurality of parking frames, the foreign matter adhesion determination unit 8 detects each of the detected objects. When the vertical frame line portion 14 has the constant formation interval in the horizontal width direction of the vertical frame line portion as the predetermined constantity, it may be determined that no foreign matter is attached to the lens 3. On the other hand, the foreign matter adhesion determination unit 8 determines that foreign matter is attached to the lens 3 when each of the detected vertical frame line portions 14 does not have a constant formation interval in the width direction. Just do it.

Further, in the case of adopting a configuration in which both the horizontal frame line portion 10 and each vertical frame line portion 14 are detected as predetermined detection objects and the foreign matter adhesion determination is individually performed based on each, one of them is used. If foreign matter is found to adhere in the determination, this may be regarded as the determination result of the entire lens 3.

According to the above configuration, the presence or absence of constantity in the length or formation interval of the lane or frame line portion detected based on the captured image and / or the presence or absence of continuity in the shape is used as a judgment criterion. The presence or absence of raindrops or dirt adhering to the lens 3 can be appropriately determined. Further, since it is only necessary to evaluate the length / shape of the lane or the frame line portion, it is possible to reduce the processing load as compared with the conventional case where the determination is performed for the entire image. Further, since it is not necessary to store the reference video as in the conventional case, it is not necessary to increase the memory capacity.

Further, in addition to the above configuration, the object detection unit 6 is capable of detecting a predetermined obstacle on the detection target, and the foreign matter adhesion determination unit 8 detects the obstacle by the object detection unit 6. If this is the case, the detection target may be configured not to be used for determining the adhesion of foreign matter. In this case, a known image recognition technique such as detection of a predetermined feature point according to the obstacle may be used for detecting a predetermined obstacle. In addition, examples of predetermined obstacles include other vehicles and pedestrians. With such a configuration, it is possible to effectively prevent erroneous determination caused by an obstacle.

Further, as a detection target on the parking area, a car stop may be adopted instead of the horizontal frame line portion.

Returning to FIG. 1, a determination result reflection processing unit 9 is connected to the foreign matter adhesion determination unit 8, and the determination result of the foreign matter adhesion determination is transmitted to the determination result reflection processing unit 9 from the foreign matter adhesion determination unit 8. It is supposed to be entered. Then, the determination result reflection processing unit 9 performs a process (hereinafter, referred to as a reflection process) that reflects the input determination result. As an example of this reflection process, when the determination result of the foreign matter adhesion determination unit 8 indicates the adhesion of foreign matter, the reliability of the image recognition result for the image captured by the in-vehicle camera 2 is lowered by reflecting this. It is possible to consider a process of outputting an alarm notifying the fact via the display unit 11 or the voice output unit 12. Further, as an example of other reflection processing, in a mode in which the lens deposit determination device 1 for an in-vehicle camera is mounted on a drive assist system provided with an alarm output function for preventing lane departure, foreign matter is detected by the foreign matter adhesion determination unit 8. When a determination result indicating that the lens is attached is obtained, it is possible to consider a process of canceling each process for alarm output by reflecting this. For example, the lane departure warning determines whether or not there is an output based on the relationship between the image-recognized lane (white line) and the position of the own vehicle, but the foreign matter adhesion determination unit 8 indicates that foreign matter is attached. If the determination is made, the image recognition processing of these lanes and the determination processing of the presence / absence of alarm output may be stopped. Incidentally, since the in-vehicle camera lens deposit determination device 1 in the present embodiment includes an object detection unit 6 that detects a lane (image recognition), the detection result of the object detection unit 6 is used for lane departure prevention. It can be used to determine whether or not the alarm is output.

By the way, in the present embodiment, it is not possible to determine the presence or absence of foreign matter adhering to a portion other than the capture portion of the lane or the frame line portion of the lens 3. However, for the output function of the lane departure warning in the drive assist system, only the positional relationship between the own vehicle and the lane or the border is important, so the influence of foreign matter in other parts can be ignored. it can.

Further, each component 4 to 9 of the above-mentioned lens deposit determination device 1 for an in-vehicle camera is, for example, a CPU that executes the functions of each component 4 to 9, and a ROM that stores an execution program and data of the CPU. It may also be configured by RAM or the like used for temporary storage of CPU processing results and the like.

Next, the operation of this embodiment will be described with reference to FIG.

In the present embodiment, first, in step 1 (ST1) of FIG. 6, the camera image acquisition unit 4 acquires the captured image of the vehicle-mounted camera 2.

Next, in step 2 (ST2), the distortion correction unit 5 performs distortion correction on the captured image acquired in step 1 (ST1).

Next, in step 3 (ST3), the object detection unit 6 determines whether or not the vehicle position is on the road based on the detection result of the vehicle position detection unit 7. Then, if a positive determination result is obtained in step 3 (ST3), the process proceeds to step 4 (ST4), and if a negative determination result is obtained, the process proceeds to step 10 (ST10).

Here, the processing after step 4 (ST4) will be described first. First, in step 4 (ST4), the object detection unit 6 detects the lane.

Next, in step 5 (ST5), the foreign matter adhesion determination unit 8 determines whether or not a lane has been detected. Then, if a positive determination result is obtained in step 5 (ST5), the process proceeds to step 6 (ST6), and if a negative determination result is obtained, the process ends.

Next, in step 6 (ST6), the foreign matter adhesion determination unit 8 determines whether or not the lane determined to be detected in step 5 (ST5) has a constant length in the longitudinal direction of the lane. .. However, the determination in step 6 (ST6) may be performed after waiting for it to be determined in step 5 (ST5) that a lane in a series of images for a predetermined number of frames has been detected. Then, if a positive determination result is obtained in step 6 (ST6), the process proceeds to step 7 (ST7), and if a negative determination result is obtained, the process proceeds to step 8 (ST8).

Here, if the process proceeds to step 7 (ST7), in this step 7 (ST7), the foreign matter adhesion determining unit 8 determines that no foreign matter is attached to the lens 3 and ends the process.

On the other hand, when the process proceeds to step 8 (ST8), the lane determined by the foreign matter adhesion determination unit 8 in step 8 (ST8) to be detected in step 5 (ST5) has a shape in the longitudinal direction of the lane. Determine if it has continuity. The order of the determination in step 8 (ST8) may be different from the determination in step 6 (ST6). Then, if a positive determination result is obtained in step 8 (ST8), the process proceeds to step 7 (ST7), and if a negative determination result is obtained, the process proceeds to step 9 (ST9).

Next, in step 9 (ST9), the foreign matter adhesion determination unit 8 determines that foreign matter is attached to the lens 3 and ends the process.

Next, the processing after step 10 (ST10) will be described. First, in step 10 (ST10), the vehicle position is set to the parking area based on the detection result of the vehicle position detection unit 7 by the object detection unit 6. Determine if it is on top. Then, if a positive determination result is obtained in step 10 (ST10), the process proceeds to step 11 (ST11), and if a negative determination result is obtained, the process ends.

Next, in step 11 (ST11), the object detection unit 6 detects each horizontal frame line portion in the plurality of parking frames.

Next, in step 12 (ST12), the foreign matter adhesion determination unit 8 determines whether or not a plurality of horizontal frame lines have been detected. Then, if a positive determination result is obtained in step 12 (ST12), the process proceeds to step 13 (ST13), and if a negative determination result is obtained, the process ends.

Next, in step 13 (ST13), whether all of the horizontal frame lines determined to be detected in step 12 (ST12) by the foreign matter adhesion determination unit 8 have a constant length in the longitudinal direction. Judge whether or not. Then, if a positive determination result is obtained in step 13 (ST13), the process proceeds to step 7 (ST7), and if a negative determination result is obtained, the process proceeds to step 9 (ST9).

The execution period of each process of FIG. 6 can be variously set according to the concept. For example, with the Acc of the own vehicle turned on, the operation may be repeated continuously, or may be performed regularly.

As described above, according to the present invention, the presence or absence of a predetermined constantity regarding the length or formation interval of the lane / frame portion detected based on the captured image and the predetermined shape of the detected lane. By using at least one of the presence or absence of continuity as a judgment criterion, it is possible to easily, lowly, and appropriately judge whether or not foreign matter is attached to the lens 3 of the vehicle-mounted camera 2.

The present invention is not limited to the above-described embodiment, and various modifications can be made as long as the features of the present invention are not impaired.

For example, the determination result of the lens deposit determination device 1 for an in-vehicle camera in the present embodiment may be reflected in the display of the so-called top view (or around view) monitor screen. That is, in general, in order to generate the monitor screen, images taken by a plurality of (for example, the above-mentioned four) in-vehicle cameras 2 are combined after performing image processing such as distortion correction and viewpoint conversion. However, the captured image of the in-vehicle camera 2 determined to have foreign matter attached by the foreign matter adhesion determination unit 8 may be hidden on the monitor screen, or the detection target object on the monitor screen ( The shape of (white line, etc.) may be corrected based on the captured image of another in-vehicle camera 2. At this time, for the video portion on the monitor screen corresponding to the in-vehicle camera 2 determined to have foreign matter attached, an alarm message indicating that the reliability is low is superimposed so as to positionally correspond to this video portion. It may be displayed. Alternatively, if there is even one in-vehicle camera 2 that is determined to have foreign matter attached, the display of the monitor screen may be completely prohibited and only the alarm message may be displayed.

1 Lens deposit judgment device for in-vehicle cameras 2 On-board camera 3 lenses 6 Object detector 8 Foreign matter adhesion judgment unit