Device for detecting the presence of objects
Abstract
Device for detecting the presence of objects, of the type that are mounted on a motor vehicle, which has at least one dead angle, where the detection device is capable of detecting an object located in the dead angle and comprises: a receiver capable of detecting electromagnetic waves, with a focusing device, and a photosensor that transforms said received electromagnetic waves into electrical signals, an electronic circuit that transforms electrical signals into digitized signals, a logical circuit that analyzes the digitized signals for analyze the presence of objects in the dead angle with a relative movement with respect to said vehicle, and that emits variable output signals depending on the result of the analysis, [d] indicating elements, activated by the output signals, suitable for being perceived by the driver.

Term
Term ended
Projected expiry passed 16 February 2021, 5.6 years ago.
- Priority
- Filed
- Published
- Projected expiry
- Today
19 claims: 1 independent, 18 dependent
- 1ES 2 273 802 T3 REIVINDICACIONES 1. Método de detección de presencia de objetos mediante el uso de un dispositivo de detección del tipo que va montado en un vehículo automóvil, presentando dicho vehículo automóvil por lo menos un ángulo muerto, donde dicho dispositivo de detección es apto para detectar un objeto tal como un vehículo que se aproxima situado en dicho ángulo muerto y para distinguirlo de otros objetos, que comprende las etapas de:detectar unas ondas electromagnéticas mediante un receptor (7) que comprende un dispositivo focalizador y un detector de luz que incluye un conjunto de elementos fotosensores activos que transforman dichas ondas electromagnéticas en señales eléctricas, definiendo dicho fotosensor una superficie de imagen;transformar dichas señales eléctricas en señales digitalizadas mediante un circuito electrónico (9);analizar dichas señales digitalizadas mediante un circuito lógico (15) para analizar la presencia de objetos en dicho ángulo muerto, que es al menos uno, con un movimiento relativo respecto de dicho vehículo, y generar unas señales de alarma, de salida, variables en función del resultado de dicho análisis, y activar unos elementos indicadores mediante dichas señales de salida, caracterizado porque dicha distinción entre un vehículo y otros objetos se realiza a base de reconocer unas aristas, claramente marcadas en al menos una primera parte de dicha superficie de imagen de interés en sentido vertical y/o horizontal, y a base de seguir el movimiento de dichas aristas a lo largo de diversas imágenes sucesivas para calcular una velocidad relativa entre un objeto detectado que proporciona dichas aristas y el vehículo a motor equipado con dicho dispositivo de detección.
- 2Método según la reivindicación 1, caracterizado porque en una segunda parte de dicha superficie de imagen se emplea una técnica diferente basada en una diferencia de fase para obtener una estimación del flujo óptico en una dirección determinada.
- 3Método según la reivindicación 1, caracterizado porque dicho circuito lógico (15) efectúa el reconocimiento de aristas realizando una convolución matemática a lo largo de al menos dicha primera parte de dicha superficie de imagen de la señal digitalizada, distinguiendo en consecuencia un vehículo de otros objetos.
- 4Método según la reivindicación 3, caracterizado porque dicha convolución matemática a lo largo de al menos dicha primera parte de dicha superficie de imagen se realiza mediante una convolución de núcleo apropiada para una detección de movimiento.
- 5Método según la reivindicación 3, caracterizado porque dicha convolución se calcula en un tiempo menor de 100 ms.
- 6Método según la reivindicación 1, caracterizado porque:dicha primera parte, que es al menos una, de dicha imagen de interés se define mediante una máscara controlada por un programa informático y porque un algoritmo calcula las aristas de la imagen y sus direcciones, siendo dichas aristas verticales y horizontales normalizadas e integradas a lo largo de los ejes vertical y horizontal, respectivamente;se ajusta dinámicamente un factor de normalización sobre la base de la densidad media de dichas aristas, empleando la proyección unidireccional de dichas aristas sobre los ejes de coordenadas correspondientes para trazar una trayectoria según dicho eje, con lo que los objetos que se aproximan generan puntos de proyección sobre dicho eje con velocidades positivas, y dichos puntos de proyección se separan de otros puntos que están fijos o tienen movimientos relativos con la ayuda de filtros de dirección, identificando y seleccionando finalmente las trayectorias resultantes.
- 7Método según la reivindicación 6, caracterizado porque:en cuanto se ha identificado la trayectoria de un vehículo que se aproxima mediante el desplazamiento de sus correspondientes proyecciones unidireccionales, se calcula y supervisa la velocidad de dicho vehículo para que sea posible detectar situaciones en las que el vehículo que se aproxima reduzca su velocidad y circule a la misma velocidad que dicho vehículo equipado con dicho dispositivo, quedándose a poca distancia del mismo, o vehículos que circulan por diversos carriles a prácticamente la misma velocidad, y se emite un tipo específico de señal de alarma hasta que se observen cambios sustanciales en el entorno de dicha imagen de un vehículo que se aproxima.
- 8Método según la reivindicación 6, caracterizado porque:ES 2 273 802 T3 dicha máscara controlada desde un programa informático que define dicha superficie de imagen de interés está colocada de tal modo que un coche ubicado en el carril de adelantamiento de una carretera recta y a mucha distancia se sitúa en el extremo superior izquierdo de dicha máscara, y la posición de la máscara en relación con la imagen puede ajustarse para conseguir un ajuste preciso del campo de visión.
- 9Método según la reivindicación 2, caracterizado porque, a fin de estimar un flujo óptico en dicha dirección de un eje de una carretera, se procesa un par de imágenes sucesivas y se filtran y eliminan los elementos visuales que no avanzan, y porque dicho procedimiento es opcional y puede no emplearse.
- 10Método según la reivindicación 7, caracterizado porque, en el caso de un giro del vehículo automóvil hacia la izquierda, se detecta el desplazamiento constante de unos elementos visuales contenidos en diversas imágenes en una parte superior de unas imágenes sucesivas mediante una técnica basada en correlaciones y se utiliza la coherencia de dicha señal durante varias imágenes sucesivas como indicación de que el vehículo equipado con dicho dispositivo está girando hacia la izquierda, con lo que, cuando el campo de detección se mueve hacia una posición más cercana a dicho vehículo equipado con dicho dispositivo, se imponen unas condiciones más restrictivas antes de activar dichas señales de alarma.
- 11Método según la reivindicación 7, que además detecta si dicho vehículo equipado con dicho dispositivo ha iniciado acciones indicadoras de una aproximación a dicho objeto.
- 12Método según la reivindicación 11, caracterizado porque dichas acciones indicadoras comprenden la puesta en marcha de una luz intermitente.
- 13Método según la reivindicación 11, caracterizado porque dichas acciones indicadoras comprenden efectuar un giro de un volante.
- 14Método según la reivindicación 1, caracterizado porque dichas señales de alarma incluyen unas señales luminosas con por lo menos dos colores, donde cada color indica un nivel de aviso diferente.
- 15Método según la reivindicación 1, caracterizado porque incluye además unos medios de detección de somnolencia del conductor utilizando dicho dispositivo de detección y emitiendo una señal de alarma en función de la posición relativa del vehículo equipado con dicho dispositivo de detección con respecto a las líneas de marcado de los carriles sobre el pavimento de la carretera.
- 16Método según la reivindicación 1, caracterizado porque dicho conjunto de elementos fotosensores se integra en una cámara digital montada en un espejo retrovisor de un vehículo y porque dicha cámara se coloca de tal modo que:un borde vertical izquierdo de dicha superficie de imagen está cerca de un borde lateral de un vehículo equipado con dicho dispositivo;un borde superior de dicha superficie de imagen está situado ligeramente por encima de la línea del horizonte, en aproximadamente una octava parte de dicha imagen, y dicha cámara está ligeramente inclinada en el sentido de las agujas del reloj para así alinear dicha superficie de imagen a lo largo de un eje de la carretera.
- 17Dispositivo de detección de presencia de objetos que pone en práctica un método según la reivindicación 1, siendo dicho dispositivo de detección del tipo que van montados en un vehículo automóvil, presentando dicho vehículo por lo menos un ángulo muerto, donde dicho dispositivo de detección es apto para detectar un objeto tal como un vehículo que se aproxima situado en dicho ángulo muerto y para distinguirlo de otros objetos, que comprende:[a] un receptor (7) apto para detectar unas ondas electromagnéticas, comprendiendo dicho receptor (7) un dispositivo focalizador y un detector de luz que incluye un conjunto de elementos fotosensores activos que transforman dichas ondas electromagnéticas en señales eléctricas, definiendo dicho detector de luz una superficie de imagen;[b] un circuito electrónico (9) que transforma dichas señales eléctricas en señales digitalizadas;[c] un circuito lógico (15) que analiza dichas señales digitalizadas para analizar la presencia de objetos en dicho ángulo muerto, que es al menos uno, con un movimiento relativo respecto de dicho vehículo, y generar unas señales de salida variables en función del resultado de dicho análisis;[d] unos elementos indicadores activados mediante dichas señales de salida, caracterizado porque ES 2 273 802 T3 dichos elementos fotosensores activos se integran en una cámara digital;dicha cámara digital se coloca de tal modo que: - un borde vertical izquierdo de dicha superficie de imagen está cerca de un borde lateral de un vehículo equipado con dicho dispositivo;- un borde superior de dicha superficie de imagen está situado ligeramente por encima de la línea del horizonte, en aproximadamente una octava parte de la imagen;- dicha cámara está ligeramente inclinada en el sentido de las agujas del reloj para así alinear dicha superficie de imagen a lo largo de un eje de la carretera;dicho dispositivo comprende una máscara controlada por un programa informático que permite realizar una selección de las partes de una superficie de interés que definen una zona de interés;dicho detector de luz está constituido por unos elementos sensores activos que presentan un rango dinámico igual o superior a seis décadas, o sea que entre un valor umbral de detección mínimo y un valor umbral de saturación hay un rango de seis décadas, expresándose la intensidad de la luz en lux, en una misma superficie de imagen, y un mismo elemento sensor presenta un rango dinámico igual o superior a las seis décadas, tal como se define anteriormente, entre dos imágenes consecutivas.
- 18Dispositivo de detección según la reivindicación 17, caracterizado porque dicha cámara digital se aloja en una unidad de espejo retrovisor exterior del vehículo.
- 19Dispositivo de detección según la reivindicación 18, que incluye una cámara digital en cada uno de dichos espejos retrovisores exteriores de dicho vehículo.
Independent claims19
85 paragraphs in 5 sections, as filed
ES 2 273 802 T3
DESCRIPTION
Object presence detection device.
Field of the invention
The invention relates to a method for detecting the presence of objects and to a device that implements said method, said device being of the type that is mounted on a motor vehicle, which has at least one blind angle, where the detection device it is able to detect an object in the blind spot. Background of the invention
Conventional motor vehicles usually have rear-view mirrors, generally one internal and one or two external, which allow the user or driver to see behind without the need for the user to turn around. However, despite having a plurality of mirrors, there are usually areas, called blind spots, that are not covered by said mirrors.
It is known to use systems that capture an image oriented towards a blind spot by means of a CCD camera and that show it to the user through a screen placed in the passenger compartment of the vehicle. These systems allow the user to see the blind spots without having to sit up, however, they have a series of drawbacks: they require image transmission systems with sufficient quality for the user to perceive a clear image, which requires working with a high number of pixels, you must have space in the cabin to be able to place the corresponding screen, the system does not process the image, but only transmits it, etc. They are, therefore, expensive systems that do not actively collaborate in the detection of risk situations.
Document US 5 424 952 A describes an apparatus for monitoring the environment of a vehicle comprising all the features of the preamble of claims 1 and 17.
Document EP 0 591 743 A describes a device for detecting relative positions between vehicles, mainly for the prevention of collisions, in which said device is associated with an exterior rear-view mirror of a motor vehicle and comprises an optical sensor of the CCD type, as well as an electronic processing unit connected to the on-board information system, to detect and indicate the presence of other approaching vehicles and the risk of collision with the vehicle on which said device is mounted, based on a prediction of the vehicle's driver's intentions.
Document US 5 521 633 A describes an obstacle monitoring system for a motor vehicle that employs optical flow processing for moving vehicles, thereby allowing the presence of danger to be determined from the magnitude of a flow vector optical.
Document US 5 699 057 A describes a warning system for a vehicle comprising a pair of stereoscopic cameras, image recognition means and two warning devices arranged on the left and right side of the driver.
All the devices and methods of these documents clearly differ from the one proposed by the present invention.
Summary and object of the invention
The object of the invention is to overcome some drawbacks and limitations of the cited state of the art. This object is achieved by means of a method and device for detecting the presence of objects according to the characterizing part of claims 1 and 17.
Indeed, such a detection method and device captures the blind spot image and analyzes it, informing the driver of the result of the analysis. This has a series of advantages: the driver is not shown the image of the blind spot, so it is not necessary to have space in the passenger compartment for a screen, additionally the driver is given information of "higher value", in the sense that, by the detection device, an analysis work has already been done, and the result of the analysis is communicated to the driver or user. On the other hand, the detection device requires receivers with a smaller number of pixels than is necessary to present the user with an image of the blind spot with a minimum quality, so the detection device can be equipped with more economical receivers without lose benefits.
The detection device not only analyzes the presence of an object in the blind spot, but gives a qualitative idea of the relative speed of the object with respect to the vehicle and, therefore, determines whether the object is approaching or moving away, and with a rough idea of speed. This allows the information provided to the driver to be more complete, since he can distinguish different levels of risk depending on the relative speed of the object.
ES 2 273 802 T3
The detection device also gives a qualitative idea of the relative distance of the object from the vehicle and thus determines the position of the object relative to the vehicle. This allows the information provided to the driver to be more complete, since he can distinguish different levels of risk depending on the position of the object.
The photosensor is a set of active photosensor elements, which are advantageously photodiodes, distributed according to a flat, two-dimensional matrix that defines rows parallel to each other. In this way, the array of sensor elements defines the image surface, which is made up of a plurality of pixels, where each pixel corresponds to a sensor element.
Photodiodes transform electromagnetic waves into an electrical current. This electrical current is preferably transformed into an electrical voltage and amplified.
Active photosensing elements preferably have a dynamic range equal to or greater than six decades (10<sup>6</sup> = 120 dB) on the same image surface, that is, between the minimum detection threshold value and the saturation threshold there is a range of six decades, expressing light intensities in lux. Likewise, a pixel that at a given moment is receiving the minimum detection value, can detect in the next image acquisition a value that is six decades higher, and vice versa. This allows the receiver to work in multiple light conditions, and even in adverse light conditions, with strong light contrasts, such as night driving. For the same reason, it is preferred that the same sensor element has a dynamic range equal to or greater than six decades between two consecutive images.
Preferably, the electronic circuit can make a selection of each of said sensor elements by activating the corresponding row and the corresponding position in said row, thus being possible to select any sensor element after any sensor element. In this way, the electrical signal from each of the sensor elements can be taken and all the pixels that make up the image surface can be sequentially amplified and digitized. Alternatively it is also possible that the electronic circuit simultaneously transforms all the electrical signals of a row of sensor elements to digitized signals. In each specific design, the higher cost of this solution must then be assessed against the faster digitization of the image.
As already indicated above, one of the objectives of the present invention is to be able to use receivers with low-cost photosensors. In this sense, it is preferred that the matrix of sensor elements is at most 512 x 512 sensor elements, and very preferably that it is at most 320 x 256 sensor elements. In general, these values refer to the number of active sensors for image processing. That is, it is possible that the sensor element array has more sensor elements, but they are not activated for image processing.
Once the image has been digitized, the logic circuit analyzes the image surface. To this end, it preferably performs a mathematical convolution, in particular a convolution kernel suitable for motion detection, along the entire image surface of the digitized signal or along a part of it.
The logic circuit preferably comprises a specialized electronic circuit that includes: [a] a sequential central processing unit (CPU) of the von Neumann type, [b] a parallel coprocessor, specialized in calculating the convolution over the entire image surface, and including at least 32 parallel accumulator multipliers with a high calculation speed suitable for calculating the convolution directly on the image surface at a calculation speed such that the convolution is completed before starting a new image acquisition, and [c] a local RAM memory. In particular, it is preferred that the calculation speed is such that it allows a convolution to be calculated in a time of less than 100 ms.
The detection device is capable of distinguishing a vehicle from other objects. This is achieved, for example, by recognizing edges or edges, forming rectangles with said edges and comparing said rectangles with patterns. When it has detected a vehicle, it analyzes from the following image the relative speed between the detected vehicle and the vehicle carrying the detection device.
A preferred embodiment of the invention provides for dividing the image surface into at least two parts, and employing different analysis techniques in each of said parts. Thus, in one of these parts the technique consists of the one already indicated in the previous paragraph, that is, in the recognition of the edges, the formation of rectangles, the comparison of the rectangles with some patterns and the comparison of two consecutive images to calculate the relative velocity, while in another part a technique based on a phase difference is used to obtain an estimate of the optical flow in a given direction, specifically in the direction of the street or road on which the vehicle is traveling.
The applied analysis technique consists of detecting clearly marked vertical and / or horizontal edges on the road image. Advantageously, this technique also includes the tracking, along successive images, of the movement of said edges, and the calculation, based on said tracking, of the relative speed between the detected object (vehicle) and the vehicle carrying the detection device. . This technique is described in more detail later.
ES 2 273 802 T3
It is also possible that in some of the parts in which the image surface has been divided, more than one analysis technique is used simultaneously.
In general, the detection device should provide a warning signal when it detects a situation where there is a risk of collision. This signal should be used to give the driver time to avoid or correct a dangerous maneuver. In this sense, it is evident that the warning signal must be activated with enough time for the driver to react appropriately. If a situation is considered in which a vehicle enters a motorway, which represents an extreme situation in terms of the relative speed between the incoming vehicle and the vehicles traveling on the motorway, it is understood that the detection device must have a high radius of action, to be able to warn the driver in good time. For this reason, it is preferable that the radius of action of the detection device is greater than 15 m, or better still, greater than 20 m. In this sense, the detection device covers a field of view larger than strictly the blind spot. Thus, the detection device can detect risk situations and alert the driver even if the risk situation is detectable through the rear-view mirror. In this way, the detection device collaborates in a broader way in safety while driving the vehicle.
The focusing device can comprise any common optical element that is obvious to a person skilled in the art. In particular, it may have a lens or a microlens integrated in the integrated circuit that includes the focusing device. It is also possible to include an electromagnetic wave transmission guide. This would, for example, make it possible to place the entire detection device at any point inside the vehicle, and connect it to the outside through said guide. However, the reduced dimensions of the detection device allow it to be placed inside a rear-view mirror, which constitutes a preferred embodiment, or it is even possible to place a detection device in each of the exterior rear-view mirrors of a vehicle.
To achieve reduced sizes, at the same time as low consumption and simplification in communications between the different components of the detection device, it is recommended that the electronic circuit and the photosensor be of CMOS, DMOS, MOS, Si-Ge technology. BiCMOS or SOI (silicon on insulator) technology, and that the photosensor and the electronic circuit are physically united in a multi-chip module (MCM, multi-chip module) on a plastic material substrate, fiberglass (FR4), ceramic or silicon.
Optionally, the ability of the detection device to analyze risk situations can be improved if, to the detection characteristics of an approaching object, the ability to detect if the vehicle on which the detection device is mounted has been added has initialized actions indicating an approach to the object. In particular, it is advantageous that the detection device is capable of detecting the start of a flashing light and / or that it is capable of detecting a turn in the steering wheel of the vehicle.
It is also interesting that the detection device is capable of communicating various signals to the user or driver of the vehicle, which allow the warning signal to be nuanced according to the risk of collision. Thus, it is preferable that the indicator elements include light signals with at least two colors, where each color indicates a different warning level. It is also advantageous to include an output element that allows the representation of pictograms, where said output element is a matrix of LEDs or a graphic display.
Likewise, there may be a risk situation if a passenger of the vehicle carrying the detection device opens a door without looking to see if another vehicle is approaching from behind. It is therefore advantageous that the detection device also indicates such risk situations to the passengers of the vehicle.
It is advantageous to allow the detection device to act on the closing of the doors. Thus, for example, you can lock a door if you detect a risky situation.
Finally, it is advantageous to add a driver drowsiness detection device to the object presence detection device. Preferably, the drowsiness device shares most of the physical devices with the object detection device and emits an alarm signal based on the relative position between the vehicle carrying the detection device and the lane marking lines of the vehicle. highway.
Other advantages and characteristics of the invention can be seen from the following description, in which, without any limitation, a preferred mode of carrying out the invention is reported, making mention of the accompanying drawings.
Brief description of the drawings
Figs. 1A-1D show a diagram of the blind spots of a vehicle, the areas of direct vision and through the left rear-view mirror, and the area covered by a detection device according to the invention;
Fig. 2 is a simplified diagram of a detection device according to the invention;
Fig. 3 is a front elevation view of a rear view mirror showing 5 possible receiver locations;
Fig. 4 is a schematic of an image surface;
Fig. 5 illustrates the image surface of Fig. 4, divided into three parts, and Fig. 6 is a block diagram of an algorithm according to the invention.
Description of preferred embodiments
By way of example, in Figs. 1A to 1D schematically shows the areas visible through the rear-view mirror 1 on the left side (driver's side), the areas visible thanks to the driver's side peripheral vision 3, and the blind spots 5. The areas visible through the mirrors Mirrors 1 must meet a number of legal requirements, for example those defined in EC 71/127 and in the following directives. In particular, as shown in Fig. 1A, the angle of view must be such that at a distance of 10 m from the rear-view mirror, the width of the viewing area is at least 2.5 m. In Fig. 1A a shaded rectangular area is shown that corresponds to the legal requirement, and a triangular area that corresponds to the actual view through a conventional rear-view mirror that meets the legal requirement.
These blind spots 5 are precisely those to be covered with the detection device. The detection device must also partially overlap with the area seen by the rear-view mirror, in order to avoid discontinuities between what the sensor detects and what the driver sees. For this same reason it is desirable that the sensor also covers part of the area seen directly by the driver. In this sense, a possible solution is to use a detection device that covers an area like the one shaded in Fig. 1C: an area in the shape of a right triangle whose legs are both 4.5 m, followed by a rectangular area of 4 , 5 m wide. The total length of the area covered may depend, depending on the performance of the detection device. As an example, in Fig. 1C has shown a range of 20 m, although the detection device described below has a range of more than 20 m.
In Fig. 1D all the above areas have been represented together. It can be seen that the blind spot is practically completely covered, as regards the area corresponding to the adjacent lane. There is also an overlap with the areas seen directly or through the rear-view mirror.
The detection device according to the invention shown in Figs. 2 to 5 comprises a receiver 7, which is formed by an array of 256 rows of photodiodes, with 320 photodiodes in each row. Receiver 7 receives electromagnetic waves from outside, in this particular case within the range of visible light, conveniently focused thanks to a lens. When the light falls on the photodiodes, they generate an electric current whose intensity is a function of the intensity of light received. This electrical current is converted to an electrical voltage. By selecting a row and a position within the row, a particular photodiode can be selected, which thus transmits the electrical signal to an electronic circuit 9. The electronic circuit 9 has an amplification stage 11, and an analog-digital conversion unit ADC, from which a digitized signal comes out.
The digitized signal is introduced into a logic circuit 15. The logic circuit 15 comprises a central processing unit CPU sequential of the Von Neumann type, a parallel coprocessor TOT that calculates the convolution and that is supported by an auxiliary memory MEM, a permanent memory FLASH and a RAM memory, fast access (SRAM). The central processing unit CPU also controls the receiver 7, sending the row 17 selection and position signals within the row 19 to the corresponding registers, and to the electronic circuit 9.
The receiver 7 captures an image, including the dead angle, which is projected onto the image surface formed by the photodiodes. This image surface is what is transmitted to logic circuit 15 as a series of digitized pixels. The receiver 7 is oriented in such a way that the lateral edge of the image surface is practically flush with the side surface of the car 21 and the upper edge of the image surface is flush with the horizon 23. The logic circuit 15 determines the direction of movement along the street or highway, allowing it to determine whether a detected movement is in the direction of the road or if it is in another direction, for example vertical. In this way you can filter “noise”, such as rain, snow, vehicles in the opposite direction, and so on.
In Fig. 3 some examples of positioning of the receiver 7 or, where appropriate, of the end of the electromagnetic waveguide, in an exterior rear-view mirror are shown.
As already indicated above, it is possible to perform various image analysis algorithms. In one case, the image surface is divided into two parts 25, 27, which have an overlapping area 29, as shown in Fig. 5. The logic circuit 15 has two independent algorithms: a vehicle detection algorithm, which is applied in part 25, and a motion detector algorithm, which is applied in part 27. In the overlap area 29 both algorithms are applied. The vehicle detection algorithm recognizes the edges of figures on the image surface, selects the horizontally and vertically arranged edges, and compares it to patterns to determine if an object with a shape similar to a vehicle exists. If so, the next image obtained by the receiver 7 is analyzed, which makes it possible to determine the direction of movement, as well as the speed of the object. The motion detector algorithm is based on a phase difference technique to obtain an estimate of the optical flow in the direction of the road. The result is compared with the results obtained in previous images, to eliminate errors and noise by means of a consistency check.
ES 2 273 802 T3
Another possible image analysis algorithm is based on the following. As has already been said, the device is designed to detect vehicles passing the vehicle carrying the device by means of the series of images captured with a digital camera, for example a CMOS camera, arranged on the rear view mirror of a vehicle.
The presence of an approaching vehicle is based on the detection and tracking of objects that move along the axis of the road (generally any public road) in the approach direction with respect to the vehicle carrying the device. From an image, the presence of a vehicle can be appreciated by the presence of edges (or edges) clearly marked vertically and horizontally on the pavement of the road. In successive images these visual elements (the vertical and horizontal edges) move forward if they are part of an approaching vehicle. On the contrary, they move backwards if they are part of static objects (such as elements of the road, protective fences, trees, traffic signs, milestones, etc.) or if they belong to vehicles that move in the opposite direction than the vehicle carrying the vehicle. device. Therefore a coherent forward movement is interpreted as a passing vehicle.
This interpretation is generally correct on motorways or similar roads, where the lanes are clearly defined and the curves are usually wide radii. In these cases the image is a simple perspective and the passing lane can be easily isolated from the rest of the scene using an appropriate mask. Hence forward movement in the passing lane is a clear indication of an approaching vehicle. Noise and interference due to potholes or sudden movements of the vehicle carrying the device can be eliminated by forcing the forward movement to be coherent throughout several successive images.
The visual image on a road other than a highway or the like is much more complex. In particular, left turns of the vehicle carrying the device can generate a consistent apparent movement that can generate false alarms. This is particularly common in urban environments, where the visual scene has a large number of objects (parked cars, buildings, various traffic signs, etc.) that have sharp edges. Furthermore, the actual distance between the approaching vehicle and the vehicle carrying the device cannot be estimated correctly from its position, since the lanes are not well defined. For this reason, it is convenient that the detector device has a specific operating module for when the carrier vehicle turns left. In this way, during a left turn, the detection field moves to a position closer to the carrier vehicle and more restrictive requirements are imposed before activating the alarm signal. As a consequence, the alarm signal will be activated when the approaching vehicle is closer to the carrier vehicle. However, this is not a problem because during tight curves vehicle speeds are slower than on motorways or other expressways. Furthermore, given the street configuration and the frequent presence of intersections, a long detection range is not necessary in the case of an urban environment.
As a concrete example, the camera may have a sensor that is a 320 x 256 CMOS matrix with a high dynamic range (120 dB). The size of the processed images is at least 128 x 128 pixels. The camera's field of view is approximately 55 °. The camera is positioned in such a way that:
- the left vertical edge of the image is close to the lateral edge of the carrier vehicle;
- the upper edge of the image is slightly above the horizon line, about one eighth of the image;
- the camera is tilted slightly clockwise so that it aligns the image along the axis of the road.
A mask is used, which is controlled by the software, which delimits the region of interest of the images. The mask is positioned in such a way that a car positioned in the overtaking lane of a straight road and at a great distance is positioned in the upper left corner of the mask. The position of the mask in the image can be adjusted to achieve a fine adjustment of the field of view.
For the detection device to function properly, the image acquisition speed should preferably be greater than 40 images per second, since in this way the device is able to follow the path of approaching vehicles with greater precision. The detection device algorithm has basically four main modules:
- An optical flow detection module. The algorithm uses a phase difference-based technique to produce a dense estimate of the optical flow in the direction of the road axis. For this, a couple of successive images are processed. Visuals that are not moving forward are filtered out and removed. The resulting images are fed to the following modules. This module is optional and may not be used.
- Vehicle detection and tracking module. In the region delimited by the mask, the algorithm calculates the edges of the image and their directions. Vertical edges and horizontal edges are normalized and are integrated along the vertical and horizontal axes, respectively. The normalization factor is dynamically adjusted based on the average edge density. The one-way projection of
ES 2 273 802 T3 said edges on the corresponding coordinate axis are used to trace the trajectory along said axis. Approaching objects generate projections with positive velocities, that is, to the right and down the image. These points are separated from other points that are stationary or that present relative movements based on directional filters. The resulting trajectories are identified and selected.
- Vehicle detection module without relative speed. Once the path of an approaching vehicle has been identified (by shifting its corresponding one-way projections) the vehicle speed is estimated and monitored. Therefore, it is possible to detect situations in which this approaching vehicle reduces its speed and circulates at the same speed as the vehicle carrying the device, staying at a short distance from the vehicle carrying the device. In these cases, it is possible to emit some type of specific alarm signal until it is observed that the environment of the image of the approaching vehicle undergoes substantial changes (moving away or overtaking the vehicle carrying the detection device). In other words, this module allows the control of traffic situations (for example dense traffic) in which parallel traffic takes place, that is, vehicles traveling on different lanes at practically equal speeds. In these cases, it is relatively common for a vehicle to be positioned in the blind spot of another vehicle, which can generate dangerous situations.
- Detection module during left turns. During turns to the left there is a constant and global displacement of the visual elements contained in different images. Using a correlation technique, constant displacements are detected in the upper part of successive images. The coherence of this signal during several successive images is used as an indication that the carrier vehicle is turning and, therefore, that the left turn detection module should be activated.
Figure 6 shows a block diagram showing the stages of the algorithm. The indicated references represent the following blocks:
6.1 - Starting the algorithm
6.2 - Image acquisition
6.3 - Estimation of the optical flow
6.4 - Left turn detection
6.5 - Vehicle detection and tracking
6.6 - Consistent forward movement?
6.7 - Activate alarm
First, the optical flow module 6.3 performs coarse filtering of the image flow based on the direction of movement. The left turn module 6.4 then alerts the system if the carrier vehicle is turning. The detection and monitoring system 6.5 then follows the trajectories of the moving objects and activates, if applicable, the corresponding alarm signal. Then, if the alarm is activated, the zero speed module is activated.
Two modes of operation can be set. If the carrier vehicle is not turning, only the projections of the trajectories along the horizontal axis are considered. If the tracking module detects a path longer than 15 images, an alarm signal is generated and gives an estimate of the relative distance and relative speed of the approaching vehicle. This indication is reliable in the case of flat and straight roads, such as motorways or the like.
If the left turn detector is activated, the requirements for the alarm to be activated are more stringent. The images are first filtered using the optical flow detector, in order to reduce noise, and the two projections (along the vertical axis and along the horizontal axis) are taken into consideration. Only if a visual effect is moving forward on both the X and Y axes is the alarm signal activated. This is done in this way since during the rotations the visual elements are characterized by having a positive speed along the X axis, but with an approximately zero speed along the Y axis, since their height is maintained. Additionally, the mask is lowered in position and moved to the right to cover the region of interest (the overtaking lane) of the images.
Next, the logic circuit, depending on the information obtained (vehicle presence, vehicle distance, and relative speed) activates, for example, a group of three LEDs (not represented in Figs.) Of three different colors (red , orange, green), which allows you to communicate different levels of warning, depending on the danger. A plurality of ways of presenting the warning levels are possible: from a single signal
ES 2 273 802 T3 red light, which is activated to indicate the presence of an object in the detection zone, up to complex devices, with various light, acoustic and tactile signals.
The detection device has a range of more than 20 m. In this way, in the situation indicated above by way of example, in which a vehicle wishes to enter a motorway, in which case there may be relative speeds of the order of 120 km / h, the driver receives the warning signal with almost 1 s of time.
In the event that the detection device is simultaneously mounted on two exterior rear-view mirrors of the vehicle (one on each side of the vehicle), it is possible to additionally add a driver drowsiness detection device to it. Preferably, the drowsiness detection device shares all the physical elements of the device for detecting the presence of objects that participate in capturing and processing images, such as the receiver, the electronic circuit and the logic circuit. Additionally, the drowsiness detection device has an algorithm that allows drowsiness to be detected in the manner described below.
By means of the images obtained through each of the object presence detection devices arranged in each of the rear-view mirrors, the marking lines of the lane on which the vehicle carrying the detection devices is traveling are detected. In this way it is possible to detect when the carrier vehicle crosses one of said marking lines. Indeed, as a result of drowsiness, the driver is no longer able to follow the lane, marked by the marking lines, and leaves it, creating a dangerous situation. The drowsiness detector is therefore capable of detecting this circumstance and of emitting an alarm signal.
To recognize these marking lines, the drowsiness detection device analyzes the image at the immediate rear of the car, extracts the edges of the marking line (the edges of the marking line) and follows them over time. The distance between the wheel and the edge of the marking line can be determined and thus it is possible to emit an alarm signal when said marking line is to be crossed. Preferably the drowsiness detection device is connected to the turn detection module, which allows it to identify the case in which an approach to the marking line occurs due to a curve being taken. Likewise, the drowsiness detection device receives information about the possible activation of the flashing lights, which allows it to discern between a voluntary crossing of the marking lines and an involuntary crossing or, at least, not notified.
If the drowsiness detection device detects an inadvertent crossing of a marking line, it activates a warning signal. This warning signal can be tactile (for example vibrations in the steering wheel), light and / or acoustic.
It is also possible to make a drowsiness detection device from a single object presence detection device, arranged in a single rear-view mirror, although in this case it is likely that its performance, in the sense of quality or relevance of the warning signals that it emits, are not the same.
The drowsiness detection device is always focused backwards, and covers exactly the same detection area of the object presence detection device, since it preferably shares with it all the physical detection and calculation elements.
Contents5
6 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6
16 members in 8 offices
Priority claims9
| Document | Office | Kind | Date |
|---|---|---|---|
| 20000000378 | Spain | – | |
| 200000378 | Spain | A | |
| 200000378 | Spain | A | |
| 0100057 | Spain | W | |
| 0100057 | Spain | W | |
| 2001ES00057 | World Intellectual Property Organization (WIPO) | – | |
| 20000037801902434 | – | – | – |
| ES20000000378 | – | – | – |
| WO2001ES00057 | – | – | – |
Members16
| Document | Office | Kind | |
|---|---|---|---|
| WO0161371A2 | World Intellectual Property Organization (WIPO) | A2 | |
| AU3026701A | Australia | A | |
| ES2158827A1 | Spain | A1 | |
| ES2158827B1 | Spain | B1 | |
| WO0161371A3 | World Intellectual Property Organization (WIPO) | A3 | |
| EP1271179A2 | European Patent Office (EPO) | A2 | |
| US2003141762A1 | United States of America | A1 | |
| JP2003523521A | Japan | A | |
| US6911642B2 | United States of America | B2 | |
| EP1271179B1 | European Patent Office (EPO) | B1 | |
| AT343144T | Austria | T | |
| ATE343144T1 | Austria | T1 | |
| DE60123936D1 | Germany | D1 | |
| ES2273802T3This record | Spain | T3 | |
| DE60123936T2 | Germany | T2 | |
| JP4235386B2 | Japan | B2 |
Numbers
- Publication
- 2273802
- Publication, DOCDB
- 2273802
- Publication, EPODOC
- ES2273802T
- Application
- 1902434
- Application, DOCDB
- 01902434
- Application, EPODOC
- ES20010902434T
Titles2
- Spanish
- DISPOSITIVO DE DETECCION DE PRESENCIA DE OBJETOS.
- English
- DEVICE FOR DETECTION OF PRESENCE OF OBJECTS.
Classification
- CPC, 4
- G08G1/167
- G01S11/12
- G06T7/20
- G06V20/58
- IPC, 10
- G01P3 36
- G01S11 00
- B60R1 00
- G01B11 24
- G01S11 12
- G01V8 10
- G06T7 20
- G08B21 00
- G08G1 16
- H04N7 18