Method and device for real-time detection, location and determination of the speed and direction of movement of an area of relative movement in a scene

The present invention relates to a method and a device for locating and locating an area in relative motion in a scene and to determine the speed and the direction of this relative motion, in real time.

By relative movement, we mean both the movement of said zone (which may be constituted by an "object", in the broadest sense including a living being or a portion of a living being, a hand for example) in an environment substantially still, that more or less complete immobility of said area (or "object") in an environment at least partial displacement.

The invention relates to the processing of a digital video signal coming from an observation system, constituted by an optical input or objective system capable of forming an image of the observed scene, and by a conversion system. Optoelectronic or sensor capable of converting said image that it receives into a digital output signal.

In general, the observation system is constituted by a video camera or camcorder, which observes the scene to be monitored (said digital output signal then being constituted by the digital video signal delivered by a digital output camera or by the output of a analog-to-digital converter whose input is connected to the output of a camera that outputs an analog video signal).

The observation system could also be constituted by the objective of an optical instrument (binoculars, observation telescope, viewfinder), from which at least a portion of the outgoing light beam is taken, and by a photoelectronic sensor. , of the CCD or CMOS type for example, with the usual associated electronics, the sensor receiving the image formed by said light beam portion and converting it, by the associated electronics, into a digital output video signal.

The invention essentially consists in processing the digital output video signal of an observation system, in particular a video camera, with a digital output to derive signals signaling the existence and location of a zone in relative displacement. in said scene, as well as the speed and the direction of movement in the case where said zone actually moves in said scene relative to a substantially immobile environment, and this in real time.

The most advanced system for locating and locating an object in relative motion and determining its speed and direction of movement is the system of animal or human vision, for example a hunter on the lookout locating the movement of a animal, as well as the direction and speed of this movement.

In the prior art artificial or artificial retina-type monitoring devices have been proposed (Giocomo Indiveri et al. in Proceedings of MicroNeuro'96 p. 15 at 22), or numerically (Pierre-François Rüedi in Proceedings of MicroNeuro'96 p. 23 to 29), but it is in the first article of detectors and analog units complex structure and in the second article of means for locating the edges of an object; furthermore, in the devices described, very fast and high-capacity memories are used to operate in real time, and limited information is obtained with regard to moving areas or objects.

It has thus been proposed to store, in a first two-dimensional memory, the signal of a frame coming from a video camera, or the like, constituted by a sequence of data concerning the pixels representative of the scene observed by the camera. a moment <i>t</i>₀, then, in a second two-dimensional memory, the video signal, for the following corresponding frame, representative of said scene at a given moment <i>t</i>₁. If an object has moved between<i>t</i>₀ and <i>t</i>₁, we determine, on the one hand, the distance <i>d</i> traveled by this one in the scene between <i>t</i>₁ and <i>t</i>₀ and, on the other hand, the duration T = <i>t</i>₁ - <i>t</i>₀ between the beginnings of two successive corresponding frames relating to the same pixels. The speed of the displacement is then equal to<i>d</i>/ T. Such a system requires a very large total memory capacity if it is desired to obtain precise indications of speed and directed direction characterizing the displacement. In addition, there is some delay in obtaining indications of speed and direction of travel; indeed such information is only available at the moment<i>t</i>₁ + R, by calling R the duration of the calculations concerning the interval <i>t</i>₀ - <i>t</i>₁. This double disadvantage (the need for a large memory capacity and delay in obtaining the desired information) limits the applications of such a system.

Furthermore, French Patent No. 2,611,063, of which one of the inventors (Mr Patrick Pirim) is the inventor of the present invention, describes a method and a device for real-time processing of a sequenced data stream, consisting of in particular by the output signal of a camcorder, in order to perform data compression. According to this prior patent, the histogram of the signal levels is formed according to a classification law for a first sequence, the representative Gauss function associated with this histogram is stored, from which we extract the maximum and minimum levels, we compare the levels of the subsequent sequence, or second sequence, at said signal levels for the first sequence, memorized with a constant time constant, identical for each pixel, a binary classification signal is generated which characterizes said following sequence with respect to the classification law, we generate, from this binary signal, an auxiliary signal representative of the duration and the position of a range of significant values ​​and finally one generates, from said auxiliary signal, a signal of location of the beach having the longest duration, so-called dominant beach; and these operations are repeated for the following sequences of the sequenced signal. This method and this classification device allow a compression of the data by retaining only the parameters of interest of the stream of data sequenced. In particular, this method makes it possible to process a digital video signal representative of a video image in order to extract and locate at least one characteristic of at least one zone of said image. It is thus possible to classify the luminance and / or chrominance levels of the signal and to characterize and locate an object in the image.

As for United States Patent No. 5,488,430, it carries out the detection and estimation of a displacement by separately determining the horizontal and vertical changes in the image of the area observed. Difference signals are used to detect movements from right to left, or vice versa and from top to bottom, Or vice versa, in the horizontal and vertical directions respectively, performing the OR EXCLUSIVE function on horizontal / vertical difference signals and frame difference signals, Firstly, and using a ratio of the horizontal / vertical sums of sums and sums of the frame difference signals with respect to a window K x 3, on the other hand. In this US patent 5 The calculated values ​​of the image according to the two horizontal and vertical orthogonal directions are used with an identical repetitive difference K in these two orthogonal directions, this difference K being defined as a function of the speeds of displacement that are to be determined. The device according to this US patent determines the direction of movement following each of the two orthogonal directions by applying to the difference signals a set of calculation operations indicated in columns 12 (at the beginning and end) and 13 (at the beginning) which requires electronic operators, including division, multiplication and summation, very complex (so difficult to achieve); additional complex operators are also needed to obtain, from projections on both horizontal and vertical axes, the speed and the direction of the displacement (square root extraction to get the amplitude of the speed and calculation of the arctg function to get the oriented direction). Lastly, in Patent No. 5,488,430, it is not possible to implement smoothing of the pixel values ​​by means of a time constant, variable for each pixel, in order to compensate for the too rapid variations of these values.

On the contrary, the implementation of the method according to the invention is carried out by means of a device, subject of the invention, which is of digital type, with a relatively simple structure and a relatively small capacity memory, and allows the fast obtaining of the desired information, with very varied results and applications (which correspond, depending on the application, to a complete half-image in the case of interlaced fields or to a complete image).

An article by Alberto Tomita Sr. and Rokuya Ishii, entitled "Hand Shape Extraction from a Sequence of Digitized Gray-Scale Images," in Institute of Electrical and Electronics Engineers, Vol. 3, 1994, p. 1925-1930, uses motion detection by subtraction between successive images, followed by the formation of histograms based on the shape of the human hand, in order to extract the shape of a human hand in a digitized scene. Histogram analysis is based on a gray scale inherent to the human hand. No formation of plane coordinate histograms is expected. The sole purpose of the authors of this article is to detect the displacements of a human hand, for example in order to replace, for the introduction of data in a computer, the usual mouse by a hand whose movements are identified.

On the contrary, the present invention is not limited to detecting the movement of a hand, but allows to detect the relative displacement of any object, in the broadest sense, in a scene and does not use histograms based on the values of gray of a hand, but histograms of certain particular numerical variables representative of the possible displacement and histograms of coordinates of the plane.

According to the present invention:<ul id="ul0001" list-style="dash" compact="compact"><li>processing a digital video signal, originating from an observation system, signal constituted, in the known manner, by a succession of frames (which correspond to a half-image in the case of two frames interlaced by image or at a complete image in the case of a single frame per image) each comprising a determined number of successive lines and, in each of these lines, a determined number of pixels or image points,</li><li>in order to obtain, by using a memory of relatively low capacity, signals able to indicate whether there exists, in the observed scene, a zone in relative displacement and, in this case, to specify the location, as well as the speed and the direction (oriented) of said zone if it actually moves relative to its environment,</li><li>and this by elaborating two characteristic digital signals, one, of a significant variation or non-variation of the pixel signal for the same pixel location between two successive corresponding frames and, the other, of the amplitude of this variation. , when it exists, and by distributing these two signals in a matrix manner for the pixels of a frame portion at the same time.</li></ul>

The invention firstly relates to a method, in real time, for locating and locating a region in relative motion in a scene observed by an output observation system constituted by a digital video signal of the type comprising a succession of corresponding frames, each composed of a succession of lines, each composed of a succession of pixels, as well as for the determination of the speed and direction of travel, said method being characterized in that it consists in performing on the digital output video signal successively:<ul id="ul0002" list-style="dash" compact="compact"><li>a smoothing process of said digital output video signal implementing a digital time constant whose numerical value can be modified for each of the pixels of said output signal, independently for each of them;</li><li>storing a frame of said output signal after the smoothing, on the one hand, and the smoothing time constant associated with said frame, on the other hand;</li><li>a consistent time processing, for each pixel position, to determine the existence, Firstly, and amplitude, on the other hand, a significant variation in the amplitude of the pixel signal between the current frame and the smoothed anterior frame, stored in memory and to generate two digital signals, the first signal being a binary or monobit signal with two possible values, one of which represents the existence of such a significant variation and the other the absence of such a variation between two successive frames, the value of said binary signal modifying the stored value of said time constant in order to decrease it if said signal represents a significant variation and in order to increase it if this signal does not represent such a variation, the decrease or increase being carried out in a quantified manner, while the second digital signal, said amplitude, is a multibit signal with a limited number of bits, quantifying the amplitude of this variation; and</li><li>a spatial processing consisting, for each input digital video signal frame,<ul id="ul0003" list-style="bullet" compact="compact"><li>to distribute, in order to characterize the values ​​of the pixels, into a matrix with a reduced number of rows and a reduced number of columns, with respect to the number of lines and the number of pixels per line in the video signal, respectively, the values, at the same time for observation, for a fraction of the pixels of a frame - a fraction which scrolls through said matrix during the duration of a frame - of said binary signal, on the one hand, and said digital amplitude signal, of somewhere else,</li><li>to be determined, in this double instant matrix representation, a particularized area, wherein both said binary signal has the desired value representing the presence, or respectively the absence, significant variation and said digital amplitude signal varies, or respectively does not vary, a significant value between neighboring pixels in the matrix in a direction oriented from an original pixel, and this for the same frame portion, therefore at the same moment of observation, and</li><li>generating signals representative of the existence and location of the region in relative displacement, as well as the relative speed interframes and the oriented direction of this displacement, if it exists, relative to its environment, from the matrix distribution instantaneous of these two digital signals, binary and amplitude. Preferably, the method according to the invention is characterized in that it consists, moreover,</li><li>to form the histograms of the values ​​of the matrix-distributed signals, on the one hand, and the histograms of the inclinations of two axes, with variable slope, of coordinates in a plane, on the other hand,</li><li>to identify, in each histogram formed, a domain of significant variation of the processed value and</li><li>to deduce, from each domain identified, the existence and the location, as well as the speed and the oriented direction, of a zone in relative motion.</li></ul></li></ul>

In particular embodiments:<ul id="ul0004" list-style="dash" compact="compact"><li>said matrix is ​​a square matrix with the same odd number (2<i>l</i> + 1) rows and columns, and we consider the nested matrices of 3 x 3, 5 x 5, 7 x 7, ... (2<i>l</i> + 1) x (2<i>l</i> + 1) elements centered on the center of this square matrix in order to determine the nested matrix of smaller size in which said digital signal varies in a direction oriented from said center, the value of said binary signal representing a crossing of the threshold in this direction ,</li><li>said matrix is ​​a hexagonal matrix and the nested hexagonal matrices of increasing size centered on the center of this hexagonal matrix are considered in order to determine the nested matrix of smaller size in which said digital signal varies in an oriented direction,</li><li>said matrix is ​​an inverted L-matrix with a single line and a single column and the nested matrices of 3 x 3 pixels, 5 x 5 pixels, 7 x 7 pixels ... (2<i>l</i> + 1) x (2<i>l +</i> 1) pixels, for the single line and column, to determine the smaller size matrix in which the signal varies in an oriented direction, namely the line of greatest slope with constant quantization.</li></ul>

Advantageously, said time constant is of the form 2^p, <i>p</i> being a number less than 16, which can therefore be expressed in not more than 4 bits, the decrease or increase of the time constant being achieved by the subtraction or addition of a unit to <i>p</i>.

In this case, if desired, successive decreasing portions of complete fields are considered according to Mallat's time-scale algorithm and selects the largest of these portions, which gives indications of displacement, speed and orientation, compatible with the value <i>p</i>.

The subject of the invention is also a device, running in real time, for locating and locating a region in relative motion in a scene observed by an output observation system constituted by a digital video signal, of the type comprising a succession of corresponding frames, successive lines in each corresponding frame and successive pixels in each line, as well as for the determination of the speed and direction of travel, by carrying out the aforementioned method, said receiving device, entrance, said output video signal and being characterized in that it comprises in combination:<ul id="ul0005" list-style="dash" compact="compact"><li>means for smoothing said digital output video signal implementing a digital time constant whose numerical value can be modified for each of the pixels of said output signal, independently for each of them;</li><li>means for storing a frame of said smoothed output signal, on the one hand, and the smoothing time constant associated with said frame, on the other hand;</li><li>a time processing unit for analyzing the temporal variations of the amplitude of the pixel signal, for the same pixel position, between the current frame and the smoothed anterior weft, stored in memory said digital video signal, said unit comprising, in association with a memory able to receive, storing and rendering information relating to the previous smoothed matching frame, comparison means for determining, for each pixel position in the frame of the incoming video signal, if the absolute value of the difference between the current pixel signal and a representative value of the pixel signal, for the same pixel position, in the previous frame, representative value stored in said memory, exceeds or not a threshold, by generating a binary or monobit signal with two values, one of which represents the existence of an overtaking and the other of which represents the absence of an overrun, and calculation means able to determine a digital signal of multibit amplitude, reduced number of bits, whose value is a function of the amplitude of the variation of the value of the same pixel between the current frame and the smoothed anterior frame, stored in memory digital video signal; and</li><li>a set of spatial processing, whose entries receive, the temporal processing unit, said successive binary and digital amplitude signals for the pixels of the same frame, said unit comprising means capable of characterizing the amplitude values ​​of the pixels, these means spreading, following a matrix with a number of rows and a reduced number of columns with respect to the number of lines and the number of pixels per line in a frame of said digital video signal, only said binary and digital signals of relative amplitude at the same instant, that is to say, to the same frame, the latter sweeping through said matrix during the duration of a frame, locating means for determining, in said matrix, an area of ​​pixels in which, at this moment, the binary signal has the desired value and means for determining, in said matrix, an area of ​​pixels in which, at that moment, the digital amplitude signal varies in a significant amount between neighboring pixels, and means that in response to the indications of the last two last means, generate signals representative of this zone of pixels, therefore the existence and location of an area in relative motion in the observed scene, as well as the relative velocity interframes and the direction oriented of this zone as it actually moves relative to its environment.</li></ul>

Preferably, the spatial processing unit furthermore comprises an output delivering a delayed digital video signal constituted by the digital input video signal delayed by a duration equal to the duration of the rows of a matrix less the duration of the delay. a line for providing a contemporaneous output signal from the analysis of the array in said time processing unit.

Preferably, in the device according to the invention, the spatial processing unit comprises cascade delay means each of which imposes a delay equal to the time difference between the beginnings of two successive lines and cascade delay means for each line each imposing a delay equal to the difference time between two successive pixels of a line, the outputs of the set of all the registers and the input of the first registers of each line at a given instant delivering the values ​​of said binary signal and said digital amplitude signal, at one moment, to said registering means.

Advantageously, the device according to the invention also comprises means forming the histograms of the output values ​​of said spatial processing unit, as well as the histograms of the inclinations of two axes, with variable slope, coordinates in a plane, means to locate, in each histogram, a domain of significant variation of the treated value, in order to validate this domain in their output and to deduce, for all the histograms, output signals which signal and locate an area of ​​the scene observed in relative motion, if it exists, as well as the speed and direction of this movement if said area actually moves relative to its environment.

If it is desired to detect the movement of an object in a substantially immobile environment, the area of ​​the matrix in which the binary signal is simultaneously determined to be equal to an overshoot and the digital amplitude signal varies by a value is determined. significant between neighboring pixels of a frame.

On the other hand, if it is desired to detect the immobility of an object in a substantially moving environment, the area of ​​the matrix in which the binary signal is simultaneously determined to be equal to a non-exceeding of the threshold and the digital signal is determined. amplitude does not vary between neighboring pixels of a frame.

Preferably, in the device for locating, locating and determining the speed and the direction of movement of an area in relative motion in a scene, by carrying out the aforementioned method:<ul id="ul0006" list-style="dash" compact="compact"><li>said smoothing means comprises an input which receives said digital video signal and calculates, for the successive pixels of a frame of this video signal, a smoothed signal, wherein the temporal variations of the input digital video signal are decreased, by implementing a threshold signal received on another input and a time constant relative to each pixel position of a frame, whose value is successively modified so that the smoothing while reducing it, the variation trend of the incoming digital video signal, these smoothing means cooperating with the memory unit which receives, stores and restores the updated values, for each pixel portion of a frame, smoothed signal and said time constant and flow, on their outings, at least the estate, for each pixel position, values ​​of the updated time constant and values ​​of a binary signal of exceeding or not exceeding said threshold by the absolute value of the difference between the value of the pixel and its smoothed value; said spatial processing unit realizing the matrix distribution, by rows and columns in small numbers, outputs of said smoothing means, i.e., successive values ​​of the time constant and said binary signal;</li><li>locating means are provided for locating, in said matrix distribution, an area of ​​pixels in which, at a time, the value of said binary signal corresponds to a threshold overshoot and said time constant varies between neighboring pixels by a significant value in one direction, and to produce output signals indicating the location of said area and the speed and the direction of travel in said area, either the value of said binary signal corresponds to a non-exceeding of the threshold and said time constant does not vary between neighboring pixels and to produce output signals indicating the location of said zone.</li></ul>

In the preferred embodiments:<ul id="ul0007" list-style="dash" compact="compact"><li>said smoothing means comprises in combination with a video memory or field memory which stores the successive values, for each frame pixel, said time constant and the smoothed digital video signal, calculation means, for each pixel, the absolute value of the difference between the value of the digital video signal from the camera and the value of the smoothed previous digital video signal, means for comparing this difference with a threshold and for generating a binary signal, where one of the two values ​​indicates an exceeding of said threshold and the other value indicates a non-exceeding thereof; means for updating the time constant receiving from said memory the just prior value of the time constant and decreasing it if it receives a binary signal whose value indicates a threshold overshoot, but increasing it if the value of the binary signal indicates a non-exceeding, the decrease or increase, however, does not take place if it leads respectively to a negative value or to a value greater than a threshold value, and means for updating the smoothed value of the digital video signal which algebraously adds to the previous value of this smoothed signal received from said memory the quotient of the difference between the digital video signal coming from the camera and the digital video signal preceding smoothing from said memory by a factor equal to the value of the previous time constant from said memory;</li><li>said time constant is in the form 2^p, <i>p</i> being an integer less than 16, which can therefore be represented by at most 4 bits, the decrease or increase in the time constant being achieved by subtracting or adding a unit to <i>p</i> ;</li><li>said matrix distribution means comprise, in combination with delay means applying successive delays, equal to the duration of a line of the video signal, to the digital video signal from the camera to output on a succession of outputs, this video signal delayed by the duration of a delay, two delays and so on until a number of delays equal to the number of rows in the spatial distribution matrix minus one unit, matrix distribution means along the successive lines of the receiving matrix, Firstly, the digital video signal of the undelayed camera and this progressively delayed signal from the outputs of the delay means and, on the other hand, the values ​​of the time constant and said binary signal from said smoothing means, to achieve a matrix distribution, by rows and columns, at a given instant of said values ​​of the time constant and the binary signal for the pixels of a frame portion of the digital video signal of dimension equal to that of the matrix;</li><li>the matrix distribution means comprise a succession of digital signal conductors in number equal to that of the rows of the distribution matrix, each with shift registers, connected in series, each imposing a delay equal to the time difference between two successive pixels of a line of the digital video signal, the position of a pixel distributed in the matrix being determined by a point of the matrix situated upstream of a delay register, whose number per line is equal to the number of columns in the matrix minus one, and by a point downstream of the most downstream shift register;</li><li>said registering means, in said matrix, an area on the move, by detecting the simultaneous presence of a value of the binary signal indicating an overshoot and the variation of the value of the time constant, include means to determine, following digitized discrete oriented directions, the slope of the variation of the value of the time constant in the vicinity of a pixel in the center of said matrix, constituting the origin for the said directions, and means for selecting the steepest slope of variation in the vicinity of said center-origin and determining the oriented direction thereof, taking into account a selection criterion for selecting the direction in case of more than one direction of the same maximum slope of variation, these last means debiting, output, signals representative of the speed and the direction of travel in the moving zone, with a validation signal indicating that these speed and direction signals are validated, as well as the value of the time constant.</li></ul>

Preferably in the case where the device according to the invention comprises them, the means forming the histograms each comprise:<ul id="ul0008" list-style="dash" compact="compact"><li>inputs receiving the signal whose histogram is formed and a validation signal from the matrix distribution means; and</li><li>means for establishing two linear one-dimensional histograms for the two coordinates of the plane and for combining these two linear histograms into a surface histogram representing the significant area of ​​variation of the input signal; and</li><li>an output delivering a signal representative of this area.</li></ul>

In addition, the means forming the histograms preferably comprise:<ul id="ul0009" list-style="dash" compact="compact"><li>marker change calculating means, the inputs of which receive a line sequence signal, a column sequence signal and a pixel clock signal and whose output is representative of the marker change;</li><li>two histogram forming means for two axes, receiving the two reference signals and forming the histograms of these axes; and</li><li>zone means receiving the outputs of the two histogram forming means for the two axes and outputting, at the output, an information signal of global slope of the two axes.</li></ul>

In certain applications, provision may be made for said time constant to be constituted by the sequence number of the intervals, in limited number, and in progressively increasing size, according to which the absolute value of the difference between the current pixel value and the just prior value of the same pixel after smoothing, for each pixel position.

Advantageously, in order to determine the binary signal for exceeding or not exceeding the threshold, a threshold is compared, for each pixel position, with the absolute value of the difference between the value of the current pixel that enters and the value of the same smoothed pixel just before from the memory.

As far as the digital amplitude signal is concerned, it is preferably generated in the form of an integer which represents the tendency to bring the current pixel value closer to the value of the same smoothed anterior pixel, for each pixel position.

One of the characteristics of the invention consists in the fact that, for determining both the binary signal and the digital amplitude signal, the value of the just prior pixel value is used, a smoothed value thereof stored in memory, in order to reduce the excessive temporal variations of this pixel signal that may exist in the input digital video signal from the video camera or other digital output observation device.

Indeed, it is known that a smoothing operation has the effect of progressively substituting, for a digital signal with large variations in amplitude over time, a signal having smaller variations and therefore more easily quantifiable with a reduced number of steps, therefore of bits, in the smoothed amplitude digital signal.

We will now describe, with reference to the accompanying drawing, a preferred embodiment, given by way of illustrative example, and in no way limiting, of a device according to the invention embodying the method according to the invention.

On this drawing :

Fig. 1 very schematically illustrates the entire system according to the invention with its input and its outputs, as well as the input signal for this system.

Fig. 2 shows, in the form of functional blocks, the main units of a device according to the invention forming a set of temporal and spatial processing.

Fig. 3 and FIG. 4 illustrate the block diagrams of the computational time processing set and the matrix division spatial processing set, respectively, which form part of the device of FIG. 2.

Fig. 5 schematizes the temporal processing and spatial processing in a system according to the invention.

Fig. 6 illustrates the numerical value, according to the Freeman code, of the discrete directions from a center of origin in the matrix of FIG. 4.

Fig. Figure 7 illustrates two small nested matrices within the time distribution matrix.

Figs. 8 and 9 describe two other types of matrix, respectively hexagonal and inverted L.

Fig. 10 schematically illustrates the assembly, according to the Z - Z line¹ of the device of FIG. 2 with an additional assembly according to a preferred embodiment of the invention.

Fig. 11 shows, in block form, said further set of FIG. 10; FIG. 2 and FIG. 11 being assembled along the Z - Z line¹, in phantom, shown in FIG. 2 and FIG. 10.

Fig. 12 illustrates the formation of two one-dimensional histograms and, from them, a surface histogram of moving area, for an input signal.

Fig. 13 shows in more detail a block or unit of formation and histogram processing and its associated linear combination block.

Fig. 14 illustrates a one-dimensional histogram.

Fig. And FIG. 16 illustrate the use of the variation of the observation slope of a scene.

Fig. 17, FIG. 18 and FIG. 19 illustrate, in the form of blocks for the first two figures, other possible applications of a device according to the invention, the first figure relating to videoconferencing, the second relating to the surveillance of motorways (or main roads) and the third concerning the control of a machine by the movement of the hand of an operator.

Figs. 20 and 21 show schematically the application of the invention to the monitoring of the sleep of a motor driver.

Fig. 22 shows the image transformed according to the Mallat diagram.

Referring first to FIG. 1, it can be seen that, in the preferred embodiment, the device 11, according to the invention, comprises first an input 12 which receives a digital video signal S from a video camera or camcorder 13, with one or more sensors CMOS type CCD, with direct digital output or with converted analog output, in an analog / digital converter, in digital output. This signal S is constituted, in the known manner, by a succession of pairs of interlaced frames such as TR₁ and TR '₁, TR₂ and TR '₂, each having a succession of horizontal scanning lines, each line (such as <i>l</i>_1.1, <i>l</i>_1.2, ... <i>l</i>_1.17... from TR₁ and <i>l</i>_2.1 from TR₂) being constituted by a succession of pixel elementary signals or image points PI representative of points (such as: <i>at</i>_1.1, <i>at</i>_1.2, and <i>at</i>_1.3 for the line <i>l</i>_1.1 ; <i>at</i>_2.1, <i>at</i>_2.2 for the line <i>l</i>_1.2 ; <i>at</i>_17.1 and <i>at</i>_17.2 for the line <i>l</i>_1.17 ; <i>at</i>_1.1 , <i>at</i>_1.2 for the line <i>l</i>_2.1) of the scene 13a monitored by the camera 13; this is why in the drawing S (PI) has been indicated, namely a signal S constituted by PI pixels.

In the known manner, S (PI) has frame synchronization signals ST at the beginning of each frame and SL line synchronization at the beginning of each line.

So we see that the signal S (PI) is constituted<ul id="ul0010" list-style="dash" compact="compact"><li>succession of sequences (the successive frames) in the context of a temporal domain and</li><li>in each sequence (in each frame) by a series of subsequences (lines, pixels) in the context of a spatial domain.</li></ul>

In the time domain, the expression "successive frames" will be used to designate successive frames of the same nature (that is to say the odd fields, for example TR₁, or respectively pairs, for example TR '₁) pairs of frames, for example TR₁ - TR '₁, forming the successive images of the digital video signal S (PI) and by the expression "successive pixels in the same position" the successive pixel values ​​(PI) in a same location of successive frames of the same nature, for example <i>at</i>_1.1 of <i>l</i>_1.1 of the TR frame₁ and <i>at</i>_1.1 of <i>l</i>_2.1 the next corresponding frame TR₂.

The device 11 furthermore includes outputs 14 delivering various digital signals, elaborated by him, useful to indicate the existence of a zone or "object" (in the most general sense indicated above) in relative displacement and its location, as well as its speed and direction of movement if it is effective with respect to a substantially immobile environment, namely the complex signal ZH schematically grouping the signaling signals of existence and the location of this zone or object, the speed V and directed direction DI of the displacement, and possibly the input digital video signal S delayed in synchronism with the previous signals, to take into account their calculation time, this delayed signal SR for viewing, on the screen of a monitor or TV 10, the image perceived by the camera 13 at the moment when information about the possible area in relative displacement is available, namely the signal ZH, (V, DI) usable in a set of processing and control 10a.

With reference to FIG. 2, we will explain the structure of the first part of the device 11 of FIG. 1, this first portion being shown inside the frame 11a in broken lines of this FIG. 2.

The assembly 11a essentially comprises, on the one hand, a temporal processing unit 15, with an associated memory unit 16, and, on the other hand, a spatial processing unit 17, with a delay unit 18 and a storage unit. associated sequencing 19, as well as a pixel clock 20 timing the time processing unit 15 and the sequencing unit 19.

The time processing unit 15, which performs, among other things, a smoothing of the video signal:<ul id="ul0011" list-style="dash" compact="compact"><li>generates, from the digital video signal S, from the video camera 13 and having a succession of pixel values ​​PI, and from the pulses HP generated by the clock 20 (from signal S) at the rate of the pixels in a frame (in particular 13.5 MHz), a certain number of quantities, as explained below with reference to FIG. 3, and exchanges with the memory 16 the values ​​of two of these magnitudes, namely the smoothed values ​​L of the digital video signal and the values ​​C of the smoothing time constant, the values ​​L and C being followed by the letter O for the values ​​entering the memory 16 from the unit 15 or the letter I for the values ​​leaving the memory 16 to reach the unit 15, and</li><li>outputting a DP binary signal for exceeding or not exceeding the threshold and a digital signal CO indicating the updated calculated value of the time constant, namely the value CO sent in the memory 16.</li></ul>

The block structure and / or comparison of the time processing unit 15 is explained in FIG. 3; the unit 15 has four blocks 15a, 15b, 15c, 15d.

The first block 15a of the unit 15, from<ul id="ul0012" list-style="dash" compact="compact"><li>an input digital video signal S, constituted by a succession of PI pixel signals, and</li><li>a smoothed value LI of this signal S for the corresponding frame just earlier, previously calculated by the unit 15 as LO and stored temporarily in the memory 16 (as explained below)</li><li>at the rate imposed by the clock signals HP of the clock 20, calculates the absolute value AB of the difference between the incoming values ​​of PI and LI for the same pixel position (for example <i>at</i>_1.1, from <i>l</i>_1.1 from TR₁ and of <i>l</i>_2.1 from TR₂)<maths id="math0001" num=""><math display="block"><mrow><mtext>AB = | PI - LI |.</mtext></mrow></math><img file="EP0912964B1_D0001.tif" /></maths></li></ul>

The second block 15b is a test block:<ul id="ul0013" list-style="dash" compact="compact"><li>it receives the abovementioned digital signal AB from the unit 15a and a threshold value digital signal SE, which could be fixed, but which is generally a function of the pixel value; it is then varied in the same direction as the latter to constitute a gamma correction (the known means for effecting the variation of SE to perform a gamma correction being represented by the optional block 15e in broken lines); and</li><li>it compares these two digital signals representative of AB and SE in order to determine a binary signal DP, that is to say that can take two values ​​1 and 0, which signal an overshoot or no overshoot, respectively, of said threshold SE by AB:</li></ul><ul id="ul0014" list-style="dash" compact="compact"><li>if AB is greater than SE, DP will be assigned in unit 15b the value 1 representative of an overshoot;</li><li>if AB is less than or equal to SE, DP will receive, in the unit 15b, the representative 0 value of no overshoot.</li></ul>

In fact, when DP = 1, there is too much difference between PI and LI, that is, between the input digital video signal and the smoothed previous digital video signal, and this difference must be reduced by reducing the smoothing time constant and conversely if DP = 0 it is necessary to increase this time constant.

The third block 15c precisely makes the desired variation in the value of the time constant as a function of the value of DP:<ul id="ul0015" list-style="dash" compact="compact"><li>if DP = 1, the block 15c decreases by a unit value U the time constant: CO (new value of this constant) = CI (old value of the constant) - U;</li><li>if DP = 0, the block 15c increases by the same unit value U the time constant: CO = CI + U. For this purpose, the block 15c receives, on an input, the above-mentioned binary overflow signal DP coming from the block 15b and, on another input, the signal CI, which is the value of the previous time constant stored in the memory 16 and decreases or increases the unit U value of the incoming time constant CI that becomes CO sent into said memory 16 instead of CI. Advantageously, the time constant on which the convergence of the smoothing (as a function of the time required for the smoothed value to reach the input value of the digital video signal) depends, is represented by a power of 2, namely a value of 2^p, and that's the whole number <i>p</i> which will be decreased or increased in block 15c by one unit, i.e. 1; then in FIG. 3 we have U = 1 for<i>p</i>,</li><li>if DP = 1, block 15c subtracts a unit (1) from the factor <i>p</i> of the time constant 2^p, which becomes 2^p-1 ;</li><li>if DP = 0, block 15c adds a unit (1) to the factor <i>p</i> of the time constant 2^p, which becomes 2^{p + 1}.</li></ul>

The choice of a time constant of type 2^p has the double advantage of matching the physiology of human vision and allowing simpler calculations, which simplifies the structure of block 15c.

Block 15c must also ensure a dual condition, namely to keep CO between two limit values: CO must not become negative (CO ≥ 0) and must not exceed a threshold N (CO ≤ N). In the particular case where CI and CO are of the form 2^p, the upper threshold N is represented by a whole number <i>not</i> which constitutes a maximum value for <i>p</i>.

The upper threshold N (or <i>not</i>) can be either constant or variable; in the latter case an optional unit 15f (in broken lines) realizes this variation of N (or<i>not</i>) on the order of the user for example. An increase in N has the effect of increasing the sensitivity of the displacement detection, while the decrease of N improves the detection of high speeds.

Finally, the fourth block 15d receives, on a first input, the value CO of the new time constant elaborated in the block 15c, on a second input, the digital input video signal S in the form of a value information of PI pixel and, on a third input, the smoothed value of the previous input digital video signal, namely LI, from the memory 16 and it calculates<maths id="math0002" num=""><math display="block"><mrow><mtext>LO = LI + (PI - LI) / CO</mtext></mrow></math><img file="EP0912964B1_D0002.tif" /></maths> which is debited on its output.

In fact the term (PI - LI) / CO represents the modification made to the smoothed value of the digital video signal, taking into account the modified value CO of the time constant and it is proportional to the algebraic difference between the effective value of the current input pixel PI from the camera 13 and its previous smoothing value LI and inversely proportional to CO.

If CO = 2^p, so<maths id="math0003" num=""><math display="block"><mrow><msup><mrow><mtext>LO = LI + (PI - LI) / 2</mtext></mrow><mrow><mtext>po</mtext></mrow></msup></mrow></math><img file="EP0912964B1_D0003.tif" /></maths> considering the fact <i>po,</i> value <i>p</i> calculated in the unit 15c, and which replaces in the memory 16 the previous value <i>pi</i> of <i>p</i>.

Therefore, the four-block computing processing unit 15a, 15b, 15c, 15d:<ul id="ul0016" list-style="dash" compact="compact"><li>receives S (PI) from the video camera 13, the clock pulses HP, for the timing of the operations, the threshold signals SE and N (or <i>not</i>);</li><li>determines, from input signals LI and CI from the associated memory 16, updated signals LO and CO which are sent in said memory instead of LI and CI respectively and which respectively represent the new calculated values ​​of the video signal digital smoothing and the time constant;</li><li>and outputting, to the spatial processing unit 17, through the delay unit 18, the aforementioned CO signal and the binary overflow signal DP which it has calculated from PI, LI and SE.</li></ul>

The aim of the smoothing operation is to normalize the variations of the digital value of the input video signal for each pixel or picture point, namely the variation of each PI, by reducing the variations of variation and substituting, for each pixel, with the actual variable values ​​of PI at this image point of the smoothed values ​​LO, less variable than the values ​​PI.

Thus, at each incoming PI, the temporal processing unit 15, in combination with the memory 16, replaces a smoothed LO value with reduced variations, by implementing a binary signal DP, whether or not a threshold is exceeded. a time constant signal CO which are updated and sent to the spatial processing unit 17 shown in FIG. 4.

Each pixel can be located inside the surface of a frame by two coordinates (in principle orthogonal) of abscissae and ordinates, namely <i>x</i> and <i>there,</i> by giving the pixel two indices <i>i</i> (line number) for the coordinate <i>there</i> and <i>j</i> (number of the pixel in the line) for the coordinate <i>x.</i> Each pixel of indices <i>i</i> and <i>j</i> has a video value (value of the video signal amplitude) PI_ij.

If we now consider evolution, in time <i>t</i>, PIs_ij for successive corresponding frames at successive times <i>t</i>₀, <i>t</i>₁, <i>t</i>₂, <i>t</i>₃ ..., separated by a period T corresponding to the image period (generally equal to two frame periods) and which may be 0.04 s in the case of a frequency of 25 Hz for the signal of video image and 0.0333 ... s in the case of a frequency of 30 Hz for this signal, or 50 Hz for progressive image sensors (1 frame / 1 image), the signal of a video pixel of location indices <i>i</i> and <i>j</i> has successive values ​​noted PI_ijt0, PI_ijt1, PI_ijt2, PI_ijt3 ... at these moments <i>t</i>₀, <i>t</i>₁, <i>t</i>₂, <i>t</i>₃ ...

In the context of the invention, in the temporal processing unit 15, P_ijt the successive smoothing values ​​LO_ijt, know LO_ijt0, LO_ijt1, LO_ijt2, LO_ijt3 ...

For each of the pixels or successive image points of coordinates <i>i</i>, <i>j</i>, just now <i>t</i>, that is to say, P_ijt, we replace its real value PI_ijt a smoothed value given by the formula<maths id="math0004" num=""><math display="block"><mrow><msub><mrow><mtext>LO</mtext></mrow><mrow><mtext>ijt</mtext></mrow></msub><msub><mrow><mtext> = LI</mtext></mrow><mrow><mtext>ij (t-1)</mtext></mrow></msub><msub><mrow><mtext> + (PI</mtext></mrow><mrow><mtext>ijt</mtext></mrow></msub><msub><mrow><mtext> - LI</mtext></mrow><mrow><mtext>ij (t-1)</mtext></mrow></msub><msub><mrow><mtext>) / CO</mtext></mrow><mrow><mtext>ijt</mtext></mrow></msub></mrow></math><img file="EP0912964B1_D0004.tif" /></maths>

The time constant is preferably of the form 2^pijt.

The calculations performed in unit 15, in particular in block 15d, for each time interval T separating <i>t</i>₁ of <i>t</i>₀, <i>t</i>₂ of <i>t</i>₁, <i>t</i>₃ of <i>t</i>₂etc. ensure a convergence of the LO value_ijt to the PI value_ijt, whose speed depends on the time constant that is variable in space (and therefore depends on <i>i</i> and <i>j</i>) and in time (and therefore depends on <i>t</i>) and that we can write CO_ijt.

In all cases more CO_ijt is big, plus the convergence of LO_ijt is slow. If CO_ijt = 1, there is no more smoothing.

We can obviously in the preceding formulas respectively substitute the Cartesian coordinates <i>there</i> and <i>x</i> to the indices <i>i</i> and <i>j</i> number of line order and pixel per line.

Block 15a calculates AB = | PI - LI |, the indices <i>i, j</i> and <i>t</i> not being explained, the magnitude AB being representative of the instantaneous variability of the digital video signal S (PI) with respect to the smoothed signal LI, for each triplet <i>i, j</i>, <i>t.</i>

This is one of the characteristics of a device according to the invention to achieve, in the time processing unit 15, a smoothing of the pixel values, to determine the successive values, for each pixel, a smoothing time constant and a binary signal for exceeding or not exceeding a threshold by the absolute value of the difference between the value of the pixel and the smoothed value of this pixel for two successive corresponding frames, to distribute, following a plane matrix, both the numerical values, at one moment, said time constant and the values ​​of said bit signal for the pixels of a limited frame portion, portion that sweeps the frame, in order to locate, thanks to the local variations of these two values ​​at this moment, a zone in relative motion and to determine the speed and direction (oriented) of actual displacement as a function of this distribution, this being done a set of spatial processing, described below with reference to FIG. 4.

The convergence of the smoothing is provided by the calculation blocks 15b and 15c which determine a variation of the value of the new incoming time constant CO (in fact CO_ijt) as it speeds up convergence. This is done by comparing AB (actually AB_ijt) at a threshold SE, which can have a constant or preferably variable value and in the latter case be a function of the pixel value to ensure a gamma correction: if the pixel value increases, the threshold increases and vice and versa. The result of comparing AB with the SE threshold (actually SE_ijt if the threshold varies as a function of the pixel value, this variation being realized in a possible calculation block 15e) generates in block 15b the binary displacement signal DP with two possible values ​​1 and 0, as explained above.

The DP bit signal arrives in block 15c to vary the value of the time constant. For this purpose the calculation unit 15c also receives the value of the incoming time constant CI from the memory 16 and updates it to a new value of the time constant CO which is, among others, sent to the memory 16 where it is substituted for the old CI value; in fact for the values ​​of CI and CO, it is CI_ij and co_ij at two successive moments, for example <i>t</i>₀ and <i>t</i>₁, separated by the interval T between two successive frames (either odd or even).

Block 15c receiving DP and CI adds or subtracts a unit value (U) to the value of the time constant CI or a unit to <i>p</i> when CO is of the form 2^p according to whether the binary signal DP represents an overshoot (DP = 1) or a non-overshoot (DP = 0).

If there is an overshoot, it is because the value of this constant is too strong and it is reduced and vice versa.

In addition, the block 15c ensures that the new value of the time constant CO, deduced from CI by addition or subtraction of a unit, remains between 0 (non-negative CO) and a threshold value N (CO no greater than NOT). If this double condition is not fulfilled, block 15c does not change the value of CI (which was actually within the range of 0 to N, limits included) and then CO = CI.

The upper limit value N or <i>not</i> can be either constant or variable; in the latter case it must not exceed a limit value N max (or<i>not</i> max); the possible variation being imposed by the block 15f under the effect of a command available to the user.

In a variant, it is possible to make N or <i>not</i> of pine, <i>not</i> and PI being actually assigned the triplet of indices <i>i</i>, <i>j, t</i>) in order to regulate the variation of LO (which is calculated in block 15d) as a function of the PI level, which can be expressed as "N_ijt or <i>not</i>_ijt is a function of PI_ijt»N's determination_ijt or <i>not</i>_ijt = <i>f</i>(PI_ijt) being performed in a calculation block substituted for the illustrated block 15f and receiving, in addition to N, the value of PI from the video camera 13.

We can advantageously impose on the time constant C (in fact on every C_ijt) the condition of having a numerical value which is a multiple of 2, more particularly a power of 2, whatever <i>i</i>, <i>j</i> and <i>t</i> ; in this particular case C_ijt = 2^{p (ijt)}, <i>p</i> being a small whole number, depending on <i>i, j</i> and <i>t</i>, representable by a reduced number of bits. This condition provides the additional benefits mentioned above:<ul id="ul0017" list-style="dash" compact="compact"><li>the law of convergence of the smoothing is close to the laws of the physiology of the human vision</li><li>the electronic realization of the blocks 15c and 15d is simpler; in particular, in block 15d, which is intended to determine the evolution of the value of the time constant, for each pair<i>i</i>, <i>j</i>, by the formula<maths id="math0005" num=""><math display="block"><mrow><mtext>LO = LI + (PI - LI) / CO,</mtext></mrow></math><img file="EP0912964B1_D0005.tif" /></maths></li></ul>

the calculations are simplified if CO is of the form 2^p (<i>p</i> being a small integer), the threshold value <i>not</i> of <i>p</i> being itself a small integer, representable by a limited number of bits.

In all cases, the new LO smoothing value of the input digital video signal S is sent to the memory 16 in which it substitutes for LI (and this for each pair of indices <i>ij</i>).

It can be seen from examining Figs. 2 and 3 that the temporal processing unit 15, which comprises the calculation blocks 15a, 15b, 15c, 15d, and possibly the block 15e and / or the block 15f, and which cooperates with the memory 16, determines, as exposed above, and outputs, for each triplet<i>i</i>, <i>j</i>, <i>t</i>,<ul id="ul0018" list-style="dash" compact="compact"><li>on the one hand, the smoothed smoothed value LO which is transferred to the memory 16 instead of the previous smoothing value LI</li><li>on the other hand, two digital signals, namely:</li><li>a binary signal DP, which indicates either the exceeding (DP = 1) or the not exceeding (DP = 0) of a certain threshold by the variation in absolute value of the incoming pixel signal coming from the video camera relative to the smoothed front pixel signal for the same point; and</li><li>a digital amplitude signal, constituted by the value of the updated time constant CO;</li></ul> these digital signals DP and CO are received, through the delay unit 18, by the spatial processing unit 17 which will be described hereinafter, the signal CO being also received by memory 16 in which the value CO replaces the previous CI value for the same pixel.

It is thus seen that the capacity of the memory 16 for storing the successive values ​​of the smoothed pixel signal, on the one hand, and the time constant, on the other hand, that is to say, assuming that there are R pixels in a frame, so 2R pixels per full frame, must be at least 2R (<i>e</i> + <i>f</i>) by calling <i>e</i> and <i>f</i> the number of bits respectively allocated to a pixel signal and a time constant. In fact the memory capacity does not need to be much larger; it suffices that it exceeds 2R (<i>e</i> + <i>f</i>) bits of the number of bits necessary to ensure its correct operation, in particular for the addressing and the extraction of the bits of the smoothed pixel signals and the bits of the time constants according to the indices <i>i</i> and <i>j</i>. If each video image is constituted by a single frame, it will suffice from R (<i>e</i> + <i>f</i>) bits instead of 2R (<i>e</i> + <i>f</i>) bits.

DP outputs_ij and co_ijat a moment <i>t</i>of the time processing set 15 are analyzed and used in a spatial processing set shown in FIG. 4, the assembly of FIGS. 3 and 4 being illustrated in FIG. 2.

In fact, the time processing unit processes the frame signals, while the spatial processing unit 17 of FIG. 4 processes subsequences of lines and pixels in a frame.

In FIG. The temporal processing of the sequences of successive corresponding frames TR has been schematized₁, TR₂, TR₃ superimposed on the figure and the spatial processing in the first of these frames, that is to say TR₁, illustrating the Cartesian coordinates <i>x</i> and <i>there</i> and a pixel PI of coordinates <i>there</i>, <i>x</i>, that is to say, indices <i>i, j</i> at the time <i>t</i>₁ ; successive pixels of the same indices<i>ij</i> on the three TR frames₁, TR₂, TR₃ are indexed <i>i, j, t</i>₁, <i>ijt</i>₂ and <i>ijt</i>₃ respectively and they have PI pixel values_ijt1, PI_ijt2 and PI_ijt3 respectively. A plane of FIG. 5 corresponds to a spatial processing of a frame, while the superposition of the planes corresponds to the temporal processing (the time<u>t</u> being the variable)

The spatial processing unit 17, which is associated with a delay unit 18 (also shown in Fig. 4), cooperates with a control unit 19 which is controlled by a clock 20 which outputs an HP clock pulse to each of the successive pixel signals (Fig 2 for the set)

DP outputs_ij and co_ij of the temporal processing unit 15 are distributed in the unit 17 according to a reduced-size matrix 21 comprising a number of rows and a number of columns much smaller than the number of rows L and pixels M per row of the DPs respectively._ij and co_ij at a moment <i>t</i> given. In particular, the matrix can comprise 2<i>l</i> + 1 lines along the axis of <i>there</i> and 2<i>m</i> + 1 columns along the axis of <i>x</i> (in Cartesian coordinates), <i>l</i> and <i>m</i> being whole numbers small. Advantageously we choose<i>l</i> and <i>m</i> among the powers of 2, <i>l</i> being equal to 2^at and <i>m</i> at 2^b, <i>at</i> and <i>b</i> being integers of the order of 2 to 5 for example. To simplify the drawing and the explanation, we take, as an example,<i>m</i> = <i>l</i> (although they may be different) and <i>m</i> = <i>l</i> = 2³ = 8; in this case the matrix 21 will have 2 x 8 + 1 = 17 rows and 17 columns.

In FIG. 4 a part of the 17 lines Y is represented₀, Y₁, ... Y₁₅. Y₁₆ and part of the 17 columns X₀, X₁, ... X₁₅. X₁₆ constituting the matrix 21 of the unit 17.

It is a question of distributing according to the matrix 21 to <i>l</i> + 1 lines, in particular 17 lines, and <i>m</i> + 1 columns, in particular 17 columns, the incoming flows of the DPs_ijt and co_ijt, i.e. DP overflow binary signals and amplitude digital signals representing the CO time constant, which arrive from the time processing unit 15, according to a larger matrix distribution for a frame, to namely L lines, in particular 312.5, and M pixels per line, in particular of the order of 250 to 800 depending on the TV standard used.

To distinguish the two matrices namely that of the video signal of L x M and that of the unit 17 of <i>l</i> x <i>m,</i> referenced 21, the indices will be used <i>i</i> and <i>j</i> following the two coordinates of the first (which only appears when viewing the digital video signal on a TV screen or monitor) and the indices <i>x and y</i> following the two coordinates of the second (shown in Fig. 4); at a given moment, a pixel of instantaneous value PI_ijt is characterized at the input of the spatial processing unit 17 by two digital signals DP_ijt and co_ijt. The matrix of the L x M of these two signals moves by scanning through the matrix 21 of (2<i>l</i> + 1) x (2<i>m</i> + 1) much smaller, as explained hereinafter with reference to FIG. 4, the matrix 21 materializing (2<i>l</i> + 1) x (2<i>m</i> + 1) pixels corresponding to the same frame.

In this matrix 21, each pixel is defined by a line order number between 0 and 16 (inclusive) for the Y lines₀ at Y₁₆ respectively and a column order number between 0 and 16 (including terminals) for columns X₀ to X₁₆ respectively, where <i>l</i> = <i>m</i> = 8, ie 2<i>l</i> + 1 = 2<i>m</i> + 1 = 17. In this case the matrix 21 will perform a representation in the plane of 17 x 17 = 289 pixels, while the matrix of the video signal will comprise several tens or hundreds of thousands of pixels or even more.

In FIG. 4, illustrated by horizontal elongated rectangles Y₀ at Y₁₆, of which only four have been represented, namely Y₀, Y₁, Y₁₅ and Y₁₆, and by vertical lines X₀ to X₁₆, of which only four have been represented, namely X₀, X₁, X₁₅ and X₁₆this matrix 21 (of unit 17) at 17 x 17 image points or index pixels defined at the intersection of an ordinate line and an abscissa column. For example, the pixel position P₈₈ is at the intersection of column 8 and line 8, as shown in FIG. <u>e</u>, center of the matrix 21.

To realize the successive spatial distribution of portions of this matrix of L x M according to the matrix 21 of (2<i>l</i> + 1) (2<i>m</i> + 1), the unit 17 is associated with a delay unit 18 which receives, on the one hand, the signals DP and CO (assigned indices <i>ijt</i>) and, on the other hand, the input pixel signal S, i.e. PI (also indicative <i>ijt</i>), as well as an HP signal from a clock 20 and SL line and SC column sequence signals (Figures 2 and 4).

As shown in FIG. 1, the signal S (PI) comprises, in addition to signals of pixel values ​​such as<i>at</i>_1.1, <i>at</i>_1.2 constituting a temporal sequence (successive frames) and spatial subsequences (pixels per lines in each frame), synchronization signals ST, SL, whose clock unit 20 deduces not only a clock signal, at the for example, a peak for each pixel of a video frame, but also BL blanking signals that render the unit 19 non-operative during the aforementioned synchronization signals.

In response to these HP and BL signals from the clock 20 (Fig. 2), the timing unit 19 outputs on the delay unit 18 a line SL signal at a frequency equal to the quotient of 13.5. MHz by the number of columns per frame, for example 400, and a frame signal SC whose frequency is equal to the aforesaid quotient 13.5 / 400 MHz divided by the number of video image lines, for example 312.5 , as well as HP.

It is from these signals SL and SC and the clock signal HP that the unit 18 provides the spatial distribution line by line following the matrix 21.

For this purpose the successive lines Y₀ at Y₁₆ receive the DP and CO signals:<ul id="ul0019" list-style="dash" compact="compact"><li>not delayed (line Y₀);</li><li>delayed by a period TP, equal to the duration of a frame line (line Y₁)</li><li>delayed by 2TP (Y line₂) and so on until</li><li>delayed by 16TP (line Y₁₆).</li></ul>

The successive delays of the duration of a frame line, namely TP, are realized in a cascade of sixteen delay circuits. <i>r</i>₁, <i>r</i>₂, ... <i>r</i>₁₆ serving the Y lines₁, Y₂ ... Y₁₆ respectively, line Y₀ being served directly by undelayed DP and CO signals from Unit 15.

All the circuits <i>r</i>₁, <i>r</i>₂, <i>r</i>₁₆ can be constituted by a delay line with sixteen outputs, the delay imposed by any section between two successive outputs being constant and equal to TP.

The scrolling of the whole of the L x M frame matrix on the matrix 21 of (2<i>l +</i> 1) (2)<i>m +</i> 1) is ensured, with respect to the successive lines of successive frames in rotation, by the timing unit 19 by means of line sequence signals SL, as follows.

With regard to the rolling motion of the pixels of a row of the matrix of a frame on the 17 × 17 matrix, for example X₀ to X₁₆ on line Y₀, it is realized by a cascade of sixteen shift registers <i>d</i> on each of the 17 lines of Y₀ at Y₁₆ (a total of 16 x 17 = 272 shift registers) arranged in each line between two successive positions of pixels, or between the positions PI₀₀ and PI₀₁ the register <i>d</i>₀₁, between the PI positions₀₁ and PI₀₂ the register <i>d</i>₀₂etc. Each register imposes a delay of TS equal to the time difference between two successive pixels of a line, thanks to the column sequence signals SC.

It should be noted that since the lines <i>l</i>_<i>1</i><i>, l</i>_<i>2</i>...<i>l</i>_<i>17</i> a TR frame₁ (Fig. 1), both for S (PI) and for DP and CO, arrive shifted by TP (complete duration of a line) one after the other to the unit 18 and that it distributes them with progressively increasing delays of TP on the Y lines₀, Y₁ ... Y₁₇, these display at a given moment the signals of DP and CO for the lines <i>l</i>_<i>1</i>, <i>l</i>_<i>2</i>, .... <i>l</i>_<i>17</i> of the same frame portion.

Similarly in a given line, such as <i>l</i>_<i>1</i>, the successive pixel signals <i>at</i>_1.1, <i>at</i>_1.2... arrive offset from TS and shift registers <u>d</u> impose a delay also equal to TS; it follows therefore that the DP and CO signals for pixels of a given line Y₀ at Y₁₆ of the matrix 21, pixels available on this line, are contemporary, that is to say that they correspond to the same portion of frame.

It is therefore, for both the lines and the pixels of these lines of a frame portion, a purely spatial processing, because the matrix 21 displays, in its 17 x 17 pixel positions, the values ​​of DP and CO for the 17 pixels of each of the 17 lines of the same matrix of the digital video signal S (PI), although these pixels, such as <i>at</i>_1.1, arrive successively line by line and pixel by pixel in each of the successive lines (Fig. 1) in the unit 18, as well as the corresponding signals DP and CO which are displayed.

The signals representative of the contemporaneous CO and DP of the matrix 21 are available, at a given instant, on the 16 x 17 = 272 outputs of the shift registers, as well as upstream of the 17 registers at the head of the 17 lines, c ' that is to say, registers <i>d</i>_0.1, <i>d</i>_1.1 ... <i>d</i>_16.1, which makes a total of 16 x 17 + 17 = 17 x 17 outputs for the 17 x 17 P positions_0.0, P_0.1, ... P_8.8 ... P_16.16.

Inside the matrix 21, around the center of it <i><u>e</u></i> Coordinates <i>x</i> = 8, <i>there</i> = 8 (and that is why the number of rows and the number of columns of the matrix 21 is preferably odd: 2<i>l</i> + 1 and 2<i>m</i> + 1 respectively), we can consider in particular a small matrix of 3 rows and 3 columns whose central element of its 9 elements is precisely the pixel <i><u>e</u></i> Coordinates <i>x</i> = 8, <i>there</i> = 8. Either <tables id="tabl0001" num="0001"><table frame="all"><tgroup cols="3" colsep="1" rowsep="0"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left"><i>at</i></entry><entry namest="col2" nameend="col2" align="left"><i>b</i></entry><entry namest="col3" nameend="col3" align="left"><i>c</i></entry></row><row><entry namest="col1" nameend="col1" align="left"><i>d</i></entry><entry namest="col2" nameend="col2" align="left"><u><i>e</i></u></entry><entry namest="col3" nameend="col3" align="left"><i>f</i> (M3)</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left"><i>g</i></entry><entry namest="col2" nameend="col2" align="left"><i>h</i></entry><entry namest="col3" nameend="col3" align="left"><i>i</i></entry></row></tbody></tgroup></table></tables> this small matrix, whose central element <i><u>e</u></i> was emphasized.

To this matrix of 3 x 3 elements, comprising 8 locations <i>at</i>, <i>b, c,</i> d, <i>f</i>, <i>g, h</i>, <i>i</i> all around the element or central location <i><u>e</u></i>, one can match 8 oriented directions starting each of the central location <i><u>e</u></i> and ending in one of the 8 others.

For this purpose, the 8 directions can be identified by means of the Freeman code illustrated in FIG. 6, the directions being coded from 0 to 7, from the axis of<i>x</i>, from 45 ° to 45 °. In Freeman code, the 8 possible oriented directions numbered from 0 to 7, can be represented by a 3-bit number, because 2³ = 8, ie 8 possibilities.

If we take the previous small matrix M3, the 8 directions according to Freeman's code from the central position <i><u>e</u></i> are the following <tables id="tabl0002" num="0002"><table frame="all"><tgroup cols="3" colsep="1" rowsep="0"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">3</entry><entry namest="col2" nameend="col2" align="left">2</entry><entry namest="col3" nameend="col3" align="left">1</entry></row><row><entry namest="col1" nameend="col1" align="left">4</entry><entry namest="col2" nameend="col2" align="left"><i><u>e</u></i></entry><entry namest="col3" nameend="col3" align="left">0</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">5</entry><entry namest="col2" nameend="col2" align="left">6</entry><entry namest="col3" nameend="col3" align="left">7</entry></row></tbody></tgroup></table></tables> as explained in FIG. 6.

Returning to matrix 21 of FIG. 4 to 17 x 17 image points or pixels, we will first expose how one locates there an area in relative displacement with respect to a substantially immobile environment in the scene, observed by the video camera 13 and thus represented in the signal digital video S composed of PI pixels_ijt, and how the velocity and the direction of an effective displacement is determined with respect to a substantially immobile environment.

Between two successive frames, such as TR₁ and TR₂ (Fig. 5), the IPs_ij pixels of the signal S will be characterized, as regards their variation between the instant <i>t</i>₁ (first frame) and the moment <i>t</i>₂ (second frame) by the two DP signals_ij and co_ij distributed by scanning according to the matrix 21.

There is a significant variation of the pixel value at a point of this matrix if DP = 1 for this point. Thus a zone in actual displacement is marked by the area of ​​the matrix within which DP = 1 at each point.

In fact, we simultaneously examine, in a computing unit 17a, inside the matrix 21 at 17 × 17 locations, the different square matrices nested centered on <i><u>e</u></i> dimensions 15 x 15, 13 x 13, 11 x 11, 9 x 9, 7 x 7, 5 x 5 and 3 x 3, the latter being the matrix M3 above. It is determined which of these is the smallest matrix to have DP = 1 aligned along a line that determines the direction of movement of the area in which DP = 1 and which determines a variation of + 1 and - 1 around CO. For a variation of + 1 and -1 around CO, DP = 1 at each value must be accepted to accept the test. It is the smallest matrix participating in the test that is chosen (line of greatest slope).

Then inside this zone moving in one of the nested matrices, for example in the small matrix M3 of 3 x 3 elements, it is determined whether CO varies on each side of the central position, in a given direction, of + 1 in an oriented direction and - 1 in the same direction but in the opposite direction. For example if we have - 1, 0, + 1 in the direction (oriented) 1, that is to say in the positions<i>g</i>, <i>e, c</i> respectively of the small matrix M3, then the displacement exists in this matrix from right to left in the direction (oriented) 1 of the Freeman code (Fig. 6). Of course simultaneously in this direction of the small matrix DP = 1. The more CO varies by + 1 or - 1 between two neighboring positions in a direction in a larger matrix among the nested matrices of 3 x 3 to 15 x 15, the greater the speed of displacement. For example if we have - 1, 0, + 1 in the oriented direction 1, that is to say <i>g, e, c,</i> in the matrix of 9 x 9, referenced M₉ the displacement will be faster than in the case where we have - 1, 0, + 1 in the matrix M₃ 3 x 3 (Fig. 7).

Since CO is a power of 2 and is represented by this power in the preferred embodiments, we can locate a wide range of speeds using only a few bits for the power of 2, while even finding a relatively low speed (which can be chosen by increasing the deviation for the test, for example - 2, 0 + 2 in the matrix M3 of 3 x 3 indicates a speed two times lower compared to the speed corresponding to the matrix M3 of -1, 0 + 1, and for the same positions <i>g, e</i>,<i>c</i>).

In addition, two tests are required to remove the uncertainties:<ul id="ul0020" list-style="dash" compact="compact"><li>a first test chooses the highest variation, that is to say the highest time constant, in the case where one has in one of the nested matrices, for example in the small matrix M3 of 3 x 3 elements, variations of CO in several directions;</li><li>a second test chooses, arbitrarily, one of two (or more) directions in which the CO variation is identical, for example by choosing the lowest value of the Freeman code; in fact such a case occurs most of the time when the actual (directed) direction of the displacement is substantially between two successive coded directions in the Freeman code, for example between directions 1 and 2, which corresponds to a direction (oriented) that can be noted 1, 5 (Fig. 6), about 67.5 ° with that of the axis of <i>x</i> (direction 0 in the Freeman code).</li></ul> The determination of the oriented direction and the speed of a zone in effective displacement in the manner just described is carried out by the calculation unit 17a (FIG 4) associated with the unit 17 and which receives the 17 x 17 aforementioned outputs of the matrix 21, for both CO and DP (two outputs of 21, namely <i>s</i>_0.1 and <i>s</i>_0.16, are represented). The unit 17a processes the values ​​of CO and DP for the successive nested matrices and determines from these values ​​the oriented direction (in the Freeman code) and the speed of the displacement (according to the matrix to be retained among the nested matrices), possibly by applying the above tests.

The scrolling of the entire frame of the digital video signal through the matrix 21 takes place<ul id="ul0021" list-style="dash" compact="compact"><li>first for the group of the first 17 lines, lines 1 to 17, of the frame: from the left to the right of the frame considering the relative movement, as illustrated for the TR frame₂ in FIG. 5: of the TM portion₁ on the far left, then TM₂ offset by one column from TM₁ and this up to TM_M (M being the number of pixels per raster line) on the far right;</li><li>then, from the left to the column column right, for lines 2 to 18 of the frame; and</li><li>and so on each time going down one line: line 3 to 19 ... until the last group at the bottom of the frame, namely that of lines L - 16 ... L (L being the number of lines per frame).</li></ul>

Considering Figs. 2 and 4 we see that the outputs of the units 17, 18 and 19, that is to say of the set of spatial treatments, are:<ul id="ul0022" list-style="dash" compact="compact"><li>a signal V representing the speed of the displacement, according to the amplitude of the maximum variation of CO in the marked zone, whose value can be for example represented by a scale of eight integers from 0 to 7 if the speed is under the numerical form of powers of 2, therefore has 3 bits;</li><li>a signal DI representing the direction of this displacement, according to the direction of this maximum variation, the value of DI may also be represented by one of eight values ​​from 0 to 7 in the Freeman code, and therefore comprise 3 bits;</li><li>a validation signal VL specifying that the result for the speed and the direction (oriented) is valid, in order to be able to distinguish a valid output with V = 0 and DI = 0, the absence of an output due to an incident, this signal being either 1 (valid output) or O (no output); therefore only one bit is needed for VL;</li><li>advantageously a signal CO of time constant, therefore of 3 bits for example;</li><li>(the 3 or 4 V, DI, VL and possibly CO signals being output by the spatial processing unit 17 and its associated electronics)</li><li>a delayed video signal SR constituted by the input video signal S delayed in the delay unit 18 of the 16 consecutive lengths of lines TR and therefore the duration of the distribution of the signal S in the matrix 21 of 17 × 17, in order to have the digital video signal contemporary to the matrix representation in the matrix 21, a signal whose content can be displayed in clear on the screen of a television or a monitor;</li><li>all three output signals of the unit 19, namely the HP clock signals, SL line sequence and SC column sequence.</li></ul>

To the nested rectangular matrices of FIGS. 4 and 7, nested hexagonal matrices (Fig. 8) or an inverted L-shaped matrix (Fig. 9) can be substituted.

In the case of FIG. 8, the nested matrices (of which only the most central matrices MR1 ​​and MR2 have been represented) are all centered on the point MR0 which corresponds to the central point (in which the binary signal is "0") of the matrices M3, M9 of FIG. 7. The advantage of a system of hexagonal matrices is that it allows, on the one hand, to use oblique coordinate axes,<i>x</i>_at, <i>there</i>_at and, on the other hand, a decomposition into triangles with identical sides, which calculates the isotropic velocities.

The matrix of FIG. 9 is composed of a single line L_u and a single column C_u from the MR central box_u in which the two signals DP and CO are respectively "1" for DP and increases or decreases by one unit for CO, in case of displacement.

It is thus determined if the direction of displacement (relative) is<ul id="ul0023" list-style="dash" compact="compact"><li>in the sense of the coordinate <i>x</i> : the CO signal is identical in all the boxes of column C_u, and the binary signal DP is equal to 1 in the cells of the line L_u, of origin MR_u at CO value_u to the box where CO is CO_u + 1 or - 1 inclusive;</li><li>in the sense of the coordinate <i>there</i> : the CO signal is identical in all the boxes of the line L_u, and the binary signal DP is equal to 1 in the boxes of column C_u, of origin MR_u at CO value_u to the box where CO is CO_u + 1 or - 1 inclusive;</li><li>or finally obliquely relative to <i>x</i> and <i>there</i> : the binary signal DP is equal to 1 and CO is CO_u in the boxes of L_u and in the boxes of C_u, the slope being determined by the perpendicular to the line passing through the two boxes in which the CO signal_u changes the value of one unit, the signal DP always being equal to 1.</li></ul>

In FIG. 9 we have illustrated the case where DP = 1 and CO_u changes the value of a unit in two particular boxes L_u3 and C_u5 and indicated the corresponding slope P_p.

In all cases, the speed of displacement is a function of the box in which CO changes value by one unit.

If CO changes a unit only in L_u or C_u , it corresponds to the value of the CO variation box.

If CO changes a unit in a box of L_u and in a box of C_u, the speed is proportional to the distance between MR_u summer_x (intersection of the line perpendicular to C_u -L_u passing through MR_u).

The system so far described with reference to FIGS. 1 9 is advantageously completed by a complementary system which will be described with reference to FIGS. 11 to 16 to form a global system 22 illustrated in FIG. 10, on which is shown, on the one hand, the assembly 11a of FIG. 2 with indication of the signals V, DI, VL, C, SR and composite F (HP, SL, SC) sent by the set 11 (which receives the input digital video signal S) to the assembly 22a (which outputs a composite output ZH).

The assembly line Z - Z₁ between the assemblies 11a and 22a is illustrated in FIGS. 2, 10 and 11, the outputs of the assembly 11a being connected to the inputs of the assembly 22a according to Z - Z₁ to transmit the aforementioned signals.

The output of the unit 22a and therefore of the global device 22 is constituted by a composite signal ZH providing the desired information on the relatively moving area of ​​the scene 13a monitored by the video camera 13.

The complementary unit 22a, connected to the outputs of the unit 11a, is illustrated in FIG. 11 in the form of functional blocks, recalling that this figure is connected along the line Z - Z₁ (at its upper part) in FIG. 2 along the line Z - Z₁ (at the bottom of this last figure).

The unit of FIG. It essentially constitutes a device for forming and using histograms and it comprises:<ul id="ul0024" list-style="dash" compact="compact"><li>a micro-bus line 23 which carries a number of digital signals, detailed below;</li><li>six blocks or units for forming and processing histograms, referenced 24, 25, 26, 27, 28, 29, respectively for the histograms of delayed digital video signals SR, of velocities V, of oriented directions (in Freeman code) D1, time constants CO, first axes <i>x</i>(<i>m</i>) and second axes <i>there</i>(<i>m</i>);</li><li>six blocks or linear combination units 30, 31, 32, 33, 34, 35 associating their inputs from the bus 23 to each form a validation signal V₁, V₂, V₃, V₄, V₅, V₆ for the six blocks 24, 25, 26, 27, 28, 29 respectively, 30 being associated with 24, 31 being associated with 25, 32 being associated with 26, 33 being associated with 27, 34 being associated with 28 and being associated at 29;</li><li>a moving block or zone unit 36 ​​coordinating the outputs of the blocks 28 and 29 for the axes <i>x</i>(<i>m</i>) and <i>there</i>(<i>m</i>); and</li><li>a reference change block or unit 37 receiving signals <i>x</i>(<i>m</i>)₀ and <i>there</i>(<i>m</i>)₀ axis orientation <i>x</i>(<i>m</i>) and <i>there</i>(<i>m</i>), as well as HP pixel, SL line and SC column clock signals (these three signals being grouped in the beam F of Figures 2, 4, 10 and 11) from the unit 19 Figs. 2 and 4, and forming the signals<i>x</i>(<i>m</i>)₁ and <i>there</i>(<i>m</i>)₁ sent to units 28 and 29 respectively.</li></ul>

The units 24, 25, 26 and 27 each output its output SSR, SV, SDI, SDO on the bus 23, while the units 28 and 29 each output a signal <i>x</i>(<i>m</i>)₂, <i>there</i>(<i>m</i>)₂ on one of the two inputs of the moving zone unit 36, which combines these two signals from the units 28 and 29 and outputs a composite signal <i>xy</i>(<i>m</i>) on the bus 23.

The operation of each of the histogram formation and processing units 24 to 29, which receives as input, on the one hand, a validation signal V₁, V₂, V₃, V₄, V₅ or V₆ of the associated linear combination unit 30 to 35 and, on the other hand, a signal SR, V, DI, CO, <i>x</i>(<i>m</i>)₁ or <i>there</i>(<i>m</i>)₁ treat, is the same and that's why, we will expose the operation of only one of these units, namely unit 25 concerning the formation and processing of speed histograms V, taken as an example; only the treated variable is different for the other similar units 24, 26 27 28 and 29, noting that the different input signals for the six units 24 to 29 are all digital signals, which allows an analogy of structure and operation of these six units.

In FIG. 12, schematically shows, each by its envelope, the histograms 38 and 39, respectively in<i>x</i> and in <i>there</i> (Cartesian coordinate axes of the matrix 21 to 17 x 17 elements of Fig. 4), velocities V of the moving zone (in Fig. 14, elements such as C are indicated₁, C₂ the envelope histogram 38); <i>x</i>_M and <i>there</i>_M represent the coordinates in <i>x</i> and in <i>there</i> maxima of both envelopes 38 and 39, respectively, while <i>l</i>_at and <i>l</i>_b, for the axis of <i>x</i> and <i>l</i>_c and <i>l</i>_d, for the axis of <i>there</i> represent the limits of the domain of significant or interesting speeds, <i>l</i>_at and <i>l</i>_c being the lower limits and <i>l</i>_b and <i>l</i>_d the upper limits of the significant portions of the histograms.

Vertical lines L_at and L_b, abscissa <i>l</i>_at and <i>l</i>_b, and the horizontal lines L_c and L_d, ordinates <i>l</i>_c and <i>l</i>_d, form a rectangle that frames the hatched area 40 significant speeds (for all directions <i>x</i> and <i>there</i>), a few microzones 41, of lower speeds and which will be ignored, existing in the vicinity of the main zone 40.

It is therefore sufficient to locate the coordinates of the four boundaries <i>l</i>_at, <i>l</i>_b, <i>l</i>_c, <i>l</i>_d and two maximas <i>x</i>_M and <i>there</i>_M to characterize the zone of greater variation of the parameter set in histogram, the speed V in the particular case considered. It is this information concerning V that the block 25 continuously outputs on the micro-bus 23.

Likewise, the similar blocks 24, 26 and 27 continuously output on this micro-bus 23 the information concerning the maximum value area for SR, DI and CO, respectively.

Finally, the similar blocks 28 and 29 continuously output at unit 36 ​​the information concerning the maximum value area of <i>x</i>(<i>m</i>)₁ and <i>there</i>(<i>m</i>)₁ respectively, the unit 36 ​​combining this information of abscissa and ordinate, referenced <i>x</i>(<i>m</i>)₂ and <i>there</i>(<i>m</i>)₂ respectively, into a composite signal <i>xy</i>(<i>m</i>) which is sent by the output of the unit 36 ​​on the bus 23.

Finally, the bus 23 conveys the information concerning the zone of maximum values ​​of SR, V, DI, CO and <i>xy</i>(<i>m</i>), that is to say <i>x</i>(<i>m</i>)₁ and <i>x</i>(<i>m</i>)₂, which makes it possible to determine the existence of a moving zone in the scene observed by the video camera 13, to locate this zone and to determine its speed and the direction (oriented) of displacement.

In FIG. 11 the output composite signal available on the bus 23 has been referenced ZH. The aforementioned components of this signal ZH can, in particular as regards V and DI, that is to say the speed and the direction of the moving zone, be displayed in digital or analog form, trigger a light signal and / or sound, especially if a threshold is exceeded by the speed V, or be transmitted by cable, optical fiber or radio path for remote use, in a control unit, such as the unit 10a. of FIG. 1, located in the vicinity or at a distance from the device 11 according to the invention.

The portion of the units of FIG. 12 above the bus 23 serves, as a result of the point-by-point processing of an entire frame and the association of the points of a frame to deduce an overall external value, to determine the existence and location of a zone of the scene observed in relative motion and, furthermore, if there is indeed displacement of this zone, the speed and the direction of this displacement. The location in the observation plane of this area in relative movement in two directions <i>x</i>, <i>there</i>, which may not be orthogonal (as in the case of the application of Figures 15 and 16 for example) is achieved by the portion of the units of FIG. 12 below the bus 23.

We will now discuss in more detail, with reference to FIGS. 12, 13 and 14, the structure and operation of a histogram formation and processing block, such as 25, and its associated linear combination block, such as 31.

Block 25 (FIG 13) comprises a portion 25a forming the histogram and a portion 25b constituting a classifier for the histogram, these two portions operate under the control of software, embodied in an integrated circuit portion 25c, which perform the extraction of the limits <i>l</i>_at, <i>l</i>_b, <i>l</i>_c, <i>l</i>_d of the histogram (Fig. 11).

The classifier 25b feeds, as well as the classifiers of the other blocks of formation and histogram processing 24, 26, 27, 28, 29 (for these last two through the unit 36 ​​of combination in <i>x</i>(<i>m</i>) and <i>there</i>(<i>m</i>)), the bus 23 and, through it, the linear combination unit 31 which therefore receives in parallel information from all the classifiers of the units 24, 25, 26, 27, 28, 29 and which , based on this information, debits or not the validation signal V₂ in unit 25.

With the software of 25c, the classifier 25b determines the different classes (each having the same number of values ​​of the speed in the case envisaged) which will define the envelope such as 38 or 39 (FIG 12).

In FIG. 14, we have illustrated, along the axis of<i>x</i>, successive classes C₁, C₂ ... C_n-1, C_not and their envelope 38 for the speed V which are determined in the classifier 25b.

Figs. 15 and 16 illustrate the role of histograms for<i>x</i>(<i>m</i>) and <i>there</i>(<i>m</i>) formed by the units 28 and 29 and combined in the unit 36 ​​to obtain a slope.

By way of example, the case of the observation of a road by means of a digital video camera embedded in a vehicle with its device according to the associated invention has been considered.

In FIG. 15 represented the two left edges Bg and right B_d of a road R, as well as the slopes numbered from 0 to 7 for example (according to a convention other than the Freeman code) of the projection P_x in <i>x</i>(<i>m</i>) performed by unit 28 and projection P_there in <i>there</i>(<i>m</i>) performed by unit 29.

To ensure the best accuracy with regard to the right edge B_d, that is to say a maximum sensitivity of the indications, for example speed, concerning this edge, it is necessary that the projection P_x has the slope closest to the optimum slope P₀ which is perpendicular to B_d, in particular the slope 5 in the representation of FIG. 15. The maximum value of the velocity histogram will therefore be obtained for the slope determined by the unit 28 (Fig. 11).

The same reasoning applies to the left edge B_g as regards the slope of the projection Py and therefore of the role of the unit 29.

The combination unit 36 ​​of the two optimum slopes provides the optimization information for both B edges together._d and B_g.

Fig. 16 illustrates the application of the determination of the optimum slope P_o of the projection P_x, taken by way of example, to ensure a correct driving of a VH vehicle in continental Europe driving on the right of the road (case a), in the UK driving on the left of the road (case b) and finally of an airplane Go for a correct landing in the middle of the runway of an airport (case c).

Therefore, to assist the driving of a land vehicle (car, truck) on a road or an air vehicle (airplane, space shuttle) in the vicinity of an airport runway, the device according to the invention comprises in in addition to means for representing the right edges B<i>d</i> and left B<i>g</i> of the road, respectively of the track, and means for orienting at least one of the axes, v ariable slope , coordinates so that it is maintained substantially orthogonal to the corresponding edge (position P₀).

Heretofore, it has been described above, the means for determining an area in actual displacement in a substantially immobile environment, by locating a region in which DP = 1. To determine an area at rest in an environment in the moving ensemble (for example when stopping a broken down vehicle or a collision on a motorway for example), it is the zones in which DP = 0 which must be localized with respect to the environment for which DP = 1. Of course in this case, the velocities will be zero in an area and the notion of direction has no meaning. The calculations in 17a will therefore be different.

The device, according to the invention, if it is only used to determine a stationary area can be simplified with deletion of units or blocks processing speeds and directions, including blocks 25, 26, 31, 32, and reduction the number of outputs of unit 11a and the inputs of unit 17.

In the operating unit 10a of the device according to the invention, it is possible to provide means for displaying, on a monitor screen, histograms and / or values ​​of the DP or CO signals.

As for the signal SR, that is to say the delayed digital video signal, it is generally applied to a television or monitor 10 in order to display, locally or remotely on the screen thereof, this signal to the television. moment when a relative motion is signaled to verify the nature of this relative motion. It will therefore suffice to observe the screen of the television or monitor 10 only when a zone in relative displacement has been signaled, for example by a visual and / or audible alarm.

It is advantageous to locate a region in relative motion on the screen by imposing arbitrary colors on the delayed digital video signal SR, each color or color shade representing a speed and / or a direction of the movement.

The various units which have just been described with reference to FIGS. 2, 3, 4, 11 and 13 are each achievable by electronic circuits of known type, in particular microprocessors performing calculations and / or comparisons or using: scanning signals; memories; units with delays; shift registers; units forming linear histograms and associating in the plane such histograms; microbuses.

The combination of these electronic circuits in separate units 11a and 22a or in an assembly 22 constituted by 11a and 22a can be made in two integrated circuits or in a single integrated circuit of very small size, of the order of 10 mm × 10 mm for example in 0.7 μm technology, all of the two interconnected integrated units or the single integrated circuit being connected by its input to the digital video output of a video camera or other observation device and by their outputs to one or more on-site distance. In a variant, if only the simplified device of the unit 11a is used, this unit, preferably in the form of a single integrated circuit, is arranged between the said digital output and one or more on-site devices. or remotely.

We will now give as non-limiting examples, additional applications of a device according to the invention, in particular the device according to FIGS. 1 and 10, that is to say according to all of Figs. 2 and 11 connected along line Z - Z¹.

A first additional application, illustrated in FIG. 17, is constituted by the automatic framing of a person moving in a room, for example in the context of a videoconference. Automatic framing eliminates the movements of the person moving, which increases the definition of the image of this person observed by a video camera with digital output and also, in the case of compression of the digital video signal, simplifies this compression .

In FIG. 17, there is the video camera 13 which observes the person P can move. The digital video signal S of the video camera is not only transmitted by cable, optical fiber or radio link to a TV screen or monitor 10b, but also received by the device 11 according to the invention, whose output ZH acts on a unit 42 which, in response to the signals received from the device 11 concerning the location and movement of the person P, controls the motors 43 of the camera 13 to direct the optical axis of the camera towards the person, especially his face F, according to the location of its movement and the speed and direction of it and possibly to act on the zoom setting, the focal length and / or the focus of the camera in case of advance or recoil of the person P.

One can also control the tracking by at least one light spot of a character (actor, singer) on a stage, the device according to the invention centering the character in the center of the image by moving the orientation of the video camera and controlling the direction of the spot, for example by one or two orientable mirrors, for each spot.

Another application, given by way of example, of the device according to the invention is illustrated in FIG. 18 on which there is the camera 13 or other observation device which delivers a digital video signal S in a device 11 according to the invention. In this application, the camera 13 monitors a portion of the highway to detect the unexpected stop of a vehicle, in particular on the emergency stop band, or a car stop following a collision.

It is therefore in this case to determine the immobility of an object (the vehicle) in a moving environment (the other vehicles), that is to say to locate in the matrix 21 to 17 x 17 elements the zone in which DP = 0. Normally the camera 13 observes a stream of vehicles that produce responses DP = 1, with values ​​of speed and direction of movement. On the other hand, if a vehicle stops, it is indicated by a response DP = 0 in its observation zone.

A unit 44 receiving ZH, as well as SR, detects the appearance of a zone in which DP = 0 in ZH and outputs a fault signal NL which, Firstly, triggers an audible and / or luminous alarm in a device 45 and, on the other hand, controls a switch 46 which applies the video signal S (or rather the delayed video signal SR) to the screen of a television or monitor 10 which allows the supervisor, alerted by the audible and / or visual alarm, observe the highway as soon as a vehicle stops or collides in order to take the necessary measures, for example following the driver reaction of the stopped vehicle.

Thanks to the invention, in a monitoring station, a supervisor can easily control a large number of sections on a highway (or a road) in each of which is disposed a camera 13 and a device 11 according to the invention, the outputs ZH and SR of each device 11 being transmitted by cable, optical fiber or radio path to a single monitoring station where there is a common unit 44; indeed the observer needs to look at the screen of 10 only in case of incident or accident, signaled by the alarm, and it is rare for several incidents and / or accidents at different locations to occur simultaneously.

In particular, the unit 44 may comprise, at the input, a rotary type switch (not shown) sending successively and cyclically the signals ZH (and SR) of the different sets 13 - 11 arranged along the highway on the portion of this unit 44 producing the NL signal.

The same system makes it possible to detect not only, as indicated, the stop, or the collision resulting in a stop of vehicles, but also a slowing down of the flow of vehicles (in the event of very slow traffic) by decreasing the speed in the different zones. in which D = 1 and conversely a vehicle at excessive speed, the speed in an observed area then exceeding the authorized speed limit.

Finally, in FIG. Another application of the invention has been illustrated, namely to the human-machine interaction, the movement of the hand M, or more particularly of the fingers DG, inside a surface SF decomposed in rectangles by a coordinate system C_x and C_there.

A video camera 13 with a digital output associated with a device 11 according to the invention, as illustrated in FIG. 1, to recognize the movement of a hand M and DG fingers and use it to control a computer (in the manner of a "mouse") or to control certain functions in a machine. For example, the set 13 - 11 could be used by a deaf-mute, using the standard code of the deaf-mute language based on movements of the hands, to enter alphanumeric data, thus a text, without having to use the computer. usual keyboard; this operation can, of course, also be carried out by a person speaking, having learned the language of the deaf-mutes, to enter a text in computer without having to use a keyboard. Such a system is not sensitive to temporal differences and does not require precise signaling of the beginning and the end of the gesture.

Figs. 20 and 21 schematically illustrate the application of the invention to the monitoring of an automobile driver to signal sleepiness thereof.

In this case, the video camera 13 is placed against the portion of the bodywork inside the vehicle, above the rearview mirror for example, and observes the driver.

The preliminary operation is to frame it as in the case of the application of FIG. 17. In FIG. 20, there is shown symbolically the image 1C of the driver on the video screen. We first delete unnecessary portions of the left and right (horizontal hatching of the image and is thus limited to the central portion of the image between these two portions.

Then in this central portion simply monitor the unhatched area AA of FIG. 21 in which is framed the head.

The interesting movements that are detected by the device according to the invention are constituted by the blinking of the eyelids of the driver (indicated by vertical movements in the area AA) whose rate changes in preliminary to falling asleep. If the speed or and the speed of these blinks becomes lower than a certain threshold, an audible alarm is triggered and the driver is awake.

In FIG. 22, a means has been indicated to overcome, when necessary, the limited number of bits<i>p</i> representative of the CO time constant to allow for a wider range of travel speeds.

For this purpose the use of the Mallat diagram is foreseen (see article by S. Mallat "A Theory for multiresolution signal decomposition" in IEEE Transactions on Pattern Analysis and Machine Intelligence, July 1989 pp. 674-693). which consists in successively breaking down the totality of the video image in half successively, labeled 1, 2, 3, 4, 5, 6, 7. A compression is thus carried out by processing only portions of images. We can thus with<i>p</i> = 4, that is 2^P = 16 determine a speed in a wider range.

If at the beginning, as part of the total image, the device according to the invention indicates that the speed of the moving object (in the broad sense) exceeds the maximum speed that can be determined with 2^P = 16 for the time constant, it suffices to pass successively by the partial images observed 1, 2, 3, 4, ... until the speed of the moving object does not exceed said maximum speed in the frame of the partial image after compression.

To implement the Mallat composition by wavelets, simply insert in the diagram of FIG. 1 a unit 13A (shown in Fig. 22) that performs this compression of the video signal. This unit may for example be constituted by the "ADV 601 Low Cost Multiformat Video Codec" component of the American company ANALOG DEVICES which is the subject of the "Adv 601 Preliminary Data Sheet" notice of January 1996. In FIG. 2, such an optional compression unit 13a.

Finally, in Figs. 2 and 3, the number of bits (1, 3 (in the case where<i>p</i> = 3), 8, 15), which shows the economy of size possible for the different functional blocks having to process a reduced number of bits.

It can finally be seen that the invention makes it possible to detect a relative movement in a scene observed by an optoelectronic device, such as a video camera, which transmits the observed scene in the form of a digital video signal constituted by a succession of frames, themselves constituted by a succession of lines composed of a succession of pixels, this digital signal being analyzed in order to identify a zone in relative displacement, with an indication of the speed and direction (oriented) of this movement if the area is in actual motion relative to a substantially immobile environment.

Since the device according to the invention determines the oriented direction and the speed of movement of an object (in the broadest sense), it can be added means for deriving from these two parameters a future position of the object to a given moment and means for directing the input video camera 13 in advance to this future position.

It should be noted that the results obtained with the device according to the invention in no way require that the camera be immobile, which makes it possible to board the camera and the associated device on a land, air or water vehicle (for the implementation of the process illustrated in Fig. 16 for example).

After a very short initialization period equal to N (of the order of less than ten corresponding successive frames), the device according to the invention determines the parameters of the relative displacement instantaneously after the end of each frame having undergone the temporal and spatial, because of the recursivity of the calculations according to the invention.

We have described a preferred embodiment of the device according to the invention and some applications thereof. Of course, this embodiment and these applications have been given by way of non-limiting examples and many variants and adaptations, which will be readily apparent to those skilled in the art, can be envisaged without departing from the scope of the invention. as defined in the claims below.

For example, it would be possible to use means for using the signals outputted by the unit 11 of FIG. 2 other than those illustrated in FIG. 11 without departing from the invention.

As for the applications of a device according to the invention, they are not limited to those given by way of example in the above description. Thus a mounting similar to that of FIG. 17 can be directly associated with a camcorder to stabilize it in relation to displacements caused by involuntary movements of the user.

One can also use one or, preferably, several devices, according to the invention, associated respectively with one or, preferably, a plurality of digital video output camcorders placed in a building room to constitute a "smart room" with such a system making it possible to detect and locate the presence and movement of one or more persons in the room, to analyze this movement, for security purposes, identification and / or to assist in the tasks to be performed, for example, or to look after children in another room or the customers of a supermarket.

Essentially, it has been assumed to use a video signal with successive interleaved frame pairs, in particular during the discussion of the capacity of the memory 16, with processing of the two frames of a pair in the device according to the invention. However, it is possible to use only one frame out of two (the odd one for example) by reducing the capacity of the memory, but with a reduction of about half the speed of obtaining the desired information. It is also possible to implement a video camera or other observation device whose digital output has only one frame per image.

It is possible, in certain applications, to associate, with the device according to the invention, specialized sensors, for example one or more acceleration sensors, so as to be able to process additional parameters of the displacement.

Naturally, the invention is not limited to the particular embodiments or to the applications described, but embraces all the variants and modifications that fall within the general definition of the invention.