Picture processing device.

This invention relates to a picture processing device which is used in controlling unmanned running vehicles or recognizing motions of objects to be observed in real time.

In controlling an unmanned movable robot or an automatic movable vehicle, for example, pictures of the lines marking off a lane, and those of the center line and the shoulder lines of a road have to be taken by a camera to be subjected to picture processing. Fig. 1 is a view explaining recognition of a road by means of a picture. Fig. 1(a) shows a picture of a road taken by a camera. Fig. 1(b) is a view in which the picture elements in Fig. 1(a) having higher brightness (or higher brightness change ratios) are shown in black points.

As shown in Fig. 1(a), in a camera picture 1 a road 3 is extended at infinity toward a horizontal line 2. Shoulder lines 4 are on both sides of the road 3. A center line 5 is at the center of the road 3. The shoulder lines 4 and the center line 5 of the road 3 have higher brightnesses than the other parts of the road 3 and are shown in continuous dots 4′, 5′ as shown in Fig. 1(b). Based on this camera picture 1, in order to recognize the direction and curves of the road 3, approximate straight lines L₁, L₂, L₃ interconnecting the dots 4′ in Fig. 1(b) are recognized.

As a method for giving the approximate straight lines L, the method called Hough transformation is conventionally known (e.g., Specification of U.S. Patent No. 3069654). This method is explained with reference to Figs. 2 to 4. Assuming that a point P (x_p,y_p) to be procssed is present on an original picture depicted by the x y coordinate system as shown in Fig. 2(a), an infinite number of straight lines l (l_a, l_b, ...) can be depicted. Straight lines passing through the origin O (0,0) and orthogonal to the respective straight lines l_a, l_b, ... can be also depicted. If a length of the straight lines passing through the origin O (0,0) to the straight lines l (l_a, l_b,...) is represented byρ, and an angle these straight lines form to the x axis is represented byϑ (ϑ_a, ϑ_b,...), the ρ and ϑ of the straight lines passing through the origin O (0,0) are given in the sine curve of Fig. 2(b), i.e., the Hough curve. The length ρ_max between the origin O (0,0) and the point to be processed P (x_p,y_p) is the longest of the straight lines passing through the point to be processed P (x_p,y_p). The length is given by ρ_max = (x_P² + Y_P²)^1/2 and when ϑ=0, ρ₀=x_p.

Next, as shown in Fig. 3(a), the Hough transform of Fig. 2 will be applied to three points P₁ ∼ P₃ on a straight line L. The sine curve (Hough curve) in the dot line in Fig. 3(b) is given with respect to the point P₁. The sine curve with respect to the point P₂ is in the one dot line in Fig. 3(b), and that with respect to the point P₃ is in the two dot line in Fig. 3(b). The peaks (ρ₁,ϑ₁), (ρ₂,ϑ₂) and (ρ₃,ϑ₃) of the respective sine curves of Fig. 3(b) correspond to the lengths ρ₁, ρ₂, ρ₃ between the origin O (0,0) and the respective points P₁, P₂ and P₃ in Fig. 3(a), and the angles ϑ₁, ϑ₂, ϑ₃ the respective straight lines passing through the origin O form to the x axis.

In Fig. 3(b), the point of intersection of the three Hough curves (sine curves) has coordinates (ρ_t,ϑ_t) which are equal to (ρ_t,ϑ_t) of a straight line passing through the origin O (0,0) and orthogonal to the straight line L. Thus by obtaining a point of intersection of such sine curves, an approximate straight line of the curves interconnecting the dots (dark points) in the x-y rectangular coordinate system of an original picture can be obtained (in Fig, 3, however, the curves and the approximate straight line are in agreement).

This will be explained with reference to Fig. 4. It is assumed that a number of dots (points to be processed) to be Hough transformed are on a curve in the x-y coordinate plane (original picture plane) in Fig. 4(a). In Fig. 4(a) three approximate straight lines L₁,L₂,L₃ interconnecting the dots can be depicted. When all the dots are subjected to Hough transformation as in Fig. 2, three points of intersections of the sine curves are obtained. The three points of intersection have coordinates (ρ_t1,ϑ_t1), (ρ_t2,ϑ_t2), (ρ_t3,ϑ_t3) in Fig. 4(a). When the occurence of the point of intersection H is expressed by a ρ,ϑ,H coordinate system, Fig. 4(b) is given. Accordingly, the approximate straight lines L₁ ∼ L₃ of Fig. 1(b) corresponding to the shoulder lines 4 of the road 3 can be given in values of ρ and ϑ at a peak of H (the occurrence of the point of intersection).

But it is not easy to apply the above described Hough transformation to high speed picture processing in real time. That is, when a Hough curve (sine curve) is given, based on coordinates (x_p,y_p) of the point P to be processed on the original picture of Fig. 2 as data, ρ = x_P · sinϑ + Y_P · cosϑ      (1) must be satisfied. For example, when ρ is divided into 512 sections, the trigonometric function calculation has to be performed 1024 times; the multiplication, 1024 times; and the addition, 512 times. Besides, when the original picture to be subjected to the computation comprises 512x512 picture elements, the computation times totally become so numerous that the processing time becomes extremely long with a usual processor.

Of course it is possible to shorten the computation time by storing values of sinϑ and cosϑ used in the above described formula 1 in a ROM or others (e.g., Hanahara et al., A realtime Processor for Hough Transform, 1985 National Conference of Information System Group, The Institute of Electronics Information and Communication engineers, No. 92 or Onda et al. A Hardware Implementation of Hough Transform with ROM , The 70th Anniversary General National Conference of the Institute of Electronics Information and Communication Engineers, No. 1587). But in Hanahara et al., in which the data of the trigonometric calculation are stored in a ROM, the multiplication tomes for the Formula 1 is still the same, and taking into account that the computation tome is considerably longer compared with the addition tome, this method cannot provide a fundamental solution.

Onda et al. will be discussed from a different viewpoint. In this method, the computing units for computing the above described Formula 1 are parallelled so as to speed up the overall signal processing. But this parallelism requires the same number of memory tables (ROMs) so as to obtain x_p · sinϑ, Y_p · cosϑ as the parallelled computing units. Consequently the system in terms of hardware becomes very large sized.

This method is not suitable for LSI. The data may be stored in a RAM in place of a ROM, but this has problems with integration.

As described above, it has been almost impossible to control in real time a vehicle running at a high speed based on the picture data and recognize in real time an object to be observed moving at a high speed based on the picture data. The speed up of the signal processing by means of parallelled computing units inevitable results in large sizes of the hardware.

A first object of this invention is to provide a picture processing device which enables not only picture data to be processed at high speed and in real time, but also the hardware to be miniaturized.

A second object of this invention is to provide a picture processing system which is able to perform the picture data processing the Hough transform is applied to at high speed and in real time.

Summary of the Invention

The picture processing device according to a first embodiment of this invention is for discriminating to extract out of a plurality of points to be processed on an original picture taken by picturing means points characteristic of the original picture. The picture processing device comprises DDA computing means comprising a plurality of DDA computing devices of the same structure serially connected; memory means for storing the computation results of the DDA computing means in accordance with the DDA computing devices; and extraction means for extracting points characteristic of the original picture, based on the stored contents of the memory means.

The picture processing device according to the first embodiment is also for deriving an approximate straight line interconnecting a plurality of points to be processed on an original picture taken by picturing means.The picture processing device comprises DDA computing means for computing a rotary motion recurrence formula subsequently every rotation by a set rotation angle in a pipe line system, the rotary recurrence formula being α_i+1 = f_α(α_i,β_i,ε) β_i+1 = f_β(α_i,β_i,ε) where one point on the circumference of an approximate circle depicted on an α β rectangular coordinate system has coordinates (α_i,β_i), and a rotation angle from the coordinates to next coordinates (α_i+1,β_i+1) on the circumference is represented by ε (but i is a positive integer); memory means for storing values of at least said β_i of the computation results given sequentially by the DDA computing means; and approximate straight line deriving means for deriving the approximate straight line of points of intersections of Hough curves which are given at the respective points to be processed based on the stored contents of the memory means.

The picture processing device according to a second embodiment of this invention is for discriminating to extract curves interconnecting a plurality of points to be processed on an original picture taken by picturing means, based on a characteristic of distributions of the points to be processed. The picture processing device comprises initial value computing means for computing an initial value based on the points to be processed on the original picture; DDA computing means comprising a plurality of DDA computing devices of the same structure serially connected for performing sequentially computing operations starting with the initial value; memory means for storing computed results of the DDA computing means in accordance with the DDA computing devices; and means for discriminating to extract the curves interconnecting the points to be processed, based on the contents stored by the memory means.

The picture processing device according to the second embodiment is also for deriving an approximate straight line of curves interconnecting a plurality of points to be processed on an original picture taken by picturing means. The picture processing device comprises initial value computing means for computing coordinates (α_i,β_i) of an initial value in a α β rectangular coordinate system, based on the points to be processed on the original picture; DDA computing means for computing a rotary motion recurrence formula subsequently every rotation by a set rotation angle from the coordinates (α_i,β_i) of the initial value in the pipe line system, the rotary recurrence formula being α_i+1 = f_α(α_i,β_i,ε) β_i+1 = f_β(α_i,β_i,ε) where one point on the circumference of an approximate circle depicted on an α β rectangular coordinate system has coordinates (α_i,β_i), and a rotation angle from the coordinates to next coordinates (α_i+1,β_i+1) on the circumference is represented by ε (but i is a positive integer); memory means for storing values of at least said β_i of the computed results given sequentially by the DDA computing means; and approximate straight line deriving means for deriving the approximate straight line of points of intersections of Hough curves which are given at the respective points to be processed based on the stored contents of the memory means.

The picture processing system according to a third embodiment of this invention is for discriminating to extract curves interconnecting a plurality of points to be processed on an original picture, based on a characteristic of distributions of the points to be processed. The system comprises a first step computing a rotary motion recurrence formula α_i+1 = f_α(α_i,β_i,ε) β_i+1 = f_β(α_i,β_i,ε) where one point on the circumference of an approximate circle depicted on an α β rectangular coordinate system has coordinates (α_i,β_i), and a rotation angle from the coordinates to next coordinates (α_i+1,β_i+1) on the circumference is represented by ε (but i is a positive integer) sequentially every rotation by said rotation angle or every plural rotations by said rotation angle along the entire circumference of said approximate circle or a part of the circumference thereof to thereby give Hough curves with respect with the respective points to be processed on the original picture; and a second step of deriving approximate straight lines of the curves interconnecting the points to be processed, based on points of intersection between at least two of the Hough curves given in the first step. A picture processing system according to the third embodiment is also for discriminating to extract curves interconnecting a plurality of points to be processed on an original picture taken by picturing means, based on a characteristic of a distribution of the points to be processed thereon. The system comprises a first step of given coordinates (α₀, β₀) of an initial value in a α β coordinate system based on the points to be processed on the original picture; and a second step of computing a recurrence formula α_i+1 = f_α(α_i,β_i,ε) β_i+1 = f_β(α_i,β_i,ε) where one point on the circumference of an approximate circle depicted on an α β rectangular coordinate system has coordinates (α_i,β_i), and a rotation angle from the coordinates to next coordinates (α_i+1,β_i+1) on the circumference is represented by ε (but i is a positive integer) sequentially every rotation by the rotation angle from the initial value given in the first step; and third step of giving Hough curves with respect to the points to be processed, based on the computed results in the second step; and a fourth step of deriving an approximate straight line of a curve interconnecting the points to be processed, based on points of intersection between at least two of the Hough curves given by repeating the steps 1 to 3 on the respective points to be processed.

The present invention will become more fully understood from the detailed description given herein below and the accompanying drawings which are given by way of illustration only, and thus are not to be considered as limiting the present invention.

Brief Description of the Drawings

<ul id="ul0001" list-style="none"><li>Figs. 1(a) and (b) are views explaining recognition of the contour of a road by the picture processing device;</li><li>Figs. 2(a) and (b), Figs. 3(a) and (b), and Fig. 4(a) and (b) are views explaining the principle of the Hough transform;</li><li>Fig. 5 is a block diagram of the overall picture processing device according to one embodiment of this invention;</li><li>Fig. 6 is a block diagram of a major part of the device of Fig. 5;</li><li>Figs. 7(a) and (b) are flow charts of the operation of the device;</li><li>Figs. 8(a)-(d) are views explaining the edge detection of the picture element signals inputted as an example of original pictures;</li><li>Fig. 9 is a flow chart of the pre-processing of the edged data;</li><li>Figs. 10(a) and (b) are views explaining the lookup table;</li><li>Figs. 11 (a)-(c) are views explaining the Hough transform in the embodiment;</li><li>Fig. 12 is a flow chart of a computing operation of the rotary motion recurrence formula in the embodiment;</li><li>Fig. 13 is a view explaining one cycle of the processing;</li><li>Fig. 14 is a view explaining the concept of the histogram memory;</li><li>Fig. 15(a)-(e) are views explaining the pipe line processing in the embodiment;</li><li>Figs. 16(a) and (b) are views explaining the 8 neighbor filtering (peak filtering);</li><li>Figs. 17(a) and (b) are flow charts explaining the sorting processing;</li><li>Figs. 18(a)-(d) are views specifically explaining the sorting processing;</li><li>Figs.19(a) and (b) are block diagrams of the sorting unit;</li><li>Fig. 20 is a block diagram of a major part of the picture processing device according to another embodiment of this invention;</li><li>Figs. 21(a)-(c) are views explaining the Hough transform in this other embodiment; and</li><li>Fig. 22 is a block diagram of the computing circuit for the rotary motion recurrence formula.</li></ul>

As shown in the block diagram of Fig. 5, a camera 11 takes a picture of an object to be processed (e.g., a road, a high speed moving object, or the like) as an original picture. The picture signals are digitized by a signal inputting unit 12 to be supplied to an edge detecting unit 13. As will be described later, the edge detecting unit 13 extracts the edges of the picture signals to produce edged data having different brightnesses, e.g., 512x512 picture element signals (edged picture element signals) and supplies the same to a multivalued memory 14. The multivalued memory 14 stores the edged data for the respective picture elements. Each time one display screen has been scanned, the edged data are supplied to a D/A converting unit 15 to be supplied to a CRT display 16 as analog signals. Thus, the edged data are displayed on the CRT display 16.

On the other hand, the edged picture element signals are supplied to a pre-processing unit 17 to be processed as will be described later. The pre-processed edged picture element signals are supplied to a DDA (digital differential analysis) computing unit 18. The DDA computing unit 18 comprises DDA computing circuits 18₀ ∼ 18_n-1 which are connected serially. A peak filter 19 and a sorting unit 20 are connected to the DDA computing unit 18 on the side of the output thereof for the processing by the peak filter and by the sorting unit (which will be described later). The above described circuit elements are connected to a CPU 22 through a VME (Versa Module European standard) bus 21 for control of the signal processing operation and synchronization of the processing timing. The pre-processing unit 17, the DDA computing unit 18, and the peak filter 19 are interconnected by a VME bus 23 for synchronization control of the transfer of the computation result of the DDA computing unit and that of brightness value data.

Fig. 6 is a block diagram of the major part which corresponds to the pre-processing unit 17, the DDA calculating unit 18 and the peak filter 19 in Fig. 5. The pre-processing unit 17 is provided by a FIFO (first in first out) board 17′. The FIFO 17′ inputs as an address signal coordinate values (X_p,X_p) in the X-Y plane of a point to be processed, and edged brightness value data D_I as a data signal. As will be described later, the FIFO 17′ transforms X-Y coordinates to x-y coordinates, sets a window, and processes a threshold value to output the results sequentially FIFO.

Each DDA computing circuit 18₀ ∼ 18_n-1 in Fig. 5 comprises three flip flops (F/Fs) 31,32,33. The respective F/Fs 31 temporarily store address signals α₀ ∼ α_n-1, β₀ ∼ β_n-1. The respective F/Fs 32 store brightness value data D₁. The F/Fs 33 temporarily store histogram data D_M0 ∼ D_M(n-1) read from respective RAMs 34 (RAM₀ ∼ RAM_n-1). Respective DDAs 37 (DDA₀ ∼ DDA_n-1) compute rotary motion recurrence formula which will be described later, every rotation by a rotation angle. The respective DDAs 37 are inputted with address signals α_i,β_i to output address signals α_i+1,β_i+1. Respective ADDs 35 (ADD₀ ∼ ADD_n-1), adders, add the brightness value data D_I from the FIFO 17′ and the histogram data D_M0 ∼ D_M(n-1). The outputs from the respective ADDs are temporarily stored in a buffer 36 and then supplied to the respective RAM₀ ∼ RAM_n-1. A timing controller 25 outputs timing pulses φ_a ∼ φ_e to control the timing of the signal processing among these circuit elements and is connected to a command/status interface not shown.

The overall operation of the picture processing device of Figs. 5 and 6 is shown by the flow chart of Fig. 7. Picture signals of respective points to be processed on the original picture taken by the camera 11 are inputted by way of the signal inputting unit 12 (Step 102). The edge detecting unit 13 detects the edges of the picture signals (Step 104) and supplies the edged data to the pre-processing unit 17 (Step 106). These Steps 102 to 106 are repeated every time picture signals are inputted. The results (edged data) are sequentially supplied to the pre-processing unit 17 as digital data.

The pre-processing unit 17 executes a set pre-­ processing (which will be described later) (Step 108) and supplies processed data to the DDA computing unit 18 (Step 110). This pre-processing is repeated sequentially every time the pre-processing unit 17 is supplied with the edged data.

Subsequently, the rotary motion recurrence formula for giving Hough curves (sine curve ) are executed by the DDA which will be explained later (Step 112). A computing operation continues until all the picture signals of one display (original picture display) to be processed except those out of a window or below a threshold value have been processed (Step 114). Then a filtering operation (Step 116) and a sorting operation (Step 118) both of which will be explained later are executed with respect to the points of intersection of the resultant Hough curves by the approximation filter 19 and the sorting unit 20. The final result is an approximate straight line of curves interconnecting the points to be processed on the original picture.

Next, the edge detection performed by the edge detecting unit 13, and the edged data are shown in Fig. 8.

It is assumed that an original picture taken by the camera is as shown in Fig. 8(a). When the line indicated by reference numeral 8 in Fig. 8(a) is set as the x′ axis, brightness S is expressed in analog form as in Fig. 8(b). That is, the brightness is lower at the outside parts of the road 3, the road 3 and the shoulders but much higher at the shoulder lines 4. The brightness distribution of Fig. 8(b) is recognized as digital data of, e.g., 256 gradation in this embodiment, but only such brightness distribution is not sufficient for accurate recognition of the configuration of the road.

Here, brightness S of the picture signals supplied by the signal inputting unit 12 are differentiated dS/dx′ on the axis x′ to give the brightnesses in change ratios. Then definite edges are obtained as shown in Fig. 8(c). The edges are given in absolute value |dS/dx′| as shown in Fig. 8(d). The edge detecting unit 13 supplies edged data (digital data) for exact recognition of the edges of the shoulder lines 4. The edged data from the edge detecting unit 13 are subjected to the pre-processing by the pre-processing unit 17 in the next step.

The pre-procesing by the pre-processing unit 17 (FIFO 17′) is shown by the flow chart of Fig. 9.

The edged data produced as in Fig. 8 are supplied from the edge detecting unit 13 to the pre-processing unit 17 (Step 122) to be judged with respect to whether or not the edged data are inside a preset window (Step 124). The window is set, for example, as indicated by reference numeral 9 in Fig. 8(a). Whether or not the edged data are within the window is judged based on the coordinate values of the edged data in the x-y coordinate plane. The next step 126 is executed on only those of the edged data which are inside the window 9.

In Step 126, the edged data inside the window 9 are digitally judged with respect to whether the edged data inside the window are above a required threshold level, based on, e.g., a lookup table (LUT) affixed to the FIFO 17′. That is, as shown in Fig. 10, without the LUT, input data to and output data from the FIFO 17′ correspond to each other in a ratio of 1:1 (Fig. 10(a)).

But, in the case where a threshold level is set at I_th, using the LUT, the output data is 0 when the input data is below I_th. In the case where the edged data are produced in 256 gradation, the threshold level I_th can be set arbitrarily in a range of0 ∼ 255. When this threshold level is expressed in analog form, it is set as indicated by the dot line in Fig. 8(d). Accordingly, the data surviving the processing of Step 126 are mainly those (digital data) corresponding to the shoulder lines 4 and the center line 5 in the original picture. That is, only major data (of, e.g., the shoulder lines and the center line of the road) are left to be signal processed in the following steps. Accordingly, there is no influence of noise components, and the overall processing speed can be improved.

Next, the coordinate transformation is performed in Step 128. That is, as shown in Fig. 8(a), the coordinate transformation is performed from the X-Y coordinate system to the x-y coordinate system. Accordingly the coordinates (X_p,Y_p) of the point to be processed is transformed to the coordinates (x_p,y_p). Thus a pre-processing finishes for the Hough transformation.

In Fig. 9, the sequence of Steps 124 ∼ 128 may be changed. For example, Step 128 of performing the coordinate transformation may be performed first. Taking the data procesing time into consideration, the sequence of Fig. 9 is most preferable.

The application of the Hough transform to this embodiment is shown in Figs. 11 and 12.

When a Hough curve (sine curve) is obtained with respect to the point P(x_p,y_p) in Fig. 11(a), the Hough curve shown in Fig. 11(c) will be given, as explained above. It is seen from the trigonometric function theorem that the trajectory of such sine curve is replaced with that of a circular motion of Fig. 11(b). In other words, it is equivalent to obtaining a trajectory of the circumference of the circular motion of Fig. 11(b) that the Hough transformation is performed with respect to the point P(x_p,y_p) in Fig. 11(a) to obtain the Hough curve of Fig. 11(c). Here, the circle of Fig. 11(b) has a radius R=ρ_max=(x_p²+y_p²)^1/2      (2) When the circular motion is started at a point Q (α₀,β₀) in Fig. 11(b) corresponding to ϑ=0° at the point p(x_p,y_p), the initial value ϑ_d is given by the formula ϑ_d = π/2-ϑ_max where tanϑ_max = x_p/y_p      (3)

From this viewpoint, the inventors have applied the rotary motion recurrence formula in depicting the circle of Fig. 11(b) and found an easy method for transforming the point P (x_P,Y_p) of Fig. 11(a) into the Hough curve of Fig. 11(c). Based on the rotary motion recurrence formula, coordinates (α_i+1,β_i+1) which is ahead of a point (α_i,β_i) by the angle ε in α-β rectangular coordinate system is given by α_i+1 = f_α(α_i,β_i,ε) β_i+1 = f_β(α_i,β_i,ε)      (4) where i is an integer. The above Formula 4 is, for example, approximately given by the following equations when the rotation angle ε=2^-m(rad) (m=0, 1, 2, ...,) α_i+1 = α_i-2^-m β_i β_i+1 = 2^-m α_i+1 + β_i      (5). α_i+1 = α_i(1-2^-2m-1)-2^-mβ_i β_i+1 = 2^-mα_i + β_i(1-2^-2m-1)      (7) or α_i+1 = α_i(1-2^-2m-1) + β_i(-2^-m + ε^-3m/6) β_i+1 = α_i(2^-m + ε^-3m/6) + β_i(1-2^-2m-1)      (8)

Before the operation of the circuit of Fig. 5 is specifically explained, this computing method will be explained in good detail.

Figure 12 is a flow chart of the computing process. The data which has been pre-processed by the FIFO 17′ in accordance with the sequence of the processing steps in Fig. 9 are supplied to the DDA computing unit 18 (Step 132), and a rotary motion is started at a position Q (α₀,β₀) corresponding to angle ϑ=0° (i.e., an angle where ρ=x_p) at the point to be processed P(x_p,y_p) in Fig. 11. At this time, (α₀,β₀) from the FIFO 17′ is α₀ = Y_p,β₀ = x_p      (9).

The initial position of the rotary motion is set by the above described method, and an initial value is substantively unnecessary for the recurrence formula computation.

Then, the value of β₀ is stored in the RAM₀ as an address, and then α₁,β₁ are obtained by Recurrence Formula 7. This computation can be performed by substituting Formula 7 with α₀,β₀ given by Formula 9 (Step 134). Every time the respective DDA₁, DDA₂, DDA₃, .... finish their computation, the results (β₁, β₂, β₃,....) are subsequently stored in RAM₁, RAM₂, RAM₃, ... as addresses (Step 138). On the other hand, between Steps 134 and 138, brightness data are accumulated. That is, the histogram data D_Mi read from the RAM 34 (RAM_i) at the address=ϑ_i, and the brightness data from the FIFO 17′ are added to be stored again in the RAM_i (Step 136).

This computation is repeated each rotation by the angle ε until one round has been finished along a circle (Step 140). When one round is has been made, a Hough curve with respect to one point to be processed on an original picture is given, based on the thus stored values β₀, β₁, β₂, β₃, ..., and values (rotation angle ε) of ϑ₀, ϑ₁, ϑ₂, ..., and also results weighted with brightness data (D_M0 , D_M1, ... D_M(n-1)) are given. When the processing of Fig. 12 is executed on all the points to be processed on the original picture, a plurality of Hough curves weighted with brightness data are given in ρ-ϑ coordinate system, and these Hough curves have points of intersection in Fig. 3(b).

The operation following the flow chart of Fig. 13 will be explained in more detail with reference to Fig. 6.

Step 132 of inputting data in Fig. 12 is executed by supplying a ready signal from the FIFO 17′ in Fig. 6 to the timing controller 25 and supplying a read strobing signal from the timing controller 25 to the FIFO 17′, then supplying address signal α₀,β₀ corresponding to the coordinate value (x_p,y_p) of the point to be processed P from the FIFO 17′ to the F/F 31, and then supplying brightness data D_I of the point to be processed P to the F/F 32. The input of an address and data to the F/Fs 31, 32 are performed synchronously with a timing pulse φ_a from a timing controller 25. Synchronously with this timing pulse φ_a, the address signals α₀,β₀ from the F/F 31 are inputted to a first DDA₀ 37.

The DDA₀ performs the processing of Step 134 in Fig. 12. That is, the computation of Recurrence Formula 7 is performed, and the computation result (address signals α₁,β₁) is supplied to a next DDA₁ (not shown). The Recurrence Formula 7 basically involves neither computation of the trigonometric function nor multiplication, and reference to a memory table (ROM) is not necessary either. The computation can be performed easily and promptly. This computation has sufficient precision (little error) for the rotary motion.

The address signal β₀ is supplied also to the RAM₀ 34. In response to this address signal the histogram data D_M0 stored in the RAM₀ is read out. That is, RAM₀ has stored at the address of β₀ the histogram data D_M0 with respect to the other points to be processed every rotation by the angle ϑ₀ (ϑ′=0) beforehand (based on the preceding computation of the ADD₀. Accordingly, synchronously with the addres signal β₀ supplied from the F/F 31 to the RAM₀, the histogram data D_M0 is supplied from the RAM₀ to the F/F 33.

Subsequently, in synchronization with a timing pulse φ_c from the timing controller 25, the histogram data D_M0 is supplied from the F/F 33 to the ADD₀. The ADD₀ has been supplied by the F/F 32 with brightness data D_I of the point to be processed (x_p,y_p). Accordingly, in the ADD₀, the rotation angle ϑ₀ which has been stored in the RAM₀, the histogram data D_M0 corresponding to the address β₀, the rotation angle ϑ₀ of the point being processed (x_p,y_p), and the brightness value data D_I corresponding to the address β₀ are added (Step 136 in Fig. 12). The addition result (D_M0′=D_M0+D_I) is temporarily held by the buffer 36 and then is supplied to the RAM₀ in synchronization with a timing pulse φ_d to be stored at an address β₀ of the RAM₀ corresponding to ϑ₀ (Step 138 in Fig. 12).

The above described one cycle of processing will be explained below in terms of the DDA computing circuit 18_i (i=1, 2, ... n-1) with reference to Fig. 13.

After address signals α_i,β_i which are a result of the computation of an address signal (initial values α₀, β₀ corresponding to coordinate values (x_p,y_p) of a point to be processed to be α₁,β₁, to be α₂,β₂, to ....., and to be α_i,β_i, the brightness value data D_I of the point to be processed (x_p,y_p) is supplied respectively from the F/Fs 31 and 32 to be held (Step 202), and address signals α_i,β_i from the F/F 31 are supplied to the DDA_i, the computation of the recurrence formula is executed, based on the address signals α_i,β_i in Step 204. Address signals α_i+1,β_i+1, a result of the computation, are supplied to the F/F 31 positioned before the DDA_i+1 in the next DDA computation circuit 18_i+1 in synchronization with the finish of one cycle of processing.

On the other hand, in response to the address signal β_i, reading of the histogram data D_Mi is performed in Step 206. This reading is performed by supplying the address signal B_i from the F/F 31 to the RAM_i to store the histogram data D_Mi at the address of β_i in the F/F 33 in Fig. 6. In Step 208 the brightness data D_I of the histogram data D_Mi is added. Step 208 is executed by the ADD₁ in Fig. 6. Then, the histogram data D_Mi′=D_Mi+D_I is written in the RAM_i at the address β₀ in Step 210.

The store of the histogram data will be explained in more detail with reference to Fig. 14.

Fig. 14 shows the concept of the histogram memory of the RAM 34 in Fig. 6. As shown, the memory has regions RAM₀ ∼ RAM_n-1 which corresponds respectively to the rotation angles ϑ₀ ∼ϑ_n-1. Each RAM region has addresses β (=ρ) +511 ∼ 0 ∼ -512. Each address can store 16 bit histogram data. Accordingly, when the DDA_i computes α_i+1,β_i+1 corresponding to the rotation angle ϑ_i+1 based on the address signal α_i,β_i, using Recurrence Formula 7, the address signal β_i is supplied to the RAM_i, and the histogram data D_M(i) is read and combined with the brightness data D_I of a point to be processed. Then the histogram data (D_M(i)+D_I) is written at the address β_i.

As described above, the computation of Recurrence Formula 7 is performed in the pipe line system in which the address signals α_i,β_i are transferred sequentially. During the computation of the addres signals α_i,β_i to the address signals α_i+1,β_i+1, the brightness data D_I stores the histogram data D_Mi, which shortens the time required for one cycle of processing.

This one cycle of processing is performed by the DDA computing circuits 18₀ ∼ 18_n-1 simultaneously and parallelly. That is, as in Fig. 15(a), data 1,none, 2, none, none, 3, 4, none is inputted by the FIFO 17′. In the first cycle the data is as shown in Fig. 15(b)., In the second cycle, the data is as shown in Fig. 15(c). In the third cycle, the data is as shown in Fig. 15(d). The same processing follows. In the eighth cycle, the data is as shown in Fig. 15(e). In Fig. 15(a), (α₀₁,β₀₁) ∼ (α₀₄,β₀₄) are address signals corresponding respectively to coordinate values (x_p1,Y_p1) ∼ (x_p4,Y_p4) of the points to be processed P₁ ∼ P₄, and D_I ∼ D_I are brightness data of the respective points to be processed, P₁ ∼ P₄, ϑ₀ ∼ ϑ_n-1 are rotation angles from the points to be processed (ϑ₀ is the points to be processed themselves) and corresponds respectively to the RAM₀ to RAM_n-1 in Fig. 6.

Next, the peak filtering following the Hough transform will be explained.

Here it is assumed that when the points of intersection of Hough curves are expressed in the ρ ϑ plane, Fig. 16(a) is given. Fig. 16(a) does not always agree with the histograms obtained by this invention, because in Fig. 16(a) histograms H which are weighted with brightness value data (brightness change ratios) of the picture elements of the original picture with respect to the points of intersection appearing every unit of the ρ-ϑ plane are expressed in the contour for easy understanding.

Here it is assumed that the histogram at the point P₁ in Fig. 16(a) is high and also high at the points P₂ and P₃. When the neighbors of the point P₁ are considered, high histograms take place at the points P₄ and P₅. But what is especially important is to to find the points P₁-P₃ spaced from each other. For example, the point P₁ corresponds to the shoulder lines of the road; and the point P₂, the center line thereof; and the point P₃, the shoulder lines of the forward curved portion of the road. But in many cases the points P₄ and P₅ correspond to partial curves of the shoulder lines, which are, in many cases, noise components in picture processing.

An eight neighbor peak filtering, for example, lessens such noise components. That is, using an 8 neighbor filter (peak filter), the histograms at the points of intersections of the Hough curves are compared between areas F₁ ∼ F₉. When the formula F₅ &gt; F₁ ∼ F₄,F₆ ∼ F₉ is satisfied, the data in the area F₅ is to be detected. Specifically, in the case, for example, where one unit (element area) of picture elements in the ρ ϑ plane is allocated to each area F₁ ∼ F₉, the formula F₅ &gt; F₁ ∼ F₄,F₆ ∼ F₉ is satisfied when the histogram number of the points of intersection are <tables id="tabl0001" num="0001"><table frame="all"><tgroup cols="3" colsep="1" rowsep="0"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">F₁ = 6,</entry><entry namest="col2" nameend="col2" align="left">F₂ = 8,</entry><entry namest="col3" nameend="col3" align="left">F₃ = 4,</entry></row><row><entry namest="col1" nameend="col1" align="left">F₄ = 2,</entry><entry namest="col2" nameend="col2" align="left">F₅ = 14,</entry><entry namest="col3" nameend="col3" align="left">F₆ = 10,</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">F₇ = 7,</entry><entry namest="col2" nameend="col2" align="left">F₈ = 9,</entry><entry namest="col3" nameend="col3" align="left">F₉ = 8.</entry></row></tbody></tgroup></table></tables> Then the point of intersection F₅ is to be detected. In contrast to this, when <tables id="tabl0002" num="0002"><table frame="all"><tgroup cols="3" colsep="1" rowsep="0"><colspec colnum="1" colname="col1" colwidth="52.50mm" /><colspec colnum="2" colname="col2" colwidth="52.50mm" /><colspec colnum="3" colname="col3" colwidth="52.50mm" /><tbody valign="top"><row><entry namest="col1" nameend="col1" align="left">F₁ = 8,</entry><entry namest="col2" nameend="col2" align="left">F₂ = 4,</entry><entry namest="col3" nameend="col3" align="left">F₃ = 3,</entry></row><row><entry namest="col1" nameend="col1" align="left">F₄ = 14,</entry><entry namest="col2" nameend="col2" align="left">F₅ = 10,</entry><entry namest="col3" nameend="col3" align="left">F₆ = 7,</entry></row><row rowsep="1"><entry namest="col1" nameend="col1" align="left">F₇ = 9,</entry><entry namest="col2" nameend="col2" align="left">F₈ = 8,</entry><entry namest="col3" nameend="col3" align="left">F₉ = 2,</entry></row></tbody></tgroup></table></tables> F₅&lt;F₄ is given, and thus the area F₅ is not data to be detected.

The above described filtering is performed by means of well-known software techniques.

The above described filtering makes it possible that the points P₂ and P₃ where histograms are high can be detected, free from being influenced by the points P₄ and P₅ in Fig. 16(a). Without the filtering, points of the second highest histogram next to the point P₁ of the first highest histogram are P₄ and P₅, and the points P₂ and P₃, which have to be detected as points of the second and the third highest histograms are of the fourth and the fifth highest histograms. This makes the subsequent signal processing very difficult.

The sorting processing of Step 118 in Fig. 7 will be explained below with reference to the flow chart of Fig. 17.

For the sorting processing, a plurality of input memories (transfer memories) M_I1∼ M_I4(4 for simplicity), and comparison memory (result memories) M_M1∼ M_M4 (4 for simplicity) are initialized (Step 152). Next, data is inputted to the input memory M_I1(Step 154). When it is judged that this input data is present in Step 156, Steps 158 ∼ 184 will follow, and when judged this input data is absent, Steps 190 ∼ 199 will follow. Steps 190, 196 and 199 perform the same processing as Steps 158 and 160 respectively. Steps 192 and 198 perform the same processing as Steps 162 ∼ 168 respectively. Step 194 performs the same processing as Steps 170 ∼ 176.

Steps 158, 162, 170, and 178 respectively compare the magnitudes of the content of the corresponding input memory M_I and that of the comparison memory M_M and find M_I≦M_M, the content of the input memory M_I is transferred to the next Steps 164,172 and 180. Contrary to this, when M_I&gt;M_M, the content of the comparison memory M_M is transferred to the next input memory M_I (Steps 168, 176, 184). Finally the memories M_MI ∼ M_M4 hold 4 input data in the sequential order of the magnitude.

This is shown specifically in Figs. 18 and 19. Four memories M_I1∼ M_I4 as the input memories, and four memories M_M1∼ M_M4 as the comparison memories are provided. The input memories and comparison memories are mated respectively in four stages of circuits. Each stage of circuits, as shown in Fig. 19, comprises a pair of the input memory M_I and the comparison memory M_M, switching circuits 81, 82, and a comparator 83. When data inputted to the input memory M₁ is larger than the data stored by the comparison memory M_M, the comparator 83 supplies an output which actuates the switching circuit 81 as indicated by a solid line in Fig. 19, and the input data is stored by the comparison memory M_M. At the same time, since the switching circuit 82 is in the state as indicated by a solid line, the data stored by the comparison memory M_M is transferred to a next stage. In contrast to this, when input data (to the input memory M_I) is smaller than the data stored in the comparison memory M_M, the comparator 83 controls the switching circuits 81, 82 to be in the states indicated by dot lines. The input data is transferred directly to a next stage. The content of the comparison memory M_M does not change.

It is assumed that in this sorting unit, the input data are 10, i.e., 5, 7, 2, 8, 4, 9, 3, 1, 6, 8 as shown in Fig. 18(a). Operations following the initialization as in Fig. 18 (b) change the data stored in the comparison memories M_I ∼ M_M4 in the sequence indicated by the arrow in Fig. 18 (c), and finally the contents <tables id="tabl0003" num="0003"><img file="EP0341985A2_D0001.tif" /></tables> are stored. This sorting processing may be executed by software. Following a series of the above described processings, the picture processing device according to this invention finishes all the steps of its signal processing. Then an approximate straight line interconnecting the points to be processed on the original picture is given in values of the above described ρ,ϑ.

The picture processing device according to another embodiment of this invention is characterized in that an initial value computing unit is inserted between the pre-­processing unit and the DDA computing unit. As shown in Fig. 20 corresponding to Fig. 6, the initial value computing unit comprises two flip flops (F/F) 41 , 42, an initial value computing circuit 43. The F/F 41 temporarily stores coordinate values (x_p,y_p) of the point to be processed in the x-y coordinate system. The F/F 42 temporarily stores brightness value data D_I of the point to be processed P. The initial value computing circuit 43 computes coordinates of an initial value in the α-β rectangular coordinate system, based on the coordinate values (x_p,y_p) so as to enable the computation of the rotary motion recurrence formula.

In this embodiment, in computing the rotary motion recurrence formula, firstly the computation of an initial value is executed. The initial value computing unit is supplied with the data pre-processed by the FIFO 17′ to give an initial value (α₀,β₀) for computing the rotary motion recurrence formula. In Fig. 21 corresponding to Fig. 11, when a rotary motion is started at the point to be processed P (x_p,y_p) at an arbitrary position (angle) ϑ′ (=ϑ₀) (rad), α₀= x_psinϑ′+y_pcosϑ′ β₀= x_pcosϑ′+y_psinϑ′      (9). The initial value (α₀,β₀) is computed by the initial value computing unit 43 to be supplied to the DDA computing unit 18 as an address signal. The initial value computing unit 43 stores the values of sinϑ′,cosϑ′ beforehand in a ROM (not shown), and referring to these data, computes Formula 9 using adders and multiplier (both not shown). Compared with the computation of the recurrence formula, that of Formula 9 requires more computing time and is more complicated but performs the computation once on one point to be processed (x_p,y_p). Accordingly, the total computing time is not much increased, nor is the hardware large sized. The initial value computation is made very easy by starting a rotary motion at ϑ=0°, 90°, 180° and 270° at the point to be processed P(x_p,y_p).

This will be explained below with reference to Fig. 20. The initial value computing unit is supplied with data when a ready signal is supplied from the FIFO17′ in Fig. 20 to the timing controller 25, then a read strobing signal is supplied from the timing controller 25 to the FIFO 17′ and then the coordinates (x_p,y_p) of the point to be processed P is stored in the F/F 41, at this time brightness value data D_I at the point to be processed P being stored by the F/F 42. Then, in synchronization with a timing pulse ϑ_s from the timing controller 25, the F/F 41 outputs the coordinates (x_p,y_p), and the initial value computing unit 43 gives the initial value (α₀,β₀) by the recurrence formula, based on the coordinate values (x_p,y_p) of the point to be processed P. The following computation of the recurrence formula is the same as in the first embodiment.

This invention is not limited to the above described embodiment and covers various modifications.

To give examples, in Step 124 in Fig. 9 a plurality of windows may be set, and when windows overlap, one of the overlapping windows may have priority of being processed. It is not essential to set a window, and instead the entire original picture may be to be processed and may be subjected to the Hough transformation. But, in this case, the peripheral region of the original picture indicated by the dot line Q in Fig. 8(a) contains so many noise components that it is necessary to remove this peripheral region beforehand for the picture processing of high precision.

In obtaining histograms at the points of intersection of Hough curves, in place of the brightness value data based on brightness change ratios(value of differentiated edged data), two valued data may be multiplexed so that the histogram is based on density of points of intersection. Instead of preparing the edged data by differentiation, data corresponding to the brightness may be digitally processed as brightness data, and then histograms may be obtained at points of intersection of Hough curves.

It is not essential that the rotary motion recurrence formula is performed along the entire circumference (one round) of an approximate circle, but the computation may be performed along a 1/2, 1/4 or 1/8 circumference. For example, DDAs are arranged serially in a number necesary for the computation along a 1/4 circumference for 0≦ϑ&lt;π/2 and π≦ϑ&lt;3π/2. The values along the other portion of circumference (π/2≦ϑ&lt;π, 3π/2≦ϑ&lt;2π) can be given immediately by the DDAs. It is not essential either that the rotation angle is always the same. It may be varied partially.

It is possible that Hough transform is performed along a half circumference.

For the computation of the rotary motion recurrence formula, the circuit of Fig. 22, may be used.

In the circuit of Fig. 22, reference numerals 51,52 represent adders; and 61 an inverter. This circuit can execute the computation of Formula 5.