Method for creating three-dimensional object model during e.g. architectural engineering design work, involves providing digital video camera on swivel base, and linking intersection points of projective rays by software

ES 2 249 162 A1

DESCRIPTION

Three-dimensional modeling procedure using boundary conditions.

Object of the invention

The present invention refers to a three-dimensional modeling method using edge conditions, which provides essential novelty characteristics and notable advantages with respect to the means known and used for the same purposes in the current state of the art.

More particularly, the invention proposes the development of a method by means of which it is possible to achieve the modeling of the external surface of certain objects or pieces, with the help of a digital video camera and a rotary base used as a. means of support for the object during the acquisition of the images from a multiplicity of points of view.

For the application of the procedure, the use of specific management software is foreseen, through conventional computer equipment, conceived exclusively for its application to the development of the proposed procedure.

The field of application of the invention is mainly within the sector related to design engineering, architecture, restoration, fine arts and geometric control, and can be extrapolated to other fields or applications that fit the purpose or result that this raises. Project Type.

Background and summary of the invention

The determination of the shape of a convex object has traditionally been solved for its use manually by applying complex robotic methods with five movements; Another more automated solution consists of the use of scanners based on the emission and reception of a laser beam, all these methods having a very high acquisition cost due to the necessary instrumentation for them.

Therefore, with this invention, a new methodology based on image analysis is proposed, using an amateur video camera of the highest possible quality (digital camera), providing this method as a result the final shape of the object with a resolution more than sufficient to any type of requirement. This resolution allows to observe an object from all its points of view and to analyze the edge of the image (edge ​​of the object).

The proposed procedure allows from the edge of the object (determined in the image) to build the complete three-dimensional model fully automatically with millimeter precision for an object with considerable dimensions, obtaining greater precision if the object is smaller in size.

The method uses a rotating base on which the object will be placed and a digital video camera to obtain all the points of view of the object, generating a three-dimensional model of it with the original textures on the model and a characterization of the precisions obtained in the determination of the model (all this applying a series of algorithms developed for the treatment of the images).

This modeling method has certain advantages compared to what is known, such as that it can be used by any user, being very economical, it does not have any limitation of space or interaction with the environment, it does not have special computer requirements and it can work on any current computer, and above all, it is not an aggressive method with the environment.

The investigation and application of the algorithm developed for the treatment of the captured images has generated a deep analysis, in which it is concluded that by using this procedure the digital model of any object can be obtained very cheaply, being able to reach be a very useful tool mainly in design engineering, architecture, restoration, fine arts and geometric control.

Brief description of the drawings

These and other characteristics and advantages of the invention will become more clearly apparent from the detailed description that follows of a preferred embodiment, given solely by way of illustrative and non-limiting example, with reference to the accompanying drawings. , in which:

Figure 1 represents by means of a schematic drawing the view of the plan in which the outline of the base of the rotating base can be seen, on which there is a figurative object surrounded by the arrows that represent the object area covered in every moment of video shooting.

Figure 2 shows, like the previous figure and by means of a schematic drawing, the view of the plan in which the outline of the base of the rotating base can be seen on which there is a figurative object surrounded by a pair. of arrows representing the limit or right edge of the object defined by the camera image.

ES 2 249 162 A1

Figure 3 represents by means of a schematic drawing, like the previous Figure, the view of the plan in which the outline of the base of the rotating base can be seen, on which there is a figurative object surrounded by border lines. defined by the image.

Figure 4 schematically shows a possible sector in which the cylindrical coordinates that delimit it are represented by double arrows.

Figure 5 represents an initial cylinder to be modeled by the edge lines that model it, which is defined by eliminating the outer material by successively applying the cuts suggested by the edge vectors that define the images.

Description of a preferred embodiment

As indicated above, the detailed description of the invention will be carried out taking into consideration the representations of the attached drawings, through which the same numerical references are used to designate the equal parts or similar. In this sense, and taking into account the procedure of the invention itself, an example is set out in the following paragraphs in which determining aspects for the development and understanding of the invention itself are clarified.

The central objective of the procedure is to obtain the three-dimensional modeling of the object automatically, for this the following data obtained according to the algorithms developed during the development of the invention are available: external orientation parameters of all the images corresponding to the first round, internal geometry of the camera, main distance and correction of radial distortion, mapping or equivalence between the images of the first and second round, to be able to use the orientation parameters obtained in the first round, and the data tables with the pixel coordinates of the edges of the object in each of the images of the second round.

With all these data it is intended to automatically obtain the three-dimensional modeling of the object and its precision. In addition, the results are complemented with the three-dimensional graphic output (using VRML) of the model obtained both in wire, face or texture format.

As previously described, to check the correct operation of the proposed algorithms, a help software has been developed, minimizing user performance in tests made with different objects.

The objective is to automatically obtain the three-dimensional modeling of the object. The concept of "automatic" implies not using stereoscopic vision at any time to identify points on the object (with which this modeling procedure would not make sense as it is a classical photogrammetry work with a peculiar way of guiding the procedure ), nor the correlation for the automatic identification of homologous points (since the object may lack appropriate textures for this purpose, like most sculptures).

The methodology presented consists of using only the points on the edges of the object to define it completely automatically, without identifying at any time homologous points with any technique such as correlation.

Therefore, there are no homologous points, but a series of lines in space that surround the points on the edges of the images, defining the outer shape of the object. In an approach to the problem, by having a single image with its object edge points, an inclusion zone and another object exclusion zone are achieved, as can be seen in Figure 1, where you can see the Rotating base, referenced with the number 1, the object to be modeled, with the number 2, and the different representations by arrows of the possible projective rays, referenced with the number 3 in the figure.

The lines at the edge of the image behave as if the object were the envelope of the bundle of lines defined by the edge. Having the double condition of tangency and inclusion to the object, so that both properties will be able to define the object in the work. However, the first condition (tangency) has a geometric definition that can be approached in the direct problem, where once the object is known, it is desired to know the tangents. Its solution will be obtained through Frenet's trihedral in space, applied on the surface in question, studying the tangent plane (the perpendicular to the normal at the point of the studied surface). On the contrary, the problem posed in this procedure is the reverse, that is, the envelopes are known and it is desired to obtain the object.

The problem of obtaining bundles of lines and their enveloping figure is always posed as a differential equation that has to be integrated. The differential equation represents the lines of the beam and its envelope. If we know the differential equation that contains all the lines and its enveloping object, we could obtain the equation of the lines, as well as the object. But the lines available do not adapt to a simple differential equation, in the same way that the object does not have a simple differential equation to represent it, therefore it is useless to continue in the direction of integration of differential equations.

ES 2 249 162 A1

The resolution of the problem is approached from the point of view of inclusion: The object is contained within the space defined by the edges. If a cylindrical coordinate system is defined in the object, so that a point on its surface has coordinates P (f, H, r) where: f polar angle, H the height and r the polar distance.

The position of point P, as indicated in Figure 2 with the number 4, will fulfill in all images that the distance from a point on the Z axis (f, H, r = 0) to any line of the edge that is find at that height and in that direction is greater than r. There is only one subtraction that equals and corresponds to the line that passes through that point on the edge.

A value of r is obtained for each value of the height H (second coordinate) and for each direction f (first coordinate). The value of r (polar distance) is obtained by applying a minimum function.

In each frame i for each pixel j (of the edge of the object in the image), the projective line that it produces and its relationship with a height H and any direction f is studied.

The first coordinate r of the model will be the minimum value of the distance D (i, j) (referenced in Figure 3 with a double arrow numbered with the number 5) from the axis of rotation to the straight line (at the height H and at the direction f), as shown in Figure 3. The distance will be obtained by calculating the minimum value in each direction of the distance from the axis: r = min [D (i, j)].

The object will be defined by the intersection or common zone between all the inclusion zones produced by all the images in the video sequence.

Going further into the applied methodology, as previously stated, the coordinate system used to define the object model will be a cylindrical coordinate system. That is, the model will be made up of sectors that will have a certain resolution (number of sectors in each horizontal plane and number of horizontal planes).

Now looking at Figure 4, the sector of coordinates (f, H, r) can be seen, each parameter f, H and r being referenced in the drawing, through a double arrow, with the numbers 6,7 and 8 respectively. Each sector will intersect the object at a point on its surface, which will be defined by an angle on the X axis (f), a height above the lower platform (H), and a radius (r) on the Z axis.

For the application of the algorithm, the number of sectors used for modeling is initially specified in the software that has been developed for subsequent calculations. It is also a good idea to limit modeling to a height margin or horizontal sector.

As described below, the modeling of the object must be carried out in two consecutive steps: a first step that consists of the actual modeling of the object without entering any evaluation, and a second step in which an improvement of the obtained modeling is carried out. in the previous step and a series of indicators are calculated to evaluate the characteristics of the model achieved.

In the first step, the initial modeling of the object starts with a cylinder determined by a height interval, with as many sectors per plane, and as many horizontal planes as those indicated in the parameters entered by the user in the software. Therefore, in a first example if there are 90,000 sectors (300 X 300, 300 sectors and 300 planes) so that when the value of r (polar distance) is known for each one, they will correspond after modeling to as many points on the surface of the object.

For each sector according to its spatial position, we have by definition the coordinates f and H. The coordinate r will be the distance to be calculated to locate the point on the surface of the object. Initially, this coordinate is initialized with an upper coordinate, which can be, for example, the radius of the platform.

The algorithm to be applied is twofold and it needs to define the projective rays that pass through each sector and the minimum distance to these rays.

The software as graphic outputs of this first model can display several images, in which each sector is represented with a point in space located in its central position. Later, graphical outputs will be created using wire, face, and texturing modeling by meshing and triangulating the obtained surface. In these obtained figures it can be seen that there are only points on the surface of the object and not within it.

In a second step, to improve and assess the modeling, it is observed that in the previous section each sector was defined by its coordinates f, H and the radius r that was calculated as the minimum distance to the closest line of all the lines that they crossed this sector. By selecting the minimum distance, the line that defines it is exposed to be created from a poorly defined edge point (1 error pixel is easy to commit when detecting an edge point in an image). This section proposes the solution to this possible error, as well as the determination of the precision in the obtained modeling.

ES 2 249 162 A1

In each sector obtained (f, H, r), it is determined which lines have been the closest (according to a previously set value) and with these, statistical operations are carried out with their minimum radii, being necessary to carry out this process in two phases: one first phase where there is a determination of the closest lines within an established margin (accumulation of lines), and a second phase where a statistical study of their geometric distribution is carried out.

As a first phase, in the accumulation of lines, it will be decided which is the minimum distance for the accumulation of lines, which will normally correspond to 1 pixel measured on the object, the minimum distance for the accumulation of lines in each sector being specified by the software.

In this case, for example, if a line passes less than 3 millimeters from a sector, its identification is stored in a list belonging to that sector. In this way, the software after performing the calculation with all the sectors will create an informative table.

In the table a table will be formed with all the sectors represented, in this case they will be 40,000, organized by rows according to the H coordinate and by columns according to the f coordinate. In the table you can see, for each sector, the number of lines that have been found at a distance less than the specified 3 mm.

The software produced presents tools for visual verification of the correct operation of the algorithm. When selecting a cell in the table, for example the cell corresponding to a certain sector f and height H, the information on the total number of lines that pass less than 3 millimeters from said sector will appear, for example 409 lines, in addition to a graph in the one that appears the distribution of these lines according to the distances less than 3 millimeters.

The horizontal and vertical axes of the graph referred to above, shown by the software, will appear multiplied by a certain factor, for example a factor of 10, so that it is observed that, for example, 4 lines appear at the distance minimum object, 31 lines pass between 0 and 0.3 mm, 57 lines pass between 0.3 and 0.6 mm, etc.

In addition, the definition of the sector according to these lines indicated above is visualized in space, by means of a graphic representation, where it is shown from different points of view, how these 409 lines touch the point whose coordinates define the sector studied (f , H, r).

At this point, it can be determined that the studied sector is a well-defined sector, since 409 lines corresponding to a number of edge points of a large number of different images are responsible for this.

However, when the study is not focused on an edge point or discontinuity of the object, but rather a point that presents a certain spatial generality in its environment, such as a point on a wall of the box where the number of Concurrent lines on that edge decrease notably, in that case the definition of the sector will not be as optimal as in the previous case.

For example, a sector in the previous situation would be a sector that would have coordinates H = 0.225175 and f = 0.125664, which only counts for its definition with 24 available lines. Therefore, the definition of this sector will have less quality than the sector seen previously, which had 409 lines.

It can be concluded by saying that the number of lines that surround the sector is a binding factor in its good definition, and that the methodology presented here correctly and more accurately detects the areas of edges and discontinuities of the objects, when contrary to other techniques that are precisely not applicable in these cases (image matching through correlation).

Finally, comment that the statistics calculated in the case of the lines that cross a sector are: the mean of the distance to the object, the standard deviation and the standard deviation of the mean.

The procedure outlined so far, accumulated a straight line in the sector each time a projective beam crossed said sector. The proposed improvement consists of accumulating the lines, but using weights to give more importance to some lines than to others.

According to Figure 5, the line R, referenced with the number 9, crosses 5 different sectors whose distance is less than 3 millimeters from the object considered.

Without using weights as up to now, each of these sectors would increase the number of lines that define it by one unit (as long as the line passes less than 3 millimeters from the object).

Using weights, the amount to add to the sector counter that indicates the number of lines that define it would be 1 divided by 5 (the line R helps define 5 sectors).

In the cells of the table provided by the developed software, the number of lines accumulated in each sector without using weights is shown first, and below the number of lines with weights.

ES 2 249 162 A1

In the studied sector of 409 accumulated lines, of all of them, for example, it will show that only 26.8 accumulated lines are real. The most precise sector defined will be given by: presenting a high number of accumulated lines and presenting a quality factor, with values ​​greater than 0.2.

The quality factor will be defined by the quotient between the number of accumulated lines and the number of accumulated lines using weights, obtaining a quality factor result that will oscillate between 0 and 1. As the last step in the improvement of the modeling, the straight lines whose residual is too high.

At this point, the software provides for clarification, some graphs distributed according to the number of sectors that present certain values ​​for the number of lines, quality factor and corrections to be applied. In some graphs, the number of lines that surround each sector will be shown, both with the accumulated lines and with the accumulated lines using weights. And in other graphs the number of sectors that present a certain quality factor is shown, this factor being able to oscillate between 0 and 1. The sectors above 0.2 are very well defined.

In another section of the modeling visualization software, the aforementioned results are also offered but in a three-dimensional way on the object by means of color legends on their faces.

Considering now the precision obtained in the model, the quality of the model can be encrypted from two points of view that, as described below, are opposed: the number of points used in the modeling and their quality.

From the point of view of the number of points that define the model, it can be said that the precision is directly defined by the number of heights and the number of sectors, although the number of points that define the model is the product of both, also the operating time will depend on the number of this product. A poorly defined model with a number of sectors of 200 x 200, requires 40,000 points, a very defined model of 2,000 x 2,000 points, requires 4,000,000 points. Since the cumulative process of lines is repetitive, we will say that the operating time grows proportionally to the number of sections of the modeling.

After checking the time it takes to evaluate a simple model (200 x 200), you can easily predict the reasonable maximum dimension based on the time you want to invest in generating the model.

From the point of view of the quality of the points there are two factors that influence, the size of the pixel and the number of accumulated lines.

The images obtained that have approximately 200,000 pixels, using a higher resolution image (4,000,000 pixels) would obtain greater detail in the definition of the edge. If the support has a size of 500 x 500 mm and a border resolution of 0.25 mm is desired, an image of 2,000 x 2,000 pixels will be needed that is easy to obtain with a digital camera, that is, the necessary image size could be calculated based on the expected edge resolution.

On the other hand, the definition of the model is improved if the edge obtained is defined by a large number of projective rays (pixels). The number of projective rays that are going to strike on average in any area will depend on the pixel size and the size of the sectors. If the size of the sectors is 3 mm and the size of the pixel 1 mm, an average of 9 pixels per sector would be expected; with more accumulated pixels in well-defined edges.

If you want well-defined points, you can choose (without changing the camera) to define the model with few points, which will accumulate many pixels at each point of the model and you will be sure that this point is correct. If, on the contrary, a model with many points is desired, it will be necessary to settle for the definition of the point from a few pixels. A minimum figure in terms of the average number of accumulated pixels that guarantee the result would be at least 4 pixels per point, reaching optimal results with values ​​greater than 16 pixels per point.

The exposed procedure presents a limitation by its own definition: the impossibility of detecting the surface of the object in the concave areas of the same.

Indeed, if the object that is arranged has a cavity or concave area, since the process is based on which the projective rays turn out to be the envelopes of the object, the concave areas do not have an external tangent. Even though all object edge points from all available images are considered, the object will appear modeled without that cavity or concave area.

Even so, the number of points calculated automatically can constitute a high percentage of the total surface of the object, and in addition the procedure is able to recognize those areas where an incorrect modeling may have been created, by means of the study of the sectors that present a small quality factor.

Therefore, another complementary method is proposed to correct these irregularities, using stereoscopic vision, performing multiple resections in space, taking advantage of the large number of images available.

ES 2 249 162 A1

To this end, in a following section the software allows calculating, from the sectors that define the obtained modeling, quadrangular or triangular meshes, in the format of wires, faces or textures of the object itself, with the option that the representation presents some colored faces of the model according to some parameters that help to study the object, such as the number of lines that define the sector, the quality factor or the standard deviation of these lines.

In the wireframe model, the meshing of these models is created by joining the mean coordinates of four contiguous sectors. Since four points cannot be coplanar, triangular models can also be created, especially for the purpose of creating faces and later applying textures to them.

In the face model, an established color differentiation is performed based on the quality factor [0-1]. Applying a logarithmic scale may be suitable for increasing resolution at low values. As indicated above, the areas on the edge of the object will have values ​​closer to unity.

As for the texturing of the object, it consists of applying the portion of the image corresponding to each triangle of the model. Due to the large number of triangles that make up the models (tests have been carried out with a resolution of 1,440,000 triangles), it is currently not feasible to create an image for each texture of each triangle, since the file created in VRML would be huge and the computer couldn't handle it easily.

The provisional solution adopted consists of giving a solid color to each triangle, which will correspond to the color of the pixel of the coordinate point of the center of each triangle or that of its average. The downside of this method is that if you model at a small resolution (less than 300 x 300 sectors, 360,000 triangles), the size of the faces is too large to apply only one color and the textures appear grainy.

The color of the texture is taken from the most appropriate image. For each sector of coordinates (f, H) the image whose projection center is perpendicular to it is calculated, and the value of the pixel is obtained by means of the collinearity condition.

Finally, the application of the developed software allows the comparison of two models made on the same object under different conditions, in order to graphically show the differences that appear between one form of modeling and another based on different conditions in the video capture. , and where parameters such as those mentioned below are taken into consideration:

- Shots taken at different distances from the object,

- Different value of the focal length of the camera when using different optical zoom factor in the two shots,

- Different lighting conditions,

- Different inclinations in the camera angle, and

- Coordinate system between the two shots by not placing the object in the same position on the platform.

The difficulty that arises when comparing different models of the same object is that they have to be reduced to the same ground coordinate system. Next, they will also have to be reduced to the same cylindrical coordinate system, so that the sectors of both models coincide in coordinates (F, H) and the differences between the r coordinates of both models can be evaluated. For the purposes of comparison of the two models, both must be calculated with the same resolution and present the same ranges of heights and angles.

To carry out the comparison of the two models and have them in the same coordinate system, a similarity transformation is applied without deformation. The transformation to be applied is a Helmert transformation in space (Pérez, 2001). The transformation parameters are 6, namely: rotation (w, j, k) and translation (T_X, T_Y, T_Z), since the homothecy factor H is equal to unity in this case.

If the coordinates of a control point in the two models are, respectively, (X₀, Y₀, Z₀) and (X_i, Y_i, Z © the transformation is given by:

^Xi ^{- [(r}ii x ^X0 + ^ri2 X ^Y0 + ^ri3 X ^Z0) ^{- T}X^] = ⁰

Yi - [(^r2i X X0 + ^r22 X Y0 + ^r2. 3 X Z0) - Τγ] = 0

Zi - [(r₃i X X0 + r₃two X Y0 + r₃₃ X Z0) - Tz] = 0 materializing the adjustment after linearizing the system by:

ES 2 249 162 A1

Au, n X ^Xn, 1 ^{- K}u, 1 - R_n,I where:

u = equations n = unknowns

In an application example, 6 control points are used, so the system comprises 18 equations and 6 unknowns.

To be able to carry out the comparison of the coordinates of the sectors of the two models, it is necessary to carry out an adjustment of the transformed points (the Helmert transformation has been applied to convert the coordinates of the points of the second model on the first), on the sectors of the first model. This process introduces an error in the comparison of up to half a sector, since each transformed coordinate (X, Y, Z) is searched for the sector of the closest first model.

The numerical comparison of the values ​​of the unknowns obtained from the shots taken is done by subtracting the r coordinates of the two models. The average of the differences is preferably slightly more than 1 millimeter.

Finally, a visual presentation of these differences in space is made. The differences represent the subtraction of the r coordinates of the second model with respect to those of the first model. The visual display may show the positive differences colored in a different color than the negative differences.

As will be understood, although these differences are small, they can be considerably reduced with the use of models with higher resolution than that of the example considered (in the example described, 300 x 300 sectors have been considered).