Method for generating a model of a flat object from views of the object
Abstract
Procedure for generating a model of a flat object from views of the flat object, in which the procedure generates a selected representation from a depth map of the flat object, a volumetric representation of the flat object and a mesh representation of the object flat, by at least the following steps: i) calibrate (1) at least a first chamber (12) and a second chamber (13); ii) calculate (2) the 3D coordinates of at least three points belonging to a plane of the plane object; iii) calculate (3) an equation of the plane comprised in the plane object (10); iv) select (4) at least one region that represents the surface of the planar object, in at least one image plane provided by at least one camera; and, v) calculate (5) an intersection between the selected region representing the surface of the object and the plane equation, in which step ii) additionally comprises: - selecting (30) a point and a first image point, wherein the first image point represents the selected point in the image plane provided by the first camera, and in which said selected point is comprised in the flat object; - calculate (32) a first ray that joins the center of the first camera with the first image point, and also with the selected point; - calculate (31) the projection of the selected point on the image plane provided by the second camera; - calculate (32) at least one second ray by means of at least the image plane of the second camera, which connects the center of the second camera with the second image point and also with the selected point; - determine (33) the 3D coordinates of the selected point by calculating the point of intersection between the first ray and the least one second ray; - repeat (34) the previous steps for at least two more points, obtaining at least the 3D coordinates of three points belonging to the plane of the plane object, characterized in that the projection of the selected point on the image plane of the second camera is calculates by means of a calculation option selected between manual and semi-automatic and because the semi-automatic calculation option is selected from semi-automatic levels as a set comprising a low level, a medium level and a high level.
Term
4.8 yearsto projected expiry
Projected expiry 11 July 2031, counted from filing; an application has no term until it is granted.
- Priority
- Filed
- Published
- Today
- Projected expiry
15 claims: 3 independent, 12 dependent
- 1ES 2 532 452 T3 REIVINDICACIONES 1. Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto plano, en el que el procedimiento genera una representación seleccionada de entre un mapa de profundidad del objeto plano, una representación volumétrica del objeto plano y una representación en malla del objeto plano, mediante al menos las siguientes etapas:i) calibrar (1) al menos una primera cámara (12) y una segunda cámara (13);ii) calcular (2) las coordenadas en 3D de al menos tres puntos que pertenecen a un plano del objeto plano;iii) calcular (3) una ecuación del plano comprendido en el objeto plano (10);iv) seleccionar (4) al menos una región que represente la superficie del objeto plano, en al menos un plano de imagen proporcionado por al menos una cámara;y, v) calcular (5) una intersección entre la región seleccionada que representa la superficie del objeto y la ecuación del plano, en el que la etapa ii) comprende adicionalmente: • seleccionar (30) un punto y un primer punto de imagen, en el que el primer punto de imagen representa el punto seleccionado en el plano de imagen proporcionado por la primera cámara, y en el que dicho punto seleccionado está comprendido en el objeto plano;• calcular (32) un primer rayo que une el centro de la primera cámara con el primer punto de imagen, y también con el punto seleccionado;• calcular (31) la proyección del punto seleccionado sobre el plano de imagen proporcionado por la segunda cámara;• calcular (32) al menos un segundo rayo por medio de al menos el plano de imagen de la segunda cámara, que conecta el centro de la segunda cámara con el segundo punto de imagen y también con el punto seleccionado;• determinar (33) las coordenadas en 3D del punto seleccionado por medio del cálculo del punto de intersección entre el primer rayo y el menos un segundo rayo;• repetir (34) las etapas anteriores para al menos dos puntos más, obteniendo al menos las coordenadas en 3D de tres puntos que pertenecen al plano del objeto plano, caracterizado porque la proyección del punto seleccionado sobre el plano de imagen de la segunda cámara se calcula por medio de una opción de cálculo seleccionada entre manual y semi-automática y porque la opción de cálculo semi-automático se selecciona de entre niveles semi-automáticos como un conjunto que comprende un nivel bajo, un nivel medio y un nivel alto.
- 2Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 1, en el que la etapa i) adicionalmente comprende calcular los parámetros extrínsecos e intrínsecos de al menos la primera cámara y segunda cámara resolviendo las ecuaciones para tantos puntos en 2D/3D como grados de libertad comprendan las cámaras.
- 3Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 1, en el que la opción de cálculo manual comprende que un usuario, mediante una interfaz gráfica de usuario que muestra al menos el plano de imagen proporcionado por la segunda cámara, seleccione el punto de imagen que mejor representa el punto previamente seleccionado mediante el plano de imagen proporcionado por la primera cámara.
- 4Procedimiento de generación de un modelo de un objeto a partir de vistas del objeto, de acuerdo con la reivindicación 1, en el que el nivel bajo comprende las siguientes etapas:• calcular una línea epipolar en al menos el plano de imagen proporcionado por la segunda cámara, en el que dicha línea epipolar es la línea dibujada sobe el plano de imagen de la segunda cámara que representa el primer rayo por medio de la calibración de las cámaras;• representar la línea epipolar por medio de la interfaz gráfica de usuario;y, • seleccionar un segundo punto de imagen sobre la interfaz gráfica de usuario, en el que el segundo punto de imagen es un punto seleccionado a lo largo de la línea epipolar que mejor representa el punto seleccionado mediante el plano de imagen proporcionado por la primera cámara. ES 2 532 452 T3
- 5Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 1, en el que el nivel medio comprende las siguientes etapas:• determinar una primera área de imagen que contiene al menos el primer punto de imagen y almacenar al menos un valor de información del punto de imagen seleccionado de entre el color, la intensidad de color y una combinación de los mismos, de al menos el primer punto de imagen y cada uno de los puntos de imagen que rodean el primer punto de imagen;• calcular una línea epipolar en al menos el plano de imagen proporcionado por la segunda cámara, en el que dicha línea epipolar es la línea dibujada sobre el plano de imagen de la segunda cámara que representa el primer rayo por medio de la calibración de las cámaras;• determinar una segunda área que contiene al menos un punto de imagen comprendido en la línea epipolar, y almacenar al menos un valor de información del punto de imagen seleccionado de entre el color, la intensidad de color y una combinación de los mismos, de al menos dicho primer punto de imagen comprendido en la línea epipolar y cada uno de los puntos de imagen que rodean el punto de imagen comprendido en la línea epipolar;• comparar el valor de la información del punto de imagen de la primera área de imagen con el valor de la información del punto de imagen de la segunda área de imagen;• repetir las dos etapas previas para cada uno de los puntos de imagen que forman la línea epipolar;y, • seleccionar un segundo punto de imagen de entre un grupo de puntos de imagen formado por los puntos de imagen obtenidos en cada repetición por medio de una opción seleccionada entre manual y automática;la opción automática comprende seleccionar el segundo punto de imagen como el punto de imagen del total de puntos de imagen comprendidos en la línea epipolar, para lo que la segunda área de imagen del punto de imagen comprende un índice de reproducción de la primera área de imagen mayor que un umbral;la forma manual comprende resaltar con un color predeterminado el segundo punto de imagen, que se selecciona como el punto de imagen del total de puntos de imagen comprendidos en la línea epipolar, para el que la segunda área de imagen del segundo punto de imagen comprende un índice de reproducción de la primera área de imagen mayor que un umbral, y seleccionar un punto de entre los resaltados por la interfaz gráfica del usuario.
- 6Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 1, en el que el nivel alto comprende las siguientes etapas:• determinar un conjunto de puntos característicos en al menos los planos de imagen proporcionados por la primera cámara y la segunda cámara, siendo dichos puntos característicos los situados en esquinas y bordes del objeto, y en las superficies de objetos con textura, los puntos obtenidos por un algoritmo de detección de esquinas;• determinar una primera área de imagen que contenga al menos el primer punto de imagen y almacenar al menos un valor de información del punto de imagen seleccionado de entre el color, la intensidad de color y una combinación de los mismos para al menos el primer punto de imagen y cada uno de los puntos de imagen que rodean al primer punto de imagen;• calcular una línea epipolar en al menos el plano de imagen proporcionado por la segunda cámara, en el que la línea epipolar es la línea dibujada sobre el plano de imagen de la segunda cámara que representa el primer rayo por medio de la calibración de las cámaras;• determinar una segunda área de imagen que comprende al menos un punto de imagen contenido en la línea epipolar, y almacenar al menos un valor de información del punto de imagen seleccionado de entre el color, la intensidad de color y una combinación de los mismos, de al menos el punto de imagen contenido en la línea epipolar y cada punto de imagen que rodea el punto de imagen contenido en la línea epipolar;• comparar el valor de información del punto de imagen de la primera área de imagen con el valor de información del punto de imagen de la segunda área de imagen;• repetir las dos etapas previas para cada uno de los puntos de imagen que forman la línea epipolar;y, • seleccionar un segundo punto de imagen de entre un grupo de puntos de imagen formado por los puntos de imagen obtenidos en cada una de las repeticiones por medio de una opción seleccionada entre manual y automática;la opción automática comprende seleccionar el segundo punto de imagen como el punto de imagen del total de puntos de imagen comprendidos en la línea epipolar, para lo que la segunda área de imagen del punto de imagen comprende un índice de reproducción de la primera área de imagen mayor que un umbral;la forma manual comprende resaltar con un color predeterminado el segundo punto de imagen, que se selecciona como el punto de imagen del total de puntos de imagen comprendidos en la línea epipolar, ES 2 532 452 T3 para lo que la segunda área de imagen del segundo punto de imagen comprende un índice de reproducción de la primea área de imagen mayor que un umbral y seleccionar un punto de entre los resaltados mediante la interfaz gráfica del usuario.
- 7Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 4, 5 o 6, en el que el rayo que une el centro de la primera cámara con el punto seleccionado y al menos el rayo que une el centro de la segunda cámara con el punto seleccionado están definidos por las ecuaciones de línea respectivas mediante las siguientes etapas:• obtener la localización en 3D del centro de la cámara del sistema de coordenadas calibrado;• obtener la localización en 3D del primer punto de imagen y del segundo punto de imagen que representan el mismo punto seleccionado en al menos el plano de imagen de la primera cámara y el plano de imagen de la segunda cámara, respectivamente;y, • determinar al menos una primera ecuación de línea y una segunda ecuación de línea, estando la primera ecuación de línea descrita por el vector que une la localización en 3D del centro de la primera cámara con la localización en 3D del primer punto de imagen y estando la segunda ecuación descrita por el vector que conecta la localización en 3D del centro de la segunda cámara con la localización en 3D del segundo punto de imagen.
- 8Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 7, en el que la determinación de las coordenadas de al menos un punto en 3D mediante el cálculo del punto de intersección entre el primer rayo y al menos el segundo rayo comprende adicionalmente:• establecer las coordenadas de al menos un punto en 3D cuando el punto de intersección entre el primer rayo y el al menos segundo rayo pertenece simultáneamente a la primera ecuación y a la segunda ecuación que determinan dicho primer rayo y dicho al menos segundo rayo respectivamente;y, • calcular las coordenadas de un punto en 3D cuya distancia a dicho primer rayo y dicho al menos segundo rayo es mínima cuando no existe punto de intersección entre dicho primer rayo y dicho al menos segundo rayo.
- 9Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 7 u 8, en el que la etapa iii) adicionalmente comprende:• calcular la expresión del plano determinada por la siguiente ecuación: en la que Xd, Yd y Zd son las coordenadas del punto en 3D que pertenece al plano del objeto y aN, bN y cn son las coordenadas de un vector N normal al plano;el vector N normal al plano es el producto vectorial de dos vectores pertenecientes al plano y definidos por al menos tres puntos en el plano del objeto plano.
- 10Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 1, en el que calcular una intersección entre la región que representa la superficie del objeto y la ecuación del plano comprende adicionalmente las siguientes cuatro subetapas:• encontrar, para cada región formada en el plano de imagen de la primera cámara, al menos un rayo que pasa a través del centro de la primera cámara y un punto comprendido en cada una de las regiones;• encontrar el punto de intersección entre dicho al menos un rayo y el plano, en el que dicho punto de intersección representa las coordenadas en 3D de un punto que pertenece al objeto plano real;• repetir las dos sub-etapas anteriores para cada uno de los puntos que forman la región y formar una nube de puntos cuando se forma la región para al menos dos puntos;• unir los puntos de intersección encontrados por al menos la primera cámara dentro de la región seleccionada.
- 11Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con cualquiera de las reivindicaciones anteriores, en el que el procedimiento adicionalmente comprende, para llevar a cabo la representación del mapa de profundidad del objeto plano, las siguientes etapas:• seleccionar una región plana perteneciente al objeto a modelar;• analizar en un plano de imagen de una cámara específica, los puntos de imagen de las regiones correspondientes a dicha región plana;ES 2 532 452 T3 • calcular, para cada punto, la distancia desde el centro de la cámara a cada punto de la región plana;• repetir las dos etapas anteriores para cada punto perteneciente a la región plana;y, • representar el mapa de profundidad con las distancias anteriormente calculadas y otorgando un valor seleccionado entre cero y un primer valor predeterminado a todos los puntos que estén comprendidos en la región plana seleccionada.
- 12Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con cualquiera de las reivindicaciones anteriores, en el que el procedimiento comprende, para realizar la representación volumétrica del objeto plano, las siguientes etapas:• definir un sistema común de coordenadas mediante la calibración de una disposición de cámaras formada por al menos dos cámaras;• definir una caja delimitante que comprende el objeto plano a modelar con referencia al sistema de coordenadas definido;• dividir la caja delimitante en pequeños elementos de volumen denominados vóxeles;• determinar para cada vóxel comprendido en la caja delimitante si dicho vóxel está ocupado mediante un algoritmo seleccionado entre un algoritmo sencillo y un algoritmo perfeccionado;y, • representar la representación volumétrica mediante la representación de los vóxeles ocupados.
- 13Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 12, en el que el algoritmo sencillo comprende:• calcular las coordenadas del centroide de cada vóxel;• calcular la distancia desde el centroide a un punto de la nube de puntos;y, • etiquetar el vóxel como ocupado si la distancia calculada anteriormente entre el centroide y el punto de la nube de puntos es inferior a un umbral predeterminado, y etiquetar el vóxel como no ocupado si dicha distancia es superior al umbral predeterminado.
- 14Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 12, en el que el algoritmo comprende adicionalmente:• calcular las coordenadas del centroide de cada vóxel;• calcular la distancia desde el centroide al centro de la cámara;• identificar el punto en la nube de puntos con la distancia mínima al centroide y la longitud del rayo que pasa a través del vóxel y el centro de la cámara;• calcular la distancia entre el punto identificado en la nube de puntos y el centro de la cámara;• etiquetar el vóxel como ocupado si la distancia calculada anteriormente entre el centroide y el centro de la cámara es al menos igual que la distancia entre el punto identificado en la nube de puntos y el centro de la cámara y etiquetar el vóxel como no ocupado en caso contrario;y • aplicar mejoras en el algoritmo mejorado seleccionadas entre: o una intersección por volúmenes que se obtiene mediante las dos siguientes sub-etapas: • determinar la ocupación de los vóxeles por medio del centro de la cámara proporcionado por la primera cámara y al menos la segunda cámara independientemente;y, • etiquetar el vóxel como ocupado si y sólo si está etiquetado como ocupado en cada una de las vistas proporcionadas por la primera cámara y al menos la segunda cámara;o una ocupación por grosor, que comprende las siguientes sub-etapas: • seleccionar el grosor T del objeto plano;• calcular las coordenadas del centroide de cada vóxel;• calcular la distancia desde el centroide al centro de la cámara;ES 2 532 452 T3 • identificar el punto en la nube de puntos con la distancia mínima al centroide y la longitud del rayo que pasa a través del vóxel y el punto en el centro de la cámara;• calcular la distancia entre el punto identificado en la nube de puntos y el centro de la cámara;y • etiquetar el vóxel como ocupado si la distancia calculada anteriormente entre el centroide y el centro de la cámara es al menos igual que la distancia entre el punto identificado en la nube de puntos y el centro de la cámara, y más pequeño que la distancia entre el punto identificado en la nube de puntos y el centro de la cámara más el grosor T y en caso contrario etiquetar el vóxel como no ocupado;o intersección por volúmenes con ocupación por el grosor que obtiene la intersección de los diferentes volúmenes mediante las siguientes subetapas: • determinar la ocupación de los vóxeles por medio del centro de la cámara proporcionado por la primera cámara y al menos la segunda cámara independientemente;• seleccionar el grosor T del objeto plano;• calcular las coordenadas del centroide de cada vóxel;• calcular la distancia desde el centroide al centro de la cámara;• identificar el punto en la nube de puntos con la distancia mínima al centroide y la longitud del rayo que pasa a través del vóxel y el punto en el centro de la cámara;• calcular la distancia entre el punto identificado en la nube de puntos y el centro de la cámara;• etiquetar, en cada vista proporcionada por la primera cámara y la segunda cámara el vóxel como ocupado si la distancia calculada anteriormente entre el centroide y el centro de la cámara es al menos igual que la distancia entre el punto identificado en la nube de puntos y el centro de la cámara y menor que la distancia entre el punto identificado en la nube de puntos y el centro de la cámara más el grosor T y etiquetar el vóxel como no ocupado en caso contrario;y, • etiquetar el vóxel como ocupado si y sólo si está etiquetado como ocupado en cada una de las vistas proporcionadas por la primera cámara y al menos la segunda cámara;
- 15Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto, de acuerdo con la reivindicación 1, en el que el procedimiento adicionalmente comprende las siguientes etapas para realizar la representación en malla del objeto plano:• determinar tres puntos por superficie localmente plana comprendida en el objeto plano;• unir los tres puntos formando un triángulo;• unir los triángulos obtenidos formando una malla;y, • representar la malla.
Independent claims15
152 paragraphs in 6 sections, as filed
ES 2 532 452 T3
DESCRIPTION
Procedure for generating a model of a flat object from views of the object
Object of the invention
The present invention, as expressed in the title of this specification, refers to a method of generating a model of a flat object from views of the object provided by at least two cameras that take views of the object. By means of the method of the present invention, representations such as a depth map of the flat object, a volumetric representation of the flat object and a mesh representation of the flat object can be obtained. The method of the present invention is especially aimed at modeling objects in three dimensions by analyzing views in two dimensions for their subsequent reconstruction in three dimensions. Its application ranges from tele-presence by videoconference to the generation of models for various purposes: analysis, education, recreation, etc.
Background of the invention
The existing state of the art discloses different types of procedures for modeling an object. These procedures are mainly classified into passive procedures and active procedures. In the area of active procedures, sensors such as lasers or structured light scanners, or also Time-of-Flight cameras are used. There are other possibilities such as projecting, with the help of a video projector, a known pattern onto an object and deducing the shape of the object by analyzing the deformation suffered by the pattern due to the shape of the object.
In the area of passive procedures, most techniques exploit geometric triangulation that relates two or more views of the object of interest.
Some examples of procedures for modeling objects are present, among others, in the following works:
BLEYER M ET AL: “A layered stereo matching algorithm using image segmentation and global visibility constraints”, ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING, AMSTERDAM [UA]: ELSEVIER, vol. 59, n ° 3, May 1, 2005 (05/01/2005), pages 128-150, ISSN: 0924-2716, describes a stereo algorithm that takes advantage of image segmentation, assuming that disparity varies smoothly within a homogeneous color segment and that the intensity discontinuities coincide with the segment edges. The disparity within a segment is represented by a flat equation and, to derive the flat model, an initial disparity map is generated. This technique achieves good results in areas with intensity discontinuities and related occlusions, where lost stereo information is replaced by surrounding regions.
SHUM HY ET AL: "Interactive 3D modeling from multiple images using scene regularities", LECTURE NOTES IN COMPUTER SCIENCE / MICCAI 2000, SPRINGER, DE, June 1, 1998 (06-01-1998), pages 236-252, DOI: 10. 1007 / 3-540-49437-5_16 ISBN: 978-3-540-24128-7 discloses two image-based interactive 3D modeling systems. The first system builds 3D models from a collection of panoramic image tiles, which consists of a set of images taken from the same point of view. The second system extracts stereo structures by representing the scene as a collection of flat layers.
TREUILLET S ET AL: Three dimensional Assessment of Skin Wounds using a Standard Digital Camera ”, IEEE TRANSACTIONS ON MEDICAL IMAGING, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 28, n ° 5, May 1, 2009 (05-01-2009), pages 752-762, ISSN: 0278-0062 discloses the construction of 3D models of skin lesions in color images. Images are uncalibrated images acquired with a pocket variable focal length digital camera.
The present invention is in the field of passive procedures that include the following view-based object modeling approaches:
• Structure from Movement, SfM, which consists of estimating the scene model in front of a moving camera. However, the technique is only applicable to a set of multiple static cameras. Generally, an SfM algorithm maps the views of a set of points in the scene. By establishing this correspondence, it is possible to triangulate the position of the points in the three dimensions of space in front of one or more cameras. From this point, there are several possibilities to generate a model of an object. One possibility is to take advantage of triangulation to calibrate the position of the camera along its movement or the position of each static camera. A dense model of the shape of the scene can be obtained, for example, by means of Shape-from-Stereo. Another possibility is to assume that the surface between any three points is locally flat. This model is obtained, therefore, by connecting points in groups of three, by a triangle. The set of 3D triangles forms a mesh that represents the shape of the object. In this sense, procedures that reconstruct parts of the flat object are known from the state of the art. First, maps are established between flat segments. Four points are found per segment or region and then a homography is induced. This homography
ES 2 532 452 T3 allows establishing the epipolar geometry between the views. Finally, the set of segments can be positioned in 3D.
• 3D volumetric reconstruction. This approach ranges from the least to the most accurate modeling. For example, the box that encloses the real object would be too rough a model. There are more precise models such as the Convex Wrap (CH), the Visual Wrap (VH) and the Photo Wrap (PH). One of the most widespread volumetric models due to its good relationship between precision and low computational cost is the Visual Envelope (VH). The Visual Envelope is obtained through a procedure called Shape-from-Silhouette (SfS). In a first phase, the Form-from-Silhouette extracts the active entities of the scene (silhouettes of the object) by means of a set of cameras. The Visual Envelope corresponds, therefore, with the volume within the intersection of the cones that go from the optical center of the cameras through the silhouettes in the optical planes of the cameras. The set of cameras must be intrinsically and extrinsically calibrated beforehand. In this way, calibration can be obtained using the set of control points whose coordinates are automatically known as a set of key feature points, as in the approach of Structure from Motion.
• Shape from Shading, SfSh, takes care of recovering the shape from a gradual variation of the shading in the view. The idea behind Shape from Shading is that the intensity of the color can be described as a function of the surface, the shape, and the direction of the light source. Most SfSh algorithms assume that the direction of the light source is known.
The passive procedures described above have several drawbacks depending on the procedure used. In the case of procedures based on Structure from Motion (SfM), the drawbacks arise from objects without texture. In fact, in the absence of texture on the surface of the object, the resulting model is very rough. In the case of very limited texture but with enough points to calibrate the set of cameras, the Shape-from-Stereo procedure can be used. However, the result of the above procedure has the drawback that it is not capable of isolating the object from the objects that form the background or that surround the object that is being modeled. In the particular case of the procedures described above and which are based on finding four points of a segment and generating a homography, the entire calibration process depends on the possibility of establishing correspondence between the detected planes, which is not feasible for objects without texture.
On the other hand, the Visual Hull obtained with a generic SfS procedure depends mainly on two aspects. First, the camera positions determine the performance of the SfS procedure. Another limitation of the applicability of this procedure is that the silhouettes are extracted by comparison with a known static background. This implies that the object cannot be present in the scene when the background is captured. Consequently, this procedure is only valid for objects that can be easily obtained or introduced into the scene, but not for modeling a part of a room, such as a wall or a fixed board.
Therefore, it would be desirable to find a procedure for generating a model of a flat object from views of the object that does not depend on the texture of the object to be modeled or the consequent limitation implied by the correct calibration of the cameras, as well as the ability to translate the object to be modeled with respect to the background or the location of the object.
Description of the invention
To achieve the objectives and avoid the drawbacks indicated above, the present invention consists of a method for generating a model of a flat object from the views of the object. This procedure is based on two concepts: epipolar geometry and image segmentation. Epipolar geometry establishes the geometric relationships between two or more cameras that capture the same scene. Image segmentation consists of generating segments (regions) or areas of an image that have similar characteristics: such as color.
The present invention covers a method that generates a volumetric model of a flat object. The method of the present invention takes advantage of the relationships of the epipolar geometry between two or more cameras, and the segmentation of the surface of the object seen by said cameras. The procedure is separated into two main stages. The first stage consists of calculating the equation of the plane in 3D that defines the surface of the plane object. For this, the method of the present invention uses the triangulation of three points that belong to the flat object. The second stage consists of finding the segment (region), in one or more views, that best represents the object. With the calibration of the cameras it is possible to find the ray defined by each image point (pixel) that belongs to the segment. Finally, the volumetric representation of the object is determined by the intersection of all the rays with the plane defined by the equation of the plane in three dimensions.
The procedure for generating a model of a flat object, from views of the object of the present invention, generates a representation selected from a depth map of the flat object, a three-dimensional representation of the flat object, and a mesh representation of the object. plane, through at least the following stages:
ES 2 532 452 T3
i) calibrating at least a first camera and a second camera;
ii) calculating 3D coordinates of at least three points that belong to a plane of the plane object;
iii) calculating an equation of the plane comprised in the plane object;
iv) selecting at least one region representing the surface of the flat object, in at least one optical or image plane provided by at least one camera; Y,
v) calculate the intersection between the selected region representing the surface of the object and the equation of the plane.
Step i) additionally comprises calculating the extrinsic and intrinsic parameters of at least the first camera and the second camera solving the equations for as many points in 2D / 3D, as degrees of freedom include the cameras.
Stage ii) additionally comprises:
• selecting a point and a first image point, in which the first image point represents the selected point in the plane of the image provided by the first camera, and in which the selected point is comprised in the flat object;
• calculate a first ray that joins the center of the first camera with the first image point, and also with the selected point. (Obviously, the center of the first camera, the first image point, and the selected point are aligned.)
• calculate the projection of the selected point on the image plane provided by the second camera.
• calculate at least a second ray by means of at least the image plane of the second camera, which connects the center of the second camera with the second image point, and also with the selected point;
• determine the 3D coordinates of the selected point by calculating the point of intersection between the first ray and the at least one second ray;
• repeat the previous steps for at least two more points, obtaining at least the 3D coordinates of three points that belong to the plane of the flat object.
The aforementioned projection of the selected point on the image plane of the second camera is calculated by means of a calculation option selected between manual and semi-automatic.
The manual calculation option comprises that a user, by means of a graphical user interface that shows at least the image plane provided by the second camera, selects the image point that best represents the previously selected point by means of the image plane provided by the first camera.
The semi-automatic calculation option is selected from a set of semi-automatic levels comprised of a low level, a medium level and a high level.
To calculate the projection for the low level of the semi-automatic calculation option, the following steps are carried out:
• calculating an epipolar line in at least the image plane provided by a second camera, where the epipolar line is the line drawn on the image plane of the second camera that represents the first ray by means of the calibration of the cameras;
• represent the epipolar line by means of the graphical user interface; and • selecting a second image point on the graphical user interface, where the second image point is the selected point along the epipolar line that best represents the selected point by means of the image plane provided by the first camera.
To calculate the projection by the mean level of the semi-automatic calculation option, the following steps are carried out:
a) determining a first image area containing at least the first image point and storing at least one image point information value selected from color, color intensity and a combination thereof, of at least the first image point and each of the image points surrounding the first image point;
ES 2 532 452 T3
b) calculating an epipolar line in at least the image plane provided by a second camera, where the epipolar line is the line drawn on the image plane of the second camera that represents the first ray by means of the calibration of the cameras;
c) determining a second image area comprising at least one image point comprised in the epipolar line, and storing at least one information value of the image point selected from among the color, the color intensity and a combination of the themselves, of at least the image point contained in the epipolar line and each of the image points surrounding the image point contained in the epipolar line;
d) comparing the information value of the image point of the first image area with the information value of the image point of the second image area;
e) repeating steps c) and d) for each of the image points that form the epipolar line; Y
f) selecting a second image point from a group of image points formed by the image points obtained in each repetition by means of an option selected between manual and automatic. The automatic way comprises selecting the second image point as the image point of the total number of image points comprised in the epipolar line, for which the second image area of the image point comprises a reproduction index of the first image area greater than a threshold. The manual way comprises highlighting with a predetermined color the second image point, which is selected as the image point of the total of image points comprised in the epipolar line, for which the second image area of the second image point comprises a rendering index of the first image area greater than a threshold, and select a point from the points highlighted by the graphical user interface.
To calculate the projection by the high level of the semi-automatic calculation option, the following steps are carried out:
a) determining a group of characteristic points in at least the image planes provided by the first camera and the second camera. These characteristic points being those located at the corners and edges of any object (flat or non-flat), and, on the surface of textured objects, the points obtained by the corner detection algorithm selected from the corner detection algorithm. Moravec's, Harris and Stephens / Plessey's corner detection algorithm, Harris's multi-scale operator, Shi and Tomasi's corner detection algorithm, the approximation of the contour curvature, the characteristic detection of DoH (Determinant of Hessians), DoG (Difference of Gaussians) and LaG (Laplacian of Gaussians), the algorithm of detection of corners of Brady and Wang, the algorithm of SUSAN corner detection, Hedley and Trajkovic corner detection algorithm, FAST feature detector, automatic synthesis point detector with genetic programming, the operators of points of interest with adapted affinity and any other of the state of the art;
b) matching the first image point with a previously determined characteristic point;
c) determining a first image area containing at least the first image point and storing at least one information value of the selected image point from among the color, the color intensity and a combination thereof for at least the first image point and each of the image points surrounding the first image point;
d) calculating an epipolar line in at least the image plane provided by the second camera, where the epipolar line is the line drawn on the image plane of the second camera that represents the first ray by means of the calibration of the cameras;
e) determining a second area of the image comprising at least one image point comprised in the epipolar line, and storing at least one information value of the image point selected from color, color intensity and a combination of the themselves, of at least the image point contained in the epipolar line and each of the image points surrounding the image point contained in the epipolar line;
f) comparing the information value of the image point of the first image area with the information value of the image point of the second image area;
g) repeating steps e) and f) for each of the image points that form the epipolar line; Y
f) selecting a second image point from a group of image points formed by the image points obtained in each repetition by means of an option selected between manual and automatic. The automatic way comprises selecting the second image point as the image point of the total number of image points comprised in the epipolar line, for which the second image area of the image point comprises a reproduction index of the first image area greater than a threshold. The manual way comprises highlighting with a predetermined color the second image point, which is selected as the image point of the total of image points comprised in the epipolar line, for which the second area of
ES 2 532 452 T3 image of the second image point comprises a reproduction index of the first image area greater than a threshold, and selecting a point from among the points highlighted by the graphical user interface.
The first image point and at least the second image point make it possible to define the two rays mentioned above in the second sub-stage and the fourth sub-stage of stage ii). A ray (first ray) is defined by three points (camera center, first image point, and the selected point) that belong to a line. The other at least one ray (one per camera) is defined by three points (center of the camera, second image point, and the selected point) that belong to a line.
In this way, the ray that connects the center of the first camera with the selected point and the ray that connects the center of the second camera with the selected point are defined by the respective line equations through the following steps:
• obtain the 3D location of the center of the camera from the calibrated coordinate system;
• obtaining the 3D location of the first image point and the second image points representing the same selected point in at least the image plane of the first camera and the image plane of the second camera respectively;
• determine at least a first line equation and a second line equation. The first equation of the line is described by the vector that connects the 3D location of the center of the first camera with the 3D location of the first image point and the second equation of the line is described by the vector that connects the location in 3D of the center of the second camera with the 3D location of the second image point.
Next, to determine the coordinates of at least one 3D point by calculating the point of intersection between the at least two rays, it further comprises:
• establish the coordinates of at least one point in 3D when the point of intersection between the two rays belongs simultaneously to the first equation and the second equation that determine the two rays;
• calculate the coordinates of a point in 3D whose distance to the first ray and at least the second ray is minimal when there is no point of intersection between the first ray and at least the second ray.
On the other hand, step iii) of the process of the present invention additionally comprises calculating the expression of the plane determined by the following equation:
<img file="ES2532452T3_D0001.tif" />
where Xd, Yd and Zd are the coordinates of a point in 3D that belongs to the plane of the object and aN, bN and cn are the coordinates of a vector N normal to the plane. The vector N normal to the plane is the vector product of two vectors belonging to the plane and defined by at least three points in the plane of the plane object.
Furthermore, step iv) of the process of the present invention is carried out by means of an option selected between manual and automatic;
The manual option comprises at least the following stages:
• calculating a sub-region comprising selecting each of the image points surrounding a selected image point by means of the graphical user interface. These image points meet a similarity criterion. The similarity criterion is a predetermined comparison between information values of the selected image points from between the color, the color intensity and the information values of the selected image points between the color and the color intensity of each of said image points surrounding the image point;
• repeat the previous step for as many sub-regions as the user establishes: and • form a region of the flat object by connecting the previously calculated sub-regions.
On the other hand, the automatic option begins with the calculation of the selected point, the first image point and at least the second image point defined previously in the semi-automatic calculation option to calculate the projection of the selected point on the image plane. provided by the second chamber (third sub-stage of stage ii of the procedure). The automatic option comprises at least the following stages:
• select at least two image points on the same image plane that best represent the at least two points of the plane object, in which these two image points are selected from the first two
ES 2 532 452 T3 image points in the image plane of the first camera and the two second image points of the image plane of at least the second camera;
• storing the values of the image point information selected from among the color and color intensity of the image points comprised in the line segment connecting the two image points selected in the previous step;
• calculate a two-dimensional histogram that represents in one dimension all the possible information values of the image point that has given the image point in the domain of space, and in the other dimension, the number of times that a certain value of Information of the selected image point between color and color intensity has appeared in the line segment. Therefore, the histogram shows peaks for the information values of the selected image point between the most repeated color and color intensity;
• identify, for each peak of the histogram, the image points, which included in the line segment, have the information values of the color image point and color intensity represented in each of the peaks;
• calculating a sub-region for each of the identifying image points that are formed by the image points surrounding the identifying image point and meeting the similarity criterion;
• repeat the previous step for as many sub-regions as identifying image points that have been identified; and • forming a region of the planar object by connecting the previously calculated subregions.
To calculate the intersection between the region representing the surface of the object and the equation of the plane, the method of the present invention additionally comprises the following four sub-steps:
• finding, for each region formed in the image plane of the first camera, at least one ray that passes through the center of the first camera and a point comprised in each region;
• find the point of intersection between said at least one ray and the plane. This intersection point represents the 3D coordinates of a point that belong to a real plane object;
• repeat the two previous sub-stages for each point that forms the region and form a point cloud when the region is formed for at least two points;
• join the intersection points found by at least the first camera within the selected region.
The four previous sub-stages are optionally repeated replacing the first chamber with the at least second chamber. Furthermore, the four sub-stages mentioned above are executed only once with at least the second camera. Therefore the first camera can be replaced with the second camera or any other camera comprised in a system based on two or more cameras.
With all the stages and sub-stages of the process of the present invention described in this way, the elements to generate any of the selected representations are obtained from a depth map of the object plane, a volumetric representation (also known as three-dimensional representation) of the object and a mesh representation of the object plane.
The procedure additionally comprises the following steps to perform the representation of the depth map of the flat object:
• select a flat region belonging to the object to be modeled;
• analyze in an image plane of a specific camera, the image points of the region corresponding to said flat region;
• calculate, for each point, the distance from the center of the camera to each point in the flat region;
• repeat the two previous steps for each point belonging to the flat region; and, • represent the depth map with the previously calculated distances and giving a selected value between zero and a first predetermined value to all the points that are included in the selected planar region. For all points that do not fall within the selected planar region, but do belong to the image plane, a second default value is assigned.
The procedure includes the following steps to perform the volumetric representation of the flat object:
ES 2 532 452 T3 • define a common coordinate system by calibrating a camera arrangement formed by at least two cameras;
• define a bounding box that includes the object to be modeled with reference to the defined coordinate system;
• divide the bounding box into small volume elements called voxels;
• determining for each voxel included in the bounding box whether said voxel is occupied by an algorithm selected from a simple algorithm and a sophisticated algorithm; and, • represent the volumetric representation by representing the occupied voxels.
The simple algorithm comprises performing the following sub-stages:
• calculate the coordinates of the centroid of each voxel;
• calculate the distance from the centroid to a point in the point cloud; and, • labeling the voxel as occupied if the previously calculated distance between the centroid and the point of the point cloud is less than a predetermined threshold, and labeling the voxel as unoccupied if the previously calculated distance is greater than the predetermined threshold.
While the improved algorithm comprises performing the following sub-stages:
• calculate the coordinates of the centroid of each voxel;
• calculate the distance from the centroid to the center of the camera;
• identify the point in the point cloud with the minimum distance to the centroid and the length of the ray that passes through the voxel and the center of the camera;
• calculate the distance between the point identified in the point cloud and the center of the camera; labeling the voxel as occupied if the previously calculated distance between the centroid and the center of the camera is greater than or equal to the distance between the point identified in the point cloud and the center of the camera and otherwise labeling the voxel with no occupied; and • apply enhancements to the enhanced algorithm through enhancements selected from:
° an intersection by volumes that is obtained through the following two sub-stages:
• determine the occupation of voxels using the camera center provided by the first camera and at least the second camera independently; and, • label the voxel as busy if and only if it is labeled as 'busy' in each of the views provided by the first camera and the at least second camera;
° an occupation by thickness, comprising the following sub-stages:
• select the thickness T of the flat object;
• calculate the coordinates of the centroid of each voxel;
• calculate the distance from the centroid to the center of the camera;
• identify the point in the point cloud with the minimum distance to the centroid and the length of the ray passing through the voxel and the point in the center of the camera;
• calculate the distance between the point identified in the point cloud and the center of the camera; and • label the voxel as occupied if the previously calculated distance between the centroid and the center of the camera is greater than or equal to the distance between the identified point in the point cloud and the center of the camera, and smaller than the distance between the point identified in the point cloud and the center of the chamber plus the thickness T and otherwise label the voxel as unoccupied;
or intersection by volumes with occupation by the thickness obtained by the intersection of the different volumes through the following sub-stages:
• determine voxel occupancy using the camera center provided by the first camera and at least the second camera independently;
ES 2 532 452 T3 • select the thickness T of the flat object;
• calculate the coordinates of the centroid of each voxel;
• calculate the distance from the centroid to the center of the camera;
• identify the point in the point cloud with the minimum distance to the centroid and the length of the ray passing through the voxel and the point in the center of the camera;
• calculate the distance between the point identified in the point cloud and the center of the camera;
• label, in each view provided by the first camera and the second camera, the voxel as occupied if the previously calculated distance between the centroid and the center of the camera is greater than or equal to the distance between the point identified in the point cloud and the center of the chamber and less than the distance between the point identified in the point cloud and the center of the chamber plus the thickness T and label the voxel as unoccupied otherwise; and, • label the voxel as busy if and only if it is labeled as busy in each of the views provided by the first camera and the at least second camera;
On the other hand, to perform the mesh representation of the flat object, the following steps are carried out:
• determine three points for each locally flat surface included in the flat object;
• join the three points forming a triangle;
• join the triangles obtained by forming a mesh; and, • represent the mesh.
The description in this way described has been made mainly on the basis of two cameras, extending to any number of cameras where the references to the second camera and the image plane produced by the second camera are replaced by the third camera and the image plane. of the third camera, the fourth camera and the image plane of the fourth camera, etc. The same would happen with the second image point.
A rapid modeling of any object comprising flat segments is obtained by the above-described steps of the process of the present invention. This is due to the easy, one-time calibration process for a given camera setup.
Modeling of objects using view-based techniques is closely related to the SfS and SfM procedures. The great advantage of the present invention over said methods is that it makes it possible to select the surface of the object manually or automatically, making it possible to overcome the limitation of the texture that the SfM method has. The present invention can be applied to a room not prepared with static furniture, which implies a limitation of the SfM procedure. Such procedures based on SfS techniques can only be used after manual selection of the segments. It is not possible to determine the plane, and therefore a rough representation of the Visual Envelope is obtained.
The present invention includes two ways to obtain a model of an object, manual or semi-automatic. The present invention allows control at each stage of a semi-automatic process in contrast to prior art procedures based on SfM or SfS techniques, allowing a more controlled result of the procedure.
Brief description of the figures
Figure 1 shows a flow chart of the process of the present invention in which the main steps of the process are shown.
Figure 2 shows an implementation of the method of the present invention in which a planar object, a point P comprised in the planar object, three cameras and a global coordinate system are shown.
Figure 3 shows a flow chart of the method of the present invention for calculating the three-dimensional coordinates of three points belonging to the same plane.
Figure 4 shows an implementation of the method of the present invention in which the manual option to select the projection of a point is shown.
Figure 5 shows an implementation for the steps of the method of the present invention corresponding to the low level of semi-automatic calculation of the projection of a point.
ES 2 532 452 T3
Figure 6 shows an embodiment for the steps of the method of the present invention for a medium and high level of all semi-automatic levels for the calculation of the projection of the selected point.
Figure 7 shows an embodiment of the method of the present invention for calculating the regions and sub-regions comprised in the flat object by means of an automatic mode.
Figure 8 shows an embodiment of the method of the present invention applied to the calculation of the flat object.
Description of an embodiment of the invention
Hereinafter, a description of various embodiments of the invention is made, by way of illustration and not limitation, with reference to the numbering used in the figures.
Figure 1 shows a flow chart of the process of the present invention in which the main steps of the process are shown. The stages of the procedure are: calibrate at least two cameras (1), calculate the 3D coordinates of at least three points that belong to the same plane of the plane object (2), calculate the equation of the plane included in the plane object (3), select at least one region representing the surface of the plane object, in at least one image plane provided by at least one camera (4) and calculating the intersection between the selected region representing the surface of the object and the equation of the plane (5). By means of these steps, a representation selected from among a depth map (6) of the flat object, a volumetric representation (7) of the flat object and a mesh representation (8) of the flat object is obtained.
Figure 2 shows an implementation of the method of the present invention in which a flat object (10), a point P (11) comprised in the flat object, a first camera (12), a second camera (13), are shown. a third camera (14) and a global coordinate system (15). To calibrate the three previous cameras with respect to the global coordinate system, the extrinsic parameters are calculated by means of the translation and rotation matrices that relate the interior coordinate system of each camera (12A, 13A, 14A) with the coordinate system global (15). Additionally, the centers of each camera Ol (12B), Oc (13B) and OR (14B) that can optionally be the focal point of each camera, as well as the image planes (12C, 13C, 14C) provided by each of the three chambers (12, 13, 14) are obtained with the intrinsic calibration of each chamber. On each of the image planes (12C, 13C, 14C), the image point (Pl, Pc, Pr) representing the selected point P (11) is also shown. In addition, it shows how the parts that comprise epipolar geometry are calculated. First, a ray (13E) is calculated that joins, for example, the center (Oc) of the second camera with the image point (PC) of the image plane representing the selected point P (11). Another ray (12E, 14E) can be obtained for each chamber with a similar procedure. Next, the epipolar lines (12D, 14D) are calculated on the image plane of the first camera and on the image plane of the third camera respectively, the epipolar lines being the calculated projections of the ray (13E) on the image plane. of the first camera and on the image plane of the third camera respectively. The epipolar lines (12D and 14D) are drawn on the image plane (12C, 14C) of each of the cameras. The projections of the ray (13E) on the image plane are calculated with the calibration of the cameras which is based in turn on the global position system.
Figure 3 shows a flow chart of the method of the present invention to calculate the 3D coordinates of at least three points belonging to the same plane. The first step is to select a point on the plane comprised in the flat object by means of the image plane provided by the first camera (30). The second stage is to calculate the projection of the selected point on the image plane provided by the second camera by a manual or semi-automatic mode (31). The third stage is to define two rays by means of the image plane of the first camera and the image plane of the second camera, one per camera and the selected point, which connects the center of the first camera with the selected point and the center. of the second camera with the selected point (32). The fourth stage is to determine the coordinates of the selected point in 3D by calculating the point of intersection between the two rays (33). The fifth stage is to repeat the previous stages until the 3D coordinates of the three points that belong to the plane of the plane object (34) are obtained.
Figure 4 shows an implementation of the method of the present invention for the manual option of calculating the projection of a selected point on the image plane of the second camera. The implementation comprises a plane object (40) which in turn comprises three points P1, P2 and P3, a first camera (41) which comprises an image plane (41A) displayed by a graphical user interface (43), a second camera (42) comprising an image plane (42A) displayed by means of the graphical user interface (44). A user (45), through the graphical user interface (44) that shows the image plane (42A) of the second camera (42), selects the image point that best represents the previously selected point by means of the image plane (41A) of the first chamber (41). The centers of the first and second cameras Ol and Or are also shown, as well as the rays (46A, 46B, 46C, 47A, 47B, 47C) that join the centers (O1, O2) of the cameras with the image points displayed (P1 ', P2', P3 ') or selected (P1 ”, P2”, P3' ') by the user (45), which also join the points (P1, P2, P3) included in the flat object (40 ).
ES 2 532 452 T3
Figure 5 shows an implementation for the steps of the method of the present invention corresponding to the low level of the semi-automatic calculation of the projection of a point P selected by means of the image plane (51 A) of the first camera (51) on the plane image (52a) from the second camera (52). The first stage is to select a first image point (P1) of the image plane (51 A) that represents a selected point (P) of the plane object (50) by means of an interface (53) that represents the selected point in the image plane of the first camera (51). The second stage is to calculate a first infinite ray that passes through the center (O1) of the first camera (51) AND the first image point (P1). To improve understanding, the first ray (55) and the second ray (56) are shown with a solid line from the center of the cameras to point P, and with dashed lines from point P to infinity. Since the 3D coordinates of the center (O1) and the first image point (P1) are known, the coordinates of the first ray are calculated automatically. The third stage is to calculate the epipolar line (57) on the image plane of the second camera (52). The epipolar line is the line on the image plane of the second camera that represents the first ray. The epipolar line (57) is represented as a line (57A) on the user interface (54). In the fourth stage, the user selects a second image point (P2) as an image point along the epipolar line (57) that best represents the selected point (P) by means of the user interface (54) connected to the second chamber (52).
Since the difference between the medium level and the high level of all semi-automatic levels is mainly based on the fact that the high level additionally comprises calculating the characteristic points by means of the corner detection algorithms of the state of the art , the implementation of figure 6 serves to show the steps of the procedure comprised in the medium level as well as in the high level.
Figure 6 shows an embodiment for the steps of the method of the present invention for the medium level and the high level of the semi-automatic calculation option for the calculation of the projection of the point selected by the image plane provided by the first camera. on the image plane provided by the second camera. The first stage of the procedure shown in the embodiment of figure 6 is to select a point P of the plane object (60) and a first image point (P1) that represents the selected point in the image plane (61A) of the first camera (61) displayed in a graphical user interface (63). The second step is to determine a first image area (68) containing the first image point (P1) and the image points surrounding the first image point, and store the selected image point information values from among the color, the color intensity, and a combination thereof (not shown), of each of the image points contained in the first image area. The third stage is to calculate a first infinite ray (65) that passes through the center (O1) of the first camera (61) and the first image point (P1). To improve understanding, the first ray (65) and the second ray (66) are shown with a solid line from the centers (O1, 02) of the cameras to point P, and with dashed lines from point P to infinity. Since the 3D coordinates of the center (O1) of the first camera and the first image point (P1) are known, the coordinates of the first ray are calculated automatically. The fourth step is to calculate the epipolar line (67) on the image plane (62A) of the second camera (62). The epipolar line is the line on the image plane of the second camera that represents the first ray. The epipolar line (67) is represented as a line (67A) on the user interface (64). The fifth stage is to determine a second image area (68A, ..., 68N) containing an image point of the epipolar line and the image points around the image point of the epipolar line and store the information values of the image points selected from color, color intensity, and a combination thereof (not shown), of each of the image points contained within the second image area. The sixth step is to compare the information values of the image point of the second area with the information values of the image point of the first area. The seventh stage is to repeat the fifth stage and the sixth stage for each of the image points comprised in the epipolar line. The eighth stage is to select an image point (P2), called the second image point, that best represents the selected point (P) by means of an option selected between manual and automatic.
The automatic way comprises selecting the second image point as the image point of the total number of image points that comprise the epipolar line, for which its second image area comprises a reduction rate of the first image area greater than a threshold . The manual way comprises highlighting with a predetermined color the second image point, which is selected as the image point of the total of image points comprised in the epipolar line, for which the second image area of the second image point comprises a reduction rate of the first image area greater than a threshold, and selecting the highlighted point (P2) by the graphical user interface (64A).
Figure 7 shows an embodiment of the method of the present invention for calculating the regions and sub-regions comprised in the flat object (70) by means of the automatic mode. The first stage is to select at least two points (P1, P2) belonging to the plane, the image points (P11, P21) of the image plane of the first camera (71) that represent the selected points, the projections of these points ( P12, P22) on the image plane (72) of the second camera, these projections being some points selected from the second image points. The second step is to draw the line segment (73) between the image points (P22, P12) comprised in the image plane of the second camera (72). The third stage is to store some selected image point information values between the color and color intensity of the image points comprised in the line segment. The fourth stage is to calculate a two-dimensional histogram (76) that represents in one dimension (X) all the possible information values of an image point that could have been given to an image point in the space domain, and in another dimension (Y), the number of times a
ES 2 532 452 T3 certain information value of an image point, selected between color and color intensity has appeared in the line segment. Thus, the histogram shows some peaks for the most repeated values of color or color intensity. The fifth step is to identify, for each peak of the histogram (77), some identifying image points that, comprised in the line segment (73), have color values and color intensity represented in each peak. The sixth step is to calculate a sub-region (74, 75), for each of the identifying image points, which are formed by the image points that surround the identifying image point and that meet a predetermined similarity criterion. The seventh stage is to repeat the previous stage for as many subregions as identifying image points have been identified. Finally, the eighth stage is to form a region of the flat object by connecting the previously calculated sub-regions.
Figure 8 shows an implementation of the method of the present invention applied to the calculation of an arbitrarily shaped flat object. After the camera calibration of the first camera (81) with respect to the global coordinate system (85), the equation of the plane (84) is calculated by calculating the 3D coordinates of the three points (for example, P1 , P2 and P4) that belong to plane (84). Next, a region (83) is selected in the image plane (82) of the first camera (81) as well as the points P1, P2, P3, and P4 included in the object, forming a cloud of 5 points. Finally, the arbitrarily shaped flat object (80) remains defined by the intersection between the rays (for example: 86a, 868, 86C, 86D) defined by the center of the camera (C) and any point (for example, P1 ', P<sub>2</sub>', P3', P4 ') comprised in the selected region (83) that represents the surface of the object and the equation of the plane (84). In other 10 words, of the entire infinite region that comprises the equation of the plane, the sought region delimits the plane obtaining the plane object and therefore its coordinates in 3D.
Contents6
9 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9
9 members in 5 offices
Priority claims3
| Document | Office | Kind | Date |
|---|---|---|---|
| 201031291 | Spain | A | |
| 201031291 | Spain | – | |
| 2011061731 | European Patent Office (EPO) | W |
Members9
| Document | Office | Kind | |
|---|---|---|---|
| WO2012025288A1 | World Intellectual Property Organization (WIPO) | A1 | |
| ES2392229A1 | Spain | A1 | |
| EP2609570A1 | European Patent Office (EPO) | A1 | |
| US2013208098A1 | United States of America | A1 | |
| ES2392229B1 | Spain | B1 | |
| EP2609570B1 | European Patent Office (EPO) | B1 | |
| ES2532452T3This record | Spain | T3 | |
| BR112013004682A2 | Brazil | A2 | |
| US9380293B2 | United States of America | B2 |
Numbers
- Publication
- 2532452
- Application
- 11735622
Titles2
- Spanish
- Procedimiento de generación de un modelo de un objeto plano a partir de vistas del objeto
- English
- Procedure for generating a model of a flat object from object views
Classification
- CPC, 6
- H04N13/239
- G06T2200/24
- G06T2207/10012
- G06T2207/20101
- G06T7/593
- G06T15/08
- IPC, 2
- G06T7 00
- H04N13 239