Apparatus and method for automatically arranging three dimensional scan data using optical marker

The present invention relates to a three-dimensional measurement data automatic alignment device using an optical marker and a method thereof. More specifically, the three-dimensional measurement object is measured at various positions and angles, and the three-dimensional measurement is performed. It relates to a three-dimensional measurement data automatic alignment device using an optical marker for automatically aligning the relative positions of these three-dimensional measurement data in one coordinate system after obtaining the data, and a method thereof. ..

In general, when measuring a predetermined measurement object, the optical coordinate measuring device can obtain three-dimensional data on the surface only in a region that can be seen by the three-dimensional measuring device.

Therefore, in order to measure other areas of an object that the CMM cannot see, the part to be measured is rotated or moved, or the measuring device itself is moved to measure. It must be positioned so that it can be seen and then measured. In addition, in order to obtain perfect 3D data through such measurement data, the process of measuring 3D measurement objects in various directions and angles, aligning these data in one coordinate system, and synthesizing them is performed. Should be done.

At this time, the reason for aligning to one coordinate system is generally defined based on the coordinate system in which each measurement data is fixed to the measuring device, but three-dimensional measurement is performed to photograph an object from different angles. When the device is moved to a different position and angle, the 3D data measured at the moved position is defined in the coordinate system together with the CMM, and the data taken at different positions is defined in the different coordinate system. This is because it has been done.

In order to match such a coordinate system, it is necessary to know the amount of movement of the measuring device, and the method of calculating the amount of movement is to move the measuring device using a numerically controlled device. There is a method of finding the absolute amount of movement and a method of calculating the amount of movement by the measuring device based only on the measured data. When calculating the amount of movement of the measuring device based on the measured data alone, as in the latter case, when measuring an object, the measured data should be measured so that they overlap each other, and the user is measured at different angles. Corresponding points are input at the parts of the measured data that overlap each other, and coordinate conversion is performed so that the input corresponding points match.

At this time, the work of manually inputting the corresponding points for the overlapping portions in the measurement data may cause an error when the operator inputs the corresponding points. In particular, if there is no characteristic figure on the surface of the object, more errors will occur. In addition, when the object is large and the shape is complicated, the operator must input the corresponding points because the measurement must be repeated from tens to hundreds of times while changing the angle and position of the three-dimensional measuring device. At the same time as the time increases, manual errors are accumulated, the corresponding points are entered incorrectly due to the mistake of the operator, and the corresponding points are accumulated, resulting in incorrect alignment. become.

To compensate for the above drawbacks, it is possible to attach to the surface of a measurement object to be measured, which is a small object that can be labeled, recently called a marker or target. Therefore, a method has been developed that recognizes a marker or a target sign and helps the operator to input an accurate corresponding point. In addition, a technology for automating the recognition of corresponding points through a video processing algorithm is being developed. That is, FIG. 1 is a drawing schematically showing a state in which a conventional sticker-type marker is attached to a measurement object and measured three-dimensionally. In the same drawing, a measurement object having a predetermined three-dimensional image is shown. A plurality of markers (4) are irregularly attached along the entire surface of the object (2), and at the same time, the surface of the measurement object (2) to which each marker (4) is attached is attached to each region. It will be taken repeatedly so that they overlap separately.

When it becomes possible to acquire a plurality of measurement data by photographing the surfaces of the measurement objects (2) to which a plurality of markers (4) are attached so as to overlap each other, the marker (marker between each data) (manually performed by the operator) Recognize the corresponding points according to 4). This is as illustrated in FIG.

As shown in FIG. 2, when the first and second measurement data (I1, I2) obtained by photographing the overlapping regions with respect to the surface of the measurement object (2) are acquired, the first measurement data (I1) is obtained. ) And the second measurement data (I2), respectively, the operator searches for a common number between (M1) and (M2) of the marker video data, and matches them as corresponding points for each measurement. The data can be aligned.

On the other hand, in the case of technology that automates the recognition of corresponding points, a method is used in which different patterns are inserted into each marker, unique markers are searched for through video processing, and different measurement data are automatically aligned based on these. There is also.

<p> In the case of the conventional point recognition technology for measurement data using a marker, the marker physically occupies a certain volume with respect to the measurement object. Therefore, when the marker adheres to the surface of the measurement object, the measurement object is measured. There is a drawback that only a part of the surface is blocked, and the part blocked by the marker is lost from the measurement data.</p><p> In such a case, the data lost in the part where the marker was located is later used by the interpolation method to guess and fill it, or the marker is peeled off and then the measurement is performed again to obtain the measurement data of the lost part. There are methods to acquire again, but all of these methods have the disadvantage that they not only require a lot of work time, but also reduce the accuracy of measurement.</p>

<p> The present invention is intended to solve the above-mentioned problems, and an object of the present invention is to use a non-contact marker that can be optically generated without losing a measurement portion of a measurement object. It is an object of the present invention to provide a three-dimensional measurement data automatic alignment device using an optical marker capable of automatically aligning three-dimensional data measured at different angles, and a method thereof.</p><p> In order to achieve the above object, according to the apparatus of the present invention, a plurality of optical markers are used in a three-dimensional data measuring apparatus for aligning three-dimensional measurement data acquired by photographing from a predetermined measurement object at various angles. An optical marker generating means for projecting a pattern on the surface of the measurement object, a three-dimensional projection means for projecting a pattern on the surface of the measurement object for three-dimensional measurement on the measurement object, and optical from the measurement object. An image acquisition means for acquiring a two-dimensional image including a marker projected by the expression marker generating means and acquiring three-dimensional measurement data of a measurement object projected by the three-dimensional projection means, and an image acquisition means. A control means that extracts the three-dimensional position of the marker from the relationship between the two-dimensional image and the three-dimensional measurement data, and searches for the relative position of the three-dimensional measurement data from the position of the marker based on each three-dimensional measurement data. Provided is a three-dimensional measurement data automatic alignment device using an optical marker characterized by being composed of.</p><p> In order to achieve the above object, according to the method according to the present invention, the step of moving the image acquisition means to a position suitable for acquiring the image of a specific area of the measurement object and the blinking of the marker generation means. In the stage of acquiring a two-dimensional image for a specific area of the measurement object on which the optical marker is projected by driving the optical marker so that the optical marker is projected on the surface of the measurement object, the three-dimensional projection means The pattern pattern is projected on the surface of the measurement object, and the three-dimensional measurement data for a specific region of the measurement object on which the pattern is projected by the image acquisition means is acquired and acquired by the image acquisition means. The three-dimensional position of the marker is extracted from the relationship between the two-dimensional image and the three-dimensional measurement data, the relative position of the three-dimensional measurement data is searched from the position of the marker based on each three-dimensional measurement data, and each measurement data is obtained. Provided is a method for automatically aligning three-dimensional measurement data using an optical marker, which is characterized by the step of aligning.</p>

<p> According to the present invention, the three-dimensional measurement data obtained from different angles and positions are automatically aligned, and the conventional method is such as a paper sticker or a magnet on which a pattern for image recognition is printed. A marker with a volume is used to find the relative position of different measurement data. The surface of the object to be measured is blocked by the marker at the part where the marker is located, and the measurement data is lost at the part where the marker is located. Although it has the drawback of being distorted or distorted, in the present invention, since optical markers having no physical volume are used to find the relative positions of different measurement data, the object to be measured is to be measured. There is an effect that the measurement data is not lost by the marker even in the portion where the marker is located.</p><p> Further, since the optical marker according to the present invention does not require a process of attaching or detaching the marker to or from the object to be measured, the measurement of the three-dimensional measurement data can be performed quickly. Therefore, it is very convenient to use, it is safe for measurement objects that may be damaged, and it has the effect of being able to be used semi-permanently.</p>

Hereinafter, the first embodiment of the present invention configured as described above will be described in detail with reference to the accompanying drawings.

That is, FIG. 3 is a drawing showing the configuration of the three-dimensional measurement data automatic alignment device using the optical marker according to the first embodiment of the present invention.

As illustrated in FIG. 3, the three-dimensional measurement data automatic alignment device according to the first embodiment of the present invention includes a marker generator (12), a projection unit (16), an image acquisition unit (18), and a moving drive unit (20). ), Movement mechanism (22), video input unit (24), marker blinking control unit (26), projection control unit (28), microprocessor (30), and buffer (32).

The marker generator (12) is for projecting a pattern that can be recognized by the image acquisition unit (18) on the surface of the measurement object (10) to be measured optically. A plurality of marker output units (14) are installed so that a plurality of optical markers are simultaneously projected in mutually irregular survey directions over the entire surface pointing to the object to be measured (10).

It is desirable that the plurality of marker output units (14) of the marker generator (12) apply a laser point capable of projecting a plurality of red points on the surface of the measurement object (10). When such a laser point is applied, it is possible to easily grasp the position of the point projected on the surface of the measurement object (10) with the image acquired by the image acquisition unit (18) such as a camera. Become.

Here, the marker generator (12) is not limited to the laser point, and can fix the relative position between the measurement object to be measured and the marker generator (12) during the measurement. It is possible to repeat the steps of making the marker appear and disappear at the same position on the surface of the object to be measured (10). Also, any optical marker may be applied that has a deep optical depth and allows the marker to be well focused on the surface of the object to be measured (10).

The marker generator (12) allows a plurality of optical markers to be arranged along the periphery of the object so that the optical markers are evenly projected on the surface of the object to be measured (10). The number of can be changed according to the size of the object to be measured (10) and the measurement area. In addition, the marker generator (12) may project a plurality of the measurement object (10) in various directions so that the optical marker can be projected on the entire surface of the measurement object (10). It is desirable to be able to arrange the image, and it is fixed so that there is no relative movement with the object to be measured (10) during shooting by the image acquisition unit (18).

In the drawing, the projection unit (16) projects a predetermined pattern or a laser stripe on the surface of the measurement object (10) so that three-dimensional data can be acquired. This involves projecting space-encoded light onto the surface of the object to be measured (10) using a projection device such as an LCD projector, or projecting laser light onto the surface of the object to be measured (10). It is made possible to acquire as three-dimensional data through the acquisition section (18).

Here, the projection unit (16) employs a slide projector or an electronic LCD projector composed of a light source capable of projecting a predetermined pattern, a pattern film, and a lens, or a laser diode capable of projecting a laser stripe. Is desirable, and a series of stripes is projected onto the object to be measured (10) while the patterned film with stripes is transferred between the light source and the lens by a predetermined transfer means.

In short, the pattern film is made of a film in which a striped pattern having a plurality of sections such as five is printed long in the horizontal direction. The pattern film of the projection unit (16) is a domestic patent application No. 2002-10839 (title of the invention: a three-dimensional measuring device using a multiple stripe pattern) filed by the applicant on February 28, 2002. And method), the film may be in the form of multiple stripes formed for each section.

The same can be said for a measuring device using a laser stripe.

Further, when the projection unit (16) optically performs the three-dimensional measurement, the measurement data is damaged when the marker is projected onto the measurement object (10) at the time when the three-dimensional measurement is performed. It is desirable to prevent the marker from being projected onto the object to be measured (10) during the measurement because of the risk.

The image acquisition unit (18) is composed of an image sensor capable of receiving an image like a CCD camera or a CMOS camera, and is optically optically attached to the surface of the measurement object (10) from the marker generator (12). When the marker is projected by the method, the image obtained by this is taken and acquired.

The image acquisition unit (18) can be installed as a separate camera with respect to the projection unit (16), but it is desirable that the image acquisition unit (18) is integrated with the projection unit (16). This not only acquires the two-dimensional image projected from the optical marker, but also acquires the three-dimensional image projected from the projection unit (16), which simplifies the equipment configuration. Instead, by using the same image acquisition means, any point in the 2D image and the corresponding point on the 3D measurement data are matched without any separate calibration (coordinate system correction) work. It becomes possible.

Here, the image acquisition unit (18) synchronizes with the blinking cycle of the marker generator (12) with respect to the optical marker and the blinking cycle of the projection unit (16), and is applied to each surface region of the measurement object (10). On the other hand, two-dimensional video data and three-dimensional measurement data are photographed and acquired, which is the same as that shown in FIG. 4a or FIG. 4c.

As illustrated in FIG. 4a, when the marker generator (12) is turned on in the image acquisition unit (18) and a plurality of laser markers are projected on the surface of the measurement object (10), a plurality of laser markers are projected. The first video data (40) of a predetermined area of the measurement object on which the optical laser marker (RM) of the above is projected irregularly is acquired.

Next, as illustrated in FIG. 4b, the image acquisition unit (18) is in a state where the marker generator (12) is turned off and the laser marker is not projected on the surface of the measurement object (10). The second video data (42) consisting only of the video of the measurement target (10) will be acquired.

Further, as shown in FIG. 4c, in the image acquisition unit (18), the marker generator (12) is turned off and the laser marker is not projected on the measurement object (10). The three-dimensional measurement data by the stripe projected on the measurement object (10) from 16) is photographed and acquired. This is the 1st to 5th measurement data (44a to 44e) in which 3D images by the 1st to 5th stripes (PT1 to PT5) in which five sections exist at different intervals depending on the pattern film are sequentially photographed. Will be acquired in the form of.

Here, in the present invention, as an example, it is set so that there are five sections of the pattern film to be adopted, but the present invention is not limited to this, and five or more patterns may exist.

The moving drive unit (20) measures the projection unit (16) and the image acquisition unit (18) in order to acquire the image of the measurement object (10) by the drive control of the microprocessor (30). Drive to move the object (10) relatively.

The movement mechanism (22) has the power transmitted by the drive of the movement drive unit (20) transmitted, and the projection unit (16) and the image acquisition unit (18) are constant with respect to the measurement object (10). It has a structure for moving in the direction.

Here, in the present invention, the moving drive unit (20) is applied so that the projection unit (16) and the image acquisition unit (18) can be moved by electrical drive, but the movement mechanism (22) ) Can be manually operated so that the operator can move it arbitrarily. In the drawing, the video input unit (24) is for inputting video data acquired from the video acquisition unit (18), and the marker blinking control unit (26) is the microprocessor (30). ) Will cause the optical marker of the marker generator (12) to blink.

The projection control unit (28) controls the transfer speed and transfer direction of the pattern film constituting the projection unit (16), and also controls the blinking of the light source that projects the pattern film.

The image acquisition is performed by driving the dedicated software program for analyzing the measurement data acquired from the measurement object (10) and automatically aligning the measurement data in one coordinate system. Two-dimensional video data and three-dimensional measurement data taken at various angles are input from the unit (18) through the video input unit (24), analyzed, and the measurement data taken at various angles are combined into one. Performs arithmetic processing to automatically align with the coordinate system.

Here, as shown in FIG. 5, the microprocessor (30) has a first video data (40) containing a laser marker (RM) and a second video data (42) not including a laser marker. On the other hand, the image processing for searching the position of the marker is performed, and the third image data (46) from which only the laser marker (RM) is extracted is acquired.

In that state, as shown in FIG. 6, the microprocessor (30) has two-dimensional image data (52) in which the lens center (50) of the camera of the image acquisition unit (18) and the position of the optical marker are extracted. Since it becomes possible to acquire the three-dimensional coordinate values corresponding to the positions of the markers due to the relationship with, the coordinate values for any three markers from the lens center (50) of the camera of the image acquisition unit (18). Corresponds to the position of the marker by estimating an arbitrary three-dimensional coordinate value (a', b', c') on the three-dimensional measurement data (54) located in line with (a, b, c). Three-dimensional coordinate values (a', b', c') can be obtained.

When the projection unit (16) and the image acquisition unit (18) are integrated and installed together, the three-dimensional coordinate value of the pixel corresponding to the position of the marker can be immediately obtained. If the unit (18) is installed separately from the projection unit (16), the projection unit (16) and the image acquisition unit (18) can be calibrated (coordinate system correction). The three-dimensional coordinate value corresponding to the marker can be obtained. Through the above steps, it is possible to know the three-dimensional coordinate value of the position where the marker is located through the three-dimensional measurement data obtained at various angles.

At this time, it is desirable that 4 to 5 or more markers are included in one measurement data, and it is desirable that two adjacent measurement data include three or more common markers.

This is because not only three or more points are required to define the unique position of the coordinate system in the three-dimensional space, but also it is a matter required when searching for a corresponding marker described later.

Further, in the present invention, when an optical marker is used through the marker generator (12), it is possible to use a different pattern for each marker so that the markers can be distinguished from each other. However, in such a case, dozens or hundreds of marker output units (14) must be able to project markers with different patterns, which may complicate the configuration and manufacture of equipment. is there.

In the present invention, in order to automatically align the three-dimensional measurement data of the adjacent region acquired by the continuous imaging of the measurement object (10) in the microprocessor (30), it is calculated from each three-dimensional measurement data. Make it possible to distinguish between markers by using the relative position information between markers.

For example, if there are three points in the space formed by the marker, they can be used to form one triangle. In general, a triangle consisting of three different points has a different shape, so each triangle can be distinguished by comparing the internal angle of the triangle with the length of the side. Then we will use this to distinguish each marker that corresponds to the apex of the triangle. A more detailed explanation of this process is as follows.

That is, as illustrated in FIGS. 7a and 7b, one measurement data (60) has M points obtained from the marker, and the other measurement data (62) has N points obtained from the marker. When there are points, the measurement data (60) is generally_MC₃Each of the different triangles can be constructed, and the other measurement data (62)_MC₃Each of the different triangles can be constructed. After constructing these triangles, total_MC₃×_NC₃Find a pair of triangles that match each other through a comparison of times.

First, as illustrated in FIG. 7a, in the microprocessor (30), points obtained by markers included in one measurement data (60) form a plurality of triangles (T1, T2), and other measurements are made. The points obtained by the markers included in the data (62) will form multiple triangles (T3, T4).

Next, as illustrated in FIG. 7b, the microprocessor (30) is a pair of triangles that match each triangle (T1, T2) (T3, T4) obtained from the markers of each measurement data (60,62). Will be sought out.

Here, there are various methods for determining whether or not each triangle matches, but it becomes possible to search for a pair of triangles by comparing the lengths of the sides. That is, for the two triangles (T1, T3) to be compared, the lengths (a1, a2, a3) (b1, b2, b3) of each of the three sides are obtained, and the lengths of the sides are all the same. , If the order of each side is the same, it can be determined that the triangles match.

First, arrange the lengths of each side in descending order, find a triangle with all three sides having the same length, and if at least two such triangles are detected, check the order of each side. By comparing the lengths of the counterclockwise sides of the longest side of each triangle, it is possible to judge that the triangles having the same side lengths are a pair.

As described above, after distinguishing the corresponding markers in each measurement data, the measurement data is moved so that these markers can be located at the same point in one coordinate system. Using the data (60) as the reference coordinate, the triangle in the reference coordinate system is moved by applying a transformation that matches the same triangle contained in the measurement data.

That is, the transformations that match two triangles at different positions are as shown in the attached FIGS. 8a to 8d.

As shown in FIG. 8a, when two triangles that are congruent and have different positions are given, the information about the corresponding vertices and sides has already been acquired while seeking the congruence of the triangles.

As shown in Fig. 8b, the process of performing transformation to match the position of one vertex of each of the two triangles, setting the reference coordinate system for one triangle to A, and matching the coordinate system B of the remaining triangles to A. The relational expression of the Translation Matrix (T) for it is as shown in Equation 1 below.

<maths num="1"><img file="JP2005534026A_D0001.tif" /></maths> Next, as shown in Fig. 8c, when performing a rotation transformation to match the sides containing the matched vertices for two triangles, the rotation transformation to match any two vectors sharing one point is rotation. It is performed as in Equation 2 which represents the matrix (R1).

<maths num="2"><img file="JP2005534026A_D0002.tif" /></maths> As described above, when the rotation transformation is performed, it becomes as shown in FIG. 8d, and the relationship of the rotation matrix (R2) for matching this with one vertex is as shown in Equation 3 below.

<maths num="3"><img file="JP2005534026A_D0003.tif" /></maths> As mentioned above, if you use the matching side as the axis of rotation, draw a perpendicular line from one vertex that is not included in that side, and then rotate it to the corner of the two lines, you can match the two triangles. Is arranged by Transform Matrix (M) as shown in Equation 4 below.

<maths num="4"><img file="JP2005534026A_D0004.tif" /></maths> In addition, one point (P) of the measurement data that includes each triangle can be moved to a new position by the following mathematical formula 5.

<maths num="5"><img file="JP2005534026A_D0005.tif" /></maths> On the other hand, as described above, the microprocessor (30) adjusts some measurement data to the marker reference by the conversion performed based on the congruence of triangles, and then performs the work for more accurately matching the data. , Since the size of the marker is not a mathematical point, some errors may occur, and the positions between each other are adjusted based on the mesh data that is not the marker data. This is called registering, and through such a process, each measurement data can be more accurately adjusted to one coordinate system.

When trying to match the predetermined point cloud data A with the coordinate system of the point cloud data B, it is adjusted to the coordinate system of B by forcibly moving and rotating A (Rigid Body Tranformation). At this time, the n points of A to be matched are P = [p._i], The corresponding point B is Q = [x_i], The movement and rotation transformations are obtained using the least squares method that can minimize the corresponding distances of P and Q, and such transformations are applied to A. Therefore, the average distance between the point cloud data A and B, which are the opposite sides of P and Qn, is minimized, and the average distance between P and Q is tolerant when the above corresponding points are searched for and applied for movement and rotation conversion. It will repeat until it gets inside.

As a method for obtaining the corresponding point pairs as described above, the point B closest to the normal direction of each point A is found and these two points are designated as point pairs.

The transformation that minimizes the distance between the two corresponding points is calculated using the least squares method, and the function for that is as shown in Equation 6 below.

<maths num="6"><img file="JP2005534026A_D0006.tif" /></maths> Where Q is the registering state vector, and Q = [Q_R| Q_T]²become. However, Q_RIs a Quaternion Vector, which is Q_R= [q₀q₁q₂q₃]^t(q&gt; = 0, q₀² + q₁² + q₂² + q₃²= 1). Q_TIs the Translation Vector, which is Q_T= [q₄q₅q₆]^tCorresponds to.

In Equation 6 above, f (Q) is pi and R (Q)_R) And Q_TWhen applying the transformation, x_iRepresents the root mean square with, and at this time, R (Q) that can minimize f (Q)_R) And Q_TIs obtained by the method of least squares.

R (Q) in the above mathematical formula 6_R) Can be represented by a 3 × 3 rotation matrix, which is defined by Equation 7 below.<maths num="7"><img file="JP2005534026A_D0007.tif" /></maths>

If the set of measurement data (P) is defined as P = [pi] and the reference data set (X) is defined as X = [xi], the center of mass of P and X is , Defined by Equation 8 below.

<maths num="8"><img file="JP2005534026A_D0008.tif" /></maths> Also, the Cross-Covariance Matrix of P and X (Σ)_px) Is defined by Equation 9 below.

<maths num="9"><img file="JP2005534026A_D0009.tif" /></maths> Here, the Cyclic Components of the Anti-symmetric Matrix (A)_ij) Is used to form a column vector (Δ), which is a symmetric 4 × 4 matrix Q (Σ)._px) Is obtained, and it is defined as the following formula 10.

<maths num="10"><img file="JP2005534026A_D0010.tif" /></maths> Where I₃Is a 3x3 Identity Matrix.

In the above formula, Q_TIs Q (Σ)_px) Is the eigenvector (Eigen Vector) corresponding to the maximum eigenvalue (Quaternion) (q) using the eigenvector value.₀, q₁, q₂, q₃), And substitute it into the above equation 7 to obtain the rotation transformation. Meanwhile, Q_T(q₄, q₅, q₆) Is R (Q) obtained from the above formula 7._R) Is used to align the volume center, which can be obtained from the following formula 11.

<maths num="11"><img file="JP2005534026A_D0011.tif" /></maths> As a result, the analytic value finally expressed is as shown in Equation 12 below.

<maths num="12"><img file="JP2005534026A_D0012.tif" /></maths> In the microprocessor (30), when the markers corresponding to the remaining three-dimensional measurement data acquired from the image acquisition unit (18) are distinguished, the one measurement data (60) is obtained. Find the matrix for movement as the reference coordinates and enable it to be automatically aligned.

On the other hand, unlike the method shown in Fig. 8a or Fig. 8d, as a method of moving the measurement data after searching for congruent triangles, a method of matching the coordinate system using the least squares method is used. Will be done.

Since there is information on the corresponding vertices while asking whether the two triangles can be congruent, if P and X are the corresponding vertices in the above mathematical formula 6, the optimum rotation and movement transformation matrix (T) is as follows. It can be calculated by the above formula 13.

<maths num="13"><img file="JP2005534026A_D0013.tif" /></maths> The equation for moving the matrix by applying the point cloud data (P) for matching the coordinate system with the mathematical formula 13 is defined by the following formula 14.

<maths num="14"><img file="JP2005534026A_D0014.tif" /></maths> On the other hand, in the microprocessor (30), in the case of the method of searching for a pair of triangles, three or more markers must be included in the area where each measurement data overlaps, but there are two markers in each measurement data. It is difficult to measure the markers, and at this time, the markers can be distinguished by another method.

That is, since each measurement data including the markers has three-dimensional figurative information, it is possible to distinguish them by using only two markers, and as shown in FIG. 9, one device having overlapping regions. The measurement data (64) has two markers (RM1, RM2) forming two points in space, and the different measurement data (66) has two markers (RM3, RM4) forming two points in space. Then, by using the vector perpendicular to each point, different markers can be distinguished by comparing the two points and the two vectors for each measurement data (64) (66) with each other.

On the other hand, in the microprocessor (30), when the number of markers projected for each measurement data is large and the markers are arranged uniformly, the set may be mistaken. In such cases, markers and two-dimensional data are used to generate and compare additional reference points. For example, if there are three things in common, construct a triangle with these points, draw a perpendicular line perpendicular to the triangle at the center of the weight of the triangle, and then find the intersection of that perpendicular line with the three-dimensional data. 4 reference points are obtained. After that, the average vertical vector information of the object surface can be used from the marker or the periphery of the marker to search for matching pairs.

If the measurement data has only two common points, draw a straight line connecting these two points, draw a circle at the midpoint of the straight line on a plane perpendicular to the straight line, and then draw a circle with the circle. By finding the intersection with the measurement data, the 4th and 5th reference points can be obtained.

According to the preferred embodiment of the present invention, in addition to the method described above, any method can be adopted in which an additional reference point is created around the marker to automatically align the three-dimensional measurement data. Of course, it has become like this.

In FIG. 3, the buffer (32) registers new marker information obtained by automatic alignment processing on the three-dimensional measurement data of the microprocessor (30) in each register.

Next, the operation according to the first embodiment of the present invention performed as described above will be described in detail with reference to the flowcharts of FIGS. 10a and 10b.

First, the microprocessor (30) drives the moving drive unit (20) to operate the moving mechanism (22) with the predetermined measurement object (10) resting on the marker generator (12) side. As a result, the image acquisition unit (18) integrated with the projection unit (16) is moved to a position suitable for measurement of the measurement object (10) (step S10).

In that state, the microprocessor (30) controls the marker blinking control unit (26), lights a plurality of marker output units (14) provided in the marker generator (12), and measures the object. Make sure that multiple markers are projected irregularly on the surface of (10) (step S11). With the optical marker from the marker generator (12) projected onto the measurement object (10), the image acquisition unit (18) photographs a specific area of the measurement object (10), and the optical marker is used. When the two-dimensional image including the marker is acquired, the microprocessor (30) inputs the two-dimensional image data acquired by the image acquisition unit (18) through the image input unit (24) ( Stage S12).

Next, the microprocessor (30) controls the marker blinking control unit (26) to turn off the marker generator (12) so that the optical marker is not projected on the measurement object (10). (Step S13) In that state, when the same area of the measurement object (10) is photographed from the image acquisition unit (18) and a two-dimensional image that does not include an optical marker is acquired, the two-dimensional image data is acquired. It will be input through the video input unit (24) (step S14).

Further, the microprocessor (30) controls the projection control unit (28) to operate the projection unit (16) in a state where the marker generator (12) is turned off and the optical marker is not projected. Then, a pattern pattern for three-dimensional measurement (for example, a stripe pattern of a plurality of sections having different intervals or a multiple stripe pattern) is projected from the projection unit (16) on the surface of the measurement object (10). ..

At this time, when the measurement object (10) on which the pattern is projected is photographed by the image acquisition unit (18) to acquire the three-dimensional measurement data, the microprocessor (30) is three-dimensional through the image input unit (24). You will be asked to enter the measurement data (step S15).

In that state, the microprocessor (30) performs image processing on the two-dimensional image data including the optical marker and the two-dimensional image data not including the marker, and extracts the two-dimensional position of the marker (step). S16).

Next, the microprocessor (30) uses the markers extracted from the two-dimensional image data to coordinate the coordinates of the image acquisition unit (18) from the lens center of the camera to any three markers in the two-dimensional image data. By estimating the value and an arbitrary 3D coordinate value on the 3D measurement data located in a straight line, the 3D position of the marker is searched for (step S17).

On the other hand, the microprocessor (30) determines whether or not the register of the buffer (32) is free (step S18).

As a result of the above determination, if it is determined that the register of the buffer (32) is not free, the previous tertiary registered in the register of the buffer (32) with respect to the three-dimensional position of the marker searched for in the step S17. Markers based on the original measurement data (that is, data that overlaps the current 3D measurement data) are compared, and markers that are paired with each other are searched for (step S19).

When the marker included in the current three-dimensional measurement data is compared with the marker registered in the register of the buffer (32) to find the marker to be paired by the marker search process as described above, the microprocessor ( In 30), the matrix for movement is obtained from the position of the marker paired with the three-dimensional measurement data (step S20), and the position of the three-dimensional measurement data registered in the register of the buffer (32) is used as a reference. The coordinate system will be used to move the current measurement data (step S21).

As a result, the microprocessor (30) registers the newly searched marker from the current measurement data in the register of the buffer (32) and aligns the previous measurement data with a different marker (step S22).

Next, the microprocessor (30) determines whether or not the automatic alignment with respect to the acquired three-dimensional measurement data with respect to the measurement object (10) is completed (step S23).

As a result of the above determination, if it is determined that the automatic alignment with respect to the three-dimensional measurement data acquired from the measurement object (10) has not been completed, the control proceeds to the step S10 and the movement drive unit (20) ), While the movement mechanism (22) is operated, the process from step S10 to step S22 is repeatedly executed while moving the projection unit (16) and the image acquisition unit (18) to appropriate positions. It will be.

Next, a second embodiment of the present invention will be described in detail with reference to the accompanying drawings.

That is, FIG. 11 is a drawing showing a configuration for a three-dimensional measurement data automatic alignment device using an optical marker according to a second embodiment of the present invention, and is a second embodiment of the present invention with the first embodiment. The same reference numerals are given to the components having the same function and operation, and detailed description thereof will be omitted.

The three-dimensional measurement data automatic alignment device according to the second embodiment of the present invention includes a marker generator (70), a projection unit (16), an image acquisition unit (18), a movement drive unit (20), and a movement mechanism (22). , Video input unit (24), individual marker blinking control unit (74), projection control unit (28), microprocessor (76), buffer (32).

The marker generator (70) is for projecting a pattern recognizable by the image acquisition unit (18) on the surface of the measurement object (10) to be measured optically, which is the measurement object (10). A plurality of marker output units (72) are installed so that a plurality of optical markers are projected in mutually irregular survey directions over the entire surface pointing to 10).

Under the control of the individual marker blinking control unit (74), the marker generator (70) sequentially individually lights a plurality of marker output units (72) from the 1st to the Nth positions one by one to acquire the image. One different marker is projected for each image acquired from part (18).

The individual marker blinking control unit (74) sequentially connects a plurality of marker output units (72) provided in the marker generator (70) under the control of the microprocessor (76) in a predetermined order. , And individually blink control.

The microprocessor (76) inputs two-dimensional video data and three-dimensional measurement data taken at various angles from the video acquisition unit (18) through the video input unit (24), and analyzes them. Although the arithmetic processing for automatically aligning the measurement data taken at various angles into one coordinate system is performed, this was taken with all the markers turned off by the image acquisition unit (18). The image is set as the basic image, and a plurality of images taken with the markers lit sequentially from the 1st to the Nth are compared, and the two-dimensional position of each marker is searched and extracted.

Next, the microprocessor (76) compares the two-dimensional measurement data from which the marker position is extracted with the three-dimensional measurement data, and searches for a marker that is paired with the function for searching the three-dimensional position of the marker. , The function of obtaining the movement matrix according to the movement matrix and the function of moving the three-dimensional measurement data to the reference coordinate system can be advanced in the same manner as the function shown in the first embodiment of the present invention.

Subsequently, the operation according to the different embodiment of the present invention performed as described above will be described in detail with reference to the flowchart of FIG.

First, the microprocessor (76) drives the moving drive unit (20) and activates the moving mechanism (22) with the predetermined measurement object settled on the marker generator (70) side. The image acquisition unit (18) integrated with the projection unit (16) is moved to a position suitable for measurement of the measurement object (10) (step S30).

In that state, the microprocessor (76) turns off all the optical markers from the marker generator (70), and acquires the video data captured from the video acquisition unit (18) as a basic video. Will be done.

Next, the microprocessor (76) controls the individual marker blinking control unit (74), and is designated first among the plurality of marker output units (72) provided in the marker generator (70). The marker output unit (72) is turned on so that the first marker is projected on the surface of the measurement object (10) (step S31), and the image captured from the image acquisition unit (18) is the first. It will be acquired as the video data of (Stage S32).

Further, the microprocessor (76) controls the individual marker blinking control unit (74), and is the Nth among the plurality of marker output units (72) of the marker generator (70) in a predetermined order. That is, the second designated marker output unit is lit, the second optical marker is projected onto the measurement object (10) (step S33), and the image is photographed from the image acquisition unit (18). The video is acquired as the Nth video, that is, the second video data (stage S34).

In that state, the microprocessor (76) determines whether or not the marker included in the image captured in the step is the last marker among the plurality of markers specified in advance (step S35).

As a result of the above determination, if it is determined that the marker included in the image captured by the image acquisition unit (18) is not the last marker among the plurality of markers, the operations of the step S33 and the step S34 are repeated. When executed, the plurality of marker output units (72) of the marker generator (70) are sequentially and individually lit in the third, fourth, ... Nth order, and the optical marker is the object to be measured. It is projected on (10), and each time each marker is individually lit, the individual images taken from the image acquisition unit (18) are acquired as the 3rd, 4th, ... Nth images. To do.

On the other hand, when the microprocessor (76) determines that the marker included in the image acquired from the image acquisition unit (76) is the last marker, the marker generator (70) turns off the light. Then, the projection control unit (28) is controlled and the projection unit (16) is operated in a state where the optical marker is not projected, and a predetermined pattern for three-dimensional measurement is performed from the projection unit (16). A pattern (for example, a striped pattern of a plurality of sections having different intervals or a multi-striped pattern) is projected on the surface of the measurement object (10).

At this time, when the measurement object (10) on which the pattern pattern is projected is photographed by the image acquisition unit (18) and the three-dimensional measurement data is acquired, the microprocessor (76) is tertiary through the image input unit (24). The original measurement data will be input (step S36).

On the other hand, in the microprocessor (76), the 1st to Nth video data captured with the 1st to Nth markers turned off sequentially and individually is in a state where the markers are turned off. The two-dimensional position of the marker can be easily extracted by comparing it with the acquired basic image and by searching the passed region formed by the optical marker (step S37).

Next, in the microprocessor (76), the three-dimensional position of the marker can be detected by comparing the two-dimensional position of the marker extracted by comparing the two-dimensional images with the three-dimensional measurement data, and such a tertiary position can be detected. A pair of markers is searched from each video data taken adjacently from the original position, a moving matrix is obtained, and the 3D measurement data is moved to the reference coordinate system (step S38).

On the other hand, the microprocessor (76) determines whether or not the automatic alignment with respect to the acquired three-dimensional measurement data with respect to the measurement object (10) has been completed (step S39).

As a result of the determination, if it is determined that the automatic alignment with respect to the three-dimensional measurement data acquired from the measurement object (10) has not been completed, the control is re-progressed to the step S30, and the moving drive unit (20) re-progresses. The process from step S30 to step S38 is repeatedly executed while moving the projection unit (16) and the image acquisition unit (18) to appropriate positions while the movement mechanism (22) operates under driving. ..

Next, a third embodiment of the present invention will be described in detail with reference to the accompanying drawings.

That is, the configuration of the two-dimensional measurement data automatic alignment device according to the third embodiment of the present invention is the same as the component shown in FIG.

However, in the second embodiment of the present invention, when each of the N markers is individually projected by the marker generator (70), the N images should be individually captured, but the third embodiment. Then, because the N markers generated in the marker generator (70) have evolved into two, the number of times the image is taken by dividing them into groups and lighting them is log.₂It decreases to (N + 1).

The individual marker blinking control unit (74) divides a plurality of marker output units (72) installed in the marker generation unit (70) under the control of the microprocessor (76) into groups for bi-evolution. Control is performed to selectively light only the output unit of the marker corresponding to the group.

In the marker individual blinking control unit (74), for example, when the number of marker output units (72) installed in the marker generation unit (70) is 16, and a total of 16 markers can be generated, the two evolution of the markers. 16 markers are set in 4 groups in duplicate.

That is, the markers contained in the first group correspond to the 9th to 16th markers, the markers contained in the 2nd group correspond to the 5th to 8th markers, the 13th to 6th markers, and 3 The markers in the second group are the 3,4,7,8,11,12,15,16th markers, and the markers in the fourth group are even numbers (2,4,6,8,12,14). , 16) Corresponding to markers, such relationships are shown in Table 1 below.

<tables num="1"><img file="JP2005534026A_D0015.tif" /></tables> However, "0" indicates that the marker is extinguished, and "1" indicates that the marker is lit. As shown in Table 1 above, the first marker is always off, while the 16th marker is always on, and all markers have their own unique lighting values. Will be.

The microprocessor (76) controls the individual marker blinking control unit (76) so that the markers divided into groups already set can be sequentially lit, and the markers acquired through the image acquisition unit (18). The two-dimensional position of the marker is extracted by comparing the number of video data corresponding to the number of groups.

Here, the microprocessor (76) sequentially lights 16 markers for each group as shown in Table 1, and when acquiring the first to fourth video data, the tenth marker is set to the binary number 1001. 16 different unique IDs, i.e. two-dimensional position values, can be detected for 16 markers in the same way that the 13th marker is recognized as the binary number 1100. become. At this time, since the No. 1 marker is always kept extinguished, it cannot be actually used, and a total of 15 markers can be used in practice.

Therefore, if 10 video data are acquired, 1024 different unique IDs can be distinguished, and 1023 markers can be practically used.

Further, the microprocessor (76) has a function of comparing the two-dimensional measurement data from which the marker position is extracted and the three-dimensional measurement data to search for the three-dimensional position of the marker, and a function of searching for a pair of markers. The function of obtaining the movement matrix and the function of moving the three-dimensional measurement data to the reference coordinate system are performed in the same manner as the function shown in the first embodiment of the present invention.

Subsequently, the operation according to the third embodiment of the present invention performed as described above will be described in detail with reference to the flowchart of FIG.

First, as shown in Table 1 above, there are 16 marker output units (72) installed in the marker generator (70), a total of 16 markers are projected, and a total of 4 markers are projected by the image acquisition unit (18). Acquiring two two-dimensional video data will be described as an example.

First, with the predetermined measurement object (10) resting on the marker generator (70) side, the microprocessor (76) drives the moving drive unit (20) to operate the moving mechanism (22). As a result, the image acquisition unit (18) integrated with the projection unit (16) is moved to a position suitable for measurement of the measurement object (10) (step S40).

In that state, the microprocessor (76) controls the individual marker blinking control unit (74), and the marker included in the first group among the group-specific markers preset by the marker generation unit (70). The marker output unit (72) is turned on so that (9th to 16th markers) can be projected (step S41), and the first video data captured by the video acquisition unit (18) is input to the video input unit (stage S41). It will be acquired through 24) (Stage S42).

Next, the microprocessor (76) controls the individual marker blinking control unit (74), and among the group-specific markers preset by the marker generation unit (70), the Nth group, that is, 2 The marker output unit (72) is turned on so that the markers (5th to 8th, 13th to 16th) included in the third group can be projected (step S43), and the image acquisition unit (18) The Nth video data taken in, that is, the second video data will be acquired (stage S44).

At this time, the microprocessor (76) determines whether or not the group of markers included in the video data acquired in the step is the last among the previously set groups (step S45).

As a result of the determination, when the microprocessor (76) determines that the group of markers included in the video data acquired from the video acquisition unit (18) is not the last group, the process proceeds again to the step S43. By repeating the process up to step S44, the markers in the 3rd group (3,4,7,8,1,12,15,16th markers) are turned on to acquire the 3rd video data. Then, the markers of the 4th group (2nd, 4th, 6th, 8th, 10th, 12th, 14th, 16th markers) are turned on to acquire the 4th video data.

On the other hand, when it is determined that the group of markers included in the video data acquired from the video acquisition unit (18) is the last group based on the determination result of the stage S45, the marker generator (70) determines. The projection control unit (28) is controlled and the projection unit (16) is operated in a state where the light is turned off and the optical marker is not projected. (For example, a stripe pattern of a plurality of sections having different intervals or a multiple stripe pattern) is projected on the surface of the measurement object (10).

At this time, when the measurement object (10) on which the pattern is projected is photographed by the image acquisition unit (18) and the three-dimensional measurement data is acquired, the microprocessor (76) is tertiary through the image input unit (24). The original measurement data will be input (step S46).

Next, the microprocessor (76) compares the first to Nth images acquired by the image acquisition unit (18), that is, the two-evolution information from the first image to the fourth image data, and each marker. The unique ID of, that is, the two-dimensional position is extracted (step S47).

On the other hand, the microprocessor (76) detects the three-dimensional position of the marker by comparing the two-dimensional position of the marker extracted by comparing the two-evolution data of the two-dimensional video data with the three-dimensional measurement data, and the marker. Search for a pair of markers in each video data taken adjacently from the 3D position of, and find the movement matrix by that, and move the 3D measurement data to the reference coordinate system ( Stage S47).

On the other hand, the microprocessor (76) determines whether or not the automatic matrix for the three-dimensional measurement data acquired for the measurement object (10) can be completed (step S49).

As a result of the determination, when it is determined that the automatic alignment with respect to the three-dimensional measurement data acquired from the measurement object (10) has not been completed, the control re-progresses to the step S40 and the movement drive unit (20). ), While the movement mechanism (22) is operating, the process from step S40 to step S48 is repeatedly executed while moving the projection unit (16) and the image acquisition unit (18) to appropriate positions. Will be.

Hereinafter, the fourth embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The configuration of the three-dimensional measurement data automatic alignment device according to the fourth embodiment of the present invention is as shown in FIG. 14, and as shown in the drawing, the measurement object (10) and the projection unit (16). , Image acquisition unit (18), movement drive unit (20), movement mechanism (22), image input unit (24), projection control unit (28), buffer (32), marker generator (80), and individual markers It is composed of a blinking control unit (84) and a microprocessor (86).

Here, the same reference numerals are given to the components that perform the same functions and operations as the configurations of the first embodiment shown in FIG. Moreover, in order to avoid duplicate description, the specific description about each component is omitted.

The marker generator (80) is for projecting a pattern that can be recognized by the image acquisition unit (18) on the surface of the measurement object (10) to be measured optically, and this is for projecting a pattern that can be recognized by the image acquisition unit (18). A plurality of marker output units (82) are installed so that a plurality of optical markers can be projected in mutually irregular survey directions over the entire surface pointing to 10).

The marker generator (80) is configured to selectively blink and drive a plurality of marker output units (82) under the control of the individual marker blinking control unit (84).

The individual marker blinking control unit (84) individually blinks and controls a plurality of marker output units (82) provided in the marker generator (80) under the control of the microprocessor (86).

The image acquisition is performed by driving the dedicated software program for analyzing the measurement data acquired from the measurement object (10) and automatically aligning the measurement data in one coordinate system. Two-dimensional video data and three-dimensional measurement data taken from various angles are input from the unit (18) through the video input unit (24), and the measurement data taken from various angles is analyzed. Although the arithmetic processing for automatically aligning to one coordinate system is performed, the specific operation process for this is the same as the operation process of the microprocessor in the first embodiment described above.

However, the microprocessor (86) according to the fourth embodiment of the present invention acquires the two-dimensional video data and the three-dimensional measurement data for one region of the measurement object (10), and after performing arithmetic processing on the two-dimensional video data and the three-dimensional measurement data. When the marker generator (80) is turned on to acquire 2D video data and 3D measurement data for different regions, the marker projected on the region where the video and measurement data have already been acquired has a predetermined period (for example, While blinking repeatedly (about 0.5 seconds), the marker projected on the remaining area controls the individual marker blinking control unit (84) so that the lighting state can be maintained.

On the contrary, the marker projected on the area where the image and the measurement data have already been acquired can maintain the lighting state, while the marker projected on the remaining area can be repeatedly blinked at a predetermined cycle. ..

In this way, by making the blinking state of the marker projected on the area where the image and measurement data have already been acquired and the marker projected on the remaining area different, the tester in charge of measurement and the image and measurement data It is easy to visually confirm the area where the data was acquired and the remaining area, which makes it possible to improve the convenience of measurement.

Next, a fifth embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The configuration of the three-dimensional measurement data automatic alignment device according to the fifth embodiment of the present invention is as shown in FIG. 15, and as shown in the drawing, the measurement object (10), the projection unit (16), and the image Acquisition unit (18), movement drive unit (20), movement mechanism (22), video input unit (24), projection control unit (28), buffer (32), marker generator (90), and individual marker blinking / It includes a hue control unit (94) and a microprocessor (96).

Here, the same reference numerals are given to the components having the same functions and operations as those of the configuration of the first embodiment shown in FIG. 3, and specific description thereof will be omitted in order to avoid duplicate descriptions.

The marker generator (90) is for projecting a pattern recognizable by the image acquisition unit (18) on the surface of the measurement object (10) to be measured optically, which is the measurement object (10). A plurality of marker output units (92) are installed so that a plurality of optical markers can be projected in mutually irregular survey directions over the entire surface pointing to 10).

Here, in the marker generator, each marker output unit (92) can selectively convert and investigate at least two types of light of different colors by controlling the individual marker blinking / hue control unit (94). It is configured as follows. For example, each marker output unit (92) is provided with at least two different color light sources, and by selectively lighting these light sources, it is possible to generate light having different hues.

The individual marker blinking / hue control unit (94) controls the blinking and individual hues of a plurality of marker output units (92) provided in the marker generator (90) under the control of the microprocessor (96). Will be done.

The microprocessor (96) acquires the video while driving a dedicated software program for analyzing the measurement data acquired from the measurement object (10) and automatically aligning the measurement data in one coordinate system. Although the arithmetic processing for automatically aligning the two-dimensional video data and the three-dimensional measurement data taken from the part (18) at various angles in one coordinate system is performed, the specific operation process for this is described in the above-mentioned first. It is the same as the operation process of the microprocessor in 1 Example.

However, the microprocessor (96) according to the fifth embodiment of the present invention acquires the two-dimensional video data and the three-dimensional measurement data for one region of the measurement object (10), and after performing arithmetic processing on the two-dimensional video data and the three-dimensional measurement data. When the marker generator (90) is turned on to acquire the two-dimensional image data and the three-dimensional measurement data for different areas, the marker projected on the area where the image and the measurement data have already been acquired and the remaining area are projected. The individual marker blinking / hue control unit (94) is controlled so that the colors of the marked markers are projected differently.

Therefore, by making the markers that have already acquired the image and measurement data and controlled in the area and the markers projected on the remaining area different colors, the tester in charge of measurement has acquired the image and measurement data and the remaining area. It is easy to check the area of the above with the naked eye, and through this, the convenience of measurement can be achieved.

Next, the sixth embodiment of the present invention will be described in detail with reference to the attached drawings. The configuration of the three-dimensional measurement data automatic alignment device according to the sixth embodiment of the present invention is as shown in FIG. 16, and as shown in the drawing, the measurement object (10), the marker generator (12), and the projection. Unit (16), image acquisition unit (18), image input unit (24), marker blinking control unit (26), projection control unit (28), buffer (32), rotary table (100), rotation drive unit (102) ), Rotation mechanism (104), and microprocessor (106).

Here, the same reference numerals are given to the components that perform the same functions and operations as the configurations of the first embodiment shown in FIG. Further, in order to avoid duplicate description, specific description of these components will be omitted.

The rotary table (100) has a structure capable of rotating with the object to be measured (10) placed on the upper plate thereof, and at the same time, a plurality of marker generators (1) on the outer peripheral portion of the upper plate. 12) is fixed and installed so that it can rotate with the object to be measured (10).

The rotation drive unit (102) is driven by the drive control of the microprocessor (106) to rotate the rotary table (100) by a target angle, and the rotation mechanism (104) is a rotation drive unit (102). It has a structure to rotate the rotary table (100) by the target angle by receiving the power driven by.

At this time, in the sixth embodiment of the present invention, the case where the rotary drive unit (102) is used to rotate the rotary table (100) by electric drive as described above will be described as an example. However, it is also possible to manually operate the rotation mechanism (104) so that the operator can move it arbitrarily.

Further, not limited to the rotary table (100), any device may be applied as long as the marker generator and the measurement target can be rotated together in a fixed state.

The microprocessor (106) analyzes the measurement data acquired from the object to be measured (10) and drives a dedicated software program for automatically aligning the measurement data in one coordinate system to acquire the image. Two-dimensional video data and three-dimensional measurement data taken at various angles are input from the unit (18) through the video input unit (24), analyzed, and the measurement data taken at various angles is input. Arithmetic processing for automatically aligning to one coordinate system is performed, and the specific operation process for this is the same as the operation process of the microprocessor in the first embodiment described above.

However, the microprocessor (106) according to the sixth embodiment of the present invention acquires the two-dimensional video data and the three-dimensional measurement data relating to one region of the measurement object (10), and after performing arithmetic processing on the two-dimensional video data and the three-dimensional measurement data. , When trying to acquire 2D video data and 3D measurement data related to other areas, the rotation drive unit (102) is controlled so as to rotate the rotation table (100).

The operation process of the three-dimensional measurement data automatic alignment device according to the sixth embodiment of the present invention configured as described above will be described in detail with reference to the flowcharts of FIGS. 17a and 17b attached.

First, with the object to be measured (10) resting on the upper plate of the rotary table (100), the microprocessor (106) drives the rotary drive unit (102) to operate the rotary mechanism (104). As a result, the rotary table (100) is rotated by a predetermined angle, and the object to be measured (10) is rotated to a position suitable for measurement (step S50).

In such a state, the microprocessor (106) controls the marker blinking control unit (26), lights a plurality of marker output units (14) provided in the marker generator (12), and measures the measurement target. Allow multiple markers to be projected irregularly on the surface of the object (10) (step S51).

With the optical marker from the marker generator (12) projected onto the measurement object (10), the image acquisition unit (18) photographs a specific area of the measurement object (10) and performs optical observation. When the two-dimensional image including the expression marker is acquired, the microprocessor (106) inputs the two-dimensional image data acquired from the image acquisition unit (18) through the image input unit (24). You will receive it (stage S52).

Next, the microprocessor (106) controls the marker blinking control unit (26) to turn off the marker generator (12) so that the optical marker is not projected onto the measurement object (10) ( In step S53), in that state, the same area of the measurement object (10) is photographed by the image acquisition unit (18), and when a two-dimensional image that does not include an optical marker is acquired, the two-dimensional image data is input. It will be entered through part (24) (step S54).

Further, the microprocessor (106) controls the projection control unit (28) and operates the projection unit (16) in a state where the marker generator (12) is turned off and the optical marker is not projected. , A predetermined pattern pattern for three-dimensional measurement (for example, a stripe pattern of a plurality of sections having different intervals or a multiple stripe pattern) is projected from the projection unit (16) on the surface of the measurement object (10).

At this time, when the measurement object (10) on which the pattern is projected is photographed by the image acquisition unit (18) and the three-dimensional measurement data is acquired, the microprocessor (106) is three-dimensional through the image input unit (24). Measurement data will be input (step S55).

In such a state, the microprocessor (106) performs image processing on the two-dimensional image data including the optical marker and the two-dimensional image data not including the marker, and extracts the two-dimensional position of the marker. Becomes (stage S56).

Next, the microprocessor (106) uses markers extracted from the two-dimensional video data, and the coordinate values for any three markers in the two-dimensional video data from the lens center of the camera of the video acquisition unit (18). Then, by estimating an arbitrary three-dimensional coordinate value on the three-dimensional measurement data located in a straight line, the three-dimensional position of the marker is searched for (step S57).

On the other hand, the microprocessor (106) determines whether or not the register of the buffer (32) is free (step S58).

As a result of the above determination, if it is determined that the register of the buffer (32) is not free, the microprocessor (106) of the buffer (32) with respect to the three-dimensional position of the marker found in the above step S57. The markers from the previous 3D measurement data (that is, the data that overlaps the 3D measurement data) registered in the register are compared, and the markers that form a mutual pair are searched for (step S59).

By the marker search process as described above, it becomes possible to search for a pair of markers by comparing the optical markers contained in the current CMM measurement data with the markers registered in the register of the buffer (32). , The microprocessor (106) obtains the position conversion matrix for movement from the position of the paired marker in each of the three-dimensional measurement data (step S60), and is registered in the register of the buffer (32). The position of the three-dimensional measurement data is used as the reference coordinate system, and the current measurement data is moved (step S61).

As a result, the microprocessor (106) registers the newly searched marker from the current measurement data in the register of the buffer (32) and aligns the previous measurement data with a different marker (step S62).

Next, the microprocessor (106) determines whether or not the automatic alignment in the acquired three-dimensional measurement data has been completed for the measurement object (10) (step S63).

As a result of the determination, if it is determined that the automatic alignment in the three-dimensional measurement data acquired from the measurement object (10) has not been completed, the microprocessor (106) returns to the step S50 and the rotation drive unit By operating the rotation mechanism (104) by (102) and rotating the rotation table (100) by a predetermined angle, the projection unit (16) and the image are acquired with respect to the other measurement areas of the measurement object (10). To be able to acquire 2D video and 3D measurement data through part (18).

After that, the control unit (106) repeatedly executes the process from the above steps S50 to S62.

As described above, the sixth embodiment of the present invention is configured so that the object to be measured moves, and the first embodiment of the present invention configured so that the projection unit and the image acquisition unit move. It is suitable for acquiring and aligning three-dimensional measurement data from a measurement object having a relatively small size.

At this time, since the marker generator and the object to be measured must not move relative to each other until the measurement is completed, the marker generator is fixed to the rotary table to prevent the relative movement.

On the other hand, the alignment method using the reference coordinate system used in the above-described embodiment of the present invention is the three-dimensional measurement data of the previous measurement area stored in the register of the buffer when the three-dimensional measurement data is aligned. Since the position of is used as the reference coordinate system and the position of the three-dimensional measurement data of the newly measured area is moved and attached, the larger the object to be measured and the larger the number of measurement areas, the more precise the image acquisition unit. A minute error due to the degree is amplified, and the error value may become considerably large.

For example, (a) and (b) in FIG. 18 show data obtained by measuring adjacent different measurement areas where boundary sites overlap with respect to the same measurement object, and the portion shown by the dotted line is the measurement object. Assuming the actual data of the object, the data obtained through the image acquisition unit will have an error value like the part displayed by the solid line.

As a result, one of the measurement data in (a) and (b) of FIG. 18 is used as the reference coordinate system, and the other data is moved to this reference coordinate system and attached. Since the error value between the measurement data in a) and (b) is added, the measurement data in which the error value becomes large as shown by the solid line shown in (c) of FIG. 18 can be obtained.

That is, as the number of regions to be measured increases, the risk of error increases.

In order to solve the above problems, in the seventh and eighth embodiments of the present invention, a method of aligning the positions of the three-dimensional measurement data in an absolute coordinate system other than the reference coordinate system is presented.

Here, unlike the reference coordinate system, the absolute coordinate system uses three-dimensional position data with respect to the entire measurement area of the measurement object, and the error value of the overall measurement data is an image with respect to the entire measurement area of the measurement object. The error range of the video acquisition device that acquires the data will not be exceeded.

For example, assuming that (a) and (b) of FIG. 19 show data obtained by measuring adjacent different measurement regions where boundary sites overlap with respect to the same measurement object, FIG. 19 (c) When the measurement data of (a) and (b) of FIG. 19 is moved and attached to the absolute coordinate system as shown in FIG. 19, the measurement data is an error of the absolute coordinate system as shown in (d) of FIG. Since the combined value of the range and the error range of the measuring device is not exceeded, it is possible to prevent the error from being amplified by the precision of the image acquisition unit as described above.

First, a seventh embodiment of the present invention will be described in detail with reference to the attached drawings.

The configuration of the three-dimensional measurement data automatic alignment device according to the seventh embodiment of the present invention is as shown in FIG. 20, and as shown in the drawing, the measurement object (10), the marker generator (12), and the projection. Unit (16), image acquisition unit (18), movement drive unit (20), movement mechanism (22), marker blinking control unit (26), projection control unit (28), buffer (32), large area image acquisition unit It is composed of (110), a video input unit (112), a second moving drive unit (114), a second moving mechanism (116), a microprocessor (118), and a reference object (120).

Here, the same reference numerals are given to the components that perform the same functions and operations as the configurations of the first embodiment shown in FIG. Moreover, in order to avoid duplicate description, the specific description for each component is omitted.

The large-area image acquisition unit (110) is an image sensor capable of receiving images like a CCD camera or a CMOS camera, and is applied from the marker generator (12) to the surface of the object to be measured (10). When the marker is projected by an optical method, the image obtained by the marker is captured and acquired, but it is provided separately from the image acquisition unit (10) and the image of the entire measurement area of the measurement object (10) is captured. You will get it by shooting.

Here, it is desirable that the large-area image acquisition unit (110) employs an image sensor having a relatively higher accuracy than the image acquisition unit (10) that acquires images in a subdivided measurement area.

The video input unit (112) is for inputting video data acquired from the video acquisition unit (18) and the large area video acquisition unit (110).

The second moving drive unit (114) is driven by the drive control of the microprocessor (118) to move the image acquisition unit (110) relative to the measurement object (10). The second movement mechanism (116) receives power from the drive of the second drive unit (114) and moves the large-area image acquisition unit (110) in a certain direction with respect to the measurement object (10). It has a structure.

In the seventh embodiment of the present invention, the second moving drive unit (114) is applied so that the large area image acquisition unit (110) can be moved by electrical drive, but the second movement It is also possible to manually operate the mechanism (116) so that the operator can move it arbitrarily.

The microprocessor (118) differs from the large area image acquisition unit (110) by two or more in a state where a plurality of optical markers are projected from the marker generator (12) onto the surface of the measurement object (10). The three-dimensional position of each marker in the entire area of the measurement target is obtained from the video data of the measurement object (10) and the reference object (120) taken in the direction, and the three-dimensional position of each obtained marker is absolute. Performs arithmetic processing to be set in the coordinate system.

At the same time, the microprocessor (118) inputs two-dimensional video data and three-dimensional measurement data taken at various angles from the video acquisition unit (18) through the video input unit (112) and analyzes them. , The measurement object (10) is subjected to arithmetic processing for aligning various measurement data obtained by photographing the subdivided measurement areas in the above absolute coordinate system.

The reference object (120) is arranged adjacent to the measurement object (10) as a predetermined object in which dimensional information for its size has already been input to the microprocessor (118), and a large-area image is acquired. The image is acquired together with the measurement object (10) through the part (110).

The following will explain in detail the operation process of the three-dimensional measurement data automatic alignment device according to the seventh embodiment of the present invention configured as described above with reference to the flowcharts of the attached FIGS. 21a and 21b.

First, the measurement object (10) is placed on the marker generator (12), and the reference object (120) is placed at a predetermined point adjacent to the measurement object (10). 2 By driving the moving drive unit (114) and operating the movement mechanism (116), the large area image acquisition unit (110) is moved to a position suitable for measurement of the measurement object (10).

Next, the microprocessor (118) controls the marker blinking control unit (26), lights up a plurality of marker output units (14) provided in the marker generator (12), and controls the measurement object (10). Allow multiple markers to be projected irregularly on the surface (step S70).

With the optical marker from the marker generator (12) projected onto the measurement object (10), the measurement object (10) including the reference object (120) is passed through the large-area image acquisition unit (110). ) Is photographed and a two-dimensional image including an optical marker is acquired, the microprocessor (118) passes through the image input unit (112) to acquire the large area image acquisition unit (110). The two-dimensional video data obtained from) will be input (step S71).

As a reference, FIG. 22 illustrates an example of an image including the entire measurement target area (10) and the reference object (120) of the measurement object (10) acquired by the large area image acquisition unit (110). In the same drawing, the reference code "RM" represents an optical marker projected on the surface of the object to be measured (10), and the reference code "BI" is the image acquired by the large area image acquisition unit (110). Represents. Next, the microprocessor (118) drives the second moving drive unit (114) and activates the movement mechanism (116), so that the large area image acquisition unit (110) is used to measure the object to be measured (10). Move to another suitable location (step S72).

Then, the microprocessor (118) controls the large-area image acquisition unit (110) at the above-mentioned other moved points, and captures the entire measurement target area of the measurement object (10) including the reference object. , A two-dimensional image including an optical marker is acquired from a direction different from that of the step S71, and this is input through the video input unit (112) (step S73).

Next, the microprocessor (118) controls the marker blinking control unit (26) and turns off the marker generator (12) to prevent the optical marker from being projected onto the object to be measured (step S74). ).

The microprocessor (118) combines the two-dimensional images of the entire area of the measurement target in different directions acquired through the large area image acquisition unit (110), and at the same time, the reference object (120) included in this image. The three-dimensional position of each marker included in the entire measurement target area is calculated by performing the calculation according to the dimensions already known in (Step S75). Then, the microprocessor (118) registers the three-dimensional position of each of the above-calculated markers in the register of the buffer (32) (step S76).

Next, the microprocessor (118) drives the moving drive unit (20) and activates the moving mechanism (22) to measure the image acquisition unit (18) integrated with the projection unit (16). The object (10) will be moved to a position suitable for measurement (step S77).

In that state, the microprocessor (118) controls the marker blinking control unit (26), lights a plurality of marker output units (14) provided in the marker generator (12), and causes the measurement object (10). Allow multiple markers to be projected irregularly on the surface of the (step S78).

With the optical marker from the marker generator (12) projected onto the measurement object (10), the image acquisition unit (18) subdivides the entire measurement target area of the measurement object (10). When the area (see "NI" in FIG. 23) is photographed and a two-dimensional image including an optical marker is acquired, the microprocessor (118) acquires the image through the image input unit (112). The two-dimensional video data acquired from the part (18) will be input (step S78).

Next, the microprocessor (118) controls the marker blinking control unit (26) so that the marker generator (12) is turned off and the optical marker is not projected onto the measurement object (10) (step S80). ), In that state, the same area of the measurement object (10) is photographed from the image acquisition unit (18), and when a two-dimensional image that does not include an optical marker is acquired, the two-dimensional image data is input to the image input unit (image input unit (10). It will be entered through 112) (step S81).

Further, the microprocessor (118) controls the projection control unit (28) and operates the projection unit (16) in a state where the marker generator (12) is turned off and the optical marker is not projected. Then, a predetermined pattern pattern for three-dimensional measurement (for example, a stripe pattern of a plurality of sections having different intervals or a multiple stripe pattern) is projected from the projection unit (16) on the surface of the measurement object (10).

At this time, when the measurement object (10) on which the pattern is projected is photographed by the image acquisition unit (18) and the three-dimensional measurement data is acquired, the microprocessor (118) performs the three-dimensional measurement through the image input unit (112). You will be asked to enter the data (step S82).

In such a state, the microprocessor (118) performs image processing on the two-dimensional image data including the optical marker and the two-dimensional image data not including the marker, and extracts the two-dimensional position of the marker. Becomes (stage S83).

At the same time, the microprocessor (118) uses markers extracted from the two-dimensional video data, and the coordinate values for any three markers in the two-dimensional video data from the lens center of the camera of the video acquisition unit (18). , By estimating an arbitrary three-dimensional coordinate value on the three-dimensional measurement data located in a straight line, the three-dimensional position of the marker is searched for (step S84).

The microprocessor (118) then compares the three-dimensional position of the marker found in step S84 above with the three-dimensional position of the marker stored in the register of buffer (32) in step S76 above and pairs with each other. That is, search for markers that have the same three-dimensional position (step S85).

By the marker search process as described above, it becomes possible to search for a pair of markers by comparing the optical marker contained in the current CMM measurement data with the marker registered in the register of the buffer (32). , The microprocessor (118) obtains a position conversion matrix for movement from the position of the paired marker in each of the three-dimensional measurement data (step S86). While moving the current measurement data to this position transformation matrix, the three-dimensional position of the marker registered in the register of the buffer (32) is set in the absolute coordinate system and aligned with this absolute coordinate system (step S87).

Next, the microprocessor (118) determines whether or not the automatic alignment in the acquired three-dimensional measurement data has been completed with respect to the measurement object (10), that is, in the measurement target area of the measurement object (10). , Determine if all the three-dimensional data of the subdivided regions are aligned (step S88).

As a result of the above determination, if it is determined that the automatic alignment in the three-dimensional measurement data acquired from the measurement object (10) has not been completed, the microprocessor (106) returns to the step S50 and the moving drive unit ( By driving 20) and activating the movement mechanism (22), the projection unit (16) and the image acquisition unit (18) are moved to a position suitable for measuring an unmeasured region, while moving the above. The process from stage S77 to stage S88 will be repeated.

In the seventh embodiment, an example is taken in which a large-area video acquisition unit that acquires an image of the entire measurement target area and an image acquisition unit that acquires an image of a subdivided measurement area are separately provided. As described above, it is also possible to use one of the image acquisition units to acquire all the images of the entire measurement target area and the subdivided measurement area.

Next, the eighth embodiment of the present invention will be described in detail with reference to the attached drawings.

The configuration of the three-dimensional measurement data automatic alignment device according to the eighth embodiment of the present invention is as shown in FIG. 24, and as shown in the drawing, the measurement object (10), the marker generator (12), and the projection. Unit (16), image acquisition unit (18), movement drive unit (20), movement mechanism (22), marker blinking control unit (26), projection control unit (28), buffer (32), one set or more It includes a large-area video acquisition unit (130, 132), a video input unit (134), and a microprocessor (136).

Here, the same reference numerals are given to the components that perform the same functions and operations as the configurations of the first embodiment shown in FIG. Then, in order to avoid duplicate description, a specific description thereof will be omitted.

The set of large-area video acquisition units (130, 132) are video sensors that can receive video, such as CCD cameras and CMOS cameras, and at the same time, they are separated and fixed by the mutual set distance. This is a method of acquiring an image by shooting the same measurement target area at different angles, so-called Stereo Vision.

Here, it is desirable that each large-area image acquisition unit (130, 132) employs an image sensor having a relatively higher accuracy than the image acquisition unit (10) that acquires images in a subdivided measurement area.

The video input unit (134) is for inputting video data acquired from the video acquisition unit (18) and the large area video acquisition unit (130, 132).

The microprocessor (136) captures images in different directions from the large-area image acquisition unit (130, 132) in a state where a plurality of optical markers are projected from the marker generator (12) onto the surface of the measurement object (10). The three-dimensional position with respect to each marker in the entire region of the measurement target is obtained from the video data of the measurement target (10), and the markers obtained again from the video obtained by the large region video acquisition unit (130, 132). Use the three-dimensional position of for the absolute coordinates.

At the same time, the microprocessor (136) inputs two-dimensional video data and three-dimensional measurement data taken at various angles from the video acquisition unit (18) through the video input unit (134) and analyzes them. Then, arithmetic processing is performed to align the measurement data obtained by photographing the measurement area subdivided with respect to the measurement object (10) in the above absolute coordinate system.

This method is the same process as registering different objects in the first embodiment above, only the absolute coordinates already obtained by the target object.

The operation process of the three-dimensional measurement data automatic alignment device according to the eighth embodiment of the present invention configured as described above will be described in detail with reference to the flowcharts of FIGS. 25a and 25b attached.

First, the microprocessor (136) controls the marker blinking control unit (26) in a state where a predetermined measurement object (10) is arranged on the marker generator (12), and is provided in the marker generator (12). A plurality of marker output units (14) are turned on so that a plurality of markers are irregularly projected on the surface of the measurement object (10) (step S90).

In a state where the optical marker from the marker generator (12) is projected onto the measurement object (10), the entire measurement target area of the measurement object (10) is measured through the large-area image acquisition unit (130, 132). When two-dimensional images including optical markers are superimposed and acquired by shooting from different directions, the microprocessor (118) passes through the image input unit (134) to acquire the large-area image acquisition unit (130, The two-dimensional video data acquired from 132) will be input respectively (step S91).

As a reference, FIG. 26 illustrates an example of the image of the entire measurement target area of the measurement object (10) acquired by the large area image acquisition unit (130, 132). Then, in the drawing, the reference code "RM" represents an optical marker projected on the surface of the object to be measured (10), and the reference code "BI" is acquired by the large area image acquisition unit (130, 132). Represents an image.

Next, the microprocessor (136) controls the marker blinking control unit (26) and turns off the marker generator (12) to prevent the optical marker from being projected onto the object to be measured (step S92).

The microprocessor (136) performs calculations based on the two-dimensional image information about the entire area of the measurement target in different directions acquired through the large area image acquisition unit (130, 132), and each of them included in the total measurement target area. Calculate the three-dimensional position of the marker (step S93).

That is, in the above stage S93, the distance between the large area image acquisition units (130, 132) is fixed and invariant, and the distance information is stored in the microprocessor (136), so that a set of sets. The three-dimensional position between each marker is obtained by calculating the relationship between the position of the large area image acquisition unit (130, 132) and the position of each marker projected on the measurement object (10) by the triangulation method. Be done.

Then, the microprocessor (136) registers the three-dimensional position of each of the calculated markers in the register of the buffer (32) (step S94).

Next, the microprocessor (136) drives the moving drive unit (20) and activates the movement mechanism (22) to measure the image acquisition unit (18) integrated with the projection unit (16). The object (10) will be moved to a position suitable for measurement (step S95).

In that state, the microprocessor (136) controls the marker blinking control unit (26), lights a plurality of marker output units (14) provided in the marker generator (12), and causes the measurement object (10). Allow multiple markers to be projected irregularly on the surface of the (step S96).

With the optical marker from the marker generator (12) projected onto the measurement object (10), the image acquisition unit (18) subdivides the entire measurement target area of the measurement object (10). When the area (see "NI" in FIG. 27) is photographed and a two-dimensional image including an optical marker is acquired, the microprocessor (136) acquires the image through the image input unit (134). The two-dimensional video data acquired from the part (18) will be input (step S97).

Next, the microprocessor (136) controls the marker blinking control unit (26), turns off the marker generator (12), and prevents the optical marker from being projected onto the measurement object (10) (step S98). In that state, when the same area of the measurement object (10) is photographed by the image acquisition unit (18) and a two-dimensional image that does not include the optical marker is acquired, the two-dimensional image data is input to the image input unit (24). ) Will be entered (step S99).

Further, the microprocessor (136) controls the projection control unit (28) and operates the projection unit (16) in a state where the marker generator (12) is turned off and the optical marker is not projected. , A predetermined pattern pattern for three-dimensional measurement (for example, a stripe pattern of a plurality of sections having different intervals or a multiple stripe pattern) is projected from the projection unit (16) on the surface of the measurement object (10).

At this time, when the measurement object (10) on which the pattern is projected is photographed by the image acquisition unit (18) and the three-dimensional measurement data is acquired, the microprocessor (136) is three-dimensional through the image input unit (112). Measurement data will be input (step S100).

In such a state, the microprocessor (136) performs image processing on the two-dimensional image data including the optical marker and the two-dimensional image data not including the marker, and extracts the two-dimensional position of the marker. Becomes (stage S101).

At the same time, the microprocessor (136) uses markers extracted from the two-dimensional video data, and the coordinate values for any three markers in the two-dimensional video data from the lens center of the camera of the video acquisition unit (18). , By estimating an arbitrary three-dimensional coordinate value on the three-dimensional measurement data located in a straight line, the three-dimensional position of the marker is searched for (step S102).

Next, the microprocessor (136) compares the three-dimensional position of the marker found in the above step S102 with the three-dimensional position of the marker stored in the register of the buffer (32) in the above step S94, and sets each other. That is, search for markers that have the same three-dimensional position (step S103).

By the marker search process as described above, it becomes possible to search for a pair of markers by comparing the optical marker contained in the current CMM measurement data with the marker registered in the register of the buffer (32). , The microprocessor (136) obtains the position conversion matrix for movement from the position of the paired marker in each of the three-dimensional measurement data (step S104). While moving the current measurement data to this position transformation matrix, the three-dimensional position of the marker registered in the register of the buffer (32) is set in the absolute coordinate system and aligned with this absolute coordinate system (step S105).

Next, the microprocessor (136) determines whether or not the automatic alignment in the acquired three-dimensional measurement data has been completed for the measurement object (10) (step S106).

As a result of the above determination, if it is determined that the automatic alignment in the three-dimensional measurement data acquired from the measurement object (10) has not been completed, the microprocessor (136) returns to the step S95 and the mobile drive unit (20) By driving the movement mechanism (22) and moving the projection unit (16) and the image acquisition unit (18) to a position suitable for measuring an unmeasured region, the above steps. The process from S95 to step S106 will be repeated.

In the eighth embodiment, a set of a large-area image acquisition unit that acquires an image of the entire measurement target area, an image acquisition unit that acquires an image of a subdivided measurement area, and a marker generator are separately configured. Although the case has been described with an example, it is also possible to integrally configure a set of a large-area image acquisition unit and a marker generator as a modified example. However, with such a configuration, it becomes unnecessary to set the position value of a set of large-area image acquisition units according to the area where the optical marker generated by the marker generator is projected, and it is easier to use. it can.

As another different modification of the eighth embodiment, it is possible to integrally configure a set of a large-area video acquisition unit and a video acquisition unit, but in such a configuration, the absolute coordinates are The obtained area may be slightly smaller and the accuracy may be slightly reduced. On the other hand, when acquiring a plurality of images of the subdivided measurement area, the subdivision is performed without superimposing the boundary portion for each image. It is possible to reduce the number of times the image of the measured measurement area is acquired, that is, the number of times of scanning.

For reference, the principle of the eighth embodiment of the present invention will be additionally described as follows.

The geometric model of the large-area image acquisition unit (130, 132) provided in the eighth embodiment of the present invention has a structure in which two cameras are looking at one object. At the same time, various forms are shown depending on the field of application, but Fig. 27 shows the structure in which two cameras are arranged in parallel.

In Figure 27 above, the variables are defined as follows:

"X: Coordinates of the position to be obtained, b: Base line distance f: Camera's focal length A, B: Image plane acquired by each camera (image plane) X₁, X_r : Coordinates to be obtained from the origin of each image plane Coordinates of the image image with respect to the image of X P, Q: Lens center of each camera In Fig. 27 above, it is to be obtained from the stereo image. The method of obtaining the coordinates (X) of the position is as shown in the following mathematical formulas 15 and 16.

<maths num="15"><img file="JP2005534026A_D0016.tif" /></maths>

<maths num="16"><img file="JP2005534026A_D0017.tif" /></maths>Next, the ninth embodiment of the present invention will be described with reference to the attached drawings.

In the ninth embodiment of the present invention, by arranging a plurality of projection units, image acquisition units, and marker generators around the measurement target, a two-dimensional image and a two-dimensional image in the entire area of the measurement target of the measurement target can be obtained. It is not necessary to move the projection unit and the image acquisition unit to acquire the 3D measurement data, and it is possible to acquire the 2D image and 3D measurement data with a single scan, which makes the work easy and the required time. The configuration for a three-dimensional measurement data automatic alignment device using an optical marker that can be shortened is presented.

FIG. 28 is a drawing showing a configuration for a three-dimensional measurement data automatic alignment device using an optical marker according to a ninth embodiment of the present invention, and as can be seen by referring to the drawing, it is based on the ninth embodiment of the present invention. The CMM automatic alignment device includes N marker generators (142), M projection units (146), L image acquisition units (148), image input units (150), and projection control units (150). It consists of a marker blinking control unit (154), a microprocessor (156), and a buffer (158).

The N marker generators (142) project a pattern that can be recognized by the image acquisition unit (148) on the surface of the measurement object (10) to be measured optically, and the measurement target is projected on the surface. A plurality of marker output units (144) are installed so that a plurality of optical markers are simultaneously projected with irregular survey directions over the entire surface pointing to the object (10).

As shown in FIG. 28, the N marker generators (142) direct the measurement object (10) around the measurement object (10) at regular intervals, while each marker generator (142) directs the measurement object (10). The area of the optical marker projected from is arranged so as to cover the entire area of the measurement object of the measurement object (10).

The M projection units (146) project a predetermined pattern and a laser stripe on the surface of the measurement object (10) so that three-dimensional data can be acquired. This involves projecting space-encoded light onto the surface of the object to be measured (10) using a projection device such as an LCD projector, or projecting laser light onto the surface of the object to be measured (10). It is made possible to acquire as three-dimensional data through the acquisition unit (148).

As shown in FIG. 28, the M projection units (146) direct the measurement object (10) around the measurement object (10) at regular intervals, while projecting from each projection unit (146). The spatially coded region of light to be measured is arranged so as to cover the entire region of the object to be measured (10) to be measured.

The L image acquisition units (148) consist of an image sensor capable of receiving images, such as a CCD camera or a CMOS camera, from the marker generator (142) to the surface of the object to be measured (10). When a marker is projected by an optical method, each image is captured and acquired.

Each of the L image acquisition units (148) is not installed as a separate camera for each projection unit (146), but can be integrated with each projection unit (146). desirable. Then, as shown in FIG. 28, the L image acquisition units (148) point the measurement object (10) at regular intervals around the measurement object (10), while each image acquisition unit (148) The imaging area of 148) is arranged so as to cover the entire area to be measured by the object to be measured (10).

The video input unit (150) is for inputting video data acquired from each of the L video acquisition units (148), and the projection control unit (152) is used for M projection units (152). In addition to controlling the transfer speed and transfer direction of the pattern film constituting 146), the blinking cycle of the light source that projects the pattern film is controlled.

The marker blinking control unit (154) periodically blinks the optical markers of each of the N marker generators (142) under the control of the microprocessor (156). The microprocessor (156) is a three-dimensional marker for each region from the two-dimensional image and the three-dimensional measurement data acquired by the M projection unit (146) and the L image acquisition unit (148), respectively. The position is extracted, and a set of markers is searched for by the overlapping area from the three-dimensional position of the extracted marker. At the same time as obtaining the position conversion matrix by the paired markers, the arithmetic processing is performed in which the positions of the three-dimensional measurement data are converted and aligned by the obtained position conversion matrix.

The buffer (158) stores data and result data necessary for arithmetic processing of the microprocessor (156).

The operation process of the three-dimensional measurement data automatic alignment device according to the ninth embodiment of the present invention configured as described above will be described in detail with reference to the flowcharts of FIGS. 25a and 25b attached.

First, the measurement object (10) is placed at an appropriate position, and N marker generators (142), M projection units (146), and L images are acquired around the measurement object (10). With each unit (148) arranged, the microprocessor (156) controls the marker blinking control unit (154) and lights a plurality of marker output units (144) provided in the marker generator (142). , Make sure that multiple markers are projected irregularly on the surface of the object to be measured (10) (step S110).

With the optical marker from the marker generator (142) projected onto the measurement object (10), each of the L image acquisition units (148) photographs the measurement target area of the measurement object (10). Then, when the two-dimensional image including the optical marker is acquired, the microprocessor (156) receives the two-dimensional image data acquired from the L image acquisition units (148) through the image input unit (150). It will be entered (step S111).

Next, the microprocessor (156) controls the marker blinking control unit (154) to turn off the N marker generators (142) so that the optical markers are not projected onto the object to be measured (10) (steps). S112), in that state, when the same area of the measurement object (10) is photographed by each of the L image acquisition units (148) and a two-dimensional image that does not include the optical marker is acquired, these L images are acquired. Dimensional video data will be input through the video input unit (150) (step S113).

Further, the microprocessor (156) controls the projection control unit (152) in a state where the N marker generators (142) are turned off and the optical markers are not projected, and the M projection units (146) are controlled. A predetermined pattern pattern (for example, a stripe pattern or a multiple stripe pattern of a plurality of sections having different intervals) for each three-dimensional measurement is generated from the M projection units (146) of the measurement object (for example). It is projected on the surface of 10).

At this time, when the measurement object (10) on which the pattern is projected is photographed by the L image acquisition units (148) and the L three-dimensional measurement data is acquired, the microprocessor (156) is subjected to the image input unit (156). Through 150), L 3D measurement data will be input (step S114).

In such a state, the microprocessor (156) video-processes the two-dimensional video data including the optical marker and the two-dimensional video data without the marker, and extracts the two-dimensional position of the marker. Becomes (stage S115).

Next, the microprocessor (156) uses markers extracted from the 2D video data, and the coordinates of the L image acquisition units (148) from the lens center of each camera to any three markers in the 2D video data. By estimating the value and any three-dimensional coordinate value on the three-dimensional measurement data located in a straight line, the three-dimensional position of the marker for each of the L three-dimensional measurement data is searched for (step S116). ..

Next, the microprocessor (156) compares the three-dimensional positions of the markers for each of the L three-dimensional measurement data found in the above step S116, and searches for markers that are paired with each other (step S117). ).

When it becomes possible to search for a pair of markers by the marker search process as described above, the microprocessor (156) uses the respective three-dimensional measurement data to move from the position of the paired marker to the position for movement. The transformation matrix is calculated (step S118), one of the L coordinate measurement data is set as the reference coordinate system, and the current measurement data is moved by the above-determined position transformation matrix. It will be aligned (step S119).

Next, a tenth embodiment of the present invention will be described with reference to the attached drawings.

The tenth embodiment of the present invention has the same hardware configuration as the ninth embodiment described above, but the configuration of the operation process is different.

Therefore, the tenth embodiment of the present invention is based on the hardware configuration of the ninth embodiment shown in FIG. 28, and the operation process thereof will be described in detail with reference to the flowchart of FIG.

First, a reference object whose dimensions are already known is placed at an appropriate position, and N marker generators (142), M projection units (146), and L image acquisition units (L) centering on this reference object ( 148) are placed respectively. At this time, the reference object can be an actual measurement object if it is separately manufactured for calibration or if the dimensions are already known.

In this state, the microprocessor (156) controls the marker blinking control unit (154), lights each marker output unit (14) provided in the N marker generators (142), and turns on the surface of the reference object. Allow multiple markers to be projected irregularly (step S120).

Next, the microprocessor (156) will perform the calibration work to find the correlation between the L image acquisition units (148) and the reference object (step S121), but the specific operation process for this will be described in detail. explain.

In the above step S121, when the optical markers from the N marker generators (142) are projected onto the reference object, the measurement target (10) of the measurement target (10) is measured by each of the L image acquisition units (148). When the area was photographed and a two-dimensional image containing an optical marker was acquired, the microprocessor (156) received L images from the image acquisition unit (148) through the image input unit (150). 2D video data will be input.

Next, the microprocessor (156) controls the projection control unit (152) to operate the M projection units (146), and the M projection units (146) are used for each of the three-dimensional measurements. A predetermined pattern pattern (for example, a stripe pattern of a plurality of sections having different intervals or a multiple stripe pattern) is projected on the surface of the reference object.

At this time, when the reference object on which the pattern is projected is photographed by the L image acquisition units (148) and the L three-dimensional measurement data is acquired, the microprocessor (156) passes the L through the image input unit (150). Three-dimensional measurement data will be input.

In that state, the microprocessor (156) uses the two-dimensional position of the marker containing the optical marker and the dimensional information of the reference object that it already knows, and uses the dimensional information of the L image acquisition units (148) of each camera. By estimating the coordinate values for any three markers in the two-dimensional video data from the center of the lens and the arbitrary three-dimensional coordinate values on the three-dimensional measurement data located in a straight line, each of the L three-dimensional data You will be looking for the three-dimensional position of the marker.

Next, the microprocessor (156) compares the three-dimensional positions of the markers for each of the L three-dimensional measurement data, and searches for markers that are paired with each other. Then, in each of the three-dimensional measurement data, the position conversion matrix for movement is obtained from the position of the paired marker. The microprocessor (156) will register the obtained position transformation matrix in the register of the buffer (158), and the calibration work of step S121 is completed. As described above, when the calibration work of step S121 is completed, the reference object is removed and the measurement object (10) is placed at the place where the reference object was, and the microprocessor (156) blinks the marker. The control unit (154) is controlled to turn off the N marker generators (142) so that the optical markers are not projected onto the measurement object (10) (step S122).

In this state, when the measurement target area of the measurement target (10) is photographed by each of the L image acquisition units (148) and the L two-dimensional images that do not include the optical marker are acquired, This two-dimensional video data will be input through the video input unit (150) (step S123).

In addition, the microprocessor (156) controls the projection control unit (152) and operates the M projection units (146) with the N marker generators (142) turned off and the optical markers not projected. From the M projection units (146), predetermined pattern patterns (for example, stripe patterns or multiple stripe patterns of a plurality of sections having different intervals) for each three-dimensional measurement are displayed on the measurement object (10). ) Is projected on the surface.

At this time, when the measurement object (10) on which the pattern is projected is photographed by the L image acquisition units (148) and the L three-dimensional measurement data is acquired, the microprocessor (156) is subjected to the image input unit (156). Through 150), L 3D measurement data will be input (step S124). Next, the microprocessor (156) reads the position transformation matrix obtained by the calibration in the step (S121) from the register of the buffer (158), and reads a certain position among the L three-dimensional measurement data. The current measurement data is moved and aligned by the position transformation matrix read from the register of the buffer (158) in the reference coordinate system (step S125).

Later, when measuring another measurement object or measuring the same measurement object again, the calibration work from step S121 to step S123 described above is omitted, and the buffer (158) is omitted. Since the three-dimensional measurement data is aligned by the position conversion matrix stored in the register of), the work time is shortened.

However, if necessary, it is possible to perform the above-mentioned calibration work from step S121 to step S123 for each measurement, which can be easily changed and carried out according to the intention of the operator and the configuration of the system. ..

Next, the eleventh embodiment of the present invention will be described.

The eleventh embodiment of the present invention presents different examples of the marker generator and its peripheral device (hereinafter, referred to as marker generator) used in the first or tenth embodiment of the present invention. ..

As shown in FIG. 31, the marker generator according to the eleventh embodiment of the present invention can rotate around a plurality of X-axis light sources (160), an X-axis blinking control unit (162), and a hinge axis. X-axis polygonal polygon mirror (164) (Polygon Mirror), X-axis rotation drive unit (166), X-axis rotation mechanism (168), and multiple Y-axis light sources (170). ), Y-axis blinking control unit (172), Y-axis polygonal polygon mirror (174) configured to rotate around the hinge axis, Y-axis rotation drive unit (166), and Y-axis Consists of including the rotation mechanism (178) of.

The plurality of light sources (160) on the X-axis generate a beam having excellent straightness such as a laser, and diverge on the reflecting surface of the polygonal polygon mirror (164) on the X-axis. For example, a laser point can be used. Then, the X-axis blinking control unit (162) blinks and controls each X-axis light source (160) by control from a microprocessor (not shown).

The X-axis polygonal polygon mirror (164) has a plurality of reflecting surfaces, and while being rotationally driven by the X-axis rotation mechanism (168), a plurality of light sources (160) radiate from the plurality of light sources (160). The beam is reflected by the above-mentioned plurality of reflecting surfaces and projected onto the measurement area of the object to be measured (OB).

The X-axis rotation drive unit (166) drives the X-axis polygonal polygon mirror (164) to rotate in one direction under the drive control of the microprocessor, and the X-axis rotation mechanism (168). ) Is powered by the X-axis rotation drive unit (166) and has a structure for rotating the X-axis polygonal polygon mirror (164) in one direction.

The plurality of Y-axis light sources (170) generate a beam having excellent straightness such as a laser and diverge to the reflecting surface of the Y-axis polygonal polygon mirror (174). For example, a laser point. Etc. are used. Then, the Y-axis blinking control unit (172) blinks and controls a plurality of Y-axis light sources (170) by control from a microprocessor (not shown).

The Y-axis polygonal polygon mirror (174) has a plurality of reflecting surfaces, and is driven to rotate by a Y-axis rotation mechanism (178), and is emitted from a plurality of light sources (170). Each beam is reflected by the above-mentioned plurality of reflecting surfaces and projected onto the measurement area of the measurement object (OB).

The Y-axis rotation drive unit (176) drives the Y-axis polygonal polygon mirror (174) to rotate in one direction under the drive control of the microprocessor, and the Y-axis rotation mechanism (178). ) Is powered by the Y-axis rotation drive unit (166) and has a structure for rotating the Y-axis polygonal polygon mirror (174) in one direction.

Hereinafter, the operation process of the marker generator according to the eleventh embodiment of the present invention configured as described above will be described in detail.

First, according to the control signal from the microprocessor, the drive power supply from the X-axis rotation drive unit (166) and the Y-axis rotation drive unit (176) is the X-axis rotation mechanism (168) and the Y-axis rotation mechanism (178). ), And the X-axis rotation mechanism (168) and Y-axis rotation mechanism (178) are driven by the drive power supply approved by the X-axis rotation drive unit (166) and the Y-axis rotation drive unit (176), respectively. Driven to rotate X-axis and Y-axis polygonal polygon mirrors (164, 174).

At the same time, the X-axis blinking control unit (162) and the Y-axis blinking control unit (172) turn on the X-axis light source (160) and the Y-axis light source (170), respectively, by the control signal from the microprocessor. As a result, the beams generated from the plurality of X-axis and Y-axis light sources (160, 170) are the reflection surfaces of the X-axis polygonal polygon mirror (164) and the Y-axis polygonal polygon mirror (174). Is incident on.

The beams incident on the reflection surfaces of the X-axis polygonal polygon mirror (164) and the Y-axis polygonal polygon mirror (174) from the X-axis and Y-axis light sources (160 and 170) are X. It is reflected from each reflecting surface of the axis and Y-axis polygonal polygon mirrors (164, 174) and projected onto the surface of the object to be measured (OB).

At this time, the X-axis and Y-axis polygonal polygon mirrors (164, 174) rotate at high speed, and the angles of the reflecting surfaces are different, so that the surface of the object to be measured (OB) is X. A large number of beam lines are formed on each of the axes and the Y-axis, and at the same time, each intersection of the X-axis and Y-axis lines becomes an individual optical marker (RM).

For example, assuming that there are m X-axis light sources (160) and n Y-axis light sources (170), m * n intersections are formed on the surface of the object to be measured (OB). Since each of these m * n intersections becomes an optical marker (RM), a relatively large number of optical markers can be generated using a smaller number of light sources. It is.

The examples according to the present invention as described above are not limited to those described above, and the techniques described in the claims attached within the scope of obvious to a person having ordinary knowledge in the technical field to which the present invention belongs. It goes without saying that various corrections and changes can be made as long as the gist is not exceeded.

<figref num="1">It is a figure which illustrated the state for attaching the conventional sticker type marker to the object of measurement, and measuring three-dimensionally.</figref><figref num="2">It is a figure which illustrated the state which arranges different measurement data with each other using a sticker type marker as a reference marker.</figref><figref num="3">It is a figure which showed the structure for the 3D measurement data automatic alignment apparatus using the optical marker by 1st Example of this invention.</figref><figref num="4a">FIG. 5 is a diagram illustrating a state in which two-dimensional video data is acquired using an optical marker and a state in which three-dimensional measurement data is acquired using a pattern pattern according to a desirable first embodiment of the present invention.</figref><figref num="4b">FIG. 5 is a diagram illustrating a state in which two-dimensional video data is acquired using an optical marker and a state in which three-dimensional measurement data is acquired using a pattern pattern according to a desirable first embodiment of the present invention.</figref><figref num="4c">FIG. 5 is a diagram illustrating a state in which two-dimensional video data is acquired using an optical marker and a state in which three-dimensional measurement data is acquired using a pattern pattern according to a desirable first embodiment of the present invention.</figref><figref num="5">FIG. 5 is a diagram illustrating a state in which a two-dimensional position of a marker is extracted from two-dimensional image data acquired through turning on and off an optical marker according to a desirable first embodiment of the present invention.</figref><figref num="6">It is a figure which illustrated the state which extracts the 3D position of a marker from the lens center of a camera, and the 2D position of a marker.</figref><figref num="7a">FIG. 5 is a diagram illustrating an operation of finding a pair of markers through comparison of triangles with different video data according to a desirable first embodiment of the present invention.</figref><figref num="7b">FIG. 5 is a diagram illustrating an operation of finding a pair of markers through comparison of triangles with different video data according to a desirable first embodiment of the present invention.</figref><figref num="8a">FIG. 5 is a diagram exemplifying a conversion operation of matching pairs of triangles at different positions in a comparison of triangles with different video data according to a desirable first embodiment of the present invention.</figref><figref num="8b">FIG. 5 is a diagram exemplifying a conversion operation of matching pairs of triangles at different positions in a comparison of triangles with different video data according to a desirable first embodiment of the present invention.</figref><figref num="8c">FIG. 5 is a diagram exemplifying a conversion operation of matching pairs of triangles at different positions in a comparison of triangles with different video data according to a desirable first embodiment of the present invention.</figref><figref num="8d">FIG. 5 is a diagram exemplifying a conversion operation of matching pairs of triangles at different positions in a comparison of triangles with different video data according to a desirable first embodiment of the present invention.</figref><figref num="9">FIG. 5 is a diagram exemplifying an operation of finding a pair of markers by finding virtual markers with different video data according to a desirable first embodiment of the present invention.</figref><figref num="10a">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 1st Example of this invention.</figref><figref num="10b">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 1st Example of this invention.</figref><figref num="11">It is a figure which showed the structure in the 3D measurement data automatic alignment apparatus using the optical marker by 2nd Example of this invention.</figref><figref num="12">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 2nd Example of this invention.</figref><figref num="13">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 3rd Example of this invention.</figref><figref num="14">It is a figure which showed the structure in the 3D measurement data automatic alignment apparatus using the optical marker by 4th Example of this invention.</figref><figref num="15">It is a figure which showed the structure in the 3D measurement data automatic alignment apparatus using the optical marker by 5th Example of this invention.</figref><figref num="16">It is a figure which showed the structure in the 3D measurement data automatic alignment apparatus using the optical marker by the 6th Example of this invention.</figref><figref num="17a">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by the 6th Example of this invention.</figref><figref num="17b">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by the 6th Example of this invention.</figref><figref num="18">It is a figure for demonstrating the error at the time of arranging the measurement data by the reference coordinate system.</figref><figref num="19">It is a figure for demonstrating the error at the time of arranging the measurement data by an absolute coordinate system.</figref><figref num="20">It is a figure which showed the structure in the 3D measurement data automatic alignment apparatus using the optical marker by 7th Example of this invention.</figref><figref num="21a">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 7th Example of this invention.</figref><figref num="21b">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 7th Example of this invention.</figref><figref num="22">FIG. 20 is a diagram illustrating an example of an image that can be obtained by using the large area image acquisition unit shown in FIG. 20.</figref><figref num="23">FIG. 20 is a diagram illustrating an example of an image obtained by using a large-area image acquisition unit and an image acquisition unit shown in FIG. 20.</figref><figref num="24">It is a figure which showed the structure in the 3D measurement data automatic alignment apparatus using the optical marker by 8th Example of this invention.</figref><figref num="25a">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 8th Example of this invention.</figref><figref num="25b">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by 8th Example of this invention.</figref><figref num="26a">FIG. 24 is a diagram illustrating an example of an image that can be obtained by using a set of large-area image acquisition units illustrated in FIG. 24.</figref><figref num="26b">FIG. 20 is a diagram illustrating an example of an image that can be obtained by using a set of a large-area image acquisition unit and an image acquisition unit shown in FIG. 20.</figref><figref num="27">It is the schematic for demonstrating the principle of 8th Example of this invention.</figref><figref num="28">It is a figure which showed the structure in the 3D measurement data automatic alignment apparatus using the optical marker by the 9th Example of this invention.</figref><figref num="29">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by the 9th Example of this invention.</figref><figref num="30">It is a flowchart for demonstrating the operation in the 3D measurement data automatic alignment method using the optical marker by the tenth embodiment of this invention.</figref><figref num="31">It is a figure which showed the structure in the marker generator according to the eleventh embodiment of this invention.</figref>

Code description

10: Measurement object, 12: Marker generator, 14: Marker output unit, 16: Projection unit, 18: Image acquisition unit, 20: Movement drive unit, 22: Movement mechanism, 24: Image input unit, 26: Marker blinking Control unit, 28: Projection control unit 30: Microprocessor, 32: Buffer