Method and apparatus for the correction of nonlinear field of view distortion of a digital imaging system
Summary by NHIP
Nonlinear Field of View Distortion Correction
The method quantifies nonlinear field of view distortions by comparing monitored image space point changes with monitored object space test artifact movements. This process determines corrective values independently of the test artifact's exact dimensions while the artifact moves through the imaging system's field of view.
Claim Score by NHIP
Abstract
Nonlinear distortions of imaging optics within a computer vision system are quantified as corrective values within a viewing window defined by a pixel array. A test artifact is displaced within an object plane through increments of measured distances and the corresponding displacements of the test artifact within an image plane are recorded within the pixel array. Comparisons are made between the actual appearances of the test artifact within the image plane and predicted appearances of the test artifact within the image plane based on the measured displacements in the object plane for determining the corrective values.

Term
0.8 yearsleft in the term
Expires 20 July 2027, including 996 days of term adjustment.
- Priority and filed
- Granted
- Today
- Expires
26 claims: 1 independent, 25 dependent
- 1Broadest claimClaim Score 40, average(NHIP)A method of quantifying nonlinear field of view distortions in a digital imaging system comprising steps of:mounting a test artifact having an imageable feature in a position for undergoing relative motion with respect to a digital imaging system;optically imaging the imageable feature of the test artifact with the digital imaging system;relatively moving the imageable feature of the test artifact through a field of view of the digital imaging system;monitoring changes in the locations of a plurality of points along the imageable feature of the test artifact appearing within an image space of the digital imaging system;monitoring changes in the location of the test artifact within an object space of the digital imaging system as a measure of the relative motion imparted to the test artifact;and comparing changes in the monitored locations of the plurality of points along the imageable feature of the test artifact appearing in the image space with changes in the monitored location of the test artifact in the object space to quantify nonlinear field of view distortions in the digital imaging system, wherein the step of comparing changes includes quantifying nonlinear field of view distortions in the digital imaging system independently of exact dimensions of the test artifact.
47 paragraphs in 4 sections, as filed
BACKGROUND OF THE INVENTION
p-00021. Field of the Invention
p-0003Computer vision applications involving digital optical metrology provide for precisely imaging test pieces for purposes of measurement or comparison. Calibration techniques accommodate imaging system errors.
p-00042. Description of Related Art
p-0005Nonlinear field of view distortions are particularly troublesome for computer vision applications that require precise measurements or comparisons of imaged test objects. The distortions do not affect image quality, such as the sharpness of the image, but do affect image geometry, which can complicate the accurate determination of size, shape, and location of the imaged test objects.
p-0006For example, curvilinear distortions are manifest as so-called “barrel” or “pincushion” distortions. Focal length varies as a function of radial distance from a center of the field. Straight tangential lines tend to curve, while radial lines remain straight. “Barrel” distortion decreases focal length as the field angle increases, and “pin cushion” distortion increases focal length as the field angle increases. Other distortions in which the resulting image is not a true-to-scale reproduction of an object include radial distortion in which magnification varies from the center of the field, anamorphic distortions in which magnification varies with direction, and tangential distortions such as resulting from centration errors.
p-0007Among the most significant problems caused by such nonlinear field of view distortions for the imaging of test objects are size, shape, and location distortions. The same test object can appear to be differently sized or shaped depending upon its position within the field of view. The apparent distance an object moves within the field of view can differ from the actual distance that the object moves. For example, the location of an object's edge from the center of the field of view can differ from the actual location of the object's edge depending upon such factors as the position and orientation of the object's edge within the field of view.
p-0008Solutions for measuring and correcting for such nonlinear field of view distortions include the use of calibrated artifacts having known size and shape. The calibrated artifacts are moved to different locations within the field of view and deviations in the measured size and shape are used to generate local error corrections within the field of view. For digital cameras, lookup tables are created for recording pixel-by-pixel corrections.
p-0009Another technique for compensating for nonlinear field of view distortions involves placing a calibrated grid within the field of view. Typically, the calibrated grid fills the entire field of view, and the imaged locations of the grid lines are compared to their known locations to generate corrections required for constructing a more accurate image of the calibrated grid.
p-0010The manufacture and certification of calibrated artifacts and grids involves considerable expense. The accuracy with which the calibrated artifacts are known affects the accuracy of the corrections that can be made to compensate for the distortions of the field of view. Multiple artifacts and grids are sometimes needed for measuring distortions of the field of view at different magnifications.
BRIEF SUMMARY OF THE INVENTION
p-0011The invention is particularly applicable to computer vision systems of the type that include a digital imaging system in combination with a motion stage for relatively moving test artifacts within the field of view of the digital imaging system. In one or more of the preferred embodiments, the digital imaging system and motion stage provide separate measurements of the test artifacts. The digital imaging system can be associated with image processing capabilities for monitoring the location of test artifacts or their features within the field of view, and the motion stage can be associated with instrumentation for measuring relative displacements of the test artifacts imparted by the motion stage. Distortion related error corrections can be determined by comparing displacements of images of the test artifact within the field of view with the measured displacements the motion stage.
p-0012For example, a test artifact can be placed on the motion stage and moved by steps through the field of view. The artifact displacements can be recorded in both image space and object space, and a comparison can be made between the two. The digital imaging system preferably detects displacements of the test artifact in image space, and instrumentation associated with the motion stage preferably detects displacements of the test artifact in object space. Differences between scaled measures of displacement in image and object spaces can be attributed to errors in the imaging system when accuracy and repeatability of the motion stage instrumentation are significantly smaller than the detected error.
p-0013The test artifact can be a simple structure of undetermined size, but preferably has one or more edges that can be clearly imaged by the imaging system. For example, the calibration artifact can have two orthogonally related edges. One of the edges preferably spans one entire dimension of the field of view at the lowest working magnification of the imaging system, and the other of the edges preferably spans an orthogonal dimension of the field of view at the same working magnification. Both edges are preferably straight, but allowances can be made for variations in form, orthogonal tilt, and perpendicularity.
p-0014Within a vertical (Y axis) and horizontal (X axis) coordinate system corresponding to both (a) vertical and horizontal dimensions of the image field and (b) coordinate directions of motion imparted by the motion stage, the edges of the reticle can be approximately aligned for rectilinear motion. For example, a vertical edge can be aligned with the Y-axis, and a horizontal edge can be aligned with the X-axis. The vertical edge spanning the entire Y axis coordinate dimension of the image field can be stepped by known increments in the X coordinate dimension, and the horizontal edge spanning the entire X axis coordinate dimension of the image field can be separately stepped by known increments in the Y coordinate dimension. The successive locations of the edges can be measured in the field of view (image space) in units of image pixels. The same displacements can be measured in units of distance by the motion stage. Absent any misalignment or departures from straightness of the edges, all points along the two edges should undergo similar amounts of displacement in units of image pixels for each step that the edges are actually displaced in their respective directions. After resolving any issues of edge misalignment or departures from straightness, deviations in the amount of displacement of the edges measured in units of pixels can be attributed to distortions of the field of view.
p-0015For example, after determining where the vertical and horizontal edges appear in the image space at the incrementally displaced positions, a comparison can be made against where the edges should have appeared based upon the known increments of displacement. Each of the pixels within the field of view can be associated with a correction in the vertical and horizontal directions for reproducing a digital image corrected for the measured distortions.
p-0016An embodiment of the invention as a method of quantifying nonlinear field of view distortions in a digital imaging system includes mounting a test artifact in a position for undergoing relative motion with respect to a digital imaging system that optically images the test artifact. The relative motion of the test artifact through a field of view of the digital imaging system is monitored in two ways. First, the location of the test artifact appearing within an image space of the digital imaging system is monitored. Second, the location of the test artifact within an object space of the digital imaging system is monitored as a measure of the relative motion imparted to the test artifact. Comparing changes in the monitored location of the test artifact appearing in the image space with changes in the monitored location of the test artifact in the object space enables nonlinear field of view distortions to be quantified for the digital imaging system.
p-0017Another embodiment of the invention as method of compensating for non-linear distortion of a computer vision system combines a digital imaging system with a motion stage. A test artifact having an imageable feature is positioned within a field of view of the digital imaging system. The motion stage relatively moves the test artifact together with the imageable feature with respect to the digital imaging system through a succession of different positions within the field of view of the digital imaging system. Noted at each position are (a) a location of an image of the imageable feature within the digital imaging system as a set of pixels within a pixel array and (b) a location of the imageable feature with respect to the digital imaging system as a relatively displaced position in coordinate space. Predicted locations of the image of the imageable feature are calculated based on the relative location of the imageable feature with respect to the digital imaging system in coordinate space. Corrections to the field of view are then determined based on differences between the predicted locations of the image of the imageable feature and the noted locations of the image of the imageable feature.
p-0018Another embodiment of the invention as method of determining distortions in a vision system also combines a digital imaging system with a motion stage. An orientation of an artifact feature is separately measured with respect to the digital imaging system and the motion stage. The artifact feature is relatively stepped with respect to the digital imaging system through a succession of positions within a field of view of the digital imaging system. Associated changes in the positions of the artifact feature with respect to the digital imaging system are also measured. Appearances of the artifact feature within the digital imaging system are recorded at each of the succession of positions. Predicted appearances of the artifact feature within the digital imaging system are calculated based on: (a) the measure of the orientation of the artifact feature with respect to the digital imaging system, (b) the measure of the orientation of the artifact feature with respect to the motion stage, and (c) the measure of the change in position of the artifact feature with respect to the digital imaging system. Distortions in the digital imaging system are then determined based on differences between the recorded appearances of the artifact feature within the digital imaging system and the predicted appearances of the artifact feature within the digital imaging system.
p-0019Yet another embodiment of the invention as a system for compensating for non-linear field of view distortions in a computer vision system includes a digital imaging system having a set of optics for imaging an artifact feature within a field of view and a motion stage for relatively moving the artifact feature with respect to the digital imaging system. A controller associated with the motion stage relatively moves the artifact feature with respect to the digital imaging system through a succession of different positions. Instrumentation associated with the motion stage measures the relative displacement of the artifact feature with respect to the digital imaging system through the succession of different positions. A processor (a) records locations of a succession of images of the artifact feature within the field of view at the succession of different relative positions of the artifact feature, (b) calculates predicted locations of the images of the artifact feature based on the measured relative displacement of the artifact feature, and (c) compares the recorded locations of the succession of images of the artifact feature within the field of view with the predicted locations of the images of the artifact feature within the field of view for determining corrections required to compensate for non-linear field of view distortions.
p-0020The invention among its embodiments obviates the need for a calibrated test artifact or test grid. In fact, the preferred test artifact for the invention has an undetermined size or a size that, even if known to some accuracy, is not relied upon to quantify imaging system distortions. The preferred test artifact also has a simple shape, such as a square with two right angle edges that participate in the characterization of image distortions. Each of the two edges preferably bisects one of two orthogonal dimensions of the field of view so that information concerning distortions in the two dimensions can be collected efficiently.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)
p-0021<figref idrefs="DRAWINGS">FIG. 1</figref> shows the layout of computer vision system in a side view with a test artifact mounted on a motion stage within view of a digital imaging system for quantifying nonlinear field of view distortions in the digital imaging system.
p-0022<figref idrefs="DRAWINGS">FIG. 2</figref> overlays a square test artifact over a field of view sampled by a detector array.
p-0023<figref idrefs="DRAWINGS">FIG. 3</figref> is a flow chart describing a first part of a data acquisition system including data relating to displacements of the test artifact along a first coordinate axis.
p-0024<figref idrefs="DRAWINGS">FIG. 4</figref> is a flow chart describing a second part of a data acquisition system including data relating to displacements of the test artifact along a second coordinate axis.
p-0025<figref idrefs="DRAWINGS">FIG. 5</figref> is a flow chart describing a process for determining tilt of the test artifact with respect to the two coordinate axes.
DETAILED DESCRIPTION OF THE INVENTION
p-0026A computer vision system <b>10</b> of a type subject to the improvements of the invention as shown in <figref idrefs="DRAWINGS">FIG. 1</figref> includes a digital imaging system <b>12</b> and a motion stage <b>14</b> for relatively moving a test artifact <b>16</b> through a field of view of the digital imaging system <b>12</b>. The test artifact <b>16</b> as shown in <figref idrefs="DRAWINGS">FIG. 2</figref> is an alignment reticle in the form of a chrome square with two edges <b>18</b> and <b>20</b> that are perpendicular to each other and of substantial straightness. The horizontal edge <b>18</b> (as shown in the plane of <figref idrefs="DRAWINGS">FIG. 2</figref>) is approximately aligned with an X coordinate axis of a video window <b>22</b> of the digital imaging system <b>12</b>, and the vertical edge <b>20</b> (also as shown in the plane of <figref idrefs="DRAWINGS">FIG. 2</figref>) is approximately aligned with a Y coordinate axis of the video window <b>22</b>. Although it is not necessary to know the exact size of the square artifact <b>16</b>, the square artifact <b>16</b> is preferably large enough to completely bisect the video window <b>22</b> along both the X and Y coordinate axes at the lowest magnification of the digital imaging system <b>12</b>.
p-0027The motion stage <b>14</b> includes a mounting surface <b>24</b> for mounting the square artifact <b>16</b> and provides two axes of motion for the square artifact <b>16</b> aligned with the X and Y coordinate axes of the video window <b>22</b>. A first translation stage <b>26</b> moves the square artifact <b>16</b> along the X coordinate axis under the control of a first drive <b>28</b> that is instrumented to provide position feedback information for measuring motion of the stage <b>26</b> along the X coordinate axis. A second translation stage <b>30</b> moves the square artifact <b>16</b> along the Y coordinate axis under the control of a second drive <b>32</b> that is instrumented to provide position feedback information for measuring motion of the stage <b>30</b> along the Y coordinate axis. A light source <b>34</b> within a base <b>36</b> supporting the translation stages <b>26</b> and <b>30</b> illuminates the mounting surface <b>24</b> through light transmitting portions of the translation stages <b>26</b> and <b>30</b>, allowing the edges <b>18</b> and <b>20</b> of the square artifact <b>16</b> to be brought into sharp focus by the digital imaging system <b>12</b>.
p-0028The digital imaging system <b>12</b> includes a digital camera <b>40</b> cantilevered from a pedestal <b>38</b> that extends above the base <b>36</b> along with a similarly supported objective lens <b>42</b> for imaging illuminated edges of the square artifact <b>16</b> in an object plane <b>44</b> onto a detector pixel array <b>46</b> in an image plane <b>48</b> within the digital camera <b>40</b>. The detector pixel array <b>46</b> corresponds in size to the video window <b>22</b> and sets limits of a field of view for a given magnification of the square artifact <b>16</b> in the object plane <b>44</b>. Resolution is determined in part by the number of pixels in the array, which can number 640 by 480 for example. The digital camera <b>40</b> can be translated along a Z-axis of the pedestal <b>38</b> for achieving the desired focusing objectives.
p-0029A controller <b>50</b> under the instruction of a computer processor <b>52</b> controls operation of the motion stage <b>14</b> providing desired translations of the square artifact <b>16</b> through a succession of positions along the two coordinate axes X and Y. The associated instrumentation of the coordinate axes X and Y provides position feedback information that can be used for better controlling the movements of the stages <b>26</b> and <b>30</b> and for relating movements of the test artifact <b>16</b> in the object plane <b>44</b> to the apparent movements of the test artifact <b>16</b> in the image plane <b>48</b>.
p-0030The processor <b>52</b> provides a number of functions including (a) recording locations of artifact edges <b>18</b> and <b>20</b> within the image plane at a succession of displaced positions of the artifact <b>16</b> within the object plane <b>48</b>, (b) predicting locations of the artifact edges <b>18</b> and <b>20</b> within the image plane <b>48</b> based on the measured displacements of the artifact within the object plane <b>44</b>, and (c) comparing the recorded locations of the artifact edges <b>18</b> and <b>20</b> within the image plane to the predicted locations of the artifact edges <b>18</b> and <b>20</b> within the image plane for determining corrections required to compensate for nonlinear field of view distortions of the digital imaging system <b>12</b>. The error corrections can be recorded in a lookup table for the pixels subject to measurement. Interpolation techniques can be used for making error corrections for the remaining pixels.
p-0031Flow charts describing an example of the type of operations performed within the processor <b>52</b> are presented in <figref idrefs="DRAWINGS">FIGS. 3-5</figref>. A first part of a data acquisition routine is described by <figref idrefs="DRAWINGS">FIG. 3</figref>, beginning with an initialization step <b>60</b> that set the bounds of the detector pixel array <b>40</b>, which includes by way of example a matrix of 640 pixels along the X coordinate axis and 480 pixels along the Y coordinate axis. The square test artifact <b>16</b>, which is referred to as an “alignment reticle”, is positioned at step <b>62</b> with its upper right corner formed by the intersection of edges <b>18</b> and <b>20</b> within the center of the field of view as shown in <figref idrefs="DRAWINGS">FIG. 2</figref>. A video display <b>54</b> connected to the computer processor <b>52</b> includes graphics for guiding the square artifact <b>16</b> to the desired position within the field of view.
p-0032Step <b>64</b> determines the approximate pixel size in the two coordinate directions along with approximate dimensions of the pixel array. The motion stage displaces the edges <b>18</b> and <b>20</b> by measured amounts along the two coordinate axis X and Y, and the corresponding displacements in pixels are measured along limited regions of the two edges <b>18</b> and <b>20</b>. The field of view can be approximated from a previously performed and saved simple (linear) calibration done on the instrument, or the field of view can be approximated from a nominal magnification value for the optical components being used (i.e., from the optical specifications). Step <b>66</b> determines the approximate tilt of the two edges <b>18</b> and <b>20</b> with respect to the X and Y coordinate axes. For measuring the tilt of the horizontal edge <b>18</b> with respect to the X coordinate axis, the edge <b>18</b> is moved to a position that horizontally bisects the field of view within the video window <b>22</b>. Pixel displacements of the edge <b>18</b> along the Y coordinate axis near the opposite side borders <b>55</b> and <b>56</b> of the video window <b>22</b> are compared to the pixel distance between the measured displacements to estimate the tilt of edge <b>18</b>. The tilt of the vertical edge <b>20</b> is similarly estimated with respect to the Y coordinate axis by moving the edge <b>20</b> to a position that vertically bisects the field of view within the video window <b>22</b> and measuring relative displacements of the edge <b>20</b> near the top and bottom borders <b>57</b> and <b>58</b> of the video window <b>22</b>.
p-0033Based on these preliminary measurements, enough information has been collected to begin acquiring data relating to the appearance of the edges <b>18</b> and <b>20</b> at incrementally shifted positions throughout the field of view. Step <b>68</b> starts a sequence for recording incrementally displaced positions of the vertical edge <b>20</b> beginning near one side border <b>56</b> of the video window <b>22</b>. Decision step <b>70</b> checks if the vertical edge <b>20</b> is approaching the center of the field of view, and if so, a regular incremental displacement of the edge <b>20</b> is modified by step <b>72</b> so that the edge <b>20</b> is displaced by equal amounts on opposite sides of the center of the field of view. Decision step <b>74</b> checks if the vertical edge <b>40</b> is approaching the opposite side border <b>55</b> of the video window <b>22</b>, and if so, the step <b>76</b> does not allow the vertical edge <b>40</b> to be moved beyond the border <b>55</b>. Otherwise step <b>78</b> supports a regular incremental displacement of the vertical edge <b>20</b> across the field of view.
p-0034Step <b>80</b> calculates instructions for the controller <b>50</b> for carrying out the displacements determined in the preceding steps. The units of pixel displacements are converted into stage motions using the estimates of pixel size also determined in the preceding steps. Actual data gathering occurs at step <b>82</b>. After moving the translation stage <b>26</b> to the next determined position and waiting for the translation stage <b>26</b> to settle, the stage location is retrieved and a conventional edge detection procedure, referred to as a weak edge algorithm, is applied to identify the pixels along which the vertical edge <b>20</b> is found.
p-0035Decision step <b>84</b> queries whether enough of the vertical edge <b>20</b> has been identified. Identification of at least 90 percent of the edge is preferred for taking a measurement. Near the borders <b>55</b> and <b>56</b>, the tilt of the edge <b>20</b> can reduce the number of pixels straddling a point along the edge below a threshold for the processing algorithm. If at least 90 percent of the vertical edge <b>20</b> is not detected, decision steps <b>86</b> and <b>88</b> check to be sure the vertical edge <b>20</b> is near one of the two horizontal borders <b>55</b> or <b>56</b>, because otherwise an error condition exists. Steps <b>90</b> and <b>92</b> move the vertical edge <b>20</b> by small increments away from the borders <b>55</b> or <b>56</b> until at least 90 percent of the edge is detected. Decision step <b>96</b> returns processing to the decision step <b>70</b> for displacing the vertical edge <b>20</b> through another incremental displacement unless the vertical edge <b>20</b> has arrived at the opposite border <b>55</b> completing the scan.
p-0036A second part of the data acquisition routine is described by the flow chart of <figref idrefs="DRAWINGS">FIG. 4</figref>. Block <b>100</b> of the flow chart bordered by steps <b>102</b> and <b>104</b> follows a sequence corresponding to the steps <b>68</b> through <b>96</b> in the first part of the data acquisition routine adapted for using the horizontal edge <b>18</b> for scanning the field of view along the Y coordinate axis. At each measured incremental displacement of the horizontal edge <b>18</b>, both the location of the motion stage <b>14</b> and the locations of the pixels along which the edge <b>18</b> is found are recorded.
p-0037Decision step <b>110</b> determines if the digital imaging system <b>12</b> is operating at highest magnification, which is the initial condition. If so, tilt of the square artifact <b>16</b>, referred to as the alignment reticle, is determined to a higher precision beginning at step <b>112</b> as described in more detail by the flow chart of <figref idrefs="DRAWINGS">FIG. 5</figref>. At step <b>114</b>, the upper right corner of the square artifact <b>16</b> is moved to the center of the field of view as shown in <figref idrefs="DRAWINGS">FIG. 2</figref>. A sequence of steps referenced generally as block <b>116</b> provides for gathering the location of fifty points along the vertical edge <b>20</b> near the upper right corner of the test artifact <b>16</b> at twenty different stage positions straddling the center of the field of view. The data is averaged to determine an accurate position of the upper end of the vertical edge <b>20</b>.
p-0038Step <b>118</b> translates the test artifact <b>16</b> vertically along the Y-axis until the lower right corner of the test artifact <b>16</b> is located at the center of the field of view. The distance translated is recorded. Step <b>120</b> provides for repeating the steps of block <b>116</b> to determine an accurate position of the lower end of the vertical edge <b>20</b>. Calculations for precisely determining the tilt of the vertical edge <b>20</b> are performed at step <b>122</b> based on the horizontal displacements measured at the opposite ends of the vertical edge (i.e., the upper right and lower right corners of the square test artifact <b>16</b>) and the vertical distance measured between the two ends. Similar measurements can be performed to determine the tilt of the horizontal edge <b>18</b>, or if the horizontal and vertical edges <b>18</b> and <b>20</b> are known to be exactly orthogonal (as is preferred), the same inclination to the orthogonal coordinate axes X and Y can be assumed,
p-0039Step <b>130</b> generates correction data based on the data acquired in the preceding steps, which includes (a) vertical lines of data representing the appearance of the vertical edge <b>20</b> within the field of view at measured horizontal displacements of the motion stage <b>14</b> along the X coordinate axis, (b) horizontal lines of data representing the appearance of the horizontal edge <b>18</b> within the field of view at measured vertical displacements of the motion stage <b>14</b> along the Y coordinate axis, and (c) a measure of the inclination of the horizontal and vertical edges <b>18</b> and <b>20</b> with respect to the X and Y coordinate axes.
p-0040A more accurate estimate of pixel size is determined by comparing the displacements of the two edges <b>18</b> and <b>20</b> recorded in pixels within the image plane <b>48</b> with the corresponding displacements of the artifact <b>16</b> recorded in distance within the object plane <b>44</b>. The large amount of collected data reduces the influence of camera noise on the determination of pixel size. For measuring horizontal distortions of the digital imaging system <b>12</b>, the vertical line of data closest to the center of the field of view is treated as being affected by the least distortion (i.e., best represents the true shape of the edge <b>20</b>). The vertical edge data is then compensated for the measured tilt, representing how the edge <b>20</b> would appear if not tilted.
p-0041Based on the tilt compensated center line of data, the measured horizontal displacements of the artifact <b>16</b>, and the estimated pixel size, new vertical lines of data are predicted, representing how the vertical edge <b>20</b> should have appeared within the field of view of the digital imaging system at the measured displacements of the artifact <b>16</b>. The differences between the predicted and actual vertical lines of data are treated as measures of the horizontal distortions of the digital imaging system <b>12</b>. Similar predictions can be made concerning the appearance of the horizontal edge <b>18</b> at its measured vertically displaced positions for measuring the vertical distortions of the digital imaging system <b>12</b>.
p-0042For example, the preferred procedure includes determining the stage displacement of the artifact <b>16</b> between the center line of data and another measured line of data within the field of view. Considering the calculated pixel size, a simple transform predicts how the center line of data should appear within the field of view if so displaced. The difference between the measured line of data and the predicted line of data is a direct measure of the local distortions of the digital imaging system <b>12</b>.
p-0043Both the measured and predicted horizontal and vertical lines of data can be represented in units of pixels within the pixel array of the digital camera <b>40</b>. Each pixel has both a horizontal component and a vertical component for locating its position within a matrix of pixels. Each measured pixel is also associated with at least one of a horizontal or vertical correction value based on the difference between its actual location within the pixel array and its predicted location within the pixel array but for optical distortion of the imaging system <b>12</b>. The correction values can be averaged of otherwise filtered based on the values of nearby pixels. A lookup table can be generated for storing the correction values for the given magnification.
p-0044Decision step <b>130</b> determines if the digital imaging system is operating at its lowest magnification. If not, processing continues with step <b>134</b> for acquiring new data and generating new correction values for the lower magnification. It is not necessary to re-measure tilt if the artifact <b>16</b> is not disturbed between the measurements. A similar lookup table can be generated for each different magnification. Data acquisition is complete after the correction values for the lowest magnification are recorded.
p-0045Nonlinear distortion related errors can be removed from new images intended for measurement or comparison within the computer vision system <b>10</b> by reconstructing the considered features of the image using the correction values. For example, a considered pixel having X and Y coordinates of 330 and 200 along with correction values of −10 and 5 is treated for measurement purposes as having X and Y coordinates of 320 and 205. If the considered pixel does not correspond to one of the pixels assigned with correction values, interpolation techniques can be used to calculate a correction value for the pixel based on surrounding pixels that do have assigned correction values.
p-0046Although the test artifact <b>16</b> is shown as a square, the artifact <b>16</b> can also take other shapes, but preferably includes at least one substantially straight edge and more preferably includes two orthogonal edges capable of bisecting the entire field of view at the lowest magnification. The motion stage <b>14</b> preferably supports two rectilinear axes of motion. However, additional or alternative axes of motion can also be used including a combination of rotational and translational axes. The horizontal and vertical orientations and their associated coordinate axes as used herein are intended as relative orthogonal orientations and do not limit the digital imaging system <b>12</b> and the motion stage <b>14</b> to any particular spatial orientation.
p-0047Preferably, the two axes of orthogonal translational motion supported by the motion stage <b>16</b> are exactly aligned with the X and Y coordinate axes of the video window <b>22</b>. However, any departures from alignment could be measured by comparing the same point on the test artifact <b>16</b> at two physically displaced positions within the field of view effected by motions along one of the axes of motion. Assumptions are also made that the distances measured in connection with the movements of the motion stage <b>14</b> are exact. Further calibrations can be carried out to support this assumption or compensate for any known errors.
p-0048Although the invention has been described in greatest detail with respect to its preferred embodiment, the teaching of the invention can be applied generally to the quantification of non-linear field of view distortions and has particular applicability to vision systems combining a digital imaging system with a motion stage for moving test objects through the field of view. The motion stage can include one or more axes for moving the test objects or the digital imaging systems.
Contents4
5 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5
Every citation, both waysCites: the store holds 20 of 21
| Document | Relation | Office | Cited during |
|---|---|---|---|
| WO2013158393A1 | Cited by | World Intellectual Property Organization (WIPO) | International search |
| US9093047B2 | Cited by | United States of America | Applicant |
| US9019316B2 | Cited by | United States of America | Applicant |
| US10441313B2 | Cited by | United States of America | Search report |
| US2008187239A1 | Cited by | United States of America | Pre-grant |
| US9418628B2 | Cited by | United States of America | Applicant |
| US2015297312A1 | Cited by | United States of America | Search report |
| US9093048B2 | Cited by | United States of America | Applicant |
| US2015297312A1 | Cited by | United States of America | Pre-grant |
| US9348111B2 | Cited by | United States of America | Search report |
| US2012050488A1 | Cited by | United States of America | Pre-grant |
| US2002041383A1 | Cites | United States of America | Applicant |
| US2002141636A1 | Cites | United States of America | Applicant |
| US2002164083A1 | Cites | United States of America | Applicant |
| US2003038933A1 | Cites | United States of America | Applicant |
| US2003048436A1 | Cites | United States of America | Applicant |
| US2003118227A1 | Cites | United States of America | Applicant |
| US2007229665A1 | Cites | United States of America | Applicant |
| US4256364A | Cites | United States of America | Applicant |
| US4298944A | Cites | United States of America | Applicant |
| US4825394A | Cites | United States of America | Applicant |
| US5070401A | Cites | United States of America | Applicant |
| US6026172A | Cites | United States of America | Applicant |
| US6538691B1 | Cites | United States of America | Applicant |
| US6567570B1 | Cites | United States of America | Applicant |
| US6573986B2 | Cites | United States of America | Applicant |
| US6618494B1 | Cites | United States of America | Applicant |
| US6707998B2 | Cites | United States of America | Applicant |
| US6816625B2 | Cites | United States of America | Applicant |
| US6819789B1 | Cites | United States of America | Search report |
| US7146033B2 | Cites | United States of America | Applicant |
| Janez Per{hacek over (s)} , Stanislav Kova{hacek over (c)}i{hacek over (c)}, "Nonparametric, Model-Based Radial Lens Distortion Correction Using Tilted Camera Assumption", Workshop, Feb. 7, 2002, 10 pages, University of Ljubljana, Tr{hacek over (z)}a{hacek over (s)}ka , Slovenia. | Non-patent | – | Applicant |
| Tsai, Roger Y, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses"; IEEE Journal of Robotics and Automation, vol. RA-3, No. 4, Aug. 1987, pp. 323-344. | Non-patent | – | Applicant |
| Reimann, David A. and Flynn, Michael J., "Automated Distortion Correction of X-Ray Image Intensifier Images", Nuclear Science Symposium and Medical Imaging Conference, 1992., Conference Record of the 1992 IEEE Orlando, FL, USA Oct. 25-31, 1992, New York, NY USA, IEEE, US, Oct. 25, 1992, pp. 1339-1341. | Non-patent | – | Applicant |
| Zhengyou, Zhang, "Camera Calibration with One-Dimensional Objects" IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE, New York, NY, US, vol. 26, No. 7, Jul. 2004, pp. 892-899. | Non-patent | – | Applicant |
| European Patent Office Search Report, Application No. 05256444.0, Date Mailed Mar. 19, 2007. | Non-patent | – | Applicant |
| McFawn, P.K., Forkert. L., Fisher, J.T. "A new method to perform quantitative measurements of bronchoscopic images"; Eur Respir J 2001; 18: 817-826, Copyright European Respiratory Journal, Copyright 2001, pp. 817-826. | Non-patent | – | Applicant |
9 members in 5 offices
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 97464504 | United States of America | A | |
| US20040974645 | – | – | – |
Members9
| Document | Office | Kind | |
|---|---|---|---|
| US2006087645A1 | United States of America | A1 | |
| EP1653406A2 | European Patent Office (EPO) | A2 | |
| JP2006129473A | Japan | A | |
| EP1653406A3 | European Patent Office (EPO) | A3 | |
| US7536053B2This record | United States of America | B2 | |
| EP1653406B1 | European Patent Office (EPO) | B1 | |
| AT470918T | Austria | T | |
| ATE470918T1 | Austria | T1 | |
| DE602005021735D1 | Germany | D1 |
50 transactions on the USPTO file
Allowed after 1 non-final rejection.
- Non-final rejections
- 1
- Final rejections
- 0
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Patent Issue Date Used in PTA CalculationAllowedPTAC | PTAC | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Dispatch to FDCD1935 | D1935 | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Non-Final ActionA... | A... | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response to Election / Restriction FiledELC. | ELC. | |
| Mail Restriction RequirementMCTRS | MCTRS | |
| Restriction/Election RequirementCTRS | CTRS | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Reference capture on IDSRCAP | RCAP | |
| Electronic Information Disclosure StatementEIDS. | EIDS. | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Reference capture on IDSRCAP | RCAP | |
| Electronic Information Disclosure StatementEIDS. | EIDS. | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Withdraw Flagged for 5/25W525 | W525 | |
| Flagged for 5/25F525 | F525 | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| IFW TSS Processing by Tech Center CompleteTSSCOMP | TSSCOMP | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Transfer Inquiry to GAUTI1050 | TI1050 | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Reference capture on IDSRCAP | RCAP | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Application Return from OIPEWROIPE | WROIPE | |
| Application Return TO OIPEROIPE | ROIPE | |
| Application Return from OIPEWROIPE | WROIPE | |
| Application Is Now CompleteCOMP | COMP | |
| Application Return TO OIPEROIPE | ROIPE | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Application Is Now CompleteCOMP | COMP | |
| Cleared by L&R (LARS)L128 | L128 | |
| Referred to Level 2 (LARS) by OIPE CSRL198 | L198 | |
| IFW Scan & PACR Auto Security ReviewSCAN | SCAN | |
| Initial Exam Team nnIEXX | IEXX |
5 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Maintenance fee paymentMAFP | MAFP | |
| Fee paymentFPAY | FPAY | |
| Fee paymentFPAY | FPAY | |
| Information on status: patent grantGrantedPATENTED CASESTCF | STCF | |
| AssignmentAS | AS |
Numbers
- Publication, DOCDB
- 7536053
- Publication, EPODOC
- US7536053
- Application
- 10974645
- Application, DOCDB
- 97464504
- Application, EPODOC
- US20040974645
Titles
- English
- Method and apparatus for the correction of nonlinear field of view distortion of a digital imaging system
Patent term adjustment
- A delay
- +996 daysthe office missed an examination deadline
- Net adjustment
- 996 days
Classification
- CPC, 1
- G06T5/80
- IPC, 1
- G06K9 68
- USPC, 4
- 382218000
- 382100000
- 382255000
- 382286000