Systems and methods for detecting the movement of an object
Summary by NHIP
Marker Movement Detection System
The system captures sequential images of a circular marker to detect object movement. A controller calculates position changes by intersecting arcs from grayscale images and summing pixel visual intensity values to generate composite data.
Claim Score by NHIP
Abstract
Systems and methods are provided for detecting a movement of an object marked with a marker. The system includes a sensor configured to capture a first image of the marker and to capture a second image of the marker after the first image, each of the first and second images having pixels each having a visual intensity. A controller is configured to compare the first image and the second image by comparing the visual intensity of each of the pixels of the first image and the second image, determine an area of overlap between the first image and the second image based on the comparison, calculate a change in position of the marker in the second image relative to the marker in the first image based on the area of overlap, and detect the movement of the object based on the change in position of the marker.

Term
5.5 yearsleft in the term
Expires 18 March 2032, including 328 days of term adjustment.
- Priority and filed
- Granted
- Today
- Expires
17 claims: 3 independent, 14 dependent
- 1A system for detecting a movement of an object marked with a marker, comprising:a sensor configured to capture a first image of the marker and to capture a second image of the marker after the first image, each of the first and second images comprising pixels each having a visual intensity;and a controller coupled to the sensor and configured to: compare the first image and the second image by comparing the visual intensity of each of the pixels of the first image with the visual intensity of each of the pixels of the second image, determine an area of overlap between the first image and the second image based on the comparison, calculate a change in position of the marker in the second image relative to the marker in the first image based on the area of overlap, and detect the movement of the object based on the change in position of the marker, wherein the marker is a circle and the controller is configured to determine the area of overlap as an intersection of a first arc associated with the marker in the first image and a second arc associated with the marker in the second image, wherein the controller is configured to compare the first image and the second image by summing values associated with the visual intensity of the pixels of the first image with values associated with the visual intensity of the pixels of the second image to generate composite pixel values.
- 7A system for detecting a movement of an object marked with a marker, comprising:a sensor configured to capture a first image of the marker and to capture a second image of the marker after the first image, each of the first and second images comprising pixels each having a visual intensity;and a controller coupled to the sensor and configured to: compare the first image and the second image by comparing the visual intensity of each of the pixels of the first image with the visual intensity of each of the pixels of the second image, determine an area of overlap between the first image and the second image based on the comparison, calculate a change in position of the marker in the second image relative to the marker in the first image based on the area of overlap, and detect the movement of the object based on the change in position of the marker, wherein the marker is a circle and the controller is configured to determine the area of overlap as an intersection of a first arc associated with the marker in the first image and a second arc associated with the marker in the second image, wherein the sensor defines a field of view, and wherein the field of view is selected based on an anticipated change of position of the marker in the second image relative to the first image.
- 10Broadest claimClaim Score 53, average(NHIP)A method for detecting a movement of an object marked with a marker, the method comprising the steps of:capturing a first image of the marker;capturing a second image of the marker, the first and second images comprising pixels, each with a respective visual intensity;comparing the first image and the second image by comparing the visual intensity of each of the pixels of the first image with the visual intensity of each of the pixels of the second image;determining an area of overlap between the first image and the second image based on the comparison;calculating a change in position of the marker in the second image relative to the marker in the first image based on the area of overlap;and detecting the movement of the object based on the change in position of the marker, wherein the step of capturing the second image includes selecting a field of view based on an anticipated change of position of the marker in the second image relative to the first image, wherein the marker is a circle, and wherein the step of determining the area of overlap includes identifying an intersection of a first arc associated with the marker in the first image and a second arc associated with the marker in the second image.
Independent claims3
46 paragraphs in 6 sections, as filed
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT
This invention was made with Government support under Contract FA9453-08-C-0162 awarded by the Air Force Research Laboratory. The Government has certain rights in the invention.
TECHNICAL FIELD
The present invention generally relates to systems and methods for detecting the movement of an object, and more particularly relates to systems and methods for detecting the movement of an object in a navigation system.
BACKGROUND
An inertial navigation system (INS) is a navigation aid that uses motion and rotation sensors, such as accelerometers and gyroscopes, to continuously calculate the position, orientation, and velocity of a vehicle without the need for external references. INSs may be used on vehicles such as ships, aircraft, submarines, guided missiles, and spacecraft.
As examples, the accelerometers and gyroscopes used in navigation systems may have freely rotating masses that are monitored to determine the kinematic state changes of the vehicle. Given the sensitivity of the accelerometers and gyroscopes, it is generally undesirable for the monitoring sensors to have physical contact with the rotating masses. However, non-contact sensors or other devices that monitor the movement of the subject objects may not have sufficient accuracy or speed and may be subject to high costs and large processing requirements.
Accordingly, it is desirable to provide improved systems and methods for monitoring the position and movement of an object, particularly in a navigation system. Furthermore, other desirable features and characteristics of the present invention will become apparent from the subsequent detailed description of the invention and the appended claims, taken in conjunction with the accompanying drawings and this background of the invention.
BRIEF SUMMARY
In accordance with an exemplary embodiment, a system is provided for detecting a movement of an object marked with a marker. The system includes a sensor configured to capture a first image of the marker and to capture a second image of the marker after the first image, each of the first and second images including pixels each having a visual intensity; and a controller coupled to the sensor. The controller is configured to compare the first image and the second image by comparing the visual intensity of each of the pixels of the first image with the visual intensity of each of the pixels of the second image, determine an area of overlap between the first image and the second image based on the comparison, calculate a change in position of the marker in the second image relative to the marker in the first image based on the area of overlap, and detect the movement of the object based on the change in position of the marker.
In accordance with an exemplary embodiment, a method is provided for detecting a movement of an object marked with a marker. The method includes capturing a first image of the marker; capturing a second image of the marker, the first and second images comprising pixels, each with a respective visual intensity; comparing the first image and the second image by comparing the visual intensity of each of the pixels of the first image with the visual intensity of each of the pixels of the second image; determining an area of overlap between the first image and the second image based on the comparison; calculating a change in position of the marker in the second image relative to the marker in the first image based on the area of overlap; and detecting the movement of the object based on the change in position of the marker.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will hereinafter be described in conjunction with the following drawing figures, wherein like numerals denote like elements, and
<figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram of a system for detecting the movement of an object in accordance with an exemplary embodiment;
<figref idrefs="DRAWINGS">FIG. 2</figref> is a flowchart depicting a method for detecting the movement of an object in accordance with an exemplary embodiment;
<figref idrefs="DRAWINGS">FIG. 3</figref> is a first visual image of a marker captured by the system of <figref idrefs="DRAWINGS">FIG. 1</figref> in accordance with an exemplary embodiment;
<figref idrefs="DRAWINGS">FIG. 4</figref> is a second visual image of the marker captured later in time by the system of <figref idrefs="DRAWINGS">FIG. 1</figref> in accordance with an exemplary embodiment;
<figref idrefs="DRAWINGS">FIG. 5</figref> is a representation of an evaluation of the visual images of <figref idrefs="DRAWINGS">FIGS. 3 and 4</figref>; and
<figref idrefs="DRAWINGS">FIG. 6</figref> is a further representation of the evaluation of the visual images of <figref idrefs="DRAWINGS">FIGS. 3 and 4</figref>.
DETAILED DESCRIPTION
The following detailed description is merely exemplary in nature and is not intended to limit the invention or the application and uses of the invention. Furthermore, there is no intention to be bound by any theory presented in the preceding background or the following detailed description.
Broadly, exemplary embodiments discussed herein provide system and methods for detecting the movement and tracking the position of an object. Particularly, the object is marked or overlaid with a reference marker, for example, a circular reference marker. A sensor captures a grayscale image of the marker, and a controller identifies the marker and its position. The sensor subsequently captures a second grayscale image to determine the change of position of the marker during the time period between the first and second images. For example, the comparison identifies the intersection points, particularly by identifying the intersecting arcs. The change in position is calculated from the length of the chord formed by the intersecting arcs. Finally, the movement and resulting position of the object is calculated from the change in position of the marker. Such systems and methods may be incorporated into inertial navigation systems or into any type of system that involves monitoring the position of any marked object.
<figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram of a system <b>100</b> for detecting the change in position of a movable object <b>150</b> in accordance with an exemplary embodiment. The system <b>100</b> includes a controller <b>110</b>, memory <b>120</b>, a sensor <b>130</b>, and a user interface <b>140</b>. As described below, the system <b>100</b> is configured to detect the movement of the object <b>150</b> by tracking the position of a known reference marker <b>160</b> etched, drawn, affixed or otherwise mounted or overlaid on the object <b>150</b>. In the discussion below, the term “marked” or “marker” may refer to any of these techniques. In one exemplary embodiment, the marker <b>160</b> is circular in the xy-plane, although other shapes may be provided. Additionally, although only marker <b>160</b> is shown, the object <b>150</b> may be provided with more than one marker <b>160</b> that collectively form a pattern.
Typically, the object <b>150</b> and system <b>100</b> are used in a controlled environment, and the object <b>150</b> may generally move only relatively small distances, e.g., relative to dimension of marker <b>160</b> between sequential frames captured by the sensor <b>130</b>, as discussed below. However, the system <b>100</b> may also monitor any type of object <b>150</b>, including those in an unpredictable environment.
In one exemplary embodiment, the system <b>100</b> and object <b>150</b> may be part of the larger inertial navigation system (INS) that uses motion sensors (e.g., accelerometers) and rotation sensors (e.g., gyroscopes) to continuously calculate the position, orientation, velocity, and acceleration of a vehicle, such as ships, aircraft, submarines, guided missiles, and spacecraft. In other embodiments, the system <b>100</b> may be part of any position detection device for detecting the change in position of any object <b>150</b> marked with the reference marker <b>160</b>. Examples of suitable applications may include a near-to-eye head mounted display that tracks the position of the head of the user and thus the line of sight, a manufacturing assembly line that tracks products, a navigation system that tracks the position of an approaching marked object, or a testing system that monitors the oscillation of a slip table.
The controller <b>110</b> generally includes one or more processing units configured to implement the functions described herein. Particularly, and as discussed below, the controller <b>110</b> is configured to determine the movements and resulting positions of the marker <b>160</b> and the object <b>150</b> based on the image data from the sensor <b>130</b>. Additionally, if the system <b>100</b> is part of navigation processing, the controller <b>110</b> may further include a kinematic state estimation module (not shown) for estimating the kinematic state (e.g., position, velocity, and acceleration) of a vehicle based on the changes in position of the object <b>150</b>. In one exemplary embodiment, the system <b>100</b> may be applied to sense the relative motion of subassemblies (e.g., as an object <b>150</b>) internal to the navigation systems to provide relative attitude information to the state estimation. In another exemplary embodiment, the system <b>100</b> may be applied to sensing the external environment (e.g., in which objects <b>150</b> in the environment are marked) to provide rate estimates to aid the state estimation.
The controller <b>110</b> also has access to data, such as program instructions or computational data, from memory <b>120</b>. As examples, the memory <b>120</b> may store known characteristics of the object <b>150</b>, such as shape and dimensions; known characteristics of the marker <b>160</b>, such as shape and dimensions; and known characteristics about the relationship between the marker <b>160</b> and the object <b>150</b>, such as the position of the marker <b>160</b> on the object <b>150</b>. The memory <b>120</b> may include any suitable type of memory or data storage, such as for example, RAM, ROM, EEPROM, flash memory, CD, DVD, or other optical storage, magnetic storage devices, or any other medium that can be used to store and access desired information by the controller <b>110</b> and other portions of the system <b>100</b>.
The sensor <b>130</b> is an optical sensor configured to capture images of the marker <b>160</b> within a field of view (FOV) <b>132</b>. For example, the sensor <b>130</b> may be a charge-coupled device (CCD), indium gallium arsenide (InGaAs), PIN (p-type/intrinsic/n-type structure) diode array or other type of camera or image capturing device sensing in a spectrum able to detect marker <b>160</b>, including in an optical spectrum not visible to the human eye. As discussed below, the sensor <b>130</b> may have a resolution suitable for capturing grayscale images of the marker <b>160</b>. The sensor <b>130</b> may capture the images at a sample rate selected or otherwise controlled by the controller <b>110</b> for subsequent evaluation of the position of the marker <b>160</b>. For example and as discussed in greater detail below, the sensor <b>130</b> may capture a first image at a first time (t<sub>0</sub>), a second image at a second time (t<sub>1</sub>), and additional images at subsequent times (t<sub>n+1</sub>), each of which are sequentially separated by a known period of time (dt). The sample rate may be based on, for example, the anticipated movement of the object <b>150</b>, the dynamics of the associated vehicle, or the processing resources allocated to the system <b>100</b>. In one exemplary embodiment, the sample rate is selected such that the predicted change in position of the marker <b>160</b> does not exceed a distance of greater than a radius of the marker <b>160</b> during the elapsed time period (dt) between sequential images. Other design parameters that may be selected under these considerations include the dimensions of the marker <b>160</b> and the selected time period
Although not shown, the sensor <b>130</b> may be pivotable such that the FOV <b>132</b> may be adjusted. Moreover, the sensor <b>130</b> may have a “zoom” feature that enables the system <b>100</b> to adjust the size of the FOV <b>132</b>. In one exemplary embodiment, the controller <b>110</b> may adjust the FOV <b>132</b> of the sensor <b>130</b> based on predictions of the change in position of the marker <b>160</b> such that the marker <b>160</b> does not move outside of the FOV <b>132</b> during one or more sequential images.
The sensor <b>130</b> or the system <b>100</b> may include further optical components for capturing and processing the visual images. For example, the system <b>100</b> may include illumination sources, mirrors, light pipes, fiber optics, and magnifying devices.
The system <b>100</b> further includes the user interface <b>140</b>. The user interface <b>140</b> is any component, including hardware and software, that enables the user to interact with the system <b>100</b>, particularly the controller <b>110</b>. Such components may include keyboards, mouse devices, buttons, switches, levers, and knobs. In one exemplary embodiment, the user interface <b>140</b> may include a display, such as a touch screen display, for displaying the images captured by the sensor <b>130</b> and other visual data representing the position or movement of the object <b>150</b>.
<figref idrefs="DRAWINGS">FIG. 2</figref> is a flowchart depicting a method <b>200</b> for detecting the position of an object in accordance with an exemplary embodiment. Particularly, the method <b>200</b> may be implemented with the system <b>100</b> of <figref idrefs="DRAWINGS">FIG. 1</figref> to determine the changes in the position of the marker <b>160</b>, and thus, the changes in the position and the resulting positions of the object <b>150</b> over time. As such, <figref idrefs="DRAWINGS">FIG. 2</figref> will be described with reference to <figref idrefs="DRAWINGS">FIG. 1</figref> as well as <figref idrefs="DRAWINGS">FIGS. 3-6</figref>, which illustrate the various steps of the method <b>200</b>.
In step <b>210</b> of the method <b>200</b>, the system <b>100</b> determines the initial position of the marker <b>160</b> on the object <b>150</b>, e.g., the position of the marker <b>160</b> at a first instance of time (t<sub>0</sub>). The initial position of the marker <b>160</b> may be determined in any suitable manner, including a known initial calibration of the object <b>150</b> and system <b>100</b>. In one exemplary embodiment, the system <b>100</b> determines the initial position of the marker <b>160</b> by capturing a first visual image of the object and calculating the initial position based on the visual image.
As one exemplary embodiment, <figref idrefs="DRAWINGS">FIG. 3</figref> is a first visual image <b>300</b> captured by the sensor <b>130</b> and depicts a marker <b>360</b> on the object <b>150</b>. In <figref idrefs="DRAWINGS">FIG. 3</figref>, the visual representation of the marker <b>160</b> from <figref idrefs="DRAWINGS">FIG. 1</figref> is labeled as marker <b>360</b> to represent the initial position of the marker <b>160</b> (e.g., at time=t<sub>0</sub>) and to provide clarity in the subsequent discussions of the visual representations of the marker <b>160</b> at other time periods (e.g., at time=t<sub>1</sub>).
The sensor <b>130</b> may provide the visual image <b>300</b> to the controller <b>110</b>. In response, the controller <b>110</b> may process the visual image <b>300</b> to identify the marker <b>360</b>, for example, using pattern recognition, and to determine the position of the marker <b>360</b>. For example, the controller <b>110</b> may determine the center and boundaries of the marker <b>360</b>. The controller <b>110</b> may contain known characteristics of the marker <b>360</b> and the object <b>150</b> stored in memory to aid in this determination and/or use standard image processing algorithms for calculation, such a centroid determination or edge detection.
Particularly, the controller <b>110</b> may generate or evaluate the visual image <b>300</b> as a grayscale image. In other words, each of the pixels of the visual image <b>300</b> may have an associated visual intensity expressed as a quantifiable grayscale value. For example, each pixel of the visual image <b>300</b> may have a grayscale value that ranges from 0 to 255 (e.g., in an 8-bit code unit). Pure white may be quantified as a grayscale value of 0, and pure black may be quantified as a grayscale value of 255. In one exemplary embodiment, the marker <b>360</b> is black and the object <b>150</b> is otherwise white such that the pixels having a value of 255 represent the image area associated with the marker <b>360</b> and the pixels having a value of 0 represent the image area associated with portions of the object <b>150</b> not covered by the marker <b>360</b>. In a further embodiment, the controller <b>110</b> may implement threshold measurement and contrast algorithms to translate the non-ideal image in to a nearly ideal image in which only the pure dark and pure white values are represented. As such, this embodiment reduces the bit resolution to a grayscale value of 1 bit per pixel with 1 representing dark areas and 0 representing white or vice versa. The grayscale values may be stored in memory <b>120</b> for subsequent processing. Other mechanisms for quantifying the visual intensity of the marker <b>360</b> in the visual image <b>300</b> may be provided.
In step <b>220</b> of the method <b>200</b>, the system <b>100</b> captures a subsequent (or second) visual image of the marker <b>160</b> on the object <b>150</b>. <figref idrefs="DRAWINGS">FIG. 4</figref> is a second visual image <b>400</b> of the marker <b>460</b> on the object <b>150</b>. In <figref idrefs="DRAWINGS">FIG. 4</figref>, the marker <b>460</b> in the second visual image <b>400</b> is labeled as marker <b>460</b> to represent the visual image <b>400</b> of the marker <b>460</b> captured after the visual image <b>300</b> of <figref idrefs="DRAWINGS">FIG. 3</figref> (e.g., at time=t<sub>1</sub>).
The sensor <b>130</b> may provide the visual image <b>400</b> to the controller <b>110</b>. In response, the controller <b>110</b> may process the visual image <b>400</b> to quantify the visual image <b>400</b> in grayscale, particularly to represent the marker <b>460</b> grayscale, as described above. In one exemplary embodiment, the controller <b>110</b> does not use pattern recognition or other processing techniques to determine the identity and position of the marker <b>460</b> based only on the visual image <b>400</b>. Instead, only the grayscale values must be stored for further processing, in effect, as a “reduced image.”
In step <b>230</b> of the method <b>200</b>, the system <b>100</b> evaluates or otherwise compares the grayscale values of the first visual image <b>300</b> and the grayscale values of the subsequent, second visual image <b>400</b>. Particularly, the controller <b>110</b> may use a simple logical AND operation to compare the grayscale representations. For example, in the 1 bit per pixel image reduction discussed above, the result of the logical AND operation will be a 1 where both images <b>300</b> and <b>400</b> were dark and a 0 in all other cases. Alternatively, in the embodiment in which the 256-scale grayscale values are determined, the controller <b>110</b> may sum grayscale values of respective pixels in the visual images <b>300</b> and <b>400</b>. The resulting values of the summed pixels may be referred to as composite pixel values. <figref idrefs="DRAWINGS">FIG. 5</figref> is a visual image <b>500</b> representing the comparison between the visual images <b>300</b> and <b>400</b> of <figref idrefs="DRAWINGS">FIGS. 3 and 4</figref>.
Using the 256-bit grayscale example discussed above, the sum of the visual images <b>300</b> and <b>400</b> represented by visual image <b>500</b> includes portions <b>570</b> in which the marker <b>360</b> or <b>460</b> did not appear in either visual image <b>300</b> or <b>400</b> (e.g., composite pixel values near 0); portions <b>572</b> in which the marker <b>360</b> or <b>460</b> appeared in one of the visual images <b>300</b> or <b>400</b> (e.g., composite pixel values of about 255); and portions <b>560</b> in which the markers <b>360</b> and <b>460</b> overlap (e.g., large composite pixel values of about 510). As shown in <figref idrefs="DRAWINGS">FIG. 5</figref>, the overlap portion <b>560</b> is represented by two intersecting arcs associated with the marker <b>360</b> (<figref idrefs="DRAWINGS">FIG. 3</figref>) and marker <b>460</b> (<figref idrefs="DRAWINGS">FIG. 4</figref>).
In step <b>240</b> of the method <b>200</b>, the system <b>100</b> evaluates the overlap portion <b>560</b> to determine the change in position of the marker <b>160</b> during the time period, e.g., the position of marker <b>460</b> in visual image <b>400</b> (<figref idrefs="DRAWINGS">FIG. 4</figref>) relative to marker <b>360</b> in visual image <b>300</b> (<figref idrefs="DRAWINGS">FIG. 3</figref>). <figref idrefs="DRAWINGS">FIG. 6</figref> is a further representation <b>600</b> to illustrate the calculation of the relative movement between visual images <b>300</b> and <b>400</b> of <figref idrefs="DRAWINGS">FIGS. 3 and 4</figref> based on the overlap portion <b>560</b>.
In one exemplary embodiment, the controller <b>110</b> identifies the intersecting arcs <b>600</b> and <b>602</b> that define the overlap portion <b>560</b>. Particularly, the intersecting arcs <b>600</b> and <b>602</b> define a chord <b>604</b> extending between the points of intersection <b>606</b> and <b>608</b> (e.g., x<sub>1</sub>,y<sub>1 </sub>and x<sub>2</sub>,y<sub>2 </sub>on a Cartesian coordinate system representing the xy plane in <figref idrefs="DRAWINGS">FIG. 6</figref>). Upon identification of the points of intersection <b>606</b> and <b>608</b>, the controller <b>110</b> calculates the length <b>610</b> of chord <b>604</b>, for example as represented by Equation (1) <br /><i>l</i>=√{square root over ((<i>x</i><sub>2</sub><i>−g</i><sub>1</sub>)<sup>2</sup>+(<i>y</i><sub>2</sub><i>−y</i><sub>1</sub>)<sup>2</sup>)}{square root over ((<i>x</i><sub>2</sub><i>−g</i><sub>1</sub>)<sup>2</sup>+(<i>y</i><sub>2</sub><i>−y</i><sub>1</sub>)<sup>2</sup>)} (1)
As discussed above, the diameter of the marker <b>160</b> is known. As such, the length <b>610</b> of the chord <b>604</b> corresponds to the distance that the marker <b>160</b> moved in the visual images <b>300</b> and <b>400</b> (e.g., as represented by markers <b>360</b> and <b>460</b>). A long chord <b>610</b> indicates that the marker <b>460</b> moved a relatively short distance during the elapsed time, and a relatively short chord <b>604</b> indicates that the marker <b>460</b> moved a relatively long distance during the elapsed time. For example, if the length <b>610</b> of the chord <b>604</b> is equal to the diameter of the marker <b>160</b>, it indicates to the controller <b>110</b> that the marker <b>460</b> did not move during the time period between images <b>300</b> and <b>400</b>. If, however, the length <b>610</b> of the chord <b>604</b> is very close to zero, it indicates to the controller <b>110</b> that the marker <b>460</b> has moved almost one diameter length from the initial position of marker <b>360</b>. In one exemplary embodiment, the distance moved may be derived from the chord length <b>610</b> based on tables stored in memory <b>120</b>. In another exemplary embodiment, the distance moved may be derived from the chord length <b>610</b> and known characteristics of the marker <b>160</b> such as its radius and initial position in the prior sensed image (e.g., marker <b>360</b>). One exemplary mechanism for determining the distance travelled is provided in Equation (2): <br />dist_travelled=√{square root over (<i>D</i><sup>2</sup><i>−l</i><sup>2</sup>)} (2),<br /> where D is the diameter of the marker <b>160</b> and l is the length of the chord <b>604</b>.
As such, the length <b>610</b> of the chord <b>604</b> determines the extent of movement between the visual images <b>300</b> and <b>400</b> (<figref idrefs="DRAWINGS">FIGS. 3 and 4</figref>). The direction of movement corresponds to the direction of a perpendicular vector <b>612</b> extending from the chord <b>604</b>. On exemplary mechanism for expressing the position change vector is provided in Equation (3):
<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><mi>positionchangevector</mi><mo>=</mo><mrow><mo>±</mo><mrow><mo>[</mo><mtable><mtr><mtd><mfrac><mrow><mrow><mo>-</mo><mrow><mo>(</mo><mrow><msub><mi>y</mi><mn>2</mn></msub><mo>-</mo><msub><mi>y</mi><mn>1</mn></msub></mrow><mo>)</mo></mrow></mrow><mo>*</mo><mi>dist_travelled</mi></mrow><mi>l</mi></mfrac></mtd></mtr><mtr><mtd><mfrac><mrow><mrow><mo>(</mo><mrow><msub><mi>x</mi><mn>2</mn></msub><mo>-</mo><msub><mi>x</mi><mn>1</mn></msub></mrow><mo>)</mo></mrow><mo>*</mo><mi>dist_travelled</mi></mrow><mi>l</mi></mfrac></mtd></mtr></mtable><mo>]</mo></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>3</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths>
Accordingly, the controller <b>110</b> determines the distance and direction of movement of the marker <b>460</b> based on the chord <b>604</b> of the overlap portion <b>560</b> during the time period (dt). Upon consideration of the initial position, the direction of movement, and the distance moved, the controller <b>110</b> may also determine the position of the marker <b>460</b> at the subsequent time (t<sub>1</sub>), as well as its rate of motion as a function of the time interval.
In step <b>250</b> of method <b>200</b>, the controller <b>110</b> determines the movement and subsequent position of the object <b>150</b> during the time period between the images <b>300</b> and <b>400</b> (<figref idrefs="DRAWINGS">FIGS. 3 and 4</figref>) based on the distance and direction of movement the marker <b>160</b> and the previously known relationship between the marker <b>160</b> and the object <b>150</b>. These relationships include but are not limited to the exact position of the marker on the object <b>150</b>, orientation of the marker <b>160</b> on the object <b>150</b>, relative position of the marker <b>160</b> to other similar or dissimilar markers and other sources of information embedded in marker <b>160</b> and separately decoded by the sensor <b>130</b> and controller <b>110</b> or similar supporting system. In embodiments with such encoding, details embedded in the marker itself may include shape, such as line thickness, characters, or bar codes, or spectral encoding such as color or luminosity in a spectrum not detected by, or filtered out of the image captured by, sensor <b>130</b>. The position and movement information may be resolved in either a vehicle reference frame or an absolute reference frame. In one exemplary embodiment, the position, velocity, and acceleration calculations based on the position information may be performed by a state estimation module for subsequent use in a navigation solution. For example, the velocity of the object <b>150</b> may be determined based on the distance moved over time between two or more images, and acceleration may be determined based on the change in velocity over time between two or more images.
In step <b>260</b> of method <b>200</b>, the controller <b>110</b> determines if the observation of the object <b>150</b> is complete. If the result of step <b>260</b> is yes, the method <b>200</b> concludes. However, if the observation of the object <b>150</b> is ongoing, the method <b>200</b> returns to step <b>220</b> and the system <b>100</b> captures a subsequent (or third) visual image of the marker <b>160</b>. Steps <b>230</b>-<b>260</b> continue as the third visual image is compared to the second visual image to determine the subsequent movement of the marker <b>160</b> and object <b>150</b>.
Although the system <b>100</b> and method <b>200</b> are described with respect to the movement of the object <b>150</b> in two dimensions (e.g., in the xy-plane), an additional sensor (not shown) may be provided to determine the movement of the object <b>150</b> in a third dimension. For example, the additional sensor may be arranged orthogonally relative to the other sensor <b>130</b> to determine movement of the object <b>150</b> in the xz-plane or the yz-plane such that the three-dimensional movement of the object <b>150</b> may be calculated. Similarly, the object <b>150</b> is depicted as being flat in the xy-plane. However, other object shapes, such as a rotating shaft may be monitored with a single sensor, or by adding additional sensors and/or additional markers to image a spherical surface or other complex shape.
Accordingly, the system <b>100</b> and method <b>200</b> enable monitoring of an object without physical contact between an optical sensor and the object. One advantage of such exemplary embodiments includes reduced processing resources as compared to conventional techniques such as those that require pattern recognition on each visual image to determine object or marker position.
Embodiments of the present invention may be described in terms of functional block diagrams and various processing steps. It should be appreciated that such functional blocks may be realized in many different forms of hardware, firmware, and or software components configured to perform the various functions. For example, embodiments of the present invention may employ various integrated circuit components, e.g., memory elements, digital signal processing elements, look-up tables, and the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Such general techniques are known to those skilled in the art and are not described in detail herein.
While at least one exemplary embodiment has been presented in the foregoing detailed description of the invention, it should be appreciated that a vast number of variations exist. It should also be appreciated that the exemplary embodiment or exemplary embodiments are only examples, and are not intended to limit the scope, applicability, or configuration of the invention in any way. Rather, the foregoing detailed description will provide those skilled in the art with a convenient road map for implementing an exemplary embodiment of the invention. It being understood that various changes may be made in the function and arrangement of elements described in an exemplary embodiment without departing from the scope of the invention as set forth in the appended claims.
Contents6
6 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6
Every citation, both waysCites: the store holds 31 of 32
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US2021270948A1 | Cited by | United States of America | Search report |
| US2017262552A1 | Cited by | United States of America | Search report |
| US11768281B2 | Cited by | United States of America | Search report |
| US10657183B2 | Cited by | United States of America | Search report |
| US12196891B2 | Cited by | United States of America | Search report |
| US11393102B2 | Cited by | United States of America | Search report |
| US2024270282A1 | Cited by | United States of America | Search report |
| US10762638B2 | Cited by | United States of America | Search report |
| US10410357B1 | Cited by | United States of America | Search report |
| US2017262552A1 | Cited by | United States of America | Search report |
| US2002076105A1 | Cites | United States of America | Search report |
| US2002091466A1 | Cites | United States of America | Applicant |
| US2008082468A1 | Cites | United States of America | Search report |
| US2008284790A1 | Cites | United States of America | Applicant |
| US2008288207A1 | Cites | United States of America | Search report |
| US2009059041A1 | Cites | United States of America | Search report |
| US2010100321A1 | Cites | United States of America | Applicant |
| US2010103323A1 | Cites | United States of America | Search report |
| US2011004405A1 | Cites | United States of America | Search report |
| US2011178658A1 | Cites | United States of America | Search report |
| US2011280454A1 | Cites | United States of America | Search report |
| US2012169871A1 | Cites | United States of America | Search report |
| US2012170865A1 | Cites | United States of America | Search report |
| US4238828A | Cites | United States of America | Applicant |
| US4281342A | Cites | United States of America | Applicant |
| US4297725A | Cites | United States of America | Applicant |
| US4805127A | Cites | United States of America | Search report |
| US4887223A | Cites | United States of America | Applicant |
| US5521843A | Cites | United States of America | Applicant |
| US5596365A | Cites | United States of America | Applicant |
| US5767922A | Cites | United States of America | Search report |
| US6321137B1 | Cites | United States of America | Applicant |
| US6380986B1 | Cites | United States of America | Applicant |
| US6616244B2 | Cites | United States of America | Applicant |
| US6954544B2 | Cites | United States of America | Search report |
| US7184586B2 | Cites | United States of America | Applicant |
| US7231063B2 | Cites | United States of America | Applicant |
| US7430455B2 | Cites | United States of America | Applicant |
| US7437226B2 | Cites | United States of America | Applicant |
| US7702420B2 | Cites | United States of America | Applicant |
| US7765027B2 | Cites | United States of America | Applicant |
| Guna Seetharaman and Ha V. Le, "Video-Assisted Global Positioning in Terrain Navigation with Known Landmarks" International Journal of Distributed Sensor Networks, vol. 2 (2006), Issue 2, pp. 103-119doi:10.1080/15501320500201235. | Non-patent | – | Search report |
| Weisstein, Eric W. "Circle-Circle Intersection." From MathWorld-A Wolfram Web Resource. http://mathworld.wolfram.com/Circle-CircleIntersection.html, Date Accessed Oct. 26, 2013, Internet Archive Capture from Apr. 2, 2007, Available Online at: https://web.archive.org/web/20070402085130/http://mathworld.wolfram.com/Circle-CircleIntersection.htm. | Non-patent | – | Search report |
2 members in 1 office
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 201113093548 | United States of America | A | |
| US201113093548 | – | – | – |
Members2
| Document | Office | Kind | |
|---|---|---|---|
| US2012269387A1 | United States of America | A1 | |
| US8737685B2This record | United States of America | B2 |
49 transactions on the USPTO file
Allowed after 1 non-final rejection and 1 final rejection.
- Non-final rejections
- 1
- Final rejections
- 1
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Payment of Maintenance Fee, 12th Year, Large EntityM1553 | M1553 | |
| 7.5 yr surcharge - late pmt w/in 6 mo, Large EntityM1555 | M1555 | |
| Payment of Maintenance Fee, 8th Year, Large EntityM1552 | M1552 | |
| Maintenance Fee Reminder MailedREM. | REM. | |
| Payment of Maintenance Fee, 4th Year, Large EntityM1551 | M1551 | |
| Recordation of Patent Grant MailedPGM/ | PGM/ | |
| Patent Issue Date Used in PTA CalculationAllowedPTAC | PTAC | |
| Email NotificationEML_NTR | EML_NTR | |
| Issue Notification MailedAllowedWPIR | WPIR | |
| Dispatch to FDCD1935 | D1935 | |
| Application Is Considered Ready for IssuePILS | PILS | |
| Issue Fee Payment VerifiedN084 | N084 | |
| Issue Fee Payment ReceivedIFEE | IFEE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Printer Rush- No mailingTCPB | TCPB | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Notice of AllowanceAllowedMN/=. | MN/=. | |
| Notice of Allowance Data Verification CompletedAllowedN/=. | N/=. | |
| Reasons for AllowanceEX.R | EX.R | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Final ActionA.NE | A.NE | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Final Rejection (PTOL - 326)Final rejectionMCTFR | MCTFR | |
| Final RejectionFinal rejectionCTFR | CTFR | |
| Date Forwarded to ExaminerFWDX | FWDX | |
| Response after Non-Final ActionA... | A... | |
| Electronic ReviewELC_RVW | ELC_RVW | |
| Email NotificationEML_NTF | EML_NTF | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Email NotificationEML_NTR | EML_NTR | |
| PG-Pub Issue NotificationPG-ISSUE | PG-ISSUE | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Application Is Now CompleteCOMP | COMP | |
| Email NotificationEML_NTR | EML_NTR | |
| Email NotificationEML_NTR | EML_NTR | |
| Change in Power of Attorney (May Include Associate POA)PA.. | PA.. | |
| Filing ReceiptFLRCPT.O | FLRCPT.O | |
| Sent to Classification ContractorPGPC | PGPC | |
| Cleared by OIPE CSRL194 | L194 | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Electronic Information Disclosure StatementEIDS. | EIDS. | |
| Applicants have given acceptable permission for participating foreignAPPERMS | APPERMS | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| IFW Scan & PACR Auto Security ReviewSCAN | SCAN | |
| Initial Exam Team nnIEXX | IEXX |
7 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Maintenance fee paymentMAFP | MAFP | |
| Fee payment procedure7.5 YR SURCHARGE - LATE PMT W/IN 6 MO, LARGE ENTITY (ORIGINAL EVENT CODE: M1555); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Maintenance fee paymentMAFP | MAFP | |
| Fee payment procedureMAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP | FEPP | |
| Maintenance fee paymentMAFP | MAFP | |
| Information on status: patent grantGrantedPATENTED CASESTCF | STCF | |
| AssignmentAS | AS |
Numbers
- Publication
- 08737685
- Publication, DOCDB
- 8737685
- Publication, EPODOC
- US8737685
- Application
- 13093548
- Application, DOCDB
- 201113093548
- Application, EPODOC
- US201113093548
Titles
- English
- Systems and methods for detecting the movement of an object
Patent term adjustment
- A delay
- +296 daysthe office missed an examination deadline
- B delay
- +32 dayspendency past three years
- Net adjustment
- 328 days
Classification
- CPC, 4
- G06T7/254
- G06T2207/10016
- G06T2207/30204
- G06T2207/30248
- IPC, 1
- G06K9 00
- USPC, 1
- 382103000