Enhancement of aimpoint in simulated training systems

Enhancement of aimpoint in simulated training systems

Reference to related applications

This application claims priority to co-pending U.S. Provisional Application No. 60/521,877, filed on July 15, 2004 by Page entitled "Aiming Point Enhancement by Extrapolated Shift", which provisional application is incorporated herein by reference. do.

The present invention relates to an improvement of a target tracking system, particularly for tracking multiple targets within a video camera based weapon aimpoint tracking system.

As simulated training becomes more important, significant efforts are underway to fabricate and improve simulated training systems. In particular, in order to provide efficient combat training, it is very important to track the target, since any training value is reduced without precise target training since the trainee cannot accurately detect whether various techniques improve aiming or not.

Many methods have been used in the past to track goals. Simply by way of example, some systems have used video cameras to track a target. Some systems have used the luminance and/or chromaticity of the imaged video, for example, to determine where a target appears in the video image. Such systems may calculate a center of gravity for an image with reference to a confined area, such as by matching pattern values with chromaticity and/or luminance, wherein the pattern values match patterns to flash tones and/or other applicable patterns. can do it Another type of video tracking system relies on a specific window to isolate target areas to determine a target. Analog comparison techniques may be used to perform tracking.

In other systems, a live digitized image is compared to a digitized background image. Based on the pixel differences, a center of gravity with respect to the center of a large amount of difference images is calculated. The velocity of the differential image can be calculated using video frame differences. Other systems use correlations between gated regions on successive frames to match positions on the moving region.

Another set of tracking systems uses digital spatial correlation to suppress false target signals fed to a point target tracking device. The search field of the tracking system is partitioned into a matrix consisting of rows and columns of elementary fields of view. Each base field of view is checked to determine if the target is within that field of view, and matrix neighbors are compared to determine if the target signal is within adjacent basic fields of view. The system rejects a signal if its adjacent matrix neighbor contains a signal.

In some tracking systems, a video processor is coupled to a television camera and limits the system in response to signals indicative of an internal intensity profile of possible targets. The digital processor is responsive to the video processor output signals to determine a difference between the angular position of the designated object and a pre-stored estimate of that position, and updates the stored position. The update function is normalized by the target image determination so that the tracking response of the system is independent of the target image size. The video processor unit estimates signals that do not represent a specified target based on the signal amplitude comparisons. Digital logic circuits differentiate between design goals and false goals based on angular position.

However, these systems do not provide sufficient precision when determining targets, particularly when tracking a target target in the field of multiple targets. Moreover, it is difficult for conventional systems to efficiently predict the location of targets, especially during periods of intense movement.

Accordingly, embodiments of the present invention provide improved systems and methods for tracking goals in a simulation environment. By way of example simply, a typical embodiment provides a reflective laser target tracking system that uses a video camera and associated computational logic device to track a target. In some embodiments, a closed-loop algorithm may be used to predict the positions of targets in advance based on formulas derived from conventional tracking points. Therefore, the next position of the target can be predicted. In some cases, goals may be filtered and/or ordered based on predicted positions. In some embodiments, the equations (a linear equation and a quadratic equation) may be derived from one or more video frames. These equations may be applied to one or more consecutive frames of video received and/or generated by the system. In some embodiments, these formulas may be used to compute predicted positions relative to targets, which in some cases may compensate for inherent delays in the processing pipeline.

Accordingly, one set of embodiments provides systems for tracking goals. A typical system may include a video display system that may be configured to display an image of a target and/or at least one simulated weapon and project an aiming point onto the displayed image. In some embodiments, the system further comprises a video acquisition system configured to acquire a plurality of video frames. Some or all of the video frames may include at least one aimpoint.

Some systems include a computer system capable of communicating with a video capture system. A computer system includes a processor and a computer-readable storage medium having instructions executable by the processor. The instructions may be executable to analyze each of the plurality of video frames to determine a location of the aiming point in each of the plurality of video frames. The instructions may further be executable to determine a formula for movement of the aim point, the formula may in some cases be satisfied by a position of the aim point in at least one of the plurality of video frames. In some embodiments, the formula may be a quadratic formula that may take into account the location of the aiming point, the speed of the aiming point, and the acceleration of the aiming point.

Using at least this formula, the position of the aimpoint in subsequent video frames can be predicted. In certain embodiments, the aim point may be identified in a subsequent video frame based on the predicted position of the aim point. In other embodiments, such an aiming point may be distinguished from a plurality of aiming points.

In some cases, predicting the location of the aim point in a subsequent video frame comprises predicting the location of the aim point in a second subsequent video frame (again using the formula) and at least one known position of the aim point in at least the current video frame. and averaging the predicted position of the aim point in the second subsequent video frame. The position of the aim point in the first subsequent video frame may be predicted based on an average of the at least one known position of the aim point and the predicted position of the aim point in the second subsequent video frame. Subsequent video frames need not be contiguous (ie, there may be one or more intervening frames between the current frame, the subsequent frame and the second subsequent frame).

Another set of embodiments provides methods including, but not limited to, a method for tracking target aim points. In a typical method, a computer may receive a plurality of video frames (from a video acquisition system). Each frame of the plurality of video frames may include at least one aiming point projected by the at least one simulated weapon. In some embodiments, the computer analyzes each frame of the plurality of video frames to determine a location of the aiming point in each frame of the plurality of video frames. In other embodiments, the computer determines a formula for movement of the aim point, the formula may be generally satisfied by the location of the aim point in at least one of the plurality of video frames. The computer may be a computer that tracks an aiming point across a plurality of video frames.

Another exemplary embodiment provides a method for predicting the location of an aim point. The method, in some embodiments, comprises a computer receiving (from a video capture system) a plurality of video frames, each of the plurality of video frames comprising at least one weapon projected by the simulated weapon. The plurality of video frames may include a current video frame and at least one past video frame. The computer, in some embodiments, analyzes each of the plurality of video frames to determine a plurality of known positions of the aimpoint. The plurality of known positions of the aimpoint includes a current known position of the aimpoint in a current video frame and/or at least one historically known position of the aimpoint in at least one past video frame. In some embodiments, the computer is configured to determine the point of aim in the first subsequent video frame by, for example, averaging, using a weighted average, at least one of a plurality of known positions of the aimpoint and the predicted position of the aimpoint in the second subsequent video frame. predict the location The average may be a weighted average that may depend on the speed of the aiming point and/or the acceleration of the aiming point.

By way of example simply, if the aimpoint moves at a relatively high speed, the plurality of historically known positions of the aimpoint includes a relatively large number of historically known positions of the aimpoint, and if the aimpoint moves at a relatively slow speed, If so, the plurality of historically known locations of the aimpoint includes a relatively small number of historically known locations of the aimpoint. As another example, if the speed of the aim point is relatively slow, the weighted average weights the at least one known position of the aim point relatively high (eg, the predicted position of the aim point is the second subsequent video frame).

As another example, if the speed of the aiming point is relatively slow, the second subsequent video frame may be relatively temporally close to the first subsequent video frame, and if the speed of the aiming point is relatively fast, the second subsequent video frame may be related to the first subsequent video frame. It may be relatively temporally distant from the video frame (ie, when the aimpoint moves at a relatively high speed, the method can be extrapolated into the future to compute the predicted position, which can be used for averaging).

Other embodiments provide systems including, but not limited to, a system configured to perform the methods of the present invention. Still other embodiments provide software programs including programs embodied on one or more computer-readable media. Some of these programs may be executable by computer systems for carrying out the methods of the present invention .

A further understanding of the nature and advantages of the present invention may become more apparent when reference is made to the following detailed description, and drawings in which like reference numbers indicate like elements throughout. In some examples, a sublabel is associated with a reference sign and is delimited by a hyphen to indicate one of a plurality of similar elements. When a reference number is given without a description with respect to an existing sublabel, the reference number refers to all pluralities of like elements.

1 is a schematic diagram illustrating structural components of a system for tracking goals in accordance with various embodiments of the present invention;

2 is a general schematic diagram illustrating a computer system in accordance with various embodiments of the present invention;

3 is a block diagram illustrating functional components of a system for tracking goals in accordance with various embodiments of the present invention;

4 is a flow diagram illustrating a method for tracking aim points in accordance with various embodiments of the present invention.

5 is a flow diagram illustrating a method for estimating predicted aim points in accordance with various embodiments of the present invention.

6 is a diagram illustrating a center of an aiming point in accordance with various embodiments of the present invention.

7 illustrates an aiming point reflection superimposed on a video image in accordance with various embodiments of the present invention.

Accordingly, embodiments of the present invention provide improved systems and methods for tracking goals in a simulation environment. By way of example simply, exemplary embodiments provide a reflective laser target tracking system that uses a video camera and associated computational logic to track a target. In some embodiments, a closed loop algorithm may be used to predict future positions of targets based on formulas derived from previous tracking points. Therefore, the next position of the target can be predicted. In some cases, goals may be filtered and/or ordered based on predicted positions. In some embodiments, equations (including, but not limited to, linear equations and quadratic equations) may be derived from one or more video frames. These equations may be applied to one or more consecutive frames of video received and/or generated by the system. In some embodiments, these formulas may be used to compute predicted positions for targets, which in some cases compensate for inherent delays in the processing pipeline.

In one aspect, some embodiments of the present invention provide improved methods and systems for determining the weapon aiming point of small arm trainers. In many cases, small-armed trainers use visible or infrared ("IR") lasers mounted on the gun bore of a training weapon to illuminate targets on a projected image of the firing range. Using feedback from the camera (or any other suitable video capture device), the computer system can identify points of sight from various weapons, and in some cases a location at which one or more points may appear in future video frames. Prediction of , which facilitates identification, as well as tracking of aim points in future frames. Thus, in some embodiments, the aimbeam radiation of the various weapons may not need to be pulsed to identify each weapon (however, in other embodiments the radiations may be modulated (i.e., , can be turned on or off)). By way of example simply, in some embodiments radiation from one or more weapons is modulated at a frequency equal to the frame rate of the video capture system that can easily track and/or identify the aiming points.

An example of a simulation system 100 is shown in FIG. 1 . System 100 includes one or more simulated weapons 105b, each weapon having an aiming beam (frequency visible to the human eye) that can be detected by suitable detection equipment as described below. or a device configured to radiate (which may be an invisible frequency such as IR). The aiming beam can be thought of as representing an assumed trajectory along which the projectile travels when fired from a simulated weapon (ignoring ballistic and environmental effects in some cases). In some cases, the simulated weapon may be (but not necessarily) a real weapon that has been reconfigured to emit an aiming beam.

The system 100 includes a projector 100 configured to display one or more targets on a projection screen 115 (the display is a moving image, eg, a video image of one or more moving targets, and/or a still image, eg, of one or more targets). may be photos). When the simulated weapon 105 is aimed at the projection screen 115 and emits an aiming beam, the intersection of the projection screen 115 and the aiming beam creates an aiming point, which is directed from the simulated weapon 105 . It is the point at which the projected assumed trajectory intersects the projection screen 115 (in some cases ignoring ballistic and/or environmental effects). The aiming point can then be detected as a reflection of the aiming beam from the projection screen 115 as described in more detail below.

The system may include a stationary digital imaging device (eg, a digital video camera, or any other device including a photovoltaic cell, one or more charge coupled devices ("CCD) or similar technology), and/or one or more aiming points on the projection screen 110 . It further comprises a video capture device 120, which may be any other suitable device capable of capturing video images (or still images having a sufficiently high speed) to record the location of the video. The acquisition device may be a progressive scanning video camera that avoids the problems associated with frame interleaving (described in more detail below).

The video capture device 120 is arranged to capture a video image (and/or a series of still images) of the projection screen. In one aspect of the present invention, the video capture device 120 is configured to sense the frequency of the aiming beam/point of aim (which may or may not be visible depending on the embodiment). In some embodiments, video capture device 120 may be configured to sense visible frequencies such that a displayed target image (usually visible) is captured by video capture device 120 (in other embodiments). For example, when the collimating beam is not at a visible frequency, the plurality of video capture devices 120 determines that one (or more than one) of the devices 120 captures the target image and one (or more) of the devices 120 . more than one) device may be used to capture the aiming points).

The video capture device 120 is in communication with a tracking computer 125 that may be used to perform the methods of the present invention, as will be described in detail below. In certain embodiments, for example, the tracking computer 125 determines where the weapon 105 points to the projected image by identifying the location of the aiming point. By way of example simply, the tracking computer 125 captures an image (such as one or more video frames) from the video capture device 120 and then executes threshold tests and/or central calculations (eg, as described in more detail below). can be used to determine the location of the aiming points on the projection screen 115 in a particular video frame (or still image). The position may be an absolute position relative to the capture frame of the projection screen 115 and/or the video capture device 120 and/or may be relative to the projected target image.

In one set of embodiments, each of the simulated weapons 105 communicates with a data interface device 130 that provides data communication between the weapons 105 and the estimation computer 125 . In this way, for example, the weapon 105 can be set with details about the weapon 105, such as when the weapon emits an aiming beam when the trigger is pulled, the aiming beam (aiming beam frequency, modulation frequency, etc.) details and the like may be communicated to the tracking computer 130 . Likewise, in some embodiments, the tracking computer 130 may communicate with the weapon 105 to instruct the weapon 105 to emit an aiming beam, aiming beam frequency and/or modulation frequency, etc. for use.

The term "collimation beam frequency" refers to the frequency of light emitted from a weapon, while "modulation frequency" refers to the frequency at which the aiming beam is modulated or pulsed. In some cases, the aiming beam may be unmodulated, but in other cases it may be modulated with a frequency matching the frame rate of the video capture device that can assist in identifying the aiming beams. In still other cases, different weapons 105a, 105b may be configured to modulate to different frequencies to identify their aiming beam. In many cases, however, this modulation difference may be unnecessary, as system 100 has the ability to identify aim points as described below.

In some embodiments, the system 100 is a display computer 135 that can be used to display goals, sights, etc. (or, for example, a still and/or video image of the goals), for example, to analyze trainee performance, etc. ) is further included. Display computer 135 may be configured for use as a control workstation for the system (although separate control workstations (not shown) and/or tracking computer 125 are not used). Display computer 135 may then communicate with tracking computer 135 (eg, via network 140 such as an Ethernet network, the Internet, an intranet, a wireless network, or any other suitable communication network) with the tracking computer 135 (other In some cases, display computer 135 and tracking computer 125 may be the same computer). Display computer 135 may in some cases be used to provide projected images to projector 110 . The display computer 135 may in some cases communicate with the data interface device(s) 130 directly to the projector 110 or via the tracking computer 125 , such that the data interface 130 may communicate with the tracking computer ( It can communicate with the weapons 105 in a manner similar to that described above for 125 .

In accordance with some embodiments, standard computers and/or devices are used in system 100 (eg, as tracking computer 125 , control workstation, display computer 135 and/or data interface device 130 ). be configurable and/or to implement the methods of the present invention. 2 depicts one embodiment of a computer system 200, which may typically be some of the computers/devices described above. Computer system 200 is shown to include hardware elements that may be electrically connected via bus 205 . The hardware elements may include one or more central processing units (CPUs) 210 , ie, one or more input devices 215 (eg, mouse, keyboard, etc.) and/or one or more output devices 220 (eg, a display device; printer, etc.). Computer system 200 may include one or more storage devices 225 . By way of example, storage(s) 225 may include disk drives, optical storage devices, random access memory ("RAM"), which may be programmable and/or flash-updatable, and/or read-only memory ("ROM"). ) may be a solid state storage device, such as

The computer system 200 includes a computer-readable storage medium reader 230 , ie, one or more communication system(s) 235 (serial and/or parallel ports, USB ports, IEEE 1934 ports, modems, network may include (but are not limited to) any suitable device-to-device communications equipment such as cards and/or chip sets (wireless or wired), infrared communications devices, and wireless communications systems including Bluetooth and the like. ) and working memory 240 , which may include, but is not limited to, the RAM and ROM devices described above. In some embodiments, computer system 200 may include a processing acceleration unit 245, which may include a DSP, special purpose processor, or the like.

The computer-readable storage medium reader 230 is configured with a remote, local, fixed and/or removable storage device plus a storage medium (optionally with a storage device) that temporarily and/or more permanently contain computer-readable information. s) 225 ) may be connected to a computer-readable storage medium 250 . Communication system(s) 235 may include networks (including, but not limited to, network 145 described above) and/or other computers and/or devices (described above with respect to system 100 ). data may be exchanged with (but not limited to) devices and computers.

The computer system 200 has a working memory including an operating system 255 (such as a part of an operating system available among UNIX or UNIX-like operating systems such as Microsoft Windows operating system, Linux, BSD, etc., mainframe operating system, etc.). software elements shown as currently disposed within 240 . The software elements may be one or more application programs (such as an application configured to perform procedures in accordance with embodiments of the present invention, as well as a client application, server application, web browser, web server, mid-tier application, RDBMS, etc.) may further include other codes 260 such as). Application programs may be designed to implement the methods of the present invention.

It should be appreciated that alternative embodiments of computer system 200 may have a number of variations other than those described above. For example, custom hardware may be used and/or certain elements may be implemented in hardware, software (including portable software such as applets), or both. In addition, connection to other computing devices such as data interface device 130 , projector 110 , video capture device 120 , network input/output devices, etc. may be implemented using any suitable standard and/or proprietary connections. can be

In one set of embodiments, the tracking computer is configured (in some cases) with a software application (and/or set of software applications) that enables the tracking computer to track and/or identify aim points and perform other methods of the present invention. , the software application(s) may be distributed to a plurality of computers that jointly perform functions embedded in the tracking computer). 3 shows a functional diagram of a tracking application 300 including various components that may be included in the application(s) in accordance with some embodiments (while specific functional structures are described by FIG. 3 , other Embodiments may feature different structures, and those skilled in the art will appreciate in the description set forth herein that the functions of the tracking server may be distributed as needed among any number of software components, applications and/or devices. will know based on it).

In the described embodiments, the tracker application 300 includes a video capture interface 305 that is used to capture video and/or still images from a video capture device. The video capture interface 305 is used to receive the image(s) and/or format the image(s) as needed for processing by the system. Application 300 includes an aim locator module 310 that scans image(s) for patterns that match possible aim points. In one set of embodiments, the target locator module searches for luminance and/or chromaticity values that match a set of pattern values (within a set of defined and/or configurable thresholds).

According to some embodiments, the radiative filtering module 315 applies a filtering algorithm to analyze patterns matching possible aim points. In one embodiment, the filtering algorithm compares the algorithms with the expected shape so that patterns that do not match the expected shape and/or size of the aimpoint can be ignored as artifacts of the video capture process (although the described implementation Although in the examples it is expected that the aim points are generally radial, other embodiments may use aim points having other shapes and suitable filtering algorithms may be used to filter based on these shapes).

Patterns that are not ignored (or filtering module, in embodiments that do not use all identified patterns) may be considered valid aimpoints. For one or more of the aiming points, the central locator 320 is located at the center of the aiming point to precisely determine the location of the center of the aiming point. One exemplary embodiment for determining the center is described below with respect to FIG. 6 . Other procedures may also be used. The position of the center may then be determined as the position of the aiming point.

In some embodiments, spatial transformer 325 may be used to correct for defects in the image capture process. As a simple example, lens distortion may exist in a video capture device such that the captured image does not accurately reflect the actual position of the aiming point on the projected image. The spatial transformer may then be one or more formulas (or arrays of formulas) that correct for these imperfections. In many cases, the formula(s) may be determined and/or corrected based on empirical comparisons of the recorded aim point position and the actual aim point position of a particular system. Accordingly, these formulas (and/or their coefficients) may be system specific (of course, depending on the spatial characteristics of the video capture device, the spatial converter 325 may not be necessary).

After performing any necessary spatial transformations, the recorded aimpoints are identified using the predicted position window 350 (described in more detail below) as an aimpoint identifier ( 330). 4, described below, describes one exemplary method for identifying an aiming point. Once an aim point has been identified, the aim point's location is added to the aim point history log 335 for this aim point. Aim history log 335 (which may be stored in a database, flat file, and/or any suitable data structure) tracks the historical locations (eg, by X, Y coordinates) of each individual aim point.

Based on the aimpoint history, the formula generator 340 generates a formula describing the movement of a particular aimpoint. In a particular set of embodiments, the formula describing the movement of the aimpoint takes the form

aT^2 + bT + c formula (1)

where T is the time value (which may be calculated based on the frame rate of captured video and/or still images), a is acceleration (eg pixels per frame), and b is velocity (pixels per frame) , and c is the original position of the tracked aimpoint. In some embodiments, the movement formula is derived by fitting a curve to a selected number of historical aimpoint positions (ie, positions stored in the aimpoint history log 335 ) for the tracked aimpoint. As described in more detail below, following some embodiments, the number of historical aimpoints used when generating the formula may vary (eg, based on the current velocity of the aimpoint and/or other factors). In many embodiments, the formula describing the position of any particular aimpoint is updated continuously (ie, in every captured frame) or frequently.

In a set of embodiments, the formula generated with the current position of the aimpoint (possibly transformed position) predicts that the location of the aimpoint will be found in subsequent frames (which may be, but not necessarily consecutive frames). may be used to determine the predicted location window 350 which is the radius from the current location aim point. This predicted position window may be used to identify and/or track an aim point of a subsequent frame, as described in more detail below.

In a set of embodiments, the generated formula and/or the aim point history determines how to weight the average of the set of historical positions for the aim point and the predicted position of the aim point in future frames to more accurately predict the aim point's pre-position. may be provided to a variable-width averaging module 345 that may As a simple example, if the current velocity of the aiming point is relatively slow, relatively more historical values may be considered averaged, and/or historical values may be weighted relatively higher (higher weights on average) given), a relatively small number of historical positions may be considered when the current velocity of the aiming point is relatively fast, and/or the historical positions may be given a low weight to the average value.

This is because if the current velocity is relatively low, the trainee will focus their aiming on one point, and changes in velocity or acceleration may be artifacts of the video capture system and/or involuntary movements of the weapon so that these changes are relative predictive values. It is based on the principles that make it possible to have less Conversely, if the current velocity is relatively high, the trainee traverses the weapon's aim, making the movement formula relatively more predictable.

5 depicts an exemplary procedure for performing weight determination in accordance with some embodiments of the present invention. In some cases, this weighted average may be used to determine the predicted location window as described above.

While typical functional components are described by FIG. 3 , those skilled in the art will appreciate that various similar functional components may be substituted for the components described in detail above and/or the configuration of the above-mentioned components may be modified as appropriate. you will realize that you can In certain embodiments, for example, certain functions do not need to be performed and may be omitted from tracking applications.

Another set of embodiments provides methods including, but not limited to, a method of identifying and/or tracking one or more aimpoints. 4 describes an exemplary method 400 in accordance with some embodiments. The methods of the present invention, including but not limited to the methods described by FIGS. 4 and 5 , are performed by various components (eg, a tracking computer) of the system 100 described with respect to FIG. 1 . However, it should be appreciated that other structural systems may be used in other embodiments. Accordingly, the methods of the present invention are not limited by any particular apparatus or systems.

The method 400 of FIG. 4 includes acquiring a video frame (block 405). As described above, capturing a video frame (which may be a still image in alternative embodiments) includes recording the video frame using a video capture device and/or recording using a video capture interface of a software application. (in an aspect, a video capture device, a video capture interface, and/or a combination thereof may be considered a video capture system). Other procedures may also be used. Generally, a video frame will contain at least one aimpoint (ie, the captured image includes a representation and reflection of the aimpoint). At block 410, one or more possible aiming points are identified. In a set of embodiments, as described above, a pattern-matching algorithm may be used to compare luminance and/or chromaticity values of various regions of a captured frame to identify possible aimpoints.

The video frame is analyzed to determine the location of the aimpoint (block 415). In a set of embodiments, determining the location of the aim point comprises filtering possible aim points to ignore artifacts, locating the center of the aim point and/or assigning the location of the center as the location of the aim point, and and/or one or more procedures including, but not limited to, performing any necessary spatial transform (eg, to correct for any imperfections in the video capture system).

In a set of embodiments, the method 400 further comprises storing the location of the aimpoint. In some embodiments, the aim point position is stored in the aim point history log together with an identifier in the order of occurrence (which may be a time stamp, frame number, etc.) that may allow analysis of the aim point position over a period of time. The procedures in blocks 405 - 420 may then be repeated to acquire a plurality of video frames and/or develop a historical record of aimpoint positions in each frame of the plurality of video frames.

At block 425, a formula is determined for the movement of the aimpoint. As described above, in some embodiments, the formula is a quadratic formula, such as Equation (1). In some embodiments, the formula is determined by curve-fitting a selected number of historical positions of the aimpoint (which may or may not include the current position of the aimpoint). In one aspect, the formula may be satisfied by the location of the aimpoint in one or more previously recorded video frames in general.

In block 430, a future location of the aim point is predicted. The prediction may be based on a formula for the movement of the aim point, the known position of the aim point in one or more previous frames and/or the current frame, and/or an average thereof. In some cases, as described above, the average is a weighted average, and/or the weighting may depend on a movement of the aimpoint (eg, velocity of the aimpoint, acceleration of the aimpoint, etc.). 5 describes an exemplary embodiment of a method for calculating a predicted position based on a weighted average.

In some embodiments, the aimpoint may be tracked (block 435). Simply by way of example, the aim points in each frame of a series of video frames may be used to evaluate a trainee by tracking the aim point over a period of time.

In other embodiments, an aiming point may be identified (block 440). Once the predicted position of the aim point is established, the predicted position can be used to identify the aim point in subsequent frames. For example, as noted above, in some cases a predicted position window may be computed for subsequent frames. Aiming points that fall within the window in subsequent frames can then be identified as tracked aimpoints. Moreover, if necessary and/or appropriate (eg, if multiple aim points are within the predicted position window), the aim points from two or more subsequent frames may be analyzed to determine a movement pattern. If this movement pattern satisfies the movement formula for the tracked boresight, the boresight in subsequent frames may be satisfied as the tracked boresight.

In some cases, the predicted position of the aimpoint is used to distinguish one of the plurality of aimpoints (block 445). For example, if there are a plurality of aim points disposed in a given frame, the aim point belonging to the predicted location window may be distinguished from the aim points belonging to the predicted location window (as the tracked aim point).

As mentioned above, in many cases the weighted average of the travel formula for the aim point and the aim point's location history can be used to predict the future location of the aim point. 5 depicts an exemplary method 500 that can determine the weighted average that should be used. The method 500 includes maintaining a history of aimpoint positions for a particular aim (block 505). A history of the aiming point positions may be maintained in the aiming point history as described above.

In block 510, a history averaging depth is determined. The historical averaging depth describes how many of the known aim positions should be used to generate the average predicted position. The historical averaging depth may be based on a variable-gradient aperture window where the speed and/or acceleration of the aimpoint (in any case based on a movement formula determined as described above) determines how much of the position history should be used. By way of example simply, some embodiments describe that a greater number of historical positions (ie, positions from past recorded video frames) are used at the average history depth when the velocity and/or acceleration is relatively slow. .

The method 500 may include determining an extrapolated distance (block 515). In some embodiments, when the aim point's velocity and/or acceleration is relatively fast (eg, as determined by a movement formula for the aim point), it is useful to extrapolate several frames into the future to predict the aim point's location. When the velocity and/or acceleration is relatively slow, such extrapolation may not be necessary. Once the extrapolated distance is determined, the extrapolated position is predicted using the move formula for the aim point (block 520).

A weighted average is then determined at block 525 . In one set of embodiments, the weighted average will take into account the historical averaging depth and the extrapolated distance. Thus, the weighted average will depend on the velocity and/or acceleration of the aimpoint. At lower velocities, for example, more historical positions will be used for averaging and the extrapolation distance will be shorter. Therefore, the movement formula is given a relatively low weight, and the historical positions are given a relatively high weight. This is to smooth out the overestimations of the position change produced using the movement formula alone, since, as mentioned earlier, movements can be the result of system artifacts and/or small involuntary movements of the trainee when slowing down. can be used for Conversely, when the speed is high, the extrapolation distance increases and fewer (or no history positions) are used for the weighted average. This allows the system to more accurately predict the position when aiming is high because the movement formula has a relatively high weight (in other embodiments, one of the two measurement methods may be used separately).

At block 530, the weighted average is used to determine the predicted position (which may be the predicted position window as described above). This predicted position may be used to track, identify and/or differentiate the aiming point as described with respect to FIG. 4 . In one set of embodiments, the weighting formulas (ie, formulas that determine the historical averaging depth and/or extrapolation distance based on a variable gradient aperture window) may be computed as run-time. In other embodiments, formulas may be pre-computed and/or calibrated based on system specific operations.

For ease of explanation, embodiments of the present invention identify, process and/or process a plurality of aim points, whereas methods 400 and 500 described above refer to identifying and processing a single aim point in each frame. Note that it can be traced. Indeed, one advantage of any of the embodiments may be the ability to simultaneously identify and/or track many aim points. By way of example simply, certain embodiments may track 15 aimpoints (50 aimpoints in any case) on a given video image. The number of aim points that can be tracked is limited by the processing power of the system and/or the ability to analyze individual aim points in a video image.

Additionally, various embodiments of the present invention include (but are not limited to) systems (including but not limited to the systems described above), and methods described above with respect to FIGS. 4 and 5 . (but not limited to) computer programs that can be configured to implement the methods of the present invention.

As mentioned above, in many embodiments the center of the aiming point may be identified. 6 depicts a typical aim point reflection 600 and shows how the center can be identified. As shown by plots 605 , 610 , the reflected energy (IR energy in this example) can be measured. center point (X<sb>I</sb>,Y<sb>I</sb>), the reflected energy E is high, while the reflected energy decreases with the distance from the central point. To find the center of the aiming point, the area surrounding the aiming point may be divided into regions (as described by typical reflection 600 ), and the reflected energy E<sb>It's</sb>can be determined for each region on the X-axis and the Y-axis. To find the center of the aiming point on each axis, the following equations can be used.

<img file="KR101222447B1_D0001.tif" /> Formula (2)

<img file="KR101222447B1_D0002.tif" /> Formula (3)

Those skilled in the art will recognize that in some embodiments a video capture device is based on a description of generating an interlaced video stream (however, other such as embodiments in which the video capture device is a progressive scanning camera or a fixed digital imaging device) embodiments will not be able to create an interlaced video stream). A person skilled in the art will show a typical representation of the aim point reflection 700 and due to the interlacing effect as shown by FIG. 7 which shows how the odd and even lines of the video return the displaced centers and separate parts of the reflection. It will be appreciated that an uncorrected interlaced video stream will produce these slightly vertically spaced alternative video frames. Accordingly, embodiments of the present invention take this displacement into account when identifying possible aiming points and locating the centers of the aiming points via, for example, correction filters applied to the video image.

In conclusion, embodiments of the present invention provide improved tracking and/or identification of aimpoints using specifically generated formulas that predict the aimpoint's location in future frames. These generated formulas improve accuracy in determining the aiming point by combining the results of multiple frames into the correct formulas of the target movement. Applying these formulas, the aim points can be accurately identified at any time, including during the aim point movement. Certain embodiments may be significantly improved in accuracy over conventional systems.

In addition, the generated movement formulas can compensate for pipeline-line delays inherent in acquisition and computational models. Because the formulas provide an accurate model of a linear system (cantilever movement of a human arm), the formulas can be extrapolated before tracking inaccuracies are observed (eg up to 20 frames in some embodiments). Additionally, the equations can be applied to precise aiming and partial frame positions at which user shots are initiated.

Accordingly, various embodiments of the present invention provide progressive methods, systems and software that enhance tracking and/or prediction of weapon aiming points in simulated environments. However, other embodiments may implement similar methods, procedures and/or systems of the present invention for tracking any suitable form of target, indicator, or the like. By way of example simply, embodiments of the present invention may be used for gaming systems and the like. Therefore, while the foregoing description identifies certain exemplary embodiments embodying the present invention, those skilled in the art should recognize that many modifications and variations may be made within the scope of the present invention.

It should be noted that the described methods and systems have been described by way of example only. Consequently, various embodiments may omit, substitute, and/or add various procedures and/or components as appropriate. Similarly, in the foregoing detailed description, for purposes of explanation, various methods have been described in a specific order. It should be appreciated that in alternative embodiments the methods may be performed in an order other than that described above. It should also be appreciated that the methods described above may be performed by hardware components and/or implemented as sequences of machine-executable instructions, the machine-executable instructions being general purpose programmed instructions. or a machine, such as a special purpose processor or logic circuits, to perform the methods. These machine-executable instructions may be on a CD-ROM or other type of optical disk, floppy diskette, ROM, RAM, EPROM, EEPROM, magnetic or optical card, flash memory, or other type of machine-readable medium suitable for storing electronic instructions. It may be stored on one or more machine-readable media, such as a compatible medium. By way of example only, some embodiments of the present invention provide software programs executable on one or more computers to perform the methods described above. In certain embodiments, for example, there may be multiple software components configured to run on various hardware devices. Alternatively, the methods may be performed by a combination of hardware and software.

Therefore, while the foregoing detailed description describes certain exemplary embodiments for implementing the present invention, those skilled in the art should recognize that many modifications and variations are possible within the scope of the present invention. Accordingly, the invention is limited only by the claims set forth below.