US11829112B2

Machine control using real-time model

Summary by NHIP

Real-time machine control method

The method controls a machine subsystem using a qualified predictive model generated from prior geo-referenced data and real-time sensor data. The system calculates an error value to derive a model quality metric, which must meet a specific threshold to authorize operation control.

Claim Score by NHIP

Read claim 1, the broadest

Abstract

A priori geo-referenced data is obtained for a worksite, along with field data that is collected by a sensor on a work machine that is performing an operation at the worksite. A predictive model is generated, while the machine is performing the operation, based on the geo-referenced data and the field data. A model quality metric is generated for the predictive model and is used to determine whether the predictive model is a qualified predicative model. If so, a control system controls a subsystem of the work machine, using the qualified predictive model, and a position of the work machine, to perform the operation.

US11829112B2, drawing sheet 1 — Sheet 1 of 12

Term

12.5 yearsleft in the term

Expires 10 April 2039.

21 claims: 3 independent, 18 dependent

1
Broadest claimClaim Score 58, broad(NHIP)A method of controlling a machine on a worksite to perform an operation comprising:identifying geo-referenced characteristic data for the worksite that was generated prior to the machine performing the operation at the worksite;collecting worksite data, with a sensor on the machine, as the machine is performing the operation at the worksite, the worksite data corresponding to a portion of the worksite;generating a predictive model based on the geo-referenced characteristic data and the worksite data;calculating an error value indicative of model error based on a comparison of model values from the generated predictive model to worksite values in the collected worksite data;calculating a model quality metric, for the predictive model, indicative of model accuracy, based on the error value;determining, while the machine is performing the operation at the worksite, whether the predictive model is a qualified predictive model based on the calculated model quality metric and if so, controlling a subsystem of the machine, using the qualified predictive model, to perform the operation.
12
A computing system on a work, machine comprising:a communication system: configured to identify a priori, geo-referenced characteristic data for a worksite;an in situ data collection system that collects worksite data, with a sensor on a machine, as the work machine is performing an operation at the worksite the worksite data corresponding to a portion of the work site;a model generator system configured to receive the a priori, geo-referenced characteristic data and the worksite data and generate a predictive Model based on the a priori, geo-referenced characteristic data and the worksite data, as the work machine is performing the operation the worksite, using a model generation mechanism, the predictive model including predictive values of a characteristic f the worksite;and a control system that controls subsystem of the machine using the predictive model.
20
A work machine comprising:a communication system configured receive geo-referenced characteristic data for a worksite;an in situ data collection system configured to collect worksite data, with a sensor on the work machine, as the work machine is performing an operation at the worksite, for a portion of the worksite, a plurality of controllable subsystems: a model generator system configured to generate a plurality of different predictive models, based on the geo-referenced characteristic data and the worksite data, each corresponding to a different controllable subsystem of the plurality of controllable subsystems of the work machine, the plurality, of different predictive models including predictive values of an at least one characteristic of the worksite to be encountered by the work machine;and a control system that generates control signals to control each of the controllable subsystems using corresponding predictive model.

Independent claims3

130 paragraphs in 6 sections, as filed

The present application is a continuation of and claims priority of U.S. patent application Ser. No. 17/344,517, filed Jun. 10, 2021, which is a continuation of U.S. patent application Ser. No. 16/380,531, filed Apr. 10, 2019, the contents of which are hereby incorporated by reference in their entirety.

FIELD OF THE DESCRIPTION

The present description relates to work machines. More specifically, the present description relates to a control system that dynamically, during runtime, senses data and generates and qualifies a predictive model and controls the work machine using that model.

BACKGROUND

There are a wide variety of different types of work machines. Those machines can include construction machines, turf management machines, forestry machines, agricultural machines, etc.

Some current systems have attempted to use a priori data to generate a predictive model that can be used to control the work machine. For instance, agricultural harvesters can include combine harvesters, forage harvesters, cotton harvesters, among other things. Some current systems have attempted to use a priori data (such as aerial imagery of a field) in order to generate a predictive yield map. The predicative yield map predicts yields at different geographic locations in the field being harvested. The current systems have attempted to use that predictive yield map in controlling the harvester.

The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.

SUMMARY

A priori geo-referenced vegetative index data is obtained for a worksite, along with field data that is collected by a sensor on a work machine that is performing an operation at the worksite. A predictive model is generated, while the machine is performing the operation, based on the geo-referenced vegetative index data and the field data. A model quality metric is generated for the predictive model and is used to determine whether the predictive model is a qualified predicative model. If so, a control system controls a subsystem of the work machine, using the qualified predictive model, and a position of the work machine, to perform the operation.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.

BRIEF DESCRIPTION OF THE DRAWINGS

<figref idref="DRAWINGS">FIG. 1</figref> is a partial schematic, partial pictorial illustration of a combine harvester.

<figref idref="DRAWINGS">FIG. 2</figref> is a block diagram showing one example of a computing system architecture that includes the combine harvester shown in <figref idref="DRAWINGS">FIG. 1</figref>.

<figref idref="DRAWINGS">FIGS. 3A-3C</figref> (collectively referred to herein as <figref idref="DRAWINGS">FIG. 3</figref>) show a flow chart illustrating one example of the operation of the computing system architecture shown in <figref idref="DRAWINGS">FIG. 2</figref>.

<figref idref="DRAWINGS">FIG. 4</figref> is a flow chart illustrating another example of the operation of the architecture illustrated in <figref idref="DRAWINGS">FIG. 2</figref>, dynamically generating actuator-specific or subsystem-specific control models.

<figref idref="DRAWINGS">FIG. 5</figref> shows a block diagram of the architecture illustrated in <figref idref="DRAWINGS">FIG. 1</figref>, deployed in a remote server environment.

<figref idref="DRAWINGS">FIGS. 6-8</figref> show examples of mobile devices that can be used in the architectures shown in the previous figures.

<figref idref="DRAWINGS">FIG. 9</figref> is a block diagram showing one example of a computing environment that can be used in the architectures shown in the previous figures.

DETAILED DESCRIPTION

As discussed above, some current systems attempt to use a priori data (such as aerial images) in order to generate a predictive map that can be used to control a work machine. By way of example, there has been a great deal of work done in attempting to generate a predictive yield map for a field, based upon vegetation index values generated from aerial imagery. Such predictive yield maps attempt to predict a yield at different locations within the field. The systems attempt to control a combine harvester (or other harvester) based upon the predicted yield.

Also, some systems attempt to use forward looking perception systems, which can involve obtaining optical images of the field, forward of a harvester in the direction of travel. A yield can be predicted for the area just forward of the harvester, based upon those images. This is another source of a priori data that can be used to generate a form of a predictive yield map.

All of these types of systems can present difficulties.

For instance, none of the models generated based on a priori data represent actual, ground truth data. For instance, they only represent predictive yield, and not actual ground truthed yield values. Therefore, some systems have attempted to generate multiple different models, and then assign them a quality score based upon historic performance. For instance, a remote server environment can obtain a priori aerial image data and generate a predictive yield map. The remote server environment can then receive actual yield data generated when that field was harvested. It can determine the quality or accuracy of the model, based upon the actual yield data. The predictive yield model, or the algorithm used to create the model, can then be modified to improve it.

However, this does not help in controlling the harvester, during the harvesting operation. Instead, the actual yield data is provided to the remote server environment, after the harvesting operation is completed, so that the model can be improved for the next harvesting season, for that field.

In contrast, the following description describes a system and method for generating a predictive model based not only on a priori data, but based upon in situ, field data that represents actual values being modeled. For instance, where the predictive map is a predictive yield map, the model used to generate that map is dynamically generated based upon a priori data (such as aerial imagery data) and in situ data, such as actual yield data sensed on the harvester during the harvesting operation. Once the predictive yield map is generated, the model (e.g., the predictive map) used to generate it is evaluated to determine its accuracy (or quality). If the quality of the model is sufficient, it is used for controlling the combine, during the harvesting operation, and it is dynamically, and iteratively, evaluated using in situ data, collected from the combine during the harvesting operation. If the model does not have a high enough quality, then the system can dynamically switch to an alternate model, or it can switch back to manual operation or preset values, or it can generate and evaluate other alternative models.

<figref idref="DRAWINGS">FIG. 1</figref> is a partial pictorial, partial schematic, illustration of an agricultural machine 100, in an example where machine 100 is a combine harvester (or combine). It can be seen in <figref idref="DRAWINGS">FIG. 1</figref> that combine 100 illustratively includes an operator compartment 101, which can have a variety of different operator interface mechanisms, for controlling combine 100, as will be discussed in more detail below. Combine 100 can include a set of front end equipment that can include header 102, and a cutter generally indicated at 104. It can also include a feeder house 106, a feed accelerator 108, and a thresher generally indicated at 110. Thresher 110 illustratively includes a threshing rotor 112 and a set of concaves 114. Further, combine 100 can include a separator 116 that includes a separator rotor. Combine 100 can include a cleaning subsystem (or cleaning shoe) 118 that, itself, can include a cleaning fan 120, chaffer 122 and sieve 124. The material handling subsystem in combine 100 can include (in addition to a feeder house 106 and feed accelerator 108) discharge beater 126, tailings elevator 128, clean grain elevator 130 (that moves clean grain into clean grain tank 132) as well as unloading auger 134 and spout 136. Combine 100 can further include a residue subsystem 138 that can include chopper 140 and spreader 142. Combine 100 can also have a propulsion subsystem that includes an engine (or other power source) that drives ground engaging wheels 144 or tracks, etc. It will be noted that combine 100 may also have more than one of any of the subsystems mentioned above (such as left and right cleaning shoes, separators, etc.).

In operation, and by way of overview, combine 100 illustratively moves through a field in the direction indicated by arrow 146. As it moves, header 102 engages the crop to be harvested and gathers it toward cutter 104. After it is cut, it is moved through a conveyor in feeder house 106 toward feed accelerator 108, which accelerates the crop into thresher 110. The crop is threshed by rotor 112 rotating the crop against concave 114. The threshed crop is moved by a separator rotor in separator 116 where some of the residue is moved by discharge beater 126 toward the residue subsystem 138. It can be chopped by residue chopper 140 and spread on the field by spreader 142. In other implementations, the residue is simply dropped in a windrow, instead of being chopped and spread.

Grain falls to cleaning shoe (or cleaning subsystem) 118. Chaffer 122 separates some of the larger material from the grain, and sieve 124 separates some of the finer material from the clean grain. Clean grain falls to an auger in clean grain elevator 130, which moves the clean grain upward and deposits it in clean grain tank 132. Residue can be removed from the cleaning shoe 118 by airflow generated by cleaning fan 120. That residue can also be moved rearwardly in combine 100 toward the residue handling subsystem 138.

Tailings can be moved by tailings elevator 128 back to thresher 110 where they can be re-threshed. Alternatively, the tailings can also be passed to a separate re-threshing mechanism (also using a tailings elevator or another transport mechanism) where they can be re-threshed as well.

<figref idref="DRAWINGS">FIG. 1</figref> also shows that, in one example, combine 100 can include ground speed sensor 147, one or more separator loss sensors 148, a clean grain camera 150, and one or more cleaning shoe loss sensors 152. Ground speed sensor 147 illustratively senses the travel speed of combine 100 over the ground. This can be done by sensing the speed of rotation of the wheels, the drive shaft, the axel, or other components. The travel speed and position of combine 100 can also be sensed by a positioning system 157, such as a global positioning system (GPS), a dead reckoning system, a LORAN system, or a wide variety of other systems or sensors that provide an indication of travel speed.

Cleaning shoe loss sensors 152 illustratively provide an output signal indicative of the quantity of grain loss by both the right and left sides of the cleaning shoe 118. In one example, sensors 152 are strike sensors (or impact sensors) which count grain strikes per unit of time (or per unit of distance traveled) to provide an indication of the cleaning shoe grain loss. The strike sensors for the right and left sides of the cleaning shoe can provide individual signals, or a combined or aggregated signal. It will be noted that sensors 152 can comprise only a single sensor as well, instead of separate sensors for each shoe.

Separator loss sensor 148 provides a signal indicative of grain loss in the left and right separators. The sensors associated with the left and right separators can provide separate grain loss signals or a combined or aggregate signal. This can be done using a wide variety of different types of sensors as well. It will be noted that separator loss sensors 148 may also comprise only a single sensor, instead of separate left and right sensors.

It will also be appreciated that sensor and measurement mechanisms (in addition to the sensors already described) can include other sensors on combine 100 as well. For instance, they can include a residue setting sensor that is configured to sense whether machine 100 is configured to chop the residue, drop a windrow, etc. They can include cleaning shoe fan speed sensors that can be configured proximate fan 120 to sense the speed of the fan. They can include a threshing clearance sensor that senses clearance between the rotor 112 and concaves 114. They include a threshing rotor speed sensor that senses a rotor speed of rotor 112. They can include a chaffer clearance sensor that senses the size of openings in chaffer 122. They can include a sieve clearance sensor that senses the size of openings in sieve 124. They can include a material other than grain (MOG) moisture sensor that can be configured to sense the moisture level of the material other than grain that is passing through combine 100. They can include machine setting sensors that are configured to sense the various configurable settings on combine 100. They can also include a machine orientation sensor that can be any of a wide variety of different types of sensors that sense the orientation or pose of combine 100. Crop property sensors can sense a variety of different types of crop properties, such as crop type, crop moisture, and other crop properties. They can also be configured to sense characteristics of the crop as they are being processed by combine 100. For instance, they can sense grain feed rate, as it travels through clean grain elevator 130. They can sense yield as mass flow rate of grain through elevator 130, correlated to a position from which it was harvested, as indicated by position sensor 157, or provide other output signals indicative of other sensed variables. Some additional examples of the types of sensors that can be used are described below.

<figref idref="DRAWINGS">FIG. 2</figref> is a block diagram showing one example of a computing system architecture 180 that includes work machine 100, a priori data collection systems 182, alternate data collection systems 184, and a priori data store 186 which is connected to work machine 100 by network 188. Some items shown in <figref idref="DRAWINGS">FIG. 2</figref> are similar to those shown in <figref idref="DRAWINGS">FIG. 1</figref>, and they are similarly numbered.

Network 188 can be any of a wide variety of different types of networks. For instance, it can be a wide area network, a local area network, a near field communication network, a cellular communication network, or any of a wide variety of other networks, or combinations of networks.

A priori data collection systems 182 illustratively collect a priori data that can be used by work machine 100 to generate a model (such as a predictive map) that can be used to control work machine 100. Thus, in one example, systems 182 can include normalized difference vegetation index imager 190, thermal imager 192, radar/microwave imager 194, crop model data 196, soil model data 198, and it can include a wide variety of other items 200. NDVI imager 190 can include such things as aerial imaging systems (e.g., satellite systems, manned or unmanned aerial vehicle imaging systems, etc.) that can be used to take images from which NDVI values can be generated. Thermal imager 192 illustratively includes one or more thermal imaging sensors that generate thermal data. Radar/microwave imager 194 illustratively generates radar or microwave images. A crop model 196 can be used to generate data which is predictive of certain characteristics of the crop, such as yield, moisture, etc. Soil model 198 is illustratively a predictive model that generates characteristics of soil at different locations in a field. Such characteristics can include soil moisture, soil compaction, soil quality or content, etc.

All of these systems 182 can be used to generate data directly indicative of metric values, or from which metric values can be derived, and used in controlling work machine 100. They can be deployed on remote sensing systems, such as unmanned aerial vehicles, manned aircraft, satellites, etc. The data generated by systems 182 can include a wide variety of other things as well, such as weather data, soil type data, topographic data, human-generated maps based on historical information, and a wide variety of other systems for generating data corresponding to the worksite on which work machine 100 is currently deployed.

Alternate data collection systems 184 may be similar to systems 182, or different. Where they are the same or similar, they may collect the same types of data, but at different times during the growing season. For instance, some aerial imagery generated during a first time in the growing season may be more helpful that other aerial imagery that was captured later in the growing season. This is just one example.

Alternate data collection systems 184 can include different collection systems as well, that generate different types of data about the field where work machine 100 is deployed. In addition, alternate data collection systems 184 can be similar to systems 182, but they can be configured to collect data at a different resolution (such as at a higher resolution, a lower resolution, etc.). They can also be configured to capture the same type of data using a different collection mechanism or data capturing mechanism which may be more or less accurate under different criteria.

A priori data store 186 thus includes geo-referenced a priori data 202 as well as alternate geo-referenced a priori data 204. It can include other items 206 as well. Data 202 may be, for example, vegetation index data which includes vegetation index values that are geo-referenced to the field being harvested. The vegetation index data may include such things as NDVI data, leaf area index data, soil adjusted vegetation index (SAVI) data, modified or optimized SAVI data, simple ratio or modified simple ratio data, renormalized difference vegetation index data, chlorophyll/pigment related indices (CARI), modified or transformed CARI, triangular vegetation index data, structural insensitive pigment index data, normalized pigment chlorophyll index data, photochemical reflectance index data, red edge indices, derivative analysis indices, among a wide variety of others.

<figref idref="DRAWINGS">FIG. 2</figref> also shows that work machine 100 can include one or more different processors 208, communication system 210, sensors 212 (which can include yield sensors 211, position/route sensors 157, speed sensors 147, and a wide variety of other sensors 214 (which can be those described above with respect to <figref idref="DRAWINGS">FIG. 1</figref> or different ones)), in situ data collection system 216, model generator system 218, model evaluation system 220, data store 222, control system 224, controllable subsystems 226, operator interface mechanisms 228, and it can include a wide variety of other items 230.

<figref idref="DRAWINGS">FIG. 2</figref> shows that operator 232 can interact with operator interface mechanisms 228 in order to control and manipulate machine 100. Thus, operator interface mechanisms 228 can include such things as a steering wheel, pedals, levers, joysticks, buttons, dials, linkages, etc. In addition, they can include a display device that displays user actuatable elements, such as icons, links, buttons, etc. Where the display is a touch sensitive display, those user actuatable items can be actuated by touch gestures. Similarly, where mechanisms 228 include speech processing mechanisms, then operator 232 can provide inputs and receive outputs through a microphone and speaker, respectively. Operator interface mechanisms 228 can include any of a wide variety of other audio, visual or haptic mechanisms.

In situ data collection system 216 illustratively includes data aggregation logic 234, data measure logic 236, and it can include other items 238. Model generator system 218 illustratively includes a set of different model generation mechanisms 240-242 that may use different schemes to generate predictive models that can be used in controlling machine 100. For example, they may generate predictive models using a linear function, different functions, such as a curve, or they may be used to generate different types of predictive models, such as a neural network, a Bayesian model, etc. System 218 can include other items 244 as well.

Model evaluation system 220 illustratively receives one or more predictive models generated by model generator system 218 and evaluates the accuracy of that model. Thus, it includes model evaluation trigger 246, model quality metric generator 248, model evaluator logic 250 (which, itself, includes threshold logic 252, sorting logic 254, and other items 256), model selection logic 258, and it can include other items 260.

Evaluation trigger logic 246 detects an evaluation trigger which indicates that model evaluation system 220 is to evaluate the accuracy of one or more predictive models. Those models may be currently in use in controlling work machine 100, or they may be different models that are generated, as alternative models which may be used to replace the current model, if the alternate model is more accurate. Once triggered, model quality metric generator 248 illustratively generates a model quality metric for a model under analysis. An example may be helpful.

Assume that the predictive model generated by system 218 is a predictive yield model that predicts a yield at different locations in the field being harvested. Evaluation trigger logic 246 will be triggered based on any of a variety of different types of criteria (some of which are described below) so that model evaluation system 220 iteratively, and dynamically evaluates the accuracy of the predictive yield model, during the harvesting operation. In that case, model quality metric generator 248 will obtain actual yield data from yield sensors 211 and determine the accuracy of the predictive yield model that it is evaluating. Based on that accuracy, it generates an accuracy score or quality score. It can do this for one or more different models.

Model evaluator logic 250 then determines whether the model is qualified to be used in order to control machine 100. It can do this in a number of different ways. Threshold logic 252 can compare the model quality metric generated by generator 248 to a threshold to determine whether the model is performing (or will perform) adequately. Where multiple models are being evaluated simultaneously, sorting logic 254 can sort those models based upon the model quality metric generated for each of them. It can find the best performing model (for which the model quality metric is highest) and threshold logic 252 can then determine whether the model quality metric for that model meets the threshold value.

Model selection logic 258 then selects a model, where one is performing (or will perform) adequately based on the model quality metric and its evaluation. It provides the selected predictive model to control system 224 which uses that model to control one or more of the different controllable subsystems 226.

Thus, control system 224 can include feed rate control logic 262, settings control logic 264, route control logic 266, power control logic 268, and it can include other items 270. Controllable subsystems 226 can include sensors 212, propulsion subsystem 272, steering subsystem 274, one or more different actuators 276 that may be used to change machine settings, machine configuration, etc., power utilization subsystem 278, and it can include a wide variety of other systems 280, some of which were described above with respect to <figref idref="DRAWINGS">FIG. 1</figref>.

Feed rate control logic 262 illustratively controls propulsion system 272 and/or any other controllable subsystems 226 to maintain a relatively constant feed rate, based upon the yield for the geographic location that harvester 100 is about to encounter, or other characteristic predicted by the predictive model. By way of example, if the predictive model indicates that the predicted yield in front of the combine (in the direction of travel) is going to be reduced, then feed rate control logic 262 can control propulsion system 272 to increase the forward speed of work machine 100 in order to maintain the feed rate relatively constant. On the other hand, if the predictive model indicates that the yield ahead of work machine 100 is going to be relatively high, then feed rate control logic 262 can control propulsion system 272 to slow down in order to, again, maintain the feed rate at a relatively constant level.

Similarly, settings control logic 264 can control actuators 276 in order to change machine settings based upon the predicted characteristic of the field being harvested (e.g., based upon the predicted yield, or other predicted characteristic). By way of example, settings control logic 264 may actuate actuators 276 that change the concave clearance on a combine, based upon the predicted yield or biomass to be encountered by the harvester.

Route control logic 266 can control steering subsystem 274, also based upon the predictive model. By way of example, operator 232 may have perceived that a thunderstorm is approaching, and provided an input through operator interface mechanisms 228 indicating that operator 232 wishes the field to be harvested in a minimum amount of time. In that case, the predictive yield model may identify areas of relatively high yield and route control logic 266 can control steering subsystem 274 to preferentially harvest those areas first so that a majority of the yield can be obtained from the field prior to the arrival of the thunderstorm. This is just one example. In another example, it may be that the predictive model is predicting a soil characteristic (such as soil moisture, the presence of mud, etc.) that may affect traction. Route control logic 266 can control steering subsystems 274 to change the route or direction of work machine 100 based upon the predicted traction at different routes through the field.

Power control logic 268 can generate control signals to control power utilization subsystem 278 based upon the predicted value as well. For instance, it can allocate power to different subsystems, generally increase power utilization or decrease power utilization, etc., based upon the predictive model. These are just examples and a wide variety of other control signals can be used to control other controllable subsystems in different ways as well.

<figref idref="DRAWINGS">FIGS. 3A-3C</figref> (collectively referred to herein as <figref idref="DRAWINGS">FIG. 3</figref>) illustrate a flow diagram showing one example of the operation of architecture 180, shown in <figref idref="DRAWINGS">FIG. 2</figref>. It is first assumed that work machine 100 is ready to perform an operation at a worksite. This is indicated by block 290 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. The machine can be configured with initial machine settings that can be provided by the operator or that can be default settings, for machine operation. This is indicated by block 292. A predictive model, that may be used for controlling work machine 100, may be initialized as well. In that case, the model parameters can be set to initial values or default values that are empirically determined or determined in other ways. Initializing the predictive model is indicated by block 294.

In another example, a predictive model can be used, during the initial operation of work machine 100 in the field, based upon historical use. By way of example, it may be that the last time this current field was harvested, with this crop type, a predictive model was used and stored. That model may be retrieved and used as the initial predictive model in controlling work machine 100. This is indicated by block 296. The work machine can be configured and initialized in a wide variety of other ways as well, and this is indicated by block 298.

Communication system 210 is illustratively a type of system that can be used to obtain a priori data over network 188 from a priori data store 186. It thus obtains a priori data which is illustratively geo-referenced vegetation index data for the field that is being harvested (or that is about to be harvested). Obtaining the a priori data is indicated by block 300. The a priori data, as discussed above with respect to <figref idref="DRAWINGS">FIG. 2</figref>, can be generated from a wide variety of different types of data sources, such as from aerial images 302, thermal images 304, temperature from a sensor on a seed firmer that was used to plant the field, as indicated by block 306, or a wide variety of other data sources 308.

Once the a priori data is obtained, it is provided to model generator system 218, and work machine 100 begins (or continues) to perform the operation (e.g., the harvesting operation). This is indicated by block 310. Again, control system 224 can begin to control controllable subsystems 226 with a default set of control parameters 312, under manual operation 314, using an initial predictive model (as discussed above) 316, or in other ways, as indicated by block 318.

As machine 100 is performing the operation (e.g., the harvesting operation) sensors 212 are illustratively generating in situ data (or field data) indicative of the various sensed variables, during the operation. Obtaining in situ (or field) data from sensors on work machine 100 during the operation is indicated by block 320 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. In the example discussed herein, the in situ data can be actual yield data 322 generated from yield sensors 211. The yield sensors 211, as discussed above, may be mass flow sensors that sense the mass flow of grain entering the clean grain tank on machine 100. That mass flow can then be correlated to a geographic position in the field from which it was harvested, to obtain an actual yield value for that geographic position. Of course, depending upon the type of predictive model being generated, the in situ (or field) data can be any of a wide variety of other types of data 324 as well.

Before model generation system 218 can dynamically generate a predictive model (e.g., map) or before model evaluation system 220 can adequately evaluate the accuracy of a predictive model, sensors 212 must generate sufficient in situ field data to make the model generation and/or evaluation meaningful. Therefore, in one example, in situ data collection system 216 includes data aggregation logic 234 that aggregates the in situ data generated by, or based on, the output from sensors 212. Data measure logic 236 can track that data along various different criteria, to determine when the amount of in situ data is sufficient. This is indicated by block 326 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. Until that happens, processing reverts to block 320 where machine 100 continues to perform the operation and data aggregation logic 234 continues to aggregate in situ (field) data based on the outputs from sensors 212 (and possibly other information as well). In one example, data measure logic 236 generates a data collection measure that may be indicative of an amount of in situ data that has been collected. This is indicated by block 328. By way of example, the particular type of predictive model that is being generated or evaluated may best be generated or evaluated after a certain amount of data has been generated. This may be indicated by the data collection measure 328.

Data measure logic 236 may measure the distance that machine 100 has traveled in the field, while performing the operation. This may be used to determine whether sufficient in situ (field) data has been aggregated, and it is indicated by block 330.

Data measure logic 236 may measure the amount of time that machine 100 is performing the operation, and this may give an indication as to whether sufficient in situ data has been obtained. This is indicated by block 332. Data measure logic 236 may quantify the number of data points that have been aggregated by data aggregation logic 234 to determine whether it is sufficient. This is indicated by block 334 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. Determining whether sufficient in situ data has been collected can be determined in a wide variety of other ways as well, and this is indicated by block 336.

Once sufficient in situ data has been collected, it is provided to model generator system 218 (which has also received the a priori data). System 218 uses at least one of the model generation mechanisms 240-242 in order to generate a predictive model using the a priori data and the in situ data. This is indicated by block 338 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. It will also be noted that, as discussed below, even after a predictive model has been generated and is being used to control work machine 100, it can be iteratively evaluated and updated (or refined) based upon the continued receipt of in situ data. Thus, at block 338, where a predictive model has already been generated, it can be dynamically and iteratively updated and improved.

In one example, the predictive model is generated by splitting the in situ data into training data and validation data sets. This is indicated by block 340. The training data, along with the a priori data can be supplied to a model generation mechanism (such as mechanism 240) to generate the predictive model. This is indicated by block 342. It will be noted that additional model generation mechanisms 242 can be used to generate alternate predictive models. Similarly, even the same model generation mechanism 240 that generated the predictive model under analysis can be used to generate alternate predictive models using a different set of a priori data. Using an alternate set of a priori data or an alternate model generation mechanism to generate alternate models is indicated by block 344.

The model generation mechanisms 240-242 can include a wide variety of different types of mechanisms, such as a linear model, polynomial curve model, neural network, Bayesian model, or other models. This is indicated by block 346. The predictive model can be generated and/or dynamically updated in a wide variety of other ways as well, and this is indicated by block 348.

Once a predictive model has been generated or updated, model evaluation system 220 evaluates that model by generating a model quality metric for the predictive model. This is indicated by block 350 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. By way of example, evaluation trigger logic 246 can detect an evaluation trigger indicating that a model is to be evaluated. This is indicated by block 352. For example, evaluation system 220 may be triggered simply by the fact that model generator system 218 provides a predictive model to it for evaluation. In another example, a predictive model may already be in use in controlling work machine 100, but it is to be evaluated intermittently or periodically. In that case, if the interval for evaluation has passed, this may trigger evaluation trigger logic 246. In yet another example, it may be that a predictive model is currently being used to control work machine 100, but a number of different alternate models have also been generated and are now available for evaluation. In that case, the alternate models can be evaluated to determine whether they will perform better than the predictive model currently in use. This may be a trigger for evaluation trigger logic 246 as well. Model evaluation can be taking place continuously, during operation, as well.

In another example, the evaluation trigger can be detected, indicating that a predictive model is to be evaluated, based upon the presence of an aperiodic event. For instance, it may be that operator 232 provides an input indicating that the operator wishes to have an alternate model evaluated. Similarly, it may be that model generator system 218 receives new a priori data, or a new model generation mechanism. All of these or other events may trigger model evaluation system 220 to evaluate a predictive model. Similarly, even though the current predictive model may be operating sufficiently, an alternate model interval may be set at which available alternate models are evaluated to ensure that the model currently being used is the best one for controlling machine 100. Thus, when the alternate model evaluation interval has run, this may trigger the model evaluation logic to evaluate a new model as well.

In order to calculate a model quality metric for the predictive model under analysis, model quality metric generator 248 illustratively applies the in situ validation data set to the model under analysis. This is indicated by block 354. It then illustratively generates an error metric that measures the error of the model. This is indicated by block 356. In one example, the error metric is the r2 error metric that measures the square of the error of the model. The model quality metric for the predictive model under analysis can be generated using a wide variety of quality metric mechanisms as well. This is indicated by block 358.

Once the model quality metric has been generated for the predictive model under analysis, the model evaluator logic determines whether that model should be used for controlling machine 100. This is indicated by block 360. For example, threshold logic 252 can determine whether the model quality metric meets a threshold value. This is indicated by block 362. The threshold value may be set based on factors such as the particular application in which machine 100 is being used, historical experience, etc. In one example where the r2 value is used as the quality metric, a threshold of 0.7 or above may be used. This is just one example, and the threshold can be less than or greater than 0.7 as well.

Where multiple different predictive models have been generated, sorting logic 254 can sort the models based upon the quality metric. A decision as to whether the model under analysis should be used can be based on its rank in the sorted list of models. This is indicated by block 364. Model evaluator logic 250 can determine whether the model under analysis is to be used in other ways as well, and this is indicated by block 366.

If, at block 360, it is determined that the model under analysis is to be used, then model selection logic 358 selects that model and provides it to control system 224 for use in controlling machine 100. Control system 224 then generates control signals to control one or more controllable subsystems 226, using the qualified model. This is indicated by block 368 (<figref idref="DRAWINGS">FIG. 3B</figref>) in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. By way of example, the predictive model may be used to predict yield or biomass or other characteristics to be encountered by work machine 100. This is indicated by block 370. The various different type of logic in control system 224 can generate control signals based upon the prediction provided by the predictive model. This is indicated by block 372. The qualified model can be used to generate control signals in a wide variety of other ways as well, and this is indicated by block 374.

The control signals are then applied to one or more of the controllable subsystems in order to control machine 100. This is indicated by block 376 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. For example, as discussed above, feed rate control logic 262 can generate control signals and apply them to propulsion system 272 to control the speed of machine 100 to maintain a feed rate. This is indicated by block 378. Settings control logic 264 can generate control signals to control settings actuators 276 to adjust the machine settings or configuration. This is indicated by block 380. Route control logic 266 can generate control signals and apply them to steering subsystem 274 to control steering of machine 100. This is indicated by block 382. Power control logic 268 can generate control signals and apply them to power utilization subsystem 278 to control power utilization of machine 100. This is indicated by block 384. A wide variety of other control signals can be generated and applied to a wide variety of other controllable subsystems to control machine 100 as well. This is indicated by block 386.

Unless the operation is complete, as is indicated by block 388, in situ data collection system illustratively resets the in situ data collection measure generated by data measure logic 236 so that it can be determined whether a sufficient amount of in situ data has been collected in order to re-evaluate the current model (or a different model). Resetting the in situ data collection measure is indicated by block 390 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. As discussed above, even where a current model has been evaluated and is sufficiently accurate to control work machine 100, that same model is iteratively evaluated and refined, as more in situ (field) data is obtained. As the field conditions change, it may be that the model is no longer as accurate as it was initially. Thus, it is iteratively and dynamically evaluated, while machine 100 is performing the operation, to ensure that it is accurate enough to be used in control of machine 100. Thus, once the in situ data collection measure is reset at block 390, processing reverts to block 320 where data aggregation logic 234 continues to aggregate in situ data until enough has been aggregated to perform another evaluation or model generation step.

Returning again to block 360 in <figref idref="DRAWINGS">FIG. 3</figref>, if model evaluation system 220 determines that the predictive model under analysis is not of high enough quality to be used by control system 224 in controlling machine 100, then this triggers evaluation trigger logic 246 to determine whether there are any alternate models that may be generated, or evaluated, to determine whether they should be used, instead of the model that was just evaluated. Determining whether there are any other models is indicated by block 392 (<figref idref="DRAWINGS">FIG. 3C</figref>) in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. Again, an alternate model may be generated or available because different a priori data (e.g., alternate a priori data 204) has been received so that an alternate model can, or already has been, generated. This is indicated by block 392. In addition, it may be that a different model generation mechanism 240-242 can be used (even on the same a priori data as was previously used) to generate an alternate model that can be evaluated. This is indicated by block 394.

In another example, it may be that both a new model generation mechanism has been received, and new a priori data has been received, so that an alternate model can be generated (or already has been generated) using the new mechanism and new a priori data. This is indicated by block 396 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>. Determining whether there are any alternative models to be generated or evaluated can be done in a wide variety of other ways as well, and this is indicated by block 398.

If, at block 392, it is determined that there are no alternative models to generate or evaluate, then model evaluation logic 220 indicates this to operator interface mechanisms 228 and a message is displayed to operator 232 indicating that control of machine 100 is reverting to manual or preset control. In that case, control system 224 receives control inputs from operator 232 through operator interface mechanisms 228, or it can receive preset inputs or it can revert to control using a default model. This is all indicated by block 400 in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>.

However, if, at block 392, it is determined that there are alternate models that can be generated or that have been generated and are ready for evaluation, then processing proceeds at block 402 where one or more of the alternate models are generated and/or evaluated to determine whether they are of sufficient quality to be used in work machine 100. The evaluation can be done as described above with respect to <figref idref="DRAWINGS">FIGS. 338-360</figref> in the flow diagram of <figref idref="DRAWINGS">FIG. 3</figref>.

Model selection logic 258 then determines whether any of the models being evaluated have a high enough quality metric to be used for controlling machine 100. This is indicated by block 404. If not, processing reverts to block 400. It should also be noted that, in one example, multiple alternate models are all evaluated substantially simultaneously. In that case, model selection logic 258 can choose the best alternate model (assuming that its quality is good enough) for controlling machine 100. In another example, only one alternate model is evaluated at a given time.

In either example, if, at block 404, model evaluator logic 250 identifies a model that has a high enough quality metric for controlling machine 100, then model selection logic 258 selects that model for control based upon the selection criteria. This is indicated by block 406. Again, where multiple models are being evaluated, model selection logic 258 may simply select the first model that has a quality metric above a threshold value. This is indicated by block 408. In another example, sorting logic 254 can sort all of the models being evaluated based upon their quality metric, and model selection logic 258 can select the model with the best quality metric value. This is indicated by block 410. The model can be selected in other ways as well, and this is indicated by block 412. Once the model is selected, processing proceeds at block 368 where that model is used to generate control signals for controlling machine 100.

Thus far in the description, it has been assumed that one predictive model is used by control system 224 to control the controllable subsystems 226. However, it may be that different predictive models are used by control system 224 to control different controllable subsystems. In addition, it may be that the outputs of a plurality of different predictive models are used to control a plurality of different controllable subsystems. Similarly, it may be that the models are specific to a given actuator or set of actuators. In that example, a predictive model may be used to generate control signals to control a single actuator or a set of actuators.

<figref idref="DRAWINGS">FIG. 4</figref> is a flow diagram illustrating one example of the operation of architecture 180 in an example where multiple different predictive models are used to control different controllable subsystems. It is thus first assumed that model generator system 218 identifies that a set of specific predictive models is to be used for controlling machine 100, instead of a single predictive model. This is indicated by block 420 in the flow diagram of <figref idref="DRAWINGS">FIG. 4</figref>. The predictive models may be subsystem-specific models so that a different predictive model is used to control each of the different controllable subsystems 226. This is indicated by block 422 in the flow diagram of <figref idref="DRAWINGS">FIG. 4</figref>. They may be actuator-specific models so that a different predictive model is used by control system 224 to control a different actuator or set of actuators. This is indicated by block 424. The models may be configured in other ways, so that, for instance, the output of a plurality of a different models is used to control a single subsystem, or so that a single model is used to control a subset of the controllable subsystems while another model is used to control the remaining controllable subsystems, or a different subset of those subsystems. Identifying the set of specific predictive models in other ways is indicated by block 426 in the flow diagram of <figref idref="DRAWINGS">FIG. 4</figref>.

In that example, model generator system 218 then generates a set of specific predictive models to be evaluated, and model evaluation system 220 evaluates those specific predictive models. Model evaluation system 220 illustratively identifies a qualified model corresponding to each subsystem/actuator (or subset of the subsystems/actuators) on work machine 100. This is indicated by block 428 in the flow diagram of <figref idref="DRAWINGS">FIG. 4</figref>.

Model selection logic 258 selects a model for each of the systems/actuators and provides it to control system 224. Control system 224 uses the qualified models to generate signals for the corresponding subsystems/actuator (or subset of subsystems/actuators). This is indicated by block 430 in the flow diagram of <figref idref="DRAWINGS">FIG. 4</figref>.

By way of example, it may be that the traction in the field is modeled by a predictive model. The output of that model may be used by control system 224 to control the steering subsystem 274 to steer around muddy or wet areas where traction is predicted to be insufficient. The in situ data, in that case, may be soil moisture data which is sensed by a soil moisture sensor on machine 100 and provided as the actual, in situ, field data for the predictive traction model. In another example, the header lift actuator may be controlled by a separate predictive model that predicts topography. The in situ data may indicate the actual topography over which machine 100 is traveling. Of course, there are a wide variety of other types of predictive models that can be used by control system 224 to control individual actuators, sets of actuators, individual subsystems, sets of subsystems, etc.

As with the single model example discussed above with respect to <figref idref="DRAWINGS">FIG. 3</figref>, each of the plurality of different predictive models will illustratively be dynamically and iteratively evaluated. Similarly, they can each be replaced by an alternate model, if, during the evaluation process, it is found that an alternate model performs better. Thus, in such an example, multiple predictive models are continuously, dynamically, and iteratively updated, improved, and evaluated against alternate models. The models used for control can be swapped out with alternate models, based upon the evaluation results, in near real time, during operation of the work machine in the field. Continuing the runtime evaluation, in this way, is indicated by block 432 in the flow diagram of <figref idref="DRAWINGS">FIG. 4</figref>.

The present discussion has mentioned processors and servers. In one embodiment, the processors and servers include computer processors with associated memory and timing circuitry, not separately shown. They are functional parts of the systems or devices to which they belong and are activated by, and facilitate the functionality of the other components or items in those systems.

It will be noted that the above discussion has described a variety of different systems, components and/or logic. It will be appreciated that such systems, components and/or logic can be comprised of hardware items (such as processors and associated memory, or other processing components, some of which are described below) that perform the functions associated with those systems, components and/or logic. In addition, the systems, components and/or logic can be comprised of software that is loaded into a memory and is subsequently executed by a processor or server, or other computing component, as described below. The systems, components and/or logic can also be comprised of different combinations of hardware, software, firmware, etc., some examples of which are described below. These are only some examples of different structures that can be used to form the systems, components and/or logic described above. Other structures can be used as well.

Also, a number of user interface displays have been discussed. They can take a wide variety of different forms and can have a wide variety of different user actuatable input mechanisms disposed thereon. For instance, the user actuatable input mechanisms can be text boxes, check boxes, icons, links, drop-down menus, search boxes, etc. They can also be actuated in a wide variety of different ways. For instance, they can be actuated using a point and click device (such as a track ball or mouse). They can be actuated using hardware buttons, switches, a joystick or keyboard, thumb switches or thumb pads, etc. They can also be actuated using a virtual keyboard or other virtual actuators. In addition, where the screen on which they are displayed is a touch sensitive screen, they can be actuated using touch gestures. Also, where the device that displays them has speech recognition components, they can be actuated using speech commands.

A number of data stores have also been discussed. It will be noted they can each be broken into multiple data stores. All can be local to the systems accessing them, all can be remote, or some can be local while others are remote. All of these configurations are contemplated herein.

Also, the figures show a number of blocks with functionality ascribed to each block. It will be noted that fewer blocks can be used so the functionality is performed by fewer components. Also, more blocks can be used with the functionality distributed among more components.

<figref idref="DRAWINGS">FIG. 5</figref> is a block diagram of harvester 100, shown in <figref idref="DRAWINGS">FIG. 2</figref>, except that it communicates with elements in a remote server architecture 500. In an example, remote server architecture 500 can provide computation, software, data access, and storage services that do not require end-user knowledge of the physical location or configuration of the system that delivers the services. In various examples, remote servers can deliver the services over a wide area network, such as the internet, using appropriate protocols. For instance, remote servers can deliver applications over a wide area network and they can be accessed through a web browser or any other computing component. Software or components shown in <figref idref="DRAWINGS">FIG. 2</figref> as well as the corresponding data, can be stored on servers at a remote location. The computing resources in a remote server environment can be consolidated at a remote data center location or they can be dispersed. Remote server infrastructures can deliver services through shared data centers, even though they appear as a single point of access for the user. Thus, the components and functions described herein can be provided from a remote server at a remote location using a remote server architecture. Alternatively, they can be provided from a conventional server, or they can be installed on client devices directly, or in other ways.

In the example shown in <figref idref="DRAWINGS">FIG. 5</figref>, some items are similar to those shown in <figref idref="DRAWINGS">FIG. 2</figref> and they are similarly numbered. <figref idref="DRAWINGS">FIG. 5</figref> specifically shows that model generation system 218, model evaluation system 220 and a priori data store 186 can be located at a remote server location 502. Therefore, harvester 100 accesses those systems through remote server location 502.

<figref idref="DRAWINGS">FIG. 5</figref> also depicts another example of a remote server architecture. <figref idref="DRAWINGS">FIG. 4</figref> shows that it is also contemplated that some elements of <figref idref="DRAWINGS">FIG. 2</figref> are disposed at remote server location 502 while others are not. By way of example, data store 186 can be disposed at a location separate from location 502, and accessed through the remote server at location 502. Regardless of where they are located, they can be accessed directly by harvester 100, through a network (either a wide area network or a local area network), they can be hosted at a remote site by a service, or they can be provided as a service, or accessed by a connection service that resides in a remote location. Also, the data can be stored in substantially any location and intermittently accessed by, or forwarded to, interested parties. For instance, physical carriers can be used instead of, or in addition to, electromagnetic wave carriers. In such an example, where cell coverage is poor or nonexistent, another mobile machine (such as a fuel truck) can have an automated information collection system. As the harvester comes close to the fuel truck for fueling, the system automatically collects the information from the harvester or transfers information to the harvester using any type of ad-hoc wireless connection. The collected information can then be forwarded to the main network as the fuel truck reaches a location where there is cellular coverage (or other wireless coverage). For instance, the fuel truck may enter a covered location when traveling to fuel other machines or when at a main fuel storage location. All of these architectures are contemplated herein. Further, the information can be stored on the harvester until the harvester enters a covered location. The harvester, itself, can then send and receive the information to/from the main network.

It will also be noted that the elements of <figref idref="DRAWINGS">FIG. 2</figref>, or portions of them, can be disposed on a wide variety of different devices. Some of those devices include servers, desktop computers, laptop computers, tablet computers, or other mobile devices, such as palm top computers, cell phones, smart phones, multimedia players, personal digital assistants, etc.

<figref idref="DRAWINGS">FIG. 6</figref> is a simplified block diagram of one illustrative example of a handheld or mobile computing device that can be used as a user's or client's hand held device 16, in which the present system (or parts of it) can be deployed. For instance, a mobile device can be deployed in the operator compartment of harvester 100 for use in generating, processing, or displaying the stool width and position data. <figref idref="DRAWINGS">FIGS. 7-8</figref> are examples of handheld or mobile devices.

<figref idref="DRAWINGS">FIG. 6</figref> provides a general block diagram of the components of a client device 16 that can run some components shown in <figref idref="DRAWINGS">FIG. 2</figref>, that interacts with them, or both. In the device 16, a communications link 13 is provided that allows the handheld device to communicate with other computing devices and under some embodiments provides a channel for receiving information automatically, such as by scanning. Examples of communications link 13 include allowing communication though one or more communication protocols, such as wireless services used to provide cellular access to a network, as well as protocols that provide local wireless connections to networks.

In other examples, applications can be received on a removable Secure Digital (SD) card that is connected to an interface 15. Interface 15 and communication links 13 communicate with a processor 17 (which can also embody processors or servers from previous FIGS.) along a bus 19 that is also connected to memory 21 and input/output (I/O) components 23, as well as clock and location system 27.

I/O components 23, in one example, are provided to facilitate input and output operations. I/O components 23 for various embodiments of the device 16 can include input components such as buttons, touch sensors, optical sensors, microphones, touch screens, proximity sensors, accelerometers, orientation sensors and output components such as a display device, a speaker, and or a printer port. Other I/O components 23 can be used as well.

Clock 25 illustratively comprises a real time clock component that outputs a time and date. It can also, illustratively, provide timing functions for processor 17.

Location system 27 illustratively includes a component that outputs a current geographical location of device 16. This can include, for instance, a global positioning system (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning system. It can also include, for example, mapping software or navigation software that generates desired maps, navigation routes and other geographic functions.

Memory 21 stores operating system 29, network settings 31, applications 33, application configuration settings 35, data store 37, communication drivers 39, and communication configuration settings 41. Memory 21 can include all types of tangible volatile and non-volatile computer-readable memory devices. It can also include computer storage media (described below). Memory 21 stores computer readable instructions that, when executed by processor 17, cause the processor to perform computer-implemented steps or functions according to the instructions. Processor 17 can be activated by other components to facilitate their functionality as well.

<figref idref="DRAWINGS">FIG. 7</figref> shows one example in which device 16 is a tablet computer 600. In <figref idref="DRAWINGS">FIG. 7</figref>, computer 600 is shown with user interface display screen 602. Screen 602 can be a touch screen or a pen-enabled interface that receives inputs from a pen or stylus. It can also use an on-screen virtual keyboard. Of course, it might also be attached to a keyboard or other user input device through a suitable attachment mechanism, such as a wireless link or USB port, for instance. Computer 600 can also illustratively receive voice inputs as well.

<figref idref="DRAWINGS">FIG. 8</figref> shows that the device can be a smart phone 71. Smart phone 71 has a touch sensitive display 73 that displays icons or tiles or other user input mechanisms 75. Mechanisms 75 can be used by a user to run applications, make calls, perform data transfer operations, etc. In general, smart phone 71 is built on a mobile operating system and offers more advanced computing capability and connectivity than a feature phone.

Note that other forms of the devices 16 are possible.

<figref idref="DRAWINGS">FIG. 9</figref> is one example of a computing environment in which elements of <figref idref="DRAWINGS">FIG. 2</figref>, or parts of it, (for example) can be deployed. With reference to <figref idref="DRAWINGS">FIG. 9</figref>, an example system for implementing some embodiments includes a computing device in the form of a computer 810. Components of computer 810 may include, but are not limited to, a processing unit 820 (which can comprise processors or servers from previous FIGS.), a system memory 830, and a system bus 821 that couples various system components including the system memory to the processing unit 820. The system bus 821 may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. Memory and programs described with respect to <figref idref="DRAWINGS">FIG. 2</figref> can be deployed in corresponding portions of <figref idref="DRAWINGS">FIG. 9</figref>.

Computer 810 typically includes a variety of computer readable media. Computer readable media can be any available media that can be accessed by computer 810 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media is different from, and does not include, a modulated data signal or carrier wave. It includes hardware storage media including both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 810. Communication media may embody computer readable instructions, data structures, program modules or other data in a transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

The system memory 830 includes computer storage media in the form of volatile and/or nonvolatile memory such as read only memory (ROM) 831 and random access memory (RAM) 832. A basic input/output system 833 (BIOS), containing the basic routines that help to transfer information between elements within computer 810, such as during start-up, is typically stored in ROM 831. RAM 832 typically contains data and/or program modules that are immediately accessible to and/or presently being operated on by processing unit 820. By way of example, and not limitation, <figref idref="DRAWINGS">FIG. 9</figref> illustrates operating system 834, application programs 835, other program modules 836, and program data 837.

The computer 810 may also include other removable/non-removable volatile/nonvolatile computer storage media. By way of example only, <figref idref="DRAWINGS">FIG. 9</figref> illustrates a hard disk drive 841 that reads from or writes to non-removable, nonvolatile magnetic media, an optical disk drive 855, and nonvolatile optical disk 856. The hard disk drive 841 is typically connected to the system bus 821 through a non-removable memory interface such as interface 840, and optical disk drive 855 is typically connected to the system bus 821 by a removable memory interface, such as interface 850.

Alternatively, or in addition, the functionality described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (e.g., ASICs), Application-specific Standard Products (e.g., ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

The drives and their associated computer storage media discussed above and illustrated in <figref idref="DRAWINGS">FIG. 9</figref>, provide storage of computer readable instructions, data structures, program modules and other data for the computer 810. In <figref idref="DRAWINGS">FIG. 9</figref>, for example, hard disk drive 841 is illustrated as storing operating system 844, application programs 845, other program modules 846, and program data 847. Note that these components can either be the same as or different from operating system 834, application programs 835, other program modules 836, and program data 837.

A user may enter commands and information into the computer 810 through input devices such as a keyboard 862, a microphone 863, and a pointing device 861, such as a mouse, trackball or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 820 through a user input interface 860 that is coupled to the system bus, but may be connected by other interface and bus structures. A visual display 891 or other type of display device is also connected to the system bus 821 via an interface, such as a video interface 890. In addition to the monitor, computers may also include other peripheral output devices such as speakers 897 and printer 896, which may be connected through an output peripheral interface 895.

The computer 810 is operated in a networked environment using logical connections (such as a local area network—LAN, or wide area network—WAN or a controller area network—CAN) to one or more remote computers, such as a remote computer 880.

When used in a LAN networking environment, the computer 810 is connected to the LAN 871 through a network interface or adapter 870. When used in a WAN networking environment, the computer 810 typically includes a modem 872 or other means for establishing communications over the WAN 873, such as the Internet. In a networked environment, program modules may be stored in a remote memory storage device. <figref idref="DRAWINGS">FIG. 9</figref> illustrates, for example, that remote application programs 885 can reside on remote computer 880.

Example 1 is a method of controlling a machine on a worksite to perform an operation comprising: <ul id="ul0001" list-style="none"><li id="ul0001-0001" num="0000"><ul id="ul0002" list-style="none"><li id="ul0002-0001" num="0109">identifying geo-referenced characteristic data for the worksite that was generated prior to the machine performing the operation at the worksite;</li><li id="ul0002-0002" num="0110">collecting worksite data, with a sensor on the machine, as the machine is performing an operation at the worksite, the worksite data corresponding to a portion of the worksite;</li><li id="ul0002-0003" num="0111">generating a predictive model based on the geo-referenced characteristic data and the worksite data;</li><li id="ul0002-0004" num="0112">calculating an error value indicative of model error based on a comparison of model values from the generated predictive model to worksite values in the collected worksite data;</li><li id="ul0002-0005" num="0113">calculating a model quality metric, for the predictive model, indicative of model accuracy, based on the error value;</li><li id="ul0002-0006" num="0114">determining, while the machine is performing the operation at the worksite, whether the predictive model is a qualified predictive model based on the calculated model quality metric and</li><li id="ul0002-0007" num="0115">if so, controlling a subsystem of the machine, using the qualified predictive model, to perform the operation.</li></ul></li></ul>

Example 2 is the method of any or all previous examples, wherein determining whether the predictive model is a qualified predictive model comprises: <ul id="ul0003" list-style="none"><li id="ul0003-0001" num="0000"><ul id="ul0004" list-style="none"><li id="ul0004-0001" num="0117">determining whether the model quality metric meets a model quality threshold.</li></ul></li></ul>

Example 3 is the method of any or all previous examples, and further comprising: <ul id="ul0005" list-style="none"><li id="ul0005-0001" num="0000"><ul id="ul0006" list-style="none"><li id="ul0006-0001" num="0119">if the predictive model is determined not to be a qualified predictive model, then</li><li id="ul0006-0002" num="0120">identifying different geo-referenced characteristic data for generating a different predictive model;</li><li id="ul0006-0003" num="0121">generating the different predictive model;</li><li id="ul0006-0004" num="0122">determining, while the machine is performing the operation at the worksite, whether the different predictive model is a qualified model; and</li><li id="ul0006-0005" num="0123">if so, controlling a subsystem of the machine, using the different predictive model to perform the operation.</li></ul></li></ul>

Example 4 is the method of any or all previous examples, wherein identifying geo-referenced characteristic data comprises: <ul id="ul0007" list-style="none"><li id="ul0007-0001" num="0000"><ul id="ul0008" list-style="none"><li id="ul0008-0001" num="0125">obtaining a priori georeferenced characteristic data, from a remote system, corresponding to the worksite.</li></ul></li></ul>

Example 5 is the method of any or all previous examples, wherein collecting worksite data comprises: <ul id="ul0009" list-style="none"><li id="ul0009-0001" num="0000"><ul id="ul0010" list-style="none"><li id="ul0010-0001" num="0127">aggregating worksite data corresponding to a portion of the worksite, as the machine is performing an operation at the worksite.</li></ul></li></ul>

Example 6 is the method of any or all previous examples, wherein the machine has a plurality of different subsystems and wherein controlling a subsystem comprises: <ul id="ul0011" list-style="none"><li id="ul0011-0001" num="0000"><ul id="ul0012" list-style="none"><li id="ul0012-0001" num="0129">controlling the plurality of different subsystems on the machine.</li></ul></li></ul>

Example 7 is the method of any or all previous examples, wherein generating a predictive model comprises: <ul id="ul0013" list-style="none"><li id="ul0013-0001" num="0000"><ul id="ul0014" list-style="none"><li id="ul0014-0001" num="0131">generating a plurality of different predictive models, based on a priori data and the worksite data, for the different subsystems of the work machine.</li></ul></li></ul>

Example 8 is the method of any or all previous examples, wherein calculating a model quality metric comprises: <ul id="ul0015" list-style="none"><li id="ul0015-0001" num="0000"><ul id="ul0016" list-style="none"><li id="ul0016-0001" num="0133">calculating a plurality of different model quality metrics values for the different predictive models, wherein determining whether the predictive model is a qualified predictive model comprises determining whether each of the plurality of predictive models are qualified predictive models based on the calculated model quality metrics values.</li></ul></li></ul>

Example 9 is the method of any or all previous examples and further comprising: <ul id="ul0017" list-style="none"><li id="ul0017-0001" num="0000"><ul id="ul0018" list-style="none"><li id="ul0018-0001" num="0135">if the plurality of different predictive models are qualified predictive models, then controlling each of the different subsystems of the work machine on the worksite, using a different one of the plurality of different predictive models.</li></ul></li></ul>

Example 10 is the method of any or all previous examples and further comprising: <ul id="ul0019" list-style="none"><li id="ul0019-0001" num="0000"><ul id="ul0020" list-style="none"><li id="ul0020-0001" num="0137">iteratively repeating steps of collecting worksite data, updating the predictive model based on the worksite data, calculating a model quality metric for the updated predictive model and determining whether the predictive model is a qualified predicative model, while the machine is performing the operation.</li></ul></li></ul>

Example 11 is the method of any or all previous examples, wherein geo-referenced characteristic data for the worksite comprises one of: <ul id="ul0021" list-style="none"><li id="ul0021-0001" num="0000"><ul id="ul0022" list-style="none"><li id="ul0022-0001" num="0139">geo-referenced topographic characteristic data for the worksite;</li><li id="ul0022-0002" num="0140">geo-referenced soil characteristic data for the worksite;</li><li id="ul0022-0003" num="0141">geo-referenced crop characteristic data for the worksite; or</li><li id="ul0022-0004" num="0142">geo-referenced weather characteristic data for the worksite.</li></ul></li></ul>

Example 12 is a computing system on a work machine comprising: <ul id="ul0023" list-style="none"><li id="ul0023-0001" num="0000"><ul id="ul0024" list-style="none"><li id="ul0024-0001" num="0144">a communication system configured to identify a priori, geo-referenced characteristic data for a worksite;</li><li id="ul0024-0002" num="0145">an in situ data collection system that collects worksite data, with a sensor on a machine, as the work machine is performing an operation at the worksite, the worksite data corresponding to a portion of the worksite;</li><li id="ul0024-0003" num="0146">a model generator system configured to receive the a priori, geo-referenced characteristic data and worksite data and generate a predictive model based on the a priori, geo-referenced characteristic data and the worksite data, as the work machine is performing the operation at the worksite, using a model generation mechanism, the predictive model including predictive values of a characteristic of the worksite; and</li><li id="ul0024-0004" num="0147">a control system that, controls a subsystem of the machine using the predictive model.</li></ul></li></ul>

Example 13 is the computing system of any or all previous examples and further comprising: <ul id="ul0025" list-style="none"><li id="ul0025-0001" num="0000"><ul id="ul0026" list-style="none"><li id="ul0026-0001" num="0149">a model evaluation system configured to calculate a model quality metric for the predictive model and determine whether the predictive model is a qualified predictive model; and</li><li id="ul0026-0002" num="0150">wherein the control system controls the subsystem using the predictive model if the predictive model is a qualified predictive model.</li></ul></li></ul>

Example 14 is the computing system of any or all previous examples, wherein the model evaluation system comprises: <ul id="ul0027" list-style="none"><li id="ul0027-0001" num="0000"><ul id="ul0028" list-style="none"><li id="ul0028-0001" num="0152">a model quality metric generator configured to calculate the model quality metric for the predictive model.</li></ul></li></ul>

Example 15 is the computing system of any or all previous examples, wherein the model evaluation system comprises: <ul id="ul0029" list-style="none"><li id="ul0029-0001" num="0000"><ul id="ul0030" list-style="none"><li id="ul0030-0001" num="0154">evaluation trigger logic configured to detect an evaluation trigger and, in response, generate a trigger output for the model evaluation system to evaluate an alternative predictive model.</li></ul></li></ul>

Example 16 is the computer system of any or all previous examples, wherein the model generation system is configured to, in response to the trigger output, generate the alternative predictive model using alternative a priori, geo-referenced characteristic data for the worksites.

Example 17 is the computer system of any or all previous examples, wherein the model generation system is configured to, in response to the trigger output, generate the alternative predictive model using an alternative model generation mechanism.

Example 18 is the computing system of any or all previous examples wherein the model generator system is configured to generate a plurality of predictive models each corresponding to a specific controllable subsystem, and wherein the control system uses each of the predictive models to control the corresponding specific controllable subsystem.

Example 19 is the computing system of any or all previous examples, wherein the a priori, geo-referenced characteristic data are generated prior to the work machine performing the operation at the worksite.

Example 20 is a work machine comprising: <ul id="ul0031" list-style="none"><li id="ul0031-0001" num="0000"><ul id="ul0032" list-style="none"><li id="ul0032-0001" num="0160">a communication system configured to receive geo-referenced characteristic data for a worksite;</li><li id="ul0032-0002" num="0161">an in situ data collection system configured to collect worksite data, with a sensor on the work machine, as the work machine is performing an operation at the worksite, for a portion of the worksite;</li><li id="ul0032-0003" num="0162">a plurality of controllable subsystems;</li><li id="ul0032-0004" num="0163">a model generator system configured to generate a plurality of different predictive models, based on the geo-referenced characteristic data and the worksite data, each corresponding to a different controllable subsystem of the plurality of controllable subsystems of the work machine, the plurality of different predictive models including predictive values of an at least one characteristic of the worksite to be encountered by the work machine; and</li><li id="ul0032-0005" num="0164">a control system that generates control signals to control each of the controllable subsystems using a corresponding predictive model.</li></ul></li></ul>

Example 21 is the work machine of any or all previous examples, and further comprising: <ul id="ul0033" list-style="none"><li id="ul0033-0001" num="0000"><ul id="ul0034" list-style="none"><li id="ul0034-0001" num="0166">a model evaluation system configured to calculate a model quality metric for each of the predictive models and determine whether each of the predictive models is a qualified predictive model based on the model quality metrics, wherein the control system generates the control signals to control each of the controllable subsystems using a corresponding qualified predictive model.</li></ul></li></ul>

It should also be noted that the different examples described herein can be combined in different ways. That is, parts of one or more examples can be combined with parts of one or more other examples. All of this is contemplated herein.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Contents6

Every citation, both waysCites: the store holds 1,000 of 1,763

Citations
Document	Relation	Office	Cited during
EP0070219B1	Cites	European Patent Office (EPO)	Applicant
WO0152160A1	Cites	World Intellectual Property Organization (WIPO)	Applicant
WO0152160A1	Cites	World Intellectual Property Organization (WIPO)	Applicant
WO0215673A1	Cites	World Intellectual Property Organization (WIPO)	Applicant
WO0215673A1	Cites	World Intellectual Property Organization (WIPO)	Applicant
WO03005803A1	Cites	World Intellectual Property Organization (WIPO)	Applicant
WO03005803A1	Cites	World Intellectual Property Organization (WIPO)	Applicant
EP0355049A2	Cites	European Patent Office (EPO)	Applicant
EP0532146B1	Cites	European Patent Office (EPO)	Applicant
EP0845198B2	Cites	European Patent Office (EPO)	Applicant
EP0845198B2	Cites	European Patent Office (EPO)	Applicant
US10004176B2	Cites	United States of America	Applicant
US10015928B2	Cites	United States of America	Applicant
US10019018B2	Cites	United States of America	Applicant
US10019790B2	Cites	United States of America	Applicant
US10025983B2	Cites	United States of America	Applicant
US10028435B2	Cites	United States of America	Applicant
US10028451B2	Cites	United States of America	Applicant
US10034427B2	Cites	United States of America	Applicant
US10039231B2	Cites	United States of America	Applicant
CN100416590C	Cites	China	Applicant
DE10050224A1	Cites	Germany	Applicant
US10064331B2	Cites	United States of America	Applicant
US10064335B2	Cites	United States of America	Applicant
US10078890B1	Cites	United States of America	Applicant
US10085372B2	Cites	United States of America	Applicant
US10091925B2	Cites	United States of America	Applicant
KR100974892B1	Cites	Republic of Korea	Applicant
KR100974892B1	Cites	Republic of Korea	Applicant
KR101067576B1	Cites	Republic of Korea	Applicant
KR101067576B1	Cites	Republic of Korea	Applicant
KR101134075B1	Cites	Republic of Korea	Applicant
KR101134075B1	Cites	Republic of Korea	Applicant
DE10120173A1	Cites	Germany	Applicant
CN101236188A	Cites	China	Applicant
US10126153B2	Cites	United States of America	Applicant
US10129528B2	Cites	United States of America	Applicant
CN101303338A	Cites	China	Applicant
CN101363833A	Cites	China	Applicant
US10143132B2	Cites	United States of America	Applicant
KR101447197B1	Cites	Republic of Korea	Applicant
KR101447197B1	Cites	Republic of Korea	Applicant
US10152035B2	Cites	United States of America	Applicant
US10154624B2	Cites	United States of America	Applicant
KR101653750B1	Cites	Republic of Korea	Applicant
KR101653750B1	Cites	Republic of Korea	Applicant
US10165725B2	Cites	United States of America	Applicant
US10178823B2	Cites	United States of America	Applicant
US10183667B2	Cites	United States of America	Applicant
CN101839906A	Cites	China	Applicant
KR101873657B1	Cites	Republic of Korea	Applicant
KR101873657B1	Cites	Republic of Korea	Applicant
US10188037B2	Cites	United States of America	Applicant
CN101929166A	Cites	China	Applicant
DE102005000770B3	Cites	Germany	Applicant
DE102005000771A1	Cites	Germany	Applicant
DE102008021785A1	Cites	Germany	Applicant
DE102009041646A1	Cites	Germany	Applicant
DE102010004648A1	Cites	Germany	Applicant
DE102010038661A1	Cites	Germany	Applicant
DE102011005400A1	Cites	Germany	Applicant
DE102011052688A1	Cites	Germany	Applicant
US10201121B1	Cites	United States of America	Applicant
DE102012211001A1	Cites	Germany	Applicant
DE102012220109A1	Cites	Germany	Applicant
DE102012223768A1	Cites	Germany	Applicant
DE102013019098B3	Cites	Germany	Applicant
DE102013212151A1	Cites	Germany	Applicant
BR102014007178A2	Cites	Brazil	Applicant
DE102014108449A1	Cites	Germany	Applicant
DE102014201203A1	Cites	Germany	Applicant
DE102014208068A1	Cites	Germany	Applicant
DE102015006398B3	Cites	Germany	Applicant
DE102015109799A1	Cites	Germany	Applicant
DE102017204511A1	Cites	Germany	Applicant
DE102019114872A1	Cites	Germany	Applicant
DE102019206734A1	Cites	Germany	Applicant
CN102080373A	Cites	China	Applicant
US10209179B2	Cites	United States of America	Applicant
CN102138383A	Cites	China	Applicant
CN102277867B	Cites	China	Applicant
US10231371B2	Cites	United States of America	Applicant
US10254147B2	Cites	United States of America	Applicant
US10254765B2	Cites	United States of America	Applicant
US10255670B1	Cites	United States of America	Applicant
US10275550B2	Cites	United States of America	Applicant
US10295703B2	Cites	United States of America	Applicant
CN103088807A	Cites	China	Applicant
US10310455B2	Cites	United States of America	Applicant
US10314232B2	Cites	United States of America	Applicant
US10315655B2	Cites	United States of America	Applicant
US10317272B2	Cites	United States of America	Applicant
CN103181263A	Cites	China	Applicant
US10351364B2	Cites	United States of America	Applicant
US10368488B2	Cites	United States of America	Applicant
CN103954738A	Cites	China	Applicant
US10398084B2	Cites	United States of America	Applicant
US10408545B2	Cites	United States of America	Applicant
US10412889B2	Cites	United States of America	Applicant
US10426086B2	Cites	United States of America	Applicant

Showing 100 of 1,452

280 members in 7 offices

This recordMember, by publication datePriority claim

Priority claims2

Priority claims
Document	Office	Kind	Date
201916380531	United States of America	A	2019-04-10
202117344517	United States of America	A	2021-06-10

Members280

Family members, oldest first
Document	Office	Kind
US2019347143A1	United States of America	A1	2019-11-14
US10572316B2	United States of America	B2	2020-02-25
EP3643159A1	European Patent Office (EPO)	A1	2020-04-29
EP3643160A1	European Patent Office (EPO)	A1	2020-04-29
US2020128734A1	United States of America	A1	2020-04-30
US2020128737A1	United States of America	A1	2020-04-30
BR102019016876A2	Brazil	A2	2020-05-05
BR102019021567A2	Brazil	A2	2020-05-05
CN111096143A	China	A	2020-05-05
US2020174853A1	United States of America	A1	2020-06-04
DE102020204462A1	Germany	A1	2020-10-15
DE102020204478A1	Germany	A1	2020-10-15
US2020326674A1	United States of America	A1	2020-10-15
US2020326727A1	United States of America	A1	2020-10-15
BR102020001022A2	Brazil	A2	2020-10-20
BR102020002789A2	Brazil	A2	2020-10-20
CN111802061A	China	A	2020-10-23
CN111857120A	China	A	2020-10-30
US2021022283A1	United States of America	A1	2021-01-28
US2021029877A1	United States of America	A1	2021-02-04
US2021029878A1	United States of America	A1	2021-02-04
US11079725B2	United States of America	B2	2021-08-03
CA3107877A1	Canada	A1	2021-08-06
CA3108290A1	Canada	A1	2021-08-06
CN113207410A	China	A	2021-08-06
CN113207411A	China	A	2021-08-06
US11086693B2	United States of America	B2	2021-08-10
EP3861842A1	European Patent Office (EPO)	A1	2021-08-11
EP3861842A1	European Patent Office (EPO)	A1	2021-08-11
EP3861843A1	European Patent Office (EPO)	A1	2021-08-11
DE102021200028A1	Germany	A1	2021-08-12
US2021243936A1	United States of America	A1	2021-08-12
US2021243938A1	United States of America	A1	2021-08-12
US2021243950A1	United States of America	A1	2021-08-12
US2021243951A1	United States of America	A1	2021-08-12
BR102020023225A2	Brazil	A2	2021-08-17
CN113298670A	China	A	2021-08-24
AU2021200024A1	Australia	A1	2021-08-26
AU2021200338A1	Australia	A1	2021-08-26
US2021271530A1	United States of America	A1	2021-09-02
US2021302925A1	United States of America	A1	2021-09-30
EP3643160B1	European Patent Office (EPO)	B1	2021-10-20
US11178818B2	United States of America	B2	2021-11-23
CN113748832A	China	A	2021-12-07
US11240961B2	United States of America	B2	2022-02-08
CA3131182A1	Canada	A1	2022-04-08
CN114287229A	China	A	2022-04-08
CA3128972A1	Canada	A1	2022-04-09
CA3130117A1	Canada	A1	2022-04-09
CA3130194A1	Canada	A1	2022-04-09
CA3130197A1	Canada	A1	2022-04-09
CA3130230A1	Canada	A1	2022-04-09
CA3130532A1	Canada	A1	2022-04-09
CA3131202A1	Canada	A1	2022-04-09
BR102021001240A2	Brazil	A2	2022-04-12
BR102021001462A2	Brazil	A2	2022-04-12
BR102021015207A2	Brazil	A2	2022-04-12
BR102021015555A2	Brazil	A2	2022-04-12
BR102021016051A2	Brazil	A2	2022-04-12
BR102021016067A2	Brazil	A2	2022-04-12
BR102021016085A2	Brazil	A2	2022-04-12
BR102021016228A2	Brazil	A2	2022-04-12
BR102021017151A2	Brazil	A2	2022-04-12
CN114303588A	China	A	2022-04-12
CN114303589A	China	A	2022-04-12
CN114303590A	China	A	2022-04-12
CN114303591A	China	A	2022-04-12
CN114303592A	China	A	2022-04-12
CN114303593A	China	A	2022-04-12
CN114303594A	China	A	2022-04-12
CN114303595A	China	A	2022-04-12
CN114303596A	China	A	2022-04-12
CN114303597A	China	A	2022-04-12
CN114303598A	China	A	2022-04-12
CN114303599A	China	A	2022-04-12
CN114303602A	China	A	2022-04-12
CN114303606A	China	A	2022-04-12
CN114303607A	China	A	2022-04-12
CN114303608A	China	A	2022-04-12
CN114303609A	China	A	2022-04-12
CN114303610A	China	A	2022-04-12
CN114303611A	China	A	2022-04-12
CN114303612A	China	A	2022-04-12
CN114303613A	China	A	2022-04-12
CN114303614A	China	A	2022-04-12
CN114303615A	China	A	2022-04-12
CN114303616A	China	A	2022-04-12
CN114303617A	China	A	2022-04-12
CN114303618A	China	A	2022-04-12
CN114303619A	China	A	2022-04-12
CN114330808A	China	A	2022-04-12
EP3981231A1	European Patent Office (EPO)	A1	2022-04-13
EP3981232A1	European Patent Office (EPO)	A1	2022-04-13
EP3981233A1	European Patent Office (EPO)	A1	2022-04-13
EP3981234A1	European Patent Office (EPO)	A1	2022-04-13
EP3981235A1	European Patent Office (EPO)	A1	2022-04-13
EP3981236A1	European Patent Office (EPO)	A1	2022-04-13
EP3981243A1	European Patent Office (EPO)	A1	2022-04-13
EP3981244A1	European Patent Office (EPO)	A1	2022-04-13
DE102021101230A1	Germany	A1	2022-04-14

Showing 100 of 280

128 transactions on the USPTO file

Allowed after 1 non-final rejection.

Non-final rejections: 1
Final rejections: 0
RCEs: 0
Appeals: 0

Over time

Point at a mark for the transaction

Transactions

Prosecution transactions, newest first
	Event	Code
2023-11-29	Email NotificationEML_NTR	EML_NTR
2023-11-28	Mail Patent eGrant NotificationMEPG_NTF	MEPG_NTF
2023-11-28	Recordation of Patent Grant MailedPGM/	PGM/
2023-11-28	Patent eGrant NotificationEPG_NTF	EPG_NTF
2023-11-28	Recordation of Patent eGrantEPG/	EPG/
2023-11-28	Patent Issue Date Used in PTA CalculationAllowedPTAC	PTAC
2023-11-09	Email NotificationEML_NTR	EML_NTR
2023-11-08	Issue Notification MailedAllowedWPIR	WPIR
2023-10-19	Dispatch to FDCD1935	D1935
2023-10-19	Application Is Considered Ready for IssuePILS	PILS
2023-10-16	Issue Fee Payment VerifiedN084	N084
2023-10-16	Issue Fee Payment ReceivedIFEE	IFEE
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Electronic ReviewELC_RVW	ELC_RVW
2023-08-07	Email NotificationEML_NTF	EML_NTF
2023-08-07	Mail Notice of AllowanceAllowedMN/=.	MN/=.
2023-08-03	Notice of Allowance Data Verification CompletedAllowedN/=.	N/=.
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-28	Information Disclosure Statement consideredIDSC	IDSC
2023-07-23	Date Forwarded to ExaminerFWDX	FWDX
2023-07-20	Response after Non-Final ActionA...	A...
2023-07-20	Paralegal or electronic terminal disclaimer approvedP574	P574
2023-07-20	Terminal Disclaimer FiledDIST	DIST
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-19	Electronic ReviewELC_RVW	ELC_RVW
2023-07-18	Email NotificationEML_NTF	EML_NTF
2023-07-18	Mail Non-Final RejectionNon-final rejectionMCTNF	MCTNF
2023-07-14	Electronic Information Disclosure StatementEIDS.	EIDS.
2023-07-14	Electronic Information Disclosure StatementEIDS.	EIDS.
2023-07-14	Electronic Information Disclosure StatementEIDS.	EIDS.
2023-07-14	Electronic Information Disclosure StatementEIDS.	EIDS.
2023-07-14	Electronic Information Disclosure StatementEIDS.	EIDS.
2023-07-14	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-14	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-14	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-14	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-14	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Information Disclosure Statement (IDS) FiledWIDS	WIDS
2023-07-13	Non-Final RejectionNon-final rejectionCTNF	CTNF
2023-07-13	Information Disclosure Statement consideredIDSC	IDSC
2023-07-13	Information Disclosure Statement consideredIDSC	IDSC
2023-07-12	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-12	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-12	Information Disclosure Statement (IDS) FiledM844	M844
2023-07-12	Information Disclosure Statement (IDS) FiledM844	M844

Showing 100 of 128

6 legal events, as the office reported them to INPADOC

Over the term

Point at a mark for the event

Events

Legal events, newest first
	Event	Code
2023-11-08	Information on status: patent grantGrantedPATENTED CASESTCF	STCF
2023-10-19	Information on status: patent application and granting procedure in generalPUBLICATIONS -- ISSUE FEE PAYMENT VERIFIEDSTPP	STPP
2023-10-16	Information on status: patent application and granting procedure in generalPUBLICATIONS -- ISSUE FEE PAYMENT RECEIVEDSTPP	STPP
2023-08-03	Information on status: patent application and granting procedure in generalNOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONSSTPP	STPP
2023-03-17	AssignmentAS	AS
2023-03-17	Fee payment procedureENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITYFEPP	FEPP

Machine control using real-time model

Summary by NHIP

Claim Score by NHIP

Abstract