Code framework for generic data extraction, analysis and reduction

CROSS REFERENCE TO RELATED APPLICATION

Pursuant to 35 U.S.C. §119, the benefit of priority from provisional application 61/192,869, with a filing date of Sep. 17, 2008, is claimed for this non-provisional application.

STATEMENT OF GOVERNMENT INTEREST

The invention described was made in the performance of official duties by one or more employees of the Department of the Navy, and thus, the invention herein may be manufactured, used or licensed by or for the Government of the United States of America for governmental purposes without the payment of any royalties thereon or therefor.

BACKGROUND

The invention relates generally to data extraction techniques. In particular, the invention relates to data extraction reduction and analysis, together with interfaces that operate to present the information from the data as events.

Modern software systems typically consist of a large number of interdependent software applications that run simultaneously on one or more physical hardware systems. These applications employ well-defined software interfaces in order to communicate data and invoke commands on remote software processes that enable the proper execution of the software system as a whole. The success of large software systems is directly tied to the correct implementation of these software interfaces.

SUMMARY

Conventional data extraction techniques yield disadvantages addressed by various exemplary embodiments of the present invention. In particular, these suffer from deficiencies related to portability to a variety of platforms.

Various exemplary embodiments provide a system and a method for processing data from a computer that executes an application program. The system includes an extraction engine, a reduction program and an analysis program. The extraction engine retrieves execution data from the application program and records the data to nonvolatile memory as extract data. The reduction program reads the extract data from the memory and reduces the data into human readable format as reduce data. The analysis program analyzes the reduce data in accordance with operator-provided instructions and produces event data that identify at least one sequence associated with the execution data.

In various exemplary embodiments a management console displays the implemented processes. The extraction engine can further include a classes assigner for categorizing said execution data into classes. The reduction program can further include a structure builder for formatting said extract data. The reduction program can further include an event layout builder for formatting said event data.

BRIEF DESCRIPTION OF THE DRAWINGS

These and various other features and aspects of various exemplary embodiments will be readily understood with reference to the following detailed description taken in conjunction with the accompanying drawings, in which like or similar numbers are used throughout, and in which:

<figref idrefs="DRAWINGS">FIG. 1</figref> is a block diagram view of a data extraction process;

<figref idrefs="DRAWINGS">FIG. 2</figref> is a class diagram view for the structure of objects within the data reduction component of the framework;

<figref idrefs="DRAWINGS">FIG. 3</figref> is a process structure view of a sequence diagram for loading events;

<figref idrefs="DRAWINGS">FIG. 4</figref> is a process structure view of a sequence diagram for loading extracted data;

<figref idrefs="DRAWINGS">FIG. 5</figref> is a process structure view of a sequence diagram for data reduction;

<figref idrefs="DRAWINGS">FIG. 6</figref> is a class diagram view for the structure of objects within the data extraction component of the framework;

<figref idrefs="DRAWINGS">FIG. 7</figref> is a process structure view of a data extraction server;

<figref idrefs="DRAWINGS">FIG. 8</figref> is a class diagram view the structure of objects within the extraction management console component of the framework;

<figref idrefs="DRAWINGS">FIG. 9</figref> is a process structure view for a management console;

<figref idrefs="DRAWINGS">FIGS. 10-15</figref> are tabular views of definition lists of parameters and operators; and

<figref idrefs="DRAWINGS">FIGS. 16-23</figref> are exemplary window views of a scenario that employs the data reduction program.

DETAILED DESCRIPTION

In the following detailed description of exemplary embodiments of the invention, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific exemplary embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be utilized, and logical, mechanical, and other changes may be made without departing from the spirit or scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.

The importance of the interfaces used in conventional display systems has led software projects to typically perform data extraction, data reduction, and data analysis. These operations provide verification of the data and command instructions provided via software interfaces, and isolate problems should these occur. Traditionally, software projects have developed support applications that perform the data extraction, data reduction, and data analysis functions required by the project. However, these support applications were focused on the data formats and interfaces used by the particular software project and could not be reused on other projects without expending significant amounts time, money, and resources on code modification, rework, testing, and maintenance.

Various exemplary embodiments provide a software framework that enables data extraction, data reduction and data analysis technology applications across a wide variety of software development projects. The framework clearly and deliberately separates the data extraction, data reduction, and data analysis functions from the specific computer hardware, computer software, interface protocols, and data formats used within a particular software development project. This facilitates the development of additional value-added data extraction, data reduction, and data analysis functions independent of the specific technologies used by a particular software development project.

Additionally, the framework does not enforce the use of a particular set of hardware, software, interface protocols, or data formats on a software development project. The framework executes on a variety of computer hardware running a variety of software operating systems, and allows the end- user (i.e., operator) to control the specific technologies used within their projects without adding additional dependencies upon them. The framework thereby enhances the productivity of software projects by allowing the operator to gain the advantages of data extraction, data reduction, and data analysis technologies without incurring the expense of designing, developing, testing, and maintaining a custom set of tools that implement these functions.

Modern software systems typically consist of a large number of interdependent software applications running simultaneously on one or more physical hardware systems. These applications employ well-defined software interfaces in order to communicate data and invoke commands on remote software processes that enable the proper execution of the software system as a whole. The success of large software systems is directly related to the correct implementation of these software interfaces. Due to the importance of these interfaces, software projects typically perform data extraction, data reduction, and data analysis to verify the data and command provided in software interfaces and to isolate problems if they occur.

Traditionally, a software project supports applications that perform the data extraction, data reduction, and data analysis functions required by the project for a specific platform. However, these support applications have been focused on the data formats an interlaces used by the particular software project and could not be reused on other projects without expending significant amounts time, money, and resources on code modification, rework, testing, and maintenance.

Due to these limitations, development of a set of tools was initiated to overcome these limitations by utilizing the capabilities of the C++ computer programming language, freely available open source software, and object oriented design principles. Their concept involved the development of a framework that clearly separated the data extraction, data reduction, and data analysis functions from the software interlaces and data formats used by these functions. The actual implementation of the software interfaces and data formats would be left to the users of the framework.

The framework itself only contains functionality that is common across multiple software projects, and enables the operator to customize the features specific to their individual needs. Additionally, this framework can be highly portable in order to execute on a variety of computer hardware and software systems. By doing so, the framework could be usable by many different software projects thereby dramatically reducing the time, money, and resources required to implement data extraction, data reduction, and data analysis capabilities in a software system.

A framework called Generic Data Extraction, Analysis and Reduction (GeDEAR) is currently being used by the Tomahawk Weapon Control System and the Littoral Combat Ship: Surface Warfare Mission Package: Module Engagement Controller software systems. The GeDEAR framework is being considered by other programs as well, however at the time of this writing no other formal commitments have been received.

Software systems that utilize data extraction, data reduction, and data analysis capabilities typically organization these functions into several software applications. The first is an application capable of receiving data from other software applications and recording the data to some form of persistent storage. In the GeDEAR framework, this is referred to as the Extraction Server (ES). The second is an application that reads the data from persistent storage, reduces the data into a human readable format, and analyzes the data according to rules specified by the operator. In the GeDEAR framework, this is referred to as the Data Reduction Program (DRP).

Some systems may include an additional application that controls the operation of the ES. In the GeDEAR framework, this is referred to as the Management Console (MC). The interactions between the software system and the data extraction, data reduction, and data analysis capabilities are outlined in the following system example of using the GeDEAR framework.

In this example, the software system consists of two physical hardware machines, computer-A and computer-B. Within these machines, several software applications are executing and communicating with each other. Additionally, one application is communicating with some external interface. This interface may be hardware or software. The ES runs on a machine and records any data provided thereto. The ES stores these data to some persistence (i.e., nonvolatile) storage. Within this configuration, the state of the ES is monitored and controlled from the MC application. At some point in the future, the DRP opens the persistent storage created by the ES, reads the stored events, applies some logic to decode the events, and outputs the events to the operator.

<figref idrefs="DRAWINGS">FIG. 1</figref> shows a block diagram <b>100</b> of the data extraction technique. An external interface (I/F) communicates with computer-A <b>110</b> that executes an executable program application-A <b>115</b>. The computer-A <b>110</b> communicates with computer-B <b>120</b> that executes application-B <b>125</b> and application-C <b>130</b>. The computer-A <b>110</b> also communicates with an extraction server <b>140</b> through an interface block <b>145</b>. A management console <b>150</b> having an interface <b>155</b> communicates with the server <b>140</b>.

Application-B <b>125</b> communicates with application-A <b>115</b> and application-C <b>130</b>, and all three programs communicate with the server <b>140</b> through the interface block <b>145</b>. The server <b>140</b> provides data to memory storage <b>160</b> through a format block <b>165</b>. The data can be extracted from the storage <b>160</b> to a reduction program <b>170</b> via a format block <b>175</b>. The program <b>170</b> can receive initial condition information from event descriptions <b>180</b> to provide an event output <b>190</b>.

The external interface <b>105</b>, applications <b>115</b>, <b>125</b> and <b>130</b> are specific to a project that uses the GeDEAR framework, which imposes no additional dependency. The project using GeDEAR is at liberty to implement these sections using any means necessary to complete the project. The server <b>140</b>, console <b>150</b> and program <b>170</b> represent core elements of the GeDEAR framework and provide the functionality common across all data formats and interface protocols.

The interface and format blocks <b>145</b>, <b>155</b>, <b>165</b>, <b>175</b> are customizable by the operator to meet their specific needs. Within the GeDEAR framework, these are referred to as plug-ins. Although the GeDEAR framework does provide some out-of-the-box reference implementations for these customizable sections, the project using the framework may treat these as optional and thereby may implement its own.

This separation between the interface/format blocks and the core elements in <figref idrefs="DRAWINGS">FIG. 1</figref> represents the result of the careful and deliberate isolation of the software interfaces and data formats used by data extraction, data reduction, and data analysis capabilities from the particular data being extracted. This separation provides the GeDEAR framework with the flexibility required to enable generic data extraction, data reduction, and data analysis capabilities. This enables the GeDEAR framework to focus on providing value-added data extraction, data reduction, and data analysis functions without delving into the intricacies of the underlying software interfaces and data formats.

The components of the GeDEAR framework operate with extracted data through a collection of abstract C++ classes that define the interface that all software interfaces and data formats conform to in order to be supported by the framework. These interfaces are further explained subsequently in this disclosure. From an end-user perspective, these interfaces take the form of plug-ins. These plug-ins can be written by inheritance from a base C++ class and implementing the required C++ methods. The code for the plug-in can be compiled into a standalone library that is loaded dynamically at run-time by the GeDEAR plug-in framework. All of the GeDEAR applications contain support for extension and modification through the use of plug-ins.

The following sections of this document provide additional details of the three main components of the GeDEAR framework, the Data Reduction Program (DRP), Extraction Server (ES), and Management Console (MC). Operators of the various framework components specify the plug-ins to load into the application as well as the software interfaces and data formats that should be used for any given execution of the DRP, ES, or MC through the use of an extended markup language (XML) configuration file. This configuration file is loaded during application startup processing. For example: <ul><li id="ul0001-0001" num="0000"><ul><li id="ul0002-0001" num="0039">&lt;drp&gt; <ul><li id="ul0003-0001" num="0040">&lt;plugins&gt; <ul><li id="ul0004-0001" num="0041">&lt;plugin <ul><li id="ul0005-0001" num="0042">location=“/usr/lib”</li><li id="ul0005-0002" num="0043">name=“gedear_binary_format” /&gt;</li></ul></li><li id="ul0004-0002" num="0044">&lt;plugin <ul><li id="ul0006-0001" num="0045">name=“gedear_output_mods” /&gt;</li></ul></li></ul></li><li id="ul0003-0002" num="0046">&lt;/plugins&gt;</li><li id="ul0003-0003" num="0047">&lt;eventdatasourceloader <ul><li id="ul0007-0001" num="0048">name=“GeDear::Plugins::Formats::Binary”</li><li id="ul0007-0002" num="0049">extension=“DX*.dx” /&gt;</li></ul></li><li id="ul0003-0004" num="0050">&lt;structurebuilder <ul><li id="ul0008-0001" num="0051">name=“GeDear::Plugins::Formats::Binary” /&gt;</li></ul></li></ul></li><li id="ul0002-0002" num="0052">&lt;/drp&gt;</li></ul></li></ul>

In this example, the operator instructs the DRP to load two plug-in libraries. The first library is the named “gedear_binary_format” and is located in /usr/lib. The second library is named “gedear_output_mods” and located in the current working directory.

The default EventDataSourceLoader can be set to “GeDear::Plugins::Formats::Binary” as the type. Additionally the DRP is informed the file extension of this file type is in the format “Dx* . dx” This example also sets the default StructureBuilder to the type “GeDear::Plugins::Formats::Binary”.

There are no dependences between the components of the GeDEAR framework. The DRP may be used to reduce and analyze data that were not extracted from the ES, and the ES does not require use of the DRP for reduction and analysis. Likewise, the ES does not require the MC be used to monitor and control it. The GeDEAR framework does not make use of any custom written inter-application protocols.

<figref idrefs="DRAWINGS">FIG. 2</figref> shows a class diagram <b>200</b> for the objects within the data reduction component of the framework. A set of DRP Human Computer Interface (HCI) classes <b>210</b> consists of at least one interface <b>215</b> (for multiple loaders), which utilizes an implementation of the event data source <b>220</b>, and to at least one interface structure builder <b>230</b>, which utilizes on an interface event layout structure <b>235</b>. The interface event layout structure consists of at least one interface event layout item <b>250</b>. The interface includes at least two implementations: Interface event layout field <b>255</b> and event layout array <b>260</b>. The interface event layout typed field <b>265</b> inherits from interface event layout field <b>255</b>. The interface event layout array <b>260</b> may include one or more objects of the interface event layout item <b>250</b>.

The object event layout <b>240</b> consists of one or more interface event layout structure <b>235</b> objects and one or more interface displayable <b>245</b> objects. There are at least three implementations of interface displayable <b>245</b>. These are object constant <b>270</b>, object field <b>275</b>, and object array <b>280</b>. Object field <b>275</b> consists of a single interface event layout field <b>255</b> object. Object array <b>280</b> consists of one or more interface Displayable <b>245</b> objects.

In the flowchart <b>200</b>, solid diamonds denote aggregation between items, open triangles denote inheritance and dash arrows indicate dependency. For example, the classes set <b>210</b> is aggregated with the interfaces <b>215</b> and <b>230</b> to combine their functionalities at its penultimate block. Each interface provides block for aggregation represents a shell to implement the instructions or establish formats therein. The classes provide methods, definitions and implementation procedures (i.e., executable functions) for reporting and analyzing data. The interface <b>250</b> inherits definitions from the interface <b>255</b> and <b>260</b> to permit operation using lower levels of instructions without controlling details of those functions. The interface <b>220</b> provides an operator-defined declaration for retrieving information fields within the data.

The Loading Events sequence begins with the operator informing the DRP Human Computer Interface (HCI) to load events. The DRP HCI responds by invoking the build method on the StructureBuilder instance to construct an EventLayout::Structure in accordance with the XML definition provided. The structure of the XML and the conversion of the XML into an EventLayout::Structure occurs via some implementation-specific means.

The DRP HCI then constructs an instance of EventLayout with the newly built EventLayout::Structure. EventLayout iterates through the event display definitions contained with a separate XML structure and constructs all the Displayable elements required by this event layout. EventLayout accomplishes this by invoking methods on the EventLayout::Structure in order to obtain the correct EventLayout::Item needed for display.

The Loading Data File sequence begins with the operator informing the DRP human computer interface to load a data file. The DRP HCI responds by invoking the build method on each successive EventDataSourceLoader object until identifying a loader that can parse the provided data in the file. The EventDataSourceLoader, through some implementation-specific means, returns a new instance of EventDataSource able to parse these data.

The processing events sequence begins with the DRP operator informing the DRP HCI that processing should being. The DRP HCI then calls the EventDataSource for the MoreEvents method. If this source has more events, the DRP HCI retrieves the next event from the EventDataSource. From the event data, the DRP HCI locates the event definition able to display the extracted data. The DRP HCI then determines whether this event is valid for display, or else the event should be skipped as part of the filtering processing. This filtering is performed based upon user-supplied criteria such as event number, event sequence number, event extraction time, or the actual values within fields within the events.

For the condition in which the event is valid and should be displayed, the DRP HCI invokes the print method on the correct EventLayout object. The EventLayout object iterates through the Displayable objects contained therein and invoke the print method on each respective Displayable object. If this Displayable object actually represents a Field, the underlying EventLayout::Field contained within the Displayable object is called to return the actual value contained within the extracted data.

<figref idrefs="DRAWINGS">FIG. 3</figref> shows an exemplary sequence diagram <b>300</b> for loading events into the data reduction component of the framework. The sequence diagram <b>300</b> details the general flow of execution within the DRP. A DRP Operator mark <b>310</b> initiates a first trigger <b>312</b> for loading events within a first cycle <b>314</b> until completion. A DRP HCI Classes mark <b>320</b> initiates a second trigger <b>322</b> for second and third cycles <b>324</b>, <b>326</b>. A StructureBuilder mark <b>330</b> initiates a third trigger <b>332</b> for a fifth cycle <b>334</b>. An EventLayout mark <b>340</b> initiates a fourth trigger <b>342</b> having a sub-trigger <b>344</b> for sixth and seventh cycles <b>346</b>, <b>348</b>.

The second cycle <b>324</b> builds structure until completion at the third trigger <b>332</b>, whereas the third trigger <b>326</b> creates classes until completion at the fourth trigger <b>342</b>. An EventLayout::Structure mark <b>350</b> initiates a fifth trigger <b>352</b> with a construction loop <b>354</b> and a sixth trigger <b>356</b>. The fourth cycle <b>334</b> creates event layouts until completion at the fifth trigger <b>352</b>. The fifth cycle <b>346</b> obtains names until completion at the sixth trigger <b>356</b>. A Display mark <b>360</b> provides a seventh trigger <b>362</b>. The sixth cycle <b>348</b> creates layout structures until completion at the seventh trigger <b>362</b>.

<figref idrefs="DRAWINGS">FIG. 4</figref> shows an exemplary sequence diagram <b>400</b> for loading extract data into the DRP. A DRP Operator mark <b>410</b> initiates a first trigger <b>412</b> for loading data within a first cycle <b>414</b> until completion. A DRP HCI Classes mark <b>420</b> initiates the second trigger <b>422</b> for building the classes within a second cycle <b>424</b> until completion. An Event Data Source Loader mark <b>430</b> initiates a third trigger <b>432</b> that creates an Event Data Source <b>440</b>.

<figref idrefs="DRAWINGS">FIG. 5</figref> shows an exemplary sequence diagram <b>500</b> for the reduction of events within the DRP. A DRP Operator mark <b>510</b> initiates a first trigger <b>512</b> for processing events within a first cycle <b>514</b> until completion. A DRP HCI Classes mark <b>520</b> initiates a second trigger <b>521</b> with a sub-trigger <b>522</b> for building classes until completion. The first cycle <b>514</b> processes events until completion at the second trigger <b>521</b>. The sub-trigger <b>522</b> initiates a third cycle <b>523</b> for ascertaining More Events, a fourth cycle <b>524</b> for sequencing the Next Event, a find Event Definition loop <b>525</b>, a validation fifth cycle <b>526</b> for determining event validity, and a sixth cycle <b>527</b> for printing events, all until completion.

An Event Data Source mark <b>530</b> initiates third and fourth triggers <b>532</b>, <b>534</b> that respectively terminate the third and fourth cycles <b>523</b>, <b>524</b>. An Event Layout mark <b>540</b> initiates a fifth trigger <b>541</b> validates events in a seventh cycle <b>542</b>, together with a sixth trigger <b>543</b> with a sub-trigger <b>544</b> that prints in an eighth cycle <b>545</b>.

A Display mark <b>550</b> initiates a seventh trigger <b>552</b> for obtaining display values in a ninth cycle <b>554</b>. An EventLayout::Structure mark <b>660</b> initiates an eighth trigger <b>562</b>; and an Event Layout::file <b>570</b> initiates a ninth trigger <b>572</b>. The seventh cycle <b>542</b> validates until completion at the eighth trigger <b>562</b>. The eighth cycle <b>545</b> prints until completion at the seventh trigger <b>552</b>, while the ninth cycle <b>554</b> obtains values until completion at the ninth trigger <b>572</b>.

<figref idrefs="DRAWINGS">FIG. 6</figref> shows an exemplary class diagram <b>600</b> for identifying interface operations and arguments. An Interface <b>610</b> operates in conjunction with an Extraction Engine <b>620</b>, which interacts with an EventDataSink <b>630</b>. The Interface <b>610</b> includes operations for handling status, starting and stopping. The Engine <b>620</b> obtains Data, Status and establishes a repository. The Sink <b>630</b> provides closing, extraction, status and opening operations for receiving data for storage.

<figref idrefs="DRAWINGS">FIG. 7</figref> shows an exemplary sequence model <b>700</b> for GeDEAR Extraction Server Details. A GeDEAR Extraction Server <b>710</b> receives a user request from an ExternalObject mark <b>720</b> through an external cycle <b>722</b>. Within the Server <b>710</b>, an Interface mark <b>730</b> initiates a first trigger <b>732</b> for handling requests in a first (internal) cycle <b>734</b>, and a second trigger <b>736</b> for handling status in a second cycle <b>738</b>.

An ExtractionEngine mark <b>740</b> initiates a third trigger <b>741</b> for sequencing requests in queuing loop <b>742</b>, and a fourth trigger <b>743</b> for performing requests in a third cycle <b>744</b> together with updating status in an updating loop <b>745</b>. An EventDataSink <b>750</b> initiates a fifth trigger <b>752</b>, which completes the third cycle <b>744</b> to perform requests. A separate thread <b>754</b> can also interact with the fourth trigger <b>743</b>.

<figref idrefs="DRAWINGS">FIG. 8</figref> shows an exemplary class diagram <b>800</b> for identifying HCI classes. A Console for HCI Classes <b>810</b> operates in conjunction with a ServerFactory <b>820</b>, which forwards information to a Server <b>830</b>. The Factory <b>820</b> builds the server, and obtains corresponding fields and names. The Server <b>830</b> obtains status and submits requests for connecting, disconnecting or obtaining exchange files.

<figref idrefs="DRAWINGS">FIG. 9</figref> shows an exemplary sequence diagram <b>900</b> for Management Console (MC) Details. An MC Operator mark <b>910</b> initiates a first trigger <b>912</b> for selecting server type within a first cycle <b>914</b> until completion, enabling operator input in a parameter entry loop <b>916</b>, and monitor beginning in a second cycle <b>918</b>. An MC HCI Classes mark <b>920</b> initiates a third trigger <b>921</b> for getting fields in third and fourth cycles <b>922</b>, <b>923</b> until completion, and initiates a fourth trigger <b>924</b> for validating input in a loop <b>925</b> and building a server in a fifth cycle <b>926</b> until completion.

The first cycle <b>914</b> selects server types until completion at the second trigger <b>921</b>, and the second cycle <b>918</b> begins monitoring until completion at the third trigger <b>924</b>. A ServerFactory <b>930</b> initiates fourth, fifth and sixth triggers <b>932</b>, <b>934</b> and <b>936</b>, the last of which creating a server in a sixth cycle <b>936</b> until completion. A Server <b>940</b> initiates a seventh trigger that provides a terminus for the sixth cycle <b>936</b>.

The parameters provided or created by these operations can be summarized, as provided by <figref idrefs="DRAWINGS">FIGS. 10-15</figref>, by a series of descriptions in tabular form, with identification by name, description and implementation. These lists, or portions of an overall list are described briefly as follows.

<figref idrefs="DRAWINGS">FIG. 10</figref> shows a first list <b>1000</b> that includes parameters with names Array <b>1010</b>, Constant <b>1020</b>, Displayable <b>1030</b>, DRP HCI Classes <b>1040</b> and EventDataSource <b>1050</b>. For example, the first two parameters represent objects that are implemented by the GeDEAR framework. <figref idrefs="DRAWINGS">FIG. 11</figref> provides a second list <b>1100</b> that includes parameters with names EventDataSourceLoader <b>1110</b>, EventLayout <b>1120</b> and EventLayout::Array <b>1130</b>, all representing classes.

<figref idrefs="DRAWINGS">FIG. 12</figref> shows a third list <b>1200</b> that includes parameters with names EventLayout::Field <b>1210</b>, EventLayout::ltem <b>1220</b> and EventLayout::Structure <b>1230</b>. <figref idrefs="DRAWINGS">FIG. 13</figref> indicates a fourth list <b>1300</b> that includes parameters with names EventLayout::TypedField <b>1310</b>, Field <b>1320</b> and StructureBuilder <b>1330</b>.

<figref idrefs="DRAWINGS">FIG. 14</figref> shows a fifth list <b>1400</b> that includes parameters with names Interface <b>1410</b>, ExtractionEngine <b>1420</b> and EventDataSink <b>1430</b>. <figref idrefs="DRAWINGS">FIG. 15</figref> shows a sixth list <b>1500</b> that includes parameters with names Console HCI Classes <b>1510</b>, ServerFactory <b>1520</b> and Server <b>1530</b>.

Various exemplary embodiments describe the design that enables development of GeDEAR tools, rather than particular GeDEAR applications. The following examples provide a typical usage scenario of the GeDEAR Data Reduction Program (DRP) to demonstrate the context of when processing occurs and how the operator is informed of the processing status.

The first step involves loading the extracted data. In this sequence, the operator loads a file that contains extracted data. The first phase is to inform the DRP that a data file should be loaded by selecting a file from the directory control or through the File menu. Upon selecting the appropriate file, the DRP automatically opens the file and performs the processing described in the sequence diagram <b>300</b>.

<figref idrefs="DRAWINGS">FIG. 16</figref> shows a first exemplary GeDEAR DRP window <b>1600</b>. An upper menu <b>1610</b> identifies four selections: input, events, filters and output. An upper files contents box <b>1620</b> displays the directories and files in a cascading list <b>1630</b> under the upper menu <b>1610</b>, which selects “input” for this example. An auxiliary contents box <b>1640</b> is disposed to the right of the box <b>1620</b> to provide file description, with an output range provided underneath. A lower events header <b>1650</b> identifies six descriptors as a tabular header. A lower events contents box <b>1660</b> provides an area for display of summary information about the loaded file.

Upon completion of file loading, the operator may select a menu command to generate an event summary. This summary lists the events within the loaded data file, when the events occur and how often the events occur.

<figref idrefs="DRAWINGS">FIG. 17</figref> shows a second exemplary GeDEAR DRP window <b>1700</b>. A highlighted file <b>1710</b> can be selected within the box <b>1620</b>. Events associated with the file <b>1710</b> can be displayed in the box <b>1660</b> as a list <b>1720</b> with parameters corresponding to the descriptors in the header <b>1650</b>.

The second step involves loading the events. In this sequence, the operator loads events into the DRP to allow for their reduction and display. The event file is an XML file that describes the data and optionally defines the formatting for display in the box <b>1660</b>. The first phase is to inform the DRP that events should be loaded. This can be accomplished through a file menu above the upper menu <b>1610</b> or through a load toolbar icon. <figref idrefs="DRAWINGS">FIG. 18</figref> shows a third exemplary GeDEAR DRP window <b>1800</b> with the file menu <b>1810</b>, which can be expanded. This results in the DRP providing a standard file selection box to the operator enabling selection of the event file or files to load into the DRP.

The third step is performed internal to the DRP. This step involves loading the event definitions described within the XML as prescribed in the flowchart <b>300</b>. <figref idrefs="DRAWINGS">FIG. 19</figref> shows a fourth exemplary GeDEAR DRP window <b>1900</b> in which events in the upper menu <b>1610</b> is selected. A window box <b>1910</b> provides a list of event XML files, including a highlighted line that identifies drp.dat.xml as the selected file.

Upon loading these events, the operator can be provided with a listing of all the events known to the DRP. <figref idrefs="DRAWINGS">FIG. 20</figref> shows a fifth exemplary GeDEAR DRP window <b>2000</b>. The events are displayed as a list <b>2010</b> within an expanded events content box <b>1660</b>, with associated parameters listed under their corresponding parameters in the header <b>1650</b>.

The fourth step involves the operator activating at least one event after the events have been loaded. An active event represents the event to be reduced from the extracted data and displayed to the operator. An event may be activated by either selecting the activate button or through right-click context menus. <figref idrefs="DRAWINGS">FIG. 21</figref> shows a sixth exemplary GeDEAR DRP window <b>2100</b>. A selection <b>2110</b> of the events shown in the list <b>2010</b> can be distinguished by highlighting. Activation opens a menu box <b>2120</b> that includes an “activate” command, from which <figref idrefs="DRAWINGS">FIG. 22</figref> displays a seventh exemplary GeDEAR DRP window <b>2200</b>. The DRP denotes those events that are active in the events table.

The fifth step involves reducing the data. With the extracted data file selected and events loaded the operator may reduce the data. This causes the DRP to display the active events within the data file in by performing the processing described in sequence diagram <b>500</b>. <figref idrefs="DRAWINGS">FIG. 23</figref> shows an eighth exemplary GeDEAR DRP window <b>2300</b>. Selection of the “output” in upper menu <b>1610</b> yields within a window box <b>2310</b> a descriptor list <b>2320</b> identifying actions performed from activation of the selected files.

The GeDEAR framework has the following advantages over the old methods: <ul><li id="ul0009-0001" num="0000"><ul><li id="ul0010-0001" num="0091">a. Well defined separation between the data formats and software interfaces used within the system and the actual extraction, reduction, and analysis capabilities;</li><li id="ul0010-0002" num="0092">b. Decoupling of the extraction and the reduction and analysis capabilities;</li><li id="ul0010-0003" num="0093">c. Independent of the hardware used by the system;</li><li id="ul0010-0004" num="0094">d. Independent of the software used by the system; including operating system, interface protocols, or programming language</li><li id="ul0010-0005" num="0095">e. Dynamic definition and application of the event descriptions to the data reduction and analysis capabilities;</li><li id="ul0010-0006" num="0096">f. Run-time loading of data formats, software interfaces, and output customizations used by the framework; and</li><li id="ul0010-0007" num="0097">g. Run-time modification of extraction, reduction, and analysis functionality provided by the framework.</li></ul></li></ul>

While certain features of the embodiments of the invention have been illustrated as described herein, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the embodiments.