Framework for managing data that provides correlation information in a distributed computing system

TECHNICAL FIELD

[0001] This invention relates to distributed computing systems, more particularly to a framework for managing data that provides correlation information in a distributed computing system.

BACKGROUND OF THE INVENTION

[0002] Distributed computing is a widely prevalent model for computing systems today. Yet, when problems occur with such systems, analysis is often hampered by the distributed nature of the computing.

[0003] Most individual servers in a distributed computing environment are configured, via a logging or other service provider, to generate reasonably useful logs of their own activity. Servers further provide tools to assist a system administrator to analyze the server logs for problem identification. Most middleware applications that facilitate communication between applications also provide a logging service and analysis tools. However, it is common today for a distributed application configuration to include six or more independent servers located on a multitude of physical machines. In such a case, often the best available method to correlate the various error or other event logs from each server is to do so manually, using a time stamp typically associated with each log entry to attempt to recreate a chain of events. The sheer volume of log entries often makes this task impossible. Even where the volume of log entries is low enough not to render the task of manual correlation impossible, the task is at best approximate because the correlator (i.e. the time stamp) is often not sufficiently granular, for example, providing millisecond accuracy when nanosecond accuracy is required. Frequently, the clocks from which the time stamps are derived on the different platforms are not synchronized. It is now recognized that the difficulties in correlating event logs, for example, in order to identify which system is the source of a problem, are a source of difficulty.

[0004] Error and other event logs generally lack sufficient correlation information that may enable the log from one system to be meaningfully compared to the log from another, particularly in a distributed computing environment. At present, such systems do not have access to sufficient contextual data particularly as it relates to other systems in the distributed computing environment. Such contextual data may be useful as a correlator between distributed systems. A correlator is data used by a service, tool or other application to associate a first event (e.g. a step taken by a process) with at least one other event. A correlator typically comprises data representative of an event identifier selected from one or more geographical and logical identifiers to specify what event occurred and where and a temporal identifier to specify when the event occurred.

[0005] Some service provider applications recognize the need for correlators between events that occur within the same or on separate servers in a distributed application environment. For example, one service application, Tivoli® ARM (application response measurement) measures service response levels for transactions in a distributed environment. Tivoli is a registered trademark of International Business Machines Corporation. ARM employs transaction correlators to provide a capability to break down a transaction into its component parts, so that the contribution of each part to the total response time can be analyzed. In a client/server model of a distributed computing system, for example, transactions often consist of one transaction visible to the user, and any number of nested component transactions that are invoked by the one visible transaction. These component transactions may be referred to as the children of the parent transaction (or the child of another child component transaction). As each component transaction contributes to the total response time of the visible transaction, it is useful to know the response times for each component. This function can be used across components within a single computer, or across multiple computers. The function further assists with transaction path tracing. In accordance with ARM, each application responsible for a component of the overall transaction to be measured is modified to include calls to ARM via an application programming interface (API). The calls may request correlators for transactions with one or more child transactions, send the assigned correlators to the child transaction(s) along with the data needed to invoke the child transaction(s) and pass correlators received from parent transactions to the ARM measurement agents.

[0006] ARM measurement agents follow conventions when creating correlators in accordance with a defined format. Included within the correlator is information identifying the computer, the transaction class, the transaction instance, and some flags. The ARM correlator format is somewhat flexible and extendible; however, the correlator and the framework for handling it are specific to the needs of the ARM service. That is, it is not a generic correlator per se for use by one or more varied service applications. Moreover, ARM correlators provide identification only to the level of a transaction instance.

[0007] There is therefore a need to provide a framework for managing data that provides improved correlation information in a distributed computing system.

SUMMARY OF THE INVENTION

[0008] An object of the invention is to provide a framework for managing data that provides correlators in a distributed computing system.

[0009] In accordance with a first aspect of the invention, there is a method for providing a correlator for an event of a series of events. The method comprises generating a correlator comprising an event identifier and a temporal identifier. The event identifier comprises one or more identifiers selected from the group comprising a geographical identifier of said event; and a logical identifier of said event. The temporal identifier is representative of the order of the event in the series of events.

[0010] In accordance with an embodiment of the invention, there is a method for providing a correlator for an event of a series of events occurring on a computer and for which correlators are desired. In accordance with the method, an event identifier is determined from data representative of one or more identifiers selected from the group consisting of geographical identifiers and logical identifiers for the event. A temporal identifier for the event is assigned from data representative of the unique temporal order of the event in the series of events. The correlator is generated from the event identifier and the temporal identifier for the event.

[0011] Preferably, to assign the temporal identifier, each event is counted using one or more sequential counters that are responsive to a characteristic of the event. The temporal identifier comprises a count from one or more of the one or more sequential counters. Its process may define a first characteristic of the event. One of the sequential counters may define a process sequential counter responsive to the first characteristic to count all the events of the process. In such a case, the temporal identifier comprises a count of the process sequential counter. Further, a thread of the process generating the event may define a second characteristic of the event. One of the sequential counters thus defines a thread sequential counter responsive to the second characteristic to count all the events of the particular thread of the process. The temporal identifier may additionally comprise a count of the thread sequential counter. In instances where an event may require a correlator for use by more than one provider instrumenting the process, a third characteristic of the event may be defined by the provider requiring the correlator. The thread sequential counter is further responsive to the third characteristic to count all of the events of the said thread for the said provider.

[0012] In accordance with a preferred method, there is included a step of providing a generic correlator data structure for defining the correlator. The data structure comprises means for defining the event identifier and the temporal identifier and the step of generating comprises configuring an instance of the data structure. The generic correlator data structure may further comprise means for defining optional context information. As such, the method further comprises the steps of determining data representative of an instance of the optional context information; and configuring the instance of the data structure with the data representative of the instance of the optional context information.

[0013] The generic correlator data structure may further comprise means for defining an association between the correlator for the event defined by the data structure and a partner correlator comprising a correlator for a second event. Accordingly, the method further comprises the steps of determining data representative of the association with the partner correlator and configuring the instance of the data structure for the event with the association. Moreover, steps are provided for receiving data for defining a partner correlator for the event and generating an instance of the generic data structure for the partner correlator in accordance with the data received.

[0014] In accordance with a preference of the invention, when the computer is coupled to one or more other computers in a distributed computer system, the method further includes the step of providing the correlator for transporting to a first one of the other computers for association with a correlator of a second one of the other computers. It is understood that the first and second other computers need not be different from one another.

[0015] Preferably, the method of the invention includes a step of providing the correlator for use to identify the event.

[0016] In accordance with a further aspect of the invention, there is provided a computer readable medium for implementing the methods of the invention on a computer. For example, the invention provides a computer readable medium containing executable program instructions for providing a correlator for an event of a series of events occurring on a computer and for which correlators are desired. The computer program medium comprises programming instructions for determining an event identifier comprising data representative of one or more identifiers selected from the group consisting of geographical identifiers and logical identifiers for the event. Programming instructions are further provided for assigning a temporal identifier comprising data representative of the unique temporal order of the event in the series of events from one or more sequential counters each responsive to a characteristic of the event; and generating the correlator comprising the event identifier and the temporal identifier for the event.

[0017] In accordance with a further aspect of the invention, there is provided a correlator for an event in a series of events. The correlator comprises an event identifier comprising one or more identifiers selected from the group comprising: a geographical identifier of said event; and a logical identifier of said event; and a temporal identifier representative of the order of said event in said series of events.

[0018] In accordance with a preference of the invention, there is a data structure stored on a computer readable medium representing a correlator. The data structure comprises a field for an event identifier comprising data representative of one or more identifiers selected from the group consisting of geographical identifiers and logical identifiers for the event; and a field for a temporal identifier comprising data representative of the unique temporal order of the event in the series of events from one or more sequential counters each responsive a characteristic of the event. In accordance with preferences of the invention, the data structure may further comprise at least one of a field for optional correlation information configured in accordance with a protocol for the exchange of information and a field for providing an association to a partner correlator.

BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Further features and advantages of the present invention will become apparent from the following detailed description, taken in combination with the appended drawings, in which:

[0020]FIG. 1 illustrates a system for managing data that provides correlation information on an exemplary distributed computing environment configured in accordance with the invention;

[0021]FIG. 2 illustrates a system for managing data that provides correlation information on another exemplary distributed computing environment configured in accordance with the invention;

[0022]FIG. 3 illustrates an exemplary correlator class defining a correlator data structure in accordance with an embodiment of the invention;

[0023]FIGS. 4A, 4B and <b>4</b>C show exemplary event occurrence time lines and counter assignments to the events of various process/thread/provider scenarios;

[0024]FIGS. 5A and 5B illustrate an object model for a distributed correlator service in accordance with an embodiment of the invention; and

[0025]FIGS. 6A, 6B and <b>6</b>C show a calling sequence for the classes of FIGS. 5A and 5B.

DETAILED DESCRIPTION OF THE INVENTION

[0026] Reference is made to FIG. 1 which shows an exemplary distributed or networked computing system <b>40</b>, namely an Internet web portal having a database, configured in accordance with the invention. The distributed computing environment <b>40</b> comprises a plurality of server platforms, collectively <b>42</b>-<b>50</b>. The respective server platforms are configured to provide various sever applications, namely, an HTTP server <b>52</b>, a servlet <b>54</b>, an Enterprise JavaBeans™ (EJB™) server <b>56</b> client database application <b>58</b> and a DB2™ store procedure <b>60</b> (i.e. database). Enterprise JavaBeans and EJB are trademarks of Sun Microsystems Inc. DB2 is a trademark of International Business Machines Corporation. In exemplary system <b>40</b>, servlet <b>54</b> and EJB server <b>56</b> are configured to provide a Java™ (trademark of Sun Microsystems Inc.) technology-based web application server, integrating enterprise data and transactions for networked communication.

[0027] As is well understood to persons skilled in the art, a server platform typically comprises a computing station having a powerful processor and high capacity mass storage. The server typically utilizes a multi-tasking operating system allowing multiple applications to run concurrently. While each server platform <b>42</b>-<b>50</b> is illustrated as providing a particular server application component, (collectively <b>52</b>-<b>60</b>) of the distributed computer system <b>40</b>, it may be understood that an individual server platform may be configured to provide more than one such component.

[0028] As illustrated, the server platforms <b>42</b>-<b>50</b> are further configured to provide transport or other communication mechanisms, as applicable, to the server applications <b>52</b>-<b>58</b>. For example, HTTP server <b>52</b> is coupled for network communication via HTTP transport <b>62</b> to servlet <b>54</b>. Servlet <b>54</b> is in turn coupled to EJB Server <b>56</b> via a Java Object Request Broker (ORB) <b>64</b>. EJB Server <b>56</b> facilitates connection of the servlet <b>54</b> to client application <b>58</b> via DB2 Java database connectivity (JDBC) <b>66</b> while the client application <b>58</b> and DB2 store procedure <b>60</b> are coupled through DB2 Distributed Relational Database Architecture (DRDA®) middleware <b>68</b>. DRDA is a trademark of International Business Machines Corporation.

[0029] In accordance with the invention, there is provided for each component <b>52</b>-<b>60</b> of system <b>40</b> a distributed computing system (DCS) service <b>70</b>-<b>78</b> for managing correlation data that provides correlators in a distributed computing system. Access to each DCS service <b>70</b>-<b>78</b> is made via one or more provider plug-ins and one or more transport plug-ins for each server. For example, HTTP server <b>52</b> includes HTTP DCS provider plug-in <b>80</b> for providing access to DCS service <b>70</b> by HTTP server <b>52</b>. DCS HTTP interceptor plug-in <b>94</b> provides a request interceptor to DCS <b>70</b> for HTTP transport <b>62</b>. Similarly, HTTP DCS provider plug-in <b>82</b> and Web DCS provider plug-in <b>84</b> are provided for servlet <b>54</b> while two transport plug-ins, namely DCS HTTP interceptor <b>96</b> and DCS ORB Interceptor <b>98</b> are provided, one for each of the transport mechanisms <b>62</b> and <b>64</b> provided by server <b>44</b> for accessing DCS service <b>72</b>. Likewise, respective DCS ORB interceptor <b>100</b> and DCS JDBC plug-in <b>102</b> are provided for DCS service <b>74</b> on server <b>46</b>. Server <b>46</b> also includes Web DCS provider <b>86</b> and DB2 DCS provider <b>88</b> for respective EJB and DB2 components. Server <b>48</b> includes DB2 DCS provider plug-in <b>90</b>, DCS JDBC transport plug-in <b>104</b> and DCS DRDA transport plug-in <b>106</b>. Server <b>50</b> is provided with DB2 DCS provider plug-in <b>92</b> and DCS DRDA transport plug-in <b>108</b>.

[0030] As described further herein below, the respective DCS Service, provider plug-ins and transport plug-ins for each particular application provide a framework for managing data providing correlator information. The framework may be invoked to generate a correlator for a particular event of an application. A service provider defined by the provider plug-ins may use the correlator locally, for example, as a part of a log entry for the application. The correlator may be transported to another component of the distributed computing system for association with another event. For example, a correlator of a first event of a server application on a first server may be transported via the transport mechanism coupling the first server application to another server application, particularly one such application implemented on a remote server platform. The sharing of such correlators provides a means to link events occurring on different components of a distributed computing system.

[0031]FIG. 2 illustrates another exemplary distributed computing environment comprising a client/server model configured in accordance with the invention. Distributed computing system <b>120</b> includes a client system <b>122</b> and a server <b>124</b> coupled for communication through a network <b>142</b>, for example, a local area network (LAN). The network <b>142</b>, in turn, may be connected to other LANs or a wide area network via a gateway (not shown). Client system <b>122</b> includes a client side component <b>126</b> of a client/server application. Client side component <b>126</b> is configured for communication between the client system <b>122</b> and the server <b>124</b>. Client side component <b>126</b> may include, for example, interfaces to network <b>142</b> providing a transport for handling messages for a function call over network <b>142</b>. Server <b>124</b> includes a server side component <b>128</b> configured for communication via network <b>142</b> with client side component <b>126</b>.

[0032] In accordance with the invention there is provided on client system <b>122</b> a DCS provider plug-in <b>130</b> for client side component <b>126</b>. DCS provider plug-in <b>130</b> defines a provider service for the client side component such as a logging service requiring logging data artifacts, for example artifact <b>132</b> for client side events and, in some instances, logging data artifact <b>132</b> includes a DCS correlator <b>134</b>, and a partner correlator (not shown) for a corresponding server side event. Provider plug-in <b>130</b> is coupled to a storage device <b>136</b> for recording an event log including the artifact and correlator <b>132</b> and <b>134</b> and partner correlators, as applicable. Storage device <b>136</b> may be local or remote relative to client system <b>122</b>.

[0033] Provider plug-in <b>130</b> is further coupled to a DCS service <b>138</b> which generates, stores and provides the DCS correlators (<b>134</b>) when invoked by provider plug-in <b>130</b> or a transport plug-in such as DCS Middleware plug-in <b>140</b>. DCS Middleware plug-in <b>140</b> is provided for communicating with DCS service <b>138</b> to obtain correlators for transporting to server <b>124</b> in network communications over network <b>142</b>. DCS service <b>138</b> also associates correlators received in inbound communications (i.e. partner correlators) with appropriate correlators generated by DCS service <b>138</b>.

[0034] Server <b>124</b> includes corresponding middleware plug-in <b>146</b>, DCS service <b>144</b> and DCS provider plug-in <b>152</b> that function in a manner similar to their corresponding client side equivalents. Provider plug-in <b>152</b> defines a provider service for the server side component <b>128</b>. The provider service may be a logging service requiring logging data artifacts, such as artifact <b>150</b> for server side events including a DCS correlator <b>147</b> and, as applicable, a partner correlator <b>148</b> for a corresponding client side event. A storage device <b>154</b> is coupled, either remotely or locally, to provider plug-in <b>152</b> for storing logging data artifact <b>150</b> including DCS and partner correlators <b>147</b> and <b>148</b> in log entries.

[0035] When a transport plug-in receives an inbound network communication including a correlator, whether from a client system <b>122</b> or server system <b>124</b>, the correlator is removed and passed to the respective DCS service with which the transport plug-in is coupled in order that the DCS service may associate the inbound correlator as a partner correlator for a correlator generated by the DCS service. This partner correlator is linked to a correlator for an event generated by the DCS Service in response to an invocation by a provider plug-in on the system receiving the partner correlator. This basic mechanism provides a framework for arbitrary applications and systems to create and communicate correlators in a distributed computing environment.

[0036] The correlation information is therefore available to a provider such as a logging/tracing; Reliability, Availability, Serviceability (RAS); Application Response Measurement (ARM); application service provider (ASP) subscriber context; or performance management service tool or application for collection and correlation with other data. In addition to providing a service for generating and transporting correlators in a distributed computing environment, the present invention provides a generic correlator for identifying events in such an environment. FIG. 3 illustrates an exemplary class data structure diagram for a correlator <b>10</b> including optional extensible context data, namely SOAP Parameter <b>15</b> described further herein below and an optional partner correlator. In accordance with the invention, the generic correlator <b>10</b> comprises the following identifiers for characteristics of the event:

[0037] Hostname—identifying the host machine where a particular process (or Java Virtual Machine (JVM)) is running;

[0038] Process Identifier (processID or jvmID)—uniquely identifying the particular process (or JVM) on a particular machine;

[0039] Thread Identifier (threadID)—identifying the current thread for a particular process;

[0040] Thread Sequence Counter (Thread_seq_cntr)—identifying an event counter for a particular thread and for a particular service provider to assist in ordering the events, for example, when clock granularity is insufficient. Also it permits the identification of the reuse of a particular thread within a process, for example, when the counter is reset to zero;

[0041] Process Sequence Counter (Process_seq_cntr)—identifying an event counter for all threads on a particular process and for all service providers serving this process to assist in ordering the events, for example, when clocks on different machines in the distributed computing environment are out of synchronization or when clock granularity is insufficient;

[0042] Provider Identifier (providerID) (not shown)—identifying the service provider associated with a particular correlator;

[0043] Unique Identifier (UniqueID)—representing the UUID for an object/event instance that is unique within the process/JVM; and

[0044] Unique Identifier Format (UniqueID_format)—identifying the format of the UUID (e.g. product, version and XML data representation, etc.).

[0045] The hostname provides a geographic location identifier indicating where the event originated while the process identifier and thread identifiers provide a logical location reference for an event specifying which event occurred. The process sequence counter and, for processes executing in more than one thread, the thread sequence counter provide temporal references giving order to particular events as described further herein below.

[0046] Additional event characteristics are useful for identifying the event. Preferably, the generic correlator data structure provides for a providerID for distinguishing correlators in situations where more than one service provider is instrumented in an application. Further the generic correlator data structure preferably includes the unique identifier and unique identifier format that provide additional instance identification capabilities to the generic correlator. The hostname and other event identifiers together with the temporal identifiers provide a generic and hierarchal convention for uniquely specifying an event. Persons skilled in the art will understand that particular events may be identified sufficiently without configuring all of the event identifier fields of the data structure, selecting geographic and location identifiers as desired.

[0047] It is further preferred that the generic correlator data structure <b>10</b> is extensible to include optional data the need for which may be determined by the service provider, for transporting between processes. In accordance with the preference SOAP Parameter class <b>15</b> defines:

[0048] Context Data (context_data)—identifying the additional information required to be transported across processes for a particular service provider;

[0049] Context Data Format (context_data_format)—identifying the format of the context data information. Exemplary context data formats include data formats defined in accordance with Simple Object Access Protocol (SOAP), an extensible markup language XML-based protocol for the exchange of information in a decentralized, distributed environment; Java Message Service (JMS) protocol or other suitable protocols.

[0050] This additional data is not required to establish a correlator, per se, but may provide additional contextual information or aid with a particular feature of the provider.

[0051] It is further preferred that the generic correlator data structure includes correlator information from a partner process. The generic data structure may include such information as follows:

[0052] Partner Correlator (partner_correlator)—identifying, in relation to a current correlator, the correlator for an event (i.e. a partner event) that triggered the particular event of the current correlator.

[0053] Correlator <b>10</b> also includes exemplary methods for setting and getting various correlator information as is understood to persons skilled in the art. Though not shown, additional methods may be included to convert or format the correlator information as desired. For example, a ToString method may be provided for converting all of the correlator information of a particular correlator to a single String data type for convenient handling. A method for incrementing the process and thread sequence counters enumerates events such that for a particular process, each event is assigned a unique count while for each thread of that particular process, and for each provider instrumented on the thread, each event is also assigned a unique count. Thus, one or more sequence counters that are responsive to characteristics of the event may be used to represent the unique temporal order of the event in a series of events requiring correlators.

[0054] With reference to FIGS. 4A, 4B and <b>4</b>C, there is illustrated the assignment of process and thread sequence counters to exemplary event occurrences for, respectively, a single provider instrumented on a multi-threaded process, multiple providers instrumented on a single threaded process and multiple providers instrumented on a multi-threaded process. For the purposes of simplification, FIGS. 4A, 4B and <b>4</b>C illustrate at most two threads and two providers but it is understood that additional threads or providers are contemplated by the invention.

[0055] With reference to FIG. 4A there is illustrated an event line for each of two threads TH<b>1</b> and TH<b>2</b> of process ProcA instrumented by single provider, Prov<b>1</b>. The process sequence counter is represented by counter Proc# counting each of the nine sample events. The respective thread sequence counters TH<b>1</b># and TH<b>2</b># assign respective counts to the events that occur on the respective threads. Collectively, the nine events produce the following process sequence count and thread sequence count pairs (Proc#, THn#): (<b>1</b>,<b>1</b>) (<b>2</b>,<b>1</b>) (<b>3</b>,<b>2</b>) (<b>4</b>,<b>2</b>) (<b>5</b>,<b>3</b>) (<b>6</b>,<b>3</b>) (<b>7</b>,<b>4</b>) (<b>8</b>,<b>4</b>) and (<b>9</b>,<b>5</b>). As understood to persons skilled in the art, the assignment of a unique count to a process sequence counter, for example, of a correlator may be accomplished through well known steps to lock access to the counter supplying the count.

[0056] With reference to FIG. 4B there is illustrated an event line for each of two providers Prov<b>1</b> and Prov<b>2</b> instrumented on one thread TH<b>1</b> of a process ProcB. It is noted that for the same nine sample events as depicted in FIG. 4A, the process sequence count and thread sequence count pairs are the same. With reference to FIG. 4C there is shown an event line for each of two providers Prov<b>1</b> and Prov<b>2</b> instrumented on two threads TH<b>1</b> and TH<b>2</b> of process ProcA to illustrate a further example of the assignment of a temporal identifier.

[0057] Thus the process sequence counter and thread sequence counter assign unique identifiers to event occurrences generally identified by geographic and logical identifiers within the generic correlator structure. The counters provide temporal granularity independent of a time stamp or other system clock reference.

[0058] In accordance with an embodiment of the invention, there is provided an object-oriented programming model for managing correlators <b>10</b> of FIG. 3 in a distributed computing environment. FIGS. <b>5</b>A-<b>5</b>B illustrate simplified class diagrams indicating basic functional objects and characteristics of an exemplary embodiment of a DCS service in accordance with the present invention. With reference to FIG. 5A, there is illustrated a client class <b>300</b>, provider class <b>310</b>, middleware class <b>320</b>, logging data (LoggingData) class <b>330</b> and parameter class (SOAP Parameter) <b>15</b>. With reference to FIG. 5B, there is illustrated a DCS service class (Service) <b>350</b>, thread table entry class (ThreadTableEntry) <b>360</b>, middleware table entry class (MwareTableEntry) <b>370</b>, Correlator Table Handler class CT_Handler) <b>380</b>, Correlator Table Entry class (CorrelatorTableEntry) <b>390</b> and Correlator class <b>10</b>.

[0059] Briefly, client class <b>300</b> from FIG. 5A defines an exemplary client object coupled via a middleware object defined by middleware class <b>320</b> to another object (not shown) in a distributed computing environment. Provider class <b>310</b> implements an exemplary service provider, namely a distributed logging service capable of tracking and recording to a log the various events of an application such as the exemplary client objects defined by client class <b>300</b>. Provider <b>310</b> provides an application programming interface (API) configured for correlating client events with corresponding events of the other process (not shown) with which the client operates via the middleware object as described. Each of the provider and middleware objects access a DCS Service defined by service class <b>350</b> of FIG. 5B for generating, storing, accessing and transporting correlators for the events of the client <b>300</b> of FIG. 5A.

[0060] As is understood to persons skilled in the art, one preferred manner of incorporating a service such as a logging provider into an application is to include invocations of the provider service at selected points in the application code that define an event of interest to be logged by the logging provider. In a distributed computing environment, particular events of interest are those involving communications, via middleware or other transport mechanisms, between the application (e.g. client) executing on one machine and another process (e.g. server) running on second machine located remotely from the first machine. These events of interest often require correlation.

[0061] Provider <b>310</b> of FIG. 5A includes various methods such as event and record for use with a logging data object defined by logging data class <b>330</b> to provide a logging service API. Logging data class <b>330</b> defines a data structure and methods for distributed logging where there is one instance of the logging data class per execution thread. Logging data class <b>330</b> provides methods for setting and getting various exemplary logging data for each event. In the example, the logging data of interest is defined in accordance with the data structures and methods of parameter class <b>15</b>. In addition to setting data for local logging purposes, provider <b>310</b> configures optional data for inclusion in an event correlator <b>10</b> of FIG. 5B for transport with the generic correlation data. For example, provider <b>310</b> may configure an instance of parameter class <b>15</b> to represent a local time stamp for an event for inclusion as an element of context_data of a correlator <b>10</b> of FIG. 5B by the set_context_data method.

[0062] In order to generate, store and access event correlators for client events, provider <b>310</b> of FIG. 5A may invoke various methods of DCS Service class <b>350</b> of FIG. 5B as described further herein below.

[0063] As noted, middleware objects defined by middleware class <b>320</b> of FIG. 5A provide transport between client objects <b>300</b> and one or more other processes (e.g. server objects not shown) for the events of interest logged by the provider objects. Exemplary middleware class <b>320</b> is configured to provide objects for either client side or server side communication.

[0064] In addition to communicating the data normally required for client/server communications to complete an event, middleware class <b>320</b> is further configured in accordance with a preference of the invention to provide for transport of the correlators for the event. The correlators are preferably prepended as a header or otherwise formatted in accordance with a protocol for the middleware.

[0065] Middleware class <b>320</b> thus includes methods for typical client/server communication, including object initialization and termination and to receive and transmit in accordance with a protocol established for the middleware as is understood to persons skilled in the art. Middleware class <b>320</b> is further configured to include additional code for handling correlators and invoking methods for accessing DCS Service <b>350</b>. Such additional code may remove a correlator from an inbound communication or include a correlator in an outbound communication. A correlator received on an inbound communication is configured as a partner correlator for partnering with a correlator for the thread of the process of the application receiving the communication. Middleware class <b>320</b> thus invokes methods of correlator class <b>10</b> and correlatorTableEntry class <b>390</b> to configure received data as a correlator object. Thereafter, the correlator object may be presented to DCS Service <b>350</b> for storing as a partner correlator and for linking with the appropriate correlator. Additionally, methods of the Service may be invoked to obtain the appropriate correlator for an outbound communication and for registering the middleware for the DCS Service as explained further below.

[0066] Middleware class <b>320</b> includes the following exemplary methods. InitMiddleware initializes the middleware communications streams for the server application side. Init_DCS_Mware initializes the DCS Service for a particular middleware instance. SendToMiddleware sends a data stream to a client application and send_DCS_Mware prepends a correlator data stream as a header to a data stream for transport. RcvFromMware receives the data stream sent by the client application into the server application. Rcv_DCS_Mware extracts a correlator data stream header from that data stream to establish a correlator object for partnering with another correlator. TerminateMware terminates the middleware communication stream for the server application side. Exit_DCS_Mware de-registers a middleware instance from the DCS service. Correspondingly, initClientMware initializes the middleware communications streams for the client application side. RcvFromMwareClient receives a data stream sent by the server application to the client application while sendToMwareClient sends a data stream to the server application, invoking send_DCS_Mware to prepend an appropriate correlator data stream header. TerminateMwareClient terminates the middleware communication stream for the client application side.

[0067] Referring to FIG. 5B, DCS service <b>350</b> illustrates a base class used to provide a distributed context service, managing the correlators <b>10</b> for each provider <b>310</b> into entries of a table as defined by correlator table entry class <b>390</b>. Provider class <b>310</b> and middleware class <b>320</b> access Service <b>350</b>.

[0068] Service <b>350</b> includes start and stop methods for respectively initiating and terminating the DCS service class. To facilitate the registration and de-registration with the DCS Service of middleware identified by an instanceID, there is provided a registerMiddleware method and an unregisterMiddleware method as well as a middleware table defined by MwareTableEntry <b>370</b>.

[0069] Register and unregister methods facilitate registration and de-registration with the DCS service of a provider identified by a providerID. Preferably, the DCS service class is started before a register method is invoked. Otherwise, the DCS service may be started by the register method. The unregister method may stop the DCS service if there are no providers or middleware that remain registered with the DCS Service.

[0070] Other methods are provided for storing and accessing a correlator with the DCS Service. For example, pushCorrelator enters a correlator into the correlator table (via classes CT_Handler <b>380</b> and CorrelatorTableEntry <b>390</b>) to make the correlator available to middleware (via the DCS Service) for transporting on an outbound call from client <b>300</b>. PullCorrelator retrieves a provider's correlator from the DCS service correlator table. By pulling a correlator from the service class, the thread sequence counter and process sequence counter are incremented by the DCS service class as discussed previously.

[0071] The removeAllCorrelator method removes all correlator entries (regardless of threadID) from the correlator table for a particular provider. The suspendcorreletor method (assisted by ThreadTableEntry class <b>360</b>) suspends a provider from updating a correlator for a particular thread until the resumeCorrelator method is invoked for the particular correlator. Preferably, a correlator may be updated even if suspended, to include a partner correlator.

[0072] SetPartnerDataStream method, invoked by a middleware, sets the partner correlator for all providers identified in the correlator received by the middleware on an inbound call. The GetDataStream method provides to a middleware the correlator for a particular threadID in a byte stream, preferably in a format supported by the middleware, to transport on an outbound middleware call.

[0073]FIGS. 6A, 6B and <b>6</b>C illustrate a calling sequence for the classes of FIGS. 5A and 5B, for generating and transporting correlators. With reference to the call flow of FIG. 6A, an instance of an application such as client class <b>300</b> of FIG. 5A performs a self-initialization at call <b>1</b> (ClientStart( )) and at call <b>2</b> performs a first call to the transport mechanism (a middleware plug-in instance of class <b>320</b>). Middleware plug-in <b>320</b> of FIG. 5A starts an instance of service class <b>350</b> at call <b>3</b> and if such is successful, registers with the service instance at call <b>4</b>, providing a middleware instanceID. An instance of provider service class <b>310</b>, is instantiated by client <b>300</b> (not shown) and the provider calls start( ) of the DCS service <b>350</b> of FIG. 5B at call <b>5</b> to ensure the service is started. If so, the provider instance registers with the service at call <b>6</b>. Provider proceeds to configure a new logging data object including a correlator at calls <b>7</b> and <b>8</b>. Provider desires a certain optional SOAP parameter <b>15</b> (e.g. a time stamp) to be included in the new correlator <b>10</b> of FIG. 5A and invokes a method of the logging data <b>330</b> to add the parameter <b>15</b> of FIG. 5B (call <b>9</b>) which in turn calls correlator class object <b>10</b> to set the optional data into the correlator at call <b>10</b>. Once a correlator is populated with the parameter, the correlator is stored with the service <b>350</b> by provider <b>310</b> at call <b>11</b>. The provider passes the threadID and providerID for the correlator to the DCS service to facilitate the completion of a new correlator. The service in turn invokes the CT_EntryHandler class <b>390</b> to store the correlator instance <b>10</b>. The provider <b>310</b> is now ready to handle particular events of client <b>300</b>.

[0074]FIG. 6B shows a calling sequence for an exemplary cross product/process calling sequence. Application <b>300</b> signals an event to provider <b>310</b> (call <b>1</b>) which pulls a correlator from the service (call <b>2</b>) for the process and thread instance of application <b>300</b>. In turn, CT_EntryHandler pulls the identified correlator from the table (call <b>3</b>) and invokes a method of the correlator class <b>10</b> (call <b>4</b>) to increment the thread sequence counter and process sequence counter, storing the values in the correlator before returning same back to the requesting provider plug-in. Provider plug-in updates particular desired optional parameters through a logging data class call <b>5</b> that in turn updates the correlator at call <b>6</b>. The updated correlator is stored with the service at calls <b>7</b> and <b>8</b>. Provider plug-in records the logging data including the correlator at call <b>9</b> and requests service to suspend activity for the particular correlator at call <b>10</b>.

[0075] At call <b>11</b> of FIG. 6B, application <b>300</b> invokes the transport via middleware plug-in to perform a cross process/product function call or communication. At call <b>12</b>, middleware requests all correlators for the particular threadID from service <b>350</b> as a data stream to transport with the outgoing communication. At call <b>13</b>, service requests all context for the threadID from the CT_EntryHandler. At call <b>14</b>, middleware sends the communication including the correlator. Upon return of a response to the communication at call <b>15</b>, middleware removes the returning correlator and passes same to service at call <b>16</b> for setting as a partner correlator at call <b>17</b>. Application receives the response at call <b>18</b> to the send of call <b>11</b> and invokes the provider plug-in to note a new event. Provider resumes the previously suspended correlator at call <b>20</b> and pulls the correlator from service at call <b>21</b>. This provokes a pull from the CT_entryHandler at call <b>22</b> and a corresponding increment of the thread sequence counter and provider sequence counter to uniquely identify the event. Call <b>24</b> to logging data and corresponding call <b>25</b> to the correlator object sets context data into the correlator while calls <b>26</b> and <b>27</b> push the updated correlator back into the service and its table. Call <b>28</b> by provider plug-in records the logging data including the correlator. Note that a retrieval of a correlator (i.e. pullcorrelator( )) having an associated partner correlator provides an opportunity to flush the partner correlator from the correlator, once the association has been logged, so that subsequent uses of the correlator do not include references to a potentially misleading partner correlator.

[0076]FIG. 6C shows a calling sequence to bring down the service and the provider for a particular thread of an application. Calls <b>1</b> through <b>3</b> show a final pull of a correlator from the service and its table, triggering an increment of the counters. Provider plug-in calls to set context data into the correlator through logging data class at calls <b>4</b> and <b>5</b>. The updated correlator is returned to the service and table at calls <b>6</b> and <b>7</b> and the logging data recorded at call <b>8</b>. To bring down the service for the thread, provider plug-in requests that all correlators for the thread be removed by the service at call <b>9</b>. Following removal of the correlators through the table entry handler <b>390</b> of FIG. 5C at call <b>10</b>, the provider de-registers with the service at call <b>11</b> and exits from the transport at call <b>12</b>. Middleware also de-registers from service at call <b>13</b>. Service stops at call <b>14</b>, having nothing further to serve. Application may also invoke a self-stop method at call <b>15</b>.

[0077] The embodiment(s) of the invention described above is (are) intended to be exemplary only. The scope of the invention is therefore intended to be limited solely by the scope of the appended claims.