Data processing arrangement and method

CROSS-REFERENCE TO RELATED APPLICATION

[0001] This application is related to the commonly owned copending application, incorporated herein by reference, filed on the same date as the present application, titled “DATA PROCESSING ARRANGEMENT AND METHOD” and having applicant's docket GB920020001 GB1.

FIELD OF THE INVENTION

[0002] This invention relates to data processing and more particularly to a data processing arrangement and method for fuzzy backup and restore of objects therein.

BACKGROUND OF THE INVENTION

[0003] IBM's MQSeries® is a set of middleware products that allow user applications to intercommunicate using messages, without having to know the complexities of the underlying hardware and software platform. Applications communicate using the MQSeries application programming interface (API), issuing such calls as MQPUT to put a message onto a queue and MQGET to get a message from a queue. In this specification, the terms “get” and “put” are used generally interchangeably with the terms “read” and “write”, respectively. MQSeries is described in such publications as MQSeries Planning Guide, IBM GC33-1349-07 (January 1999), incorporated herein by reference.

[0004] The IBM S/390® Parallel Sysplex® configuration is a cluster of interconnected processing nodes with attachments to shared storage devices, network controllers, and core cluster technology components, consisting of coupling facilities, coupling support facilities, and external time references (ETRs). A coupling facility (CF) enables high-performance read/write sharing of data by applications running on each node of the cluster through global locking and cache coherency management mechanisms. It also provides cluster-wide queuing mechanisms for workload distribution and message passing between nodes.

[0005] The coupling facility is described in the following patents and publications, incorporated herein by reference:

[0006] “IN A MULTIPROCESSING SYSTEM HAVING A COUPLING FACILITY, COMMUNICATING MESSAGES BETWEEN THE PROCESSORS AND THE COUPLING FACILITY IN EITHER A SYNCHRONOUS OPERATION OR AN ASYNCHRONOUS OPERATION”, by D. A. Elko et al., Ser. No. 08/420,893 (docket P0991006X), filed Apr. 11, 1995, now U.S. Pat. No. 5,561,809;

[0007] “Sysplex Shared Data Coherency Method And Means”, by D. A. Elko et al., Ser. No. 07/860,805 (docket P0991052), filed Mar. 30, 1992, now U.S. Pat. No. 5,537,574;

[0008] “Method And Apparatus For Distributed Locking Of Shared Data, Employing A Central Coupling Facility”, by D. A. Elko et al., Ser. No. 07/860,808 (docket P0991059), filed Mar. 30, 1992, now U.S. Pat. No. 5,339,427;

[0009] “Command Quiesce Function”, by D. A. Elko et al., Ser. No. 07/860,330 (docket P0991062), filed Mar. 30, 1992, now U.S. Pat. No. 5,339,405;

[0010] “Software Cache Management Of A Shared Electronic Store In A Sysplex”, by D. A. Elko et al., Ser. No. 07/860,807 (docket P0991078), filed Mar. 30, 1992, now U.S. Pat. No. 5,457,793;

[0011] “Multiple Processor System Having Software For Selecting Shared Cache Entries Of An Associated Castout Class For Transfer To A DASD With One I/O Operation”, by D. A. Elko et al , Ser. No. 07/860,806 (docket P0991079), filed Mar. 30, 1992, now U.S. Pat. No. 5,493,668;

[0012] “Command Execution System For Using First And Second Commands To Reserve And Store Second Command Related Status Information In Memory Portion Respectively”, by D. A. Elko et al., Ser. No. 07/860,378 (docket P0992002), filed Mar. 30, 1992, now U.S. Pat. No. 5,392,397;

[0013] “Integrity Of Data Objects Used To Maintain State Information For Shared Data At A Local Complex”, by D. A. Elko et al., Ser. No. 07/860,800 (docket P0992003), filed Mar. 30, 1992, now U.S. Pat. No. 5,331,673;

[0014] “Management Of Data Objects Used To Maintain State Information For Shared Data At A Local Complex”, by J. A. Frey et al., Ser. No. 07/860,797 (docket P0992004), filed Mar. 30, 1992, now U.S. Pat. No. 5,388,266;

[0015] “Clearing Data Objects Used To Maintain State Information For Shared Data At A Local Complex When At Least One Message Path To The Local Complex Cannot Be Recovered”, by J. A. Frey et al., Ser. No. 07/860,647 (docket P0992005), filed Mar. 30, 1992, now U.S. Pat. No. 5,394,542;

[0016] “Coupling Facility For Receiving Commands From Plurality Of Hosts For Activating Selected Connection Paths To I/O Devices And Maintaining Status Thereof”, by D. A. Elko et al., Ser. No. 08/324,447 (docket P0992006X), filed Oct. 18, 1994, now U.S. Pat. No. 5,463,736;

[0017] “Data Processing System And Method For Providing Notification In A Central Processor Of State Changes For Shared Data Structure On External Storage”, by J. A. Frey et al., Ser. No. 07/860,809 (docket P0992007), filed Mar. 30, 1992, now U.S. Pat. No. 5,390,328;

[0018] “Method And Apparatus For Performing Conditional Operations On Externally Shared Data”, by J. A. Frey et al., Ser. No. 08/383,532 (docket P0992008X), filed Feb. 1, 1995, now U.S. Pat. No. 5,742,830;

[0019] “Apparatus And Method For List Management In A Coupled Data Processing System”, by J. A. Frey et al., Ser. No. 07/860,633 (docket P0992009), filed Mar. 30, 1992, now U.S. Pat. No. 5,410,695;

[0020] “Interdicting I/O And Messaging Operations In A Multi-System Complex”, by D. A. Elko et al., Ser. No. 07/860,489 (docket P0992010), filed Mar. 30, 1992, now U.S. Pat. No. 5,394,554;

[0021] “Method And Apparatus For Coupling Data Processing Systems”, by D. A. Elko et al., Ser. No. 07/860,803 (docket P0992012), filed Mar. 30, 1992, now U.S. Pat. No. 5,317,739;

[0022] “Authorization Method For Conditional Command Execution”, by D. A. Elko et al., Ser. No. 08/408,446 (docket P0992018X), filed Mar. 22, 1995, now U.S. Pat. No. 5,450,590;

[0023] “Dynamically Assigning a Dump Space in a Shared Data Facility to Receive Dumping Information to be Captured”, by D. A. Elko et al., Ser. No. 08/471,895 (docket P0992068B), filed Jun. 7, 1995, now U.S. Pat. No. 5,664,155;

[0024] “Method And System For Capturing and Controlling Access To Information In A Coupling Facility”, by D. E. Neuhard et al., Ser. No. 08/146,647 (docket KI993004A), filed Nov. 1, 1993, now U.S. Pat. No. 5,630,050;

[0025] “Method and System for Determining and Overriding Information Unavailability Time at a Coupling Facility”, by D. A. Neuhard et al., Ser. No. 08/779,196 (docket KI993004B), filed Jan. 6, 1997, now U.S. Pat. No. 5,875,484;

[0026] “Requesting a Dump of Information Stored within a Coupling Facility, in Which the Dump Includes Serviceability Information from an Operating System That Lost Communication with the Coupling Facility”, by D. A. Neuhard et al., Ser. No. 08/779,195 (docket KI993004C), filed Jan. 6, 1997, now U.S. Pat. No. 5,860,115;

[0027] “Method and Apparatus for Expansion, Contraction, and Reapportionment of Structured External Storage Structures”, by D. J. Dahlen et al., Ser. No. 08/304,458 (docket KI993006A), filed Sep. 12, 1994, now U.S. Pat. No. 5,581,737;

[0028] “Method of Managing Resources in One or More Coupling Facilities Coupled to One or More Operating Systems in One or More Central Programming Complexes Using a Policy”, by R. A. Allen et al., Ser. No. 08/607,053 (docket KI993008X), filed Feb. 26, 1996, now U.S. Pat. No. 5,634,072;

[0029] “Method and System for Managing Data and Users of Data in a Data Processing System”, by R. A. Allen, Ser. No. 08/146,727 (docket KI993009), filed Nov. 1, 1993, now U.S. Pat. No. 5,465,359;

[0030] “Method and System for Reconfiguring a Storage Structure Within a Structure Processing Facility”, by R. A. Allen et al., Ser. No. 08/544,941 (docket KI993010AX), filed Oct. 18, 1995, now U.S. Pat. No. 5,515,499;

[0031] “Method for Coordinating Executing Programs in a Data Processing System”, by R. A. Allen et al., Ser. No. 08/439,269 (docket KI993010B), filed May 9, 1995, now U.S. Pat. No. 5,604,863;

[0032] “Coherence Controls for Store-Multiple Shared Data Coordinated by Cache Directory Entries in a Shared Electronic Storage”, by K. S. Carpenter et al., Ser. No. 08/148,707 (docket P0993047), filed Nov. 8, 1993, now U.S. Pat. No. 5,544,345;

[0033] “Method and System for Log Management in a Coupled Data Processing System”, by R. V. Geiner et al., Ser. No. 08/632/683 (docket KI994019X), filed Apr. 15, 1996, now U.S. Pat. No. 5,737,600; and

[0034] J. M. Nick et al., “S/390 cluster technology: Parallel Sysplex”, IBM Systems Journal, vol. 36, no. 2, 1997, pages 172-201.

[0035] In the field of this invention it is known that fuzzy backup of a database can be achieved by copying the database in parallel with updates by a database engine to create a ‘fuzzy’ backup, and then (if required) restoring the database from that fuzzy backup and forward recovering it by application of log records cut by the database engine which describe the updates that were performed since the start of the fuzzy backup.

[0036] In MQSeries systems it is possible to exploit the Coupling Facility List Structure architecture for the storage of messages on shared queues.

[0037] However, this approach has the disadvantages that:

[0038] 1. For queues consisting largely of messages which exist for a short time the cost, size and duration of that fuzzy backup is much higher than necessary.

[0039] 2. High fuzzy backup costs result in less frequent fuzzy backups for a given cost. Less frequent and longer fuzzy backup results in more update data on the log between each fuzzy backup. Restore time is a function of the fuzzy backup size and the amount of update data to be replayed from the log since the beginning of the last complete fuzzy backup. So restore time is also longer than necessary.

[0040] STATEMENT OF INVENTION

[0041] Accordingly, the invention provides a data processing arrangement comprising: at least one data processing system; store means coupled to the at least one data processing system and arranged to store data associated with the at least one processing system; backup means for holding a backup of data held in the store means; record log means for recording changes in data stored in the store means; and restore means coupled to the store means, the backup means and the record log means for restoring store means data from the backup means and the record log means in event of failure; and backup testing means for testing the age of elements of store means data and for backing up the elements of store means data only if the ages are greater than a predetermined threshold.

[0042] According to another aspect, the invention provides a method of operation in a data processing arrangement comprising: at least one data processing system; store means coupled to the at least one data processing system and arranged to store data associated with the at least one processing system; backup means for holding a backup of data held in the store means; record log means for recording changes in data stored in the store means; and restore means coupled to the store means, the backup means and the record log means for restoring store means data from the backup means and the record log means in event of failure, the method comprising in the backup means performing backup only of elements of store means data which are older than a predetermined threshold.

[0043] According to another aspect, the invention provides a computer program for backing up data in a data processing system, the computer program comprising program code means adapted to perform the steps of: backing up only elements which are older than a predetermined threshold; and recording changes to the data in the data processing system.

[0044] Preferably, the program code means is further adapted to perform the step of: restoring data from the backup and the changes recorded.

[0045] Preferably the program code means is further adapted to perform a restore of messages from the backup and from replay of the recorded changes from a time substantially equal to the predetermined threshold before start of the backup.

BRIEF DESCRIPTION OF THE DRAWINGS

[0046] A preferred embodiment of the present invention will now be described, by way of example only, and with reference to the following drawings:

[0047]FIG. 1 shows an MQSeries multiple-system complex (sysplex) incorporating a preferred embodiment of the present invention;

[0048]FIG. 2 shows a block-schematic illustration of the coupling facility list structure used in the sysplex of FIG. 1; and

[0049]FIG. 3 shows a flowchart illustrating the ‘message backup’ method sequence in the CF used in the system of FIG. 1.

DESCRIPTION OF PREFERRED EMBODIMENT

[0050] The following terms, used throughout this description, are reproduced here for ready reference.

[0051] A “backout” (or “abort”) is an operation that reverses all the changes made during the current unit of work. After the operation is complete, a new unit of work begins.

[0052] A “commit” is an operation that applies all the changes made during the current unit of work. After the operation is complete, a new unit of work begins.

[0053] A “syncpoint” is an intermediate or end point during processing of a transaction at which the transaction's protected resources are consistent. At a syncpoint, changes to the resources can safely be committed, or they can be backed out to the previous syncpoint.

[0054] A “unit of work” is a recoverable sequence of operations performed by an application between two points of consistency. A unit of work begins when a transaction starts or after a user-requested syncpoint. It ends either at a user-requested syncpoint or at the end of a transaction.

[0055]FIG. 1 shows the general configuration of an MQSeries system complex (sysplex) <b>100</b> incorporating the present invention. Sysplex <b>100</b> comprises a plurality of systems <b>102</b> (an exemplary two of which, System A and System B, are shown), each of which is connected to a coupling facility (CF) <b>104</b> within which one or more shared message queues <b>106</b> are maintained. As is conventional, each system <b>102</b> comprises one or more processors and an operating system (not separately shown) and may constitute either a separate physical machine or a logical partition of a logically partitioned machine. Similarly, coupling facility <b>104</b> may constitute either a separate physical machine or a logical partition of a logically partitioned machine. Although the invention is not so limited, in a preferred embodiment the physical machines may be IBM S/390 Parallel Enterprise Server® processors, while the operating system is the IBM OS/390® operating system.

[0056] Within each system <b>102</b>, user applications <b>110</b> perform sequences of operations, referred to herein as units of work (UOWs), on shared message queues <b>106</b> via a shared queue manager (SQM) <b>108</b> resident on that system. Shared queue managers <b>108</b> on different systems <b>102</b> that share the same message queue <b>106</b> are referred to herein as a queue-sharing group (QSG). Each shared queue manager <b>108</b> in turn interacts with the coupling facility via a component <b>112</b> of the OS/390 operating system known as Sysplex Services for Data Sharing (XES). Sysplex Services for Data Sharing (XES) is described more fully in the IBM publication MVS/ESA Programming: Sysplex Services Guide, GC28-1495-02 (June 1995), incorporated herein by reference.

[0057] User applications <b>110</b> on a system <b>102</b> interact with the shared queue manager <b>108</b> on that system via a series of Message Queue Interface (MQI) calls, the most important of which are MQPUT (to put a message onto a message queue <b>106</b>) and MQGET (to get a message from a message queue <b>106</b>). In response to such a call, shared queue manager <b>108</b> issues a call to XES <b>112</b> to invoke a requested system service to communicate with the coupling facility <b>104</b>. In response to receiving a call from a shared queue manager <b>108</b>, XES <b>112</b> sends an appropriate message command block (MCB) containing data and other operands to the coupling facility <b>104</b> to perform the requested operation. XES <b>112</b> receives back from the coupling facility <b>104</b> a message response block (MRB) containing data and/or a response code. As part of its list-processing capability, coupling facility <b>104</b> performs requested operations on lists composed of one or more entries.

[0058] Each shared queue manager <b>108</b> maintains a record log <b>114</b> of all message transactions involving the CF <b>104</b>. Periodically, one of the shared queue managers <b>108</b> uses XES services <b>112</b> to read the contents of the coupling facility <b>104</b> and record them in a fuzzy backup <b>116</b>.

[0059] Shared queue messages are stored as list entries in one or more coupling facility (CF) list structures in the CF <b>104</b>. Applications running on multiple queue managers in the same queue sharing group (QSG) anywhere in the sysplex <b>100</b> can then access these shared queue messages. This provides continuous availability, scalable capacity, and automatic pull workload balancing.

[0060] Support for persistent shared queue messages can be provided by periodically taking nondisruptive fuzzy backups of each CF list structure, and having each queue manager in the QSG log (i) the message identification and message content of each shared queue message it MQPUTs and (ii) the message identification of each shared queue message it MQGETs. If the CF list structure should fail its shared queue messages can then be recovered by restoring a latest fuzzy backup of that CF structure, positioning the recovery log of each queue manager in the QSG to the time of the fuzzy backup, and then replaying the shared queue update operations from these logs in time stamp order across the QSG into the CF structure.

[0061] If a CF structure should fail then a user will want to recover it from the fuzzy backup and recovery logs as quickly as possible. The time taken to replay the recovery logs is typically the major portion of this recovery time. This can be minimised by taking frequent fuzzy backups and by efficient processing during the recovery log replay process.

[0062] The present invention preferably allows fuzzy backups of coupling facility list structures containing shared queue messages to be made smaller and more efficient to minimise restore time. The advantage is that the size of the fuzzy backup is smaller and takes less time and resource to create. This therefore affords the opportunity to take more frequent fuzzy backups, which reduces the amount of recovery log to replay and thus minimises restore time.

[0063] It has been recognised by the inventor(s) that, in a well behaved queue manager system such as an MQSeries queue manager system, most shared queue messages reside on a queue for a short period of time only, and can safely be excluded from the fuzzy backup if at any subsequent restore the recovery log is replayed from a time slightly before the fuzzy backup start time.

[0064] Referring now to FIG. 2, in a list structure <b>200</b> in the CF <b>106</b>, each shared queue corresponds to a list header <b>210</b>. Each shared queue message corresponds to a list entry <b>220</b> within that list header. A unique time of creation indicator <b>230</b> is present in the key of each CF list structure element.

[0065] As will be described in more detail below, based on the unique time of creation indicator <b>230</b> in each message, during backup only messages older than some given age (in the example of this preferred embodiment, approximately 30 seconds) are written to the fuzzy backup <b>116</b>. This is straightforward and efficient as the unique time of creation indicator <b>230</b> is present in the key of each CF list structure element.

[0066] Referring now to FIG. 3, the procedure used in the sysplex <b>100</b> to produce a fuzzy backup <b>116</b> consists of the following steps: <tables id="TABLE-US-00001" num="1"><table frame="none" colsep="0" rowsep="0"><tgroup align="left" colsep="0" rowsep="0" cols="3"><colspec colname="OFFSET" colwidth="21PT" align="left" /><colspec colname="1" colwidth="49PT" align="left" /><colspec colname="2" colwidth="147PT" align="left" /><thead><row><entry /><entry /></row><row><entry /><entry namest="OFFSET" nameend="2" align="center" rowsep="1" /></row></thead><tbody valign="top"><row><entry /><entry>Step 310</entry><entry>The unique time of creation indicator</entry></row><row><entry /><entry /><entry>230 of the next CF message is obtained</entry></row><row><entry /><entry /><entry>from the CF list structure</entry></row><row><entry /><entry>Step 320</entry><entry>The age of the message is calculated</entry></row><row><entry /><entry /><entry>from the unique time of creation</entry></row><row><entry /><entry /><entry>indicator 230, and tested to determine</entry></row><row><entry /><entry /><entry>if the age of the message is greater</entry></row><row><entry /><entry /><entry>than 30 seconds. If the message is not</entry></row><row><entry /><entry /><entry>older than 30 seconds, the procedure</entry></row><row><entry /><entry /><entry>returns to step 310.</entry></row><row><entry /><entry>Step 330</entry><entry>If the message is older than 30</entry></row><row><entry /><entry /><entry>seconds, the message is backed up in</entry></row><row><entry /><entry /><entry>the fuzzy backup 116.</entry></row><row><entry /><entry>Step 340</entry><entry>The CF is tested to determine whether</entry></row><row><entry /><entry /><entry>there are further messages in the CF</entry></row><row><entry /><entry /><entry>104 that may require backing up. If the</entry></row><row><entry /><entry /><entry>end of messages has not been reached,</entry></row><row><entry /><entry /><entry>the procedure returns to step 310. If</entry></row><row><entry /><entry /><entry>the end of messages has been reached,</entry></row><row><entry /><entry /><entry>the fuzzy backup ends and the procedure</entry></row><row><entry /><entry /><entry>terminates.</entry></row><row><entry /><entry namest="OFFSET" nameend="2" align="center" rowsep="1" /></row></tbody></tgroup></table></tables>

[0067] Typically, more than 80% of messages on a shared queue at a given instant remain on that queue for a time of the order of seconds rather than minutes. Thus, fuzzy backup of messages older than a few seconds (e.g., 30 seconds is significantly cheaper (in terms of resource cost) and quicker than a fuzzy backup of all messages.

[0068] Thus, more frequent fuzzy backups can be afforded. Typically four or five times as many ‘except most recent message’ backups can be performed for the cost of a full fuzzy backup. Without this technique, the interval between fuzzy backups would typically be of the order of several tens of minutes (if not hours). With this technique, the interval between affordable fuzzy backups can be significantly reduced. This reduction is of the order of four to five times for the typical case where more than 80% of messages exist for less than a few seconds. Thus, the reduction is typically of the order of minutes (if not tens of minutes).

[0069] For the less typical case where messages are mostly long-lived, the cost is not significantly greater than it would have been without this technique, although the advantage is also significantly reduced.

[0070] In the event of a CF failure, the scope of recovery log which must be replayed without this technique is on average half the interval between fuzzy backups. With this technique it is on average half of the shorter (by minutes, if not tens of minutes) interval between fuzzy backups plus a few extra seconds of recovery log replay necessary to restore messages younger than the given age at the time of last fuzzy backup.

[0071] For the less typical case where messages are mostly long lived the restore time is longer but only by the few extra seconds of recovery log replay.

[0072] Thus this technique reduces restore time, because the scope of recovery log which must be replayed to restore a CF list structure containing shared queue messages is reduced by the order of minutes (if not tens of minutes).

[0073] It will be understood that although the invention has been described above in the context of backing up messages from a coupling facility list structure in an MQSeries sysplex, the invention is not necessarily limited to such backup and may be used more generally in other backup arrangements in which data records have an indication of age. Further, it will be understood that the ‘threshold age’ of 30 seconds used in the example described above may alternatively be lesser or greater than 30 seconds, as desired.

[0074] Thus, in summary, it will be understood that the technique described above for reducing fuzzy backup costs and restore time by not backing up most recent objects described above provides the advantages of reduction of fuzzy backup cost and reduction of restore time.