Managing operation requests using different resources

RELATED APPLICATIONS

0001This application is a continuation application of copending parent application Ser. No. 10/719,070, filed Nov. 20, 2003, assigned to the assignee of the present application, and incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

00021. Field of the Invention

0003The present invention relates to a method, system, and program for managing resources for competing tasks including operations, operation requests, or processes.

00042. Description of the Related Art

0005Computing systems often include one or more host computers (“hosts”) for processing data and running application programs, direct access storage devices (DASDs) for storing data, and a storage controller for controlling the transfer of data between the hosts and the DASD. Storage controllers, also referred to as control units or storage directors, manage access to a storage space often comprised of numerous hard disk drives connected in a loop architecture, otherwise referred to as a Direct Access Storage Device (DASD). Hosts may communicate Input/Output (I/O) requests to the storage space through the storage controller.

0006To maintain availability in the event of a failure, many storage controllers known in the prior art provide redundant hardware clusters. Each hardware cluster comprises a processor complex, cache, non-volatile storage (NVS), such as a battery backed-up Random Access Memory (RAM), and separate power supply to provide connection paths to the attached storage. The NVS in one cluster backs up write data from the cache in the other cluster so that if one cluster fails, the write data in the cache of the failed cluster is stored in the NVS of the surviving cluster. After one cluster fails, all Input/Output (I/O) requests would be directed toward the surviving cluster. When both clusters are available, each cluster may be assigned to handle I/O requests for specific logical storage devices configured within the physical storage devices.

0007In performing these and other tasks, a cluster can not only execute operations locally using the capabilities of the local cluster itself, but can also make a request to have an operation executed on a remote cluster in the storage controller system. Moreover, each storage controller may have an adapter to communicate over a network with a host computer or another storage controller. For example, the write instructions from the primary storage controller to a secondary storage controller may be generated by a processor of the adapter of the primary storage controller and issued over the network to the secondary storage controller.

0008Each operation may consume a significant portion of the resources of the cluster or adapter which is performing the requested operation. These resources may include the bandwidth of a bus or the bandwidth of a network connection, as well as the computational capabilities of the processors of the cluster or adapter.

0009Various techniques have been used to manage the allocation of resources so that operations proceed in an appropriate fashion. For example, operations may be assigned a priority such as low, medium or high. A high priority operation tends to receive a resource more quickly than a medium priority operation which tends to receive a resource more quickly than a low priority operation. However, in order to prevent higher priority operations from blocking the execution of all lower priority operations, various schemes have been used to ensure that a certain minimum number or proportion of lower priority operations are permitted to proceed. For example, the number of operations of each priority type may be counted and higher priority operations may be deferred on occasion to permit a minimum number or proportion of lower priority operations to proceed. Another scheme keeps track of how long a lower priority operation is caused to wait and may increase the priority of the operation as time passes until the operation has sufficient priority to proceed.

0010Further improvements in allocating resources amongst competing operations may be useful in a variety of applications.

SUMMARY OF ILLUSTRATED EMBODIMENTS

0011Provided is a method, system and program for managing tasks using different resources. In one embodiment, a first attempt is made to acquire a first resource for a task requiring both a first resource and a second resource. The task is enqueued on a first queue if the first attempt to acquire the first resource for the task fails. Upon acquiring the first resource in a subsequent attempt, the task is removed from the first queue and an attempt is made to acquire the second resource for the task. The task is enqueued on a second queue if the attempt to acquire the second resource fails and the first resource is released.

0012There are additional aspects to the present inventions. It should therefore be understood that the preceding is merely a brief summary of some embodiments and aspects of the present inventions. Additional embodiments and aspects of the present inventions are referenced below. It should further be understood that numerous changes to the disclosed embodiments can be made without departing from the spirit or scope of the inventions. The preceding summary therefore is not meant to limit the scope of the inventions. Rather, the scope of the inventions is to be determined by appended claims and their equivalents.

BRIEF DESCRIPTION OF THE DRAWINGS

0013Referring now to the drawings in which like reference numbers represent corresponding parts throughout:

0014<figref idref="DRAWINGS">FIG. 1</figref> illustrates an example of a computing environment in which aspects of the invention are implemented;

0015<figref idref="DRAWINGS">FIG. 2</figref> illustrates queuing of requests when a request which utilizes a first resource is received, in accordance with aspects of the invention;

0016<figref idref="DRAWINGS">FIG. 3</figref> illustrates queuing of requests when a request which utilizes both a first and second resource is received, in accordance with aspects of the invention;

0017<figref idref="DRAWINGS">FIG. 4</figref> illustrates dispatching a request from a queue when a second of two resources is freed, in accordance with aspects of the invention;

0018<figref idref="DRAWINGS">FIGS. 5A-5B</figref> illustrate dispatching a request from a queue when a first of two resources is freed, in accordance with aspects of the invention;

0019<figref idref="DRAWINGS">FIG. 6</figref> illustrates queues for queuing requests which utilizes one or both of a first and second resource, in accordance with aspects of the invention; and

0020<figref idref="DRAWINGS">FIG. 7</figref> illustrates an architecture of computing components in a network environment, such as the hosts, storage controllers, clusters, and any other computing devices.

DETAILED DESCRIPTION OF ILLUSTRATED EMBODIMENTS

0021In the following description, reference is made to the accompanying drawings which form a part hereof and which illustrate several embodiments of the present invention. It is understood that other embodiments may be utilized and structural and operational changes may be made without departing from the scope of the present invention.

0022<figref idref="DRAWINGS">FIG. 1</figref> illustrates a computing architecture in which aspects of the invention are implemented. One or more hosts <b>102</b><i>a</i>, <b>102</b><i>b </i>. . . <b>102</b><i>n </i>are in data communication with a primary storage system <b>104</b><i>a</i>, such as a DASD or any other storage system known in the art, via a primary storage controller <b>106</b><i>a</i>. In a similar manner, the hosts <b>102</b><i>a</i>, <b>102</b><i>b </i>. . . <b>102</b><i>n </i>are in data communication with a secondary storage system <b>104</b><i>b</i>, such as a DASD or any other storage system known in the art, via a secondary storage controller <b>106</b><i>b</i>. The host <b>102</b> may be any computing device known in the art, such as a server, mainframe, workstation, personal computer, hand held computer, laptop, telephony device, network appliance, etc. The storage controllers <b>106</b><i>a</i>, <b>106</b><i>b </i>and host system(s) <b>102</b> communicate via a network <b>108</b>, which may comprise a Storage Area Network (SAN), Local Area Network (LAN), Intranet, the Internet, Wide Area Network (WAN), etc. The primary and secondary storage systems <b>104</b><i>a</i>, <b>104</b><i>b </i>may each be comprised of hard disk drives, tape cartridge libraries, optical disks, or any suitable non-volatile storage medium known in the art.

0023The primary and secondary storage systems <b>104</b><i>a</i>, <b>104</b><i>b </i>may each be arranged as an array of storage devices, such as a Just a Bunch of Disks (JBOD), DASD, Redundant Array of Independent Disks (RAID) array, virtualization device, etc. The primary and secondary storage controllers <b>106</b><i>a</i>, <b>106</b><i>b </i>may comprise any storage controller or server known in the art, such as the IBM Enterprise Storage Server (ESS) or any other storage controller known in the art. In certain implementations, the storage space in each of the storage systems <b>104</b><i>a</i>, <b>104</b><i>b </i>may be configured as a plurality of logical devices (LD) <b>110</b><i>a</i>, <b>110</b><i>b </i>. . . <b>110</b><i>n. </i>

0024The primary storage <b>104</b><i>a </i>or the logical devices <b>110</b><i>a</i>, <b>110</b><i>b </i>. . . <b>110</b><i>n </i>may each be divided into blocks of storage containing blocks of data, and the blocks of storage can be further divided into sub-blocks of storage that contain sub-blocks of data. In certain implementations, the blocks of data are contents of tracks, while the sub-blocks of data are contents of sectors of tracks. For ease of reference, the terms tracks and sectors will be used herein as examples of blocks of data and sub-blocks of data, but use of these terms is not meant to limit the technique of the invention to tracks and sectors. The techniques of the invention are applicable to any type of storage, block of storage or block of data divided in any manner.

0025The storage controllers <b>106</b><i>a</i>, <b>106</b><i>b </i>each include two separate clusters <b>120</b><i>a</i>, <b>120</b><i>b </i>of hardware components to provide redundancy for improved availability. Although the illustrated embodiment is described in connection with a storage controller having clusters, it is appreciated that the invention may be applicable to a variety of systems utilizing a single processor or multiple processors in which one processor can direct operations to be executed by another processor.

0026Each cluster <b>120</b><i>a</i>, <b>120</b><i>b </i>of the illustrated embodiment may be maintained on a separate power boundary, and includes a processor complex <b>122</b><i>a</i>, <b>122</b><i>b</i>, a cache <b>124</b><i>a</i>, <b>124</b><i>b</i>, and a non-volatile storage unit (NVS) <b>126</b><i>a</i>, <b>126</b><i>b</i>. The NVS <b>126</b><i>a</i>, <b>126</b><i>b </i>may comprise a battery backed-up RAM or any other type of non-volatile or volatile backup cache used to backup data in cache. The hosts <b>102</b><i>a</i>, <b>102</b><i>b </i>. . . <b>102</b><i>n </i>would submit application I/O requests directed to a target logical device (LD) <b>110</b><i>a</i>, <b>110</b><i>b</i>, <b>110</b><i>n</i>, including write data, to the cluster <b>120</b><i>a</i>, <b>120</b><i>b </i>to which the target logical device (LD) <b>110</b><i>a</i>, <b>110</b><i>b</i>, <b>110</b><i>n </i>is assigned. The NVS <b>126</b><i>a</i>, <b>126</b><i>b </i>in one cluster <b>120</b><i>a</i>, <b>120</b><i>b </i>is used to backup write data in the cache <b>124</b><i>b</i>, <b>124</b><i>a </i>in the other cluster <b>120</b><i>b</i>, <b>120</b><i>a</i>, e.g., NVS <b>126</b><i>a </i>backs up write data in cache <b>124</b><i>b. </i>

0027The storage controller <b>106</b><i>a</i>, <b>106</b><i>b </i>includes a read process for reading data, and a write process for writing data. The primary storage controller <b>106</b><i>a </i>also includes a sync process for transferring data from the primary storage controller <b>106</b><i>a </i>to remote storage, such as storage <b>104</b><i>b </i>at the secondary storage controller <b>106</b><i>b</i>. Each task or operation of each process is dispatched by an operation dispatcher, for execution, when the task acquires the controller resources needed to execute the task. The operation dispatcher may be implemented in software, firmware, hardware or a combination of these within the processor <b>122</b><i>a</i>, <b>122</b><i>b </i>or an adapter <b>130</b>.

0028In certain implementations, the sync process may perform synchronization of data for the synchronous PPRC® and for PPRC® Extended Distance. In such cases, the sync process runs continuously for PPRC® Extended Distance, and starts up and completes for host writes for the synchronous PPRC®. In certain implementations, there may be multiple sync processes (e.g., different sync processes for synchronous PPRC® and for PPRC® Extended Distance or one sync process for each volume on primary storage <b>104</b><i>a </i>for PPRC® Extended Distance). Some IBM mirroring systems are described in IBM publication “Remote Copy: Administrator's Guide and Reference,” IBM document no. SC35-0169-02 (IBM Copyright 1994, 1996).

0029In certain implementations, the read process, write process, and sync process are implemented as firmware. In certain implementations, the read process, write process and sync process are implemented as separate software programs for each process. In certain implementations, the read process, write process and sync process may be combined with each other or other software programs.

0030Secondary storage controller <b>106</b><i>b </i>provides one or more recovery systems access to disk storage, such as secondary storage <b>104</b><i>b</i>, which maintains back-up copies of all or a subset of the volumes of the primary storage <b>104</b><i>a</i>. Secondary storage may be a Direct Access Storage Device (DASD). Secondary storage <b>104</b><i>b </i>is also divided into blocks of storage containing blocks of data, and the blocks of storage are further divided into sub-blocks of storage that contain sub-blocks of data. In certain implementations, the blocks of data are tracks, while the sub-blocks of data are sectors of tracks. For ease of reference, the terms tracks and sectors will be used herein as examples of blocks of data and sub-blocks of data, but use of these terms is not meant to limit the technique of the invention to tracks and sectors. The techniques of the invention are applicable to any type of storage, block of storage or block of data divided in any manner.

0031In certain implementations, removable storage (instead of or in addition to remote storage, such as secondary storage <b>104</b><i>b </i>may be used to maintain back-up copies of all or a subset of the primary storage <b>104</b><i>a</i>, and the techniques of the invention transfer data to the removable storage rather than to the remote storage. The removable storage may reside at the primary storage controller <b>106</b><i>a. </i>

0032In certain implementations, the primary storage controller <b>106</b><i>a </i>and secondary storage controller <b>106</b><i>b </i>may be comprised of the IBM 3990, Model 6 Storage Controller, Enterprise Storage Server, or any other control unit known in the art.

0033In certain implementations, the primary storage controller <b>106</b><i>a </i>and/or secondary storage controller <b>106</b><i>b </i>may comprise any computing device known in the art, such as a mainframe, server, personal computer, workstation, laptop, handheld computer, telephony device, network appliance, virtualization device, storage controller, etc.

0034A primary site may include multiple primary storage controllers, primary storage, and host computers. A secondary site may include multiple secondary storage controllers, recovery systems, and secondary storage.

0035In certain implementations of the invention, data is maintained in volume pairs. A volume pair is comprised of a volume in a primary storage device (e.g., primary storage <b>104</b><i>a</i>) and a corresponding volume in a secondary storage device (e.g., secondary storage <b>104</b><i>b</i>) that includes a consistent copy of the data maintained in the primary volume. For example, primary storage <b>104</b><i>a </i>may include Volume1 and Volume2, and secondary storage <b>104</b><i>b </i>may contain corresponding Volume1 and Volume2. A primary storage controller may be provided to control access to the primary storage and a secondary storage controller may be provided to control access to the secondary storage.

0036In certain implementations, the primary storage controller <b>106</b><i>a </i>and secondary storage controller <b>106</b><i>b </i>communicate via communication paths, such as direct high speed transmission lines (e.g., an Enterprise System Connection (ESCON®), or Fiber connection (FICON) or Fibre Connection Protocol (FCP) link). However, the communication paths may be comprised of any other communication means known in the art, including network transmission lines, fiber optic cables, etc.

0037Each storage controller <b>106</b><i>a</i>, <b>106</b><i>b </i>of the illustrated embodiment further includes a network adapter <b>130</b> which permits each storage controller <b>106</b><i>a</i>, <b>106</b><i>b </i>to communicate with each of the hosts <b>102</b><i>a</i>, <b>102</b><i>b </i>. . . <b>102</b><i>n </i>as well as the other storage controller. For communication among the clusters and adapter of each storage controller <b>106</b><i>a</i>, <b>106</b><i>b</i>, each cluster <b>120</b><i>a</i>, <b>120</b><i>b </i>and adapter <b>130</b> has a mail manager process.

0038To write data to a target logical device <b>110</b><i>a</i>, <b>110</b><i>b </i>. . . <b>110</b><i>n</i>, a host such as host <b>102</b><i>a </i>issues a write command such as a SCSI (small computer system interface) embedded in a suitable network protocol frame as a Fibre Channel Protocol (FCP) frame to the primary storage controller <b>106</b><i>a</i>. The network adapter <b>130</b> of the primary storage controller <b>106</b><i>a </i>receives the write command and sends an appropriate write operation via its mail manager to the mail manager of the one of the clusters <b>120</b><i>a</i>, <b>120</b><i>b </i>of the primary storage controller <b>106</b><i>a. </i>

0039As previously mentioned, it is often useful for data to be written onto more than one storage device so that if the data is lost on one device, the data may be retrieved from the other device. One technique for achieving this redundant data writing is to pair a logical storage device <b>110</b><i>a</i>, <b>110</b><i>b </i>. . . <b>110</b><i>n </i>of one storage controller <b>106</b><i>a </i>with a logical storage device <b>110</b><i>a</i>, <b>110</b><i>b </i>. . . <b>110</b><i>n </i>of another storage controller <b>106</b><i>b</i>, for example. Thus, for example, the logical storage device <b>110</b><i>a </i>of the storage controller <b>106</b><i>a </i>may be paired with the logical storage device <b>110</b><i>a </i>of the storage controller <b>106</b><i>b</i>. In such an arrangement, the cluster <b>120</b><i>a </i>of the storage controller <b>106</b><i>a </i>can be programmed to recognize that a write operation to the logical storage device <b>110</b><i>a </i>of the storage controller <b>106</b><i>a </i>should be repeated on the logical storage device <b>110</b><i>a </i>of the storage controller <b>106</b><i>b. </i>

0040For synchronous PPRC®, before a host such as host <b>102</b><i>a </i>completes writing a chain of tracks to the primary storage controller <b>106</b><i>a</i>, all tracks in the chain will typically have also been transferred to the secondary storage controller <b>106</b><i>b</i>. In certain implementations, this transfer would be accomplished by a synchronization process which will typically have been completed before an acknowledgment of completion of the write process is sent to the host <b>102</b><i>a. </i>

0041For PPRC® Extended Distance, the host <b>102</b><i>a </i>will complete writing a track to the primary storage controller <b>106</b><i>a </i>without the track having been sent to the secondary storage controller <b>106</b><i>b</i>. After the track has been written to the primary storage controller <b>106</b><i>b</i>, the sync process will discover that an indicator corresponding to the track is set to indicate that the track is out of sync with a corresponding track at the secondary storage controller <b>106</b><i>b </i>and will send the track to the secondary storage controller <b>106</b><i>b</i>. That is, the track is sent asynchronously with respect to the track written by the host.

0042To send a track to the secondary storage controller <b>106</b><i>b</i>, the cluster <b>120</b><i>a </i>can send a mail message to the network adapter <b>130</b> of the primary storage controller <b>106</b><i>a</i>. In response, the network adapter <b>130</b> of the primary storage controller <b>106</b><i>a </i>can, in a manner similar to that of a host <b>102</b><i>a</i>, <b>102</b><i>b </i>. . . <b>102</b><i>n </i>issue a write command such as a SCSI (small computer system interface) command embedded in a suitable network protocol frame as a fiber channel Protocol (FCP) frame to the secondary storage controller <b>106</b><i>b</i>. The network adapter <b>130</b> of the secondary storage controller <b>106</b><i>b </i>receives the write command and sends an appropriate write operation via its mail manager to the mail manager of the one of the clusters <b>120</b><i>a</i>, <b>120</b><i>b </i>of the secondary storage controller <b>106</b><i>b. </i>

0043<figref idref="DRAWINGS">FIGS. 2-5B</figref> shows operations of an operation dispatcher of a cluster or adapter such as cluster <b>120</b><i>a </i>which can facilitate the allocation of a variety of resources to a variety of operations. In this example, two resources, resource A and resource B are being allocated for operations which utilize either resource A or both resource A and resource B. It is appreciated that more than two resources may be allocated utilizing the principles described herein. Also, in this example, resource A is a count of the number of permissible ongoing low and medium priority operations. This count is limited to a maximum number to limit the total number of low and medium priority operations which are permitted to be ongoing at a particular time. Once resource A is exhausted, no more low or medium priority operations will be dispatched until resource A is made available again by the completion of additional low or medium priority operations.

0044In this example, resource B is the bandwidth of a port and may be allocated in variable sized portions of the total bandwidth of the port. Once resource B is exhausted, that is the bandwidth of the port is fully utilized, no more operations which utilize resource B will be dispatched until resource B is made available again by the completion of operations which utilize the bandwidth of the port. Again, it is appreciated that other types of resources such as task control block (TCB) may be allocated as well. A TCB can function as a “ticket” needed for an operation to proceed in which the supply of TCB's is limited and is replenished by the completion of operations.

0045In one aspect of the illustrated embodiment, to facilitate the allocation of resource A and resource B for operations which utilize resource A and resource B, the operation dispatcher maintains a first set of wait queues <b>200</b> for resource A and a second set of wait queues <b>202</b> for resource B as shown in <figref idref="DRAWINGS">FIG. 6</figref>. The first set of wait queues <b>200</b> is provided for operations which utilize resource A, which in the illustrated embodiment, is a maximum number of permissible ongoing low and medium priority operations. In one aspect of the illustrated embodiment, each set of wait queues for each resource includes a wait queue for operations of a particular priority. For example, one wait queue <b>200</b><i>a </i>of wait queue set <b>200</b> is used to queue low priority operations which are waiting to acquire resource A. Another wait queue <b>200</b><i>b </i>of wait queue set <b>200</b> is used to queue medium priority operations which are waiting to acquire resource A. Similarly, another wait queue <b>202</b><i>a </i>is used to queue low priority operations which are waiting to acquire resource B. Another wait queue <b>202</b><i>b </i>of wait queue set <b>200</b> is used to queue medium priority operations which are waiting to acquire resource B. Still another wait queue <b>202</b><i>c </i>of wait queue set <b>202</b> is used to queue high priority operations which are waiting to acquire resource B. In the illustrated embodiments, the wait queues <b>200</b><i>a</i>-<b>202</b><i>c </i>are each FIFO (first in, first out) queues, although other types of queues may be used as well.

0046Referring first to the allocation operations of <figref idref="DRAWINGS">FIG. 2</figref>, when the operation dispatcher receives (block <b>204</b>) for dispatching a new operation request which utilizes resource A, a determination is made (block <b>206</b>) whether the wait queue for resource A for the priority of the received operation is empty. Thus, if the received operation is a low priority operation, the resource A low priority wait queue <b>200</b><i>a </i>is checked to see if other operations having the same priority, that is, low priority, are waiting to acquire resource A. If so, that is, if the low priority wait queue <b>200</b><i>a </i>is not empty, the received low priority operation is enqueued (block <b>208</b>) at the end of the resource A wait queue of the same priority, that is, wait queue <b>200</b><i>a </i>in this example.

0047If the resource A wait queue for operations of the same priority is empty (block <b>206</b>), a check (block <b>210</b>) is made to determine whether the resource A wait queue set <b>200</b> wait queues for higher priority operations are empty. Thus, if the received operation is a low priority operation, the resource A medium priority wait queue <b>200</b><i>b </i>is checked to see if other operations having a higher priority, that is, medium priority in this example, are waiting to acquire resource A. If so, that is, if the medium priority wait queue <b>200</b><i>b </i>is not empty, the received low priority operation is enqueued (block <b>208</b>) at the end of the resource A wait queue <b>200</b><i>a </i>for low priority resource A operations.

0048If the resource A wait queues for operations of higher priority are empty (block <b>206</b>) as well, a determination (block <b>212</b>) is made as to whether resource A is available for the received operation. If so, the received operation is permitted to acquire (block <b>214</b>) resource A. If resource A is not available (block <b>212</b>), the received low priority operation is enqueued (block <b>208</b>) at the end of the resource A wait queue <b>200</b><i>a </i>for low priority resource A operations.

0049Once resource A has been acquired (block <b>214</b>), the operation may be ready to be dispatched. Alternatively, the operation may need another resource such as a resource C to be dispatched. In one embodiment, the operation, once resource A is acquired, would be permitted to be dispatched when the next available resource C, such as a task control block (TCB), for example, becomes available and acquired.

0050However, some operations may utilize both resource A and resource B, both of which are to be allocated in accordance with priority. <figref idref="DRAWINGS">FIG. 3</figref> shows the operations of the operation dispatcher for such operations. When an operation request which utilizes both resource A and resource B is received (block <b>250</b>) by the operation dispatcher, an attempt (block <b>252</b>) is made to acquire resource A for the received operation in the manner described above in connection with <figref idref="DRAWINGS">FIG. 2</figref>. Accordingly, if the received operation request is not able to acquire resource A, the received operation is enqueued (block <b>254</b>) at the end of the resource A wait queue for operations of the same priority. Thus for example, if the received operation is a low priority operation, the operation will be enqueued at the end of the resource A wait queue <b>200</b><i>a </i>for low priority resource A operations if resource A could not be acquired.

0051If resource A was acquired (block <b>252</b>), a determination is made (block <b>256</b>) whether the wait queues of wait queue set <b>202</b> for resource B for the same priority of the received operation, or for a higher priority, are empty. Thus, if the received operation is a low priority operation, the resource B low priority wait queue <b>202</b><i>a </i>is checked to see if other operations having the same priority, that is, low priority, are waiting to acquire resource B. If so, that is, if the low priority wait queue <b>202</b><i>a </i>is not empty, the resource A is released (block <b>258</b>) and the received low priority operation is enqueued (block <b>260</b>) at the end of the resource B wait queue of the same priority, that is, wait queue <b>202</b><i>a </i>in this example.

0052If the resource B wait queue for operations of the same priority is empty, a check (block <b>256</b>) is also made to determine whether the resource B wait queue set <b>202</b> wait queues for higher priority operations are also empty. Thus, if the received operation is a low priority operation, the resource B medium priority wait queue <b>202</b><i>b </i>and the resource B high priority wait queue <b>202</b><i>c </i>are checked to see if other operations having a higher priority, that is, medium priority or high priority in this example, are waiting to acquire resource B. If so, that is, if either of the medium priority wait queue <b>202</b><i>b </i>or the high priority wait queue <b>202</b><i>c </i>is not empty, the resource A is released (block <b>258</b>) and the received low priority operation is enqueued (block <b>260</b>) at the end of the resource B wait queue of the same priority, that is, wait queue <b>202</b><i>a </i>in this example.

0053If the resource B wait queues for operations of higher priority are empty (block <b>256</b>) as well, a determination (block <b>262</b>) is made as to whether resource B is available for the received operation. If so, the received operation is permitted to acquire (block <b>264</b>) resource B. If resource B is not available (block <b>262</b>), the resource A is released (block <b>258</b>) and the received operation is enqueued (block <b>260</b>) at the end of the resource B wait queue of the same priority, that is, wait queue <b>202</b><i>a </i>in this example for a received low priority operation.

0054Once resource B has been acquired (block <b>264</b>), the operation request may be ready to be dispatched since resource A was previously acquired (block <b>252</b>). Alternatively, the operation request may need another resource such as a resource C to be dispatched. In one embodiment, the operation, once resource A and resource B are acquired, would be permitted to be dispatched when the next available resource C, such as a task control block (TCB), for example, becomes available and acquired.

0055It is seen from the above, in one aspect, operation requests are enqueued on wait queues for operations of various priorities, when an operation request is unable to acquire the resources needed to complete the requested operation. In addition, acquired resources can be released when an operation request enqueued on a wait queue is unable to acquire other needed resources. In this manner, the released resources are made available to other operations while the enqueued operation request awaits the freeing of the needed resources.

0056<figref idref="DRAWINGS">FIG. 4</figref> shows operations of the operation dispatcher when a resource such as resource B becomes available. In one aspect, wait queues are “walked” to identify waiting operations enqueued on the wait queues and to dispatch those enqueued waiting operation requests as resource availability permits. In the illustrated embodiment, the process of <figref idref="DRAWINGS">FIG. 4</figref> is initiated (block <b>300</b>) when resource B becomes freed, typically by the completion of a prior operation which was utilizing resource B. Thus, for example, if an operation is utilizing 100 megabytes per second of a total port bandwidth of 1000 megabytes per second, a bandwidth portion of 100 megabytes per second may be released for use by other operations upon completion of that operation.

0057Upon a release (block <b>300</b>) of resource B, one of the waiting operation requests enqueued on one of the wait queues of resource B wait queue set <b>202</b> is selected (block <b>302</b>), taking into account the different priority levels of the waiting operations in the wait queue set <b>202</b>. Various techniques may be used to select a waiting operation having a particular priority level. In general, a high priority operation may be selected more frequently than a medium priority operation which may be selected more frequently than a low priority operation. However, in order to prevent higher priority operations from blocking the selection of all lower priority operations, the number of operations of each priority type being dispatched may be counted and higher priority operations may be deferred as needed to permit a minimum number or proportion of lower priority operations to proceed. Another technique may be used in which the time a lower priority operation is caused to wait may be tracked and the priority of the operation may increase as time passes until the operation has sufficient priority to proceed. Other techniques may be used to select the priority level of the next operation to be selected. In the illustrated embodiment, once a priority level has been selected for the next operation to be selected, in general, the operation at the front of the wait queue for that priority level is selected since the wait queues of the illustrated embodiment are FIFO queues. However, other queues types and other techniques for selecting operations of different priority levels may be used as well.

0058Once one of the waiting operations enqueued on one of the wait queues of resource B wait queue set <b>202</b> is selected (block <b>302</b>), taking into account the different priority levels of the waiting operations in the wait queue set <b>202</b>, the selected operation is marked (block <b>304</b>) for potential dispatching. In addition, an attempt (block <b>306</b>) is made to acquire, in order, first resource A for the selected operation and then resource B for the selected operation. A determination (block <b>308</b>) is made to determine if the attempt was successful.

0059If the attempt was unsuccessful, that is, resource A could not be acquired or resource A could be acquired but resource B could not be acquired for the selected operation, resource A is released (block <b>310</b>) (if acquired), the selected operation is marked (block <b>312</b>) as waiting, and the process of <figref idref="DRAWINGS">FIG. 4</figref> is exited (block <b>314</b>). In one aspect, the selected operation remains on the wait queue of the associated priority level of the wait queue set <b>202</b> for resource B and, in one embodiment, need not return to a wait queue of the wait queue set <b>200</b> to compete for resource A again.

0060If the attempt (block <b>306</b>) to acquire both resource A and resource B, in order, is determined (block <b>308</b>) to have been successful, the selected operation is removed (block <b>316</b>) from the wait queue for dispatching. A determination (block <b>318</b>) is made as to whether the wait queues of the resource B wait queue set <b>202</b> are empty. If not, that is, additional operations are enqueued on one or more of the wait queues of the resource B wait queue set <b>202</b>, another one of the waiting operations enqueued on one of the wait queues of resource B wait queue set <b>202</b> is selected (block <b>302</b>), taking into account the different priority levels of the waiting operations in the wait queue set <b>202</b>. The process of steps <b>304</b>-<b>316</b> is repeated, acquiring (block <b>306</b>) both resource A and resource B in order for the selected operation, removing (block <b>316</b>) the selected operation from the wait queue for dispatching and selecting (block <b>302</b>) another waiting operation on one of the wait queues of the resource B wait queue set <b>202</b> until either the operation dispatcher is unable (block <b>308</b>) to acquire both resource A and resource B for the selected operation, or the wait queues of the resource B wait queue set <b>202</b> are empty (block <b>318</b>). Once the wait queues of the resource B wait queue set <b>202</b> are empty (block <b>318</b>), the process of <figref idref="DRAWINGS">FIG. 4</figref> may be exited (block <b>314</b>).

0061In one aspect, it is noted that once an operation request acquires resource A but fails to acquire resource B, the operation may be enqueued on the wait queue of the associated priority level of the wait queue set <b>202</b> for resource B and, in one embodiment, need not return to a wait queue of the wait queue set <b>200</b> to compete for resource A again.

0062<figref idref="DRAWINGS">FIGS. 5A and 5B</figref> show operations of the operation dispatcher when a resource such as resource A becomes available. Here, the wait queues of both wait queue sets <b>200</b> and <b>202</b> are “walked” to identify waiting operations enqueued on the wait queues of sets <b>200</b> and <b>202</b> and to dispatch those enqueued waiting operations as resource availability permits. In the illustrated embodiment, in a manner similar to the process of <figref idref="DRAWINGS">FIG. 4</figref>, the process of <figref idref="DRAWINGS">FIG. 5A</figref> is initiated (block <b>400</b>) when resource A becomes freed, typically by the completion of a prior operation which was utilizing resource A. As previously mentioned, in this example, resource A is a count of the number of permissible ongoing low and medium priority operations. Thus, resource A could be freed when a low or medium priority operation is completed.

0063Upon a release (block <b>400</b>) of resource A, one of the waiting operations enqueued on one of the wait queues of resource B wait queue set <b>202</b> is selected (block <b>402</b>), taking into account the different priority levels of the waiting operations in the wait queue set <b>202</b>. Once one of the waiting operations enqueued on one of the wait queues of resource B wait queue set <b>202</b> is selected (block <b>402</b>), the selected operation is marked (block <b>404</b>) for potential dispatching. In addition, an attempt (block <b>406</b>) is made to acquire, in order, first resource A for the selected operation and then resource B for the selected operation. A determination (block <b>408</b>) is made to determine if the attempt was successful.

0064If the attempt was unsuccessful, that is, resource A could not be acquired or resource A could be acquired but resource B could not be acquired for the selected operation, resource A is released (block <b>410</b>) (if acquired), the selected operation is marked (block <b>412</b>) as waiting, and the process of <figref idref="DRAWINGS">FIG. 5A</figref> is exited (block <b>414</b>). In one aspect, the selected operation remains on the wait queue of the associated priority level of the wait queue set <b>202</b> for resource B and, in one embodiment, need not return to a wait queue of the wait queue set <b>200</b> to compete for resource A again.

0065If the attempt (block <b>406</b>) to acquire both resource A and resource B, in order, is determined (block <b>408</b>) to have been successful, the selected operation is removed (block <b>416</b>) from the wait queue for dispatching. A determination (block <b>418</b>) is made as to whether the wait queues of the resource B wait queue set <b>202</b> are empty. If not, that is, additional operations are enqueued on one or more of the wait queues of the resource B wait queue set <b>202</b>, another one of the waiting operations enqueued on one of the wait queues of resource B wait queue set <b>202</b> is selected (block <b>402</b>), taking into account the different priority levels of the waiting operations in the wait queue set <b>202</b>. The process of steps <b>404</b>-<b>416</b> is repeated, acquiring (block <b>406</b>) both resource A and resource B in order for the selected operation, removing (block <b>416</b>) the selected operation from the wait queue for dispatching and selecting (block <b>402</b>) another waiting operation on one of the wait queues of the resource B wait queue set <b>202</b> until either the operation dispatcher is unable (block <b>408</b>) to acquire both resource A and resource B for the selected operation, or the wait queues of the resource B wait queue set <b>202</b> are empty (block <b>418</b>). Once the wait queues of the resource B wait queue set <b>202</b> are empty (block <b>418</b>), the process of <figref idref="DRAWINGS">FIG. 5A</figref> continues (block <b>420</b>, <figref idref="DRAWINGS">FIGS. 5A</figref>, <b>5</b>B) with the process of <figref idref="DRAWINGS">FIG. 5B</figref> in which the wait queues of wait queue set <b>200</b> for resource A are walked.

0066In a manner similar to that described above in connection with <figref idref="DRAWINGS">FIG. 5A</figref>, one of the waiting operations enqueued on one of the wait queues of resource A wait queue set <b>200</b> is selected (block <b>502</b>), taking into account the different priority levels of the waiting operations in the wait queue set <b>200</b>. Once one of the waiting operations enqueued on one of the wait queues of resource A wait queue set <b>200</b> is selected (block <b>502</b>), the selected operation is marked (block <b>504</b>) for potential dispatching. In addition, an attempt (block <b>506</b>) is made to acquire, in order, first resource A for the selected operation and then resource B for the selected operation. A determination (block <b>508</b>) is made to determine if the attempt was successful.

0067If the attempt was unsuccessful, that is, resource A could not be acquired or resource A could be acquired but resource B could not be acquired for the selected operation, resource A is released (block <b>510</b>) (if acquired), and the selected operation is marked (block <b>512</b>) as waiting. If resource A could be acquired but resource B could not be acquired for the selected operation, the operation is moved (block <b>513</b>) to the wait queue of the appropriate priority level of wait queue set <b>202</b> for resource B. The process of <figref idref="DRAWINGS">FIG. 5B</figref> is exited (block <b>514</b>).

0068If the attempt (block <b>506</b>) to acquire both resource A and resource B, in order, is determined (block <b>508</b>) to have been successful, the selected operation is removed (block <b>516</b>) from the wait queue for dispatching. A determination (block <b>518</b>) is made as to whether the wait queues of the resource A wait queue set <b>200</b> are empty. If not, that is, additional operations are enqueued on one or more of the wait queues of the resource A wait queue set <b>200</b>, another one of the waiting operations enqueued on one of the wait queues of resource A wait queue set <b>200</b> is selected (block <b>502</b>), taking into account the different priority levels of the waiting operations in the wait queue set <b>200</b>. The process of steps <b>504</b>-<b>516</b> is repeated, acquiring (block <b>506</b>) both resource A and resource B in order for the selected operation, removing (block <b>516</b>) the selected operation from the wait queue for dispatching and selecting (block <b>502</b>) another waiting operation on one of the wait queues of the resource A wait queue set <b>200</b> until either the operation dispatcher is unable (block <b>508</b>) to acquire both resource A and resource B for the selected operation, or the wait queues of the resource A wait queue set <b>200</b> are empty (block <b>518</b>). Once the wait queues of the resource A wait queue set <b>200</b> are empty (block <b>518</b>), the process of <figref idref="DRAWINGS">FIG. 5B</figref> may be exited (block <b>514</b>).

0069Additional Implementation Details

0070The described techniques for managing resources may be implemented as a method, apparatus or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The term “article of manufacture” as used herein refers to code or logic implemented in hardware logic (e.g., an integrated circuit chip, Programmable Gate Array (PGA), Application Specific Integrated Circuit (ASIC), etc.) or a computer readable medium, such as magnetic storage medium (e.g., hard disk drives, floppy disks, tape, etc.), optical storage (CD-ROMs, optical disks, etc.), volatile and non-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs, SRAMs, firmware, programmable logic, etc.). Code in the computer readable medium is accessed and executed by a processor complex. The code in which preferred embodiments are implemented may further be accessible through a transmission media or from a file server over a network. In such cases, the article of manufacture in which the code is implemented may comprise a transmission media, such as a network transmission line, wireless transmission media, signals propagating through space, radio waves, infrared signals, etc. Thus, the “article of manufacture” may comprise the medium in which the code is embodied. Additionally, the “article of manufacture” may comprise a combination of hardware and software components in which the code is embodied, processed, and executed. Of course, those skilled in the art will recognize that many modifications may be made to this configuration without departing from the scope of the present invention, and that the article of manufacture may comprise any information bearing medium known in the art.

0071In the described implementations, certain operations were described as performed by the processors of the processor complexes <b>122</b><i>a</i>, and <b>122</b><i>b </i>or the adapter <b>130</b>. In alternative implementations, certain operations described as performed by the processors may be performed by a variety of computers, processors, and controllers.

0072The described implementations for resource management were described for use with systems deployed in a data environment where high availability is of value. However, those skilled in the art will appreciate that the resource management operations described herein may apply to computational and storage systems used for non-critical data.

0073In the described implementations, dual clusters were provided. In alternative implementations, there may be one or more than two clusters and resource management may be practiced in any one cluster.

0074The illustrated logic of <figref idref="DRAWINGS">FIGS. 2-5B</figref> show certain events occurring in a certain order. In alternative implementations, certain operations may be performed in a different order, modified or removed. Moreover, steps may be added to the above described logic and still conform to the described implementations. Further, operations described herein may occur sequentially or certain operations may be processed in parallel. Yet further, operations may be performed by a single processing unit or by distributed processing units.

0075The variable N is used to denote any integer variable for certain of the described elements and may indicate a same or different integer value when used in different instances.

0076<figref idref="DRAWINGS">FIG. 7</figref> illustrates one implementation of a computer architecture <b>800</b> of the network components, such as the hosts and storage controller clusters shown in <figref idref="DRAWINGS">FIG. 1</figref>. The architecture <b>800</b> may include a processor <b>802</b> (e.g., a microprocessor), a memory <b>804</b> (e.g., a volatile memory device), and storage <b>806</b> (e.g., a non-volatile storage, such as magnetic disk drives, optical disk drives, a tape drive, etc.). The storage <b>806</b> may comprise an internal storage device or an attached or network accessible storage. Programs in the storage <b>806</b> are loaded into the memory <b>804</b> and executed by the processor <b>802</b> in a manner known in the art. The architecture further includes a network card <b>808</b> to enable communication with a network. An input device <b>810</b> is used to provide user input to the processor <b>802</b>, and may include a keyboard, mouse, pen-stylus, microphone, touch sensitive display screen, or any other activation or input mechanism known in the art. An output device <b>812</b> is capable of rendering information transmitted from the processor <b>802</b>, or other component, such as a display monitor, printer, storage, etc.

0077The foregoing description of various implementations of the invention has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the invention be limited not by this detailed description, but rather by the claims appended hereto. The above specification, examples and data provide a complete description of the manufacture and use of the composition of the invention. Since many embodiments of the invention can be made without departing from the spirit and scope of the invention, the invention resides in the claims hereinafter appended.