Apparatus and method of cascading backup logical volume mirrors

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is related to co-pending U.S. patent application Ser. No. ______ (IBM Docket No. AUS920010936), entitled APPARATUS AND METHOD OF MAINTAINING RELIABLE OFFLINE MIRROR COPIES IN VIRTUAL VOLUME GROUPS, filed on even date herewith and assigned to the common assignee of this application.

BACKGROUND OF THE INVENTION

[0002] 1. Technical Field

[0003] The present invention is directed to a method and apparatus for managing data storage systems. More specifically, the present invention is directed to a method and apparatus for cascading logical volume mirrors.

[0004] 2. Description of Related Art

[0005] Most computer systems are made up of at least one processor and one physical storage system. The processor processes, stores and retrieves data from the physical storage system under the guidance of an application program.

[0006] Application programs generally run atop an operating system. Among the many tasks of an operating system is that of allowing an application program to have a rather simplistic view of how data (i.e., data files) are stored within a physical storage system. Typically, an application program views the physical storage system as containing a number of hierarchical partitions (i.e., directories) within which entire data files are stored. This simplistic view is often referred to as a logical view since most files are not really stored as unit bodies into directories but rather are broken up into data blocks that may be strewn across the entire physical storage system.

[0007] The operating system is able to allow an application program to have this simplistic logical view with the help of a file management system. The file management system stores directory structures, breaks up data files into their constituent data blocks, stores the data blocks throughout a physical storage system and maintains data logs of where every piece of data is stored. Thus, the file management system is consulted whenever data files are being stored or retrieved from storage.

[0008] Computer systems that have a plurality of physical storage systems (e.g., servers) use an added layer of abstraction when storing and retrieving data. The added layer of abstraction is a logical volume manager (LVM). Volume, in this case, is the storage capacity of a physical storage system. Thus, volume and physical storage system will henceforth be used interchangeably.

[0009] The LVM arranges the physical storage systems into volume groups in order to give the impression that storage systems having each a much more voluminous storage capacity are being used. Within each volume group, one or more logical volumes may be defined. Data stored in a logical volume appears to be stored contiguously. However in actuality, the data may be interspersed into many different locations across all the physical storage systems that make up the volume group.

[0010] Stated differently, each logical volume in a logical volume group is divided into logical partitions. Likewise, each physical volume in a physical volume group is divided into physical partitions. Each logical partition corresponds to at least one physical partition. But, although the logical partitions in a logical volume are numbered consecutively or appear to be contiguous to each other, the physical partitions to which they each correspond, need not be contiguous to each other. And indeed, most often, the physical partitions are not contiguous to each other. Thus, one of the many tasks of the LVM is to keep tab on the location of each physical partition that corresponds to a logical partition.

[0011] For fault tolerance and performance, some servers store at least one extra copy of each piece of data onto the physical storage systems they use. Storing more than one copy of a piece of data is called mirroring the data. In order to store mirrored data, each logical partition used must correspond to as many physical partitions as there are mirrors (or copies) of the data. In other words, if the data is mirrored three times, for example, each logical partition will correspond to three physical partitions.

[0012] Writing data in mirrors is quite a time-consuming and CPU-intensive endeavor. Thus when there is more than two mirrors, some system administrators sometimes designate one of the mirrors as a backup mirror and the others as working mirrors. As alluded to above, data is usually written concurrently into all the working mirrors. However, updates are made to the backup mirror periodically (e.g., once a day). One mirror is usually designated as the mirror that will provide the updates. Using data from a working mirror to update a backup mirror is referred to as synchronizing the backup mirror to the designated working mirror.

[0013] For disaster recovery, some computer systems may have another mirror located at a remote location. This mirror may also be designated as another backup mirror. It should be noted however, that during the time a backup mirror is being synchronized with a working mirror, no application programs may have access to any of the working mirrors. Therefore, it may not be practical to synchronize a remote backup mirror with a working mirror.

[0014] Thus, what is needed is an apparatus and method of synchronizing one backup mirror to another backup mirror.

SUMMARY OF THE INVENTION

[0015] The present invention provides a method, system and apparatus for cascading backup mirrors. A mirroring map is created. The mirroring map includes at least three mirrors. A first mirror of the three mirrors is set to synchronize to a second mirror and a third mirror is set to synchronize to the first mirror. The first and the third mirror are backup mirrors and the second mirror is a working mirror. One of the backup mirrors is located remotely and the other locally.

BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:

[0017]FIG. 1 is an exemplary block diagram illustrating a distributed data processing system according to the present invention.

[0018]FIG. 2 is an exemplary block diagram of a server apparatus according to the present invention.

[0019]FIG. 3 is an exemplary block diagram of a client apparatus according to the present invention.

[0020]FIG. 4 is a conceptual view of an LVM.

[0021]FIG. 5 illustrates a mirroring map that may be used with the present invention.

[0022]FIG. 6 illustrates a first modification table that may be used with the present invention.

[0023]FIG. 7 illustrates a second modification table that may be used with the invention.

[0024]FIG. 8 illustrates a third modification table that may be used with the invention.

[0025]FIG. 9 is a flow chart of a process that may be used by the present invention to write data into an active mirror.

[0026]FIG. 10 is a flow chart of a process that may be used by the present invention to write data into a backup mirror.

[0027]FIG. 11 is a flow chart of a process that may be used by the present invention to synchronize one mirror to another mirror.

[0028]FIG. 12 is a flow chart of a process that may be used by the present invention to disassociate one mirror from another.

[0029]FIG. 13 is a flow chart of a process that may be used by the present invention to read data from a primary volume group when a mirror has been taken offline.

[0030]FIG. 14 is a flow chart of a process that may be used by the present invention to write data from a primary volume group when a mirror has been taken offline.

[0031]FIG. 15 is a flow chart of a process that may be used by the present invention to read data from a secondary volume group when a mirror has been taken offline.

[0032]FIG. 16 is a flow chart of a process that may be used by the present invention to write data to a secondary volume group when a mirror has been taken offline.

[0033]FIG. 17 is a flow chart of a process that may be used by the present invention to rejoin an offline mirror to an active mirror.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0034] With reference now to the figures, FIG. 1 depicts a pictorial representation of a network of data processing systems in which the present invention may be implemented. Network data processing system <b>100</b> is a network of computers in which the present invention may be implemented. Network data processing system <b>100</b> contains a network <b>102</b>, which is the medium used to provide communications links between various devices and computers connected together within network data processing system <b>100</b>. Network <b>102</b> may include connections, such as wire, wireless communication links, or fiber optic cables.

[0035] In the depicted example, server <b>104</b> is connected to network <b>102</b> along with storage unit <b>106</b>. In addition, clients <b>108</b>, <b>110</b>, and <b>112</b> are connected to network <b>102</b>. These clients <b>108</b>, <b>110</b>, and <b>112</b> may be, for example, personal computers or network computers. In the depicted example, server <b>104</b> provides data, such as boot files, operating system images, and applications to clients <b>108</b>, <b>110</b> and <b>112</b>. Clients <b>108</b>, <b>110</b> and <b>112</b> are clients to server <b>104</b>. Network data processing system <b>100</b> may include additional servers, clients, and other devices not shown. In the depicted example, network data processing system <b>100</b> is the Internet with network <b>102</b> representing a worldwide collection of networks and gateways that use the TCP/IP suite of protocols to communicate with one another. At the heart of the Internet is a backbone of high-speed data communication lines between major nodes or host computers, consisting of thousands of commercial, government, educational and other computer systems that route data and messages. Of course, network data processing system <b>100</b> also may be implemented as a number of different types of networks, such as for example, an intranet, a local area network (LAN), or a wide area network (WAN). FIG. 1 is intended as an example, and not as an architectural limitation for the present invention.

[0036] Referring to FIG. 2, a block diagram of a data processing system that may be implemented as a server, such as server <b>104</b> in FIG. 1, is depicted in accordance with a preferred embodiment of the present invention. Data processing system <b>200</b> may be a symmetric multiprocessor (SMP) system including a plurality of processors <b>202</b> and <b>204</b> connected to system bus <b>206</b>. Alternatively, a single processor system may be employed. Also connected to system bus <b>206</b> is memory controller/cache <b>208</b>, which provides an interface to local memory <b>209</b>. I/O bus bridge <b>210</b> is connected to system bus <b>206</b> and provides an interface to I/O bus <b>212</b>. Memory controller/cache <b>208</b> and I/O bus bridge <b>210</b> may be integrated as depicted.

[0037] Peripheral component interconnect (PCI) bus bridge <b>214</b> connected to I/O bus <b>212</b> provides an interface to PCI local bus <b>216</b>. A number of modems may be connected to PCI local bus <b>216</b>. Typical PCI bus implementations will support four PCI expansion slots or add-in connectors. Communications links to network computers <b>108</b>, <b>110</b> and <b>112</b> in FIG. 1 may be provided through modem <b>218</b> and network adapter <b>220</b> connected to PCI local bus <b>216</b> through add-in boards. Additional PCI bus bridges <b>222</b> and <b>224</b> provide interfaces for additional PCI local buses <b>226</b> and <b>228</b>, from which additional modems or network adapters may be supported. In this manner, data processing system <b>200</b> allows connections to multiple network computers. A memory-mapped graphics adapter <b>230</b> and hard disk <b>232</b> may also be connected to I/O bus <b>212</b> as depicted, either directly or indirectly.

[0038] Those of ordinary skill in the art will appreciate that the hardware depicted in FIG. 2 may vary. For example, other peripheral devices, such as optical disk drives and the like, also may be used in addition to or in place of the hardware depicted. The depicted example is not meant to imply architectural limitations with respect to the present invention.

[0039] The data processing system depicted in FIG. 2 may be, for example, an IBM e-Server pSeries system, a product of International Business Machines Corporation in Armonk, N.Y., running the Advanced Interactive Executive (AIX) operating system or LINUX operating system.

[0040] With reference now to FIG. 3, a block diagram illustrating a data processing system is depicted in which the present invention may be implemented. Data processing system <b>300</b> is an example of a client computer. Data processing system <b>300</b> employs a peripheral component interconnect (PCI) local bus architecture. Although the depicted example employs a PCI bus, other bus architectures such as Accelerated Graphics Port (AGP) and Industry Standard Architecture (ISA) may be used. Processor <b>302</b> and main memory <b>304</b> are connected to PCI local bus <b>306</b> through PCI bridge <b>308</b>. PCI bridge <b>308</b> also may include an integrated memory controller and cache memory for processor <b>302</b>. Additional connections to PCI local bus <b>306</b> may be made through direct component interconnection or through add-in boards. In the depicted example, local area network (LAN) adapter <b>310</b>, SCSI host bus adapter <b>312</b>, and expansion bus interface <b>314</b> are connected to PCI local bus <b>306</b> by direct component connection. In contrast, audio adapter <b>316</b>, graphics adapter <b>318</b>, and audio/video adapter <b>319</b> are connected to PCI local bus <b>306</b> by add-in boards inserted into expansion slots. Expansion bus interface <b>314</b> provides a connection for a keyboard and mouse adapter <b>320</b>, modem <b>322</b>, and additional memory <b>324</b>. Small computer system interface (SCSI) host bus adapter <b>312</b> provides a connection for hard disk drive <b>326</b>, tape drive <b>328</b>, and CD-ROM drive <b>330</b>. Typical PCI local bus implementations will support three or four PCI expansion slots or add-in connectors.

[0041] An operating system runs on processor <b>302</b> and is used to coordinate and provide control of various components within data processing system <b>300</b> in FIG. 3. The operating system may be a commercially available operating system, such as Windows 2000, which is available from Microsoft Corporation. An object oriented programming system such as Java may run in conjunction with the operating system and provide calls to the operating system from Java programs or applications executing on data processing system <b>300</b>. “Java” is a trademark of Sun Microsystems, Inc. Instructions for the operating system, the object-oriented operating system, and applications or programs are located on storage devices, such as hard disk drive <b>326</b>, and may be loaded into main memory <b>304</b> for execution by processor <b>302</b>.

[0042] Those of ordinary skill in the art will appreciate that the hardware in FIG. 3 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash ROM (or equivalent nonvolatile memory) or optical disk drives and the like, may be used in addition to or in place of the hardware depicted in FIG. 3. Also, the processes of the present invention may be applied to a multiprocessor data processing system.

[0043] As another example, data processing system <b>300</b> may be a stand-alone system configured to be bootable without relying on some type of network communication interface, whether or not data processing system <b>300</b> comprises some type of network communication interface. As a further example, data processing system <b>300</b> may be a Personal Digital Assistant (PDA) device, which is configured with ROM and/or flash ROM in order to provide non-volatile memory for storing operating system files and/or user-generated data.

[0044] The depicted example in FIG. 3 and above-described examples are not meant to imply architectural limitations. For example, data processing system <b>300</b> may also be a notebook computer or hand held computer in addition to taking the form of a PDA. Data processing system <b>300</b> also may be a kiosk or a Web appliance.

[0045] The present invention provides an apparatus and method of synchronizing one backup mirror to another backup mirror. Although the invention may preferably be local to server <b>104</b>, it may nonetheless, be local to client systems <b>108</b>, <b>110</b> and <b>112</b> of FIG. 1 or to both the server <b>104</b> and clients <b>108</b>, <b>110</b> and <b>112</b>. Consequently, the present invention may reside on any data storage medium (i.e., floppy disk, compact disk, hard disk, ROM, RAM, etc.) used by a computer system.

[0046] To better understand the invention, a more detailed explanation of the LVM is needed. The LVM interacts with application programs and the physical storage devices as shown in FIG. 4. In FIG. 4 three layers are depicted, an application layer <b>400</b>, a logical layer <b>410</b> and a physical layer <b>420</b> each having one or more devices. It should be noted that the devices shown in the three layers are not all-inclusive. There may be more devices in use in each of the application layer <b>412</b>, the logical layer <b>410</b> and the physical layer <b>430</b>. Thus, the devices in FIG. 4 should be taken only as an example of devices that may be used.

[0047] The logical layer <b>410</b>, for all intent and purpose, is the LVM. The LVM may be regarded as being made up of a set of operating system commands, library subroutines or other tools that allow a user to establish and control logical volume storage. The LVM controls physical storage system resources by mapping data between a simple and flexible logical view of storage space and the actual physical storage system. The LVM does this by using a layer of device driver code that runs above traditional device drivers. This logical view of the disk storage is provided to application programs and is independent of the underlying physical disk structure.

[0048] The logical layer <b>410</b> contains a logical volume <b>412</b> that interacts with logical volume device driver <b>414</b>. A device driver, as is well known in the art, acts as a translator between a device and programs that use the device. That is, the device driver accepts generic commands from programs and translates them into specialized commands for the device. In this case, the logical volume device driver <b>414</b> translates commands from an application program that may be executing on the computer system for device driver <b>430</b>. Thus, when an application program sends commands to file system manager <b>402</b> to store or retrieve data from logical volume <b>412</b>, the file system manager <b>402</b> informs the logical volume manager <b>412</b> of the application program's wish. The logical volume manager <b>412</b> then conveys the wish to the logical volume device driver <b>414</b>. The logical volume device driver <b>414</b> then consults the appropriate map and instructs the device driver <b>430</b> which ones of physical storage systems <b>422</b>, <b>424</b>, <b>426</b> and <b>428</b> to use for the data.

[0049] When a system administrator wants to mirror a piece of data, the administrator has to devise a map (or mirroring scheme) to correlate the logical volume being used to the actual physical storage systems in which the data is to be stored. Generally, this map correlates the logical partitions to the physical partitions of the physical storage systems that are to be used. This map is stored in the LVM.

[0050]FIG. 5 illustrates an example of a mirroring map that may be stored in the LVM. In this example, four physical storage systems (PSS) are used. The physical storage systems are PSS-1 <b>500</b>, PSS-2 <b>510</b>, PSS-3 <b>520</b> and PSS-4 <b>530</b>. The four physical storage systems are divided into partitions (see partitions <b>502</b>, <b>504</b> and <b>506</b> of PSS-1 <b>500</b>, partitions <b>512</b>, <b>514</b> and <b>516</b> of PSS-2 <b>510</b>, partitions <b>522</b>, <b>524</b> and <b>526</b> of PSS-3 <b>520</b> and partitions <b>532</b> and <b>534</b> of PSS-4 <b>530</b>). Mirrored data is stored in the three physical storage systems as illustrated by data A in partitions <b>502</b> and <b>512</b> of PSS-1 <b>500</b> and PSS-2 <b>510</b>. PSS-1 <b>500</b> and PSS-2 <b>510</b> are local working mirrors. PSS-3 <b>520</b> is a local backup mirror and PSS-4 <b>530</b> is a remote backup mirror.

[0051] When the computer system effectuates a write operation into PSS-1 <b>500</b>, a write operation is also effectuated into PSS-2 <b>510</b> and vice versa and the data is written into appropriate locations as per the mirroring map. However, a write operation is not performed into PSS-3 <b>520</b> and PSS-4 <b>530</b>. Instead, a table of modifications is updated in the LVM. This table is consulted periodically. Specifically, the table is consulted before PSS-<b>520</b> is to be synchronized to a designated local working mirror (i.e., either PSS-1 <b>500</b> or PSS-2 <b>510</b>) to determine which pieces of data are to be written into PSS-3 <b>520</b> to perform the synchronization.

[0052]FIG. 6 illustrates a modification table that may be used by the present invention. Based on the table, PSS-3 <b>520</b> (see FIG. 5) is to be synchronized to PSS-1 <b>500</b> at every hour on the hour. The last time PSS-3 <b>520</b> was synchronized to PSS-1 <b>500</b> was at 5:00 AM. Since the synchronization, data A <b>502</b> was modified at 5:15 AM and new data B <b>504</b> was written into PSS-1 <b>500</b> at 5:45 AM, at 6:00 AM, modified data A <b>502</b> as well as new data B <b>504</b> will be written into PSS-3 <b>520</b> as data A <b>522</b> and new data B <b>524</b>. Note that if since the last synchronization no modified data or new data was written in PSS-1 <b>500</b> (i.e., no new entries in the modification table), then no data will be sent to PSS-3 <b>520</b> at the scheduled time of synchronization, since the PSS-1 <b>500</b> and PSS-3 <b>520</b> are already synchronized. Note also that PSS-1 <b>500</b> is used as the physical volume to which PSS-3 <b>520</b> is synchronized; however, PSS-3 <b>520</b> may instead be synchronized to PSS-2 <b>510</b> since data written in PSS-1 <b>500</b> is concurrently modified in PSS-2 <b>510</b> as per the mirroring map. Note further that the synchronizations may occur less often or more often depending on the synchronization time schedule enters in the table. Indeed, most backup mirrors are updated once a day, usually sometime during the night. Thus, the hourly backups are used just for illustration purposes.

[0053] As stated in the Background of the Invention, when the working mirrors are being backed up by the local backup mirror (i.e., when PSS-3 <b>520</b> is being synchronized to PSS-1 <b>500</b>) application programs that are running on the computer system do not have access to the working mirrors (e.g., cannot read or write into either PSS-1 <b>500</b> or PSS-2 <b>510</b>). Some application programs may have time-sensitive information that may need to be stored or read from the working mirrors. Thus, a method that allows the application programs to have constant access to the data as well as to modify and write new data into the physical systems must be devised.

[0054] One way to allow the application programs to continue reading and writing data is to split off the working mirrors. So, just before the time that PSS-3 <b>520</b> is to be synchronized to PSS-1 <b>500</b>, PSS-1 may be disassociated with PSS-2 <b>510</b>. The application programs will continue to read or write from PSS-2 <b>510</b> but not from PSS-1 <b>500</b>. After PSS-3 <b>520</b> is synchronized to PSS-1 <b>500</b>, PSS-1 <b>500</b> may be re-associated with PSS-2 <b>510</b>.

[0055] Ordinarily, when the two working mirrors are disassociated, the working mirror to which the backup mirror is to be synchronized will be ported to another computer system. There, the file systems on the mirror will be mounted. To mount a file system is to make the file systems available for use. Once mounted, the synchronization procedure may be initiated. When the file systems are mounted, some data may be written into the mirrors. The written data may be metadata such as date and time the file systems were mounted etc. Any data written into the mirror should be marked stale before re-associating the mirrors together. Upon re-association, all data marked stale will be discarded.

[0056] As mentioned before, during disassociation new data may have been written into PSS-2 <b>510</b> or existing data in PSS-2 <b>510</b> may have been modified. Thus upon re-association, PSS-1 <b>500</b> will not be a true mirror of PSS-2 <b>510</b>. To ascertain that PSS-1 <b>500</b> remains a true mirror of PSS-2 <b>510</b> after re-association, new data and modified data written into PSS-2 <b>510</b> when the mirrors were disassociated are entered into a modification table. After re-association, the new data and the modified data are copied from PSS-2 <b>510</b> into PSS-1 <b>500</b>. When this occurs, PSS-1 <b>500</b> and PSS-2 <b>510</b> become true mirrors of each other again.

[0057] To disassociate PSS-1 <b>500</b> from PSS-2 <b>510</b> the mapping mirror must be modified. That is, partitions from the logical volume that originally correspond to physical partitions in both PSS-1 <b>500</b> and PSS-2 <b>510</b> will only correspond to only partitions in PSS-2 <b>510</b>. After re-association, the mapping mirror may be modified once more to correlate each partition in the logical volume to the physical partitions in both PSS-1 <b>500</b> and PSS-2 <b>510</b>.

[0058]FIG. 7 illustrates a modification table that may be used to record changes in PSS-2 <b>510</b> while PSS-1 <b>500</b> and PSS-2 <b>510</b> are no longer mirrors of each other. It is assumed that at 11:00 PM, PSS-1 <b>500</b> is disassociated from PSS-2 <b>510</b> for backup purposes and at 12:00 AM it is re-associated with PSS-2 <b>510</b>. Between 11:00 PM and 12:00 AM data B in PSS-2 <b>510</b> is modified and new data C is written into PSS-2 <b>510</b>. When PSS-1 <b>500</b> is re-associated with PSS-2 <b>510</b>, modified data B and new data C will be copied from PSS-2 <b>510</b> and written into PSS-1 <b>500</b>. At that point, PSS-1 <b>500</b> and PSS-2 <b>510</b> are again mirrors of each other. From that point forward data written in PSS-1 <b>500</b> will concurrently be written into PSS-2 <b>510</b> and vice versa until they are disassociated from each other again for backup purposes.

[0059]FIG. 8 is another modification table that may be used with the invention. In this modification table, the time that a piece of data was modified as well as the time that a new piece of data was written into the local backup mirror (i.e., PSS-3 <b>520</b>) including the data itself are noted. To better understand the invention, FIG. 8 should be used in conjunction with FIG. 7. In FIG. 7, let us suppose that PSS-3 <b>520</b> was synchronized with PSS-1 <b>500</b> at 11:00 PM. Thus, the modification time entered for both modified data A and new data B is 11:00 PM. At 1:00 AM, remote backup PSS-4 <b>530</b> is to be synchronized with local backup PSS-3 <b>520</b>. Thus, modified data A and new data B will be copied into remote backup PSS-4 <b>530</b>

[0060] As described, the remote backup mirror is synchronized to the local backup mirror. The invention can therefore be extended to have a second remote or local backup mirror be synchronized to the first remote backup mirror and to have a third remote backup mirror be synchronized to the second remote backup mirror and so on to obtain a cascaded backup mirrors.

[0061]FIG. 9 is a flow chart of a process that may be used by the present invention to write data into an active mirror. The process starts when a request to write data into an active mirror is received. A check is then made to determine whether there are active backup mirrors. If so the data is written into the mirror and the process ends. If there is a backup mirror being active, then all backup mirrors are marked as stale with respect to the active mirror. Then a check is made to determine whether a cascaded backup mirror is active. If not, the data is written into the active mirror and the process ends. If there is a cascaded active backup mirror, it is marked as stale with respect to the first backup mirror before writing the data into the mirror (steps <b>900</b>-<b>914</b>).

[0062]FIG. 10 is a flow chart of a process that may be used by the present invention to write data into a backup mirror. The process starts when a request to write data onto a backup mirror is received. The first backup mirror is marked as stale and a check is made to determine whether there is a cascaded backup mirror that has been marked as stale with respect to the first backup mirror. If so, the data is written in the backup mirror and the process ends. If not, the cascaded backup mirror is marked as stale with respect to the first backup mirror before writing the data onto the mirror (steps <b>1000</b>-<b>1012</b>).

[0063]FIG. 11 is a flow chart of a process that may be used by the present invention to synchronize one mirror to another mirror. The process starts when a request to synchronize one mirror to another is received. Data is read from the mirror to which the other mirror is to be synchronized and written into the mirror being synchronized. Then a check is made to determine whether the data is read from an active mirror. If so, backup mirrors, if there are any, are marked as active with respect to the active mirror and as stale with respect to cascaded backup mirrors and the process ends. If the data was read from a non-active mirror, then the mirror is a first backup mirror and cascaded backup mirrors are marked as active with respect to first backup mirror and the process ends (steps <b>1100</b>-<b>1120</b>).

[0064]FIG. 12 is a flow chart of a process that may be used by the present invention to disassociate one mirror from another. The process starts when a mirror is to be taken offline. All PSSs on primary volume group are marked as backup PSS and are not used anymore. Then, a secondary volume group is created using the backup PSSs (the ones that have just been marked as backup PSSs). Status allocations are made to keep track of regions that may have changed while the PSSs make up the secondary volume group and the process ends (steps <b>1200</b>-<b>1210</b>).

[0065]FIG. 13 is a flow chart of a process that may be used by the present invention to read data from a primary volume group when a mirror has been taken offline. The process <b>10</b> starts when data is to be read. Then a check is made to determine whether there is a backup disk. If not, any mirror can be chosen from which to read the requested data. If yes, the backup disk is removed from the list of mirrors from which the data can be read. A mirror from which the data is to be read is chosen from the remaining disk. The data is then read and the process ends (steps <b>1300</b>-<b>1316</b>).

[0066]FIG. 14 is a flow chart of a process that may be used by the present invention to write data from a primary volume group when a mirror has been taken offline. The process starts when a request to read data is received. A check is made to determine whether there is a backup mirror. If not, the data is written to all the (active) mirrors and the process ends. If there is a backup mirror, the backup mirror is removed from the list of mirrors that the data is to be written onto. The regions on the active mirrors' status area are flagged to indicate that the corresponding regions on the backup mirror are stale. The data is then written to all the remaining mirrors in the list and the process ends (steps <b>1400</b>-<b>1414</b>).

[0067]FIG. 15 is a flow chart of a process that may be used by the present invention to read data from a secondary volume group when a mirror has been taken offline. The process starts when a read request is received to read from a secondary volume group. The primary mirror is removed from the list of possible mirrors to read from. Then a mirror is selected from the remaining mirrors from the list. The read is then effectuated and the process ends (steps <b>1500</b>-<b>1508</b>).

[0068]FIG. 16 is a flow chart of a process that may be used by the present invention to write data to a secondary volume group when a mirror has been taken offline. The process starts when a request to write data to a secondary volume group is received. Flag regions on backup mirrors' status area to indicate that corresponding regions on the primary mirror are stale. The data is then written onto the backup mirror and the process ends (steps <b>1600</b>-<b>1606</b>).

[0069]FIG. 17 is a flow chart of a process that may be used by the present invention to rejoin an offline mirror to an active mirror. The process starts when an offline mirror is to be re-associated with a working mirror. The secondary volume group, which contains the offline mirror, is deactivated and de-configured. Stale regions of the previously offline mirror are read or activated. A list of the stale regions is created. All corresponding regions in the primary volume group are flagged. The backup mirror is then re-activated. Then the flagged regions are copied into the backup mirror and the process ends (steps <b>1700</b>-<b>1714</b>).

[0070] The description of the present invention has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiment was chosen and described in order to best explain the principles of the invention, the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.