Storage system and remote copy control method for storage system
6 claims: 2 independent, 4 dependent
- 1複数の記憶制御装置間でデータをコピーさせるストレージシステムであって、 複数の第1記憶制御装置と、これら各第1記憶制御装置にそれぞれ接続される複数の第2記憶制御装置とを備え、 前記各第1記憶制御装置は、 上位装置により入出力されるデータを記憶するための第1論理ボリュームと、 前記上位装置による前記第1論理ボリュームへのデータ更新を、更新データと該更新データの更新順序を指示するための更新順序情報とを対応付けて前記第2記憶制御装置に送信することにより通知する更新通知部と、をそれぞれ備えており、 前記各第2記憶制御装置は、互いに接続されており、かつ、 前記第1論理ボリュームに対応付けられ、前記第1論理ボリュームに記憶されるデータのコピーが記憶される第2論理ボリュームと、 前記更新通知部から受信した前記更新データを記憶する更新データ記憶部と、 前記更新通知部から受信した前記更新順序情報を管理する更新順序情報管理部と、 前記更新順序情報に基づいて、前記更新データ記憶部に記憶された前記更新データを前記第2論理ボリュームに順番に書き込むことにより、前記第2論理ボリュームに記憶されるデータを更新させる更新部とを それぞれ備えており、 前記各第2記憶制御装置のうちいずれか1つの第2記憶制御装置は、更新制御部をさらに備えており、 前記各第1記憶制御装置内では、1つまたは複数の前記第1論理ボリュームによって第1ボリュームグループがそれぞれ形成されており、 前記各第2記憶制御装置内では、1つまたは複数の前記第2論理ボリュームによって第2ボリュームグループがそれぞれ形成されており、 前記各第1ボリュームグループと前記各第2ボリュームグループとが対応付けられており、 前記複数の第2ボリュームグループから少なくとも1つ以上の拡張ボリュームグループが形成されており、 前記更新制御部は、前記拡張グループ毎に前記更新可能時点を決定し、この決定された更新可能時点を前記拡張グループに属する前記各第2ボリュームグループを有する前記各第2記憶制御装置にそれぞれ通知するようになっており、 前記更新制御部は、前記各第1記憶制御装置のうちいずれかの第1記憶制御装置を介してサスペンド要求を受けた場合に、前記各更新順序情報管理部からそれぞれ収集される前記各更新順序情報に基づいて前記各第2論理ボリュームを更新可能な時点を決定し、この決定された更新可能時点を前記各更新部にそれぞれ通知することにより、前記各更新部によって前記各第2論理ボリュームに記憶されるデータを前記更新可能時点までそれぞれ更新させ、 前記各第2記憶制御装置は、前記更新可能時点よりも後の前記更新データについては、前記各第2論理ボリュームに書き込まずに差分管理し、前記各第2論理ボリュームの状態をサスペンド状態に変更し、前記各第2論理ボリュームがサスペンド状態に変更された旨を示す状態変更通知を前記第1記憶制御装置に送信し、 前記第1記憶制御装置は、前記状態変更通知を受信すると、前記サスペンド要求に対応する前記第1論理ボリュームの状態をサスペンド状態に変更し、前記サスペンド要求を前記第1記憶制御装置に発行した前記上位装置にサスペンド処理が完了した旨を報告する、 ストレージシステム。
- 2前記各更新部は、前記第1論理ボリュームにおける更新順序と同一の順序で、前記通知された更新可能時点まで、前記第2論理ボリュームに記憶されるデータを更新させる請求項1に記載のストレージシステム。
- 3前記更新制御部は、前記各第2記憶制御装置に受信されて前記各更新順序情報管理部によりそれぞれ管理されている最新の前記各更新順序情報のうち、最も古い前記更新順序情報に基づいて、前記更新可能時点を決定する請求項1に記載のストレージシステム。
- 4前記更新順序情報には、前記上位装置による更新要求の発行時刻が含まれている請求項1に記載のストレージシステム。
- 5前記更新順序情報には、前記上位装置による更新要求の発行時刻及び前記更新要求の発行順序が含まれている請求項1に記載のストレージシステム。
- 6複数の第1記憶制御装置と該各第1記憶制御装置にそれぞれ接続される複数の第2記憶制御装置とを備えたストレージシステムにおいて、前記各第1記憶制御装置から前記各第2記憶制御装置へデータをそれぞれ転送してコピーさせるリモートコピー制御方法であって、 前記各第1記憶制御装置がそれぞれ備える第1論理ボリュームと前記各第2記憶制御装置がそれぞれ備える第2論理ボリュームとを対応付けることにより、リモートコピーのペアを形成 し、前記各第1記憶制御装置内では、1つまたは複数の前記第1論理ボリュームによって第1ボリュームグループをそれぞれ形成し、前記各第2記憶制御装置内では、1つまたは複数の前記第2論理ボリュームによって第2ボリュームグループをそれぞれ形成し、前記各第1ボリュームグループと前記各第2ボリュームグループとを対応付け、前記複数の第2ボリュームグループから少なくとも1つ以上の拡張ボリュームグループが形成され るステップと、 上位装置から前記第1論理ボリュームに対する更新要求を受信するステップと、 前記更新要求に係る更新データを前記第1論理ボリュームに書き込んで、前記第1論理ボリュームに記憶されるデータを更新させるステップと、 前記更新データと該更新データの更新順序を指示するための更新順序情報とを対応付けて、前記各第1記憶制御装置から前記各第2記憶制御装置に転送させるステップと、 前記更新順序情報に基づいて、前記第2論理ボリュームに前記更新データを順番に書き込み、前記第2論理ボリュームに記憶されるデータを更新させるステップと、 前記各第2記憶制御装置から前記各更新順序情報をそれぞれ収集するステップと、 前記各第1記憶制御装置のうち前記上位装置からサスペンド要求 を受けた場合に、前記収集された各更新順序情報に基づいて、 前記拡張グループ毎に 前記各第2論理ボリュームを更新可能な時点を決定するステップと、 前記決定された更新可能時点を前記各第2記憶制御装置にそれぞれ通知し、前記各第2論理ボリュームに記憶されているデータを前記更新可能時点までそれぞれ更新させる同期ステップと、 前記更新可能時点よりも後に発行された前記更新データについては、前記各第2論理ボリュームに書き込まずに更新箇所を差分管理するステップと、 前記各第2論理ボリュームの状態をサスペンド状態に変更するステップと、 前記各第2論理ボリュームがサスペンド状態に変更された旨を示す状態変更通知を前記第1記憶制御装置に送信するステップと、 前記状態変更通知を受信した前記第1記憶制御装置が、前記サスペンド要求に対応する前記第1論理ボリュームの状態をサスペンド状態に変更し、前記サスペンド要求を発行した前記上位装置にサスペンド処理が完了した旨を報告するステップと、 を実行するストレージシステムのリモートコピー制御方法。
Independent claims6
285 paragraphs, as filed
The present invention relates to a storage system and a remote copy control method for the storage system.
In recent computer systems, the amount of data used has increased, and the frequency of data updates has also increased. Therefore, how to back up this data and how quickly it can be restored to the normal operating state in the event of a failure are important issues in storage technology.
As one solution to this, for example, a plurality of memory control devices are installed at the primary site and the secondary site that are separated from each other, and each memory control device (regular memory control device) of the primary site and each memory of the secondary site are installed. A technique has also been proposed in which a control device (sub-memory control device) is connected by a transfer path to synchronize the data managed at both sites (Patent Document 1). This conventional technology uses a so-called remote copy technology in which data updated by a positive storage control device is transferred to a sub storage control device and copied without going through a host computer (hereinafter, "host"). To do.
As a technique for matching data between a normal memory control device and a sub-memory control device, a technique using write time has also been proposed (Patent Document 2). In this prior art, the positive storage controller that receives the write data from the positive host reports the completion of receipt of the write data to the positive host immediately after receiving the write data. After receiving the report from the positive storage controller, the positive host reads a copy of the write data from the positive storage controller. The time (write time) at which the write request was issued by the positive host is assigned to the read write data. The primary host transfers the read write data and the write time to the secondary host. The sub-host refers to the write time assigned to each write data and writes the write data to the sub-storage control device in the order of the write time. As a result, the consistency between the data stored in the primary storage control device and the data stored in the secondary storage control device is maintained.<patcit num="1"><text>Japanese Unexamined Patent Publication No. 2004-13367</text></patcit><patcit num="2"><text>European Patent No. 0671686</text></patcit>
<p> In the above-mentioned conventional technique, since the primary host and the secondary host constantly perform data transfer and reflection processing (processing of writing data to the volume in order of write time), the load on each host is large. Each host needs to be always running, and the software for executing the above-mentioned data transfer processing and reflection processing needs to be always running.</p><p> Further, when a remote copy route is set between a plurality of normal storage control devices and a plurality of sub-memory control devices and a plurality of remote copies are processed in parallel at the same time, each sub-memory control device receives the data at a certain point in time. The light data is different. The remote copy process is an asynchronous process performed at a timing different from the timing at which the write request is received from the positive host. Therefore, the plurality of remote copy processes are executed at different arbitrary timings. As a result, even if the write data has been received by one sub-memory control device, it may not be received by the other sub-memory control device yet.</p><p> As described above, when a plurality of remote copy processes are executed at the same time between the primary site and the secondary site, it is difficult to maintain data integrity because the progress of each remote copy process is different.</p><p> The present invention has been made in view of the above problems, and an object of the present invention is a storage system capable of maintaining data consistency when data is transferred and updated between a plurality of storage control devices. And to provide a remote copy control method for the storage system. Another object of the present invention is to suspend while maintaining the consistency of data between each storage control device when a failure is detected in the case where remote copying is performed between a plurality of storage control devices. It is an object of the present invention to provide a storage system and a remote copy control method for the storage system. Further objections of the present invention will become clear from the description of embodiments described below.</p>
<p> In order to solve the above problems, the storage system according to the present invention is a storage system for copying data between a plurality of storage control devices, and is connected to the plurality of first storage control devices and each of these first storage control devices. Each of the first storage control devices is provided with a plurality of second storage control devices to be used, and each first storage control device has a first logical volume for storing data input / output by the higher-level device and data to the first logical volume by the higher-level device. Each of the update notification units is provided with an update notification unit that notifies the update by transmitting the update data to the second storage control device in association with the update data and the update order information for instructing the update order of the update data. Each second storage control device is connected to each other, is associated with the first logical volume, and is stored in the first logical volume from the second logical volume and the update notification unit. The update data storage unit that stores the received update data, the update order information management unit that manages the update order information received from the update notification unit, and the update data stored in the update data storage unit based on the update order information. It is provided with an update unit for updating the data stored in the second logical volume by writing to the second logical volume in order. Further, when a predetermined notification is received, each update order information is collected from each update order information management unit, and each second logical volume can be updated based on each collected update order information. Is provided, and by notifying each update unit of the determined updateable time point, each update unit updates the data stored in each second logical volume until the updateable time point. Storage system.</p><p> In one aspect of the present invention, each update unit updates the data stored in the second logical volume in the same order as the update order in the first logical volume until the notified updateable time point. ..</p><p> In one aspect of the present invention, the predetermined notification is an instruction to register a volume group composed of a plurality of second logical volumes.</p><p> In one aspect of the present invention, the predetermined notification is a notification of detection of a suspend factor.</p><p> In one aspect of the present invention, the predetermined notification is a suspension factor detection notification, and the update control unit receives each update order information management unit when receiving the suspend factor detection notification. Data stored in each second logical volume by each update unit by determining the time when each second logical volume can be updated based on the information and notifying each update unit of the determined updateable time. Is updated until the time when it can be updated, and each second storage control device stores a copy of the data of the first logical volume for the second logical volume related to the detection notification of the suspend factor in each second logical volume. To stop.</p><p> In one aspect of the invention, the suspend factor is when a failure is detected.</p><p> In one aspect of the present invention, a first volume group is formed by one or more first logical volumes in each first storage control device, and one or more in each second storage control device. The second volume group is formed by the second logical volume of the above, each first volume group and each second volume group are associated with each other, and the update control unit can update each second volume group at the time when it can be updated. Is determined, and the determined updateable time is notified to each update unit related to the second volume group.</p><p> In one aspect of the present invention, at least one or more extended volume groups are formed from a plurality of second volume groups, and the update control unit determines an updateable time point for each extended group, and the determined updateable time is determined. Notify each second volume group belonging to the extended group of the time point.</p><p> In one aspect of the present invention, when the update control unit is instructed to form a volume group containing at least one second logical volume, the update control unit determines an updateable time point for the volume group, and the determined updateable time is determined. Notify the update section of the second logical volume belonging to the volume group of the time point.</p><p> In one aspect of the present invention, the update control unit is based on the oldest update order information among the latest update order information received by each second storage control device and managed by each update order information management unit. To determine when it can be updated.</p><p> In one aspect of the present invention, the update order information includes the issue time of the update request by the host device.</p><p> In one aspect of the present invention, the update order information includes the issue time of the update request by the host device and the issue order of the update request.</p><p> In one aspect of the present invention, the update control unit determines an updateable time point when a suspend factor is detected, notifies each update unit of the determined updateable time point and the request for stopping the update, and each of them. The update unit updates the data stored in each second logical volume until it can be updated, and then stops updating to each second logical volume.</p><p> In one aspect of the present invention, when the occurrence of a failure in which update data and update order information cannot be transferred from each first storage control device to each second storage control device is detected, a suspend factor is detected. Judged.</p><p> In one aspect of the present invention, the update control unit determines the updatable time only for the predetermined second logical volume related to the failure among the second logical volumes, and determines the updatable time and the update stop. Notify the request to the update section related to the specified second logical volume.</p><p> In one aspect of the present invention, the update control unit is provided in at least one of the second storage control devices.</p><p> A remote copy control method for a storage system according to another aspect of the present invention is in a storage system including a plurality of first storage control devices and a plurality of second storage control devices connected to each of the first storage control devices. , A remote copy control method in which data is transferred and copied from each first storage control device to each second storage control device, and the first logical volume and each second storage control provided by each first storage control device are provided. The step of forming a remote copy pair by associating the second logical volume provided with each device, the step of receiving an update request for the first logical volume from the higher-level device, and the first logic of updating data related to the update request. From each first storage control device, the step of writing to the volume and updating the data stored in the first logical volume is associated with the update data and the update order information for instructing the update order of the update data. A step of transferring to each second storage control device, a step of writing update data to the second logical volume in order based on the update order information, and a step of updating the data stored in the second logical volume, and each second storage. A step of collecting each update order information from the control device, a step of determining when each second logical volume can be updated based on each collected update order information when a predetermined notification is received, and a step of determining when each second logical volume can be updated. A synchronization step of notifying each second storage control device of the determined updatable time point and updating the data stored in each second logical volume up to the updatable time point is executed.</p><p> Furthermore, when a failure is detected in which update data and update order information cannot be transferred from each first storage control device to each second storage control device, the updateable time point is determined based on each update order information. , The determined updateable time point and the remote copy stop request are notified to each second storage control device, and the data stored in each second logical volume is updated to the updateable time point, and then each first. You can also perform a step to stop the remote copy process from the logical volume to each second logical volume.</p><p> A storage system according to yet another aspect of the present invention is a storage system for copying data between a plurality of storage control devices, comprising a first site and a second site, each of which has a plurality of first logics. It is equipped with a plurality of first storage control devices having volumes and a higher-level device connected to each first storage control device and inputting / outputting data to each first logical volume, and the second site has a plurality of each. It has a second logical volume and includes a plurality of second storage controls connected to each other, each first logical volume constituting at least one or more first volume groups, and each second logical volume. Consists of at least one second volume group, and each first logical volume belonging to the first volume group and the second logical volume belonging to the second volume group are associated as a remote copy pair. Each first storage control device stores the data updated to each first logical volume by the higher-level device in association with the update data and the update order information for instructing the update order of the update data. It is equipped with an update notification unit that notifies by transmitting to the control device and a suspend request notification unit that notifies each second storage control device of a suspend request for stopping the operation of remote copy. The storage control device has an update data storage unit that stores update data received from the update notification unit, an update order information management unit that manages update order information received from the update notification unit, and update data based on the update order information. Each of the update units includes an update unit for updating the data stored in the second logical volume by sequentially writing the update data stored in the storage unit to the second logical volume. Further, a predetermined second storage control device among the second storage control devices includes an update control unit for synchronizing the update processing of each second logical volume by each second storage control device, and is provided with an update control unit. When receiving a predetermined notification, collects each update order information from each update order information management unit, and each based on each of the collected update order information.</p><p> The present invention can also be expressed as, for example, a control method for a remote copy system as follows. That is, it is a control method of a remote copy system that copies data between a plurality of storage device systems. The remote copy system includes a plurality of first storage device systems in which data is input and output from a computer, and a first storage device system. Each of the first storage system has a first logical volume in which data to be input and output is stored, and each second storage system has a plurality of second storage systems connected to each of the two storage systems. , Has a second logical volume that stores a copy of the data stored in the first logical volume, at least one of the second storage system has a master processing unit, and receives a registration instruction to store the second storage. The step of instructing the device system to change the reflection process, the step of receiving the change instruction and performing the reflection process based on the reflectable time, and the arrival time are collected from each second storage device system to determine the reflectable time. The second logic is the step, the step of notifying the second storage system of the reflectable time, and the write data transferred from the first storage system by the second storage system to the notified reflectable time. How to control a remote copy system, including steps to store it in a volume.</p><p> The present invention can also be expressed as a remote copy system. That is, it is a remote copy system that copies data between a plurality of storage device systems, and is a plurality of first storage device systems in which data is input and output from a computer, and a plurality of devices connected to each of the first storage device systems. It has a second storage system, each first storage system has a first logical volume in which data to be input and output is stored, and each second storage system is stored in the first logical volume. It has a second logical volume in which a copy of the data is stored, and at least one of the second storage device systems has a master processing unit, and the master processing unit receives a registration instruction and reflects it in the second storage device system. A process change instruction is given, the second storage device system performs reflection processing based on the reflectable time in response to the change instruction, and the master processing unit collects the arrival time from each second storage device system and sets the reflectable time. Determined and notified to the second storage system, the second storage system is a remote copy that stores the write data transferred from the first storage system up to the notified reflectable time in the second logical volume. system.</p><p> At least a portion of each means, part, and step of the present invention may be feasible by a computer program. Then, this computer program can be distributed in a state of being fixed to various recording media, or can be transmitted via a communication medium.</p>
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is an explanatory diagram schematically showing the whole concept of the present invention. The storage system of the present embodiment includes a primary site and a secondary site that are separated from each other. For example, in case of a disaster, the primary site and the secondary site are located in different areas. The primary site and the secondary site are connected so that bidirectional communication is possible, for example, by a dedicated line or a communication line such as a WAN (Wide Area Netwrok).
The primary site is provided with a primary host H1 and a plurality of primary storage control devices 1, and the secondary site is provided with a secondary host H2 and a plurality of secondary storage control devices 2. Each positive storage control device 1 includes a first volume 1A and a write data transfer unit 1B, respectively.
Each sub-storage control device 2 includes a second volume 2A, a write data storage unit 2B, an update unit 2C, and an update order information management unit 2D. Further, at least one sub-memory control device 2 includes an update control unit 3 in addition to the above configurations 2A to 2D.
For convenience of explanation, FIG. 1 shows one host H1 and one H2 and two storage control devices 1 and 2, respectively. However, not limited to this, more hosts and storage control devices can be placed at each site.
Normally, the user performs various business processes and the like using the regular site. Examples of business processing include ordering processing, inventory management processing, customer management processing, e-mail management processing, document management processing, and the like. The user accesses each positive storage control device 1 via the positive host H1 and writes data to the first volume 1A, or reads and uses the data stored in the first volume 1A.
It is also possible to back up the data (user data) used by the user in the regular site. For example, user data can be protected by storing the stored contents of the first volume 1A, which is normally used, in a backup device such as a tape device or a disk device. In addition, by transferring the old data whose utility value has decreased after a predetermined time has passed from the creation to the backup device from the first volume 1A, the free space of the first volume 1A is secured and the operating cost of the storage system is reduced. It can also be reduced.
Even if the user data is protected in the regular site, there is a possibility that the operation of the regular site will be hindered due to a large-scale disaster or the like. In this case, the user's business processing will be stopped for a long period of time until the positive site is restored. Therefore, a secondary site is set up at a place that is an appropriate distance from the primary site. The secondary site stores the same data as the user data managed on the primary site just in case. A system that manages user data at a plurality of physically separated bases and prepares for a disaster is called a disaster recovery system.
It is also conceivable to back up the stored contents of the first volume 1A of the primary site to a tape device or the like, bring this tape device to the secondary site, and update the stored contents of the second volume 2A. However, with such a method, it takes time to match the stored contents of the primary site and the secondary site, and it takes a lot of time and effort.
Therefore, when the primary site and the secondary site are connected by a communication line and the data of the first volume 1A is updated, the newly written write data (updated data) D1 is transferred to the secondary site, and the second volume 1A is updated. Update the data on volume 2A. A technique for transferring and copying data between different storage control devices is called remote copying.
The plurality of positive memory control devices 1 installed at the primary site are connected to the plurality of secondary memory control devices 2 installed at the secondary site via separate communication paths. When the data in the first volume 1A is updated by the positive host H1, each write data transfer unit 2B transfers the write data D1 and the update order information D2 to the sub-storage control device 2.
The update order information D2 is information for instructing the update order, and can include, for example, the time (write time), the sequential number, and the like when the positive host H1 requests the writing of write data. The update order information D2 is information for correctly reflecting the update contents of the first volume 1A, which is the primary volume, on the second volume 2A, which is the secondary volume.
When the sub-storage control device 2 receives the write data D1 and the update order information D2 from the normal storage control device 1, the sub-storage control device 2 stores the write data D1 in the write data storage unit 2B. The update order information D2 is managed by the update order information management unit 2D. The update unit 2C reads the write data D1 stored in the write data storage unit 2B based on the update order information D2, and rewrites the data of the second volume 2A according to the update order.
Each first volume 1A in each normal storage control device 1 stores data related to each other or the same data. One first volume 1A is associated with one second volume 2A to form a remote copy pair. The other first volume 1A is associated with the other second volume to form another remote copy pair. Since the data of each 1st volume 1A is related to each other, the data of each 2nd volume 2A must also be consistent.
The remote copy is executed by each positive storage control device 1 at an arbitrary timing. When a write request is issued from the positive host H1, each positive storage controller 1 notifies the positive host H1 that the processing of the write request is completed when the write data D1 is received, and then another timing. Transfers the write data D1 to the sub-memory control device 2. It is also possible to configure the sub-memory control device 2 to take the initiative in performing remote copying. In this case, for example, the sub-memory control device 2 periodically inquires of the normal memory control device 1 to confirm the presence or absence of update, and reads the write data D1 from the normal memory control device 1.
In this way, the issuance of the write request and the execution of the remote copy are executed at different timings. Therefore, even if the write request is issued from the positive host H1 at the same time, the write data arrives at one sub-memory control device 2, but the write data arrives at the other sub-memory control device 2. It is possible that you have not.
Therefore, in the storage system of the present embodiment, the update control unit 3 is provided in the predetermined sub-memory control device among the plurality of sub-memory control devices 2. The update control unit 3 is for synchronizing the progress status of the remote copy of each sub-memory control device 2.
The update control unit 3 includes, for example, an update order information collection unit 3A, an updateable time point determination unit 3B, and an update instruction unit 3C. The update order information collection unit 3A collects the latest update order information D2 from each update order information management unit 2D. The updatable time point determination unit 3B selects the write time indicated in the oldest update order information among the latest collected update order information as the updatable time. The update instruction unit 3C permits each update unit 2C to update the write data up to the point when it can be updated.
For convenience of explanation, here, for example, in one of the sub-storage control devices 2, the write data D1 (N1a) of the write request issuance time (write time) N1a, and the write data D1 (N2a) and N3a of the same N2a It is assumed that the light data D1 (N3a) has arrived respectively. It is assumed that only the write request issuance time N1b and the write data D1 (N1b) and D1 (N2b) of the same N2b have arrived at the other sub-memory control device 2, respectively. Here, in the order of arrival of the write data, the numerical values "1" and "2" are added to the code "N", but N1a and N1b and N2a and N2b are different times. It is assumed that the time elapses in the order of light time N1 N2 N3. Since the time when each positive storage controller 1 receives the write request from the host H1 is different, the write time of the write data D1 received by each sub-memory controller 2 may coincide with each other, but it is basic. Are different from each other.
The latest update order information managed by one sub-memory controller 2 is information D2 (N3a) related to N3a, and the latest update sequence information managed by the other sub-memory controller 2 is N2b. Information D2 (N2b). Comparing the update order information D2 (N3a) and the update order information D2 (N2b), D2 (N2b) is earlier in time.
Therefore, the updatable time point determination unit 3B determines the time N2b indicated in the oldest update order information D2 (N2b) among the latest update order information D2 (N3a) and D2 (N2b) as the updatable time point. .. The updateable time point is the time point when the write data is allowed to be updated until that time point (N2b in this example). That is, the updateable time point indicates the time when the stored contents of each sub-storage control device 2 can be synchronized.
The update instruction unit 3C notifies the update unit 2C of each sub-memory control device 2 of the determined update possibility time. Each update unit 2C that receives this notification reads the write data D1 stored in the write data storage unit 2B and writes it to the second volume 2A according to the update order.
In the above example, each update unit 2C writes the write data D1 (N1) at the write time N1 to the second volume 2A, and then writes the write data D1 (N2) at the write time N2 to the second volume 2A. On the other hand, the sub-memory control device 2 also obtains the write data D1 (N3) at the write time N3, but this write data D1 (N3) is not written to the second volume 2A. This is because the write data D1 (N3) arrives after the updateable time point N2, and writing to the second volume 2A is not permitted. As a result, the stored contents of each second volume 2A match at the time of write time N2.
When the time elapses and the write data D1 (N3) arrives at the other sub-memory controller 2, N3 is selected as the new updatable time point, and the contents of each second volume 2A match at the write time N3. To do.
Since this embodiment is configured as described above, it has the following effects. In the present embodiment, when remote copying is performed via a plurality of routes, an update control unit 3 is provided for synchronizing the remote copying of each route on the receiving side (secondary storage control device side). Therefore, each update order information D2 is collected from each update order information management unit 2D, the time point at which each second volume 2A can be updated is determined based on each collected update order information D2, and each second volume 2A is determined. It is possible to update the stored contents of each until the time when it can be updated.
As a result, data can be matched between the storage control devices 2 without increasing the processing load of each host H1 and H2. That is, it is so-called host-free, and the results of a plurality of remote copy processes can be synchronized, which improves usability and reliability. Then, when a failure occurs at the primary site and failover is performed to the secondary site, the user's business processing can be continuously executed using the consistent data, and the reliability of the storage system can be improved. .. Hereinafter, the present embodiment will be described in detail.
FIG. 2 is an explanatory diagram showing the overall configuration of the storage system according to this embodiment. This storage system includes, for example, a primary site Ps and a secondary site Ss, and a management server 43 for managing each site Ps and Ss.
The positive site Ps can include at least one positive host 31, a plurality of positive storage control devices 10, and a management terminal 41 that manages each positive storage control device 10. Similarly, the secondary site Ss can include at least one or more secondary hosts 32, a plurality of secondary storage control devices 20, and a management terminal 42. Here, to describe the correspondence with FIG. 1, the primary host 31 and the secondary host 32 correspond to the primary host H1 and the secondary host H2 in FIG. 1, respectively. The normal memory control device 10 corresponds to the normal memory control device 1 in FIG. 1, and the sub-memory control device 20 corresponds to the sub-memory control device 2 in FIG. The primary volume 530A corresponds to the first volume 1A in FIG. 1, and the secondary volume 530B corresponds to the second volume 2A in FIG.
First, the connection configuration of the positive site Ps will be described. The positive host 31 is connected to each positive storage control device 10 via a data input / output path CN11 such as a SAN (Storage Area Network). The positive host 31 and each positive storage control device 10 can be directly connected or can be connected via a relay device such as a Fiber Channel switch.
Each management terminal 41 is for managing the state of the positive storage control device 10 connected to the management terminal 41 and instructing a configuration change. Each positive storage control device 10 is connected to the management network CN14 via the management terminal 41. The management network CN14 is configured as a network such as a LAN (Local Area Network) or WAN, for example. The positive storage control device 10 can be directly connected to the management network CN14 without going through the management terminal 41.
Similar to the primary site Ps, the secondary site Ss is also connected to each secondary storage control device 20 via the data input / output path CN11 such as SAN. Each sub-storage control device 20 is connected to the management network CN14 via the management terminal 42. Further, the sub-memory control devices 20 are connected to each other via an update control path CN13 such as a fiber channel.
The network configuration between each site Ps and Ss will be explained. Each site Ps and Ss are connected via the management network CN14. Each positive storage control device 10 and each sub storage control device 20 are connected to each other via a remote copy path CN12 such as a SAN. The primary volume 530A in each primary storage controller 10 and the secondary volume 530B in each secondary storage controller 20 form a remote copy pair, respectively.
Although the details will be described later, the stored contents of the primary volume 530A are also stored in the secondary volume 530B forming a copy pair by the initial copy. Then, when the data is updated on the primary volume 530A, the information regarding the data update is also transferred to the secondary volume 530B. By updating the stored contents of the secondary volume 530B, it matches the stored contents of the primary volume 530A.
Strictly speaking, the timing when the positive storage controller 10 processes the write request from the primary host 31 and the timing when the remote copy processing is started in response to this write request are not synchronized, so the primary volume 530A and the secondary volume The period in which the stored contents of the 530B completely match is short. However, the data update for the primary volume 530A is also reflected in the secondary volume 530B after a slight time delay. Therefore, after the primary host 31 stops issuing the write request, the stored contents of the primary volume 530A and the secondary volume 530B eventually match with each other over time.
The secondary site Ss may be provided exclusively for backing up the primary site, or may perform business processing unique to the secondary site. That is, the sub-host 32 can also perform another business process by using another volume that is not used for remote copy. In this case, the business process independently executed by the secondary site Ss may be configured so that the primary site Ps can back it up.
Of the plurality of sub-memory control devices 20 provided at the sub-site Ss, any predetermined one is used as the master sub-memory control device 20 (M). The master sub-memory control device 20 (M) centrally manages the progress status of the remote copy of all the sub-memory control devices 20 related to the remote copy.
The management server 43 is connected to the primary host 31 and the secondary host 32, and the primary storage control device 10 and the secondary storage control device 20, respectively, via the management network CN14. The management server 43 manages the states of the hosts 31 and 32 and the storage control devices 10 and 20.
The normal memory control device 10 and the sub-memory control device 20 do not need to be connected one-to-one, and N normal memory control devices 10 may be connected to M sub-memory control devices. That is, as described later, it is sufficient that the logical volume corresponds between the copy source (source side) and the copy destination (destination side) of the remote copy, and the storage control devices 10 and 20 are provided. The numbers of volumes 530A and 530B do not have to be the same.
Further, the sub-host 32 and the positive storage control device 10 may be connected by another network for data input / output. In such a configuration, when the primary host 31 fails, the secondary host 32 can directly access the primary volume 530A and take over the processing performed by the primary host 31.
FIG. 3 is an explanatory diagram showing a part of the hardware configuration of the storage system. Figure 3 shows a part of the configuration of the secondary site Ss. Since the secondary site Ss and the primary site Ps can have the same hardware configuration, the description on the primary site Ps side will be omitted.
The subhost 32 is, for example, a computer device such as a server machine or a mainframe machine. The sub-host 32 can be configured to include, for example, a processor 301, a memory 302, an interface for data input / output (hereinafter, I / F) 303, and an I / F 304 for management.
The processor 301 realizes a predetermined function by reading and executing various programs stored in the memory 302. Examples of various programs include a management unit 312, an OS (Operating System) 322, and an application program (APP) 332, which will be described later together with FIG.
The sub-host 32 is connected to each sub-storage control device 20 from each data input / output I / F 303 via each data input / output path CN11. The management I / F 304 is connected to the management network CN14.
The hardware configuration of the sub-memory control device 20 will be described. The sub-storage control device 20 can be roughly divided into a controller 510 and a storage unit 520. The controller 510 and the storage unit 520 may be provided in the same housing, or may be provided in separate housings.
The controller 510 controls the operation of the sub-memory control device 20. The controller 510 includes, for example, a plurality of channel adapters (hereinafter, CHA) 511, a plurality of disk adapters (hereinafter, DKA) 512, a cache memory 513, a shared memory 514, connection control units 515, 516, and an SVP 517. Can be configured.
Each CHA 511 controls data transfer between the sub-host 32 and each of the storage control devices 10 and 20, and includes a plurality of communication ports 511A. In the example shown in FIG. 3, one CHA511 is used to communicate with the secondary host 32. The other CHA511 is connected to communication with the sub-memory controller 20. Yet another CHA511 is used to communicate with the positive memory controller 10. As described above, instead of CHA511, the sub-memory control devices 20 may be connected by a LAN I / F.
Each DKA512 controls data communication with each disk drive 521. Each DKA512 and each disk drive 521 are connected via a connection control unit 516, and data transfer is performed in block units according to the Fiber Channel protocol. Each DKA512 monitors the status of the disk drive 521 at any time, and the monitoring result is transmitted to the SVP517 via the internal network. Each CHA511 and each DKA512 can be configured as separate control circuit boards, or a CHA function and a DKA function can be provided on one control circuit board, respectively. Each disk drive 521 and each DKA512 are connected to each other via a plurality of communication paths. Even if one of the communication paths fails, each DKA512 can access each disk drive 521 via the other communication path. Furthermore, even if one of the DKA512s fails, each disk drive 521 can be accessed through a normal DKA512.
The cache memory 513 stores the write data transferred from the positive storage control device 10 and the write data received from the sub-host 32. The cache memory 513 can be configured from, for example, a non-volatile memory, but can also be configured from a volatile memory. If the cache memory 513 is composed of volatile memory, the cache memory 513 is backed up by the battery.
The shared memory (or control memory) 514 stores various control information, management information, and the like for controlling the operation of the sub-storage control device 20. The shared memory 514 is composed of, for example, a non-volatile memory. Control information and the like can be multiple-managed by a plurality of shared memories 514.
The cache memory 513 and the shared memory 514 may be configured as separate memory circuit boards, or the cache memory 513 and the shared memory 514 may be mounted in one memory circuit board. Further, a configuration may be configured in which a part of the cache memory is used as a control area for storing control information and the other part is used as a cache area for storing data.
The connection control unit 515 connects each CHA511, each DKA512, a cache memory 513, and a shared memory 514, respectively. All CHA511 and DKA512 can access the cache memory 513 and the shared memory 514 by the connection control unit 515, respectively. The connection control unit 515 is configured as, for example, a crossbar switch or the like.
The SVP517 is connected to each CHA511 via an internal network such as a LAN. The SVP517 can acquire the status of each CHA511 and each DKA512, the usage status of the cache memory 513, and the like. The SVP517 is connected to the management network CN14 via the management terminal 42.
As described above, the controller 510 can be configured by mounting a plurality of types of boards (CHA511, DKA512, etc.) on the controller housing. Not limited to this, each of the above-mentioned functions (communication function with the sub-host 32, communication function with the disk drive 521, data processing function, etc.) may be mounted on a single control board. In this case, it is preferable to provide a plurality of control boards to form a redundant configuration from the viewpoint of improving reliability.
The configuration of the storage unit 520 will be described. The storage unit 520 includes a plurality of disk drives 521. As the disk drive 521, for example, various storage devices such as a hard disk drive, a semiconductor memory drive, an optical disk drive, a magneto-optical disk drive, and a magnetic tape drive can be used. When using a hard disk drive, various types of hard disk drives can be used, for example, FC (Fibre Channel) disk, SATA (Serial AT Attachment) disk, and SCSI (Small Computer System Interface) disk. Further, different types of storage devices can be mixed in the storage unit 520.
A RAID group (parity group) is composed of a predetermined number of disk drives 521, although it differs depending on the RAID level and the like. For example, a RAID 5 configuration can be obtained from three data disks and one parity disk (3D + 1P).
A RAID group is a redundant physical storage area and can also be called a physical storage device. One or more logical volumes 530B, which are logical storage devices, can be set on the physical storage area provided by the RAID group. In the following description, a logical volume may be abbreviated as LU (Lugical Unit). The journal volume 531B is configured as a logical storage device like the logical volume 530B. The journal volume 531B can store write data and the like transferred from the positive storage control device 10.
Since the primary host 31 and the primary storage control device 10 can have the same hardware configuration as the secondary host 32 and the secondary storage control device 20, the description thereof will be omitted. However, the primary host 31 and the secondary host 32, and the primary storage control device 10 and the secondary storage control device 20 do not need to have the same hardware configuration. For example, the physical configuration of the primary site Ps and the secondary site Ss, as in the case where the latest memory control device is used as the positive memory control device 10 and the old memory control device is used as the secondary memory control device 20. May be different. Note that the journal volume 531A of the normal storage control device 10 is the untransferred data when the transfer between the normal storage control device 10 and the sub-memory control device 20 is temporarily impossible due to a link failure (path failure) or the like. Can be used to store. In this case, after recovering from the link failure, the untransferred data accumulated in the journal volume 531A is transferred to the sub-storage control device 20. By using the journal volume 531A on the positive side in this way, resistance to link failure can be improved.
FIG. 4 is an explanatory diagram showing the software configuration of the storage system. The primary host 31 is a computer device used for normal operation, and the secondary host 32 is a computer device that is on standby in preparation for the occurrence of a failure or the like. On the positive host 31, for example, the management unit 311, the OS 321, and the application program (abbreviated as "APP" in the figure) 331 are operating. Similarly, the management unit 312, the OS 322, and the application program 332 are operating on the sub-host 32, respectively. Application programs 331,332 also include software such as a database management system (DBMS), for example. As will be described later, the management units 311, 312 perform processing such as registering a volume in a group that performs remote copying, and instructing suspend processing.
The controller 510 of the positive storage control device 10 realizes the write data receiving unit 110, the write data transfer unit 120, and the marker creating unit 130, respectively. The write data receiving unit 110 is a function for receiving a write request and write data transmitted from the positive host 31. The write data transfer unit 120 is a function for transferring write data or the like to the copy destination sub-storage control device 20. The marker creation unit 130 is a function of transmitting control information for controlling the suspend process to the sub-memory control device 20. Each of these functions 110, 120, 130 is realized, for example, by the processor in CHA511 reading and executing a predetermined program.
The shared memory 514 of the normal storage control device 10 stores, for example, the volume group management information T11, the other party volume information T12, the write data management information T13, the latest write time management information T14, and the difference bitmap T15. be able to. Information for these controls will be further described later.
The cache memory 513 of the positive storage control device 10 stores the write data received from the positive host 31. The journal volume 531A may be used as a light data storage device. By storing the write data in the journal volume 531A, the usage rate of the cache memory 513 can be reduced, and a large amount of write data can be stored. The size of the journal volume 531A may be a fixed value set in advance, or may be set as needed at that time.
The configuration of the sub-memory control device 20 will be described. The controller 510 of the sub-memory control device 20 realizes the write data receiving unit 210, the write data reflecting unit 220, and the master processing unit 230, respectively. The write data receiving unit 210 is a function for receiving the write data or the like transmitted from the positive storage control device 10. The write data reflection unit 220 is a function for writing the write data received from the positive storage control device 10 to the sub volume 530B in order.
The master processing unit 230 operates by any one of the plurality of sub-memory control devices 20. The master processing unit 230 is a function for controlling the operation of the write data reflection unit 220 of each sub-storage control device 20 that performs remote copying. The master processing unit 230 can give a predetermined instruction to the write data reflection unit 220 in the other sub-storage control device 20 via the update control path CN13. It should be noted that the master processing unit 230 may be mounted in each sub-storage control device 20 in advance, and only one of the master processing units 230 may be activated.
In the shared memory 514 of the sub-memory control device 20, for example, the volume group management information T21, the other party volume information T22, the write data management information T23, the arrived write time management information T24, and the reflectable write time management information T25 , Master information T26, extended group management information T27, master time information T28, and difference bitmap T29 are stored, respectively.
The difference bitmaps T29 and T15 are for managing the update status of the volumes 530B and 530A, respectively. For example, the difference bitmaps T29 and T15 are composed of a sequence of bit data corresponding to each block of the volume, and the updated block is set with bit "1" as an update flag. Therefore, by using the difference bitmaps T29 and T15, it is possible to determine which part of the volume has been updated.
Here, in the storage system of this embodiment, by grouping the primary volume 530A and the secondary volume 530B, remote copying can be performed in group units. The user can appropriately set from which volume the group is to be formed. This group setting can be performed from, for example, hosts 31, 32 or management terminals 41, 42 or management server 43.
FIG. 5 is an explanatory diagram showing the concept of a logical volume group. In this embodiment, the relationships between volumes 530A and 530B are grouped. In Fig. 5, volumes 530A and 530B forming a remote copy pair are connected by wavy lines. The primary volume 530A is the copy source, and the secondary volume 530B is the copy destination.
Volume group 540A is formed by one or more positive volumes 530A. Similarly, one or more secondary volumes 530B form a volume group 540B. The volume group (primary volume group) 540A on the source side and the volume group (secondary volume group) 540B on the destination side are associated with each other, and remote copy is performed between the volumes 530A and 530B constituting each group 540A and 540B. Each is to be done. Further, as described later, the remote copy reflection process is controlled in units of the sub-volume group 540B.
Computer resources required for transferring write data and writing write data to volume 530B (reflection processing) are allocated to each volume group 540A and 540B, respectively.
If remote copy is managed for each volume 530A and 530B or resources are allocated individually, the number of management targets increases and the management work becomes complicated. In addition, since the number of processing targets increases, the resources required for processing may increase, and the load on each of the storage control devices 10 and 20 increases.
On the other hand, if the entire normal memory control device 10 and the sub-memory control device 20 are regarded as one management unit, it is difficult to manage the volumes 530A and 530B according to their characteristics. The performance required for volumes 530A and 530B differs depending on the type of application program 331 and the type of business processing.
For example, one application program 331 requires a higher performance and more reliable volume, and another application program 331 requires a lower cost volume. Therefore, it is preferable to group the volumes 530A and 530B according to their characteristics (for example, the type and volume size of the disk drive 521) and manage the remote copy for each volume group 540A and 540B.
Therefore, by grouping and managing the volumes as in the present embodiment, the user can set the remote copy configuration and conditions for each of the volume groups 540A and 540B, which improves convenience.
Further, in this embodiment, the expansion group 550 can be configured by selecting an arbitrary sub-volume group 540B from each sub-volume group 540B. The expansion group 550 can be configured across housings. That is, one expansion group 550 can be configured from the sub-volume group 540B in one sub-memory control device 20 and the sub-volume group 540B in another one sub-memory control device 20. Although only one extension group 550 is shown in FIG. 5, the user can configure multiple extension groups 550.
Each sub-volume group 540B belonging to a certain expansion group 550 is controlled so that the progress of remote copying is synchronized even if they exist in different sub-storage controllers 20 from each other. That is, as described later, in the state where the remote copy is stopped (suspended state), the write data is reflected between the plurality of sub-volume groups 540B belonging to the extended group 550 until a predetermined time (reflectable write time). Therefore, the integrity of the data is maintained.
Next, various information for controlling remote copying will be described. First, FIG. 6 is an explanatory diagram showing an example of volume group management information T11 (T21) and counterparty volume information T12 (T22).
The volume group management information T11 (T21) is information for managing the volume groups 540A and 540B shown in FIG. The volume group management information T11 stored in the normal storage control device 10 manages the configuration of the normal volume group 540A provided on the normal storage control device 10 side. The volume group management information T21 stored in the sub-memory control device 20 manages the configuration of the sub-volume group 540B provided on the sub-memory control device 20 side.
The volume group management information T11 and T21 have the same structure. Therefore, the volume group management information T11 will be mainly described below. However, even if the structure itself is common, the values set for each item may be different.
The volume group management information T11 is configured by, for example, associating a volume group ID, a sequential number, a number of volumes, a volume ID, a remote storage control device ID, a remote group ID, and an untransferred sequential number.
The volume group ID is identification information for identifying the volume group. In the volume group management information T11 on the positive side, identification information for identifying the primary volume group 540A is set in the volume group ID. In the volume group management information T21 on the secondary side, identification information for identifying the secondary volume group 540B is set in the volume group ID.
The sequential number is information that manages the issuance order and the number of issuances of write data to the volumes belonging to the volume group specified by the volume group ID. Sequential numbers are given continuously for write data to the volume. Assuming that the initial value is "0", the sequential number value is incremented by 1 each time the write data is issued.
The number of volumes is information indicating the total number of volumes belonging to the volume group specified by the volume group ID.
The volume ID is identification information for uniquely identifying one or more volumes belonging to the volume group specified by the volume group ID. The volume IDs of all the volumes belonging to the volume group are registered in the volume group management information T11.
The remote storage control device ID is identification information for identifying a storage control device having a remote volume group that forms a pair of a volume group specified by the volume group ID and a remote copy. For the other party's memory control device ID, for example, a serial number or the like preset in each memory control device can be used. In the volume group management information T11 on the positive side, the serial number of the secondary storage control device 20 is set as the remote storage control device ID. In the volume group management information T21 on the secondary side, the serial number of the positive storage control device 10 is set.
The remote party group ID is identification information for identifying the volume group that forms a pair of the volume group specified by the volume group ID and the remote copy. In the volume group management information T11 on the positive side, identification information for identifying the secondary volume group 540B is set as the counterparty group ID. In the volume group management information T21 on the secondary side, identification information for identifying the primary volume group 540A is set as the partner group ID.
The untransferred sequential number is information used in the write data transfer process described later, and indicates the minimum sequential number of the write data that needs to be transferred. The untransferred sequential number is used only by the normal storage control device 10 and not by the sub-memory control device 20. Therefore, the untransferred sequential number is not used in the volume group management information T21.
The other party's volume information T12 (T22) is shown at the bottom of FIG. The other party volume information T12 (the other party volume information on the primary side) used by the primary storage control device 10 and the other party volume information T22 (the other party volume information on the secondary side) used by the secondary storage control device 20 have the same structure. It has. Therefore, the explanation will be centered on the other party volume information T12 on the positive side.
The remote copy pair relationship is defined in the remote volume information T12. The remote party volume information T12 is configured by, for example, associating a volume ID, a remote party storage control device ID, and a remote party volume ID.
Information for identifying the primary volume 530A as the copy source is set in the volume ID. Information for identifying the sub-memory control device 20 having the sub-volume 530B forming a copy pair with the primary volume 530A specified by the volume ID is set in the remote memory control device ID.
Information for identifying the secondary volume 530B that forms a copy pair with the primary volume 530A specified by the volume ID is set in the remote volume ID.
In the case of the counterparty volume information T22 on the secondary side, the volume ID contains information for identifying the secondary volume 530B, the counterparty memory controller ID contains information for identifying the primary memory controller 10, and the counterparty volume ID contains information for identifying the primary volume 530B. Information for identifying the positive volume 530A is set respectively.
FIG. 7 shows the write data management information T13 (T23) and the latest write time management information T14, respectively. The write data management information T13 on the primary side and the write data management information T23 on the secondary side have the same structure. Therefore, the light data management information T13 on the positive side will be mainly described.
The write data management information T13 is information for managing the write data written in the primary volume 530A which is the copy source. The write data management information T13 is created and managed in a list format for each primary volume group 540A, for example.
One write data management information T13 is configured by, for example, associating a volume ID, a write address, a write data length, a write data pointer, a sequential number, a write time, a marker attribute bit, and a marker type. Will be done.
Information for identifying the positive volume 530A is set in the volume ID. The write address indicates the start address to which the write data is written in the specified positive volume. The write data length is the size of the write data. The write data pointer is the start address of the write data stored in the cache memory 513.
The positive storage control device 10 notifies the positive host 31 of the completion of writing when the write data received from the positive host 31 is stored in the cache memory 513. After that, the positive storage control device 10 writes the write data stored in the cache memory 513 to the positive volume 530A at an appropriate timing. Therefore, the write data is first stored in the cache memory 513 and then stored in the positive volume 530A. Therefore, the write data management information T13 includes a write data pointer.
The status of the write data before being written to the primary volume 530A is "dirty". The status of the write data after being written to the primary volume 530A is "clean". When the write data on the cache memory 513 changes to the clean status, the cache area in which the write data is stored can be reused for storing other write data.
The sequential number is a number continuously assigned to the write data written to the positive volume 530A specified by the volume ID. The write time is the time when the positive host 31 requests the write of write data. That is, this write time indicates the time when a write request is issued from the positive host 31 for the specified positive volume 530A.
The marker attribute bit is information for identifying whether the light data for controlling the remote copy is special light data (hereinafter, the light data for this control is referred to as a "marker") or the normal light data. Is. Details will be described later, but in the case of normal write data, "0" is set for the marker attribute bit. In the case of a marker, "1" is set in the marker attribute bit.
The marker type is information for specifying the marker type. Details will be described later, but examples of the marker type include "flash suspend", "purge suspend", and "swap suspend".
The latest light time management information T14 is shown at the bottom of FIG. The latest write time management information T14 manages the write time of the latest write data requested to be written to the primary volume 530A in each remote copy. Therefore, the latest write time management information T14 is prepared for each positive volume 530A. The latest write time management information T14 is configured by, for example, associating a group ID for identifying the primary volume group 540A with the latest write time.
FIG. 8 shows the extended group management information T27 and the master information T26. This will be explained from the extended group management information T27. The expansion group management information T27 is information for managing the expansion group 550 that can be formed across the plurality of sub-memory control devices 20. Extended group management information T27 is prepared for each extended group 550.
The extended group management information T27 can be configured, for example, by associating the extended group ID, the number of volume groups, the storage control device ID, and the volume group ID.
The extension group ID is identification information for identifying the extension group 550. The number of volume groups indicates the number of sub-volume groups 540B contained in the specified expansion group 550. The storage control device ID is identification information for identifying the sub-storage control device 20 in which the sub-volume group 540B included in the expansion group 550 exists. The volume group ID is identification information for identifying the sub-volume group 540B included in the extension group 550.
The master information T26 is shown at the bottom of FIG. The master information T26 is information for specifying the master processing unit 230. As will be described later, the master processing unit 230 gives a predetermined instruction to each sub-memory control device 20 regarding synchronization of remote copy and the like. Therefore, each sub-memory control device 20 holds the master information T26 as information indicating where a predetermined instruction is received from.
The master information T26 is configured by, for example, associating a volume group ID with an extended group ID and a master storage control device ID. The volume group ID is identification information for identifying the secondary volume group 540B. The extension group ID is candy identification information that identifies the extension group 550 to which the identified sub-volume group 540B belongs. The master storage control device ID is identification information for identifying the sub storage control device 20 having the master processing unit 230 for issuing a predetermined instruction to the specified sub volume group 540B.
In this embodiment, one master processing unit 230 is configured to arbitrate the progress of remote copying in each sub-memory control device 20. Instead of this, for example, a master processing unit 230 may be provided in different sub-memory control devices 20, and each master processing unit 230 may be in charge of a separate expansion group 550.
FIG. 9 shows the arrived light time management information T24, the master time information T28, and the reflectable light time management information T25, respectively.
The arrival write time management information T24 is information for managing the write time of the latest write data arriving at the sub-memory control device 20. The arrival write time management information T24 is configured by, for example, associating the volume group ID, the arrival write time, the marker type, and the marker time.
The volume group ID is identification information for identifying the secondary volume group 540B. In the arrived write time, the write time of the latest write data among the arrived write data related to the specified sub-volume group 540B is recorded. The initial value of the arrival light time is "0". The marker type is information indicating the type of the marker detected by the write data receiving unit 210, and the initial value is an invalid value. The marker time is the time given to the marker, and the initial value is "0".
The master time information T28 is information used by the master processing unit 230. The master time information T28 is configured by, for example, associating an extended group ID, a storage control device ID, a volume group ID, an arrived write time, a marker type, and a marker time.
The extension group ID is identification information for identifying the extension group 550. The storage control device ID is identification information for identifying the sub-memory control device 20 having the sub-volume group 540B belonging to the specified expansion group 550. The volume group ID is identification information for identifying the sub-volume group 540B belonging to the specified extended group 550. The arrived write time indicates the write time of the latest write data among the arrived write data for the sub-volume group 540B. The initial value of the arrival light time is "0". Each sub-memory control device 20 notifies the master processing unit 230 of the write time of the latest write data arriving at the sub-volume group 540B. The marker type is information indicating the marker type, and the initial value is an invalid value. The marker time is the time given to the marker, and the initial value is 0.
The reflectable write time management information T25 is information for managing the reflectable write time for each sub-volume group 540B. The reflectable write time is information that specifies the range of write data that can be reflected in each sub-volume 530B belonging to the sub-volume group 540B. The reflectable write time corresponds to the "updateable time".
Write data can be written to the secondary volume 530B in order until the time indicated as the reflectable write time. Take the case where there are four light data of write time N1, N2, N3, and N4 as an example. Suppose light time N4 is the latest light time. If the master processing unit 230 selects N3 as the reflectable write time, the write data reflection unit 220 writes the write data of the write time N1 to the secondary volume 530B, and then writes the write data of the write time N3 to the secondary volume. The write data can be written to the 530B, and finally, the write data of the write time N3 can be written to the secondary volume 530B.
The reflectable write time management information T25 is configured by, for example, associating the volume group ID with the reflectable write time and the marker type. The volume group ID is identification information for identifying the secondary volume group 540B. The reflectable write time is as described above, and its initial value is "0". The marker type indicates the marker type, and the initial value is an invalid value.
FIG. 10 is a flowchart showing the processing of the write request. This write request processing is executed by the controller 510 (specifically, the write data receiving unit 110) of the positive storage control device 10. For convenience of explanation, the subject of operation is the positive memory control device 10. However, in practice, the write request is processed by one or more processors in the controller 510 operating according to a predetermined program. The same applies to each of the following flowcharts, but each flowchart shows the content of each process to the extent necessary for understanding and implementing the present invention, and is different from the actual program. Further, in the following description, the step is abbreviated as "S".
The positive storage controller 10 receives a write request for writing the positive volume 530A from the positive host 31 (S11). The write time is assigned to this write request by the positive host 31.
The positive storage control device 10 stores the write data received from the positive host 31 in the cache memory 513 (S12). The positive storage control device 10 refers to the volume group management information T11, increments the already stored sequential number by one, and acquires a new sequential number (abbreviated as SEQ number in the figure) (S13). ..
The positive storage control device 10 generates write data management information T13 by associating the acquired sequential number and write time with the write data received from the positive host 31 (S14). As described above, the volume ID of the positive volume 530A designated as the write destination of the write data, the write address, and the like are also set in the write data management information T13.
The positive storage control device 10 stores the latest write time management information T14 in the write time indicated in the write request received in S11 (S15). Then, the positive storage control device 10 notifies the positive host 31 that the processing of the write request is completed (S16).
That is, the positive storage control device 10 reports to the positive host 31 that the writing of the write data is completed before the write data is actually written to the positive volume 530A. After reporting the completion of writing to the primary host 31, writing to the primary volume 530A and transfer to the secondary volume 530B are performed, so that the response performance of the positive storage control device 10 can be improved. Further, as a result, even when the primary site Ps and the secondary site Ss are far apart, the write request processing can be promptly performed without causing the primary host 31 to recognize the delay due to the propagation time between the two sites. Therefore, the asynchronous processing described above is suitable for a disaster recovery system in which the primary site Ps and the secondary site Ss are separated from each other.
Finally, the positive storage control device 10 writes the write data stored in the cache memory 513 to each disk drive 521 constituting the positive volume 530A at an appropriate timing (S17). The method of writing to disk drive 521 depends on the RAID configuration of the primary volume 530A. If the primary volume 530A is set to RAID1, the same write data is written to both the primary disk drive and the secondary disk drive. If the positive volume 530A is set to use parity data such as RAID5, the write data and the parity data are written to the predetermined disk drive 521 after the new parity data is calculated.
FIG. 11 is a flowchart showing a process for transferring write data from the primary storage control device 10 to the secondary storage control device 20 and copying the write data to the secondary volume 530B. This process can be called a write data transfer process or a remote copy process.
The process shown in FIG. 11 is executed by the write data transfer unit 120 of the normal storage control device 10 and the write data receiving unit 210 of the sub storage control device 20. More specifically, as described above, the processing shown in FIG. 11 is realized by executing a predetermined program by one or more processors of the storage control devices 10 and 20.
First, the normal storage control device 10 refers to the untransferred sequential number stored in the volume group management information T11, and identifies the write data that needs to be transferred to the sub-storage control device 20. Then, the positive storage control device 10 generates write data-related information by referring to the write data management information T13, the volume group management information T11, and the other party's volume information T12, respectively (S21).
The write data-related information is information including various information necessary for managing the write data in the sub-memory control device 20 of the transfer destination (copy destination). The write data-related information includes a write address, a write data length, a sequential number, a write time, a marker attribute bit, and a marker type. These write addresses, sequential numbers, etc. are obtained from the write data management information T13, respectively. The write data-related information also includes information for identifying the transfer destination of the write data. The information for specifying the transfer destination of the write data is the other party's memory control device ID and the other party's volume ID acquired from the other party's volume information T12. Further, the write data related information also includes the counterparty volume group ID acquired from the volume group management information T11.
Then, the normal storage control device 10 associates the write data stored in the cache memory 513 with the write data-related information created in S22, and transfers the write data to the sub-storage control device 20 (S22).
When the sub-storage control device 20 receives the write data and the write data-related information from the normal storage control device 10 (S23), the sub-storage control device 20 stores the write data in the cache memory 513 in the sub-storage control device 20 (S24).
The sub-memory control device 20 generates the write data management information T23 based on the write data-related information received from the normal memory control device 10 (S25). Then, the sub-memory control device 20 determines whether or not the sequential number set in the write data received in S23 is missing (S26).
When the sequential numbers are consecutive (S26: YES), the sub-storage controller 20 uses the write time of the latest write data received in S23 and the arrival write time stored in the arrival write time management information T24. And, it is determined whether or not the arrived write time is earlier than the write time of the light data received in S23 (S27). The judgment steps of S26 and S27 can be interchanged.
If the arrived write time stored in the arrived write time management information T24 is older than the write time of the write data received in S23 (S27: YES), the sub-storage controller 20 determines the arrived write time. The arrival write time registered in the management information T24 is replaced with the write time of the write data received in S23 (S28). The sub-memory control device 20 reports to the normal memory control device 10 that the reception of the write data is completed (S29).
When the primary storage controller 10 receives the completion report from the secondary storage controller 20, it updates the untransferred sequential number in the volume group management information T11 (S30). When the write data transfer to the sub storage controller 20 is completed, if the transferred write data has already been written to the primary volume 530A, the primary storage controller 10 writes the write stored in the cache memory 513. Data can be discarded. If write data is held for transfer to the sub-memory control device 20 separately from for writing to the primary volume 530A, write data for this transfer is received after the completion report from the sub-memory control device 20 is received. Can be destroyed.
The process shown in FIG. 11 is configured to transmit write data or the like from the normal memory control device 10 to the sub-memory control device 20 (S22). Instead of this, the sub-memory control device 20 issues a write data transfer request to the normal memory control device 10, and the normal memory control device 10 sends write data or the like to the sub-memory control device 20 in response to the transfer request. It may be configured to transmit. In this way, when the configuration is such that write data or the like is transmitted from the normal memory control device 10 to the sub-memory control device 20 in response to the transfer request from the sub-memory control device 20, the status of the sub-memory control device 20 is determined. , Write data transfer processing can be performed. That is, for example, the normal storage control device 10 writes to the sub-memory control device 20 according to the write data processing status, the load amount, the write data storage amount (the usage rate of the cache memory 513), and the like in the sub-memory control device 20. Data etc. can be transferred.
Further, in the process shown in FIG. 11, the sub-memory control device 20 is described as storing the write data in the cache memory 513 (S24). Instead of this, when the sub-storage control device 20 has the journal volume 531B, the journal volume 531B may be configured to store the write data. In general, the journal volume 531B can be formed to have a larger capacity than the cache memory 513, so that more write data can be stored. When the journal volume 531B is used, the cache memory 513 is used as the cache area of the secondary volume 530B and the journal volume 531B. Similarly, in the positive storage control device 10, the journal volume 531A can be used as a cache area.
The volume size of the journal volume 531B can also be set variably. For example, if the size of the journal volume 531B is initially set small and the amount of write data transferred from the positive storage controller 10 increases, the journal volume responds to the increase in the amount of write data transferred. It may be configured to expand the size of 531B.
FIG. 12 is a flowchart showing a remote copy reflection process executed by each sub-memory control device 20 and an arbitration process for synchronizing a plurality of reflection processes. In this embodiment, the data consistency between each sub-volume 530B is guaranteed by this arbitration process. The master processing unit 230 is operating on any one of the sub-memory control devices 20. The sub-memory control device 20 in which the master processing unit 230 is operating is shown as a master sub-memory control device, and the other sub-memory control device 20 that receives an instruction from the master processing unit 230 is shown as a slave sub-memory control device in the figure. ing.
The master processing unit 230 inquires about the latest arrived write time stored in the arrived write time management information T24 for each of the sub-volume groups 540B belonging to the extended group 550 (S41). The master processing unit 230 inquires the controller 510 of each sub-memory control device 20 about the latest arrived write time via the update control path CN13.
When each sub-storage control device 20 receives an inquiry from the master processing unit 230, it refers to the arrival write time management information T24 (S42) and transmits the arrival write time stored therein to the master processing unit 230. (S43).
The master processing unit 230 records the arrived write time received from each sub-memory control device 20 in the master time information T28 (S44). The master processing unit 230 detects one of the oldest arrived write times by comparing each arrived write time recorded in the master time information T28 (S45). The earliest detected arrival light time is selected as the reflectable light time. Then, the master processing unit 230 permits each sub-memory control device 20 to write the write data arriving by the reflectable write time to the sub-volume 530B (S46).
Each sub-storage control device 20 stores the reflectable write time notified from the master processing unit 230 in the reflectable write time management information T25 (S47). With the arbitration processing of S41 to S47 as one cycle, the master processing unit 230 repeats this arbitration processing periodically or irregularly.
As described above, in the arbitration process, an instruction is given between the sub-memory control device (master sub-memory control device) in which the master processing unit 230 operates and the other sub-memory control device 20 via the update control path CN13. And responses are exchanged. Therefore, by monitoring the exchange status of these instructions and responses, it is possible to detect whether or not a failure has occurred in each sub-memory control device 20 or the update control path CN13. For example, if there is no response from the other sub-memory control device 20 within a certain period of time in response to the instruction from the master processing unit 230, it can be estimated that a failure has occurred.
FIG. 13 is a flowchart showing the write data reflection processing on the sub-volume 530B in the sub-memory control device 20. The reflection process is a process of updating the contents of the secondary volume 530B by writing the write data received from the positive storage control device 10 to the secondary volume 530B. The write data reflection unit 220 performs reflection processing for each sub-volume group 540B. Hereinafter, the main body of operation will be described as the sub-memory control device 20.
The sub-memory control device 20 refers to the write data management information T23 for the sub-volume 530B belonging to a certain sub-volume group 540B (S51), and selects the write data management information in the order of write time and sequential number (S52). As a result, the order in which the arrived write data is stored in the secondary volume 530B is determined.
The sub-memory controller 20 refers to the master information T26 of the sub-volume group 540B (S53), and determines whether or not the sub-volume group 540B belongs to the expansion group 550 (S54). That is, the sub-memory control device 20 determines whether or not the master information T26 is set for the sub-volume group 540B.
When the secondary volume group 540B belongs to the extended group 550 (S54: YES), the secondary volume group 540B is the target of the synchronous reflection processing. Therefore, the sub-memory control device 20 compares the write time of the write data selected in S52 with the reflectable write time stored in the reflectable write time management information T25. When the write time of the selected write data is earlier than the reflectable write time, the sub-memory control device 20 writes the selected write data to the sub-volume 530B (S56). That is, the sub-storage control device 20 refers to the write data management information T23 for the write data to be written, specifies the write address and the write data length, and stores the write data in a predetermined location of the sub-volume 530B.
On the other hand, when the sub-volume group 540B does not belong to the expansion group 550 (S54: NO), the sub-memory controller 20 skips S55 and moves to S56, and writes the write data to be written to the sub-volume 530B. .. This is because if the secondary volume group 540B does not belong to the extended group 550, the secondary volume group 540B is not subject to the synchronous reflection processing.
With the reflection processing of S51 to S56 described above as one cycle, the sub-memory control device 20 performs the reflection processing for each sub-volume 530B. As a result, the sub-memory control device 20 can write the write data received by the reflectable write time to the sub-volume 530B in the order of the write time and the sequential number for the sub-volume group 540B belonging to the expansion group 550. it can.
Here, the operation of comparing the write time and the reflectable write time and writing the write data up to the reflectable write time to the sub-volume 530B in the order of the write time and the sequential number is described in the following description as "By instruction". It is called "reflection processing mode". On the other hand, the operation of writing all the arrived write data to the sub volume 530B in the order of write time and sequential number is called "automatic reflection processing mode". In the "reflection processing mode by instruction", the update time of each sub-volume 530B belonging to the expansion group 550 matches the time permitted by the master processing unit 230 (reflectable write time), so that the "synchronous reflection processing mode" is used. Can also be called. Further, for example, the "automatic reflection processing mode" may be referred to as a first reflection processing mode, and the "reflection processing mode by instruction" may be referred to as a second reflection processing mode.
As described above, in the "reflection processing mode by instruction", the write data is reflected in each sub-volume 530B belonging to the expansion group 550 until the reflection-enabled write time instructed by the master processing unit 230.
Therefore, even when a plurality of remote copies in which data transfer timing and buffering on the data transfer path operate asynchronously are performed in parallel, the storage contents of the sub-volume 530B can be made consistent.
As a result, for example, even if a failure occurs in the positive memory control device 10 or the like and some write data does not arrive at the secondary volume 530B, the write data issued after the undelivered write data will be the secondary volume. Reflection to 530B is not allowed. Therefore, in each sub-volume 530B belonging to the expansion group 550, the update order is not changed and some write data is not lost, and the consistency is ensured. Therefore, the secondary host 32 can take over the business processing of the primary host 31 by using the secondary volume 530B whose consistency is ensured, and can realize a highly reliable failover.
FIG. 14 is a flowchart showing a process for registering the sub-volume group 540B in the expansion group 550. As described above, when the sub-volume group 540B is registered in the expansion group 550, the "reflection processing mode by instruction" is applied to the sub-volume 530B belonging to the sub-volume group 540B.
First, the management unit 312 (hereinafter referred to as the sub-host 32) of the sub-host 32 requests the master sub-memory control unit 20 in which the master processing unit 230 is operating to register the sub-volume group 540B in the expansion group 550 (hereinafter, the sub-host 32). S61). In this registration request, for example, an expansion group ID of the expansion group 550 to be registered, a volume group ID indicating a sub-volume group 540B desired to be registered in the expansion group 550, and a sub-volume group 540B thereof are provided. The memory control device ID of the sub-memory control device 20 is included.
When the master processing unit 230 receives the registration request from the secondary host 32, the master processing unit 230 requests the secondary storage controller 20 having the designated secondary volume group 540B to register the designated secondary volume group 540B in the expansion group 550. (S62). The request issued from the master processing unit 230 includes, for example, the volume group ID and the extended group ID specified by the sub-host 32.
The sub-memory control device 20 that has received the request from the master processing unit 230 sets the master information T26 (S63). That is, the sub-memory control device 20 having the sub-volume group 540B to be registered has the sub-storage control device 20 having the designated extended group ID and the master processing unit 230 in the master information T26 corresponding to the designated volume group ID. Set the storage control device ID of (S63).
The sub-memory control device 20 in which the master information T26 is set changes the processing mode of the sub-volume group 540B registered in the expansion group 550 from the "automatic reflection processing mode" to the "reflection processing mode by instruction" (S64). The sub-memory control device 20 that has changed the processing mode reports to the master sub-memory control device 20 that the registration process has been completed (S65).
The master sub-memory controller 20 refers to the extended group management information T27 (S66), and determines whether or not the extended group ID set in the master information T26 in S63 is registered in the extended group management information T27 (S66). S67).
If the extended group ID is not registered in the extended group management information T27 (S67: NO), it means that the first secondary volume group 540B is registered in the extended group ID. Therefore, the master sub-memory control device 20 sets a new expansion group ID, a storage control device ID of the sub-memory control device 20, and a volume group ID of the sub-volume group 540B in the expansion group management information T27 ( S68). In addition, "1" is set in the "number of volume groups".
Subsequently, the master sub-memory control device 20 executes the above-mentioned arbitration process for the sub-volume group 540B belonging to the newly established expansion group 550 (S69). That is, when the expansion group 550 is newly set (S67: NO), the master sub-storage control device 20 matches the storage contents of each sub-volume 530B belonging to the expansion group 550.
On the other hand, when the extended group ID set in the master information T26 in S63 is already registered in the extended group management information T27 (S67: YES), the master sub-storage controller 20 updates the contents related to the extended group ID. (S70). That is, the master sub-memory control device 20 obtains the volume group ID of the added sub-volume group 540B and the storage control device ID of the sub-memory control device 20 having the added sub-volume group 540B in the extended group management information T27. Set to each. Further, the master sub-memory control device 20 increases the "number of volume groups" by one.
After completing the setting change of the extended group management information T27, the master sub-storage controller 20 reports to the sub-host 32 that the registration of the sub-volume group 540B to the extended group 550 is completed (S71). As a result, the sub-host 32 confirms that the registration to the extension group 550 is completed. For convenience of explanation, FIG. 14 shows that the master sub-memory control unit 20 reports the completion of the setting to the sub-host 32 after the arbitration process (S69). However, the extended group management information T27 After the setting of (S68), the setting completion may be reported.
FIG. 15 is a flowchart showing a process for deleting the sub-volume group 540B from the expansion group 550.
The secondary host 32 requests the master secondary storage controller 20 to delete the secondary volume group 540B from the expansion group 550 (S81). In this deletion request, for example, the volume group ID of the sub-volume group 540B to be deleted, the storage control device ID of the sub-storage control device 20 having the sub-volume group 540B to be deleted, and the sub-volume group 540B to be deleted are included. The extension group ID of the extension group 550 to which it belongs is included.
When the master sub-storage controller 20 receives the deletion request from the sub-host 32, the master sub-memory controller 20 requests the sub-storage controller 20 specified by the storage controller ID to delete the sub-volume group 540B from the extended group 550 (S82). ). The deletion request includes, for example, the volume group ID of the secondary volume 530B to be deleted and the extension group ID of the extension group 550.
Upon receiving the deletion request from the master sub-memory controller 20, the sub-memory controller 20 initializes the master information T26 associated with the designated sub-volume group 540B (S83). Subsequently, the sub-memory control device 20 changes the processing mode of the sub-volume group 540B, which has been deregistered in the expansion group 550, from the instructed reflection processing mode to the automatic reflection mode (S84). The sub-memory controller 20 whose processing mode has been changed reports to the master sub-memory controller 20 that the deletion from the expansion group 550 has been completed (S85).
When the master sub-storage controller 20 receives the completion report issued in S85, it refers to the expansion group management information T27 (S86), and is set to the deleted expansion group 550 of the sub-volume group 540B. It is determined whether or not the "number" is "1" (S87).
When the "number of volume groups" of the expansion group 550 is "2" or more (S87: NO), the master sub-storage controller 20 reduces the value of the "number of volume groups" by 1 and also deletes the sub. Delete the volume group ID of volume group 540B from the extended group management information T27 (S88).
When the "number of volume groups" set in the deleted expansion group 550 of the sub-volume group 540B is "1" (S87: YES), the master sub-storage controller 20 performs the expansion group management information regarding the expansion group 550. Initialize T27 (S90). This is because no secondary volume group 540B is registered in the expansion group 550. Then, the master sub-memory control device 20 stops the arbitration process (S91).
The master secondary storage controller 20 notifies the secondary host 32 that the deletion of the secondary volume group 540B from the expansion group 550 has been completed (S89). As a result, the secondary host 32 confirms that the deletion of the secondary volume group 540B from the expansion group 550 is completed (S92).
Here, for each sub-volume group 540B belonging to the expansion group 550, the conditions for ensuring the consistency of the stored contents will be considered. First, the conditions for the formed copy (initial copy) are examined.
The formed copy is a copy for matching the difference between the primary volume 530A and the secondary volume 530B at the start of the remote copy. In the formation copy, the data of the difference is copied from the primary volume 530A to the secondary volume 530B.
The formation copy can be performed in parallel with the remote copy of the write data described above. That is, the write data newly written from the primary host 31 to the primary volume 530A can be transferred to the secondary volume 530B for copying while eliminating the difference between the primary volume 530A and the secondary volume 530B.
However, the data transmitted from the primary volume 530A to the secondary volume 530B by the formed copy is the data written to the primary volume 530A, and the write time (the time when the write request is issued by the primary host 31) is not given. Therefore, during the execution of the formed copy, the write order (reflection order) of the write data cannot be grasped on the sub-storage control device 20 side, so that the data consistency cannot be ensured.
In this case, the integrity of the data between the primary and secondary volumes will be ensured when the formed copy is completed. For example, if the state of the secondary volume 530B during formation copy is expressed as "Duplex Pending state" and the state of the secondary volume 530B after the formation copy is completed is expressed as "Duplex state", the state is changed from "Duplex Pending state" to "Duplex state". At that point, data integrity is ensured.
However, when the sub-volume group 540B for which the formation copy is completed is registered in the extended group 550, the arbitration process is executed from the time when the sub-volume group 540B is registered in the extended group 550 as described in S69 in FIG.
Therefore, when the sub-volume group 540B is registered in the expansion group 550, the storage contents of each sub-volume 530B in each sub-volume group 540B belonging to the expansion group 550 may be different. That is, even if they belong to the same expansion group 550, the stored contents of each sub-volume 530B may be time-shifted.
Such a "deviation" continues until the sub-volume group 540B whose remote copy processing is delayed catches up with the other sub-volume group 540B which is preceded by the remote copy processing. Therefore, in this case, the consistency of the data is ensured when the following equation (1) is satisfied.
Reflectable write time Arrival light time of the secondary volume group registered in the expansion group ... (Equation 1)
That is, Equation 1 indicates that the reflectable write time indicated for extended group 550 is equal to or later than the latest arrived write time of subvolume group 540B registered with extended group 550.
Therefore, the sub-memory control device 20 monitors whether or not the condition of Eq. (1) is satisfied, and the state of the sub-volume 530B registered in the expansion group 550 until the condition of Eq. (1) is satisfied. Is treated as "Duplex Pending state (during forming copy)".
That is, in the sub-memory control device 20, even when the sub-volume 530B of the sub-volume group 540B registered in the expansion group 550 actually completes the formation copy (Duplex state), the condition of the equation (1) is satisfied. During the unsatisfied period, the state of the secondary volume 530B is treated as the "Duplex Pending state", and when the condition of the equation (1) is satisfied, it is set as the "Duplex state". As a result, it can be known that the consistency is not ensured in the case of the "Duplex Pending state" and the consistency is ensured in the case of the "Duplex state" together with the above-mentioned conditions for the formed copy. The sub-memory control device 20 and the normal storage control device 10 that form a remote copy pair provide information on whether or not data integrity is ensured to, for example, the primary host 31, the secondary host 32, the management server 43, and the like. Can be provided.
Since this embodiment is configured as described above, even when a plurality of remote copies are executed asynchronously, the stored contents of a plurality of sub-volumes 530B are stored in a predetermined single time (reflectable write time). Can be unified to. As a result, consistency between a plurality of volumes 530B to which different remote copies are applied can be maintained, and usability and reliability can be improved.
Next, the process of stopping the remote copy will be described. The process of stopping remote copying is also called the suspend process. In a system that performs remote copying, there are cases where you want to stop remote copying. For example, when maintaining each device of the storage system, or when changing the configuration of the storage system. In such a case, it is necessary to stop the remote copy.
In this embodiment, as described above, the sub-memory control devices 20 cooperate with each other to coordinate the execution of the remote copy process. Therefore, it is necessary for each sub-memory control device 20 to cooperate with each other to perform the suspend process.
Here, there are several types of suspend processing depending on the copy stop instruction source (suspend instruction source), the specifications related to the suspend processing (for example, the processing time and the processing result specifications), and the like. There are a primary host 31 and a secondary host 32 as suspend instruction sources. Specifications related to suspend processing include "flash", "purge", and "swap". Hereinafter, each type of suspend will be described.
FIG. 16 is a flowchart showing a flash suspend process when a flash suspend process is requested by the positive host 31.
The management unit 311 of the positive host 31 (hereinafter referred to as the regular host 31) issues a suspend request for which "flash" is specified as the suspend type to the positive storage control device 10 (S101). The suspend request includes, for example, the suspend type and the issue time of the suspend request, respectively. This suspend request is transmitted from the positive host 31 to the positive storage control device 10 having the positive volume 530A corresponding to the extended group 550 to be suspended via the data input / output path CN11.
The marker creation unit 130 of the normal storage control device 10 creates a marker and transmits the marker to the sub-memory control device 20 having the sub-volume group 540B to be suspended (S102). That is, the marker creation unit 130 acquires one new sequential number with reference to the volume group management information T21, and generates write data management information used as a marker. The marker creation unit 130 sets the marker attribute bit to on and sets "flash suspend" as the marker type. As described above, the marker attribute bit is a bit indicating that the write data is a marker, and is set to off in normal write data, but is set to on in the case of a marker. Further, in the write time in the marker, the time when the positive host 31 issues the suspend request is recorded.
Similar to the write data receiving unit 110, the marker creating unit 130 acquires a new sequential number obtained by increasing the previous value by 1 and sets it in the marker, and manages the newly acquired sequential number in the volume group. Record in information T11. The marker is configured in the same manner as the write data management information and is transmitted to the sub-memory control device 20. However, since the marker is control information for controlling the operation of the remote copy, it is not written to the secondary volume 530B.
After the marker is created, the positive storage control device 10 stops the new generation of write data management information regarding the positive volume 530A to be suspended (S103), and manages the subsequent updates by the positive host 31 by the difference bitmap T15. (S104). That is, the positive storage control device 10 manages a new write request to the positive volume 530A at which the remote copy is stopped by the difference bitmap T15 until the remote copy is restarted.
When the sub-memory control device 20 having the sub-volume group 540B to be suspended receives the marker from the normal storage control device 10 (S105), the write data receiving unit 210 causes the write data management information T23 to be created (S106). The write-data management information T23 created based on the marker, write time (suspend request issue time) and marker attribute bits and the marker type pixels respectively are recorded.
The write data receiving unit 210 updates the arrived write time with respect to the arrived write time management information T24 regarding the sub-volume group 540B to be suspended, and records the marker type and the marker time, respectively (S107). The arrived light time is recorded at the light time set in the flash suspend request.
As described above, in the arbitration process, the master processing unit 230 collects the latest arrived write times from each sub-memory control device 20 (S41 to S44 in FIG. 12). When the sub-memory control device 20 that has received the suspend request marker receives an inquiry about the arrival write time from the master processing unit 230, in addition to the write time recorded in the arrival write time management information T24, the marker type and the marker type and Notify the master processing unit 230 of the marker time (S108).
The master processing unit 230 records the write time, marker type, and the like received from the sub-memory control device 20 in the master time information T28 (S109). As described above, the master processing unit 230 compares the arrived write times collected from each sub-memory control device 20 and selects the oldest arrived time as the reflectable write time. The master processing unit 230 notifies each sub-storage control device 20 of the selected reflectable write time and marker type (S110).
When each sub-memory control device 20 receives the reflectable write time and the marker type from the master processing unit 230, each sub-storage control device 20 records the reflectable write time and the marker type in the reflectable write time management information T25 (S111).
Each sub-memory control device 20 writes each write data up to the reflectable write time to the suspend target sub-volume 530B (S112), and does not write the write data after the reflectable write time to the sub-volume 530B. , Managed by difference bitmap T29 (S113). That is, when the flash suspend is requested, the sub-memory control device 20 reflects the write data up to the reflectable write time on the sub-volume 530B, but the sub-volume 530B reflects the write data after the reflectable write time. The update location of the secondary volume 530B is managed by the difference bitmap T29 without writing to. A difference management method using a difference bitmap is disclosed in, for example, Japanese Patent Application Laid-Open No. 2004-13367.
Then, each sub-memory control device 20 changes the state of each sub-volume 530B in the sub-volume group 540B designated as the suspend target to the "suspend state" (S114), and the state change of the sub-volume 530B is completed. Notify the positive memory control device 10 of the fact (S115). This status change notification is transmitted from the sub-memory control device 20 to the normal storage control device 10 via the remote copy path CN12.
Upon receiving the state change notification from the sub-memory control device 20, the normal memory control device 10 changes the state of the normal volume 530A to the suspend state (S116). The positive memory control device 10 notifies the positive host 31 that the suspend process is completed (S117). As a result, the positive host 31 confirms that the remote copy has stopped (S118).
By performing such flash suspend, each write data updated before the issue time of the flash suspend request is reflected in each sub-volume 530B that is the target of arbitration processing, and then the remote copy is stopped. Can be made to. The write data after the flash suspend request is issued is differentially managed. Therefore, remote copying can be stopped while maintaining data integrity.
For example, when the primary host 31 stops issuing the write request and then issues the flash suspend request, the suspend process is completed with the stored contents of the primary volume 530A and the stored contents of the secondary volume 530B matched. be able to.
FIG. 17 is a flowchart showing a process when a flash suspend request is issued from the sub-host 32. The secondary host 32 is a standby host that is not used all the time and is used when a failure occurs in the primary site Ps. Flash suspend can also be requested from this sub-host 32.
The management unit 312 of the sub-host 32 issues a flash suspend request to the predetermined sub-memory control device 20 via the data input / output path CN11 (S121). The predetermined sub-memory control device 20 is a sub-memory control device 20 having any sub-volume group 540B belonging to the extended group 550 to be suspended. Here, since the sub-host 32 cannot use the time information (reference timer) used by the primary host 31, the flash suspend request issued in S121 does not include the issue time of the suspend request.
Upon receiving the suspend request from the secondary host 32, the secondary storage controller 20 notifies the positive memory controller 10 of the suspend request via the remote copy path CN12 (S122). The marker creation unit 130 of the positive memory control device 10 creates a marker in the same manner as the process for the suspend request from the positive host 31 described with reference to FIG. 16, and notifies the sub-memory control device 20 of the created marker (S123). However, unlike the marker creation step shown in FIG. 16, the write time recorded in the latest write time management information T14 is used as the write time included in the marker.
Hereinafter, as in the process shown in FIG. 16, the positive storage control device 10 stops the generation of new write data management information (S124), and manages the subsequent updates from the positive host 31 with the difference bitmap T15 (S). S125). Then, when receiving from the sub-memory control device 20 that the sub-volume 530B has been changed to the "suspend state", the normal memory control device 10 changes the primary volume 530A to the "suspend state" (S126). The positive memory controller 10 notifies the sub-host 32 of the completion of the suspend process (S127). By this notification, the sub-host 32 confirms the completion of the suspend process (S128).
In this way, even if the flash suspend request cannot be issued from the primary host 31, the flash suspend can be requested from the secondary host 32 using the latest write time managed by the positive storage control device 10. As a result, each write data before the latest write time can be reflected on the secondary volume 530B, and then the remote copy can be stopped.
FIG. 18 is a flowchart showing the purge suspend process. The purge suspend process is a process of stopping copying while maintaining consistency, similar to the flash suspend process. However, in the purge suspend process, the update to the primary volume 530A performed before the issue of the suspend request is completed with the flash suspend in that the copy stop is completed without waiting for the update of the write data to the secondary volume 530B. different.
The management unit 311 of the positive host 31 issues a suspend request for which "purge" is specified as the suspend type to the predetermined positive storage control device 10 via the data input / output path CN11 (S131). Here, the predetermined positive storage control device 10 is any one of the positive storage control devices 10 having the positive volume group 540A corresponding to the extended group 550 to be suspended.
Upon receiving this purge suspend request, the positive storage control device 10 notifies the sub-memory control device 20 that the purge suspend request has been received via the remote copy path CN12 (S132).
In addition, in response to a write data transfer request (write data read request) from the sub-memory control device 20 to the normal storage control device 10, the normal storage control device 10 may transfer the write data to the sub-memory control device 20. Conceivable. In this case, the normal memory control device 10 may be configured to notify the sub-memory control device 20 of the receipt of the purge suspend request by rejecting the transfer request from the sub-memory control device 20. Alternatively, the sub-memory control device 20 may be notified of the receipt of the purge suspend request by using the return value of the transfer request from the normal storage control device 10 to the sub-memory control device 20.
Upon notifying the secondary storage controller 20 of the receipt of the purge suspend request, the positive storage controller 10 stops generating new write data management information for the primary volume group 540A (S133). This write data management information generation stop continues until remote copying is resumed. After that, the positive storage controller 10 manages the update location of the positive volume 530A by the difference bitmap T15 for the write request received from the positive host 31 (S134).
The write data receiving unit 210 of the sub-memory control device 20 notified of the receipt of the purge suspend request sets "purge suspend" as the marker type of the arrived write time management information T24, and also sets the arrived write time management information T24. Updates the arrival light time of (S135) with the marker time.
After that, the same steps as the flash suspend process described above are executed (S136 to S146). That is, the master processing unit 230 collects the marker type and the marker time from the sub-memory control device 20 (S136, S137) together with the arrived write time, and determines the earliest arrived write time as a reflectable write time. The master processing unit 230 notifies each sub-memory control device 20 of the reflectable write time and the marker type (S138).
Each sub-memory control device 20 updates the reflectable write time management information T25 (S139), and reflects each write data up to the reflectable write time on the sub-volume 530B (S140). Each sub-memory control device 20 does not reflect the write data after the reflectable write time on the sub-volume 530B, and manages it by the difference bitmap T29 (S141).
Then, the sub-memory control device 20 changes the sub-volume 530B to the suspend state (S142), and notifies the normal storage control device 10 of this state change via the remote copy path CN12 (S143).
When the secondary storage controller 20 notifies the primary volume 530B of the status change, the primary memory controller 10 changes the primary volume 530A to the "suspend state" (S144), and completes the purge suspend process to the primary host 31. Report (S145). As a result, the positive host 31 confirms the completion of the purge suspend process (S146).
By performing such purge suspend processing, each write data up to a single time (reflectable write time) can be reflected in the sub volume 530B of each sub volume group 540B belonging to the expansion group 550. .. Then, the copy processing can be stopped without reflecting the write data after the single time on the sub volume 530B. That is, the data integrity can be maintained and the remote copy can be stopped.
Until the master processing unit 230 notifies the sub-memory control device 20 of the marker type (purge suspend) and the reflectable write time in S138, other than the sub-volume group 540B that has been notified of the receipt of the purge suspend request. Arrived write times may be updated in other sub-volume groups 540B.
Therefore, when the master processing unit 230 notifies each sub-memory control device 20 of the write time that can be reflected, a failure or the like occurs in the master processing unit 230, and the write time that can be reflected in some of the sub-memory control devices 20 occurs. If the marker type cannot be notified, it may not be possible to stop copying at a unified single time simply by performing arbitration processing after restoration.
Therefore, in this embodiment, when the master processing unit 230 detects the marker type of purge suspend in S132, the master processing unit 230 instructs the write data transfer stop of all the sub-storage control devices 20 to be arbitrated. Upon receiving this instruction, the sub-memory control device 20 performs a process of temporarily stopping the write data transfer. As a result, even if the write time and marker type that can be reflected to some sub-memory control devices 20 cannot be notified due to a failure of the master processing unit 230, the arbitration process is performed only after the master processing unit 230 is restored. The remote copy process can be stopped at a single time.
As described above, in the case of the configuration in which the transfer of write data is requested from the sub-memory control device 20 to the normal memory control device 10, the issuance of the write data transfer request from the sub-memory control device 20 to the normal memory control device is stopped. By doing so, it is possible to stop the transfer of write data from the primary storage control device 10 to the secondary storage control device 20.
Further, in the process shown in FIG. 18, even if the sub volume group 540B is notified of the receipt of the purge suspend request, it is reflected until the notification of the write time and the marker type that can be reflected is received from the master processing unit 230 (S138). Processing is not stopped and difference management is not performed (S140, S141). Instead of this, when the notification of the receipt of the purge suspend request is received from the positive storage control device 10 (S132), the purge suspend process can be shortened by stopping the reflection process and performing the difference management.
FIG. 19 is a flowchart showing a process when requesting purge suspend from the sub-host 32. The management unit 312 of the sub-host 32 notifies the predetermined sub-memory control device 20 of the suspend request for which "purge" is specified as the suspend type via the data input / output path CN11 (S151). The predetermined sub-memory control device 20 is any one of the sub-memory control devices 20 having the sub-volume group 540B belonging to the extended group 550 to be suspended.
Upon receiving the purge suspend request, the sub-memory controller 20 notifies the forward memory controller 10 of the suspend request via the remote copy path CN12 (S152). Upon receiving the notification of the suspend request, the positive storage control unit 10 notifies the sub-memory control unit 20 of the receipt of the purge suspend request via the remote copy path CN12 (S153).
Subsequently, the positive storage controller 10 stops creating write data management information regarding a write request to the positive volume group 540A (S154), and for write requests received from the positive host 31 thereafter, the positive volume 530A The updated part is managed by the difference bitmap T15 (S155).
Subsequent processing is performed in the same step as the step related to the purge suspend request issued from the positive host 31. When the primary storage control device 10 receives a notification from the secondary storage control device 20 that the secondary volume 530B has been changed to the suspend state, the primary memory control device 10 changes the primary volume 530A to the suspend state (S168). The positive memory controller 10 reports to the sub-host 32 that the suspend process has been completed (S169). As a result, the secondary host 32 confirms that the purge suspend process is completed (S170).
In this way, even in the case of a purge suspend request issued from the secondary host 32, the write data before a single time is reflected in each secondary volume 530B of each secondary volume group 540B belonging to the expansion group 550. The copy process can be stopped without reflecting the write data after the single time on the sub-volume 530B. That is, the data integrity can be maintained and the remote copy can be stopped.
Next, the process when swap suspend is performed based on the instruction from the sub-host 32 will be described. The swap suspend request is issued to enable the data stored in the secondary volume 530B of the secondary storage controller 20 when the secondary secondary host 32 of the standby system is used. That is, when the business process is inherited from the primary site Ps stopped due to a failure or the like to the secondary site Ss, the secondary host 32 issues a swap suspend request.
When using the secondary host 32, it is possible that the primary host 31, which is used for normal operation, cannot be used. Therefore, suspend processing is performed without using the positive host 31. Further, in order to take over the business processing of the primary host 31 to the secondary host 32, all the write data received by the primary storage control device 10 is reflected in each secondary volume 530B. That is, when the swap suspend processing described later is performed while the positive host 31 has stopped issuing the write request, the positive storage control device 10 determines the positive volume group 540A corresponding to the extended group 550 to be suspended. All the write data received up to this point is reflected in each sub-volume 530B of the sub-volume group 540B of the sub-memory control device 20.
There are three types of swap suspend processing described below. Therefore, the first swap suspend process to the third swap suspend process will be described in order.
FIG. 20 is a flowchart showing the first swap suspend process. The management unit 312 of the sub-host 32 collects the latest write time for the predetermined positive volume group 540A from the positive storage controller 10, and the latest write time (most advanced in time) from each collected write time. Get one (write time) (S161).
The predetermined positive volume group 540A is a volume group 540A corresponding to the suspended expansion group 550 as a source. Further, the sub-host 32 can collect the latest write time from the positive storage control device 10 via a plurality of routes. As the first collection path, the secondary host 32 receives the latest write time from each positive storage controller 10 via the data input / output path CN11 in the secondary site Ss, the secondary storage controller 20, and the remote copy path CN12. Can be collected. As a second collection route, the secondary host 32 collects the latest write time from the positive storage controller 10 via the management network CN14, the primary host 31, and the data input / output route CN11 in the primary site Ps. It is possible.
The sub-host 32 requests a swap suspend from the predetermined sub-memory controller 20 (S162). The predetermined sub-memory control device 20 is any one sub-memory control device 20 having a sub-volume group 540B belonging to the extended group 550 to be suspended. The swap suspend request issued by S162 includes the latest write time acquired by S161, and "swap" is set as the suspend type. This swap suspend request is transmitted to the predetermined sub-storage control device 20 via the data input / output path CN11 in the sub-site Ss.
The sub-storage controller 20 that has received the swap suspend request notifies the forward storage controller 10 of the swap suspend request via the remote copy path CN12 (S163). The marker creation unit 130 of the positive storage control device 10 that has received the swap suspend request compares the latest write time set in the suspend request with the write time recorded in the latest write time management information T14. , Update the write time recorded in the latest write time management information T14 (S164). That is, the marker creation unit 130 selects the newer of the write time set in the swap suspend request and the write time recorded in the latest write time management information T14, and this selection is made. The light time of the latest write time management information T14 is updated with the light time.
Then, the marker creation unit 130 creates a marker in the same manner as the processing in the case of the flash suspend request issued from the sub-host 32 described above, and transmits the created marker to the sub-memory control device 20 (S165). That is, the marker creation unit 130 sets the light time selected in S164 as the marker. The marker type set for this marker is "swap suspend".
The subsequent processing is the same as the processing for the flash suspend request issued from the positive host 31 described above (S166 to S170). That is, the positive storage control device 10 stops the generation of write data management information regarding the positive volume 530A (S166), and starts the difference management by the difference bitmap T15 (S167). Then, when the primary storage control device 10 receives a notification from the secondary storage control device 20 that the secondary volume 530B has been changed to the "suspended state", the primary memory control device 10 changes the primary volume 530A to the "suspended state" (S168) and swaps. Notify secondary host 32 of the completion of suspend (S169). As a result, the secondary host 32 confirms the completion of the swap suspend processing (S170).
FIG. 21 is a flowchart showing the second swap suspend process. The management unit 312 of the sub-host 32 issues a suspend request for which "swap" is specified as the suspend type to the predetermined sub-memory control unit 20 (S181). Similarly to the above, the predetermined sub-memory control device 20 is any one of the sub-memory control devices 20 having the sub-volume group 540B belonging to the extended group 550 to be suspended. Further, since the secondary host 32 cannot use the clock used by the primary host 31, the issue time is not set in the swap suspend request.
The sub-memory controller 20 that has received the swap suspend request sets "swap suspend request receipt" as the marker type in the arrival write time management information T24 for the sub-volume group 540B to be suspended (S182).
When the master processing unit 230 inquires the sub-memory control device 20 of the arrival write time for arbitration processing, the sub-memory control device 20 that has received the swap suspend request sets the arrival write time, the marker type, and the marker time. , Notify the master processing unit 230 (S183). The master processing unit 230 notifies each sub-memory controller 20 related to the extended group 550 to be suspended of the swap suspend request (S184).
The sub-memory control device 20 notified of the swap suspend request from the master processing unit 230 notifies the positive storage control device 10 of the swap suspend request (S185). The marker creation unit 130 of the positive storage control device 10 that has received the swap suspend request creates a marker in the same manner as the process for the flash suspend request from the sub-host 32 described above. This marker is transmitted to the sub-memory controller 20 (S186).
That is, the marker creation unit 130 sets the write time recorded in the latest write time management information T14 as a marker, and sets "swap suspend" as the marker type. Here, the positive storage control device 10 does not stop the creation of the write data management information regarding the positive volume group 540A at the time of creating the marker, and does not perform the difference management by the difference bitmap T15.
The sub-memory control device 20 (S187), which has received the marker requesting swap suspend from the normal memory control device 10, receives an inquiry of the arrival write time from the master processing unit 230 again. The sub-memory control device 20 notifies the master processing unit 230 of the arrival write time, the marker type, and the marker time (S188).
The master processing unit 230 records the arrived write time, the marker type, and the marker time received from the sub-memory control device 20 in the master time information T28 (S189). When the "swap suspend" is set for the marker type of the master time information T28 and the marker time is set for all the sub volume groups 540B belonging to the extended group 550 to be suspended, the master processing unit 230 sets each sub. Compares the marker times in volume group 540B and finds one of the newest marker times (S190).
The master processing unit 230 compares the arrived write times using the master time information T28, and obtains the earliest arrived write time among them. Here, if the latest marker time detected by S190 is past or the same as the obtained arrival write time, the master processing unit 230 can reflect the detected marker time. Select as light time (S191). The master processing unit 230 notifies each sub-memory control device 20 of the reflectable write time and the marker type (swap suspend) (S192).
Subsequent processing is the same as the processing of the flash suspend request described above (S193 to S199). That is, the sub-memory control device 20 that has received the notification of the reflective light time and the marker type updates the reflective light time of the reflective light time management information T25 (S193), and outputs the write data up to the reflective light time. Reflect on the secondary volume 530B (S194).
Then, the sub-memory control device 20 does not write the write data after the reflectable write time to the sub-volume 530B, but manages it by the difference bitmap T29 (S195). Then, the sub-memory control device 20 changes the sub-volume 530B to the "suspend state (both read and write are possible)" (S196).
The sub-memory control device 20 notifies the normal storage control device 10 of the state change of the sub-volume 530B via the remote copy path CN12 (S197). Upon receiving this notification, the positive storage controller 10 stops creating write data management information for the positive volume group 540A (S198), and manages write requests that occur thereafter with the difference bitmap T15 (S199). ..
FIG. 22 is a flowchart showing the third swap suspend process.
The management unit 312 of the secondary host 32 sends a suspend request for which "swap" is set as the suspend type via the data input / output path CN11, and has a secondary storage control device having a secondary volume group 540B belonging to the extended group 550 to be suspended. Issue to any one of 20 (S201). However, the sub-host 32 does not set the issue time in the swap suspend request.
The sub-storage controller 20 that has received the suspend request from the sub-host 32 sets "swap suspend request receipt" as the marker type of the arrival write time management information T24 for the sub-volume group 540B (S202). When the sub-memory control device 20 receives an inquiry about the arrival write time from the master processing unit 230, the sub-memory control device 20 notifies the master processing unit 230 of the arrival write time, the marker type, and the marker time (S203).
The master processing unit 230 notifies each sub-memory controller 20 related to the extended group 550 to be suspended of the swap suspend request (S204). Upon receiving this suspend request, the sub-storage control device 20 inquires of the positive storage control device 10 whether or not untransferred write data exists for the primary volume group 540A, which is the copy source (S205). This query can be made on a regular or irregular basis.
When the secondary storage controller 20 detects that there is no untransferred or unarrival write data based on the untransferred sequential number of the primary volume group 540A, the arrival write time for the secondary volume group 540B Record the arrival write time of the management information T24 in the marker time, and set "swap suspend" for the marker type (S206).
Similarly to the above, in the arbitration process, the arrival light time, the marker type, and the like are notified to the master processing unit 230 (S207). The master processing unit 230 selects the arrived write time as the reflectable write time (S208), and notifies each sub-storage control device 20 of the reflectable write time and the marker type (S209).
Upon receiving this notification, the sub-memory control device 20 updates the reflectable write time management information T25 (S210) and performs the same processing as described above (S211 to S214).
That is, the sub-memory control device 20 sequentially writes the write data up to the reflectable write time to the sub-volume 530B (S211). The sub-memory control device 20 does not write the write data after the reflectable write time to the sub-volume 530B, and manages the update location with the difference bitmap T29 (S212).
Further, the sub-memory control device 20 changes the sub-volume 530B to the "suspend state (both read / write possible)" (S213), and notifies the normal memory control device 10 of this state change (S214).
Upon receiving the status change notification of the secondary volume 530B, the positive storage control device 10 stops creating the write data management information for the write request for the primary volume group 540A (S215). The positive storage control device 10 manages the update location of the positive volume 530A with the difference bitmap T15 for the write request received after stopping the creation of the write data management information (S216).
By processing as described above, for the primary volume group 540A corresponding to the extended group 550 to be suspended, all the write data received by the positive storage controller 10 up to that point is transferred to the secondary volume 530B of the secondary volume group 540B. All can be reflected and the remote copy process can be stopped. The positive host 31 is not used for this suspend process.
In each of the suspend processes described above, it is stated that the report of information on the marker from the sub-memory control device 20 to the master processing unit 230 and the notification from the master processing unit 230 to the sub-memory control device 20 are performed together with the arbitration process. It was. However, the present invention is not limited to this, and a configuration in which a report on the marker and a notification from the master processing unit 230 to the sub-memory control device 20 may be performed by an independent communication separate from the arbitration processing may be used.
Further, in each of the above suspend processes, the extension group 550 to be suspended can be specified by explicitly specifying the extension group ID as a parameter of the suspend request, or a subvolume belonging to the extension group 550. By specifying either group 540B or sub-volume 530B, suspend processing may be triggered for the extended group 550 to which they belong.
Next, FIG. 23 is a flowchart showing a process in which a suspend factor such as a failure is automatically detected and the suspend process is performed.
The various suspend processes described above are executed based on the suspend request issued by either the primary host 31 or the secondary host 32. However, it is also necessary to perform suspend processing due to other factors, except when based on the explicit instructions from each host 31 and 32.
For example, if the remote copy process cannot be continued due to a failure of the remote copy route CN12 or the like, it is necessary to automatically perform the suspend process. In addition, in the third swap suspend process described above, when it is not possible to detect that there is no untransferred or unarrival write data within a certain period of time (this time can be set by the user), After all, it is necessary to automatically stop the remote copy process.
Therefore, in the process of FIG. 23, when the suspend factor is detected, the suspend process is automatically executed. First, the suspend factor is detected by either the normal memory control device 10 or the sub-memory control device 20. When the normal storage control unit 10 detects the suspend factor (S221: YES), the sub-memory control unit 20 is notified of the detection of the suspend factor via the remote copy path CN12 (S222).
The method of notifying the suspend factor is the same as the method of notifying the receipt of the suspend request in the processing of the purge suspend request issued from the positive host 31. The sub-memory control device 20 that detected the suspend factor, or the sub-memory control device 20 that received the notification of the suspend factor, sets the marker type in the arrived write time management information T24 to "failure suspend" (or swap suspend processing). If it is medium, set "Swap Suspend"). Further, the sub-memory control device 20 records the arrived write time recorded in the arrived write time management information T24 in the marker time (S223).
Subsequent processing is the same as the above-mentioned purge suspend processing (S224 to S235). That is, the master processing unit 230 collects the arrived write time, the marker type, and the marker time from the sub-memory control device 20 (S224, S225). The master processing unit 230 determines the oldest arrived write time as the reflectable write time, and notifies each sub-storage control device 20 of the reflectable write time and the marker type (S226).
The sub-memory control device 20 updates the reflectable write time in the reflectable write time management information T25 (S227). Then, the sub-memory control device 20 writes the write data up to the reflectable write time to the sub-volume 530B in order (S228). The sub-memory control device 20 does not write the write data after the reflectable write time to the sub-volume 530B, and manages the difference with the difference bitmap T29 (S229).
The sub-memory controller 20 changes the sub-volume 530B to the "suspended state" (S230). However, in the case of swap suspend, the sub-memory control device 20 changes the sub-volume 530B to "suspend state (both read / write are possible)". The status change of the secondary volume 530B is notified to the positive storage control device 10. (S231).
When the positive storage control device 10 receives the status change notification of the secondary volume 530B, it stops creating the write data management information related to the primary volume 530A (S232), and manages the write requests issued thereafter by the difference bitmap T15. (S233). Then, the positive memory control device 10 changes the positive volume 530A to the "suspend state" (S234) and notifies the outside of the completion of the suspend process (S235).
By the above processing, each write data arriving at each sub-storage control device 20 before a single time is reflected in the sub-volume 530B of each sub-volume group 540B belonging to the expansion group 550, and the above-mentioned single unit is used. The remote copy process can be stopped without reflecting the write data after one time. That is, it can be suspended while maintaining consistency.
When recovering from the suspended state, if necessary, remove the suspend factor and instruct each sub-volume group 540B to restart remote copy. In response to the instruction to resume remote copy, the formation copy process is performed, and the state of the secondary volume 530B belonging to the secondary volume group 540B is restored.
In the above-mentioned automatic suspend processing, the remote copy processing is stopped for the sub-volume 530B of all the sub-volume groups 540B belonging to the expansion group 550. Alternatively, the remote copy processing may be stopped only for the sub-volume group 540B or the sub-volume 530B whose remote copy processing cannot be continued due to a failure. In this case, it is also possible to continue the remote copy for the other sub-volume group 540B that can continue the remote copy process. This is realized, for example, by the master processing unit 230 notifying the sub-volume group 540B related to the failure occurrence only for the marker type and excluding the sub-volume group 540B related to the failure occurrence from the targets of the subsequent arbitration processing. It is possible. By processing in this way, remote copying can be continued in another normal sub-volume group 540B, and the usage of the differential bitmap T29 can be reduced. As a result, the amount of formed copy and the time required for formed copy when the remote copy process is restarted can be reduced.
Further, in the above-mentioned automatic suspend processing, the marker type is used as the notification content or instruction content in the notification or instruction between the master processing unit 230 and the sub-memory control device 20. Instead of this, another kind of notification or instruction such as "suspend processing instruction" may be used.
The present invention is not limited to the above-described embodiment. A person skilled in the art can make various additions and changes within the scope of the present invention. For example, each embodiment can be combined as appropriate.
Further, the case where the management unit 311 is provided on the primary host 31 and the management unit 312 is provided on the secondary host 32 has been described, but the present invention is not limited to this, and the primary storage control device 10, the secondary storage control device 20, and the management terminals 41, 42 are not limited to this. , The management unit 43 may be provided with a management unit.
Further, the case where the master processing unit 230 is provided in any one of the sub-storage control devices 20 has been described, but the present invention is not limited to this, and for example, each host 31, 32, the regular storage control device 10, the management terminal 42, and the management server A master processing unit for performing arbitration processing may be provided at 43 or the like. Further, each sub-memory control device 20 may be provided with a master processing unit in advance, and one or a plurality of master processing units may be automatically or manually selected and activated.
In addition, the case where the reflection order is controlled by using the write time given to the write data by the positive host 31 at the time of write request has been described, but instead of this, it is also possible to control the reflection order only by the sequential number. is there. For example, each time the storage control devices 10 and 20 receive the write data or the suspend request, the write data can be grasped from each other by setting a continuous number without omission. The number may be a serial number commonly used in the primary site Ps and the secondary site Ss.
Further, although each of the storage control devices 10 and 20 has a built-in disk drive 521, the present invention is not limited to this, and an external storage resource may be used. That is, at least one of the normal memory control device 10 and the sub-memory control device 20 uses a storage device existing outside its own housing as if it were its own internal storage device. You can also. For example, by providing a virtual intermediate device in the storage control devices 10 and 20, and mapping a logical volume provided in another external housing to this intermediate storage device, the external storage device can be used as its own. It can be used as if it were a built-in storage device.
<figref num="1">It is explanatory drawing which shows the whole concept of this invention.</figref><figref num="2">It is explanatory drawing which shows the whole of the storage system of embodiment.</figref><figref num="3">It is explanatory drawing which shows the hardware configuration of a host and a storage control device.</figref><figref num="4">It is explanatory drawing which shows the software structure of a host and each storage control device.</figref><figref num="5">It is explanatory drawing which shows typically the mode that the remote copy is performed by grouping a plurality of logical volumes.</figref><figref num="6">It is explanatory drawing which shows the structure of the volume group management information and the other party volume information.</figref><figref num="7">It is explanatory drawing which shows the structure of the write data management information and the latest write time management information.</figref><figref num="8">It is explanatory drawing which shows the structure of extended group management information and master information.</figref><figref num="9">It is explanatory drawing which shows the structure of the arrival light time management information, the master time information, and the can reflect light time management information.</figref><figref num="10">It is a flowchart which shows the processing method of a write request.</figref><figref num="11">It is a flowchart which shows the remote copy processing (write data transfer processing).</figref><figref num="12">It is a flowchart which shows the arbitration process for controlling the reflection time of each sub-volume.</figref><figref num="13">It is a flowchart which shows the process for writing write data to a secondary volume and reflecting it.</figref><figref num="14">It is a flowchart which shows the process for registering a volume in the expansion group which is the target of the arbitration process.</figref><figref num="15">It is a flowchart which shows the process for deleting a volume from an extension group.</figref><figref num="16">It is a flowchart which shows the flash suspend process executed in the storage system which concerns on 2nd Example.</figref><figref num="17">It is a flowchart when the flash suspend processing is executed by the instruction from a subhost.</figref><figref num="18">It is a flowchart which shows the purge suspend process executed in the storage system which concerns on 3rd Example.</figref><figref num="19">It is a flowchart when the purge suspend process is executed by the instruction from a subhost.</figref><figref num="20">It is a flowchart which shows the 1st swap suspend processing executed in the storage system which concerns on 4th Example.</figref><figref num="21">It is a flowchart which shows the 2nd swap suspend processing.</figref><figref num="22">It is a flowchart which shows the 3rd swap suspend processing.</figref><figref num="23">It is a flowchart which shows the suspend process which is executed automatically when the suspend factor such as a failure is detected.</figref>
Code description
1 ... Positive storage controller, 1A ... 1st volume, 1B ... Write data transfer unit, 2 ... Secondary storage controller, 2A ... 2nd volume, 2B ... Write data storage Department, 2C ... Update Department, 2D ... Update Order Information Management Department, 3 ... Update Control Department, 3A ... Update Order Information Collection Department, 3B ... Updateable Time Determination Department, 3C .. Update indicator, D1 ... write data, D2 ... update order information, H1 ... primary host, H2 ... secondary host, 10 ... primary memory controller, 20 ... secondary memory control Device, 31 ... primary host, 32 ... secondary host, 41,42 ... management terminal, 43 ... management server, CN11 ... data input / output route, CN12 ... remote copy route , CN13 ... Update control route, CN14 ... Management network, Ps ... Primary site, Ss ... Secondary site, 110 ... write data receiving unit, 120 ... write data transfer unit, 130 ... marker creation unit, 210 ... write data receiving unit, 220 ... write data reflection unit, 230 ... master processing Department, 311,312 ... Administration Department, 321,322 ... Operating System (OS), 331,332 ... Application Program, 510 ... Controller, 511 ... Channel Adapter (CHA), 512 ... Disk Adapter (DKA) ), 513 ... cache memory, 514 ... shared memory, 515,516 ... connection, 517 ... service processor (SVP), 521 ... disk drive, 530A ... positive volume, 530B .. .Secondary volume, 531A, 531B ... Journal volume, 540 ... Secondary volume group, 540A ... Primary volume group, 540B ... Secondary volume group, 550 ... Extended group, T11 ... Volume group Management information, T12 ... other party volume information, T13 ... write data management information, T14 ... latest write time management information, T15 ... difference bitmap, T21 ... volume group management information, T21 .. .Volume group management information, T22 ... other party volume information, T23 ... write data management information, T24 ... arrived write time management information, T25 ... reflectable write time management information, T26 ... master Information, T27 ... Extended group management information, T28 ... Master time information, T29 ... Difference bitmap
23 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22 Sheet 23
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2006039976A | Cites | Japan |
| JP2002189570A | Cites | Japan |
45 members in 4 offices
Priority claims2
| Document | Office | Kind | Date |
|---|---|---|---|
| 2006087983 | Japan | A | |
| JP20060087983 | – | – | – |
Members45
| Document | Office | Kind | |
|---|---|---|---|
| EP1538527A2 | European Patent Office (EPO) | A2 | |
| US2005122817A1 | United States of America | A1 | |
| US2005125465A1 | United States of America | A1 | |
| US2005125618A1 | United States of America | A1 | |
| JP2005190456A | Japan | A | |
| US2005213389A1 | United States of America | A1 | |
| EP1538527A3 | European Patent Office (EPO) | A3 | |
| US7085788B2 | United States of America | B2 | |
| EP1691291A1 | European Patent Office (EPO) | A1 | |
| CN1821974A | China | A | |
| JP2006221487A | Japan | A | |
| US2007043870A1 | United States of America | A1 | |
| US2007174352A1 | United States of America | A1 | |
| US2007192555A1 | United States of America | A1 | |
| EP1837768A2 | European Patent Office (EPO) | A2 | |
| CN101046759A | China | A | |
| EP1840747A1 | European Patent Office (EPO) | A1 | |
| US2007233981A1 | United States of America | A1 | |
| JP2007264946A | Japan | A | |
| US7293050B2 | United States of America | B2 | |
| US7330861B2 | United States of America | B2 | |
| JP2008040536A | Japan | A | |
| US7437389B2 | United States of America | B2 | |
| US2009024815A1 | United States of America | A1 | |
| CN100559351C | China | C | |
| JP4425728B2 | Japan | B2 | |
| US7724599B2 | United States of America | B2 | |
| US2010191864A1 | United States of America | A1 | |
| US7945750B2 | United States of America | B2 | |
| EP1691291B1 | European Patent Office (EPO) | B1 | |
| US2011219189A1 | United States of America | A1 | |
| US8032726B2 | United States of America | B2 | |
| CN1821974B | China | B | |
| JP4845627B2 | Japan | B2 | |
| EP1837768A3 | European Patent Office (EPO) | A3 | |
| JP4915775B2This record | Japan | B2 | |
| US8176010B2 | United States of America | B2 | |
| CN102446124A | China | A | |
| US8200928B2 | United States of America | B2 | |
| US2012191652A1 | United States of America | A1 | |
| US8250240B2 | United States of America | B2 | |
| US2012246429A1 | United States of America | A1 | |
| US8347053B2 | United States of America | B2 | |
| US8375000B2 | United States of America | B2 | |
| CN102446124B | China | B |
12 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Cancellation because of no payment of annual feesLAPS | LAPS | |
| Renewal fee payment (event date is renewal date of database)FPAY | FPAY | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Written amendmentJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 4915775
- Publication, DOCDB
- 4915775
- Publication, EPODOC
- JP4915775B
- Application
- 87983
- Application, DOCDB
- 2006087983
- Application, EPODOC
- JP20060087983
Titles2
- Japanese
- ストレージシステム及びストレージシステムのリモートコピー制御方法
- English
- Storage system and remote copy control method for storage system
Classification
- CPC, 3
- G06F11/2074
- G06F11/2064
- G06F2201/855
- IPC, 1
- G06F3 06
